AI column

Data@donghyeok99Staff

XERJ: a local search engine agents use instead of grep

A coding agent that read whole files through grep got passages from XERJ instead: 2.7x fewer output tokens, same 16/16 solved

xerj autoindex sorts code, PDFs, CSVs and logs into indices; one xerj search brings back a passage with its file:line. The bars are the README's output tokens.
xerj autoindex sorts code, PDFs, CSVs and logs into indices; one xerj search brings back a passage with its file:line. The bars are the README's output tokens.
Who
An Apache-2.0 search engine from xerj-org, for developers who want their coding agents to find code and docs fast.
When
Public since June 2026; per the README the latest is rc.101, which added the XERJ Query-1 model. Corpus packs shipped on 9 November.
Where
One Rust binary on Linux, macOS and Windows, running at localhost:9200; plugs into Claude Code, Cursor and other MCP clients.

What was hard

Give an agent a codebase it has never seen and it greps for files and reads them whole; the README measured up to 1.06M input tokens on one corpus. In front of an API it has not memorised, it guesses from memory and goes round the fix loop again and again.

What it does about it

Point XERJ at a folder and it reads each file to work out what it is, then builds an index per dataset. Code arrives with its definitions and line numbers. The agent asks through MCP tools and gets the passage with its file:line, not the whole file. The main use is reference coding: fetch and index the open-source projects closest to your task and look up how they solved it first.

The README demo: boot, bulk ingest, search, vector kNN, dashboards, no cuts
xerj autoindex ~/my-projectphase A: 593 datasets inferred, 1955 junk/skipped filesphase B: indexing 25329 files with 8 workersdone in 158.1s, 593 datasets, 83103 records live, 790 junk records
The README's own run

What people made with it

GitHub@Bowen-Man2026-11-01

Indexed the scientific Python repo rctd-py (97 files) in 7.4 s and SCTransform's 26 CSVs, 179,374 records, in 33.8 s with 4 workers; at 16 and 32 workers it stalled, and they reported that too.

Successfully indexed 179,374 records in 33.8 seconds.
likes 0 · comments 3 (2026-11-11 as of)GitHub ↗
GitHub@GHOST-AKU2026-10-28

Ran autoindex, lexical search and definition lookup on Windows x64 and wrote up how it went as a field report.

This report describes a small Windows x64 trial of autoindex, lexical search, and definition lookup
likes 0 · comments 4 (2026-11-11 as of)GitHub ↗

How it runs inside

  1. 1

    You → Claude Code

    the one-prompt install (llms.txt)Paste the README's prompt and the agent follows llms.txt: install, start, index.landing/llms.txt
  2. 2

    Claude Code → xerj node (localhost:9200)

    curl -fsSL https://xerj.org/get | shDownloads the release binary for your OS and CPU, checks its SHA-256, installs it, starts the node.landing/get
  3. 3

    Claude Code → xerj mcp

    xerj init → .mcp.jsonFinds the agents installed here, registers the MCP server and backs up any file it edits as .bak.engine/crates/xerj-autoindex/src/init.rs
  4. 4

    xerj autoindex → xerj node (localhost:9200)

    xerj autoindex ~/my-projectNever trusts extensions: magic bytes and content decide the format, one index per dataset.engine/crates/xerj-autoindex/src/sniff.rs
  5. 5

    xerj autoindex → xerj node (localhost:9200)

    code → symbols {name, kind, line}Source is parsed with tree-sitter, and every function, class or other definition gets its own document.engine/crates/xerj-autoindex/src/extract/code.rs
  6. 6

    Claude Code → xerj mcp

    xerj_search · xerj_def callsAn MCP tool call becomes a search request to the node.engine/crates/xerj-mcp/src/lib.rs
  7. 7

    xerj mcp → Passages with file:line

    definition at file:line + signaturexerj_def runs the same query as the CLI's xerj def and returns the definition.engine/crates/xerj-mcp/src/def.rs

What goes in, what comes out

8 tasks, 4 languages, 16 runs per arm (claude -p)Output tokens: grep 26,477, XERJ 9,982, both 16/16 solved (the README's case study)
xerj def "euler_to_rotationmatrix"The definition's file:line and signature
A Unity project folderOne record per GameObject/Component; "which scenes use this script?" is one query
/_memory/{namespace}What the agent learns, stored and recalled by meaning next session

Is it for you

Good for

  • Handing agents private, internal or niche code
  • Agents that spend many tokens on grep and reading files
  • Asking questions across a folder of logs, CSVs, PDFs and mail
  • Teams already using Elasticsearch clients or dashboards

Not for

  • Per the README it wins on code the model has not memorised, and is neutral to harmful on popular public libraries it already knows.
  • --insecure turns off TLS and auth, so anyone who reaches the port can read and delete every index; it is for one user on a dev machine.
  • The engine source is Apache-2.0, but the XERJ Query-1 crates in release binaries since rc.101 are proprietary; the model is off by default.
  • Of the 26 MCP clients listed, six have actually been registered and run, Claude Code among them.

Reviewed by Devalley’s editor before publishing. How the column is made

Sign in to post, comment and like.Sign in

Report

Reason

Read the community rules