docbert is a local document search tool. You point it at your files, it indexes them, and you search from the terminal, a web app, or an MCP-connected agent. Nothing leaves your machine.

v1.0 makes GPU indexing about 4x faster by actually using CUDA’s fast paths, swaps the default model for one that indexes three times faster, and stops re-reading the semantic index from disk on every query. Out of the box on my 3080 Ti, indexing went from 35 chunks a second under v0.9 to about 400. The embedding store takes half the disk it did, and BM25-only search is exposed everywhere the hybrid search already was. The major version is doing actual work, too: every code path that read pre-1.0 data is deleted. Old stores get one reset, docbert clean, and what v1.0 writes is what docbert reads from now on.

GPU indexing is ~4x faster

docbert embeds documents with a ColBERT model on a ModernBERT trunk. On CUDA, that trunk was leaving most of the hardware idle: everything ran in F32, which candle’s matmul never routes to tensor cores; all ~45 bias-free LayerNorms per forward missed candle’s fused kernel, which only dispatches when a bias is present; and the fourteen sliding-window attention layers unpacked the batch, applied rotary embeddings row by row, and repacked it — every layer, every forward.

v1.0 works through that list one step at a time:

The first three compound to roughly 3.6x on the encode stage against the F32 baseline, and overlapping pooling and LMDB writes with the GPU encoding the next batch adds another 1.11x — call it 4x on the embedding phase. The flash routing matters most for incremental syncs, where the tail batches are small. Query encoding rides the same kernels, so interactive search got faster as a side effect. A one-slot cache also replaced three grow-only per-batch caches that leaked 1-2 GB of VRAM over a full indexing pass; on a 12 GB card that’s headroom you notice.

None of this was allowed in on vibes. Every step had to hold parity against an eager CPU F32 ground truth, compared at a batch size that forces the same varlen path production uses. That gate earned its keep immediately: the sorted varlen path had been reading per-row token counts after padding extended every row in place, so flash attention attended [PAD] positions as real keys. Measured damage: min token cosine 0.9347 against ground truth, and 8 of 16 nearest-match flips on the probe batch. This wasn’t a bug the speedup work introduced — the same code shipped in v0.9 and ran in any CUDA build that indexed mixed-length batches, which is most of bulk indexing. (The prebuilt binaries never had it; they’re CPU and Metal only.) Nothing caught it before because integration tests encode corpora small enough to take the eager path, and the old parity protocol compared the GPU path against itself. The new harness is a permanent gate now: every CUDA attention path has to stay above 0.999 token cosine against the eager reference at F32. If you indexed on CUDA under v0.9, see the upgrade notes below.

The numerics stay conservative. The trunk computes in BF16 but the projection casts back to F32, rope angle tables are computed in F32 and cast once, and the attention mask uses per-dtype finite minima so a fully-masked padding row can’t softmax to NaN and poison the batch. Final parity against the F32 reference is max MaxSim delta 0.019 / min token cosine 0.996 at batch 64 (0.074 / 0.970 at batch 16), and the one top-1 flip in the probe was a 0.005-margin tie in the F32 reference itself. DOCBERT_EMBEDDING_DTYPE=f32 brings back full F32 compute if you want it. CPU-only builds are untouched by all of this.

Search stops paying a 790 ms tax

On my production corpus (155k chunks, a 761 MB PLAID index) encoding a query takes about 3 ms. Loading the index that query runs against took 790 ms, and docbert paid it on every semantic query: the whole file was re-read and re-deserialized from disk each time.

Two changes kill that. A process-wide cache now holds the last deserialized index behind an Arc, keyed by path, mtime, and length. In a long-lived process, the MCP server or the web UI, every query after the first is a stat plus a pointer clone. A rewrite by another process (say a docbert sync from a cron job) changes the key and gets picked up on the next query — which is the kind of thing the v0.9 move to LMDB was for.

And for the loads that still happen — one-shot CLI searches, the first query after a server starts — the on-disk format got reworked. The old format interleaved every token as a (centroid id, residual) record, so loading 21M tokens meant ~42M tiny reads. Format v3 stores centroid ids and residuals as two contiguous sections and loads them as bulk slice copies, and the inverted-file rebuild that runs on every load swapped per-document sort-and-dedup for a last-doc-seen stamp array (182 → 76 ms). Cold load on the same index: 790 ms → 314 ms.

v3 is also the only format v1.0 reads. The interleaved layouts from earlier releases are refused outright; more on that below.

Per-query metadata lookups also went from scanning the entire document table to point reads on the candidates that survived scoring. Honestly a wash on today’s corpus, the full scan cost under a millisecond, but it’s one less cost that scales with corpus size instead of result count.

A faster default model, and BM25 gets a front door

The default embedding model moves from lightonai/LateOn to lightonai/GTE-ModernColBERT-v1, and this one is a walk-back. v0.7 made LateOn the default on the strength of its BEIR numbers: 57.22 NDCG@10, first place on the decontaminated split, comfortably ahead of GTE-ModernColBERT-v1’s 54.75. Three releases later I can report what those 2.5 points cost. LateOn pushes every chunk through a three-layer Dense projection chain (the one that took until v0.8 to load correctly) at a 519-token document length; on my 3080 Ti it embedded about 35 chunks a second, and GTE, with a single Dense layer and 300-token documents, does about 110 on the same card. The benchmark edge never showed up in my actual searches: on my corpus LateOn was a little worse. Triple the indexing time for slightly worse results is not a trade, so it’s undone.

Stacked on the CUDA work from the first section, the embedding meter on this card now reads about 400 chunks a second, against 35 for v0.9 out of the box. Shorter chunks mean the two meters aren’t counting identical work, but however you slice it, a full sync takes a fraction of the time it did.

Along with the model, docbert stops dictating sequence length. It used to force a hardcoded 519-token document length onto every model; now the length declared in the model’s own config_sentence_transformers.json wins, the old constant became a 300-token fallback for models that don’t declare one, and the default chunk size derives from that (roughly 2K characters down to 1.2K).

Nothing migrates silently. The model that built your index is recorded in the config database, and v1.0 checks it: docbert sync refuses to run on a mismatch, and docbert status tells you exactly what’s wrong. Coming from v0.9 the question resolves itself during the upgrade; see the notes below.

BM25-only search (exact terms, identifiers, verbatim strings) existed in the CLI as --bm25-only. v1.0 gives it a front door everywhere else: mode=bm25 on the web search API, a search_bm25 tool for the web chat agent plus a rewritten system prompt that steers identifier-shaped queries to it, and a bm25_search tool on the MCP server next to semantic_search and the hybrid search. BM25-only skips query encoding and the PLAID index entirely, so it needs no GPU and no model download, and it works before you’ve ever built a semantic index.

The MCP tools also lost their redundant prefix. Clients already namespace tools by server, so every call read mcp__docbert__docbert_search: the server name twice. The tools are now bare verbs — search, get, multi_get, status, plus the query prompt (semantic_search and bm25_search were already bare, and rustbert’s rustdocs_* got the same treatment). Agents that discover tools dynamically won’t notice; anything that hardcoded the old names needs updating.

The embedding store halves

Stored embeddings switch from F32 to BF16. embeddings.db is docbert’s dominant artifact on disk; for my corpus that’s 11.5 GB, and a re-embed drops it to roughly half.

The discarded 16 bits per component were never real precision. The encoder trunk computes in BF16 on CUDA, so the low mantissa bits of the stored F32s are normalization noise; on a sample of the live corpus, plain zstd manages only 1.11x on them, which is about what noise compresses like. And the only reader of this store is the PLAID bridge, which re-quantizes every token down to 2 bits per dimension anyway. Search never touches the raw values at query time. Half the disk, identical results.

The pre-1.0 formats are gone

Calling this 1.0 had to mean something, and here it means the formats hold still and the code stops reading their ancestors. Deleted outright: the redb-to-heed database migration (a 692-line module, plus the redb dependency it existed for), the F32 embedding read path, PLAID v1 and v2 support, and the fallback decoder for pre-1.0 conversation JSON. That last one turned out to be dead code anyway; the old JSON shapes only ever lived inside redb-format files, which are now refused wholesale.

Transparent migration is over. An old store fails fast, and every one of those errors names the command that fixes it: docbert clean. Clean dispatches before any database opens, so it can’t be blocked by the very errors it exists to fix. It deletes redb-format files outright, strips leftover F32 embedding rows, and wipes document state (manifests, tantivy entries, merkle snapshots, the recorded model key) so the next docbert sync re-embeds everything instead of skipping documents it thinks are unchanged. Registered collections survive as long as the config database itself is readable.

The docs got the same discipline as the data. One pass rewrote them as a spec of how v1.0 works (not how it got here), with every claim checked against the code: crate layout, table lists, flag behavior, error types. A second pass normalized the prose into ASD-STE100 Simplified Technical English, the controlled language aircraft maintenance manuals are written in. Active voice, one instruction per sentence, a hard cap on sentence length, an approved word list, no semicolons. The docs read like a manual now, because they are one.

The rest

docbert clean is a new maintenance command, and the legacy reset above is only its second job. Its day job is reclaiming space. Deleting a document deliberately leaves its embeddings on disk (content-addressed since v0.9, so re-adding the same text is a cache hit), and clean drops every embedding no document references anymore, with --dry-run to preview and --json for scripting. If the stored model doesn’t match the current one it goes further and clears everything the old model built, so a fresh docbert sync starts from a clean slate.

The web chat can draw now. The agent is told it may emit mermaid diagrams, and the UI renders mermaid code fences as SVG themed to match the app: Catppuccin Mocha in dark mode, Latte in light, in chat transcripts and document previews both. The mermaid bundle is lazy-loaded, so you only pay for it when a diagram actually appears.

docbert now logs at info level to stderr by default instead of staying silent; -v means debug, -vv means trace, and DOCBERT_LOG overrides everything (DOCBERT_LOG=warn restores the old quiet). tantivy and pdf_oxide are pinned one level below whatever you pick, because they’re chatty.

cargo build works on machines without Nix now: a build script compiles the web UI with bun (npm as a fallback) before it gets embedded into the binary, skippable with DOCBERT_SKIP_UI_BUILD. On the Nix side, the runtime closure lost about 2 GiB of dead weight. rustc bakes toolchain paths into debug info and docbert ships unstripped, so every install dragged the whole nightly toolchain along; those references are now scrubbed, with a build-time guard so they can’t sneak back. The web UI’s dependencies got locked down against supply-chain attacks, and react-router was bumped past a high-severity DoS advisory.

candle moved from 0.10.2 to 0.11.0 with byte-identical similarity scores across the upgrade, and the MCP server migrated to rmcp 2.2.

Getting started

Download a binary from GitHub releases. Prebuilt for Linux (x86_64 and aarch64, static musl builds) and macOS (Apple Silicon, with Metal). For CUDA, install through Nix or Cargo:

# Nix
nix profile install github:cfcosta/docbert

# Nix, for CUDA support (NVIDIA gpus)
nix profile install github:cfcosta/docbert#docbert-cuda

# Nix, for Metal support on Mac OS
nix profile install github:cfcosta/docbert#docbert-metal

# Cargo
cargo install --git https://github.com/cfcosta/docbert

# Cargo, for CUDA support (NVIDIA gpus)
cargo install --git https://github.com/cfcosta/docbert --features cuda

From zero to searching, three commands:

# 1. register a folder as a collection (nothing is indexed yet)
docbert collection add ~/notes --name notes

# 2. index it — the first run downloads the embedding model
#    (~600 MB, cached under ~/.cache/huggingface)
docbert sync

# 3. search
docbert search "how did I configure backups"

docbert search runs hybrid BM25 + semantic. docbert ssearch is semantic-only, docbert search --bm25-only skips the model entirely, and docbert status shows what’s indexed. Indexing handles Markdown, plain text, and PDF, and respects gitignore rules when the collection root is a git repo.

The other two frontends are one command each:

docbert web    # web UI + HTTP API on http://127.0.0.1:3030
docbert mcp    # MCP server on stdio

For MCP, the config is the standard stdio shape. With Claude Code it’s claude mcp add docbert -- docbert mcp:

{ "mcpServers": { "docbert": { "command": "docbert", "args": ["mcp"] } } }

Upgrading from v0.9

No decision tree this time. Pre-1.0 data is unreadable to v1.0 on purpose: the F32 embeddings, the old PLAID layouts, anything still in redb. The first thing the new binary does with an old store is stop and tell you to run exactly this:

docbert clean   # wipe pre-1.0 data; registered collections survive
docbert sync    # full re-index with the current default model

The sync is a from-scratch re-encode of the corpus, which is the part that used to hurt. It doesn’t anymore. On CUDA, the meter that read 35 chunks a second when I first indexed this corpus reads about 400 now; the re-index that used to take over an hour here takes minutes.

The forced re-encode also closes two accounts from earlier in this post. Indexes built by a v0.9 CUDA binary carry the [PAD]-attention bug from the first section, with embeddings slightly off wherever a batch mixed document lengths. Since everything goes back through the fixed path, that’s gone. And the model swap needs nothing from you: the re-index picks up GTE-ModernColBERT-v1 on its own. One caveat if you had pinned a model deliberately: clean wipes the recorded model along with the rest of the document state, so run docbert model set <your-model> again before the sync.

Finally, if any agent config hardcodes the old MCP tool names (docbert_search and friends), update them to the bare verbs, and set DOCBERT_LOG=warn if you preferred your stderr silent.