Your document chunker is a memory chunker, and it quadruples your store

Every memory system I have looked at, mine included, grew a document ingest path by pointing the existing chunker at a folder. It splits on bullets, it caps at a few hundred characters, it has worked for months. Then a real directory goes in and the store gets four times denser, the router starts ranking a 0.000 above a 0.746, and nobody can say which of two scorers produced the number on screen. None of that is a model problem. All of it is a seam between two things that were quietly sharing one rule.

This is what shipped over eight days in Vodou’s library lane, and more usefully, what was wrong first.

A library is not a memory: whole files, one card each, two ways to match

The capability is plain. You add a document from the Library page (a file, a folder, or a URL), it is extracted and indexed as a whole document, it gets a one-row “card” summarising what it is about, and it becomes addressable in any chat as an @doc: token. Two matching lanes answer two different questions. The subject lane asks “is this document what you are asking ABOUT” and scores with a cross-encoder. The topic lane asks “does this document DISCUSS what you asked” by finding the best passage in the chunk index and mapping it back to its file. The two are never sorted against each other, because a cross-encoder probability and a raw cosine are not the same kind of number.

The public half is MCP-servers/Vodou-Console/src/api/library.ts (the routes), extension/Store-vodou-bridge/background.js (the URL and page-text lanes) and MCP-servers/Vodou-Console/src/__tests__/library-e2e.test.ts, which is the harness that should have existed first.

181 chunks from one 15,869-character file

The directory ingest command already existed. Its first real run turned a 15,869-character plan file into 181 chunks, about 88 characters each, against a 600-character ceiling. The reason was correct behaviour in the wrong place. For memory files, a bullet is a fact: ”- The user’s name is Chad.” has to be its own retrieval unit. For a document, ”- [ ] Run npm run build” was becoming its own embedding row. Across the 643 files I actually wanted in, that density would have roughly quadrupled a 38k-chunk store.

The engine now carries a chunk mode, memory or document, and the same file went from 181 to 50 chunks with nothing dropped. One test pins the difference between modes rather than either mode alone.

The first fix changed nothing, and that miss is worth keeping. I patched the function that applies the ceiling, which is the probe. The writer re-chunked in memory mode on its own. Probe and writer had two copies of the decision, so fixing one left the store exactly as dense as before.

0.70 was calibrated on cosine and applied to a cross-encoder

Then the router. The card lane had a 0.68 floor justified by a code comment citing one pair of observations. Measured against a 22-query labelled set, the relevant population (0.553 to 0.845) and the ordinary-browsing population (0.455 to 0.686) overlap outright. “Sourdough Starter Recipe” scored 0.686, above a genuine contract query, because a bi-encoder rewards surface overlap between a short title and a long summary. No constant separates those two populations. Switching the card scorer to the cross-encoder collapsed every noise query to about 0.000 and routed all 8 relevant queries to the correct document. It also took match from 0.36s to 8.3s cold, so a cosine gate below every floor skips the model for ordinary browsing.

That gate created the next bug. The score field was overwritten with a cross-encoder probability when the reranker ran and left as a cosine when the gate skipped it. The 0.70 floor was calibrated on cosine. So whether a document was eligible depended on whether a latency gate happened to fire. Three queries, each ranking the right document first under both scorers, each rejected under only one:

query                       cosine   cross-encoder
launching on hacker news     0.703       0.000
console redesign             0.720       0.001
go to market launch plan     0.676       0.052

Meanwhile sub-floor subject hits were never dropped, and because topic hits are appended after subject hits, a 0.000 non-answer printed above the 0.746 right answer. I first filtered them out, which turned “go to market launch plan” (right document, 0.052) into “no documents matched”. Losing the answer is worse than mis-ordering it, so sub-floor hits now survive as a tagged weak fallback used only when neither lane produced anything real.

The split that stuck: cosine decides eligibility and is the number shown; the cross-encoder decides order among the eligible; score is restored to the cosine after reranking. Cosine alone gets “go to market launch plan” wrong (0.687 for the wrong plan). The cross-encoder alone rejects it. Each does the one thing it is calibrated for.

There was a third leak under load. The reranker had four early-return paths and only one was safe. When the model failed to load under memory pressure, raw cosine stood and “Getting Started with React” cleared a floor that means nothing on a cosine. Cosine ordering is a weaker signal; cosine thresholding is a broken one. The result now carries an explicit calibrated-or-not marker, and an uncalibrated result is suppressed rather than answered. On a machine with no inference runtime the lane goes silent and says so. All 11 noise queries: 0 leaks, from 1 to 2 under load.

The failure class: one field, two producers, one floor

Two invariants fell out, and both are checkable against a codebase rather than advice.

First: the chunk-size decision has exactly one owner, and the probe that predicts the chunk count and the writer that produces it call it. If they can disagree, the ceiling trims the wrong amount and a test on the probe passes while the store fills.

Second: a threshold is bound to the distribution it was calibrated on, so any field a floor reads must have exactly one producer or a calibration tag. If a rerank step can be skipped, disabled, or fail, and on every one of those paths the same field holds a different scorer’s output, then eligibility is a function of latency and memory pressure, not of relevance.

Two queries to run on your own store tonight

Both take five minutes on any SQLite or Postgres chunk table. First, chunk density per source. Documents and facts should not share a distribution.

SELECT source_id,
       COUNT(*)                          AS chunks,
       SUM(LENGTH(content))              AS chars,
       SUM(LENGTH(content)) / COUNT(*)   AS avg_chunk
FROM chunks
GROUP BY source_id
ORDER BY chunks DESC
LIMIT 20;

Passing: your long documents sit near your configured ceiling (a 600 ceiling should show 400 to 600). Failing looks like mine did, 181 | 15869 | 87 for a single file. If your average chunk is a fifth of your ceiling, your document chunker is a fact chunker.

Second, the scale check. Log the retriever’s final score alongside which scorer produced it, then look at the floor’s neighbourhood.

SELECT scorer, COUNT(*),
       MIN(score), AVG(score), MAX(score)
FROM retrieval_log
WHERE score BETWEEN 0.60 AND 0.80
GROUP BY scorer;

Passing: one row, or two rows whose ranges do not both straddle your floor. Failing: cosine averaging 0.71 and cross_encoder averaging 0.03 in the same band, which means the same floor is admitting one and rejecting the other by accident. If you have no scorer column, that is the finding: add it before you tune the threshold again.

The retrieval-quality posts say measure recall; the seam is inside one score

The current writing is right that context-building fails before generation does. The New Stack’s piece on retrieval as the defining bottleneck and Weaviate’s retrieval-quality overview both land on the same point: when retrieval breaks, the model extrapolates with full fluency. Shaped’s stochastic parrot post names the semantic trap where a vector store finds fifty near-misses. What none of them say is that the trap can be a data-shape bug: a single score column with two producers, or a chunker with two callers. Recall benchmarks report the symptom. The mem.nowledge library docs get the product split right (documents stay whole, memories are takeaways), and that split has to reach the chunker and the scorer too, not stop at the UI.

Still open: the topic lane leaks on vocabulary, and there is no number

The topic lane matched react.dev pages to implementation plans and doc.rust-lang.org to anything with a Rust paragraph. Two threshold calibrations and a cross-encoder swap failed: true interior hits sat at 0.702 to 0.741, false ones at 0.703 to 0.743, interleaved point for point. The fix that held is a document-level veto (the cited passage’s heading must mention what was asked), 3 of 3 false positives rejected, 0 of 4 true hits killed. But that labelled set is 22 queries on one library, and the subject lane still scores real paraphrases near 0.0, so weak ordering is barely better than chance. The e2e suite reports the topic leak count on every run instead of thresholding it, because a cap calibrated on a live library breaks within hours. I have the number for mine. I do not yet have one I would trust on yours.