building vodou.

Pipeline counts lie: 2 turns STORED, 126 characters saved

A capture reported 2 turns STORED and wrote 126 characters. Per-stage counts never compare output length to source length. The ratio check that catches it.

Chad Priest / / 6 min read

On 2026-07-31 my console printed 2 turn(s) STORED by Vodou ✓ after capturing a chat. The row it wrote held 126 characters, and the check I ran afterward printed ratio 0.031.

Those two numbers give you the denominator: 126 ÷ 0.031 puts the source node at roughly four thousand characters. I am handing you the division instead of a stored number because nothing in the pipeline stored the source length: that is the entire post. The ratio exists only because I computed it by hand, after the fact, on a page I had to go back and re-open.

Every stage was honest about itself. The site adapter matched. The parser returned two turns. The gateway wrote two rows and counted two rows. This is the failure class: a multi-stage pipeline where each stage counts its own output and no stage compares that output against the size of what it was handed. Nothing in the chain ever measured the source, so nothing could notice that three percent of it survived.

My parser test passed on a fixture built from the truncated row

The parser was the only stage that touched text, so it got the day.

The specific waste: I built a fixture out of the captured row. I copied the stored text into a .txt beside the parser tests, wrote a case asserting it produced two turns with non-empty bodies, and it went green on the first run. I took that as the parser being cleared and went looking downstream at the writer.

That test could not have failed. The fixture was the bug’s output. I had fed the parser the thing the parser produced, asserted it produced that thing, and called it evidence. Any test whose input you sourced from the incident is a test of your copy-paste. The fixture has to come from the page, upstream of the stage you suspect, or it proves nothing about how the stage behaves on real input.

The parser was fine. It had been handed a spliced fragment of a streamed reply, and it turned that fragment into a turn, because a fragment shaped like a turn is a turn.

The selector matched one slice of a message still being streamed

This is the part that transfers. The selector upstream had matched a node holding one slice of an incrementally rendered message. The UI was still writing into the DOM when I read it. Everything downstream of a bad selector behaves perfectly, which is what makes it expensive.

Any selector run against a UI that streams can match a partial node, and nothing about the match tells you so. You get a string. It parses. It is a third of a sentence.

The check is cheaper than the ratio and catches the bug one stage earlier. Query, wait, query again, compare:

const SEL = 'main';                      // your selector
const len = () => document.querySelector(SEL)?.innerText.length ?? -1;
const a = len();
setTimeout(() => {
  const b = len();
  console.log(a, '->', b, a === b ? 'settled' : 'CAPTURED MID-STREAM');
}, 1500);

If the two lengths differ, you read the node while something was still appending to it, and whatever you captured is a prefix. On a static page they are equal and you move on. Run it once against a page that streams, an assistant mid-reply, a log tail, an infinite feed, and watch it fire, so you know what the positive looks like.

The fix is the same shape everywhere: don’t capture on a mutation, capture when mutations stop. Poll the length until it holds steady across two reads, or attach a MutationObserver and debounce the capture behind it. Do not pick a fixed delay and hope; a slow reply outlives your timeout and you are back to a prefix.

The ratio check: what you kept over what you were given

The streaming check tells you the DOM was still moving. The ratio tells you how much of it made it to disk, no matter what ate the rest.

At the boundary where you hand raw text to a parser, measure both sides. You can watch it work right now. Open devtools on this page and paste:

// A "parser" that splits body text into paragraphs on blank lines.
// Swap in your real selector and your real parse().
const parse = (s) => s.split(/\n\s*\n/).map(t => t.trim()).filter(Boolean);

const raw  = document.querySelector('main')?.innerText ?? document.body.innerText;
const kept = parse(raw).join('');
console.log('raw', raw.length, 'kept', kept.length,
            'ratio', (kept.length / Math.max(raw.length, 1)).toFixed(3));

You should see a ratio in the high 0.9s: the only thing that parser drops is the whitespace between paragraphs. That is what healthy looks like: ratio 0.972 is what my own passing captures printed. Then replace 'main' with the selector you actually ship and parse with the function you actually ship, and run it on the page you actually scrape. Mine printed 0.031, and the run still exited 0.

Where to put the alarm

Not at 1.0, and not at a number I pick for you. A healthy parser legitimately lands well under 1.0 when it:

  • strips markdown scaffolding: fences, list bullets, table pipes, heading hashes
  • drops UI chrome the selector swept in: “Copy”, “Regenerate”, timestamps, avatar alt text
  • dedupes a UI that re-renders the whole message on every token, where the raw node can contain the same sentence many times over and the correct ratio is a small fraction
  • discards attachments, code blocks, or citations on purpose

Any of those gives you a stable low number on correct input. If you alarm on an absolute threshold you will page yourself on day one and delete the check on day two.

So: log the ratio for a week before it can wake anyone. Then alarm at half your own median, with a hard floor at 0.5: below half of what your pipeline normally keeps, something changed that you did not change. A parser that normally sits at 0.62 should page at 0.31. Mine normally sat above 0.95, which is why 0.031 was not a judgement call.

Sort by stored length: does your bottom row hold 126 characters?

You do not need instrumentation to start. Sort by what you stored and look at the bottom:

SELECT id, length(body) FROM your_table ORDER BY 2 ASC LIMIT 20;

Read those twenty against your own sense of what a typical captured reply looks like. You know whether your users write 126-character messages. Rows that are an order of magnitude below your median are either real one-word replies or amputations, and you can tell which by opening three of them.

That is the whole check on a store with no extra columns. The instrumented version is the follow-on: record the raw length at the capture boundary, store it beside the body, and the eyeball becomes a query.

SELECT id, length(body) AS kept, source_len,
       ROUND(1.0 * length(body) / NULLIF(source_len, 0), 3) AS ratio
FROM your_table
ORDER BY ratio ASC
LIMIT 20;

If source_len does not exist, that is the finding, not an obstacle. Add the column, backfill nothing, you cannot recover a denominator you never measured, and watch the bottom twenty accumulate for a week before you trust any threshold you set on it.

A pipeline that only counts its own output will always report that it produced all of it.