# Two writers, one judge: how a review queue became 90% noise > A memory-conflict queue hit 1,377 rows on a vault with 33 real conflicts. One producer had an LLM judge and dismissed 97% of its own work. The other had none. - Author: Chad Priest - Published: 2026-08-31 - Canonical URL: https://blog.vodou.ai/review-queue-two-writers-one-judge/ - Tags: ai-agents, memory, architecture, observability --- If your agent has persistent memory, it will eventually store two things that cannot both be true. Last week's price and this week's price. The old address and the new one. Something a document said and something the user said. Every system that stores facts over time grows a contradiction problem, and every one of them ends up with the same surface: a queue of rows where a human is asked to pick a side. Mine had 286 open rows and roughly 90% of them were garbage. Not wrong in an interesting way. Garbage: line numbers, array indices, shell snippets, CPU floats with fifteen decimal places, and the same date written two different ways. I want to describe how that happened, because the cause was not the detector. It was the queue having two writers and only one of them being judged. ## Ten cards, each with the evidence folded behind it What shipped is small on the surface. The conflicts panel now shows one card per disputed value pair, capped at ten, with a source count. If a card says "5 pieces of evidence," you can expand it and see the five, fetched on expand rather than up front, because most cards are 1x and paying to load evidence nobody opens is how a "very few, highly relevant" queue turns slow. There is a resolved view, so a decision you made is a thing you can go back and read. There is a collapsed receipt for what the judge settled without asking you. And a card whose losing side no longer exists is not offered at all. Under that, the change that mattered: **Diagram: Who is allowed to write 'needs a human'** Two detectors both write candidate rows, a judge promotes some to open, and only open rows reach the user ```text [Import scan] --> [status = candidate] [Value mismatch (problem)] --> [status = candidate] [status = candidate] --> [Judge (fixed)] [Judge (fixed)] --promote--> [status = open] [Judge (fixed)] --or settle silently--> [The queue a human reads (fixed)] [status = open] --> [The queue a human reads (fixed)] notes: Import scan: pairwise, semantic Value mismatch: cosine + differing number sets Judge: filters, then decides The second detector used to write straight into the box on the right. ``` ## 932 dismissed and 33 open, against 143 dismissed and 253 open The measurement that changed my mind took one query. Two producers wrote into the conflicts table. I grouped by producer and by status: **Diagram: Rows by producer and status** The judged producer left 33 rows open after dismissing 932. The unjudged producer left 253 open after dismissing 143. ```text import scan · dismissed : 932 rows ################################ (fixed) import scan · open : 33 rows # (fixed) value mismatch · dismissed : 143 rows ##### (problem) value mismatch · open : 253 rows ######### (problem) ``` The judged producer dismisses 97% of its own candidates. That is the number I keep coming back to. An LLM pass over a pair of chunks throws away nineteen out of twenty things a similarity function thought were worth a human's attention. The second producer had no such pass. It wrote `status='open'` directly, on evidence that amounted to "these two chunks are cosine-similar and their number sets differ." And the thing deciding what a "number" was would call any digit-ish token a value. So `1.0` from a run log and `1.0` from a different run log were a conflict. `[0]` and `[1]` were a conflict. `8/31/2026` and `2026-08-31` were a conflict, which is the one that made me stop and look properly, because it is not a data problem at all. It is the same day written two ways, compared with exact string equality. The fix for that one is a canonical fold before the set difference, and it has a constraint I got wrong in an earlier draft: the fold must never change a value's *shape*. I had a version that stripped currency symbols, which quietly turned `$20/mo` into a bare number and broke a comparison that had been working. Both date notations are already the same shape, so folding them is safe. A yearless `7/24` is not the same shape as anything and stays alone. ## 1,377 cards on a vault with 33 real conflicts Then the console. The panel's query returned every row of every status, ungrouped. That is 1,377 cards, 1,075 of which were already dismissed, on a vault that had 33 conflicts a human should look at. The part I find embarrassing is that the CLI had been right the whole time. The command-line lister had filtered to `open` and grouped by value pair with a source count since the day it was written. Two readers of one table, one of them correct, and the wrong one was the one users actually saw. So the console adopted the existing grouping rather than growing a second one, and the card key is now spelled once in the engine and mirrored in `MCP-servers/Vodou-Console/src/brain/queries.ts` instead of being invented twice. That mirroring is not tidiness. Keying the resolve cascade on the value pair while the console keyed on the slot left four buttons sitting behind "r/AI_Agents subscriber count" after the user had already decided it. Two readers deciding separately what a card *is* produced a card you could resolve and then resolve again. ## Fourteen cards with two dead buttons, and 29 writes with no surface at all Two more, both found by looking rather than by a test failing. Resolving a conflict demotes the losing chunk. So when the losing chunk is already gone, there is nothing to demote. The card still renders, because the texts are denormalized and it outlives its chunk, and it renders with two buttons that do nothing. Fourteen of those were open. The predicate for "is this row actually actionable" is now spelled once and shared by both readers, for exactly the reason the card key is. It tests existence rather than `archived = 0`, because an archived chunk is still there and still resolvable, and filtering it out would hide conflicts the user can act on. The sweep also settles orphans now, so they do not sit open forever behind a filter. Invisible is not finished. And the judge settles some conflicts without asking, by demoting the older side. On the first run, 29 of those landed with no surface anywhere. The only way to see them was to open the database. Which is how I found out that an early keep-newest rule had been inverted: nothing told me, because nothing showed me. That receipt is now in the panel, collapsed by default, because it is a receipt and not a queue. **Diagram (beforeafter)** Before: 1377 ungrouped cards including dismissed ones and dead buttons. After: ten grouped cards, evidence on demand, a resolved view and a receipt. ```text BEFORE: Before - 1,377 cards, every status - 1,075 already dismissed - 253 open with no judge - 14 cards with dead buttons - 29 silent writes, no surface AFTER: After - 10 cards, open only, grouped - evidence fetched on expand - unactionable rows not offered - resolved view for what you decided - collapsed receipt for auto-settled ``` ## The property: one writer of "needs a human", everyone else writes "candidate" Here is the checkable version, and it is either true or false of your codebase right now. **Exactly one code path may set the state that means "a human must look at this." Every detector writes a candidate state. Promotion is a separate pass that can refuse.** If you have two writers of the terminal state, one of them is going to be the one nobody wrote a filter for, and it will out-produce the good one by an order of magnitude while looking identical in the UI. The second one, which is the same disease from my earlier notes: **a rule that exists only in a document is not implemented.** My project instructions have said for months that ephemeral tool output is not a fact and must not feed the contradiction detector. Nothing enforced it. Every one of those 1.0-versus-1.0 conflicts was a run-log line, and the rule that would have stopped them had been written down and never turned into code. Filters now run at the producer and again at the judge. ## Run this on your own queue: group by producer, then divide Five minutes, no tooling of mine, against any review or moderation or flagging table you have. ```sql SELECT producer, COUNT(*) AS total, SUM(status = 'open') AS still_open, ROUND(1.0 * SUM(status != 'open') / COUNT(*), 2) AS settle_rate FROM review_queue GROUP BY producer ORDER BY still_open DESC; ``` Passing looks like every producer landing in the same band: each one throwing away most of what it proposes, at rates within a factor of two of each other. Failing looks like mine, one row at 0.97 and one row at 0.36, and the low one holding almost all of your open volume. If your table has no `producer` column at all, that is the finding: you cannot tell which detector is filling your queue, so you cannot tell whether any of them is being judged. Then the orphan check, which found my fourteen dead cards: ```sql SELECT COUNT(*) FROM review_queue q WHERE q.status = 'open' AND NOT EXISTS (SELECT 1 FROM facts f WHERE f.id = q.loser_id); ``` Anything above zero is a card in your UI with a button that cannot do its job. Passing is `0`. Last, count how many places spell the predicate for "this row is actionable": ```bash grep -rn "status *= *'open'\|status *== *['\"]open" \ --include=*.ts --include=*.js --include=*.py --include=*.sql . \ | grep -v test | wc -l ``` One or two is a shared definition. Five is two readers that have already started disagreeing and have not noticed yet. ## Deterministic adjudication only helps the rows that reach the adjudicator The current advice on this class is good and it did not save me. Simon Foster's [agent-dialectic-resolver](https://download.plaud.ai/spotrunner1/agent-dialectic-resolver) states the rule plainly: "LLMs can propose. LLMs can object. LLMs can request evidence. LLMs cannot adjudicate." Mahamudul Hasan Rubel's [three-tier write-up](https://rubel.dev/blog/llm-agents-conflict-resolution-merging-divergent-workflow-outputs) draws the same seam, extraction then deterministic validation then arbitration. I agreed with both. I had built both. The bug was that a second extraction path bypassed the arbitration tier entirely, and every diagram in this literature is drawn as a single pipeline, so there is no place in the picture for "the other producer." That is why the [Semantic Consensus](https://arxiv.org/abs/2604.16339) finding lands for me: roughly 79% of multi-agent production failures come from specification and coordination rather than model capability. Mine was a coordination failure between two pieces of code I wrote, in one process, on one table. No agents disagreed. Two `INSERT` sites did. The transferable correction to the diagram is one arrow: the resolver must be the only thing with write access to the terminal state, not merely the recommended path to it. ## The tool that lists conflicts still cannot settle one Live limitation, unfixed today. The MCP tool surface can list conflicts and their statuses and it cannot resolve one. Resolving is still a CLI action. So an agent connected over MCP can see that two of your facts disagree and has no way to act on it, which means the read and the write live on different surfaces and I have not reconciled them. Related, and deliberate: the license and entity conflicts in my own vault are still sitting open. Those are judgments I want to make myself, so I tested the resolve path on a run-log conflict instead. Which is a fair description of the whole feature: the machine is now good at knowing which arguments it is not qualified to end. --- Source: [Two writers, one judge: how a review queue became 90% noise](https://blog.vodou.ai/review-queue-two-writers-one-judge/) by Chad Priest, from Building Vodou in Public.