# Your source-trust score is grading the pipe, not the source > Two records about one announcement, two different dates, both stored at trust_mult 1.0. The trust multiplier keyed on ingest channel, not on who asserted the claim. - Author: Chad Priest - Published: 2026-08-29 - Canonical URL: https://blog.vodou.ai/your-source-trust-score-is-grading-the-pipe-not-the-source/ - Tags: rag, sqlite, ai, debugging --- This morning my competitor-intel lane wrote two records about the same announcement, four minutes apart, into the same store: > Claudeforce announced 2026-08-26: Salesforce made Claude the default reasoning engine across its ecosystem... > Claudeforce dates conflict across sources: Salesforce's own press release is dated 2026-08-26 while one aggregator listed it 2026-08-28; the ~$300M token-spend figure comes from secondary coverage, not the press release text The second record is correct and useful. It is also completely disconnected from the first. There is no foreign key, no supersedes edge, no shared claim id. They are two chunks in a vector index that happen to be about the same event, and the only thing deciding which one a question gets back is cosine similarity. Both were stored with `trust_mult: 1.0`. ## Two dates for one event, both stamped `trust_mult: 1.0` Here is the scoring breakdown my retriever emitted for the two chunks on the same query set. I keep the full breakdown because the numbers are the argument: ```json { "chunk_tag": "GOTCHA", "final_score": 1.5672, "reranker_logit": 0.9999, "trust_mult": 1.0, "key_text": "what date was Claudeforce announced?" } { "chunk_tag": "RESEARCH", "final_score": 1.4484, "reranker_logit": 0.9343, "trust_mult": 1.0, "key_text": "what is Claudeforce?" } { "chunk_tag": "METRIC", "final_score": 0.4320, "reranker_logit": 0.0172, "trust_mult": 1.0, "key_text": "what did Salesforce stock do on Claudeforce?" } ``` Ask "what date was Claudeforce announced" and the caveat wins by 0.12. Ask "what is Claudeforce" and you get the bare assertion, no caveat, at 1.4484. The caveat does not ride along, because nothing says it should. The `~$300M` figure, which the note itself flags as secondary coverage, scores 0.4320 and is retrieved as an ordinary fact when the question is about spend. ## I thought the trust multiplier was about the source. It was about the door. I had `trust_mult` in the scorer for months and told myself provenance was handled. Then I dumped the actual values by scope and found this: - `import:obsidian` → 0.85 - `capture:ide:claude-code` → 0.925 - `web` → 1.0 - `workbench:skill-console:daily-competitor-intel` → 1.0 That is not a source trust score. That is a *transport* trust score. It grades the pipe the bytes came through. Everything the intel lane wrote arrived through one door, so a corporate press release and a third-hand aggregator summary of that press release came out the other side indistinguishable. The engine is doing exactly what it was told: it scores the channel, and the channel was one channel. I had a warning about this on 2026-07-19 and did not act on it. Same lane, same shape: *"Aggregators falsely assert 'Hermes v0.19.0 released July 1' by conflating it with v0.18.0's date."* I wrote the gotcha down as a note and moved on, which is a very efficient way to keep a bug. ## The class: a channel score wearing a source score's name The general failure is this. **A system ingests documents that each contain claims from several upstream parties, stamps trust metadata at document or pipeline granularity, and then serves claims at claim granularity.** The granularity mismatch is the whole bug. Web-scrape RAG hits it when one blog post quotes a primary filing and adds a wrong gloss. LangChain document loaders hit it when `metadata["source"]` is the loader name, not the asserting party. Entity-resolution graphs over multi-source feeds hit it when two vendors disagree on a number and the merge takes the newest write. Any MCP server aggregating third-party APIs hits it the moment two of those APIs return a different value for the same field. The standard advice is source-class tagging at ingest. Tian Pan's [provenance debt post](https://tianpan.co/blog/2026-05-07-provenance-debt-ai-knowledge-bases) is right that source class must be stamped at write time and is expensive to recover later, and right that `human_primary` and `third_party` should be distinct. What it does not cover is the case where a single document *contains both*. My intel note is one file, one ingest, one timestamp, and it carries a press release date and an aggregator date in adjacent sentences. There is no document-level value for `source_class` that is not a lie. [nuggetindex](https://github.com/searchsim-org/nuggetindex) gets closer by marking disagreements `Contested`, but it needs two competing records to contest. My extractor collapsed the conflict into one prose sentence, so the pair never existed as a pair. **The invariant: a claim's trust score must be a function of who asserted it, not of which pipeline carried it. If two claims from different upstream sources can share a trust value because they entered through the same door, your system has provenance metadata but no provenance.** ## Run this against your own index: one GROUP BY and one query Five minutes, your stack, nothing of mine. ```sql -- 1. Is your provenance a party, or a pipeline? SELECT source, COUNT(*) FROM chunks GROUP BY 1 ORDER BY 2 DESC LIMIT 20; ``` Failing output looks like `web_scrape 40213`, `slack_export 9822`, `nightly_job 4110`. Those are doors. Passing output identifies an asserting party and a retrievable locator: `sec.gov/Archives/... 88`, `reuters.com/... 41`. If your top value is the name of a cron job, stop here, you have found it. ```bash # 2. Pick an entity with a contested number or date. Retrieve, print provenance. curl -s localhost:8080/search -d '{"q":"when was announced","k":8}' \ | jq -r '.results[] | [.score, .meta.source_class // "MISSING", .meta.asserted_by // "MISSING", .text[0:80]] | @tsv' ``` Failing output: every row prints `MISSING MISSING`, or every row prints the same `source_class`, and at least two rows contain different values for the same field. That is two contradictory facts at equal confidence, and whichever one your reranker likes today is the one your user gets. Passing output: the rows carry different `asserted_by` values, and your ranker can order them. Then run the same probe with the *definitional* question instead of the *date* question ("what is X"). If the caveat chunk drops out of the top-k, you have my exact bug: the correction exists, and it only surfaces for people who already suspected something was wrong. If a claim in your store cannot answer "who said this, and what else says otherwise," it is not a fact. It is a sentence that scored well. --- Source: [Your source-trust score is grading the pipe, not the source](https://blog.vodou.ai/your-source-trust-score-is-grading-the-pipe-not-the-source/) by Chad Priest, from Building Vodou in Public.