# Our registry called three context lanes dead. One was live.

> We re-measured a declared registry of prompt lanes against 3,148 logged rows and found 23 side LLM calls outside the trace. Here's the check to run on yours.

- Author: Chad Priest
- Published: 2026-09-16
- Canonical URL: https://blog.vodou.ai/llm-calls-missing-from-agent-trace/
- Tags: ai-agents, observability, llm, architecture

---

If you run an agent in production, you probably have a trace per turn. You log the prompt, the tool calls and the reply. You may also keep a file that lists what goes into the context window: system prompt, memory, tool results, summaries. Do both of those match what your process actually sends to a model? I assumed ours did. Neither one did.

I build [Vodou](https://vodou.ai/register?utm_source=blog&utm_medium=feature&utm_campaign=workflows), a local-first AI system with memory, MCP tools and multi-step workflows. A workflow turns one sentence into a plan: resolved tools, parallel steps, checks with fresh context, and your approval before anything sends. Planning a workflow takes more than the one model call you see in chat. Some helpers write the step prose, some draft a skill, some generate trigger phrases. Each is a one-shot completion, and until September 3 none of them showed up on the turn that caused them.

## What a turn record has to hold before you can trust a plan

The idea is simple. When you approve a workflow plan, the turn record should hold every model call that shaped the plan, with its full prompt, since the model saw that prompt. It also has to say which world the run executes in. On a dev machine that is either the real install or an isolated test instance I break on purpose. A test harness whose output gets read as the live system is a bad kind of wrong.

![Infographic: same brain, it can also act. Skills, MCP tools, automations and workflows with approvals.](https://blog.vodou.ai/img/infographics/vodou-acts-on-your-stack.jpg)

Deciding the world used to be a guess based on the path. Now it follows one precedence order, in `MCP-servers/Vodou-Console/src/exec-world.ts`:

**Diagram: Which world is this process running in?**

Execution world resolution: operator override first, then the launching stack's declared world, then a path heuristic as a last resort

```text
  [Operator override] --set--> [local or lab (fixed)]
  [Operator override] --unset--> [Launching stack]
  [Launching stack] --declared--> [local or lab (fixed)]
  [Launching stack] --no declaration--> [Path heuristic (problem)]
  [Path heuristic (problem)] --> [local or lab (fixed)]

  notes:
    Operator override: explicit setting in the process env
    Launching stack: declared exec_world in stacks.toml
    Path heuristic: temp-dir project root means test lab

  The guess still exists, but it only answers when nobody declared anything.
```

Each process now carries the name of the stack that launched it. The build inspector reads that name from the live process environment (`ps -E` on macOS, `/proc` on Linux) instead of trusting a config file. It prints two things: the stack the process is actually running as, and the stack that declares it. It says `undeclared` only after it has read an environment and found nothing there. It prints a dash when there is no pid to ask. Those are two different facts, and the old code gave both the same blank.

## The milestone marked COMPLETE had three of five changes unbuilt

The build story isn't flattering. The plan for this work said "complete" in one section, while three of its five stated changes didn't exist. There were no stack keys, no way to start a stack by name, and the exec seam never read the stack registry. My own notes filed it as an issue: *P4 marked "COMPLETE" with three of five stated changes unbuilt.* The plan header was worse. It still said the later phases had "not started" while the tree showed several of them shipped. A reader who trusted the header would rebuild finished work, and one who trusted the body would skip unfinished work.

Then I followed the spec and got it wrong. The spec said each process row and each lane row should "gain `stacks = [...]`". I started writing that and stopped, because which processes belong to a stack is already written down once, in the stack registry. A second key on every row would copy one fact N times, and that is the kind of drift this whole plan exists to remove. So I didn't add the keys. The engine derives "which stacks is this process in" and "which world does this stack run in" from the one declaration, and tests pin that derivation. It was the only deliberate deviation from the spec, and I wrote it into the commit so nobody later "fixes" it back.

The lane registry was the bigger surprise. `lanes.toml` lists nine sources of context that can reach the model. Each lane has a budget, a trust label and an `emits` field: `log`, `receipt` (the user sees it, the log doesn't) or `none` (declared, written by nothing). On August 30 a source search had marked three lanes dead and two as receipt-only. On September 3 I re-measured against 3,148 logged inject rows instead of grepping for names.

**Diagram (beforeafter)**

Lane registry before re-measurement called hook_intent, lenses and rolling_summary dead and skill and automation receipt-only; after measurement hook_intent is wired, lenses and rolling_summary are retired, skill and automation are unexercised

```text
  BEFORE: Registry, from a source grep (Aug 30)
    - hook_intent: dead
    - lenses: dead
    - rolling_summary: dead
    - skill: receipt-only
    - automation: receipt-only

  AFTER: Registry, from 3,148 rows (Sep 3)
    - hook_intent: wired, bytes were inside another lane
    - lenses: retired, logged under system_prompt
    - rolling_summary: retired, logged as late context
    - skill: logs, but zero traffic since the log began
    - automation: logs, but zero traffic
```

None of the five labels held up. `hook_intent` was called dead because its name appeared nowhere in the source. Its bytes were reaching the model the whole time, inside the memory hook's payload, as a suggested-skill hint. Nobody gave them a name. `lenses` and `rolling_summary` were called dead, but their text was already logged under other lanes, so a separate lane would have counted the same bytes twice. I deleted them. `skill` and `automation` weren't receipt-only. Both have emitters that write to the log. The log simply had no rows, because no turn had taken those paths since logging began. The only two skill receipts ever recorded were older than the log.

Dead, unexercised and double-counted all look the same in a registry: a lane with no rows. Only measurement tells them apart.

## 23 one-shot completions, and a test that fails on the undeclared ones

Then the side calls. My notes said there were 33 raw LLM call sites. The real count was 23. The wrapper had always accepted a conversation id, and the pooled variant always carried one, but it only used it for a log line. Now, whenever it gets a conversation id, the wrapper writes a `tool/call` event with the full prompt and a `tool/result` event with the completion. Both are tagged with the world and the calling agent (step prose, skill author, trigger generator).

Nine sites really have no turn. The setup wizard, for example, runs before any conversation exists. I didn't ban those. I banned leaving them unexplained:

**Diagram: One-shot LLM call sites after the change**

Of 23 raw LLM call sites, 14 now log onto a turn and 9 are declared turnless

```text
  on a turn         : 14  ################################ (fixed)
  declared TURNLESS :  9  #####################
```

The gate lives in `MCP-servers/Vodou-Console/src/__tests__/oneshot-llm-gate.test.ts`. It walks every source file, finds each call, and fails unless the call passes a conversation id or has a `// TURNLESS: <why>` comment within the three lines above it. A new side call can't join the codebase silently. Its author has to either put it on the record or write down why it can't be.

## The invariant: every model-visible byte has a named producer that logs it

As a property you can check, it's this: for every call that sends text to a model, the text is written to the trace under a name, or the call site says in code why it isn't. The companion rule: whether a context source is dead gets decided from logged rows, never from a name search.

Both halves are true or false of a codebase today. An agent that takes the "simplest solution" approach from [Anthropic's agent guidance](https://www.anthropic.com/engineering/building-effective-agents) still ends up with a helper call that summarizes, classifies or rewrites. Those helpers are exactly the calls that tend to skip the tracing wrapper.

## Find your untraced model calls in five minutes

First, list every place your code calls a model client and count the ones that carry no trace or request id nearby. Change the call pattern and the id name to match your SDK:

```bash
grep -rnE "(chat\.completions\.create|messages\.create|generateText|invoke)\(" src \
  --include='*.ts' --include='*.py' -A8 \
  | awk '/(chat\.completions\.create|messages\.create|generateText|invoke)\(/{if(site&&!ok)print site; site=$0; ok=0} /trace_id|traceId|run_id|conversation_id|conversationId/{ok=1} END{if(site&&!ok)print site}'
```

Passing output is empty. Failing output is one line per call site whose next eight lines never mention an id. Each of those calls can put text in front of a model with no record attached.

Second, if your trace store has a span or event table, compare the context sources you declare with the ones that appear:

```sql
-- declared_sources: the names in your context config
SELECT d.name,
       COUNT(e.id)            AS rows_logged,
       MAX(e.created_at)      AS last_seen
FROM declared_sources d
LEFT JOIN trace_events e ON e.source = d.name
GROUP BY d.name
ORDER BY rows_logged ASC;
```

A row with `rows_logged = 0` isn't proof of dead code. Before you delete it, answer three questions. Does an emitter exist? Has any traffic taken that path since logging started? Are its bytes already logged under another name? Then do the reverse: `SELECT DISTINCT source FROM trace_events` minus your declared names. Anything left over reaches the model without a declaration.

## Procedural Graphs and skill papers audit structure, not the prompt bytes

Recent work is clear about structure. The [hierarchical skill architecture paper](https://arxiv.org/pdf/2607.11138.pdf) argues for auditability and capability boundaries. [Procedural Graphs](https://arxiv.org/abs/2609.09153) turns implicit procedure into explicit triples so agents stop calling tools out of order. [LangChain's multi-agent piece](https://www.langchain.com/blog/how-and-when-to-build-multi-agent-systems) says context engineering is the hard part. All three are about what the planner should see. None of them treats the planner's own helper completions as context that needs a record. Nor do they ask whether the document declaring the structure matches the running system. Our registry was a thoughtful declaration, and the measurement disagreed with it five times out of five.

## Still open: `skill` and `automation`, two lanes with zero rows

`skill` and `automation` are still unexercised. They have emitters, and I believe they log. But "believe" is exactly the word this post argues against, and neither lane has produced a single row since the log began. Workflows themselves are at Phase 1. Later phases are open, and the stack registry can declare a world for a process that `service` can't start yet (it names the owning command instead). Until a real automation-scoped turn runs, one registry line is still a claim.

If you want multi-step agent work where the plan, every model call that shaped it, and the world it runs in all land on one turn you can read before approving, that's what workflows in [vodou.ai](https://vodou.ai/register?utm_source=blog&utm_medium=feature&utm_campaign=workflows) are built to give you.

---

Source: [Our registry called three context lanes dead. One was live.](https://blog.vodou.ai/llm-calls-missing-from-agent-trace/) by Chad Priest, from Building Vodou in Public.