building vodou.

Hallucinated tool URLs don't error. They render empty cards

An LLM invented an Allrecipes URL, the site returned a 404 page, and our card rendered empty with no error. Why schema checks miss it, and a 5-minute test.

Chad Priest / / 4 min read

A user asked the chat for a potato salad recipe. The gateway rendered a recipe card with empty ingredients, empty steps and a small ⚠ some fields missing in the footer. There was no error and nothing turned red. The model had built the card from allrecipes.com/recipe/14838/potato-salad/, and that recipe does not exist.

/recipe/14838/potato-salad/ passed every check we had

My first read was wrong. Allrecipes has redesigned its markup twice in two years (the lens file says so in its header comment), and back in May I had written myself a note that card selectors break easily. “Some fields missing” is exactly what stale selectors look like, so I went hunting through cheerio selectors for a scraper bug that wasn’t there.

Nothing had searched for that URL. The model wrote it from memory. It looked right because Allrecipes URLs really are /recipe/<number>/<slug>/, and the model has seen thousands of them.

Here is how it got through. The lens at MCP-servers/Vodou-Console/src/lenses/recipe.allrecipes/index.ts has a validate() that asks two things: does the hostname end in allrecipes.com, and does the path contain /recipe/? Yes and yes. The URL matched the registered pattern *.allrecipes.com/recipe/**. Bytes came back. Then came this line: const { body } = await ctx.fetchStatic(sourceUrl);

fetchStatic in src/lenses/_lib/fetch_ctx.ts returns { status, body, headers }. The lens kept one of the three. Whatever status Allrecipes sent got thrown away. Cheerio parsed a not-found page, every ingredient selector matched zero nodes, and an empty array is a perfectly legal list of ingredients.

Validating against the real system doesn’t help when the fetch is the lookup

The Hallucinated Tool Argument That Passed Schema Validation gets the core point right: a schema checks shape, not existence. Our validate() was a shape check.

Bitsfolio’s piece on bad tool arguments gives the standard advice: validate identifiers against authoritative systems before execution, and make the model use only information it retrieved. The first half doesn’t cover a fetch tool, where running the tool is the lookup. We did ask the authoritative system, and it answered with a page. The failure survived because we parsed that answer without checking what it said.

We also did the second half. The system prompt in src/llm.ts now says a source_url must come from the user, from a tool result, or be something the model “know[s] with high confidence to exist”, and that recipes need an exa search first. I wrote that third clause myself, and it’s the hole. A model that invents /recipe/14838/ is confident. That’s how it happened. Prompt rules lower the rate, but they can’t catch a guess.

The fix that actually catches it: the lens now looks for the Allrecipes not-found template (<html id="404Template_1-0"> or a “page not found” title) and returns an error telling the model to search for a real URL. To be honest, that fix sniffs one site’s template, and the status code was sitting in the return value the whole time.

An ID the model typed is a guess until a tool returned it

The general problem: a model writes a resource identifier (a URL, an order ID, a file path, an entity ID) from training memory instead of taking it from a retrieval result. Then the code downstream treats “found nothing” as a valid empty result instead of an error. The same bug shows up in OpenAI or Anthropic function calling with a fetch_url tool, in LangChain and LlamaIndex agents with scrape tools, in MCP servers that take entity IDs, and in any generative UI where an empty list renders cleanly.

Every identifier in a tool call’s arguments must appear verbatim in the user’s message or in an earlier tool result in the same trace, or the call is recorded as ungrounded.

To check your own agent, point this at your stored traces. Adjust the field names to match your provider’s message format:

import json, re

ID = re.compile(r'https?://[^\s"\\<>]+|\b[A-Z]{2,5}-\d{4,}\b')

for trace in load_traces():               # your logger: ordered list of messages
    seen = ""
    for m in trace:
        if m["role"] in ("user", "tool"):  # Anthropic: tool_result blocks live in user turns
            seen += json.dumps(m["content"])
        for call in m.get("tool_calls") or []:
            for ident in ID.findall(json.dumps(call["arguments"])):
                if ident not in seen:
                    print("UNGROUNDED", call["name"], ident)

If it passes, it prints nothing, or only IDs you can explain. If it fails, you get lines like UNGROUNDED fetch_url https://.../recipe/14838/potato-salad/. For each one, run curl -s -o /dev/null -w '%{http_code}\n' '<url>' and open whatever your UI rendered for that call. If curl says 404 and your trace logged the tool call as a success, you have this bug. Then grep your fetch wrappers for const { body } or const { data } destructures that never mention status. That’s where the 404 disappeared for us.

When a tool returns nothing for an ID the model made up, show that as an error, not an empty result.