tag
caching
2 posts.
-
Why My LLM Agent Fabricated Numbers From Stale Context
An agent replayed a ten-minute-old CPU reading as live. The bug wasn't the model: a guard checked whether cached tool output existed, not whether it was fresh.
-
cached_tokens is 0 because your system prompt isn't stable
Our provider reported cached_tokens 0 of ~16K prompt tokens every turn. The cause: per-turn memory glued into the system prompt. Freeze the prefix instead.