Your LLM provider list is five lists, and they already disagree

If your AI system supports more than one model provider, you probably have a list of providers. You probably have more than one. There’s the type union, or the enum. There’s a map of context windows the prompt builder reads, a price table the usage tracker reads, a switch that turns an id into a display name, and a hard-coded array in the settings page that decides which model dropdowns to fill. Each one was right on the day someone wrote it. Nothing makes them agree after that.

A missing entry doesn’t throw. It falls through to a default, and the default always looks reasonable: zero dollars, 200,000 tokens, the raw id as a label. Your dashboards stay green while the numbers under them are made up.

I built a single provider table into Vodou, a local-first AI OS that routes turns across about nineteen LLM providers. Before building it I counted, and what I found is the reason for this post.

19, 16, 18, 15, 17: the five lists, measured before any refactor

Before touching any code, I wrote a test that asked one question: do the five places that know about providers list the same ids? It reads the source files directly and pulls the ids out of each. It failed straight away.

Four missing ids from a price table sounds like a housekeeping chore. Here is what the gaps actually did.

The hosted tier we pay for reported $0.00 per turn, for weeks

vodou is our hosted, paid tier. It showed up in the settings menu and in the type union. It had no rows in the price table at all. The pricing lookup ends in ?? { input: 0, output: 0 }, so every turn through the one provider where we pay the upstream bill logged zero cost. The revenue side of the product had been reporting nothing, and nothing raised an error, because a zero is a valid price.

The fix taught me something too. The hosted tier is Fireworks underneath. Its default model is a Fireworks model path, and the display label already stripped that prefix. The obvious move was to copy the Fireworks price rows under a second id. I didn’t, because two tables holding the same numbers is exactly the drift I was trying to end. Next time Fireworks changes a price, someone would update one copy. So the row declares a pricingAlias pointing at fireworks, and the test checks behavior, not table contents: a million tokens through the hosted tier must cost more than $0, and must cost the same as Fireworks.

One provider, two context windows: 200,000 at two call sites, 64,000 at the third

vodou (and together) were also missing from the context-limit map. Three call sites read that map, and each had its own inline fallback. Two used 200,000. One used 64,000. Which context window the provider had depended on which line of code asked.

I added the real window (131,072 for the default model) and one named DEFAULT_CONTEXT_LIMIT, and I picked the conservative number for it. If you overestimate a window, the vendor truncates or rejects the prompt, and that failure is easy to miss. If you underestimate it, you compact a little early. The test now also checks that the fallback is a named constant and not a literal that just happens to match today, because three literals that matched once is how this started.

Once the consolidation was finished, six drift bugs had come out of it. Besides the two above: cheap-model routing was enabled and logged success while doing nothing on the default provider. The release secret scanner couldn’t recognize the shape of a Fireworks key. The settings page rejected API keys the runtime was already using. And the model dropdown read a different key than the actual call did, so it listed 24 Gemini models for a provider that couldn’t authenticate. The key bug was a twelve-arm validation switch where five arms checked exported environment keys and seven didn’t. The loader and the validator answered the same question two different ways.

A switch can’t be incomplete, so I used a table

A switch statement has no concept of coverage across files. Nothing forces the price switch and the label switch to handle the same ids. That’s how vodou got into the menu and the union while missing from pricing and the context map: no case was missing, because no case was ever required.

providers.ts is now one row per provider, and three of the old copies read from it:

export const PROVIDERS = [
  { id: 'fireworks', kind: 'openai-compat', label: 'Fireworks ({model})',
    contextLimit: 131_072, endpoint: 'https://api.fireworks.ai/inference/v1' },
  { id: 'vodou', kind: 'openai-compat', label: 'Vodou ({model})',
    contextLimit: 131_072, pricingAlias: 'fireworks', hostedTier: true,
    labelTrimPrefix: 'accounts/fireworks/models/' },
  // ...
] as const satisfies readonly ProviderSpec[];

The compiler proved the point right away. A filter I wrote to find rows missing a context window narrowed to never. The property is enforced by the type system instead of by a test.

The last copy was the frontend. settings.js had a literal array of seventeen ids and fetched models for each one. It was the copy missing custom. Now it asks /api/providers, which returns each row’s id, kind, label template, context limit, whether it’s local-only, and its default model. The endpoint deliberately returns no secrets and no per-user state. It says which providers exist, not which ones the operator has set up. The literal array stays behind as the .catch fallback: if that request fails, rendering a slightly stale list of model pickers beats rendering none. After a hard reload, the network log showed 18 model fetches, all 200, including custom.

The check that passed because of a newline

My first frontend check was “no literal array of provider ids remains in settings.js.” It went green. It went green because the regex didn’t match across newlines, and the fallback array spans three lines. The check passed only because of how the regex was written. I rewrote it to check what is actually true: there’s exactly one such array, and the 200 characters before it contain .catch(.

I also found six provider === 'vodou' checks that were really asking “is this the managed tier?” They covered quota enforcement, summary defaults and cache-prefix defaults. A second hosted tier would have quietly skipped all six. That question is now hostedTier: true on the row, answered by isHostedTier(). I checked that it returns false for fireworks (the user brings their own key and pays), for claude-cli, and for the empty string.

The invariant: a selectable provider id must never reach a fallback

This is the failure class, stated so you can check it:

For every provider id a user can select, every per-provider lookup (price, context window, label, endpoint, credential source) has to resolve from a real entry. No fallback branch should be reachable for a known id.

Fallbacks are fine for ids you don’t know about. For ids you ship, a fallback is a missing entry with a believable value filled in.

Run this against your own stack

This takes five minutes and needs nothing from our code. First, collect every provider id your UI can select (from the enum, the settings schema, or the frontend array). Then call your own lookups for each one, with the fallbacks turned into markers:

# check_providers.py: adapt the three imports to your codebase
from myapp.providers import SELECTABLE_IDS          # enum / union / UI list
from myapp.pricing import PRICING                    # dict keyed by provider
from myapp.context import CONTEXT_LIMITS             # dict keyed by provider

missing = []
for pid in sorted(SELECTABLE_IDS):
    if pid not in PRICING and not pid.startswith(("ollama", "local")):
        missing.append((pid, "price"))
    if pid not in CONTEXT_LIMITS:
        missing.append((pid, "context"))
for pid, fact in missing:
    print(f"FAIL {pid}: no {fact} entry, a fallback answers instead")
print("PASS" if not missing else f"{len(missing)} gaps")

If everything is covered it prints only PASS. If not, you get lines like FAIL together: no context entry, a fallback answers instead. If your fallbacks are inline literals, grep for them too. More than one distinct number is a finding by itself:

grep -rnE '\?\?\s*[0-9_]{5,}|\|\|\s*[0-9_]{5,}|get\([^)]*,\s*[0-9_]{5,}\)' src/ | grep -iE 'context|limit|window|token'

Then check the ledger, which is where a $0 bug actually shows up:

SELECT provider, COUNT(*) AS turns, SUM(input_tokens) AS tokens_in, SUM(cost_usd) AS cost
FROM llm_usage
WHERE created_at > datetime('now', '-30 days')
GROUP BY provider
HAVING SUM(input_tokens) > 0 AND COALESCE(SUM(cost_usd), 0) = 0;

Every row this returns should be a local model. Any hosted provider in the output is spending tokens at a price your system made up.

Context budgeting assumes the catalog is right

Most current advice on context windows is about the budget math. Mako’s provider-agnostic compaction PR derives a token budget from the catalog contextWindow minus system prompt, thinking and output reserve. That’s the right design, and it’s only as good as that one catalog field. With two copies of the catalog, the budget is right at one call site and wrong at another. The window isn’t even fixed per model: the Claude Code context budget notes describe the same Sonnet model with a 200,000 standard window and a separately gated 1M tier. Pricing works the same way. SitePoint’s tiered pricing breakdown shows prices changing at context thresholds and with cache writes, so a price is several numbers, and every extra copy is another place for them to drift. Anthropic’s advice to start with the simplest composable pattern applies here too. The simplest version of provider knowledge is one table, not a lookup that each module writes for itself.

Still open: 30 id comparisons and a price alias that trusts Fireworks

When the gate went in, 30 currentProvider === checks, 11 switches and 26 module globals per provider were still in place. Some were collapsed later, and some I left alone on purpose, because a claude-cli auth probe really is specific to that binary. The price alias assumes Fireworks keeps pricing by model family, and that needs rechecking whenever the hosted tier moves to a new model snapshot. One consolidated openai-compat path has also not been exercised live, because testing it means switching the global provider on a running system.

If you’d rather not keep five lists in sync yourself, vodou.ai routes across these providers from one table. The price, the context window and the settings page all read the same row.