# Your LLM provider list is five lists, and they already disagree > We counted provider ids in five places: 19, 16, 18, 15 and 17. Our paid tier billed $0 and one model had two context windows. Here's the check to run on yours. - Author: Chad Priest - Published: 2026-09-16 - Canonical URL: https://blog.vodou.ai/llm-provider-config-drift-five-lists/ - Tags: llm, architecture, ai-agents, observability --- If your AI system supports more than one model provider, you probably have a list of providers. You probably have more than one. There's the type union, or the enum. There's a map of context windows the prompt builder reads, a price table the usage tracker reads, a switch that turns an id into a display name, and a hard-coded array in the settings page that decides which model dropdowns to fill. Each one was right on the day someone wrote it. Nothing makes them agree after that. A missing entry doesn't throw. It falls through to a default, and the default always looks reasonable: zero dollars, 200,000 tokens, the raw id as a label. Your dashboards stay green while the numbers under them are made up. I built a single provider table into [Vodou](https://vodou.ai/register?utm_source=blog&utm_medium=feature&utm_campaign=providers-api-providers-the-frontend-stops-keeping-its-own-l), a local-first AI OS that routes turns across about nineteen LLM providers. Before building it I counted, and what I found is the reason for this post. ## 19, 16, 18, 15, 17: the five lists, measured before any refactor Before touching any code, I wrote a test that asked one question: do the five places that know about providers list the same ids? It reads the source files directly and pulls the ids out of each. It failed straight away. **Diagram: Provider ids per list, same codebase, same day** Provider ids known to each of five places: type union 19, display names 18, frontend list 17, context limits 16, pricing 15 ```text type union : 19 ids ################################ display names : 18 ids ############################## frontend list : 17 ids ############################# (problem) context limits : 16 ids ########################### (problem) pricing : 15 ids ######################### (problem) ``` Four missing ids from a price table sounds like a housekeeping chore. Here is what the gaps actually did. ## The hosted tier we pay for reported $0.00 per turn, for weeks `vodou` is our hosted, paid tier. It showed up in the settings menu and in the type union. It had no rows in the price table at all. The pricing lookup ends in `?? { input: 0, output: 0 }`, so every turn through the one provider where we pay the upstream bill logged zero cost. The revenue side of the product had been reporting nothing, and nothing raised an error, because a zero is a valid price. The fix taught me something too. The hosted tier is Fireworks underneath. Its default model is a Fireworks model path, and the display label already stripped that prefix. The obvious move was to copy the Fireworks price rows under a second id. I didn't, because two tables holding the same numbers is exactly the drift I was trying to end. Next time Fireworks changes a price, someone would update one copy. So the row declares a `pricingAlias` pointing at `fireworks`, and the test checks behavior, not table contents: a million tokens through the hosted tier must cost more than $0, and must cost the same as Fireworks. ## One provider, two context windows: 200,000 at two call sites, 64,000 at the third `vodou` (and `together`) were also missing from the context-limit map. Three call sites read that map, and each had its own inline fallback. Two used 200,000. One used 64,000. Which context window the provider had depended on which line of code asked. I added the real window (131,072 for the default model) and one named `DEFAULT_CONTEXT_LIMIT`, and I picked the conservative number for it. If you overestimate a window, the vendor truncates or rejects the prompt, and that failure is easy to miss. If you underestimate it, you compact a little early. The test now also checks that the fallback is a named constant and not a literal that just happens to match today, because three literals that matched once is how this started. Once the consolidation was finished, six drift bugs had come out of it. Besides the two above: cheap-model routing was enabled and logged success while doing nothing on the default provider. The release secret scanner couldn't recognize the shape of a Fireworks key. The settings page rejected API keys the runtime was already using. And the model dropdown read a different key than the actual call did, so it listed 24 Gemini models for a provider that couldn't authenticate. The key bug was a twelve-arm validation switch where five arms checked exported environment keys and seven didn't. The loader and the validator answered the same question two different ways. ## A switch can't be incomplete, so I used a table A switch statement has no concept of coverage across files. Nothing forces the price switch and the label switch to handle the same ids. That's how `vodou` got into the menu and the union while missing from pricing and the context map: no case was missing, because no case was ever required. `providers.ts` is now one row per provider, and three of the old copies read from it: ```ts export const PROVIDERS = [ { id: 'fireworks', kind: 'openai-compat', label: 'Fireworks ({model})', contextLimit: 131_072, endpoint: 'https://api.fireworks.ai/inference/v1' }, { id: 'vodou', kind: 'openai-compat', label: 'Vodou ({model})', contextLimit: 131_072, pricingAlias: 'fireworks', hostedTier: true, labelTrimPrefix: 'accounts/fireworks/models/' }, // ... ] as const satisfies readonly ProviderSpec[]; ``` The compiler proved the point right away. A filter I wrote to find rows missing a context window narrowed to `never`. The property is enforced by the type system instead of by a test. **Diagram (beforeafter)** Before: five independent provider lists with silent fallbacks. After: one table read by the backend and served to the frontend ```text BEFORE: Before - type union, 19 ids - context map, 16, fallback 200k or 64k - label switch, 18 - price table, 15, fallback $0 - settings.js array, 17, no custom AFTER: After - providers.ts: one row per provider - contextLimitFor(), providerLabel(), pricingAlias - GET /api/providers serves the rows - settings.js keeps a literal only as .catch fallback ``` The last copy was the frontend. `settings.js` had a literal array of seventeen ids and fetched models for each one. It was the copy missing `custom`. Now it asks `/api/providers`, which returns each row's id, kind, label template, context limit, whether it's local-only, and its default model. The endpoint deliberately returns no secrets and no per-user state. It says which providers exist, not which ones the operator has set up. The literal array stays behind as the `.catch` fallback: if that request fails, rendering a slightly stale list of model pickers beats rendering none. After a hard reload, the network log showed 18 model fetches, all 200, including `custom`. ## The check that passed because of a newline My first frontend check was "no literal array of provider ids remains in settings.js." It went green. It went green because the regex didn't match across newlines, and the fallback array spans three lines. The check passed only because of how the regex was written. I rewrote it to check what is actually true: there's exactly one such array, and the 200 characters before it contain `.catch(`. I also found six `provider === 'vodou'` checks that were really asking "is this the managed tier?" They covered quota enforcement, summary defaults and cache-prefix defaults. A second hosted tier would have quietly skipped all six. That question is now `hostedTier: true` on the row, answered by `isHostedTier()`. I checked that it returns false for `fireworks` (the user brings their own key and pays), for `claude-cli`, and for the empty string. ## The invariant: a selectable provider id must never reach a fallback This is the failure class, stated so you can check it: **For every provider id a user can select, every per-provider lookup (price, context window, label, endpoint, credential source) has to resolve from a real entry. No fallback branch should be reachable for a known id.** Fallbacks are fine for ids you don't know about. For ids you ship, a fallback is a missing entry with a believable value filled in. ## Run this against your own stack This takes five minutes and needs nothing from our code. First, collect every provider id your UI can select (from the enum, the settings schema, or the frontend array). Then call your own lookups for each one, with the fallbacks turned into markers: ```python # check_providers.py: adapt the three imports to your codebase from myapp.providers import SELECTABLE_IDS # enum / union / UI list from myapp.pricing import PRICING # dict keyed by provider from myapp.context import CONTEXT_LIMITS # dict keyed by provider missing = [] for pid in sorted(SELECTABLE_IDS): if pid not in PRICING and not pid.startswith(("ollama", "local")): missing.append((pid, "price")) if pid not in CONTEXT_LIMITS: missing.append((pid, "context")) for pid, fact in missing: print(f"FAIL {pid}: no {fact} entry, a fallback answers instead") print("PASS" if not missing else f"{len(missing)} gaps") ``` If everything is covered it prints only `PASS`. If not, you get lines like `FAIL together: no context entry, a fallback answers instead`. If your fallbacks are inline literals, grep for them too. More than one distinct number is a finding by itself: ```bash grep -rnE '\?\?\s*[0-9_]{5,}|\|\|\s*[0-9_]{5,}|get\([^)]*,\s*[0-9_]{5,}\)' src/ | grep -iE 'context|limit|window|token' ``` Then check the ledger, which is where a $0 bug actually shows up: ```sql SELECT provider, COUNT(*) AS turns, SUM(input_tokens) AS tokens_in, SUM(cost_usd) AS cost FROM llm_usage WHERE created_at > datetime('now', '-30 days') GROUP BY provider HAVING SUM(input_tokens) > 0 AND COALESCE(SUM(cost_usd), 0) = 0; ``` Every row this returns should be a local model. Any hosted provider in the output is spending tokens at a price your system made up. ## Context budgeting assumes the catalog is right Most current advice on context windows is about the budget math. Mako's [provider-agnostic compaction PR](https://github.com/mako-ai/mako/pull/541) derives a token budget from the catalog `contextWindow` minus system prompt, thinking and output reserve. That's the right design, and it's only as good as that one catalog field. With two copies of the catalog, the budget is right at one call site and wrong at another. The window isn't even fixed per model: the [Claude Code context budget notes](https://unpkg.com/aiwg@2026.9.5/docs/providers/claude-context-budget.md) describe the same Sonnet model with a 200,000 standard window and a separately gated 1M tier. Pricing works the same way. SitePoint's [tiered pricing breakdown](https://www.sitepoint.com/modeling-llm-context-costs-tiered-pricing-beyond-200k-tokens/) shows prices changing at context thresholds and with cache writes, so a price is several numbers, and every extra copy is another place for them to drift. Anthropic's [advice](https://www.anthropic.com/engineering/building-effective-agents?lang=en-US) to start with the simplest composable pattern applies here too. The simplest version of provider knowledge is one table, not a lookup that each module writes for itself. ## Still open: 30 id comparisons and a price alias that trusts Fireworks When the gate went in, 30 `currentProvider ===` checks, 11 switches and 26 module globals per provider were still in place. Some were collapsed later, and some I left alone on purpose, because a `claude-cli` auth probe really is specific to that binary. The price alias assumes Fireworks keeps pricing by model family, and that needs rechecking whenever the hosted tier moves to a new model snapshot. One consolidated openai-compat path has also not been exercised live, because testing it means switching the global provider on a running system. If you'd rather not keep five lists in sync yourself, [vodou.ai](https://vodou.ai/register?utm_source=blog&utm_medium=feature&utm_campaign=providers-api-providers-the-frontend-stops-keeping-its-own-l) routes across these providers from one table. The price, the context window and the settings page all read the same row. --- Source: [Your LLM provider list is five lists, and they already disagree](https://blog.vodou.ai/llm-provider-config-drift-five-lists/) by Chad Priest, from Building Vodou in Public.