feat: keep every Bot's provider defaults in one spec file

[Hieuej147]

What this changes

shared/model-providers.json is now the single place that names a provider's
facts — which environment variable carries its key, which carries its endpoint,
and which model runs when nothing else does — plus one row per Bot for the
provider and model it runs when its environment names neither. The TypeScript
Bots read it through shared/model-providers.ts, the ten Python Bots through
shared/model_providers.py, and any other language can read the file directly:
a spec file does not need a loader written for it.

• Both loaders validate the whole file at startup. A row missing a field, or a
  bots entry naming a provider the file does not know, stops the Bot with the
  path of the offending key rather than at its first model call; a missing
  bots entry throws rather than silently answering on somebody else's model.
• Precedence is exactly what docs/configuration.md already promised:
  BOT_PROVIDER and BOT_MODEL beat the file, the file beats the hard-coded
  default, AGENT_BOT_MODEL still applies to the proof-of-concept Bot on its
  own, and model_key() keeps splitting provider/model overrides.
• The defaults written into the file are the ones each Bot hard-coded before,
  so a deployment that changes nothing answers on the same models it did. The
  proof-of-concept and TypeScript LangGraph Bots keep gpt-5.5, the Mastra
  Bot keeps gpt-4o-mini, and the Python Bots keep gpt-4o-mini or gpt-5.5
  as each one was. Moving a Bot to a different model, or giving a Bot written in any
  other language its first one, is editing one row in one file instead of one
  line per language.
• API keys never live in the file: each provider row names the environment
  variable that carries its key, and every refusal message names that variable
  and where the provider row lives.
• Compose no longer substitutes gpt-5.5 for agent-langgraph whenever
  BOT_MODEL is unset: it passes the unset value through, so the Bot's row —
  or the moved provider's default row — is what answers. An OpenAI deployment
  keeps gpt-5.5 either way; a Google or Anthropic one stops being handed a
  model its vendor has never heard of. The picked harness keeps its own
  default, pinned by tests/compose.test.ts.
• Scope: the change covers the thirteen agent-* Bots — the processes that
  call a model provider themselves. The built-in Bots that run inside the
  server keep the server's own default (server/src/copilot.ts) and are not
  touched here. A remote_ag_ui coworker such as risk-analyst has no model
  of its own: it answers through the agent endpoint it forwards to (here
  agent-langgraph, whose row is in the file), so it inherits this change
  indirectly rather than gaining a row of its own.

Wired: three TypeScript Bots through botSettings(), ten Python Bots through
bot_settings(), ten new agent-*/tests/test_model_spec.py pinning each
wiring and its source-level binding, and the Dockerfiles copy the loader and
the file into the images (the TypeScript Bots already copy the whole shared/
directory, so they needed no change).

Where it runs

• New state that outlives a request? None — the file is read once at
  startup into process memory and never written.
• What happens on the second replica? Every replica reads the same
  read-only file from its own image; replicas never coordinate over it.
• Anything serialised? Nothing new is written, so nothing needs
  arbitrating.
• Anything fanned out to a browser? Nothing.
• New listener, port, or schedule? None.

Boundary and audit

• Every acting call still goes through the gateway — this change only reads
  configuration before serving, on the same path the Bots already used.
• New refusals and new failures each write a row — the startup validation
  throws before any request exists; it acts on nothing.
• Nothing new is trusted from the client that the server can resolve itself
  — the file ships inside the image and is never client input.

Changelog

• Two lines under Unreleased ("The Bots agree on one set of provider
  defaults" and "One spec file, in every language"), the second covering the
  Compose default fix.

Original contributor proof (before repair)

After merging main (146 files, @copilotkit/runtime 1.70.1 → 1.73.1,
bun install first):

1. bun run format:check (962 files), bun run lint (977 files, no warnings),
   bun run typecheck (app, server, worker), bun run build — all clean.
2. bun run test:ci — 5355 pass, 0 fail, 38 skip across 415 files, floor
   of 400 tests satisfied.
3. pytest agent-*/tests/test_model_spec.py across the ten Python harnesses —
   40/40, four assertions each: provider and model defaulting, env beating
   file, and the AGENT_BOT_MODEL override with its separation.
4. bun test tests/compose.test.ts — 17 pass: the Compose default is now
   an empty BOT_MODEL, the picked-harness default is untouched, and the
   rendered YAML for the Unset case shows BOT_MODEL: "".
5. docker build of agent-mastra and agent-adk (both sources changed by the
   merge) — succeed; the merged agent-langgraph image rebuilt and picked the
   spec file up from shared/.
6. Live stack: with BOT_PROVIDER=google and a real key, agent-langgraph
   answered /health with {"provider":"google","model":...} straight from
   the file with no BOT_MODEL set, and answered a chat through the managed
   coworker on the first run (Risk Analyst persona, 17 x 23 = 391). Both
   Google candidates, gemini-2.5-flash and gemini-3.1-flash-lite, verified
   HTTP 200 against the same key.
7. Final-tree check after the merge, .env restored to its committed state and
   no BOT_MODEL anywhere: /health reports
   {"status":"ok","provider":"openai","model":"gpt-5.5"} — the value from
   agent-langgraph's row in the spec file, not from a Compose substitution,
   which is the same model an OpenAI deployment answered with before this
   change.

Local environment notes, not code: the start script's Google gap and the
AGENT_COMPUTER_ALLOW_PRIVATE_HOSTS flag for local coworker endpoints are
pre-existing behaviour, documented in the test session rather than changed
here.

Repair verification

The Python LangGraph harness now passes the spec-resolved model into its SDK, including edited Bot rows and provider defaults. The existing google_genai alias, explicit BOT_MODEL values, provider-qualified model names, and ChatGPT authentication branch are preserved. Mastra retains its prior gpt-4o-mini default; this consolidation does not change its default model or cost.

Verified locally on the repaired tree:

• Full Python LangGraph suite: 90 passed, including 13 new cases that check real Anthropic SDK request bodies and Google loopback request paths. The initial Anthropic regression run demonstrated five failures before the runtime fix.
• Shared provider and Mastra tests: 62 passed, including Mastra's absent/empty/whitespace model defaults.
• Full repository format and lint checks passed; scoped TypeScript --noEmit and Python compilation passed.
• Mastra's production CLI build passed.

The original contributor's broader validation above predates this repair. Current CI results should be read from this PR's checks.

[Hotragn]

Nobody has reviewed this yet and it is the kind of change where the risk is not in any one file, so here is a pass over the parts that a 13-runtime consolidation usually gets wrong. Two of the three came out clean, and I want to say so rather than only list problems.

Checked and clean

Packaging. My first worry was an image that ships the reader and not the data. All eleven changed Dockerfiles copy both (shared/model_providers.py + shared/model-providers.json, and shared/model-providers.ts + the JSON for Mastra), and agent-bot and agent-langgraph need no change because they already COPY shared ./shared wholesale. shared/model-providers.ts reaches the data through a static import specJson from "./model-providers.json", so it has to be on disk in the image, and it is.

The defaults match main, bot for bot. I extracted each Bot's current default and compared against the bots table: all thirteen agree, including the split between the gpt-4o-mini Bots and the gpt-5.5 ones. The one change is agent-mastra, and you disclosed it in both the changelog and the PR body, so that is a decision rather than a slip. I will only note that it is the single behaviour change in a PR framed as consolidation — every deployment running the Mastra Bot with no BOT_MODEL moves model and cost on upgrade — and it would be easy for a maintainer to approve the refactor and not register that. It might deserve its own line at the top of the entry rather than a parenthesis.

providers.*.default_model is reachable, which I checked because a brand-new contract file with a dead field ages badly. modelFor falls back to it when the environment moves the provider off the Bot's own row (model-providers.ts:266). Live and load-bearing.

The two loaders do not enforce the same contract

This is the one I would want fixed before it lands, because it is invisible until the file changes and the file is now the thing everybody edits.

The module doc says the JSON "is the contract every language in the box reads" and that the loaders add "types and validation". They add different validation, because they answer to different authorities:

	TypeScript (model-providers.ts)	Python (model_providers.py)
Which providers must exist	all of PROVIDER_IDS — openai, anthropic, google	openai only (:70)
A row not in that list	refuses: "providers has a … row this module has never heard of" (:117-121)	accepted, validated, ignored
bots.*.provider checked against	the hardcoded PROVIDER_IDS	the file's own providers map (:82)

Two consequences, both reachable by the workflow this PR is for:

Adding a provider breaks the TypeScript Bots. The _readme says "providers = facts per provider" and the changelog says a new provider is a row in one file. Add a mistral row and the ten Python Bots take it happily, while all three TypeScript Bots refuse to start. The TS module docblock does say adding one means "one entry to PROVIDER_IDS and one row to the JSON file" — so this is deliberate on that side — but a contributor working in Python, following the _readme, never reads that module. Whatever the rule is, the JSON's own _readme should state it, because that string is what the next person reads.

Removing a provider is silent on the Python side. Drop the anthropic row and the TS Bots stop at startup naming the missing key, which is precisely the behaviour the docblock argues for: "stops the process at startup with the path of the offending key rather than at the first model call." The Python Bots start clean, and a deployment on BOT_PROVIDER=anthropic then fails at its first model call — the exact failure this validation exists to prevent, on ten of the thirteen Bots.

The comment at model_providers.py:70 reads "The same fallback the TypeScript loader makes", which is true of the fallback and not of the validation around it, and I think that sentence is how the gap survived review by its own author.

Either direction fixes it. Give Python the same explicit tuple and check both ways; or drop PROVIDER_IDS and have TypeScript derive its union from the file the way Python does, which costs the compile-time type. I would take the first — the whole point of the file is that a mistake in it stops every Bot the same way, and "every Bot" currently means three of them.

One small thing

The two changelog entries contradict each other. The first says the Mastra Bot's default moved from gpt-4o-mini to gpt-5.5; the second says "the defaults themselves are unchanged". I can see that the second means moving to the file changed nothing by itself, but they sit in the same Unreleased section, and the top of CHANGELOG.md says it is written "for somebody deciding whether to upgrade". A reader who hits the second sentence has been told the thing they most need to check is not worth checking.

I have not run this: thirteen runtimes, ten of them Python, and I have no Python toolchain set up for this repo. The claims above are all read off the diff and off main, and I would rather be told I have misread one than have you find out I guessed.

[Hieuej147]

@Hotragn Thanks — the loader gap was the one I wanted fixed too, and both changelog points are in.

The loaders now enforce the same contract. shared/model_providers.py keeps its own PROVIDER_IDS and checks the file both ways in the same words: providers is missing its anthropic row., providers has a mistral row this module has never heard of., and bots.agent-x.provider is "mistral", not one of openai, anthropic, google. A bots entry now checks against the list rather than against the file's own providers map — the same check once the two agree. The comment you flagged sat on the openai-only check and described the fallback rather than the validation beside it; it goes with the check.

New agent-langgraph-agui/tests/test_spec_validation.py: the shipped file accepted, each of the three provider rows dropped in turn, an unknown row, and a bots entry naming an unknown provider. Each case imports the loader in a subprocess against a file in tmp_path, so a refusal cannot leave a half-loaded module beside the tests beside it. It lives in that harness's tests because that is where CI already runs pytest for anything under shared/. The rule is in the JSON's _readme, both module docstrings, and an "Adding a provider" paragraph in docs/configuration.md beside "Adding a Bot".

Changelog: the Mastra line is now the first line of its entry, with what a deployment running that Bot without BOT_MODEL should do. The second entry says reading the file moved no default on its own and points at the one that did move.

Two things I would read back differently: providers.*.default_model is reachable as you say, but the function is botSettings (model-providers.ts:243-269, the fallback at :266) — there is no modelFor. And on "silent on the Python side": shared/model-providers.test.ts:313-320 asserts Object.keys(MODEL_PROVIDERS) equals PROVIDER_IDS and readSpec() throws at module load, so adding or dropping a provider row already failed CI for the TypeScript Bots. What was missing is the part you are really pointing at — the docs/configuration.md guarantee ("stops every Bot at startup") held for three of them, and a Python Bot's own startup did not enforce it. The tuple fixes that.

Verified locally: format:check, lint, typecheck; bun test shared/ tests/compose.test.ts (106 pass); pytest, 50 pass across test_spec_validation.py and the ten test_model_spec.py; bun run test against a stashed baseline, unchanged at 4376 pass / 60 fail before and after, every failure the missing TEST_DATABASE_URL on this machine.

CI has been sitting at action_required since the first push with 0 jobs — could a maintainer approve the workflow run? I'll post the result here once it goes.