fix(custom-model): isolate Claude config dir and inject real context length

Addresses two live-validation findings on the Run-menu custom-model picker:

1. Both claude.ai and ANTHROPIC_API_KEY set warning. Claude Code still
   coexists an OAuth login with an injected ANTHROPIC_API_KEY in the same
   config directory and warns about it (confirmed cosmetic - the API key
   wins for actual requests, verified via a real session's own API Usage
   Billing line). A custom-model claude session now gets an isolated
   CLAUDE_CONFIG_DIR (registry-declared via a new configDirVar field, empty,
   no files written into it) so there is nothing to conflict with. projects
   is symlinked (junction on Windows) back into the real config dir so the
   response viewer, subagent windows and Read My Mind keep working for that
   session, best-effort.

2. Context-window overflow. Claude Code assumes a large default context
   window for a model id it doesn't recognise and never compacts, so a
   custom endpoint's real, much smaller context (verified live: a 400
   exceeding a 16384-token llama-swap model with a stock ~33.7K-token system
   prompt) silently overflows. Discovery now also learns each model's real
   context length from llama.cpp/llama-swap's GET /props?model=<id> (n_ctx),
   but ONLY for a model llama-swap's own /v1/models response already marks
   status.value === 'loaded' - never an unloaded one, since llama-swap
   treats ?model= as a routing hint and probing an unloaded model risks
   triggering an actual, slow, GPU-swapping load as a side effect of
   read-only discovery. A server with no status field at all gets no
   enrichment rather than a guess; a model not probed this round keeps its
   previously-learned value until it disappears from the list entirely.
   Stored per model (CustomModelHost.modelContextLengths) and applied via a
   new contextLengthVar registry field, set to
   CLAUDE_CODE_MAX_CONTEXT_TOKENS for claude.

Both new fields live on the existing env-kind customModelInjection
capability shape, declared only on claude's registry entry - every other
CLI's injection is unaffected (pinned by test).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
This commit is contained in:
Devvyn
2026-09-16 12:08:26 +08:00
co-authored by Claude Sonnet 5
parent 5c25a52f95
commit 0e8b1981af
14 changed files with 504 additions and 20 deletions
+7
View File
@@ -340,6 +340,13 @@ const capabilitiesSchema = z
// an env var, so it declares baseUrl/apiKey injection with no model var at all.
modelVars: z.array(envName).max(8),
launchModel: launchModelTemplate,
// Optional: the env var to carry a discovered per-model context-window size
// (claude's CLAUDE_CODE_MAX_CONTEXT_TOKENS), and/or the env var that isolates
// this session's config/credential directory from the user's real one (claude's
// CLAUDE_CONFIG_DIR) so an injected API key never collides with a stored OAuth
// session. See the customModelInjection doc comment in cli-registry/types.ts.
contextLengthVar: envName.optional(),
configDirVar: envName.optional(),
})
.strict(),
z
+17
View File
@@ -237,6 +237,12 @@ const CLAUDE: CliEntry = {
'ANTHROPIC_DEFAULT_SONNET_MODEL',
'ANTHROPIC_DEFAULT_HAIKU_MODEL',
'ANTHROPIC_DEFAULT_OPUS_MODEL',
// CLAUDE_CODE_MAX_CONTEXT_TOKENS already matches the CLAUDE_CODE_* allowedPrefix, and
// CLAUDE_CONFIG_DIR is already an allowed exact key (docs/wiki/Agent-CLIs.md), so both
// were already reachable via plain envOverrides before this pair existed — listed here
// only so the custom-model route clamps them the same way as every other injected var.
'CLAUDE_CODE_MAX_CONTEXT_TOKENS',
'CLAUDE_CONFIG_DIR',
],
gates: { nameFlag: { minVersion: '2.1.224', failClosed: true } },
// Custom Model Endpoint Profiles (docs/custom-model-endpoints-plan.md) — verified by hand against a real
@@ -247,6 +253,17 @@ const CLAUDE: CliEntry = {
baseUrlVar: 'ANTHROPIC_BASE_URL',
apiKeyVar: 'ANTHROPIC_API_KEY',
modelVars: ['ANTHROPIC_DEFAULT_SONNET_MODEL', 'ANTHROPIC_DEFAULT_HAIKU_MODEL', 'ANTHROPIC_DEFAULT_OPUS_MODEL'],
// Verified via Claude Code's own docs: CLAUDE_CODE_MAX_CONTEXT_TOKENS overrides the
// assumed context window and applies directly for a model name Claude Code doesn't
// recognize as one of its own — exactly the custom-model case. Without it, Claude Code
// assumes a large (200k) window for any unrecognized model id and never compacts,
// eventually overflowing a much smaller real local context (see plan doc reasoning
// above the interface for the confirmed failure).
contextLengthVar: 'CLAUDE_CODE_MAX_CONTEXT_TOKENS',
// Isolates this session's config/credential directory so an injected ANTHROPIC_API_KEY
// never shares a directory with a stored claude.ai OAuth login — see the doc comment on
// customModelInjection in cli-registry/types.ts for the traded-off side effect.
configDirVar: 'CLAUDE_CONFIG_DIR',
},
},
overlays: {
+25 -1
View File
@@ -496,9 +496,33 @@ export interface CliCapabilities {
* declares). Absent = the config alone selects the model (claude's env vars,
* opencode's blob, codex's top-level `model` key). Applied by the session's
* respawn options through the entry's `legacyConfigField`, never by id.
*
* `contextLengthVar` (env kind only): the env var a discovered per-model context-window
* size is written to when known (claude's `CLAUDE_CODE_MAX_CONTEXT_TOKENS`) — without it,
* a CLI that assumes a large default window for an unrecognized model name keeps sending
* full-size prompts against a much smaller local server and eventually overflows its real
* context (verified: a 33.7K-token system prompt against a 16384-token llama-swap model).
* Absent when the CLI has no such override, or the value is unknown for this model.
*
* `configDirVar` (env kind only): the env var that redirects this session's config/
* credential directory to an isolated, per-session one (claude's `CLAUDE_CONFIG_DIR`), so
* an injected API key never coexists with a stored claude.ai OAuth session in the same
* directory — the CLI still warns "both claude.ai and ANTHROPIC_API_KEY set" when they
* share a directory even though the API key wins for actual requests. Isolating it trades
* that cosmetic warning for a documented side effect: a relocated config directory writes
* transcripts outside `~/.claude/projects`, blinding the response viewer, subagent
* windows, and Read My Mind for that session (see docs/wiki/Agent-CLIs.md).
*/
customModelInjection:
| { kind: 'env'; baseUrlVar: string; apiKeyVar: string; modelVars: string[]; launchModel?: string }
| {
kind: 'env';
baseUrlVar: string;
apiKeyVar: string;
modelVars: string[];
launchModel?: string;
contextLengthVar?: string;
configDirVar?: string;
}
| { kind: 'configContentEnv'; envVar: string; template: 'opencode-json'; launchModel?: string }
| {
kind: 'configDir';