mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-04 14:39:42 +02:00
feat(custom-model): Custom Model Endpoint Profiles (local or cloud, all harnesses)
Point any Codeman-supported harness (Claude, opencode, Codex, Gemini, Pi, Grok, DeepSeek, OMP) at a custom OpenAI-compatible endpoint instead of its native cloud backend, for a given session. Covers local hardware (llama.cpp, Ollama, vLLM, DGX Spark, Strix Halo) and cloud (Azure AI Foundry, OpenRouter). Off by default (customModelEndpointsEnabled, synced, default OFF). - Registry: capabilities.customModelInjection per CLI entry (env / configContentEnv / configDir / unsupported kinds) - Pure injection builder (custom-model-injection.ts) turning an endpoint + model id into the real env vars / config content per CLI - Endpoint store + CRUD routes (custom-model-hosts.ts, custom-model-routes.ts), discovery via GET /v1/models, SSRF-guarded - Session integration: Session.setCustomModel()/restartCli() (POST /api/sessions/:id/custom-model), reusing the existing respawn-pane -k primitive to restart the CLI process with new env - Multi-user hardening: every new redirect-capable env var added to its CLI's privilegedEnvKeys, closing a pre-existing gap where several were already reachable via the generic envOverrides field's prefix allowlist - Standalone scripts/test-local-llm-harnesses.mjs: spawns real CLI binaries against a real endpoint outside the web UI, independent of tmux/sessions - Mock-server contract tests (test/fixtures/mock-openai-server.ts) replaying every CLI's injected values through a real HTTP shape Real end-to-end validation against a live llama-swap server (inside a codeman/agent:llm-test Docker image with all 9 CLI binaries) found and fixed three real bugs before they shipped: - Codex's config.toml schema was wrong ([model].default table instead of a top-level model string + [model_providers.custom]); fixing it then surfaced a genuine, documented protocol incompatibility (Codex only speaks the Responses API since Feb 2026, which llama.cpp/llama-swap don't implement) - Claude Code's async session-title-generation call validates ANTHROPIC_DEFAULT_HAIKU_MODEL against its own internal model list and hangs the whole -p invocation on an unrecognized name; documented for chunk 6, worked around in the standalone script only (--bare is NOT safe for a real interactive session, which needs hooks) - The discovery route's authStyle: 'both' option (send both Authorization and api-key headers) reliably hung a real server; removed the option entirely rather than just changing the default Status: draft. Chunk 6 (frontend toolbar/settings UI) not yet built — see PR.md and deployment_plan.md for the full chunk breakdown and confidence table. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017HqNWfmtBU2KN29SvSVWB3
This commit is contained in:
Devvyn
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
a017e9a8e0
commit
41416566aa
@@ -310,6 +310,34 @@ const capabilitiesSchema = z
|
||||
privilegedEnvKeys: z.array(envName).max(8),
|
||||
gates: z.record(z.string(), z.object({ minVersion: z.string().max(20), failClosed: z.boolean() }).strict()),
|
||||
maxFrameBytes: z.number().int().positive().optional(),
|
||||
customModelInjection: z.discriminatedUnion('kind', [
|
||||
z
|
||||
.object({
|
||||
kind: z.literal('env'),
|
||||
baseUrlVar: envName,
|
||||
apiKeyVar: envName,
|
||||
// Empty is valid: deepseek's model routing is a profile-composition concern, not
|
||||
// an env var, so it declares baseUrl/apiKey injection with no model var at all.
|
||||
modelVars: z.array(envName).max(8),
|
||||
})
|
||||
.strict(),
|
||||
z
|
||||
.object({
|
||||
kind: z.literal('configContentEnv'),
|
||||
envVar: envName,
|
||||
template: z.literal('opencode-json'),
|
||||
})
|
||||
.strict(),
|
||||
z
|
||||
.object({
|
||||
kind: z.literal('configDir'),
|
||||
dirEnvVar: envName,
|
||||
fileName: z.string().min(1).max(80),
|
||||
template: z.enum(['codex-toml', 'pi-models-json', 'omp-models-yml']),
|
||||
})
|
||||
.strict(),
|
||||
z.object({ kind: z.literal('unsupported') }).strict(),
|
||||
]),
|
||||
})
|
||||
.strict();
|
||||
|
||||
|
||||
@@ -178,6 +178,12 @@ const CLAUDE: CliEntry = {
|
||||
unset: ['CLAUDECODE', 'COLORTERM'],
|
||||
tmuxSetenvKeys: [],
|
||||
dockerExecEnvNames: [],
|
||||
// Deliberately excludes ANTHROPIC_* (base URL / API key / default-model overrides):
|
||||
// custom-model-injection.ts's claude recipe uses those names, but they must reach a
|
||||
// session ONLY through the admin-configured, SSRF-guarded custom-model route, never
|
||||
// through a plain client-supplied envOverrides field. Widening this prefix would let
|
||||
// any session-create caller redirect a session's Anthropic traffic and credentials to
|
||||
// an arbitrary, unvalidated URL.
|
||||
allowedPrefixes: ['CLAUDE_CODE_'],
|
||||
allowedKeys: ['CLAUDE_CONFIG_DIR'],
|
||||
},
|
||||
@@ -209,8 +215,28 @@ const CLAUDE: CliEntry = {
|
||||
statusLineTelemetry: true,
|
||||
model: { source: 'claude-settings-file' },
|
||||
privilegedParams: [],
|
||||
privilegedEnvKeys: [],
|
||||
// ANTHROPIC_* is NOT in allowedPrefixes/allowedKeys above (deliberately — see the
|
||||
// allowedPrefixes comment nearby), so these are unreachable via plain envOverrides
|
||||
// today; listed here only so the dedicated custom-model route (deployment_plan.md
|
||||
// chunk 5) clamps them for a non-granted multi-user owner the same way every other
|
||||
// CLI's injection vars are clamped, the day that route widens who can set them.
|
||||
privilegedEnvKeys: [
|
||||
'ANTHROPIC_BASE_URL',
|
||||
'ANTHROPIC_API_KEY',
|
||||
'ANTHROPIC_DEFAULT_SONNET_MODEL',
|
||||
'ANTHROPIC_DEFAULT_HAIKU_MODEL',
|
||||
'ANTHROPIC_DEFAULT_OPUS_MODEL',
|
||||
],
|
||||
gates: { nameFlag: { minVersion: '2.1.224', failClosed: true } },
|
||||
// Custom Model Endpoint Profiles (deployment_plan.md) — verified by hand against a real
|
||||
// llama.cpp server. Claude reads these at process start only, so switching requires a
|
||||
// respawn, never a live hot-swap.
|
||||
customModelInjection: {
|
||||
kind: 'env',
|
||||
baseUrlVar: 'ANTHROPIC_BASE_URL',
|
||||
apiKeyVar: 'ANTHROPIC_API_KEY',
|
||||
modelVars: ['ANTHROPIC_DEFAULT_SONNET_MODEL', 'ANTHROPIC_DEFAULT_HAIKU_MODEL', 'ANTHROPIC_DEFAULT_OPUS_MODEL'],
|
||||
},
|
||||
},
|
||||
overlays: {
|
||||
// Mirrors the local default so the remote/in-container agent runs non-interactively
|
||||
@@ -276,6 +302,7 @@ const SHELL: CliEntry = {
|
||||
privilegedParams: [],
|
||||
privilegedEnvKeys: [],
|
||||
gates: {},
|
||||
customModelInjection: { kind: 'unsupported' }, // a raw shell has no "model" concept
|
||||
},
|
||||
overlays: {
|
||||
// No `remote` entry: defaultRemoteCommandForMode special-cases kind==='shell' directly
|
||||
@@ -355,6 +382,15 @@ const OPENCODE: CliEntry = {
|
||||
...agentDefaults(),
|
||||
altScreen: 'strip-mux-only',
|
||||
echo: { policy: 'buffer', anchor: { kind: 'cursor' }, predictProfile: undefined },
|
||||
// Verified by hand against a real llama.cpp server. Reuses the SAME env var opencode's
|
||||
// own `env.configContentVar` already declares — the builder in custom-model-injection.ts
|
||||
// must merge into whatever opencode config Codeman would otherwise send, not clobber it.
|
||||
customModelInjection: { kind: 'configContentEnv', envVar: 'OPENCODE_CONFIG_CONTENT', template: 'opencode-json' },
|
||||
// OPENCODE_CONFIG_CONTENT already matches the OPENCODE_ allowedPrefix above, so it was
|
||||
// ALREADY reachable via plain envOverrides before this feature existed — it replaces
|
||||
// opencode's whole config, provider api keys included, so a non-granted multi-user owner
|
||||
// sending it is a pre-existing credential-redirection gap, not one this feature opens.
|
||||
privilegedEnvKeys: ['OPENCODE_CONFIG_CONTENT'],
|
||||
},
|
||||
overlays: {
|
||||
credStore: { rel: '.config/opencode', seedWhole: true },
|
||||
@@ -444,6 +480,23 @@ const CODEX: CliEntry = {
|
||||
// `dangerouslyBypassApprovals` on the wire), so it is the one that would have caught a
|
||||
// regression; `schema.ts` now rejects a name that is not a declared param.
|
||||
privilegedParams: [{ param: 'bypassApprovals', clampTo: false }],
|
||||
// Verified by hand against a real llama.cpp server. Written to an isolated CODEX_HOME
|
||||
// so the user's real ~/.codex/config.toml is never touched.
|
||||
customModelInjection: {
|
||||
kind: 'configDir',
|
||||
dirEnvVar: 'CODEX_HOME',
|
||||
fileName: 'config.toml',
|
||||
template: 'codex-toml',
|
||||
},
|
||||
// CODEX_HOME already matches the CODEX_ allowedPrefix above, so it was ALREADY
|
||||
// reachable via plain envOverrides before this feature existed. It is arguably
|
||||
// MORE sensitive than a bare base-url var: a redirected CODEX_HOME points codex at a
|
||||
// config.toml a non-granted owner fully controls, which can restate sandbox/approval
|
||||
// policy INSIDE that file — a path the argv-level `bypassApprovals` clamp above
|
||||
// cannot see or stop.
|
||||
// CODEMAN_CUSTOM_MODEL_API_KEY: the credential config.toml's env_key references
|
||||
// (see custom-model-injection.ts) — same reasoning as CODEX_HOME above.
|
||||
privilegedEnvKeys: ['CODEX_HOME', 'CODEMAN_CUSTOM_MODEL_API_KEY'],
|
||||
},
|
||||
overlays: {
|
||||
credStore: {
|
||||
@@ -527,6 +580,20 @@ const GEMINI: CliEntry = {
|
||||
// MATERIALIZE a config (not just touch an already-sent one) or a non-granted owner who
|
||||
// sends no geminiConfig at all would still get yolo for free.
|
||||
privilegedParams: [{ param: 'approvalMode', clampTo: 'auto_edit', materializeWhenAbsent: true }],
|
||||
// Web-researched, unverified — needs a restart to pick up (CLI reads these at process
|
||||
// start). Confirm the exact model-override env var name against the installed
|
||||
// gemini-cli version before shipping.
|
||||
customModelInjection: {
|
||||
kind: 'env',
|
||||
baseUrlVar: 'GOOGLE_GEMINI_BASE_URL',
|
||||
apiKeyVar: 'GEMINI_API_KEY',
|
||||
modelVars: ['GEMINI_MODEL'],
|
||||
},
|
||||
// All three already match the GEMINI_/GOOGLE_ allowedPrefixes above, so they were
|
||||
// ALREADY reachable via plain envOverrides before this feature existed — a non-granted
|
||||
// multi-user owner redirecting a gemini session's endpoint/credentials is a
|
||||
// pre-existing gap this feature's analysis surfaced, not one it opens.
|
||||
privilegedEnvKeys: ['GOOGLE_GEMINI_BASE_URL', 'GEMINI_API_KEY', 'GEMINI_MODEL'],
|
||||
},
|
||||
overlays: {
|
||||
credStore: { rel: '.gemini', seedWhole: true }, // also covers antigravity — see its own entry
|
||||
@@ -592,6 +659,10 @@ const ANTIGRAVITY: CliEntry = {
|
||||
// Like codex: an ABSENT config already defaults safe (no bypass flag), so only a
|
||||
// SENT config needs the flag forced off — nothing is materialized.
|
||||
privilegedParams: [{ param: 'dangerouslySkipPermissions', clampTo: false }],
|
||||
// No known CLI/env/config mechanism — Antigravity's own docs describe a GUI-only
|
||||
// custom-endpoint setting and explicitly say it "cannot currently" become the core
|
||||
// reasoning model. Toolbar entry stays disabled for this mode.
|
||||
customModelInjection: { kind: 'unsupported' },
|
||||
},
|
||||
overlays: {
|
||||
// No credStore of its own: agy nests its whole state under ~/.gemini/antigravity-cli/,
|
||||
@@ -678,6 +749,20 @@ const PI: CliEntry = {
|
||||
// just answer "yes" to, so omitting --approve is not itself a clamp — MATERIALIZE
|
||||
// approveProjectTrust:false so buildPiCommand emits --no-approve outright.
|
||||
privilegedParams: [{ param: 'approveProjectTrust', clampTo: false, materializeWhenAbsent: true }],
|
||||
// Web-researched, unverified. pi's models.json hot-reloads, but this feature always
|
||||
// restarts the CLI on switch for consistency with the other 8 harnesses. Written to an
|
||||
// isolated PI_CONFIG_DIR so the user's real ~/.pi/agent/models.json is never touched.
|
||||
customModelInjection: {
|
||||
kind: 'configDir',
|
||||
dirEnvVar: 'PI_CONFIG_DIR',
|
||||
fileName: 'agent/models.json',
|
||||
template: 'pi-models-json',
|
||||
},
|
||||
// PI_CONFIG_DIR already matches the PI_ allowedPrefix above, so it was ALREADY
|
||||
// reachable via plain envOverrides before this feature existed — and pi executes
|
||||
// repo-local .pi/extensions TypeScript (see the External CLI modes note in CLAUDE.md),
|
||||
// so redirecting this dir is a code-execution surface, not just a config swap.
|
||||
privilegedEnvKeys: ['PI_CONFIG_DIR'],
|
||||
},
|
||||
overlays: {
|
||||
credStore: {
|
||||
@@ -773,6 +858,16 @@ const GROK: CliEntry = {
|
||||
// already its safe interactive ask-mode, so the multi-user clamp only needs to force an
|
||||
// EXPLICITLY-SENT bypass flag back off — nothing is materialized when config is absent.
|
||||
privilegedParams: [{ param: 'alwaysApprove', clampTo: false }],
|
||||
// Web-researched, unverified.
|
||||
customModelInjection: {
|
||||
kind: 'env',
|
||||
baseUrlVar: 'GROK_BASE_URL',
|
||||
apiKeyVar: 'XAI_API_KEY',
|
||||
modelVars: ['GROK_MODEL'],
|
||||
},
|
||||
// All three already match the GROK_/XAI_ allowedPrefixes above, so they were ALREADY
|
||||
// reachable via plain envOverrides before this feature existed.
|
||||
privilegedEnvKeys: ['GROK_BASE_URL', 'XAI_API_KEY', 'GROK_MODEL'],
|
||||
},
|
||||
overlays: {
|
||||
// ~/.grok also holds sessions/, memory/, downloads/ (the ~160MB binary), completions/,
|
||||
@@ -924,7 +1019,19 @@ const DEEPSEEK: CliEntry = {
|
||||
// The half no other CLI needs. `DSH_*` is an allowlisted envOverrides prefix and
|
||||
// applyEnvOverrides() runs LAST, so without this a non-granted owner could send
|
||||
// DSH_PERMISSION_MODE on the same request and land after the config clamp.
|
||||
privilegedEnvKeys: ['DSH_PERMISSION_MODE', 'DSH_HOME', 'DEEPSEEK_BASE_URL'],
|
||||
// DEEPSEEK_API_KEY added alongside DEEPSEEK_BASE_URL for the custom-model-injection.ts
|
||||
// recipe (deployment_plan.md) — the pair travels together, same reasoning as base URL.
|
||||
privilegedEnvKeys: ['DSH_PERMISSION_MODE', 'DSH_HOME', 'DEEPSEEK_BASE_URL', 'DEEPSEEK_API_KEY'],
|
||||
// Web-researched, unverified, partial: reuses the already-existing DEEPSEEK_BASE_URL/
|
||||
// DEEPSEEK_API_KEY keys above. No modelVars — dsh's model is a profile-composition
|
||||
// entry (see `model: { source: 'none' }` above), not an env var, so forcing a specific
|
||||
// model name may not fully work; verify against a real profile before shipping.
|
||||
customModelInjection: {
|
||||
kind: 'env',
|
||||
baseUrlVar: 'DEEPSEEK_BASE_URL',
|
||||
apiKeyVar: 'DEEPSEEK_API_KEY',
|
||||
modelVars: [],
|
||||
},
|
||||
},
|
||||
overlays: {
|
||||
// No credStore: dsh keeps everything under $DSH_HOME (default ~/.dsh), which is
|
||||
@@ -1026,7 +1133,20 @@ const OMP: CliEntry = {
|
||||
// Where omp resolves its auth from. No known concrete exfiltration path today (omp
|
||||
// forwards no operator-held key into a pane), but a non-granted owner redirecting where
|
||||
// a shared multi-tenant deployment resolves auth is not something to allow silently.
|
||||
privilegedEnvKeys: ['OMP_AUTH_BROKER_URL', 'OMP_AUTH_BROKER_TOKEN'],
|
||||
// PI_CONFIG_DIR added for custom-model-injection.ts's omp recipe, which reuses pi's
|
||||
// dir-redirect mechanism (see the customModelInjection comment below) — already
|
||||
// reachable via the PI_ allowedPrefix (pi's own entry), so this closes the same
|
||||
// pre-existing gap for an omp session that PI's own entry closes for a pi session.
|
||||
privilegedEnvKeys: ['OMP_AUTH_BROKER_URL', 'OMP_AUTH_BROKER_TOKEN', 'PI_CONFIG_DIR'],
|
||||
// Web-researched, unverified. omp's ~/.omp tree is itself relocatable via PI_CONFIG_DIR
|
||||
// (see the DeepSeek/OMP note in CLAUDE.md), so this reuses that same redirect rather
|
||||
// than inventing an OMP-specific dir env var.
|
||||
customModelInjection: {
|
||||
kind: 'configDir',
|
||||
dirEnvVar: 'PI_CONFIG_DIR',
|
||||
fileName: 'agent/models.yml',
|
||||
template: 'omp-models-yml',
|
||||
},
|
||||
},
|
||||
overlays: {
|
||||
// `~/.omp/agent` also holds agent.db/history.db/models.db (SQLite caches) and
|
||||
|
||||
@@ -441,6 +441,37 @@ export interface CliCapabilities {
|
||||
gates: Record<string, { minVersion: string; failClosed: boolean }>;
|
||||
/** Cap on a single terminal frame, when this CLI needs a tighter one than the default. */
|
||||
maxFrameBytes?: number;
|
||||
/**
|
||||
* How this CLI is pointed at a user-supplied custom OpenAI-compatible
|
||||
* endpoint (local, e.g. llama.cpp, or cloud, e.g. Azure AI Foundry) — the
|
||||
* Custom Model Endpoint Profiles feature (`deployment_plan.md`). Declared
|
||||
* per entry, never branched on id, same as every other capability here.
|
||||
*
|
||||
* `env`: plain env vars (claude's `ANTHROPIC_BASE_URL`/`ANTHROPIC_API_KEY`/
|
||||
* `ANTHROPIC_DEFAULT_*_MODEL`). `configContentEnv`: a full config blob
|
||||
* carried in one env var (opencode's `OPENCODE_CONFIG_CONTENT`).
|
||||
* `configDir`: a generated config file under an isolated, dir-redirect-env-
|
||||
* pointed directory so the user's real CLI config is never touched
|
||||
* (codex's `CODEX_HOME`/`config.toml`, pi/omp's `PI_CONFIG_DIR`).
|
||||
* `unsupported`: no known mechanism (antigravity) — the toolbar entry
|
||||
* stays disabled for this CLI.
|
||||
*
|
||||
* Every env var name this introduces that can redirect a session's
|
||||
* traffic MUST also appear in `privilegedEnvKeys` above, exactly like
|
||||
* `DEEPSEEK_BASE_URL` — a non-granted multi-user owner redirecting a
|
||||
* session to their own endpoint is a credential-exfiltration path, not
|
||||
* just a mischief redirect.
|
||||
*/
|
||||
customModelInjection:
|
||||
| { kind: 'env'; baseUrlVar: string; apiKeyVar: string; modelVars: string[] }
|
||||
| { kind: 'configContentEnv'; envVar: string; template: 'opencode-json' }
|
||||
| {
|
||||
kind: 'configDir';
|
||||
dirEnvVar: string;
|
||||
fileName: string;
|
||||
template: 'codex-toml' | 'pi-models-json' | 'omp-models-yml';
|
||||
}
|
||||
| { kind: 'unsupported' };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user