mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-06 07:29:42 +02:00
feat(custom-model): Custom Model Endpoint Profiles (local or cloud, all harnesses)
Point any Codeman-supported harness (Claude, opencode, Codex, Gemini, Pi, Grok, DeepSeek, OMP) at a custom OpenAI-compatible endpoint instead of its native cloud backend, for a given session. Covers local hardware (llama.cpp, Ollama, vLLM, DGX Spark, Strix Halo) and cloud (Azure AI Foundry, OpenRouter). Off by default (customModelEndpointsEnabled, synced, default OFF). - Registry: capabilities.customModelInjection per CLI entry (env / configContentEnv / configDir / unsupported kinds) - Pure injection builder (custom-model-injection.ts) turning an endpoint + model id into the real env vars / config content per CLI - Endpoint store + CRUD routes (custom-model-hosts.ts, custom-model-routes.ts), discovery via GET /v1/models, SSRF-guarded - Session integration: Session.setCustomModel()/restartCli() (POST /api/sessions/:id/custom-model), reusing the existing respawn-pane -k primitive to restart the CLI process with new env - Multi-user hardening: every new redirect-capable env var added to its CLI's privilegedEnvKeys, closing a pre-existing gap where several were already reachable via the generic envOverrides field's prefix allowlist - Standalone scripts/test-local-llm-harnesses.mjs: spawns real CLI binaries against a real endpoint outside the web UI, independent of tmux/sessions - Mock-server contract tests (test/fixtures/mock-openai-server.ts) replaying every CLI's injected values through a real HTTP shape Real end-to-end validation against a live llama-swap server (inside a codeman/agent:llm-test Docker image with all 9 CLI binaries) found and fixed three real bugs before they shipped: - Codex's config.toml schema was wrong ([model].default table instead of a top-level model string + [model_providers.custom]); fixing it then surfaced a genuine, documented protocol incompatibility (Codex only speaks the Responses API since Feb 2026, which llama.cpp/llama-swap don't implement) - Claude Code's async session-title-generation call validates ANTHROPIC_DEFAULT_HAIKU_MODEL against its own internal model list and hangs the whole -p invocation on an unrecognized name; documented for chunk 6, worked around in the standalone script only (--bare is NOT safe for a real interactive session, which needs hooks) - The discovery route's authStyle: 'both' option (send both Authorization and api-key headers) reliably hung a real server; removed the option entirely rather than just changing the default Status: draft. Chunk 6 (frontend toolbar/settings UI) not yet built — see PR.md and deployment_plan.md for the full chunk breakdown and confidence table. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017HqNWfmtBU2KN29SvSVWB3
This commit is contained in:
Devvyn
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
a017e9a8e0
commit
41416566aa
@@ -0,0 +1,188 @@
|
||||
/**
|
||||
* @fileoverview Pure builder for the Custom Model Endpoint Profiles feature
|
||||
* (deployment_plan.md): turns a CLI registry entry's
|
||||
* `capabilities.customModelInjection` declaration, a configured endpoint,
|
||||
* and a chosen model id into the concrete env vars / config-file content
|
||||
* that would redirect that CLI's session at the endpoint.
|
||||
*
|
||||
* No IO here on purpose (mirrors `session-cli-builder.ts`) — a caller
|
||||
* writes `ConfigDirInjection.files` to disk under an isolated per-session
|
||||
* directory and points `dirEnvVar` at it; this module only computes what
|
||||
* those files/env vars should contain.
|
||||
*
|
||||
* Confidence: `claude` and `opencode` are verified end-to-end against a real
|
||||
* llama-swap server (a real "hello world" reply came back). `codex`'s
|
||||
* config.toml STRUCTURE is now verified (an earlier `[model].default` table
|
||||
* shape was rejected by a real codex binary with "invalid type: map,
|
||||
* expected a string" — caught by `scripts/test-local-llm-harnesses.mjs`),
|
||||
* but `wire_api = "responses"` is the only value codex still accepts
|
||||
* (support for `"chat"` was dropped in Feb 2026), and a plain OpenAI
|
||||
* Chat-Completions server (llama.cpp, llama-swap, most local setups) does
|
||||
* NOT implement the Responses API — so codex may still fail at the
|
||||
* PROTOCOL level even with a correctly-shaped config file. That gap is
|
||||
* real and current, not a stale warning; see deployment_plan.md. The rest
|
||||
* (gemini/pi/grok/deepseek/omp) have their ONE-SHOT INVOCATION flags
|
||||
* confirmed against real installed binaries' own `--help` output, but
|
||||
* their custom-endpoint env/config conventions remain web-researched,
|
||||
* unverified.
|
||||
*/
|
||||
|
||||
import type { CliEntry } from './config/cli-registry/types.js';
|
||||
|
||||
export interface CustomModelEndpoint {
|
||||
id: string;
|
||||
label: string;
|
||||
/** Root URL, no trailing slash required — e.g. "http://192.168.1.50:8080" or an Azure AI Foundry URL. */
|
||||
baseUrl: string;
|
||||
/** Falls back to a harmless placeholder for endpoints (llama.cpp) that don't check it. */
|
||||
apiKey?: string;
|
||||
}
|
||||
|
||||
export interface EnvInjection {
|
||||
kind: 'env';
|
||||
/** Ready to merge into a session's envOverrides. */
|
||||
envOverrides: Record<string, string>;
|
||||
}
|
||||
|
||||
export interface ConfigDirInjection {
|
||||
kind: 'configDir';
|
||||
/** Env var that must be set to the directory the caller writes `files` under. */
|
||||
dirEnvVar: string;
|
||||
files: Array<{ relPath: string; content: string }>;
|
||||
/**
|
||||
* Env vars the written config file REFERENCES by name rather than embedding a
|
||||
* literal value (codex's `env_key = "..."` convention: config.toml never carries
|
||||
* the API key itself, only the name of an env var codex reads it from). Merge
|
||||
* these into the session's envOverrides alongside `dirEnvVar` — never skip them,
|
||||
* or the config points at a credential that was never actually set.
|
||||
*/
|
||||
extraEnv?: Record<string, string>;
|
||||
}
|
||||
|
||||
export interface UnsupportedInjection {
|
||||
kind: 'unsupported';
|
||||
}
|
||||
|
||||
export type CustomModelInjectionResult = EnvInjection | ConfigDirInjection | UnsupportedInjection;
|
||||
|
||||
const DEFAULT_API_KEY = 'local-dummy-key';
|
||||
|
||||
/** Normalizes a base URL to end in exactly one trailing `/v1`, for CLIs whose config expects the OpenAI-style suffix. */
|
||||
export function withV1Suffix(baseUrl: string): string {
|
||||
const trimmed = baseUrl.replace(/\/+$/, '');
|
||||
return /\/v1$/.test(trimmed) ? trimmed : `${trimmed}/v1`;
|
||||
}
|
||||
|
||||
/** JSON-escapes a string for embedding in a TOML/YAML double-quoted scalar — a safe superset of both grammars' basic escapes. */
|
||||
function quoted(value: string): string {
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
export function buildCustomModelInjection(
|
||||
entry: Pick<CliEntry, 'capabilities'>,
|
||||
endpoint: CustomModelEndpoint,
|
||||
modelId: string
|
||||
): CustomModelInjectionResult {
|
||||
const cap = entry.capabilities.customModelInjection;
|
||||
const apiKey = endpoint.apiKey?.trim() || DEFAULT_API_KEY;
|
||||
|
||||
switch (cap.kind) {
|
||||
case 'env': {
|
||||
const envOverrides: Record<string, string> = {
|
||||
[cap.baseUrlVar]: endpoint.baseUrl,
|
||||
[cap.apiKeyVar]: apiKey,
|
||||
};
|
||||
for (const modelVar of cap.modelVars) envOverrides[modelVar] = modelId;
|
||||
return { kind: 'env', envOverrides };
|
||||
}
|
||||
|
||||
case 'configContentEnv': {
|
||||
const content = renderConfigContent(cap.template, endpoint, modelId, apiKey);
|
||||
return { kind: 'env', envOverrides: { [cap.envVar]: content } };
|
||||
}
|
||||
|
||||
case 'configDir': {
|
||||
const { content, extraEnv } = renderConfigFile(cap.template, endpoint, modelId, apiKey);
|
||||
return { kind: 'configDir', dirEnvVar: cap.dirEnvVar, files: [{ relPath: cap.fileName, content }], extraEnv };
|
||||
}
|
||||
|
||||
case 'unsupported':
|
||||
return { kind: 'unsupported' };
|
||||
}
|
||||
}
|
||||
|
||||
function renderConfigContent(
|
||||
template: 'opencode-json',
|
||||
endpoint: CustomModelEndpoint,
|
||||
modelId: string,
|
||||
apiKey: string
|
||||
): string {
|
||||
switch (template) {
|
||||
case 'opencode-json':
|
||||
return JSON.stringify({
|
||||
$schema: 'https://opencode.ai/config.json',
|
||||
provider: {
|
||||
custom: {
|
||||
options: { baseURL: withV1Suffix(endpoint.baseUrl), apiKey },
|
||||
models: { [modelId]: {} },
|
||||
},
|
||||
},
|
||||
model: `custom/${modelId}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const CODEX_API_KEY_ENV_VAR = 'CODEMAN_CUSTOM_MODEL_API_KEY';
|
||||
|
||||
function renderConfigFile(
|
||||
template: 'codex-toml' | 'pi-models-json' | 'omp-models-yml',
|
||||
endpoint: CustomModelEndpoint,
|
||||
modelId: string,
|
||||
apiKey: string
|
||||
): { content: string; extraEnv?: Record<string, string> } {
|
||||
const baseUrl = withV1Suffix(endpoint.baseUrl);
|
||||
switch (template) {
|
||||
case 'codex-toml': {
|
||||
// Verified against real codex (>= Feb 2026): `model` is a top-level STRING, never
|
||||
// a `[model].default` table — codex rejects that with "invalid type: map, expected
|
||||
// a string" (caught by scripts/test-local-llm-harnesses.mjs against a real llama-swap
|
||||
// server). The API key is NEVER a literal TOML field: codex's schema only supports
|
||||
// `env_key`, the NAME of an env var it reads the credential from at runtime, so the
|
||||
// actual value must ride along as an extra env var, never embedded in the file.
|
||||
// ⚠️ `wire_api = "responses"` is the only value codex still accepts (it dropped
|
||||
// `"chat"` support in Feb 2026) — a plain OpenAI Chat-Completions server (llama.cpp,
|
||||
// llama-swap, most local setups) does NOT implement the Responses API, so this
|
||||
// recipe may still fail at the PROTOCOL level even though the file now parses
|
||||
// correctly. That is a real, currently-unresolved compatibility gap, not a syntax
|
||||
// bug — track it before calling codex support done.
|
||||
const content = [
|
||||
`model = ${quoted(modelId)}`,
|
||||
`model_provider = "custom"`,
|
||||
'',
|
||||
'[model_providers.custom]',
|
||||
`name = "Custom Endpoint"`,
|
||||
`base_url = ${quoted(baseUrl)}`,
|
||||
`env_key = ${quoted(CODEX_API_KEY_ENV_VAR)}`,
|
||||
`wire_api = "responses"`,
|
||||
'',
|
||||
].join('\n');
|
||||
return { content, extraEnv: { [CODEX_API_KEY_ENV_VAR]: apiKey } };
|
||||
}
|
||||
case 'pi-models-json':
|
||||
return {
|
||||
content: JSON.stringify(
|
||||
{
|
||||
providers: {
|
||||
custom: { baseUrl, apiKey, api: 'openai-completions', models: { [modelId]: {} } },
|
||||
},
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
};
|
||||
case 'omp-models-yml':
|
||||
return {
|
||||
content: `providers:\n custom:\n baseUrl: ${quoted(baseUrl)}\n apiKey: ${quoted(apiKey)}\n models:\n - ${quoted(modelId)}\n`,
|
||||
};
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user