mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-08 16:39:42 +02:00
Custom Model Endpoint Profiles (#393) let a session point its CLI at a custom OpenAI-compatible endpoint by injecting env vars or a config file and restarting the CLI in place. Review of the apply path found four things, two of them destructive. This lands all four plus the smaller items from the same review. 1. Clearing a selection did not clear it. The injected vars reach the CLI via `tmux setenv`, which persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so deleting the keys from the session's envOverrides relaunched the CLI still pointed at the old endpoint, and for the configDir kinds at a HOME/CODEX_HOME/GROK_HOME that had just been deleted. `Session.setCustomModel()` now reports the removed keys, queues them (`_pendingEnvUnsets`), and `RespawnPaneOptions.unsetEnvKeys` carries them into `applyEnvOverrides()`, which `setenv -u`s them before re-applying the live overrides, on the same path that already unsets the legacy CLAUDE_CODE_EFFORT_LEVEL. Verified on a private tmux socket that `setenv -u HOME` hands the next respawn the global HOME back. 2. Applying a model to a local claude session killed the pane. The relaunch was `claude --session-id <id>` and Claude refuses an id that already has a transcript, and unlike the dead-pane respawn this one kills a working pane first. `restartCli()` now pins the live conversation id as the resume id for that respawn when the CLI's launch declares a `fallback` chain, which renders the same `--resume <id> || --session-id <id>` shape the docker and remote pane commands use. Gated on the registry shape, not the CLI id: an entry whose resume id is minted by the CLI itself never declares that chain. 3. pi, omp and grok wrote their config file and then launched without the `--model` that selects it, so the file was ignored. The registry entry now declares `customModelInjection.launchModel` (`custom/{modelId}` for pi and omp, grok's `[model.codeman-custom]` block name), the builder renders it, and `_withCustomModelLaunchModel()` applies it onto the respawn options through `legacyConfigField`, leaving the stored <Mode>Config untouched so a clear falls back to the user's own model. A model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. 4. Remote (SSH) and Docker sessions reported `restarted: true` and changed nothing: their `restartCli()` reattaches the durable tmux rather than relaunching the agent, and the env lands on the local pane. Both are refused with a 400 until those paths are plumbed. Smaller items from the same review: - The selection survives a Codeman restart as the disk-only `__customModel` bookkeeping (endpoint, model, injected key NAMES, config dir, launch model; never the values, which carry the API key). Recovery re-derives the values from the endpoint store through the same apply path the route uses and keeps the bookkeeping even when the endpoint is gone, so a later clear still has keys to unset. - Discovery goes through `webviewFetch()`, so the RESOLVED address is judged by the same egress guard the web-tab proxy uses, and `baseUrl` reuses `webviewUrlSchema` (http(s) only, no embedded credentials, link-local and cloud-metadata addresses refused). undici's `fetch failed` wrapper is unwrapped so the user sees the ECONNREFUSED underneath. - `custom-model-hosts.json` is written 0600 via tmp+rename, the per-session config dir 0700/0600 (pi and omp embed the key literally), and that dir is removed with the session. - `PR.md` is gone from the repo root and the design doc moved to `docs/custom-model-endpoints-plan.md` with the LAN address and the personal name scrubbed; every reference follows. The guide's `authStyle` text matches the shipped schema (`bearer | api-key`, default `bearer`) and says that `customModelEndpointsEnabled` is read by nothing until the picker lands. - `config/tsconfig.scripts.json` typechecks `scripts/test-local-llm-harnesses.ts` (four real type errors fixed). It is not yet wired into `npm run typecheck` because that line differs on master; adding `&& tsc -p config/tsconfig.scripts.json` there is the one-line follow-up. Tests: `test/session-custom-model-restart.test.ts` drives a real Session and fails on the unfixed code for items 1 to 3; the route suite covers item 4 and the pattern refusal; `test/tmux-manager.test.ts` pins that the unsets run before the overrides and that a shell-metachar key never reaches tmux. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
147 lines
6.9 KiB
TypeScript
147 lines
6.9 KiB
TypeScript
/**
|
|
* @fileoverview Custom Model Endpoint Profiles CRUD + discovery
|
|
* (docs/custom-model-endpoints-plan.md). Endpoints are machine-level infra,
|
|
* like remote/docker hosts, so writes are admin-only in multi-user mode
|
|
* (`case-routes.ts`'s `/api/remote-hosts` is the pattern this mirrors).
|
|
*
|
|
* Discovery (`POST /:id/discover-models`) fetches `${baseUrl}/v1/models`
|
|
* through `webviewFetch()` (`webview-egress.ts`), the same guarded dispatcher
|
|
* the web-tab proxy uses: `baseUrl` is refused at save time by the schema's
|
|
* hostname check (link-local / cloud-metadata literals and names), and the
|
|
* undici lookup hook refuses a name that RESOLVES into one of those ranges at
|
|
* connect time, redirects included — a save-time hostname check alone would
|
|
* let `models.example` resolve to 169.254.169.254 later. The endpoint is
|
|
* admin-configured, so this is defence in depth rather than the only gate.
|
|
*/
|
|
|
|
import type { FastifyInstance, FastifyRequest } from 'fastify';
|
|
import { ApiErrorCode, createErrorResponse, type ApiResponse } from '../../types.js';
|
|
import { isAdmin, parseBody } from '../route-helpers.js';
|
|
import { isMultiUserMode } from '../../config/multiuser.js';
|
|
import { getDataDir } from '../../config/instance.js';
|
|
import { isBlockedWebviewUrl } from '../webview-egress-policy.js';
|
|
import { egressBlockedReason, webviewFetch } from '../webview-egress.js';
|
|
import { CustomModelHostSchema } from '../schemas.js';
|
|
import { readCustomModelHosts, writeCustomModelHosts, type CustomModelHost } from '../../custom-model-hosts.js';
|
|
|
|
const CODEMAN_CONFIG_DIR = getDataDir();
|
|
const DISCOVER_TIMEOUT_MS = 8000;
|
|
|
|
function adminOnly(req: FastifyRequest, reply: { code: (n: number) => unknown }): ApiResponse<never> | null {
|
|
if (!isMultiUserMode() || isAdmin(req)) return null;
|
|
reply.code(403);
|
|
return createErrorResponse(ApiErrorCode.FORBIDDEN, 'Admin only in multi-user mode');
|
|
}
|
|
|
|
async function discoverModels(host: Pick<CustomModelHost, 'baseUrl' | 'apiKey' | 'authStyle'>): Promise<string[]> {
|
|
const headers: Record<string, string> = {};
|
|
const apiKey = host.apiKey?.trim();
|
|
// Exactly ONE header, never both — see custom-model-hosts.ts's CustomModelAuthStyle
|
|
// doc comment for why: sending both reliably HANGS some real servers.
|
|
const style = host.authStyle ?? 'bearer';
|
|
if (apiKey && style === 'bearer') headers.Authorization = `Bearer ${apiKey}`;
|
|
if (apiKey && style === 'api-key') headers['api-key'] = apiKey;
|
|
|
|
const res = await webviewFetch(new URL(`${host.baseUrl.replace(/\/+$/, '')}/v1/models`), {
|
|
headers,
|
|
signal: AbortSignal.timeout(DISCOVER_TIMEOUT_MS),
|
|
});
|
|
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
|
const body = (await res.json()) as { data?: Array<{ id?: unknown }> };
|
|
return (body.data ?? []).map((m) => m.id).filter((id): id is string => typeof id === 'string' && id.length > 0);
|
|
}
|
|
|
|
/**
|
|
* undici reports every network failure as `TypeError('fetch failed', { cause })`, with the
|
|
* useful part (`connect ECONNREFUSED 127.0.0.1:8080`) one level down; surface the deepest
|
|
* message so the user sees the refused connection, not the wrapper.
|
|
*/
|
|
function describeFetchError(err: unknown): string {
|
|
let message = err instanceof Error ? err.message : String(err);
|
|
let current: unknown = err;
|
|
for (let depth = 0; depth < 5 && current instanceof Error && current.cause !== undefined; depth++) {
|
|
current = current.cause;
|
|
if (current instanceof Error && current.message) message = current.message;
|
|
}
|
|
return message;
|
|
}
|
|
|
|
export function registerCustomModelRoutes(app: FastifyInstance): void {
|
|
app.get('/api/model-endpoints', async (req) =>
|
|
isMultiUserMode() && !isAdmin(req) ? [] : readCustomModelHosts(CODEMAN_CONFIG_DIR)
|
|
);
|
|
|
|
app.post('/api/model-endpoints', async (req, reply): Promise<ApiResponse<{ host: CustomModelHost }>> => {
|
|
const denied = adminOnly(req, reply);
|
|
if (denied) return denied;
|
|
const host = parseBody(CustomModelHostSchema, req.body);
|
|
if (isBlockedWebviewUrl(host.baseUrl)) {
|
|
return createErrorResponse(ApiErrorCode.INVALID_INPUT, 'Endpoint base URL is not allowed');
|
|
}
|
|
const hosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
|
|
if (hosts.some((item) => item.id === host.id)) {
|
|
return createErrorResponse(ApiErrorCode.ALREADY_EXISTS, 'Model endpoint already exists');
|
|
}
|
|
await writeCustomModelHosts(CODEMAN_CONFIG_DIR, [...hosts, host]);
|
|
return { success: true, data: { host } };
|
|
});
|
|
|
|
app.put('/api/model-endpoints/:id', async (req, reply): Promise<ApiResponse<{ host: CustomModelHost }>> => {
|
|
const denied = adminOnly(req, reply);
|
|
if (denied) return denied;
|
|
const { id } = req.params as { id: string };
|
|
const host = parseBody(CustomModelHostSchema, { ...(req.body as object), id });
|
|
if (isBlockedWebviewUrl(host.baseUrl)) {
|
|
return createErrorResponse(ApiErrorCode.INVALID_INPUT, 'Endpoint base URL is not allowed');
|
|
}
|
|
const hosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
|
|
const index = hosts.findIndex((item) => item.id === id);
|
|
if (index === -1) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found');
|
|
const next = [...hosts];
|
|
next[index] = host;
|
|
await writeCustomModelHosts(CODEMAN_CONFIG_DIR, next);
|
|
return { success: true, data: { host } };
|
|
});
|
|
|
|
app.delete('/api/model-endpoints/:id', async (req, reply): Promise<ApiResponse<{ id: string }>> => {
|
|
const denied = adminOnly(req, reply);
|
|
if (denied) return denied;
|
|
const { id } = req.params as { id: string };
|
|
const hosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
|
|
await writeCustomModelHosts(
|
|
CODEMAN_CONFIG_DIR,
|
|
hosts.filter((item) => item.id !== id)
|
|
);
|
|
return { success: true, data: { id } };
|
|
});
|
|
|
|
app.post(
|
|
'/api/model-endpoints/:id/discover-models',
|
|
async (req, reply): Promise<ApiResponse<{ models: string[] }>> => {
|
|
const denied = adminOnly(req, reply);
|
|
if (denied) return denied;
|
|
const { id } = req.params as { id: string };
|
|
const hosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
|
|
const index = hosts.findIndex((item) => item.id === id);
|
|
if (index === -1) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found');
|
|
const host = hosts[index];
|
|
if (isBlockedWebviewUrl(host.baseUrl)) {
|
|
return createErrorResponse(ApiErrorCode.INVALID_INPUT, 'Endpoint base URL is not allowed');
|
|
}
|
|
try {
|
|
const models = await discoverModels(host);
|
|
const next = [...hosts];
|
|
next[index] = { ...host, models, lastDiscoveredAt: new Date().toISOString() };
|
|
await writeCustomModelHosts(CODEMAN_CONFIG_DIR, next);
|
|
return { success: true, data: { models } };
|
|
} catch (err) {
|
|
const blocked = egressBlockedReason(err);
|
|
return createErrorResponse(
|
|
ApiErrorCode.OPERATION_FAILED,
|
|
blocked ? `Endpoint refused: ${blocked}` : `Could not reach endpoint: ${describeFetchError(err)}`
|
|
);
|
|
}
|
|
}
|
|
);
|
|
}
|