BitSquare/Codeman
2
0
Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-10-07 07:59:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(custom-model): unset injected env on clear, resume on restart, select the model for pi/omp/grok

Browse Source 
Custom Model Endpoint Profiles (#393) let a session point its CLI at a
custom OpenAI-compatible endpoint by injecting env vars or a config file
and restarting the CLI in place. Review of the apply path found four
things, two of them destructive. This lands all four plus the smaller
items from the same review.

1. Clearing a selection did not clear it. The injected vars reach the CLI
   via `tmux setenv`, which persists at the tmux-session level and is
   inherited by `respawn-pane` (measured: `setenv FOO bar` survived two
   successive `respawn-pane -k`), so deleting the keys from the session's
   envOverrides relaunched the CLI still pointed at the old endpoint, and
   for the configDir kinds at a HOME/CODEX_HOME/GROK_HOME that had just
   been deleted. `Session.setCustomModel()` now reports the removed keys,
   queues them (`_pendingEnvUnsets`), and `RespawnPaneOptions.unsetEnvKeys`
   carries them into `applyEnvOverrides()`, which `setenv -u`s them before
   re-applying the live overrides, on the same path that already unsets
   the legacy CLAUDE_CODE_EFFORT_LEVEL. Verified on a private tmux socket
   that `setenv -u HOME` hands the next respawn the global HOME back.

2. Applying a model to a local claude session killed the pane. The
   relaunch was `claude --session-id <id>` and Claude refuses an id that
   already has a transcript, and unlike the dead-pane respawn this one
   kills a working pane first. `restartCli()` now pins the live
   conversation id as the resume id for that respawn when the CLI's launch
   declares a `fallback` chain, which renders the same
   `--resume <id> || --session-id <id>` shape the docker and remote pane
   commands use. Gated on the registry shape, not the CLI id: an entry
   whose resume id is minted by the CLI itself never declares that chain.

3. pi, omp and grok wrote their config file and then launched without the
   `--model` that selects it, so the file was ignored. The registry entry
   now declares `customModelInjection.launchModel` (`custom/{modelId}` for
   pi and omp, grok's `[model.codeman-custom]` block name), the builder
   renders it, and `_withCustomModelLaunchModel()` applies it onto the
   respawn options through `legacyConfigField`, leaving the stored
   <Mode>Config untouched so a clear falls back to the user's own model.
   A model id the CLI's `model` token pattern cannot carry is refused
   with a 400 rather than silently dropped by the argv engine.

4. Remote (SSH) and Docker sessions reported `restarted: true` and changed
   nothing: their `restartCli()` reattaches the durable tmux rather than
   relaunching the agent, and the env lands on the local pane. Both are
   refused with a 400 until those paths are plumbed.

Smaller items from the same review:

- The selection survives a Codeman restart as the disk-only `__customModel`
  bookkeeping (endpoint, model, injected key NAMES, config dir, launch
  model; never the values, which carry the API key). Recovery re-derives
  the values from the endpoint store through the same apply path the route
  uses and keeps the bookkeeping even when the endpoint is gone, so a
  later clear still has keys to unset.
- Discovery goes through `webviewFetch()`, so the RESOLVED address is
  judged by the same egress guard the web-tab proxy uses, and `baseUrl`
  reuses `webviewUrlSchema` (http(s) only, no embedded credentials,
  link-local and cloud-metadata addresses refused). undici's `fetch failed`
  wrapper is unwrapped so the user sees the ECONNREFUSED underneath.
- `custom-model-hosts.json` is written 0600 via tmp+rename, the per-session
  config dir 0700/0600 (pi and omp embed the key literally), and that dir
  is removed with the session.
- `PR.md` is gone from the repo root and the design doc moved to
  `docs/custom-model-endpoints-plan.md` with the LAN address and the
  personal name scrubbed; every reference follows. The guide's `authStyle`
  text matches the shipped schema (`bearer | api-key`, default `bearer`)
  and says that `customModelEndpointsEnabled` is read by nothing until
  the picker lands.
- `config/tsconfig.scripts.json` typechecks `scripts/test-local-llm-harnesses.ts`
  (four real type errors fixed). It is not yet wired into `npm run typecheck`
  because that line differs on master; adding `&& tsc -p config/tsconfig.scripts.json`
  there is the one-line follow-up.

Tests: `test/session-custom-model-restart.test.ts` drives a real Session and
fails on the unfixed code for items 1 to 3; the route suite covers item 4
and the pattern refusal; `test/tmux-manager.test.ts` pins that the unsets
run before the overrides and that a shell-metachar key never reaches tmux.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer
2026-09-14 23:46:28 +02:00
parent 1e42cb4e2d
commit 942bf37e48
31 changed files with 988 additions and 457 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/routes/custom-model-routes.test.ts
+56 -11
View File
@@ -1,18 +1,34 @@
/**
* @fileoverview Route tests for Custom Model Endpoint Profiles CRUD + discovery.
*
* Discovery is mocked at `webviewFetch()` (webview-egress.ts), NOT at the global
* `fetch`: the route deliberately goes through the guarded undici dispatcher whose
* lookup hook refuses a name that resolves into a link-local / cloud-metadata range,
* so a global-fetch stub that still satisfied these tests would mean the guard had
* been bypassed.
* Port: N/A (app.inject, no real port needed)
*/
import { describe, it, expect, vi, afterEach } from 'vitest';
import { registerCustomModelRoutes } from '../../src/web/routes/custom-model-routes.js';
import { webviewFetch } from '../../src/web/webview-egress.js';
import { createRouteTestHarness } from './_route-test-utils.js';
vi.mock('../../src/web/webview-egress.js', async () => {
const actual = await vi.importActual<typeof import('../../src/web/webview-egress.js')>(
'../../src/web/webview-egress.js'
);
return { ...actual, webviewFetch: vi.fn() };
});
const fetchMock = vi.mocked(webviewFetch);
async function setup() {
return createRouteTestHarness(registerCustomModelRoutes);
}
describe('custom model endpoint CRUD', () => {
afterEach(() => {
vi.unstubAllGlobals();
fetchMock.mockReset();
});
it('starts empty', async () => {
@@ -88,18 +104,18 @@ describe('custom model endpoint CRUD', () => {
payload: { id: 'ep1', label: 'A', baseUrl: 'http://localhost:8080', apiKey: 'k' },
});
const fetchMock = vi.fn(async (url: string, init?: RequestInit) => {
expect(url).toBe('http://localhost:8080/v1/models');
fetchMock.mockImplementation(async (url: URL, init?: RequestInit) => {
expect(url.href).toBe('http://localhost:8080/v1/models');
const headers = init?.headers as Record<string, string>;
// Exactly ONE auth header — never both (a real server hung when sent both).
expect(headers.Authorization).toBe('Bearer k');
expect(headers['api-key']).toBeUndefined();
return new Response(JSON.stringify({ data: [{ id: 'qwen3' }, { id: 'llama3' }] }), { status: 200 });
});
vi.stubGlobal('fetch', fetchMock);
const res = await app.inject({ method: 'POST', url: '/api/model-endpoints/ep1/discover-models' });
expect(res.json().data.models).toEqual(['qwen3', 'llama3']);
expect(fetchMock).toHaveBeenCalledTimes(1);
// Data dir is shared across this WHOLE test file (one temp HOME per file, not per
// test — test/setup.ts), so find by id rather than assuming index 0.
@@ -117,13 +133,12 @@ describe('custom model endpoint CRUD', () => {
payload: { id: 'ep-azure', label: 'A', baseUrl: 'http://localhost:8080', apiKey: 'k', authStyle: 'api-key' },
});
const fetchMock = vi.fn(async (_url: string, init?: RequestInit) => {
fetchMock.mockImplementation(async (_url: URL, init?: RequestInit) => {
const headers = init?.headers as Record<string, string>;
expect(headers['api-key']).toBe('k');
expect(headers.Authorization).toBeUndefined();
return new Response(JSON.stringify({ data: [] }), { status: 200 });
});
vi.stubGlobal('fetch', fetchMock);
await app.inject({ method: 'POST', url: '/api/model-endpoints/ep-azure/discover-models' });
expect(fetchMock).toHaveBeenCalledTimes(1);
@@ -136,11 +151,9 @@ describe('custom model endpoint CRUD', () => {
url: '/api/model-endpoints',
payload: { id: 'ep-err', label: 'A', baseUrl: 'http://localhost:8080' },
});
vi.stubGlobal(
'fetch',
vi.fn(async () => {
throw new Error('connect ECONNREFUSED');
})
// undici's shape: a bare `fetch failed` with the real reason one level down.
fetchMock.mockRejectedValue(
new TypeError('fetch failed', { cause: new Error('connect ECONNREFUSED 127.0.0.1:8080') })
);
const res = await app.inject({ method: 'POST', url: '/api/model-endpoints/ep-err/discover-models' });
@@ -148,4 +161,36 @@ describe('custom model endpoint CRUD', () => {
expect(res.json().errorCode).toBe('OPERATION_FAILED');
expect(res.json().error).toContain('ECONNREFUSED');
});
it('names the egress refusal when the endpoint resolves into a blocked range', async () => {
const { app } = await setup();
await app.inject({
method: 'POST',
url: '/api/model-endpoints',
payload: { id: 'ep-meta', label: 'A', baseUrl: 'http://models.example:8080' },
});
const { WebviewEgressBlockedError } = await vi.importActual<typeof import('../../src/web/webview-egress.js')>(
'../../src/web/webview-egress.js'
);
fetchMock.mockRejectedValue(
new TypeError('fetch failed', { cause: new WebviewEgressBlockedError('resolves to 169.254.169.254') })
);
const res = await app.inject({ method: 'POST', url: '/api/model-endpoints/ep-meta/discover-models' });
expect(res.json().success).toBe(false);
expect(res.json().error).toMatch(/refused.*169\.254\.169\.254/);
});
it('refuses a baseUrl with embedded credentials or a non-http scheme at save time', async () => {
const { app } = await setup();
for (const baseUrl of ['http://user:pw@host:8080', 'ftp://host/models', 'http://169.254.169.254']) {
const res = await app.inject({
method: 'POST',
url: '/api/model-endpoints',
payload: { id: 'bad', label: 'A', baseUrl },
});
expect(res.json().success, baseUrl).toBe(false);
expect(res.json().errorCode, baseUrl).toBe('INVALID_INPUT');
}
});
});
test/routes/session-custom-model.test.ts
+100 -1
View File
@@ -1,5 +1,5 @@
/**
* @fileoverview Tests for POST /api/sessions/:id/custom-model (deployment_plan.md
* @fileoverview Tests for POST /api/sessions/:id/custom-model (docs/custom-model-endpoints-plan.md
* chunk 5 — applying/clearing a session's custom model endpoint + CLI restart).
* Port: N/A (app.inject, no real port needed)
*/
@@ -8,6 +8,8 @@ import { registerSessionRoutes } from '../../src/web/routes/session-routes.js';
import { createRouteTestHarness } from './_route-test-utils.js';
import { getDataDir } from '../../src/config/instance.js';
import { writeCustomModelHosts, type CustomModelHost } from '../../src/custom-model-hosts.js';
import { existsSync, statSync } from 'node:fs';
import { join } from 'node:path';
const CLAUDE_ENDPOINT: CustomModelHost = {
id: 'ep1',
@@ -102,6 +104,103 @@ describe('POST /api/sessions/:id/custom-model', () => {
expect(res.json().errorCode).toBe('OPERATION_FAILED');
});
it('refuses a remote (SSH) session before touching it: restartCli would only reattach the remote tmux', async () => {
const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'claude';
session.remote = { hostId: 'h1', remotePath: '/srv/case' };
const res = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep1', modelId: 'qwen3' },
});
expect(res.json().success).toBe(false);
expect(res.json().errorCode).toBe('INVALID_INPUT');
expect(res.json().error).toMatch(/remote/i);
expect(session.setCustomModel).not.toHaveBeenCalled();
expect(session.restartCli).not.toHaveBeenCalled();
});
it('refuses a Docker session the same way, for clear as well as apply', async () => {
const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'claude';
session.docker = { containerName: 'codeman-case' };
for (const payload of [{ endpointId: 'ep1', modelId: 'qwen3' }, { clear: true }]) {
const res = await app.inject({ method: 'POST', url: '/api/sessions/test-session-1/custom-model', payload });
expect(res.json().success).toBe(false);
expect(res.json().errorCode).toBe('INVALID_INPUT');
}
expect(session.setCustomModel).not.toHaveBeenCalled();
expect(session.restartCli).not.toHaveBeenCalled();
});
it('pi: writes the config dir AND forces --model custom/<id>, since the file alone does not select the model', async () => {
const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'pi';
const res = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep1', modelId: 'qwen3.5-0.8b' },
});
expect(res.statusCode).toBe(200);
const [next, envOverrides] = session.setCustomModel.mock.calls[0];
expect(next.launchModel).toBe('custom/qwen3.5-0.8b');
expect(next.configDir).toBe(join(getDataDir(), 'custom-model-configs', 'test-session-1'));
expect(envOverrides.HOME).toBe(next.configDir);
const written = join(next.configDir, '.pi', 'agent', 'models.json');
expect(existsSync(written)).toBe(true);
// pi embeds the key literally, so the file is private to the server account.
expect(statSync(written).mode & 0o777).toBe(0o600);
expect(session.restartCli).toHaveBeenCalledTimes(1);
});
it('refuses a model id the CLI cannot carry on its command line instead of launching without it', async () => {
const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'pi';
const res = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep1', modelId: 'qwen 3 with spaces' },
});
expect(res.json().success).toBe(false);
expect(res.json().errorCode).toBe('INVALID_INPUT');
expect(session.setCustomModel).not.toHaveBeenCalled();
expect(session.restartCli).not.toHaveBeenCalled();
// The config dir written before the check is cleaned up again.
expect(existsSync(join(getDataDir(), 'custom-model-configs', 'test-session-1'))).toBe(false);
});
it('clear removes the previous config dir the session reports', async () => {
const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'pi';
await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep1', modelId: 'qwen3' },
});
const dir = join(getDataDir(), 'custom-model-configs', 'test-session-1');
expect(existsSync(dir)).toBe(true);
const res = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { clear: true },
});
expect(res.statusCode).toBe(200);
expect(existsSync(dir)).toBe(false);
});
it('refuses to touch a busy session', async () => {
const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!;