mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-03 22:19:42 +02:00
DevvynandClaude Sonnet 5
60e1bd52f7
Addresses every blocker, both majors, and all but one minor from the
maintainer's review of the draft PR.
Blockers:
1. Every generated inline onclick was unparseable. JSON.stringify's own
double quotes terminated the double-quoted HTML attribute at the first
one, leaving btn.onclick null on every picker entry and every Discover/
Edit/Delete button. Fixed with escapeHtml(JSON.stringify(...)) per
argument, the same idiom deleteCase's onclick already uses four lines
away in session-ui.js. This also closes the live-HTML-injection route
through modelId (server-controlled, from the endpoint's own /v1/models
reply): with quoting intact, a `>` inside it can no longer terminate the
<button> tag early.
2. GET /api/model-endpoints wraps its body in the {success,data} envelope
like every other /api route (server.ts's preSerialization hook applies
to arrays too), so Array.isArray(hosts) was always false in production
and the picker/settings panel silently saw nothing. Both call sites now
go through _apiJson(), which already exists for exactly this.
3. A failed or declined run*() (missing CLI, isBusy, a caught exception)
returns normally without ever changing activeSessionId, so the apply
step used to silently re-point and restart whatever session the user was
already looking at. runCustomModelEntry() now snapshots activeSessionId
before the launch and requires it to have actually changed.
Majors:
4. Routes the launch through run() itself via a temporary _runMode swap
(never persisted — setRunMode() would sync it to the server) instead of
a parallel hardcoded dispatch table, so a custom-model launch now holds
the same _runInFlight lock every other Run click gets. This also
resolves the "hardcoded runners map contradicts the PR's own design"
minor: dispatch is run()'s own, so a CLI whose customModelInjection
recipe lands later needs no update here.
5. New test/custom-model-run-menu-ui.test.ts drives the real session-ui.js
against a JSDOM window (runScripts:"dangerously" — this JSDOM only ever
parses markup this module generated itself) for exactly the DOM-level
facts the review said needed no Playwright and no tmux: a generated
button's onclick genuinely compiles and fires, a dangerous modelId never
produces a live element, the envelope unwrap works, the session-changed
guard holds, run() actually gets called (proving the in-flight lock
engages), and _runMode is restored afterward. Confirmed against the
pre-fix code first (reproduces btn.onclick === null exactly) so this
isn't a vacuous pass. Plus new tests in custom-model-routes.test.ts and
render-index-html.test.ts for the other fixes below.
Minors:
- Generated entries now filter through isCliAvailable(), matching
_refreshRunModeAvailability's own gating of the stock entries.
- The CRUD panel is now gated on customModelEndpointsEnabled
(applyCustomModelEndpointsVisibility(), wired to the toggle's onchange
and to settings-modal open) instead of always rendering; the endpoint GET
no longer fires unconditionally either.
- API keys are never handed back to the browser on GET, POST or PUT —
redactApiKey() replaces the field with a computed apiKeySet: boolean, and
a PUT with no apiKey now keeps the stored one server-side
(applyStoredApiKey()) instead of the client resending a value it was
never given. New tests cover both directions (kept vs. replaced) by
observing the actual auth header a subsequent discovery request sends.
- "+ Add endpoint" hides for a non-admin in multi-user mode
(_applyCustomModelAdminGate(), also wired to admin-ui.js's codeman:me
event, since the real role can resolve after settings were first opened)
— endpoint writes were already admin-only server-side, but the button
used to render for everyone and eat a 403.
- design doc (custom-model-endpoints-plan.md §4) now says up front that its
toolbar-button design was superseded by the Run-menu picker.
- docs/api-reference.md gained a Custom Model Endpoints section (every
route, the apiKeySet/defaultModelId contract, the restart mechanics).
- Wiki page now covers un-pointing a session (curl/delete, no UI yet) and
that the picker is desktop-only for now.
- .set-inline-form uses --control-bg instead of a hardcoded black alpha
(CLAUDE.md already records that exact literal turning the settings
preview into a grey slab on light skins), .run-mode-custom-models gets
the same gap: 2px .run-mode-menu's own flex gap only applies one level
up, and the index.html comment naming the wrong function is fixed.
- __codemanCustomModelClis's JSON is now escaped against a literal
</script> (CliEntry.label is user-clis.json-settable, unlike
__codemanCliAvailable's booleans-only payload) via a new exported
escapeScriptJson(), pure and unit-tested without needing a WebServer.
- Added defaultModelId + the new /v1/model-endpoints routes to
docs/api-reference.md; left the "no zh-CN for the new Models-section
group" minor unaddressed only insofar as the wider Models section (task
routing, thinking effort, etc.) has never had zh-CN coverage either —
everything this PR itself introduces (labels, hints, button text, the
Run-menu's "Custom Endpoints" header) IS translated in i18n.js.
Regression caught while fixing #4: the admin-gate's codeman:me listener is
a module-level document.addEventListener() call, which threw in
run-mode-ui.test.ts's minimal vm-context fake document and failed all 10
of that file's tests. Fixed with optional chaining before it ever reached
the branch this commit lands on; full targeted suite (route tests,
structural guards, every settings-ui.js-loading frontend test) reverified
green afterward.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
81 lines
4.7 KiB
Markdown
81 lines
4.7 KiB
Markdown
# Custom Model Endpoints
|
|
|
|
Point a harness at your own OpenAI-compatible server instead of its native cloud backend, for
|
|
one session at a time. "Custom endpoint" covers **local** hardware (llama.cpp, Ollama, vLLM,
|
|
a home GPU rig, DGX Spark, Strix Halo) and **cloud** services (Azure AI Foundry's
|
|
OpenAI-compatible endpoint, OpenRouter, a company gateway) alike, anything answering
|
|
`GET /v1/models` and `POST /v1/chat/completions` in the standard shape.
|
|
|
|
**Off by default.** Turn it on in App Settings → Models → **Custom model endpoints**.
|
|
|
|
## Adding an endpoint
|
|
|
|
Still in App Settings → Models → Custom model endpoints:
|
|
|
|
1. **+ Add endpoint** — give it an id, a label, and the base URL (`http://192.168.1.50:8080`,
|
|
say). An API key is optional; most local servers don't check one.
|
|
2. **Discover** — fetches the endpoint's own model list over `GET /v1/models` and stores it.
|
|
3. Pick a **default model** from what was discovered. This is the model the Run-menu entry
|
|
below applies with no further choice, so set it once you know which one you want.
|
|
|
|
Endpoint management is admin-only in multi-user mode, the same as remote hosts and Docker
|
|
hosts — these are machine-level infra, not a per-user setting.
|
|
|
|
## Running a session against one
|
|
|
|
With the setting on and at least one endpoint carrying a usable default model, the **Run**
|
|
dropdown grows a **Custom Endpoints** section: one entry per harness that can redirect to a
|
|
custom endpoint, per saved endpoint, e.g. "Claude Code (llama.cpp)". Picking one starts a
|
|
session on that harness exactly the way its own entry would, then points it at the
|
|
endpoint's default model. It is a one-off "try this endpoint" action, not a sticky mode — the
|
|
plain **Run** button still means "this harness, native cloud" afterward, and a fresh session
|
|
never inherits whatever the last one was pointed at.
|
|
|
|
Applying a selection **restarts the harness's process in place** — same tab, same
|
|
conversation where the harness supports resuming one, fresh environment. That restart is
|
|
necessary, not incidental: every supported harness reads its endpoint config at process
|
|
start, never per turn, so there is no live hot-swap while a turn is running.
|
|
|
|
Entries are hidden entirely for a session in a **remote (SSH) or Docker case** — support for
|
|
redirecting those hasn't landed yet, see below. The picker also only appears in the desktop
|
|
**Run** dropdown; the phone home screen builds its own run picker separately and does not
|
|
currently offer these entries.
|
|
|
|
## Which harnesses actually work
|
|
|
|
| Harness | Status |
|
|
| ------- | ------ |
|
|
| **Claude Code, opencode, Pi, Grok, OMP** | Verified end-to-end against a real local server. |
|
|
| **Codex** | Config is correct, but Codex only speaks the Responses API, which llama.cpp-style servers don't implement. A protocol gap, not a Codeman bug. |
|
|
| **Gemini** | Fails with an auth error gemini-cli raises once redirected. Unresolved; don't rely on it yet. |
|
|
| **DeepSeek** | Reaches the server but gets a consistent 404. Root cause not identified. |
|
|
| **Antigravity** | No known custom-endpoint mechanism at all. Not offered. |
|
|
|
|
Which harnesses show up in the Run-menu picker is read live off Codeman's own CLI registry,
|
|
not a fixed list here, so this table can go stale before this page does — a greyed-out or
|
|
missing entry is the more current answer.
|
|
|
|
## What it does not do
|
|
|
|
- **No remote or Docker sessions yet.** Both restart their agent differently under the hood
|
|
(reattaching a durable tmux session rather than relaunching the process), so redirecting
|
|
them needs its own plumbing that hasn't been built.
|
|
- **No live hot-swap mid-conversation.** Applying a selection always restarts the process.
|
|
- **No button to un-point a session from the UI yet.** Clearing back to native cloud is an
|
|
HTTP call (`POST .../custom-model {"clear": true}`) or deleting the session; the settings
|
|
panel manages saved endpoints, not what a running session is currently pointed at.
|
|
- **Nothing is shared with your real cloud credentials.** The endpoint's own key, if any,
|
|
never touches your Anthropic/OpenAI/Google login — a custom endpoint is a separate,
|
|
explicit choice per session.
|
|
|
|
## Security
|
|
|
|
An endpoint's base URL can't point at a link-local or cloud-metadata address (both at save
|
|
time and against the address it actually resolves to), the same guard Web Tabs uses for
|
|
saved dashboards. Endpoint records and any per-session config files a harness needs are
|
|
written with owner-only permissions. See
|
|
[custom-model-endpoints-plan.md](https://github.com/Ark0N/Codeman/blob/master/docs/custom-model-endpoints-plan.md)
|
|
in the repository for the full design reasoning, including why this feature closed a
|
|
pre-existing gap in how session environment overrides were guarded rather than opening a new
|
|
one.
|