Codeman/.changeset/run-menu-custom-model-picker.md

aicodeman

aicodeman
minor

Custom model endpoints: Run-menu picker, and hardening from real llama-swap validation (#430, follow-up to #393's HTTP-API-only cut). With Custom model endpoints on (App Settings → Models) and at least one saved endpoint carrying a discovered model, the Run dropdown grows a Custom Endpoints section generated live off the CLI registry's own capabilities.customModelInjection — one entry per (harness that can redirect to a custom endpoint, saved endpoint). Picking one launches that harness and applies the endpoint to it; with two or more discovered models a small, scrollable dialog asks which one first, the endpoint's defaultModelId marked but never auto-chosen. Endpoints also now re-discover themselves automatically every 5 minutes in the background, one unreachable endpoint never blocking the others.

Everything below was found and fixed against a real llama-swap server, not just unit tests:

• Session-busy false refusal. A freshly launched CLI reports itself busy for its own startup (spinner, workspace-trust check) well before the apply call would reach it, and the apply route correctly refuses to restart a session mid-turn — indistinguishable from a fresh boot. The picker now waits for the new session to go idle (bounded at 20s, never an error on timeout) before applying.
• Errors and confirmations you can actually read. Toasts now default to sticky with a close button (errors always were meant to stay, but a fixed 3s timer silently hid them); a failed apply's real server-side reason (not a generic message) reaches the toast.
• "Both claude.ai and ANTHROPIC_API_KEY set" warning. A custom-model Claude session now runs with an isolated CLAUDE_CONFIG_DIR (empty, no real credentials in it) so the injected API key never coexists with a stored OAuth login — projects is symlinked back to the real config dir so the response viewer/subagent windows/Read My Mind keep working. That isolated, otherwise-empty directory has none of a real profile's prior "Detected a custom API key — use it?" approvals either, which would otherwise re-ask on every launch with nobody at a TTY to answer (and silently refuse the key on its own default); the apply step now pre-seeds that exact approval field the same way answering the prompt once by hand would.
• Context-window overflow. Claude Code assumes a large default context window for a model id it doesn't recognize and never compacts, so a real local model's much smaller context silently overflowed (confirmed live: a stock ~33.7K-token system prompt against a 16384-token model). Discovery now also learns each model's real context length from llama.cpp/llama-swap's GET /props?model=, but only for a model llama-swap's own /v1/models response already reports loaded — never an unloaded one, since asking about one risks triggering an actual, slow, GPU-swapping load as a side effect of read-only discovery — and applies it as CLAUDE_CODE_MAX_CONTEXT_TOKENS.
• The real root cause of "it still says opus, not my model." llama.cpp runs exactly one model at a time; llama-swap unloads and reloads it on demand, which can take anywhere from a few seconds to well over a minute — long enough that a session mid-swap is indistinguishable from one that never left the native backend. Applying a selection now checks llama-swap's own GET /running first (feature-detected; a plain llama.cpp/OpenAI-compatible server has no such endpoint and is never checked); if switching would unload a model another live session is actively using, the apply is refused with a warning naming that session instead of silently switching, and a confirmation retry proceeds anyway. Either way, a sticky "loading model…" toast now covers the actual swap window until llama-swap reports the target model ready, so a prompt sent mid-swap reads as "loading," never as silence or an answer from whatever was loaded a moment before.

Remote (SSH) and Docker sessions are refused for now (400) — their restart reattaches the durable remote/in-container tmux rather than relaunching the agent.

One more, from watching it launch live: opencode, Codex, Gemini, Pi, Grok, DeepSeek and OMP now launch directly on the endpoint, with no restart at all. Picking one of these seven from the Run-menu picker used to launch natively first, wait for it to settle, then restart it in place with the endpoint applied — a deliberate two-step design, but visibly a native boot immediately followed by a second one, worst on a CLI whose TUI fully reinitializes on a restart (confirmed live on Codex). POST /api/quick-start now accepts a customModel field and computes the same injection before the session exists, launching straight onto the endpoint the first time — no visible relaunch, and it also runs the same llama-swap conflict check (warns before unloading a model another live session is using) at create time. Claude still uses the original launch-then-restart path for now (its own --resume-based restart is far less jarring, and runClaude()'s multi-tab and docker-config-drift-retry logic make folding it into the one-shot path separate work).