mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-08 16:39:42 +02:00
feat(custom-model): warn before launching Claude on a model too small for its own overhead
Claude Code's own fixed per-turn overhead (system prompt + tool schemas,
~36.4K tokens measured live) can exceed a small local model's entire real
context before any conversation history exists to compact — confirmed
live twice as an in:0 out:0 failure on the very first message sent.
CLAUDE_CODE_MAX_CONTEXT_TOKENS cannot fix this: it only governs when
history gets compacted, and there is none on message one.
- exceedsSafeContextFloor() (custom-model-routes.ts): true when a CLI's
registry entry declares contextLengthVar (currently only claude) and
the model's discovered context is below CLAUDE_MIN_SAFE_CONTEXT_TOKENS
(40000). A no-op for every other CLI by construction.
- Both apply routes (POST /api/sessions/:id/custom-model and the
quick-start customModel path) check this before the swap-conflict
check and before launching/restarting anything, returning
{requiresContextWarning, modelId, contextLength, minSafeContextTokens}
— skipped when confirmed:true.
- Frontend: #customModelContextWarningModal + _confirmContextWarning/
_resolveContextWarningConfirm (session-ui.js), wired into both
_quickStartWithCustomModelConfirm and _runCustomModelEntryViaRestart
(the path Claude actually uses) ahead of the swap-confirmation check.
Explains the fix in-modal: give the model an explicit larger -c/
--ctx-size in llama-swap instead of relying on --fit-ctx, which
optimizes for the biggest model that fits rather than the biggest
context.
Tests added for the route-level warning/confirm/skip cases and the
frontend modal + launch-flow wiring. Docs updated (custom-model-
endpoints.md, wiki/Custom-Model-Endpoints.md) and the PR's running
changeset extended.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
993710263d
commit
b45a96358e
@@ -948,6 +948,29 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Custom Model Endpoint Profiles: context-window-too-small warning
|
||||
(docs/custom-model-endpoints-plan.md) — shown before launching a CLI
|
||||
whose own fixed system-prompt/tool-schema overhead exceeds the
|
||||
model's real discovered context, which guarantees a first-message
|
||||
failure regardless of CLAUDE_CODE_MAX_CONTEXT_TOKENS. See
|
||||
_confirmContextWarning() in session-ui.js. -->
|
||||
<div class="modal" id="customModelContextWarningModal">
|
||||
<div class="modal-backdrop" onclick="app._resolveContextWarningConfirm(false)"></div>
|
||||
<div class="modal-content modal-sm">
|
||||
<div class="modal-header">
|
||||
<h3>Context window too small</h3>
|
||||
<button class="modal-close" onclick="app._resolveContextWarningConfirm(false)" aria-label="Cancel">×</button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<p class="form-hint" id="customModelContextWarningMessage" style="white-space: pre-wrap;"></p>
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button class="btn-toolbar" onclick="app._resolveContextWarningConfirm(false)">Cancel</button>
|
||||
<button class="btn-toolbar btn-primary" onclick="app._resolveContextWarningConfirm(true)">Launch anyway</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Cron Jobs Modal -->
|
||||
<div class="modal" id="cronModal">
|
||||
<div class="modal-backdrop" onclick="app.closeCron()"></div>
|
||||
|
||||
Reference in New Issue
Block a user