mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-04 14:39:42 +02:00
feat(custom-model): show real-time llama.cpp backend status in the loading banner
Answers the underlying request behind investigating llama.cpp log
access: surface what the backend is actually doing, live, on top of
the existing countdown timer during a model load.
- getLatestLlamaSwapLogLine()/pruneIdleLlamaSwapLogTails()
(custom-model-routes.ts): one persistent GET /api/events (SSE)
connection held open per endpoint, parsing logData frames and
keeping the latest source:"upstream" (backend llama-server) line —
filtering out llama-swap's own source:"proxy" request-access lines.
Idle-closed after 30s of no polling, same 20s sweep as the existing
swap-displacement check.
- running-status route now returns logLine alongside the existing
isLlamaSwap/running fields.
- Frontend: _watchLlamaSwapLoading's banner gains a second line
("llama.cpp: <line>", bootlog timestamp/level/component prefix
stripped for display) that stays on the last real thing llama.cpp
said rather than clearing to blank between polls.
⚠️ Caught and fixed before merge, not after: the first cut targeted
GET /logs (the endpoint the name suggests), shipped a working-looking
implementation with passing tests, and only failed a live check against
the real Nemesis llama-swap deployment — /logs turns out to carry ONLY
llama-swap's own proxy request-access log and never once showed a
single backend line, even seconds after a real, confirmed model swap
triggered via a direct API call. GET /api/events's logData frames
(with an explicit source field distinguishing upstream from proxy) are
the only source that actually has backend output; corrected and
re-verified live end-to-end through an actual forced swap before
writing this commit, confirmed live to hold its connection open
indefinitely (unlike /logs, which closes after a fixed ~100KB).
12 tests for the corrected /api/events parsing (SSE frame buffering
across chunk boundaries, source filtering, malformed/wrong-type frames,
connection reuse, idle pruning) plus 2 for the frontend banner
rendering. Typecheck/lint/frontend-syntax clean; full suite shows no
new regressions (14 more passing than baseline, matching the new
tests; same pre-existing Windows-environment failures).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
5ddc028a2f
commit
2d3fc65758
@@ -192,6 +192,7 @@ import {
|
||||
registerCustomModelRoutes,
|
||||
refreshAllCustomModelHosts,
|
||||
detectCustomModelSwapDisplacements,
|
||||
pruneIdleLlamaSwapLogTails,
|
||||
tryWebviewRefererFallback,
|
||||
} from './routes/index.js';
|
||||
import { isLostWebviewFrameNavigation } from './webview-proxy.js';
|
||||
@@ -2788,6 +2789,10 @@ export class WebServer extends EventEmitter {
|
||||
.catch((err) => {
|
||||
console.error('[custom-model] swap-displacement check failed:', getErrorMessage(err));
|
||||
});
|
||||
// Same cadence, unrelated concern: close any /logs tail (see
|
||||
// getLatestLlamaSwapLogLine) nothing has polled in a while, so a loading banner
|
||||
// that finished (or was abandoned) doesn't leave a connection open forever.
|
||||
pruneIdleLlamaSwapLogTails();
|
||||
},
|
||||
CUSTOM_MODEL_SWAP_CHECK_INTERVAL_MS,
|
||||
{ description: 'custom model swap-displacement check' }
|
||||
|
||||
Reference in New Issue
Block a user