mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-04 14:39:42 +02:00
fix(custom-model): poll llama-swap readiness every 1s, check immediately, extend the cap
Reported: the "Loading..." banner stayed up past 2 minutes even though llama-swap itself had already finished loading the model. Three fixes: 1. pollIntervalMs default 3000ms -> 1000ms (as asked). 2. The loop now checks readiness IMMEDIATELY on entry rather than sleeping a full interval first - a model that's already ready (a fast load, or a re-apply onto one already loaded) shouldn't sit on "Loading..." at all. 3. maxWaitMs default 120000ms (2 min) -> 300000ms (5 min): a large (20GB+) model reading from disk can genuinely take longer than 2 minutes, which would have looked identical to the reported symptom - "still stuck past the point it should have resolved" - except it would have actually flipped to a "still waiting" warning toast at the 2-minute mark rather than staying on "Loading" indefinitely, so this alone doesn't explain what was reported, but is a real, separate improvement worth making. Also fixes a real, separate bug this surfaced while reasoning through the report: _showCenterStatus's banner is ONE shared, reused DOM node. A second call to _watchLlamaSwapLoading (e.g. switching models again before the first switch's loop had finished) would take over that shared banner, but the FIRST loop was still running and would eventually dismiss or overwrite it once ITS OWN deadline or readiness check resolved - clobbering whatever the second, current loop had put there. A generation counter (_watchLlamaSwapGeneration) now lets each call recognise when it no longer owns the banner and stop touching it silently, rather than only the last call to actually start ever safely reading or writing it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
0929694012
commit
0af233c96c
@@ -928,36 +928,51 @@ Object.assign(CodemanApp.prototype, {
|
||||
|
||||
/**
|
||||
* Polls llama-swap's own `/running` (via the read-only running-status route) until
|
||||
* `modelId` reports `state: 'ready'`, showing a sticky toast the whole time so a slow
|
||||
* `modelId` reports `state: 'ready'`, showing a sticky banner the whole time so a slow
|
||||
* unload/reload (measured well over a minute for a large model) reads as "loading",
|
||||
* never as silence or a wrong answer from whatever was loaded before. Bounded at 2
|
||||
* minutes; still not ready by then gets a toast saying so rather than polling forever.
|
||||
* never as silence or a wrong answer from whatever was loaded before. Checks immediately
|
||||
* (a fast load, or a re-apply onto an already-ready model, shouldn't wait a full interval
|
||||
* to say so), then every `pollIntervalMs`. Bounded at `maxWaitMs`; still not ready by then
|
||||
* gets a toast saying so rather than polling forever — 5 minutes by default, since a large
|
||||
* (20GB+) model reading from disk can genuinely take longer than the 2 minutes this used
|
||||
* to allow.
|
||||
*
|
||||
* `_watchLlamaSwapGeneration` guards against two overlapping calls (a second launch
|
||||
* started before the first one's loop finished) clobbering each other's banner:
|
||||
* `_showCenterStatus` reuses one shared DOM node, so an older loop's `dismiss()`/message
|
||||
* update firing after a newer one has already taken over the banner would otherwise hide
|
||||
* or overwrite the WRONG one. Each call claims the counter as its own "generation" and
|
||||
* checks it still owns it before touching the banner.
|
||||
*
|
||||
* `pollIntervalMs`/`maxWaitMs` exist to let a test drive this in milliseconds instead of
|
||||
* minutes — real callers never pass them, which is what keeps the defaults live here
|
||||
* rather than only in a test fixture.
|
||||
*/
|
||||
async _watchLlamaSwapLoading(endpointId, modelId, pollIntervalMs = 3000, maxWaitMs = 120000) {
|
||||
async _watchLlamaSwapLoading(endpointId, modelId, pollIntervalMs = 1000, maxWaitMs = 300000) {
|
||||
const generation = (this._watchLlamaSwapGeneration = (this._watchLlamaSwapGeneration || 0) + 1);
|
||||
const isCurrent = () => this._watchLlamaSwapGeneration === generation;
|
||||
// Prominent and screen-centred, not a corner toast — a real llama-swap model load can
|
||||
// sit on screen for well over a minute, easy to mistake for nothing happening there.
|
||||
const toast = this._showCenterStatus(`Loading ${modelId} on ${endpointId}… this can take a while`);
|
||||
const deadline = Date.now() + maxWaitMs;
|
||||
while (Date.now() < deadline) {
|
||||
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
|
||||
const status = await this._apiJson(`/api/model-endpoints/${encodeURIComponent(endpointId)}/running-status`);
|
||||
if (!status) continue; // transient failure — keep waiting rather than giving up early
|
||||
if (!status.isLlamaSwap) {
|
||||
if (!isCurrent()) return; // a newer launch took over the banner — this loop is done
|
||||
if (!status) {
|
||||
// transient failure — keep waiting rather than giving up early
|
||||
} else if (!status.isLlamaSwap) {
|
||||
// Endpoint changed under us, or wasn't llama-swap after all — nothing more to
|
||||
// watch for, and not a failure worth a toast of its own.
|
||||
toast?.dismiss();
|
||||
return;
|
||||
}
|
||||
if (status.running.some((r) => r.model === modelId && r.state === 'ready')) {
|
||||
} else if (status.running.some((r) => r.model === modelId && r.state === 'ready')) {
|
||||
toast?.dismiss();
|
||||
this.showToast(`${modelId} is ready`, 'success', { duration: 2500 });
|
||||
return;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
|
||||
}
|
||||
if (!isCurrent()) return;
|
||||
toast?.dismiss();
|
||||
this.showToast(`Still waiting for ${modelId} to finish loading on ${endpointId} — check the llama-swap server`, 'warning');
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user