fix(tiles): cap each tile's live-output backlog and recover dropped output

The server applies no WebSocket backpressure (16 KB / 8 ms batches, no
bufferedAmount check), and a tile wrote every live frame straight into
xterm. A flood a tile could not parse as fast (a shell tile running cat on
a huge log) piled up in xterm's own write queue without bound, on a main
thread up to six tiles share, until xterm's WriteBuffer threw past 50M
code units; onmessage's empty catch then dropped every frame silently and
nothing recaptured the screen. The primary pane caps its queues and drops
then recaptures (_onSessionTerminal, _scheduleDroppedOutputRecovery).

Each tile now writes live output through _writeLive:
- unparsed code units are counted, each write's callback counting its own
  back down; frames held behind a replay (_liveQueue) count too;
- the budget is TerminalTile.LIVE_BACKLOG_BUDGET, 4 MiB, deliberately not
  the primary pane's 128 KB: that caps its own rAF-paced queues, while
  xterm itself paces a tile, and a tight cap would trip on ordinary bursts
  and blank-and-reload the tile over and over;
- past it a frame is dropped, the tile stops writing onto the hole, and one
  refresh is scheduled, debounced and bounded by the primary pane's own
  rule (CodemanDroppedOutput: 2 s, DROP_RECOVERY_MAX_ATTEMPTS, never retried
  after a deadline abort). It is an ordinary refresh, so single-flight,
  bounded by lines=/tail= and paced by the grid's TileLoadQueue. The flag
  clears once a capture taken after the last dropped frame has replayed;
- a write that throws is the same drop, never a "malformed frame";
- past the bound the flag is released, so a tile is never left frozen;
- a reconnect starts the accounting over (an epoch makes callbacks from
  before it count nothing) and drops a pending recovery, since its own
  refresh replaces the screen; destroy() cancels it.
The live-queue flush after a pull or a refresh goes through the same path,
so a throwing write there cannot skip the load's marker and trailing
refresh either.

Tests (input harness, real constants and fake timers): the default budget
lets a 1 MiB unparsed burst through, parsed bytes stop counting, a trip
stops writing and ONE debounced refresh recaptures, a write throw takes the
same recovery, a hole in the held queue is recovered by another refresh,
bounded retries then release, no retry after a deadline, a reconnect resets
the count, and destroy cancels. The fake xterm can now hold and release
parses and throw on a write. Live writes now carry a callback, so the unit
tests match them on the data argument (a `.not.toHaveBeenCalledWith(data)`
would otherwise pass for nothing).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer
2026-10-09 09:26:51 +02:00
parent cdb3c34ae0
commit 1def7de146
5 changed files with 444 additions and 19 deletions
+18 -2
View File
@@ -149,9 +149,23 @@ export class FakeTerminal {
}
registerLinkProvider() {}
writes: string[] = [];
/** Set by a test: write callbacks never run, as on a disposed xterm. */
/**
* Set by a test: write callbacks do not run, as while xterm is still parsing
* (or never, on a disposed xterm). They wait in `heldParses` until `parse()`.
*/
holdParse = false;
heldParses: Array<() => void> = [];
/** xterm catches up: runs every write callback held so far, in order. */
parse() {
for (const cb of this.heldParses.splice(0)) cb();
}
/** Set by a test: the next write of exactly this data throws, as xterm's WriteBuffer does past 50M. */
throwOnWrite: string | null = null;
write(data: string, cb?: () => void) {
if (this.throwOnWrite !== null && data === this.throwOnWrite) {
this.throwOnWrite = null;
throw new Error('write data discarded, use flow control to avoid losing data');
}
// An empty write puts nothing on screen; the replay queues one only to hear
// (its callback) that everything before it has been parsed.
if (data) this.writes.push(data);
@@ -162,7 +176,9 @@ export class FakeTerminal {
this.lineCount += data.slice(reset === -1 ? 0 : reset + 2).split('\n').length - 1;
this.settleRows();
}
if (!this.holdParse) cb?.();
if (!cb) return;
if (this.holdParse) this.heldParses.push(cb);
else cb();
}
clear() {
this.writes.push('<CLEAR>');