diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index 4cf4b5b7..a6dc7e64 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -801,7 +801,7 @@ Further detail: with many sessions the horizontal strip stops being scannable, w ### Split-pane sessions -**Split-pane sessions** (`showSplitButton`, header button, default OFF, per-device like `showFileViewerButton` — not in `SettingsUpdateSchema`, `displayKeys` in settings-ui.js): shows two live sessions side-by-side in one Codeman window. Pane A is the untouched, existing singleton terminal (`this.terminal`/`this._ws` in terminal-ui.js); Pane B is an independent `TerminalTile` (terminal-tile.js, constructed by the orchestration in terminal-split.js) with its own xterm instance and its own `/ws/sessions/:id/terminal` WebSocket, whose `cid` is the tab's identity with a `:tile` suffix, so it can never supersede the primary pane's socket (the registry supersedes by cid per session). ⚠️ **Pane B is deliberately plainer than Pane A** — no local-echo overlay, no CJK IME, no touch/mobile handlers, no keyboard accessory bar (it does carry the primary pane's hollow-buffer wheel paging and desktop click report, through terminal-ui.js's own gates aimed at the tile, and its own keyCode-229 soft-keyboard controller from terminal-keycode229-recovery.js, the #441 next-keydown drain and #541's edit-based diff for Android autocorrect, since the 1180px width gate is reachable by a wide Android tablet; see terminal-tile.js's fileoverview) — since this is a desktop-only feature (a split view needs a wide viewport) and those features exist for mobile/touch input; `.btn-split` is hard-hidden below 1180px regardless of the setting by the `@media (max-width: 1179px)` rule in styles.css (mobile.css only carries a comment pointing at it: that file loads up to 1023px, so it cannot cover the 1024-1179px tablet range the feature also needs to stay off), and the per-device setting means turning it on at a desk can never sync it onto a phone in the first place. No persistence: closing the browser tab or reloading always returns to the normal single-pane view; there is no localStorage key for split state. ⚠️ Splitting a session against itself is disallowed (the picker excludes the active session), **as is splitting against a popped-out (detached) session** — `buildSplitPickerSessions()` excludes `detachedSessions` because a detached session's own window is already claiming its PTY size, and `MAX_WS_PER_SESSION` needs no change since Pane A/B are always two different sessions. ⚠️ Either pane's session ending (deleted locally or from another client) auto-collapses the split — Pane A's session ending promotes Pane B to the new single pane via `selectSession(id, { auto: true })` (an app-driven selection, so it must not spend the session's idle alert — see the Approvals Inbox note above), never by trying to hot-swap the lightweight `TerminalTile` object into the primary singleton state. ⚠️ Pane B refits on every window/sidebar/tab-rail resize via the SAME trailing-edge `ResizeObserver` callback that resizes Pane A (`throttledResize` in terminal-ui.js) — it only ever measured Pane A's own container, so without the explicit `this._forEachTile?.((tile) => tile.fit(), { grid: false })` call there (grid tiles are left out: the grid's own observer refits them on the same resize) Pane B silently kept its stale PTY size through every resize that did not happen to be a divider drag. ⚠️ A dropped WebSocket writes a `[disconnected, reconnecting…]` marker into Pane B and reconnects on the primary pane's backoff ladder (`CodemanWsReconnect` plus jitter; the attempt count resets only on a successful open), then refreshes the buffer to close the output gap. ⚠️ The open handler clears `_wsClosed`/`_markerOwed` BEFORE that refresh, or the refresh re-owes the marker and stamps it under a healthy pane. ⚠️ 4003/4004/4009/4010 stop the pane for good, report once through `onExit(code)` and write a marker saying why (`CodemanWsReconnect` alone would retry 4003). ⚠️ A replacement socket detaches the old one first and every handler ignores a socket that is no longer `this.ws`, so a late 4010 from a superseded socket cannot stop its successor; `destroy()` cancels a pending reconnect. Related but distinct: `detachSession()` already opens one session in a separate OS-level browser window (`isSoloWindow`) — that is prior art for "two sessions visible at once" but not for one window with a draggable in-page divider, which is what this feature adds. ⚠️ Pane B installs its own `attachCustomKeyEventHandler` gating the same app-level chords Pane A's own handler gates (command palette, Alt+1-9/[/] tab nav, Alt+B sidebar toggle, Ctrl+Z suspend, Shift/Ctrl+Enter newline, smart-copy Ctrl+C) — without it the document capture-phase handler's `preventDefault()` (which never stops xterm) let each chord ALSO write its raw byte/escape sequence into Pane B's live PTY on top of whatever the app action did to Pane A (COD-153). Ctrl+Z is swallowed unless Pane B's own session is `mode === 'shell'`, mirroring terminal-ui.js's reasoning: in a plain shell it is the user's own job-control tool, everywhere else it silently suspends an unattended agent loop. Shift/Ctrl+Enter POSTs to `/api/sessions/:id/send-key` (`{key:'S-Enter'|'C-Enter'}`, typed by tmux `send-keys -H`: Ctrl+Enter is always a real 0x0a, Shift+Enter is the CLI's declared `capabilities.newline` chord, also 0x0a unless the CLI declares another; sent on keydown only, with the keypress and keyup swallowed too, see the keypress rule under Command palette and shortcut registry) targeting THIS pane's own `sessionId` rather than the primary pane's `activeSessionId` — without it xterm's plain `\r` would submit an incomplete prompt instead of adding a line to it. Smart-copy Ctrl+C/Ctrl+Shift+C is re-implemented against `this.terminal` (Pane B's own) rather than reusing `app.copyTerminalSelection()`, which reads Pane A's terminal and would copy the wrong pane's selection; Ctrl+Shift+C never falls through even with nothing to copy, mirroring terminal-ui.js's own `ev.shiftKey` branch. ⚠️ **This is a UX-parity fix, not an interrupt-safety one** — verified live in a real browser: xterm's `evaluateKeyboardEvent` routes a shifted ctrl-letter into a branch that assigns `c.key` only for two special cases (`_`→US, `@`→NUL), so it emits no data for Ctrl+Shift+C at all regardless of any application gate; a synthetic keydown with the gate removed produces zero WS frames, proving no accidental interrupt reaches the PTY either way. What gating the whole copy block on `hasSelection()` (an earlier draft) actually cost: with no selection, a selection-less Ctrl+Shift+C fell straight to `return true`, silently ceding the keystroke to the BROWSER's own handling (e.g. Chrome's Inspect-Element binding) with no feedback and no copy attempt — Pane A always intercepts it. Ctrl+V goes through the primary pane's paste trap aimed at Pane B (`_handleImagePaste({ terminal, sessionId })`), so a pasted image uploads to Pane B's session. ⚠️ `buildSplitPickerSessions()` also excludes any session with `pid === null` (an exited CLI, a crash-looped session whose breaker tripped, a restore that never re-attached): Pane B has no equivalent of `selectSession()`'s auto re-attach POST, so a pane opened onto one has nothing reading its tmux pane — no `terminal` events ever arrive, and `Session.write()` silently drops every keystroke with no ack either way, so the loss is invisible behind a socket that reports healthy. Pane B's keystrokes go through the exactly-once queue (`_sendInputAsync`) over its own socket, registered in the app's input-socket map (`_registerInputSocket` / `_inputSocketFor`), so they are seq-tagged, ACKed (`{t:'ia'}` is routed by the RECEIVING socket's session, since the frame names none), redelivered after a drop, and the ACK clears the idle alert. ⚠️ What xterm generates never enters that persisted queue: query replies are dropped (`shouldSuppressTerminalQueryResponse`, as the primary pane drops them) and focus/mouse reports go through `_sendInputEphemeral`, or a reload would replay them as typed text. That includes the click report a tile hand-encodes for a mouse-strip CLI: `TerminalTile._installClickListener` passes `ephemeral: true` in its target, which `_sendSyntheticSgrTap` honours, while the primary pane's own report (no flag) stays on `_sendInputAsync` as before. ⚠️ A SHELL Pane B pulls scrollback itself when the wheel goes up at the top of its buffer (`_maybeLoadMoreHistory`/`_pullHistory`): tmux repaints a burst of output instead of scrolling it, so Pane B's own xterm holds about one screen of scrollback while tmux holds every line, and it loaded history exactly once at connect and never again. It is the same bounded pull as Pane A's (`?full=1&tail=TERMINAL_TAIL_SIZE`, no rewrite when the window holds no more rows than the pane already has or the pane is at its `scrollback + rows` cap, and a 60 s back-off instead of 4 s when that skipped window was truncated or the pane is full, since each ask costs the server a whole-history `capture-pane`), against Pane B's OWN terminal rather than `app.terminal`, so it cannot share `_maybeRefetchFullHistory`. The wheel listener is capture-phase because xterm `stopPropagation()`s the events it consumes; the alternate-screen skip (nano, vim, less) only matters for a direct-PTY shell, since under tmux the browser xterm never enters the alternate buffer; skipped too for a detached session (mirrors `_sendResize()`'s own check and app.js's `_maybeRefetchFullHistory`), since its own window already owns its PTY size and scrollback. Live frames arriving from the response onward are held with their arrival time (`_liveQueue`, opened right after `await fetch(...)` beside `capturedAt`; a frame from before it is replaced by the capture or written unchanged, so the pane keeps painting during the round trip) and replayed in order only if they arrived after the capture (the response's arrival stands in for the capture instant, as in `_finishBufferLoad`, so a frame inside that one round trip can be lost or doubled); the request uses the primary pane's budget (`CodemanFetchDeadline.terminalFetchDeadlineMs({ full: true })`, 45 s, 10 s only if that helper is absent) and the body read gets 10 s once the headers land, because from then on the pull holds the pane's live output. The request phase holds no live output, but it does hold the single-flight flag, so a coalesced `{t:'r'}` refresh and a marker owed by a close (below) wait for the response, at worst for that whole budget (accepted: a Codeman restart resets an in-flight request along with the socket, so that pull fails at once and stamps the marker). ⚠️ The "disconnected" marker must be the LAST thing on screen. A replay's own `\x1bc` would otherwise wipe a marker written before the pull and paint a fresh, current-looking history while `onData` keeps silently dropping every keystroke on the dead socket (a Codeman restart drops the socket while the tmux session, and so the HTTP pull, still succeeds), so `_pullHistory()` re-stamps it after the live-frame flush. A close DURING any load (a pull, a refresh; `connect()` awaits the initial load before it creates the socket, so no close lands in that one) writes nothing: `_onSocketClosed()` sets `_markerOwed` while a load's work runs (`_loadRunning`; a load that only waits in the tile grid's queue writes the marker at once, and its refresh resets the screen only once its own capture is in hand), since written there it would sit above the held frames the pull flushes after a skip, a downgrade, a failed fetch or the deadline, above a refresh's replay, or land mid-way through a chunked replay. Each load settles the marker in its OWN `finally` (`_stampMarkerIfOwed()`), after the queue flush, EXCEPT when a trailing refresh is pending: that refresh settles it itself (its replay's queued `\x1bc` would wipe a marker written now, so it re-owes the marker on a closed socket and writes the one copy below its replay, and a refresh that writes nothing stamps the one still owed). The marker stays owed instead of being written twice; the default fake terminal in `test/terminal-tile-unit.test.ts` writes synchronously, so an async-parse fake there pins what ends up on screen. Anything that wipes the terminal on a closed socket (a pull's or a refresh's `\x1bc`) sets `_markerOwed` too, so the marker is rewritten whether or not the close landed during the load. Tracked via `_wsClosed`/`_markerOwed` rather than routed through `_onLiveOutput()`, since a close landing before the response is stamped before the cutoff and would be dropped with the rest of the pre-capture queue. There is no "Load full history" banner in Pane B, so a shell history past that 1 MiB window stays out of reach there. Non-shell Pane B is unchanged: it already loads `full=1`, and its history is out of scope for this pull (codex and Claude's inline renderer do grow tmux history; this just isn't how they recover it). ⚠️ A server `{t:'c'}` frame is a REFRESH, never a bare `clear()` (`_onLiveClear()` calls `_refreshBuffer()`): its one emitter is a fresh Claude pane's first prompt (`Session.startInteractive`, "refresh after startup"), the primary pane answers it with a refetch and replay (`_onSessionClearTerminal`, which stands aside while the grid is open), and xterm's `clear()` keeps only the cursor's row, so a Claude session Run into the grid or Attached in a tile came up as a near-empty tile that an idle Claude never repainted. As a refresh it coalesces behind a load already running (a pull's held frames included) and waits its turn in the grid's load queue. ⚠️ A refresh (`{t:'r'}`, `{t:'c'}`, a reconnect) runs in the primary pane's order (`_onSessionNeedsRefresh`): fetch FIRST, so the pane keeps its last frame through the round trip and through a grid tile's wait; hold live frames from the response on (`_liveQueue`; the body read of a bounded window, a grid tile's or a shell's, gets the 10 s budget, while Pane B's unbounded `full=1` keeps the request's); then the queued in-stream `\x1bc` immediately before the replay, then the held frames that arrived after the response. Never xterm's `clear()`: it is synchronous while queued bytes are parsed after it (they fused into the snapshot) and it keeps the cursor's row and column, where the capture (raw rows, no home) then started. A failed, aborted or empty fetch writes nothing and resets nothing. ⚠️ App-level terminal actions find their pane through `_focusedPane()`, the terminal focused LAST (not `document.activeElement`, which the mic or a header button takes): Ctrl+L, Ctrl+Shift+R, voice and image paste act on Pane B while it holds the keyboard. Ctrl+W is not an app shortcut at all (Close Session has no default key), so it reaches whichever pane is focused as delete-word. ⚠️ Geometry: `TerminalTile.fit()` sends the size the xterm actually holds, unfloored (the divider's 20% clamp leaves about 28 columns), skips an unchanged size, re-sends on every fresh socket, and adopts the PTY's column count from `{t:'zc'}`; a forced fit (Redraw: Ctrl+Shift+R, the header button, through `restoreTerminalSize`) carries the primary pane's `f` flag, without which `Session.resize` skips a size equal to the one it last applied, and `fit()` returns whether a frame went out, so Redraw reports success only for a resize it sent (a detached session or a socket that is down gets a warning instead); the font, family and weight setters call `tile.fit()`, never a local refit alone (#464). Every capture a pane fetches carries a deadline covering the body (`CodemanFetchDeadline`), so a capture that never answers cannot hold its single-flight flag forever. Design: `docs/split-pane-sessions-plan.md`; the tile class: `docs/tile-grid-plan.md`. +**Split-pane sessions** (`showSplitButton`, header button, default OFF, per-device like `showFileViewerButton` — not in `SettingsUpdateSchema`, `displayKeys` in settings-ui.js): shows two live sessions side-by-side in one Codeman window. Pane A is the untouched, existing singleton terminal (`this.terminal`/`this._ws` in terminal-ui.js); Pane B is an independent `TerminalTile` (terminal-tile.js, constructed by the orchestration in terminal-split.js) with its own xterm instance and its own `/ws/sessions/:id/terminal` WebSocket, whose `cid` is the tab's identity with a `:tile` suffix, so it can never supersede the primary pane's socket (the registry supersedes by cid per session). ⚠️ **Pane B is deliberately plainer than Pane A** — no local-echo overlay, no CJK IME, no touch/mobile handlers, no keyboard accessory bar (it does carry the primary pane's hollow-buffer wheel paging and desktop click report, through terminal-ui.js's own gates aimed at the tile, and its own keyCode-229 soft-keyboard controller from terminal-keycode229-recovery.js, the #441 next-keydown drain and #541's edit-based diff for Android autocorrect, since the 1180px width gate is reachable by a wide Android tablet; see terminal-tile.js's fileoverview) — since this is a desktop-only feature (a split view needs a wide viewport) and those features exist for mobile/touch input; `.btn-split` is hard-hidden below 1180px regardless of the setting by the `@media (max-width: 1179px)` rule in styles.css (mobile.css only carries a comment pointing at it: that file loads up to 1023px, so it cannot cover the 1024-1179px tablet range the feature also needs to stay off), and the per-device setting means turning it on at a desk can never sync it onto a phone in the first place. No persistence: closing the browser tab or reloading always returns to the normal single-pane view; there is no localStorage key for split state. ⚠️ Splitting a session against itself is disallowed (the picker excludes the active session), **as is splitting against a popped-out (detached) session** — `buildSplitPickerSessions()` excludes `detachedSessions` because a detached session's own window is already claiming its PTY size, and `MAX_WS_PER_SESSION` needs no change since Pane A/B are always two different sessions. ⚠️ Either pane's session ending (deleted locally or from another client) auto-collapses the split — Pane A's session ending promotes Pane B to the new single pane via `selectSession(id, { auto: true })` (an app-driven selection, so it must not spend the session's idle alert — see the Approvals Inbox note above), never by trying to hot-swap the lightweight `TerminalTile` object into the primary singleton state. ⚠️ Pane B refits on every window/sidebar/tab-rail resize via the SAME trailing-edge `ResizeObserver` callback that resizes Pane A (`throttledResize` in terminal-ui.js) — it only ever measured Pane A's own container, so without the explicit `this._forEachTile?.((tile) => tile.fit(), { grid: false })` call there (grid tiles are left out: the grid's own observer refits them on the same resize) Pane B silently kept its stale PTY size through every resize that did not happen to be a divider drag. ⚠️ A dropped WebSocket writes a `[disconnected, reconnecting…]` marker into Pane B and reconnects on the primary pane's backoff ladder (`CodemanWsReconnect` plus jitter; the attempt count resets only on a successful open), then refreshes the buffer to close the output gap. ⚠️ The open handler clears `_wsClosed`/`_markerOwed` BEFORE that refresh, or the refresh re-owes the marker and stamps it under a healthy pane. ⚠️ 4003/4004/4009/4010 stop the pane for good, report once through `onExit(code)` and write a marker saying why (`CodemanWsReconnect` alone would retry 4003). ⚠️ A replacement socket detaches the old one first and every handler ignores a socket that is no longer `this.ws`, so a late 4010 from a superseded socket cannot stop its successor; `destroy()` cancels a pending reconnect. Related but distinct: `detachSession()` already opens one session in a separate OS-level browser window (`isSoloWindow`) — that is prior art for "two sessions visible at once" but not for one window with a draggable in-page divider, which is what this feature adds. ⚠️ Pane B installs its own `attachCustomKeyEventHandler` gating the same app-level chords Pane A's own handler gates (command palette, Alt+1-9/[/] tab nav, Alt+B sidebar toggle, Ctrl+Z suspend, Shift/Ctrl+Enter newline, smart-copy Ctrl+C) — without it the document capture-phase handler's `preventDefault()` (which never stops xterm) let each chord ALSO write its raw byte/escape sequence into Pane B's live PTY on top of whatever the app action did to Pane A (COD-153). Ctrl+Z is swallowed unless Pane B's own session is `mode === 'shell'`, mirroring terminal-ui.js's reasoning: in a plain shell it is the user's own job-control tool, everywhere else it silently suspends an unattended agent loop. Shift/Ctrl+Enter POSTs to `/api/sessions/:id/send-key` (`{key:'S-Enter'|'C-Enter'}`, typed by tmux `send-keys -H`: Ctrl+Enter is always a real 0x0a, Shift+Enter is the CLI's declared `capabilities.newline` chord, also 0x0a unless the CLI declares another; sent on keydown only, with the keypress and keyup swallowed too, see the keypress rule under Command palette and shortcut registry) targeting THIS pane's own `sessionId` rather than the primary pane's `activeSessionId` — without it xterm's plain `\r` would submit an incomplete prompt instead of adding a line to it. Smart-copy Ctrl+C/Ctrl+Shift+C is re-implemented against `this.terminal` (Pane B's own) rather than reusing `app.copyTerminalSelection()`, which reads Pane A's terminal and would copy the wrong pane's selection; Ctrl+Shift+C never falls through even with nothing to copy, mirroring terminal-ui.js's own `ev.shiftKey` branch. ⚠️ **This is a UX-parity fix, not an interrupt-safety one** — verified live in a real browser: xterm's `evaluateKeyboardEvent` routes a shifted ctrl-letter into a branch that assigns `c.key` only for two special cases (`_`→US, `@`→NUL), so it emits no data for Ctrl+Shift+C at all regardless of any application gate; a synthetic keydown with the gate removed produces zero WS frames, proving no accidental interrupt reaches the PTY either way. What gating the whole copy block on `hasSelection()` (an earlier draft) actually cost: with no selection, a selection-less Ctrl+Shift+C fell straight to `return true`, silently ceding the keystroke to the BROWSER's own handling (e.g. Chrome's Inspect-Element binding) with no feedback and no copy attempt — Pane A always intercepts it. Ctrl+V goes through the primary pane's paste trap aimed at Pane B (`_handleImagePaste({ terminal, sessionId })`), so a pasted image uploads to Pane B's session. ⚠️ `buildSplitPickerSessions()` also excludes any session with `pid === null` (an exited CLI, a crash-looped session whose breaker tripped, a restore that never re-attached): Pane B has no equivalent of `selectSession()`'s auto re-attach POST, so a pane opened onto one has nothing reading its tmux pane — no `terminal` events ever arrive, and `Session.write()` silently drops every keystroke with no ack either way, so the loss is invisible behind a socket that reports healthy. Pane B's keystrokes go through the exactly-once queue (`_sendInputAsync`) over its own socket, registered in the app's input-socket map (`_registerInputSocket` / `_inputSocketFor`), so they are seq-tagged, ACKed (`{t:'ia'}` is routed by the RECEIVING socket's session, since the frame names none), redelivered after a drop, and the ACK clears the idle alert. ⚠️ What xterm generates never enters that persisted queue: query replies are dropped (`shouldSuppressTerminalQueryResponse`, as the primary pane drops them) and focus/mouse reports go through `_sendInputEphemeral`, or a reload would replay them as typed text. That includes the click report a tile hand-encodes for a mouse-strip CLI: `TerminalTile._installClickListener` passes `ephemeral: true` in its target, which `_sendSyntheticSgrTap` honours, while the primary pane's own report (no flag) stays on `_sendInputAsync` as before. ⚠️ A SHELL Pane B pulls scrollback itself when the wheel goes up at the top of its buffer (`_maybeLoadMoreHistory`/`_pullHistory`): tmux repaints a burst of output instead of scrolling it, so Pane B's own xterm holds about one screen of scrollback while tmux holds every line, and it loaded history exactly once at connect and never again. It is the same bounded pull as Pane A's (`?full=1&tail=TERMINAL_TAIL_SIZE`, no rewrite when the window holds no more rows than the pane already has or the pane is at its `scrollback + rows` cap, and a 60 s back-off instead of 4 s when that skipped window was truncated or the pane is full, since each ask costs the server a whole-history `capture-pane`), against Pane B's OWN terminal rather than `app.terminal`, so it cannot share `_maybeRefetchFullHistory`. The wheel listener is capture-phase because xterm `stopPropagation()`s the events it consumes; the alternate-screen skip (nano, vim, less) only matters for a direct-PTY shell, since under tmux the browser xterm never enters the alternate buffer; skipped too for a detached session (mirrors `_sendResize()`'s own check and app.js's `_maybeRefetchFullHistory`), since its own window already owns its PTY size and scrollback. Live frames arriving from the response onward are held with their arrival time (`_liveQueue`, opened right after `await fetch(...)` beside `capturedAt`; a frame from before it is replaced by the capture or written unchanged, so the pane keeps painting during the round trip) and replayed in order only if they arrived after the capture (the response's arrival stands in for the capture instant, as in `_finishBufferLoad`, so a frame inside that one round trip can be lost or doubled); the request uses the primary pane's budget (`CodemanFetchDeadline.terminalFetchDeadlineMs({ full: true })`, 45 s, 10 s only if that helper is absent) and the body read gets 10 s once the headers land, because from then on the pull holds the pane's live output. The request phase holds no live output, but it does hold the single-flight flag, so a coalesced `{t:'r'}` refresh and a marker owed by a close (below) wait for the response, at worst for that whole budget (accepted: a Codeman restart resets an in-flight request along with the socket, so that pull fails at once and stamps the marker). ⚠️ The "disconnected" marker must be the LAST thing on screen. A replay's own `\x1bc` would otherwise wipe a marker written before the pull and paint a fresh, current-looking history while `onData` keeps silently dropping every keystroke on the dead socket (a Codeman restart drops the socket while the tmux session, and so the HTTP pull, still succeeds), so `_pullHistory()` re-stamps it after the live-frame flush. A close DURING any load (a pull, a refresh; `connect()` awaits the initial load before it creates the socket, so no close lands in that one) writes nothing: `_onSocketClosed()` sets `_markerOwed` while a load's work runs (`_loadRunning`; a load that only waits in the tile grid's queue writes the marker at once, and its refresh resets the screen only once its own capture is in hand), since written there it would sit above the held frames the pull flushes after a skip, a downgrade, a failed fetch or the deadline, above a refresh's replay, or land mid-way through a chunked replay. Each load settles the marker in its OWN `finally` (`_stampMarkerIfOwed()`), after the queue flush, EXCEPT when a trailing refresh is pending: that refresh settles it itself (its replay's queued `\x1bc` would wipe a marker written now, so it re-owes the marker on a closed socket and writes the one copy below its replay, and a refresh that writes nothing stamps the one still owed). The marker stays owed instead of being written twice; the default fake terminal in `test/terminal-tile-unit.test.ts` writes synchronously, so an async-parse fake there pins what ends up on screen. Anything that wipes the terminal on a closed socket (a pull's or a refresh's `\x1bc`) sets `_markerOwed` too, so the marker is rewritten whether or not the close landed during the load. Tracked via `_wsClosed`/`_markerOwed` rather than routed through `_onLiveOutput()`, since a close landing before the response is stamped before the cutoff and would be dropped with the rest of the pre-capture queue. There is no "Load full history" banner in Pane B, so a shell history past that 1 MiB window stays out of reach there. Non-shell Pane B is unchanged: it already loads `full=1`, and its history is out of scope for this pull (codex and Claude's inline renderer do grow tmux history; this just isn't how they recover it). ⚠️ A server `{t:'c'}` frame is a REFRESH, never a bare `clear()` (`_onLiveClear()` calls `_refreshBuffer()`): its one emitter is a fresh Claude pane's first prompt (`Session.startInteractive`, "refresh after startup"), the primary pane answers it with a refetch and replay (`_onSessionClearTerminal`, which stands aside while the grid is open), and xterm's `clear()` keeps only the cursor's row, so a Claude session Run into the grid or Attached in a tile came up as a near-empty tile that an idle Claude never repainted. As a refresh it coalesces behind a load already running (a pull's held frames included) and waits its turn in the grid's load queue. ⚠️ A refresh (`{t:'r'}`, `{t:'c'}`, a reconnect) runs in the primary pane's order (`_onSessionNeedsRefresh`): fetch FIRST, so the pane keeps its last frame through the round trip and through a grid tile's wait; hold live frames from the response on (`_liveQueue`; the body read of a bounded window, a grid tile's or a shell's, gets the 10 s budget, while Pane B's unbounded `full=1` keeps the request's); then the queued in-stream `\x1bc` immediately before the replay, then the held frames that arrived after the response. Never xterm's `clear()`: it is synchronous while queued bytes are parsed after it (they fused into the snapshot) and it keeps the cursor's row and column, where the capture (raw rows, no home) then started. A failed, aborted or empty fetch writes nothing and resets nothing. ⚠️ Live output is flow-controlled per pane (`_writeLive`), because the server applies no backpressure and a flood a tile cannot parse as fast (a shell tile running `cat` on a huge log) used to pile up in xterm's own queue until its WriteBuffer threw past 50M code units and onmessage's catch lost the frames silently: unparsed output is counted through each write's callback, and held output by the queue, against `TerminalTile.LIVE_BACKLOG_BUDGET` (4 MiB; deliberately NOT the primary pane's 128 KB, which caps its own rAF-paced queues, while xterm itself paces a tile and a tight cap would blank-and-reload it on ordinary bursts). Past it a frame is dropped, the tile stops writing onto the hole, and one refresh, debounced and bounded by the primary pane's own rule (`CodemanDroppedOutput`: 2 s, at most `DROP_RECOVERY_MAX_ATTEMPTS`, never retried after a deadline abort), recaptures the screen; the flag clears only once a capture taken after the last dropped frame has replayed. A write xterm throws on is the same drop. Past the bound the flag is released (the tile is never left frozen), a reconnect starts the accounting over (an epoch makes callbacks from before it count nothing) and drops a pending recovery, since its own refresh replaces the screen, and `destroy()` cancels it. ⚠️ App-level terminal actions find their pane through `_focusedPane()`, the terminal focused LAST (not `document.activeElement`, which the mic or a header button takes): Ctrl+L, Ctrl+Shift+R, voice and image paste act on Pane B while it holds the keyboard. Ctrl+W is not an app shortcut at all (Close Session has no default key), so it reaches whichever pane is focused as delete-word. ⚠️ Geometry: `TerminalTile.fit()` sends the size the xterm actually holds, unfloored (the divider's 20% clamp leaves about 28 columns), skips an unchanged size, re-sends on every fresh socket, and adopts the PTY's column count from `{t:'zc'}`; a forced fit (Redraw: Ctrl+Shift+R, the header button, through `restoreTerminalSize`) carries the primary pane's `f` flag, without which `Session.resize` skips a size equal to the one it last applied, and `fit()` returns whether a frame went out, so Redraw reports success only for a resize it sent (a detached session or a socket that is down gets a warning instead); the font, family and weight setters call `tile.fit()`, never a local refit alone (#464). Every capture a pane fetches carries a deadline covering the body (`CodemanFetchDeadline`), so a capture that never answers cannot hold its single-flight flag forever. Design: `docs/split-pane-sessions-plan.md`; the tile class: `docs/tile-grid-plan.md`. ### Tile grid diff --git a/src/web/public/terminal-tile.js b/src/web/public/terminal-tile.js index 54a34968..222ebc65 100644 --- a/src/web/public/terminal-tile.js +++ b/src/web/public/terminal-tile.js @@ -159,7 +159,19 @@ this._historyPullAt = 0; this._historyPullUseless = false; this._liveQueue = null; + this._liveQueueBytes = 0; this._markerOwed = false; + // Live-output flow control (_writeLive, TerminalTile.LIVE_BACKLOG_BUDGET): + // code units written into this xterm and not yet parsed (each write's + // callback counts its own back down, unless a reset bumped `_liveEpoch` + // since), whether output was dropped and not yet recovered, when the last + // frame was dropped, and the debounced, bounded recovery refresh. + this._liveInFlight = 0; + this._liveEpoch = 0; + this._liveDropped = false; + this._liveDropAt = 0; + this._dropRecoveryTimer = null; + this._dropRecoveryAttempt = 0; this._onWheel = null; // `{ ws, lastRecvAt }`, registered with the app's input-socket map while // this pane's socket is open, so the exactly-once input queue delivers this @@ -507,7 +519,12 @@ this._lastSentDims = null; this._registerInputSocket(); this._sendResize(); - if (reconnected) this._refreshBuffer(); + if (reconnected) { + // The gap already cost output, and the refresh below replaces the + // screen, so live-output accounting starts over with it. + this._resetLiveFlow(); + this._refreshBuffer(); + } } // Opens a replacement socket now instead of waiting out the backoff (for an @@ -762,7 +779,7 @@ // TERMINAL_TAIL_SIZE) gets the pull's short budget; an unbounded // capture (the split's Pane B, up to 32 MB) keeps the request's. capturedAt = performance.now(); - this._liveQueue = []; + this._openLiveQueue(); if (shell || this.boundedLoad) armDeadline(HISTORY_PULL_TIMEOUT_MS); } payload = (await res.json())?.data ?? {}; @@ -795,6 +812,9 @@ clearTimeout(abortTimer); this._loadAbort = null; this._loadRunning = false; + // Before the flush: a refresh that recovered dropped output lets its + // held frames through. + if (refresh) this._settleDropRecovery({ replayed, capturedAt, timedOut: !!controller?.signal?.aborted }); // Held frames before the marker, so the marker stays the last thing on // screen (see _pullHistory()). this._flushLiveQueue(replayed ? capturedAt : 0); @@ -846,24 +866,149 @@ // tmux took it, so a frame arriving mid-replay is held with its arrival time // and written behind the snapshot by that load's _flushLiveQueue() (the // primary pane's _finishBufferLoad `since` rule), never underneath it. + // Held frames count against the same budget as unparsed ones: a pull or a + // refresh holds output for up to its body budget, and a flood meanwhile + // must not grow the queue without bound either. _onLiveOutput(data) { - if (this._liveQueue) this._liveQueue.push({ at: performance.now(), data }); - else this.terminal?.write(data); + if (!data) return; + if (this._liveQueue) { + if (this._liveQueueBytes + data.length > TerminalTile.LIVE_BACKLOG_BUDGET) { + this._noteLiveDrop(); + return; + } + this._liveQueueBytes += data.length; + this._liveQueue.push({ at: performance.now(), data }); + return; + } + this._writeLive(data); + } + + _openLiveQueue() { + this._liveQueue = []; + this._liveQueueBytes = 0; } // Releases the frames a load held (_liveQueue) and closes the queue. After a // replay only those that arrived after the capture are news (`cutoff`, the // response's arrival; earlier ones are already in it); with no replay - // (`cutoff` 0) every one is. + // (`cutoff` 0) every one is. Through _writeLive(), so they are counted (and + // a write that throws cannot skip the load's marker and trailing refresh). _flushLiveQueue(cutoff) { const queued = this._liveQueue ?? []; this._liveQueue = null; + this._liveQueueBytes = 0; for (const entry of queued) { if (entry.at < cutoff) continue; - this.terminal?.write(entry.data); + this._writeLive(entry.data); } } + // Writes one live frame into this xterm, under flow control. The server + // applies no backpressure (16 KB / 8 ms batches, never a bufferedAmount + // check), so a flood a tile cannot parse as fast as it arrives (a shell + // tile running `cat` on a huge log, `yes`) used to pile up in xterm's own + // write queue without bound, on a main thread six tiles share, until + // xterm's WriteBuffer throws past 50M code units and the frames were + // silently lost in onmessage's catch. The primary pane caps its queues + // and drops then recaptures (_onSessionTerminal, app.js); this is the + // tile's equivalent. Past TerminalTile.LIVE_BACKLOG_BUDGET unparsed, a + // frame is dropped and the tile stops writing until a refresh recaptures + // the screen (_scheduleDropRecovery): every byte after a hole is written + // onto a screen out of step with the PTY, which that refresh replaces + // anyway. A write that throws is the same drop, never a malformed frame. + _writeLive(data) { + const terminal = this.terminal; + if (!terminal || this._destroyed || !data) return; + if (this._liveDropped) { + this._noteLiveDrop(); + return; + } + const n = data.length; + if (this._liveInFlight + n > TerminalTile.LIVE_BACKLOG_BUDGET) { + this._noteLiveDrop(); + return; + } + const epoch = this._liveEpoch; + this._liveInFlight += n; + try { + terminal.write(data, () => { + if (epoch === this._liveEpoch) this._liveInFlight -= n; + }); + } catch { + if (epoch === this._liveEpoch) this._liveInFlight -= n; + this._noteLiveDrop(); + } + } + + // A live frame was dropped. Marks the tile out of step and arms ONE + // recovery; later drops only move the stamp the recovery has to beat. + _noteLiveDrop() { + this._liveDropAt = performance.now(); + if (this._liveDropped) return; + this._liveDropped = true; + this._scheduleDropRecovery(); + } + + // The primary pane's dropped-output recovery (_scheduleDroppedOutputRecovery, + // app.js), aimed at this tile: debounced by DROP_RECOVERY_DELAY_MS so a + // sustained flood collapses into one attempt, and run as an ordinary + // refresh, which is single-flight, bounded (`lines=`/`tail=`) and waits its + // turn in the grid's load queue. _settleDropRecovery() decides what the + // refresh it starts achieved. + _scheduleDropRecovery() { + if (this._dropRecoveryTimer || this._destroyed) return; + const delay = global.CodemanDroppedOutput?.DROP_RECOVERY_DELAY_MS ?? 2000; + this._dropRecoveryTimer = setTimeout(() => { + this._dropRecoveryTimer = null; + if (this._destroyed || !this._liveDropped) return; + this._dropRecoveryAttempt++; + this._refreshBuffer(); + }, delay); + } + + // A refresh finished while output was marked dropped. Recovered when its + // replay's capture was taken after the last dropped frame (the response's + // arrival, the cutoff every load uses): output flows again. Otherwise one + // more attempt, bounded by the primary pane's rule + // (shouldRetryDroppedOutputRecovery: DROP_RECOVERY_MAX_ATTEMPTS, and never + // after a capture cut off at its deadline, a stalled link); past that the + // flag is released so the tile is never left frozen, and it writes on, out + // of step, as every tile did before this existed. + _settleDropRecovery({ replayed, capturedAt, timedOut }) { + if (!this._liveDropped || this._destroyed) return; + if (replayed && capturedAt >= this._liveDropAt) { + this._liveDropped = false; + this._dropRecoveryAttempt = 0; + return; + } + // Another try is already on its way: the debounce, or a trailing refresh. + if (this._dropRecoveryTimer || this._bufferRefreshPending) return; + const retry = + global.CodemanDroppedOutput?.shouldRetryDroppedOutputRecovery?.({ + repainted: false, + timedOut, + attempt: Math.max(0, this._dropRecoveryAttempt - 1), + stillActive: true, + }) === true; + if (retry) { + this._scheduleDropRecovery(); + return; + } + this._liveDropped = false; + this._dropRecoveryAttempt = 0; + } + + // Starts live-output accounting over (a reconnect, destroy): write callbacks + // still pending from before carry the old epoch and count nothing. + _resetLiveFlow() { + this._liveEpoch++; + this._liveInFlight = 0; + this._liveDropped = false; + this._dropRecoveryAttempt = 0; + clearTimeout(this._dropRecoveryTimer); + this._dropRecoveryTimer = null; + } + // The server's `{t:'c'}` frame, which is a refresh, not a wipe. Its one // emitter (Session.startInteractive, session.ts) sends it once a fresh Claude // pane first shows its prompt: the server has just trimmed its own buffer and @@ -1099,7 +1244,7 @@ // Opened only now: a frame from before the response is either replaced by // the capture or written unchanged, so holding it for the round trip // would buy nothing and freeze the pane for as long as the fetch took. - this._liveQueue = []; + this._openLiveQueue(); const payload = (await res.json())?.data; clearTimeout(abortTimer); const buffer = payload?.terminalBuffer; @@ -1307,6 +1452,9 @@ this.mountEl?.removeEventListener('click', this._onClick); this._onClick = null; } + // A disposed xterm never runs its write callbacks, and a pending + // recovery would refresh a pane nobody can see. + this._resetLiveFlow(); // Page keys still waiting for their flush go nowhere: the pane is gone. clearTimeout(this._scrollFlushTimer); this._scrollFlushTimer = null; @@ -1332,6 +1480,15 @@ } } + // Code units of live output a pane lets sit unparsed in its xterm (or held + // behind a replay) before it drops a frame and recaptures (_writeLive). Not + // the primary pane's 128 KB: that caps its own rAF-paced queues, about two + // frames of them, while here xterm itself is the pacer, a burst normally + // parses within a frame or two, and a tight cap would trip on ordinary + // bursts and blank-and-reload the tile over and over. A few MB keeps a flood + // far below xterm's 50M code-unit throw and bounds each tile's memory. + TerminalTile.LIVE_BACKLOG_BUDGET = 4 * 1024 * 1024; + // The marker a pane writes when its socket drops: a transient drop says it is // reconnecting; a permanent stop says why, keyed by close code. All start // with `[disconnected` so a reader (and a test) can tell any of them apart diff --git a/test/mocks/terminal-tile-fakes.ts b/test/mocks/terminal-tile-fakes.ts index eaddc2f1..2b70614a 100644 --- a/test/mocks/terminal-tile-fakes.ts +++ b/test/mocks/terminal-tile-fakes.ts @@ -149,9 +149,23 @@ export class FakeTerminal { } registerLinkProvider() {} writes: string[] = []; - /** Set by a test: write callbacks never run, as on a disposed xterm. */ + /** + * Set by a test: write callbacks do not run, as while xterm is still parsing + * (or never, on a disposed xterm). They wait in `heldParses` until `parse()`. + */ holdParse = false; + heldParses: Array<() => void> = []; + /** xterm catches up: runs every write callback held so far, in order. */ + parse() { + for (const cb of this.heldParses.splice(0)) cb(); + } + /** Set by a test: the next write of exactly this data throws, as xterm's WriteBuffer does past 50M. */ + throwOnWrite: string | null = null; write(data: string, cb?: () => void) { + if (this.throwOnWrite !== null && data === this.throwOnWrite) { + this.throwOnWrite = null; + throw new Error('write data discarded, use flow control to avoid losing data'); + } // An empty write puts nothing on screen; the replay queues one only to hear // (its callback) that everything before it has been parsed. if (data) this.writes.push(data); @@ -162,7 +176,9 @@ export class FakeTerminal { this.lineCount += data.slice(reset === -1 ? 0 : reset + 2).split('\n').length - 1; this.settleRows(); } - if (!this.holdParse) cb?.(); + if (!cb) return; + if (this.holdParse) this.heldParses.push(cb); + else cb(); } clear() { this.writes.push(''); diff --git a/test/terminal-tile-input.test.ts b/test/terminal-tile-input.test.ts index ec77e81d..41231a77 100644 --- a/test/terminal-tile-input.test.ts +++ b/test/terminal-tile-input.test.ts @@ -802,6 +802,252 @@ describe('TerminalTile claims the keyboard for the app-level shortcuts', () => { }); }); +describe('TerminalTile live-output flow control: a flood cannot pile up in xterm', () => { + // The server applies no backpressure, and a tile used to write every live + // frame straight into xterm: a flood it could not parse as fast piled up in + // xterm's own queue without bound, until xterm's WriteBuffer threw past 50M + // code units and onmessage's catch silently lost the frames. Now unparsed + // (and held) output is counted per tile; past the budget a frame is dropped, + // the tile stops writing onto the hole, and one debounced, bounded refresh + // recaptures the screen (the primary pane's _scheduleDroppedOutputRecovery). + const TileStatics = TerminalTile as unknown as { LIVE_BACKLOG_BUDGET: number }; + const defaultBudget = TileStatics.LIVE_BACKLOG_BUDGET; + afterEach(() => { + TileStatics.LIVE_BACKLOG_BUDGET = defaultBudget; + delete windowStub.AbortController; + }); + const out = (ws: FakeSocket, d: string) => ws.receive({ t: 'o', d }); + const inFlight = (tile: Tile) => (tile as unknown as { _liveInFlight: number })._liveInFlight; + function serve(terminalBuffer: string) { + fetchMock.mockClear(); + fetchMock.mockImplementation(async () => ({ + ok: true, + status: 200, + json: async () => ({ data: { terminalBuffer } }), + })); + } + + it("the budget is a few MB, not the primary pane's 128 KB: a 1 MiB burst xterm has not parsed yet is still written", async () => { + expect(defaultBudget).toBeGreaterThanOrEqual(2 * 1024 * 1024); + expect(defaultBudget).toBeLessThanOrEqual(16 * 1024 * 1024); + vi.useFakeTimers(); + const { ws, term } = await connectTile(makeApp()); + ws.open(); + serve(''); + term.holdParse = true; + + out(ws, 'z'.repeat(1024 * 1024)); + + expect(term.writes.at(-1)).toHaveLength(1024 * 1024); + await vi.advanceTimersByTimeAsync(10_000); + expect(fetchMock).not.toHaveBeenCalled(); + }); + + it('counts output until xterm has parsed it, so parsed bytes no longer hold the budget', async () => { + TileStatics.LIVE_BACKLOG_BUDGET = 100; + const { tile, ws, term } = await connectTile(makeApp()); + ws.open(); + term.holdParse = true; + + out(ws, 'a'.repeat(40)); + out(ws, 'b'.repeat(40)); + expect(inFlight(tile)).toBe(80); + term.parse(); + expect(inFlight(tile)).toBe(0); + out(ws, 'c'.repeat(90)); + + expect(term.writes.slice(-3)).toEqual(['a'.repeat(40), 'b'.repeat(40), 'c'.repeat(90)]); + }); + + it('stops writing past the budget, and ONE debounced refresh recaptures the screen', async () => { + TileStatics.LIVE_BACKLOG_BUDGET = 100; + vi.useFakeTimers(); + const { ws, term } = await connectTile(makeApp()); + ws.open(); + serve('recovered screen'); + term.holdParse = true; // xterm falls behind + + out(ws, 'a'.repeat(60)); + out(ws, 'b'.repeat(60)); // 120 unparsed: past the budget, dropped + out(ws, 'c'); + term.parse(); // xterm catches up, but the stream already has a hole: + out(ws, 'd'); // nothing more is written onto it + expect(term.writes.filter((w) => /^[abcd]/.test(w))).toEqual(['a'.repeat(60)]); + expect(fetchMock).not.toHaveBeenCalled(); + + term.holdParse = false; + await vi.advanceTimersByTimeAsync(1999); + expect(fetchMock).not.toHaveBeenCalled(); // debounced, like the primary pane's + await vi.advanceTimersByTimeAsync(1); + await settle(); + expect(fetchMock).toHaveBeenCalledTimes(1); + expect(fetchMock.mock.calls[0][0]).toBe('/api/sessions/s-tile/terminal?full=1'); + expect(term.writes.slice(-2)).toEqual(['\x1bc', 'recovered screen']); + + out(ws, 'live again'); + expect(term.writes.at(-1)).toBe('live again'); + await vi.advanceTimersByTimeAsync(10_000); + expect(fetchMock).toHaveBeenCalledTimes(1); // one recovery for the whole burst + }); + + it("a write xterm refuses (its 50M throw) is a drop that schedules the same recovery, never a swallowed 'malformed frame'", async () => { + vi.useFakeTimers(); + const { ws, term } = await connectTile(makeApp()); + ws.open(); + serve('recovered screen'); + term.throwOnWrite = 'boom'; + + out(ws, 'boom'); + out(ws, 'after the hole'); + expect(term.writes).not.toContain('after the hole'); + + await vi.advanceTimersByTimeAsync(2000); + await settle(); + expect(fetchMock).toHaveBeenCalledTimes(1); + expect(term.writes.slice(-2)).toEqual(['\x1bc', 'recovered screen']); + }); + + it('frames held behind a replay count against the budget too, and a hole there is recovered by another refresh', async () => { + TileStatics.LIVE_BACKLOG_BUDGET = 100; + vi.useFakeTimers(); + const { ws, term } = await connectTile(makeApp()); + ws.open(); + fetchMock.mockClear(); + let answer!: (body: string) => void; + fetchMock.mockImplementationOnce(async () => ({ + ok: true, + status: 200, + json: () => + new Promise((resolveBody) => { + answer = (terminalBuffer) => resolveBody({ data: { terminalBuffer } }); + }), + })); + fetchMock.mockImplementation(async () => ({ + ok: true, + status: 200, + json: async () => ({ data: { terminalBuffer: 'second capture' } }), + })); + + ws.receive({ t: 'r' }); + await settle(); // the headers landed: from here frames are held + out(ws, 'x'.repeat(60)); + out(ws, 'y'.repeat(60)); // the held queue would pass the budget: dropped + answer('first capture'); + await settle(); + + // The capture predates the hole, so nothing after it is written onto it. + expect(term.writes.slice(-2)).toEqual(['\x1bc', 'first capture']); + expect(term.writes).not.toContain('y'.repeat(60)); + await vi.advanceTimersByTimeAsync(2000); + await settle(); + expect(fetchMock).toHaveBeenCalledTimes(2); + expect(term.writes.slice(-2)).toEqual(['\x1bc', 'second capture']); + out(ws, 'live again'); + expect(term.writes.at(-1)).toBe('live again'); + }); + + it('a recovery that keeps failing is retried a bounded number of times, then lets live output through again', async () => { + vi.useFakeTimers(); + const { ws, term } = await connectTile(makeApp()); + ws.open(); + fetchMock.mockClear(); + fetchMock.mockImplementation(async () => { + throw new Error('offline'); + }); + term.throwOnWrite = 'boom'; + + out(ws, 'boom'); + out(ws, 'held back'); + for (let i = 0; i < 10; i++) { + await vi.advanceTimersByTimeAsync(2000); + await settle(); + } + + const { DROP_RECOVERY_MAX_ATTEMPTS } = windowStub.CodemanDroppedOutput as { DROP_RECOVERY_MAX_ATTEMPTS: number }; + expect(fetchMock).toHaveBeenCalledTimes(DROP_RECOVERY_MAX_ATTEMPTS); + expect(term.writes).not.toContain('held back'); + out(ws, 'flowing'); + expect(term.writes.at(-1)).toBe('flowing'); // never left frozen + }); + + it('a recovery cut off at its deadline is not retried (a stalled link), and lets live output through again', async () => { + windowStub.AbortController = AbortController; + vi.useFakeTimers(); + const { ws, term } = await connectTile(makeApp()); + ws.open(); + fetchMock.mockClear(); + fetchMock.mockImplementation( + (_url: string, init?: { signal?: AbortSignal }) => + new Promise((_resolveFetch, rejectFetch) => { + init?.signal?.addEventListener('abort', () => rejectFetch(new Error('aborted'))); + }) + ); + term.throwOnWrite = 'boom'; + + out(ws, 'boom'); + await vi.advanceTimersByTimeAsync(2000); + expect(fetchMock).toHaveBeenCalledTimes(1); + await vi.advanceTimersByTimeAsync(45_000); // the full-capture budget runs out + await settle(); + + out(ws, 'flowing'); + expect(term.writes.at(-1)).toBe('flowing'); + await vi.advanceTimersByTimeAsync(20_000); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it('a reconnect starts the count over, and callbacks from before it count nothing', async () => { + TileStatics.LIVE_BACKLOG_BUDGET = 100; + vi.useFakeTimers(); + const { tile, ws, term } = await connectTile(makeApp()); + ws.open(); + term.holdParse = true; + out(ws, 'a'.repeat(60)); + expect(inFlight(tile)).toBe(60); + + ws.drop(1006); + await vi.advanceTimersByTimeAsync(300); + FakeSocket.instances.at(-1)!.open(); // reconnected: its refresh replaces the screen + expect(inFlight(tile)).toBe(0); + term.parse(); // the old write's callback lands after the reset + expect(inFlight(tile)).toBe(0); + }); + + it('a reconnect drops a pending recovery: its own refresh replaces the screen', async () => { + vi.useFakeTimers(); + const { ws, term } = await connectTile(makeApp()); + ws.open(); + serve(''); + term.throwOnWrite = 'boom'; + out(ws, 'boom'); + + ws.drop(1006); + await vi.advanceTimersByTimeAsync(300); + const ws2 = FakeSocket.instances.at(-1)!; + ws2.open(); + await settle(); + expect(fetchMock).toHaveBeenCalledTimes(1); // the reconnect's refresh + out(ws2, 'flowing'); + expect(term.writes.at(-1)).toBe('flowing'); + await vi.advanceTimersByTimeAsync(5000); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it('destroy() cancels a pending recovery', async () => { + vi.useFakeTimers(); + const { tile, ws, term } = await connectTile(makeApp()); + ws.open(); + fetchMock.mockClear(); + term.throwOnWrite = 'boom'; + out(ws, 'boom'); + + tile.destroy(); + await vi.advanceTimersByTimeAsync(5000); + + expect(fetchMock).not.toHaveBeenCalled(); + }); +}); + describe('the server coming back kicks Pane B', () => { it("handleInit's reconnect branch asks the split pane's tile to reconnect without waiting out its backoff", () => { // handleInit needs a whole app to run, so the wiring is pinned by source; diff --git a/test/terminal-tile-unit.test.ts b/test/terminal-tile-unit.test.ts index 55f95e50..e24f6680 100644 --- a/test/terminal-tile-unit.test.ts +++ b/test/terminal-tile-unit.test.ts @@ -185,6 +185,10 @@ const isMarker = (data: unknown) => typeof data === 'string' && data.includes('[ */ const screenWrites = (pane: { terminal: FakeTerminal }) => pane.terminal.write.mock.calls.map((call) => call[0]).filter((data) => data !== ''); +// Live frames are written with a callback (the tile counts them until xterm +// has parsed them, _writeLive), so they are matched on the data argument +// alone, never with toHaveBeenCalledWith(data): that would also miss, and a +// `.not` on it would then pass for nothing. /** * Holds xterm's write callbacks, as a real xterm still parsing a replay does: @@ -738,7 +742,7 @@ describe('TerminalTile scroll-to-top history pull', () => { // keeps painting during the round trip), and the replay then replaces it. clock = 1; pane._onLiveOutput('early'); - expect(term.write).toHaveBeenCalledWith('early'); + expect(screenWrites(pane)).toContain('early'); await settle(); // 200 rows (more than the pane holds, so it replays) of 400 columns each: @@ -748,12 +752,14 @@ describe('TerminalTile scroll-to-top history pull', () => { clock = 2; // the response arrives: this is the cutoff response.resolve(jsonResponse(bigReplay)); await settle(); - expect(xterm.held).toHaveLength(1); + // Two parses pending: 'early' (a live write, counted until xterm parses it) + // and the replay's end marker. + expect(xterm.held).toHaveLength(2); // Arrives while the snapshot is still being parsed: must not land under it. clock = 3; pane._onLiveOutput('late'); - expect(term.write).not.toHaveBeenCalledWith('late'); + expect(screenWrites(pane)).not.toContain('late'); // The replay parsed, then the pull's own settle write before it scrolls. xterm.parse(); @@ -778,12 +784,12 @@ describe('TerminalTile scroll-to-top history pull', () => { pane._maybeLoadMoreHistory(); await settle(); pane._onLiveOutput('held'); - expect(pane.terminal.write).not.toHaveBeenCalledWith('held'); + expect(screenWrites(pane)).not.toContain('held'); held.release(rowsOf(30)); // nothing to gain: no replay await settle(); // Nothing replaced the terminal, so the held frame is news. - expect(pane.terminal.write).toHaveBeenCalledWith('held'); + expect(screenWrites(pane)).toContain('held'); }); it('a failed fetch releases the flag and the queue, so live output flows again', async () => { @@ -799,9 +805,9 @@ describe('TerminalTile scroll-to-top history pull', () => { expect(pane._bufferLoading).toBe(false); expect(pane._liveQueue).toBeNull(); - expect(pane.terminal.write).toHaveBeenCalledWith('held'); + expect(screenWrites(pane)).toContain('held'); pane._onLiveOutput('after'); - expect(pane.terminal.write).toHaveBeenLastCalledWith('after'); + expect(screenWrites(pane).at(-1)).toBe('after'); }); it('a refresh frame during the pull runs once behind it', async () => { @@ -921,7 +927,7 @@ describe('TerminalTile scroll-to-top history pull', () => { expect(pane._liveQueue).toBeNull(); expect(pane.terminal).toBeNull(); expect(term.write).not.toHaveBeenCalledWith('\x1bc'); - expect(term.write).not.toHaveBeenCalledWith('held'); + expect(term.write.mock.calls.map((call) => call[0])).not.toContain('held'); }); it('a pull whose request is aborted (the deadline) frees the pane', async () => { @@ -937,7 +943,7 @@ describe('TerminalTile scroll-to-top history pull', () => { expect(pane._bufferLoading).toBe(false); expect(pane._liveQueue).toBeNull(); - expect(pane.terminal.write).toHaveBeenCalledWith('held'); + expect(screenWrites(pane)).toContain('held'); }); it('the wheel listener is capture-phase, and only a wheel UP can trigger a pull', async () => {