COD-108 auto-reconnect remote tmux sessions on SSH drop

Continuous remote-only reconnect watcher closing the COD-104 durability
arc: when a remote session's local ssh pane dies mid-run, re-establish it
automatically instead of leaving a dead pane until the user pokes it.

Design decisions (per cod108 design doc):
- D1 event->owner: TmuxManager watcher DETECTS a dead remote pane and emits
  `remoteSessionDropped`; the session owner (server) reassembles the same
  RespawnPaneOptions and calls Session.reattachRemote() -> respawnPane, which
  re-runs the idempotent remote command (owned new-session -A / non-owned
  attach) and REJOINS the still-running durable remote tmux session. The
  watcher never reassembles options itself, and never routes through the
  Claude-idle respawn-controller.
- D2 bounded backoff: per-session exponential backoff [5s,15s,45s,2m,5m,5m],
  reset on a successful reattach, `remoteReconnectExhausted` emitted once after
  the cap. Pure, unit-tested schedule + eligibility decision.
- D3 always-on + kill-switch: `remoteAutoReconnect` app setting (default ON),
  read each tick; when false the watcher does nothing.

Guards: killSession() (incl. the non-owned DETACH early-return) and shutdown
add the session to an intentional-teardown guard set + clear its backoff
BEFORE teardown, so a closed/killed tab is never auto-revived. Exactly one
reconnect in flight per session (inFlight guard prevents stacked respawns).
Per-session reconnect/guard state cleared on session removal.

New: src/remote-reconnect.ts (pure backoff + decideReconnect), TmuxManager
startRemoteReconnectWatcher/stop + runRemoteReconnectTick + noteRemoteReconnect
+ guardRemoteReconnect + clearRemoteReconnectState; Session.reattachRemote()
(+ extracted _buildRespawnPaneOptions, shared with interactive start); server
wiring + watcher start; 3 SSE events (sse-events.ts + constants.js in sync,
broadcast + app.js exhausted "Reconnect" affordance); remoteAutoReconnect
schema + settings-ui toggle.

Tests: test/remote-auto-reconnect.test.ts (21) - pure schedule, eligibility
(guarded never reconnects, non-remote/pane-alive/not-due skip, over-cap
exhaust), and manager-level integration (dead remote pane -> dropped ->
backoff -> exhausted; guarded emits nothing; reset-on-success; kill-switch
off; state-cleared-on-remove). Verified real-remote against aa-desktop: drop
local ssh pane -> watcher emitted -> respawnPane reattached the SAME remote
session (remote pane_pid unchanged 3939->3939); test session cleaned up, the
real host sessions left untouched.

Checks: tsc, eslint, check:frontend-syntax, check:public-assets, prettier
--check, build all green; tmux-manager/session-routes/session-manager/
sse-registry-parity suites pass.

(cherry picked from commit d13d58b1994eb6594fd2eadea208104d36204f9d)
This commit is contained in:
Aamer Akhter
2026-07-17 15:59:36 -04:00
parent 897bfdff59
commit 6dba8b5227
12 changed files with 824 additions and 17 deletions
+64 -17
View File
@@ -1211,6 +1211,68 @@ export class Session extends EventEmitter {
return { isRestored };
}
/**
* COD-108 — re-establish a dropped REMOTE session. Triggered by the
* `TmuxManager` remote-reconnect watcher (via `remoteSessionDropped`): the
* watcher detects a dead remote pane, the session owner reassembles the SAME
* `RespawnPaneOptions` used for Claude-idle respawns and calls
* `respawnPane()` directly. For a remote session that re-runs
* `buildRemoteSessionCommand` (owned → `new-session -A`, non-owned →
* `attach`), which idempotently REATTACHES the still-running durable remote
* tmux session — scrollback + agent intact (proven COD-104/105).
*
* Deliberately does NOT route through the Claude-idle respawn-controller —
* this is a transport re-establish, not a `/clear`/`/compact` cycle.
*
* @returns true if the pane was respawned (reattach issued), false otherwise.
*/
async reattachRemote(): Promise<boolean> {
if (!this._remote) return false; // not a remote session
if (!this._useMux || !this._mux || !this._muxSession) return false;
const mux = this._mux;
// If tmux lost the whole session (not just a dead pane), there is nothing to
// respawn into — a genuine death, leave it for normal recovery/reconcile.
if (!mux.muxSessionExists(this._muxSession.muxName)) {
console.log('[Session] reattachRemote: mux session gone, skipping:', this._muxSession.muxName);
return false;
}
const newPid = await mux.respawnPane(this._buildRespawnPaneOptions());
if (!newPid) {
console.error('[Session] reattachRemote: respawnPane failed for', this._muxSession.muxName);
return false;
}
console.log('[Session] reattachRemote: reattached remote session', this._muxSession.muxName, 'pid', newPid);
return true;
}
/**
* Assemble the {@link RespawnPaneOptions} for this session. Single source of
* truth shared by interactive start, shell start (via their inline copies),
* and {@link reattachRemote} so the remote reattach path can never drift from
* the spawn path.
*/
private _buildRespawnPaneOptions(): import('./mux-interface.js').RespawnPaneOptions {
return {
sessionId: this.id,
workingDir: this.workingDir,
mode: this.mode,
niceConfig: this._niceConfig,
model: this._model,
claudeMode: this._claudeMode,
allowedTools: this._allowedTools,
openCodeConfig: this._openCodeConfig,
codexConfig: this._codexConfig,
geminiConfig: this._geminiConfig,
resumeSessionId: this._resumeSessionId,
envOverrides: this._envOverrides,
effort: this._effort,
historyLimit: this._tmuxHistoryLimit,
remote: this._remote,
};
}
private _handleTerminalOutput(data: string): void {
// Codex AND Claude Code emit sequences that wipe xterm.js scrollback, plus
// mouse-tracking enables that hijack the scroll wheel so the user can't reach
@@ -1334,23 +1396,8 @@ export class Session extends EventEmitter {
if (this._useMux && this._mux) {
try {
const { isRestored } = await this._setupOrAttachMuxSession({
respawnPaneOptions: {
sessionId: this.id,
workingDir: this.workingDir,
mode: this.mode,
niceConfig: this._niceConfig,
model: this._model,
claudeMode: this._claudeMode,
allowedTools: this._allowedTools,
openCodeConfig: this._openCodeConfig,
codexConfig: this._codexConfig,
geminiConfig: this._geminiConfig,
resumeSessionId: this._resumeSessionId,
envOverrides: this._envOverrides,
effort: this._effort,
historyLimit: this._tmuxHistoryLimit,
remote: this._remote,
},
// Single source of truth shared with reattachRemote() (COD-108).
respawnPaneOptions: this._buildRespawnPaneOptions(),
createSessionOptions: {
sessionId: this.id,
workingDir: this.workingDir,