COD-108 auto-reconnect remote tmux sessions on SSH drop

Continuous remote-only reconnect watcher closing the COD-104 durability
arc: when a remote session's local ssh pane dies mid-run, re-establish it
automatically instead of leaving a dead pane until the user pokes it.

Design decisions (per cod108 design doc):
- D1 event->owner: TmuxManager watcher DETECTS a dead remote pane and emits
  `remoteSessionDropped`; the session owner (server) reassembles the same
  RespawnPaneOptions and calls Session.reattachRemote() -> respawnPane, which
  re-runs the idempotent remote command (owned new-session -A / non-owned
  attach) and REJOINS the still-running durable remote tmux session. The
  watcher never reassembles options itself, and never routes through the
  Claude-idle respawn-controller.
- D2 bounded backoff: per-session exponential backoff [5s,15s,45s,2m,5m,5m],
  reset on a successful reattach, `remoteReconnectExhausted` emitted once after
  the cap. Pure, unit-tested schedule + eligibility decision.
- D3 always-on + kill-switch: `remoteAutoReconnect` app setting (default ON),
  read each tick; when false the watcher does nothing.

Guards: killSession() (incl. the non-owned DETACH early-return) and shutdown
add the session to an intentional-teardown guard set + clear its backoff
BEFORE teardown, so a closed/killed tab is never auto-revived. Exactly one
reconnect in flight per session (inFlight guard prevents stacked respawns).
Per-session reconnect/guard state cleared on session removal.

New: src/remote-reconnect.ts (pure backoff + decideReconnect), TmuxManager
startRemoteReconnectWatcher/stop + runRemoteReconnectTick + noteRemoteReconnect
+ guardRemoteReconnect + clearRemoteReconnectState; Session.reattachRemote()
(+ extracted _buildRespawnPaneOptions, shared with interactive start); server
wiring + watcher start; 3 SSE events (sse-events.ts + constants.js in sync,
broadcast + app.js exhausted "Reconnect" affordance); remoteAutoReconnect
schema + settings-ui toggle.

Tests: test/remote-auto-reconnect.test.ts (21) - pure schedule, eligibility
(guarded never reconnects, non-remote/pane-alive/not-due skip, over-cap
exhaust), and manager-level integration (dead remote pane -> dropped ->
backoff -> exhausted; guarded emits nothing; reset-on-success; kill-switch
off; state-cleared-on-remove). Verified real-remote against aa-desktop: drop
local ssh pane -> watcher emitted -> respawnPane reattached the SAME remote
session (remote pane_pid unchanged 3939->3939); test session cleaned up, the
real host sessions left untouched.

Checks: tsc, eslint, check:frontend-syntax, check:public-assets, prettier
--check, build all green; tmux-manager/session-routes/session-manager/
sse-registry-parity suites pass.

(cherry picked from commit d13d58b1994eb6594fd2eadea208104d36204f9d)

src/tmux-manager.ts

@@ -62,6 +62,13 @@ import type {
   RespawnPaneOptions,
   PaneCaptureOptions,
 } from './mux-interface.js';
+import {
+  decideReconnect,
+  advanceBackoff,
+  freshReconnectState,
+  resetReconnectState,
+  type RemoteReconnectState,
+} from './remote-reconnect.js';
 
 // ============================================================================
 // Timing Constants
@@ -94,6 +101,9 @@ const GRACEFUL_SHUTDOWN_WAIT_MS = 100;
 /** Default stats collection interval (2 seconds) */
 const DEFAULT_STATS_INTERVAL_MS = 2000;
 
+/** Default remote-reconnect watcher poll interval (5 seconds) — COD-108 */
+const DEFAULT_REMOTE_RECONNECT_INTERVAL_MS = 5000;
+
 /** Stable cwd for tmux server/pane launch; actual session cwd is reached inside the pane. */
 const TMUX_LAUNCH_CWD = '/tmp';
 
@@ -118,6 +128,20 @@ const IS_TEST_MODE = !!process.env.VITEST;
 /** Path to persisted mux session metadata */
 const MUX_SESSIONS_FILE = dataPath('mux-sessions.json');
 
+/**
+ * COD-108 kill-switch: `remoteAutoReconnect` app setting (default ON). Read at
+ * call time (like headroom routing) so a settings change takes effect without a
+ * restart. Absent/non-boolean ⇒ true (feature on).
+ */
+function isRemoteAutoReconnectEnabled(): boolean {
+  try {
+    const s = JSON.parse(readFileSync(dataPath('settings.json'), 'utf8')) as Record<string, unknown>;
+    return typeof s.remoteAutoReconnect === 'boolean' ? s.remoteAutoReconnect : true;
+  } catch {
+    return true;
+  }
+}
+
 /** Regex to validate tmux session names (only allow safe characters) */
 const SAFE_MUX_NAME_PATTERN = /^codeman-[a-f0-9-]+$/;
 
@@ -1008,6 +1032,17 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
   /** Track last-known pane count per session to avoid unnecessary tmux set-option calls */
   private lastPaneCount: Map<string, number> = new Map();
 
+  // ── COD-108 remote-reconnect watcher state ────────────────────────────────
+  /** Periodic watcher that re-establishes dropped remote sessions. */
+  private remoteReconnectInterval: NodeJS.Timeout | null = null;
+  /** Per-session backoff/attempt bookkeeping (sessionId → state). */
+  private reconnectState: Map<string, RemoteReconnectState> = new Map();
+  /**
+   * Sessions excluded from auto-reconnect because they are being intentionally
+   * torn down (killed/detached/stopping). A guarded session is NEVER revived.
+   */
+  private reconnectGuard: Set<string> = new Set();
+
   private trueColorConfigured = false;
 
   constructor() {
@@ -1679,9 +1714,16 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
       return false;
     }
 
+    // COD-108: an intentional kill/detach must NEVER be auto-revived by the
+    // remote-reconnect watcher. Guard BEFORE any teardown so a tick that fires
+    // mid-kill (especially the non-owned DETACH early-return below, where the
+    // dead local pane would otherwise look reconnectable) sees the guard.
+    this.guardRemoteReconnect(sessionId);
+
     // TEST MODE: Remove from memory only — NEVER touch real tmux sessions
     if (IS_TEST_MODE) {
       this.sessions.delete(sessionId);
+      this.clearRemoteReconnectState(sessionId);
       this.emit('sessionKilled', { sessionId });
       return true;
     }
@@ -1721,6 +1763,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
       }
       this.lastPaneCount.delete(session.muxName);
       this.sessions.delete(sessionId);
+      this.clearRemoteReconnectState(sessionId);
       this.saveSessions();
       this.emit('sessionKilled', { sessionId });
       return true;
@@ -1818,6 +1861,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
 
     this.lastPaneCount.delete(session.muxName);
     this.sessions.delete(sessionId);
+    this.clearRemoteReconnectState(sessionId);
     this.saveSessions();
     this.emit('sessionKilled', { sessionId });
 
@@ -1884,6 +1928,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
       } else {
         dead.push(sessionId);
         this.sessions.delete(sessionId);
+        this.clearRemoteReconnectState(sessionId);
         this.emit('sessionDied', { sessionId });
       }
     }
@@ -2155,9 +2200,118 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
     this.lastPaneCount.clear();
   }
 
+  // ── COD-108 remote-session auto-reconnect watcher ─────────────────────────
+
+  /**
+   * Start the remote-reconnect watcher (COD-108). Each tick, for every tracked
+   * session with `session.remote` whose local pane is DEAD, not intentionally
+   * guarded, and within its backoff budget, emit `remoteSessionDropped` so the
+   * session owner reattaches (re-running the idempotent remote command rejoins
+   * the durable remote tmux session). After the attempt cap, emit
+   * `remoteReconnectExhausted` once and go quiet.
+   *
+   * No-op tick body under `IS_TEST_MODE` (mirrors `startMouseModeSync`): tests
+   * drive the logic deterministically via {@link runRemoteReconnectTick}.
+   */
+  startRemoteReconnectWatcher(intervalMs: number = DEFAULT_REMOTE_RECONNECT_INTERVAL_MS): void {
+    if (this.remoteReconnectInterval) {
+      clearInterval(this.remoteReconnectInterval);
+    }
+    this.remoteReconnectInterval = setInterval(() => {
+      if (IS_TEST_MODE) return;
+      try {
+        this.runRemoteReconnectTick(Date.now(), isRemoteAutoReconnectEnabled());
+      } catch (err) {
+        console.error('[TmuxManager] Remote reconnect watcher error:', err);
+      }
+    }, intervalMs);
+  }
+
+  stopRemoteReconnectWatcher(): void {
+    if (this.remoteReconnectInterval) {
+      clearInterval(this.remoteReconnectInterval);
+      this.remoteReconnectInterval = null;
+    }
+  }
+
+  /**
+   * Run ONE watcher tick. Extracted (and given an injected `now`/`enabled`) so
+   * the reconnect logic is deterministically testable even though the live
+   * `setInterval` body no-ops under test mode. For each remote session it
+   * applies the pure {@link decideReconnect} decision and translates the result
+   * into events + backoff/state transitions. Public for tests + the watcher.
+   */
+  runRemoteReconnectTick(now: number, enabled: boolean): void {
+    for (const session of this.sessions.values()) {
+      if (!session.remote) continue;
+      const sessionId = session.sessionId;
+      const state = this.reconnectState.get(sessionId);
+      const action = decideReconnect({
+        session: {
+          sessionId,
+          isRemote: true,
+          paneDead: this.isPaneDead(session.muxName),
+        },
+        state,
+        guarded: this.reconnectGuard.has(sessionId),
+        enabled,
+        now,
+      });
+
+      if (action.kind === 'emit') {
+        const base = state ?? freshReconnectState();
+        // Mark in-flight + advance backoff BEFORE emitting so a re-entrant tick
+        // (or a synchronous listener) can never stack a second reconnect.
+        this.reconnectState.set(sessionId, { ...advanceBackoff(base, now), inFlight: true });
+        this.emit('remoteSessionDropped', { sessionId, attempt: action.attempt });
+      } else if (action.kind === 'exhaust') {
+        const base = state ?? freshReconnectState();
+        if (!base.exhaustedEmitted) {
+          this.reconnectState.set(sessionId, { ...base, exhausted: true, exhaustedEmitted: true });
+          this.emit('remoteReconnectExhausted', { sessionId });
+        }
+      }
+      // 'skip' → nothing to do.
+    }
+  }
+
+  /**
+   * Tell the watcher a reattach attempt for `sessionId` finished. On success,
+   * reset the backoff so the session is healthy again; on failure, just clear
+   * the in-flight flag so the next due tick can retry under the existing
+   * backoff schedule. Called by the session owner after `respawnPane`.
+   */
+  noteRemoteReconnect(sessionId: string, success: boolean): void {
+    if (success) {
+      this.reconnectState.set(sessionId, resetReconnectState());
+      return;
+    }
+    const state = this.reconnectState.get(sessionId);
+    if (state) this.reconnectState.set(sessionId, { ...state, inFlight: false });
+  }
+
+  /**
+   * Exclude a session from auto-reconnect (intentional teardown). Adds it to the
+   * guard set and drops any backoff state so a closed/killed tab — especially a
+   * non-owned remote DETACH — is never auto-revived. Idempotent.
+   */
+  guardRemoteReconnect(sessionId: string): void {
+    this.reconnectGuard.add(sessionId);
+    this.reconnectState.delete(sessionId);
+  }
+
+  /** Clear all per-session reconnect + guard state (e.g. when a session is removed). */
+  clearRemoteReconnectState(sessionId: string): void {
+    this.reconnectState.delete(sessionId);
+    this.reconnectGuard.delete(sessionId);
+  }
+
   destroy(): void {
     this.stopStatsCollection();
     this.stopMouseModeSync();
+    this.stopRemoteReconnectWatcher();
+    this.reconnectState.clear();
+    this.reconnectGuard.clear();
   }
 
   registerSession(session: MuxSession): void {