COD-108 auto-reconnect remote tmux sessions on SSH drop

Continuous remote-only reconnect watcher closing the COD-104 durability
arc: when a remote session's local ssh pane dies mid-run, re-establish it
automatically instead of leaving a dead pane until the user pokes it.

Design decisions (per cod108 design doc):
- D1 event->owner: TmuxManager watcher DETECTS a dead remote pane and emits
  `remoteSessionDropped`; the session owner (server) reassembles the same
  RespawnPaneOptions and calls Session.reattachRemote() -> respawnPane, which
  re-runs the idempotent remote command (owned new-session -A / non-owned
  attach) and REJOINS the still-running durable remote tmux session. The
  watcher never reassembles options itself, and never routes through the
  Claude-idle respawn-controller.
- D2 bounded backoff: per-session exponential backoff [5s,15s,45s,2m,5m,5m],
  reset on a successful reattach, `remoteReconnectExhausted` emitted once after
  the cap. Pure, unit-tested schedule + eligibility decision.
- D3 always-on + kill-switch: `remoteAutoReconnect` app setting (default ON),
  read each tick; when false the watcher does nothing.

Guards: killSession() (incl. the non-owned DETACH early-return) and shutdown
add the session to an intentional-teardown guard set + clear its backoff
BEFORE teardown, so a closed/killed tab is never auto-revived. Exactly one
reconnect in flight per session (inFlight guard prevents stacked respawns).
Per-session reconnect/guard state cleared on session removal.

New: src/remote-reconnect.ts (pure backoff + decideReconnect), TmuxManager
startRemoteReconnectWatcher/stop + runRemoteReconnectTick + noteRemoteReconnect
+ guardRemoteReconnect + clearRemoteReconnectState; Session.reattachRemote()
(+ extracted _buildRespawnPaneOptions, shared with interactive start); server
wiring + watcher start; 3 SSE events (sse-events.ts + constants.js in sync,
broadcast + app.js exhausted "Reconnect" affordance); remoteAutoReconnect
schema + settings-ui toggle.

Tests: test/remote-auto-reconnect.test.ts (21) - pure schedule, eligibility
(guarded never reconnects, non-remote/pane-alive/not-due skip, over-cap
exhaust), and manager-level integration (dead remote pane -> dropped ->
backoff -> exhausted; guarded emits nothing; reset-on-success; kill-switch
off; state-cleared-on-remove). Verified real-remote against aa-desktop: drop
local ssh pane -> watcher emitted -> respawnPane reattached the SAME remote
session (remote pane_pid unchanged 3939->3939); test session cleaned up, the
real host sessions left untouched.

Checks: tsc, eslint, check:frontend-syntax, check:public-assets, prettier
--check, build all green; tmux-manager/session-routes/session-manager/
sse-registry-parity suites pass.

(cherry picked from commit d13d58b1994eb6594fd2eadea208104d36204f9d)
This commit is contained in:
Aamer Akhter
2026-07-17 15:59:36 -04:00
parent 897bfdff59
commit 6dba8b5227
12 changed files with 824 additions and 17 deletions
+4
View File
@@ -216,6 +216,10 @@ const _SSE_HANDLER_MAP = [
[SSE_EVENTS.MUX_DIED, '_onMuxDied'],
[SSE_EVENTS.MUX_STATS_UPDATED, '_onMuxStatsUpdated'],
// Remote auto-reconnect (COD-108)
[SSE_EVENTS.REMOTE_SESSION_RECONNECTED, '_onRemoteSessionReconnected'],
[SSE_EVENTS.REMOTE_RECONNECT_EXHAUSTED, '_onRemoteReconnectExhausted'],
// Ralph
[SSE_EVENTS.SESSION_RALPH_LOOP_UPDATE, '_onRalphLoopUpdate'],
[SSE_EVENTS.SESSION_RALPH_TODO_UPDATE, '_onRalphTodoUpdate'],
+5
View File
@@ -379,6 +379,11 @@ const SSE_EVENTS = {
MUX_DIED: 'mux:died',
MUX_STATS_UPDATED: 'mux:statsUpdated',
// Remote auto-reconnect (COD-108)
REMOTE_SESSION_DROPPED: 'remote:sessionDropped',
REMOTE_SESSION_RECONNECTED: 'remote:sessionReconnected',
REMOTE_RECONNECT_EXHAUSTED: 'remote:reconnectExhausted',
// Ralph
SESSION_RALPH_LOOP_UPDATE: 'session:ralphLoopUpdate',
SESSION_RALPH_TODO_UPDATE: 'session:ralphTodoUpdate',
+8
View File
@@ -1471,6 +1471,14 @@
</label>
<span class="form-hint">Use 1M token context window (model: opus[1m]) for all new sessions — ignored when a Claude Model is selected above</span>
</div>
<div class="form-row form-row-switch">
<label>Remote auto-reconnect</label>
<label class="switch">
<input type="checkbox" id="appSettingsRemoteAutoReconnect">
<span class="slider"></span>
</label>
<span class="form-hint">Automatically re-establish remote (SSH) sessions when the connection drops, reattaching to the durable remote tmux session (on by default; bounded backoff)</span>
</div>
<div class="form-row">
<label>Thinking Effort</label>
<select id="appSettingsThinkingEffort" class="form-select">
+27
View File
@@ -82,6 +82,33 @@ Object.assign(CodemanApp.prototype, {
}
},
// Remote auto-reconnect (COD-108)
_onRemoteSessionReconnected(data) {
const id = this.getShortId(data.sessionId);
this.showToast(`Remote session ${id} reconnected`, 'success');
},
_onRemoteReconnectExhausted(data) {
const sessionId = data.sessionId;
const id = this.getShortId(sessionId);
// Auto-reconnect gave up after the bounded backoff. Surface a manual
// "Reconnect" affordance that re-triggers the attach path (force-reload the
// session, which re-runs the create/attach flow against the durable remote).
this.showToast(`Remote session ${id} dropped — auto-reconnect gave up`, 'error', {
duration: 15000,
action: {
label: 'Reconnect',
onClick: () => {
if (this.sessions && this.sessions.has(sessionId)) {
this.selectSession(sessionId, { forceReload: true });
} else {
this.showToast('Session no longer available', 'warning');
}
},
},
});
},
// Bash tools
_onBashToolStart(data) {
+4
View File
@@ -359,6 +359,7 @@ Object.assign(CodemanApp.prototype, {
document.getElementById('appSettingsAgentTeams').checked = settings.agentTeamsEnabled ?? false;
document.getElementById('appSettingsClaudeModel').value = settings.claudeModel ?? '';
document.getElementById('appSettingsOpusContext1m').checked = settings.opusContext1mEnabled ?? false;
document.getElementById('appSettingsRemoteAutoReconnect').checked = settings.remoteAutoReconnect ?? true;
document.getElementById('appSettingsThinkingEffort').value = settings.thinkingEffort ?? '';
// CPU Priority settings
const niceSettings = settings.nice || {};
@@ -1453,6 +1454,7 @@ Object.assign(CodemanApp.prototype, {
agentTeamsEnabled: document.getElementById('appSettingsAgentTeams').checked,
claudeModel: document.getElementById('appSettingsClaudeModel').value,
opusContext1mEnabled: document.getElementById('appSettingsOpusContext1m').checked,
remoteAutoReconnect: document.getElementById('appSettingsRemoteAutoReconnect').checked,
thinkingEffort: document.getElementById('appSettingsThinkingEffort').value,
// CPU Priority settings
nice: {
@@ -1770,6 +1772,8 @@ Object.assign(CodemanApp.prototype, {
showPlanUsageLimits: false,
showAttachmentsButton: false,
showRedrawButton: false,
// Remote auto-reconnect (COD-108) — on by default
remoteAutoReconnect: true,
// Input
gestureControlEnabled: false,
// Feature toggles - keep tracking on even on mobile