fix(session): persist an exit retraction, and let tests reach the watcher

Ten findings from a two-model review of this branch. Both reviewers cleared the
detection logic itself; everything here is a gap around it.

A route that starts a command in a pane now PERSISTS as well as broadcasts.
`/interactive` and `/shell` did neither before, and the pane-exit watcher cannot
cover for them: its next tick finds `paneExit` already cleared in memory,
reports no change and writes nothing, so `state.json` kept saying the agent had
exited for as long as the session stayed quiet. Nothing reads that record for a
decision yet, which is exactly why it had to be fixed now — part 2 is designed
to read it. The `clearPaneExitForNewPane()` docstring claimed its callers
already persisted; that claim was false for these two, and now says what the
caller owes instead.

The watcher's four guards were unreachable by any test. `refreshPaneExits()`
opened with `if (IS_TEST_MODE) return;`, so the read gate, the in-flight
suppression, the generation counter and the empty-read rule could each be
deleted with the whole suite green. The tmux call moves into `readPaneRows()`,
which a test subclass overrides — the shape `runRemoteReconnectTick` already
uses in this file for the same reason — and the test-mode gate moves with it, so
what a test cannot do is spawn a process rather than exercise the bookkeeping.
Each of the four guards now has a test that fails when it is deleted.

The muted status dot turned out to be a specificity fight on three surfaces, not
two. `.tab-status.error` was not excluded, so a session whose agent exited and
whose PTY-exit breaker then tripped lost its red dot to the mute — the state the
browser answers with a "restart it?" confirm, and a needs-you colour by the same
argument that protects the two alert classes. And mobile.css gives a `busy` dot
a 9px size and a green glow with `!important`, while `status` stays `busy` for a
pane whose agent died mid-turn, so a phone rendered a grey dot still wearing the
green halo beside a badge reading "exited". Both measured against the real
stylesheets, both now excluded, and the CSS test reads mobile.css too instead of
being structurally blind to half the problem.

Six comments said things that were not true. Two named the stats collector as
what replaces a restored reading, which is the opposite of the design. The
interval constant argued that 2000 ms keeps a read inside a tick, when the
5000 ms exec timeout means it cannot — which is why the in-flight guard exists.
`MuxSession.discovered` did not say the flag is permanent, though `saveSessions()`
serializes it. The empty-read docstring claimed a distinction that `|| true`
makes impossible. The invariants doc promised more than its drift test delivers.
And CLAUDE.md had no pointer at all, leaving its two hardest prohibitions
("never set `status: 'error'`", "never null the pid") only in the file it is
meant to route people to.

Refs Ark0N/Codeman#446.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/mux-interface.ts

@@ -65,6 +65,13 @@ export interface MuxSession {
    * arrives with no `remote`/`docker` metadata and looks local. Anything that
    * would be WRONG about such a session rather than merely vague must fail
    * closed on this flag.
+   *
+   * ⚠ It is PERMANENT, not merely true for the boot that rediscovered the
+   * session: `saveSessions()` serializes the whole record to
+   * `mux-sessions.json` and `loadSessions()` restores it, so a genuinely local
+   * session rediscovered once stays opted out of everything keyed on this for
+   * the life of that record. That is the safe direction to fail, and it costs
+   * only the guess Codeman is declining to make.
    */
   discovered?: boolean;
 }

src/session.ts

@@ -895,8 +895,9 @@ export class Session extends EventEmitter {
     // because the scoping reads `_remote`, `_docker` and the mux fields, all of
     // which are set by now. It is a claim about a pane this process has not
     // looked at yet, so every path that starts or re-attaches a pane drops it
-    // (see `_setupOrAttachMuxSession`) and the stats tick replaces it with a
-    // first-hand reading.
+    // (see `_setupOrAttachMuxSession`) and the pane-exit watcher's own tick
+    // replaces it with a first-hand reading. NOT the stats collector, which a
+    // browser panel arms and disarms — see `startPaneExitWatcher`.
     this.setPaneExit(config.paneExit);
     // Never self-parent: a session pointing at itself would draw a zero-length
     // lineage arc under its own tab. Only reachable via the recovery path, where
@@ -1156,9 +1157,13 @@ export class Session extends EventEmitter {
    * Every path that starts or relaunches a command in the pane calls it, and
    * the mux half also invalidates a pane read already in flight.
    *
-   * It does not persist or broadcast by itself. Each caller is already followed
-   * by the route's or recovery's own persist, and the pane-exit watcher would
-   * reach the same answer within one interval regardless.
+   * It does not persist or broadcast by itself; the caller owns both. ⚠ That
+   * caller MUST persist, and the pane-exit watcher is not a fallback for it:
+   * the watcher's next tick reads UNKNOWN, finds this field already cleared,
+   * reports no change and therefore writes nothing, so a caller that only
+   * broadcasts leaves `state.json` saying the agent exited for as long as the
+   * session stays quiet. `/interactive` and `/shell` did exactly that until
+   * Ark0N/Codeman#446 review; both now persist on their success path.
    */
   private clearPaneExitForNewPane(): void {
     this.setPaneExit(undefined);

src/tmux-manager.ts

@@ -157,8 +157,9 @@ const DEFAULT_STATS_INTERVAL_MS = 2000;
  * How often the pane-exit watcher re-reads every pane on the socket. The
  * watcher owns this cadence: it does NOT ride `startStatsCollection()`, whose
  * lifetime a browser panel controls (see {@link TmuxManager.startPaneExitWatcher}).
- * Matched to the stats cadence above because both cost one batched tmux read,
- * and kept well under EXEC_TIMEOUT_MS so a normal read finishes inside a tick.
+ * Matched to the stats cadence above because both cost one batched tmux read.
+ * ⚠ It does NOT bound a read: EXEC_TIMEOUT_MS is 5000 ms, so a slow read can
+ * outlive two ticks, which is exactly why `paneExitReadInFlight` exists.
  */
 const DEFAULT_PANE_EXIT_INTERVAL_MS = 2000;
 
@@ -3079,9 +3080,13 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
    * polls rather than probing per session.
    *
    * A failed or empty probe leaves the previous answers ALONE rather than
-   * clearing them. An empty read is "tmux did not answer", and clearing on it
-   * would turn a transient failure into a silent retraction of a death Codeman
-   * had already observed. A NON-empty read is different: `list-panes -a` lists
+   * clearing them, because the two cannot be told apart: the command ends in
+   * `|| true`, so a tmux that errored and a socket with genuinely no panes both
+   * arrive as empty output. Treating that as "tmux did not answer" is the
+   * conservative reading — clearing on it would turn a transient failure into a
+   * silent retraction of a death Codeman had already observed, and the cost of
+   * being wrong the other way is one stale entry for a socket that no longer
+   * has the pane. A NON-empty read is different: `list-panes -a` lists
    * every pane on the socket, so it is authoritative and {@link applyPaneExits}
    * prunes against it.
    *
@@ -3096,8 +3101,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
    * command in a pane calls {@link clearPaneExit} itself.
    */
   async refreshPaneExits(now: number = Date.now()): Promise<void> {
-    if (IS_TEST_MODE) return;
-    // Nothing on this socket could answer, so do not exec tmux to find that
+    // Nothing on this socket could answer, so do not read tmux to find that
     // out. See `hasObservablePaneSession`: the watcher above still ticks.
     if (!hasObservablePaneSession(this.sessions.values())) return;
     if (this.paneExitReadInFlight) return;
@@ -3106,18 +3110,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
     this.paneExitReadInFlight = true;
     let rows: PaneRow[];
     try {
-      // execAsync, not execSync: this runs on a 2000 ms timer, and a synchronous
-      // exec freezes the port while the process stays alive (see the
-      // event-loop-monitor note in CLAUDE.md). The three `isPaneDead()` callers
-      // stay synchronous because each is answering one request right then.
-      const { stdout } = await execAsync(`${this.tmux()} list-panes -a -F '${PANE_LIST_FORMAT}' 2>/dev/null || true`, {
-        encoding: 'utf-8',
-        timeout: EXEC_TIMEOUT_MS,
-      });
-      rows = parsePaneRows(stdout.trim());
-    } catch (err) {
-      console.error('[TmuxManager] Failed to read pane exit state:', err);
-      return;
+      rows = await this.readPaneRows();
     } finally {
       this.paneExitReadInFlight = false;
     }
@@ -3130,6 +3123,37 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
     this.applyPaneExits(derivePaneExits(rows, now));
   }
 
+  /**
+   * Read every pane on the socket. The ONLY part of the pane-exit watcher that
+   * touches tmux, which is what lets a test subclass drive the guards in
+   * {@link refreshPaneExits} — the in-flight suppression, the generation
+   * check, the empty-read retraction rule and the read gate — against rows it
+   * chooses. Split out for the reason `runRemoteReconnectTick` is: a guard no
+   * test can reach is a guard that can be deleted without anything failing.
+   *
+   * A failed read answers with NO rows, which the caller treats as "tmux did
+   * not answer" and which therefore retracts nothing.
+   */
+  protected async readPaneRows(): Promise<PaneRow[]> {
+    // The test-mode gate lives HERE rather than at the top of the tick, so that
+    // what tests cannot do is spawn a process, not exercise the bookkeeping.
+    if (IS_TEST_MODE) return [];
+    try {
+      // execAsync, not execSync: this runs on a 2000 ms timer, and a synchronous
+      // exec freezes the port while the process stays alive (see the
+      // event-loop-monitor note in CLAUDE.md). The three `isPaneDead()` callers
+      // stay synchronous because each is answering one request right then.
+      const { stdout } = await execAsync(`${this.tmux()} list-panes -a -F '${PANE_LIST_FORMAT}' 2>/dev/null || true`, {
+        encoding: 'utf-8',
+        timeout: EXEC_TIMEOUT_MS,
+      });
+      return parsePaneRows(stdout.trim());
+    } catch (err) {
+      console.error('[TmuxManager] Failed to read pane exit state:', err);
+      return [];
+    }
+  }
+
   /**
    * Fold one authoritative observation into {@link paneExits}. Split out from
    * the tmux call so the merge rules are unit-testable.
@@ -3190,7 +3214,9 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
       clearInterval(this.paneExitInterval);
     }
     this.paneExitInterval = setInterval(() => {
-      if (IS_TEST_MODE) return;
+      // No IS_TEST_MODE guard: `readPaneRows()` is the only thing that would
+      // spawn a process and it refuses under test, so a test can drive this
+      // whole loop with fake timers instead of being locked out of it.
       void this.refreshPaneExits()
         .then(() => this.emit('paneExitsUpdated'))
         .catch((err) => console.error('[TmuxManager] Pane exit watcher error:', err));

src/web/public/mobile.css

@@ -694,6 +694,22 @@ html.mobile-init .file-browser-panel {
     display: none;
   }
 
+  /* The exited-agent mute, a third time, for the phone (Ark0N/Codeman#446).
+     The rule above enlarges the working dot and gives it a green glow with
+     !important, and `status` deliberately stays `busy` for a pane whose agent
+     died mid-turn — so without this a phone renders a 9px grey dot still
+     wearing the green halo, beside a badge reading "exited". Measured; the
+     desktop rules cannot reach it, since they declare no box-shadow and lose
+     to !important anyway. !important here for the reason the glow needs it:
+     the skin block in styles.css outranks any plain class rule in this file.
+     ⚠ The alert classes and the error state are excluded exactly as they are
+     on desktop — a colour that means "this needs you" outranks "it exited". */
+  .session-tab.tab-agent-exited:not(.tab-alert-action):not(.tab-alert-idle) .tab-status:not(.error) {
+    background: var(--text-muted) !important;
+    box-shadow: none !important;
+    opacity: 0.5;
+  }
+
   /* Truncate tab names more aggressively on mobile */
   .session-tab .tab-name {
     max-width: 50px;

src/web/public/styles.css

@@ -2458,18 +2458,23 @@ html[data-tab-orientation='vertical'] .tab-rail .session-tab .tab-name-prefix {
    `error` is the PTY-exit breaker's value and makes the browser offer a restart
    — so this is a rendering rule only, and it matches `.tab-status.ended`.
 
-   ⚠ The two alert classes are excluded BY HAND rather than by cascade, the same
-   way the rich-rail dot rules below do it: a dot turning red or yellow because
-   a session is blocked on a human outranks "the agent exited", and this
-   selector is specific enough (0,5,0) to have beaten those (0,3,0) rules
-   otherwise. The spinner ring is a pseudo-element the skin blocks cannot reach,
-   so it needs hiding explicitly rather than by unsetting the animation. */
-.session-tab.tab-agent-exited:not(.tab-alert-action):not(.tab-alert-idle) .tab-status {
+   ⚠ Three states are excluded BY HAND rather than by cascade, the same way the
+   rich-rail dot rules below do it: a dot turning red or yellow because a
+   session is blocked on a human outranks "the agent exited", and this selector
+   is specific enough (0,5,0) to have beaten those (0,3,0) rules otherwise. The
+   third is `.tab-status.error` (0,2,0), which is the PTY-exit breaker's state
+   and the one the browser answers with a "restart it?" confirm — the same
+   argument that protects the two alert classes, and it reaches the dot rather
+   than the tab, which is why the `:not()` sits on `.tab-status` here and on
+   `.session-tab` there. The spinner ring is a pseudo-element the skin blocks
+   cannot reach, so it needs hiding explicitly rather than by unsetting the
+   animation. */
+.session-tab.tab-agent-exited:not(.tab-alert-action):not(.tab-alert-idle) .tab-status:not(.error) {
   background: var(--text-muted);
   opacity: 0.5;
   animation: none;
 }
-.session-tab.tab-agent-exited:not(.tab-alert-action):not(.tab-alert-idle) .tab-status::after {
+.session-tab.tab-agent-exited:not(.tab-alert-action):not(.tab-alert-idle) .tab-status:not(.error)::after {
   display: none;
 }
 
@@ -18511,7 +18516,7 @@ html[data-tab-orientation='vertical'][data-tab-rail-detail='rich']:not(.tab-rail
 html[data-tab-orientation='vertical'][data-tab-rail-detail='rich']:not(.tab-rail-compact)
   .tab-rail
   .session-tab.tab-agent-exited:not(.tab-alert-action):not(.tab-alert-idle)
-  .tab-status {
+  .tab-status:not(.error) {
   background: var(--text-muted);
   opacity: 0.5;
   box-shadow: none;

src/web/routes/session-routes.ts

@@ -1583,6 +1583,12 @@ export function registerSessionRoutes(
         name: session.name,
         mode: session.mode,
       });
+      // Persist, not just broadcast. Starting a command in the pane changes
+      // `pid` and retracts any `paneExit` (Ark0N/Codeman#446), and the pane-exit
+      // watcher cannot write that retraction to disk for us: its next tick finds
+      // the in-memory field already cleared, reports no change and persists
+      // nothing, so `state.json` would keep saying the agent had exited.
+      ctx.persistSessionState(session);
       ctx.broadcast(SseEvent.SessionInteractive, { id });
       ctx.broadcast(SseEvent.SessionUpdated, { session: ctx.getSessionStateWithRespawn(session) });
 
@@ -1612,6 +1618,9 @@ export function registerSessionRoutes(
         name: session.name,
         mode: 'shell',
       });
+      // Persist for the same reason /interactive does: a started pane retracts
+      // `paneExit`, and the watcher's next tick cannot write that retraction.
+      ctx.persistSessionState(session);
       ctx.broadcast(SseEvent.SessionInteractive, { id, mode: 'shell' });
       ctx.broadcast(SseEvent.SessionUpdated, { session: ctx.getSessionStateWithRespawn(session) });
       return {};

src/web/server.ts

@@ -3373,8 +3373,9 @@ export class WebServer extends EventEmitter {
               claudeSessionChain: savedState?.claudeSessionChain,
               // What the previous run last observed of this pane's agent. Carried
               // over so the first persist after boot does not blank a record that
-              // says the agent exited; the attach below drops it, and the stats
-              // tick replaces it with a first-hand reading.
+              // says the agent exited; the attach below drops it, and the
+              // pane-exit watcher's own tick replaces it with a first-hand
+              // reading (not the stats collector — see `startPaneExitWatcher`).
               paneExit: savedState?.paneExit,
               // A record rebuilt from the socket has no provenance, so its
               // apparent locality is a guess (see `MuxSession.discovered`).