BitSquare/Codeman
2
0
Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-09-30 12:39:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #442 from irisitymichaelgrundberg/feat/restore-sessions-after-reboot

Browse Source 
feat(sessions): offer to rebuild the sessions a host reboot destroyed
This commit is contained in:
Ark0N
2026-09-18 13:41:24 +02:00
committed by GitHub
parent 9a503872d9 62ceb4e87b
commit f32c4f60d5
23 changed files with 2678 additions and 143 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.changeset/reboot-restore-banner.md
+5
View File
@@ -0,0 +1,5 @@
---
'aicodeman': minor
---
Offer to rebuild the sessions a host reboot destroyed. A reboot takes the tmux server down with it, so every pane dies and the board comes up empty. Codeman now works out what was running, and the board offers to restore it behind a click. The conversations come back; the terminal scrollback does not, and the banner says so.
docs/api-reference.md
+126 -76
View File
@@ -66,17 +66,17 @@ The single source of truth is `ErrorStatus` / `httpStatusForErrorCode()` in
`src/types/api.ts`. Clients should branch on `errorCode` (stable) and may rely on
the HTTP status.
| `errorCode` | HTTP | Meaning |
|-------------|------|---------|
| `INVALID_INPUT` | 400 | Malformed request / failed validation |
| `UNAUTHORIZED` | 401 | Authentication required or failed |
| `NOT_FOUND` | 404 | Resource does not exist |
| `SESSION_BUSY` | 409 | Session is busy |
| `CONFLICT` | 409 | Conflicts with current state (e.g. already running) |
| `ALREADY_EXISTS` | 409 | Resource already exists |
| `OPERATION_FAILED` | 422 | Well-formed but could not be completed |
| `RATE_LIMITED` | 429 | Too many requests |
| `INTERNAL_ERROR` | 500 | Unexpected server error |
| `errorCode` | HTTP | Meaning |
| ------------------ | ---- | --------------------------------------------------- |
| `INVALID_INPUT` | 400 | Malformed request / failed validation |
| `UNAUTHORIZED` | 401 | Authentication required or failed |
| `NOT_FOUND` | 404 | Resource does not exist |
| `SESSION_BUSY` | 409 | Session is busy |
| `CONFLICT` | 409 | Conflicts with current state (e.g. already running) |
| `ALREADY_EXISTS` | 409 | Resource already exists |
| `OPERATION_FAILED` | 422 | Well-formed but could not be completed |
| `RATE_LIMITED` | 429 | Too many requests |
| `INTERNAL_ERROR` | 500 | Unexpected server error |
Adding a new error code is non-breaking; removing or renaming one is a major change.
@@ -87,10 +87,10 @@ exist because SSE is Codeman's only other "tell me when" channel, and an agent
driving the API from a shell tool cannot practically hold a stream and parse
events inline.
| Call | Blocks until |
|------|--------------|
| `GET /api/v1/sessions/:id/wait` | one of a set of lifecycle signals fires |
| `GET /api/v1/sessions/:id/wait-output` | a literal string appears in the session's output |
| Call | Blocks until |
| --------------------------------------------- | -------------------------------------------------- |
| `GET /api/v1/sessions/:id/wait` | one of a set of lifecycle signals fires |
| `GET /api/v1/sessions/:id/wait-output` | a literal string appears in the session's output |
| `POST /api/v1/sessions/:id/input` with `wait` | the input is delivered **and then** a signal fires |
`POST .../input` with `wait` is not the same as a `POST` followed by a separate
@@ -140,13 +140,13 @@ contract is a **marker unique to each call** (`MARK="DONE_$RANDOM"`, send
### Signals
| Signal | Source | Actually fires for |
|--------|--------|--------------------|
| `idle` | the session's own `idle` event | `claude`: yes, on ❯-prompt detection after activity. `shell`: **once only**, ~500 ms after start, and never again. External CLIs: not guaranteed (they render their own TUIs and readiness is output stabilization) |
| `working` | the session's own `working` event | `claude` only in practice (spinner and work-keyword detection are Claude output formats) |
| `stop` | the Claude Code `stop` hook, the definitive end-of-turn signal | `claude` only |
| `blocked` | a `permission_prompt` or `elicitation_dialog` hook | `claude` only, and rarer than it looks: see below |
| `exit` | no process is behind the session | every mode |
| Signal | Source | Actually fires for |
| --------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `idle` | the session's own `idle` event | `claude`: yes, on ❯-prompt detection after activity. `shell`: **once only**, ~500 ms after start, and never again. External CLIs: not guaranteed (they render their own TUIs and readiness is output stabilization) |
| `working` | the session's own `working` event | `claude` only in practice (spinner and work-keyword detection are Claude output formats) |
| `stop` | the Claude Code `stop` hook, the definitive end-of-turn signal | `claude` only |
| `blocked` | a `permission_prompt` or `elicitation_dialog` hook | `claude` only, and rarer than it looks: see below |
| `exit` | no process is behind the session | every mode |
`stop` is the signal to orchestrate on where it exists; `idle` is a heuristic
fallback that can flap mid-turn when a spinner pauses. The default set when `until`
@@ -156,12 +156,12 @@ can no longer happen). On a `claude` worker, prefer an explicit `until=stop,exit
once the session is up: the default set's `idle` also resolves on a spinner pause,
and on a fresh session the **startup** `idle` (emitted when the CLI first comes up)
can land inside your first wait window and report a turn that never ran. Measured:
a session parked on the trust dialog emits no *further* `idle`, so it is the
a session parked on the trust dialog emits no _further_ `idle`, so it is the
startup transition, not the dialog, that produces the false success below.
⚠️ **`exit` means "nothing is running", which includes "not started yet".** The
server answers from `pid === null` plus a mux-layer pane-death probe, and that
covers a session that exited — including a worker that died *inside* its tmux pane
covers a session that exited — including a worker that died _inside_ its tmux pane
while the local attach client (and therefore `pid`) lives on — one that was
detached, and one that was **created but never started**. So the first wait
after `POST /api/v1/sessions` returns `{"signal":"exit","immediate":true}` in
@@ -184,7 +184,7 @@ blocked, and polling `blocked` alone will sit at its timeout.
⚠️ **On a `shell` session, only `exit` and marker-matching are dependable.** A shell
session emits its one `idle` at startup and then stays `status: "idle"` forever,
whatever the pane is doing, so it never emits a *transition*. Since send-and-wait
whatever the pane is doing, so it never emits a _transition_. Since send-and-wait
requires a transition (and so does `fresh=1`), both can only time out there:
a documented default `wait` on a shell worker running `sleep 4` times out at the
full 25 s. Synchronize hook-less sessions with `wait-output` and a unique marker
@@ -218,11 +218,11 @@ with `from=buffer` keeps matching long after the dialog is gone. A worked versio
### `GET /api/v1/sessions/:id/wait`
| Param | Type | Default | Notes |
|-------|------|---------|-------|
| `until` | comma-separated list of `idle,working,stop,blocked,exit` | `stop,idle,exit` | resolves on the first to fire. An unknown token is a `400` naming it, never a silent fallback |
| `timeout` | positive integer ms | `60000` | **validated first, clamped second.** `0`, a negative value and a fractional value are all `400`s, not clamps; a valid value outside `[1000, 600000]` is clamped and echoed as `wait.timeoutMs` |
| `fresh` | `0` \| `1` \| `false` \| `true` | `0` | `1` requires an actual transition, ignoring the state at call time |
| Param | Type | Default | Notes |
| --------- | -------------------------------------------------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `until` | comma-separated list of `idle,working,stop,blocked,exit` | `stop,idle,exit` | resolves on the first to fire. An unknown token is a `400` naming it, never a silent fallback |
| `timeout` | positive integer ms | `60000` | **validated first, clamped second.** `0`, a negative value and a fractional value are all `400`s, not clamps; a valid value outside `[1000, 600000]` is clamped and echoed as `wait.timeoutMs` |
| `fresh` | `0` \| `1` \| `false` \| `true` | `0` | `1` requires an actual transition, ignoring the state at call time |
```bash
curl -s "$API/api/v1/sessions/$SID/wait?until=stop,exit&timeout=60000"
@@ -239,12 +239,12 @@ a plain signal wait, so check the endpoint path before blaming the parameters.
### `GET /api/v1/sessions/:id/wait-output`
| Param | Type | Default | Notes |
|-------|------|---------|-------|
| `match` | literal string, 1 to 200 chars | required | substring match against the PTY stream with ANSI escapes stripped. A match spanning two PTY chunks is found |
| `nocase` | `0` \| `1` \| `false` \| `true` | `0` | case-insensitive compare. The returned snippet keeps the terminal's original casing |
| `from` | `now` \| `buffer` | `now` | `buffer` scans the tail of the existing terminal buffer (bounded, 256 KB by default) before blocking |
| `timeout` | positive integer ms | `60000` | same validation and clamp as `/wait` |
| Param | Type | Default | Notes |
| --------- | ------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------- |
| `match` | literal string, 1 to 200 chars | required | substring match against the PTY stream with ANSI escapes stripped. A match spanning two PTY chunks is found |
| `nocase` | `0` \| `1` \| `false` \| `true` | `0` | case-insensitive compare. The returned snippet keeps the terminal's original casing |
| `from` | `now` \| `buffer` | `now` | `buffer` scans the tail of the existing terminal buffer (bounded, 256 KB by default) before blocking |
| `timeout` | positive integer ms | `60000` | same validation and clamp as `/wait` |
**Matching is literal, never a pattern.** A `regex` parameter is rejected with a
`400` rather than ignored, so a caller that assumed otherwise finds out immediately
@@ -296,10 +296,10 @@ hand-written query string decodes to a space.
Two optional fields on the existing endpoint:
| Field | Type | Notes |
|-------|------|-------|
| `wait` | `true` or the same comma grammar as `until` | `true` means the default signal set. Omitted keeps the historical fire-and-forget behavior, unchanged. `null`, `false` and an empty string are all read as **absent**, not as an error and not as "wait for the default" |
| `waitTimeout` | positive integer ms | same validation **and** clamp as `timeout`: `0`, a negative and a fractional value are `400`s, anything valid is clamped into `[1000, 600000]` and echoed as `wait.timeoutMs` |
| Field | Type | Notes |
| ------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `wait` | `true` or the same comma grammar as `until` | `true` means the default signal set. Omitted keeps the historical fire-and-forget behavior, unchanged. `null`, `false` and an empty string are all read as **absent**, not as an error and not as "wait for the default" |
| `waitTimeout` | positive integer ms | same validation **and** clamp as `timeout`: `0`, a negative and a fractional value are `400`s, anything valid is clamped into `[1000, 600000]` and echoed as `wait.timeoutMs` |
Both are `nullish`, so an explicit `null` from `JSON.stringify` is accepted as
"absent" rather than failing validation. That is deliberate: `.optional()` would
@@ -330,16 +330,24 @@ All three nest the wait result under `data.wait`, so one client helper works aga
any of them:
```json
{ "success": true, "data": {
"sessionId": "28325fd3-caa7-4178-82bf-87dfebf0f464",
"status": "idle",
"limitPaused": false,
"wait": {
"signal": "stop", "until": ["stop", "idle", "exit"],
"timedOut": false, "immediate": false, "ended": false, "aborted": false,
"waitedMs": 8421, "timeoutMs": 60000
{
"success": true,
"data": {
"sessionId": "28325fd3-caa7-4178-82bf-87dfebf0f464",
"status": "idle",
"limitPaused": false,
"wait": {
"signal": "stop",
"until": ["stop", "idle", "exit"],
"timedOut": false,
"immediate": false,
"ended": false,
"aborted": false,
"waitedMs": 8421,
"timeoutMs": 60000
}
}
}}
}
```
`POST .../input` returns the same `wait` object alongside `delivered`, `duplicate`,
@@ -353,21 +361,21 @@ redelivery (harmless, the turn it refers to may be long over), while with
client that reads `delivered === false` as "duplicate" silently treats a failed send
as a success.
| Field | Type | Meaning |
|-------|------|---------|
| `wait.signal` | signal \| `null` | the signal that fired (`/wait` and `/input` only) |
| `wait.until` | array of signals | what the server actually waited on, after narrowing the default set for the session's mode (`/wait` and `/input` only) |
| `wait.matched` | boolean | the string appeared (`/wait-output` only) |
| `wait.match` | string | the literal that was searched for (`/wait-output` only) |
| `wait.snippet` | string \| `null` | bounded window of output around the match, blank runs collapsed for readability (`/wait-output` only) |
| `wait.timedOut` | boolean | the wait hit its timeout. Still a `200` |
| `wait.immediate` | boolean | the condition already held at call time, so nothing was waited for (`waitedMs` is 0) |
| `wait.ended` | boolean | the session went away (deleted or torn down) before the condition was met |
| `wait.aborted` | boolean | the client hung up, so the waiter was released without resolving — and by that definition a client never reads `true`. When the **server** abandons a wait itself (send-and-wait against a session with no PTY), it answers in about a millisecond with `ended: true`, `delivered: false`, `duplicate: false` and `aborted: false`: `delivered`/`ended` carry that story, and `aborted` stays the transport flag. Present for completeness; treat a `true` as "this wait answered nothing", never as an outcome |
| `wait.waitedMs` | number | wall-clock ms actually spent waiting |
| `wait.timeoutMs` | number | the timeout **after clamping**, which is what was applied |
| `status` | `SessionStatus` | the session's status after the wait, so a caller that timed out still learns where things stand |
| `limitPaused` | boolean | the session is paused on a usage limit and will emit nothing until its reset, so a timeout here is expected rather than a stall worth retrying hard |
| Field | Type | Meaning |
| ---------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `wait.signal` | signal \| `null` | the signal that fired (`/wait` and `/input` only) |
| `wait.until` | array of signals | what the server actually waited on, after narrowing the default set for the session's mode (`/wait` and `/input` only) |
| `wait.matched` | boolean | the string appeared (`/wait-output` only) |
| `wait.match` | string | the literal that was searched for (`/wait-output` only) |
| `wait.snippet` | string \| `null` | bounded window of output around the match, blank runs collapsed for readability (`/wait-output` only) |
| `wait.timedOut` | boolean | the wait hit its timeout. Still a `200` |
| `wait.immediate` | boolean | the condition already held at call time, so nothing was waited for (`waitedMs` is 0) |
| `wait.ended` | boolean | the session went away (deleted or torn down) before the condition was met |
| `wait.aborted` | boolean | the client hung up, so the waiter was released without resolving — and by that definition a client never reads `true`. When the **server** abandons a wait itself (send-and-wait against a session with no PTY), it answers in about a millisecond with `ended: true`, `delivered: false`, `duplicate: false` and `aborted: false`: `delivered`/`ended` carry that story, and `aborted` stays the transport flag. Present for completeness; treat a `true` as "this wait answered nothing", never as an outcome |
| `wait.waitedMs` | number | wall-clock ms actually spent waiting |
| `wait.timeoutMs` | number | the timeout **after clamping**, which is what was applied |
| `status` | `SessionStatus` | the session's status after the wait, so a caller that timed out still learns where things stand |
| `limitPaused` | boolean | the session is paused on a usage limit and will emit nothing until its reset, so a timeout here is expected rather than a stall worth retrying hard |
Read the outcome by discriminator, in this order:
@@ -390,12 +398,12 @@ read the timeout as "the worker is wedged" and kill a session that was working f
### Errors
| `errorCode` | HTTP | When |
|-------------|------|------|
| `INVALID_INPUT` | 400 | unknown `until` / `wait` token; `stop` or `blocked` requested explicitly on a mode that installs no hooks (the message names the mode); `regex=` on `/wait-output`; `match` outside 1 to 200 chars; a non-numeric `timeout` |
| `NOT_FOUND` | 404 | no such session, or one this caller does not own |
| `SESSION_BUSY` | 409 | this session's waiter cap is full |
| `RATE_LIMITED` | 429 | a per-owner or process-wide waiter cap is full. Retry later; the session you named is not the problem |
| `errorCode` | HTTP | When |
| --------------- | ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `INVALID_INPUT` | 400 | unknown `until` / `wait` token; `stop` or `blocked` requested explicitly on a mode that installs no hooks (the message names the mode); `regex=` on `/wait-output`; `match` outside 1 to 200 chars; a non-numeric `timeout` |
| `NOT_FOUND` | 404 | no such session, or one this caller does not own |
| `SESSION_BUSY` | 409 | this session's waiter cap is full |
| `RATE_LIMITED` | 429 | a per-owner or process-wide waiter cap is full. Retry later; the session you named is not the problem |
The two capacity codes are deliberately different. A process-wide cap reported as
`SESSION_BUSY` would tell the caller to switch sessions, which cannot help. The
@@ -446,9 +454,9 @@ Design: [`approvals-inbox-plan.md`](approvals-inbox-plan.md).
- `GET /api/v1/approvals` → `{ approvals: ApprovalItem[] }`, oldest first,
ownership-scoped in multi-user mode. `ApprovalItem`: `{ id, sessionId,
sessionName, kind: 'permission'|'question'|'idle', createdAt, toolName?,
toolSummary?, message?, cwd?, context?, options?: {n, label}[],
acknowledgedAt? }`. `context` is the ANSI-stripped visible pane frame;
sessionName, kind: 'permission'|'question'|'idle', createdAt, toolName?,
toolSummary?, message?, cwd?, context?, options?: {n, label}[],
acknowledgedAt? }`. `context` is the ANSI-stripped visible pane frame;
`options` is present only when the dialog's numbered choices parsed
confidently; `acknowledgedAt` marks an item a human has already looked at
(see `/viewed` below) and tells clients not to re-arm its tab alert. Listing
@@ -466,7 +474,7 @@ Design: [`approvals-inbox-plan.md`](approvals-inbox-plan.md).
first, `422 OPERATION_FAILED` when the session refused input.
- `POST /api/v1/approvals/:id/dismiss` removes the item without keystrokes.
- `POST /api/v1/approvals/session/:sessionId/viewed` → `{ sessionId,
acknowledged: itemId | null }`. Marks the session's pending **idle** item as
acknowledged: itemId | null }`. Marks the session's pending **idle** item as
seen by a human (the web UI calls it when you open the session's tab): the
item stays pending and answerable, but stops arming the yellow tab alert on
every client, including after a reload. Permission/question items are never
@@ -479,6 +487,48 @@ re-captured, or the item acknowledged), `approval:resolved` (`{ id, sessionId, k
`resolution` one of `answered | resolved_in_terminal | superseded |
session_ended | dismissed | expired`).
## Reboot restore
A host reboot takes the tmux server down with it, so every pane dies and the
board comes up empty. At boot Codeman works out which sessions the reboot
destroyed and holds that plan in memory, and these endpoints let a client offer
it to the user. Nothing creates a pane until the user asks: the boot-time reboot
heuristic decides whether to ASK, never whether to act.
Claude-mode sessions only (others carry their conversation id in their own
config object); remote and docker sessions are never offered, because both need
another host or container to be up. The plan is in-memory, so a server restart
drops it and the offer is gone — the conversations themselves are unaffected,
since they live in the CLI's own transcript store and stay reachable from the
Resume list. A plan nobody spends expires after 24 hours.
- `GET /api/v1/reboot-restore` → `{ sessions: RestorableSession[],
scrollbackRestored: false }`, ownership-scoped in multi-user mode.
`RestorableSession`: `{ id, name?, workingDir, mode, owner? }`. The persisted
record itself is never sent. `scrollbackRestored` is always `false` and exists
so a client states it: a restored session is a NEW pane, so the conversation
continues and the terminal history does not.
- `POST /api/v1/reboot-restore/restore` with `{ sessionIds?: string[] }` (omit
to restore everything the caller can see) → `{ restored: RestorableSession[],
skipped: { sessionId, reason }[] }`. `reason` is one of `workspace-missing`
(the directory is gone), `workspace-forbidden` (in multi-user mode it is
outside the workspace of the user the session belongs to, re-checked against
that owner's current grant rather than the caller's), `already-live` (the conversation is already
open, typically resumed by hand from the Resume list), `capacity-reached`
(the global or per-user session cap), or `rebuild-failed` (the agent would not
start, most often a CLI binary missing from the server's PATH).
`409 CONFLICT` when that caller already has a restore running. Entries are
removed from the plan before any pane is built, so a double-click cannot put
two panes on one conversation; anything that never became a pane goes back on
offer, except `already-live`, which cannot stop being true. A restored session
comes back attached, idle and disarmed — respawn controllers and Ralph loops
are never re-armed automatically.
- `POST /api/v1/reboot-restore/dismiss` → `{ dismissed: n }`. Drops the offer
for everything the caller can see.
Each rebuilt session also emits the ordinary `session:created` SSE event, so
clients other than the one that clicked pick it up without refetching.
## Read My Mind intent profiles
Per-case profiles of what the user is trying to accomplish: user/agent-stated
@@ -491,7 +541,7 @@ user guide: [`readmymind.md`](readmymind.md).
- `GET /api/v1/sessions/:id/intent` -> `{ intent: IntentProfile }` for the
session's case. `IntentProfile`: `{ key, workingDir, updatedAt, goals,
recentPrompts: { ts, sessionId, text }[] }` (prompts oldest first, FIFO cap
recentPrompts: { ts, sessionId, text }[] }` (prompts oldest first, FIFO cap
50, each <= 500 chars). A case with nothing recorded answers an empty
profile with `updatedAt: 0`; nothing is persisted by reads.
- `PUT /api/v1/sessions/:id/intent` with `{ goals }` (<= 8192 chars, strict
@@ -524,7 +574,7 @@ same speech-to-text service the CLI's own `/voice` mode uses. Gated on the synce
[`claude-voice-plan.md`](claude-voice-plan.md).
- `GET /api/v1/voice/status` -> `{ available, reason?, subscriptionType?,
expiresAt? }`. `reason` is `disabled` (setting off), `no-credentials` (nobody
expiresAt? }`. `reason` is `disabled` (setting off), `no-credentials` (nobody
signed in to Claude Code on the server), `expired` (the access token elapsed;
running any Claude session refreshes it) or `malformed`. The OAuth token
itself is never returned by this or any other endpoint.
src/reboot-restore.ts
+272
View File
@@ -0,0 +1,272 @@
/**
* @fileoverview Decide which sessions a host reboot destroyed and may be rebuilt.
*
* A server restart and a host reboot both leave `reconcileSessions()` reporting
* dead sessions, and they need opposite handling. A server restart leaves the
* tmux panes running, so recovery ATTACHES to them. A host reboot takes the tmux
* server down with it, so there is nothing to attach to and the pane has to be
* created again. This module holds the decision half of that second case, kept
* free of tmux and disk access so it can be unit tested without either. Every
* observation it reads is gathered by the caller and passed in.
*
* "Eligible" here means a session the user did not end on purpose. The rule that
* an intentional kill or detach is never auto-revived is enforced at runtime by
* an in-memory guard in `TmuxManager`, and memory does not survive a reboot. The
* durable equivalent is the record `cleanupSession()` leaves behind. An unpinned
* kill deletes the record outright, so it is already absent here. A pinned kill
* goes through `demoteOrRemoveSession()` and lands as `status: 'stopped'`, which
* is the marker this module refuses. Pruning keeps a pinned record WITHOUT
* touching its status, so a pinned session a reboot killed still reads `idle` or
* `busy` and stays eligible.
*
* ⚠️ Ending the AGENT rather than the session is a shape this module CANNOT
* recognise today, and a reboot restores it. `/exit` ends the CLI inside the
* pane, `remain-on-exit` keeps the pane, and the PTY Codeman owns is the
* `tmux attach-session` process, which stays alive throughout — so no exit
* handler runs, no lifecycle `exit` is logged, and the record keeps both its pid
* and `status: 'idle'`. Nothing durable distinguishes it from a session that was
* simply idle when the power went. Ark0N/Codeman#446 covers making Codeman
* notice the dead pane; until a record can say the agent is gone, this pass will
* offer those sessions back, and the user dismisses or closes them.
*
* The `pid` check below is therefore NOT that rule. It refuses a record whose
* attach process was already gone, which is a session that never started or
* whose pane died outright.
*
* @dependencies types (SessionState), config/cli-registry
* @consumedby web/server (plan build at boot), web/routes/reboot-restore-routes
*
* @module reboot-restore
*/
import type { SessionState } from './types.js';
import { getCli } from './config/cli-registry/registry.js';
/** Session statuses a reboot restore may rebuild. `stopped` is the kill marker. */
const RESTORABLE_STATUSES: ReadonlySet<string> = new Set(['idle', 'busy', 'error']);
/** Observations the reboot heuristic reads. Gathered by the caller, never here. */
export interface RebootEvidence {
/** Sessions that still had a live pane during reconciliation. */
livePaneCount: number;
/** Sessions reconciliation just marked dead. */
deadSessionCount: number;
/** `os.uptime()`, in seconds. */
uptimeSeconds: number;
/** Newest `lastActivityAt` across the persisted records, in ms since the epoch. */
newestPersistedActivityAt: number;
/** `Date.now()` when the evidence was gathered, in ms. */
now: number;
}
/**
* Decide whether the machine plausibly rebooted rather than the server restarting.
*
* Two signals have to agree. The socket must hold no panes at all while state
* still lists sessions, which rules out an ordinary server restart. The host
* must also have booted after the newest persisted session activity, which is
* the corroboration `os.uptime()` provides cheaply. A wiped tmux socket on a
* long-uptime host fails the second test, so a user who killed the tmux server
* by hand does not get every session offered back to them.
*
* This heuristic decides whether to ASK, never whether to act. A wrong yes costs
* the user a banner they dismiss, because the restore itself waits for a click.
*
* ⚠️ `os.uptime()` reports the HOST's uptime, which a container shares. A Codeman
* running in Docker therefore sees a long uptime after its own container restarts,
* the boot test fails, and no banner appears. The feature is effectively off for
* containerized installs. That is the safe direction to fail in, and fixing it
* needs a boot signal the container actually owns rather than a wider heuristic.
*/
export function looksLikeHostReboot(evidence: RebootEvidence): boolean {
if (evidence.deadSessionCount === 0) return false;
if (evidence.livePaneCount > 0) return false;
if (evidence.newestPersistedActivityAt <= 0) return false;
const bootedAt = evidence.now - evidence.uptimeSeconds * 1000;
return bootedAt > evidence.newestPersistedActivityAt;
}
/**
* Pick the conversation the rebuilt pane should resume.
*
* The chain's tail is the newest conversation the session was holding, which is
* what a compact or a clear leaves behind; `resumeSessionId` covers a session
* that was itself started as a resume, and the session id is the original
* conversation for everything else.
*/
export function resolveResumeConversationId(state: SessionState): string {
const chain = state.claudeSessionChain;
const chainTail = Array.isArray(chain) && chain.length > 0 ? chain[chain.length - 1] : undefined;
return chainTail || state.resumeSessionId || state.id;
}
/**
* Why one session was passed over. Reported for logging and shown to the user.
*
* The first seven are decided before anything is built. `capacity-reached` and
* `rebuild-failed` can only happen once a click is spending the plan, and they
* are the two the banner must not confuse with a missing workspace: one means
* "try again after closing something", the other means the CLI would not start.
*/
export interface RebootRestoreRejection {
sessionId: string;
reason:
| 'no-persisted-record'
| 'intentionally-ended'
| 'not-running'
| 'respawn-blocked'
| 'remote-or-docker'
| 'unsupported-mode'
| 'no-working-dir'
| 'workspace-missing'
| 'workspace-forbidden'
| 'already-live'
| 'capacity-reached'
| 'rebuild-failed';
}
/** One restorable session, as the banner shows it and the rebuild replays it. */
export interface RebootRestoreEntry {
sessionId: string;
name?: string;
workingDir: string;
owner?: string;
mode: string;
/** The conversation the rebuilt pane resumes. */
resumeConversationId: string;
/**
* The persisted record, kept whole so the rebuild can replay what it held.
* Read at boot, before pruning deletes it, and held in memory until the click.
*/
state: SessionState;
}
export interface RebootRestorePlan {
restore: RebootRestoreEntry[];
skipped: RebootRestoreRejection[];
}
/**
* Split the sessions reconciliation just killed into the ones a reboot restore
* may offer and the ones it must leave alone.
*
* @param deadSessionIds Session ids `reconcileSessions()` reported as dead.
* @param persisted The `state.json` session records, which `cleanupStaleSessions()`
* has not pruned yet at the point this runs.
* @param workspaceExists Whether a working directory is still on disk. A tmux
* session can outlive its deleted repo, and rebuilding one there would scaffold
* an empty tree. The caller owns the disk access; the click re-checks, because
* a repo can be deleted between the boot and the click.
*/
export function planRebootRestore(
deadSessionIds: readonly string[],
persisted: Readonly<Record<string, SessionState>>,
workspaceExists: (workingDir: string) => boolean
): RebootRestorePlan {
const restore: RebootRestoreEntry[] = [];
const skipped: RebootRestoreRejection[] = [];
for (const sessionId of deadSessionIds) {
const state = persisted[sessionId];
if (!state) {
// An unpinned kill already deleted the record, so absence IS the guard.
skipped.push({ sessionId, reason: 'no-persisted-record' });
continue;
}
if (!RESTORABLE_STATUSES.has(state.status)) {
// A pinned kill was demoted to `stopped`. Reviving it would undo the kill.
skipped.push({ sessionId, reason: 'intentionally-ended' });
continue;
}
if (state.pid === null || state.pid === undefined) {
// No attach process when the record was last written: the session never
// started, or its pane died outright rather than its agent exiting inside a
// surviving pane. Either way there was nothing running to bring back.
//
// ⚠️ This does NOT catch a session the user ended with `/exit`. See the
// module header: that leaves the pid in place, because the pid is the tmux
// attach process and `remain-on-exit` keeps it alive.
//
// Conservative on purpose. A session that somehow persisted no pid while
// genuinely running is not offered, and its conversation stays reachable
// from the Resume list, which is where every session would be without this
// feature.
skipped.push({ sessionId, reason: 'not-running' });
continue;
}
if (state.respawnBlocked === true) {
// The crash-loop breaker tripped on this pane. Re-creating it restarts the loop.
skipped.push({ sessionId, reason: 'respawn-blocked' });
continue;
}
if (state.remote || state.docker) {
// Both need another host or a container to be up, which a just-booted machine
// cannot promise. The remote reconnect watcher owns the remote case already.
skipped.push({ sessionId, reason: 'remote-or-docker' });
continue;
}
// Capability, not a CLI id: this pass resumes by handing the CLI a conversation
// id through the top-level `resumeSessionId`, which only a CLI whose history the
// claude-jsonl reader understands can consume that way. Others carry their thread
// id in their own `<Mode>Config`, which this pass does not thread through.
if (getCli(state.mode ?? 'claude')?.capabilities.transcript !== 'claude-jsonl') {
skipped.push({ sessionId, reason: 'unsupported-mode' });
continue;
}
if (!state.workingDir) {
skipped.push({ sessionId, reason: 'no-working-dir' });
continue;
}
if (!workspaceExists(state.workingDir)) {
skipped.push({ sessionId, reason: 'workspace-missing' });
continue;
}
restore.push({
sessionId,
name: state.name,
workingDir: state.workingDir,
owner: state.owner,
mode: state.mode ?? 'claude',
resumeConversationId: resolveResumeConversationId(state),
state,
});
}
return { restore, skipped };
}
/**
* Drop the entries whose conversation is already on screen.
*
* Hours can pass between the boot that built the plan and the click that spends
* it, and the Resume list can reach the same conversation in the meantime. Two
* panes running `claude --resume` on one conversation is the failure this
* prevents, so a match on either the session id or the conversation id is enough
* to skip the entry.
*/
export function rejectAlreadyLive(
entries: readonly RebootRestoreEntry[],
liveSessionIds: ReadonlySet<string>,
liveConversationIds: ReadonlySet<string>
): RebootRestorePlan {
const restore: RebootRestoreEntry[] = [];
const skipped: RebootRestoreRejection[] = [];
for (const entry of entries) {
if (liveSessionIds.has(entry.sessionId) || liveConversationIds.has(entry.resumeConversationId)) {
skipped.push({ sessionId: entry.sessionId, reason: 'already-live' });
continue;
}
restore.push(entry);
}
return { restore, skipped };
}
/** Newest `lastActivityAt` across persisted records, or 0 when there are none. */
export function newestPersistedActivity(persisted: Readonly<Record<string, SessionState>>): number {
let newest = 0;
for (const state of Object.values(persisted)) {
const stamp = state.lastActivityAt ?? state.createdAt ?? 0;
if (stamp > newest) newest = stamp;
}
return newest;
}
src/session-env-clamp.ts
+97
View File
@@ -0,0 +1,97 @@
/**
* @fileoverview The env-var half of the multi-user privilege clamp.
*
* A session's `envOverrides` can hand back privilege that the per-CLI config
* clamp removed, so a non-granted owner's overrides get the privileged keys
* stripped before the session is built. The create and resume routes are what
* this bites on: they clamp what a request asked for.
*
* The reboot-restore route calls it as defence in depth, and today it can strip
* nothing. `Session.getEnvOverridesForPersist()` keeps only `CLAUDE_CODE_*` and
* `CLAUDE_CONFIG_DIR` out of a session's overrides, claude's `privilegedEnvKeys`
* are the five `ANTHROPIC_*` names, and that pass admits claude alone — so a
* persisted record cannot carry a clamped key. The call is there for the day the
* persisted set widens. The grant re-resolution that does bite on that path is
* `resolveClaudeModeForUsername`, which recomputes the permission mode.
*
* This lives outside `web/routes` on purpose. The question it answers is about
* session privilege rather than about HTTP, and `cron/cron-service.ts` sets the
* precedent by importing `canUsernameRunPrivilegedCommands` from `user-store.ts`
* directly and re-resolving the owner's grant when a job fires. Every caller here
* re-resolves the grant at the moment it builds a session, for the same reason.
*
* @dependencies user-store (canUsernameRunPrivilegedCommands), config/cli-registry
* @consumedby web/routes/session-routes, web/routes/reboot-restore-routes
*
* @module session-env-clamp
*/
import { canUsernameRunPrivilegedCommands } from './user-store.js';
import { enabledClis } from './config/cli-registry/registry.js';
/**
* Env-var keys a non-granted owner must not be able to set, because each one
* hands back privilege `clampExternalCliBypassForOwner()` just removed, or redirects a
* credential-resolution endpoint.
*
* The DeepSeek three are reachable because `DSH_*` and `DEEPSEEK_*` are
* allowlisted `envOverrides` prefixes (schemas.ts) — which they have to be, since
* that is also how a user configures the harness's non-privileged knobs.
*
* - `DSH_PERMISSION_MODE` IS the harness's permission switch. Every other CLI's
* bypass is a command-line FLAG, reachable only through the per-CLI config the
* clamp already owns; this one is an env var, so the config clamp alone is
* half a gate.
* - `DSH_HOME` points the launcher at a profile tree, and a profile's plugin code
* executes at BOOT, before any approval row can apply. A user who can write a
* workspace can put a profile in it, so this is the wider of the two.
* - `DEEPSEEK_BASE_URL` aims the provider endpoint, and `_configureCliEnv()`
* forwards the SERVER's own `DEEPSEEK_API_KEY` into every dsh pane before
* `applyEnvOverrides()` runs — so a non-granted owner who could set the base
* URL would have the operator's API key sent as a bearer credential to a host
* of their choosing. (`DEEPSEEK_API_KEY` itself stays overridable: supplying
* your OWN key removes privilege rather than granting it.)
* - `OMP_AUTH_BROKER_URL`/`OMP_AUTH_BROKER_TOKEN` are where omp resolves
* credentials from — the same shape as `DEEPSEEK_BASE_URL` above, reachable
* because `OMP_*` is an allowlisted prefix. Unlike DeepSeek, Codeman does not
* forward any operator-held key into an omp pane today (omp's provider
* credentials live in `~/.omp` config files, not env vars), so there is no
* known concrete exfiltration path yet — clamped defensively anyway, since a
* non-granted owner redirecting where a shared multi-tenant deployment
* resolves auth from is not something to allow silently (found in
* Ark0N/Codeman#353 review; omp's own knobs are otherwise mostly `PI_*`,
* already allowlisted for pi and not addressed here — see resolveOmpHome()).
*/
export function ownerClampedEnvKeys(): string[] {
return enabledClis().flatMap((entry) => entry.capabilities.privilegedEnvKeys);
}
/**
* Env-var half of the multi-user bypass clamp.
*
* `clampExternalCliBypassForOwner()` in `web/routes/session-routes.ts` clamps the
* per-CLI CONFIG, and for every CLI
* but DeepSeek that is the whole story. Here it is not: `applyEnvOverrides()` runs
* AFTER `_configureCliEnv()` in tmux-manager, so an override sent on the SAME
* request lands last and wins, and a non-granted owner could restore
* `danger-full-access` on the very request the config clamp downgraded.
*
* Keys are DROPPED rather than rewritten: dropping falls through to what
* `_configureCliEnv()` exports, which is the clamped config and the server's own
* `DSH_HOME`, i.e. exactly the intended state. No-op in single-user mode and for a
* granted owner, like every other clamp here
* (`canUsernameRunPrivilegedCommands()` returns true when `!isMultiUserMode()`),
* and it returns the caller's own object untouched when there is nothing to strip.
*/
export async function clampEnvOverridesForOwner(
owner: string | undefined,
envOverrides: Record<string, string> | undefined
): Promise<Record<string, string> | undefined> {
if (!envOverrides) return envOverrides;
const keys = ownerClampedEnvKeys();
if (!keys.some((key) => key in envOverrides)) return envOverrides;
if (await canUsernameRunPrivilegedCommands(owner)) return envOverrides;
const clamped = { ...envOverrides };
for (const key of keys) delete clamped[key];
return clamped;
}
src/session.ts
+13
View File
@@ -1499,6 +1499,19 @@ export class Session extends EventEmitter {
this._pinnedAt = pinned ? Date.now() : null;
}
/**
* Restore a pin from a persisted record, keeping the moment it was pinned.
*
* `setPinned()` stamps `pinnedAt` with now, which is right for a user pinning a
* session and wrong for a restore: the session-manager orders its pinned group
* by that stamp, so a restored session would jump to the front of a list it had
* been sitting further down.
*/
restorePin(pinned: boolean, pinnedAt?: number): void {
this._pinned = pinned;
this._pinnedAt = pinned ? (pinnedAt ?? Date.now()) : null;
}
get flickerFilterEnabled(): boolean {
return this._flickerFilterEnabled;
}
src/web/ports/session-port.ts
+24
View File
@@ -4,6 +4,7 @@
*/
import type { Session } from '../../session.js';
import type { SessionState } from '../../types.js';
export interface SessionPort {
readonly sessions: ReadonlyMap<string, Session>;
@@ -12,5 +13,28 @@ export interface SessionPort {
setupSessionListeners(session: Session): Promise<void>;
persistSessionState(session: Session): void;
persistSessionStateNow(session: Session): void;
/**
* Re-apply the persisted state a freshly CONSTRUCTED session does not carry.
*
* A `Session` built from a record holds only what its constructor takes, so
* persisting it would otherwise REPLACE the fuller record with the reduced one.
* Two phases: `before-spawn` shapes the pane (the custom-model environment and
* the nice priority) and must precede `startInteractive()`; `after-spawn` is
* the session's own history (the pin, token and cost totals, auto-compact,
* auto-clear, auto-resume, colour, image watcher, flicker filter) and must NOT
* land on a session whose pane failed to start.
*/
reapplyPersistedSessionState(
session: Session,
saved: SessionState,
phase: 'before-spawn' | 'after-spawn'
): Promise<void>;
/**
* Undo a session that was registered but never got a working pane: the map
* entry, its tab-layout slot, and any pane the launch created before throwing.
* Unlike {@link cleanupSession} it leaves the persisted record, the lifetime
* token totals, the Ralph state and the workspace's own files untouched.
*/
discardPartiallyBuiltSession(sessionId: string): Promise<void>;
getSessionStateWithRespawn(session: Session): unknown;
}
src/web/public/app.js
+10
View File
@@ -957,6 +957,10 @@ class CodemanApp {
this.registerServiceWorker();
// Fetch tunnel status for header indicator (desktop only)
this.loadTunnelStatus();
// Ask whether a host reboot left sessions worth rebuilding (banner, never
// automatic). handleInit() re-reads it on every SSE init; this covers the
// path where that event never arrives.
this.initRebootRestoreBanner?.();
// Share a single settings fetch between both consumers
const settingsPromise = fetch('/api/settings').then(r => r.ok ? r.json() : null).then(env => env?.data ?? null).catch(() => null);
this.loadQuickStartCases(null, settingsPromise);
@@ -3759,6 +3763,12 @@ class CodemanApp {
// a fresh load / reconnect (authoritative; wins over the localStorage restore).
if (data.planUsage) this.updatePlanUsageChip(data.planUsage);
// A board left open across a host reboot reconnects HERE, to a server that came
// back with an empty session list. The reboot-restore offer is built at boot,
// before any client could be listening, so re-read it on every init rather than
// only on the page-load path.
this.refreshRebootRestoreBanner?.();
// Update version displays (header and toolbar)
if (data.version) {
const versionEl = this.$('versionDisplay');
src/web/public/index.html
+19
View File
@@ -213,6 +213,24 @@
<button class="offline-banner-retry" id="offlineBannerRetry" onclick="app.retryConnection()">Retry now</button>
</div>
<!-- Reboot-restore offer: shown when the server found sessions a host reboot
killed and is asking whether to rebuild them. Populated by
reboot-restore-ui.js; nothing is created until the user clicks. -->
<div class="reboot-restore-banner" id="rebootRestoreBanner" role="status" hidden>
<span class="reboot-restore-banner-icon" aria-hidden="true">↺</span>
<span class="reboot-restore-banner-text" id="rebootRestoreBannerText"></span>
<span class="reboot-restore-banner-detail" id="rebootRestoreBannerDetail"></span>
<span class="reboot-restore-banner-note">Conversations return; terminal history does not.</span>
<button
class="reboot-restore-banner-accept"
id="rebootRestoreBannerAccept"
onclick="app.restoreRebootSessions()"
>
Restore
</button>
<button class="reboot-restore-banner-dismiss" onclick="app.dismissRebootRestore()">Dismiss</button>
</div>
<!-- Timer Banner (shown when timed run is active) -->
<div class="timer-banner" id="timerBanner" style="display: none;">
<div class="timer-content">
@@ -3535,6 +3553,7 @@
<script defer src="readmymind-ui.js"></script>
<script defer src="ultracode-panel.js"></script>
<script defer src="approvals-ui.js"></script>
<script defer src="reboot-restore-ui.js"></script>
<script defer src="admin-ui.js"></script>
<script defer src="session-ui.js"></script>
<script defer src="webview-tabs.js"></script>
src/web/public/mobile.css
+37
View File
@@ -3241,6 +3241,43 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
other banners. The overlay is fixed and handles its own insets.
============================================================================ */
@media (max-width: 599px) {
/* Reboot-restore banner: the same treatment as the offline banner below. Its
text and note are nowrap and the two buttons cannot shrink, so without this
the actions are pushed off a phone-width viewport and become unreachable. */
.reboot-restore-banner {
padding: 0.4rem 0.5rem;
padding-left: calc(0.5rem + var(--safe-area-left));
padding-right: calc(0.5rem + var(--safe-area-right));
font-size: 0.7rem;
gap: 0.4rem;
}
/* The session names and the scrollback note are the first things to go. The
count plus the two buttons carry the message on their own, and the note
survives as the accept button's title. */
.reboot-restore-banner-detail,
.reboot-restore-banner-note {
display: none;
}
/* A flex item will not shrink below its content width at the default
`min-width: auto`, so without this the nowrap text pushes the buttons off a
360px viewport and the ellipsis never engages. */
.reboot-restore-banner-text {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
}
.reboot-restore-banner-accept,
.reboot-restore-banner-dismiss {
padding: 0.25rem 0.5rem;
}
.reboot-restore-banner-accept {
margin-left: auto;
}
.offline-banner {
padding: 0.4rem 0.5rem;
padding-left: calc(0.5rem + var(--safe-area-left));
src/web/public/reboot-restore-ui.js
+142
View File
@@ -0,0 +1,142 @@
/**
* @fileoverview Reboot-restore banner: offer back the sessions a host reboot destroyed.
*
* A host reboot takes the tmux server down with it, so every session's pane dies
* and the board comes up empty. The server works out what was running from the
* records it still holds at boot, and this banner asks the user whether to
* rebuild them. Nothing is created until they click, because the server's
* reboot guess is a heuristic and a wrong automatic restore would spawn CLI
* processes nobody asked for.
*
* Seeded from `GET /api/reboot-restore` on init and again on every SSE reconnect,
* because the tab most likely to want this is one that was open across the reboot
* and reconnects to a server that came back up with an empty board. Restore posts to
* `POST /api/reboot-restore/restore` and Dismiss posts to
* `POST /api/reboot-restore/dismiss`. Dismiss always clears the banner; Restore
* re-reads the plan afterwards, because the server puts back anything it could
* not build for a reason that may pass, such as a session limit or an agent that
* would not start. The restored sessions arrive as ordinary `session:created`
* events, so no extra rendering is needed here.
*
* The banner says that terminal history did not survive, because a restored
* session is a new pane: the conversation continues and the scrollback does not.
* Saying so is what keeps an empty pane from reading as a broken restore.
* Backend: src/web/reboot-restore-registry.ts, src/web/routes/reboot-restore-routes.ts.
*
* @mixin Extends CodemanApp.prototype via Object.assign
* @dependency app.js (CodemanApp class, showToast)
* @dependency api-client.js at runtime (this._api / this._apiJson)
* @loadorder 11.7 of 17, after approvals-ui.js
*/
/** Plain-language wording for one skip reason, for the toast after a restore. */
function rebootSkipReason(reason) {
switch (reason) {
case 'workspace-missing':
return 'workspace is gone';
case 'workspace-forbidden':
return 'workspace is outside your space';
case 'already-live':
return 'already open';
case 'capacity-reached':
return 'session limit reached';
case 'rebuild-failed':
return 'the agent would not start';
default:
return reason;
}
}
Object.assign(CodemanApp.prototype, {
/** Ask the server whether a reboot left anything on offer, and show the banner if so. */
async initRebootRestoreBanner() {
const data = await this._apiJson('/api/reboot-restore');
const sessions = data?.sessions ?? [];
if (sessions.length === 0) return;
this._rebootRestoreSessions = sessions;
this.renderRebootRestoreBanner();
},
renderRebootRestoreBanner() {
const banner = this.$('rebootRestoreBanner');
if (!banner) return;
const sessions = this._rebootRestoreSessions ?? [];
if (sessions.length === 0) {
banner.hidden = true;
return;
}
const count = sessions.length;
const text = this.$('rebootRestoreBannerText');
if (text) {
const noun = count === 1 ? 'session' : 'sessions';
text.textContent = `Restore ${count} ${noun} from before the reboot`;
}
const detail = this.$('rebootRestoreBannerDetail');
if (detail) {
// Names, so the user can tell what they are about to relaunch.
const names = sessions
.map((s) => s.name || s.workingDir?.split('/').pop() || s.id.slice(0, 8))
.slice(0, 4)
.join(', ');
detail.textContent = count > 4 ? `${names}, …` : names;
detail.title = sessions.map((s) => `${s.name || s.id}\n${s.workingDir}`).join('\n\n');
}
const accept = this.$('rebootRestoreBannerAccept');
// The note is hidden at phone width, so the warning travels on the button too.
if (accept) accept.title = 'Conversations return; terminal history does not.';
banner.hidden = false;
},
/** Rebuild everything on offer. The panes are new, so scrollback does not come back. */
async restoreRebootSessions() {
const button = this.$('rebootRestoreBannerAccept');
if (button) button.disabled = true;
const res = await this._api('/api/reboot-restore/restore', { method: 'POST', body: {} });
if (res && res.status === 409) {
if (button) button.disabled = false;
this.showToast?.('A restore is already running', 'info');
return;
}
// The uniform envelope wraps every /api payload; reading the outer object
// would report every count as zero.
const body = res && res.ok ? (await res.json().catch(() => null))?.data : null;
if (!body) {
if (button) button.disabled = false;
this.showToast?.('Could not restore the sessions', 'error');
return;
}
const restored = body.restored?.length ?? 0;
const skipped = body.skipped?.length ?? 0;
// Re-read rather than clearing: the server puts back anything it could not
// build for a reason that may pass, such as a session limit or an agent that
// would not start, and blanking the banner here would put those entries out
// of reach until a reload.
await this.refreshRebootRestoreBanner();
if (button) button.disabled = false;
if (restored > 0) {
const noun = restored === 1 ? 'conversation' : 'conversations';
this.showToast?.(`Restored ${restored} ${noun}. Terminal history did not survive the reboot.`, 'success');
}
if (skipped > 0) {
// Each reason means a different next step for the user, so they are not
// collapsed into one message: capacity clears by closing something, a
// failed start usually means the CLI is not on the server's PATH.
const reasons = new Set((body.skipped ?? []).map((s) => s.reason));
this.showToast?.(`${skipped} not restored: ${[...reasons].map(rebootSkipReason).join('; ')}`, 'warning');
}
},
/** Re-read the offer after a reconnect, for a tab that was open across the reboot. */
async refreshRebootRestoreBanner() {
const data = await this._apiJson('/api/reboot-restore');
this._rebootRestoreSessions = data?.sessions ?? [];
this.renderRebootRestoreBanner();
},
/** Drop the offer. The Resume list still reaches every one of these conversations. */
async dismissRebootRestore() {
this._rebootRestoreSessions = [];
this.renderRebootRestoreBanner();
await this._apiPost('/api/reboot-restore/dismiss', {});
},
});
src/web/public/styles.css
+79
View File
@@ -15243,6 +15243,85 @@ html[data-skin="daylight-blue"] .welcome-btn-tunnel.active:hover {
skin, including the light ones. Visibility is driven by the `hidden`
attribute, so the display rules need !important to lose to it. */
/* Reboot-restore offer. Amber rather than red: nothing is wrong, the board is
asking a question, and the user can ignore it. See reboot-restore-ui.js. */
.reboot-restore-banner {
display: flex;
align-items: center;
gap: 0.6rem;
padding: 0.45rem 1rem;
background: linear-gradient(90deg, #b45309, #92400e);
border-bottom: 1px solid rgba(0, 0, 0, 0.35);
color: #fff;
font-size: 0.78rem;
font-weight: 600;
letter-spacing: 0.01em;
flex-shrink: 0;
z-index: 1250;
}
.reboot-restore-banner[hidden] {
display: none !important;
}
.reboot-restore-banner-icon {
flex-shrink: 0;
font-size: 0.95rem;
line-height: 1;
}
.reboot-restore-banner-text {
white-space: nowrap;
}
.reboot-restore-banner-detail {
color: rgba(255, 255, 255, 0.8);
font-weight: 500;
/* A flex item will not shrink below its content width at the default
`min-width: auto`, so without this the session names push the buttons out of
the line between the phone breakpoint and full width. */
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.reboot-restore-banner-note {
color: rgba(255, 255, 255, 0.75);
font-weight: 500;
white-space: nowrap;
margin-left: auto;
}
.reboot-restore-banner-accept,
.reboot-restore-banner-dismiss {
flex-shrink: 0;
padding: 0.2rem 0.6rem;
border-radius: 5px;
border: 1px solid rgba(255, 255, 255, 0.55);
background: rgba(255, 255, 255, 0.12);
color: #fff;
font-size: 0.72rem;
font-weight: 600;
cursor: pointer;
}
.reboot-restore-banner-accept:hover,
.reboot-restore-banner-dismiss:hover {
background: rgba(255, 255, 255, 0.24);
}
.reboot-restore-banner-accept:disabled {
opacity: 0.6;
cursor: default;
}
.reboot-restore-banner-dismiss {
border-color: rgba(255, 255, 255, 0.3);
background: transparent;
font-weight: 500;
}
.offline-banner {
display: flex;
align-items: center;
src/web/reboot-restore-registry.ts
+198
View File
@@ -0,0 +1,198 @@
/**
* @fileoverview The pending restore plan: what a host reboot destroyed, waiting on a click.
*
* The boot pass builds this plan inside `restoreMuxSessions()`, in the window
* where reconciliation has reported the dead sessions and `cleanupStaleSessions()`
* has not pruned their records yet. The board then offers "restore N sessions
* from before the reboot", and `web/routes/reboot-restore-routes` spends the plan
* when the user clicks.
*
* Invariants:
* - Entries are in-memory only. A server restart drops the plan, and nothing
* re-builds it, because the records it was built from are pruned by then.
* That costs the convenience this feature adds and never the conversation:
* the conversation IS the transcript under `~/.claude/projects`, which
* `services/unified-session-service.ts` reads for the Welcome screen's Resume
* list and the Session Manager, and `resumeHistorySession()` in
* `web/public/terminal-ui.js` resumes from a row there with no persisted
* session record involved. A dropped plan therefore returns the user to
* resuming by hand, one at a time, which is where they are without this
* feature. What the plan held that a transcript does not is the owner, the
* name, the env overrides, the effort and the lineage.
* - Module-level singleton in the style of `web/approval-inbox.ts`: no `Session`
* import and no IO, which keeps it unit-testable and cycle-free.
* - Spending is take-then-build: `take()` removes entries synchronously, before
* the route's first `await`, so a double-click or two devices cannot both
* reach the same entry and put two panes on one conversation.
* - One restore runs at a time per owner. `beginSpending()` single-flights the
* route, so two concurrent clicks cannot interleave pane creation for the same
* user, while two different users never block each other.
*
* @dependencies reboot-restore (RebootRestoreEntry)
* @consumedby web/server (plan build at boot), web/routes/reboot-restore-routes
*
* @module web/reboot-restore-registry
*/
import type { RebootRestoreEntry } from '../reboot-restore.js';
/**
* A plan older than this is dropped on read. A machine that rebooted yesterday
* has moved on, and an offer nobody took by then is noise rather than a rescue.
*/
const PLAN_TTL_MS = 24 * 60 * 60 * 1000;
export class RebootRestoreRegistry {
/** Keyed by session id, in the order the boot pass found them. */
private entries = new Map<string, RebootRestoreEntry>();
/** When the boot pass built the plan, in ms since the epoch. */
private builtAt = 0;
/**
* Entries handed to a restore that has not finished, by session id, each
* remembering which caller is spending it.
*
* A taken entry is still part of the offer until its restore resolves it, so
* it has to stay reachable by everything that can invalidate an offer. Holding
* the entries themselves — rather than a counter to compare against later —
* means `clear()` filters them by the SAME `canAccess(entry.owner)` predicate
* it already applies to the plan. A counter cannot do that, because the caller
* spending an entry need not be its owner: an admin may restore another user's
* sessions, and then the spender and the owner are different keys.
*/
private parked = new Map<string, { entry: RebootRestoreEntry; spender: string | undefined }>();
/**
* Owners with a restore in flight, between its take and its last pane.
* Keyed by owner so one user's restore does not turn another user's click into
* a conflict; `take()` already guarantees no two callers get the same entry.
* Single-user mode has one key, `undefined`, so it behaves as one global flight.
*/
private spending = new Set<string | undefined>();
/** Replace the plan with what the boot pass found. An empty list clears it. */
set(entries: readonly RebootRestoreEntry[]): void {
this.entries = new Map(entries.map((entry) => [entry.sessionId, entry]));
this.builtAt = entries.length > 0 ? Date.now() : 0;
// A fresh boot plan supersedes anything an in-flight restore still holds.
this.parked.clear();
}
/**
* The entries a viewer may see, newest plan first-come order preserved.
*
* @param canAccess Ownership predicate, so a user sees their own entries and
* an admin sees all. Applied here rather than in the route so the count the
* banner shows and the entries a click spends come from one filter.
*/
list(canAccess: (owner: string | undefined) => boolean): RebootRestoreEntry[] {
this.dropIfExpired();
return [...this.entries.values()].filter((entry) => canAccess(entry.owner));
}
/**
* Remove and return the entries a click is about to spend.
*
* Synchronous and total: an entry leaves the plan here, before any pane is
* created, so a second click finds nothing to spend. Entries a caller may not
* access are left in place, and unknown ids are ignored.
*
* @param sessionIds The ids to spend, or undefined for every visible entry.
*/
take(
canAccess: (owner: string | undefined) => boolean,
sessionIds: readonly string[] | undefined,
spender: string | undefined
): RebootRestoreEntry[] {
this.dropIfExpired();
const wanted = sessionIds ? new Set(sessionIds) : undefined;
const taken: RebootRestoreEntry[] = [];
for (const entry of [...this.entries.values()]) {
if (wanted && !wanted.has(entry.sessionId)) continue;
if (!canAccess(entry.owner)) continue;
this.entries.delete(entry.sessionId);
// Parked rather than forgotten: until this restore resolves the entry, a
// dismiss still has to be able to reach and cancel it.
this.parked.set(entry.sessionId, { entry, spender });
taken.push(entry);
}
return taken;
}
/**
* Put entries back after a rebuild never got as far as creating a pane.
*
* Used for the click-time rejections that may resolve themselves: a workspace
* that comes back, a capacity limit the user makes room under, a CLI that
* starts once its binary is on the PATH. A conversation the user resumed by
* hand is NOT put back, because that one cannot stop being true, and an entry
* the banner keeps re-offering forever is noise only Dismiss can clear.
*/
releaseFlight(spender: string | undefined, keep: readonly RebootRestoreEntry[]): void {
const wanted = new Set(keep.map((entry) => entry.sessionId));
let added = 0;
for (const [sessionId, held] of [...this.parked]) {
if (held.spender !== spender) continue;
this.parked.delete(sessionId);
// Still parked means nothing cancelled it while the restore ran. A dismiss,
// an expiry or a fresh boot plan removes it from `parked`, and then it does
// not come back however the restore ended.
if (wanted.has(sessionId)) {
this.entries.set(sessionId, held.entry);
added += 1;
}
}
if (added > 0 && this.builtAt === 0) this.builtAt = Date.now();
}
/** Drop the entries a viewer can see. Returns how many went. */
clear(canAccess: (owner: string | undefined) => boolean): number {
const removable = [...this.entries.values()].filter((entry) => canAccess(entry.owner));
for (const entry of removable) this.entries.delete(entry.sessionId);
// Entries a restore is holding are dismissed by the same rule, so a dismiss
// that lands mid-restore wins. Judged on the ENTRY's owner, exactly as above,
// rather than on who happens to be restoring it.
let parkedRemoved = 0;
for (const [sessionId, held] of [...this.parked]) {
if (!canAccess(held.entry.owner)) continue;
this.parked.delete(sessionId);
parkedRemoved += 1;
}
if (this.entries.size === 0) this.builtAt = 0;
return removable.length + parkedRemoved;
}
/**
* Claim the right to run a restore for one owner, or report that owner already
* has one running. Callers that get `true` must call `endSpending()` in a
* `finally` with the same owner.
*/
beginSpending(owner?: string): boolean {
if (this.spending.has(owner)) return false;
this.spending.add(owner);
return true;
}
endSpending(owner?: string): void {
this.spending.delete(owner);
}
/** Test hook: forget everything, including the single-flight claim. */
reset(): void {
this.entries.clear();
this.parked.clear();
this.builtAt = 0;
this.spending.clear();
}
private dropIfExpired(): void {
if (this.builtAt > 0 && Date.now() - this.builtAt > PLAN_TTL_MS) {
// A restore that took entries just before the expiry must not hand them
// back afterwards and give an expired plan another full day of life.
this.parked.clear();
this.entries.clear();
this.builtAt = 0;
}
}
}
/** Process-wide singleton, mirroring `approvalInbox`. */
export const rebootRestoreRegistry = new RebootRestoreRegistry();
src/web/routes/index.ts
+1
View File
@@ -11,6 +11,7 @@ export { registerCronRoutes } from './cron-routes.js';
export { registerSystemRoutes } from './system-routes.js';
export { registerHookEventRoutes } from './hook-event-routes.js';
export { registerApprovalRoutes } from './approval-routes.js';
export { registerRebootRestoreRoutes } from './reboot-restore-routes.js';
export { registerReadMyMindRoutes } from './readmymind-routes.js';
export { registerStatusTelemetryRoutes } from './status-telemetry-routes.js';
export { registerCaseRoutes } from './case-routes.js';
src/web/routes/reboot-restore-routes.ts
+273
View File
@@ -0,0 +1,273 @@
/**
* @fileoverview Reboot-restore routes: offer back the sessions a host reboot destroyed.
*
* The boot pass leaves a plan in `web/reboot-restore-registry` when the machine
* plausibly rebooted. The board reads it, shows a banner, and the user decides:
* - `GET /api/reboot-restore`: what is on offer, ownership-scoped
* - `POST /api/reboot-restore/restore`: rebuild some or all of it
* - `POST /api/reboot-restore/dismiss`: drop the offer
*
* A click, not the heuristic, is what creates panes. The heuristic only decides
* whether the banner appears, so a wrong yes costs a line of text the user
* dismisses rather than N CLI processes nobody asked for.
*
* Rebuilding is take-then-build: entries leave the plan synchronously at the top
* of the route, before the first `await`, and the whole route is single-flighted,
* so a double-click or two devices cannot put two panes on one conversation.
* Three things are re-checked at click time rather than trusted from boot: the
* owner's privilege grant, the workspace still being on disk, and the
* conversation not already being live because the user resumed it by hand.
*
* A rebuilt session comes back attached, idle and disarmed. Respawn controllers
* and Ralph loops are deliberately not re-armed, and its terminal scrollback is
* gone, because the pane is new. The banner says so.
*/
import { FastifyInstance } from 'fastify';
import { existsSync } from 'node:fs';
import { ApiErrorCode, createErrorResponse, getErrorMessage } from '../../types.js';
import { RebootRestoreRequestSchema } from '../schemas.js';
import {
parseBody,
getAuthUser,
canAccessOwned,
ownerFor,
isWorkingDirAllowedForUsername,
sessionCapacityMessage,
} from '../route-helpers.js';
import { rebootRestoreRegistry } from '../reboot-restore-registry.js';
import { rejectAlreadyLive, type RebootRestoreEntry, type RebootRestoreRejection } from '../../reboot-restore.js';
import { clampEnvOverridesForOwner } from '../../session-env-clamp.js';
import { Session } from '../../session.js';
import { resolveClaudeModeForUsername } from '../../user-store.js';
import { getCli } from '../../config/cli-registry/registry.js';
import { applyWorkspaceHooks } from '../../hooks-config.js';
import { getLifecycleLog } from '../../session-lifecycle-log.js';
import { STATS_COLLECTION_INTERVAL_MS } from '../../config/server-timing.js';
import { SseEvent } from '../sse-events.js';
import type { SessionAttachmentHistoryItem } from '../../types.js';
import type { SessionPort, EventPort, ConfigPort, InfraPort } from '../ports/index.js';
type RebootRestoreCtx = SessionPort & EventPort & ConfigPort & InfraPort;
/** The banner's view of one restorable session. The record itself never leaves the server. */
function toBannerItem(entry: RebootRestoreEntry) {
return {
id: entry.sessionId,
name: entry.name,
workingDir: entry.workingDir,
mode: entry.mode,
owner: entry.owner,
};
}
export function registerRebootRestoreRoutes(app: FastifyInstance, ctx: RebootRestoreCtx): void {
const accessorFor = (req: Parameters<typeof getAuthUser>[0]) => {
const user = getAuthUser(req);
return (owner: string | undefined) => canAccessOwned(user, owner);
};
// ========== What is on offer ==========
app.get('/api/reboot-restore', async (req) => {
const entries = rebootRestoreRegistry.list(accessorFor(req));
return {
sessions: entries.map(toBannerItem),
// Said plainly here so the banner never implies a full restore: the pane is
// new, so the conversation continues and the terminal history does not.
scrollbackRestored: false,
};
});
// ========== Spend it ==========
app.post('/api/reboot-restore/restore', async (req, reply) => {
const body = parseBody(RebootRestoreRequestSchema, req.body, 'Invalid reboot restore request');
const canAccess = accessorFor(req);
const owner = ownerFor(req);
// Take BEFORE the first await: a second click must find nothing to spend.
// The flight is per owner, because `take()` already guarantees two callers
// never receive the same entry, so one user's restore need not block another's.
if (!rebootRestoreRegistry.beginSpending(owner)) {
return reply.code(409).send(createErrorResponse(ApiErrorCode.CONFLICT, 'A reboot restore is already running'));
}
const taken = rebootRestoreRegistry.take(canAccess, body.sessionIds, owner);
// Entries nothing built a pane for, returned to the plan on every exit path
// including a throw. Without this a failure between here and the loop would
// spend the offer and rebuild nothing, and the plan cannot be rebuilt.
const unspent = new Set(taken);
try {
if (taken.length === 0) return { restored: [], skipped: [] };
// The plan was built at boot and the board has moved on since. A conversation
// the user resumed by hand from the Resume list is already on screen, and a
// second pane on it would fight the first for the same transcript. This one
// is never re-offered: unlike a missing workspace, it cannot stop being true.
const liveSessionIds = new Set(ctx.sessions.keys());
const liveConversationIds = new Set(
[...ctx.sessions.values()].map((session) => session.claudeSessionId).filter((id): id is string => !!id)
);
const { restore, skipped } = rejectAlreadyLive(taken, liveSessionIds, liveConversationIds);
for (const entry of taken) {
if (skipped.some((s) => s.sessionId === entry.sessionId)) unspent.delete(entry);
}
const restored: ReturnType<typeof toBannerItem>[] = [];
const failures: RebootRestoreRejection[] = [...skipped];
const workspaceHooksEnabled = await ctx.getWorkspaceHooksEnabled();
for (const entry of restore) {
// Capacity is re-checked per iteration, because this loop is itself
// creating the sessions it counts. The offer can be a day old, so the
// board may be fuller now than the plan assumed.
const capMsg = sessionCapacityMessage(ctx.sessions, entry.owner);
if (capMsg) {
failures.push({ sessionId: entry.sessionId, reason: 'capacity-reached' });
continue;
}
// A repo can be deleted between the boot that planned this and the click.
if (!existsSync(entry.workingDir)) {
failures.push({ sessionId: entry.sessionId, reason: 'workspace-missing' });
continue;
}
// Multi-user workspace separation: the create route confines a non-admin's
// workingDir to their own case space, and a grant can be withdrawn between
// the session's creation and this restore, so the confinement is re-run
// rather than inherited from the record. Keyed on the OWNER, not on the
// caller: an admin spending another user's entry must be held to that
// user's confinement, and `isWorkingDirAllowed` would wave an admin
// through. The same reason the two grant re-checks below read
// `saved.owner`.
if (!(await isWorkingDirAllowedForUsername(entry.owner, entry.workingDir))) {
// Left on offer: a withdrawn grant can be restored, unlike an already-open
// conversation, so this is not the permanent kind of refusal.
failures.push({ sessionId: entry.sessionId, reason: 'workspace-forbidden' });
continue;
}
try {
const saved = entry.state;
const claudeModeConfig = await ctx.getClaudeModeConfig();
const session = new Session({
// The old id is reused on purpose: a pinned record, subagent parents,
// window states and the lifecycle log all key off it, and the unpinned
// record is gone, so there is nothing to collide with.
id: saved.id,
workingDir: saved.workingDir,
mode: saved.mode,
name: saved.name,
createdAt: saved.createdAt,
mux: ctx.mux,
useMux: true,
// No `muxSession`: the reboot took the pane with it, so `startInteractive()`
// takes its create branch and makes a fresh one.
claudeMode: await resolveClaudeModeForUsername(claudeModeConfig.claudeMode, saved.owner),
allowedTools: claudeModeConfig.allowedTools,
resumeSessionId: entry.resumeConversationId,
// Re-resolved against the owner's CURRENT grant, never replayed from the
// record: a grant held when the record was written may be gone now.
envOverrides: await clampEnvOverridesForOwner(
saved.owner,
(saved as { __envOverrides?: Record<string, string> }).__envOverrides
),
effort: saved.effort,
attachmentHistory:
(saved as { __attachmentHistory?: SessionAttachmentHistoryItem[] }).__attachmentHistory ??
saved.attachmentHistory,
lastSubmitAt: saved.lastSubmitAt,
claudeSessionChain: saved.claudeSessionChain,
lastActivityAt: saved.lastActivityAt,
owner: saved.owner,
parentSessionId: saved.parentSessionId,
});
await ctx.addSession(session);
// Before the listeners, because setupSessionListeners() reads the
// image-watcher flag this phase restores; before the spawn, because the
// custom-model environment and the nice priority shape the process.
await ctx.reapplyPersistedSessionState(session, saved, 'before-spawn');
await ctx.setupSessionListeners(session);
await session.startInteractive();
// The session's own history, applied only once the pane exists: on a
// failed start these totals would belong to a session that never ran.
// Both halves precede the route's OWN persist, which matters because a
// constructed session carries none of this and `toState()` is written
// wholesale, so persisting first would replace the fuller record with
// the reduced one and drop the pin that keeps it from being pruned. A
// listener-driven persist can still land inside the debounce window
// while the pane starts; the write below repairs the record.
await ctx.reapplyPersistedSessionState(session, saved, 'after-spawn');
ctx.persistSessionState(session);
// A session without its workspace hooks goes silently blind: no stop or
// idle events for respawn, no Approvals Inbox item, no red tab on a
// blocking dialog. The boot-time sweep finished hours ago, so the click
// path installs them itself. `hooks: 'always'` is the capability that says
// this CLI installs Codeman's hooks into the workspace.
if (workspaceHooksEnabled && getCli(session.mode)?.capabilities.hooks === 'always') {
await applyWorkspaceHooks(session.workingDir, true).catch((err: unknown) =>
console.warn(`[reboot-restore] hook install failed for ${session.workingDir}: ${getErrorMessage(err)}`)
);
}
getLifecycleLog().log({ event: 'recovered', sessionId: session.id, name: session.name });
// Every other open tab and phone needs this; the clicking tab already has
// the response, and the client's handler is an idempotent upsert.
ctx.broadcast(SseEvent.SessionCreated, ctx.getSessionStateWithRespawn(session));
restored.push(toBannerItem(entry));
} catch (err) {
// One entry that will not start must not stop the rest of the pass, and
// must not leave a registered session with no pane behind it: by this
// point the session is in `ctx.sessions`, holds a tab-layout slot and has
// listeners.
//
// Reaching this is rarer than it looks, measured against a real server:
// the CLI resolver finds its binary by absolute path rather than through
// PATH, and tmux falls back to another directory rather than failing when
// it cannot enter the workspace, so neither of the two obvious "freshly
// booted machine" failures throws. What is left is the mux layer itself
// failing, which is why this path is defended rather than expected.
console.error(`[reboot-restore] failed to rebuild ${entry.sessionId}:`, err);
// Not cleanupSession(): that is the user-initiated delete, and it would
// count this session's historical tokens into the lifetime totals, demote
// a pinned record to `stopped` (which this pass reads as an intentional
// kill, making the session permanently unrestorable) and delete the
// workspace's `.claude-images`. This undoes only the construction.
await ctx
.discardPartiallyBuiltSession(entry.sessionId)
.catch((discardErr: unknown) =>
console.error(`[reboot-restore] discarding a failed rebuild failed: ${getErrorMessage(discardErr)}`)
);
failures.push({ sessionId: entry.sessionId, reason: 'rebuild-failed' });
// Left on offer: the user can put the binary back and click again.
continue;
}
unspent.delete(entry);
}
if (restored.length > 0) {
// A reboot leaves recovery with nothing alive to find, so its own block never
// started the stats collector. This clears and re-arms its interval, so it is
// safe to call whether or not the collector is already running.
ctx.mux.startStatsCollection(STATS_COLLECTION_INTERVAL_MS);
}
return { restored, skipped: failures };
} finally {
// Anything that never became a pane goes back on offer, including after a
// throw, so a transient failure costs a retry rather than the whole plan.
// Ends the flight: entries still parked for it come back if they are in
// `unspent`, and a Dismiss that unparked them meanwhile wins.
rebootRestoreRegistry.releaseFlight(owner, [...unspent]);
rebootRestoreRegistry.endSpending(owner);
}
});
// ========== Drop it ==========
app.post('/api/reboot-restore/dismiss', async (req) => {
const dismissed = rebootRestoreRegistry.clear(accessorFor(req));
return { dismissed };
});
}
src/web/routes/session-routes.ts
+1 -66
View File
@@ -89,6 +89,7 @@ import {
} from '../route-helpers.js';
import { buildAgentCaseMarker, writeAgentCaseMarker } from '../../agent-case-marker.js';
import { canUsernameRunPrivilegedCommands, resolveClaudeModeForUsername } from '../../user-store.js';
import { clampEnvOverridesForOwner } from '../../session-env-clamp.js';
import { enabledClis, getCli } from '../../config/cli-registry/registry.js';
import { resolveCliLaunchError } from '../../utils/cli-launcher.js';
import { legacyConfigForMode } from '../../session-cli-registry-bridge.js';
@@ -442,72 +443,6 @@ export async function _clampExternalCliBypassForOwner(
};
}
/**
* Env-var keys a non-granted owner must not be able to set, because each one
* hands back privilege the config clamp above just removed, or redirects a
* credential-resolution endpoint.
*
* The DeepSeek three are reachable because `DSH_*` and `DEEPSEEK_*` are
* allowlisted `envOverrides` prefixes (schemas.ts) — which they have to be, since
* that is also how a user configures the harness's non-privileged knobs.
*
* - `DSH_PERMISSION_MODE` IS the harness's permission switch. Every other CLI's
* bypass is a command-line FLAG, reachable only through the per-CLI config the
* clamp already owns; this one is an env var, so the config clamp alone is
* half a gate.
* - `DSH_HOME` points the launcher at a profile tree, and a profile's plugin code
* executes at BOOT, before any approval row can apply. A user who can write a
* workspace can put a profile in it, so this is the wider of the two.
* - `DEEPSEEK_BASE_URL` aims the provider endpoint, and `_configureCliEnv()`
* forwards the SERVER's own `DEEPSEEK_API_KEY` into every dsh pane before
* `applyEnvOverrides()` runs — so a non-granted owner who could set the base
* URL would have the operator's API key sent as a bearer credential to a host
* of their choosing. (`DEEPSEEK_API_KEY` itself stays overridable: supplying
* your OWN key removes privilege rather than granting it.)
* - `OMP_AUTH_BROKER_URL`/`OMP_AUTH_BROKER_TOKEN` are where omp resolves
* credentials from — the same shape as `DEEPSEEK_BASE_URL` above, reachable
* because `OMP_*` is an allowlisted prefix. Unlike DeepSeek, Codeman does not
* forward any operator-held key into an omp pane today (omp's provider
* credentials live in `~/.omp` config files, not env vars), so there is no
* known concrete exfiltration path yet — clamped defensively anyway, since a
* non-granted owner redirecting where a shared multi-tenant deployment
* resolves auth from is not something to allow silently (found in
* Ark0N/Codeman#353 review; omp's own knobs are otherwise mostly `PI_*`,
* already allowlisted for pi and not addressed here — see resolveOmpHome()).
*/
function ownerClampedEnvKeys(): string[] {
return enabledClis().flatMap((entry) => entry.capabilities.privilegedEnvKeys);
}
/**
* Env-var half of the multi-user bypass clamp.
*
* `clampExternalCliBypassForOwner()` clamps the per-CLI CONFIG, and for every CLI
* but DeepSeek that is the whole story. Here it is not: `applyEnvOverrides()` runs
* AFTER `_configureCliEnv()` in tmux-manager, so an override sent on the SAME
* request lands last and wins, and a non-granted owner could restore
* `danger-full-access` on the very request the config clamp downgraded.
*
* Keys are DROPPED rather than rewritten: dropping falls through to what
* `_configureCliEnv()` exports, which is the clamped config and the server's own
* `DSH_HOME`, i.e. exactly the intended state. No-op in single-user mode and for a
* granted owner, like every other clamp here
* (`canUsernameRunPrivilegedCommands()` returns true when `!isMultiUserMode()`),
* and it returns the caller's own object untouched when there is nothing to strip.
*/
async function clampEnvOverridesForOwner(
owner: string | undefined,
envOverrides: Record<string, string> | undefined
): Promise<Record<string, string> | undefined> {
if (!envOverrides) return envOverrides;
const keys = ownerClampedEnvKeys();
if (!keys.some((key) => key in envOverrides)) return envOverrides;
if (await canUsernameRunPrivilegedCommands(owner)) return envOverrides;
const clamped = { ...envOverrides };
for (const key of keys) delete clamped[key];
return clamped;
}
/** Test hook: the env-var half of the same multi-user safety gate. */
export const _clampEnvOverridesForOwner = clampEnvOverridesForOwner;
src/web/schemas.ts
+14
View File
@@ -1161,6 +1161,20 @@ const NotificationEventSchema = z
})
.optional();
/**
* Body of `POST /api/reboot-restore/restore`.
*
* `sessionIds` restores a subset, and omitting it restores everything the caller
* can see. The ids are session ids from `GET /api/reboot-restore`, and an id the
* caller does not own is ignored rather than refused, matching how the session
* list scopes rather than 403s.
*/
export const RebootRestoreRequestSchema = z
.object({
sessionIds: z.array(z.string().max(128)).max(200).optional(),
})
.strict();
export const SettingsUpdateSchema = z
.object({
// User-facing product branding. This changes browser/UI copy only; package,
src/web/server.ts
+229 -1
View File
@@ -39,7 +39,9 @@ import { fileURLToPath } from 'node:url';
import { existsSync, mkdirSync, readFileSync, chmodSync, rmSync, statSync } from 'node:fs';
import fs from 'node:fs/promises';
import { execSync } from 'node:child_process';
import { hostname as getHostname } from 'node:os';
import { hostname as getHostname, uptime as osUptime } from 'node:os';
import { looksLikeHostReboot, newestPersistedActivity, planRebootRestore } from '../reboot-restore.js';
import { rebootRestoreRegistry } from './reboot-restore-registry.js';
import { dataPath, getDataDir, CODEMAN_INSTANCE } from '../config/instance.js';
import { normalizeBasePath, stripBasePath, joinBasePath } from '../config/base-path.js';
import { GLYPH, palette } from '../cli-style.js';
@@ -171,6 +173,7 @@ import {
registerScheduledRoutes,
registerHookEventRoutes,
registerApprovalRoutes,
registerRebootRestoreRoutes,
registerReadMyMindRoutes,
registerStatusTelemetryRoutes,
registerSystemRoutes,
@@ -668,6 +671,8 @@ export class WebServer extends EventEmitter {
setupSessionListeners: this.setupSessionListeners.bind(this),
persistSessionState: this.persistSessionState.bind(this),
persistSessionStateNow: this._persistSessionStateNow.bind(this),
reapplyPersistedSessionState: this.reapplyPersistedSessionState.bind(this),
discardPartiallyBuiltSession: this.discardPartiallyBuiltSession.bind(this),
getSessionStateWithRespawn: this.getSessionStateWithRespawn.bind(this),
// EventPort
broadcast: this.broadcast.bind(this),
@@ -1060,6 +1065,7 @@ export class WebServer extends EventEmitter {
registerScheduledRoutes(this.app, ctx);
registerHookEventRoutes(this.app, ctx);
registerApprovalRoutes(this.app, ctx);
registerRebootRestoreRoutes(this.app, ctx);
registerReadMyMindRoutes(this.app, ctx);
registerStatusTelemetryRoutes(this.app, ctx);
registerSystemRoutes(this.app, ctx);
@@ -2857,6 +2863,212 @@ export class WebServer extends EventEmitter {
return false;
}
/**
* Work out what a host reboot destroyed, and leave it on offer for the board.
*
* Runs inside `restoreMuxSessions()`, in the window after `reconcileSessions()`
* has reported the dead sessions and before `finalizeRestoredState()` prunes
* their records, so `state.json` is still the full picture here. That window is
* the only place the plan can be built, which is why the boot pass builds it
* even though nothing is rebuilt until a user clicks.
*
* Nothing is created here. The plan goes to `rebootRestoreRegistry`, the board
* offers it as a banner, and `web/routes/reboot-restore-routes` rebuilds what
* the user asks for. A wrong reboot guess therefore costs a line of text the
* user dismisses, not N CLI processes nobody asked for.
*
* @returns how many sessions are on offer.
*/
private planRebootRestoreOffer(dead: string[], livePaneCount: number): number {
if (dead.length === 0) return 0;
const persisted = this.store.getSessions();
if (
!looksLikeHostReboot({
livePaneCount,
deadSessionCount: dead.length,
uptimeSeconds: osUptime(),
newestPersistedActivityAt: newestPersistedActivity(persisted),
now: Date.now(),
})
) {
return 0;
}
const { restore, skipped } = planRebootRestore(dead, persisted, (workingDir) => existsSync(workingDir));
if (skipped.length > 0) {
console.log(`[Server] Reboot restore is passing over ${skipped.length} dead session(s):`);
for (const rejection of skipped) {
console.log(`[Server] ${rejection.sessionId}: ${rejection.reason}`);
}
}
rebootRestoreRegistry.set(restore);
if (restore.length > 0) {
console.log(`[Server] Host reboot detected; offering ${restore.length} session(s) for restore`);
}
return restore.length;
}
/**
* Re-apply the persisted state that a `Session` constructor does not take.
*
* The reboot-restore route builds a session from a record rather than
* attaching to a surviving pane, so everything the constructor has no
* parameter for starts at its default. Persisting such a session writes
* `toState()` wholesale, which would REPLACE the record with the reduced
* version — and for a pinned session that is worse than losing a setting,
* because `cleanupSessionsByIds()` keeps a record only while it is pinned, so
* dropping the pin hands the record to the next stale sweep.
*
* Split in two phases because the two halves have opposite timing needs:
*
* - `before-spawn` shapes the pane itself, so it has to land before the CLI
* process starts, and before `setupSessionListeners()`, which reads the
* image-watcher flag. The custom-model selection is an environment injection
* and the nice priority is applied to the spawn.
* - `after-spawn` is the session's own accumulated history. It must NOT land
* on a session whose pane failed to start: the totals would then belong to a
* session that never ran, and any later cleanup would add them to the
* lifetime figures a second time.
*
* Respawn and Ralph are deliberately NOT re-armed: a machine that just came up
* is the worst moment to turn an autonomous run loose, and the user re-arms
* what they want. Ralph's loop CONFIGURATION does not survive either, because
* `toState()` reads `ralphEnabled` and the completion phrase off a live
* tracker, and there is no way to hold them without arming the loop.
*/
async reapplyPersistedSessionState(
session: Session,
saved: SessionState,
phase: 'before-spawn' | 'after-spawn'
): Promise<void> {
if (phase === 'before-spawn') {
// The custom-model env has to be rebuilt from the endpoint store: the persist
// deliberately keeps the injected VALUES out of state.json, so only the
// bookkeeping survives a restart and the values are re-derived here.
const savedCustomModel = (saved as { __customModel?: CustomModelBookkeeping }).__customModel;
if (savedCustomModel) {
session.setCustomModel(savedCustomModel, await this._rebuildCustomModelEnv(session, savedCustomModel));
}
if (saved.niceEnabled !== undefined || saved.niceValue !== undefined) {
session.setNice({ enabled: saved.niceEnabled, niceValue: saved.niceValue });
}
// `setupSessionListeners()` READS this flag to decide whether to start the
// watcher, so setting it later would leave the session reporting the feature
// as on with nothing watching.
if (saved.imageWatcherEnabled !== undefined) session.imageWatcherEnabled = saved.imageWatcherEnabled;
return;
}
if (saved.pinned) session.restorePin(true, saved.pinnedAt);
if (saved.autoCompactEnabled !== undefined || saved.autoCompactThreshold !== undefined) {
session.setAutoCompact(saved.autoCompactEnabled ?? false, saved.autoCompactThreshold, saved.autoCompactPrompt);
}
if (saved.autoClearEnabled !== undefined || saved.autoClearThreshold !== undefined) {
session.setAutoClear(saved.autoClearEnabled ?? false, saved.autoClearThreshold);
}
if (saved.autoResumeEnabled) {
session.restoreAutoResume(true, saved.autoResumeAt);
}
if (saved.inputTokens !== undefined || saved.outputTokens !== undefined || saved.totalCost !== undefined) {
session.restoreTokens(saved.inputTokens ?? 0, saved.outputTokens ?? 0, saved.totalCost ?? 0);
// Seed the daily-usage baseline, or the restored totals are counted again as new usage.
this.lastRecordedTokens.set(session.id, {
input: saved.inputTokens ?? 0,
output: saved.outputTokens ?? 0,
});
}
if (saved.color) session.setColor(saved.color);
if (saved.flickerFilterEnabled !== undefined) session.flickerFilterEnabled = saved.flickerFilterEnabled;
}
/**
* Undo a session that was registered but never got a working pane.
*
* Deliberately NOT `cleanupSession()`, which is the user-initiated delete: that
* path adds the session's token totals to the lifetime figures, demotes a
* pinned record to `stopped` (the durable marker of an intentional kill, which
* would make the session permanently ineligible for a reboot restore), drops
* the persisted Ralph state, and recursively removes `.claude-images` from the
* WORKING DIRECTORY, which belongs to the workspace rather than to this session
* and may hold another live session's pasted images.
*
* Everything else `_doCleanupSession()` does, this has to do as well. It is the
* inverse of `registerSessionWithLayout()` plus `setupSessionListeners()`, and
* every registration those two make has to come back out — above all
* `sessionListenerRefs`, whose presence makes `setupSessionListeners()` return
* early. Leaving that entry behind is worse than the leak this function exists
* to prevent: the retry reuses the same session id, wires no listeners at all,
* and the user gets a tab that never shows output.
*
* The persisted record, the lifetime totals, the stored Ralph state and the
* workspace's own files are left exactly as they were, so the session stays
* restorable on the next attempt.
*/
async discardPartiallyBuiltSession(sessionId: string): Promise<void> {
const session = this.sessions.get(sessionId);
if (!session) return;
this.sessions.delete(sessionId);
// --- the inverse of setupSessionListeners(), in reverse order ---
// Listeners first: while they are attached, one of them can still reach a
// tracker this is about to stop.
const listeners = this.sessionListenerRefs.get(sessionId);
if (listeners) {
detachSessionListeners(session, listeners);
this.sessionListenerRefs.delete(sessionId);
}
// An FSWatcher on the workspace that nothing else closes.
imageWatcher.unwatchSession(sessionId);
// An fs.watch on the workspace (or on @fix_plan.md), likewise.
session.ralphTracker.stopWatchingFixPlan();
const summaryTracker = this.runSummaryTrackers.get(sessionId);
if (summaryTracker) {
summaryTracker.stop();
this.runSummaryTrackers.delete(sessionId);
}
// --- what anything else may have attached to this id in the meantime ---
// A rebuild can fail AFTER startInteractive() resolved, and a restored
// workspace still carries Codeman's hooks, so the CLI can post a hook event
// within milliseconds. Each of these outlives the listeners and would
// otherwise meet the retry, which reuses the same session id by design.
this.stopTranscriptWatcher(sessionId);
attachmentRegistry.clearSession(sessionId);
sessionWaits.notifySignal(sessionId, 'exit');
sessionWaits.cancelAll(sessionId);
approvalInbox.resolveForSession(sessionId, 'session_ended');
// --- the inverse of the construction itself ---
this.sse.cleanupSessionBatches(sessionId);
this.persistDeb.cancelKey(sessionId);
fileStreamManager.closeSessionStreams(sessionId);
// `lastRecordedTokens` is deliberately NOT deleted: the `after-spawn` phase
// seeds it as the daily-usage baseline for these restored totals, and the
// retry reuses the id, so dropping it would count them as new usage.
// The per-session custom-model config dir carries the endpoint's API key, and
// `before-spawn` may already have written it. Nothing else would ever remove
// it: the stale sweep only touches state.json. A retry rewrites it.
removeConfigDir(customModelConfigDir(sessionId));
try {
session.removeAllListeners();
await session.stop(true);
} catch (err) {
console.warn(`[Server] stopping a partially built session failed: ${getErrorMessage(err)}`);
// `stop()` kills the mux session in its last block, after destroying its
// trackers, so a throw on the way there leaves the pane running.
await this.mux.killSession(sessionId).catch(() => {});
}
try {
await this.tabLayouts.sessionsRemoved([{ id: sessionId, owner: session.owner }]);
} catch (err) {
console.warn(`[Server] releasing the tab layout slot failed: ${getErrorMessage(err)}`);
}
// Any `session:updated` the half-built session emitted before it failed left a
// tab on every other open board, and the client's handler is an upsert.
this.broadcast(SseEvent.SessionDeleted, { id: sessionId });
}
private async restoreMuxSessions(): Promise<boolean> {
try {
// Reconcile mux sessions to find which ones are still alive (also discovers unknown ones)
@@ -2866,6 +3078,22 @@ export class WebServer extends EventEmitter {
console.log(`[Server] Discovered ${discovered.length} unknown mux session(s)`);
}
// Build the reboot-restore offer HERE: `dead` is only known after
// reconciliation, and the records it reads are pruned by
// `cleanupStaleSessions()` as soon as `finalizeRestoredState()` runs.
//
// Guarded on its own, because this runs inside the try that decides whether
// RECOVERY succeeded. A throw here would otherwise be caught below, report
// restoration as failed, and block the stale cleanup and layout
// reconciliation that follow — turning an optional convenience into a
// failure of the thing it is supposed to help. An offer nobody gets is the
// correct way for this to fail.
try {
this.planRebootRestoreOffer(dead, alive.length);
} catch (err) {
console.error('[Server] Building the reboot-restore offer failed; continuing recovery:', err);
}
if (alive.length > 0 || discovered.length > 0) {
console.log(`[Server] Found ${alive.length + discovered.length} alive mux session(s) from previous run`);
test/discard-partially-built-session.test.ts
+131
View File
@@ -0,0 +1,131 @@
/**
* `WebServer.discardPartiallyBuiltSession()` against the real server object.
*
* The reboot-restore route calls this when a rebuild registers a session and
* then fails to start its pane. It has to be the exact inverse of
* `registerSessionWithLayout()` plus `setupSessionListeners()`, and it must NOT
* be the user-initiated delete: banking the session's token totals, demoting a
* pinned record or deleting the workspace's files would all be wrong for a
* session that never ran.
*
* These tests drive the real method rather than the route, because the route
* tests run against a mock context whose `discardPartiallyBuiltSession` is a
* one-line stub — an earlier version of this function left four registrations
* behind and every route test still passed.
*
* The retry assertion is the important one. `setupSessionListeners()` returns
* early when `sessionListenerRefs` still holds the session id, so a discard that
* leaves that entry makes the next attempt wire nothing at all, and the user
* gets a tab that never shows output.
*/
import { mkdirSync } from 'node:fs';
import { homedir } from 'node:os';
import { join } from 'node:path';
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest';
import { safeRmHomeTree } from './mocks/test-helpers.js';
import { WebServer } from '../src/web/server.js';
import { Session } from '../src/session.js';
import { TmuxManager } from '../src/tmux-manager.js';
/** Reach the private collections the discard is responsible for emptying. */
interface ServerInternals {
sessions: Map<string, Session>;
sessionListenerRefs: Map<string, unknown>;
runSummaryTrackers: Map<string, unknown>;
registerSessionWithLayout(session: Session): Promise<void>;
setupSessionListeners(session: Session): Promise<void>;
discardPartiallyBuiltSession(sessionId: string): Promise<void>;
}
const WORKSPACE = join(homedir(), '.codeman-test-discard');
const SESSION_ID = 'a1b2c3d4e5f60718';
let server: WebServer;
let internals: ServerInternals;
let mux: TmuxManager;
function buildSession(): Session {
return new Session({
id: SESSION_ID,
workingDir: WORKSPACE,
mode: 'claude',
name: 'rebuilt session',
mux,
useMux: true,
});
}
beforeAll(() => {
mkdirSync(WORKSPACE, { recursive: true });
// Test mode: no port is opened and no CLI is launched. One server for the file,
// stopped at the end: the constructor registers handlers on the module-level
// image, subagent, team and workflow watchers, and only stop() removes them.
server = new WebServer(0, false, true);
internals = server as unknown as ServerInternals;
mux = new TmuxManager();
});
afterEach(async () => {
await internals.discardPartiallyBuiltSession(SESSION_ID).catch(() => {});
});
afterAll(async () => {
await server.stop().catch(() => {});
safeRmHomeTree(WORKSPACE);
});
describe('discarding a session whose pane never started', () => {
it('takes the session back out of the server', async () => {
const session = buildSession();
await internals.registerSessionWithLayout(session);
await internals.setupSessionListeners(session);
expect(internals.sessions.has(SESSION_ID)).toBe(true);
await internals.discardPartiallyBuiltSession(SESSION_ID);
expect(internals.sessions.has(SESSION_ID)).toBe(false);
});
it('releases the listener registration, so a retry can wire itself again', async () => {
const first = buildSession();
await internals.registerSessionWithLayout(first);
await internals.setupSessionListeners(first);
expect(internals.sessionListenerRefs.has(SESSION_ID)).toBe(true);
const firstRefs = internals.sessionListenerRefs.get(SESSION_ID);
await internals.discardPartiallyBuiltSession(SESSION_ID);
expect(internals.sessionListenerRefs.has(SESSION_ID)).toBe(false);
// The retry reuses the id by design. `setupSessionListeners()` returns early
// while the refs are still there, so a session built now would run blind: no
// terminal output, no status updates, no exit broadcast. Asserting a DIFFERENT
// refs object is what distinguishes wiring the retry from finding the corpse
// of the first attempt still in place.
const retry = buildSession();
await internals.registerSessionWithLayout(retry);
await internals.setupSessionListeners(retry);
const retryRefs = internals.sessionListenerRefs.get(SESSION_ID);
expect(retryRefs).toBeDefined();
expect(retryRefs).not.toBe(firstRefs);
});
it('stops the run-summary tracker, whose interval would otherwise keep firing', async () => {
const session = buildSession();
await internals.registerSessionWithLayout(session);
await internals.setupSessionListeners(session);
const tracker = internals.runSummaryTrackers.get(SESSION_ID) as { stop: () => void };
expect(tracker).toBeDefined();
// Dropping the map entry is not enough: the tracker arms a setInterval in its
// constructor, and only stop() clears it, so a discard that merely forgot the
// entry would leave the timer running for the life of the process.
const stopped = vi.spyOn(tracker, 'stop');
await internals.discardPartiallyBuiltSession(SESSION_ID);
expect(stopped).toHaveBeenCalled();
expect(internals.runSummaryTrackers.has(SESSION_ID)).toBe(false);
});
it('does nothing at all for a session it never registered', async () => {
await expect(internals.discardPartiallyBuiltSession('never-existed')).resolves.toBeUndefined();
});
});
test/mocks/mock-route-context-completeness.test.ts
+28
View File
@@ -0,0 +1,28 @@
/**
* The mock route context must offer everything the real one does.
*
* Route tests pass their context as `ctx as never`, and `tsconfig.json` includes
* only `src/**`, so no type check ever compares the mock against the ports. A
* port that gained a method left this mock missing it twice; both times the
* route under test threw a TypeError inside its own catch, and the suite
* reported a plausible-looking failure for an unrelated reason.
*
* So the comparison is made at runtime, against `WebServer.createRouteContext()`
* rather than against the port types, which is what keeps it from drifting: the
* server's own context object is the thing route modules are really given.
*/
import { describe, expect, it } from 'vitest';
import { WebServer } from '../../src/web/server.js';
import { createMockRouteContext } from './mock-route-context.js';
describe('the mock route context', () => {
it('offers every member the real route context does', () => {
const server = new WebServer(0, false, true);
const real = (server as unknown as { createRouteContext(): Record<string, unknown> }).createRouteContext();
const mock = createMockRouteContext() as unknown as Record<string, unknown>;
const missing = Object.keys(real).filter((key) => !(key in mock));
expect(missing, `mock-route-context.ts is missing: ${missing.join(', ')}`).toEqual([]);
});
});
test/mocks/mock-route-context.ts
+5
View File
@@ -61,6 +61,10 @@ export function createMockRouteContext(options?: {
setupSessionListeners: vi.fn(async () => {}),
persistSessionState: vi.fn(),
persistSessionStateNow: vi.fn(),
reapplyPersistedSessionState: vi.fn(async () => {}),
discardPartiallyBuiltSession: vi.fn(async (id: string) => {
sessions.delete(id);
}),
getSessionStateWithRespawn: vi.fn((s: MockSession) => s.toState()),
// -- EventPort --
@@ -149,6 +153,7 @@ export function createMockRouteContext(options?: {
clearRespawnConfig: vi.fn(),
updateRespawnConfig: vi.fn(),
setHistoryLimit: vi.fn(async () => {}),
startStatsCollection: vi.fn(),
},
runSummaryTrackers: new Map(),
activePlanOrchestrators: new Map(),
test/reboot-restore.test.ts
+395
View File
@@ -0,0 +1,395 @@
/**
* @fileoverview The decision half of reboot restore, and proof that the existing
* recovery construction path can CREATE a resumed pane.
*
* Three things are under test. `src/reboot-restore.ts` decides whether the
* machine rebooted and which dead sessions may be offered back. The plan
* registry in `src/web/reboot-restore-registry.ts` holds that offer between the
* boot that builds it and the click that spends it. The third is the claim the
* whole feature rests on: a `Session` built the way `restoreMuxSessions()`
* already builds one, but given no `muxSession` and a `resumeSessionId`, creates
* a fresh pane that resumes the old conversation. If that holds, the restore
* needs no new session-creation service.
*
* `reconcileSessions()` reports every session ALIVE under vitest, so the
* server's own boot pass cannot be reached from here. The decision logic is
* therefore driven directly, and the construction claim is driven through a real
* `Session` against the in-memory tmux layer vitest substitutes.
*/
import { mkdirSync, rmSync } from 'node:fs';
import { homedir } from 'node:os';
import { join } from 'node:path';
import { afterEach, describe, expect, it } from 'vitest';
import { Session } from '../src/session.js';
import { TmuxManager } from '../src/tmux-manager.js';
import type { SessionState } from '../src/types.js';
import {
looksLikeHostReboot,
newestPersistedActivity,
planRebootRestore,
rejectAlreadyLive,
resolveResumeConversationId,
type RebootRestoreEntry,
} from '../src/reboot-restore.js';
import { RebootRestoreRegistry } from '../src/web/reboot-restore-registry.js';
const HOUR = 60 * 60 * 1000;
const NOW = 1_760_000_000_000;
function persistedSession(overrides: Partial<SessionState> & { id: string }): SessionState {
return {
// A live agent's record carries its process id; `/exit` persists null instead.
pid: 99999,
status: 'idle',
workingDir: '/tmp/spike',
currentTaskId: null,
createdAt: NOW - 4 * HOUR,
lastActivityAt: NOW - 2 * HOUR,
mode: 'claude',
...overrides,
} as SessionState;
}
describe('reboot detection', () => {
const base = {
livePaneCount: 0,
deadSessionCount: 2,
// The host came up 10 minutes ago, well after the sessions were last active.
uptimeSeconds: 600,
newestPersistedActivityAt: NOW - 2 * HOUR,
now: NOW,
};
it('calls it a reboot when the socket is empty and the host booted after the last activity', () => {
expect(looksLikeHostReboot(base)).toBe(true);
});
it('refuses when some panes survived, which is an ordinary server restart', () => {
expect(looksLikeHostReboot({ ...base, livePaneCount: 3 })).toBe(false);
});
it('refuses on a long-uptime host, where someone wiped the tmux socket by hand', () => {
// Up for 30 days: the sessions were active long AFTER this boot, so the panes
// went away for some reason other than the machine restarting.
expect(looksLikeHostReboot({ ...base, uptimeSeconds: 30 * 24 * 60 * 60 })).toBe(false);
});
it('refuses when nothing died', () => {
expect(looksLikeHostReboot({ ...base, deadSessionCount: 0 })).toBe(false);
});
it('reads the newest activity stamp across the persisted records', () => {
const persisted = {
a: persistedSession({ id: 'a', lastActivityAt: NOW - 5 * HOUR }),
b: persistedSession({ id: 'b', lastActivityAt: NOW - 1 * HOUR }),
};
expect(newestPersistedActivity(persisted)).toBe(NOW - 1 * HOUR);
});
});
describe('which dead sessions may be rebuilt', () => {
it('rebuilds a session that was simply running when the power went out', () => {
const persisted = { live: persistedSession({ id: 'live', status: 'busy' }) };
const plan = planRebootRestore(['live'], persisted, () => true);
expect(plan.restore.map((s) => s.sessionId)).toEqual(['live']);
});
it('never revives a session the user killed while pinned (COD-142 demotes it to stopped)', () => {
const persisted = { killed: persistedSession({ id: 'killed', status: 'stopped', pinned: true }) };
const plan = planRebootRestore(['killed'], persisted, () => true);
expect(plan.restore).toEqual([]);
expect(plan.skipped).toEqual([{ sessionId: 'killed', reason: 'intentionally-ended' }]);
});
it('never revives a session whose record an unpinned kill already deleted', () => {
const plan = planRebootRestore(['gone'], {}, () => true);
expect(plan.restore).toEqual([]);
expect(plan.skipped).toEqual([{ sessionId: 'gone', reason: 'no-persisted-record' }]);
});
it('never revives a pane whose PTY-exit breaker had tripped', () => {
const persisted = { crashy: persistedSession({ id: 'crashy', respawnBlocked: true }) };
expect(planRebootRestore(['crashy'], persisted, () => true).skipped[0].reason).toBe('respawn-blocked');
});
it('leaves remote sessions to the COD-108 reconnect watcher', () => {
const persisted = {
r: persistedSession({
id: 'r',
remote: { hostId: 'h', host: 'example.test', username: 'u', sessionName: 'n', owned: true },
} as Partial<SessionState> & { id: string }),
};
expect(planRebootRestore(['r'], persisted, () => true).skipped[0].reason).toBe('remote-or-docker');
});
it('leaves docker sessions alone, since the container may not be up', () => {
const persisted = {
d: persistedSession({ id: 'd', docker: { containerId: 'abc', caseId: 'c' } } as Partial<SessionState> & {
id: string;
}),
};
expect(planRebootRestore(['d'], persisted, () => true).skipped[0].reason).toBe('remote-or-docker');
});
it('skips a CLI whose history the claude transcript reader does not understand', () => {
const persisted = { c: persistedSession({ id: 'c', mode: 'codex' }) };
expect(planRebootRestore(['c'], persisted, () => true).skipped[0].reason).toBe('unsupported-mode');
});
});
describe('a session with no attach process in its record', () => {
it('is refused, because there was nothing running to bring back', () => {
// A session that never started, or whose pane died outright. NOT a session
// the user ended with `/exit`: that keeps its pid, because the pid is the
// tmux attach process and `remain-on-exit` keeps the pane alive.
const persisted = { exited: persistedSession({ id: 'exited', status: 'idle', pid: null }) };
const plan = planRebootRestore(['exited'], persisted, () => true);
expect(plan.restore).toEqual([]);
expect(plan.skipped).toEqual([{ sessionId: 'exited', reason: 'not-running' }]);
});
it('still restores the session beside it that was attached when the power went', () => {
const persisted = {
exited: persistedSession({ id: 'exited', pid: null }),
running: persistedSession({ id: 'running', pid: 4242 }),
};
const plan = planRebootRestore(['exited', 'running'], persisted, () => true);
expect(plan.restore.map((entry) => entry.sessionId)).toEqual(['running']);
expect(plan.skipped.map((s) => s.reason)).toEqual(['not-running']);
});
it('refuses a record with no pid field at all', () => {
const persisted = { odd: persistedSession({ id: 'odd', pid: undefined as unknown as null }) };
expect(planRebootRestore(['odd'], persisted, () => true).skipped[0].reason).toBe('not-running');
});
});
describe('a workspace that is no longer on disk', () => {
it('is kept out of the offer, so a click cannot scaffold a deleted repo', () => {
const persisted = { gone: persistedSession({ id: 'gone', workingDir: '/tmp/deleted-repo' }) };
const plan = planRebootRestore(['gone'], persisted, () => false);
expect(plan.restore).toEqual([]);
expect(plan.skipped).toEqual([{ sessionId: 'gone', reason: 'workspace-missing' }]);
});
it('is judged per session, not for the batch', () => {
const persisted = {
kept: persistedSession({ id: 'kept', workingDir: '/tmp/still-here' }),
gone: persistedSession({ id: 'gone', workingDir: '/tmp/deleted-repo' }),
};
const plan = planRebootRestore(['kept', 'gone'], persisted, (dir) => dir === '/tmp/still-here');
expect(plan.restore.map((entry) => entry.sessionId)).toEqual(['kept']);
expect(plan.skipped.map((s) => s.reason)).toEqual(['workspace-missing']);
});
});
describe('a conversation that came back on its own before the click', () => {
const entry: RebootRestoreEntry = {
sessionId: 'abc',
workingDir: '/tmp/spike',
mode: 'claude',
resumeConversationId: 'conv-1',
state: persistedSession({ id: 'abc' }),
};
it('is skipped when the user resumed it by hand from the Resume list', () => {
// Same conversation, different session id: the Resume list creates a NEW id.
const result = rejectAlreadyLive([entry], new Set(['other']), new Set(['conv-1']));
expect(result.restore).toEqual([]);
expect(result.skipped).toEqual([{ sessionId: 'abc', reason: 'already-live' }]);
});
it('is skipped when a session with that id is already on the board', () => {
const result = rejectAlreadyLive([entry], new Set(['abc']), new Set());
expect(result.skipped).toEqual([{ sessionId: 'abc', reason: 'already-live' }]);
});
it('is rebuilt when neither its id nor its conversation is live', () => {
const result = rejectAlreadyLive([entry], new Set(['other']), new Set(['conv-other']));
expect(result.restore.map((e) => e.sessionId)).toEqual(['abc']);
expect(result.skipped).toEqual([]);
});
});
describe('the plan the banner spends', () => {
const all = () => true;
const entryFor = (sessionId: string, owner?: string): RebootRestoreEntry => ({
sessionId,
owner,
workingDir: '/tmp/spike',
mode: 'claude',
resumeConversationId: `conv-${sessionId}`,
state: persistedSession({ id: sessionId, owner }),
});
it('hands an entry to the first caller and nothing to the second', () => {
const registry = new RebootRestoreRegistry();
registry.set([entryFor('a'), entryFor('b')]);
expect(registry.take(all, undefined, undefined).map((e) => e.sessionId)).toEqual(['a', 'b']);
// The double-click: two panes on one conversation is what this prevents.
expect(registry.take(all, undefined, undefined)).toEqual([]);
});
it('spends only the ids a caller asked for', () => {
const registry = new RebootRestoreRegistry();
registry.set([entryFor('a'), entryFor('b')]);
expect(registry.take(all, ['b'], undefined).map((e) => e.sessionId)).toEqual(['b']);
expect(registry.list(all).map((e) => e.sessionId)).toEqual(['a']);
});
it("shows a user their own sessions and leaves another owner's alone", () => {
const registry = new RebootRestoreRegistry();
registry.set([entryFor('mine', 'alice'), entryFor('theirs', 'bob')]);
const asAlice = (owner: string | undefined) => owner === 'alice';
expect(registry.list(asAlice).map((e) => e.sessionId)).toEqual(['mine']);
expect(registry.take(asAlice, undefined, 'alice').map((e) => e.sessionId)).toEqual(['mine']);
// Bob's entry is still on offer for Bob.
expect(registry.list(() => true).map((e) => e.sessionId)).toEqual(['theirs']);
});
it('puts back an entry that no pane was created for', () => {
const registry = new RebootRestoreRegistry();
registry.set([entryFor('a')]);
const taken = registry.take(all, undefined, undefined);
registry.releaseFlight(undefined, taken);
expect(registry.list(all).map((e) => e.sessionId)).toEqual(['a']);
});
it('runs one restore at a time', () => {
const registry = new RebootRestoreRegistry();
expect(registry.beginSpending()).toBe(true);
expect(registry.beginSpending()).toBe(false);
registry.endSpending();
expect(registry.beginSpending()).toBe(true);
});
it('drops what a dismiss cleared', () => {
const registry = new RebootRestoreRegistry();
registry.set([entryFor('a'), entryFor('b')]);
expect(registry.clear(all)).toBe(2);
expect(registry.list(all)).toEqual([]);
});
it('forgets a plan nobody took for a day', () => {
const registry = new RebootRestoreRegistry();
registry.set([entryFor('a')]);
const dayLater = Date.now() + 25 * HOUR;
const realNow = Date.now;
Date.now = () => dayLater;
try {
expect(registry.list(all)).toEqual([]);
} finally {
Date.now = realNow;
}
});
});
describe('which conversation a rebuilt pane resumes', () => {
it('prefers the chain tail, the conversation the CLI reported last', () => {
const state = persistedSession({
id: 'sess-1',
resumeSessionId: 'launch-id',
claudeSessionChain: ['launch-id', 'after-clear'],
});
expect(resolveResumeConversationId(state)).toBe('after-clear');
});
it('falls back to the id the session originally resumed', () => {
const state = persistedSession({ id: 'sess-1', resumeSessionId: 'resumed-id' });
expect(resolveResumeConversationId(state)).toBe('resumed-id');
});
it('falls back to the session id, which is what Claude was launched with', () => {
expect(resolveResumeConversationId(persistedSession({ id: 'sess-1' }))).toBe('sess-1');
});
});
describe('the recovery construction path can create a resumed pane', () => {
const workingDir = join(homedir(), 'codeman-cases', 'reboot-restore-spike');
const sessions: Session[] = [];
afterEach(() => {
for (const s of sessions.splice(0)) s.stop();
rmSync(workingDir, { recursive: true, force: true });
});
/** Built exactly as the reboot pass builds one: no `muxSession`, plus a resume id. */
function rebuildFromPersistedState(state: SessionState, mux: TmuxManager): Session {
mkdirSync(workingDir, { recursive: true });
const session = new Session({
id: state.id,
workingDir,
mode: state.mode,
name: state.name,
createdAt: state.createdAt,
mux,
useMux: true,
resumeSessionId: resolveResumeConversationId(state),
owner: state.owner,
lastActivityAt: state.lastActivityAt,
claudeSessionChain: state.claudeSessionChain,
});
sessions.push(session);
return session;
}
it('creates a NEW mux session rather than needing one to attach to', async () => {
const mux = new TmuxManager();
const state = persistedSession({ id: 'aaaaaaa1-1111-4111-8111-111111111111', name: 'w1-spike' });
const session = rebuildFromPersistedState(state, mux);
expect(mux.getSessions()).toHaveLength(0);
await session.startInteractive();
const created = mux.getSessions();
expect(created).toHaveLength(1);
expect(created[0].sessionId).toBe('aaaaaaa1-1111-4111-8111-111111111111');
expect(created[0].workingDir).toBe(workingDir);
});
it('comes back pointed at the conversation the pane was holding', async () => {
const mux = new TmuxManager();
const state = persistedSession({
id: 'aaaaaaa2-2222-4222-8222-222222222222',
resumeSessionId: 'launch-id',
claudeSessionChain: ['launch-id', 'after-clear'],
});
const session = rebuildFromPersistedState(state, mux);
await session.startInteractive();
// The chain tail wins: a `/clear` before the reboot moved the CLI off the launch id.
expect(session.claudeSessionId).toBe('after-clear');
});
it('comes back idle, with no prompt sent and no autonomous loop armed', async () => {
const mux = new TmuxManager();
const state = persistedSession({
id: 'aaaaaaa3-3333-4333-8333-333333333333',
ralphEnabled: true,
respawnEnabled: true,
});
const session = rebuildFromPersistedState(state, mux);
await session.startInteractive();
// No prompt was queued: nothing is waiting on a task. The status itself is not
// assertable here, because the test PTY echoes and the activity detector reads
// that echo as work; in production the pane settles once the CLI finishes booting.
expect(session.currentTaskId).toBeNull();
// The pass never touches the tracker, so a persisted Ralph loop stays cold.
expect(session.ralphTracker.enabled).toBe(false);
});
it('keeps the owner it was persisted with, there being no request to read one from', async () => {
const mux = new TmuxManager();
const state = persistedSession({ id: 'aaaaaaa4-4444-4444-8444-444444444444', owner: 'alice' });
const session = rebuildFromPersistedState(state, mux);
await session.startInteractive();
expect(session.owner).toBe('alice');
expect(mux.getSessions()[0].owner).toBe('alice');
});
});
test/routes/reboot-restore-rebuild-failure.test.ts
+377
View File
@@ -0,0 +1,377 @@
/**
* Reboot-restore route: what happens when a rebuild gets part-way and then fails.
*
* The other route test file deliberately uses workspaces that do not exist, so it
* never reaches `new Session()`. This one mocks the `Session` module so the route
* runs its whole construction path — `addSession`, `setupSessionListeners`,
* `reapplyPersistedSessionState`, `startInteractive` — and then throws.
*
* The mock is the only way in. Driven against a real server, `startInteractive()`
* does not throw for either obvious cause: the CLI resolver finds its binary by
* absolute path rather than through PATH, and tmux falls back to another
* directory rather than failing when it cannot enter the workspace. A mux-layer
* failure is what is left, and it cannot be provoked from a test. Without the
* mock this path would go unexercised, which is how the original version of this
* route shipped a session leak the tests could not see.
*
* It also covers the session caps, because those too are only reachable once the
* route is actually willing to build something.
*/
import { describe, it, expect, afterEach, vi, beforeEach } from 'vitest';
import Fastify, { type FastifyInstance } from 'fastify';
import fastifyCookie from '@fastify/cookie';
/** Set per test: whether the mocked `startInteractive()` rejects. */
let startShouldThrow = false;
/** Ordering log, so a test can assert what ran before the pane spawned. */
const callOrder: string[] = [];
vi.mock('../../src/session.js', () => ({
Session: class {
id: string;
mode: string;
name?: string;
workingDir: string;
owner?: string;
claudeSessionId: string | null = null;
constructor(config: { id: string; mode?: string; name?: string; workingDir: string; owner?: string }) {
this.id = config.id;
this.mode = config.mode ?? 'claude';
this.name = config.name;
this.workingDir = config.workingDir;
this.owner = config.owner;
}
async startInteractive() {
callOrder.push('startInteractive');
if (startShouldThrow) throw new Error('spawn claude ENOENT');
}
/** The mock route context projects a session through this on broadcast. */
toState() {
return { id: this.id, mode: this.mode, name: this.name, workingDir: this.workingDir, owner: this.owner };
}
},
}));
const { registerRebootRestoreRoutes } = await import('../../src/web/routes/reboot-restore-routes.js');
const { rebootRestoreRegistry } = await import('../../src/web/reboot-restore-registry.js');
const { installRouteErrorHandler } = await import('../../src/web/route-error-handler.js');
const { httpStatusForErrorCode } = await import('../../src/types.js');
const { createMockRouteContext } = await import('../mocks/index.js');
type ApiErrorCode = import('../../src/types.js').ApiErrorCode;
type RebootRestoreEntry = import('../../src/reboot-restore.js').RebootRestoreEntry;
type SessionState = import('../../src/types.js').SessionState;
/** A real directory, so the route's workspace checks pass and it reaches the build. */
const WORKSPACE = process.cwd();
function offerEntry(sessionId: string, owner?: string): RebootRestoreEntry {
return {
sessionId,
name: `session ${sessionId}`,
workingDir: WORKSPACE,
owner,
mode: 'claude',
resumeConversationId: `conv-${sessionId}`,
state: {
id: sessionId,
pid: null,
status: 'idle',
workingDir: WORKSPACE,
currentTaskId: null,
createdAt: 1_760_000_000_000,
mode: 'claude',
owner,
} as SessionState,
};
}
async function createHarness(ctx: ReturnType<typeof createMockRouteContext>): Promise<FastifyInstance> {
const app = Fastify({ logger: false });
await app.register(fastifyCookie);
registerRebootRestoreRoutes(app, ctx as never);
app.addHook('preSerialization', (req, reply, payload: unknown, done) => {
if (!req.url.startsWith('/api')) return done(null, payload);
if (payload === null || typeof payload !== 'object') return done(null, payload);
const p = payload as { success?: unknown; errorCode?: unknown };
if (p.success === false) {
if (reply.statusCode === 200 && typeof p.errorCode === 'string') {
reply.code(httpStatusForErrorCode(p.errorCode as ApiErrorCode));
}
return done(null, payload);
}
if (p.success === true) return done(null, payload);
return done(null, { success: true, data: payload });
});
installRouteErrorHandler(app);
await app.ready();
return app;
}
beforeEach(() => {
startShouldThrow = false;
callOrder.length = 0;
});
afterEach(() => {
rebootRestoreRegistry.reset();
vi.clearAllMocks();
});
describe('a rebuild that fails after the session is registered', () => {
it('reports why it failed rather than blaming the workspace', async () => {
startShouldThrow = true;
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
const app = await createHarness(ctx);
const res = (await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} })).json().data;
expect(res.restored).toEqual([]);
// Not `workspace-missing`: the directory is there, the agent would not start.
expect(res.skipped).toEqual([{ sessionId: 'a', reason: 'rebuild-failed' }]);
await app.close();
});
it('does not leave a registered session with no pane behind it', async () => {
startShouldThrow = true;
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
const app = await createHarness(ctx);
await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
// The session reached ctx.sessions via addSession; the route has to take it
// back out, or the board shows a tab whose pane never existed.
expect(ctx.discardPartiallyBuiltSession).toHaveBeenCalledWith('a');
expect(ctx.sessions.has('a')).toBe(false);
// NOT the user-initiated delete: that would bank this session's historical
// tokens into the lifetime totals, demote a pinned record to `stopped`, and
// delete the workspace's .claude-images.
expect(ctx.cleanupSession).not.toHaveBeenCalled();
await app.close();
});
it('keeps the entry on offer, so the user can fix the PATH and click again', async () => {
startShouldThrow = true;
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
const app = await createHarness(ctx);
await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
const left = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions.map((s: { id: string }) => s.id)).toEqual(['a']);
await app.close();
});
});
describe('a rebuild that succeeds', () => {
it('re-applies the persisted state before the record is written again', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
const app = await createHarness(ctx);
const res = (await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} })).json().data;
expect(res.restored.map((s: { id: string }) => s.id)).toEqual(['a']);
// A session built from a record carries none of the pin, token totals or
// custom-model selection, so persisting it first would replace the fuller
// record with the reduced one.
expect(ctx.reapplyPersistedSessionState).toHaveBeenCalled();
const reapplyOrder = (ctx.reapplyPersistedSessionState as ReturnType<typeof vi.fn>).mock.invocationCallOrder[0];
const persistOrder = (ctx.persistSessionState as ReturnType<typeof vi.fn>).mock.invocationCallOrder[0];
expect(reapplyOrder).toBeLessThan(persistOrder);
await app.close();
});
it('shapes the pane before it spawns, and restores the history after', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
(ctx.reapplyPersistedSessionState as ReturnType<typeof vi.fn>).mockImplementation(
async (_s: unknown, _saved: unknown, phase: string) => {
callOrder.push(`reapply:${phase}`);
}
);
(ctx.setupSessionListeners as ReturnType<typeof vi.fn>).mockImplementation(async () => {
callOrder.push('setupSessionListeners');
});
const app = await createHarness(ctx);
await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
// `setupSessionListeners()` READS the image-watcher flag that `before-spawn`
// restores, so the phase has to precede it or the session comes back
// reporting the watcher as on with nothing watching. The custom-model
// environment has to reach the process, and the token totals must not land
// on a session whose pane never started.
expect(callOrder).toEqual([
'reapply:before-spawn',
'setupSessionListeners',
'startInteractive',
'reapply:after-spawn',
]);
await app.close();
});
it('tells every other board about the rebuilt session', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
const app = await createHarness(ctx);
await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
expect(ctx.broadcast).toHaveBeenCalledWith('session:created', expect.anything());
await app.close();
});
it('spends the entry, so it is no longer on offer', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
const app = await createHarness(ctx);
await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
const left = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions).toEqual([]);
await app.close();
});
});
describe('the session caps', () => {
it('counts the sessions it is itself creating, not just the ones it started with', async () => {
rebootRestoreRegistry.set([offerEntry('a'), offerEntry('b')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
// One seat short of the documented maximum of 50, counting the session the
// mock context seeds. A check that ran once before the loop would restore
// BOTH entries; only a per-iteration check refuses the second.
for (let i = 0; i < 48; i += 1) {
ctx.sessions.set(`filler-${i}`, { id: `filler-${i}`, owner: undefined } as never);
}
const app = await createHarness(ctx);
const res = (await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} })).json().data;
expect(res.restored.map((s: { id: string }) => s.id)).toEqual(['a']);
expect(res.skipped).toEqual([{ sessionId: 'b', reason: 'capacity-reached' }]);
// Refused rather than lost: closing a session and clicking again works.
const left = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions.map((s: { id: string }) => s.id)).toEqual(['b']);
await app.close();
});
it('refuses every entry when the board is already at the cap', async () => {
rebootRestoreRegistry.set([offerEntry('a'), offerEntry('b')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
for (let i = 0; i < 50; i += 1) {
ctx.sessions.set(`filler-${i}`, { id: `filler-${i}`, owner: undefined } as never);
}
const app = await createHarness(ctx);
const res = (await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} })).json().data;
expect(res.restored).toEqual([]);
expect(res.skipped.map((s: { reason: string }) => s.reason)).toEqual(['capacity-reached', 'capacity-reached']);
await app.close();
});
});
describe('a failure before any entry is considered', () => {
it('returns the whole plan rather than spending it', async () => {
rebootRestoreRegistry.set([offerEntry('a'), offerEntry('b')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
(ctx.getWorkspaceHooksEnabled as ReturnType<typeof vi.fn>).mockRejectedValue(new Error('settings unreadable'));
const app = await createHarness(ctx);
const res = await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
expect(res.statusCode).toBeGreaterThanOrEqual(500);
// The plan cannot be rebuilt once boot has pruned the records, so a throw
// anywhere in the route has to hand the entries back.
const left = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions.map((s: { id: string }) => s.id).sort()).toEqual(['a', 'b']);
await app.close();
});
it('releases the single flight, so the next click is not refused', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
(ctx.getWorkspaceHooksEnabled as ReturnType<typeof vi.fn>).mockRejectedValue(new Error('settings unreadable'));
const app = await createHarness(ctx);
await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
expect(rebootRestoreRegistry.beginSpending(undefined)).toBe(true);
rebootRestoreRegistry.endSpending(undefined);
await app.close();
});
});
describe('a dismiss that lands while a restore is running', () => {
it('wins when an admin is restoring the entries and their owner dismisses', async () => {
const theirs = offerEntry('theirs', 'bob');
rebootRestoreRegistry.set([theirs]);
// An admin may spend another user's entries, so the caller doing the restore
// and the owner of what is being restored are different people.
const taken = rebootRestoreRegistry.take(() => true, undefined, 'admin');
expect(taken.map((e) => e.sessionId)).toEqual(['theirs']);
// Bob dismisses his own banner. Nothing of his is in the plan any more, and
// the restore is running under a different name than his.
rebootRestoreRegistry.clear((owner) => owner === 'bob');
rebootRestoreRegistry.releaseFlight('admin', taken);
expect(rebootRestoreRegistry.list(() => true)).toEqual([]);
});
it('wins when an admin dismisses everything mid-restore', async () => {
rebootRestoreRegistry.set([offerEntry('theirs', 'bob')]);
const taken = rebootRestoreRegistry.take(() => true, undefined, 'admin');
rebootRestoreRegistry.clear(() => true);
rebootRestoreRegistry.releaseFlight('admin', taken);
expect(rebootRestoreRegistry.list(() => true)).toEqual([]);
});
it('wins, rather than being undone when the route hands its entries back', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
const ctx = createMockRouteContext({ workspaceHooksEnabled: false });
// The user clicks Dismiss while the restore is between its take and its
// return. Driven through the ROUTE, so removing the generation argument from
// the route would make this fail.
(ctx.getWorkspaceHooksEnabled as ReturnType<typeof vi.fn>).mockImplementation(async () => {
rebootRestoreRegistry.clear(() => true);
throw new Error('settings unreadable');
});
const app = await createHarness(ctx);
await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
const left = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions).toEqual([]);
await app.close();
});
it('reaches an in-flight restore the dismisser can see, even once its entries are taken', async () => {
const mine = offerEntry('mine', 'alice');
rebootRestoreRegistry.set([mine]);
const taken = rebootRestoreRegistry.take((owner) => owner === 'alice', undefined, 'alice');
expect(taken).toHaveLength(1);
// The plan is empty now, so the dismiss has nothing of Alice's left in the
// plan; it has to reach the entry the restore is holding.
rebootRestoreRegistry.clear((owner) => owner === 'alice');
rebootRestoreRegistry.releaseFlight('alice', taken);
expect(rebootRestoreRegistry.list(() => true)).toEqual([]);
});
it('does not reach another owner, whose unspent entries still come back', async () => {
const mine = offerEntry('mine', 'alice');
const theirs = offerEntry('theirs', 'bob');
rebootRestoreRegistry.set([mine, theirs]);
// Bob is mid-restore, holding his own entry.
const bobsTaken = rebootRestoreRegistry.take((owner) => owner === 'bob', undefined, 'bob');
expect(bobsTaken.map((e) => e.sessionId)).toEqual(['theirs']);
// Alice dismisses her own banner meanwhile.
rebootRestoreRegistry.clear((owner) => owner === 'alice');
// Bob's restore finishes and hands his entry back. Alice's dismiss covered
// her entries, not his, so his offer survives.
rebootRestoreRegistry.releaseFlight('bob', bobsTaken);
expect(rebootRestoreRegistry.list(() => true).map((e) => e.sessionId)).toEqual(['theirs']);
});
});
test/routes/reboot-restore-routes.test.ts
+202
View File
@@ -0,0 +1,202 @@
/**
* Reboot-restore route tests (src/web/routes/reboot-restore-routes.ts) via
* app.inject(), no live port.
*
* Every entry these tests put on offer names a workspace that does not exist, so
* the route's click-time workspace check rejects it before any `Session` is
* constructed. That keeps the tests on the route's own guards — taking, scoping,
* single-flighting and re-checking — and leaves pane creation to
* test/reboot-restore.test.ts, which drives a real `Session` for it.
*
* The routes read the process-wide `rebootRestoreRegistry` singleton, so every
* test resets it; a leaked entry would bleed into the next one.
*/
import { describe, it, expect, afterEach } from 'vitest';
import Fastify, { type FastifyInstance } from 'fastify';
import fastifyCookie from '@fastify/cookie';
import { registerRebootRestoreRoutes } from '../../src/web/routes/reboot-restore-routes.js';
import { rebootRestoreRegistry } from '../../src/web/reboot-restore-registry.js';
import { installRouteErrorHandler } from '../../src/web/route-error-handler.js';
import { httpStatusForErrorCode, type ApiErrorCode } from '../../src/types.js';
import { createMockRouteContext } from '../mocks/index.js';
import type { RebootRestoreEntry } from '../../src/reboot-restore.js';
import type { SessionState } from '../../src/types.js';
async function createHarness(authUser?: { username: string; role: 'admin' | 'user' }): Promise<FastifyInstance> {
return createHarnessWithCtx(createMockRouteContext(), authUser);
}
async function createHarnessWithCtx(
ctx: ReturnType<typeof createMockRouteContext>,
authUser?: { username: string; role: 'admin' | 'user' }
): Promise<FastifyInstance> {
const app = Fastify({ logger: false });
await app.register(fastifyCookie);
if (authUser) {
app.addHook('onRequest', async (req) => {
(req as unknown as { authUser: typeof authUser }).authUser = authUser;
});
}
registerRebootRestoreRoutes(app, ctx as never);
app.addHook('preSerialization', (req, reply, payload: unknown, done) => {
if (!req.url.startsWith('/api')) return done(null, payload);
if (payload === null || typeof payload !== 'object') return done(null, payload);
const p = payload as { success?: unknown; errorCode?: unknown };
if (p.success === false) {
if (reply.statusCode === 200 && typeof p.errorCode === 'string') {
reply.code(httpStatusForErrorCode(p.errorCode as ApiErrorCode));
}
return done(null, payload);
}
if (p.success === true) return done(null, payload);
return done(null, { success: true, data: payload });
});
installRouteErrorHandler(app);
await app.ready();
return app;
}
/** An entry whose workspace is deliberately absent, so no pane is ever created. */
function offerEntry(sessionId: string, owner?: string): RebootRestoreEntry {
return {
sessionId,
name: `session ${sessionId}`,
workingDir: `/tmp/codeman-reboot-restore-missing/${sessionId}`,
owner,
mode: 'claude',
resumeConversationId: `conv-${sessionId}`,
state: {
id: sessionId,
pid: null,
status: 'idle',
workingDir: `/tmp/codeman-reboot-restore-missing/${sessionId}`,
currentTaskId: null,
createdAt: 1_760_000_000_000,
mode: 'claude',
owner,
} as SessionState,
};
}
afterEach(() => {
rebootRestoreRegistry.reset();
});
describe('GET /api/reboot-restore', () => {
it('reports nothing when no reboot left anything behind', async () => {
const app = await createHarness();
const res = await app.inject({ method: 'GET', url: '/api/reboot-restore' });
expect(res.statusCode).toBe(200);
expect(res.json().data.sessions).toEqual([]);
await app.close();
});
it('names what is on offer, and says the scrollback is not coming back', async () => {
rebootRestoreRegistry.set([offerEntry('a'), offerEntry('b')]);
const app = await createHarness();
const body = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(body.sessions.map((s: { id: string }) => s.id)).toEqual(['a', 'b']);
expect(body.scrollbackRestored).toBe(false);
await app.close();
});
it('never carries the persisted record itself to the browser', async () => {
rebootRestoreRegistry.set([offerEntry('a', 'alice')]);
const app = await createHarness({ username: 'alice', role: 'admin' });
const body = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(Object.keys(body.sessions[0]).sort()).toEqual(['id', 'mode', 'name', 'owner', 'workingDir']);
expect(body.sessions[0].state).toBeUndefined();
await app.close();
});
});
describe('POST /api/reboot-restore/restore', () => {
it('reports a workspace that is gone, and keeps offering it in case it comes back', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
const app = await createHarness();
const first = (await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} })).json().data;
expect(first.restored).toEqual([]);
expect(first.skipped).toEqual([{ sessionId: 'a', reason: 'workspace-missing' }]);
// Nothing was built, so the entry goes back: a repo can be restored from a
// backup between two clicks, and losing the offer would be unrecoverable.
const left = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions.map((s: { id: string }) => s.id)).toEqual(['a']);
await app.close();
});
it('never re-offers a conversation that is already open', async () => {
const entry = offerEntry('a');
rebootRestoreRegistry.set([entry]);
const app = await createHarness();
const ctx = createMockRouteContext({ sessionId: entry.sessionId });
// A session with that id is live, which is what the Resume list would produce.
const liveApp = await createHarnessWithCtx(ctx);
const res = (await liveApp.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} })).json().data;
expect(res.skipped).toEqual([{ sessionId: 'a', reason: 'already-live' }]);
// Unlike a missing workspace, this one is dropped: it cannot stop being true.
const left = (await liveApp.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions).toEqual([]);
await liveApp.close();
await app.close();
});
it('spends only the sessions the click named', async () => {
rebootRestoreRegistry.set([offerEntry('a'), offerEntry('b')]);
const app = await createHarness();
const res = await app.inject({
method: 'POST',
url: '/api/reboot-restore/restore',
payload: { sessionIds: ['b'] },
});
expect(res.json().data.skipped).toEqual([{ sessionId: 'b', reason: 'workspace-missing' }]);
// 'a' was never taken, and 'b' came back because no pane was built for it.
const left = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(left.sessions.map((s: { id: string }) => s.id).sort()).toEqual(['a', 'b']);
await app.close();
});
it('refuses a body it does not recognise rather than guessing', async () => {
const app = await createHarness();
const res = await app.inject({
method: 'POST',
url: '/api/reboot-restore/restore',
payload: { sessionIds: 'not-an-array' },
});
expect(res.statusCode).toBeGreaterThanOrEqual(400);
await app.close();
});
it('turns a second concurrent restore away rather than interleaving it', async () => {
rebootRestoreRegistry.set([offerEntry('a')]);
// Claimed by a restore already in flight for this same owner (undefined in
// single-user mode, which is what the harness runs as).
expect(rebootRestoreRegistry.beginSpending(undefined)).toBe(true);
const app = await createHarness();
const res = await app.inject({ method: 'POST', url: '/api/reboot-restore/restore', payload: {} });
expect(res.statusCode).toBe(409);
rebootRestoreRegistry.endSpending(undefined);
await app.close();
});
});
describe('POST /api/reboot-restore/dismiss', () => {
it('drops the offer and leaves the banner with nothing to show', async () => {
rebootRestoreRegistry.set([offerEntry('a'), offerEntry('b')]);
const app = await createHarness();
const res = await app.inject({ method: 'POST', url: '/api/reboot-restore/dismiss', payload: {} });
expect(res.json().data.dismissed).toBe(2);
const after = (await app.inject({ method: 'GET', url: '/api/reboot-restore' })).json().data;
expect(after.sessions).toEqual([]);
await app.close();
});
});