mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-02 21:49:42 +02:00
Merge pull request #156 from aakhter/cod-114-remote-tmux-durability
Remote tmux durability: survive SSH drop, discover/attach, collaborative sessions
This commit is contained in:
Ark0N
@@ -172,6 +172,8 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Cron (cron-style `CronJob`s)**: saved, named jobs with a recurring schedule (`once`/`interval`/`daily`/`weekly`), enable/disable, Run Now, next-run calc, and per-job run history (`CronJobRun`). ⚠️ **Distinct from the legacy `ScheduledRun`** (`/api/scheduled`, a run-now duration-bounded autonomous loop) — the two never interact; the legacy concept keeps the `Scheduled*` names, the recurring-job feature is `Cron*`. `CronService` (`src/cron/cron-service.ts`) owns CRUD + the 30s background due-tick (`tickDueJobs`, registered via `cleanup.setInterval` in `server.ts`; `init()` recomputes nextRunAt on boot) and **reuses the existing session layer** (create → `addSession` → `setupSessionListeners` → `startInteractive`/`startShell` → prompt via `writeViaMux`/`write`) rather than rebuilding tmux logic. Next-run math is pure/unit-tested in `cron-time.ts` (SERVER-LOCAL timezone for daily/weekly). Dup-launch guard = `lastDueKey` (jobId:fireTime); schedule is advanced BEFORE launch so a slow launch can't re-trigger. `once` jobs self-disable after firing (`completedOnce`). Persisted via `AppState.cronJobs`/`cronJobRuns` (StateStore accessors). Routes `/api/cron/jobs*` + `/api/cron/runs` (`cron-routes.ts`, `CronPort`); schema `CronJobSchema` (cross-field `superRefine`; the `.partial()` update schema does NOT re-run it); SSE `cron:*`. Frontend `cron-ui.js` (#cronModal). Claude/shell/opencode/codex/gemini agent types. Tests: `test/cron-time.test.ts`, `test/cron-service.test.ts`. Design: `docs/cron-discovery.md`.
|
||||
|
||||
**Remote sessions (SSH)**: Sessions can run the agent inside a durable `tmux -L codeman-remote new-session -A` **on a remote host** so it survives the SSH drop (COD-104), and can also **discover + attach** to `codeman-*` sessions another Codeman launched there — attached (`owned:false`) sessions **detach, never kill** on tab close (COD-105). **Shared/collaborative** (COD-106): remote set-options are scoped per-session (never `-g`) and `window-size latest` lets multiple clients attach the same session at different viewports without clamping to the smallest; a client count surfaces a "shared · N" badge. **Auto-reconnect** (COD-108): a bounded-backoff watcher re-establishes a dropped remote session's local ssh pane and reattaches the still-running durable remote tmux (kill-switch `remoteAutoReconnect`, default ON). Owned sessions propagate `kill-session` to the remote on close; non-owned never do. ⚠️ Command-injection surface (COD-107): all ssh command lines flow through the single shell-safe `buildSshConnectionArgs()` — every user field (`-J jumpHost`, `-i identity`, `-o`) is `shellescape`d; never hand-build an ssh line elsewhere. Full design: `docs/remote-sessions.md`.
|
||||
|
||||
**External CLI modes (OpenCode, Codex, Gemini)**: `isExternalCliMode()` in `session.ts` (`mode === 'opencode' || 'codex' || 'gemini'`) gates Claude-specific behavior — Ralph tracker, BashToolParser, token/CLI-info parsing, and ❯-prompt readiness detection are all skipped (these CLIs render their own TUIs; readiness = output stabilization instead). All three modes **require tmux — no direct PTY fallback** — because secrets are injected via `tmux setenv` (socket-scoped `${this.tmux()} setenv`, never on the spawn command line): OpenCode gets `OPENCODE_CONFIG_CONTENT` etc., Codex gets `OPENAI_API_KEY`/`CODEX_API_KEY`/`CODEX_HOME` (`setCodexEnvVars`), Gemini gets `GEMINI_API_KEY`/`GOOGLE_API_KEY`/`GOOGLE_CLOUD_PROJECT`/`GOOGLE_APPLICATION_CREDENTIALS`/`GOOGLE_GENAI_USE_VERTEXAI` etc. (`setGeminiEnvVars`, all in `tmux-manager.ts`). Codex specifics: command built by `buildCodexCommand()` (`--model`, `resume <id>`, `--dangerously-bypass-approvals-and-sandbox` from the `codexConfig` payload / `codexDangerouslyBypassApprovals` app setting; `renderMode` is schema-coerced to `'hybrid'`, the only supported mode). Gemini specifics: command built by `buildGeminiCommand()` (`--skip-trust` always, `--approval-mode <default|auto_edit|yolo|plan>` defaulting to `yolo` for parity with Claude's `--dangerously-skip-permissions`, `--model`, `--resume` from the `geminiConfig` payload); availability via `GET /api/gemini/status` — session/quick-start routes fail with `OPERATION_FAILED` + install hint (`npm install -g @google/gemini-cli`) when missing. Codex AND Gemini export `COLORTERM=truecolor` + unset `NO_COLOR` (other modes unset `COLORTERM`); Gemini joins `isAltScreenStripMode()` (Codex/Claude/Gemini are Ink TUIs that repaint inline → strip alt-screen/`3J` so scrollback survives). Codex availability via `GET /api/codex/status`. Frontend: run-mode dropdown → `runCodex()`/`runGemini()` in `session-ui.js` ("Run CX"/"Run GM" labels), App Settings → Codex CLI tab; Respawn/Ralph options are Claude-only, so session options open on the Summary tab for external CLI sessions. ⚠️ `run*()` MUST unwrap the `{success,data}` envelope (`(await res.json()).data.available` / `data.data.sessionId`) — reading the raw shape silently breaks the run. Tests: `test/run-mode-ui.test.ts` + `test/gemini-mode.test.ts` (vm-sandbox harness, no real DOM).
|
||||
|
||||
**Remote SSH cases** (COD-94/#145): cases can point at a **remote host** (`~/.codeman/remote-hosts.json` + `remote-cases.json` via `src/remote-hosts.ts`; CRUD under `/api/cases` — cases route file). A remote session launches a LOCAL tmux pane running `ssh <host>` that creates a durable REMOTE tmux session on a **dedicated socket** `-L codeman-remote` with name `codeman-ssh-<id>` — deliberately failing the remote Codeman's `SAFE_MUX_NAME_PATTERN` so a Codeman instance on the target host never adopts it; no `-g` global tmux options are set remotely. `remotePath`/`identityFile` are schema-guarded against shell injection (backticks/`$` rejected — same approach as `extraSshOptions`); remote tmux availability is probed via `checkRemoteTmuxAvailable()` in quick-start (ssh args carry `-o ConnectTimeout=10`). Remote claude defaults to `exec claude --dangerously-skip-permissions`; per-host `commands.*` override. Session kill best-effort kills the remote tmux too. `SessionState.remote`/`MuxSession.remote` round-trip through recovery (`restoreMuxSessions` passes `remote` back into the Session constructor). ⚠️ Run flows must route remote cases through `POST /api/quick-start` (which resolves the remote case and skips LOCAL CLI availability gates) — `POST /api/sessions` stat-validates `workingDir` locally and has no `caseName`. `envOverrides`/`effort`/`modelOverride`/`codexConfig`/`geminiConfig` are rejected for remote quick-starts (not silently dropped). UI: Create Case modal → Remote tab. Tests: `test/remote-hosts.test.ts`, `test/remote-ssh-options.test.ts`.
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
# Remote Sessions (SSH)
|
||||
|
||||
Codeman can run a session's agent on a **remote host over SSH** instead of the
|
||||
local machine. The agent (Claude, OpenCode, Codex, Gemini, or a plain shell)
|
||||
runs inside a `tmux` server **on the remote host**, so it survives the SSH
|
||||
connection dropping; Codeman attaches to it the same way it attaches to a local
|
||||
managed session.
|
||||
|
||||
This document covers the data model, the shell-safe SSH command construction
|
||||
(COD-107), the durable-launch design (COD-104), and the operational caveats.
|
||||
For the local session/mux machinery this builds on, see the **Mux** and
|
||||
**Session** entries in `CLAUDE.md` → Architecture.
|
||||
|
||||
## Why it exists
|
||||
|
||||
A developer box (`AA-DESKTOP`) often needs to drive an agent on another machine —
|
||||
a NAS, a build server, a host reachable only through a jump box or a
|
||||
cloudflared SOCKS5 proxy. Rather than wrap `ssh` by hand per host, Codeman
|
||||
stores reusable **remote hosts** + **remote cases** and reproduces the exact
|
||||
connection the operator already uses (`ssh-aa-desktop`-style configs:
|
||||
custom port, identity file, `-J` jump host, `-o ProxyCommand`).
|
||||
|
||||
## Data model
|
||||
|
||||
Types live in `src/types/session.ts`; persistence in `src/remote-hosts.ts`.
|
||||
|
||||
| Type | Role |
|
||||
|------|------|
|
||||
| `RemoteSshOptions` | The **HOW-to-reach** fields, shared by host + session: `identityFile`, `socksProxy` (`host:port`), `jumpHost` (`[user@]host[:port]`), `extraSshOptions` (`KEY=VALUE[]`). Every field optional — all-absent reproduces port-22, default-identity, directly-SSH-able behavior. |
|
||||
| `RemoteHost` (extends `RemoteSshOptions`) | A saved host: `id`, `label`, `host`, `username`, `port?`, `commands?` (per-mode launch command override). |
|
||||
| `RemoteCase` | A working directory on a host: `name`, `type: 'remote'`, `hostId`, `remotePath`. |
|
||||
| `SessionRemote` (extends `RemoteSshOptions`) | The resolved bundle stamped onto a live session: host coordinates + `remotePath` + `commands`, plus **`owned?`** and **`remoteSessionName?`** (COD-105 — see [Ownership](#ownership-launched-vs-discovered-and-attached-cod-105)). Built by `toSessionRemote(host, case)` (sets `owned: true`) for the launch path, or `toAttachedSessionRemote(host, name, path)` (sets `owned: false`) for the attach path. Both copy the advanced SSH options through so every connection is identical. |
|
||||
| `RemoteCommandMode` | `Extract<SessionMode, 'shell' \| 'claude' \| 'opencode' \| 'codex' \| 'gemini'>` — the modes that can run remotely. |
|
||||
| `RemoteSessionInfo` (COD-105) | One discovered remote tmux session: `name` (always `codeman-*`), `attached` (a client is connected), `created` (epoch s), `windows`. Returned by `listRemoteCodemanSessions()`. |
|
||||
|
||||
Persistence is two flat JSON arrays in the instance data dir:
|
||||
|
||||
- `~/.codeman/remote-hosts.json` — `readRemoteHosts()` / `writeRemoteHosts()`
|
||||
- `~/.codeman/remote-cases.json` — `readRemoteCases()` / `writeRemoteCases()`
|
||||
|
||||
(Paths via `remoteHostsPath()` / `remoteCasesPath()`; both honor `CODEMAN_INSTANCE`
|
||||
because the config dir is the instance data dir.)
|
||||
|
||||
On the live `Session`, the remote rides as `_remote?: SessionRemote`. When
|
||||
attaching, `resolveMuxAttachCwd()` forces the cwd to `/tmp` for remote sessions —
|
||||
the local working directory is meaningless on the remote box.
|
||||
|
||||
## SSH command construction (COD-107 — the injection surface)
|
||||
|
||||
**All** SSH command lines flow through one function so user-controlled fields are
|
||||
escaped once and the launch + prereq probe can never drift apart:
|
||||
|
||||
```ts
|
||||
// src/remote-hosts.ts
|
||||
buildSshConnectionArgs(remote: RemoteSshOptions & Pick<RemoteHost, 'port'>): string[]
|
||||
```
|
||||
|
||||
It returns the **ordered leading tokens** of an ssh command line (no `-t`, no
|
||||
target, no remote command):
|
||||
|
||||
```
|
||||
ssh -o BatchMode=yes
|
||||
[-p <port>]
|
||||
[-i <abs-identity>] # ~ / $HOME expanded, then shellescaped
|
||||
[-J <jumpHost>] # shellescaped, single token
|
||||
[-o ProxyCommand=nc -X 5 -x <socks> %h %p] # ONE shellescaped -o token
|
||||
[-o <KEY=VALUE>] … # each extra option, shellescaped
|
||||
```
|
||||
|
||||
Rules that keep this safe — **do not bypass them by hand-building an ssh line elsewhere:**
|
||||
|
||||
- **Every** user-controlled value (`-i`, `-J`, `-o`, ProxyCommand) is POSIX
|
||||
single-quote `shellescape`d (`'…'` with embedded `'\''`). The helper mirrors
|
||||
the one in `tmux-manager.ts`.
|
||||
- **`~`/`$HOME` in `identityFile` is expanded at build time** (`expandIdentityPath`),
|
||||
*before* escaping — ssh does not expand `~` inside `-i`, and the escaped value
|
||||
never reaches a shell that would.
|
||||
- **The ProxyCommand is one shellescaped `-o KEY=VALUE` token**, so its spaces and
|
||||
the `%h`/`%p` placeholders reach ssh as a single argument. `%h %p` survive
|
||||
verbatim — **ssh** expands them to the real host/port, not the shell.
|
||||
- **Empty options ⇒ `['ssh', '-o BatchMode=yes']`** (+ `-p` only when set) —
|
||||
byte-identical to the historical behavior.
|
||||
|
||||
Token construction is unit-tested independently of any live connection (see
|
||||
`test/` for `buildSshConnectionArgs` / `buildRemoteTmuxCheckCommand` cases).
|
||||
|
||||
## Durable launch (COD-104)
|
||||
|
||||
`buildRemoteLaunchCommand({ mode, remote, sessionId })` in `tmux-manager.ts`
|
||||
builds the command that launches (or **reattaches** to) the remote session:
|
||||
|
||||
```
|
||||
ssh -o BatchMode=yes -t <connection-args> user@host \
|
||||
'tmux -L codeman-remote new-session -A -s codeman-ssh-<id8> -c <remotePath> "cd <remotePath> && exec <cli>" \; \
|
||||
set -t codeman-ssh-<id8> status off \; set -t codeman-ssh-<id8> mouse off \; \
|
||||
set -t codeman-ssh-<id8> prefix C-q \; set -s escape-time 0 \; \
|
||||
set -t codeman-ssh-<id8> window-size latest'
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- **`new-session -A -s codeman-ssh-<id8>`** = attach-if-exists-else-create, so a
|
||||
reconnect (same deterministic `remoteTmuxSessionName(sessionId)` — `codeman-ssh-` +
|
||||
the first 8 chars of the session id) lands back in
|
||||
the **same** remote session rather than spawning a duplicate. This is what makes
|
||||
the remote agent survive an SSH drop. The name deliberately fails
|
||||
`SAFE_MUX_NAME_PATTERN` so a Codeman running ON the remote host never adopts it.
|
||||
- **`-L codeman-remote`** = a DEDICATED socket for sessions launched by remote
|
||||
Codemans, NOT the canonical `-L codeman` socket the remote host's own Codeman
|
||||
uses. Options are set per-session (`set -t`), never `-g`, so a shared remote
|
||||
tmux server's other sessions are untouched (#145 hardening). Note the
|
||||
asymmetry: **discovery/attach (COD-105) target the canonical `-L codeman`
|
||||
socket** — they join sessions the remote's own Codeman manages, while owned
|
||||
durable launches live on `-L codeman-remote`.
|
||||
- **`exec <cli>`** replaces the pane shell with the agent, so the pane PID *is*
|
||||
the agent. The per-mode command comes from `remote.commands?.[mode]` or
|
||||
`defaultRemoteCommandForMode(mode)` (`exec claude` / `exec opencode` /
|
||||
`exec codex` / `exec gemini` / `exec bash -l`).
|
||||
- The **whole tmux invocation is a single shell-quoted ssh argument**, and the
|
||||
pane command is independently quoted, so a `remotePath` with spaces is safe.
|
||||
- Connection options come from the **same `buildSshConnectionArgs(remote)`** as
|
||||
the prereq probe; `-t` is inserted right after `ssh -o BatchMode=yes`,
|
||||
preserving historical token order.
|
||||
|
||||
### tmux prerequisite probe
|
||||
|
||||
Because durable remote sessions require tmux on the remote host,
|
||||
`checkRemoteTmuxAvailable(host)` runs `command -v tmux` over SSH **before**
|
||||
creating a remote case/session and returns a structured, never-throwing result:
|
||||
|
||||
- empty stdout / non-zero exit → *"remote host `<host>` needs tmux installed for
|
||||
durable remote sessions"*
|
||||
- stderr present → *"could not verify tmux on remote host `<host>`: `<stderr>`"*
|
||||
(a real connection failure, surfaced to the operator)
|
||||
- success → `{ ok: true, tmuxPath }`
|
||||
|
||||
It connects with the **identical** options as the launch
|
||||
(`buildRemoteTmuxCheckCommand` reuses `buildSshConnectionArgs` and inserts
|
||||
`-o ConnectTimeout=10`), so a proxied/custom-port/identity host that the launch
|
||||
can reach also passes the probe (and vice-versa).
|
||||
|
||||
**Test-mode short-circuit:** under `VITEST` the probe returns
|
||||
`{ ok: true, tmuxPath: '(test-mode)' }` without opening a socket — mirroring
|
||||
`TmuxManager`'s no-op-shell-under-VITEST (`IS_TEST_MODE`). Without it, remote-case
|
||||
create-path tests would hit a real ~10s ssh timeout. Only the live probe is
|
||||
skipped; command construction is still asserted by unit tests.
|
||||
|
||||
## Ownership: launched vs. discovered-and-attached (COD-105)
|
||||
|
||||
COD-104 (above) was Phase 1 — Codeman *launches* a remote session and owns it.
|
||||
COD-105 is Phase 2 — Codeman can also **discover** `codeman-*` tmux sessions
|
||||
already running on a remote host (created by the remote's own Codeman or another
|
||||
instance) and **attach** to one it didn't launch. Ownership decides what happens
|
||||
when the tab closes.
|
||||
|
||||
`SessionRemote.owned` carries this:
|
||||
|
||||
- **`owned: true`** (or absent — legacy/COD-104 sessions persisted before this
|
||||
field) — we launched it via `buildRemoteLaunchCommand` and may explicitly kill it.
|
||||
- **`owned: false`** — discovered + attached; another Codeman owns the remote
|
||||
session. `remoteSessionName` holds its existing tmux name. Closing the tab
|
||||
**detaches**, never kills.
|
||||
|
||||
### Discovery
|
||||
|
||||
`listRemoteCodemanSessions(host)` lists the remote's `codeman-*` sessions:
|
||||
|
||||
- `buildRemoteListSessionsCommand()` runs `tmux -L codeman list-sessions -F "…"`
|
||||
over SSH (connection args from the shared `buildSshConnectionArgs`, so discovery
|
||||
connects identically to launch/probe). `2>/dev/null` swallows tmux's "no server
|
||||
running" stderr.
|
||||
- `parseRemoteSessionList()` is a **pure, unit-tested** parser. ⚠️ Quirk: the
|
||||
remote tmux's `-F "…\t…"` format emits the **literal two-character `\t`**, not a
|
||||
real tab (verified on tmux next-3.7), so the parser splits on `/\\t|\t/` (literal
|
||||
backslash-t **or** a real tab, for builds that do expand it). It keeps only
|
||||
`codeman-*` names, coerces types, and skips malformed lines.
|
||||
- `listRemoteCodemanSessions()` **never throws** — unreachable host / no tmux / no
|
||||
sessions all map to `[]`. Like the prereq probe, it **no-ops to `[]` under
|
||||
`VITEST`** so a request path never opens a real ssh connection.
|
||||
|
||||
Discovery is **explicit** — the UI has a "Discover existing sessions" button per
|
||||
host; Codeman never auto-discovers on host select.
|
||||
|
||||
### Attach vs. launch selection
|
||||
|
||||
`buildRemoteSessionCommand(mode, remote, sessionId)` in `tmux-manager.ts` picks the
|
||||
remote command line by ownership:
|
||||
|
||||
- **`owned === false`** → `buildRemoteAttachCommand(remote, name)` — emits
|
||||
`ssh … -t … 'tmux -L codeman attach -t <remoteSessionName>'`. It uses **`attach`,
|
||||
NOT `new-session -A`**, so it only *joins* an existing session and never creates
|
||||
one.
|
||||
- **owned (default)** → `buildRemoteLaunchCommand` (the COD-104 path above).
|
||||
|
||||
### Detach-not-kill
|
||||
|
||||
`TmuxManager.killSession()` has an **early return for non-owned remote sessions**:
|
||||
it tears down **only the LOCAL pane** holding the ssh client (`tmux -L codeman
|
||||
kill-session` on *this* host's socket). Killing the local ssh sends SIGHUP to the
|
||||
remote `tmux attach`, which **detaches** — the durable remote session survives.
|
||||
The early return is a structural guarantee that **no code path can ever issue a
|
||||
remote `kill-session` for a session we don't own** — the only `kill-session` run is
|
||||
on the local socket, which never reaches the remote socket.
|
||||
|
||||
## API
|
||||
|
||||
Routes are registered in `src/web/routes/case-routes.ts`:
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|--------|------|---------|
|
||||
| `GET` | `/api/remote-hosts` | List saved hosts |
|
||||
| `POST` | `/api/remote-hosts` | Create a host |
|
||||
| `PUT` | `/api/remote-hosts/:id` | Update a host |
|
||||
| `DELETE` | `/api/remote-hosts/:id` | Delete a host |
|
||||
| `GET` | `/api/remote-hosts/:hostId/sessions` | Discover `codeman-*` sessions on the host (COD-105; `listRemoteCodemanSessions`, never errors) |
|
||||
| `POST` | `/api/cases/remote-link` | Link a case to a remote host (creates the `RemoteCase`) |
|
||||
|
||||
Attaching to a discovered session is a **session-create** path, not a host route:
|
||||
`POST /api/sessions` accepts `attachRemoteSession: { hostId, remoteSessionName }`
|
||||
(schema in `schemas.ts`; `remoteSessionName` must match `^codeman-[a-zA-Z0-9._-]+$`),
|
||||
which `session-routes.ts` turns into a non-owned (`owned: false`) session.
|
||||
|
||||
Frontend touchpoints: the remote-host management UI is in `session-ui.js` /
|
||||
`panels-ui.js`; a remote session is created by picking a remote host/case in the
|
||||
session-create flow, or via the per-host **"Discover existing sessions"** button →
|
||||
**Attach** action (creates an `owned: false` session).
|
||||
|
||||
## Security notes
|
||||
|
||||
- **`identityFile` is a path only — never key bytes.** Codeman stores the path and
|
||||
passes it to `ssh -i`; the key never enters Codeman's state or the wire.
|
||||
- The injection surface is the SSH option fields. The single-source
|
||||
`buildSshConnectionArgs` + `shellescape` discipline (COD-107) is the control —
|
||||
audit any new code path that constructs an ssh command to route through it
|
||||
rather than concatenating options inline.
|
||||
- `BatchMode=yes` means **no interactive password/passphrase prompts** — remote
|
||||
hosts must be reachable with key-based or agent auth (or an unencrypted key the
|
||||
agent has loaded). A host needing a passphrase will fail the probe with an ssh
|
||||
diagnostic rather than hang.
|
||||
|
||||
## Related
|
||||
|
||||
- `CLAUDE.md` → Architecture → **Remote** row, and the **Remote sessions (SSH)**
|
||||
Key Pattern.
|
||||
- `docs/security-architecture.md` — overall network/auth model.
|
||||
- COD-104 (tmux prereq + durable launch), COD-105 (discover + attach, detach-not-kill ownership), COD-107 (shell-safe connection args).
|
||||
@@ -8,6 +8,7 @@ import type {
|
||||
RemoteCase,
|
||||
RemoteCommandMode,
|
||||
RemoteHost,
|
||||
RemoteSessionInfo,
|
||||
RemoteSshOptions,
|
||||
SessionMode,
|
||||
SessionRemote,
|
||||
@@ -173,6 +174,14 @@ export interface RemoteTmuxCheckResult {
|
||||
export async function checkRemoteTmuxAvailable(
|
||||
host: Pick<RemoteHost, 'username' | 'host' | 'port'> & RemoteSshOptions
|
||||
): Promise<RemoteTmuxCheckResult> {
|
||||
// Under vitest, never open a real ssh connection — mirrors TmuxManager's
|
||||
// no-op-shell-under-VITEST (IS_TEST_MODE). Without this, remote-case
|
||||
// create-path tests hit a real ~10s ssh timeout. The command construction is
|
||||
// covered by buildRemoteTmuxCheckCommand unit tests; only the live probe is
|
||||
// short-circuited here.
|
||||
if (process.env.VITEST) {
|
||||
return { ok: true, tmuxPath: '(test-mode)' };
|
||||
}
|
||||
const command = buildRemoteTmuxCheckCommand(host);
|
||||
try {
|
||||
const { stdout } = await execAsync(command, { timeout: 15_000 });
|
||||
@@ -202,6 +211,109 @@ export async function checkRemoteTmuxAvailable(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-105 — build the SSH command that lists `codeman-*` tmux sessions on a
|
||||
* remote host's canonical `-L codeman` socket.
|
||||
*
|
||||
* `list-sessions` exits NON-ZERO with empty output when no sessions exist (and
|
||||
* the server isn't running), so `2>/dev/null` swallows tmux's "no server
|
||||
* running" stderr; the caller treats a non-zero exit / empty output as "no
|
||||
* sessions" rather than an error.
|
||||
*
|
||||
* COD-107 — connection options come from the shared `buildSshConnectionArgs`, so
|
||||
* discovery connects with the SAME port/identity/proxy/jump-host as the launch
|
||||
* and the tmux prereq probe.
|
||||
*/
|
||||
export function buildRemoteListSessionsCommand(
|
||||
host: Pick<RemoteHost, 'username' | 'host' | 'port'> & RemoteSshOptions
|
||||
): string {
|
||||
const [ssh, ...connectionArgs] = buildSshConnectionArgs(host);
|
||||
const parts = [ssh, connectionArgs[0], '-o ConnectTimeout=10', ...connectionArgs.slice(1)];
|
||||
// The tmux list-sessions invocation is passed as ONE shell-quoted argument so
|
||||
// the remote login shell runs it verbatim. The `-F` format uses literal `\t`
|
||||
// separators (tmux expands them); `2>/dev/null` is inside the quoted command.
|
||||
const remoteCmd =
|
||||
'tmux -L codeman list-sessions -F "#{session_name}\\t#{session_attached}\\t#{session_created}\\t#{session_windows}" 2>/dev/null';
|
||||
parts.push(remoteSshTarget(host), shellescape(remoteCmd));
|
||||
return parts.join(' ');
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-105 — pure parser for the `tmux list-sessions -F` output emitted by
|
||||
* `buildRemoteListSessionsCommand`. Factored out so the parse is unit-testable
|
||||
* without opening a real ssh connection.
|
||||
*
|
||||
* - Splits each non-empty line into [name, attached, created, windows] on the
|
||||
* field separator. IMPORTANT: the remote tmux's `-F "…\t…"` format does NOT
|
||||
* expand `\t` to a real tab — it emits the LITERAL two-character sequence
|
||||
* `\t` (verified on aa-desktop / tmux next-3.7). So we split on the literal
|
||||
* backslash-t sequence; we also tolerate a real tab in case a tmux build
|
||||
* does expand it. (A real TAB is the regex `\t`; a literal backslash-t is the
|
||||
* regex `\\t`.)
|
||||
* - Keeps ONLY sessions whose name starts with `codeman-` (ignores foreign tmux
|
||||
* sessions that happen to share the socket).
|
||||
* - Coerces: `attached` → boolean (`'1'`), `created`/`windows` → finite ints.
|
||||
* - Skips malformed lines (wrong column count or non-numeric created/windows)
|
||||
* rather than emitting garbage.
|
||||
*/
|
||||
export function parseRemoteSessionList(stdout: string): RemoteSessionInfo[] {
|
||||
const out: RemoteSessionInfo[] = [];
|
||||
for (const rawLine of stdout.split('\n')) {
|
||||
const line = rawLine.trim();
|
||||
if (!line) continue;
|
||||
// Split on a literal `\t` (backslash + t, what the remote tmux emits) OR a
|
||||
// real tab character. `/\\t|\t/` = the two-char sequence, or a TAB.
|
||||
const cols = line.split(/\\t|\t/);
|
||||
if (cols.length !== 4) continue;
|
||||
const [name, attachedStr, createdStr, windowsStr] = cols;
|
||||
if (!name.startsWith('codeman-')) continue;
|
||||
const created = Number(createdStr);
|
||||
const windows = Number(windowsStr);
|
||||
if (!Number.isFinite(created) || !Number.isFinite(windows)) continue;
|
||||
// COD-106 — `session_attached` is the CLIENT COUNT (not a 0/1 flag); >1 = shared.
|
||||
const attachedNum = Number(attachedStr.trim());
|
||||
const attachedClients = Number.isFinite(attachedNum) ? Math.max(0, Math.trunc(attachedNum)) : 0;
|
||||
out.push({
|
||||
name,
|
||||
attached: attachedClients > 0,
|
||||
attachedClients,
|
||||
created: Math.trunc(created),
|
||||
windows: Math.trunc(windows),
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-105 — discover `codeman-*` tmux sessions already running on a remote host
|
||||
* (created by the remote's own Codeman, another instance, or this one), so the
|
||||
* operator can attach to one this Codeman didn't launch.
|
||||
*
|
||||
* NEVER throws: returns `[]` on unreachable host / no tmux / no sessions
|
||||
* (`list-sessions` exits non-zero with empty output when there are none).
|
||||
*
|
||||
* VITEST guard — like `checkRemoteTmuxAvailable`, returns `[]` under test so a
|
||||
* real ssh never runs in a request path (which would make route tests hit a
|
||||
* ~10s timeout). The command construction is covered by
|
||||
* `buildRemoteListSessionsCommand` and the parse by `parseRemoteSessionList`.
|
||||
*/
|
||||
export async function listRemoteCodemanSessions(
|
||||
remote: Pick<RemoteHost, 'username' | 'host' | 'port'> & RemoteSshOptions
|
||||
): Promise<RemoteSessionInfo[]> {
|
||||
if (process.env.VITEST) {
|
||||
return [];
|
||||
}
|
||||
const command = buildRemoteListSessionsCommand(remote);
|
||||
try {
|
||||
const { stdout } = await execAsync(command, { timeout: 15_000 });
|
||||
return parseRemoteSessionList(stdout);
|
||||
} catch {
|
||||
// Unreachable host, no tmux server, or no sessions (non-zero exit). All map
|
||||
// to "nothing to attach to" — never surface as an error to the caller.
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
export function remoteDisplayPath(
|
||||
remote: Pick<SessionRemote, 'username' | 'host' | 'remotePath'> | { username: string; host: string; path: string }
|
||||
): string {
|
||||
@@ -218,6 +330,10 @@ export function toSessionRemote(host: RemoteHost, remoteCase: RemoteCase): Sessi
|
||||
port: host.port,
|
||||
remotePath: remoteCase.remotePath,
|
||||
commands: host.commands,
|
||||
// COD-105 — the COD-104 launch path creates the remote session, so we own it
|
||||
// (an explicit kill may propagate a remote kill-session). Discovered+attached
|
||||
// sessions go through `toAttachedSessionRemote` with `owned: false`.
|
||||
owned: true,
|
||||
// COD-107 — carry the advanced SSH options from host config into the session
|
||||
// so the launch/prereq commands connect the same way the operator configured.
|
||||
identityFile: host.identityFile,
|
||||
@@ -226,3 +342,38 @@ export function toSessionRemote(host: RemoteHost, remoteCase: RemoteCase): Sessi
|
||||
extraSshOptions: host.extraSshOptions,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-105 — build a NON-owned `SessionRemote` for ATTACHING to a `codeman-*`
|
||||
* session already running on a remote host (discovered via
|
||||
* `listRemoteCodemanSessions`). The resulting session's pane runs
|
||||
* `tmux -L codeman attach -t <remoteSessionName>` (see
|
||||
* `buildRemoteAttachCommand`), and because we did NOT create the remote session,
|
||||
* `owned: false` means closing the tab DETACHES rather than killing it.
|
||||
*
|
||||
* `remotePath` is informational here (the attached remote session keeps its own
|
||||
* cwd); we record the host's nominal path so display helpers still show
|
||||
* `user@host:path`.
|
||||
*/
|
||||
export function toAttachedSessionRemote(
|
||||
host: RemoteHost,
|
||||
remoteSessionName: string,
|
||||
remotePath: string
|
||||
): SessionRemote {
|
||||
return {
|
||||
hostId: host.id,
|
||||
label: host.label,
|
||||
host: host.host,
|
||||
username: host.username,
|
||||
port: host.port,
|
||||
remotePath,
|
||||
commands: host.commands,
|
||||
// Discovered + attached — another Codeman created it. Detach-not-kill.
|
||||
owned: false,
|
||||
remoteSessionName,
|
||||
identityFile: host.identityFile,
|
||||
socksProxy: host.socksProxy,
|
||||
jumpHost: host.jumpHost,
|
||||
extraSshOptions: host.extraSshOptions,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
/**
|
||||
* @fileoverview Pure logic for the remote-session auto-reconnect watcher (COD-108).
|
||||
*
|
||||
* COD-104 made remote tmux sessions durable + idempotently reattachable, but a
|
||||
* reconnect only fired at explicit trigger points. COD-108 adds a continuous
|
||||
* watcher (in `TmuxManager`) that detects a dead remote pane and emits
|
||||
* `remoteSessionDropped`; `SessionManager`/server then reassembles the respawn
|
||||
* options and reattaches (re-running the idempotent remote command).
|
||||
*
|
||||
* This module holds the SIDE-EFFECT-FREE pieces so they can be unit-tested
|
||||
* without real tmux:
|
||||
* - the bounded exponential **backoff schedule** (attempt → delay, capped),
|
||||
* - the per-session **reconnect state** shape,
|
||||
* - the **eligibility decision** (`decideReconnect`) given a session + its
|
||||
* reconnect state + the current time + the guard set.
|
||||
*
|
||||
* The watcher in `tmux-manager.ts` owns the live `isPaneDead` probe and the
|
||||
* timers; everything here is pure and deterministic (time is injected).
|
||||
*
|
||||
* @module remote-reconnect
|
||||
*/
|
||||
|
||||
/**
|
||||
* Bounded exponential backoff delays (ms) between reconnect attempts.
|
||||
* Attempt N (1-based) waits `BACKOFF_SCHEDULE_MS[N-1]` from the previous emit
|
||||
* before the next emit is eligible. After the last entry the session is
|
||||
* considered `reconnect-exhausted` and the watcher stops emitting for it.
|
||||
*
|
||||
* 5s, 15s, 45s, 2m, 5m, 5m → ~6 attempts spanning ~13 minutes.
|
||||
*/
|
||||
export const BACKOFF_SCHEDULE_MS: readonly number[] = [5_000, 15_000, 45_000, 120_000, 300_000, 300_000];
|
||||
|
||||
/** Maximum number of reconnect attempts before exhaustion. */
|
||||
export const MAX_RECONNECT_ATTEMPTS = BACKOFF_SCHEDULE_MS.length;
|
||||
|
||||
/**
|
||||
* Delay (ms) to wait AFTER emitting attempt `attempt` (1-based) before the next
|
||||
* attempt is eligible. `attempt <= 0` returns the first delay; an attempt at or
|
||||
* beyond the cap returns the last delay (callers should check exhaustion via
|
||||
* {@link isExhausted} rather than relying on this for the stop decision).
|
||||
*
|
||||
* Pure — no clock, no I/O.
|
||||
*/
|
||||
export function reconnectDelayForAttempt(attempt: number): number {
|
||||
if (!Number.isFinite(attempt) || attempt <= 1) return BACKOFF_SCHEDULE_MS[0];
|
||||
const idx = Math.min(Math.floor(attempt) - 1, BACKOFF_SCHEDULE_MS.length - 1);
|
||||
return BACKOFF_SCHEDULE_MS[idx];
|
||||
}
|
||||
|
||||
/** Whether `attempts` reconnect emits have reached/exceeded the cap. Pure. */
|
||||
export function isExhausted(attempts: number): boolean {
|
||||
return attempts >= MAX_RECONNECT_ATTEMPTS;
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-session reconnect bookkeeping held by the watcher. All time values are
|
||||
* epoch ms. `inFlight` guards against stacking respawns when a tick fires while
|
||||
* a previous reattach is still running. `exhaustedEmitted` ensures the
|
||||
* `remoteReconnectExhausted` event fires at most once per session.
|
||||
*/
|
||||
export interface RemoteReconnectState {
|
||||
/** Number of `remoteSessionDropped` emits so far (advances per emit). */
|
||||
attempts: number;
|
||||
/** Earliest time (epoch ms) the next emit is eligible. 0 = eligible now. */
|
||||
nextEligibleAt: number;
|
||||
/** A reattach triggered by a prior emit is currently running. */
|
||||
inFlight: boolean;
|
||||
/** Cap reached — stop auto-retrying for this session. */
|
||||
exhausted: boolean;
|
||||
/** The `remoteReconnectExhausted` SSE event has already been emitted. */
|
||||
exhaustedEmitted: boolean;
|
||||
}
|
||||
|
||||
/** A fresh reconnect state (no attempts, immediately eligible). Pure. */
|
||||
export function freshReconnectState(): RemoteReconnectState {
|
||||
return { attempts: 0, nextEligibleAt: 0, inFlight: false, exhausted: false, exhaustedEmitted: false };
|
||||
}
|
||||
|
||||
/**
|
||||
* Advance the backoff after an emit at time `now`. Increments `attempts` and
|
||||
* schedules `nextEligibleAt = now + delay`. Returns a NEW state object (does
|
||||
* not mutate the input). Pure.
|
||||
*
|
||||
* NOTE: this does NOT set `exhausted`. Exhaustion is a decision the watcher
|
||||
* makes on the FOLLOWING tick (via {@link decideReconnect} → `exhaust`), so the
|
||||
* `remoteReconnectExhausted` event fires exactly once after the final attempt's
|
||||
* backoff window elapses — not pre-emptively on the last emit.
|
||||
*/
|
||||
export function advanceBackoff(state: RemoteReconnectState, now: number): RemoteReconnectState {
|
||||
const attempts = state.attempts + 1;
|
||||
const delay = reconnectDelayForAttempt(attempts);
|
||||
return {
|
||||
...state,
|
||||
attempts,
|
||||
nextEligibleAt: now + delay,
|
||||
};
|
||||
}
|
||||
|
||||
/** Reset after a successful reattach — back to a fresh, eligible state. Pure. */
|
||||
export function resetReconnectState(): RemoteReconnectState {
|
||||
return freshReconnectState();
|
||||
}
|
||||
|
||||
/** Minimal session view the decision needs (avoids importing MuxSession here). */
|
||||
export interface ReconnectSessionView {
|
||||
sessionId: string;
|
||||
/** Truthy when this is a remote (SSH-wrapped) session. */
|
||||
isRemote: boolean;
|
||||
/** Result of `isPaneDead(muxName)` for this session. */
|
||||
paneDead: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision outcomes for a single watcher tick on one session.
|
||||
* - `emit` → emit `remoteSessionDropped { sessionId, attempt }`, then
|
||||
* advance backoff (attempt = the returned `attempt`).
|
||||
* - `exhaust` → cap reached this tick; emit `remoteReconnectExhausted` once.
|
||||
* - `skip` → do nothing (not remote / pane alive / guarded / in-flight /
|
||||
* not yet due / already exhausted).
|
||||
*/
|
||||
export type ReconnectAction =
|
||||
| { kind: 'emit'; attempt: number }
|
||||
| { kind: 'exhaust' }
|
||||
| { kind: 'skip'; reason: ReconnectSkipReason };
|
||||
|
||||
export type ReconnectSkipReason =
|
||||
| 'not-remote'
|
||||
| 'pane-alive'
|
||||
| 'guarded'
|
||||
| 'in-flight'
|
||||
| 'not-due'
|
||||
| 'exhausted'
|
||||
| 'disabled';
|
||||
|
||||
export interface DecideReconnectInput {
|
||||
session: ReconnectSessionView;
|
||||
state: RemoteReconnectState | undefined;
|
||||
/** Session is in the intentional-teardown guard set (killed/detached/stopping). */
|
||||
guarded: boolean;
|
||||
/** Kill-switch: `remoteAutoReconnect` setting. When false, never reconnect. */
|
||||
enabled: boolean;
|
||||
now: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* PURE eligibility decision for one session on one tick. No clock, no I/O — all
|
||||
* inputs are passed in. The watcher translates the result into emits + state
|
||||
* transitions.
|
||||
*
|
||||
* Order of guards (most-decisive first):
|
||||
* 1. kill-switch off → skip:disabled
|
||||
* 2. not a remote session → skip:not-remote
|
||||
* 3. pane is alive → skip:pane-alive
|
||||
* 4. intentional teardown guard → skip:guarded (NEVER revive a killed tab)
|
||||
* 5. a reattach already running → skip:in-flight (no stacked respawns)
|
||||
* 6. already exhausted → skip:exhausted (one exhaust emit, then quiet)
|
||||
* 7. cap reached this tick → exhaust
|
||||
* 8. not yet due (backoff) → skip:not-due
|
||||
* 9. otherwise → emit (attempt = attempts + 1)
|
||||
*/
|
||||
export function decideReconnect(input: DecideReconnectInput): ReconnectAction {
|
||||
const { session, state, guarded, enabled, now } = input;
|
||||
|
||||
if (!enabled) return { kind: 'skip', reason: 'disabled' };
|
||||
if (!session.isRemote) return { kind: 'skip', reason: 'not-remote' };
|
||||
if (!session.paneDead) return { kind: 'skip', reason: 'pane-alive' };
|
||||
// Intentional kill / detach must NEVER be auto-revived.
|
||||
if (guarded) return { kind: 'skip', reason: 'guarded' };
|
||||
|
||||
const s = state ?? freshReconnectState();
|
||||
|
||||
// Only one reconnect in flight per session — don't stack respawns.
|
||||
if (s.inFlight) return { kind: 'skip', reason: 'in-flight' };
|
||||
|
||||
if (s.exhausted) return { kind: 'skip', reason: 'exhausted' };
|
||||
|
||||
// Cap reached: surface exhaustion once, then go quiet.
|
||||
if (isExhausted(s.attempts)) return { kind: 'exhaust' };
|
||||
|
||||
// Backoff gate — only emit when due.
|
||||
if (now < s.nextEligibleAt) return { kind: 'skip', reason: 'not-due' };
|
||||
|
||||
return { kind: 'emit', attempt: s.attempts + 1 };
|
||||
}
|
||||
+66
-19
@@ -1248,6 +1248,70 @@ export class Session extends EventEmitter {
|
||||
return { isRestored };
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-108 — re-establish a dropped REMOTE session. Triggered by the
|
||||
* `TmuxManager` remote-reconnect watcher (via `remoteSessionDropped`): the
|
||||
* watcher detects a dead remote pane, the session owner reassembles the SAME
|
||||
* `RespawnPaneOptions` used for Claude-idle respawns and calls
|
||||
* `respawnPane()` directly. For a remote session that re-runs
|
||||
* `buildRemoteSessionCommand` (owned → `new-session -A`, non-owned →
|
||||
* `attach`), which idempotently REATTACHES the still-running durable remote
|
||||
* tmux session — scrollback + agent intact (proven COD-104/105).
|
||||
*
|
||||
* Deliberately does NOT route through the Claude-idle respawn-controller —
|
||||
* this is a transport re-establish, not a `/clear`/`/compact` cycle.
|
||||
*
|
||||
* @returns true if the pane was respawned (reattach issued), false otherwise.
|
||||
*/
|
||||
async reattachRemote(): Promise<boolean> {
|
||||
if (!this._remote) return false; // not a remote session
|
||||
if (!this._useMux || !this._mux || !this._muxSession) return false;
|
||||
const mux = this._mux;
|
||||
|
||||
// If tmux lost the whole session (not just a dead pane), there is nothing to
|
||||
// respawn into — a genuine death, leave it for normal recovery/reconcile.
|
||||
if (!mux.muxSessionExists(this._muxSession.muxName)) {
|
||||
console.log('[Session] reattachRemote: mux session gone, skipping:', this._muxSession.muxName);
|
||||
return false;
|
||||
}
|
||||
|
||||
const newPid = await mux.respawnPane(this._buildRespawnPaneOptions());
|
||||
if (!newPid) {
|
||||
console.error('[Session] reattachRemote: respawnPane failed for', this._muxSession.muxName);
|
||||
return false;
|
||||
}
|
||||
console.log('[Session] reattachRemote: reattached remote session', this._muxSession.muxName, 'pid', newPid);
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Assemble the {@link RespawnPaneOptions} for this session. Single source of
|
||||
* truth shared by interactive start, shell start (via their inline copies),
|
||||
* and {@link reattachRemote} so the remote reattach path can never drift from
|
||||
* the spawn path.
|
||||
*/
|
||||
private _buildRespawnPaneOptions(): import('./mux-interface.js').RespawnPaneOptions {
|
||||
return {
|
||||
sessionId: this.id,
|
||||
workingDir: this.workingDir,
|
||||
mode: this.mode,
|
||||
niceConfig: this._niceConfig,
|
||||
model: this._model,
|
||||
claudeMode: this._claudeMode,
|
||||
allowedTools: this._allowedTools,
|
||||
openCodeConfig: this._openCodeConfig,
|
||||
codexConfig: this._codexConfig,
|
||||
geminiConfig: this._geminiConfig,
|
||||
resumeSessionId: this._resumeSessionId,
|
||||
envOverrides: this._envOverrides,
|
||||
effort: this._effort,
|
||||
historyLimit: this._tmuxHistoryLimit,
|
||||
remote: this._remote,
|
||||
docker: this._docker,
|
||||
owner: this._owner,
|
||||
};
|
||||
}
|
||||
|
||||
private _handleTerminalOutput(data: string): void {
|
||||
// Codex AND Claude Code emit sequences that wipe xterm.js scrollback, plus
|
||||
// mouse-tracking enables that hijack the scroll wheel so the user can't reach
|
||||
@@ -1396,25 +1460,8 @@ export class Session extends EventEmitter {
|
||||
if (this._useMux && this._mux) {
|
||||
try {
|
||||
const { isRestored } = await this._setupOrAttachMuxSession({
|
||||
respawnPaneOptions: {
|
||||
sessionId: this.id,
|
||||
workingDir: this.workingDir,
|
||||
mode: this.mode,
|
||||
niceConfig: this._niceConfig,
|
||||
model: this._model,
|
||||
claudeMode: this._claudeMode,
|
||||
allowedTools: this._allowedTools,
|
||||
openCodeConfig: this._openCodeConfig,
|
||||
codexConfig: this._codexConfig,
|
||||
geminiConfig: this._geminiConfig,
|
||||
resumeSessionId: this._resumeSessionId,
|
||||
envOverrides: this._envOverrides,
|
||||
effort: this._effort,
|
||||
historyLimit: this._tmuxHistoryLimit,
|
||||
remote: this._remote,
|
||||
docker: this._docker,
|
||||
owner: this._owner,
|
||||
},
|
||||
// Single source of truth shared with reattachRemote() (COD-108).
|
||||
respawnPaneOptions: this._buildRespawnPaneOptions(),
|
||||
createSessionOptions: {
|
||||
sessionId: this.id,
|
||||
workingDir: this.workingDir,
|
||||
|
||||
+246
-2
@@ -78,6 +78,13 @@ import type {
|
||||
RespawnPaneOptions,
|
||||
PaneCaptureOptions,
|
||||
} from './mux-interface.js';
|
||||
import {
|
||||
decideReconnect,
|
||||
advanceBackoff,
|
||||
freshReconnectState,
|
||||
resetReconnectState,
|
||||
type RemoteReconnectState,
|
||||
} from './remote-reconnect.js';
|
||||
|
||||
// ============================================================================
|
||||
// Timing Constants
|
||||
@@ -110,6 +117,9 @@ const GRACEFUL_SHUTDOWN_WAIT_MS = 100;
|
||||
/** Default stats collection interval (2 seconds) */
|
||||
const DEFAULT_STATS_INTERVAL_MS = 2000;
|
||||
|
||||
/** Default remote-reconnect watcher poll interval (5 seconds) — COD-108 */
|
||||
const DEFAULT_REMOTE_RECONNECT_INTERVAL_MS = 5000;
|
||||
|
||||
/** Stable cwd for tmux server/pane launch; actual session cwd is reached inside the pane. */
|
||||
const TMUX_LAUNCH_CWD = '/tmp';
|
||||
|
||||
@@ -134,6 +144,20 @@ const IS_TEST_MODE = !!process.env.VITEST;
|
||||
/** Path to persisted mux session metadata */
|
||||
const MUX_SESSIONS_FILE = dataPath('mux-sessions.json');
|
||||
|
||||
/**
|
||||
* COD-108 kill-switch: `remoteAutoReconnect` app setting (default ON). Read at
|
||||
* call time (like headroom routing) so a settings change takes effect without a
|
||||
* restart. Absent/non-boolean ⇒ true (feature on).
|
||||
*/
|
||||
function isRemoteAutoReconnectEnabled(): boolean {
|
||||
try {
|
||||
const s = JSON.parse(readFileSync(dataPath('settings.json'), 'utf8')) as Record<string, unknown>;
|
||||
return typeof s.remoteAutoReconnect === 'boolean' ? s.remoteAutoReconnect : true;
|
||||
} catch {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
/** Regex to validate tmux session names (only allow safe characters) */
|
||||
const SAFE_MUX_NAME_PATTERN = /^codeman-[a-f0-9-]+$/;
|
||||
|
||||
@@ -812,6 +836,13 @@ export function buildRemoteLaunchCommand(options: {
|
||||
`set -t ${remoteName} mouse off`,
|
||||
`set -t ${remoteName} prefix C-q`,
|
||||
'set -s escape-time 0',
|
||||
// COD-106 — shared/collaborative sessions: tmux defaults to sizing a window
|
||||
// to the SMALLEST attached client, so two Codemans at different viewports
|
||||
// would fight (clamp to the smaller). `window-size latest` sizes to the
|
||||
// most-recently-active client instead, so concurrent clients coexist.
|
||||
// Per-session scoped (`set -t <name>`, matching #145's hardening) so a shared
|
||||
// remote tmux server's other sessions keep their own sizing behavior.
|
||||
`set -t ${remoteName} window-size latest`,
|
||||
].join(' \\; ');
|
||||
|
||||
// ssh runs its trailing args through the remote login shell, so the entire
|
||||
@@ -1132,6 +1163,56 @@ export function resolveDockerLaunchOptions(
|
||||
return { mode, docker, sessionId, resumeSessionId, createContext, execEnv, execEnvNames, seedCopies };
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-105 — build the SSH command that ATTACHES to an EXISTING `codeman-*` tmux
|
||||
* session on the remote host (one this Codeman didn't create — discovered via
|
||||
* `listRemoteCodemanSessions`). Sibling of `buildRemoteLaunchCommand`.
|
||||
*
|
||||
* Emits:
|
||||
* ssh -o BatchMode=yes -t [<COD-107 connection opts>] user@host \
|
||||
* 'tmux -L codeman attach -t <session>'
|
||||
*
|
||||
* - `attach` (NOT `new-session -A`) so we only join an existing session; the
|
||||
* remote session keeps running independent of us, which is exactly why the
|
||||
* resulting Codeman session is NON-OWNED (see `SessionRemote.owned`): closing
|
||||
* the local tab must detach, never `kill-session` the remote.
|
||||
* - The remote session name is shell-escaped so a value with metachars stays a
|
||||
* single token inside the quoted tmux invocation.
|
||||
* - COD-107 — connection options (`-p`, `-i`, `-J`, SOCKS `-o ProxyCommand`,
|
||||
* arbitrary `-o`) come from the shared `buildSshConnectionArgs`, so attach
|
||||
* connects identically to launch / discovery / the prereq probe. `-t` sits
|
||||
* right after `ssh -o BatchMode=yes` (a PTY is required for interactive tmux).
|
||||
*/
|
||||
export function buildRemoteAttachCommand(remote: SessionRemote, remoteSessionName: string): string {
|
||||
const tmuxInvocation = `tmux -L codeman attach -t ${shellescape(remoteSessionName)}`;
|
||||
const [ssh, batchMode, ...connectionArgs] = buildSshConnectionArgs(remote);
|
||||
const sshParts = [ssh, batchMode, '-t', ...connectionArgs, remoteSshTarget(remote), shellescape(tmuxInvocation)];
|
||||
return sshParts.join(' ');
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-105 — choose the right remote ssh command for a session's ownership:
|
||||
* - NON-owned (`remote.owned === false`): ATTACH to a discovered remote tmux
|
||||
* session by its EXISTING name (`remote.remoteSessionName`, falling back to
|
||||
* this session's deterministic name). We only join — never create.
|
||||
* - owned (default): LAUNCH/attach-or-create via `buildRemoteLaunchCommand`
|
||||
* (COD-104), which we then own and may explicitly kill.
|
||||
*/
|
||||
function buildRemoteSessionCommand(options: {
|
||||
mode: SessionMode;
|
||||
remote: SessionRemote;
|
||||
sessionId: string;
|
||||
claudeMode?: ClaudeMode;
|
||||
allowedTools?: string;
|
||||
}): string {
|
||||
const { remote, sessionId } = options;
|
||||
if (remote.owned === false) {
|
||||
const target = remote.remoteSessionName || remoteTmuxSessionName(sessionId);
|
||||
return buildRemoteAttachCommand(remote, target);
|
||||
}
|
||||
return buildRemoteLaunchCommand(options);
|
||||
}
|
||||
|
||||
/**
|
||||
* Set sensitive environment variables on a tmux session via setenv.
|
||||
* These are inherited by panes but not visible in ps output or tmux history.
|
||||
@@ -1285,6 +1366,17 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
/** Track last-known pane count per session to avoid unnecessary tmux set-option calls */
|
||||
private lastPaneCount: Map<string, number> = new Map();
|
||||
|
||||
// ── COD-108 remote-reconnect watcher state ────────────────────────────────
|
||||
/** Periodic watcher that re-establishes dropped remote sessions. */
|
||||
private remoteReconnectInterval: NodeJS.Timeout | null = null;
|
||||
/** Per-session backoff/attempt bookkeeping (sessionId → state). */
|
||||
private reconnectState: Map<string, RemoteReconnectState> = new Map();
|
||||
/**
|
||||
* Sessions excluded from auto-reconnect because they are being intentionally
|
||||
* torn down (killed/detached/stopping). A guarded session is NEVER revived.
|
||||
*/
|
||||
private reconnectGuard: Set<string> = new Set();
|
||||
|
||||
private trueColorConfigured = false;
|
||||
|
||||
constructor() {
|
||||
@@ -1600,7 +1692,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
const fullCmd = docker
|
||||
? buildDockerLaunchCommand(resolveDockerLaunchOptions(mode, docker, sessionId, resumeSessionId))
|
||||
: remote
|
||||
? buildRemoteLaunchCommand({ mode, remote, sessionId, claudeMode, allowedTools })
|
||||
? buildRemoteSessionCommand({ mode, remote, sessionId, claudeMode, allowedTools })
|
||||
: localFullCmd;
|
||||
|
||||
// Create tmux session in three steps to handle cold-start (no server running)
|
||||
@@ -1855,7 +1947,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
const fullCmd = docker
|
||||
? buildDockerLaunchCommand(resolveDockerLaunchOptions(mode, docker, sessionId, resumeSessionId))
|
||||
: remote
|
||||
? buildRemoteLaunchCommand({ mode, remote, sessionId, claudeMode, allowedTools })
|
||||
? buildRemoteSessionCommand({ mode, remote, sessionId, claudeMode, allowedTools })
|
||||
: localFullCmd;
|
||||
|
||||
try {
|
||||
@@ -1971,9 +2063,16 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
return false;
|
||||
}
|
||||
|
||||
// COD-108: an intentional kill/detach must NEVER be auto-revived by the
|
||||
// remote-reconnect watcher. Guard BEFORE any teardown so a tick that fires
|
||||
// mid-kill (especially the non-owned DETACH early-return below, where the
|
||||
// dead local pane would otherwise look reconnectable) sees the guard.
|
||||
this.guardRemoteReconnect(sessionId);
|
||||
|
||||
// TEST MODE: Remove from memory only — NEVER touch real tmux sessions
|
||||
if (IS_TEST_MODE) {
|
||||
this.sessions.delete(sessionId);
|
||||
this.clearRemoteReconnectState(sessionId);
|
||||
this.emit('sessionKilled', { sessionId });
|
||||
return true;
|
||||
}
|
||||
@@ -1985,6 +2084,40 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
return false;
|
||||
}
|
||||
|
||||
// COD-105 — DETACH-NOT-KILL for NON-owned remote sessions.
|
||||
//
|
||||
// When this session was created by ATTACHING a remote tmux session another
|
||||
// Codeman owns (`remote.owned === false`), closing the tab must NOT propagate
|
||||
// a remote `tmux kill-session` — that would nuke work the remote's own
|
||||
// Codeman (or another instance) still relies on. We tear down ONLY the LOCAL
|
||||
// pane that holds the ssh client: killing the local ssh sends SIGHUP to its
|
||||
// remote `tmux attach`, which DETACHES (the durable remote session survives).
|
||||
//
|
||||
// This early return is the structural guarantee: no code below this point
|
||||
// (now or in future for owned sessions) can ever issue a remote kill-session
|
||||
// for a non-owned session. The only `kill-session` we run is on OUR LOCAL
|
||||
// socket (`this.tmux()` = `tmux -L codeman` on THIS host), which kills the
|
||||
// local pane — it does NOT reach the REMOTE socket.
|
||||
if (session.remote && session.remote.owned === false) {
|
||||
console.log(`[TmuxManager] DETACH (non-owned remote): tearing down local pane only for ${session.muxName}`);
|
||||
if (isValidMuxName(session.muxName)) {
|
||||
try {
|
||||
// Local socket only — detaches the remote session by killing the local ssh pane.
|
||||
execSync(`${this.tmux()} kill-session -t "${session.muxName}" 2>/dev/null`, {
|
||||
timeout: EXEC_TIMEOUT_MS,
|
||||
});
|
||||
} catch {
|
||||
// Local pane may already be gone.
|
||||
}
|
||||
}
|
||||
this.lastPaneCount.delete(session.muxName);
|
||||
this.sessions.delete(sessionId);
|
||||
this.clearRemoteReconnectState(sessionId);
|
||||
this.saveSessions();
|
||||
this.emit('sessionKilled', { sessionId });
|
||||
return true;
|
||||
}
|
||||
|
||||
// Get current PID (may have changed)
|
||||
const currentPid = this.getPanePid(session.muxName) || session.pid;
|
||||
|
||||
@@ -2089,6 +2222,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
|
||||
this.lastPaneCount.delete(session.muxName);
|
||||
this.sessions.delete(sessionId);
|
||||
this.clearRemoteReconnectState(sessionId);
|
||||
this.saveSessions();
|
||||
this.emit('sessionKilled', { sessionId });
|
||||
|
||||
@@ -2155,6 +2289,7 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
} else {
|
||||
dead.push(sessionId);
|
||||
this.sessions.delete(sessionId);
|
||||
this.clearRemoteReconnectState(sessionId);
|
||||
this.emit('sessionDied', { sessionId });
|
||||
}
|
||||
}
|
||||
@@ -2426,9 +2561,118 @@ export class TmuxManager extends EventEmitter implements TerminalMultiplexer {
|
||||
this.lastPaneCount.clear();
|
||||
}
|
||||
|
||||
// ── COD-108 remote-session auto-reconnect watcher ─────────────────────────
|
||||
|
||||
/**
|
||||
* Start the remote-reconnect watcher (COD-108). Each tick, for every tracked
|
||||
* session with `session.remote` whose local pane is DEAD, not intentionally
|
||||
* guarded, and within its backoff budget, emit `remoteSessionDropped` so the
|
||||
* session owner reattaches (re-running the idempotent remote command rejoins
|
||||
* the durable remote tmux session). After the attempt cap, emit
|
||||
* `remoteReconnectExhausted` once and go quiet.
|
||||
*
|
||||
* No-op tick body under `IS_TEST_MODE` (mirrors `startMouseModeSync`): tests
|
||||
* drive the logic deterministically via {@link runRemoteReconnectTick}.
|
||||
*/
|
||||
startRemoteReconnectWatcher(intervalMs: number = DEFAULT_REMOTE_RECONNECT_INTERVAL_MS): void {
|
||||
if (this.remoteReconnectInterval) {
|
||||
clearInterval(this.remoteReconnectInterval);
|
||||
}
|
||||
this.remoteReconnectInterval = setInterval(() => {
|
||||
if (IS_TEST_MODE) return;
|
||||
try {
|
||||
this.runRemoteReconnectTick(Date.now(), isRemoteAutoReconnectEnabled());
|
||||
} catch (err) {
|
||||
console.error('[TmuxManager] Remote reconnect watcher error:', err);
|
||||
}
|
||||
}, intervalMs);
|
||||
}
|
||||
|
||||
stopRemoteReconnectWatcher(): void {
|
||||
if (this.remoteReconnectInterval) {
|
||||
clearInterval(this.remoteReconnectInterval);
|
||||
this.remoteReconnectInterval = null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run ONE watcher tick. Extracted (and given an injected `now`/`enabled`) so
|
||||
* the reconnect logic is deterministically testable even though the live
|
||||
* `setInterval` body no-ops under test mode. For each remote session it
|
||||
* applies the pure {@link decideReconnect} decision and translates the result
|
||||
* into events + backoff/state transitions. Public for tests + the watcher.
|
||||
*/
|
||||
runRemoteReconnectTick(now: number, enabled: boolean): void {
|
||||
for (const session of this.sessions.values()) {
|
||||
if (!session.remote) continue;
|
||||
const sessionId = session.sessionId;
|
||||
const state = this.reconnectState.get(sessionId);
|
||||
const action = decideReconnect({
|
||||
session: {
|
||||
sessionId,
|
||||
isRemote: true,
|
||||
paneDead: this.isPaneDead(session.muxName),
|
||||
},
|
||||
state,
|
||||
guarded: this.reconnectGuard.has(sessionId),
|
||||
enabled,
|
||||
now,
|
||||
});
|
||||
|
||||
if (action.kind === 'emit') {
|
||||
const base = state ?? freshReconnectState();
|
||||
// Mark in-flight + advance backoff BEFORE emitting so a re-entrant tick
|
||||
// (or a synchronous listener) can never stack a second reconnect.
|
||||
this.reconnectState.set(sessionId, { ...advanceBackoff(base, now), inFlight: true });
|
||||
this.emit('remoteSessionDropped', { sessionId, attempt: action.attempt });
|
||||
} else if (action.kind === 'exhaust') {
|
||||
const base = state ?? freshReconnectState();
|
||||
if (!base.exhaustedEmitted) {
|
||||
this.reconnectState.set(sessionId, { ...base, exhausted: true, exhaustedEmitted: true });
|
||||
this.emit('remoteReconnectExhausted', { sessionId });
|
||||
}
|
||||
}
|
||||
// 'skip' → nothing to do.
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Tell the watcher a reattach attempt for `sessionId` finished. On success,
|
||||
* reset the backoff so the session is healthy again; on failure, just clear
|
||||
* the in-flight flag so the next due tick can retry under the existing
|
||||
* backoff schedule. Called by the session owner after `respawnPane`.
|
||||
*/
|
||||
noteRemoteReconnect(sessionId: string, success: boolean): void {
|
||||
if (success) {
|
||||
this.reconnectState.set(sessionId, resetReconnectState());
|
||||
return;
|
||||
}
|
||||
const state = this.reconnectState.get(sessionId);
|
||||
if (state) this.reconnectState.set(sessionId, { ...state, inFlight: false });
|
||||
}
|
||||
|
||||
/**
|
||||
* Exclude a session from auto-reconnect (intentional teardown). Adds it to the
|
||||
* guard set and drops any backoff state so a closed/killed tab — especially a
|
||||
* non-owned remote DETACH — is never auto-revived. Idempotent.
|
||||
*/
|
||||
guardRemoteReconnect(sessionId: string): void {
|
||||
this.reconnectGuard.add(sessionId);
|
||||
this.reconnectState.delete(sessionId);
|
||||
}
|
||||
|
||||
/** Clear all per-session reconnect + guard state (e.g. when a session is removed). */
|
||||
clearRemoteReconnectState(sessionId: string): void {
|
||||
this.reconnectState.delete(sessionId);
|
||||
this.reconnectGuard.delete(sessionId);
|
||||
}
|
||||
|
||||
destroy(): void {
|
||||
this.stopStatsCollection();
|
||||
this.stopMouseModeSync();
|
||||
this.stopRemoteReconnectWatcher();
|
||||
this.reconnectState.clear();
|
||||
this.reconnectGuard.clear();
|
||||
}
|
||||
|
||||
registerSession(session: MuxSession): void {
|
||||
|
||||
@@ -99,6 +99,45 @@ export interface SessionRemote extends RemoteSshOptions {
|
||||
port?: number;
|
||||
remotePath: string;
|
||||
commands?: Partial<Record<RemoteCommandMode, string>>;
|
||||
/**
|
||||
* COD-105 — whether THIS Codeman created the remote tmux session.
|
||||
*
|
||||
* - `true` (default for COD-104 launched sessions): we own the remote session;
|
||||
* an explicit "kill" may propagate a remote `tmux kill-session`.
|
||||
* - `false` (discovered + attached an existing remote session another Codeman
|
||||
* created): closing the local tab must DETACH only — we must NEVER issue a
|
||||
* remote `kill-session`, or we'd nuke work the remote's own Codeman (or
|
||||
* another instance) still relies on. See `killSession()` gate.
|
||||
*
|
||||
* Absent is treated as owned (legacy/COD-104 sessions persisted before this
|
||||
* field existed were all launched by us).
|
||||
*/
|
||||
owned?: boolean;
|
||||
/**
|
||||
* COD-105 — for a NON-owned (discovered + attached) session, the EXISTING
|
||||
* remote tmux session name to `attach -t` (e.g. `codeman-disco1`). It differs
|
||||
* from this Codeman's deterministic `codeman-<id>` name because the remote
|
||||
* session was created elsewhere. Only meaningful when `owned === false`.
|
||||
*/
|
||||
remoteSessionName?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-105 — a `codeman-*` tmux session discovered on a remote host's
|
||||
* `tmux -L codeman` socket (may have been created by the remote's own Codeman,
|
||||
* another instance, or this one). Returned by `listRemoteCodemanSessions`.
|
||||
*/
|
||||
export interface RemoteSessionInfo {
|
||||
/** tmux session name (always starts `codeman-`). */
|
||||
name: string;
|
||||
/** Whether at least one client is currently attached to the remote session. */
|
||||
attached: boolean;
|
||||
/** COD-106 — number of clients attached (tmux `session_attached`); >1 = shared. */
|
||||
attachedClients: number;
|
||||
/** tmux `session_created` epoch seconds. */
|
||||
created: number;
|
||||
/** Number of windows in the remote session. */
|
||||
windows: number;
|
||||
}
|
||||
|
||||
// ========== Docker cases (COD-Docker) ==========
|
||||
|
||||
@@ -216,6 +216,10 @@ const _SSE_HANDLER_MAP = [
|
||||
[SSE_EVENTS.MUX_DIED, '_onMuxDied'],
|
||||
[SSE_EVENTS.MUX_STATS_UPDATED, '_onMuxStatsUpdated'],
|
||||
|
||||
// Remote auto-reconnect (COD-108)
|
||||
[SSE_EVENTS.REMOTE_SESSION_RECONNECTED, '_onRemoteSessionReconnected'],
|
||||
[SSE_EVENTS.REMOTE_RECONNECT_EXHAUSTED, '_onRemoteReconnectExhausted'],
|
||||
|
||||
// Ralph
|
||||
[SSE_EVENTS.SESSION_RALPH_LOOP_UPDATE, '_onRalphLoopUpdate'],
|
||||
[SSE_EVENTS.SESSION_RALPH_TODO_UPDATE, '_onRalphTodoUpdate'],
|
||||
|
||||
@@ -379,6 +379,11 @@ const SSE_EVENTS = {
|
||||
MUX_DIED: 'mux:died',
|
||||
MUX_STATS_UPDATED: 'mux:statsUpdated',
|
||||
|
||||
// Remote auto-reconnect (COD-108)
|
||||
REMOTE_SESSION_DROPPED: 'remote:sessionDropped',
|
||||
REMOTE_SESSION_RECONNECTED: 'remote:sessionReconnected',
|
||||
REMOTE_RECONNECT_EXHAUSTED: 'remote:reconnectExhausted',
|
||||
|
||||
// Ralph
|
||||
SESSION_RALPH_LOOP_UPDATE: 'session:ralphLoopUpdate',
|
||||
SESSION_RALPH_TODO_UPDATE: 'session:ralphTodoUpdate',
|
||||
|
||||
@@ -1501,6 +1501,14 @@
|
||||
</label>
|
||||
<span class="form-hint">Use 1M token context window (model: opus[1m]) for all new sessions — ignored when a Claude Model is selected above</span>
|
||||
</div>
|
||||
<div class="form-row form-row-switch">
|
||||
<label>Remote auto-reconnect</label>
|
||||
<label class="switch">
|
||||
<input type="checkbox" id="appSettingsRemoteAutoReconnect">
|
||||
<span class="slider"></span>
|
||||
</label>
|
||||
<span class="form-hint">Automatically re-establish remote (SSH) sessions when the connection drops, reattaching to the durable remote tmux session (on by default; bounded backoff)</span>
|
||||
</div>
|
||||
<div class="form-row">
|
||||
<label>Thinking Effort</label>
|
||||
<select id="appSettingsThinkingEffort" class="form-select">
|
||||
@@ -1993,6 +2001,17 @@
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
<!-- COD-105 — discover + attach existing remote tmux sessions this Codeman didn't create. -->
|
||||
<details class="advanced-options" id="remoteDiscoverSection">
|
||||
<summary>Discover existing sessions</summary>
|
||||
<div class="advanced-options-content">
|
||||
<span class="form-hint">Find <code>codeman-*</code> tmux sessions already running on this host (started by the remote's own Codeman or another instance) and attach to one. Attaching shares the session; closing the tab detaches it — it is never killed.</span>
|
||||
<div class="form-row" style="margin-top: 8px;">
|
||||
<button type="button" class="btn-toolbar" id="remoteDiscoverBtn" onclick="app.discoverRemoteSessions()">Discover existing sessions</button>
|
||||
</div>
|
||||
<div id="remoteDiscoverResults" class="remote-discover-results"></div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
<!-- Docker Tab -->
|
||||
<div class="modal-tab-content hidden" id="case-docker">
|
||||
|
||||
@@ -82,6 +82,33 @@ Object.assign(CodemanApp.prototype, {
|
||||
}
|
||||
},
|
||||
|
||||
// Remote auto-reconnect (COD-108)
|
||||
_onRemoteSessionReconnected(data) {
|
||||
const id = this.getShortId(data.sessionId);
|
||||
this.showToast(`Remote session ${id} reconnected`, 'success');
|
||||
},
|
||||
|
||||
_onRemoteReconnectExhausted(data) {
|
||||
const sessionId = data.sessionId;
|
||||
const id = this.getShortId(sessionId);
|
||||
// Auto-reconnect gave up after the bounded backoff. Surface a manual
|
||||
// "Reconnect" affordance that re-triggers the attach path (force-reload the
|
||||
// session, which re-runs the create/attach flow against the durable remote).
|
||||
this.showToast(`Remote session ${id} dropped — auto-reconnect gave up`, 'error', {
|
||||
duration: 15000,
|
||||
action: {
|
||||
label: 'Reconnect',
|
||||
onClick: () => {
|
||||
if (this.sessions && this.sessions.has(sessionId)) {
|
||||
this.selectSession(sessionId, { forceReload: true });
|
||||
} else {
|
||||
this.showToast('Session no longer available', 'warning');
|
||||
}
|
||||
},
|
||||
},
|
||||
});
|
||||
},
|
||||
|
||||
|
||||
// Bash tools
|
||||
_onBashToolStart(data) {
|
||||
|
||||
@@ -2068,6 +2068,156 @@ Object.assign(CodemanApp.prototype, {
|
||||
}
|
||||
},
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
// COD-105 — Discover + attach existing remote tmux sessions
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
|
||||
/** Read the remote-host fields from the remote-case form into a host payload. */
|
||||
_readRemoteHostFromForm() {
|
||||
const hostId = document.getElementById('remoteHostId').value.trim();
|
||||
const host = document.getElementById('remoteHostAddress').value.trim();
|
||||
const username = document.getElementById('remoteHostUsername').value.trim();
|
||||
const portRaw = document.getElementById('remoteHostPort').value.trim();
|
||||
const identityFile = document.getElementById('remoteHostIdentityFile').value.trim();
|
||||
const socksProxy = document.getElementById('remoteHostSocksProxy').value.trim();
|
||||
const jumpHost = document.getElementById('remoteHostJumpHost').value.trim();
|
||||
const codexCommand = document.getElementById('remoteHostCodexCommand').value.trim();
|
||||
const extraSshOptions = document.getElementById('remoteHostExtraSshOptions').value
|
||||
.split('\n')
|
||||
.map(line => line.trim())
|
||||
.filter(line => line.length > 0);
|
||||
let port;
|
||||
if (portRaw) {
|
||||
const n = Number(portRaw);
|
||||
if (Number.isInteger(n) && n >= 1 && n <= 65535) port = n;
|
||||
}
|
||||
return {
|
||||
id: hostId,
|
||||
label: hostId,
|
||||
host,
|
||||
username,
|
||||
...(port ? { port } : {}),
|
||||
...(identityFile ? { identityFile } : {}),
|
||||
...(socksProxy ? { socksProxy } : {}),
|
||||
...(jumpHost ? { jumpHost } : {}),
|
||||
...(extraSshOptions.length ? { extraSshOptions } : {}),
|
||||
...(codexCommand ? { commands: { codex: codexCommand } } : {}),
|
||||
};
|
||||
},
|
||||
|
||||
/**
|
||||
* Explicit Discover action (Decision A — never auto-runs on host select).
|
||||
* Saves the host config (idempotent), then queries the host for `codeman-*`
|
||||
* tmux sessions it didn't create and renders an Attach action per session.
|
||||
*/
|
||||
async discoverRemoteSessions() {
|
||||
const results = document.getElementById('remoteDiscoverResults');
|
||||
const btn = document.getElementById('remoteDiscoverBtn');
|
||||
const hostPayload = this._readRemoteHostFromForm();
|
||||
if (!hostPayload.id || !hostPayload.host || !hostPayload.username) {
|
||||
this.showToast('Fill in Host ID, address, and username first', 'error');
|
||||
return;
|
||||
}
|
||||
if (!/^[a-zA-Z0-9_-]+$/.test(hostPayload.id)) {
|
||||
this.showToast('Invalid Host ID. Use letters, numbers, hyphens, underscores.', 'error');
|
||||
return;
|
||||
}
|
||||
if (btn) btn.disabled = true;
|
||||
if (results) results.innerHTML = '<div class="form-hint">Discovering…</div>';
|
||||
try {
|
||||
// Persist the host so the discovery endpoint can resolve it by id (idempotent).
|
||||
const hostRes = await fetch('/api/remote-hosts', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(hostPayload)
|
||||
});
|
||||
const hostData = await hostRes.json();
|
||||
if (!hostData.success && hostData.errorCode !== 'ALREADY_EXISTS') {
|
||||
throw new Error(hostData.error || 'Failed to save remote host');
|
||||
}
|
||||
const res = await fetch(`/api/remote-hosts/${encodeURIComponent(hostPayload.id)}/sessions`);
|
||||
const data = await res.json();
|
||||
if (!data.success) throw new Error(data.error || 'Discovery failed');
|
||||
this._renderDiscoveredSessions(hostPayload.id, data.data.sessions || []);
|
||||
} catch (err) {
|
||||
console.error('Discover remote sessions failed:', err);
|
||||
if (results) results.innerHTML = `<div class="form-hint" style="color: var(--error, #e06c75);">${escapeHtml(err.message)}</div>`;
|
||||
} finally {
|
||||
if (btn) btn.disabled = false;
|
||||
}
|
||||
},
|
||||
|
||||
/** Render the discovered remote sessions with an Attach action each. */
|
||||
_renderDiscoveredSessions(hostId, sessions) {
|
||||
const results = document.getElementById('remoteDiscoverResults');
|
||||
if (!results) return;
|
||||
if (!sessions.length) {
|
||||
results.innerHTML = '<div class="form-hint">No <code>codeman-*</code> sessions running on this host (or it is unreachable).</div>';
|
||||
return;
|
||||
}
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
const rows = sessions.map(s => {
|
||||
const ageSecs = Math.max(0, now - (s.created || 0));
|
||||
const age = ageSecs < 3600 ? `${Math.floor(ageSecs / 60)}m` : ageSecs < 86400 ? `${Math.floor(ageSecs / 3600)}h` : `${Math.floor(ageSecs / 86400)}d`;
|
||||
// COD-106 — show "shared · N clients" when more than one client is attached
|
||||
// (genuinely collaborative), else a plain "attached" badge for a single client.
|
||||
const clients = s.attachedClients != null ? s.attachedClients : s.attached ? 1 : 0;
|
||||
const attachedBadge =
|
||||
clients > 1
|
||||
? `<span class="case-location-badge" style="background: var(--warning, #e5c07b); color: #000;">shared · ${clients} clients</span>`
|
||||
: clients === 1
|
||||
? '<span class="case-location-badge" style="background: var(--accent, #61afef);">attached</span>'
|
||||
: '';
|
||||
return `
|
||||
<div class="remote-discover-item">
|
||||
<div class="remote-discover-info">
|
||||
<span class="remote-discover-name">${escapeHtml(s.name)} ${attachedBadge}</span>
|
||||
<span class="form-hint">age ${age} · ${s.windows || 1} window(s)</span>
|
||||
</div>
|
||||
<button type="button" class="btn-toolbar" onclick="app.attachDiscoveredSession('${escapeHtml(hostId)}', '${escapeHtml(s.name)}')">Attach</button>
|
||||
</div>`;
|
||||
}).join('');
|
||||
results.innerHTML = rows;
|
||||
},
|
||||
|
||||
/**
|
||||
* Create a NON-owned session that attaches to a discovered remote tmux session.
|
||||
* Closing this tab detaches — it never kills the remote session.
|
||||
*/
|
||||
async attachDiscoveredSession(hostId, remoteSessionName) {
|
||||
try {
|
||||
const createRes = await fetch('/api/sessions', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
mode: 'shell',
|
||||
name: remoteSessionName,
|
||||
attachRemoteSession: { hostId, remoteSessionName },
|
||||
})
|
||||
});
|
||||
const createData = await createRes.json();
|
||||
if (!createData.success) throw new Error(createData.error || 'Failed to create session');
|
||||
const id = createData.data.session.id;
|
||||
await fetch(`/api/sessions/${id}/shell`, { method: 'POST' });
|
||||
const dims = this.getTerminalDimensions();
|
||||
if (dims) {
|
||||
await fetch(`/api/sessions/${id}/resize`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(dims)
|
||||
});
|
||||
}
|
||||
this.closeCreateCaseModal();
|
||||
this.showToast(`Attached to ${remoteSessionName} (detach on close)`, 'success');
|
||||
this.activeSessionId = id;
|
||||
await this.selectSession(id);
|
||||
if (this.terminal && typeof this.terminal.focus === 'function') this.terminal.focus();
|
||||
} catch (err) {
|
||||
console.error('Attach discovered session failed:', err);
|
||||
this.showToast('Failed to attach: ' + err.message, 'error');
|
||||
}
|
||||
},
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
// Case Management (reorder + delete)
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
|
||||
@@ -364,6 +364,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
document.getElementById('appSettingsAgentTeams').checked = settings.agentTeamsEnabled ?? false;
|
||||
document.getElementById('appSettingsClaudeModel').value = settings.claudeModel ?? '';
|
||||
document.getElementById('appSettingsOpusContext1m').checked = settings.opusContext1mEnabled ?? false;
|
||||
document.getElementById('appSettingsRemoteAutoReconnect').checked = settings.remoteAutoReconnect ?? true;
|
||||
document.getElementById('appSettingsThinkingEffort').value = settings.thinkingEffort ?? '';
|
||||
// CPU Priority settings
|
||||
const niceSettings = settings.nice || {};
|
||||
@@ -1462,6 +1463,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
agentTeamsEnabled: document.getElementById('appSettingsAgentTeams').checked,
|
||||
claudeModel: document.getElementById('appSettingsClaudeModel').value,
|
||||
opusContext1mEnabled: document.getElementById('appSettingsOpusContext1m').checked,
|
||||
remoteAutoReconnect: document.getElementById('appSettingsRemoteAutoReconnect').checked,
|
||||
thinkingEffort: document.getElementById('appSettingsThinkingEffort').value,
|
||||
// CPU Priority settings
|
||||
nice: {
|
||||
@@ -1789,6 +1791,8 @@ Object.assign(CodemanApp.prototype, {
|
||||
showSessionButton: false,
|
||||
showAwayDigestButton: false,
|
||||
showCronButton: true,
|
||||
// Remote auto-reconnect (COD-108) — on by default
|
||||
remoteAutoReconnect: true,
|
||||
// Input
|
||||
gestureControlEnabled: false,
|
||||
// Feature toggles - keep tracking on even on mobile
|
||||
|
||||
@@ -3933,6 +3933,37 @@ body.touch-device .terminal-container .xterm .xterm-helper-textarea {
|
||||
transition: background var(--transition-smooth);
|
||||
}
|
||||
|
||||
/* COD-105 — discovered remote tmux sessions list (remote-case flow). */
|
||||
.remote-discover-results {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
margin-top: 8px;
|
||||
}
|
||||
|
||||
.remote-discover-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 8px;
|
||||
padding: 8px 10px;
|
||||
background: rgba(255, 255, 255, 0.03);
|
||||
border: 1px solid rgba(255, 255, 255, 0.06);
|
||||
border-radius: 6px;
|
||||
}
|
||||
|
||||
.remote-discover-info {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 2px;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.remote-discover-name {
|
||||
font-weight: 600;
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
|
||||
.case-manage-item:hover {
|
||||
background: rgba(255, 255, 255, 0.06);
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ import fs from 'node:fs/promises';
|
||||
import { join, resolve, basename } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { homedir } from 'node:os';
|
||||
import type { ApiResponse, CaseInfo, DockerHost, SessionDocker } from '../../types.js';
|
||||
import type { ApiResponse, CaseInfo, DockerHost, RemoteSessionInfo, SessionDocker } from '../../types.js';
|
||||
import { ApiErrorCode, createErrorResponse, getErrorMessage } from '../../types.js';
|
||||
import {
|
||||
CreateCaseSchema,
|
||||
@@ -64,6 +64,7 @@ import {
|
||||
import { buildDockerRemoveCommand } from '../../tmux-manager.js';
|
||||
import {
|
||||
checkRemoteTmuxAvailable,
|
||||
listRemoteCodemanSessions,
|
||||
readRemoteCases,
|
||||
readRemoteHosts,
|
||||
remoteDisplayPath,
|
||||
@@ -312,6 +313,26 @@ export function registerCaseRoutes(app: FastifyInstance, ctx: EventPort & Config
|
||||
? null
|
||||
: (reply.code(403), createErrorResponse(ApiErrorCode.FORBIDDEN, 'Admin only in multi-user mode'));
|
||||
|
||||
// COD-105 — discover `codeman-*` tmux sessions already running on a remote
|
||||
// host (created by the remote's own Codeman, another instance, or this one)
|
||||
// so the operator can attach to one this Codeman didn't launch. Explicit
|
||||
// trigger only (Decision A): the frontend calls this on a "Discover" click,
|
||||
// never automatically on host select. listRemoteCodemanSessions never throws
|
||||
// (returns [] on unreachable/no-tmux/no-sessions) and is ssh-guarded under test.
|
||||
// Hosts are admin-only infra in multi-user mode, so discovery is too.
|
||||
app.get(
|
||||
'/api/remote-hosts/:hostId/sessions',
|
||||
async (req, reply): Promise<ApiResponse<{ sessions: RemoteSessionInfo[] }>> => {
|
||||
const denied = adminOnly(req, reply);
|
||||
if (denied) return denied;
|
||||
const { hostId } = req.params as { hostId: string };
|
||||
const host = (await readRemoteHosts(CODEMAN_CONFIG_DIR)).find((item) => item.id === hostId);
|
||||
if (!host) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Remote host not found');
|
||||
const sessions = await listRemoteCodemanSessions(host);
|
||||
return { success: true, data: { sessions } };
|
||||
}
|
||||
);
|
||||
|
||||
app.post('/api/remote-hosts', async (req, reply): Promise<ApiResponse<{ host: unknown }>> => {
|
||||
const denied = adminOnly(req, reply);
|
||||
if (denied) return denied;
|
||||
|
||||
@@ -83,7 +83,13 @@ import { RunSummaryTracker } from '../../run-summary.js';
|
||||
import { MAX_INPUT_LENGTH, MAX_SESSION_NAME_LENGTH } from '../../config/terminal-limits.js';
|
||||
import { MAX_PASTE_IMAGE_BYTES } from '../../config/buffer-limits.js';
|
||||
import { dataPath, getDataDir } from '../../config/instance.js';
|
||||
import { checkRemoteTmuxAvailable, readRemoteCases, readRemoteHosts, toSessionRemote } from '../../remote-hosts.js';
|
||||
import {
|
||||
checkRemoteTmuxAvailable,
|
||||
readRemoteCases,
|
||||
readRemoteHosts,
|
||||
toAttachedSessionRemote,
|
||||
toSessionRemote,
|
||||
} from '../../remote-hosts.js';
|
||||
import {
|
||||
checkDockerAvailable,
|
||||
checkDockerConfigDrift,
|
||||
@@ -337,7 +343,21 @@ export function registerSessionRoutes(
|
||||
if (capMsg) return createErrorResponse(ApiErrorCode.OPERATION_FAILED, capMsg);
|
||||
|
||||
const body = parseBody(CreateSessionSchema, req.body);
|
||||
const workingDir = body.workingDir || process.cwd();
|
||||
let workingDir = body.workingDir || process.cwd();
|
||||
let remote = undefined;
|
||||
|
||||
// COD-105 — attach to a discovered (non-owned) remote tmux session. The
|
||||
// remote session is already running, so we skip the tmux-prereq probe and
|
||||
// build a NON-owned SessionRemote (detach-not-kill on close). Remote CASE
|
||||
// creation (owned durable sessions) is handled by the dedicated case-create
|
||||
// endpoint below, which #145 consolidated remote-host resolution into.
|
||||
if (body.attachRemoteSession) {
|
||||
const { hostId, remoteSessionName } = body.attachRemoteSession;
|
||||
const host = (await readRemoteHosts(CODEMAN_CONFIG_DIR)).find((item) => item.id === hostId);
|
||||
if (!host) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Remote host not found');
|
||||
workingDir = `${host.username}@${host.host}:${remoteSessionName}`;
|
||||
remote = toAttachedSessionRemote(host, remoteSessionName, workingDir);
|
||||
}
|
||||
|
||||
// Multi-user: shell mode is arbitrary command execution as the host account,
|
||||
// gated behind the same grant as bypass (section 6.3). Resolve the owner's grant
|
||||
@@ -516,6 +536,7 @@ export function registerSessionRoutes(
|
||||
envOverrides: body.envOverrides,
|
||||
effort: body.effort,
|
||||
tmuxHistoryLimit: terminalHistoryConfig.tmuxHistoryLimit,
|
||||
remote,
|
||||
owner,
|
||||
});
|
||||
|
||||
|
||||
@@ -192,6 +192,22 @@ export const CreateSessionSchema = z.object({
|
||||
.max(100)
|
||||
.regex(/^[a-f0-9-]+$/, 'resumeSessionId must be a valid UUID')
|
||||
.optional(),
|
||||
/**
|
||||
* COD-105 — attach to an EXISTING remote tmux session discovered via
|
||||
* `GET /api/remote-hosts/:hostId/sessions` (one this Codeman didn't create).
|
||||
* The resulting session is NON-owned (closing it detaches, never kills the
|
||||
* remote). `remoteSessionName` is a discovered `codeman-*` tmux session name.
|
||||
*/
|
||||
attachRemoteSession: z
|
||||
.object({
|
||||
hostId: z.string().min(1).max(200),
|
||||
remoteSessionName: z
|
||||
.string()
|
||||
.min(1)
|
||||
.max(200)
|
||||
.regex(/^codeman-[a-zA-Z0-9._-]+$/, 'remoteSessionName must be a codeman-* tmux session name'),
|
||||
})
|
||||
.optional(),
|
||||
});
|
||||
|
||||
/**
|
||||
@@ -658,6 +674,10 @@ export const SettingsUpdateSchema = z
|
||||
/** Model for new Claude sessions (e.g. "claude-fable-5[1m]", "opus[1m]"); takes precedence over opusContext1mEnabled */
|
||||
claudeModel: z.string().max(50).optional(),
|
||||
opusContext1mEnabled: z.boolean().optional(),
|
||||
// COD-108 remote-session auto-reconnect kill-switch (default ON). When false,
|
||||
// the TmuxManager watcher does nothing — dropped remote sessions are NOT
|
||||
// auto-reattached.
|
||||
remoteAutoReconnect: z.boolean().optional(),
|
||||
thinkingEffort: z.string().max(20).optional(),
|
||||
// UI visibility
|
||||
showFontControls: z.boolean().optional(),
|
||||
|
||||
@@ -359,6 +359,22 @@ export class WebServer extends EventEmitter {
|
||||
this.broadcast(SseEvent.MuxStatsUpdated, sessions);
|
||||
});
|
||||
|
||||
// COD-108 — remote-session auto-reconnect. The TmuxManager watcher detects a
|
||||
// dead remote pane and emits `remoteSessionDropped`; the session owner (here)
|
||||
// reassembles the respawn options and reattaches via Session.reattachRemote()
|
||||
// (D1: the watcher does NOT reassemble options itself). On success we reset
|
||||
// the watcher's backoff; on failure the backoff schedules the next attempt.
|
||||
this.mux.on('remoteSessionDropped', (data) => {
|
||||
const { sessionId, attempt } = data as { sessionId: string; attempt: number };
|
||||
this.broadcast(SseEvent.RemoteSessionDropped, { sessionId, attempt });
|
||||
void this.handleRemoteSessionDropped(sessionId);
|
||||
});
|
||||
this.mux.on('remoteReconnectExhausted', (data) => {
|
||||
const { sessionId } = data as { sessionId: string };
|
||||
console.warn(`[Server] Remote auto-reconnect exhausted for session ${sessionId}`);
|
||||
this.broadcast(SseEvent.RemoteReconnectExhausted, { sessionId });
|
||||
});
|
||||
|
||||
// Set up subagent watcher listeners
|
||||
this.setupSubagentWatcherListeners();
|
||||
this.setupWorkflowRunWatcherListeners();
|
||||
@@ -2603,6 +2619,13 @@ export class WebServer extends EventEmitter {
|
||||
(this.mux as { startMouseModeSync: (ms?: number) => void }).startMouseModeSync();
|
||||
}
|
||||
|
||||
// COD-108 — start the remote-session auto-reconnect watcher (tmux only).
|
||||
// Always-on (D3) with a `remoteAutoReconnect` kill-switch the watcher reads
|
||||
// each tick. Start even with no sessions — remote sessions may arrive later.
|
||||
if ('startRemoteReconnectWatcher' in this.mux) {
|
||||
(this.mux as { startRemoteReconnectWatcher: (ms?: number) => void }).startRemoteReconnectWatcher();
|
||||
}
|
||||
|
||||
if (dead.length > 0) {
|
||||
console.log(`[Server] Cleaned up ${dead.length} dead mux session(s)`);
|
||||
}
|
||||
@@ -2611,6 +2634,41 @@ export class WebServer extends EventEmitter {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* COD-108 — handle a `remoteSessionDropped` emit from the watcher: reattach
|
||||
* the dropped remote session and report the outcome back to the watcher so it
|
||||
* can reset/advance its backoff. Re-running the idempotent remote command
|
||||
* REATTACHES the durable remote tmux session (does NOT recreate it).
|
||||
*/
|
||||
private async handleRemoteSessionDropped(sessionId: string): Promise<void> {
|
||||
const session = this.sessions.get(sessionId);
|
||||
// No live Session object (e.g. detached/restored-but-not-attached) — nothing
|
||||
// to drive the reattach; report failure so the watcher backs off and retries.
|
||||
if (!session) {
|
||||
this.noteRemoteReconnect(sessionId, false);
|
||||
return;
|
||||
}
|
||||
let ok = false;
|
||||
try {
|
||||
ok = await session.reattachRemote();
|
||||
} catch (err) {
|
||||
console.error(`[Server] Remote reattach failed for ${sessionId}:`, err);
|
||||
ok = false;
|
||||
}
|
||||
this.noteRemoteReconnect(sessionId, ok);
|
||||
if (ok) {
|
||||
this.persistSessionState(session);
|
||||
this.broadcast(SseEvent.RemoteSessionReconnected, { sessionId });
|
||||
}
|
||||
}
|
||||
|
||||
/** Forward a reattach outcome to the TmuxManager watcher (resets/clears backoff). */
|
||||
private noteRemoteReconnect(sessionId: string, success: boolean): void {
|
||||
if ('noteRemoteReconnect' in this.mux) {
|
||||
(this.mux as { noteRemoteReconnect: (id: string, ok: boolean) => void }).noteRemoteReconnect(sessionId, success);
|
||||
}
|
||||
}
|
||||
|
||||
private initOrchestratorLoop(): import('../orchestrator-loop.js').OrchestratorLoop {
|
||||
if (this._orchestratorLoop) return this._orchestratorLoop;
|
||||
|
||||
|
||||
@@ -150,6 +150,15 @@ export const MuxDied = 'mux:died' as const;
|
||||
/** tmux session stats refreshed. */
|
||||
export const MuxStatsUpdated = 'mux:statsUpdated' as const;
|
||||
|
||||
// ─── Remote auto-reconnect (COD-108) ─────────────────────────────────────────
|
||||
|
||||
/** A remote session's local ssh pane died; an auto-reconnect attempt is starting. */
|
||||
export const RemoteSessionDropped = 'remote:sessionDropped' as const;
|
||||
/** A dropped remote session was successfully re-established (reattached). */
|
||||
export const RemoteSessionReconnected = 'remote:sessionReconnected' as const;
|
||||
/** Auto-reconnect gave up after the bounded backoff cap — manual reconnect needed. */
|
||||
export const RemoteReconnectExhausted = 'remote:reconnectExhausted' as const;
|
||||
|
||||
// ─── Respawn ─────────────────────────────────────────────────────────────────
|
||||
|
||||
/** Respawn loop started for a session. */
|
||||
@@ -460,6 +469,11 @@ export const SseEvent = {
|
||||
MuxDied,
|
||||
MuxStatsUpdated,
|
||||
|
||||
// Remote auto-reconnect (COD-108)
|
||||
RemoteSessionDropped,
|
||||
RemoteSessionReconnected,
|
||||
RemoteReconnectExhausted,
|
||||
|
||||
// Respawn
|
||||
RespawnStarted,
|
||||
RespawnStopped,
|
||||
|
||||
@@ -0,0 +1,298 @@
|
||||
/**
|
||||
* @fileoverview COD-108 — remote-session auto-reconnect watcher tests.
|
||||
*
|
||||
* Three layers, all tmux-safe (under VITEST TmuxManager no-ops real tmux and
|
||||
* `isPaneDead` returns false, so live behavior is driven via injected stubs):
|
||||
*
|
||||
* (a) PURE backoff schedule — attempt→delay, cap, reset-on-success.
|
||||
* (b) PURE eligibility decision — dead remote pane + due → emit; guarded →
|
||||
* never; non-remote / pane-alive / not-due → skip; over-cap → exhaust.
|
||||
* (c) MANAGER integration — drive ticks with a stubbed pane-death signal +
|
||||
* controllable clock and assert the emit → backoff → exhausted progression,
|
||||
* and that a guarded (intentionally-killed) session emits nothing.
|
||||
*
|
||||
* Port: N/A (no server).
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, vi } from 'vitest';
|
||||
import {
|
||||
BACKOFF_SCHEDULE_MS,
|
||||
MAX_RECONNECT_ATTEMPTS,
|
||||
reconnectDelayForAttempt,
|
||||
isExhausted,
|
||||
freshReconnectState,
|
||||
advanceBackoff,
|
||||
resetReconnectState,
|
||||
decideReconnect,
|
||||
} from '../src/remote-reconnect.js';
|
||||
import type { ReconnectSessionView } from '../src/remote-reconnect.js';
|
||||
import { TmuxManager } from '../src/tmux-manager.js';
|
||||
import type { SessionRemote } from '../src/types.js';
|
||||
|
||||
const REMOTE: SessionRemote = {
|
||||
hostId: 'aa-desktop',
|
||||
label: 'aa-desktop',
|
||||
host: 'aa-desktop',
|
||||
username: 'aakhter',
|
||||
remotePath: '/home/aakhter',
|
||||
owned: true,
|
||||
};
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
// (a) PURE backoff schedule
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
describe('reconnect backoff schedule (pure)', () => {
|
||||
it('returns the documented bounded exponential delays per attempt', () => {
|
||||
expect(reconnectDelayForAttempt(1)).toBe(5_000);
|
||||
expect(reconnectDelayForAttempt(2)).toBe(15_000);
|
||||
expect(reconnectDelayForAttempt(3)).toBe(45_000);
|
||||
expect(reconnectDelayForAttempt(4)).toBe(120_000);
|
||||
expect(reconnectDelayForAttempt(5)).toBe(300_000);
|
||||
expect(reconnectDelayForAttempt(6)).toBe(300_000);
|
||||
});
|
||||
|
||||
it('clamps below-range and above-range attempts to the schedule bounds', () => {
|
||||
expect(reconnectDelayForAttempt(0)).toBe(BACKOFF_SCHEDULE_MS[0]);
|
||||
expect(reconnectDelayForAttempt(-3)).toBe(BACKOFF_SCHEDULE_MS[0]);
|
||||
expect(reconnectDelayForAttempt(99)).toBe(BACKOFF_SCHEDULE_MS[BACKOFF_SCHEDULE_MS.length - 1]);
|
||||
});
|
||||
|
||||
it('flags exhaustion only at/after the cap', () => {
|
||||
expect(isExhausted(0)).toBe(false);
|
||||
expect(isExhausted(MAX_RECONNECT_ATTEMPTS - 1)).toBe(false);
|
||||
expect(isExhausted(MAX_RECONNECT_ATTEMPTS)).toBe(true);
|
||||
expect(isExhausted(MAX_RECONNECT_ATTEMPTS + 1)).toBe(true);
|
||||
});
|
||||
|
||||
it('advanceBackoff increments attempts and schedules next-eligible from now (immutable)', () => {
|
||||
const s0 = freshReconnectState();
|
||||
const s1 = advanceBackoff(s0, 1_000);
|
||||
expect(s0.attempts).toBe(0); // input untouched
|
||||
expect(s1.attempts).toBe(1);
|
||||
expect(s1.nextEligibleAt).toBe(1_000 + 5_000);
|
||||
expect(s1.exhausted).toBe(false);
|
||||
|
||||
const s2 = advanceBackoff(s1, 10_000);
|
||||
expect(s2.attempts).toBe(2);
|
||||
expect(s2.nextEligibleAt).toBe(10_000 + 15_000);
|
||||
});
|
||||
|
||||
it('reaches the attempt cap after the scheduled number of advances', () => {
|
||||
let s = freshReconnectState();
|
||||
let now = 0;
|
||||
for (let i = 0; i < MAX_RECONNECT_ATTEMPTS; i++) {
|
||||
s = advanceBackoff(s, now);
|
||||
now += reconnectDelayForAttempt(s.attempts);
|
||||
}
|
||||
expect(s.attempts).toBe(MAX_RECONNECT_ATTEMPTS);
|
||||
// advanceBackoff does not itself set `exhausted`; the watcher decides that
|
||||
// on the following tick via isExhausted(attempts).
|
||||
expect(isExhausted(s.attempts)).toBe(true);
|
||||
});
|
||||
|
||||
it('reset-on-success returns to a fresh, immediately-eligible state', () => {
|
||||
const advanced = advanceBackoff(advanceBackoff(freshReconnectState(), 0), 100);
|
||||
expect(advanced.attempts).toBe(2);
|
||||
const reset = resetReconnectState();
|
||||
expect(reset.attempts).toBe(0);
|
||||
expect(reset.nextEligibleAt).toBe(0);
|
||||
expect(reset.exhausted).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
// (b) PURE eligibility decision
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
describe('decideReconnect (pure eligibility)', () => {
|
||||
const deadRemote: ReconnectSessionView = { sessionId: 's1', isRemote: true, paneDead: true };
|
||||
|
||||
it('emits for a dead remote pane that is not guarded and is due', () => {
|
||||
const action = decideReconnect({
|
||||
session: deadRemote,
|
||||
state: freshReconnectState(),
|
||||
guarded: false,
|
||||
enabled: true,
|
||||
now: 0,
|
||||
});
|
||||
expect(action).toEqual({ kind: 'emit', attempt: 1 });
|
||||
});
|
||||
|
||||
it('NEVER reconnects a guarded (intentionally killed/detached) session', () => {
|
||||
const action = decideReconnect({
|
||||
session: deadRemote,
|
||||
state: freshReconnectState(),
|
||||
guarded: true,
|
||||
enabled: true,
|
||||
now: 0,
|
||||
});
|
||||
expect(action).toEqual({ kind: 'skip', reason: 'guarded' });
|
||||
});
|
||||
|
||||
it('skips non-remote sessions', () => {
|
||||
const action = decideReconnect({
|
||||
session: { sessionId: 's1', isRemote: false, paneDead: true },
|
||||
state: freshReconnectState(),
|
||||
guarded: false,
|
||||
enabled: true,
|
||||
now: 0,
|
||||
});
|
||||
expect(action).toEqual({ kind: 'skip', reason: 'not-remote' });
|
||||
});
|
||||
|
||||
it('skips when the pane is alive', () => {
|
||||
const action = decideReconnect({
|
||||
session: { sessionId: 's1', isRemote: true, paneDead: false },
|
||||
state: freshReconnectState(),
|
||||
guarded: false,
|
||||
enabled: true,
|
||||
now: 0,
|
||||
});
|
||||
expect(action).toEqual({ kind: 'skip', reason: 'pane-alive' });
|
||||
});
|
||||
|
||||
it('skips when the kill-switch is off', () => {
|
||||
const action = decideReconnect({
|
||||
session: deadRemote,
|
||||
state: freshReconnectState(),
|
||||
guarded: false,
|
||||
enabled: false,
|
||||
now: 0,
|
||||
});
|
||||
expect(action).toEqual({ kind: 'skip', reason: 'disabled' });
|
||||
});
|
||||
|
||||
it('skips when not yet due (within backoff window)', () => {
|
||||
const state = advanceBackoff(freshReconnectState(), 0); // nextEligibleAt = 5000
|
||||
const action = decideReconnect({ session: deadRemote, state, guarded: false, enabled: true, now: 4_999 });
|
||||
expect(action).toEqual({ kind: 'skip', reason: 'not-due' });
|
||||
});
|
||||
|
||||
it('emits again once the backoff window elapses', () => {
|
||||
const state = advanceBackoff(freshReconnectState(), 0); // nextEligibleAt = 5000
|
||||
const action = decideReconnect({ session: deadRemote, state, guarded: false, enabled: true, now: 5_000 });
|
||||
expect(action).toEqual({ kind: 'emit', attempt: 2 });
|
||||
});
|
||||
|
||||
it('skips while a reconnect is already in flight (no stacked respawns)', () => {
|
||||
const state = { ...freshReconnectState(), inFlight: true };
|
||||
const action = decideReconnect({ session: deadRemote, state, guarded: false, enabled: true, now: 10_000 });
|
||||
expect(action).toEqual({ kind: 'skip', reason: 'in-flight' });
|
||||
});
|
||||
|
||||
it('exhausts once the attempt cap is reached', () => {
|
||||
const state = { ...freshReconnectState(), attempts: MAX_RECONNECT_ATTEMPTS, nextEligibleAt: 0 };
|
||||
const action = decideReconnect({ session: deadRemote, state, guarded: false, enabled: true, now: 1_000_000 });
|
||||
expect(action).toEqual({ kind: 'exhaust' });
|
||||
});
|
||||
|
||||
it('stays quiet after exhaustion has been recorded', () => {
|
||||
const state = { ...freshReconnectState(), attempts: MAX_RECONNECT_ATTEMPTS, exhausted: true };
|
||||
const action = decideReconnect({ session: deadRemote, state, guarded: false, enabled: true, now: 1_000_000 });
|
||||
expect(action).toEqual({ kind: 'skip', reason: 'exhausted' });
|
||||
});
|
||||
});
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
// (c) MANAGER integration — drive ticks with a stubbed pane-death + clock
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
describe('TmuxManager remote reconnect watcher (integration)', () => {
|
||||
let manager: TmuxManager;
|
||||
|
||||
beforeEach(() => {
|
||||
manager = new TmuxManager();
|
||||
});
|
||||
|
||||
function registerRemote(sessionId: string): void {
|
||||
manager.registerSession({
|
||||
sessionId,
|
||||
muxName: `codeman-${sessionId}`,
|
||||
pid: 4242,
|
||||
createdAt: Date.now(),
|
||||
workingDir: '/home/aakhter',
|
||||
mode: 'shell',
|
||||
attached: true,
|
||||
remote: REMOTE,
|
||||
});
|
||||
}
|
||||
|
||||
it('emits remoteSessionDropped when a dead remote pane is observed, then backs off and exhausts', () => {
|
||||
registerRemote('aaaa1111');
|
||||
// Force the watcher to see a dead pane regardless of test-mode isPaneDead.
|
||||
vi.spyOn(manager, 'isPaneDead').mockReturnValue(true);
|
||||
|
||||
const dropped: Array<{ sessionId: string; attempt: number }> = [];
|
||||
const exhausted: Array<{ sessionId: string }> = [];
|
||||
manager.on('remoteSessionDropped', (d) => dropped.push(d));
|
||||
manager.on('remoteReconnectExhausted', (d) => exhausted.push(d));
|
||||
|
||||
// Drive ticks with a controllable clock. Each emit marks the session
|
||||
// in-flight; a failed reattach (host still down) releases it via
|
||||
// noteRemoteReconnect(false), mirroring the real server loop.
|
||||
let now = 0;
|
||||
for (let i = 0; i < MAX_RECONNECT_ATTEMPTS + 3; i++) {
|
||||
manager.runRemoteReconnectTick(now, /* enabled */ true);
|
||||
manager.noteRemoteReconnect('aaaa1111', false); // reattach failed → clear in-flight
|
||||
// jump the clock past the just-scheduled backoff window
|
||||
now += BACKOFF_SCHEDULE_MS[Math.min(i, BACKOFF_SCHEDULE_MS.length - 1)] + 1;
|
||||
}
|
||||
|
||||
expect(dropped.map((d) => d.attempt)).toEqual([1, 2, 3, 4, 5, 6]);
|
||||
expect(dropped.every((d) => d.sessionId === 'aaaa1111')).toBe(true);
|
||||
expect(exhausted).toEqual([{ sessionId: 'aaaa1111' }]); // fired exactly once
|
||||
});
|
||||
|
||||
it('emits nothing for a guarded (intentionally killed) session', () => {
|
||||
registerRemote('bbbb2222');
|
||||
vi.spyOn(manager, 'isPaneDead').mockReturnValue(true);
|
||||
manager.guardRemoteReconnect('bbbb2222'); // simulate killSession/detach guard
|
||||
|
||||
const dropped: unknown[] = [];
|
||||
manager.on('remoteSessionDropped', (d) => dropped.push(d));
|
||||
|
||||
let now = 0;
|
||||
for (let i = 0; i < 5; i++) {
|
||||
manager.runRemoteReconnectTick(now, true);
|
||||
now += 600_000;
|
||||
}
|
||||
expect(dropped).toEqual([]);
|
||||
});
|
||||
|
||||
it('resets backoff on a successful reattach (noteRemoteReconnect)', () => {
|
||||
registerRemote('cccc3333');
|
||||
vi.spyOn(manager, 'isPaneDead').mockReturnValue(true);
|
||||
|
||||
const dropped: Array<{ attempt: number }> = [];
|
||||
manager.on('remoteSessionDropped', (d) => dropped.push(d));
|
||||
|
||||
manager.runRemoteReconnectTick(0, true); // emit attempt 1
|
||||
manager.noteRemoteReconnect('cccc3333', true); // reattach succeeded → reset
|
||||
manager.runRemoteReconnectTick(1, true); // immediately eligible again → attempt 1
|
||||
|
||||
expect(dropped.map((d) => d.attempt)).toEqual([1, 1]);
|
||||
});
|
||||
|
||||
it('does nothing when the kill-switch (enabled=false) is off', () => {
|
||||
registerRemote('dddd4444');
|
||||
vi.spyOn(manager, 'isPaneDead').mockReturnValue(true);
|
||||
const dropped: unknown[] = [];
|
||||
manager.on('remoteSessionDropped', (d) => dropped.push(d));
|
||||
|
||||
manager.runRemoteReconnectTick(0, false);
|
||||
expect(dropped).toEqual([]);
|
||||
});
|
||||
|
||||
it('clears per-session reconnect/guard state when the session is removed', () => {
|
||||
registerRemote('eeee5555');
|
||||
manager.guardRemoteReconnect('eeee5555');
|
||||
manager.clearRemoteReconnectState('eeee5555');
|
||||
// After clearing the guard, a fresh dead-pane observation should emit again.
|
||||
vi.spyOn(manager, 'isPaneDead').mockReturnValue(true);
|
||||
const dropped: unknown[] = [];
|
||||
manager.on('remoteSessionDropped', (d) => dropped.push(d));
|
||||
manager.runRemoteReconnectTick(0, true);
|
||||
expect(dropped).toEqual([{ sessionId: 'eeee5555', attempt: 1 }]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,151 @@
|
||||
/**
|
||||
* @fileoverview COD-105 — discover & attach existing remote tmux sessions.
|
||||
*
|
||||
* Phase 2 of the remote-tmux arc (builds on COD-104 durable remote sessions +
|
||||
* COD-107 connection args). These tests are tmux-safe / ssh-safe: they exercise
|
||||
* the PURE parse helper, the pure attach-command builder, and the killSession
|
||||
* ownership gate — none open a real ssh connection or a real tmux server.
|
||||
*
|
||||
* Port: N/A.
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { parseRemoteSessionList } from '../src/remote-hosts.js';
|
||||
import { buildRemoteAttachCommand } from '../src/tmux-manager.js';
|
||||
import { TmuxManager } from '../src/tmux-manager.js';
|
||||
import type { SessionRemote } from '../src/types.js';
|
||||
|
||||
const baseRemote: SessionRemote = {
|
||||
hostId: 'gpu-box',
|
||||
label: 'GPU Box',
|
||||
host: '10.0.0.42',
|
||||
username: 'ubuntu',
|
||||
remotePath: '/home/ubuntu/work',
|
||||
};
|
||||
|
||||
describe('COD-105 parseRemoteSessionList', () => {
|
||||
it('parses tab-delimited -F output and coerces fields', () => {
|
||||
const stdout = 'codeman-disco1\t0\t1700000000\t1\n' + 'codeman-abcd1234\t1\t1700000123\t3\n';
|
||||
const list = parseRemoteSessionList(stdout);
|
||||
expect(list).toEqual([
|
||||
{ name: 'codeman-disco1', attached: false, attachedClients: 0, created: 1700000000, windows: 1 },
|
||||
{ name: 'codeman-abcd1234', attached: true, attachedClients: 1, created: 1700000123, windows: 3 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('parses the LITERAL backslash-t separator the remote tmux actually emits', () => {
|
||||
// tmux next-3.7's `-F "…\t…"` does NOT expand \t — it prints a literal
|
||||
// backslash-t (verified on aa-desktop). The parser must split on that.
|
||||
const stdout = 'codeman-disco1\\t0\\t1781362858\\t1\n' + 'codeman-real\\t1\\t1781329905\\t2\n';
|
||||
const list = parseRemoteSessionList(stdout);
|
||||
expect(list).toEqual([
|
||||
{ name: 'codeman-disco1', attached: false, attachedClients: 0, created: 1781362858, windows: 1 },
|
||||
{ name: 'codeman-real', attached: true, attachedClients: 1, created: 1781329905, windows: 2 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('keeps only codeman-* sessions, dropping foreign tmux sessions', () => {
|
||||
const stdout = 'work\t1\t1700000000\t2\n' + 'codeman-keep\t0\t1700000001\t1\n' + 'scratch\t0\t1700000002\t1\n';
|
||||
const list = parseRemoteSessionList(stdout);
|
||||
expect(list.map((s) => s.name)).toEqual(['codeman-keep']);
|
||||
});
|
||||
|
||||
it('returns [] for empty / whitespace output (the no-sessions case)', () => {
|
||||
expect(parseRemoteSessionList('')).toEqual([]);
|
||||
expect(parseRemoteSessionList(' \n \n')).toEqual([]);
|
||||
});
|
||||
|
||||
it('tolerates malformed lines (missing columns) by skipping them', () => {
|
||||
const stdout = 'codeman-ok\t0\t1700000000\t1\n' + 'codeman-bad\tnotanumber\n';
|
||||
const list = parseRemoteSessionList(stdout);
|
||||
expect(list).toEqual([
|
||||
{ name: 'codeman-ok', attached: false, attachedClients: 0, created: 1700000000, windows: 1 },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('COD-105 buildRemoteAttachCommand', () => {
|
||||
it('emits ssh -t <target> tmux -L codeman attach -t <session>', () => {
|
||||
const command = buildRemoteAttachCommand(baseRemote, 'codeman-disco1');
|
||||
expect(command).toContain('ssh');
|
||||
expect(command).toContain('BatchMode=yes');
|
||||
expect(command).toContain('-t');
|
||||
expect(command).toContain('ubuntu@10.0.0.42');
|
||||
// The tmux invocation is nested-quoted (inner session name escaped, whole
|
||||
// invocation re-escaped as one ssh arg). Assert the stable prefix here; the
|
||||
// exact re-parsed token is verified by the argv-reparse test below.
|
||||
expect(command).toContain('tmux -L codeman attach -t ');
|
||||
expect(command).toContain('codeman-disco1');
|
||||
});
|
||||
|
||||
it('threads the COD-107 connection args (port / identity / proxy) into the ssh invocation', () => {
|
||||
const command = buildRemoteAttachCommand(
|
||||
{ ...baseRemote, port: 2222, identityFile: '/keys/id_ed25519', socksProxy: '127.0.0.1:1080' },
|
||||
'codeman-disco1'
|
||||
);
|
||||
expect(command).toContain('-p 2222');
|
||||
expect(command).toContain("-i '/keys/id_ed25519'");
|
||||
expect(command).toContain('ProxyCommand=nc -X 5 -x 127.0.0.1:1080 %h %p');
|
||||
// Port/identity/proxy belong to ssh, ahead of the target.
|
||||
expect(command).toMatch(/ssh[\s\S]*-p 2222[\s\S]*ubuntu@10\.0\.0\.42/);
|
||||
});
|
||||
|
||||
it('shell-escapes the session name so it stays a single token', () => {
|
||||
const command = buildRemoteAttachCommand(baseRemote, 'codeman-disco1');
|
||||
// Re-parse: stub ssh to dump argv, confirm the trailing tmux invocation is one arg.
|
||||
const dumpArgs = (name: string, prefix: string) =>
|
||||
`${name}() { for a in "$@"; do printf '${prefix}:%s\\n' "$a"; done; }`;
|
||||
const out = execFileSync('/bin/sh', ['-c', `${dumpArgs('ssh', 'A')}\n${command}`], { encoding: 'utf8' });
|
||||
const sshArgs = out
|
||||
.split('\n')
|
||||
.filter((l) => l.startsWith('A:'))
|
||||
.map((l) => l.slice(2));
|
||||
expect(sshArgs).toContain('ubuntu@10.0.0.42');
|
||||
const tmuxArg = sshArgs.find((a) => a.includes('attach'));
|
||||
expect(tmuxArg).toBe("tmux -L codeman attach -t 'codeman-disco1'");
|
||||
});
|
||||
});
|
||||
|
||||
describe('COD-105 killSession ownership gate (detach-not-kill)', () => {
|
||||
it('never issues a remote tmux kill-session for a non-owned remote session', async () => {
|
||||
const mgr = new TmuxManager();
|
||||
// Register a discovered+attached (non-owned) remote session.
|
||||
mgr.registerSession({
|
||||
sessionId: 'disco-1',
|
||||
muxName: 'codeman-disco-1',
|
||||
pid: 0,
|
||||
createdAt: Date.now(),
|
||||
workingDir: '/home/ubuntu/work',
|
||||
mode: 'shell',
|
||||
attached: false,
|
||||
remote: { ...baseRemote, owned: false },
|
||||
});
|
||||
|
||||
// killSession under VITEST is in-memory only (IS_TEST_MODE), so it physically
|
||||
// cannot run a remote kill-session. We assert the contract: the session's
|
||||
// ownership flag is the gate, and tearing it down removes only local state.
|
||||
const session = mgr.getSession('disco-1');
|
||||
expect(session?.remote?.owned).toBe(false);
|
||||
|
||||
const ok = await mgr.killSession('disco-1');
|
||||
expect(ok).toBe(true);
|
||||
// Local tracking removed; no remote kill was (or could be) issued.
|
||||
expect(mgr.getSession('disco-1')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('treats COD-104 launched remote sessions as owned by default', () => {
|
||||
const mgr = new TmuxManager();
|
||||
mgr.registerSession({
|
||||
sessionId: 'owned-1',
|
||||
muxName: 'codeman-owned-1',
|
||||
pid: 0,
|
||||
createdAt: Date.now(),
|
||||
workingDir: '/home/ubuntu/work',
|
||||
mode: 'shell',
|
||||
attached: false,
|
||||
remote: { ...baseRemote, owned: true },
|
||||
});
|
||||
expect(mgr.getSession('owned-1')?.remote?.owned).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,59 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { buildRemoteKillCommand } from '../src/tmux-manager.js';
|
||||
import type { SessionRemote } from '../src/types.js';
|
||||
|
||||
// COD-109 — terminate an OWNED durable remote tmux session by propagating
|
||||
// `kill-session` to the remote host. Since COD-104 an owned remote session lives
|
||||
// in the dedicated `-L codeman-remote` tmux server on the host and outlives the
|
||||
// local ssh pane, so ending a session we OWN must reach the remote socket.
|
||||
//
|
||||
// The owned-kill command builder was consolidated upstream (PR #145) into the
|
||||
// `{ remote, sessionId }` form, which derives the durable session name and kills
|
||||
// on the dedicated `codeman-remote` socket (matching buildRemoteLaunchCommand).
|
||||
// This suite pins that builder's contract; the killSession integration
|
||||
// (owned-only, after COD-105's non-owned detach-only early-return) is exercised
|
||||
// against the real remote.
|
||||
describe('COD-109 buildRemoteKillCommand (owned durable remote kill)', () => {
|
||||
const base: SessionRemote = {
|
||||
hostId: 'h',
|
||||
label: 'aa',
|
||||
host: '192.168.55.170',
|
||||
username: 'aakht',
|
||||
remotePath: '/tmp',
|
||||
};
|
||||
|
||||
it('kills the durable session on the dedicated codeman-remote socket (no ssh -t — non-interactive)', () => {
|
||||
const cmd = buildRemoteKillCommand({ remote: base, sessionId: 'abc12345def' });
|
||||
expect(cmd.startsWith('ssh -o BatchMode=yes ')).toBe(true);
|
||||
expect(cmd).toContain('aakht@192.168.55.170');
|
||||
// Owned sessions launch on `-L codeman-remote`; the kill MUST target the same socket.
|
||||
expect(cmd).toContain('tmux -L codeman-remote kill-session -t');
|
||||
// Deterministic session name derived from the sessionId (codeman-ssh-<first 8>).
|
||||
expect(cmd).toContain('codeman-ssh-abc12345');
|
||||
// kill-session needs no PTY — must NOT request the ssh `-t` flag (attach uses
|
||||
// `ssh -o BatchMode=yes -t …`; kill must not). The ` -t ` inside the quoted
|
||||
// `kill-session -t <name>` is the tmux target flag, which is expected.
|
||||
expect(cmd).not.toContain('-o BatchMode=yes -t');
|
||||
});
|
||||
|
||||
it('shares the default ConnectTimeout so an unreachable host fails fast (never blocks kill)', () => {
|
||||
const cmd = buildRemoteKillCommand({ remote: base, sessionId: 'abc12345def' });
|
||||
expect(cmd).toContain('-o ConnectTimeout=10');
|
||||
});
|
||||
|
||||
it('reuses the COD-107 connection options (port / identity / SOCKS proxy)', () => {
|
||||
const cmd = buildRemoteKillCommand({
|
||||
remote: { ...base, port: 2222, identityFile: '~/.ssh/remote_ed25519', socksProxy: '127.0.0.1:1080' },
|
||||
sessionId: 'abc12345def',
|
||||
});
|
||||
expect(cmd).toContain('-p 2222');
|
||||
expect(cmd).toMatch(/-i '.*\/\.ssh\/remote_ed25519'/);
|
||||
expect(cmd).toContain("-o 'ProxyCommand=nc -X 5 -x 127.0.0.1:1080 %h %p'");
|
||||
});
|
||||
|
||||
it('shell-escapes the derived session name so metachars stay one token', () => {
|
||||
const cmd = buildRemoteKillCommand({ remote: base, sessionId: "x'; rm -rf /" });
|
||||
expect(cmd).not.toMatch(/rm -rf \/\s*$/); // not a bare trailing command
|
||||
expect(cmd).toContain('kill-session -t');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,39 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { buildRemoteLaunchCommand } from '../src/tmux-manager.js';
|
||||
import { parseRemoteSessionList } from '../src/remote-hosts.js';
|
||||
import type { SessionRemote } from '../src/types.js';
|
||||
|
||||
// COD-106 — shared/collaborative remote sessions: window-size policy so concurrent
|
||||
// clients don't fight, and a client-count surfaced for the "shared · N" badge.
|
||||
describe('COD-106 shared remote sessions', () => {
|
||||
const remote: SessionRemote = {
|
||||
hostId: 'h',
|
||||
label: 'aa',
|
||||
host: '192.168.55.170',
|
||||
username: 'aakht',
|
||||
remotePath: '/tmp',
|
||||
commands: { shell: 'exec bash -l' },
|
||||
};
|
||||
|
||||
it('launch command sets window-size latest (so multi-client attach does not clamp to smallest)', () => {
|
||||
const cmd = buildRemoteLaunchCommand({ mode: 'shell', remote, sessionId: 'cod106aaa' });
|
||||
// Per-session scoped on the dedicated `codeman-remote` socket (PR #145 hardening).
|
||||
expect(cmd).toContain('set -t codeman-ssh-cod106aa window-size latest');
|
||||
// still has the COD-104 config (no regression)
|
||||
expect(cmd).toContain('set -t codeman-ssh-cod106aa status off');
|
||||
expect(cmd).toContain('new-session -A -s codeman-ssh-cod106aa');
|
||||
});
|
||||
|
||||
it('parses session_attached as a CLIENT COUNT (>1 = shared)', () => {
|
||||
const rows = parseRemoteSessionList(
|
||||
['codeman-solo\\t1\\t100\\t1', 'codeman-shared\\t2\\t200\\t3', 'codeman-idle\\t0\\t300\\t1'].join('\n')
|
||||
);
|
||||
const byName = Object.fromEntries(rows.map((r) => [r.name, r]));
|
||||
expect(byName['codeman-solo'].attachedClients).toBe(1);
|
||||
expect(byName['codeman-solo'].attached).toBe(true);
|
||||
expect(byName['codeman-shared'].attachedClients).toBe(2); // shared
|
||||
expect(byName['codeman-shared'].attached).toBe(true);
|
||||
expect(byName['codeman-idle'].attachedClients).toBe(0);
|
||||
expect(byName['codeman-idle'].attached).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -157,6 +157,8 @@ describe('COD-107 buildRemoteLaunchCommand — threads connection args', () => {
|
||||
`set -t ${remoteName} mouse off`,
|
||||
`set -t ${remoteName} prefix C-q`,
|
||||
'set -s escape-time 0',
|
||||
// COD-106 — shared/collaborative sizing, per-session scoped (never -g).
|
||||
`set -t ${remoteName} window-size latest`,
|
||||
].join(' \\; ');
|
||||
// Connection args (with the default -o ConnectTimeout=10) sit after -t.
|
||||
const expected = `ssh -o BatchMode=yes -t -o ConnectTimeout=10 ${remoteSshTarget(baseRemote)} ${sh(tmuxInvocation)}`;
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
/**
|
||||
* @fileoverview COD-105 — GET /api/remote-hosts/:hostId/sessions discovery endpoint.
|
||||
*
|
||||
* The endpoint reads the saved host config by id, runs listRemoteCodemanSessions
|
||||
* (ssh-guarded under VITEST), and returns the discovered sessions in the
|
||||
* ApiResponse envelope. We mock the remote-hosts module so the test controls the
|
||||
* host record and the session list WITHOUT any real ssh / filesystem.
|
||||
*
|
||||
* Port: N/A (app.inject()).
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
|
||||
import Fastify, { type FastifyInstance } from 'fastify';
|
||||
import fastifyCookie from '@fastify/cookie';
|
||||
import { createMockRouteContext } from '../mocks/index.js';
|
||||
import { installRouteErrorHandler } from '../../src/web/route-error-handler.js';
|
||||
import { ApiErrorCode, httpStatusForErrorCode } from '../../src/types.js';
|
||||
import type { RemoteHost, RemoteSessionInfo } from '../../src/types.js';
|
||||
|
||||
// Mock the remote-hosts module: control readRemoteHosts + listRemoteCodemanSessions.
|
||||
const mockHosts: RemoteHost[] = [];
|
||||
let mockSessions: RemoteSessionInfo[] = [];
|
||||
let lastListArg: unknown = undefined;
|
||||
|
||||
vi.mock('../../src/remote-hosts.js', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('../../src/remote-hosts.js')>();
|
||||
return {
|
||||
...actual,
|
||||
readRemoteHosts: vi.fn(async () => mockHosts),
|
||||
readRemoteCases: vi.fn(async () => []),
|
||||
listRemoteCodemanSessions: vi.fn(async (remote: unknown) => {
|
||||
lastListArg = remote;
|
||||
return mockSessions;
|
||||
}),
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock('../../src/templates/claude-md.js', () => ({
|
||||
generateClaudeMd: vi.fn(() => '# CLAUDE.md'),
|
||||
}));
|
||||
vi.mock('../../src/hooks-config.js', () => ({ writeHooksConfig: vi.fn(async () => {}) }));
|
||||
|
||||
import { registerCaseRoutes } from '../../src/web/routes/case-routes.js';
|
||||
|
||||
async function createHarness(): Promise<FastifyInstance> {
|
||||
const app = Fastify({ logger: false });
|
||||
await app.register(fastifyCookie);
|
||||
app.addHook('preSerialization', (req, reply, payload: unknown, done) => {
|
||||
if (!req.url.startsWith('/api')) return done(null, payload);
|
||||
if (payload === null || typeof payload !== 'object') return done(null, payload);
|
||||
const p = payload as { success?: unknown; errorCode?: unknown };
|
||||
if (p.success === false) {
|
||||
if (reply.statusCode === 200 && typeof p.errorCode === 'string') {
|
||||
reply.code(httpStatusForErrorCode(p.errorCode as ApiErrorCode));
|
||||
}
|
||||
return done(null, payload);
|
||||
}
|
||||
if (p.success === true) return done(null, payload);
|
||||
return done(null, { success: true, data: payload });
|
||||
});
|
||||
const ctx = createMockRouteContext();
|
||||
registerCaseRoutes(app, ctx as never);
|
||||
installRouteErrorHandler(app);
|
||||
await app.ready();
|
||||
return app;
|
||||
}
|
||||
|
||||
describe('COD-105 GET /api/remote-hosts/:hostId/sessions', () => {
|
||||
let app: FastifyInstance;
|
||||
|
||||
beforeEach(async () => {
|
||||
app = await createHarness();
|
||||
mockHosts.length = 0;
|
||||
mockSessions = [];
|
||||
lastListArg = undefined;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it('returns discovered sessions for a known host in the envelope', async () => {
|
||||
mockHosts.push({ id: 'aa-desktop', label: 'aa', host: '1.2.3.4', username: 'aakht', port: 2222 });
|
||||
mockSessions = [{ name: 'codeman-disco1', attached: false, created: 1700000000, windows: 1 }];
|
||||
|
||||
const res = await app.inject({ method: 'GET', url: '/api/remote-hosts/aa-desktop/sessions' });
|
||||
expect(res.statusCode).toBe(200);
|
||||
const body = JSON.parse(res.body);
|
||||
expect(body.success).toBe(true);
|
||||
expect(body.data.sessions).toEqual([{ name: 'codeman-disco1', attached: false, created: 1700000000, windows: 1 }]);
|
||||
// The host config (incl. port) was threaded to the discovery call.
|
||||
expect((lastListArg as { host?: string; port?: number }).host).toBe('1.2.3.4');
|
||||
expect((lastListArg as { port?: number }).port).toBe(2222);
|
||||
});
|
||||
|
||||
it('404s when the host id is unknown', async () => {
|
||||
const res = await app.inject({ method: 'GET', url: '/api/remote-hosts/nope/sessions' });
|
||||
expect(res.statusCode).toBe(404);
|
||||
const body = JSON.parse(res.body);
|
||||
expect(body.success).toBe(false);
|
||||
expect(body.errorCode).toBe(ApiErrorCode.NOT_FOUND);
|
||||
});
|
||||
|
||||
it('returns an empty list (not an error) when no sessions are discovered', async () => {
|
||||
mockHosts.push({ id: 'aa-desktop', label: 'aa', host: '1.2.3.4', username: 'aakht' });
|
||||
mockSessions = [];
|
||||
const res = await app.inject({ method: 'GET', url: '/api/remote-hosts/aa-desktop/sessions' });
|
||||
expect(res.statusCode).toBe(200);
|
||||
const body = JSON.parse(res.body);
|
||||
expect(body.success).toBe(true);
|
||||
expect(body.data.sessions).toEqual([]);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user