mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-09-30 12:39:42 +02:00
fix(files): fail closed on remote symlinks, guard PUT for remote cases, bound ssh fan-out
Follow-up to #421 (remote-case file reads over ssh), addressing the review. Symlink escape on a host without `readlink -f` (blocker). The probe's portable fallback canonicalized only the directory chain and returned the final component unresolved, so on macOS < 12.3 `ws/notes.txt -> ~/.ssh/id_rsa` came back as `.../ws/notes.txt` (with the target's size), passed every containment and blocklist check that runs on `realPath`, and `cat` followed the link. The fallback now walks the directory chain with `cd -P`/`pwd -P` and follows the LAST component with plain `readlink` for a bounded number of hops, and anything it cannot fully resolve (a loop, a readlink failure, the hop cap) is reported with an `x` marker that parses as null, i.e. 404. It never returns the unresolved string. Measured on a real /bin/sh with `readlink -f` shadowed: the pre-fix script reports `/ws/notes.txt`, the fixed one `/secret/id_rsa`; both branches (native and fallback) now agree. `PUT /api/sessions/:id/file-content` never had the remote guard the PR described. It sits ahead of `validateSessionFilePath`, which resolves against the LOCAL filesystem, because with a same-named directory on the Codeman host (an sshfs mount of the remote tree, the documented stop-gap) the write landed on the local twin while the viewer believed it edited the remote file. ssh fan-out is bounded. `src/remote-ssh-limiter.ts` is a document-conversion-limiter-shaped semaphore (default 4, env `CODEMAN_MAX_REMOTE_FILE_SSH`) around every probe and buffered read; the attachment-history list resolves its whole history in ONE batched probe (`probeRemoteAttachmentHistory`, threaded into `registerExternalAttachment({remoteProbes})` so the guards run unchanged) instead of one handshake per entry; and probes chunk at 40 paths because the whole script is one argv string. Terminal output in a remote session is written on the remote host, so a prompt-injected agent printing hundreds of `codeman://attach` links forked one ssh per link, each holding a 20 s timeout, and a 100-entry history re-listed on every attachment:detected tripped OpenSSH's default MaxStartups. Streams are deliberately not counted (one per browser request, held for a whole playback, and gated behind a counted probe anyway). Smaller items from the same review: probe records are NUL-terminated and index-keyed after a leading NUL (a newline in a filename can no longer shift the alignment, and the banner is fenced off without last-N-lines guessing); size comes from `stat -c %s || stat -f %z`; the three IO functions refuse under VITEST instead of opening a connection; an unreachable host now reads as unknown (missing: false) for detected AND external history entries, where external used to fold its 502 into missing; a client that aborted during the guard probe has its body's ssh child reaped (`reply.raw.destroyed` is checked before the close listener is attached); `describeExecError` never returns Node's `Command failed: <ssh line>` message, which carried the identity path and the probe script into a 502 body; and the docs note that `isSensitivePath`'s three home-anchored entries resolve against the Codeman host's home, not the remote one. Tests: the probe script runs on a real /bin/sh with a `readlink` shim that rejects `-f` (the escape, a relative chain through a symlinked directory, a loop, a newline filename, banner chatter that itself looks like a record), the limiter's cap and FIFO order, and route tests for the PUT guard (local twin untouched, no connection), the single batched history probe, the unreachable-host alignment and the aborted-client reap. All four route tests fail against the pre-fix file-routes.ts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer
@@ -215,7 +215,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Cron (`CronJob`s)**: saved, named jobs on a recurring schedule (`once`/`interval`/`daily`/`weekly`) with per-job run history. ⚠️ **Distinct from the legacy `ScheduledRun`** (`/api/scheduled`, a run-now duration-bounded loop); the two never interact and keep separate `Scheduled*` / `Cron*` names. `CronService` **reuses the existing session layer** rather than rebuilding tmux logic. Next-run math is pure and unit-tested in `cron-time.ts` (server-local timezone). The schedule is advanced BEFORE launch so a slow launch cannot re-trigger. → [architecture-invariants#cron-jobs](docs/architecture-invariants.md#cron-jobs), `docs/cron-discovery.md`
|
||||
|
||||
**Remote sessions + remote SSH cases**: a case can point at a remote host. The agent runs inside a durable remote `tmux -L codeman-remote` (session name `codeman-ssh-<id>`, deliberately failing the remote Codeman's `SAFE_MUX_NAME_PATTERN` so an instance on the target host never adopts it), fronted by a LOCAL tmux pane running `ssh`. Attached (`owned:false`) sessions **detach, never kill** on tab close; owned ones propagate `kill-session`. A bounded-backoff watcher auto-reconnects dropped sessions (`remoteAutoReconnect`, default ON). ⚠️ **It revives ONLY when the durable remote tmux session is verifiably still alive** (`remoteTmuxSessionAlive()`, a `has-session` probe over ssh, #355): a clean agent exit (Ctrl-C, Ctrl-D, `exit`) tears that session down, and `isPaneDead()` cannot tell it from a transport drop, so the watcher used to relaunch a FRESH agent after every clean exit (claude only looked fine because its `|| --resume` fallback masked it). An unreachable host answers `undefined`, which also means do not revive. ⚠️ `has-session` prints NOTHING on success, so the probe is classified by EXIT STATUS (`classifyRemoteAliveExit`: 0 alive, ssh's 255 or a timeout unknown, anything else gone); reading stdout classified every live session as gone and silently disabled transport-drop reconnects. The answer is cached per session and forgotten whenever the pane is seen alive again, or a stale `true` from one transport drop would revive the next clean exit. ⚠️ **File reads in a remote case are the second ssh surface** (#415, `src/remote-files.ts`): they go through `buildSshConnectionArgs()` as well, a browser-supplied path is only ever a `shellescape`d token, an unreachable host answers 502 (never 404), the size cap uses the REMOTE size, and no remote file is ever copied onto the server's disk — which is why writes, office previews and thumbnails are deliberately unsupported over ssh. The ATTACHMENT routes (a clicked path outside the case dir) go through the same layer, and which host a record is read from follows the SESSION, never the path string. ⚠️ **Command-injection surface: every ssh command line must flow through `buildSshConnectionArgs()`**, which `shellescape`s every user field. Never hand-build an ssh line elsewhere. ⚠️ Run flows must route remote cases through `POST /api/quick-start`, not `POST /api/sessions` (which stat-validates `workingDir` locally and has no `caseName`). → [architecture-invariants#remote-sessions-over-ssh](docs/architecture-invariants.md#remote-sessions-over-ssh), [#remote-ssh-cases](docs/architecture-invariants.md#remote-ssh-cases), `docs/remote-sessions.md`
|
||||
**Remote sessions + remote SSH cases**: a case can point at a remote host. The agent runs inside a durable remote `tmux -L codeman-remote` (session name `codeman-ssh-<id>`, deliberately failing the remote Codeman's `SAFE_MUX_NAME_PATTERN` so an instance on the target host never adopts it), fronted by a LOCAL tmux pane running `ssh`. Attached (`owned:false`) sessions **detach, never kill** on tab close; owned ones propagate `kill-session`. A bounded-backoff watcher auto-reconnects dropped sessions (`remoteAutoReconnect`, default ON). ⚠️ **It revives ONLY when the durable remote tmux session is verifiably still alive** (`remoteTmuxSessionAlive()`, a `has-session` probe over ssh, #355): a clean agent exit (Ctrl-C, Ctrl-D, `exit`) tears that session down, and `isPaneDead()` cannot tell it from a transport drop, so the watcher used to relaunch a FRESH agent after every clean exit (claude only looked fine because its `|| --resume` fallback masked it). An unreachable host answers `undefined`, which also means do not revive. ⚠️ `has-session` prints NOTHING on success, so the probe is classified by EXIT STATUS (`classifyRemoteAliveExit`: 0 alive, ssh's 255 or a timeout unknown, anything else gone); reading stdout classified every live session as gone and silently disabled transport-drop reconnects. The answer is cached per session and forgotten whenever the pane is seen alive again, or a stale `true` from one transport drop would revive the next clean exit. ⚠️ **File reads in a remote case are the second ssh surface** (#415, `src/remote-files.ts`): they go through `buildSshConnectionArgs()` as well, a browser-supplied path is only ever a `shellescape`d token, an unreachable host answers 502 (never 404), the size cap uses the REMOTE size, and no remote file is ever copied onto the server's disk — which is why writes, office previews and thumbnails are deliberately unsupported over ssh (the `PUT` guard sits BEFORE the local path validation, or a same-named local directory such as an sshfs mount takes the write). The probe's symlink resolution FAILS CLOSED (a path it cannot canonicalize is a 404, never its own unresolved string: the directory-only fallback let a `notes.txt -> ~/.ssh/id_rsa` link pass containment), and ssh children are BOUNDED by `src/remote-ssh-limiter.ts` plus one batched probe per attachment-history listing, because terminal output in a remote session is written on the remote host and a prompt-injected agent can print hundreds of `codeman://attach` links. The ATTACHMENT routes (a clicked path outside the case dir) go through the same layer, and which host a record is read from follows the SESSION, never the path string. ⚠️ **Command-injection surface: every ssh command line must flow through `buildSshConnectionArgs()`**, which `shellescape`s every user field. Never hand-build an ssh line elsewhere. ⚠️ Run flows must route remote cases through `POST /api/quick-start`, not `POST /api/sessions` (which stat-validates `workingDir` locally and has no `caseName`). → [architecture-invariants#remote-sessions-over-ssh](docs/architecture-invariants.md#remote-sessions-over-ssh), [#remote-ssh-cases](docs/architecture-invariants.md#remote-ssh-cases), `docs/remote-sessions.md`
|
||||
|
||||
**Docker cases**: a case can point at a **container**, with any of the CLI run modes running inside it. Like remote-SSH this is a **LOCATION OVERLAY on cases, never a `SessionMode` of its own**. Exactly one long-lived container **per case**, shared by all its sessions, so killing a session kills only that session's in-container tmux and **never** `docker stop` while siblings remain. The workspace is a real host dir bind-mounted at the **same absolute path**, which is what keeps file-routes/watchers on real host bytes and makes the in-container transcript projHash match the host. Credentials are **seeded** (RO mount, copied into the container once) rather than shared RW, so in-container CLIs never write refreshed tokens back to the host, and bind mounts are excluded from `docker commit` so exports stay secret-free. **NEVER a create-time `-e` for secrets, NEVER `--privileged`, NEVER the docker socket.** Config drift is detected via a label hash and a drifted launch is REFUSED rather than silently launched with stale config. ⚠️ A case may instead **ADOPT** a container the user already runs (`DockerCase.owned === false`, mirror of remote-SSH's `owned:false`): Codeman only `exec`s into it and never creates, starts, stops, restarts or removes it, so a missing or stopped container FAILS CLOSED with an actionable message instead of being fixed. Absent = owned, so existing cases are byte-identical. The guarantee is enforced at four independent layers because it cannot be observed by using the feature: `buildDockerStopCommand`/`buildDockerRemoveCommand` throw during pure STRING CONSTRUCTION, `removeDockerContainer` refuses again, drift reports "none" (an adopted container carries no `codeman.confighash` label, so a real comparison would 409 the launch forever), and the boot reaper skips it. ⚠️ Two lifecycle touches the original design missed and that are easy to re-introduce: the full-image export `docker commit`s the container (refused for an adopted case) and the workspace export `docker pause`s it first (skipped — it freezes the owner's processes for the length of the tar). ⚠️ `owned` is applied AFTER `dockerConfigHash`, which takes an explicit field list, or every pre-existing case would trip the drift gate at once. ⚠️ Run modes for a container case come from the CONTAINER (`availableModes`, live-probed): gating the run menu on HOST CLIs (#201) is right for local sessions and wrong here, since a host with no `claude` may run a container that ships one. ⚠️ **A failed probe means opposite things per ownership** — for an ADOPTED case it is a fault worth reporting, for an OWNED one it is the NORMAL state before the first session (the launch chain creates the container), so treating it as a fault hid every agent mode on every freshly linked Docker case behind "start it yourself first". That is why `CaseInfo.docker.owned` is on the wire. ⚠️ Claude is launched WITHOUT `--dangerously-skip-permissions` when the container's exec user is root (Claude Code refuses the flag as root and the refusal is visible only inside the container); which flag to drop is a per-CLI fact, so it is the registry's `overlays.docker.rootCommand`, never a branch. ⚠️ Adoption is **admin-only in multi-user mode**, unlike `docker-link`: linking creates OUR container, whose one bind mount `isWorkingDirAllowed` has already confined, while an adopted container's mounts belong to its owner and one mounting `/` hands the adopter the host. The same reasoning admin-gates the container listing and the in-container directory browser; the preflight instead admits a non-admin for a container already linked to a case they own, because the run menu probes it for every docker case. ⚠️ On the loopback-only prod bind a container cannot reach 127.0.0.1, so in-container hooks need `CODEMAN_DOCKER_BRIDGE_HOOKS=1`; otherwise idle detection falls back to output-based. → [architecture-invariants#docker-cases](docs/architecture-invariants.md#docker-cases), `docs/docker-cases.md` (user guide), `docs/docker-cases-plan.md` (design)
|
||||
|
||||
|
||||
@@ -54,7 +54,7 @@ Model is NOT a session field: it is a composition entry in the profile's config
|
||||
|
||||
### Remote SSH cases
|
||||
|
||||
**Remote SSH cases** (COD-94/#145): cases can point at a **remote host** (`~/.codeman/remote-hosts.json` + `remote-cases.json` via `src/remote-hosts.ts`; CRUD under `/api/cases` — cases route file). A remote session launches a LOCAL tmux pane running `ssh <host>` that creates a durable REMOTE tmux session on a **dedicated socket** `-L codeman-remote` with name `codeman-ssh-<id>` — deliberately failing the remote Codeman's `SAFE_MUX_NAME_PATTERN` so a Codeman instance on the target host never adopts it; no `-g` global tmux options are set remotely. `remotePath`/`identityFile` are schema-guarded against shell injection (backticks/`$` rejected — same approach as `extraSshOptions`); remote tmux availability is probed via `checkRemoteTmuxAvailable()` in quick-start (ssh args carry `-o ConnectTimeout=10`). Remote claude defaults to an idempotent `claude --session-id <id> || claude --resume <id>` pair under a login shell, so a respawn or reattach continues the SAME conversation rather than starting a fresh one (remote omp gets the same treatment via `--continue`; ⚠️ because the claude arm is an `a || b` pair under `-c`, that pane's PID is the login shell, not the agent); per-host `commands.*` override. Session kill best-effort kills the remote tmux too. `SessionState.remote`/`MuxSession.remote` round-trip through recovery (`restoreMuxSessions` passes `remote` back into the Session constructor). ⚠️ Run flows must route remote cases through `POST /api/quick-start` (which resolves the remote case and skips LOCAL CLI availability gates) — `POST /api/sessions` stat-validates `workingDir` locally and has no `caseName`. `envOverrides`/`effort`/`modelOverride`/`codexConfig`/`geminiConfig` are rejected for remote quick-starts (not silently dropped). UI: Create Case modal → Remote tab. Tests: `test/remote-hosts.test.ts`, `test/remote-ssh-options.test.ts`. ⚠️ **Reading a file in a remote case goes over ssh too** (#415): `src/remote-files.ts` is the single remote-READ layer (`buildRemoteFileCommand` = `buildSshConnectionArgs` + one shellescaped remote command; `remoteProbePaths` returns remote realpath + stat; `remoteCreateReadStream` streams a `Range` via `tail -c +N | head -c L` and its `close()` must be wired to the response's `close` or the ssh child outlives an aborted download). The guard order matches the local path exactly (`validateSessionFilePathLexical` → remote realpath of BOTH file and workspace root → containment → sensitive-path → size cap on the REMOTE size), a request path arrives from the browser and is only ever interpolated as a `shellescape`d token, and an unreachable host answers **502**, never a 404. This covers the ATTACHMENT routes too, which is the half a clicked path needs when the file is OUTSIDE the case directory (`_isExternalPreviewPath` sends it to `POST …/attachments`): registration, by-id `raw`, metadata and the history list all resolve over ssh (`registerExternalAttachment({remote})`, `resolveServableRemoteAttachment`), and what decides the host is the SESSION, never the path string — the same absolute path means a different file on each host. Deliberately NOT supported over ssh: writes (`edit=1`/`PUT` answer 400, `editable` is always false), office previews/thumbnails, the file tree/picker, `tail-file`. Tests: `test/remote-files.test.ts`, `test/routes/file-routes-remote.test.ts`.
|
||||
**Remote SSH cases** (COD-94/#145): cases can point at a **remote host** (`~/.codeman/remote-hosts.json` + `remote-cases.json` via `src/remote-hosts.ts`; CRUD under `/api/cases` — cases route file). A remote session launches a LOCAL tmux pane running `ssh <host>` that creates a durable REMOTE tmux session on a **dedicated socket** `-L codeman-remote` with name `codeman-ssh-<id>` — deliberately failing the remote Codeman's `SAFE_MUX_NAME_PATTERN` so a Codeman instance on the target host never adopts it; no `-g` global tmux options are set remotely. `remotePath`/`identityFile` are schema-guarded against shell injection (backticks/`$` rejected — same approach as `extraSshOptions`); remote tmux availability is probed via `checkRemoteTmuxAvailable()` in quick-start (ssh args carry `-o ConnectTimeout=10`). Remote claude defaults to an idempotent `claude --session-id <id> || claude --resume <id>` pair under a login shell, so a respawn or reattach continues the SAME conversation rather than starting a fresh one (remote omp gets the same treatment via `--continue`; ⚠️ because the claude arm is an `a || b` pair under `-c`, that pane's PID is the login shell, not the agent); per-host `commands.*` override. Session kill best-effort kills the remote tmux too. `SessionState.remote`/`MuxSession.remote` round-trip through recovery (`restoreMuxSessions` passes `remote` back into the Session constructor). ⚠️ Run flows must route remote cases through `POST /api/quick-start` (which resolves the remote case and skips LOCAL CLI availability gates) — `POST /api/sessions` stat-validates `workingDir` locally and has no `caseName`. `envOverrides`/`effort`/`modelOverride`/`codexConfig`/`geminiConfig` are rejected for remote quick-starts (not silently dropped). UI: Create Case modal → Remote tab. Tests: `test/remote-hosts.test.ts`, `test/remote-ssh-options.test.ts`. ⚠️ **Reading a file in a remote case goes over ssh too** (#415): `src/remote-files.ts` is the single remote-READ layer (`buildRemoteFileCommand` = `buildSshConnectionArgs` + one shellescaped remote command; `remoteProbePaths` returns remote realpath + stat; `remoteCreateReadStream` streams a `Range` via `tail -c +N | head -c L` and its `close()` must be wired to the response's `close` or the ssh child outlives an aborted download). The guard order matches the local path exactly (`validateSessionFilePathLexical` → remote realpath of BOTH file and workspace root → containment → sensitive-path → size cap on the REMOTE size), a request path arrives from the browser and is only ever interpolated as a `shellescape`d token, and an unreachable host answers **502**, never a 404. ⚠️ The probe's symlink resolution FAILS CLOSED: `readlink -f` where it exists, otherwise a `cd -P`/`pwd -P` directory walk plus a bounded plain-`readlink` loop over the last component, and anything it cannot fully resolve is reported unresolvable (404), never as the unresolved string — the first version resolved the directory chain only, so on a host without `readlink -f` a `ws/notes.txt -> ~/.ssh/id_rsa` link passed containment under its own path while `cat` served the key. Records are NUL-separated and index-keyed so a newline in a filename cannot shift the mapping. ⚠️ ssh children are BOUNDED: probes and buffered reads go through `src/remote-ssh-limiter.ts` (a `document-conversion-limiter`-shaped semaphore, default 4), the attachment-history list probes its whole history in ONE batched call (`probeRemoteAttachmentHistory`, threaded into `registerExternalAttachment({remoteProbes})`), and probes chunk at 40 paths — a prompt-injected agent printing `codeman://attach` links in a remote session used to fork one `ssh` per link. `describeExecError` never returns Node's `Command failed: <ssh line>` message (identity path + probe script in a 502 body). The `PUT /file-content` guard sits AHEAD of `validateSessionFilePath`, which resolves LOCALLY, or a same-named local directory (an sshfs mount) takes the write. Under `VITEST` the three IO functions refuse rather than connect. This covers the ATTACHMENT routes too, which is the half a clicked path needs when the file is OUTSIDE the case directory (`_isExternalPreviewPath` sends it to `POST …/attachments`): registration, by-id `raw`, metadata and the history list all resolve over ssh (`registerExternalAttachment({remote})`, `resolveServableRemoteAttachment`), and what decides the host is the SESSION, never the path string — the same absolute path means a different file on each host. Deliberately NOT supported over ssh: writes (`edit=1`/`PUT` answer 400, `editable` is always false), office previews/thumbnails, the file tree/picker, `tail-file`. Tests: `test/remote-files.test.ts`, `test/routes/file-routes-remote.test.ts`.
|
||||
|
||||
### Docker cases
|
||||
|
||||
|
||||
+33
-5
@@ -265,11 +265,12 @@ and it follows the same rule as the launch path: every ssh command line comes fr
|
||||
|---------|--------------|
|
||||
| `GET /api/sessions/:id/file-raw` | Streamed over `ssh` (`cat`, or `tail -c +N \| head -c L` for a `Range`); the same 200/206/416 contract as a local file, so `<video>`/`<audio>` seeking works |
|
||||
| `GET /api/sessions/:id/file-content` | `cat` into memory, capped by the existing text limit; `edit=1` answers `400` (see below) and `editable` is always `false` |
|
||||
| `PUT /api/sessions/:id/file-content` | `400` before any path is looked at: the guard sits AHEAD of the local path validation, because with a same-named directory on the Codeman host (an `sshfs` mount) the write would otherwise land on the local twin |
|
||||
| `GET /api/sessions/:id/file-preview` | Non-office files redirect to `file-raw` (which works remotely); docx/pptx answer `400` |
|
||||
| `GET /api/sessions/:id/file-thumbnail` | `400` for remote files |
|
||||
| `POST /api/sessions/:id/attachments` | Registers an absolute path that lives on the **remote** host (a clicked link pointing outside the case directory) by probing it there |
|
||||
| `GET /api/sessions/:id/attachments/:attachmentId/raw` | Streams the registered remote file over ssh, same 200/206/416 contract; `preview` (office) and `thumbnail` answer `400` |
|
||||
| `GET /api/sessions/:id/attachments/:attachmentId`, `GET …/attachments` (history) | Size/mtime/existence resolved over ssh, so a remote entry is not reported `missing` |
|
||||
| `GET /api/sessions/:id/attachments/:attachmentId`, `GET …/attachments` (history) | Size/mtime/existence resolved over ssh, so a remote entry is not reported `missing`; the history list resolves EVERY entry in one batched probe, never one connection per entry |
|
||||
|
||||
⚠️ The attachment route is the one a clicked path takes when it is **outside** the case
|
||||
directory (a remote `/tmp` scratchpad capture, a screenshot elsewhere in the home dir):
|
||||
@@ -284,13 +285,27 @@ by the transport:
|
||||
connection is opened.
|
||||
3. ONE ssh round trip that returns `realpath` **and** `stat` for the path **and** the
|
||||
workspace root (`remoteProbePaths`). Resolving the root remotely is what keeps the
|
||||
boundary honest for a symlinked `remotePath`; the probe uses `readlink -f` when
|
||||
available and a POSIX `cd`/`pwd -P` fallback otherwise.
|
||||
boundary honest for a symlinked `remotePath`. The probe uses `readlink -f` when
|
||||
available; on a host without it (macOS before 12.3) a POSIX fallback canonicalizes
|
||||
the directory chain with `cd -P`/`pwd -P` and then follows the LAST component with
|
||||
plain `readlink` for a bounded number of hops. ⚠️ **The fallback fails closed**: a
|
||||
path it cannot fully resolve (a loop, a `readlink` failure, the hop cap) is reported
|
||||
as unresolvable and answers 404, never as its own unresolved string. An earlier
|
||||
version resolved only the directory chain, so `ws/notes.txt -> ~/.ssh/id_rsa` passed
|
||||
containment under the link's own path while `cat` followed it to the key.
|
||||
Records come back NUL-separated and index-keyed (`<index>|kind|size|mtime|realPath`,
|
||||
after a leading NUL that fences off any login banner), so a filename containing a
|
||||
newline cannot shift the alignment.
|
||||
4. Containment of the remote realpath against the remote root. The sensitive-path
|
||||
blocklist then applies on whichever routes already apply it locally (`/api/download`,
|
||||
attachment registration, edit mode — where resolving symlinks first is what makes it
|
||||
meaningful); the remote branch neither drops a guard the local path has nor invents a
|
||||
stricter one.
|
||||
stricter one. One entry of that blocklist is host-bound by construction: the three
|
||||
home-anchored members (`~/.claude.json`, `~/.claude/settings.json`,
|
||||
`~/.claude/settings.local.json`) are compared against the **Codeman host's** home
|
||||
directory, so they do not match a remote home at a different path. Everything else in
|
||||
the list is depth-anchored (`/.ssh/`, `/.aws/credentials`, `/.claude/.credentials.json`,
|
||||
`/etc/shadow`, ...) and applies to a remote path unchanged.
|
||||
5. Size cap (`CODEMAN_MAX_DOWNLOAD_BYTES`) applied to the **remote** size, before the
|
||||
body is requested.
|
||||
|
||||
@@ -298,7 +313,20 @@ The path arrives from the browser (`?path=`) and is interpolated as a single
|
||||
`shellescape`-quoted token, in a command that is itself shellescaped into the ssh
|
||||
line; `BatchMode=yes` means a host needing a passphrase fails fast instead of hanging.
|
||||
A failed connection is reported as **502** with the remote reason — never a 404, which
|
||||
used to make an unreachable host look like a typo in the agent's output.
|
||||
used to make an unreachable host look like a typo in the agent's output. The reason is
|
||||
the first stderr line, the timeout, or the exit code; never Node's `Command failed: …`
|
||||
message, which would carry the identity-file path and the probe script into the body.
|
||||
|
||||
**Connections are bounded.** Every probe and buffered read runs through a small global
|
||||
semaphore (`src/remote-ssh-limiter.ts`, default 4, `CODEMAN_MAX_REMOTE_FILE_SSH`), the
|
||||
attachment-history list resolves its whole history in one batched probe instead of one
|
||||
handshake per entry, and probes are chunked at 40 paths per round trip. Terminal output
|
||||
in a remote session is written on the remote host, so a prompt-injected agent printing
|
||||
hundreds of `codeman://attach` links used to make the server fork one `ssh` per link,
|
||||
each holding a 20 s probe timeout, and a 100-entry history re-listed on every
|
||||
`attachment:detected` event tripped OpenSSH's default `MaxStartups 10:30:100`. Streams
|
||||
(`file-raw`, by-id `raw`) are not counted: one is held per browser request for the life
|
||||
of a playback, and each is gated behind a counted probe anyway.
|
||||
|
||||
⚠️ **There is deliberately NO local fallback.** A remote case reads the remote bytes or
|
||||
fails, even when a file with the same absolute name exists on the Codeman host — which
|
||||
|
||||
+31
-12
@@ -13,7 +13,7 @@ import { basename, extname, isAbsolute } from 'node:path';
|
||||
import { isBlockedAttachmentPath, isUnderTree, loadAttachmentGuardConfig } from './config/attachment-guard.js';
|
||||
import { EDITABLE_EXTENSIONS } from './config/file-editing.js';
|
||||
import { validateSessionFilePath } from './web/route-helpers.js';
|
||||
import { remoteProbePaths, RemoteFileAccessError } from './remote-files.js';
|
||||
import { remoteProbePaths, RemoteFileAccessError, type RemoteProbe } from './remote-files.js';
|
||||
import type { AttachmentDetectedEvent, AttachmentDetectedType } from './types.js';
|
||||
import type { SessionRemote } from './types/session.js';
|
||||
|
||||
@@ -228,6 +228,13 @@ export interface RegisterExternalAttachmentOptions {
|
||||
* so a symlinked `remotePath` does not refuse every registration.
|
||||
*/
|
||||
remote?: SessionRemote;
|
||||
/**
|
||||
* Remote only: `[file, workspaceRoot]` probes a caller already resolved in a BATCHED
|
||||
* `remoteProbePaths` call (the attachment-history list does one round trip for the
|
||||
* whole history). Skips this registration's own ssh probe; every guard below still
|
||||
* runs on the same resolved path it would have produced itself.
|
||||
*/
|
||||
remoteProbes?: readonly [RemoteProbe | null, RemoteProbe | null];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -277,17 +284,24 @@ async function resolveLocalAttachment(requestedPath: string): Promise<ResolvedAt
|
||||
async function resolveRemoteAttachment(
|
||||
requestedPath: string,
|
||||
remote: SessionRemote,
|
||||
sessionWorkingDir?: string
|
||||
sessionWorkingDir?: string,
|
||||
preResolved?: readonly [RemoteProbe | null, RemoteProbe | null]
|
||||
): Promise<ResolvedAttachmentFile> {
|
||||
const paths = sessionWorkingDir ? [requestedPath, sessionWorkingDir] : [requestedPath];
|
||||
let probes;
|
||||
try {
|
||||
probes = await remoteProbePaths(remote, paths);
|
||||
} catch (err) {
|
||||
throw new AttachmentRegistrationError(
|
||||
err instanceof RemoteFileAccessError ? err.message : 'remote host unreachable',
|
||||
502
|
||||
);
|
||||
let probes: ReadonlyArray<RemoteProbe | null>;
|
||||
if (preResolved) {
|
||||
probes = preResolved;
|
||||
} else {
|
||||
try {
|
||||
probes = await remoteProbePaths(remote, paths);
|
||||
} catch (err) {
|
||||
// 502 marks the TRANSPORT as the failure, distinct from the file's own 404/403,
|
||||
// so a history listing can report the entry as unknown rather than missing.
|
||||
throw new AttachmentRegistrationError(
|
||||
err instanceof RemoteFileAccessError ? err.message : 'remote host unreachable',
|
||||
502
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const [probe, rootProbe] = probes;
|
||||
@@ -315,7 +329,7 @@ export async function registerExternalAttachment(
|
||||
}
|
||||
|
||||
const resolved = await (options.remote
|
||||
? resolveRemoteAttachment(requestedPath, options.remote, options.sessionWorkingDir)
|
||||
? resolveRemoteAttachment(requestedPath, options.remote, options.sessionWorkingDir, options.remoteProbes)
|
||||
: resolveLocalAttachment(requestedPath));
|
||||
|
||||
// COD-53: enforce the active attachment-guard policy on the symlink-resolved
|
||||
@@ -343,7 +357,12 @@ export async function registerExternalAttachment(
|
||||
// codeman-publish and the ~/.codeman review loop keep working.
|
||||
//
|
||||
// The list is a pattern list over ABSOLUTE paths, so it is host-agnostic and holds
|
||||
// for a remote path exactly as it does for a local one.
|
||||
// for a remote path exactly as it does for a local one, with ONE exception worth
|
||||
// knowing: `isSensitivePath`'s three home-anchored members (`~/.claude.json`,
|
||||
// `~/.claude/settings.json`, `~/.claude/settings.local.json`) resolve against THIS
|
||||
// host's `homedir()`, so on a remote host with a different home they do not match.
|
||||
// Everything else in that list is depth-anchored (`/.ssh/`, `/.aws/credentials`,
|
||||
// `/.claude/.credentials.json`, ...) and applies unchanged.
|
||||
if (isBlockedAttachmentPath(resolved.resolvedPath, guard.blockedTrees)) {
|
||||
throw new AttachmentRegistrationError('Access to this file is blocked', 403);
|
||||
}
|
||||
|
||||
+154
-52
@@ -28,9 +28,10 @@
|
||||
|
||||
import { exec, spawn } from 'node:child_process';
|
||||
import { promisify } from 'node:util';
|
||||
import type { Readable } from 'node:stream';
|
||||
import { PassThrough, type Readable } from 'node:stream';
|
||||
import type { SessionRemote } from './types/session.js';
|
||||
import { buildSshConnectionArgs, remoteSshTarget, shellescape } from './remote-hosts.js';
|
||||
import { runWithRemoteSshLimit } from './remote-ssh-limiter.js';
|
||||
|
||||
const execAsync = promisify(exec);
|
||||
|
||||
@@ -50,6 +51,37 @@ const READ_BUFFER_SLACK_BYTES = 64 * 1024;
|
||||
/** Marker a probe prints when the path does not exist on the remote host. */
|
||||
const NOT_FOUND_MARKER = 'n';
|
||||
|
||||
/**
|
||||
* Marker a probe prints when the path exists but could NOT be canonicalized (no
|
||||
* `readlink -f`, and the portable fallback hit its hop cap or a `readlink` failure).
|
||||
* Parsed as `null`, i.e. 404: a path whose real target is unknown must never be
|
||||
* served, because every containment and blocklist check runs on the resolved path.
|
||||
*/
|
||||
const UNRESOLVABLE_MARKER = 'x';
|
||||
|
||||
/**
|
||||
* Paths per ssh round trip. The whole remote script is ONE shellescaped argument,
|
||||
* and Linux caps a single argv string at 128 KiB, so a 100-entry attachment history
|
||||
* of long paths is split rather than risking `E2BIG` on the local `sh`.
|
||||
*/
|
||||
const REMOTE_PROBE_CHUNK_SIZE = 40;
|
||||
|
||||
/** Symlink hops the portable resolver follows before giving up (Linux uses 40). */
|
||||
const REMOTE_SYMLINK_MAX_HOPS = 40;
|
||||
|
||||
/**
|
||||
* Under vitest no real ssh connection may ever be opened (mirrors
|
||||
* `checkRemoteTmuxAvailable` and friends in remote-hosts.ts). The route tests mock
|
||||
* this module, so nothing reaches here today; this is what keeps the NEXT
|
||||
* remote-session test that touches a file route from opening a connection from CI.
|
||||
* A clear 502-shaped error, never a fake success: there are no fake bytes to return.
|
||||
*/
|
||||
function assertNotUnderTest(): void {
|
||||
if (process.env.VITEST) {
|
||||
throw new RemoteFileAccessError('remote file access is disabled under test');
|
||||
}
|
||||
}
|
||||
|
||||
/** What a remote path turned out to be. `other` = symlink/socket/fifo/device. */
|
||||
export type RemotePathKind = 'file' | 'directory' | 'other';
|
||||
|
||||
@@ -95,40 +127,67 @@ export function buildRemoteFileCommand(remote: SessionRemote, shellCommand: stri
|
||||
* every extra `ssh` is a fresh handshake, and the file routes need the path AND the
|
||||
* workspace root canonicalized to compare them.
|
||||
*
|
||||
* Each path emits exactly one line — `n` when it does not exist, otherwise
|
||||
* `kind|size|mtime|realPath` with `realPath` LAST so a path containing `|` still
|
||||
* parses (the earlier fields are fixed and the remainder is the path).
|
||||
* Output format: the script first prints a lone NUL, then one NUL-terminated record
|
||||
* per path, `<index>|n` (missing), `<index>|x` (exists but cannot be canonicalized) or
|
||||
* `<index>|kind|size|mtime|realPath`. Records are keyed by INDEX and separated by NUL
|
||||
* rather than newline so that a remote filename containing a newline cannot shift the
|
||||
* alignment, and the leading NUL is what separates a login banner or an eager rc-file
|
||||
* `echo` (which land before the script runs) from the records without any "last N
|
||||
* lines" guesswork. `realPath` is the last field, so a `|` in a path still parses.
|
||||
*
|
||||
* Symlink resolution is portable on purpose: `readlink -f` where available (Linux,
|
||||
* macOS >= 12.3), else the POSIX `cd`/`pwd -P` fallback, which resolves the DIRECTORY
|
||||
* chain. Resolution is required here rather than optional: `isSensitivePath()`
|
||||
* demands an already-realpath'd input, so a remote read must not be able to reach a
|
||||
* blocked target through a symlink any more than a local one can.
|
||||
* Symlink resolution is portable AND fails closed. `readlink -f` where available
|
||||
* (Linux, macOS >= 12.3); otherwise the fallback canonicalizes the directory chain
|
||||
* with `cd -P`/`pwd -P` and then follows the LAST component with plain `readlink`
|
||||
* (which the systems lacking `-f` do have) for a bounded number of hops. A path the
|
||||
* fallback cannot resolve prints `x`, never the unresolved string: every containment
|
||||
* and blocklist check downstream runs on `realPath`, and an earlier version of this
|
||||
* fallback returned the directory-resolved path with the final symlink still in it,
|
||||
* so `ws/notes.txt -> ~/.ssh/id_rsa` passed containment while `cat` served the key.
|
||||
*/
|
||||
export function buildRemoteProbeCommand(paths: readonly string[]): string {
|
||||
const probes = paths.map((path) => `probe ${shellescape(path)}`).join('\n');
|
||||
const probes = paths.map((path, index) => `probe ${index} ${shellescape(path)}`).join('\n');
|
||||
return [
|
||||
'resolve_last() {',
|
||||
' q=$1',
|
||||
' hops=0',
|
||||
' while :; do',
|
||||
' d=$(cd -P "$(dirname "$q")" 2>/dev/null && pwd -P) || return 1',
|
||||
' q=$d/$(basename "$q")',
|
||||
' [ -L "$q" ] || break',
|
||||
' hops=$((hops + 1))',
|
||||
` [ "$hops" -le ${REMOTE_SYMLINK_MAX_HOPS} ] || return 1`,
|
||||
' l=$(readlink "$q" 2>/dev/null) || return 1',
|
||||
' [ -n "$l" ] || return 1',
|
||||
' case $l in /*) q=$l ;; *) q=$d/$l ;; esac',
|
||||
' done',
|
||||
' if [ -d "$q" ]; then q=$(cd -P "$q" 2>/dev/null && pwd -P) || return 1; fi',
|
||||
' printf %s "$q"',
|
||||
'}',
|
||||
'probe() {',
|
||||
' p=$1',
|
||||
' r=$(readlink -f "$p" 2>/dev/null) || r=$(cd "$(dirname "$p")" 2>/dev/null && printf %s/%s "$(pwd -P)" "$(basename "$p")")',
|
||||
' [ -n "$r" ] || r=$p',
|
||||
` if [ ! -e "$p" ]; then printf '%s\\n' ${NOT_FOUND_MARKER}; return; fi`,
|
||||
' i=$1',
|
||||
' p=$2',
|
||||
` if [ ! -e "$p" ]; then printf '%s|${NOT_FOUND_MARKER}\\0' "$i"; return; fi`,
|
||||
` r=$(readlink -f "$p" 2>/dev/null) || r=$(resolve_last "$p") || { printf '%s|${UNRESOLVABLE_MARKER}\\0' "$i"; return; }`,
|
||||
` [ -n "$r" ] || { printf '%s|${UNRESOLVABLE_MARKER}\\0' "$i"; return; }`,
|
||||
' if [ -d "$r" ]; then t=d; elif [ -f "$r" ]; then t=f; else t=o; fi',
|
||||
' s=0',
|
||||
' if [ "$t" = f ]; then s=$(wc -c < "$r" 2>/dev/null | tr -d " "); [ -n "$s" ] || s=0; fi',
|
||||
' if [ "$t" = f ]; then s=$(stat -c %s "$r" 2>/dev/null || stat -f %z "$r" 2>/dev/null); [ -n "$s" ] || s=0; fi',
|
||||
' m=$(stat -c %Y "$r" 2>/dev/null || stat -f %m "$r" 2>/dev/null || printf 0)',
|
||||
` printf '%s|%s|%s|%s\\n' "$t" "$s" "$m" "$r"`,
|
||||
` printf '%s|%s|%s|%s|%s\\0' "$i" "$t" "$s" "$m" "$r"`,
|
||||
'}',
|
||||
"printf '\\0'",
|
||||
probes,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
/** Parse one probe line. `null` for the not-found marker or anything malformed. */
|
||||
export function parseRemoteProbeLine(line: string): RemoteProbe | null {
|
||||
const trimmed = line.replace(/\r$/, '');
|
||||
if (!trimmed || trimmed === NOT_FOUND_MARKER) return null;
|
||||
/**
|
||||
* Parse one probe record (index prefix already stripped). `null` for the not-found
|
||||
* and unresolvable markers or anything malformed.
|
||||
*/
|
||||
export function parseRemoteProbeRecord(record: string): RemoteProbe | null {
|
||||
if (!record || record === NOT_FOUND_MARKER || record === UNRESOLVABLE_MARKER) return null;
|
||||
|
||||
const parts = trimmed.split('|');
|
||||
const parts = record.split('|');
|
||||
if (parts.length < 4) return null;
|
||||
|
||||
const [kindRaw, sizeRaw, mtimeRaw] = parts;
|
||||
@@ -151,48 +210,77 @@ export function parseRemoteProbeLine(line: string): RemoteProbe | null {
|
||||
|
||||
/**
|
||||
* Parse the output of {@link buildRemoteProbeCommand} into one entry per requested
|
||||
* path, in order. Throws when the output cannot be one line per path — that means the
|
||||
* transport or the remote shell did something unexpected, and silently treating it as
|
||||
* "not found" would turn an infrastructure failure into a wrong 404.
|
||||
* path, in order. Throws when a path's record is missing: that means the transport
|
||||
* or the remote shell did something unexpected, and silently treating it as "not
|
||||
* found" would turn an infrastructure failure into a wrong 404.
|
||||
*
|
||||
* The LAST `paths.length` lines are used so a login banner or an eager rc-file `echo`
|
||||
* on the remote host cannot shift the alignment.
|
||||
* Everything before the first NUL is the remote shell's own chatter (banner, rc-file
|
||||
* output) and is discarded; records are matched by their index prefix, so neither
|
||||
* extra output nor a newline inside a filename can shift the mapping.
|
||||
*/
|
||||
export function parseRemoteProbeLines(stdout: string, paths: readonly string[]): Array<RemoteProbe | null> {
|
||||
const lines = stdout.split('\n').filter((line) => line !== '');
|
||||
if (lines.length < paths.length) {
|
||||
throw new RemoteFileAccessError('remote host returned no usable file information');
|
||||
export function parseRemoteProbeOutput(stdout: string, paths: readonly string[]): Array<RemoteProbe | null> {
|
||||
const records = stdout.split('\0').slice(1);
|
||||
const byIndex = new Map<number, string>();
|
||||
for (const record of records) {
|
||||
const match = /^(\d+)\|([\s\S]*)$/.exec(record);
|
||||
if (!match) continue;
|
||||
const index = Number.parseInt(match[1], 10);
|
||||
if (!byIndex.has(index)) byIndex.set(index, match[2]);
|
||||
}
|
||||
return lines.slice(-paths.length).map((line) => parseRemoteProbeLine(line));
|
||||
return paths.map((_, index) => {
|
||||
const record = byIndex.get(index);
|
||||
if (record === undefined) {
|
||||
throw new RemoteFileAccessError('remote host returned no usable file information');
|
||||
}
|
||||
return parseRemoteProbeRecord(record);
|
||||
});
|
||||
}
|
||||
|
||||
/** Probe one or more remote paths. Entry is `null` for a path that does not exist. */
|
||||
/**
|
||||
* Probe one or more remote paths. Entry is `null` for a path that does not exist (or
|
||||
* could not be canonicalized, which is refused the same way).
|
||||
*
|
||||
* Large batches are split into round trips of {@link REMOTE_PROBE_CHUNK_SIZE}, each
|
||||
* counted against the global ssh limiter, so an attachment history of 100 entries
|
||||
* costs three connections in sequence rather than 100 at once.
|
||||
*/
|
||||
export async function remoteProbePaths(
|
||||
remote: SessionRemote,
|
||||
paths: readonly string[]
|
||||
): Promise<Array<RemoteProbe | null>> {
|
||||
const command = buildRemoteFileCommand(remote, buildRemoteProbeCommand(paths));
|
||||
let stdout: string;
|
||||
try {
|
||||
const result = await execAsync(command, { timeout: REMOTE_PROBE_TIMEOUT_MS, maxBuffer: 64 * 1024 });
|
||||
stdout = result.stdout;
|
||||
} catch (err) {
|
||||
throw new RemoteFileAccessError(
|
||||
`remote host ${remote.label || remote.host} unreachable: ${describeExecError(err)}`
|
||||
);
|
||||
assertNotUnderTest();
|
||||
const results: Array<RemoteProbe | null> = [];
|
||||
for (let offset = 0; offset < paths.length; offset += REMOTE_PROBE_CHUNK_SIZE) {
|
||||
const chunk = paths.slice(offset, offset + REMOTE_PROBE_CHUNK_SIZE);
|
||||
const command = buildRemoteFileCommand(remote, buildRemoteProbeCommand(chunk));
|
||||
let stdout: string;
|
||||
try {
|
||||
const result = await runWithRemoteSshLimit(() =>
|
||||
execAsync(command, { timeout: REMOTE_PROBE_TIMEOUT_MS, maxBuffer: 256 * 1024 })
|
||||
);
|
||||
stdout = result.stdout;
|
||||
} catch (err) {
|
||||
throw new RemoteFileAccessError(
|
||||
`remote host ${remote.label || remote.host} unreachable: ${describeExecError(err)}`
|
||||
);
|
||||
}
|
||||
results.push(...parseRemoteProbeOutput(stdout, chunk));
|
||||
}
|
||||
return parseRemoteProbeLines(stdout, paths);
|
||||
return results;
|
||||
}
|
||||
|
||||
/** Read a whole remote file into memory, capped by `maxBytes`. */
|
||||
export async function remoteReadFile(remote: SessionRemote, remotePath: string, maxBytes: number): Promise<Buffer> {
|
||||
assertNotUnderTest();
|
||||
const command = buildRemoteFileCommand(remote, `cat ${shellescape(remotePath)}`);
|
||||
try {
|
||||
const result = await execAsync(command, {
|
||||
timeout: REMOTE_READ_TIMEOUT_MS,
|
||||
maxBuffer: maxBytes + READ_BUFFER_SLACK_BYTES,
|
||||
encoding: 'buffer',
|
||||
});
|
||||
const result = await runWithRemoteSshLimit(() =>
|
||||
execAsync(command, {
|
||||
timeout: REMOTE_READ_TIMEOUT_MS,
|
||||
maxBuffer: maxBytes + READ_BUFFER_SLACK_BYTES,
|
||||
encoding: 'buffer',
|
||||
})
|
||||
);
|
||||
return Buffer.isBuffer(result.stdout) ? result.stdout : Buffer.from(result.stdout);
|
||||
} catch (err) {
|
||||
throw new RemoteFileAccessError(`failed to read remote file: ${describeExecError(err)}`);
|
||||
@@ -238,6 +326,13 @@ export function remoteCreateReadStream(
|
||||
remotePath: string,
|
||||
range?: { start: number; end: number }
|
||||
): RemoteFileStream {
|
||||
if (process.env.VITEST) {
|
||||
// Same rule as the buffered calls, in stream form: the consumer sees the error
|
||||
// through the stream's normal failure path instead of a connection attempt.
|
||||
const stream = new PassThrough();
|
||||
process.nextTick(() => stream.destroy(new RemoteFileAccessError('remote file access is disabled under test')));
|
||||
return { stream, close: () => stream.destroy() };
|
||||
}
|
||||
const command = buildRemoteFileCommand(remote, buildRemoteReadCommand(remotePath, range));
|
||||
const child = spawn(command, { shell: true, stdio: ['ignore', 'pipe', 'pipe'] });
|
||||
|
||||
@@ -277,20 +372,27 @@ export function remoteCreateReadStream(
|
||||
};
|
||||
}
|
||||
|
||||
/** First useful line of an exec/stderr error, for a user-facing message. */
|
||||
/**
|
||||
* First useful line of an exec/stderr error, for a user-facing message.
|
||||
*
|
||||
* ⚠️ Never Node's `err.message`: for a failed `exec` it is `Command failed: <the whole
|
||||
* ssh line>`, which carries the identity-file path and the probe script, and this
|
||||
* string goes out in a 502 body. stderr, the timeout flag and the exit/spawn code are
|
||||
* everything a user can act on.
|
||||
*/
|
||||
function describeExecError(err: unknown): string {
|
||||
if (typeof err === 'object' && err !== null) {
|
||||
const record = err as { stderr?: unknown; message?: unknown; code?: unknown; killed?: unknown };
|
||||
const record = err as { stderr?: unknown; code?: unknown; killed?: unknown };
|
||||
const stderr =
|
||||
typeof record.stderr === 'string' ? record.stderr : Buffer.isBuffer(record.stderr) ? String(record.stderr) : '';
|
||||
const line = stderr
|
||||
.split('\n')
|
||||
.map((entry) => entry.trim())
|
||||
.find((entry) => entry.length > 0);
|
||||
if (line) return line;
|
||||
if (line) return line.slice(0, 300);
|
||||
if (record.killed) return 'timed out';
|
||||
if (typeof record.message === 'string' && record.message.length > 0) return record.message;
|
||||
if (typeof record.code === 'string' || typeof record.code === 'number') return `ssh exit ${record.code}`;
|
||||
if (typeof record.code === 'number') return `ssh exit ${record.code}`;
|
||||
if (typeof record.code === 'string') return `ssh could not be started (${record.code})`;
|
||||
}
|
||||
return 'unknown error';
|
||||
}
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
/**
|
||||
* @fileoverview Global concurrency limiter for the short-lived `ssh` children that
|
||||
* remote-case file access spawns (`src/remote-files.ts`: the realpath+stat probe and
|
||||
* the buffered text read).
|
||||
*
|
||||
* Two paths can fan those out without a human behind each one:
|
||||
*
|
||||
* - `GET /api/sessions/:id/attachments` resolves every history entry (up to
|
||||
* `ATTACHMENT_HISTORY_LIMIT`, 100), and the attachments drawer re-runs it on every
|
||||
* `attachment:detected` event while it is open, which is exactly when an agent is
|
||||
* writing files. The route now batches the probes, but a burst of drawers is still
|
||||
* a burst.
|
||||
* - A `codeman://attach?path=` magic link in terminal output registers the path
|
||||
* fire-and-forget, once per distinct link per PTY chunk. In a remote session that
|
||||
* output is written by a process on the remote host, so a prompt-injected agent can
|
||||
* print hundreds of links and have the server fork one `ssh` per link, each holding
|
||||
* a 20s probe timeout.
|
||||
*
|
||||
* Without a cap that is the fork-bomb shape `document-conversion-limiter.ts` exists to
|
||||
* prevent, and it also trips OpenSSH's default `MaxStartups 10:30:100`, which starts
|
||||
* dropping connections at ten unauthenticated handshakes. This is that limiter for
|
||||
* ssh: a small fixed pool, FIFO queueing, and a slot handed straight to the next
|
||||
* waiter on release so the active count can never exceed the cap under interleaved
|
||||
* async resumption.
|
||||
*
|
||||
* Streams (`remoteCreateReadStream`) are deliberately NOT counted: one is opened per
|
||||
* browser request and held for the life of a media playback, so four open videos
|
||||
* would otherwise block every preview and the history list. They are already gated
|
||||
* behind a counted probe (the guard re-probe runs first), so their spawn RATE is
|
||||
* bounded here even though their concurrency is bounded by the browser.
|
||||
*
|
||||
* NOT re-entrant: never acquire from inside a task already holding a slot.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Max remote probes/reads allowed to run concurrently across the whole process.
|
||||
* Override with CODEMAN_MAX_REMOTE_FILE_SSH (clamped to >= 1). Four keeps a burst
|
||||
* well under OpenSSH's ten-handshake default.
|
||||
*/
|
||||
const MAX_CONCURRENT_REMOTE_SSH = (() => {
|
||||
const raw = Number(process.env.CODEMAN_MAX_REMOTE_FILE_SSH);
|
||||
return Number.isFinite(raw) && raw >= 1 ? Math.floor(raw) : 4;
|
||||
})();
|
||||
|
||||
let active = 0;
|
||||
const waiters: Array<() => void> = [];
|
||||
|
||||
/** Test/diagnostic hook: remote calls currently holding a slot. */
|
||||
export function getActiveRemoteSshCount(): number {
|
||||
return active;
|
||||
}
|
||||
|
||||
/** Test/diagnostic hook: remote calls queued behind the cap. */
|
||||
export function getQueuedRemoteSshCount(): number {
|
||||
return waiters.length;
|
||||
}
|
||||
|
||||
/** The configured cap, so a test can assert against the real number. */
|
||||
export function getRemoteSshLimit(): number {
|
||||
return MAX_CONCURRENT_REMOTE_SSH;
|
||||
}
|
||||
|
||||
function acquire(): Promise<void> {
|
||||
if (active < MAX_CONCURRENT_REMOTE_SSH) {
|
||||
active++;
|
||||
return Promise.resolve();
|
||||
}
|
||||
return new Promise<void>((resolve) => waiters.push(resolve));
|
||||
}
|
||||
|
||||
function release(): void {
|
||||
const next = waiters.shift();
|
||||
if (next) {
|
||||
// Hand the slot straight to the next waiter; `active` stays at the cap.
|
||||
next();
|
||||
} else {
|
||||
active--;
|
||||
}
|
||||
}
|
||||
|
||||
/** Run `task` once an ssh slot is free, releasing the slot afterward. */
|
||||
export async function runWithRemoteSshLimit<T>(task: () => Promise<T>): Promise<T> {
|
||||
await acquire();
|
||||
try {
|
||||
return await task();
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
}
|
||||
+108
-13
@@ -135,6 +135,16 @@ function sendRawStream(reply: FastifyReply, content: Readable, cleanup?: () => v
|
||||
// when the client goes away (tab closed, video seek, a cancelled fetch), or the
|
||||
// ssh process outlives the request. Registered here because this is the one place
|
||||
// that owns the response's lifecycle.
|
||||
//
|
||||
// ⚠️ Check BEFORE attaching: the guard probe that ran ahead of this is an ssh round
|
||||
// trip, and a client that gave up during it has already closed the response, so
|
||||
// `close` has already fired and a listener attached now would never run. The
|
||||
// `open()` call above still spawned the body's ssh child; reap it here instead.
|
||||
if (reply.raw.destroyed) {
|
||||
cleanup?.();
|
||||
content.destroy();
|
||||
return;
|
||||
}
|
||||
if (cleanup) {
|
||||
reply.raw.on('close', cleanup);
|
||||
}
|
||||
@@ -1010,12 +1020,64 @@ function getSessionAttachmentHistory(
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The remote probes an attachment-history listing needs, resolved in ONE batch.
|
||||
*
|
||||
* The list route used to probe each entry on its own, i.e. one ssh handshake per
|
||||
* history item, up to `ATTACHMENT_HISTORY_LIMIT` (100) of them, and the attachments
|
||||
* drawer re-runs the route on every `attachment:detected` event while it is open,
|
||||
* which is exactly when an agent is writing files. OpenSSH's default
|
||||
* `MaxStartups 10:30:100` starts dropping connections at ten concurrent handshakes,
|
||||
* so most of such a burst simply failed. `remoteProbePaths` already takes an array
|
||||
* (and chunks it), so the whole history is one call, plus the global ssh limiter
|
||||
* bounding whatever is left.
|
||||
*/
|
||||
interface RemoteHistoryProbes {
|
||||
/** The workspace root, canonicalized on the remote host. */
|
||||
root: RemoteProbe | null;
|
||||
/** Keyed by the exact path handed to the probe (a lexical resolution or an external path). */
|
||||
byPath: Map<string, RemoteProbe | null>;
|
||||
/**
|
||||
* The batch itself failed (unreachable host). Every entry is then UNKNOWN, not
|
||||
* missing: reporting "missing" would tell the user their files are gone when the
|
||||
* host is merely asleep.
|
||||
*/
|
||||
unreachable: boolean;
|
||||
}
|
||||
|
||||
async function probeRemoteAttachmentHistory(
|
||||
scope: SessionFileScope,
|
||||
history: readonly SessionAttachmentHistoryItem[]
|
||||
): Promise<RemoteHistoryProbes | undefined> {
|
||||
const remote = scope.remote;
|
||||
if (!remote || history.length === 0) return undefined;
|
||||
|
||||
const paths = new Set<string>();
|
||||
for (const item of history) {
|
||||
if (item.source === 'external') {
|
||||
if (item.externalPath) paths.add(item.externalPath);
|
||||
} else if (item.relativePath) {
|
||||
const lexical = validateSessionFilePathLexical(scope.workingDir, item.relativePath);
|
||||
if (lexical) paths.add(lexical.resolvedPath);
|
||||
}
|
||||
}
|
||||
|
||||
const list = [...paths];
|
||||
try {
|
||||
const [root, ...rest] = await remoteProbePaths(remote, [scope.workingDir, ...list]);
|
||||
return { root, byPath: new Map(list.map((path, index) => [path, rest[index] ?? null])), unreachable: false };
|
||||
} catch {
|
||||
return { root: null, byPath: new Map(), unreachable: true };
|
||||
}
|
||||
}
|
||||
|
||||
// History item for a file detected inside the workspace: re-stat for live
|
||||
// size/mtime and resolve preview/thumbnail/raw routes off the relative path.
|
||||
async function buildDetectedAttachmentRouteItem(
|
||||
sessionId: string,
|
||||
scope: SessionFileScope,
|
||||
item: SessionAttachmentHistoryItem
|
||||
item: SessionAttachmentHistoryItem,
|
||||
batch?: RemoteHistoryProbes
|
||||
): Promise<AttachmentHistoryRouteItem> {
|
||||
const safe = sanitizeAttachmentHistoryItem(item);
|
||||
if (!item.relativePath) {
|
||||
@@ -1032,15 +1094,22 @@ async function buildDetectedAttachmentRouteItem(
|
||||
// inside the workspace), executed on the host that owns the files.
|
||||
const lexical = validateSessionFilePathLexical(workingDir, item.relativePath);
|
||||
if (!lexical) return { ...safe, missing: true };
|
||||
let probes: Array<RemoteProbe | null>;
|
||||
try {
|
||||
probes = await remoteProbePaths(scope.remote, [lexical.resolvedPath, workingDir]);
|
||||
} catch {
|
||||
// Unreachable host: the entry is not "missing", it is unknown. Reporting it as
|
||||
// missing would tell the user their file is gone when its host is merely asleep.
|
||||
return { ...safe, missing: false, size, mtimeMs };
|
||||
let probe: RemoteProbe | null;
|
||||
let rootProbe: RemoteProbe | null;
|
||||
if (batch) {
|
||||
// The list route resolved the whole history in one round trip.
|
||||
if (batch.unreachable) return { ...safe, missing: false, size, mtimeMs };
|
||||
probe = batch.byPath.get(lexical.resolvedPath) ?? null;
|
||||
rootProbe = batch.root;
|
||||
} else {
|
||||
try {
|
||||
[probe, rootProbe] = await remoteProbePaths(scope.remote, [lexical.resolvedPath, workingDir]);
|
||||
} catch {
|
||||
// Unreachable host: the entry is not "missing", it is unknown. Reporting it as
|
||||
// missing would tell the user their file is gone when its host is merely asleep.
|
||||
return { ...safe, missing: false, size, mtimeMs };
|
||||
}
|
||||
}
|
||||
const [probe, rootProbe] = probes;
|
||||
if (!probe || !isPathWithinRoot(rootProbe?.realPath ?? workingDir, probe.realPath)) {
|
||||
return { ...safe, missing: true };
|
||||
}
|
||||
@@ -1090,17 +1159,24 @@ async function buildDetectedAttachmentRouteItem(
|
||||
async function buildExternalAttachmentRouteItem(
|
||||
sessionId: string,
|
||||
item: SessionAttachmentHistoryItem,
|
||||
scope: SessionFileScope
|
||||
scope: SessionFileScope,
|
||||
batch?: RemoteHistoryProbes
|
||||
): Promise<AttachmentHistoryRouteItem> {
|
||||
const safe = sanitizeAttachmentHistoryItem(item);
|
||||
if (!item.externalPath) {
|
||||
return { ...safe, missing: true };
|
||||
}
|
||||
// Same answer as the detected branch for the same event: an unreachable host makes
|
||||
// the entry unknown, never missing.
|
||||
if (batch?.unreachable) {
|
||||
return { ...safe, missing: false };
|
||||
}
|
||||
|
||||
try {
|
||||
const event = await registerExternalAttachment(sessionId, item.externalPath, {
|
||||
sessionWorkingDir: scope.workingDir,
|
||||
remote: scope.remote,
|
||||
remoteProbes: batch ? [batch.byPath.get(item.externalPath) ?? null, batch.root] : undefined,
|
||||
});
|
||||
return {
|
||||
...safe,
|
||||
@@ -1118,7 +1194,9 @@ async function buildExternalAttachmentRouteItem(
|
||||
};
|
||||
} catch (err) {
|
||||
if (err instanceof AttachmentRegistrationError) {
|
||||
return { ...safe, missing: true };
|
||||
// 502 is the transport, not the file (see resolveRemoteAttachment): unknown,
|
||||
// like the detected branch. Anything else (404, 403, wrong kind) is missing.
|
||||
return { ...safe, missing: err.statusCode === 502 ? false : true };
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
@@ -1815,6 +1893,20 @@ export function registerFileRoutes(app: FastifyInstance, ctx: SessionPort & Even
|
||||
async (req): Promise<ApiResponse<FileWriteData>> => {
|
||||
const { id } = req.params as { id: string };
|
||||
const session = findSessionOrFail(ctx, id, req);
|
||||
// Remote WRITES are out of scope by design (docs/file-viewer-edit-plan.md §6),
|
||||
// and this guard must sit ahead of `validateSessionFilePath`: that helper
|
||||
// resolves against the LOCAL filesystem, so with a directory of the same
|
||||
// absolute name on this host (an sshfs mount of the remote tree, `/srv/case`,
|
||||
// a same-named home) the write would land on the local twin while the viewer
|
||||
// believes it edited the remote file. The read-remote/write-local split is
|
||||
// exactly what the no-local-fallback rule exists to prevent.
|
||||
if (session.remote) {
|
||||
throwFileEditError(
|
||||
400,
|
||||
ApiErrorCode.INVALID_INPUT,
|
||||
'Editing is not supported for files in a remote (SSH) case'
|
||||
);
|
||||
}
|
||||
const body = parseBody(FileWriteSchema, req.body);
|
||||
|
||||
// Exact byte cap — the schema's .max() counts UTF-16 code units and is
|
||||
@@ -2056,11 +2148,14 @@ export function registerFileRoutes(app: FastifyInstance, ctx: SessionPort & Even
|
||||
return;
|
||||
}
|
||||
|
||||
// Remote: every entry's realpath + stat in one batched probe, never one ssh per
|
||||
// item (see probeRemoteAttachmentHistory). Local: undefined, each item stats itself.
|
||||
const batch = await probeRemoteAttachmentHistory(sessionHistory.scope, sessionHistory.history);
|
||||
const items = await Promise.all(
|
||||
sessionHistory.history.map((item) =>
|
||||
(item.source === 'external'
|
||||
? buildExternalAttachmentRouteItem(id, item, sessionHistory.scope)
|
||||
: buildDetectedAttachmentRouteItem(id, sessionHistory.scope, item)
|
||||
? buildExternalAttachmentRouteItem(id, item, sessionHistory.scope, batch)
|
||||
: buildDetectedAttachmentRouteItem(id, sessionHistory.scope, item, batch)
|
||||
).catch(() => ({ ...sanitizeAttachmentHistoryItem(item), missing: true }))
|
||||
)
|
||||
);
|
||||
|
||||
+174
-28
@@ -16,7 +16,7 @@
|
||||
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync, mkdirSync, rmSync, writeFileSync, existsSync, statSync } from 'node:fs';
|
||||
import { mkdtempSync, mkdirSync, rmSync, writeFileSync, existsSync, statSync, chmodSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { homedir, tmpdir } from 'node:os';
|
||||
import {
|
||||
@@ -24,8 +24,11 @@ import {
|
||||
buildRemoteFileCommand,
|
||||
buildRemoteProbeCommand,
|
||||
buildRemoteReadCommand,
|
||||
parseRemoteProbeLine,
|
||||
parseRemoteProbeLines,
|
||||
parseRemoteProbeRecord,
|
||||
parseRemoteProbeOutput,
|
||||
remoteProbePaths,
|
||||
remoteReadFile,
|
||||
remoteCreateReadStream,
|
||||
} from '../src/remote-files.js';
|
||||
import type { SessionRemote } from '../src/types/session.js';
|
||||
|
||||
@@ -103,34 +106,117 @@ describe('buildRemoteProbeCommand', () => {
|
||||
const script = buildRemoteProbeCommand(['/srv/case/a.png', '/srv/case']);
|
||||
const probeCalls = script.split('\n').filter((line) => line.startsWith('probe '));
|
||||
|
||||
expect(probeCalls).toEqual(["probe '/srv/case/a.png'", "probe '/srv/case'"]);
|
||||
// The index is what the parser keys records on, so it is part of the call.
|
||||
expect(probeCalls).toEqual(["probe 0 '/srv/case/a.png'", "probe 1 '/srv/case'"]);
|
||||
});
|
||||
|
||||
it('quotes a path with spaces, quotes and a command substitution', () => {
|
||||
const nasty = "/srv/case/it's $(touch /tmp/pwned).txt";
|
||||
const script = buildRemoteProbeCommand([nasty]);
|
||||
|
||||
expect(script).toContain(`probe '/srv/case/it'\\''s $(touch /tmp/pwned).txt'`);
|
||||
expect(script).toContain(`probe 0 '/srv/case/it'\\''s $(touch /tmp/pwned).txt'`);
|
||||
expect(shellArgv(buildRemoteFileCommand(remoteFixture(), script)).at(-1)).toBe(script);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Run the probe script through a real `/bin/sh`. With `shadowReadlinkF` the PATH is
|
||||
* fronted by a `readlink` that rejects `-f` the way macOS < 12.3 does (`illegal
|
||||
* option -- f`) and otherwise defers to the real one, which forces the portable
|
||||
* fallback branch on a host that natively has `readlink -f`.
|
||||
*/
|
||||
function runProbe(paths: string[], options: { cwd?: string; shadowReadlinkF?: boolean; shimDir?: string } = {}) {
|
||||
const env =
|
||||
options.shadowReadlinkF && options.shimDir
|
||||
? { ...process.env, PATH: `${options.shimDir}:${process.env.PATH}` }
|
||||
: process.env;
|
||||
const stdout = execFileSync('sh', ['-c', buildRemoteProbeCommand(paths)], { cwd: options.cwd, env }).toString();
|
||||
return parseRemoteProbeOutput(stdout, paths);
|
||||
}
|
||||
|
||||
describe('the probe script on a real shell', () => {
|
||||
let root: string;
|
||||
let shimDir: string;
|
||||
|
||||
beforeAll(() => {
|
||||
root = mkdtempSync(join(tmpdir(), 'codeman-remote-probe-'));
|
||||
shimDir = join(root, 'shim-bin');
|
||||
mkdirSync(shimDir);
|
||||
const realReadlink = execFileSync('sh', ['-c', 'command -v readlink']).toString().trim();
|
||||
writeFileSync(
|
||||
join(shimDir, 'readlink'),
|
||||
`#!/bin/sh\ncase "$1" in -f) echo 'readlink: illegal option -- f' >&2; exit 1;; esac\nexec ${realReadlink} "$@"\n`
|
||||
);
|
||||
chmodSync(join(shimDir, 'readlink'), 0o755);
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('resolves the fallback branch on a shell whose readlink has no -f', () => {
|
||||
// Sanity check on the shim itself: without it this whole describe would be
|
||||
// exercising the native branch twice.
|
||||
expect(() =>
|
||||
execFileSync('sh', ['-c', 'readlink -f / 2>/dev/null'], {
|
||||
env: { ...process.env, PATH: `${shimDir}:${process.env.PATH}` },
|
||||
})
|
||||
).toThrow();
|
||||
});
|
||||
|
||||
it.each([
|
||||
['readlink -f', false],
|
||||
['portable fallback', true],
|
||||
])('refuses to report a symlink by its own path (%s): the target is what is served', (_label, shadow) => {
|
||||
// The reviewer's exact reproduction: ws/notes.txt -> secret/id_rsa. The old
|
||||
// fallback resolved only the DIRECTORY chain, returned `ws/notes.txt` as the
|
||||
// realpath (with the TARGET's size), containment passed, and `cat` served the key.
|
||||
const ws = join(root, `escape-${shadow ? 'fallback' : 'native'}`);
|
||||
const secret = join(root, `secret-${shadow ? 'fallback' : 'native'}`);
|
||||
mkdirSync(ws);
|
||||
mkdirSync(secret);
|
||||
writeFileSync(join(secret, 'id_rsa'), 'KEYKEYKEYKEY1');
|
||||
execFileSync('ln', ['-s', join(secret, 'id_rsa'), join(ws, 'notes.txt')]);
|
||||
|
||||
const [probe] = runProbe([join(ws, 'notes.txt')], { shadowReadlinkF: shadow, shimDir });
|
||||
|
||||
expect(probe?.realPath).toBe(join(secret, 'id_rsa'));
|
||||
expect(probe?.size).toBe(13);
|
||||
});
|
||||
|
||||
it('follows a relative symlink chain through a symlinked directory on the fallback branch', () => {
|
||||
const ws = join(root, 'chain');
|
||||
mkdirSync(join(ws, 'sub'), { recursive: true });
|
||||
writeFileSync(join(ws, 'sub', 'real.txt'), 'inside');
|
||||
execFileSync('ln', ['-s', 'real.txt', join(ws, 'sub', 'hop1.txt')]);
|
||||
execFileSync('ln', ['-s', 'hop1.txt', join(ws, 'sub', 'hop2.txt')]);
|
||||
execFileSync('ln', ['-s', 'sub', join(ws, 'subl')]);
|
||||
|
||||
const probes = runProbe([join(ws, 'subl', 'hop2.txt'), join(ws, 'subl')], { shadowReadlinkF: true, shimDir });
|
||||
|
||||
expect(probes[0]).toMatchObject({ kind: 'file', size: 6, realPath: join(ws, 'sub', 'real.txt') });
|
||||
expect(probes[1]).toMatchObject({ kind: 'directory', realPath: join(ws, 'sub') });
|
||||
});
|
||||
|
||||
it.each([
|
||||
['readlink -f', false],
|
||||
['portable fallback', true],
|
||||
])('fails CLOSED on a symlink loop (%s), never reporting the unresolved path', (_label, shadow) => {
|
||||
const ws = join(root, `loop-${shadow ? 'fallback' : 'native'}`);
|
||||
mkdirSync(ws);
|
||||
execFileSync('ln', ['-s', 'b', join(ws, 'a')]);
|
||||
execFileSync('ln', ['-s', 'a', join(ws, 'b')]);
|
||||
|
||||
const [probe] = runProbe([join(ws, 'a')], { shadowReadlinkF: shadow, shimDir });
|
||||
|
||||
expect(probe).toBeNull();
|
||||
});
|
||||
|
||||
it('reports kind, size and realpath for a file, a directory and a missing path', () => {
|
||||
const filePath = join(root, 'image.png');
|
||||
writeFileSync(filePath, 'fake png bytes');
|
||||
|
||||
const probes = parseRemoteProbeLines(
|
||||
const probes = parseRemoteProbeOutput(
|
||||
execFileSync('sh', ['-c', buildRemoteProbeCommand([filePath, root, join(root, 'nope.png')])]).toString(),
|
||||
[filePath, root, join(root, 'nope.png')]
|
||||
);
|
||||
@@ -147,7 +233,7 @@ describe('the probe script on a real shell', () => {
|
||||
writeFileSync(target, 'x');
|
||||
execFileSync('ln', ['-s', target, link]);
|
||||
|
||||
const [probe] = parseRemoteProbeLines(execFileSync('sh', ['-c', buildRemoteProbeCommand([link])]).toString(), [
|
||||
const [probe] = parseRemoteProbeOutput(execFileSync('sh', ['-c', buildRemoteProbeCommand([link])]).toString(), [
|
||||
link,
|
||||
]);
|
||||
|
||||
@@ -161,7 +247,7 @@ describe('the probe script on a real shell', () => {
|
||||
const hostile = join(root, `it's; touch ${marker}; $(id).txt`);
|
||||
writeFileSync(hostile, 'hostile');
|
||||
|
||||
const [probe] = parseRemoteProbeLines(
|
||||
const [probe] = parseRemoteProbeOutput(
|
||||
execFileSync('sh', ['-c', buildRemoteProbeCommand([hostile])], { cwd: root }).toString(),
|
||||
[hostile]
|
||||
);
|
||||
@@ -170,11 +256,39 @@ describe('the probe script on a real shell', () => {
|
||||
expect(existsSync(join(root, marker))).toBe(false);
|
||||
});
|
||||
|
||||
it('keeps a filename containing a newline aligned with its own index', () => {
|
||||
// One record per LINE would have made this two lines, shifting every record
|
||||
// after it by one; records are NUL-terminated and index-keyed instead.
|
||||
const weird = join(root, 'a\nb.txt');
|
||||
writeFileSync(weird, 'nl');
|
||||
const after = join(root, 'after.txt');
|
||||
writeFileSync(after, 'after');
|
||||
|
||||
const probes = runProbe([weird, after, join(root, 'nope')]);
|
||||
|
||||
expect(probes[0]).toMatchObject({ kind: 'file', size: 2, realPath: weird });
|
||||
expect(probes[1]).toMatchObject({ kind: 'file', size: 5, realPath: after });
|
||||
expect(probes[2]).toBeNull();
|
||||
});
|
||||
|
||||
it('discards a login banner and rc-file chatter printed before the records', () => {
|
||||
const filePath = join(root, 'banner.txt');
|
||||
writeFileSync(filePath, 'b');
|
||||
|
||||
const stdout = execFileSync('sh', [
|
||||
'-c',
|
||||
`echo 'Welcome to box'; printf '0|f|9|9|/etc/shadow\\n'; ${buildRemoteProbeCommand([filePath])}`,
|
||||
]).toString();
|
||||
|
||||
// The chatter even LOOKS like a record; the leading NUL is what fences it off.
|
||||
expect(parseRemoteProbeOutput(stdout, [filePath])[0]).toMatchObject({ realPath: filePath, size: 1 });
|
||||
});
|
||||
|
||||
it('handles a path containing the field separator', () => {
|
||||
const pipePath = join(root, 'a|b.txt');
|
||||
writeFileSync(pipePath, 'xy');
|
||||
|
||||
const [probe] = parseRemoteProbeLines(execFileSync('sh', ['-c', buildRemoteProbeCommand([pipePath])]).toString(), [
|
||||
const [probe] = parseRemoteProbeOutput(execFileSync('sh', ['-c', buildRemoteProbeCommand([pipePath])]).toString(), [
|
||||
pipePath,
|
||||
]);
|
||||
|
||||
@@ -187,7 +301,7 @@ describe('the probe script on a real shell', () => {
|
||||
mkdirSync(nested, { recursive: true });
|
||||
writeFileSync(join(nested, 'f.txt'), 'abc');
|
||||
|
||||
const [probe] = parseRemoteProbeLines(
|
||||
const [probe] = parseRemoteProbeOutput(
|
||||
execFileSync('sh', ['-c', buildRemoteProbeCommand([join(nested, 'f.txt')])]).toString(),
|
||||
[join(nested, 'f.txt')]
|
||||
);
|
||||
@@ -197,9 +311,9 @@ describe('the probe script on a real shell', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseRemoteProbeLine', () => {
|
||||
it('parses a file line and converts mtime to milliseconds', () => {
|
||||
expect(parseRemoteProbeLine('f|1234|1700000000|/srv/case/a.png')).toEqual({
|
||||
describe('parseRemoteProbeRecord', () => {
|
||||
it('parses a file record and converts mtime to milliseconds', () => {
|
||||
expect(parseRemoteProbeRecord('f|1234|1700000000|/srv/case/a.png')).toEqual({
|
||||
realPath: '/srv/case/a.png',
|
||||
kind: 'file',
|
||||
size: 1234,
|
||||
@@ -208,34 +322,66 @@ describe('parseRemoteProbeLine', () => {
|
||||
});
|
||||
|
||||
it('keeps a path that itself contains the separator', () => {
|
||||
expect(parseRemoteProbeLine('f|7|0|/srv/ca|se/a b.txt')?.realPath).toBe('/srv/ca|se/a b.txt');
|
||||
expect(parseRemoteProbeRecord('f|7|0|/srv/ca|se/a b.txt')?.realPath).toBe('/srv/ca|se/a b.txt');
|
||||
});
|
||||
|
||||
it('maps directories, other kinds and the not-found marker', () => {
|
||||
expect(parseRemoteProbeLine('d|0|5|/srv/case')?.kind).toBe('directory');
|
||||
expect(parseRemoteProbeLine('o|0|0|/srv/case/sock')?.kind).toBe('other');
|
||||
expect(parseRemoteProbeLine('n')).toBeNull();
|
||||
expect(parseRemoteProbeLine('')).toBeNull();
|
||||
it('maps directories, other kinds, the not-found and the unresolvable markers', () => {
|
||||
expect(parseRemoteProbeRecord('d|0|5|/srv/case')?.kind).toBe('directory');
|
||||
expect(parseRemoteProbeRecord('o|0|0|/srv/case/sock')?.kind).toBe('other');
|
||||
expect(parseRemoteProbeRecord('n')).toBeNull();
|
||||
// Exists but could not be canonicalized: refused like a missing file, never
|
||||
// served under a path whose real target is unknown.
|
||||
expect(parseRemoteProbeRecord('x')).toBeNull();
|
||||
expect(parseRemoteProbeRecord('')).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects malformed lines instead of inventing a path', () => {
|
||||
expect(parseRemoteProbeLine('f|1|2')).toBeNull();
|
||||
expect(parseRemoteProbeLine('x|1|2|/p')).toBeNull();
|
||||
expect(parseRemoteProbeLine('f|1|2|')).toBeNull();
|
||||
expect(parseRemoteProbeRecord('f|1|2')).toBeNull();
|
||||
expect(parseRemoteProbeRecord('x|1|2|/p')).toBeNull();
|
||||
expect(parseRemoteProbeRecord('f|1|2|')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseRemoteProbeLines', () => {
|
||||
it('aligns the last N lines, so a login banner cannot shift the mapping', () => {
|
||||
const stdout = 'welcome to the remote box\nf|3|1|/srv/a.txt\nn\n';
|
||||
expect(parseRemoteProbeLines(stdout, ['/srv/a.txt', '/srv/b.txt'])).toEqual([
|
||||
describe('parseRemoteProbeOutput', () => {
|
||||
it('keys records by index after the leading NUL, so a login banner cannot shift the mapping', () => {
|
||||
const stdout = 'welcome to the remote box\n\x000|f|3|1|/srv/a.txt\x001|n\x00';
|
||||
expect(parseRemoteProbeOutput(stdout, ['/srv/a.txt', '/srv/b.txt'])).toEqual([
|
||||
{ realPath: '/srv/a.txt', kind: 'file', size: 3, mtimeMs: 1000 },
|
||||
null,
|
||||
]);
|
||||
});
|
||||
|
||||
it('throws when the remote shell returned too little output', () => {
|
||||
expect(() => parseRemoteProbeLines('f|3|1|/srv/a.txt\n', ['/a', '/b'])).toThrow(RemoteFileAccessError);
|
||||
it('accepts records in any order and ignores duplicates of an index', () => {
|
||||
const stdout = '\x001|d|0|0|/srv\x000|f|3|1|/srv/a.txt\x000|f|9|9|/evil\x00';
|
||||
expect(parseRemoteProbeOutput(stdout, ['/srv/a.txt', '/srv'])).toEqual([
|
||||
{ realPath: '/srv/a.txt', kind: 'file', size: 3, mtimeMs: 1000 },
|
||||
{ realPath: '/srv', kind: 'directory', size: 0, mtimeMs: 0 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('throws when a requested path has no record (transport or shell failure, never a 404)', () => {
|
||||
expect(() => parseRemoteProbeOutput('\x000|f|3|1|/srv/a.txt\x00', ['/a', '/b'])).toThrow(RemoteFileAccessError);
|
||||
expect(() => parseRemoteProbeOutput('', ['/a'])).toThrow(RemoteFileAccessError);
|
||||
// No leading NUL at all: the script never ran, whatever the shell printed.
|
||||
expect(() => parseRemoteProbeOutput('0|f|3|1|/srv/a.txt', ['/srv/a.txt'])).toThrow(RemoteFileAccessError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('under vitest', () => {
|
||||
const remote = remoteFixture();
|
||||
|
||||
it('never opens a connection: probes and reads reject with a clear error', async () => {
|
||||
// Mirrors checkRemoteTmuxAvailable's guard. The route tests mock this module, so
|
||||
// this is the backstop for the next test that reaches the real one.
|
||||
await expect(remoteProbePaths(remote, ['/srv/case'])).rejects.toThrow(/disabled under test/);
|
||||
await expect(remoteReadFile(remote, '/srv/case/a.txt', 1024)).rejects.toThrow(/disabled under test/);
|
||||
});
|
||||
|
||||
it('never opens a connection: a stream fails through its own error path', async () => {
|
||||
const { stream, close } = remoteCreateReadStream(remote, '/srv/case/a.mp4');
|
||||
const failure = await new Promise<Error>((resolveError) => stream.on('error', resolveError));
|
||||
expect(failure).toBeInstanceOf(RemoteFileAccessError);
|
||||
expect(() => close()).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* @fileoverview Tests for the remote-file ssh concurrency limiter
|
||||
* (`src/remote-ssh-limiter.ts`): the cap holds under interleaved async resumption,
|
||||
* waiters are served FIFO, and a task that throws still releases its slot.
|
||||
*
|
||||
* Port: N/A (no HTTP server).
|
||||
*/
|
||||
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import {
|
||||
getActiveRemoteSshCount,
|
||||
getQueuedRemoteSshCount,
|
||||
getRemoteSshLimit,
|
||||
runWithRemoteSshLimit,
|
||||
} from '../src/remote-ssh-limiter.js';
|
||||
|
||||
function deferred(): { promise: Promise<void>; resolve: () => void } {
|
||||
let resolve!: () => void;
|
||||
const promise = new Promise<void>((r) => {
|
||||
resolve = r;
|
||||
});
|
||||
return { promise, resolve };
|
||||
}
|
||||
|
||||
describe('runWithRemoteSshLimit', () => {
|
||||
it('never lets more than the cap run at once, and queues the rest FIFO', async () => {
|
||||
const cap = getRemoteSshLimit();
|
||||
const gates = Array.from({ length: cap + 3 }, () => deferred());
|
||||
const started: number[] = [];
|
||||
let peak = 0;
|
||||
|
||||
const runs = gates.map((gate, index) =>
|
||||
runWithRemoteSshLimit(async () => {
|
||||
started.push(index);
|
||||
peak = Math.max(peak, getActiveRemoteSshCount());
|
||||
await gate.promise;
|
||||
return index;
|
||||
})
|
||||
);
|
||||
await Promise.resolve();
|
||||
|
||||
expect(started).toEqual(Array.from({ length: cap }, (_, i) => i));
|
||||
expect(getActiveRemoteSshCount()).toBe(cap);
|
||||
expect(getQueuedRemoteSshCount()).toBe(3);
|
||||
|
||||
// Releasing one hands the slot to the OLDEST waiter; the count stays at the cap.
|
||||
gates[0].resolve();
|
||||
await runs[0];
|
||||
await Promise.resolve();
|
||||
expect(started).toEqual([...Array.from({ length: cap }, (_, i) => i), cap]);
|
||||
expect(getActiveRemoteSshCount()).toBe(cap);
|
||||
|
||||
for (const gate of gates) gate.resolve();
|
||||
expect(await Promise.all(runs)).toEqual(gates.map((_, i) => i));
|
||||
expect(peak).toBe(cap);
|
||||
expect(getActiveRemoteSshCount()).toBe(0);
|
||||
expect(getQueuedRemoteSshCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('releases the slot when the task throws', async () => {
|
||||
await expect(runWithRemoteSshLimit(async () => Promise.reject(new Error('ssh exit 255')))).rejects.toThrow(
|
||||
'ssh exit 255'
|
||||
);
|
||||
expect(getActiveRemoteSshCount()).toBe(0);
|
||||
expect(await runWithRemoteSshLimit(async () => 'after')).toBe('after');
|
||||
});
|
||||
});
|
||||
@@ -542,7 +542,9 @@ describe('file routes in a remote (SSH) case', () => {
|
||||
externalPath: outsidePath,
|
||||
},
|
||||
];
|
||||
mockedProbePaths.mockResolvedValue([fileProbe(outsidePath, 42), dirProbe]);
|
||||
mockedProbePaths.mockImplementation(async (_remote, paths) =>
|
||||
paths.map((path) => (path === outsidePath ? fileProbe(outsidePath, 42) : path === REMOTE_DIR ? dirProbe : null))
|
||||
);
|
||||
|
||||
const res = await harness.app.inject({ method: 'GET', url: `/api/sessions/${sessionId}/attachments` });
|
||||
|
||||
@@ -568,7 +570,15 @@ describe('file routes in a remote (SSH) case', () => {
|
||||
relativePath: 'out.png',
|
||||
},
|
||||
];
|
||||
mockedProbePaths.mockResolvedValue([fileProbe(`${REMOTE_DIR}/out.png`, 7), dirProbe]);
|
||||
mockedProbePaths.mockImplementation(async (_remote, paths) =>
|
||||
paths.map((path) =>
|
||||
path === `${REMOTE_DIR}/out.png`
|
||||
? fileProbe(`${REMOTE_DIR}/out.png`, 7)
|
||||
: path === REMOTE_DIR
|
||||
? dirProbe
|
||||
: null
|
||||
)
|
||||
);
|
||||
|
||||
const res = await harness.app.inject({ method: 'GET', url: `/api/sessions/${sessionId}/attachments` });
|
||||
|
||||
@@ -577,5 +587,150 @@ describe('file routes in a remote (SSH) case', () => {
|
||||
expect(item.size).toBe(7);
|
||||
expect(item.rawUrl).toContain('file-raw');
|
||||
});
|
||||
|
||||
describe('the history list probes the whole history in ONE ssh round trip', () => {
|
||||
// One connection per entry (up to ATTACHMENT_HISTORY_LIMIT, re-run on every
|
||||
// attachment:detected while the drawer is open) tripped OpenSSH's default
|
||||
// MaxStartups 10:30:100, which drops most of a burst that size.
|
||||
const history = () => [
|
||||
{
|
||||
id: 'hist-a',
|
||||
sessionId,
|
||||
fileName: 'out.png',
|
||||
extension: 'png',
|
||||
attachmentType: 'image' as const,
|
||||
size: 1,
|
||||
mtimeMs: 1,
|
||||
timestamp: 1,
|
||||
source: 'detected' as const,
|
||||
relativePath: 'out.png',
|
||||
},
|
||||
{
|
||||
id: 'hist-b',
|
||||
sessionId,
|
||||
fileName: 'shot.png',
|
||||
extension: 'png',
|
||||
attachmentType: 'image' as const,
|
||||
size: 1,
|
||||
mtimeMs: 1,
|
||||
timestamp: 1,
|
||||
source: 'external' as const,
|
||||
externalPath: outsidePath,
|
||||
},
|
||||
{
|
||||
id: 'hist-c',
|
||||
sessionId,
|
||||
fileName: 'gone.png',
|
||||
extension: 'png',
|
||||
attachmentType: 'image' as const,
|
||||
size: 1,
|
||||
mtimeMs: 1,
|
||||
timestamp: 1,
|
||||
source: 'detected' as const,
|
||||
relativePath: 'gone.png',
|
||||
},
|
||||
];
|
||||
|
||||
it('issues a single batched probe covering every entry plus the workspace root', async () => {
|
||||
harness.ctx._session.attachmentHistory = history();
|
||||
mockedProbePaths.mockImplementation(async (_remote, paths) =>
|
||||
paths.map((path) =>
|
||||
path === `${REMOTE_DIR}/out.png`
|
||||
? fileProbe(path, 7)
|
||||
: path === outsidePath
|
||||
? fileProbe(outsidePath, 42)
|
||||
: path === REMOTE_DIR
|
||||
? dirProbe
|
||||
: null
|
||||
)
|
||||
);
|
||||
|
||||
const res = await harness.app.inject({ method: 'GET', url: `/api/sessions/${sessionId}/attachments` });
|
||||
|
||||
expect(res.statusCode).toBe(200);
|
||||
expect(mockedProbePaths).toHaveBeenCalledTimes(1);
|
||||
const [, probed] = mockedProbePaths.mock.calls[0];
|
||||
expect([...probed].sort()).toEqual(
|
||||
[REMOTE_DIR, `${REMOTE_DIR}/gone.png`, `${REMOTE_DIR}/out.png`, outsidePath].sort()
|
||||
);
|
||||
// (ids are re-minted for external entries by the sanitizer, so key on the name)
|
||||
const items = JSON.parse(res.body).data.items as Array<{ fileName: string; missing: boolean; size: number }>;
|
||||
expect(items.map((item) => [item.fileName, item.missing, item.size])).toEqual([
|
||||
['out.png', false, 7],
|
||||
['shot.png', false, 42],
|
||||
['gone.png', true, 1],
|
||||
]);
|
||||
});
|
||||
|
||||
it('reports every entry as unknown (missing: false), detected AND external alike, when the host is unreachable', async () => {
|
||||
harness.ctx._session.attachmentHistory = history();
|
||||
mockedProbePaths.mockRejectedValue(new RemoteFileAccessError('remote host testhost unreachable: timed out'));
|
||||
|
||||
const res = await harness.app.inject({ method: 'GET', url: `/api/sessions/${sessionId}/attachments` });
|
||||
|
||||
expect(res.statusCode).toBe(200);
|
||||
const items = JSON.parse(res.body).data.items as Array<{ id: string; missing: boolean }>;
|
||||
// The two branches used to disagree here: detected kept missing:false while
|
||||
// external's 502 was folded into missing:true.
|
||||
expect(items.map((item) => item.missing)).toEqual([false, false, false]);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('PUT /api/sessions/:id/file-content', () => {
|
||||
// The remote guard has to come BEFORE the local path validation: with a
|
||||
// directory of the same absolute name on this host (an sshfs mount of the remote
|
||||
// tree, the documented stop-gap for #415) the write would land on the local twin
|
||||
// while the viewer believes it edited the remote file.
|
||||
let shadowRoot: string;
|
||||
let shadowFile: string;
|
||||
|
||||
beforeEach(() => {
|
||||
shadowRoot = mkdtempSync(join(tmpdir(), 'codeman-remote-put-'));
|
||||
shadowFile = join(shadowRoot, 'notes.txt');
|
||||
writeFileSync(shadowFile, 'LOCAL TEXT');
|
||||
harness.ctx._session.workingDir = shadowRoot;
|
||||
harness.ctx._session.remote = { ...remote, remotePath: shadowRoot };
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(shadowRoot, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('answers 400 for a remote case and never touches the local file of the same name', async () => {
|
||||
const res = await harness.app.inject({
|
||||
method: 'PUT',
|
||||
url: `/api/sessions/${sessionId}/file-content`,
|
||||
payload: { path: 'notes.txt', content: 'OVERWRITTEN', baseHash: 'whatever', force: true },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(400);
|
||||
expect(JSON.parse(res.body).error).toMatch(/not supported for files in a remote/);
|
||||
expect(readFileSync(shadowFile, 'utf8')).toBe('LOCAL TEXT');
|
||||
expect(mockedProbePaths).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('a client that gives up during the guard probe', () => {
|
||||
it('still has its ssh body child reaped', async () => {
|
||||
// The probe is an ssh round trip; a client that aborted during it has already
|
||||
// closed the response, so a `close` listener attached afterwards never fires.
|
||||
const controller = new AbortController();
|
||||
mockedProbePaths.mockImplementation(async () => {
|
||||
controller.abort();
|
||||
await new Promise((resolveDelay) => setTimeout(resolveDelay, 20));
|
||||
return [fileProbe(`${REMOTE_DIR}/img.png`, 9), dirProbe];
|
||||
});
|
||||
|
||||
await harness.app
|
||||
.inject({ method: 'GET', url: `/api/sessions/${sessionId}/file-raw?path=img.png`, signal: controller.signal })
|
||||
.catch(() => undefined);
|
||||
await new Promise((resolveDelay) => setTimeout(resolveDelay, 50));
|
||||
|
||||
// The body WAS opened (the route ran to completion against an already-closed
|
||||
// response), which is exactly the window the guard covers.
|
||||
expect(mockedCreateReadStream).toHaveBeenCalledTimes(1);
|
||||
expect(closeSpy).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user