Merge pull request #574: feat(uploads): write prompt uploads to a hidden, self-ignoring .codeman-uploads/ folder

Conflicts resolved against the release branch: docs/api-reference.md keeps
both new sections (codeman agent CLI, then Prompt uploads);
test/test-ports-guard.test.ts takes the release side (#570 already removed
the legacy list); test/paste-image-dir-shared.test.ts keeps #570's ephemeral
port and this PR's bounded-path-probe mock.

Also at merge: the Docker sentence in the route comment and the API reference
is narrowed to owned cases, since an adopted container mounts nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer
2026-10-10 03:39:33 +02:00
16 changed files with 522 additions and 95 deletions
+3 -1
View File
@@ -107,7 +107,9 @@ todo.md
@fix_plan.md @fix_plan.md
readme-preview.mjs readme-preview.mjs
# Uploaded images land here under each session working dir (runtime artifact) # Prompt uploads land here under each session working dir (runtime artifact);
# .claude-images/ is where they landed before the move.
.codeman-uploads/
.claude-images/ .claude-images/
# Local-LLM harness smoke-test config (real IPs/keys) — see the .example.json # Local-LLM harness smoke-test config (real IPs/keys) — see the .example.json
+3 -1
View File
File diff suppressed because one or more lines are too long
+17
View File
@@ -470,6 +470,23 @@ geometry was read. The capture runs synchronous tmux calls on the server; the
`codeman agent ls|spawn|send|wait|read|interrupt|rm` (`src/cli-agent.ts`) is the command-line client for the endpoints above, for agents in modes that never receive the claude-only skill preamble. It adds no route: `spawn` is `POST /api/v1/quick-start` (+ `wait-output` on the mode's `capabilities.composerReadyMark` from the CLI registry, where it declares one), `send` is `POST …/input` with `clientId`+`seq` (and `wait`/`waitTimeout` for `--wait` / `--until <signals>`; `delivered:false` without `duplicate` and `wait.ended` both exit 3 — the CLI never reports a dead worker as done), `wait` is `GET …/wait` (`--until`) or `GET …/wait-output` (`--match`, `from=buffer` by default), `read` is `GET …/last-response` or `GET …/terminal?tail=`, `interrupt` is `POST …/input` with a bare `\u001b`, `rm` is `DELETE …/sessions/:id`. A fire-and-forget `send` to a sleeping wake-on-LAN host reads the route's `buffered` (own line, exit 0) and `dropped` (exit 1: the chunk is gone). An id may be the 8-character form `ls` prints, resolved through `GET /api/v1/sessions`; anything shorter refuses before any request, the same floor as `PARENT_SESSION_ID_MIN_PREFIX`. Every call carries `X-Codeman-Parent-Session`; only `spawn`'s quick-start carries `X-Codeman-Agent-Origin: codeman-agent-cli` (the agent-scratch label must never reach a request that cannot create the case directory). Basic auth comes from `CODEMAN_PASSWORD` or the data dir's `.env`. Server-side error codes are shown verbatim (`INVALID_INPUT: until=stop …` on a hook-less mode is not hidden); exit codes are `0` ok, `1` error, `2` timeout, `3` the session exited, `4` refused by a client-side guard. See the README section "`codeman agent`" for the guards and `test/cli-agent.test.ts` for the pinned behaviour. `codeman agent ls|spawn|send|wait|read|interrupt|rm` (`src/cli-agent.ts`) is the command-line client for the endpoints above, for agents in modes that never receive the claude-only skill preamble. It adds no route: `spawn` is `POST /api/v1/quick-start` (+ `wait-output` on the mode's `capabilities.composerReadyMark` from the CLI registry, where it declares one), `send` is `POST …/input` with `clientId`+`seq` (and `wait`/`waitTimeout` for `--wait` / `--until <signals>`; `delivered:false` without `duplicate` and `wait.ended` both exit 3 — the CLI never reports a dead worker as done), `wait` is `GET …/wait` (`--until`) or `GET …/wait-output` (`--match`, `from=buffer` by default), `read` is `GET …/last-response` or `GET …/terminal?tail=`, `interrupt` is `POST …/input` with a bare `\u001b`, `rm` is `DELETE …/sessions/:id`. A fire-and-forget `send` to a sleeping wake-on-LAN host reads the route's `buffered` (own line, exit 0) and `dropped` (exit 1: the chunk is gone). An id may be the 8-character form `ls` prints, resolved through `GET /api/v1/sessions`; anything shorter refuses before any request, the same floor as `PARENT_SESSION_ID_MIN_PREFIX`. Every call carries `X-Codeman-Parent-Session`; only `spawn`'s quick-start carries `X-Codeman-Agent-Origin: codeman-agent-cli` (the agent-scratch label must never reach a request that cannot create the case directory). Basic auth comes from `CODEMAN_PASSWORD` or the data dir's `.env`. Server-side error codes are shown verbatim (`INVALID_INPUT: until=stop …` on a hook-less mode is not hidden); exit codes are `0` ok, `1` error, `2` timeout, `3` the session exited, `4` refused by a client-side guard. See the README section "`codeman agent`" for the guards and `test/cli-agent.test.ts` for the pinned behaviour.
## Prompt uploads (`POST /api/v1/sessions/:id/paste-image`)
A `multipart/form-data` body with one `image` part. The file is written into the
session's workspace as `<workingDir>/.codeman-uploads/paste-<ms>-<hex>.<ext>`, and
`data` carries `path` and `filename` for the client to type the path into the
prompt. The folder is Codeman's own: hidden, created on first use with a
`.gitignore` containing `*` (written once, never over a file already there), and
cleaned up the way pasted images always were: `paste-*` files older than 7 days
go in an hourly sweep, and the folder goes when the last session of that
workspace is killed. Uploads made before this release sit in `.claude-images/`;
that folder receives nothing new, and is swept and removed the same way for one
release. A remote (SSH) session answers 400, since the file would land on the
Codeman host under a path the remote agent cannot read. A Docker session of an
owned case is fine, its workspace is bind-mounted at the same absolute path; an
adopted container (`owned: false`) mounts nothing, so its agent can open the file
only if the container itself exposes that host path.
## Session lineage (`parentSessionId`) ## Session lineage (`parentSessionId`)
A create request may name the session that spawned it, which the web UI draws as a A create request may name the session that spawned it, which the web UI draws as a
+4 -2
View File
@@ -254,7 +254,7 @@ Further detail, closing: ⚠️ **Closing has the mirror-image race and one owne
- **No start, attach or relaunch may be in flight** (`Session.paneLifecycleInFlight`, raised for the whole of `_setupOrAttachMuxSession()` and `restartCli()`). The dead-pane respawn revives an exited pane on purpose, and the pane reads as dead until `clearPaneExitForNewPane()` runs after its startup delay. - **No start, attach or relaunch may be in flight** (`Session.paneLifecycleInFlight`, raised for the whole of `_setupOrAttachMuxSession()` and `restartCli()`). The dead-pane respawn revives an exited pane on purpose, and the pane reads as dead until `clearPaneExitForNewPane()` runs after its startup delay.
- **The pane must have been up for `CLEAN_EXIT_MIN_PANE_LIFETIME_MS` (10 s)** since the last start, attach or relaunch finished (`Session.paneStartedAt`). A CLI that prints a startup error and exits 0 would otherwise lose its tab, and the error with it, seconds after launch; its row stays as `exited (0)` instead. An attach to an already running pane stamps it too, so an `/exit` within seconds of a server restart leaves a row to close by hand. - **The pane must have been up for `CLEAN_EXIT_MIN_PANE_LIFETIME_MS` (10 s)** since the last start, attach or relaunch finished (`Session.paneStartedAt`). A CLI that prints a startup error and exits 0 would otherwise lose its tab, and the error with it, seconds after launch; its row stays as `exited (0)` instead. An attach to an already running pane stamps it too, so an `/exit` within seconds of a server restart leaves a row to close by hand.
Scoping needs no check of its own here: `setPaneExit()` already forces `paneExit` to UNKNOWN for direct-PTY, remote, docker and discovered sessions. There is no setting, by the maintainer's decision on #446. ⚠️ Do not flip local panes to `remain-on-exit failed` to get the same effect: a destroyed pane ends the tmux session, the PTY exit nulls the pid, and the browser's `selectSession()` then launches a fresh CLI. `cleanupSession()` keeps `{workingDir}/.claude-images` while another session still uses that directory (`pasteImageDirInUseByOtherSession()`, `paste-image-gc.ts`), since the sweep would otherwise routinely delete a live sibling's pasted images. Paths are compared by `realpath`, and a detached session counts through its persisted record, because `killMux=false` removes it from the map while its pane keeps running; only a session being KILLED is exempt. Each exit gets ONE close attempt (keyed by session id and `at`), and a session being closed refuses `startInteractive()`/`startShell()` (`Session.markClosing()`), so a start that races the close cannot orphan a tmux session. `planRebootRestore()` refuses a record whose persisted `paneExit` is clean (`agent-exited`), which covers an agent that exited just before the power went, before the sweep reached it. Tests: `test/pane-exit-sweep.test.ts`, `test/paste-image-dir-shared.test.ts`, `test/reboot-restore.test.ts`, `test/tmux-manager.test.ts`. Scoping needs no check of its own here: `setPaneExit()` already forces `paneExit` to UNKNOWN for direct-PTY, remote, docker and discovered sessions. There is no setting, by the maintainer's decision on #446. ⚠️ Do not flip local panes to `remain-on-exit failed` to get the same effect: a destroyed pane ends the tmux session, the PTY exit nulls the pid, and the browser's `selectSession()` then launches a fresh CLI. `cleanupSession()` keeps `{workingDir}/.codeman-uploads` (and the pre-move `.claude-images`) while another session still uses that directory (`pasteImageDirInUseByOtherSession()`, `paste-image-gc.ts`), since the sweep would otherwise routinely delete a live sibling's pasted images. Paths are compared by `realpath`, and a detached session counts through its persisted record, because `killMux=false` removes it from the map while its pane keeps running; only a session being KILLED is exempt. Each exit gets ONE close attempt (keyed by session id and `at`), and a session being closed refuses `startInteractive()`/`startShell()` (`Session.markClosing()`), so a start that races the close cannot orphan a tmux session. `planRebootRestore()` refuses a record whose persisted `paneExit` is clean (`agent-exited`), which covers an agent that exited just before the power went, before the sweep reached it. Tests: `test/pane-exit-sweep.test.ts`, `test/paste-image-dir-shared.test.ts`, `test/reboot-restore.test.ts`, `test/tmux-manager.test.ts`.
### Dead-pane respawn: the resume pin ### Dead-pane respawn: the resume pin
@@ -351,7 +351,7 @@ The escape hatch is the synced `workspaceHooksEnabled` setting (App Settings →
⚠️ Claude-mode only (others carry their conversation id in their own config object), and remote/docker sessions are never offered (`remote-or-docker`), because both need another host or container to be up. ⚠️ Claude-mode only (others carry their conversation id in their own config object), and remote/docker sessions are never offered (`remote-or-docker`), because both need another host or container to be up.
⚠️ A failed rebuild is undone with `discardPartiallyBuiltSession()`, deliberately NOT `cleanupSession()`: the delete path would count the session's tokens into the lifetime totals, demote a pinned record to `stopped` (which this pass reads as an intentional kill, making the session permanently unrestorable) and recursively remove the WORKSPACE's `.claude-images`. ⚠️ A failed rebuild is undone with `discardPartiallyBuiltSession()`, deliberately NOT `cleanupSession()`: the delete path would count the session's tokens into the lifetime totals, demote a pinned record to `stopped` (which this pass reads as an intentional kill, making the session permanently unrestorable) and recursively remove the WORKSPACE's upload dirs (`.codeman-uploads`, and the pre-move `.claude-images`).
⚠️ `os.uptime()` reports the HOST's uptime, which a container shares, and that cuts both ways: after a genuine host reboot a containerized Codeman does see a short uptime and the banner works, but a container-only restart is invisible to it, which is the case where this would help most. Tests: `test/reboot-restore.test.ts`, `test/routes/reboot-restore-routes.test.ts`, `test/routes/reboot-restore-rebuild-failure.test.ts`, `test/discard-partially-built-session.test.ts`. ⚠️ `os.uptime()` reports the HOST's uptime, which a container shares, and that cuts both ways: after a genuine host reboot a containerized Codeman does see a short uptime and the banner works, but a container-only restart is invisible to it, which is the case where this would help most. Tests: `test/reboot-restore.test.ts`, `test/routes/reboot-restore-routes.test.ts`, `test/routes/reboot-restore-rebuild-failure.test.ts`, `test/discard-partially-built-session.test.ts`.
@@ -1071,6 +1071,8 @@ Tests: `test/mobile-prompt-composer.test.ts` (in the CI gate, deliberately not u
Target: 20 sessions, 50 agent windows at 60fps. Limits in `src/config/`: terminal 32MB (see below), text 1MB, messages 1000, max agents 500, max sessions 50, max SSE clients 100. **Terminal history** (`src/config/terminal-history.ts`, COD-80): tmux history-limit 100k lines, PTY buffer 32MB max / 24MB trim (env `CODEMAN_MAX_TERMINAL_BUFFER`/`CODEMAN_TRIM_TERMINAL_TO`; the env-derived trim is clamped ≤75% of max — trim ≥ max would disable `BufferAccumulator` trimming entirely = unbounded memory); browser xterm scrollback stays a separate hardcoded 50k (`DEFAULT_SCROLLBACK` in constants.js — 100k/tab is a mobile-memory hazard). tmux <3.7 allocates history at pane creation, so `createSession()` sets the global default in the same command queue immediately before `new-session`; tmux 3.7+ instead creates the session and targets only that pane, because changing the global option can resize and trim unrelated live panes. A settings change resizes tracked panes only on 3.7+ and otherwise affects future panes; no version can recover lines already evicted. Settings keys `terminalScrollbackLines`/`terminalBufferMaxBytes`/`terminalBufferTrimBytes` are schema-validated but inert (only `tmuxHistoryLimit` is wired); `buffer-limits.ts` re-exports the defaults. Text/message limits are env-overridable too (`CODEMAN_MAX_TEXT_OUTPUT`/`CODEMAN_TRIM_TEXT_TO`/`CODEMAN_MAX_MESSAGES`). **Image upload** (`image-input.js` / `config/buffer-limits.ts`): up to `_maxBatchImages` 20 images/batch (bounded concurrency 3), per-file `MAX_PASTE_IMAGE_BYTES` 50MB (env `CODEMAN_MAX_PASTE_IMAGE_BYTES`); the mobile camera-roll picker auto-downscales to fit before upload. **HEIC paste uploads** (#151): converted server-side to JPEG in a `worker_threads` worker (`web/heic-jpeg-worker.ts`, resourceLimits + 30s timeout) gated by `runWithConversionLimit()`; detection is magic-byte based (covers Android/MIUI HEIFs mislabeled as JPEG); headers declaring > 64MP are rejected 415 BEFORE decode (decompression-bomb guard). Deps: `heic-decode` + `jpeg-js`. Use `LRUMap` for bounded caches, `StaleExpirationMap` for TTL cleanup. Anti-flicker pipeline: `docs/terminal-anti-flicker.md`. Target: 20 sessions, 50 agent windows at 60fps. Limits in `src/config/`: terminal 32MB (see below), text 1MB, messages 1000, max agents 500, max sessions 50, max SSE clients 100. **Terminal history** (`src/config/terminal-history.ts`, COD-80): tmux history-limit 100k lines, PTY buffer 32MB max / 24MB trim (env `CODEMAN_MAX_TERMINAL_BUFFER`/`CODEMAN_TRIM_TERMINAL_TO`; the env-derived trim is clamped ≤75% of max — trim ≥ max would disable `BufferAccumulator` trimming entirely = unbounded memory); browser xterm scrollback stays a separate hardcoded 50k (`DEFAULT_SCROLLBACK` in constants.js — 100k/tab is a mobile-memory hazard). tmux <3.7 allocates history at pane creation, so `createSession()` sets the global default in the same command queue immediately before `new-session`; tmux 3.7+ instead creates the session and targets only that pane, because changing the global option can resize and trim unrelated live panes. A settings change resizes tracked panes only on 3.7+ and otherwise affects future panes; no version can recover lines already evicted. Settings keys `terminalScrollbackLines`/`terminalBufferMaxBytes`/`terminalBufferTrimBytes` are schema-validated but inert (only `tmuxHistoryLimit` is wired); `buffer-limits.ts` re-exports the defaults. Text/message limits are env-overridable too (`CODEMAN_MAX_TEXT_OUTPUT`/`CODEMAN_TRIM_TEXT_TO`/`CODEMAN_MAX_MESSAGES`). **Image upload** (`image-input.js` / `config/buffer-limits.ts`): up to `_maxBatchImages` 20 images/batch (bounded concurrency 3), per-file `MAX_PASTE_IMAGE_BYTES` 50MB (env `CODEMAN_MAX_PASTE_IMAGE_BYTES`); the mobile camera-roll picker auto-downscales to fit before upload. **HEIC paste uploads** (#151): converted server-side to JPEG in a `worker_threads` worker (`web/heic-jpeg-worker.ts`, resourceLimits + 30s timeout) gated by `runWithConversionLimit()`; detection is magic-byte based (covers Android/MIUI HEIFs mislabeled as JPEG); headers declaring > 64MP are rejected 415 BEFORE decode (decompression-bomb guard). Deps: `heic-decode` + `jpeg-js`. Use `LRUMap` for bounded caches, `StaleExpirationMap` for TTL cleanup. Anti-flicker pipeline: `docs/terminal-anti-flicker.md`.
**Prompt uploads live in `<workspace>/.codeman-uploads/`** (`POST /api/sessions/:id/paste-image`; the names live ONLY in `src/web/paste-image-gc.ts`): IN the workspace because that is the only path that resolves identically for a local agent and a container (only the workspace is bind-mounted, at the same absolute path; `~/.codeman` is not); hidden so it stays out of `git status` and the agent's view of the repository; FLAT because a nested `<workspace>/.codeman/` IS the data dir (`join(homedir(), '.codeman')`, `src/config/instance.ts`) when the workspace is the home directory, which nothing refuses; and self-ignoring through a `.gitignore` of `*` written once with `wx`, so a file already there is the user's and stays. The pre-move `.claude-images/` receives nothing but stays readable for one release: `UPLOAD_DIR_NAMES` lists both, the hourly sweep and the delete cleanup in `cleanupSession()` go through `uploadDirs()`, and the image watcher's ignore filter reads the names. ⚠️ `uploadDirs()` is async: the working directory is a user-chosen path, so it is probed BOUNDED first (`probePathKind()`, the #516 rule; `unknown` is skipped and never touched, the hourly sweep keeps the stall cap and `cleanupSession()` passes `pastCap`, since it acts on one path at the user's request), and the lstat and realpath after it are `fs.promises` calls, never sync, because the sweep runs 30 s after boot and hourly over every live session and a linked case on a dead mount would otherwise freeze the whole server. It returns only REAL directories (lstat at the leaf, so a planted `.codeman-uploads -> /other-case/.codeman-uploads` is never listed: `readdir` follows a link to a directory, and the sweep would otherwise age out the other case's uploads through it), none that is or contains `getDataDir()` (contrived, `CODEMAN_INSTANCE=uploads` makes `~/.codeman-uploads` the data dir of a home workspace and `CODEMAN_DATA_DIR` can point inside one, but it is one check next to a recursive delete; one strictly INSIDE the data dir is listed, since it only ever holds uploads and a workspace like the `~/.codeman/app` that `install.sh` clones would otherwise never have its uploads collected), and nothing for a remote (SSH) session, whose `workingDir` is the remote path and would otherwise name a same-named LOCAL directory for the sweep and the delete. The check is made when the directories are listed: a same-user process that swaps a listed directory for a link during the sweep's awaits is accepted (Node has no `openat()`, and that actor already writes anywhere this process can). The route refuses a remote (SSH) session with 400 before any disk touch: its `workingDir` is the remote path, so the file would land on THIS host where the agent cannot read it. Tests: `test/paste-image-gc.test.ts`, `test/paste-image-dir-shared.test.ts`, the paste-image block of `test/routes/session-routes.test.ts`.
### Process-tree walks are bounded ### Process-tree walks are bounded
**Process-tree walks are bounded** (`proc-tree.ts`, pure + unit tested): `collectDescendants(pid, byParent)` is the ONE descendant traversal, fed by a single cached `ps -eo pid=,ppid=` snapshot (`refreshProcSnapshot()` in tmux-manager.ts: in-flight-shared, async because `execSync`'s timeout cannot return at all while spawnSync waits on an unkillable child, and ANY error discards the result rather than caching a truncated `ps`, which would make whole subtrees invisible to the kill path). **Process-tree walks are bounded** (`proc-tree.ts`, pure + unit tested): `collectDescendants(pid, byParent)` is the ONE descendant traversal, fed by a single cached `ps -eo pid=,ppid=` snapshot (`refreshProcSnapshot()` in tmux-manager.ts: in-flight-shared, async because `execSync`'s timeout cannot return at all while spawnSync waits on an unkillable child, and ANY error discards the result rather than caching a truncated `ps`, which would make whole subtrees invisible to the kill path).
+6 -2
View File
@@ -14,6 +14,7 @@ import { basename, extname, relative } from 'node:path';
import { statSync } from 'node:fs'; import { statSync } from 'node:fs';
import type { AttachmentDetectedEvent, AttachmentDetectedType, ImageDetectedEvent } from './types.js'; import type { AttachmentDetectedEvent, AttachmentDetectedType, ImageDetectedEvent } from './types.js';
import { KeyedDebouncer } from './utils/index.js'; import { KeyedDebouncer } from './utils/index.js';
import { UPLOAD_DIR_NAMES } from './web/paste-image-gc.js';
// ========== Types ========== // ========== Types ==========
@@ -157,12 +158,15 @@ export class ImageWatcher extends EventEmitter {
// Watch all subdirectories (images may be saved in src/, assets/, etc.) // Watch all subdirectories (images may be saved in src/, assets/, etc.)
// Ignore common heavy directories for performance // Ignore common heavy directories for performance
ignored: (path: string) => { ignored: (path: string) => {
// Skip node_modules, .git, and other heavy directories // Skip node_modules, .git, and other heavy directories, and Codeman's
// own upload folders: a pdf the user handed to the agent is not a file
// the agent produced.
if ( if (
path.includes('/node_modules/') || path.includes('/node_modules/') ||
path.includes('/.git/') || path.includes('/.git/') ||
path.includes('/dist/') || path.includes('/dist/') ||
path.includes('/.next/') path.includes('/.next/') ||
UPLOAD_DIR_NAMES.some((name) => path.includes(`/${name}/`))
) { ) {
return true; return true;
} }
+105 -30
View File
@@ -1,25 +1,99 @@
/** /**
* @fileoverview Periodic GC for paste-image files. * @fileoverview Periodic GC for prompt-upload files, and the one place that
* names the directories they live in.
* *
* Without cleanup, /api/sessions/:id/paste-image accumulates files indefinitely * Without cleanup, /api/sessions/:id/paste-image accumulates files indefinitely
* under {workingDir}/.claude-images/. The route only triggers cleanup on * under {workingDir}/.codeman-uploads/. The route only triggers cleanup on
* killMux=true session deletion, so long-lived sessions can fill disk under * killMux=true session deletion, so long-lived sessions can fill disk under
* heavy pasting. This sweeper bounds disk use by deleting `paste-*` files * heavy pasting. This sweeper bounds disk use by deleting `paste-*` files
* older than MAX_AGE_MS from each live session's image dir on an interval. * older than MAX_AGE_MS from each live session's upload dirs on an interval.
* *
* Conservative defaults — only files matching the `paste-` prefix are * Conservative defaults — only files matching the `paste-` prefix are
* considered, and we lstat (not stat) so a planted symlink cannot escape the * considered, and we lstat (not stat) so a planted symlink cannot escape the
* image dir. * upload dir.
*/ */
import fs from 'node:fs/promises'; import fs from 'node:fs/promises';
import { realpathSync } from 'node:fs'; import { realpathSync } from 'node:fs';
import { join, resolve } from 'node:path'; import { join, resolve, sep } from 'node:path';
import { getDataDir } from '../config/instance.js';
import { probePathKind, type PathProbeOptions } from '../utils/bounded-path-probe.js';
import type { SessionPort } from './ports/index.js'; import type { SessionPort } from './ports/index.js';
const MAX_AGE_MS = 7 * 24 * 60 * 60 * 1000; // 7 days const MAX_AGE_MS = 7 * 24 * 60 * 60 * 1000; // 7 days
const SWEEP_INTERVAL_MS = 60 * 60 * 1000; // 1 hour const SWEEP_INTERVAL_MS = 60 * 60 * 1000; // 1 hour
const INITIAL_DELAY_MS = 30 * 1000; // 30s after startup const INITIAL_DELAY_MS = 30 * 1000; // 30s after startup
/**
* Where a prompt upload is written, relative to the session's working
* directory. IN the workspace, because that is the only path that resolves
* identically for a local agent and a container (only the workspace is
* bind-mounted, at the same absolute path); hidden, so it stays out of
* `git status` and the agent's view of the repository; FLAT, because a nested
* `<workspace>/.codeman/` is Codeman's own data dir when the workspace is the
* home directory; and self-ignoring, through a `.gitignore` of `*` the route
* writes once.
*/
export const UPLOADS_DIR = '.codeman-uploads';
/** Where uploads landed before the move: written to by nothing, readable for one release. */
export const LEGACY_UPLOADS_DIR = '.claude-images';
/** Every upload dir name, current first. Retiring the legacy one here retires it for every reader. */
export const UPLOAD_DIR_NAMES = [UPLOADS_DIR, LEGACY_UPLOADS_DIR];
/**
* Every directory a session's uploads sit in, current first. Both consumers
* act on what this returns, the hourly sweep and the recursive delete in
* cleanupSession(), so it lists only REAL directories (a link planted by a
* workspace script, `.codeman-uploads -> /other-case/.codeman-uploads`, is
* not one; readdir follows a link to a directory), none that is or contains
* this instance's data dir (a home workspace reaches it under a contrived
* instance name, `CODEMAN_INSTANCE=uploads`, and `CODEMAN_DATA_DIR` can point
* inside one; a directory strictly below the data dir only ever holds uploads
* and is listed, or the uploads of a workspace like `~/.codeman/app` would
* never be collected), and nothing for a remote (SSH) session, whose
* workingDir is the remote path and would name a same-named LOCAL directory
* here. The working directory is a user-chosen path, so it is probed BOUNDED
* first (#516): one on a mount that stopped answering reads `unknown` and is
* skipped, never touched. The sweep keeps the probe's stall cap; the delete,
* acting on one path at the user's request, passes `pastCap`. The check is
* made when listing: a same-user process that swaps a listed directory for a
* link afterwards is accepted, since it already writes anywhere this process
* can.
*/
export async function uploadDirs(
session: { workingDir: string; remote?: unknown },
probe: PathProbeOptions = {}
): Promise<string[]> {
if (session.remote) return [];
if ((await probePathKind(session.workingDir, probe)) !== 'directory') return [];
const dataDir = await realDir(getDataDir());
const dirs: string[] = [];
for (const dir of UPLOAD_DIR_NAMES.map((name) => join(session.workingDir, name))) {
if (!(await isRealDir(dir))) continue;
const real = await realDir(dir);
if (real === dataDir || dataDir.startsWith(real + sep)) continue;
dirs.push(dir);
}
return dirs;
}
/** lstat, so a symlink is not a directory, whatever it points at. */
async function isRealDir(p: string): Promise<boolean> {
try {
return (await fs.lstat(p)).isDirectory();
} catch {
return false;
}
}
/** `canonicalDir()` for the listing, which must not block the event loop on a user path. */
async function realDir(dir: string): Promise<string> {
try {
return await fs.realpath(dir);
} catch {
return resolve(dir);
}
}
export async function sweepPasteImagesOnce( export async function sweepPasteImagesOnce(
ctx: Pick<SessionPort, 'sessions'>, ctx: Pick<SessionPort, 'sessions'>,
now: number = Date.now() now: number = Date.now()
@@ -28,26 +102,27 @@ export async function sweepPasteImagesOnce(
let scanned = 0; let scanned = 0;
let deleted = 0; let deleted = 0;
for (const session of ctx.sessions.values()) { for (const session of ctx.sessions.values()) {
const dir = join(session.workingDir, '.claude-images'); for (const dir of await uploadDirs(session)) {
let entries: string[]; let entries: string[];
try {
entries = await fs.readdir(dir);
} catch {
continue; // dir absent — nothing to do
}
for (const name of entries) {
if (!name.startsWith('paste-')) continue;
const p = join(dir, name);
scanned += 1;
try { try {
const st = await fs.lstat(p); entries = await fs.readdir(dir);
if (!st.isFile()) continue;
if (st.mtimeMs < cutoff) {
await fs.unlink(p);
deleted += 1;
}
} catch { } catch {
// best-effort: skip permission/race errors silently continue; // gone since listed — nothing to do
}
for (const name of entries) {
if (!name.startsWith('paste-')) continue;
const p = join(dir, name);
scanned += 1;
try {
const st = await fs.lstat(p);
if (!st.isFile()) continue;
if (st.mtimeMs < cutoff) {
await fs.unlink(p);
deleted += 1;
}
} catch {
// best-effort: skip permission/race errors silently
}
} }
} }
} }
@@ -55,7 +130,7 @@ export async function sweepPasteImagesOnce(
} }
/** /**
* The path two sessions must share to share a paste-image dir: the canonical * The path two sessions must share to share an upload dir: the canonical
* path when it can be resolved, so a sibling that reaches the same directory * path when it can be resolved, so a sibling that reaches the same directory
* through a symlink matches, and the normalised path otherwise (a directory * through a symlink matches, and the normalised path otherwise (a directory
* that no longer exists has nothing left to protect). * that no longer exists has nothing left to protect).
@@ -68,7 +143,7 @@ function canonicalDir(dir: string): string {
} }
} }
/** One session the paste-image guard weighs: its id, directory and, for a persisted record, its status. */ /** One session the upload-dir guard weighs: its id, directory and, for a persisted record, its status. */
export interface PasteImageDirUser { export interface PasteImageDirUser {
id: string; id: string;
workingDir: string; workingDir: string;
@@ -76,11 +151,11 @@ export interface PasteImageDirUser {
} }
/** /**
* Does another live session still use this working directory's paste-image * Does another live session still use this working directory's upload dirs?
* dir? Deleting a session removes `{workingDir}/.claude-images` recursively, * Deleting a session removes them (`uploadDirs()`) recursively, and several
* and several sessions routinely share one case directory, so without this * sessions routinely share one case directory, so without this check closing
* check closing one session deletes the pasted images a sibling in the same * one session deletes the pasted images a sibling in the same case still
* case still refers to. * refers to.
* *
* Two kinds of sibling count as live: * Two kinds of sibling count as live:
* *
+1 -1
View File
@@ -2046,7 +2046,7 @@ Object.assign(CodemanApp.prototype, {
const cmdPattern = /\b(tail|cat|head|less|grep|watch|vim|nano)\s+(?:[^\s\/]+\s+){0,4}(\/[^\s"'<>|;&\n\x00-\x1f]+)/g; const cmdPattern = /\b(tail|cat|head|less|grep|watch|vim|nano)\s+(?:[^\s\/]+\s+){0,4}(\/[^\s"'<>|;&\n\x00-\x1f]+)/g;
// Pattern 2: Paths with common extensions. Image/PDF/media extensions are // Pattern 2: Paths with common extensions. Image/PDF/media extensions are
// included so pasted-attachment paths (`.claude-images/paste-*.png`) and // included so pasted-attachment paths (`.codeman-uploads/paste-*.png`) and
// screenshots an agent just wrote are clickable; those open the file // screenshots an agent just wrote are clickable; those open the file
// preview rather than the log viewer (see addLink). // preview rather than the log viewer (see addLink).
// //
+1 -1
View File
@@ -275,7 +275,7 @@ export function registerRebootRestoreRoutes(app: FastifyInstance, ctx: RebootRes
// count this session's historical tokens into the lifetime totals, demote // count this session's historical tokens into the lifetime totals, demote
// a pinned record to `stopped` (which this pass reads as an intentional // a pinned record to `stopped` (which this pass reads as an intentional
// kill, making the session permanently unrestorable) and delete the // kill, making the session permanently unrestorable) and delete the
// workspace's `.claude-images`. This undoes only the construction. // workspace's `.codeman-uploads`. This undoes only the construction.
await ctx await ctx
.discardPartiallyBuiltSession(entry.sessionId) .discardPartiallyBuiltSession(entry.sessionId)
.catch((discardErr: unknown) => .catch((discardErr: unknown) =>
+65 -31
View File
@@ -188,6 +188,7 @@ import {
} from '../response-viewer-transcript.js'; } from '../response-viewer-transcript.js';
import { readDeepSeekLastResponse } from '../../deepseek-transcript.js'; import { readDeepSeekLastResponse } from '../../deepseek-transcript.js';
import { appendClaudeCustomTitle } from '../../claude-session-title.js'; import { appendClaudeCustomTitle } from '../../claude-session-title.js';
import { UPLOADS_DIR } from '../paste-image-gc.js';
// Path to linked-cases registry (same file used by case-routes resolveCasePath) // Path to linked-cases registry (same file used by case-routes resolveCasePath)
const LINKED_CASES_FILE = dataPath('linked-cases.json'); const LINKED_CASES_FILE = dataPath('linked-cases.json');
@@ -361,6 +362,51 @@ export function imageMagicMatchesExt(data: Buffer, ext: string): boolean {
} }
} }
/**
* Create or re-verify `{workingDir}/.codeman-uploads` for a prompt upload, or
* null when something other than a regular directory sits there. An agent or
* postinstall script could plant `.codeman-uploads -> ~/.ssh/` and redirect
* future writes outside workingDir: lstat (not stat) sees the symlink itself,
* and mkdir without `recursive` does not follow one for the leaf either;
* O_EXCL|O_NOFOLLOW on the file open makes the write itself symlink-safe.
* The folder ignores itself: a `.gitignore` of `*`, written once with O_EXCL,
* so the user's repository never sees uploads, their own ignore file is never
* touched, and a file already there is theirs and stays as it is.
*/
async function ensureUploadDir(workingDir: string): Promise<string | null> {
// workingDir is guaranteed to exist (live session).
const uploadDir = join(workingDir, UPLOADS_DIR);
try {
const dirStat = await fs.lstat(uploadDir);
if (dirStat.isSymbolicLink() || !dirStat.isDirectory()) return null;
} catch (err: unknown) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') throw err;
try {
await fs.mkdir(uploadDir);
} catch (mkErr: unknown) {
// Concurrent uploads (a batch of photos) race to create the dir — the
// losers get EEXIST. Treat an already-present REAL directory as success,
// but re-verify it isn't a symlink a racing actor planted.
if ((mkErr as NodeJS.ErrnoException).code !== 'EEXIST') throw mkErr;
const raceStat = await fs.lstat(uploadDir);
if (raceStat.isSymbolicLink() || !raceStat.isDirectory()) return null;
}
}
const ignoreFile = join(uploadDir, '.gitignore');
try {
// 'wx' = O_CREAT|O_EXCL, which also fails on a symlink at the path.
await fs.writeFile(ignoreFile, '*\n', { flag: 'wx' });
} catch (err: unknown) {
if ((err as NodeJS.ErrnoException).code === 'EEXIST') return uploadDir;
// The create can succeed before the write fails (ENOSPC): an empty ignore
// file would read as the user's on the next upload, so take it back; its own
// failure must not replace the cause.
await fs.rm(ignoreFile, { force: true }).catch(() => {});
throw err;
}
return uploadDir;
}
// Per-(IP, sessionId) token bucket for paste-image. 30 requests/minute. // Per-(IP, sessionId) token bucket for paste-image. 30 requests/minute.
// Bucket map entries are pruned when they drift > 1h stale to bound memory // Bucket map entries are pruned when they drift > 1h stale to bound memory
// against a flood of unique IP keys. // against a flood of unique IP keys.
@@ -5198,6 +5244,20 @@ export function registerSessionRoutes(
const session = findSessionOrFail(ctx, id, req); const session = findSessionOrFail(ctx, id, req);
// The file lands on THIS host under the session's working directory, which
// for a remote (SSH) session is the remote path: the agent there could never
// read it, and the write would land in a same-named local directory or fail.
// An owned Docker case is fine, its workspace is bind-mounted at the same
// absolute path; an adopted container (owned: false) mounts nothing, so its
// agent reads the file only if the container exposes that host path.
if (session.remote) {
reply.code(400);
return createErrorResponse(
ApiErrorCode.INVALID_INPUT,
'Prompt uploads are not supported for remote (SSH) sessions'
);
}
if (!req.isMultipart()) { if (!req.isMultipart()) {
reply.code(400); reply.code(400);
return createErrorResponse(ApiErrorCode.INVALID_INPUT, 'Expected multipart/form-data'); return createErrorResponse(ApiErrorCode.INVALID_INPUT, 'Expected multipart/form-data');
@@ -5294,37 +5354,11 @@ export function registerSessionRoutes(
return createErrorResponse(ApiErrorCode.INVALID_INPUT, `Image bytes do not match declared type ${ext}`); return createErrorResponse(ApiErrorCode.INVALID_INPUT, `Image bytes do not match declared type ${ext}`);
} }
// Save to {workingDir}/.claude-images/ // {workingDir}/.codeman-uploads/, see ensureUploadDir.
// Refuse symlinks at imageDir — an agent or postinstall script could plant const imageDir = await ensureUploadDir(session.workingDir);
// `.claude-images -> ~/.ssh/` and redirect future writes outside workingDir. if (!imageDir) {
// We lstat (not stat) so we see the symlink itself. Use mkdir without reply.code(403);
// `recursive` so the leaf creation does not follow a symlink either, and return createErrorResponse(ApiErrorCode.INVALID_INPUT, `${UPLOADS_DIR} is not a regular directory`);
// O_EXCL|O_NOFOLLOW on the file open so the write itself is symlink-safe.
const imageDir = join(session.workingDir, '.claude-images');
try {
const dirStat = await fs.lstat(imageDir);
if (dirStat.isSymbolicLink() || !dirStat.isDirectory()) {
reply.code(403);
return createErrorResponse(ApiErrorCode.INVALID_INPUT, '.claude-images is not a regular directory');
}
} catch (err: unknown) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') throw err;
// Non-recursive mkdir: does not follow symlinks for the leaf.
// session.workingDir is guaranteed to exist (live session).
try {
await fs.mkdir(imageDir);
} catch (mkErr: unknown) {
// Concurrent uploads (a batch of photos) race to create .claude-images —
// the losers get EEXIST. Treat an already-present REAL directory as
// success, but re-verify it isn't a symlink a racing actor planted
// (preserve the symlink-safety guarantee above).
if ((mkErr as NodeJS.ErrnoException).code !== 'EEXIST') throw mkErr;
const raceStat = await fs.lstat(imageDir);
if (raceStat.isSymbolicLink() || !raceStat.isDirectory()) {
reply.code(403);
return createErrorResponse(ApiErrorCode.INVALID_INPUT, '.claude-images is not a regular directory');
}
}
} }
// Date.now() collides on same-ms uploads from two tabs (last-write wins // Date.now() collides on same-ms uploads from two tabs (last-write wins
// silently). Append 8 hex chars so concurrent pastes get distinct names. // silently). Append 8 hex chars so concurrent pastes get distinct names.
+14 -9
View File
@@ -33,11 +33,11 @@ import fastifyCookie from '@fastify/cookie';
import fastifyStatic from '@fastify/static'; import fastifyStatic from '@fastify/static';
import fastifyWebsocket from '@fastify/websocket'; import fastifyWebsocket from '@fastify/websocket';
import fastifyMultipart from '@fastify/multipart'; import fastifyMultipart from '@fastify/multipart';
import { pasteImageDirInUseByOtherSession, startPasteImageGc } from './paste-image-gc.js'; import { pasteImageDirInUseByOtherSession, startPasteImageGc, uploadDirs } from './paste-image-gc.js';
import { CLEAN_EXIT_CLOSE_REASON, shouldCloseCleanlyExitedSession } from '../pane-exit-sweep.js'; import { CLEAN_EXIT_CLOSE_REASON, shouldCloseCleanlyExitedSession } from '../pane-exit-sweep.js';
import { join, dirname } from 'node:path'; import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url'; import { fileURLToPath } from 'node:url';
import { existsSync, mkdirSync, readFileSync, chmodSync, rmSync, statSync } from 'node:fs'; import { existsSync, mkdirSync, readFileSync, chmodSync, statSync } from 'node:fs';
import fs from 'node:fs/promises'; import fs from 'node:fs/promises';
import { execSync } from 'node:child_process'; import { execSync } from 'node:child_process';
import { hostname as getHostname, uptime as osUptime } from 'node:os'; import { hostname as getHostname, uptime as osUptime } from 'node:os';
@@ -1546,11 +1546,16 @@ export class WebServer extends EventEmitter {
killing: this.killingSessions, killing: this.killingSessions,
}) })
) { ) {
const pasteImageDir = join(session.workingDir, '.claude-images'); // Both upload dirs, the pre-move one too; uploadDirs() lists only real
try { // directories that are not the data dir and do not contain it, none for a
rmSync(pasteImageDir, { recursive: true, force: true }); // remote session, and nothing on a workspace whose bounded probe did not
} catch { // answer (pastCap: this acts on one path at the user's request).
// Best-effort cleanup for (const uploadDir of await uploadDirs(session, { pastCap: true })) {
try {
await fs.rm(uploadDir, { recursive: true, force: true });
} catch {
// Best-effort cleanup
}
} }
} }
// Drop the agent skill's preamble cache for this session (seeded at create). // Drop the agent skill's preamble cache for this session (seeded at create).
@@ -2911,7 +2916,7 @@ export class WebServer extends EventEmitter {
} }
// Bound disk use under heavy paste-image traffic: delete `paste-*` files // Bound disk use under heavy paste-image traffic: delete `paste-*` files
// older than 7 days from each live session's .claude-images/ hourly. // older than 7 days from each live session's upload dirs hourly.
if (!this.testMode) { if (!this.testMode) {
this._pasteImageGcStop = startPasteImageGc({ sessions: this.sessions }); this._pasteImageGcStop = startPasteImageGc({ sessions: this.sessions });
// Surface event-loop stalls (e.g. a slow synchronous tmux/ps call) so the // Surface event-loop stalls (e.g. a slow synchronous tmux/ps call) so the
@@ -3363,7 +3368,7 @@ export class WebServer extends EventEmitter {
* path adds the session's token totals to the lifetime figures, demotes a * path adds the session's token totals to the lifetime figures, demotes a
* pinned record to `stopped` (the durable marker of an intentional kill, which * pinned record to `stopped` (the durable marker of an intentional kill, which
* would make the session permanently ineligible for a reboot restore), drops * would make the session permanently ineligible for a reboot restore), drops
* the persisted Ralph state, and recursively removes `.claude-images` from the * the persisted Ralph state, and recursively removes the upload dirs from the
* WORKING DIRECTORY, which belongs to the workspace rather than to this session * WORKING DIRECTORY, which belongs to the workspace rather than to this session
* and may hold another live session's pasted images. * and may hold another live session's pasted images.
* *
+9
View File
@@ -31,6 +31,7 @@ vi.mock('node:fs', async (importOriginal) => {
}); });
import { ImageWatcher } from '../src/image-watcher.js'; import { ImageWatcher } from '../src/image-watcher.js';
import { watch } from 'chokidar';
import { statSync } from 'node:fs'; import { statSync } from 'node:fs';
describe('ImageWatcher', () => { describe('ImageWatcher', () => {
@@ -92,6 +93,14 @@ describe('ImageWatcher', () => {
expect(watcher.getWatchedSessions()).toHaveLength(1); expect(watcher.getWatchedSessions()).toHaveLength(1);
}); });
it("ignores Codeman's own upload folders, so a pdf the user handed over is not a detected artifact", () => {
watcher.watchSession('session-1', '/home/user/project');
const [, opts] = vi.mocked(watch).mock.calls.at(-1) as unknown as [string, { ignored: (p: string) => boolean }];
expect(opts.ignored('/home/user/project/.codeman-uploads/paste-1-ab.pdf')).toBe(true);
expect(opts.ignored('/home/user/project/.claude-images/paste-1-ab.png')).toBe(true);
expect(opts.ignored('/home/user/project/docs/report.pdf')).toBe(false);
});
it('should replace watcher when working directory changes', () => { it('should replace watcher when working directory changes', () => {
watcher.watchSession('session-1', '/home/user/project-a'); watcher.watchSession('session-1', '/home/user/project-a');
watcher.watchSession('session-1', '/home/user/project-b'); watcher.watchSession('session-1', '/home/user/project-b');
+2 -2
View File
@@ -124,11 +124,11 @@ describe('terminal link-provider regexes (shipped source)', () => {
}); });
it('the file-path pattern links pasted image/PDF/media attachment paths', () => { it('the file-path pattern links pasted image/PDF/media attachment paths', () => {
// `.claude-images/paste-*.png` is what Codeman writes for a pasted screenshot; // `.codeman-uploads/paste-*.png` is what Codeman writes for a pasted screenshot;
// without image extensions the path rendered as plain, unclickable text. // without image extensions the path rendered as plain, unclickable text.
const ext = shippedPattern('FILE_PATH_LINK_PATTERN'); const ext = shippedPattern('FILE_PATH_LINK_PATTERN');
const cases = [ const cases = [
'/home/arkon/default/claudeman/.claude-images/paste-1785164958410-d11eb7d0.png', '/home/arkon/default/claudeman/.codeman-uploads/paste-1785164958410-d11eb7d0.png',
'/tmp/shot.jpeg', '/tmp/shot.jpeg',
'/opt/app/report.pdf', '/opt/app/report.pdf',
'/home/a/diagram.svg', '/home/a/diagram.svg',
+47 -10
View File
@@ -1,8 +1,9 @@
/** /**
* @fileoverview Deleting a session keeps `.claude-images` while a sibling in the * @fileoverview Deleting a session keeps the upload dirs while a sibling in the
* same working directory is still live (Ark0N/Codeman#446). * same working directory is still live (Ark0N/Codeman#446).
* *
* `cleanupSession()` removes `{workingDir}/.claude-images` recursively. That * `cleanupSession()` removes `{workingDir}/.codeman-uploads` (and the pre-move
* `.claude-images`) recursively. That
* dir belongs to the working directory, not to the session, and several * dir belongs to the working directory, not to the session, and several
* sessions routinely share one case directory, so closing one used to delete * sessions routinely share one case directory, so closing one used to delete
* the pasted images a live sibling still referred to. The exited-agent sweep * the pasted images a live sibling still referred to. The exited-agent sweep
@@ -11,12 +12,18 @@
* *
* Port: ephemeral * Port: ephemeral
*/ */
import { existsSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; import { existsSync, lstatSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os'; import { tmpdir } from 'node:os';
import { join } from 'node:path'; import { join } from 'node:path';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'; import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { WebServer } from '../src/web/server.js'; import { WebServer } from '../src/web/server.js';
import { pasteImageDirInUseByOtherSession } from '../src/web/paste-image-gc.js'; import { pasteImageDirInUseByOtherSession } from '../src/web/paste-image-gc.js';
import { probePathKind } from '../src/utils/bounded-path-probe.js';
vi.mock('../src/utils/bounded-path-probe.js', async (importOriginal) => {
const real = await importOriginal<typeof import('../src/utils/bounded-path-probe.js')>();
return { ...real, probePathKind: vi.fn(real.probePathKind) };
});
describe('pasteImageDirInUseByOtherSession', () => { describe('pasteImageDirInUseByOtherSession', () => {
const none = new Set<string>(); const none = new Set<string>();
@@ -132,18 +139,48 @@ describe('deleting a session that shares its working directory', () => {
const remove = (id: string) => fetch(`${base}/api/sessions/${id}`, { method: 'DELETE' }); const remove = (id: string) => fetch(`${base}/api/sessions/${id}`, { method: 'DELETE' });
it('keeps the images while a sibling is live, and removes them with the last session', async () => { it('keeps the uploads while a sibling is live, and removes them with the last session', async () => {
const first = await create(); const first = await create();
const second = await create(); const second = await create();
const imageDir = join(workingDir, '.claude-images'); // Both the current dir and the pre-move one, which a case in use back then still carries.
mkdirSync(imageDir, { recursive: true }); const uploadDir = join(workingDir, '.codeman-uploads');
writeFileSync(join(imageDir, 'paste-1.png'), 'x'); const legacyDir = join(workingDir, '.claude-images');
mkdirSync(uploadDir, { recursive: true });
mkdirSync(legacyDir, { recursive: true });
writeFileSync(join(uploadDir, 'paste-1.png'), 'x');
writeFileSync(join(legacyDir, 'paste-0.png'), 'x');
expect((await remove(first)).status).toBe(200); expect((await remove(first)).status).toBe(200);
expect(existsSync(join(imageDir, 'paste-1.png'))).toBe(true); expect(existsSync(join(uploadDir, 'paste-1.png'))).toBe(true);
expect(existsSync(join(legacyDir, 'paste-0.png'))).toBe(true);
// The delete acts on one path at the user's request, so its probe goes past
// the bulk stall cap (#516); the hourly sweep keeps the cap. Cleared first:
// the create path probes the same workspace past the cap too.
vi.mocked(probePathKind).mockClear();
expect((await remove(second)).status).toBe(200); expect((await remove(second)).status).toBe(200);
expect(existsSync(imageDir)).toBe(false); expect(existsSync(uploadDir)).toBe(false);
expect(existsSync(legacyDir)).toBe(false);
expect(probePathKind).toHaveBeenCalledWith(workingDir, { pastCap: true });
});
it('does not follow a planted symlink at the upload dir into another case when the last session closes', async () => {
const other = mkdtempSync(join(tmpdir(), 'codeman-paste-other-'));
const otherUploads = join(other, '.codeman-uploads');
mkdirSync(otherUploads);
writeFileSync(join(otherUploads, 'paste-7.png'), 'x');
// Planted by a workspace script. uploadDirs() never lists a link, so the delete
// removes nothing here, and the link itself stays: it is not Codeman's to remove.
symlinkSync(otherUploads, join(workingDir, '.codeman-uploads'));
try {
const only = await create();
expect((await remove(only)).status).toBe(200);
expect(existsSync(join(otherUploads, 'paste-7.png'))).toBe(true);
expect(lstatSync(join(workingDir, '.codeman-uploads')).isSymbolicLink()).toBe(true);
} finally {
rmSync(join(workingDir, '.codeman-uploads'), { force: true });
rmSync(other, { recursive: true, force: true });
}
}); });
it('keeps the images while a sibling is only detached, since it still runs in tmux', async () => { it('keeps the images while a sibling is only detached, since it still runs in tmux', async () => {
@@ -153,7 +190,7 @@ describe('deleting a session that shares its working directory', () => {
// write, so wait for the record a long-running session would already have. // write, so wait for the record a long-running session would already have.
const store = (server as unknown as { store: { getSession: (id: string) => unknown } }).store; const store = (server as unknown as { store: { getSession: (id: string) => unknown } }).store;
await vi.waitFor(() => expect(store.getSession(detached)).toBeTruthy(), { timeout: 10_000 }); await vi.waitFor(() => expect(store.getSession(detached)).toBeTruthy(), { timeout: 10_000 });
const imageDir = join(workingDir, '.claude-images'); const imageDir = join(workingDir, '.codeman-uploads');
mkdirSync(imageDir, { recursive: true }); mkdirSync(imageDir, { recursive: true });
writeFileSync(join(imageDir, 'paste-2.png'), 'x'); writeFileSync(join(imageDir, 'paste-2.png'), 'x');
+157
View File
@@ -0,0 +1,157 @@
/**
* @fileoverview The hourly upload sweep reads BOTH upload dirs of a live session:
* `.codeman-uploads` and the `.claude-images` a case in use before the move still
* carries. Only `paste-*` regular files past the age cap go. `uploadDirs()` never
* lists a planted symlink, a directory that is or contains the data dir, anything
* under a workspace whose bounded probe did not answer, or anything for a remote
* session, since the sweep and the delete cleanup both act on what it returns.
*/
import { existsSync, lstatSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, utimesSync, writeFileSync } from 'node:fs';
import fsp from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { LEGACY_UPLOADS_DIR, UPLOADS_DIR, sweepPasteImagesOnce, uploadDirs } from '../src/web/paste-image-gc.js';
import { probePathKind } from '../src/utils/bounded-path-probe.js';
vi.mock('../src/utils/bounded-path-probe.js', async (importOriginal) => {
const real = await importOriginal<typeof import('../src/utils/bounded-path-probe.js')>();
return { ...real, probePathKind: vi.fn(real.probePathKind) };
});
const DAY_MS = 24 * 60 * 60 * 1000;
const sessionsOf = (workingDir: string, remote?: unknown) => ({
sessions: new Map([['s1', { workingDir, remote } as never]]) as never,
});
describe('sweepPasteImagesOnce', () => {
const dirs: string[] = [];
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true });
});
it('ages paste-* files out of both the current and the legacy upload dir', async () => {
const workingDir = mkdtempSync(join(tmpdir(), 'codeman-gc-'));
dirs.push(workingDir);
const now = Date.now();
const current = join(workingDir, UPLOADS_DIR);
const legacy = join(workingDir, LEGACY_UPLOADS_DIR);
mkdirSync(current);
mkdirSync(legacy);
expect(await uploadDirs({ workingDir })).toEqual([current, legacy]);
const old = (now - 8 * DAY_MS) / 1000;
const fresh = (now - 1 * DAY_MS) / 1000;
const plant = (dir: string, name: string, mtime: number) => {
writeFileSync(join(dir, name), 'x');
utimesSync(join(dir, name), mtime, mtime);
};
plant(current, 'paste-1-aa.pdf', old);
plant(current, 'paste-2-bb.png', fresh);
plant(current, 'keep.png', old); // not an upload of ours
plant(legacy, 'paste-0-cc.png', old);
symlinkSync(join(legacy, 'paste-0-cc.png'), join(current, 'paste-3-dd.png'));
utimesSync(join(current, 'paste-3-dd.png'), old, old);
const result = await sweepPasteImagesOnce(sessionsOf(workingDir), now);
expect(result.deleted).toBe(2);
expect(existsSync(join(current, 'paste-1-aa.pdf'))).toBe(false);
expect(existsSync(join(legacy, 'paste-0-cc.png'))).toBe(false);
expect(existsSync(join(current, 'paste-2-bb.png'))).toBe(true);
expect(existsSync(join(current, 'keep.png'))).toBe(true);
// A planted symlink is skipped (lstat, never followed), so the sweep leaves the link itself.
expect(lstatSync(join(current, 'paste-3-dd.png')).isSymbolicLink()).toBe(true);
});
it('never reads through a planted symlink at the upload dir', async () => {
const workingDir = mkdtempSync(join(tmpdir(), 'codeman-gc-'));
const other = mkdtempSync(join(tmpdir(), 'codeman-gc-other-'));
dirs.push(workingDir, other);
const now = Date.now();
const old = (now - 8 * DAY_MS) / 1000;
mkdirSync(join(other, UPLOADS_DIR));
writeFileSync(join(other, UPLOADS_DIR, 'paste-9-zz.png'), 'x');
utimesSync(join(other, UPLOADS_DIR, 'paste-9-zz.png'), old, old);
// readdir follows a link to a directory, so a listed link would let the sweep
// age out the other case's uploads through it.
symlinkSync(join(other, UPLOADS_DIR), join(workingDir, UPLOADS_DIR));
expect(await uploadDirs({ workingDir })).toEqual([]);
expect(await sweepPasteImagesOnce(sessionsOf(workingDir), now)).toEqual({ scanned: 0, deleted: 0 });
expect(existsSync(join(other, UPLOADS_DIR, 'paste-9-zz.png'))).toBe(true);
});
});
describe('uploadDirs', () => {
const dirs: string[] = [];
const savedDataDir = process.env.CODEMAN_DATA_DIR;
afterEach(() => {
if (savedDataDir === undefined) delete process.env.CODEMAN_DATA_DIR;
else process.env.CODEMAN_DATA_DIR = savedDataDir;
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true });
});
it('lists the upload dir of a home workspace, which sits beside the data dir, not inside it', async () => {
// The home-as-workspace case the flat name exists for: `<home>/.codeman-uploads`
// is a sibling of `<home>/.codeman`, so a prefix test without the separator would
// wrongly refuse it.
const home = mkdtempSync(join(tmpdir(), 'codeman-gc-home-'));
dirs.push(home);
process.env.CODEMAN_DATA_DIR = join(home, '.codeman');
const dir = join(home, UPLOADS_DIR);
mkdirSync(dir);
expect(await uploadDirs({ workingDir: home })).toEqual([dir]);
});
it('refuses an upload dir that is the data dir or contains it, and lists one inside it', async () => {
const root = mkdtempSync(join(tmpdir(), 'codeman-gc-data-'));
const parent = mkdtempSync(join(tmpdir(), 'codeman-gc-link-'));
dirs.push(root, parent);
const uploads = join(root, UPLOADS_DIR);
mkdirSync(join(uploads, 'state'), { recursive: true });
// `CODEMAN_INSTANCE=uploads` on a home workspace: the upload dir IS the data dir.
process.env.CODEMAN_DATA_DIR = uploads;
expect(await uploadDirs({ workingDir: root })).toEqual([]);
// A data dir pointed inside the upload dir: the recursive delete would take it.
process.env.CODEMAN_DATA_DIR = join(uploads, 'state');
expect(await uploadDirs({ workingDir: root })).toEqual([]);
// An upload dir strictly inside the data dir only ever holds uploads (install.sh
// clones the app to ~/.codeman/app), so it is listed, directly and through a
// symlinked working directory.
process.env.CODEMAN_DATA_DIR = root;
expect(await uploadDirs({ workingDir: root })).toEqual([uploads]);
symlinkSync(root, join(parent, 'ws'));
expect(await uploadDirs({ workingDir: join(parent, 'ws') })).toEqual([join(parent, 'ws', UPLOADS_DIR)]);
});
it('skips a workspace whose bounded probe does not answer, without touching it', async () => {
// A linked case on a mount that stopped answering reads `unknown` (#516): the
// sweep must not lstat under it, which would hold a threadpool worker.
const workingDir = mkdtempSync(join(tmpdir(), 'codeman-gc-stalled-'));
dirs.push(workingDir);
mkdirSync(join(workingDir, UPLOADS_DIR));
vi.mocked(probePathKind).mockResolvedValueOnce('unknown');
const lstat = vi.spyOn(fsp, 'lstat');
try {
expect(await uploadDirs({ workingDir })).toEqual([]);
expect(lstat).not.toHaveBeenCalled();
} finally {
lstat.mockRestore();
}
expect(probePathKind).toHaveBeenCalledWith(workingDir, {});
});
it('lists nothing for a remote session, whose workingDir is the remote path', async () => {
// The same path on THIS host belongs to whoever has a local case there.
const workingDir = mkdtempSync(join(tmpdir(), 'codeman-gc-remote-'));
dirs.push(workingDir);
mkdirSync(join(workingDir, UPLOADS_DIR));
writeFileSync(join(workingDir, UPLOADS_DIR, 'paste-1-aa.png'), 'x');
const old = (Date.now() - 8 * DAY_MS) / 1000;
utimesSync(join(workingDir, UPLOADS_DIR, 'paste-1-aa.png'), old, old);
const remote = { hostId: 'gpu-box' };
expect(await uploadDirs({ workingDir, remote })).toEqual([]);
expect(await sweepPasteImagesOnce(sessionsOf(workingDir, remote))).toEqual({ scanned: 0, deleted: 0 });
expect(existsSync(join(workingDir, UPLOADS_DIR, 'paste-1-aa.png'))).toBe(true);
});
});
@@ -144,7 +144,7 @@ describe('a rebuild that fails after the session is registered', () => {
expect(ctx.sessions.has('a')).toBe(false); expect(ctx.sessions.has('a')).toBe(false);
// NOT the user-initiated delete: that would bank this session's historical // NOT the user-initiated delete: that would bank this session's historical
// tokens into the lifetime totals, demote a pinned record to `stopped`, and // tokens into the lifetime totals, demote a pinned record to `stopped`, and
// delete the workspace's .claude-images. // delete the workspace's .codeman-uploads.
expect(ctx.cleanupSession).not.toHaveBeenCalled(); expect(ctx.cleanupSession).not.toHaveBeenCalled();
await app.close(); await app.close();
}); });
+87 -4
View File
@@ -18,9 +18,9 @@ import Fastify, { type FastifyInstance } from 'fastify';
import fastifyCookie from '@fastify/cookie'; import fastifyCookie from '@fastify/cookie';
import fastifyMultipart from '@fastify/multipart'; import fastifyMultipart from '@fastify/multipart';
import { dirname, join } from 'node:path'; import { dirname, join } from 'node:path';
import { mkdirSync, rmSync, writeFileSync } from 'node:fs'; import { existsSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
import { registryFilePath, reloadCliRegistry } from '../../src/config/cli-registry/registry.js'; import { registryFilePath, reloadCliRegistry } from '../../src/config/cli-registry/registry.js';
import { mkdtemp, rm, mkdir, writeFile } from 'node:fs/promises'; import fs, { mkdtemp, rm, mkdir, writeFile, readFile, readdir, symlink } from 'node:fs/promises';
import { tmpdir } from 'node:os'; import { tmpdir } from 'node:os';
import { createMockRouteContext, type MockRouteContext } from '../mocks/index.js'; import { createMockRouteContext, type MockRouteContext } from '../mocks/index.js';
import { installRouteErrorHandler } from '../../src/web/route-error-handler.js'; import { installRouteErrorHandler } from '../../src/web/route-error-handler.js';
@@ -285,7 +285,7 @@ describe('session-routes', () => {
expect(res.statusCode).toBe(200); expect(res.statusCode).toBe(200);
const body = JSON.parse(res.body); const body = JSON.parse(res.body);
expect(body.success).toBe(true); expect(body.success).toBe(true);
expect(body.data.path).toMatch(/\/\.claude-images\/paste-\d+-[a-f0-9]{8}\.jpg$/); expect(body.data.path).toMatch(/\/\.codeman-uploads\/paste-\d+-[a-f0-9]{8}\.jpg$/);
expect(heicConvert).toHaveBeenCalledWith(heic); expect(heicConvert).toHaveBeenCalledWith(heic);
}); });
@@ -314,10 +314,93 @@ describe('session-routes', () => {
expect(res.statusCode).toBe(200); expect(res.statusCode).toBe(200);
const body = JSON.parse(res.body); const body = JSON.parse(res.body);
expect(body.success).toBe(true); expect(body.success).toBe(true);
expect(body.data.path).toMatch(/\/\.claude-images\/paste-\d+-[a-f0-9]{8}\.jpg$/); expect(body.data.path).toMatch(/\/\.codeman-uploads\/paste-\d+-[a-f0-9]{8}\.jpg$/);
expect(heicConvert).toHaveBeenCalledWith(heic); expect(heicConvert).toHaveBeenCalledWith(heic);
}); });
const jpeg = Buffer.from('ffd8ffe000104a46494600010100', 'hex');
const upload = (name: string, mime: string, bytes: Buffer) =>
harness.app.inject({
method: 'POST',
url: `/api/sessions/${harness.ctx._sessionId}/paste-image`,
headers: {
host: 'codeman.test',
origin: 'http://codeman.test',
'content-type': 'multipart/form-data; boundary=codeman-test-boundary',
},
payload: imageUploadBody('codeman-test-boundary', name, mime, bytes),
});
it('writes into .codeman-uploads with a self-ignoring .gitignore, written once and never over a file already there', async () => {
const workDir = await mkdtemp(join(tmpdir(), 'codeman-uploads-'));
harness.ctx._session.workingDir = workDir;
try {
const first = await upload('shot.jpg', 'image/jpeg', jpeg);
expect(first.statusCode).toBe(200);
expect(JSON.parse(first.body).data.path).toMatch(/\/\.codeman-uploads\/paste-\d+-[a-f0-9]{8}\.jpg$/);
expect(await readdir(workDir)).toEqual(['.codeman-uploads']);
expect(await readFile(join(workDir, '.codeman-uploads', '.gitignore'), 'utf8')).toBe('*\n');
await writeFile(join(workDir, '.codeman-uploads', '.gitignore'), 'theirs\n');
expect((await upload('shot.jpg', 'image/jpeg', jpeg)).statusCode).toBe(200);
expect(await readFile(join(workDir, '.codeman-uploads', '.gitignore'), 'utf8')).toBe('theirs\n');
} finally {
await rm(workDir, { recursive: true });
}
});
it('takes back an ignore file whose write failed, so the next upload writes a real one', async () => {
const workDir = await mkdtemp(join(tmpdir(), 'codeman-uploads-'));
harness.ctx._session.workingDir = workDir;
const ignoreFile = join(workDir, '.codeman-uploads', '.gitignore');
// The exclusive create succeeds and the two-byte write fails, as on a full disk.
const spy = vi.spyOn(fs, 'writeFile').mockImplementationOnce(async (path) => {
await writeFile(path as string, '');
throw Object.assign(new Error('no space left on device'), { code: 'ENOSPC' });
});
try {
expect((await upload('shot.jpg', 'image/jpeg', jpeg)).statusCode).toBe(500);
expect(existsSync(ignoreFile)).toBe(false);
expect((await upload('shot.jpg', 'image/jpeg', jpeg)).statusCode).toBe(200);
expect(await readFile(ignoreFile, 'utf8')).toBe('*\n');
} finally {
spy.mockRestore();
await rm(workDir, { recursive: true });
}
});
it('refuses a planted symlink at .codeman-uploads and writes nothing through it', async () => {
const workDir = await mkdtemp(join(tmpdir(), 'codeman-uploads-'));
const elsewhere = await mkdtemp(join(tmpdir(), 'codeman-elsewhere-'));
harness.ctx._session.workingDir = workDir;
await symlink(elsewhere, join(workDir, '.codeman-uploads'));
try {
const res = await upload('shot.jpg', 'image/jpeg', jpeg);
expect(res.statusCode).toBe(403);
expect(JSON.parse(res.body).error).toBe('.codeman-uploads is not a regular directory');
expect(await readdir(elsewhere)).toEqual([]);
} finally {
await rm(workDir, { recursive: true });
await rm(elsewhere, { recursive: true });
}
});
it('refuses an upload for a remote session before touching disk', async () => {
const workDir = await mkdtemp(join(tmpdir(), 'codeman-uploads-'));
harness.ctx._session.workingDir = workDir;
// A remote session's workingDir is the REMOTE path: a file written here is unreadable there.
(harness.ctx._session as unknown as { remote: unknown }).remote = { hostId: 'gpu-box' };
try {
const res = await upload('shot.jpg', 'image/jpeg', jpeg);
expect(res.statusCode).toBe(400);
expect(JSON.parse(res.body).error).toMatch(/remote/);
expect(await readdir(workDir)).toEqual([]);
} finally {
(harness.ctx._session as unknown as { remote: unknown }).remote = undefined;
await rm(workDir, { recursive: true });
}
});
it('returns 415 with the error envelope when HEIC conversion fails', async () => { it('returns 415 with the error envelope when HEIC conversion fails', async () => {
heicConvert.mockClear(); heicConvert.mockClear();
heicConvert.mockRejectedValueOnce(new Error('HEIC dimensions 30000x30000 exceed the 64MP decode limit')); heicConvert.mockRejectedValueOnce(new Error('HEIC dimensions 30000x30000 exceed the 64MP decode limit'));