diff --git a/.changeset/docker-self-update.md b/.changeset/docker-self-update.md new file mode 100644 index 00000000..00c2ccc3 --- /dev/null +++ b/.changeset/docker-self-update.md @@ -0,0 +1,37 @@ +--- +'aicodeman': minor +--- + +Restore in-app self-update for the Docker Compose deployment. + +App Settings → Updates now works in the container, using the same updater, +status file and progress UI as a bare-host install. The Compose file mounts the +checkout it builds from at `/opt/codeman`, so an update's `git checkout` and +rebuild land on the host and survive container recreation, and the restart is +the server exiting — `restart: unless-stopped` relaunches it on the new build. + +An in-place container update applies application code only, since a restarted +container reuses its existing image and configuration. The updater therefore +refuses a release that changes `docker/server.Dockerfile` or +`docker/docker-compose.yaml`, or that adds keys to `docker/.env.example` the +user's `.env` has no value for, naming what changed and pointing at +`docker/Start-Codeman.sh` on the host. It also refuses when the container's +restart policy would not bring it back. The missing-key check matters most: +Compose resolves an unset `${VAR}` to the empty string and starts anyway, so a +new required setting would otherwise arrive as a silently blank variable. + +Supporting changes: + +- The runtime image keeps devDependencies and gains `python3`/`make`/`g++`, so + `npm install` and `npm run build` can run inside the container. This makes the + image larger; that is the cost of updating in place. +- Build artefacts live in `codeman-node-modules` and `codeman-dist` named + volumes so container-compiled native modules never land in the host checkout. +- The four global agent CLIs are pinned, so a release needing newer CLI + behaviour becomes a Dockerfile change the environment gate can detect. +- `docker/Start-Codeman.sh` records the Dockerfile and compose fingerprints the + container was created from, which is the baseline the gate compares against. +- New CI guard: `test/docker-compose-env-parity.test.ts` fails when a compose + variable has no `.env.example` entry, or the reverse. + +Documented in `docs/docker-self-update.md`. diff --git a/CLAUDE.md b/CLAUDE.md index 50029cb9..943ff48b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -211,7 +211,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Docker cases**: a case can point at a **container**, with any of the CLI run modes running inside it. Like remote-SSH this is a **LOCATION OVERLAY on cases, never a `SessionMode` of its own**. Exactly one long-lived container **per case**, shared by all its sessions, so killing a session kills only that session's in-container tmux and **never** `docker stop` while siblings remain. The workspace is a real host dir bind-mounted at the **same absolute path**, which is what keeps file-routes/watchers on real host bytes and makes the in-container transcript projHash match the host. Credentials are **seeded** (RO mount, copied into the container once) rather than shared RW, so in-container CLIs never write refreshed tokens back to the host, and bind mounts are excluded from `docker commit` so exports stay secret-free. **NEVER a create-time `-e` for secrets, NEVER `--privileged`, NEVER the docker socket.** Config drift is detected via a label hash and a drifted launch is REFUSED rather than silently launched with stale config. ⚠️ On the loopback-only prod bind a container cannot reach 127.0.0.1, so in-container hooks need `CODEMAN_DOCKER_BRIDGE_HOOKS=1`; otherwise idle detection falls back to output-based. → [architecture-invariants#docker-cases](docs/architecture-invariants.md#docker-cases), `docs/docker-cases.md` (user guide), `docs/docker-cases-plan.md` (design) -**Docker Compose deployment** (`docker/`, contributed): Codeman itself runs in a container and spawns Docker cases as **SIBLING** containers through the mounted host socket (Docker-outside-of-Docker), never nested. That inverts one assumption the bare-host path takes for granted: the daemon no longer shares Codeman's filesystem, so a bind source valid *inside* Codeman means nothing to it. `resolveDockerDaemonMountSource()` translates sources under HOME into the daemon's namespace via `CODEMAN_DOCKER_HOST_HOME`, and `CODEMAN_CASES_PATH` points the cases dir at a host-absolute bind mount so a workspace resolves to the SAME absolute path on both sides (which is what keeps the transcript projHash matching, per Docker cases above). ⚠️ **`CODEMAN_CASES_PATH` must move every consumer or none**: it is resolved once in `config/cases-dir.ts` because `src/cli.ts` resolves case paths too, and when only the server's `CASES_DIR` learned the override, `codeman skill install --case ` reported "Case not found" on exactly the deployment the override exists for. ⚠️ **`.dockerignore` patterns match the WHOLE context-relative path**, so a bare `.env` line excludes only the ROOT file: `docker/.env` (which holds `CODEMAN_PASSWORD` and any provider keys) rode `COPY . .` into the image until `**/.env` was added — verified in both directions with a real build context. ⚠️ A Compose LONG-form bind (`type: bind`) **creates a missing host source directory ROOT-OWNED** rather than refusing, so any bind source the runtime user must write to has to be pre-created and chowned. `CODEMAN_DOCKER_DISABLE_SWAP_LIMIT=1` drops `--memory-swap` (and filters only that one kernel warning) for hosts without swap accounting; `--memory` still applies. `docs/docker-compose.md` + `docker/README.md` (user guides) +**Docker Compose deployment** (`docker/`, contributed): Codeman itself runs in a container and spawns Docker cases as **SIBLING** containers through the mounted host socket (Docker-outside-of-Docker), never nested. That inverts one assumption the bare-host path takes for granted: the daemon no longer shares Codeman's filesystem, so a bind source valid *inside* Codeman means nothing to it. `resolveDockerDaemonMountSource()` translates sources under HOME into the daemon's namespace via `CODEMAN_DOCKER_HOST_HOME`, and `CODEMAN_CASES_PATH` points the cases dir at a host-absolute bind mount so a workspace resolves to the SAME absolute path on both sides (which is what keeps the transcript projHash matching, per Docker cases above). ⚠️ **`CODEMAN_CASES_PATH` must move every consumer or none**: it is resolved once in `config/cases-dir.ts` because `src/cli.ts` resolves case paths too, and when only the server's `CASES_DIR` learned the override, `codeman skill install --case ` reported "Case not found" on exactly the deployment the override exists for. ⚠️ **`.dockerignore` patterns match the WHOLE context-relative path**, so a bare `.env` line excludes only the ROOT file: `docker/.env` (which holds `CODEMAN_PASSWORD` and any provider keys) rode `COPY . .` into the image until `**/.env` was added — verified in both directions with a real build context. ⚠️ A Compose LONG-form bind (`type: bind`) **creates a missing host source directory ROOT-OWNED** rather than refusing, so any bind source the runtime user must write to has to be pre-created and chowned. `CODEMAN_DOCKER_DISABLE_SWAP_LIMIT=1` drops `--memory-swap` (and filters only that one kernel warning) for hosts without swap accounting; `--memory` still applies. ⚠️ The deployment ALSO self-updates in place (the repo bind mount at `/opt/codeman` + a restart-by-exiting supervisor) — see Self-update below and `docs/docker-self-update.md` before touching `server.Dockerfile`, the compose file or `.env.example`, since each is an input to the updater's environment gate. `docs/docker-compose.md` + `docker/README.md` (user guides) **External CLI modes (OpenCode, Codex, Gemini, Antigravity, Pi, Grok, DeepSeek, OMP)**: `isExternalCliMode()` in `session.ts` gates Claude-specific behavior off (Ralph tracker, BashToolParser, token/CLI-info parsing, ❯-prompt readiness); these CLIs render their own TUIs, so readiness is output stabilization instead. All eight **require tmux with no direct PTY fallback**, because secrets are injected via socket-scoped `tmux setenv` and never on the spawn command line. ⚠️ `run*()` in `session-ui.js` MUST unwrap the `{success,data}` envelope; reading the raw shape silently breaks the run. ⚠️ **Codex sessions use PREDICTIVE WRITE-THROUGH echo, never the buffer overlay** (`_localEchoPolicy` in `_updateLocalEchoState`, terminal-ui.js): codex's composer reacts per keystroke ("/" pops a live-filtering picker, arrows edit server-side state, the composer grows as it wraps), so buffer-until-Enter starved it into issues #218/#219/#220/#222 and stays disabled (`_localEchoEnabled` remains false for codex). Instead, `PredictiveEchoAddon` (separate `vendor/xterm-predictive-echo.js` bundle) paints each keystroke at the predicted cell while the wire path stays BYTE-IDENTICAL: the onData hook (`_predictHookOnData`) is a plain statement with no `return`, so control always falls through into the untouched send path — pinned by vm and E2E byte-identity tests. Predictions reconcile against the parsed buffer and only while the cursor sits on the measured composer row (`isCodexComposerRow`, `/^› /`). Codex also **drops keystrokes that share a PTY read with a bracketed paste**, so flushed text and the paste sequence must go out as separate delayed writes (mirroring the Enter branch's delayed `\r`). Tests: `test/local-echo-codex-gating.test.ts`, `test/codex-predictive-echo.test.ts` (E2E vs real codex), `packages/xterm-zerolag-input/test/codex-replay.test.ts`. ⚠️ **Pi is the opposite kind of CLI and needs the opposite instincts**: it has NO permission prompts and no sandbox, so there is no bypass flag to send and Codeman must not invent one; its privileged knob is the tri-state `approveProjectTrust` (`--approve`/`--no-approve`), which makes pi EXECUTE repo-local `.pi/extensions` TypeScript, so the multi-user clamp puts pi in the **materialize** branch (an absent config still yields `--no-approve` for a non-granted owner) and `--api-key` is never wired. Pi stays OUT of `isAltScreenStripMode()` (main-screen TUI, and its 0.84.0 fullscreen mode is runtime-switchable via `/settings`, where the alt screen is load-bearing), and lands on the `'buffer'` echo policy via the `_updateLocalEchoState` fallthrough. Pi's own tests: `test/pi-mode.test.ts`, `test/routes/external-cli-bypass-clamp.test.ts`; user guide `docs/pi-integration.md`. ⚠️ **Grok is codex-shaped on permissions but opencode-shaped on rendering**: its bypass switch is `alwaysApprove` (`--always-approve`, grok's `bypassPermissions` mode — the Run button sends it `true` like antigravity's, and the clamp's only-if-sent branch strips it for non-granted owners), while its fullscreen alt-screen TUI keeps it OUT of `isAltScreenStripMode()`; the resolver version-probes `grok --version` like pi's (npm squatters exist for the name — `GET /api/grok/status` surfaces path + version), and grok lands on the `'buffer'` echo policy via the fallthrough (UNMEASURED against a live authenticated session; if its composer turns out per-keystroke-reactive like codex, flip it to the `'off'` branch). Grok's own tests: `test/grok-mode.test.ts`, `test/grok-cli-resolver.test.ts`; user guide `docs/grok-integration.md`. ⚠️ **DeepSeek breaks three of this family's assumptions, so do not pattern-match it onto its siblings.** (1) The agent is a **PROFILE, not the binary**: `dsh` is a launcher over `$DSH_HOME/profiles/` and DeepSeek ships only `web`/`headless`/`base`, so the terminal front door is ALWAYS third-party and "installed" ≠ "runnable" — the Run button gates on `isDeepSeekRunnable()` (binary AND a pane-capable profile) while `isDeepSeekAvailable()` gates the "add a profile" affordance; a `web`/`headless` profile is refused at spawn because it cannot drive a pane. (2) The permission switch is the **`DSH_PERMISSION_MODE` env export, not a flag** (`read-only`/`workspace-write`/`danger-full-access`) — the harness has none, and this is the one legitimate exception to the effort-style env-var ban because it is read with `??` as a boot-time default, so it stays soft; absent = `workspace-write`, which asks, hence the only-if-sent clamp branch, clamping to `workspace-write` (never `read-only`, which would break the workspace). ⚠️ **That clamp needs a second half no other CLI needs**, because the switch is an env var and `DSH_*` is an allowlisted `envOverrides` prefix: `applyEnvOverrides()` runs AFTER `_configureDeepSeek()` in tmux-manager, so a non-granted owner sending `DSH_PERMISSION_MODE` on the SAME request would land last and hand back exactly the privilege the config clamp removed. `clampEnvOverridesForOwner()` (session-routes.ts) DROPS `DSH_PERMISSION_MODE`, `DSH_HOME` and `DEEPSEEK_BASE_URL` for a non-granted owner (the last because `_configureDeepSeek()` forwards the SERVER's own `DEEPSEEK_API_KEY` into the pane, so a redirected base URL would send it to a foreign host) (dropping falls through to what `_configureDeepSeek()` exports, which is the clamped value); `DSH_HOME` is there because it points the launcher at a profile tree whose plugin code runs at BOOT, before any approval row applies. Every OTHER CLI's bypass is a command-line flag reachable only through its config, which is why the config clamp alone is the whole gate for them. (3) It is the **only non-claude mode that passes `hooksAvailableForMode()`**, and for it alone that predicate is a per-SESSION question rather than a per-mode one (`deepSeekConfig.statusReporting: false` disarms the bridge, so every call site passes `sessionHookOptions(session)`; answering from the mode there re-creates the infinite-wait-dressed-as-a-timeout the guard exists to prevent). It passes because the terminal front door reports idle/working/blocked to a supervisor over a generic env-gated contract and `deepseek-status-shim.ts` makes Codeman that supervisor — real `stop`/`blocked` signals, real Approvals Inbox items, plus the `agent_working` event that clears an alert answered in the terminal. ⚠️ The resolver needs the strictest identity probe of the family (`dsh --help` must say `DeepSeek Harness`) because Debian ships an unrelated `dsh` (dancer's shell) that would pass a version probe. Model is NOT a session field (it is a profile composition entry). ⚠️ `hooksAvailableForMode()` is about hook SIGNALS and is not a stand-in for "is this a claude session": Read My Mind and intent capture read Claude's own transcript and compare `mode === 'claude'` directly, because when `deepseek` earned a yes the shared predicate silently widened both to a mode with no transcript to read (pinned by a static check in `test/deepseek-mode.test.ts`). ⚠️ **It is also the only external CLI whose answers are READ FROM DISK rather than scraped off the pane**: `deepseek-transcript.ts` reads `$DSH_HOME/sessions///session.jsonl.zstd` and backs the `last-response` route for dsh, because the pane segmenter served dsh-TUI's ASCII-art SPLASH as the worker's answer (measured), which anything polling for a first answer reads as an answer. Three traps live in that file: dsh appends **one zstd FRAME per write** and Node's `zlib` zstd decoder stops at the first (a real 56-line transcript decoded as 1 line, so the module walks frame headers itself; a Node older than 22.15 has no zstd and falls back to the pane); every turn also records a **plugin-sourced `user/message`** (the runtime-context snapshot) that must not render as the user's words; and a failed `turn/end` is surfaced as `Turn error: …` rather than as an empty string that reads as "still thinking". ⚠️ Session→transcript pairing is by the header's own `cwd` plus a ±60 s boot window, never by reproducing dsh's directory mangling (which has already changed form once) — and NEVER by newest-mtime alone, which handed a fresh worker its predecessor's answer in the same case dir. DeepSeek's own tests: `test/deepseek-mode.test.ts`, `test/deepseek-cli-resolver.test.ts`, `test/deepseek-transcript.test.ts`; user guide `docs/deepseek-integration.md`. OMP (`omp`) needs no bypass flag (the CLI's own `~/.omp` config governs trust/model routing, defaulting to `tools.approvalMode: yolo`), so `buildOmpCommand()` only ever passes `--model`/`--resume`/`--continue` — but the multi-user clamp is NOT a no-op for it: `OMP_*` is an allowlisted `envOverrides` prefix, and the two credential-resolution keys it admits, `OMP_AUTH_BROKER_URL`/`OMP_AUTH_BROKER_TOKEN`, are clamped in `clampEnvOverridesForOwner()` for a non-granted owner, the same shape as `DEEPSEEK_BASE_URL`. Separately, `PI_*` is already allowlisted (pi needs it) and omp reads several of its knobs too (`PI_CONFIG_DIR`, `PI_CODING_AGENT_DIR`, `PI_CODING_AGENT_SESSION_DIR`, `PI_SUBPROCESS_CMD`, `PI_SHELL_PREFIX`) — a redirected `PI_CONFIG_DIR` moves the `~/.omp` tree `omp-session-resolver.ts`/`omp-transcript.ts` hardcode, silently breaking pinning/history; this is a known gap shared with pi, not fixed here. → [architecture-invariants#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp](docs/architecture-invariants.md#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp) @@ -246,7 +246,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Terminal scrollback strip + wheel/touch forwarding** (#205): codex/claude/gemini get the FULL strip (alt-screen, `3J`, mouse DECSETs); tmux-backed shell/opencode/antigravity/omp get a NARROW strip (alt-screen toggles only — it removes tmux's own attach-time `smcup`, which otherwise parks xterm in the scrollback-less alt buffer and turns the wheel into arrow keys). ⚠️ Gated on `useMux`: direct-PTY fallback sessions must keep the alt screen for vim/less/htop. Wheel AND touch forward to the CLI transcript for **claude ≥ 2.1.187 ONLY** at ANY scroll position (snap-to-bottom first); Shift+wheel and the `terminalWheelLocalScrollback` setting stay local. ⚠️ Codex was in that list and must never go back without a fresh measurement: codex-cli 0.147.0 ignores SGR wheel reports entirely (`mouse_any_flag=0`, inline viewport, transcript pushed into terminal scrollback), so forwarding produced a dead wheel (#227 follow-up). `_wheelScrollLines()` reads `ev.deltaMode` (Firefox = LINE units). ⚠️ When that gate is FALSE on a claude session whose local buffer is hollow (`baseY === 0`), the gesture becomes coalesced PageUp/PageDown key sends (`_maybePageCliTranscript`) instead of a no-op; ⚠️ and `getClaudeCliVersion()` must never cache a FAILED probe (one timeout used to disable forwarding process-wide until restart). ⚠️ **A click is hand-reported to the CLI only while the CLI actually has mouse tracking on.** The full strip removes the mouse DECSETs, so xterm's `mouseTrackingMode` is permanently `none` there and the browser hand-encodes SGR reports (`_sendSyntheticSgrTap`); without state it did that on EVERY click, so a stripped-mode pane running a plain shell (CLI exited, or a shell started inside a claude-mode session) received reports it never asked for and printed them as literal text (`[<0;88;20M`), garbling the next typed line. `_recordStrippedMouseMode()` (session.ts) records what the strip removes, `toState()` publishes `cliMouseTracking`, and `_shouldReportMouseToCli()` gates all three report sites on it. Only 1000/1001/1002/1003 count (1005/1006 are encodings, 1007 is alt-scroll), and the change broadcasts UNdebounced since a dialog can be clicked inside the 500ms window. `_logScrollRouting()` prints the routing decision and its inputs once per session — read it before diagnosing a scroll report. → [architecture-invariants#terminal-scrollback-strip-flavors-and-wheeltouch-forwarding](docs/architecture-invariants.md#terminal-scrollback-strip-flavors-and-wheeltouch-forwarding) **Detached start + service install** (issue #231): `codeman web -d` relaunches the SAME entry script with `detached:true` (setsid), so there is no controlling terminal and no shell job entry. ⚠️ `nohup` is NOT what makes this work: Node re-arms SIGHUP to its default disposition even when it inherits "ignore", and `cli.ts` handles SIGHUP with a graceful shutdown, so a delivered HUP still stops the server. ⚠️ Both `-d` and `service install` must REFUSE when a server is already up on this data dir (pidfile check + `/api/status` probe): a second instance on the shared tmux socket attaches PTYs to the first one's live sessions. ⚠️ Neither may report success it has not observed — the parent polls `/api/status` until the child answers or dies, since `launchctl load` and a clean spawn are both silent about a server that starts and immediately exits. `--stop` verifies the pid still LOOKS like a Codeman server (`ps -o command=`) before signalling, because pids get recycled. Unit/label names live in `config/service-names.ts` so install.sh, `detectSupervisor()` and `service install` cannot drift into supervising two copies; they are instance-scoped, and identical to the historical names for the default instance. `service install` bakes the installing shell's PATH into the unit (launchd gives a job `/usr/bin:/bin:/usr/sbin:/sbin`, which finds neither a Homebrew/nvm `node` nor `tmux`/`claude`) and never writes `CODEMAN_PASSWORD` into it. → [architecture-invariants#detached-start-and-service-install](docs/architecture-invariants.md#detached-start-and-service-install) -**Self-update** (App Settings → System → Updates): in-app updater for git-clone installs supervised by systemd/launchd (`systemd`, `launchd`, `launchd-daemon`, else `none` → "restart manually"). The update restarts the very process running it, so the real work runs in a DETACHED `scripts/self-update.sh` that outlives the restart and writes progress to `update-status.json`, which the browser polls across the connection drop. `src/web/self-update.ts` splits pure helpers (unit-tested) from IO wrappers. npm installs report as non-updatable. → [architecture-invariants#self-update](docs/architecture-invariants.md#self-update) +**Self-update** (App Settings → System → Updates): in-app updater for git-clone installs supervised by systemd/launchd (`systemd`, `launchd`, `launchd-daemon`, `docker-compose`, else `none` → "restart manually"). The update restarts the very process running it, so the real work runs in a DETACHED `scripts/self-update.sh` that outlives the restart and writes progress to `update-status.json`, which the browser polls across the connection drop. `src/web/self-update.ts` splits pure helpers (unit-tested) from IO wrappers. npm installs report as non-updatable. ⚠️ **The Compose deployment is the one supervisor that does NOT outlive the restart**: there the restart IS the container exiting (`restart: unless-stopped` relaunches it), which kills the script too — safe only because the terminal `restarting` marker is written BEFORE the kill, so nothing may be appended after it. Two config facts make it work at all and both are load-bearing: the repo is a HOST BIND MOUNT over `/opt/codeman` (a pull into the baked image copy would land in the writable layer and be silently discarded by the next `up`), and the runtime image keeps devDependencies + a build toolchain (`npm run build` is tsc+esbuild, and node-pty has no Linux prebuild), which is why `npm prune --omit=dev` is gone and the updater passes `--include=dev` against `NODE_ENV=production`. ⚠️ An in-place container update applies CODE ONLY — a restart reuses the existing image and config — so `evaluateEnvironmentGate()` REFUSES a release that changes `server.Dockerfile`/`docker-compose.yaml` (sha256 vs the baseline `Start-Codeman.sh` writes to `docker-env-applied.json` on every start) or adds `.env.example` keys the user's `.env` lacks, and refuses when the restart policy would not bring the container back. That third check exists because **Compose resolves an unset `${VAR}` to the EMPTY STRING and starts anyway**, so a new required setting otherwise arrives as a silently blank env var. Every unknown fails OPEN (no baseline, unreadable `.env`, no socket): failing closed would permanently block containers created before the fingerprint file existed. The gate is re-evaluated on `POST /api/system/update`, so hiding the button is UX, not the control. ⚠️ The four global agent CLIs in `server.Dockerfile` are PINNED on purpose — unpinned, a user's CLI versions are a function of when their image was built rather than of any commit, which is the one environment change no diff-derived gate can see; pinning turns it into a Dockerfile change the gate already catches. `test/docker-compose-env-parity.test.ts` is the merge-side guard (every compose `${VAR}` ↔ an `.env.example` entry). → [docs/docker-self-update.md](docs/docker-self-update.md), [architecture-invariants#self-update](docs/architecture-invariants.md#self-update) **Attachments** (live external document references; all wiring in `file-routes.ts`): a **registry** maps a stable `attachmentId` to a realpath-resolved, extension-allowlisted absolute path, so browser requests never carry arbitrary absolute paths. ⚠️ The **magic-link scanner** (`codeman://attach?...` in terminal output) is **prompt-injectable**, so its scan path is force-confined to the session workspace; a hostile prompt could otherwise exfiltrate arbitrary host files over SSE. The security gate is an extension **allowlist**, not a blocklist. `document-conversion-limiter.ts` caps converter spawns globally: without it, N large docs detected at once fork N multi-minute processes, which is a resource-exhaustion vector. → [architecture-invariants#attachments](docs/architecture-invariants.md#attachments) @@ -374,7 +374,7 @@ Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. L ## State Files -All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). +All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). **Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored. diff --git a/docker/.env.example b/docker/.env.example index 55000fc6..70b128f7 100644 --- a/docker/.env.example +++ b/docker/.env.example @@ -20,6 +20,13 @@ CODEMAN_RUNTIME_USER=opencode # directory in the container. CODEMAN_APPDATA_PATH=/mnt/user/appdata/Coding/codeman +# Optional. Absolute host path of this Codeman checkout, mounted at +# /opt/codeman so App Settings -> Updates can update Codeman in place. The Bash +# start script detects it from the compose file's own location, so it only needs +# setting for direct `docker compose` use or a checkout kept elsewhere. Point it +# at a directory that is not a git checkout and in-app updates are unavailable. +# CODEMAN_REPO_PATH=/mnt/user/appdata/Coding/codeman/app + # Required for Docker cases. This must be an absolute path on the Docker host. # Codeman and each isolated case use this same path, so it cannot be a # container-only path such as /home/opencode/codeman-cases. diff --git a/docker/README.md b/docker/README.md index 190896ab..e03115e8 100644 --- a/docker/README.md +++ b/docker/README.md @@ -26,6 +26,18 @@ Codeman, Claude, OpenCode, and other local sessions run as the unprivileged acco To retain Docker-case support without root when running Compose directly, set `DOCKER_SOCKET_GID` to the numeric group ID of the host socket. On a standard Linux Docker host, obtain it with `stat -c '%g' /var/run/docker.sock`. The Bash start script detects it automatically. +## Updating + +Use **App Settings → Updates** in the web UI. The checkout Compose builds from is +also mounted at `/opt/codeman`, so an update's `git checkout` and rebuild persist +on the host, and the server exiting is what restarts the container onto the new +build. + +Releases that change `server.Dockerfile`, `docker-compose.yaml`, or add a key to +`.env.example` cannot be applied that way — the updater detects them, names what +changed, and asks you to run `Start-Codeman.sh` here on the host instead. Details: +[`../docs/docker-self-update.md`](../docs/docker-self-update.md). + ## Application data storage The default configuration uses a host-folder bind mount: diff --git a/docker/Start-Codeman.sh b/docker/Start-Codeman.sh index cb7a677e..3055ecfc 100644 --- a/docker/Start-Codeman.sh +++ b/docker/Start-Codeman.sh @@ -70,4 +70,54 @@ fi export DOCKER_SOCKET_GID=${socket_ids##*:} +repo_path=${CODEMAN_REPO_PATH:-$(cd -- "$script_dir/.." && pwd)} +if [[ ! -d "$repo_path" ]]; then + printf 'Error: CODEMAN_REPO_PATH is not a directory: %s\n' "$repo_path" >&2 + exit 1 +fi +export CODEMAN_REPO_PATH="$repo_path" + +# The in-app updater runs `git checkout` and `npm install` against this checkout +# as PUID:PGID. If the directory belongs to someone else, git refuses outright +# ("detected dubious ownership") and the update fails at the first step — so warn +# here, where the fix is obvious, rather than in a failed update hours later. +if repo_owner=$(stat -c '%u' -- "$repo_path" 2>/dev/null || stat -f '%u' "$repo_path" 2>/dev/null); then + if [[ "$repo_owner" != "$PUID" ]]; then + printf 'Warning: %s is owned by UID %s but Codeman runs as UID %s.\n' "$repo_path" "$repo_owner" "$PUID" >&2 + printf 'In-app updates will fail until the ownership matches. Codeman itself still starts.\n' >&2 + fi +fi + +if [[ ! -d "$repo_path/.git" ]]; then + printf 'Note: %s is not a git checkout, so in-app updates are unavailable.\n' "$repo_path" >&2 +fi + +# Record what the container is about to be built and created FROM. The in-app +# updater compares these against the release it wants to apply: a release that +# changes either file cannot be applied by the container restarting itself (a +# restart reuses the existing image and config), so it is refused and the user +# is sent back here. Written on every start, so the baseline always describes +# the container that is actually running. See docs/docker-self-update.md. +if command -v sha256sum >/dev/null 2>&1; then + sha256_of() { sha256sum -- "$1" | cut -d' ' -f1; } +elif command -v shasum >/dev/null 2>&1; then + sha256_of() { shasum -a 256 -- "$1" | cut -d' ' -f1; } +else + sha256_of() { printf ''; } +fi + +dockerfile_sha=$(sha256_of "$script_dir/server.Dockerfile") +compose_sha=$(sha256_of "$compose_file") +if [[ -n "$dockerfile_sha" && -n "$compose_sha" ]]; then + # $CODEMAN_APPDATA_PATH is mounted at the runtime account's home, so this is + # dataPath('docker-env-applied.json') as the server inside the container sees it. + state_dir="$appdata_path/.codeman" + mkdir -p -- "$state_dir" + printf '{\n "dockerfileSha256": "%s",\n "composeSha256": "%s"\n}\n' \ + "$dockerfile_sha" "$compose_sha" >"$state_dir/docker-env-applied.json.tmp" + mv -- "$state_dir/docker-env-applied.json.tmp" "$state_dir/docker-env-applied.json" +else + printf 'Warning: no sha256 tool found; in-app updates will not detect environment changes.\n' >&2 +fi + exec docker compose --env-file "$env_file" -f "$compose_file" up --build -d diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml index 2548ccb5..9103ad46 100644 --- a/docker/docker-compose.yaml +++ b/docker/docker-compose.yaml @@ -15,6 +15,11 @@ services: ports: - "${CODEMAN_PORT}:${CODEMAN_PORT}" environment: + # Tells the self-updater to restart by exiting (the restart policy below + # relaunches it) rather than by looking for an init system that is not + # here. Also set in the image; repeated so a container started without the + # image default still self-identifies. + CODEMAN_IN_CONTAINER: "1" CODEMAN_DOCKER_BRIDGE_HOOKS: ${CODEMAN_DOCKER_BRIDGE_HOOKS} # Host-side equivalent of the runtime user's HOME. Docker case seed, # credential and hook mounts are translated into the daemon namespace. @@ -48,6 +53,32 @@ services: - type: bind source: ${DOCKER_SOCKET} target: /var/run/docker.sock + # The application source, so App Settings -> Updates can update in place. + # This is the SAME checkout used as the build context above, mounted over + # the image's baked copy: a `git checkout` performed inside the container + # then lands on the host and survives the container being recreated. + # Without it the pull would go to the container's writable layer and be + # silently discarded by the next `up`. See docs/docker-self-update.md. + # Defaults to `..` — the build context above — which Compose resolves + # against the project directory, so plain `docker compose up` works with + # no extra configuration. Set CODEMAN_REPO_PATH only to point elsewhere. + - type: bind + source: ${CODEMAN_REPO_PATH:-..} + target: /opt/codeman + # Build artefacts live in named volumes layered OVER the repo bind mount, + # so `npm install` and `npm run build` inside the container never write + # into the host checkout. That keeps container-compiled native modules + # (node-pty is built from source here) out of a checkout that may also be + # used to run Codeman natively, and keeps `git status` clean. Docker seeds + # an EMPTY named volume from the image, so the first start inherits the + # image's already-built node_modules and dist rather than paying for a + # bootstrap build. + - type: volume + source: codeman-node-modules + target: /opt/codeman/node_modules + - type: volume + source: codeman-dist + target: /opt/codeman/dist extra_hosts: - "host.docker.internal:host-gateway" security_opt: @@ -63,3 +94,12 @@ services: timeout: 5s retries: 3 start_period: 30s + +volumes: + # Container-owned build artefacts. They persist across container recreation, + # so an in-app update's `npm install` output is not thrown away by the next + # `up`, and they are seeded from the image on first use. Removing them (or + # `docker compose down -v`) is the supported reset: the next start rebuilds + # from the image. + codeman-node-modules: + codeman-dist: diff --git a/docker/server.Dockerfile b/docker/server.Dockerfile index d346866d..ecb00d14 100644 --- a/docker/server.Dockerfile +++ b/docker/server.Dockerfile @@ -12,9 +12,12 @@ WORKDIR /opt/codeman COPY . . +# devDependencies are deliberately KEPT (no `npm prune --omit=dev`). The in-app +# updater rebuilds from inside this container, and `npm run build` is tsc + +# esbuild — both devDependencies. Pruning them saves image size and takes the +# self-updater with it. See docs/docker-self-update.md. RUN npm ci \ && npm run build \ - && npm prune --omit=dev --ignore-scripts \ && npm cache clean --force # The Docker CLI talks to the host daemon through the socket mounted by @@ -25,13 +28,21 @@ ARG CODEMAN_RUNTIME_USER=opencode ARG PUID=1000 ARG PGID=1000 +# python3/make/g++ are here for the SELF-UPDATER, not for this build. An update +# runs `npm install` inside the running container, and node-pty ships no Linux +# prebuild, so a release that bumps it compiles from source right here. Without +# a toolchain that install fails and the update rolls back — every time, on the +# releases that need it most. Same reason install.sh installs one on bare hosts. RUN apt-get update \ && apt-get install -y --no-install-recommends \ ca-certificates \ curl \ + g++ \ git \ + make \ openssh-client \ procps \ + python3 \ ripgrep \ tmux \ && rm -rf /var/lib/apt/lists/* @@ -59,11 +70,23 @@ COPY --from=docker:29-cli \ # Keep credentials out of the image. Users authenticate these CLIs at runtime # through Codeman sessions, and the configured host bind mount retains state. +# +# ⚠️ PINNED ON PURPOSE. Unpinned, the agent CLI versions a user ends up with are +# a function of WHEN their image was built, not of any commit — so a Codeman +# release that depends on newer CLI behaviour (the trust-dialog handling is +# pinned to Claude Code 2.1.252's layout; wheel forwarding to >= 2.1.187) breaks +# on an older image with no diff anywhere to explain why. In-app updates make +# rebuilds RARER, which makes that drift worse. Pinning turns "this release needs +# a newer CLI" into a Dockerfile change, which the updater's environment gate +# already detects and refuses (docs/docker-self-update.md). +# +# Bump these deliberately, in a release. `--no-cache` is still needed to rebuild +# this layer when only the pins change upstream. RUN npm install --global \ - @anthropic-ai/claude-code \ - @google/gemini-cli \ - @openai/codex \ - opencode-ai \ + @anthropic-ai/claude-code@2.1.258 \ + @google/gemini-cli@0.58.0 \ + @openai/codex@0.152.1 \ + opencode-ai@1.18.26 \ && npm cache clean --force # Keep the web server and every local Codeman session unprivileged. PUID and @@ -103,7 +126,12 @@ WORKDIR /opt/codeman COPY --from=build /opt/codeman /opt/codeman -ENV CODEMAN_PORT=3000 \ +# CODEMAN_IN_CONTAINER tells the self-updater it must restart by exiting rather +# than by asking an init system that is not here (src/web/self-update.ts). +# NODE_ENV stays `production`; the updater passes `npm install --include=dev` +# explicitly, since that value would otherwise omit the build toolchain. +ENV CODEMAN_IN_CONTAINER=1 \ + CODEMAN_PORT=3000 \ HOME=/home/${CODEMAN_RUNTIME_USER} \ NODE_ENV=production diff --git a/docs/docker-compose.md b/docs/docker-compose.md index a86bdb00..f2add195 100644 --- a/docs/docker-compose.md +++ b/docs/docker-compose.md @@ -61,6 +61,16 @@ If `docker info` reports `SwapLimit=false`, set `CODEMAN_DOCKER_DISABLE_SWAP_LIM If that directory was created by an earlier root-running image, change its ownership to the configured `PUID:PGID` before starting this version. This preserves existing CLI credentials and session state while allowing the unprivileged runtime account to use them. +## Updating + +Codeman updates itself from **App Settings → Updates**, as it does on a bare host. The checkout mounted at `/opt/codeman` is the same directory Compose builds from, so the update's `git checkout` and rebuild land on the host and survive container recreation; the restart is the server exiting, which `restart: unless-stopped` turns into a relaunch on the new build. + +That applies application code only. A release that changes `docker/server.Dockerfile`, `docker/docker-compose.yaml`, or adds a key to `docker/.env.example` needs the image rebuilt or the container recreated, which a container cannot do to itself. The updater detects each case and refuses with a message naming what changed; run `docker/Start-Codeman.sh` on the host to apply those. + +`CODEMAN_REPO_PATH` overrides which checkout is mounted. It defaults to the compose project's parent directory, so it normally needs no setting. Point it at a directory that is not a git checkout and in-app updates are reported as unavailable. + +Full detail, including the fingerprint baseline and the troubleshooting table: [`docker-self-update.md`](docker-self-update.md). + ## Docker cases The default socket path is `/var/run/docker.sock`, which works with a standard Linux Docker Engine. The Bash start script detects its numeric group ID. When running Compose directly, set `DOCKER_SOCKET_GID`, for example using `stat -c '%g' /var/run/docker.sock`, so the unprivileged `CODEMAN_RUNTIME_USER` account can create Docker cases. Docker Desktop users should set `DOCKER_SOCKET` in `docker/.env` only when their Docker installation exposes a different compatible socket path. diff --git a/docs/docker-self-update.md b/docs/docker-self-update.md new file mode 100644 index 00000000..b8a4af1c --- /dev/null +++ b/docs/docker-self-update.md @@ -0,0 +1,205 @@ +# Self-update in the Docker Compose deployment + +Codeman running as a container updates itself from **App Settings → Updates**, the +same place and the same button as a bare-host install. This document explains how +that works, what it deliberately refuses to do, and how to recover when it stops. + +The bare-host updater is documented in +[`architecture-invariants.md#self-update`](architecture-invariants.md#self-update); +this file covers only what the container changes. + +## The short version + +| Change in the release | Applied by | +| -------------------------------- | ------------------------------------------------ | +| Application code | The in-app updater | +| `docker/server.Dockerfile` | `docker/Start-Codeman.sh` on the host | +| `docker/docker-compose.yaml` | `docker/Start-Codeman.sh` on the host | +| New key in `docker/.env.example` | Add it to `docker/.env`, then `Start-Codeman.sh` | + +The in-app updater detects all three of the bottom rows itself and refuses with a +message naming what changed, so you never have to work out which case you are in. + +## Why the container needs its own path + +The bare-host updater does `git checkout && npm install && npm run build`, +then asks systemd or launchd to restart the service. Two of those assumptions are +false in a container: + +1. **There is no init system.** A container's supervisor is the Docker daemon, + which acts on the container, not on processes inside it. +2. **The image is immutable.** A `git pull` into the image's baked `/opt/codeman` + would land in the container's writable layer, survive `docker restart`, and be + silently discarded by the next `docker compose up`. + +Both are solved by configuration rather than by a second updater: + +- **The checkout is a host bind mount.** `docker-compose.yaml` mounts the repo + (the same directory used as the build context) over `/opt/codeman`, so the + updater's `git checkout` writes to the host filesystem and survives the + container being recreated. +- **The restart is the server exiting.** `restart: unless-stopped` relaunches the + container whenever its main process ends, including on a clean exit — so the + updater's final step is to signal the server, and Docker starts it again on the + freshly built `dist/`. + +Everything else — the release-tag channel, the auto-stash, the atomic +`update-status.json` the browser polls across the connection drop, the boot-time +reconcile that flips `restarting` to `completed` — is the existing machinery, +unchanged. The container path is a new `SupervisorKind`, not a new updater. + +## What the pieces are + +| Piece | Role | +| ---------------------------------------------- | ------------------------------------------------------------------- | +| Repo bind mount at `/opt/codeman` | Makes the pull persistent. Without it, self-update is unavailable. | +| `codeman-node-modules`, `codeman-dist` volumes | Container-owned build artefacts, layered over the bind mount. | +| `CODEMAN_IN_CONTAINER=1` | Tells `detectSupervisor()` to restart by exiting. | +| `restart: unless-stopped` | Turns that exit into a restart. Verified before every update. | +| Toolchain + devDependencies in the image | Lets `npm install` and `npm run build` run inside the container. | +| `docker-env-applied.json` | Fingerprint baseline, written by `Start-Codeman.sh` on every start. | + +### Why build artefacts are in named volumes + +`node_modules` and `dist` are mounted as named volumes **on top of** the repo bind +mount. Without that, an update's `npm install` would write into the host checkout, +leaving container-compiled native modules (node-pty builds from source here) in a +directory that may also be used to run Codeman natively, and leaving `git status` +permanently noisy. + +Docker seeds an empty named volume from the image, so the first start inherits the +image's already-built `node_modules` and `dist` and pays no bootstrap cost. +`docker compose down -v` is the supported reset: the next start re-seeds them. + +### Why the runtime image carries a build toolchain + +`npm run build` is `tsc` plus `esbuild`, both devDependencies, so the image no +longer runs `npm prune --omit=dev`. And `npm install` may rebuild node-pty, which +ships no Linux prebuild, so `python3`, `make` and `g++` are installed as well. + +This is the real cost of in-place updates: a noticeably larger image than a +runtime-only one. It buys an update that takes about a minute instead of a full +image rebuild, and it is why `NODE_ENV=production` is paired with an explicit +`npm install --include=dev` in the updater. + +## The environment gate + +An in-place update applies **code only**. A restarted container reuses its existing +image and configuration, so a release that changes the environment cannot take +effect that way — and would half-apply: new code against an old environment. The +updater therefore checks the **target release's own files**, read straight out of +git with `git show :` before anything is checked out. + +### 1. `server.Dockerfile` changed, so the image must be rebuilt + +Compared by sha256 against the fingerprint `Start-Codeman.sh` recorded when the +running container was built. + +### 2. `docker-compose.yaml` changed, so the container must be recreated + +Same mechanism. A restart cannot pick up a new mount, port or environment +variable; only recreating the container can. + +### 3. `.env.example` gained keys your `.env` has no value for + +The check that matters most, because **Compose will not tell you**. An unset +`${VAR}` interpolates to the empty string; Compose prints a warning to a terminal +nobody is watching and starts anyway. A new required setting therefore arrives as +a silently blank environment variable and misbehaves later, far from the cause. +The updater names the missing keys instead. + +Commented-out lines in `.env.example` are deliberately *not* keys — that is how +the file marks optional overrides such as `# PUID=1000`, and counting them would +block updates on settings you are meant to leave alone. + +### 4. A restart policy that would not bring the container back + +Before signalling the server, the updater asks the Docker daemon for its own +container's restart policy. If it is `no`, the update is refused: applying it +would take Codeman down and leave no UI to recover from. + +### What the gate deliberately does not do + +Every unknown fails **open**: + +- A missing fingerprint baseline (a container started before this feature existed) + is not treated as a change, or those installs could never update at all. +- An unreadable `.env`, an unreachable Docker socket, or a target tag whose files + cannot be read all yield "no blocker" rather than a refusal. + +The gate catches a specific, detectable class of mistake; it is not a last line of +defence. It is also re-evaluated server-side on `POST /api/system/update`, so +hiding the button in the UI is a courtesy rather than the control. + +## The one residual risk + +The gate is derived from the diff, so it cannot see a release that needs a newer +environment **without changing any of those files** — for example, code that +depends on newer agent-CLI behaviour. + +That is why the four global CLIs in `server.Dockerfile` are **pinned**. Unpinned, +the versions a user ends up with are a function of when their image was built +rather than of any commit, and in-app updates make rebuilds rarer, which makes +that drift worse over time. Pinned, "this release needs a newer CLI" becomes a +Dockerfile change, which check 1 already detects. Bump them deliberately, as part +of a release. + +The complementary merge-side guard is `test/docker-compose-env-parity.test.ts`, +which fails CI when a variable is added to `docker-compose.yaml` without an entry +in `.env.example`, or the reverse. + +## Sequence of an in-place update + +1. **Check** — `GET /api/system/update/check` finds the latest release tag, fetches + that one ref so the gate can read the target's files, and returns any blockers. +2. **Start** — `POST /api/system/update` re-evaluates the gate, writes `queued` to + `update-status.json`, stages `self-update.sh` outside the repo and runs it. +3. **Apply** — stash if dirty, fetch the tag, check it out, `npm install + --include=dev`, `npm run build`. A failure at any step rolls back to the + previous commit, rebuilds it and reports `failed`; the server is never + restarted into a broken build. +4. **Restart** — write the terminal `restarting` marker, then signal the server. + The container exits and Docker restarts it. +5. **Reconcile** — the rebooted server compares its own version against the target + and flips the status to `completed` or `failed`. The browser, still polling, + picks that up. + +Step 4 kills the updater script along with the container — unlike the systemd +path, it does not outlive the restart. That is safe only because the terminal +marker is written first, which is why nothing may be appended after the kill. + +## Troubleshooting + +**"This install can't update itself (unknown)"** — the repo bind mount is missing, +so the container is running the baked image copy. Check `CODEMAN_REPO_PATH` and +confirm the mounted directory really contains `.git`. + +**The update fails immediately with a git ownership or permission error** — the +mounted checkout belongs to a different user than the one Codeman runs as +(`PUID`), so git refuses it as "dubious ownership". `Start-Codeman.sh` warns +about this at start; fix it by chowning the checkout to the same account that +owns `CODEMAN_APPDATA_PATH`. + +**A rebuild is reported as required every time** — the fingerprint baseline does +not match the checkout. `Start-Codeman.sh` writes it on every start, so start +through that script rather than a bare `docker compose up` after either file +changes. + +**Codeman does not come back after an update** — the build succeeded, since the +updater gates the restart on it, so read the container logs with `docker compose +logs codeman`. To roll back, check out the previous tag in the host checkout and +run `docker/Start-Codeman.sh`. + +**The update failed during `npm install`** — most likely a native rebuild with no +toolchain, meaning the image predates the toolchain being added. Rebuild once from +the host and the in-app path works from then on. + +**Resetting the build artefacts** — `docker compose down -v`, then +`Start-Codeman.sh`. This discards the named volumes and re-seeds them from a fresh +image. + +## Disabling it + +Set `CODEMAN_DISABLE_SELF_UPDATE=1` in `docker/.env` and pass it through in the +compose file's `environment:` block. The Updates panel then reports that in-app +updates are disabled, and the host-side script is the only way to update. diff --git a/scripts/self-update.sh b/scripts/self-update.sh index f78f3a5c..9ad786b8 100755 --- a/scripts/self-update.sh +++ b/scripts/self-update.sh @@ -7,16 +7,22 @@ # the repo (the server stages it at ~/.codeman/self-update-runner.sh) — `git # checkout` rewrites the in-repo copy and bash reads scripts lazily. # +# ⚠️ The `docker-compose` supervisor is the exception to "outlives": there the +# restart IS the container exiting, which kills this script too. That is safe +# because the terminal "restarting" marker is written before the kill and the +# rebooted server reconciles it — but nothing may be added after that kill. +# # Reports progress by writing ~/.codeman/update-status.json atomically; the # browser polls GET /api/system/update/status across the restart drop. The # freshly-booted server reconciles the final "restarting" → "completed"/"failed". # -# Cross-platform: restarts via systemd (Linux), launchd (macOS), or prints a -# manual command (foreground installs). Linux launches inside a transient -# systemd scope so `systemctl restart codeman-web` can't kill it mid-build. +# Cross-platform: restarts via systemd (Linux), launchd (macOS), a container exit +# under Docker Compose (the restart policy relaunches it), or prints a manual +# command (foreground installs). Linux launches inside a transient systemd scope +# so `systemctl restart codeman-web` can't kill it mid-build. # # Args (all from the server, never user input — tag is validated server-side): -# --repo --tag --supervisor +# --repo --tag --supervisor # --status-file --update-id --from-version --node # --log [--prev-sha ] [--stash] # @@ -144,7 +150,7 @@ rollback_and_fail() { echo "[self-update] $msg — rolling back to ${PREV_SHA:-}" if [[ -n "$PREV_SHA" ]]; then git checkout --force "$PREV_SHA" >/dev/null 2>&1 || true - npm install --no-fund --no-audit >/dev/null 2>&1 || true + npm install --no-fund --no-audit --include=dev >/dev/null 2>&1 || true npm run build >/dev/null 2>&1 || true fi fail "$msg — rolled back to the previous version" "$msg" @@ -176,7 +182,9 @@ write_status "checkout" "Checking out $TAG…" git -c advice.detachedHead=false checkout --force "$TAG" || rollback_and_fail "Could not check out $TAG" # 4) Install dependencies (heartbeat keeps the UI live during this slow step). -run_step "installing" "Installing dependencies" npm install --no-fund --no-audit \ +# --include=dev: tsc and esbuild are devDependencies, and the Compose image sets +# NODE_ENV=production, which would otherwise omit them and fail the build below. +run_step "installing" "Installing dependencies" npm install --no-fund --no-audit --include=dev \ || rollback_and_fail "Dependency install failed" # 5) Build (gate the restart on success — never restart into a torn dist/). @@ -200,6 +208,31 @@ case "$SUPERVISOR" in || fail "Build succeeded but launchd restart failed" "launchctl" } ;; + docker-compose) + # In the Compose deployment there is no init system to ask: the "restart" is + # the server EXITING, so the container's `restart: unless-stopped` policy + # relaunches it on the dist/ we just built. The repo and dist/ live on host + # mounts, so the new build survives the container being replaced. + # + # ⚠️ This script dies WITH the container it is restarting — it is a child of + # the server process, not a survivor like the systemd-scope path. That is + # fine, and load-bearing: the terminal "restarting" marker is already written + # above, and the freshly-booted server reconciles it. Nothing may be appended + # after the kill that the update depends on. + # + # ⚠️ The server is signalled by PID rather than `docker restart`: this + # container's own Docker CLI talks to the HOST daemon, and a self-directed + # restart there races the client's own death. Exiting is the one path that + # needs no cooperation from anything outside the container. + if [[ -n "$SERVER_PID" ]] && kill "$SERVER_PID" 2>/dev/null; then + : # container exit + restart policy take it from here + else + MANUAL_CMD="docker restart \$(hostname) # from the Docker host" + write_status "completed-needs-manual-restart" "Update staged — restart the Codeman container to apply v$TO_VERSION." + echo "[self-update] docker-compose: could not signal server pid '$SERVER_PID' — manual restart required" + exit 0 + fi + ;; launchd-daemon) # System-level KeepAlive LaunchDaemon (headless Mac): kickstarting the system # domain needs root, but we don't need it — kill the server and launchd diff --git a/src/types/update.ts b/src/types/update.ts index b8751257..696d2cdf 100644 --- a/src/types/update.ts +++ b/src/types/update.ts @@ -7,6 +7,10 @@ * (see `dataPath('update-status.json')`) that the browser polls across the * restart boundary. * + * The Docker Compose deployment updates in place too (same script, same status + * file) — see `docs/docker-self-update.md` for how the container restarts itself + * and what the environment gate refuses. + * * Backend logic: `src/web/self-update.ts`. Routes: `src/web/routes/system-routes.ts` * (`/api/system/update/check`, `POST /api/system/update`, `/api/system/update/status`). * @@ -17,11 +21,56 @@ * Which init system supervises the running server (decides how we restart it). * `launchd-daemon` = a KeepAlive system-level LaunchDaemon (headless Macs, no GUI * login): restart works by killing the server and letting launchd respawn it. + * `docker-compose` = the Compose deployment (`docker/docker-compose.yaml`): the + * "restart" is the server exiting so the container's `restart: unless-stopped` + * policy relaunches it on the freshly built `dist/`. */ -export type SupervisorKind = 'systemd' | 'launchd' | 'launchd-daemon' | 'none'; +export type SupervisorKind = 'systemd' | 'launchd' | 'launchd-daemon' | 'docker-compose' | 'none'; -/** How Codeman was installed — only `git` installs can self-update in place. */ -export type InstallKind = 'git' | 'npm' | 'unknown'; +/** + * How Codeman was installed. `git` and `docker-compose` can self-update in + * place; `docker-compose` is a git checkout bind-mounted into the container, so + * the pull/build happen on the host filesystem and survive container recreation. + */ +export type InstallKind = 'git' | 'docker-compose' | 'npm' | 'unknown'; + +/** + * Why an in-place container update is refused. Each is derived mechanically from + * the target release's own files — nothing here depends on a human remembering + * to declare something at release time. + * + * - `dockerfile-changed` / `compose-changed`: the release changes the ENVIRONMENT, + * which a self-restart cannot apply (a restart reuses the existing container's + * image and config). Needs a rebuild + recreate from the host. + * - `env-keys-missing`: the release's `docker/.env.example` gained keys the user's + * `docker/.env` has no value for. Compose interpolates an unset `${VAR}` to the + * EMPTY STRING and starts anyway, so without this check a new required setting + * arrives as a silently blank env var. + * - `no-auto-restart`: the container's restart policy would not bring it back + * after the server exits, so applying the update would take Codeman down. + */ +export type EnvironmentBlockerKind = 'dockerfile-changed' | 'compose-changed' | 'env-keys-missing' | 'no-auto-restart'; + +/** One reason an in-place container update is refused, with UI-ready text. */ +export interface EnvironmentBlocker { + kind: EnvironmentBlockerKind; + /** One-line explanation shown in App Settings → Updates. */ + message: string; + /** Optional specifics (e.g. the names of the missing env keys). */ + details?: string[]; +} + +/** + * Result of the environment gate for a candidate release. `checked: false` means + * the gate did not run (not a container install, or the target tag's files could + * not be read) — callers must not treat that as "no blockers". + */ +export interface EnvironmentGate { + checked: boolean; + blockers: EnvironmentBlocker[]; + /** The host command that resolves every blocker. */ + hostCommand: string; +} /** * Lifecycle of a single update run. `idle`/`completed`/`failed`/ @@ -96,6 +145,8 @@ export interface UpdateCheckResult { /** epoch ms of the check. */ checkedAt: number; source: 'github-api' | 'git-ls-remote' | 'none'; + /** Environment gate for THIS candidate release (container installs only). */ + environment?: EnvironmentGate; error?: string; } diff --git a/src/web/public/settings-ui.js b/src/web/public/settings-ui.js index 16838851..a346b479 100644 --- a/src/web/public/settings-ui.js +++ b/src/web/public/settings-ui.js @@ -1079,10 +1079,14 @@ Object.assign(CodemanApp.prototype, { const verEl = this.$('updateCurrentVersion'); if (verEl && data.currentVersion) verEl.textContent = `v${data.currentVersion}`; - if (data.installKind && data.installKind !== 'git') { - this._setUpdateResult( - `This install can't update itself (${escapeHtml(data.installKind)}). Update with npm i -g aicodeman@latest.` - ); + // `docker-compose` self-updates in place like `git` does — the container + // restarts itself. Anything else cannot. + if (data.installKind && data.installKind !== 'git' && data.installKind !== 'docker-compose') { + const hint = + data.supervisor === 'docker-compose' + ? 'Update from the Docker host with docker/Start-Codeman.sh.' + : 'Update with npm i -g aicodeman@latest.'; + this._setUpdateResult(`This install can't update itself (${escapeHtml(data.installKind)}). ${hint}`); return; } if (data.selfUpdateEnabled === false) { @@ -1093,6 +1097,30 @@ Object.assign(CodemanApp.prototype, { this._setUpdateResult(escapeHtml(data.error)); return; } + // A container release that changes the ENVIRONMENT (Dockerfile, compose file + // or new .env keys) cannot be applied by the container restarting itself, so + // the update button is never offered — the host command is, instead. The + // server re-checks this on POST, so hiding the button is UX, not the gate. + const blockers = data.environment?.blockers || []; + if (data.updateAvailable && blockers.length > 0) { + const reasons = blockers + .map((b) => { + const details = b.details?.length ? `
${escapeHtml(b.details.join(' '))}` : ''; + return `
  • ${escapeHtml(b.message)}${details}
    • `; + }) + .join(''); + this._setUpdateResult( + `v${escapeHtml(data.latestVersion || '')} needs a rebuild on the Docker host` + + ` (current v${escapeHtml(data.currentVersion || '')}):
      ${reasons}
    ` + + `Run ${escapeHtml(data.environment?.hostCommand || 'docker/Start-Codeman.sh')} there to apply it.` + ); + if (notes && data.notes) { + notes.style.display = 'block'; + notes.textContent = data.notes; + } + return; + } + if (data.updateAvailable && data.latestVersion) { this._setUpdateResult( `Update available: v${escapeHtml(data.latestVersion)}  (current v${escapeHtml(data.currentVersion || '')})` diff --git a/src/web/routes/system-routes.ts b/src/web/routes/system-routes.ts index fbf72878..dc709324 100644 --- a/src/web/routes/system-routes.ts +++ b/src/web/routes/system-routes.ts @@ -389,6 +389,9 @@ export function registerSystemRoutes( 'in-flight': { http: 409, api: ApiErrorCode.ALREADY_EXISTS }, 'up-to-date': { http: 409, api: ApiErrorCode.ALREADY_EXISTS }, 'not-git': { http: 400, api: ApiErrorCode.INVALID_INPUT }, + // A container release that changes the ENVIRONMENT: not a client error to + // retry, it needs a host-side rebuild (docs/docker-self-update.md). + 'env-blocked': { http: 409, api: ApiErrorCode.INVALID_INPUT }, disabled: { http: 403, api: ApiErrorCode.INVALID_INPUT }, 'bad-tag': { http: 400, api: ApiErrorCode.INVALID_INPUT }, error: { http: 500, api: ApiErrorCode.INTERNAL_ERROR }, diff --git a/src/web/self-update.ts b/src/web/self-update.ts index 40335cf5..2e821d29 100644 --- a/src/web/self-update.ts +++ b/src/web/self-update.ts @@ -16,6 +16,15 @@ * tested, and IO wrappers (`getInstallInfo`, `checkForUpdate`, `startUpdate`, * `reconcileUpdateOnBoot`) that touch git/network/fs. * + * DOCKER COMPOSE installs update in place too, through the same script and the + * same status file. The repo is a host bind mount, so the pull/build land on the + * host filesystem and survive container recreation; the "restart" is the server + * EXITING so the container's restart policy relaunches it on the new `dist/`. + * That applies CODE only — a restart reuses the existing container's image and + * config — so `evaluateEnvironmentGate()` refuses a release that changes + * `server.Dockerfile`, `docker-compose.yaml` or `.env.example`, pointing at the + * host command instead. See `docs/docker-self-update.md`. + * * Related: `src/types/update.ts`, `scripts/self-update.sh`, routes in * `src/web/routes/system-routes.ts`. * @@ -26,13 +35,15 @@ import { spawn, execFileSync } from 'node:child_process'; import { existsSync, readFileSync, writeFileSync, renameSync, copyFileSync, chmodSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; -import { homedir, tmpdir } from 'node:os'; -import { randomUUID } from 'node:crypto'; +import { homedir, hostname, tmpdir } from 'node:os'; +import { randomUUID, createHash } from 'node:crypto'; import { createRequire } from 'node:module'; import { dataPath } from '../config/instance.js'; import { LAUNCHD_LABEL, SYSTEMD_UNIT } from '../config/service-names.js'; import { EXEC_TIMEOUT_MS } from '../config/exec-timeout.js'; import type { + EnvironmentBlocker, + EnvironmentGate, InstallInfo, InstallKind, SupervisorKind, @@ -215,6 +226,120 @@ export function reconcileStatusDecision( return null; } +// ───────────────────────────────────────────────────────────────────────────── +// PURE helpers — the container environment gate +// ───────────────────────────────────────────────────────────────────────────── + +/** Host command that resolves every environment blocker. */ +export const DOCKER_HOST_UPDATE_COMMAND = 'docker/Start-Codeman.sh'; + +/** + * Parse the SET keys out of a dotenv file. Commented-out lines are deliberately + * NOT keys: `docker/.env.example` uses `# PUID=1000` to document an OPTIONAL + * override, so treating those as required would block every update on settings + * the user is meant to leave alone. + */ +export function parseEnvKeys(text: string): string[] { + const keys: string[] = []; + for (const raw of text.split(/\r?\n/)) { + const line = raw.trim(); + if (!line || line.startsWith('#')) continue; + const m = line.replace(/^export\s+/, '').match(/^([A-Za-z_][A-Za-z0-9_]*)\s*=/); + if (m && !keys.includes(m[1])) keys.push(m[1]); + } + return keys; +} + +/** + * Keys the TARGET release's `.env.example` sets that the user's `.env` does not. + * + * This is the check that makes a new required setting visible: Compose resolves + * an unset `${VAR}` to the empty string and starts anyway, so a missing key is + * otherwise silent until something misbehaves at runtime. + */ +export function diffRequiredEnvKeys(targetExample: string, userEnv: string): string[] { + const have = new Set(parseEnvKeys(userEnv)); + return parseEnvKeys(targetExample).filter((k) => !have.has(k)); +} + +/** + * True when the container's restart policy relaunches it after the server exits. + * `no` and an empty policy mean an in-place update would take Codeman DOWN + * rather than restart it, so the update is refused instead. + */ +export function isAutoRestartPolicy(name: string | null | undefined): boolean { + return name === 'always' || name === 'unless-stopped' || name === 'on-failure'; +} + +export interface EnvironmentGateInput { + /** sha256 of `docker/server.Dockerfile` the running container was built from. */ + appliedDockerfileHash: string | null; + /** sha256 of `docker/server.Dockerfile` at the target release tag. */ + targetDockerfileHash: string | null; + /** sha256 of `docker/docker-compose.yaml` the running container was created from. */ + appliedComposeHash: string | null; + /** sha256 of `docker/docker-compose.yaml` at the target release tag. */ + targetComposeHash: string | null; + /** Keys from `diffRequiredEnvKeys()`. */ + missingEnvKeys: string[]; + /** Docker restart policy name of the running container, or null if unknown. */ + restartPolicy: string | null; +} + +/** + * PURE gate decision. An in-place container update applies CODE only: the server + * exits and the container's restart policy relaunches it on the new `dist/`. A + * restart reuses the existing container's image and config, so anything that + * changes the ENVIRONMENT cannot take effect that way and is refused here with + * the host command that can apply it. + * + * ⚠️ An unknown hash (null) is NOT treated as "changed": a first update from a + * container created before the fingerprint file existed has no baseline, and + * failing closed there would block every such install from ever updating. The + * baseline is written by `Start-Codeman.sh`, so it exists from the first + * host-side start onward. An unknown restart policy is likewise not a blocker — + * the shipped Compose file sets `unless-stopped`, and the probe needs the Docker + * socket, which a user may not have mounted. + */ +export function computeEnvironmentBlockers(input: EnvironmentGateInput): EnvironmentBlocker[] { + const blockers: EnvironmentBlocker[] = []; + + if ( + input.appliedDockerfileHash && + input.targetDockerfileHash && + input.appliedDockerfileHash !== input.targetDockerfileHash + ) { + blockers.push({ + kind: 'dockerfile-changed', + message: 'This release changes docker/server.Dockerfile, so the image must be rebuilt.', + }); + } + + if (input.appliedComposeHash && input.targetComposeHash && input.appliedComposeHash !== input.targetComposeHash) { + blockers.push({ + kind: 'compose-changed', + message: 'This release changes docker/docker-compose.yaml, so the container must be recreated.', + }); + } + + if (input.missingEnvKeys.length > 0) { + blockers.push({ + kind: 'env-keys-missing', + message: `This release adds ${input.missingEnvKeys.length} setting(s) your docker/.env has no value for.`, + details: input.missingEnvKeys, + }); + } + + if (input.restartPolicy !== null && !isAutoRestartPolicy(input.restartPolicy)) { + blockers.push({ + kind: 'no-auto-restart', + message: `This container's restart policy is "${input.restartPolicy}", so it would not come back after the update.`, + }); + } + + return blockers; +} + // ───────────────────────────────────────────────────────────────────────────── // Status file IO // ───────────────────────────────────────────────────────────────────────────── @@ -273,19 +398,149 @@ export function resolveInstallDir(): string { return process.cwd(); } +/** + * True when this process runs inside a container. `/.dockerenv` is created by the + * Docker daemon itself; the env var is set by our own Compose file so the check + * also holds under runtimes that omit that file. + */ +export function isRunningInContainer(): boolean { + return process.env.CODEMAN_IN_CONTAINER === '1' || existsSync('/.dockerenv'); +} + function detectInstallKind(dir: string): InstallKind { - if (existsSync(join(dir, '.git'))) return 'git'; + // A container whose code is a bind-mounted checkout updates in place (the pull + // and build land on the host filesystem and survive container recreation). A + // container WITHOUT that mount runs a baked image copy — a pull there would go + // to the writable layer and vanish on the next `up`, so it is not updatable. + if (existsSync(join(dir, '.git'))) return isRunningInContainer() ? 'docker-compose' : 'git'; // Global npm install ships only dist/ (no src/, no .git). if (!existsSync(join(dir, 'src'))) return 'npm'; return 'unknown'; } +/** Install kinds whose update is applied in place by `scripts/self-update.sh`. */ +export function canSelfUpdateInPlace(kind: InstallKind): boolean { + return kind === 'git' || kind === 'docker-compose'; +} + +/** Path of the fingerprint baseline written by `docker/Start-Codeman.sh`. */ +const DOCKER_ENV_APPLIED_FILE = dataPath('docker-env-applied.json'); + +/** Files whose content defines the container ENVIRONMENT (vs. the app's code). */ +const DOCKERFILE_REL = 'docker/server.Dockerfile'; +const COMPOSE_REL = 'docker/docker-compose.yaml'; +const ENV_EXAMPLE_REL = 'docker/.env.example'; +const ENV_REL = 'docker/.env'; + +function sha256(text: string): string { + return createHash('sha256').update(text, 'utf-8').digest('hex'); +} + +/** Read a file at a git TAG without checking it out (`git show tag:path`). */ +function gitShowAtTag(repo: string, tag: string, relPath: string): string | null { + return tryExec('git', ['show', `${tag}:${relPath}`], repo); +} + +function readFileOrNull(path: string): string | null { + try { + return readFileSync(path, 'utf-8'); + } catch { + return null; + } +} + +/** + * The fingerprints the RUNNING container was created from, recorded on the host + * by `Start-Codeman.sh` at each build/recreate. Returns nulls when absent (a + * container started before this file existed) — `computeEnvironmentBlockers()` + * deliberately treats an unknown baseline as "not a blocker". + */ +function readAppliedEnvironmentFingerprints(): { dockerfile: string | null; compose: string | null } { + const raw = readFileOrNull(DOCKER_ENV_APPLIED_FILE); + if (!raw) return { dockerfile: null, compose: null }; + try { + const parsed = JSON.parse(raw) as { dockerfileSha256?: string; composeSha256?: string }; + return { dockerfile: parsed.dockerfileSha256 ?? null, compose: parsed.composeSha256 ?? null }; + } catch { + return { dockerfile: null, compose: null }; + } +} + +/** + * Restart policy of the container we're running in, via the mounted Docker + * socket. Returns null when the socket or CLI is unavailable — an unknown policy + * is not a blocker (see `computeEnvironmentBlockers`). + */ +function detectOwnRestartPolicy(): string | null { + // Docker sets HOSTNAME to the short container id; os.hostname() is the same + // value when the env var is absent. A custom `hostname:` in the compose file + // makes both unresolvable to the daemon, which fails open (unknown is not a + // blocker) rather than refusing an update over a cosmetic setting. + const id = process.env.HOSTNAME || hostname(); + if (!id) return null; + const out = tryExec('docker', ['inspect', '--format', '{{.HostConfig.RestartPolicy.Name}}', id]); + return out && out.length > 0 ? out : null; +} + +/** + * Evaluate the environment gate for a candidate release tag. Reads the TARGET + * tag's files straight out of git (`git show`), so nothing is checked out and the + * answer is available at CHECK time — the UI can refuse before the user commits + * to an update. + */ +export function evaluateEnvironmentGate(installDir: string, tag: string): EnvironmentGate { + // `git show :` needs the tag's objects locally, and neither the + // GitHub API nor `ls-remote` fetches anything — so a check that has never seen + // this tag would read nothing and report a falsely clean gate. Fetch the one + // ref first (cheap: it deltas against what the clone already has) and only + // then read. The updater fetches the same ref again; both are idempotent. + if (tryExec('git', ['rev-parse', '--verify', '--quiet', `${tag}^{commit}`], installDir) === null) { + tryExec( + 'git', + ['fetch', '--tags', '--force', 'origin', `refs/tags/${tag}:refs/tags/${tag}`], + installDir, + CHECK_TIMEOUT_MS + ); + } + + const targetDockerfile = gitShowAtTag(installDir, tag, DOCKERFILE_REL); + const targetCompose = gitShowAtTag(installDir, tag, COMPOSE_REL); + const targetExample = gitShowAtTag(installDir, tag, ENV_EXAMPLE_REL); + + // No environment files at the target tag at all: we cannot judge, so say so + // rather than reporting a clean gate the caller would trust. + if (targetDockerfile === null && targetCompose === null && targetExample === null) { + return { checked: false, blockers: [], hostCommand: DOCKER_HOST_UPDATE_COMMAND }; + } + + const applied = readAppliedEnvironmentFingerprints(); + const userEnv = readFileOrNull(join(installDir, ENV_REL)); + + const blockers = computeEnvironmentBlockers({ + appliedDockerfileHash: applied.dockerfile, + targetDockerfileHash: targetDockerfile === null ? null : sha256(targetDockerfile), + appliedComposeHash: applied.compose, + targetComposeHash: targetCompose === null ? null : sha256(targetCompose), + // A missing/unreadable .env cannot be diffed — report no missing keys rather + // than every key, which would block on an install using a non-standard path. + missingEnvKeys: targetExample !== null && userEnv !== null ? diffRequiredEnvKeys(targetExample, userEnv) : [], + restartPolicy: detectOwnRestartPolicy(), + }); + + return { checked: true, blockers, hostCommand: DOCKER_HOST_UPDATE_COMMAND }; +} + /** * Detect which init system supervises us. Detection happens HERE (in the running * server, which has a rich env) and the result is passed to the updater script — * the detached child must not re-probe with a stripped-down environment. */ export function detectSupervisor(): SupervisorKind { + // Checked FIRST: a container has no init system of its own, and its "restart" + // is the server exiting so the Docker restart policy relaunches it. Probing + // systemd here would find nothing and report `none`, which stages the update + // and then asks the user to restart by hand for no reason. + if (isRunningInContainer()) return 'docker-compose'; if (process.platform === 'darwin') { if (existsSync(join(homedir(), 'Library', 'LaunchAgents', `${LAUNCHD_LABEL}.plist`))) return 'launchd'; // Headless Macs (no GUI login → no gui domain) run Codeman as a system-level @@ -389,17 +644,29 @@ export async function checkForUpdate(): Promise { checkedAt, source: 'none', }; - if (info.installKind !== 'git') { - return { ...base, error: 'Not a git install — self-update is unavailable.' }; + if (!canSelfUpdateInPlace(info.installKind)) { + return { + ...base, + error: + info.installKind === 'unknown' && isRunningInContainer() + ? 'This container runs a baked image copy with no repository mounted — self-update is unavailable. See docs/docker-self-update.md.' + : 'Not a git install — self-update is unavailable.', + }; } + /** Attach the container environment gate to a finished check result. */ + const withGate = (result: UpdateCheckResult): UpdateCheckResult => { + if (info.installKind !== 'docker-compose' || !result.latestTag || !result.updateAvailable) return result; + return { ...result, environment: evaluateEnvironmentGate(info.installDir, result.latestTag) }; + }; + const remote = tryExec('git', ['remote', 'get-url', 'origin'], info.installDir); const gh = remote ? parseGitHubRepo(remote) : null; if (gh) { const rel = await fetchLatestReleaseFromGitHub(gh.owner, gh.repo); if (rel) { - return { + return withGate({ ...base, latestVersion: rel.version, latestTag: rel.tag, @@ -407,20 +674,20 @@ export async function checkForUpdate(): Promise { htmlUrl: rel.htmlUrl, updateAvailable: isNewerStableVersion(info.currentVersion, rel.version), source: 'github-api', - }; + }); } } // Fallback: enumerate remote tags directly (works for non-GitHub remotes too). const viaGit = fetchLatestTagViaGit(info.installDir); if (viaGit) { - return { + return withGate({ ...base, latestVersion: viaGit.version, latestTag: viaGit.tag, updateAvailable: isNewerStableVersion(info.currentVersion, viaGit.version), source: 'git-ls-remote', - }; + }); } return { ...base, error: 'Could not reach the update server (GitHub API + git ls-remote both failed).' }; @@ -432,7 +699,11 @@ export async function checkForUpdate(): Promise { export type StartUpdateResult = | { ok: true; updateId: string; toTag: string; toVersion: string | null } - | { ok: false; code: 'disabled' | 'not-git' | 'in-flight' | 'up-to-date' | 'bad-tag' | 'error'; message: string }; + | { + ok: false; + code: 'disabled' | 'not-git' | 'in-flight' | 'up-to-date' | 'bad-tag' | 'env-blocked' | 'error'; + message: string; + }; /** * Copy the updater script OUT of the repo before running it. The script lives in @@ -497,11 +768,13 @@ export async function startUpdate(): Promise { if (!info.selfUpdateEnabled) { return { ok: false, code: 'disabled', message: 'Self-update is disabled (CODEMAN_DISABLE_SELF_UPDATE=1).' }; } - if (info.installKind !== 'git') { + if (!canSelfUpdateInPlace(info.installKind)) { return { ok: false, code: 'not-git', - message: 'This is not a git install. Update with: npm i -g aicodeman@latest', + message: isRunningInContainer() + ? 'This container has no repository mounted. Update from the host with docker/Start-Codeman.sh.' + : 'This is not a git install. Update with: npm i -g aicodeman@latest', }; } const existing = readUpdateStatus(); @@ -517,6 +790,20 @@ export async function startUpdate(): Promise { return { ok: false, code: 'bad-tag', message: `Refusing to update to an unrecognized tag: ${check.latestTag}` }; } + // Re-evaluate rather than trusting the check the browser saw: the UI hides the + // button when the gate blocks, but the endpoint is reachable directly and the + // release could have moved between the check and the click. + if (info.installKind === 'docker-compose') { + const gate = evaluateEnvironmentGate(info.installDir, check.latestTag); + if (gate.blockers.length > 0) { + return { + ok: false, + code: 'env-blocked', + message: `${gate.blockers.map((b) => b.message).join(' ')} Run ${gate.hostCommand} on the Docker host to apply this release.`, + }; + } + } + const prevSha = tryExec('git', ['rev-parse', 'HEAD'], info.installDir); const runner = stageRunner(info.installDir); if (!runner) { diff --git a/test/docker-compose-env-parity.test.ts b/test/docker-compose-env-parity.test.ts new file mode 100644 index 00000000..be6260c4 --- /dev/null +++ b/test/docker-compose-env-parity.test.ts @@ -0,0 +1,81 @@ +/** + * @fileoverview Static parity check between docker/docker-compose.yaml and + * docker/.env.example. + * + * This is the MERGE GATE for the container environment. A feature that needs a + * new setting must add it to BOTH files; forgetting one is what produces the + * failure the in-app updater cannot defend against, because Compose resolves an + * unset `${VAR}` to the EMPTY STRING and starts anyway — the container comes up + * with a silently blank setting and misbehaves later, far from the cause. + * + * Failing here costs a line in a PR. Failing in production costs a debugging + * session on someone else's server. Related: docs/docker-self-update.md. + */ + +import { describe, it, expect } from 'vitest'; +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { parseEnvKeys } from '../src/web/self-update.js'; + +const DOCKER_DIR = join(process.cwd(), 'docker'); +const compose = readFileSync(join(DOCKER_DIR, 'docker-compose.yaml'), 'utf-8'); +const example = readFileSync(join(DOCKER_DIR, '.env.example'), 'utf-8'); + +/** + * Every `${VAR}` / `${VAR:-default}` the compose file interpolates. Compose's + * own built-ins are excluded — they are supplied by Compose, not by .env. + */ +function composeVariables(text: string): string[] { + const found = new Set(); + for (const m of text.matchAll(/\$\{([A-Z_][A-Z0-9_]*)(?::?-[^}]*)?\}/g)) found.add(m[1]); + return [...found].sort(); +} + +/** Keys .env.example mentions at all, including the commented-out optional ones. */ +function documentedKeys(text: string): Set { + const keys = new Set(parseEnvKeys(text)); + for (const m of text.matchAll(/^#\s*([A-Z_][A-Z0-9_]*)=/gm)) keys.add(m[1]); + return keys; +} + +/** + * Variables Compose or the start script provides, which therefore need no entry + * in .env.example. Keep this list SHORT and justified — every addition is a + * setting the parity check stops guarding. + */ +const PROVIDED_ELSEWHERE = new Set([ + // Derived by docker/Start-Codeman.sh from the appdata dir and socket owner. + 'PUID', + 'PGID', + 'DOCKER_SOCKET_GID', +]); + +/** + * Keys .env.example sets for an OVERRIDE documented in docker/README.md (the + * macvlan networking example), which the base compose file deliberately does not + * read. They are settings for a file that is not this one, not dead entries. + */ +const EXAMPLE_ONLY_KEYS = new Set([ + 'CODEMAN_MACVLAN_NETWORK', + 'CODEMAN_IPV4_ADDRESS', + 'CODEMAN_MAC_ADDRESS', + 'CODEMAN_MACVLAN_PARENT', + 'CODEMAN_MACVLAN_SUBNET', + 'CODEMAN_MACVLAN_GATEWAY', +]); + +describe('docker compose ↔ .env.example parity', () => { + it('every variable the compose file reads is documented in .env.example', () => { + const documented = documentedKeys(example); + const undocumented = composeVariables(compose).filter((v) => !documented.has(v) && !PROVIDED_ELSEWHERE.has(v)); + expect(undocumented, `add these to docker/.env.example: ${undocumented.join(', ')}`).toEqual([]); + }); + + it('every key .env.example SETS is actually read by the compose file', () => { + // Commented-out entries are exempt: they document optional overrides and + // example-only values (the macvlan block) that the base file never reads. + const used = new Set(composeVariables(compose)); + const unused = parseEnvKeys(example).filter((k) => !used.has(k) && !EXAMPLE_ONLY_KEYS.has(k)); + expect(unused, `these are set in .env.example but unused: ${unused.join(', ')}`).toEqual([]); + }); +}); diff --git a/test/docker-self-update.test.ts b/test/docker-self-update.test.ts new file mode 100644 index 00000000..86b35b71 --- /dev/null +++ b/test/docker-self-update.test.ts @@ -0,0 +1,148 @@ +/** + * @fileoverview Unit tests for the Docker Compose self-update path. + * + * Covers the PURE half of the container environment gate: which release changes + * can be applied by the container restarting itself, and which must go back to + * the host. The IO half (`evaluateEnvironmentGate`) shells out to git and docker + * and is exercised by hand — see docs/docker-self-update.md. + */ + +import { describe, it, expect } from 'vitest'; +import { + canSelfUpdateInPlace, + computeEnvironmentBlockers, + diffRequiredEnvKeys, + isAutoRestartPolicy, + parseEnvKeys, + type EnvironmentGateInput, +} from '../src/web/self-update.js'; + +/** A gate input where nothing has changed — each test perturbs one field. */ +const CLEAN: EnvironmentGateInput = { + appliedDockerfileHash: 'aaa', + targetDockerfileHash: 'aaa', + appliedComposeHash: 'bbb', + targetComposeHash: 'bbb', + missingEnvKeys: [], + restartPolicy: 'unless-stopped', +}; + +describe('canSelfUpdateInPlace', () => { + it('accepts git and docker-compose, rejects npm and unknown', () => { + expect(canSelfUpdateInPlace('git')).toBe(true); + expect(canSelfUpdateInPlace('docker-compose')).toBe(true); + expect(canSelfUpdateInPlace('npm')).toBe(false); + // A container with no repo mounted: a pull would land in the writable layer. + expect(canSelfUpdateInPlace('unknown')).toBe(false); + }); +}); + +describe('parseEnvKeys', () => { + it('reads set keys and ignores blanks, comments and values', () => { + expect(parseEnvKeys('A=1\n\nB=two words\n')).toEqual(['A', 'B']); + }); + + it('does NOT treat a commented-out key as set', () => { + // .env.example documents optional overrides as `# PUID=1000`. Counting those + // as required would block every update on settings the user should not set. + expect(parseEnvKeys('# PUID=1000\nCODEMAN_PORT=3000')).toEqual(['CODEMAN_PORT']); + }); + + it('handles `export` prefixes and repeated keys', () => { + expect(parseEnvKeys('export A=1\nA=2\n')).toEqual(['A']); + }); + + it('ignores lines that are not assignments', () => { + expect(parseEnvKeys('just a line\n=novalue\n1BAD=x\nOK=y')).toEqual(['OK']); + }); +}); + +describe('diffRequiredEnvKeys', () => { + it('reports keys the release added that the user has no value for', () => { + expect(diffRequiredEnvKeys('A=\nB=\nC=', 'A=1\nC=3')).toEqual(['B']); + }); + + it('ignores keys the user set that the release dropped', () => { + expect(diffRequiredEnvKeys('A=', 'A=1\nOBSOLETE=2')).toEqual([]); + }); + + it('counts a key the user set to an EMPTY value as present', () => { + // `GEMINI_API_KEY=` is a deliberate opt-out, not a missing setting. + expect(diffRequiredEnvKeys('GEMINI_API_KEY=', 'GEMINI_API_KEY=')).toEqual([]); + }); +}); + +describe('isAutoRestartPolicy', () => { + it('accepts the policies that relaunch the container after the server exits', () => { + expect(isAutoRestartPolicy('unless-stopped')).toBe(true); + expect(isAutoRestartPolicy('always')).toBe(true); + expect(isAutoRestartPolicy('on-failure')).toBe(true); + }); + + it('rejects "no" and unknown values', () => { + expect(isAutoRestartPolicy('no')).toBe(false); + expect(isAutoRestartPolicy('')).toBe(false); + expect(isAutoRestartPolicy(null)).toBe(false); + }); +}); + +describe('computeEnvironmentBlockers', () => { + it('allows a code-only release', () => { + expect(computeEnvironmentBlockers(CLEAN)).toEqual([]); + }); + + it('blocks a release that changes the Dockerfile', () => { + const blockers = computeEnvironmentBlockers({ ...CLEAN, targetDockerfileHash: 'zzz' }); + expect(blockers.map((b) => b.kind)).toEqual(['dockerfile-changed']); + }); + + it('blocks a release that changes the compose file', () => { + const blockers = computeEnvironmentBlockers({ ...CLEAN, targetComposeHash: 'zzz' }); + expect(blockers.map((b) => b.kind)).toEqual(['compose-changed']); + }); + + it('blocks and NAMES missing env keys', () => { + const blockers = computeEnvironmentBlockers({ ...CLEAN, missingEnvKeys: ['CODEMAN_NEW_THING'] }); + expect(blockers[0].kind).toBe('env-keys-missing'); + expect(blockers[0].details).toEqual(['CODEMAN_NEW_THING']); + }); + + it('blocks when the container would not come back', () => { + const blockers = computeEnvironmentBlockers({ ...CLEAN, restartPolicy: 'no' }); + expect(blockers.map((b) => b.kind)).toEqual(['no-auto-restart']); + // The message says which policy, so the fix is obvious from the UI alone. + expect(blockers[0].message).toContain('"no"'); + }); + + it('reports every blocker at once rather than stopping at the first', () => { + const blockers = computeEnvironmentBlockers({ + ...CLEAN, + targetDockerfileHash: 'zzz', + targetComposeHash: 'yyy', + missingEnvKeys: ['A'], + restartPolicy: 'no', + }); + expect(blockers.map((b) => b.kind)).toEqual([ + 'dockerfile-changed', + 'compose-changed', + 'env-keys-missing', + 'no-auto-restart', + ]); + }); + + // ⚠️ Regression guards for the fail-OPEN decisions. An unknown baseline is not + // evidence of a change, and failing closed there would permanently block every + // container created before the fingerprint file existed. + it('does not block when the applied baseline is unknown', () => { + expect(computeEnvironmentBlockers({ ...CLEAN, appliedDockerfileHash: null, appliedComposeHash: null })).toEqual([]); + }); + + it('does not block when the target files cannot be read', () => { + expect(computeEnvironmentBlockers({ ...CLEAN, targetDockerfileHash: null, targetComposeHash: null })).toEqual([]); + }); + + it('does not block when the restart policy is unknown', () => { + // The probe needs the Docker socket, which a user may not have mounted. + expect(computeEnvironmentBlockers({ ...CLEAN, restartPolicy: null })).toEqual([]); + }); +});