diff --git a/.changeset/calm-files-search.md b/.changeset/calm-files-search.md deleted file mode 100644 index 018c68fa..00000000 --- a/.changeset/calm-files-search.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"aicodeman": patch ---- - -Search the full session workspace from File Viewer while keeping results scoped to the active session and hidden-file preference. diff --git a/CHANGELOG.md b/CHANGELOG.md index e481374c..42ce0d51 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,23 @@ # aicodeman +## 1.23.1 + +### Patch Changes + +- Fix a fresh-Linux install failure, and bound the browser terminal's live write queue. + + **install.sh now installs a build toolchain.** Reported against a stock Ubuntu 24 server: node-pty publishes prebuilt binaries for darwin and win32 only, so on Linux it is always compiled from source during `npm install`. The installer set up Node, tmux and git but never a compiler, so a machine without `build-essential` died deep inside node-gyp with `not found: make` — which reads like an npm bug rather than a missing system package. `make`, a C++ compiler and `python3` are now checked up front exactly like git and tmux, installed per distro (apt / dnf / pacman / apk / zypper) behind the same consent prompt, and re-verified afterwards rather than assumed. If `npm install` fails anyway — including on `install.sh update` — it now names the missing tools and the command that installs them instead of leaving a node-gyp stack trace as the last word. + + **Bounded live xterm backpressure** (#339): live output is now one chunk in flight at a time, released by xterm's own parse callback, so xterm's private WriteBuffer can no longer hide an unbounded backlog behind the browser's 128 KiB render cap; queued, loading and incoming bytes all count against that cap. Automatic drop recovery for a shell stays on the bounded 1 MiB tail — a 100k-line shell capture is tens of MiB, and parsing it on the main thread is the freeze the cap exists to prevent — while TUI modes still recover full history behind the existing downgrade guard. Duplicate SSE terminal events are dropped before JSON parsing while WebSocket owns terminal I/O, and recovery is single-flight per active session. Follow-up hardening: the three write-queue reset paths now also release the in-flight gate, so a parse callback that never lands cannot leave live output permanently stalled. + + **File Viewer searches the workspace** (#340): the File Viewer search box now queries the server-side file search endpoint with a 250 ms debounce and strict response validation, instead of filtering only the part of the tree already loaded. Tree and search state are scoped to the active session, the hidden-file preference and independent request epochs, so a stale response cannot repaint the panel; cached-tree restoration, directory results and reset behaviour survive session switches and both panel-hide paths. + + ### Thanks + - @dignfei for #339 + - @aakhter for #340 + +- 858b15e: Search the full session workspace from File Viewer while keeping results scoped to the active session and hidden-file preference. + ## 1.23.0 ### Minor Changes diff --git a/CLAUDE.md b/CLAUDE.md index 2f4f7760..97f7dd29 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -75,7 +75,7 @@ When user says "COM": CI runs `npm run check:lockfile` on every push/PR, so lockfile drift fails the build even if the `version-packages` script is bypassed. -**Version**: 1.23.0 (must match `package.json`) +**Version**: 1.23.1 (must match `package.json`) ## Project Overview @@ -135,7 +135,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph - **`xterm-zerolag-input` is single-source** — BOTH echo addons live ONLY in `packages/xterm-zerolag-input/src/`, bundled into TWO **gitignored** vendor files: `vendor/xterm-zerolag-input.js` (buffer overlay, entry `zerolag-input-addon.ts`) and `vendor/xterm-predictive-echo.js` (codex write-through, entry `predictive-echo-addon.ts`) — dev by `scripts/postinstall.js`, prod by `scripts/build.mjs`. `app.js`/terminal-ui.js only **consume** them via `new LocalEchoOverlay(terminal)` / `new PredictiveEchoOverlay(terminal)`; there is no inline copy. So: change the package source, then rerun the bundle step (`npm install` for dev, `npm run build` for prod). **Never hand-edit `app.js` for overlay behavior, and never commit the gitignored vendor bundles.** Always test on mobile after touching it. → [architecture-invariants#xterm-zerolag-input-is-single-source](docs/architecture-invariants.md#xterm-zerolag-input-is-single-source), `docs/local-echo-overlay-plan.md` - **Default bind is loopback-only; non-loopback without a password starts but warns** — the server defaults to `--host 127.0.0.1`. Binding non-loopback (`--host`/`-H`/`CODEMAN_HOST`) without `CODEMAN_PASSWORD` starts anyway but prints a loud warning; `--allow-unauthenticated-network` / `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1` acknowledges it. ⚠️ The production systemd unit passes no `--host`, so prod binds **localhost only**: reach it via `tailscale serve`/tunnel to `127.0.0.1`. A loopback bind is reachable through a same-host tunnel but NOT by a browser hitting the box's LAN IP. `install.sh` is separate and prompts for the binding (defaulting to LAN + a password), and preserves the existing binding on re-runs. → [architecture-invariants#default-bind-and-the-non-loopback-warning-path](docs/architecture-invariants.md#default-bind-and-the-non-loopback-warning-path), `docs/security-architecture.md` - **Instance isolation / multi-instance attach danger** — the data dir (`~/.codeman`) and tmux socket (`tmux -L codeman`) are PROCESS-WIDE and shared by every Codeman on the machine, derived from `CODEMAN_INSTANCE` via `src/config/instance.ts`. ⚠️ A 2nd instance on the SAME socket **discovers and attaches PTYs to the first instance's live sessions**, resizing and mutating them. `$HOME` isolation is NOT enough because tmux is system-global. To run two instances, give each a distinct `CODEMAN_INSTANCE` (scopes dir + socket together), or set `CODEMAN_TMUX_SOCKET` + `CODEMAN_DATA_DIR` individually; `scripts/run-beta.sh` does this for a beta alongside prod. **Any new `~/.codeman/...` path MUST go through `dataPath()`**, never `join(homedir(), '.codeman', …)`, and **any new `tmux -L` caller through `resolveTmuxSocketName()`** (both in `config/instance.ts`): the TUI shells out to tmux from a second process, and a hardcoded `codeman` there would point a beta instance at prod's panes. → [architecture-invariants#instance-isolation-and-the-multi-instance-attach-danger](docs/architecture-invariants.md#instance-isolation-and-the-multi-instance-attach-danger) -- **node-pty's macOS `spawn-helper` ships without `+x`** (issues #6, #204): `node-pty@1.1.0` publishes `prebuilds/darwin-/spawn-helper` as mode 0644, and macOS launches every PTY through it, so a stock macOS install fails every session start with `Error: posix_spawnp failed.` **Linux can never reproduce it**: `spawn-helper` is an `OS=="mac"` gyp target and node-pty ships no Linux prebuild, so node-gyp always emits an executable helper there. ⚠️ Look in **`prebuilds/-/`**, not just `build/Release/`, which does not exist on macOS. Repair is a chmod, never a mandatory rebuild (that would require Xcode CLI tools and deletes `prebuilds/` before compiling): `npm run fix:node-pty` chmods every helper then proves it by really opening a PTY. `spawnPtyWithHelperRepair()` (`utils/node-pty-repair.ts`) wraps every `pty.spawn()` in `session.ts` and self-heals a broken install on the first failure. → [architecture-invariants#node-ptys-macos-spawn-helper-must-be-executable](docs/architecture-invariants.md#node-ptys-macos-spawn-helper-must-be-executable) +- **node-pty's macOS `spawn-helper` ships without `+x`** (issues #6, #204): `node-pty@1.1.0` publishes `prebuilds/darwin-/spawn-helper` as mode 0644, and macOS launches every PTY through it, so a stock macOS install fails every session start with `Error: posix_spawnp failed.` **Linux can never reproduce it**: `spawn-helper` is an `OS=="mac"` gyp target and node-pty ships no Linux prebuild, so node-gyp always emits an executable helper there. ⚠️ The flip side of that: since Linux has no prebuild, `npm install` **needs a C/C++ toolchain there** (`make`, `g++`, `python3`), so `install.sh` checks for and installs one alongside Node/tmux/git — a stock Ubuntu 24 server has none and died inside node-gyp with `not found: make`. Do not drop that step. ⚠️ Look in **`prebuilds/-/`**, not just `build/Release/`, which does not exist on macOS. Repair is a chmod, never a mandatory rebuild (that would require Xcode CLI tools and deletes `prebuilds/` before compiling): `npm run fix:node-pty` chmods every helper then proves it by really opening a PTY. `spawnPtyWithHelperRepair()` (`utils/node-pty-repair.ts`) wraps every `pty.spawn()` in `session.ts` and self-heals a broken install on the first failure. → [architecture-invariants#node-ptys-macos-spawn-helper-must-be-executable](docs/architecture-invariants.md#node-ptys-macos-spawn-helper-must-be-executable) - **Headless screenshots: `deviceScaleFactor` MUST be 1, and write unique filenames** — under DSF=2 xterm's WebGL renderer draws glyphs at ~2× nominal size while still *reporting* nominal cell dims, so only the pixels reveal it and only the terminal font looks wrong. And overwriting a fixed output path leaves OS image viewers showing the old render, which reads as "the fix didn't work"; `scripts/capture-real-overview.mjs` mints a timestamped filename per run. Seed the per-device `localStorage` keys (`codeman:skin`, `codeman-font-size`, `codeman-app-settings`) so the capture matches a real device. → [architecture-invariants#headless-screenshot-capture](docs/architecture-invariants.md#headless-screenshot-capture) **Import conventions**: Utils from `./utils`, types from `./types` (barrel), config from specific `./config/*` files. diff --git a/README.md b/README.md index 2a728ebc..30d5312d 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,7 @@ The installer asks before every system change, and re-running the same line upda curl -fsSL https://getcodeman.com/install | bash ``` -This installs Node.js and tmux if missing, clones Codeman to `~/.codeman/app`, and builds it. A few things worth knowing: +This installs Node.js, tmux and a build toolchain if missing (node-pty ships no Linux prebuilds, so it compiles from source), clones Codeman to `~/.codeman/app`, and builds it. A few things worth knowing: - **It asks first.** Every system change (package installs, AI CLI download) is prompted, and a menu at the end lets you choose: run Codeman in this terminal, install it as a background service (systemd/launchd, auto-start on boot), or don't start yet. Nothing runs in the background unless you pick it. - **Network or local-only, your choice.** The installer asks whether the dashboard should be reachable from other devices on your network (`0.0.0.0`, the default, with a strongly recommended password prompt) or from this machine only (`127.0.0.1`, safest). Skipping the password on a network bind requires an explicit confirmation and ends with a loud warning. A bare `codeman web` started by hand still defaults to loopback. diff --git a/install.sh b/install.sh index d2e58ed2..1ca50718 100755 --- a/install.sh +++ b/install.sh @@ -412,6 +412,27 @@ check_tmux() { command -v tmux &>/dev/null } +# node-pty ships prebuilt binaries for darwin and win32 ONLY, so on Linux it is +# always compiled from source during `npm install`. Without a toolchain that +# fails deep inside node-gyp with `not found: make`, which reads like an npm bug +# rather than a missing system package (issue: fresh Ubuntu 24 server install). +# So the toolchain is checked up front, exactly like git and tmux. +# +# Returns a human-readable list of what is missing, empty when all present. +missing_build_tools() { + local missing="" + command -v make &>/dev/null || missing="make" + if ! command -v c++ &>/dev/null && ! command -v g++ &>/dev/null && ! command -v clang++ &>/dev/null; then + missing="${missing:+$missing, }a C++ compiler (g++)" + fi + command -v python3 &>/dev/null || missing="${missing:+$missing, }python3" + printf '%s' "$missing" +} + +check_build_tools() { + [[ -z "$(missing_build_tools)" ]] +} + check_claude() { # Check PATH first if command -v claude &>/dev/null; then @@ -934,6 +955,50 @@ install_git_suse() { run_as_root zypper install -y git } +# Build toolchain for node-pty's source compile (see missing_build_tools). +install_buildtools_debian() { + info "Installing build tools via apt (build-essential, python3)..." + ensure_sudo + run_as_root apt-get update -qq + run_as_root apt-get install -y -qq build-essential python3 +} + +install_buildtools_fedora() { + info "Installing build tools (gcc, gcc-c++, make, python3)..." + ensure_sudo + if command -v dnf &>/dev/null; then + run_as_root dnf install -y gcc gcc-c++ make python3 + else + run_as_root yum install -y gcc gcc-c++ make python3 + fi +} + +install_buildtools_arch() { + info "Installing build tools via pacman (base-devel, python)..." + ensure_sudo + run_as_root pacman -Sy --noconfirm base-devel python +} + +install_buildtools_alpine() { + info "Installing build tools via apk (build-base, python3)..." + ensure_sudo + run_as_root apk add --no-cache build-base python3 +} + +install_buildtools_suse() { + info "Installing build tools via zypper..." + ensure_sudo + run_as_root zypper install -y gcc gcc-c++ make python3 +} + +install_buildtools_macos() { + # macOS normally never gets here: node-pty ships darwin prebuilds. Only a + # forced source build needs a compiler, and Xcode CLT is its only supplier. + info "Requesting Xcode Command Line Tools..." + xcode-select --install 2>/dev/null || true + die "Finish the Xcode Command Line Tools install in the dialog, then re-run this installer." +} + install_cloudflared_macos() { info "Installing cloudflared via Homebrew..." ensure_homebrew @@ -2061,6 +2126,29 @@ setup_tunnel_service() { # Installation Helpers # ============================================================================ +# npm install with an actionable message for the failure that actually happens +# on a fresh Linux box: no toolchain, so node-pty cannot compile. +npm_install_deps() { + if npm install --quiet --no-fund --no-audit 2>/dev/null; then + return 0 + fi + if npm install --no-fund --no-audit; then + return 0 + fi + + error "npm install failed." + if [[ "$(detect_os)" == "linux" ]] && ! check_build_tools; then + error "Missing native build tools: $(missing_build_tools)" + error "node-pty has no Linux prebuilds, so it must compile from source." + error "Install them and re-run this installer:" + error " Debian/Ubuntu: sudo apt-get install -y build-essential python3" + error " Fedora/RHEL: sudo dnf install -y gcc gcc-c++ make python3" + error " Arch: sudo pacman -S --noconfirm base-devel python" + error " Alpine: sudo apk add build-base python3" + fi + exit 1 +} + install_dependency() { local dep_name="$1" local os="$2" @@ -2174,6 +2262,31 @@ main() { fi fi + # Native build toolchain. node-pty compiles from source on Linux, so this is + # a hard requirement there, not a nicety. + if [[ "$os" == "linux" ]]; then + info "Checking build tools (node-pty compiles from source on Linux)..." + local missing_tools + missing_tools="$(missing_build_tools)" + if [[ -z "$missing_tools" ]]; then + success "Build tools are installed" + else + warn "Missing build tools: $missing_tools" + headless_guard "install build tools (system package via sudo)" + if prompt_yes_no "Install the build tools now?"; then + install_dependency "buildtools" "$os" "$distro" + hash -r 2>/dev/null || true + missing_tools="$(missing_build_tools)" + if [[ -n "$missing_tools" ]]; then + die "Build tools still missing after install: $missing_tools. Install them manually and re-run." + fi + success "Build tools installed" + else + die "A build toolchain (make, g++, python3) is required: node-pty has no Linux prebuilds and compiles from source." + fi + fi + fi + # AI CLI (Codeman drives one of: Claude Code, OpenCode, Codex, Gemini, Antigravity, Pi) local has_claude=false local has_opencode=false @@ -2341,7 +2454,7 @@ main() { # ======================================================================== info "Installing dependencies..." - npm install --quiet --no-fund --no-audit 2>/dev/null || npm install --no-fund --no-audit + npm_install_deps info "Building..." npm run build --quiet 2>/dev/null || npm run build @@ -2637,7 +2750,7 @@ update() { git fetch --quiet origin git reset --hard "origin/$BRANCH" --quiet - npm install --quiet --no-fund --no-audit 2>/dev/null || npm install --no-fund --no-audit + npm_install_deps npm run build --quiet 2>/dev/null || npm run build date -u +%Y-%m-%dT%H:%M:%SZ > "$INSTALL_DIR/.install-complete" success "Updated to $(node -e "console.log(require('./package.json').version)")" diff --git a/package-lock.json b/package-lock.json index 84fe119f..514d91dc 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "aicodeman", - "version": "1.23.0", + "version": "1.23.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "aicodeman", - "version": "1.23.0", + "version": "1.23.1", "hasInstallScript": true, "license": "MIT", "workspaces": [ diff --git a/package.json b/package.json index 25fab0e9..42bcc7a7 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "aicodeman", - "version": "1.23.0", + "version": "1.23.1", "description": "Mission control for AI coding agents - run 20 autonomous agents with real-time monitoring and session persistence", "type": "module", "main": "dist/index.js", diff --git a/src/web/public/app.js b/src/web/public/app.js index bd20bcb3..327b732f 100644 --- a/src/web/public/app.js +++ b/src/web/public/app.js @@ -3491,6 +3491,14 @@ class CodemanApp { this._clearTimer('_clientDropRecoveryTimer'); this.pendingWrites = []; this.writeFrameScheduled = false; + // Release the one-chunk-in-flight gate with the rest of the write queue. + // flushPendingWrites() early-returns while this is set, so a reset that + // cleared everything EXCEPT this flag would leave live output permanently + // stalled if xterm's parse callback never lands (disposed terminal, or a + // throw inside the async parse). A late callback is harmless: it clears an + // already-clear flag and schedules a flush. + this._terminalWriteInFlight = false; + this._terminalWriteInFlightBytes = 0; this._isLoadingBuffer = false; this._loadBufferQueue = null; this._bufferLoadOwner = null; @@ -5364,6 +5372,14 @@ class CodemanApp { this._clearTimer('syncWaitTimeout'); this.pendingWrites = []; this.writeFrameScheduled = false; + // Release the one-chunk-in-flight gate with the rest of the write queue. + // flushPendingWrites() early-returns while this is set, so a reset that + // cleared everything EXCEPT this flag would leave live output permanently + // stalled if xterm's parse callback never lands (disposed terminal, or a + // throw inside the async parse). A late callback is harmless: it clears an + // already-clear flag and schedules a flush. + this._terminalWriteInFlight = false; + this._terminalWriteInFlightBytes = 0; this._isLoadingBuffer = false; this._loadBufferQueue = null; this._bufferLoadOwner = null; @@ -5677,6 +5693,8 @@ class CodemanApp { this._clearTimer('syncWaitTimeout'); this.pendingWrites = []; this.writeFrameScheduled = false; + this._terminalWriteInFlight = false; + this._terminalWriteInFlightBytes = 0; this._isLoadingBuffer = false; this._loadBufferQueue = null; this._terminalRefreshOwner = null;