mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-09-30 20:49:41 +02:00
Compare commits
38
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
15a43894f9 | ||
|
|
e20aa1d4d8 | ||
|
|
aa28ef048c | ||
|
|
67f6ed3168 | ||
|
|
2d4616f059 | ||
|
|
35f8f9d19f | ||
|
|
c992784681 | ||
|
|
3a7be356ae | ||
|
|
a6a572e635 | ||
|
|
26416f98de | ||
|
|
084d7b7328 | ||
|
|
a4cdb352be | ||
|
|
d81454b6f9 | ||
|
|
00f1b9228a | ||
|
|
13d069e1e5 | ||
|
|
fa4c36c2a5 | ||
|
|
fe2c03b2cc | ||
|
|
4e3f7ac36b | ||
|
|
089283e0b3 | ||
|
|
3b85001fed | ||
|
|
1513067a7f | ||
|
|
623fedf5b7 | ||
|
|
1410362e5b | ||
|
|
92ae46246c | ||
|
|
6831d79127 | ||
|
|
b01ed611c4 | ||
|
|
8d094b086c | ||
|
|
ecc6f30e24 | ||
|
|
7da9fb4d53 | ||
|
|
b025047cbf | ||
|
|
f11bee72f5 | ||
|
|
78356d7fd0 | ||
|
|
0da7f652b4 | ||
|
|
6ccab925b1 | ||
|
|
4b51ba306e | ||
|
|
aaad031510 | ||
|
|
a6cf4c2b2a | ||
|
|
3a106bd048 |
@@ -1,5 +1,61 @@
|
||||
# aicodeman
|
||||
|
||||
## 1.16.5
|
||||
|
||||
### Patch Changes
|
||||
|
||||
- Mobile keyboard dismissal, and a tidier Save/Close pair in the phone settings sheet.
|
||||
|
||||
**The on-screen keyboard can finally be closed from inside the app.** The terminal
|
||||
keeps focus on a hidden textarea and nothing ever released it, so once the keyboard
|
||||
was up it covered roughly half the screen with no way out but the OS back gesture.
|
||||
Two gestures now dismiss it:
|
||||
- **A tap outside the terminal** (header, tab strip, empty page chrome). Deliberately
|
||||
narrow: it only fires while the terminal input actually holds focus, never inside
|
||||
the terminal (tap classification owns that decision), and never on a control, since
|
||||
anything focusable is about to take focus itself and the keyboard accessory bar
|
||||
exists to be used _while_ the keyboard is open. A scroll ends in `touchend` too, so
|
||||
finger travel is tracked from `touchstart` and only a near-stationary gesture counts
|
||||
as a tap, sharing the terminal's own 8px threshold so both agree on tap-vs-scroll.
|
||||
Scrolling to read something mid-compose no longer drops the composer.
|
||||
- **A second tap on inert transcript content.** Every terminal tap used to re-focus,
|
||||
which left the accessory bar's chevron as the only way out. Scoped to inert rows on
|
||||
purpose: the prompt row keeps focus-then-position, so a second tap there still
|
||||
places the caret, and actionable rows (readbacks, `esc to interrupt` status rows,
|
||||
menu selections) still blur as before.
|
||||
|
||||
**Settings sheet header on phones.** Below 860px Save moves into the header, which
|
||||
left the two ways out of the sheet as a fat accent pill beside a bare glyph. Save and
|
||||
Close now share a recessed tray with matching 36px pill geometry, reading as one
|
||||
44px cluster the height of the phone header. Tray colors come from skin tokens, so
|
||||
the light skins keep their look, and the tray stays off the sheets that carry a lone
|
||||
close button.
|
||||
|
||||
Also fixes a test that could never have caught a regression: the case asserting that
|
||||
tapping a control does _not_ dismiss the keyboard was picking a button from the
|
||||
hidden welcome overlay, whose rect still measures while the hit-test lands on the
|
||||
terminal underneath, so it passed for the wrong reason and stayed green even with the
|
||||
exemption deleted. All four guards in the dismiss handler are now individually
|
||||
pinned.
|
||||
|
||||
## 1.16.4
|
||||
|
||||
### Patch Changes
|
||||
|
||||
- **Voice dictation through your Claude Code login (no API key).** The mic button can now transcribe using this machine's existing Claude Code subscription, via the same speech-to-text service the CLI's own `/voice` mode uses. Off by default (`claudeVoiceEnabled`, synced): turning it on spends the server owner's Claude subscription on transcription for anyone who can reach the UI. The OAuth token never leaves the server process, credentials are read-only (Codeman never refreshes them, which would rotate the refresh token out from under the CLI), streams are capped at 5 minutes and 4 concurrent, and the WebSocket carries the same allowed-Host + same-site Origin guard as the terminal socket. A new Speech engine picker (Auto / Claude / Deepgram / Browser) sits alongside the existing Deepgram and Web Speech paths, which are untouched.
|
||||
|
||||
**One settings surface.** Session Options and Add Case now use the same `set-*` chrome as App Settings instead of the old modal-tab chrome, with a left rail, grouped rows, per-group device/synced scope badges and a search box. App Settings leads with version + update; the Session Options rail stays a real switcher (one section at a time) because Summary and Respawn are each long enough to bury the other. Collapsed Add Case blocks gained a disclosure chevron.
|
||||
|
||||
**Read My Mind: rethink steer note (phase 3 part 2).** Rethink now carries an optional free-text note ("no, I meant the mobile bug") sent as `steer`, the highest-authority signal the predictor gets. It stays in the field across re-runs, clears on each open, and the empty-result copy points at it. The modal footer moved to the styled `btn-toolbar` convention; the bare `btn btn-*` classes it shipped with match no CSS in this codebase and rendered as unstyled browser buttons.
|
||||
|
||||
**Mobile terminal taps no longer fight the keyboard.** Taps on TUI-owned rows (expandable readbacks, tool results, decision menus, the working/status row) now act on the CLI without popping the keyboard, while a tap on inert transcript text keeps the keyboard reachable. Rows are told apart by the affordance the CLI prints (`ctrl+r to expand`, `tap to collapse`, `esc to interrupt`) rather than by row titles, which vary per CLI and per version. A tap with the viewport scrolled up sends no mouse report at all but still restores focus, so the keyboard is reachable after every tab switch. Thanks to @Lint111.
|
||||
|
||||
**Path labels abbreviate `$HOME` on both platforms.** The "show `~/project`" rule had three implementations and two were platform-specific in opposite directions: the Run menu's matched `/home/<user>/` only, so on macOS every Recent Sessions row spent its first ~19 characters on an identical `/Users/<user>/` prefix and ellipsized away the tail that identifies it (#273); the case-manage list's matched `/Users/<user>` only, so no Linux case path was ever abbreviated. Both now route through one helper, with a static guard against a fourth copy appearing.
|
||||
|
||||
**Run menu Recent Sessions rows are legible.** Rows now read as folder, worktree pill, dimmed parent path, timestamp, with only the parent path allowed to shrink, so truncation can never hide which project (or which worktree) a row refers to. `<repo>/.claude/worktrees` is dropped from the parent path as noise. Thanks to @jordan8037310. Follow-up fix: the widened menu was not actually usable by its rows, since `.run-mode-history` is a block scroller and its `<button>` rows stayed shrink-to-fit at ~250px inside a full-window-width menu; rows now fill the menu and it is capped at the 760px one full row costs.
|
||||
|
||||
**Desktop home screen** no longer clips, and shows full tab names.
|
||||
|
||||
## 1.16.3
|
||||
|
||||
### Patch Changes
|
||||
|
||||
@@ -13,7 +13,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
| Task | Command |
|
||||
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Dev server | `npm run dev` (or `npx tsx src/index.ts web`) |
|
||||
| Type check | `tsc --noEmit` |
|
||||
| Type check | `npm run typecheck` (= `tsc --noEmit`) |
|
||||
| Lint | `npm run lint` (fix: `npm run lint:fix`) |
|
||||
| Format | `npm run format` (check: `npm run format:check`) |
|
||||
| Single test | `npm test -- test/<file>.test.ts` (or `npx vitest run --config config/vitest.config.ts test/<file>.test.ts`) — ⚠ **never** run bare `npm test`, see Testing section |
|
||||
@@ -74,7 +74,7 @@ When user says "COM":
|
||||
|
||||
CI runs `npm run check:lockfile` on every push/PR, so lockfile drift fails the build even if the `version-packages` script is bypassed.
|
||||
|
||||
**Version**: 1.16.3 (must match `package.json`)
|
||||
**Version**: 1.16.5 (must match `package.json`)
|
||||
|
||||
## Project Overview
|
||||
|
||||
@@ -159,15 +159,15 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
| **Search** | `src/search-service.ts` | Pure in-memory core for `GET /api/search` |
|
||||
| **Attachments** | `src/attachment-registry.ts`, `attachment-magic`, `generated-artifact-attachments`, `session-attachment-history`, `document-preview-cache`, `document-thumbnailer`, `document-conversion-limiter`, `config/attachment-guard` | See Key Patterns |
|
||||
| **Plan** | `src/plan-orchestrator.ts`, `src/prompts/*.ts`, `src/templates/` (`claude-md.ts` + `case-template.md`) | `templates/` holds the CLAUDE.md scaffold generated into new cases |
|
||||
| **Web** | `src/web/server.ts` ★, `sse-events.ts`, `routes/*.ts` (20 modules + barrel; `session-routes.ts` ★), `route-helpers.ts`, `ports/*.ts`, `middleware/auth.ts`, `schemas.ts`, `self-update.ts`, `plan-usage-latest.ts`, `ws-connection-registry.ts`, `heic-jpeg-converter.ts` + `heic-jpeg-worker.ts` | |
|
||||
| **Frontend** | `src/web/public/app.js` (~5K lines, core) + 28 modules + `sw.js` | See Frontend section for the load order, which is authoritative |
|
||||
| **Types** | `src/types/index.ts` (barrel) → 20 domain files; also `src/types.ts` root re-export | See `@fileoverview` in index.ts |
|
||||
| **Web** | `src/web/server.ts` ★, `sse-events.ts`, `routes/*.ts` (24 modules + barrel; `session-routes.ts` ★), `route-helpers.ts`, `ports/*.ts`, `middleware/auth.ts`, `schemas.ts`, `self-update.ts`, `plan-usage-latest.ts`, `ws-connection-registry.ts`, `heic-jpeg-converter.ts` + `heic-jpeg-worker.ts` | |
|
||||
| **Frontend** | `src/web/public/app.js` (~5K lines, core) + 29 modules + `sw.js` | See Frontend section for the load order, which is authoritative |
|
||||
| **Types** | `src/types/index.ts` (barrel) → 22 domain files; also `src/types.ts` root re-export | See `@fileoverview` in index.ts |
|
||||
|
||||
★ = Large, central file (>50KB) — read its `@fileoverview` first. All files have `@fileoverview` JSDoc — read that before diving in. Discovery aid: `grep -l '@fileoverview' src/web/routes/*.ts` lists all route modules; same grep works for `src/types/`, `src/web/public/*.js`.
|
||||
|
||||
**Local packages**: `packages/xterm-zerolag-input/` (local echo overlay, single-source, see Gotchas). `packages/gesture-control/` (`codeman-gesture-control`, hand-tracking overlay source, built via `npm run build:gesture`).
|
||||
|
||||
**Config**: `src/config/` — 17 files, no barrel (`index.ts`) exists; import from the specific file.
|
||||
**Config**: `src/config/` — 20 files, no barrel (`index.ts`) exists; import from the specific file.
|
||||
|
||||
**Utilities**: `src/utils/` — re-exported via index. Key: `CleanupManager`, `LRUMap` (⚠ NOT in the barrel — import from `./utils/lru-map.js` directly), `StaleExpirationMap`, `BufferAccumulator`, `stripAnsi`, `Debouncer`, `KeyedDebouncer`. Also: `claude-cli-resolver`/`opencode-cli-resolver`/`codex-cli-resolver`/`gemini-cli-resolver` (CLI path resolution), `string-similarity` (fuzzy matching), `regex-patterns` (ANSI/token/spinner patterns), `assertNever` (exhaustive checks), `token-validation` (auth tokens), `nice-wrapper` (process priority).
|
||||
|
||||
@@ -200,7 +200,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Docker cases**: a case can point at a **container**, with any of the five CLI backends running inside it. Like remote-SSH this is a **LOCATION OVERLAY on cases, never a sixth `SessionMode`**. Exactly one long-lived container **per case**, shared by all its sessions, so killing a session kills only that session's in-container tmux and **never** `docker stop` while siblings remain. The workspace is a real host dir bind-mounted at the **same absolute path**, which is what keeps file-routes/watchers on real host bytes and makes the in-container transcript projHash match the host. Credentials are **seeded** (RO mount, copied into the container once) rather than shared RW, so in-container CLIs never write refreshed tokens back to the host, and bind mounts are excluded from `docker commit` so exports stay secret-free. **NEVER a create-time `-e` for secrets, NEVER `--privileged`, NEVER the docker socket.** Config drift is detected via a label hash and a drifted launch is REFUSED rather than silently launched with stale config. ⚠️ On the loopback-only prod bind a container cannot reach 127.0.0.1, so in-container hooks need `CODEMAN_DOCKER_BRIDGE_HOOKS=1`; otherwise idle detection falls back to output-based. → [architecture-invariants#docker-cases](docs/architecture-invariants.md#docker-cases), `docs/docker-cases.md` (user guide), `docs/docker-cases-plan.md` (design)
|
||||
|
||||
**External CLI modes (OpenCode, Codex, Gemini, Antigravity)**: `isExternalCliMode()` in `session.ts` gates Claude-specific behavior off (Ralph tracker, BashToolParser, token/CLI-info parsing, ❯-prompt readiness); these CLIs render their own TUIs, so readiness is output stabilization instead. All four **require tmux with no direct PTY fallback**, because secrets are injected via socket-scoped `tmux setenv` and never on the spawn command line. ⚠️ `run*()` in `session-ui.js` MUST unwrap the `{success,data}` envelope; reading the raw shape silently breaks the run. ⚠️ **Codex sessions use PREDICTIVE WRITE-THROUGH echo, never the buffer overlay** (`_localEchoPolicy` in `_updateLocalEchoState`, terminal-ui.js): codex's composer reacts per keystroke ("/" pops a live-filtering picker, arrows edit server-side state, the composer grows as it wraps), so buffer-until-Enter starved it into issues #218/#219/#220/#222 and stays disabled (`_localEchoEnabled` remains false for codex). Instead, `PredictiveEchoAddon` (separate `vendor/xterm-predictive-echo.js` bundle) paints each keystroke at the predicted cell while the wire path stays BYTE-IDENTICAL: the onData hook (`_predictHookOnData`) is a plain statement with no `return`, so control always falls through into the untouched send path — pinned by vm and E2E byte-identity tests. Predictions reconcile against the parsed buffer and only while the cursor sits on the measured composer row (`isCodexComposerRow`, `/^› /`). Codex also **drops keystrokes that share a PTY read with a bracketed paste**, so flushed text and the paste sequence must go out as separate delayed writes (mirroring the Enter branch's delayed `\r`). Tests: `test/local-echo-codex-gating.test.ts`, `test/codex-predictive-echo.test.ts` (E2E vs real codex), `packages/xterm-zerolag-input/test/codex-replay.test.ts`. → [architecture-invariants#external-cli-modes-opencode-codex-gemini](docs/architecture-invariants.md#external-cli-modes-opencode-codex-gemini)
|
||||
**External CLI modes (OpenCode, Codex, Gemini, Antigravity)**: `isExternalCliMode()` in `session.ts` gates Claude-specific behavior off (Ralph tracker, BashToolParser, token/CLI-info parsing, ❯-prompt readiness); these CLIs render their own TUIs, so readiness is output stabilization instead. All four **require tmux with no direct PTY fallback**, because secrets are injected via socket-scoped `tmux setenv` and never on the spawn command line. ⚠️ `run*()` in `session-ui.js` MUST unwrap the `{success,data}` envelope; reading the raw shape silently breaks the run. ⚠️ **Codex sessions use PREDICTIVE WRITE-THROUGH echo, never the buffer overlay** (`_localEchoPolicy` in `_updateLocalEchoState`, terminal-ui.js): codex's composer reacts per keystroke ("/" pops a live-filtering picker, arrows edit server-side state, the composer grows as it wraps), so buffer-until-Enter starved it into issues #218/#219/#220/#222 and stays disabled (`_localEchoEnabled` remains false for codex). Instead, `PredictiveEchoAddon` (separate `vendor/xterm-predictive-echo.js` bundle) paints each keystroke at the predicted cell while the wire path stays BYTE-IDENTICAL: the onData hook (`_predictHookOnData`) is a plain statement with no `return`, so control always falls through into the untouched send path — pinned by vm and E2E byte-identity tests. Predictions reconcile against the parsed buffer and only while the cursor sits on the measured composer row (`isCodexComposerRow`, `/^› /`). Codex also **drops keystrokes that share a PTY read with a bracketed paste**, so flushed text and the paste sequence must go out as separate delayed writes (mirroring the Enter branch's delayed `\r`). Tests: `test/local-echo-codex-gating.test.ts`, `test/codex-predictive-echo.test.ts` (E2E vs real codex), `packages/xterm-zerolag-input/test/codex-replay.test.ts`. → [architecture-invariants#external-cli-modes-opencode-codex-gemini](docs/architecture-invariants.md#external-cli-modes-opencode-codex-gemini-antigravity)
|
||||
|
||||
**Run launch synchronization**: the Run entrypoint holds an in-flight lock and disables `#runBtn` for the whole launch (≥500ms), so a double click cannot create duplicate sessions with the same `w<n>-<case>` name. `_ensureCreatedSessionVisible()` runs before `selectSession()`, and `_onSessionCreated()` stays an idempotent upsert, so POST-first and SSE-first ordering both produce exactly one rendered tab. → [architecture-invariants#run-launch-synchronization](docs/architecture-invariants.md#run-launch-synchronization)
|
||||
|
||||
@@ -212,6 +212,8 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Read My Mind intent profiles** (phase 1 of `docs/readmymind-plan.md`; `readMyMindEnabled`, SYNCED, default OFF): per-CASE profiles (user-stated `goals` + the user's recent real prompts), keyed by owner + realpath(workingDir) so they survive `/clear`/respawns and multi-user scoping is structural. Capture rides the transcript (`transcript:user_prompt` from `transcript-watcher.ts`), NOT the input paths: `POST /input` sees only programmatic prompts and the WS channel is raw keystrokes. The listener lives inside `startTranscriptWatcher()`'s `if (!watcher)` block (outside it would duplicate per hook event) and is claude-only + gated on the setting per event. Store: `src/intent-store.ts` singleton, `intents.json` written 0600 tmp+rename (prompts can contain secrets; never fed to `/api/search`). Endpoints: GET/PUT/DELETE `/api/sessions/:id/intent` + POST `/api/sessions/:id/readmymind` (`readmymind-routes.ts`, ownership via `findSessionOrFail` WITH `req`; registrations stay the bare `app.<method>('path')` shape, the endpoints.md drift scanner cannot see generics). **Phase 2 (predictor + 🧠 button)**: `readmymind-context.ts` is the PURE budgeted assembler (9 ranked sources, drop order siblings→away→workspace→tools, sections 1-4 truncate only); IO lives in `readmymind-collectors.ts` (transcript TAIL read — the live watcher keeps only a 500-char snippet — + git signals, skipped for remote-SSH cases) and the route; `readmymind-predictor.ts` reuses the AiCheckerBase spawn mechanics standalone (verdict-shaped base vs freeform JSON) as a mutable singleton routes call and tests stub. Claude-mode only (400), one in flight per session (409 CONFLICT), model = `readMyMindModel` setting defaulting to `AI_CHECK_MODEL` (opus, decided). Frontend `readmymind-ui.js`: header 🧠 marker-hidden (`btn-readmymind--hidden`) until the setting is ON; phones hide it in mobile.css and get a keyboard-accessory 🧠 key instead (ships in BOTH bar templates, revealed by the `rmm-enabled` class on the BAR element — setMode() rebuilds button innerHTML, so per-key state would be wiped; synced at init + every `applyHeaderVisibilitySettings()`). Alternate suggestions render as tappable rows that swap into the editable field without losing edits; Rethink rejects the whole shown set and carries the optional steer note (`#readMyMindSteer`, sent as `steer`, shown in ready + empty-result phases, cleared on each open). Suggestions render via value/`textContent` ONLY and Send/Insert go through `POST /input` (server-side, so the sendEnterKey/local-echo trap does not apply) — nothing auto-sends, ever. User guide: `docs/readmymind.md`.
|
||||
|
||||
**Voice dictation via Claude** (`claudeVoiceEnabled`, SYNCED, default OFF): the mic button can transcribe through this machine's Claude Code login instead of a Deepgram key, using the same speech-to-text service the CLI's own `/voice` mode uses. ⚠️ **Claude Code's voice mode itself is unusable here**: it opens the HOST's microphone (`sox`/`arecord`), and the CLI runs in a headless tmux pane while the human is in a browser elsewhere. So Codeman captures in the browser and borrows only the backend. Audio goes browser → Codeman → Anthropic (`src/web/voice-stream.ts`): the OAuth token never reaches the page, and the browser only sends PCM and receives text. ⚠️ Credentials are **read-only** (`src/claude-credentials.ts`) and Codeman never refreshes them — a refresh rotates the refresh token and could sign the user out of their own CLI; an elapsed token reports `expired` instead. ⚠️ Capture MUST be linear16/16 kHz/mono, so it uses an **AudioWorklet**, not MediaRecorder (which cannot emit raw PCM); `voice-pcm-worklet.js` is fetched from JS, so it is invisible to `cacheBustAssets` and borrows voice-input.js's `?v=` token — **edit the two together**. ⚠️ Transcript frames carry the WHOLE running transcript, not deltas: the Claude path replaces where the Deepgram path appends. Provider choice is `voiceSettings.provider` (`auto` prefers Claude → Deepgram → Web Speech). → `docs/claude-voice-plan.md`
|
||||
|
||||
**Agent Teams**: `TeamWatcher` polls `~/.claude/teams/`, matches to sessions via `leadSessionId`. Teammates are in-process threads appearing as subagents. Enable: `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`. See `docs/agent-teams/`.
|
||||
|
||||
**Circuit breakers**: the Ralph breaker prevents respawn thrashing (`CLOSED` → `HALF_OPEN` → `OPEN`; reset via `/api/sessions/:id/ralph-circuit-breaker/reset`). **Distinct: the PTY-exit breaker** (`session-pty-exit-breaker.ts`) trips after repeated rapid PTY exits and blocks auto-restarts. ⚠️ It resets ONLY via an explicit `{clearBreaker:true}` body on `POST /api/sessions/:id/interactive`; the frontend's auto-reattach in `selectSession()` sends no body and must never clear it. → [architecture-invariants#circuit-breakers-ralph--pty-exit](docs/architecture-invariants.md#circuit-breakers-ralph-and-pty-exit)
|
||||
@@ -222,7 +224,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Detached start + service install** (issue #231): `codeman web -d` relaunches the SAME entry script with `detached:true` (setsid), so there is no controlling terminal and no shell job entry. ⚠️ `nohup` is NOT what makes this work: Node re-arms SIGHUP to its default disposition even when it inherits "ignore", and `cli.ts` handles SIGHUP with a graceful shutdown, so a delivered HUP still stops the server. ⚠️ Both `-d` and `service install` must REFUSE when a server is already up on this data dir (pidfile check + `/api/status` probe): a second instance on the shared tmux socket attaches PTYs to the first one's live sessions. ⚠️ Neither may report success it has not observed — the parent polls `/api/status` until the child answers or dies, since `launchctl load` and a clean spawn are both silent about a server that starts and immediately exits. `--stop` verifies the pid still LOOKS like a Codeman server (`ps -o command=`) before signalling, because pids get recycled. Unit/label names live in `config/service-names.ts` so install.sh, `detectSupervisor()` and `service install` cannot drift into supervising two copies; they are instance-scoped, and identical to the historical names for the default instance. `service install` bakes the installing shell's PATH into the unit (launchd gives a job `/usr/bin:/bin:/usr/sbin:/sbin`, which finds neither a Homebrew/nvm `node` nor `tmux`/`claude`) and never writes `CODEMAN_PASSWORD` into it. → [architecture-invariants#detached-start-and-service-install](docs/architecture-invariants.md#detached-start-and-service-install)
|
||||
|
||||
**Self-update** (App Settings → Updates): in-app updater for git-clone installs supervised by systemd/launchd (`systemd`, `launchd`, `launchd-daemon`, else `none` → "restart manually"). The update restarts the very process running it, so the real work runs in a DETACHED `scripts/self-update.sh` that outlives the restart and writes progress to `update-status.json`, which the browser polls across the connection drop. `src/web/self-update.ts` splits pure helpers (unit-tested) from IO wrappers. npm installs report as non-updatable. → [architecture-invariants#self-update](docs/architecture-invariants.md#self-update)
|
||||
**Self-update** (App Settings → System → Updates): in-app updater for git-clone installs supervised by systemd/launchd (`systemd`, `launchd`, `launchd-daemon`, else `none` → "restart manually"). The update restarts the very process running it, so the real work runs in a DETACHED `scripts/self-update.sh` that outlives the restart and writes progress to `update-status.json`, which the browser polls across the connection drop. `src/web/self-update.ts` splits pure helpers (unit-tested) from IO wrappers. npm installs report as non-updatable. → [architecture-invariants#self-update](docs/architecture-invariants.md#self-update)
|
||||
|
||||
**Attachments** (live external document references; all wiring in `file-routes.ts`): a **registry** maps a stable `attachmentId` to a realpath-resolved, extension-allowlisted absolute path, so browser requests never carry arbitrary absolute paths. ⚠️ The **magic-link scanner** (`codeman://attach?...` in terminal output) is **prompt-injectable**, so its scan path is force-confined to the session workspace; a hostile prompt could otherwise exfiltrate arbitrary host files over SSE. The security gate is an extension **allowlist**, not a blocklist. `document-conversion-limiter.ts` caps converter spawns globally: without it, N large docs detected at once fork N multi-minute processes, which is a resource-exhaustion vector. → [architecture-invariants#attachments](docs/architecture-invariants.md#attachments)
|
||||
|
||||
@@ -264,7 +266,7 @@ Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. L
|
||||
|
||||
**Per-device vs synced settings**: the `displayKeys` set in settings-ui.js is a **client-side merge policy**, not a wire filter. A display key seeds from the server only when localStorage has no value for it, which is what prevents one device overwriting another; `showPlanUsageLimits` is additionally `delete`d from the incoming payload outright. Separately, `SettingsUpdateSchema` is `.strict()` and simply **does not declare** `skin`, `showFileViewerButton`, `showCronButton`, `webglRendererEnabled`, `localEchoEnabled`, `cjkInputEnabled`, or `extendedKeyboardBar`, so sending one of those is a validation error. The rest (`showResponseViewer`, `showPlanUsageLimits`, `language`, and most `show*` keys) ARE in the schema and do persist server-side; they are per-device by client policy only. ⚠️ Adding a new per-device setting means deciding **both** questions: membership in `displayKeys`, and presence in the schema.
|
||||
|
||||
**App Settings modal** (`#appSettingsModal`): a left rail that is a **table of contents over ONE scrolling document**, not a tab switcher. Every section stays mounted (`.set-section`, ids `settings-terminal|appearance|layout|models|clis|notifications|voice|shortcuts|system`), the rail follows the scroll, and `switchSettingsTab(id)` keeps its historical name but SCROLLS instead of hiding. Phones swap the rail for the sticky `#appSettingsJump` pill (compact layout at ≤860px in mobile.css). ⚠️ **The load/save contract is `getElementById` by id**: `openAppSettings()`/`saveAppSettings()` read every control by a fixed id, so moving a control between sections is free but renaming or dropping one silently stops it loading or saving. `test/app-settings-structure.test.ts` is the static guard. ⚠️ Model cards (`#appSettingsModelCards`) and the effort segment are **views over hidden `<select>`s** that remain the source of truth; the cards hold the BASE model and the "1M context window" switch composes `base + [1m]` back into `claudeModel`, which is what retires the old "takes precedence over the toggle below" trap. ⚠️ `.modal-tabs`/`.modal-tab-btn`/`.modal-tab-content` are still used by `#sessionOptionsModal` and `#createCaseModal`; the settings rail uses its own `set-*` classes and must not restyle them. ⚠️ `admin-ui.js` injects the multi-user Users entry into `.set-rail-items` + `.set-doc`, so those hooks must survive any restructure.
|
||||
**Settings surface** (`#appSettingsModal` + `#sessionOptionsModal` + `#createCaseModal`): the `set-*` language (left rail, groups of rows, control pinned right) is shared by all three modals through ONE `:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal)` scope in styles.css: an `:is()` list takes its most specific argument's specificity, so every rule keeps the id weight it had and nothing downstream shifts. **App Settings** is a rail that is a **table of contents over ONE scrolling document**, not a tab switcher: every section stays mounted (`.set-section`, ids `settings-updates|terminal|layout|appearance|models|clis|notifications|voice|shortcuts|system`, in that order, the version and the updater leading and the rest of the system settings tailing), and `switchSettingsTab(id)` keeps its historical name but SCROLLS instead of hiding. **Session Options** and **Add Case** use the same surface with a rail that really SWITCHES (`switchOptionsTab` / `switchCaseModalTab` show one `.set-section` and `.hidden` the rest, since Summary owns its own scroller, Respawn is long, and Add Case is six independent forms). ⚠️ They also take a deliberate **size-up** that App Settings does not (900px shell, 236px rail, `height:auto` between `min(560px,80vh)` and 88vh, vs App Settings' tight 760×620): they are short task panels, not a document you scan, and at scanning density they read as a few fields marooned in an empty frame. Those per-modal blocks are the design, not drift. Phones (≤860px) give App Settings the sticky `#appSettingsJump` pill and give the other two a horizontal rail strip, which neither has a pill for. ⚠️ The Session Options rail entry labelled **Session** still keys off `context` (`data-tab="context"`, `#context-tab`, `switchOptionsTab('context')`), the rename is label-only. Add Case keeps its legacy `.form-row` markup (six panels of it, every id read back by session-ui.js) and is mapped onto the look by an adapter block scoped to `#createCaseModal .set-doc`. Do not restructure those forms just to reach the row classes. ⚠️ That adapter's `summary { display:flex }` **kills the native disclosure triangle**, so every `<details>` there needs the explicit `.set-adv-chev` and both marker suppressions (`list-style` + `::-webkit-details-marker`); without it five collapsed blocks render as plain headings nobody clicks. ⚠️ **The load/save contract is `getElementById` by id**: `openAppSettings()`/`saveAppSettings()`/`openSessionOptions()` read every control by a fixed id, so moving a control between sections is free but renaming or dropping one silently stops it loading or saving. Static guards: `test/app-settings-structure.test.ts` + `test/session-options-structure.test.ts` (rail↔section pairing, one-visible-section, the `data-claude-only` entries external CLIs drop). ⚠️ Model cards (`#appSettingsModelCards`) and the effort segment are **views over hidden `<select>`s** that remain the source of truth; the cards hold the BASE model and the "1M context window" switch composes `base + [1m]` back into `claudeModel`, which is what retires the old "takes precedence over the toggle below" trap. ⚠️ `.modal-tabs`/`.modal-tab-btn`/`.modal-tab-content` are RETIRED: no modal uses them and their CSS is deleted, and a reappearance means a modal drifted off the shared surface. ⚠️ The **Header & Panels live preview** is a scale model rebuilt from the chips (`_syncLayoutPreview`); it owns NO icons, it CLONES `.set-chip-ico` out of the chip, so each icon has exactly one copy in index.html. A chip joins it via `data-preview` (slot) + `data-preview-order`, or `data-preview-text` for readouts that are not buttons. Its frame is painted from skin tokens only (hardcoded black alphas turned it into a grey slab on the light skins) and is `data-i18n-skip`. ⚠️ In Session Options → Respawn, auto-resume is a `.set-callout` whose `<label>` **wraps its own switch with no `for=`** (nesting associates them; the label+`for` pair has historically double-fired), and the cycle steps are real checkboxes (`.set-checks`), not chips. ⚠️ `admin-ui.js` injects the multi-user Users entry into `.set-rail-items` + `.set-doc`, so those hooks must survive any restructure. → [architecture-invariants#settings-surface-app-settings-session-options-add-case](docs/architecture-invariants.md#settings-surface-app-settings-session-options-add-case)
|
||||
|
||||
**Header button visibility**: most header controls are opt-in and hidden by a marker class (`btn-multimonitor--hidden`, `btn-response-viewer-header--hidden`, `btn-file-viewer--hidden`, `btn-cron--hidden`) that `applyHeaderVisibilitySettings()` (settings-ui.js) toggles after settings load; the multi-monitor button is instead stripped at render by `renderIndexHtml`. ⚠️ Hiding must go through the marker class: the base rules are `display:inline-flex !important`, so an inline style cannot override them. Current desktop default is WS/CPU/MEM + File Viewer + gear, with the token chip and lifecycle-log button OFF. ⚠️ New header controls must not leak onto phones; `test/mobile-header-buttons-policy.test.ts` is the static guard. → [architecture-invariants#header-button-visibility-multi-monitor-response-viewer-file-viewer-cron](docs/architecture-invariants.md#header-button-visibility-multi-monitor-response-viewer-file-viewer-cron)
|
||||
|
||||
@@ -278,6 +280,8 @@ Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. L
|
||||
|
||||
**Shell keyboard accessory bar + one-shot Ctrl** (issue #262, `keyboard-accessory.js`): a **shell**-mode session automatically swaps the mobile accessory bar for terminal controls (Ctrl, Esc, Tab, four arrows, paste, dismiss); every other mode keeps the agent bar. `setMode()` now records the user's `extendedKeyboardBar` preference as the **base** layout and `refreshForActiveSession()` (called from `selectSession`) resolves base-vs-shell, so a settings save during a shell session cannot yank the bar away and switching back restores the user's choice. ⚠️ **Ctrl is a ONE-SHOT modifier applied in `terminal.onData`, not in a keydown handler**: a virtual keyboard emits no usable key events, so the character only exists as onData text. The hook sits AFTER `shouldSuppressTerminalQueryResponse` (xterm answers DA/CPR through onData too, and one of those would silently spend the modifier) and BEFORE every send path, so the control byte follows the normal control-char route. ⚠️ **Not every onData chunk is a keystroke**, and the query filter is not enough on its own: xterm ALSO emits mouse and focus reports on its own initiative, so the hook skips them via `isTerminalFocusOrMouseReport()` (they still reach the PTY, they just don't count as the next key). The mouse half is live — a shell session keeps the NARROW strip, so mouse DECSETs reach the browser and one tap while vim/htop runs spent the armed modifier silently (measured). The focus half is defense in depth: `FOCUS_ESCAPE_FILTER` in `session.ts` strips `\x1b[?1004h` from every PTY read, so `sendFocusMode` never turns on today; if it ever did, the bar's own post-key refocus would emit `\x1b[I` and eat the modifier before the user typed. ⚠️ It must disarm on ALL of: use, second tap, any other accessory key, session switch, keyboard dismissal, and a layout swap; a modifier left armed turns the next innocent keystroke into a control byte. ⚠️ **onData is not the only input path** — with `cjkInputEnabled` on, the CJK textarea owns the keyboard (onData returns early for everything it swallows, and the focus router sends `terminal.focus()` there, which is where the bar refocuses after every key), so `_handleCjkInput()` applies the modifier too. It is that module's single choke point to the PTY, so one call covers typed characters, IME flushes, Enter, backspace and arrows. Without it an armed modifier could neither fire NOR be spent, and survived to a later keystroke. Mapping is `ctrlByteFor()` (`code & 0x1f` over @A-Z[\]^_ and a-z, plus Ctrl+Space=NUL / Ctrl+?=DEL); characters with no control equivalent pass through unchanged, like a hardware keyboard. ⚠️ The armed style is `.accessory-btn.accessory-btn-ctrl.armed` (0,3,0) in BOTH stylesheets, and it cannot outrank mobile.css's light-skin repaint at **(0,3,1)** (`:is()` inherits its most specific argument, and that list holds `.btn-toolbar.btn-shell`) — so that rule excludes the state by hand as `.accessory-btn:not(.armed)`. Without the exclusion the armed button renders identically to a resting one on all four light skins, which is worse than no armed style at all.
|
||||
|
||||
**Dismissing the on-screen keyboard** (PRs #279/#280, `terminal-ui.js`): the terminal parks focus on a hidden textarea that nothing used to release, so TWO gestures now blur it, and they own different regions. **(1)** `_installMobileKeyboardDismiss()` — a document-level `touchend` that fires only while the terminal input actually holds focus, **never inside `#terminalContainer`** (tap classification owns that) and **never on a control** (`MOBILE_KEYBOARD_DISMISS_EXEMPT_SELECTOR`, matched with `closest()` so an icon inside a button counts). Session tabs are covered by the selector's `[tabindex]:not([tabindex="-1"])` arm, which is what stops a tab tap from blurring and then being re-focused by `selectSession()`. **(2)** In `_handleMobileTerminalTap`, a second tap on **inert `content`** (`startedWithTerminalFocus`) blurs instead of re-focusing. ⚠️ Scoped to `content` on purpose: the prompt row (`input`) keeps focus-then-position so a second tap still places the caret, and actionable rows blur earlier via `_isActionableMobileTerminalTap`. ⚠️ **A scroll ends in `touchend` too** — dismissing there closes the keyboard and drops the composer mid-read, so travel is tracked from `touchstart` and multi-touch is never a tap. Both classifiers MUST share one threshold: `initTerminal`'s `TAP_THRESHOLD` reads `MOBILE_KEYBOARD_DISMISS_TAP_SLOP`, since a gesture the terminal calls a scroll and the dismiss handler calls a tap is exactly that bug. ⚠️ **`test:ci` excludes `test/mobile/**`, so CI cannot see the only test covering (1)** — run `npm test -- test/mobile/keyboard.test.ts` by hand and diff the FAIL list against master. That blind spot is why merging the two PRs, which conflicted semantically but not textually, produced a red suite with two green CI checks.
|
||||
|
||||
**Phone toolbar: Enter replaces Shell** (post-1.8.0): inside `@media (max-width: 430px)` `btn-shell` is `display:none` and `btn-enter` takes its slot (`order: 4`); starting a shell moved into the Run dropdown (`Terminal / Shell` → `setRunMode('shell')` → `run()` → `runShell()`, button label "Run SH"). `runMode` is `z.string().max(20)` server-side, so new modes need no schema change. Desktop and tablet keep the green Run Shell button unchanged.
|
||||
|
||||
⚠️ **`sendEnterKey()` MUST go through `terminal._core.coreService.triggerDataEvent('\r', true)`** — not `sendInput()`, and never a raw POST to `/api/sessions/:id/input`. `localEchoEnabled` defaults to `MobileDetection.isTouchDevice()`, so on every phone the characters you type are buffered in the `LocalEchoOverlay` and have **never reached the PTY**; the `onData` Enter branch in terminal-ui.js is what flushes `pendingText` first and only then sends `\r` (after an 80ms delay so text lands first). Sending a bare `\r` submits an empty line and strands the typed text on screen, so the button looks dead. Replaying the keypress reuses the overlay flush, the flushed-offset cleanup and the ordering instead of reimplementing them. `KeyboardAccessory.sendKey()` is for escape sequences (arrows/Esc) and is the WRONG template to copy for input.
|
||||
@@ -314,11 +318,11 @@ Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. L
|
||||
|
||||
### SSE Event Registry
|
||||
|
||||
154 event constants in `src/web/sse-events.ts` (backend) and `SSE_EVENTS` in `constants.js` (frontend). **Both must be kept in sync** — they are currently exactly in sync, and the backend file's `@fileoverview` carries the per-category breakdown.
|
||||
154 event constants in `src/web/sse-events.ts` (backend) and `SSE_EVENTS` in `constants.js` (frontend). **Both must be kept in sync**, and `test/sse-registry-parity.test.ts` is the guard that pins it (currently exactly in sync, 154 = 154, no drift either direction). The backend file's `@fileoverview` carries the per-category breakdown.
|
||||
|
||||
### API Routes
|
||||
|
||||
~200 handlers across 23 route files in `src/web/routes/`: system (45), sessions (34), cases (29), files (16), orchestrator (10), ralph (9), cron (9), admin (8), plan (8), respawn (7), webviews (6 + the `/webview/:cap/*` proxy), mux (5), push (4), scheduled (4, legacy `ScheduledRun`), approvals (3), readmymind (4), me (2), teams (2), search (1), hooks (1), clipboard (1), status-telemetry (1), ws (1 WebSocket). Each file has `@fileoverview` with endpoint details.
|
||||
~200 handlers across 24 route files in `src/web/routes/`: system (45), sessions (34), cases (29), files (16), orchestrator (10), ralph (9), cron (9), admin (8), plan (8), respawn (7), webviews (6 + the `/webview/:cap/*` proxy), mux (5), push (4), scheduled (4, legacy `ScheduledRun`), approvals (3), readmymind (4), me (2), teams (2), search (1), hooks (1), clipboard (1), status-telemetry (1), voice (1 + the `/ws/voice/stream` relay), ws (1 WebSocket). Each file has `@fileoverview` with endpoint details.
|
||||
|
||||
**HTTP contract** (stable since 0.9.x, see `docs/versioning-policy.md`; full envelope/status/error-code/SSE spec in `docs/api-reference.md`): responses use the `ApiResponse<T>` envelope — `{ success: true, data? }` or `{ success: false, error, errorCode }` (`src/types/api.ts`). `/api/v1/*` is a versioned alias of `/api/*` (URL rewrite in `server.ts`).
|
||||
|
||||
|
||||
@@ -254,7 +254,7 @@ Click **+ New Session** (or **Quick Start**). A session is one AI CLI running in
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Working directory / case** | The folder the agent operates in. A "case" is just a named working dir Codeman remembers. **Add Case** creates one from scratch, links an existing folder, or clones a GitHub repo straight into one (**Clone Repo**). |
|
||||
| **CLI / run mode** | `Claude` (default), `OpenCode`, `Codex`, `Antigravity`, `Gemini`, or `Terminal` (plain shell). |
|
||||
| **Model** | Per-session model (App Settings → Claude Model). A soft default — `/model` still works in-session. |
|
||||
| **Model** | Per-session model (App Settings → Models → New Claude sessions). A soft default — `/model` still works in-session. |
|
||||
| **Effort / Ultracode** | Reasoning effort (`low`–`max`) or `ultracode` for dynamic multi-agent workflows. Switchable anytime with `/effort`. |
|
||||
|
||||
Hit start — Codeman spawns the CLI via a real PTY and streams it to your browser over SSE.
|
||||
@@ -278,7 +278,7 @@ Hit start — Codeman spawns the CLI via a real PTY and streams it to your brows
|
||||
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| **Respawn** | Long unattended runs — auto-restarts the CLI on idle/limit, with adaptive timing. Presets: `solo-work`, `overnight-autonomous`, … | Respawn tab |
|
||||
| **Orchestrator** | Turn one goal into a phased plan and drive it to completion across agents. | Orchestrator panel |
|
||||
| **Cron** | Saved, named jobs on a schedule (`once`/`interval`/`daily`/`weekly`) that spawn a session and send a prompt when due. | ⏰ Cron button _(opt-in: App Settings → Display → Header Displays)_ |
|
||||
| **Cron** | Saved, named jobs on a schedule (`once`/`interval`/`daily`/`weekly`) that spawn a session and send a prompt when due. | ⏰ Cron button _(opt-in: App Settings → Header & Panels → Scheduling)_ |
|
||||
| **Auto-resume** | Automatically continue after a subscription rate-limit resets. | Respawn tab (top) |
|
||||
|
||||
### 6. Reach it from anywhere
|
||||
@@ -291,7 +291,7 @@ Hit start — Codeman spawns the CLI via a real PTY and streams it to your brows
|
||||
|
||||
- **App Settings** — model, effort, permission startup mode, theme/skin, notifications, display toggles, per-CLI options, a synced custom display name, and per-device English/Simplified Chinese UI language.
|
||||
- **Run it in the background** — `codeman web -d` detaches from your shell (`--status`, `--stop`); `codeman service install` makes it a systemd user unit / macOS LaunchAgent that survives reboots. Both verify the server actually answers before reporting success, and both refuse to start a second server on one data dir. See [Keep it running in the background](#quick-start---installation).
|
||||
- **Self-update** — git-clone installs update in place from **Settings → Updates**.
|
||||
- **Self-update** — git-clone installs update in place from **App Settings → System → Updates**.
|
||||
- **Deploy your own changes** — see [Development](#development).
|
||||
|
||||
> ⚠️ **Safety:** if you're working _inside_ a Codeman-managed session (`echo $CODEMAN_MUX` → `1`), never run `tmux kill-session` / `pkill claude` directly — use the web UI or `./scripts/tmux-manager.sh`.
|
||||
@@ -427,7 +427,7 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt
|
||||
## More Features
|
||||
|
||||
- **Background daemon & service install** — `codeman web -d` runs the server detached with a pidfile, `~/.codeman/web.log`, and verified startup (it polls the server until it answers, so a port clash never reads as success); `codeman service install` writes a systemd user unit (Linux) or LaunchAgent (macOS) with your shell's PATH baked in, so an nvm or Homebrew `node`, `tmux` and `claude` are actually found. Secrets are never written into unit files
|
||||
- **Self-update** — git-clone installs under systemd/launchd update in place from **App Settings → Updates**: it detects the latest release, auto-stashes a dirty tree, and streams build progress across the service restart (npm installs report as non-updatable)
|
||||
- **Self-update** — git-clone installs under systemd/launchd update in place from **App Settings → System → Updates**: it detects the latest release, auto-stashes a dirty tree, and streams build progress across the service restart (npm installs report as non-updatable)
|
||||
- **Clone a GitHub repo as a case** — paste a repository URL into **Add Case → Clone Repo** and Codeman clones it into `~/codeman-cases/<name>` and registers it as a normal case, ready to run an agent in. It preflights the URL while you type (tells you whether it can be cloned anonymously and offers the repo's real branches and tags for the optional branch/tag field), fills the case name in from the URL, and lets you pick which CLI the Run button should use. Public repositories over `https://`; Codeman never collects or stores credentials
|
||||
- **Multi-CLI** — run **Claude Code**, **OpenCode**, **Codex**, **Antigravity**, or **Gemini** per session; env-var prefixes auto-gate (`CLAUDE_CODE_*` vs `OPENCODE_*` vs `CODEX_*` vs `ANTIGRAVITY_*` vs `GEMINI_*`/`GOOGLE_*`). See [`docs/opencode-integration.md`](docs/opencode-integration.md)
|
||||
- **Docker sessions** — run a case inside an isolated, hardened container. One checkbox on **Create New** spins up a container with sensible defaults and starts the agent inside it; multiple sessions share one per-case container; export a container + its workspace to a portable `.tar.gz` to move it to another machine. See [`docs/docker-cases.md`](docs/docker-cases.md)
|
||||
@@ -435,9 +435,9 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt
|
||||
- **Effort & Ultracode** — set a per-session default effort (`low`–`max`) or enable **ultracode** (dynamic multi-agent workflows). Soft defaults only — switchable anytime with `/effort` in-session. Extended-thinking budget is configurable too
|
||||
- **Voice input** — dictate prompts with Deepgram Nova-3 (Web Speech API fallback): toggle recording, auto-silence stop, live level meter (`Ctrl+Shift+V`)
|
||||
- **Image input** — paste or drag-and-drop images straight into a session
|
||||
- **Gesture control** _(opt-in)_ — a MediaPipe hand-tracking overlay to grab/drag session windows and pinch buttons, hands-free. Enable with `CODEMAN_GESTURE=1` + App Settings → Display
|
||||
- **Gesture control** _(opt-in)_ — a MediaPipe hand-tracking overlay to grab/drag session windows and pinch buttons, hands-free. Enable with `CODEMAN_GESTURE=1` + App Settings → Terminal & Input
|
||||
- **Multi-monitor span** _(macOS)_ — one click opens a browser window maximized across all displays, so floating agent/gesture panels can cross the physical seam
|
||||
- **File Viewer button** _(opt-in)_ — a header button that toggles the built-in file browser panel with one tap; enable under App Settings → Display → Header Displays
|
||||
- **File Viewer button** _(opt-in)_ — a header button that toggles the built-in file browser panel with one tap; enable under App Settings → Header & Panels → Header buttons
|
||||
- **CJK / IME input** — full composition support for Chinese / Japanese / Korean
|
||||
- **OS notifications & hostname-aware titles** — desktop alerts and tab titles are prefixed `codeman:<host>` so multi-host setups stay unambiguous
|
||||
|
||||
@@ -521,7 +521,7 @@ The script auto-installs a systemd user service on first run. The tunnel URL is
|
||||
systemctl --user enable codeman-tunnel
|
||||
loginctl enable-linger $USER
|
||||
|
||||
# Or via the Codeman web UI: Settings → Tunnel → Toggle On
|
||||
# Or via the Codeman web UI: App Settings → System → Remote access → Cloudflare Tunnel
|
||||
```
|
||||
|
||||
</details>
|
||||
@@ -623,7 +623,7 @@ By default Codeman launches sessions with `--dangerously-skip-permissions`, so t
|
||||
- **Loopback by default** — the server binary binds `127.0.0.1`, reachable only from the same machine, so the no-password default is safe out of the box (the guided installer asks about network access and configures the binding + password for you). Binding a non-loopback host without `CODEMAN_PASSWORD` _starts but prints a loud warning_ with three concrete fixes (set a password, loopback + an authenticated tunnel, or explicitly acknowledge with `--allow-unauthenticated-network`)
|
||||
- **Optional auth, real sessions** — HTTP Basic via `CODEMAN_USERNAME` (default `admin`) / `CODEMAN_PASSWORD`. Success issues an opaque 256-bit `codeman_session` cookie (`randomBytes(32)`) — validated server-side, not client-signed, so it can't be forged offline (24h TTL, auto-extend, device-context audit log)
|
||||
- **Per-IP rate limiting** — 10 failed attempts → `429` with `Retry-After` (15-min decay). A valid cookie or correct password recovers _immediately_ even while an attacker hammers the same IP — important because all tunnel traffic shares one loopback IP. QR auth has its own separate limiter
|
||||
- **Configurable permission mode** - `--dangerously-skip-permissions` is only the default. **App Settings → Claude CLI → Startup Mode** can switch new sessions to Anthropic's classifier-guarded `auto` mode (low-prompt, needs Claude Code 2.1.207+), `normal` prompting, or an explicit allowed-tools list. In multi-user mode, non-granted users are forced to `auto`, and shell sessions / skip-permissions require an explicit per-user grant
|
||||
- **Configurable permission mode** - `--dangerously-skip-permissions` is only the default. **App Settings → Agents & CLIs → Claude → Startup Mode** can switch new sessions to Anthropic's classifier-guarded `auto` mode (low-prompt, needs Claude Code 2.1.207+), `normal` prompting, or an explicit allowed-tools list. In multi-user mode, non-granted users are forced to `auto`, and shell sessions / skip-permissions require an explicit per-user grant
|
||||
|
||||
### Always-on browser hardening (v0.9.5)
|
||||
|
||||
@@ -696,7 +696,7 @@ For AI agents and automation that control Codeman without a browser: an agent th
|
||||
>
|
||||
> - `npx skills add Ark0N/Codeman --skill codeman -g`: global, works for any skills-aware agent
|
||||
> - `codeman skill install` (global) or `codeman skill install --case <name>`: for npm installs that never cloned the repo; `codeman skill uninstall` reverses it
|
||||
> - **App Settings → Agent Skill** (`agentSkillEnabled`, default off): Codeman then injects the skill into each case on Claude session create; a user-authored `skills/codeman` in the case is never overwritten
|
||||
> - **App Settings → Agents & CLIs → Claude → Agent Skill** (`agentSkillEnabled`, default off): Codeman then injects the skill into each case on Claude session create; a user-authored `skills/codeman` in the case is never overwritten
|
||||
>
|
||||
> A global install (`codeman skill install`, or `npx skills add`) is picked up by **every new Claude Code session on the machine**, inside Codeman or not. The skill self-gates: outside a Codeman session (`CODEMAN_MUX` unset) it refuses to act, so a global install costs an idle session nothing.
|
||||
>
|
||||
|
||||
@@ -471,6 +471,28 @@ All four enforce session ownership in multi-user mode; a foreign session id
|
||||
answers `404 NOT_FOUND` (no existence leak), and profiles of two owners of the
|
||||
same directory are distinct by construction.
|
||||
|
||||
## Voice dictation
|
||||
|
||||
Browser dictation transcribed through this server's Claude Code login, i.e. the
|
||||
same speech-to-text service the CLI's own `/voice` mode uses. Gated on the synced
|
||||
`claudeVoiceEnabled` setting (default OFF). Design:
|
||||
[`claude-voice-plan.md`](claude-voice-plan.md).
|
||||
|
||||
- `GET /api/v1/voice/status` -> `{ available, reason?, subscriptionType?,
|
||||
expiresAt? }`. `reason` is `disabled` (setting off), `no-credentials` (nobody
|
||||
signed in to Claude Code on the server), `expired` (the access token elapsed;
|
||||
running any Claude session refreshes it) or `malformed`. The OAuth token
|
||||
itself is never returned by this or any other endpoint.
|
||||
- `GET /ws/voice/stream?language=&keyterms=` (WebSocket, not under `/api`)
|
||||
relays one dictation. Client sends binary frames of signed 16-bit
|
||||
little-endian PCM, 16 kHz mono (<= 64 KB per frame), plus JSON control frames
|
||||
`{"t":"finalize"}` (ask for the final transcript) and `{"t":"stop"}`. Server
|
||||
sends `{"t":"ready"}`, `{"t":"transcript","text","final"}` (each frame is the
|
||||
WHOLE running transcript, not a delta), `{"t":"error","message"}` and
|
||||
`{"t":"closed"}`. Close codes: `4003` disallowed Host/Origin, `4004`
|
||||
unavailable (reason in the close reason), `4008` too many concurrent streams.
|
||||
Streams are capped in count and length (`src/config/voice.ts`).
|
||||
|
||||
## Authentication
|
||||
|
||||
Optional HTTP Basic (`CODEMAN_USERNAME`/`CODEMAN_PASSWORD`) → opaque
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -300,6 +300,11 @@ For reference when writing browser tests:
|
||||
.xterm // Terminal container
|
||||
#helpModal // Help modal
|
||||
#appSettingsModal // Settings modal
|
||||
#sessionOptionsModal // Session Options (same set-* surface)
|
||||
#createCaseModal // Add Case (same set-* surface)
|
||||
.set-rail-item // Rail entry: scrolls in App Settings, switches in the other two
|
||||
.set-section // A settings section (`.hidden` on the inactive ones outside App Settings)
|
||||
.set-row // One setting: label + description left, control right
|
||||
.modal-content // Modal content
|
||||
.modal-close // Modal close button
|
||||
.header-brand .logo // Logo text
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
# Claude voice dictation in Codeman
|
||||
|
||||
Wire Codeman's existing mic button to the same speech-to-text service Claude Code's own
|
||||
`/voice` mode uses, so dictation works with **no third-party API key** for anyone already
|
||||
signed in to Claude Code on the server.
|
||||
|
||||
## Why the CLI's own voice mode cannot be reused directly
|
||||
|
||||
Claude Code 2.1.x ships voice input: `/voice hold|tap|off` arms it, the CLI opens the
|
||||
**host's** microphone (native `audio-capture-napi`, falling back to `sox`/`arecord` on Linux
|
||||
after probing `/proc/asound/cards`), streams PCM upstream and types the transcript into its
|
||||
own composer.
|
||||
|
||||
Every part of that is on the wrong machine for Codeman. The CLI runs inside a tmux pane on
|
||||
the server, which is typically headless and has no sound card at all, while the human is in
|
||||
a browser on a phone somewhere else. Toggling `/voice` in the pane from Codeman would arm a
|
||||
microphone nobody is sitting in front of. So Codeman keeps capturing audio in the browser,
|
||||
where the user actually is, and only borrows the CLI's **transcription backend**.
|
||||
|
||||
## The backend, as the CLI uses it
|
||||
|
||||
Extracted from the 2.1.226 binary (`connectVoiceStream`):
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| URL | `wss://api.anthropic.com/api/ws/speech_to_text/voice_stream` |
|
||||
| Query | `encoding=linear16`, `sample_rate=16000`, `channels=1`, `endpointing_ms=300`, `utterance_end_ms=1000`, `language=<lang>`, `use_conversation_engine=true`, `stt_provider=deepgram-nova3` |
|
||||
| Headers | `Authorization: Bearer <Claude Code OAuth access token>`, `User-Agent`, `x-app: cli`, `anthropic-client-platform`, optional `x-config-keyterms` |
|
||||
| Audio | raw binary frames, PCM signed 16-bit little-endian, 16 kHz, mono |
|
||||
| Keepalive | `{"type":"KeepAlive"}` on open, then every 8 s |
|
||||
| Finalize | `{"type":"CloseStream"}`, then wait for the endpoint frame |
|
||||
| Downstream | `{"type":"TranscriptText"\|"TranscriptInterim","data":"…"}` (running interim), `{"type":"TranscriptEndpoint"}` (promotes the pending interim to final), `{"type":"TranscriptError",…}`, `{"type":"error","message":…}` |
|
||||
|
||||
Deepgram Nova-3 runs server-side, so the Deepgram-quality result arrives without a Deepgram
|
||||
account. Verified against the live endpoint before this design was written: connect, stream
|
||||
PCM, receive interims and an endpoint frame.
|
||||
|
||||
## Architecture
|
||||
|
||||
The browser cannot call that endpoint itself: it would need the OAuth bearer token in page
|
||||
JavaScript (and CORS would refuse anyway). So the audio goes browser → Codeman → Anthropic,
|
||||
and Codeman is the only thing that ever touches the token.
|
||||
|
||||
```
|
||||
mic → AudioWorklet (Float32 → PCM16 @16 kHz)
|
||||
→ wss://<codeman>/ws/voice/stream [cookie/basic auth, Origin+Host guarded]
|
||||
→ VoiceStreamRelay (reads ~/.claude/.credentials.json per connect)
|
||||
→ wss://api.anthropic.com/api/ws/speech_to_text/voice_stream
|
||||
← {"t":"transcript","text":…,"final":…} → existing _insertText() path
|
||||
```
|
||||
|
||||
Nothing about the insert path changes: the transcript lands in the same preview overlay,
|
||||
the same direct/compose insert modes, the same green Send button.
|
||||
|
||||
### Server pieces
|
||||
|
||||
- **`src/claude-credentials.ts`** — locate and parse the Claude Code OAuth credentials.
|
||||
`parseClaudeCredentials()` is pure (JSON string + `now` → status) and unit-tested;
|
||||
`readClaudeOAuthToken()` wraps it with IO: `$CLAUDE_CONFIG_DIR/.credentials.json` or
|
||||
`~/.claude/.credentials.json`, and on macOS the login keychain
|
||||
(`security find-generic-password -s "Claude Code-credentials"`).
|
||||
**Read-only, always.** Codeman never writes credentials and never refreshes the token: a
|
||||
refresh rotates the refresh token, and racing Claude Code's own refresh could sign the
|
||||
user out of their CLI. An expired token surfaces as a plain "run a Claude session to
|
||||
refresh" error instead.
|
||||
The token is never logged, never returned by any endpoint, and never sent to the browser.
|
||||
|
||||
- **`src/web/voice-stream.ts`** — pure `buildVoiceStreamUrl()` / `buildVoiceStreamHeaders()` /
|
||||
`sanitizeKeyterms()` (ASCII-only, deduped, 1024-char cap, mirroring the CLI), plus
|
||||
`VoiceStreamRelay`, which owns one upstream socket: keepalive timer, audio passthrough,
|
||||
transcript translation, finalize, and the caps below.
|
||||
|
||||
- **`src/web/routes/voice-routes.ts`**
|
||||
- `GET /api/voice/status` → `{ available, reason, subscriptionType?, expiresAt? }`. Never
|
||||
the token. `available:false` with a machine-readable `reason` (`disabled`, `no-credentials`,
|
||||
`expired`) is what the settings row and the provider resolver read.
|
||||
- `GET /ws/voice/stream?language=&keyterms=` → the relay. Same upgrade guard as
|
||||
`/ws/sessions/:id/terminal`: allowed Host, same-site Origin, and the global auth hook has
|
||||
already run on the handshake.
|
||||
|
||||
Caps, because an open mic is an open pipe: one stream per connection, `MAX_VOICE_STREAMS`
|
||||
concurrent server-wide, a hard `MAX_STREAM_MS` per stream, and a per-frame size cap. A tab
|
||||
left recording cannot bill an unbounded amount of upstream audio.
|
||||
|
||||
### Frontend pieces
|
||||
|
||||
- **`voice-pcm-worklet.js`** — an `AudioWorkletProcessor` converting Float32 blocks to PCM16
|
||||
and posting ~256 ms frames back. `MediaRecorder` cannot produce raw PCM, which is why the
|
||||
existing Deepgram path (container audio, auto-detected) cannot be reused as-is. Falls back
|
||||
to `ScriptProcessorNode` where AudioWorklet is unavailable.
|
||||
- **`ClaudeVoiceProvider`** in `voice-input.js` — mirrors `DeepgramProvider`'s shape
|
||||
(`start({language, keyterms, onStream, onResult, onError, onEnd})`) so `VoiceInput` treats
|
||||
the three providers uniformly.
|
||||
- **Provider resolution** — new `voiceSettings.provider`: `auto` (default) | `claude` |
|
||||
`deepgram` | `webspeech`. `auto` picks Claude when `/api/voice/status` reports it
|
||||
available, else Deepgram when a key is set, else Web Speech. Pinning a provider always
|
||||
wins, so an existing Deepgram user can keep exactly what they have.
|
||||
|
||||
### Settings
|
||||
|
||||
- `claudeVoiceEnabled` — synced, **default OFF**, gating the whole server side. Off is the
|
||||
honest default: turning it on means this machine's Claude subscription starts paying for
|
||||
transcription for whoever can reach the UI, and the audio goes to Anthropic rather than to
|
||||
wherever it went before. One switch in Settings → Voice, and the mic works with no key.
|
||||
- `voiceSettings.provider` — per the resolution table above; joins the existing synced
|
||||
`voiceSettings` object.
|
||||
|
||||
## Things worth knowing
|
||||
|
||||
- **This uses an undocumented endpoint with subscription credentials.** It is the user's own
|
||||
token, on the user's own machine, driving the user's own Claude Code install, but it is not
|
||||
a published API and Anthropic can change or restrict it. Default-OFF is deliberate; the
|
||||
Deepgram and Web Speech paths stay untouched as the supported fallbacks.
|
||||
- **Multi-user mode**: every user's dictation would run on the server owner's Claude
|
||||
credentials, exactly as every user's *sessions* already run on them. Consistent, but worth
|
||||
stating out loud in the settings copy.
|
||||
- **Token lifetime** is about 8 hours, refreshed by Claude Code itself whenever it runs. The
|
||||
relay re-reads the file on every connect rather than caching, so a refresh is picked up on
|
||||
the next press of the mic.
|
||||
- **HTTPS or localhost**: `getUserMedia` needs a secure context. Prod is HTTPS behind
|
||||
`tailscale serve`, so this is already satisfied; the existing error copy covers the rest.
|
||||
+2
-2
@@ -11,7 +11,7 @@ Codeman's per-case memory of what you are trying to accomplish, and the 🧠 but
|
||||
|
||||
## Turning it on
|
||||
|
||||
App Settings → Panels → **Read My Mind** (synced setting `readMyMindEnabled`, default **OFF**). It gates everything: capture, the header button, and nothing shows anywhere while it is off. The API equivalent:
|
||||
App Settings → Header & Panels → Cross-session features → **Read My Mind** (synced setting `readMyMindEnabled`, default **OFF**). It gates everything: capture, the header button, and nothing shows anywhere while it is off. The API equivalent:
|
||||
|
||||
```bash
|
||||
curl -sk -X PUT https://localhost:3000/api/settings \
|
||||
@@ -93,7 +93,7 @@ Explicitly later: proactive predict-on-idle, auto-compaction of the prompt histo
|
||||
|
||||
| Symptom | Cause / fix |
|
||||
| ------- | ----------- |
|
||||
| No 🧠 button in the header | `readMyMindEnabled` is OFF (App Settings → Panels), you are on a phone (there it is a key on the keyboard accessory bar instead, visible while typing), or the active session is not claude-mode |
|
||||
| No 🧠 button in the header | `readMyMindEnabled` is OFF (App Settings → Header & Panels → Cross-session features), you are on a phone (there it is a key on the keyboard accessory bar instead, visible while typing), or the active session is not claude-mode |
|
||||
| Prediction feels generic | The profile is thin: record goals (PUT or ask your agent to), and let capture accumulate a few real prompts first |
|
||||
| "A prediction is already running" (409) | One per session at a time; wait for the current one (up to 90 s) |
|
||||
| Prediction fails (502) | The model returned no usable JSON, or the CLI could not start; retry. Check `readMyMindModel` if you overrode it |
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "aicodeman",
|
||||
"version": "1.16.3",
|
||||
"version": "1.16.5",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "aicodeman",
|
||||
"version": "1.16.3",
|
||||
"version": "1.16.5",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "aicodeman",
|
||||
"version": "1.16.3",
|
||||
"version": "1.16.5",
|
||||
"description": "Mission control for AI coding agents - run 20 autonomous agents with real-time monitoring and session persistence",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
/**
|
||||
* @fileoverview Read-only access to the Claude Code OAuth credentials.
|
||||
*
|
||||
* Claude Code stores its subscription OAuth tokens in
|
||||
* `$CLAUDE_CONFIG_DIR/.credentials.json` (default `~/.claude/.credentials.json`,
|
||||
* mode 0600) on Linux/Windows, and in the login keychain on macOS. Codeman reads
|
||||
* the access token to authenticate the voice-dictation relay
|
||||
* (`src/web/voice-stream.ts`) against the same speech-to-text service the CLI's
|
||||
* own `/voice` mode uses.
|
||||
*
|
||||
* ⚠️ READ-ONLY, deliberately. Codeman never writes this file and never performs
|
||||
* an OAuth refresh: a refresh ROTATES the refresh token, so racing Claude Code's
|
||||
* own refresh could invalidate the user's CLI login. An expired access token is
|
||||
* reported as `expired` and the caller tells the user to run a Claude session
|
||||
* (which refreshes it) instead.
|
||||
*
|
||||
* ⚠️ The token is a bearer secret: it is never logged, never persisted, never
|
||||
* included in any API response, and never sent to the browser.
|
||||
*/
|
||||
|
||||
import { readFile } from 'fs/promises';
|
||||
import { execFile } from 'child_process';
|
||||
import { homedir, userInfo } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
/** Result of inspecting the credential store. The token is present only on 'ok'. */
|
||||
export type ClaudeCredentialStatus = 'ok' | 'expired' | 'missing' | 'malformed';
|
||||
|
||||
export interface ClaudeOAuthCredentials {
|
||||
status: ClaudeCredentialStatus;
|
||||
/** Bearer token. Present only when status is 'ok'. Never log or serialize this. */
|
||||
accessToken?: string;
|
||||
/** Epoch ms the access token expires at, when the store reports one. */
|
||||
expiresAt?: number;
|
||||
/** e.g. 'max', 'pro'. Display-only, safe to surface. */
|
||||
subscriptionType?: string;
|
||||
}
|
||||
|
||||
/** Skew applied to the stored expiry so a token that dies mid-stream is refused up front. */
|
||||
const EXPIRY_SKEW_MS = 60_000;
|
||||
|
||||
/** macOS keychain service holding the same JSON blob as `.credentials.json`. */
|
||||
const KEYCHAIN_SERVICE = 'Claude Code-credentials';
|
||||
|
||||
/** Keychain lookups shell out; keep them short so a locked keychain cannot hang a request. */
|
||||
const KEYCHAIN_TIMEOUT_MS = 3000;
|
||||
|
||||
/**
|
||||
* Parse a `.credentials.json` payload. Pure: no IO, no clock read (pass `now`),
|
||||
* so the expiry and shape handling are unit-testable.
|
||||
*
|
||||
* Returns 'malformed' for anything that is not the expected `claudeAiOauth`
|
||||
* shape rather than throwing — a hand-edited or half-written file must degrade
|
||||
* to "voice unavailable", never to a 500.
|
||||
*/
|
||||
export function parseClaudeCredentials(raw: string, now: number): ClaudeOAuthCredentials {
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
return { status: 'malformed' };
|
||||
}
|
||||
if (!parsed || typeof parsed !== 'object') return { status: 'malformed' };
|
||||
|
||||
const oauth = (parsed as { claudeAiOauth?: unknown }).claudeAiOauth;
|
||||
if (!oauth || typeof oauth !== 'object') return { status: 'malformed' };
|
||||
|
||||
const record = oauth as Record<string, unknown>;
|
||||
const accessToken = typeof record.accessToken === 'string' ? record.accessToken.trim() : '';
|
||||
if (!accessToken) return { status: 'malformed' };
|
||||
|
||||
const expiresAt = typeof record.expiresAt === 'number' ? record.expiresAt : undefined;
|
||||
const subscriptionType = typeof record.subscriptionType === 'string' ? record.subscriptionType : undefined;
|
||||
|
||||
// An expired token is a real state (the CLI refreshes on its next run), not a
|
||||
// malformed store: report it separately so the UI can say something useful.
|
||||
if (expiresAt !== undefined && expiresAt - EXPIRY_SKEW_MS <= now) {
|
||||
return { status: 'expired', expiresAt, subscriptionType };
|
||||
}
|
||||
return { status: 'ok', accessToken, expiresAt, subscriptionType };
|
||||
}
|
||||
|
||||
/** Path of the credentials file, honoring CLAUDE_CONFIG_DIR like the CLI does. */
|
||||
export function claudeCredentialsPath(env: NodeJS.ProcessEnv = process.env): string {
|
||||
const configDir = typeof env.CLAUDE_CONFIG_DIR === 'string' && env.CLAUDE_CONFIG_DIR.trim();
|
||||
return join(configDir || join(homedir(), '.claude'), '.credentials.json');
|
||||
}
|
||||
|
||||
/** Read the macOS keychain entry. Resolves to null on any failure (locked, absent, non-mac). */
|
||||
function readKeychainCredentials(): Promise<string | null> {
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
'security',
|
||||
['find-generic-password', '-a', userInfo().username, '-w', '-s', KEYCHAIN_SERVICE],
|
||||
{ encoding: 'utf-8', timeout: KEYCHAIN_TIMEOUT_MS },
|
||||
(err, stdout) => resolve(err ? null : stdout.trim() || null)
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Locate and parse the Claude Code OAuth credentials.
|
||||
*
|
||||
* File first (present on every platform once the CLI has run there), keychain
|
||||
* second on macOS. Never caches: Claude Code rewrites the store roughly every
|
||||
* 8 hours, and a cached token would go stale inside a long-lived server.
|
||||
*/
|
||||
export async function readClaudeOAuthCredentials(now: number = Date.now()): Promise<ClaudeOAuthCredentials> {
|
||||
let fileResult: ClaudeOAuthCredentials | null = null;
|
||||
try {
|
||||
fileResult = parseClaudeCredentials(await readFile(claudeCredentialsPath(), 'utf-8'), now);
|
||||
} catch {
|
||||
fileResult = null;
|
||||
}
|
||||
if (fileResult && fileResult.status !== 'malformed') return fileResult;
|
||||
|
||||
if (process.platform === 'darwin') {
|
||||
const raw = await readKeychainCredentials();
|
||||
if (raw) {
|
||||
const keychainResult = parseClaudeCredentials(raw, now);
|
||||
if (keychainResult.status !== 'malformed') return keychainResult;
|
||||
}
|
||||
}
|
||||
|
||||
return fileResult ?? { status: 'missing' };
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* @fileoverview Bounds and endpoint config for Claude voice dictation.
|
||||
*
|
||||
* Backs the browser → Codeman → Anthropic dictation relay (`src/web/voice-stream.ts`,
|
||||
* `src/web/routes/voice-routes.ts`; design in `docs/claude-voice-plan.md`).
|
||||
*
|
||||
* Why everything here is bounded: an open microphone is an open pipe. Each live
|
||||
* stream holds a browser socket, an upstream socket and a keepalive timer, and
|
||||
* every second of audio is billed against the server owner's Claude subscription.
|
||||
* A tab left recording (phone in a pocket, forgotten laptop) must cost a bounded
|
||||
* amount, so streams die on their own at `MAX_STREAM_MS` and the server refuses
|
||||
* more than `MAX_CONCURRENT_STREAMS` at once.
|
||||
*
|
||||
* The audio frame cap is a memory guard on a socket that carries attacker-shaped
|
||||
* binary data: PCM16 at 16 kHz mono is 32 KB/s, so a 256 ms frame is ~8 KB and
|
||||
* anything near 64 KB is either a broken client or an attempt to make the relay
|
||||
* buffer for someone else.
|
||||
*/
|
||||
|
||||
/** Upstream speech-to-text service (the one Claude Code's own `/voice` mode uses). */
|
||||
export const VOICE_STREAM_HOST = 'wss://api.anthropic.com';
|
||||
|
||||
/** Path of the streaming speech-to-text endpoint. */
|
||||
export const VOICE_STREAM_PATH = '/api/ws/speech_to_text/voice_stream';
|
||||
|
||||
/**
|
||||
* Base override, for tests (point the relay at a local mock) and for users on an
|
||||
* Anthropic-compatible gateway. Must be a ws:// or wss:// origin.
|
||||
*/
|
||||
export function voiceStreamBase(env: NodeJS.ProcessEnv = process.env): string {
|
||||
const override = typeof env.CODEMAN_VOICE_STREAM_BASE === 'string' ? env.CODEMAN_VOICE_STREAM_BASE.trim() : '';
|
||||
if (override && /^wss?:\/\//.test(override)) return override.replace(/\/+$/, '');
|
||||
return VOICE_STREAM_HOST;
|
||||
}
|
||||
|
||||
/** Upstream drops an idle socket; the CLI pings at 8s and so do we. */
|
||||
export const KEEPALIVE_INTERVAL_MS = 8000;
|
||||
|
||||
/** Hard ceiling on one dictation. Long enough for any real utterance, short enough to bound a forgotten mic. */
|
||||
export const MAX_STREAM_MS = 5 * 60_000;
|
||||
|
||||
/** Concurrent relays server-wide. Dictation is a human-paced, one-at-a-time act. */
|
||||
export const MAX_CONCURRENT_STREAMS = 4;
|
||||
|
||||
/** Largest single audio frame accepted from the browser (~2s of PCM16 @16 kHz mono). */
|
||||
export const MAX_AUDIO_FRAME_BYTES = 64 * 1024;
|
||||
|
||||
/** How long to wait for the final transcript after the client asks to finalize. */
|
||||
export const FINALIZE_TIMEOUT_MS = 3000;
|
||||
|
||||
/** Upstream caps the keyterms header; mirrors the CLI's own limit. */
|
||||
export const MAX_KEYTERMS_HEADER_CHARS = 1024;
|
||||
|
||||
/** Audio format the endpoint is opened with. The browser worklet must match exactly. */
|
||||
export const AUDIO_SAMPLE_RATE = 16000;
|
||||
export const AUDIO_CHANNELS = 1;
|
||||
@@ -19,6 +19,8 @@ export interface ConfigPort {
|
||||
getTerminalHistoryConfig(): Promise<TerminalHistoryConfig>;
|
||||
/** Synced `agentSkillEnabled` app setting (default OFF); gates per-case agent-skill injection. */
|
||||
getAgentSkillEnabled(): Promise<boolean>;
|
||||
/** Synced `claudeVoiceEnabled` app setting (default OFF); gates the Claude voice dictation relay. */
|
||||
getClaudeVoiceEnabled(): Promise<boolean>;
|
||||
getDefaultClaudeMdPath(): Promise<string | undefined>;
|
||||
getLightState(identity?: { username: string; role: 'admin' | 'user' }): unknown;
|
||||
getLightSessionsState(): unknown[];
|
||||
|
||||
@@ -393,10 +393,14 @@ Object.assign(CodemanApp.prototype, {
|
||||
// that collide with state strings on other surfaces.
|
||||
pill.setAttribute('data-i18n-skip', '');
|
||||
pill.textContent = row.pill;
|
||||
item.appendChild(pill);
|
||||
|
||||
// Wraps onto its own line (the row is flex-wrap) so it gets the full width.
|
||||
item.appendChild(this._buildHomeSessionsMeta(row));
|
||||
// The stamps line wraps onto its own full-width line (the row is flex-wrap)
|
||||
// and the pill rides along at its right end, rather than sitting beside the
|
||||
// name: that hands the whole width of the rail to the session name, which is
|
||||
// what stops it ellipsizing.
|
||||
const meta = this._buildHomeSessionsMeta(row);
|
||||
meta.appendChild(pill);
|
||||
item.appendChild(meta);
|
||||
|
||||
return item;
|
||||
},
|
||||
@@ -439,7 +443,14 @@ Object.assign(CodemanApp.prototype, {
|
||||
pill.className = 'home-sessions-pill home-sessions-pill--web';
|
||||
pill.setAttribute('data-i18n-skip', '');
|
||||
pill.textContent = 'web';
|
||||
item.appendChild(pill);
|
||||
|
||||
// Same bottom line as a session row (minus the stamps, a dashboard has
|
||||
// none), so the pill sits in the same place on every row in the rail.
|
||||
const foot = document.createElement('span');
|
||||
foot.className = 'home-sessions-row-meta';
|
||||
foot.setAttribute('data-i18n-skip', '');
|
||||
foot.appendChild(pill);
|
||||
item.appendChild(foot);
|
||||
|
||||
return item;
|
||||
},
|
||||
|
||||
+652
-407
File diff suppressed because it is too large
Load Diff
+140
-116
@@ -223,24 +223,6 @@ html.mobile-init .file-browser-panel {
|
||||
min-height: 56px;
|
||||
}
|
||||
|
||||
.modal-tabs {
|
||||
overflow-x: auto;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scrollbar-width: none;
|
||||
flex-wrap: nowrap;
|
||||
}
|
||||
|
||||
.modal-tabs::-webkit-scrollbar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.modal-tab-btn {
|
||||
padding: 0.4rem 0.75rem;
|
||||
font-size: 0.7rem;
|
||||
white-space: nowrap;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* Settings grid stays 2-col on tablet but tighter */
|
||||
.settings-grid {
|
||||
gap: 0.4rem 0.75rem;
|
||||
@@ -2079,45 +2061,8 @@ html.mobile-init .file-browser-panel {
|
||||
|
||||
/* ---- Settings Modal: Mobile Optimizations ---- */
|
||||
|
||||
/* Scrollable tabs row - prevent overflow on small screens */
|
||||
.modal-tabs {
|
||||
overflow-x: auto;
|
||||
-webkit-overflow-scrolling: touch;
|
||||
scrollbar-width: none;
|
||||
gap: 0.25rem;
|
||||
padding: 0 0.75rem 0.5rem 0.75rem;
|
||||
flex-wrap: nowrap;
|
||||
}
|
||||
|
||||
.modal-tabs::-webkit-scrollbar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.modal-tab-btn {
|
||||
padding: 0.35rem 0.6rem;
|
||||
font-size: 0.65rem;
|
||||
white-space: nowrap;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* ---- Case Modal: Mobile Touch Optimizations ---- */
|
||||
|
||||
/* Larger tab buttons for case modal - easy to tap */
|
||||
#createCaseModal .modal-tabs {
|
||||
gap: 0.5rem;
|
||||
padding: 0.5rem 1rem 0.75rem;
|
||||
}
|
||||
|
||||
#createCaseModal .modal-tab-btn {
|
||||
flex: 1;
|
||||
min-height: 44px;
|
||||
padding: 0.6rem 1rem;
|
||||
font-size: 0.8rem;
|
||||
font-weight: 500;
|
||||
border-radius: 8px;
|
||||
justify-content: center;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
/* Touch-friendly form inputs in case modal */
|
||||
#createCaseModal .form-row {
|
||||
@@ -3172,21 +3117,21 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
stack of separate cards: at 390px the per-card borders were most of the pixels.
|
||||
============================================================================ */
|
||||
@media (max-width: 860px) {
|
||||
#appSettingsModal .modal-content.modal-lg {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .modal-content.modal-lg {
|
||||
width: 100%;
|
||||
max-width: 100%;
|
||||
height: 100%;
|
||||
max-height: 100%;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-body {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-body {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
/* Rail keeps only its search field, laid out as a bar */
|
||||
#appSettingsModal .set-rail {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-rail {
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
border-right: 0;
|
||||
@@ -3197,37 +3142,64 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-rail-items,
|
||||
#appSettingsModal .set-rail-foot {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-rail-items,
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-rail-foot {
|
||||
display: none;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-search {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-search {
|
||||
margin: 0;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-search input {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-search input {
|
||||
padding: 9px 10px 9px 30px;
|
||||
border-radius: 10px;
|
||||
}
|
||||
|
||||
/* Save moves into the header; the bottom action bar would cost 60px */
|
||||
#appSettingsModal .set-head-save {
|
||||
display: inline-flex;
|
||||
/* Save + Close are the two ways out of the sheet (save-and-close vs
|
||||
discard-and-close), hit in the same corner with the same thumb, so here —
|
||||
and only here, since Save is header-only below 860px — they share a
|
||||
recessed tray and matching pill geometry instead of reading as a fat
|
||||
accent pill parked beside a stray × glyph. Tray colors come from skin
|
||||
tokens, never a hardcoded black alpha, or the light skins get a grey slab.
|
||||
`:has()` keeps the tray off the two sheets that carry a lone × (Session
|
||||
Options and Add Case save from inside their own forms). */
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-head-actions:has(.set-head-save) {
|
||||
padding: 3px;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 12px;
|
||||
background: var(--bg-input);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-foot {
|
||||
/* Save moves into the header; the bottom action bar would cost 60px. Both
|
||||
buttons grow to a thumb-sized target and keep identical heights so the
|
||||
pair reads as one cluster — 36 + the tray's 3px padding and 1px border on
|
||||
each side is a 44px block, the same height as the phone header. */
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-head-save {
|
||||
display: inline-flex;
|
||||
height: 36px;
|
||||
padding: 0 16px;
|
||||
font-size: 0.86rem;
|
||||
}
|
||||
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-head-actions .modal-close {
|
||||
width: 36px;
|
||||
height: 36px;
|
||||
font-size: 1.35rem;
|
||||
}
|
||||
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-foot {
|
||||
display: none;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-doc {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-doc {
|
||||
padding: 0 14px 34px;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
/* ── jump control ──────────────────────────────────────────────────── */
|
||||
#appSettingsModal .set-jump {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump {
|
||||
display: flex;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
@@ -3248,30 +3220,30 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-ico {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-ico {
|
||||
color: var(--accent);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-label {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-label {
|
||||
font-weight: 580;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-chev {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-chev {
|
||||
margin-left: auto;
|
||||
color: var(--text-muted);
|
||||
flex-shrink: 0;
|
||||
transition: transform 0.18s;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump[aria-expanded='true'] .set-jump-chev {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump[aria-expanded='true'] .set-jump-chev {
|
||||
transform: rotate(180deg);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-veil {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-veil {
|
||||
display: none;
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
@@ -3279,7 +3251,7 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
background: rgba(4, 8, 13, 0.62);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-menu {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-menu {
|
||||
display: none;
|
||||
position: absolute;
|
||||
left: 14px;
|
||||
@@ -3301,7 +3273,7 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
display: block;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-row {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 11px;
|
||||
@@ -3316,43 +3288,43 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-row svg {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-row svg {
|
||||
color: var(--text-muted);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-row .set-jump-count {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-row .set-jump-count {
|
||||
margin-left: auto;
|
||||
font-size: 0.62rem;
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-row.active {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-row.active {
|
||||
background: rgba(var(--accent-rgb), 0.14);
|
||||
color: var(--text);
|
||||
font-weight: 570;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-jump-row.active svg {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-jump-row.active svg {
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
/* ── sections step down: the jump pill already names the current one ── */
|
||||
#appSettingsModal .set-section {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-section {
|
||||
padding-top: 0;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-section + .set-section {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-section + .set-section {
|
||||
border-top: 0;
|
||||
margin-top: 0;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-section-head {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-section-head {
|
||||
gap: 7px;
|
||||
margin: 18px 0 2px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-section-head svg {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-section-head svg {
|
||||
padding: 0;
|
||||
border: 0;
|
||||
background: none;
|
||||
@@ -3361,7 +3333,7 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
height: 12px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-section-head h2 {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-section-head h2 {
|
||||
font-size: 0.6rem;
|
||||
font-weight: 640;
|
||||
letter-spacing: 0.1em;
|
||||
@@ -3369,36 +3341,45 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-section-head::after {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-section-head::after {
|
||||
content: '';
|
||||
flex: 1;
|
||||
height: 1px;
|
||||
background: linear-gradient(90deg, var(--border), transparent);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-section-blurb {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-section-blurb {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* ── live layout preview ───────────────────────────────────────────── */
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-preview {
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-preview-stage {
|
||||
min-height: 62px;
|
||||
}
|
||||
|
||||
/* ── inset grouped list ────────────────────────────────────────────── */
|
||||
#appSettingsModal .set-group {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group {
|
||||
margin-top: 14px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group + .set-group {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group + .set-group {
|
||||
margin-top: 16px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group-head {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-head {
|
||||
margin-bottom: 7px;
|
||||
padding: 0 3px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group-hint {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-hint {
|
||||
padding: 0 3px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group-body {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body {
|
||||
gap: 0;
|
||||
background: rgba(255, 255, 255, 0.035);
|
||||
border: 1px solid rgba(255, 255, 255, 0.06);
|
||||
@@ -3406,7 +3387,7 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group-body > .set-row {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body > .set-row {
|
||||
background: transparent;
|
||||
border: 0;
|
||||
border-radius: 0;
|
||||
@@ -3414,104 +3395,147 @@ html:is([data-skin="paper-gray"], [data-skin="solarized-light"], [data-skin="cat
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group-body > .set-row + .set-row {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body > .set-row + .set-row {
|
||||
border-top: 1px solid rgba(255, 255, 255, 0.055);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group-body > .set-chips,
|
||||
#appSettingsModal .set-group-body > .set-modelgrid,
|
||||
#appSettingsModal .set-group-body > .set-minigrid,
|
||||
#appSettingsModal .set-group-body > #appSettingsShortcutsList {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body > .set-chips,
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body > .set-modelgrid,
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body > .set-minigrid,
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body > #appSettingsShortcutsList {
|
||||
padding: 12px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-group-body > .event-type-grid {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-group-body > .event-type-grid {
|
||||
padding: 12px;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-row-label {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row-label {
|
||||
font-size: 0.84rem;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-row-desc {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row-desc {
|
||||
font-size: 0.69rem;
|
||||
max-width: none;
|
||||
}
|
||||
|
||||
/* Fields go full width under their label instead of fighting for the row */
|
||||
#appSettingsModal .set-row.has-field {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row.has-field {
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
gap: 9px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-row.has-field .set-select,
|
||||
#appSettingsModal .set-row.has-field .set-input {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row.has-field .set-select,
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row.has-field .set-input {
|
||||
width: 100%;
|
||||
min-width: 0;
|
||||
max-width: none;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-row-actions-wide {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row-actions-wide {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-row-actions-wide .set-input {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row-actions-wide .set-input {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-num {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-num {
|
||||
width: 76px;
|
||||
}
|
||||
|
||||
/* Bigger touch targets for the toggles and chips */
|
||||
#appSettingsModal .switch-sm {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .switch-sm {
|
||||
width: 40px;
|
||||
height: 24px;
|
||||
}
|
||||
|
||||
#appSettingsModal .switch-sm .slider:before {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .switch-sm .slider:before {
|
||||
height: 18px;
|
||||
width: 18px;
|
||||
}
|
||||
|
||||
#appSettingsModal .switch-sm input:checked + .slider:before {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .switch-sm input:checked + .slider:before {
|
||||
transform: translateX(16px);
|
||||
}
|
||||
|
||||
#appSettingsModal .set-chip {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-chip {
|
||||
font-size: 0.78rem;
|
||||
padding: 9px 14px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-modelgrid {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-modelgrid {
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-minigrid {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-minigrid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-mini .set-select {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-mini .set-select {
|
||||
width: 148px;
|
||||
}
|
||||
|
||||
/* One scrollable line beats a ragged two-row wrap for 7 effort levels */
|
||||
#appSettingsModal .set-segment {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-segment {
|
||||
overflow-x: auto;
|
||||
scrollbar-width: none;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-segment::-webkit-scrollbar {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-segment::-webkit-scrollbar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
#appSettingsModal .set-segment button {
|
||||
:is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-segment button {
|
||||
flex: 0 0 auto;
|
||||
padding: 8px 12px;
|
||||
}
|
||||
}
|
||||
|
||||
/* ============================================================================
|
||||
Session Options, compact layout (<= 860px)
|
||||
|
||||
App Settings collapses its rail and hands navigation to the sticky
|
||||
#appSettingsJump pill. Session Options has no such pill (and no search), so
|
||||
its rail stays put and becomes a horizontal, scrollable strip — which is
|
||||
what its tab bar was before the two modals started sharing a surface.
|
||||
============================================================================ */
|
||||
@media (max-width: 860px) {
|
||||
:is(#sessionOptionsModal, #createCaseModal) .set-rail {
|
||||
padding: 8px 10px;
|
||||
}
|
||||
|
||||
:is(#sessionOptionsModal, #createCaseModal) .set-rail-items {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
gap: 4px;
|
||||
overflow-x: auto;
|
||||
scrollbar-width: none;
|
||||
}
|
||||
|
||||
:is(#sessionOptionsModal, #createCaseModal) .set-rail-items::-webkit-scrollbar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
:is(#sessionOptionsModal, #createCaseModal) .set-rail-item {
|
||||
white-space: nowrap;
|
||||
padding: 8px 12px;
|
||||
}
|
||||
|
||||
/* The active marker is a left bar in the vertical rail; horizontally that
|
||||
reads as a stray tick, so the strip uses a filled pill instead. */
|
||||
:is(#sessionOptionsModal, #createCaseModal) .set-rail-item.active::before {
|
||||
display: none;
|
||||
}
|
||||
|
||||
:is(#sessionOptionsModal, #createCaseModal) .set-rail-item.active {
|
||||
background: rgba(var(--accent-rgb), 0.13);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -489,23 +489,56 @@ Object.assign(CodemanApp.prototype, {
|
||||
const date = new Date(s.lastModified);
|
||||
const timeStr = date.toLocaleDateString('en', { month: 'short', day: 'numeric' })
|
||||
+ ' ' + date.toLocaleTimeString('en', { hour: '2-digit', minute: '2-digit', hour12: false });
|
||||
const shortDir = s.workingDir.replace(/^\/home\/[^/]+\//, '~/');
|
||||
// Shared helper, not a local regex: the copy that used to live here
|
||||
// matched `/home/<user>/` only, so on macOS (`/Users/<user>/`) nothing was
|
||||
// stripped and every row spent its first ~19 characters on an identical
|
||||
// prefix — with the tail ellipsized, all rows rendered as
|
||||
// `/Users/jordanryan/co…` and became indistinguishable (#273).
|
||||
const shortDir = this._shortenHomePath(s.workingDir);
|
||||
// Lead with the folder that identifies the row; the parent path trails and
|
||||
// is what gets truncated. Truncation must never eat the identity.
|
||||
const lastSlash = shortDir.lastIndexOf('/');
|
||||
const leafName = lastSlash === -1 ? shortDir : shortDir.slice(lastSlash + 1);
|
||||
// `<repo>/.claude/worktrees` in the parent path is pure noise once the pill
|
||||
// says which worktree it is — drop it so the repo stays visible instead.
|
||||
const parentDir = (lastSlash === -1 ? '' : shortDir.slice(0, lastSlash)).replace(/\/\.claude\/worktrees$/, '');
|
||||
|
||||
const btn = document.createElement('button');
|
||||
btn.className = 'run-mode-option';
|
||||
btn.className = 'run-mode-option run-mode-hist-row';
|
||||
btn.title = s.workingDir;
|
||||
btn.dataset.sessionId = s.sessionId;
|
||||
btn.dataset.workingDir = s.workingDir;
|
||||
|
||||
const dirSpan = document.createElement('span');
|
||||
dirSpan.className = 'hist-dir';
|
||||
dirSpan.textContent = shortDir;
|
||||
const nameSpan = document.createElement('span');
|
||||
nameSpan.className = 'hist-name';
|
||||
nameSpan.textContent = leafName;
|
||||
|
||||
const parts = [nameSpan];
|
||||
|
||||
// Worktree pill, same data the session rows use (#266). A worktree's
|
||||
// directory basename is often just the worktree name, so without this two
|
||||
// worktrees of one repo still read alike.
|
||||
const wt = this._worktreeLabel ? this._worktreeLabel(s) : '';
|
||||
if (wt) {
|
||||
const wtSpan = document.createElement('span');
|
||||
wtSpan.className = 'hist-wt';
|
||||
wtSpan.textContent = wt;
|
||||
parts.push(wtSpan);
|
||||
}
|
||||
|
||||
if (parentDir) {
|
||||
const dirSpan = document.createElement('span');
|
||||
dirSpan.className = 'hist-dir';
|
||||
dirSpan.textContent = parentDir;
|
||||
parts.push(dirSpan);
|
||||
}
|
||||
|
||||
const metaSpan = document.createElement('span');
|
||||
metaSpan.className = 'hist-meta';
|
||||
metaSpan.textContent = timeStr;
|
||||
parts.push(metaSpan);
|
||||
|
||||
btn.append(dirSpan, metaSpan);
|
||||
btn.append(...parts);
|
||||
btn.addEventListener('click', (e) => {
|
||||
e.stopPropagation();
|
||||
this.resumeHistorySession(s.sessionId, s.workingDir, s.name);
|
||||
@@ -1278,8 +1311,8 @@ Object.assign(CodemanApp.prototype, {
|
||||
document.getElementById('presetDescriptionHint').textContent = '';
|
||||
|
||||
// Hide Ralph/Todo tab and Respawn tab for external CLI sessions (not supported)
|
||||
const ralphTabBtn = document.querySelector('#sessionOptionsModal .modal-tab-btn[data-tab="ralph"]');
|
||||
const respawnTabBtn = document.querySelector('#sessionOptionsModal .modal-tab-btn[data-tab="respawn"]');
|
||||
const ralphTabBtn = document.querySelector('#sessionOptionsModal .set-rail-item[data-tab="ralph"]');
|
||||
const respawnTabBtn = document.querySelector('#sessionOptionsModal .set-rail-item[data-tab="respawn"]');
|
||||
if (isExternalCli) {
|
||||
if (ralphTabBtn) ralphTabBtn.style.display = 'none';
|
||||
if (respawnTabBtn) respawnTabBtn.style.display = 'none';
|
||||
@@ -1303,6 +1336,18 @@ Object.assign(CodemanApp.prototype, {
|
||||
}
|
||||
|
||||
const modal = document.getElementById('sessionOptionsModal');
|
||||
|
||||
// Chips mirror their checkbox onto the label, the same way App Settings does
|
||||
// (settings-ui.js: _syncSettingsChips). Registered once per page, never per
|
||||
// open, or a long-lived tab accumulates one listener per visit.
|
||||
if (modal.dataset.chipsReady !== '1') {
|
||||
modal.dataset.chipsReady = '1';
|
||||
modal.addEventListener('change', e => {
|
||||
if (e.target?.closest?.('.set-chip')) this._syncSettingsChips();
|
||||
});
|
||||
}
|
||||
this._syncSettingsChips();
|
||||
|
||||
modal.classList.add('active');
|
||||
|
||||
// Activate focus trap
|
||||
@@ -1500,18 +1545,32 @@ Object.assign(CodemanApp.prototype, {
|
||||
// Session Options Modal Tabs
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
|
||||
/**
|
||||
* Show one section of the Session Options modal.
|
||||
*
|
||||
* The chrome is the shared `set-*` settings surface, but unlike App Settings
|
||||
* (whose rail is a table of contents over one scrolling document) this rail
|
||||
* is a real switcher: exactly one `.set-section` is visible and the rest
|
||||
* carry `.hidden`. Summary owns its own scroller and Respawn is long, so
|
||||
* stacking them into a single document would bury both.
|
||||
*/
|
||||
switchOptionsTab(tabName) {
|
||||
// Toggle active class on tab buttons
|
||||
document.querySelectorAll('#sessionOptionsModal .modal-tab-btn').forEach(btn => {
|
||||
// Toggle active class on rail entries
|
||||
document.querySelectorAll('#sessionOptionsModal .set-rail-item').forEach(btn => {
|
||||
btn.classList.toggle('active', btn.dataset.tab === tabName);
|
||||
});
|
||||
|
||||
// Toggle hidden class on tab content
|
||||
// Toggle hidden class on the sections
|
||||
document.getElementById('respawn-tab').classList.toggle('hidden', tabName !== 'respawn');
|
||||
document.getElementById('context-tab').classList.toggle('hidden', tabName !== 'context');
|
||||
document.getElementById('ralph-tab').classList.toggle('hidden', tabName !== 'ralph');
|
||||
document.getElementById('summary-tab').classList.toggle('hidden', tabName !== 'summary');
|
||||
|
||||
// A switched-to section starts at its own top, not at the scroll offset the
|
||||
// previous one was left at.
|
||||
const doc = document.getElementById('sessionOptionsDoc');
|
||||
if (doc) doc.scrollTop = 0;
|
||||
|
||||
// Load run summary data when switching to summary tab
|
||||
if (tabName === 'summary' && this.editingSessionId) {
|
||||
this.loadRunSummary(this.editingSessionId);
|
||||
@@ -1782,7 +1841,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
this.switchCaseModalTab('case-create');
|
||||
// Wire up tab buttons
|
||||
const modal = document.getElementById('createCaseModal');
|
||||
modal.querySelectorAll('.modal-tabs .modal-tab-btn').forEach(btn => {
|
||||
modal.querySelectorAll('.set-rail-item').forEach(btn => {
|
||||
btn.onclick = () => this.switchCaseModalTab(btn.dataset.tab);
|
||||
});
|
||||
// Scroll-into-view on focus for mobile keyboard visibility
|
||||
@@ -1803,14 +1862,17 @@ Object.assign(CodemanApp.prototype, {
|
||||
switchCaseModalTab(tabName) {
|
||||
this.caseModalTab = tabName;
|
||||
const modal = document.getElementById('createCaseModal');
|
||||
// Toggle active class on tab buttons
|
||||
modal.querySelectorAll('.modal-tabs .modal-tab-btn').forEach(btn => {
|
||||
// Toggle active class on rail entries
|
||||
modal.querySelectorAll('.set-rail-item').forEach(btn => {
|
||||
btn.classList.toggle('active', btn.dataset.tab === tabName);
|
||||
});
|
||||
// Toggle hidden class on tab content
|
||||
modal.querySelectorAll('.modal-tab-content').forEach(content => {
|
||||
// Toggle hidden class on the panels
|
||||
modal.querySelectorAll('.set-section').forEach(content => {
|
||||
content.classList.toggle('hidden', content.id !== tabName);
|
||||
});
|
||||
// A switched-to panel starts at its own top.
|
||||
const doc = document.getElementById('createCaseDoc');
|
||||
if (doc) doc.scrollTop = 0;
|
||||
// Update submit button (hide for manage tab)
|
||||
const submitBtn = document.getElementById('caseModalSubmit');
|
||||
if (tabName === 'case-manage') {
|
||||
@@ -2676,7 +2738,9 @@ Object.assign(CodemanApp.prototype, {
|
||||
cases.forEach((c, idx) => {
|
||||
const isFirst = idx === 0;
|
||||
const isLast = idx === cases.length - 1;
|
||||
const pathDisplay = c.path ? c.path.replace(/^\/Users\/[^/]+/, '~') : '';
|
||||
// Was `/Users/<user>` only, the mirror image of the Run menu's bug: every
|
||||
// case path on a Linux host rendered in full, unabbreviated.
|
||||
const pathDisplay = c.path ? this._shortenHomePath(c.path) : '';
|
||||
html += `
|
||||
<div class="case-manage-item" data-case="${escapeHtml(c.name)}">
|
||||
<div class="case-manage-info">
|
||||
|
||||
+133
-10
@@ -482,17 +482,18 @@ Object.assign(CodemanApp.prototype, {
|
||||
const voiceCfg = VoiceInput._getDeepgramConfig();
|
||||
document.getElementById('voiceDeepgramKey').value = voiceCfg.apiKey || '';
|
||||
document.getElementById('voiceLanguage').value = voiceCfg.language || 'en-US';
|
||||
document.getElementById('voiceKeyterms').value = voiceCfg.keyterms || 'refactor, endpoint, middleware, callback, async, regex, TypeScript, npm, API, deploy, config, linter, env, webhook, schema, CLI, JSON, CSS, DOM, SSE, backend, frontend, localhost, dependencies, repository, merge, rebase, diff, commit, com';
|
||||
document.getElementById('voiceKeyterms').value = voiceCfg.keyterms || DEFAULT_VOICE_KEYTERMS;
|
||||
document.getElementById('voiceInsertMode').value = voiceCfg.insertMode || 'direct';
|
||||
document.getElementById('voiceProvider').value = voiceCfg.provider || 'auto';
|
||||
document.getElementById('appSettingsClaudeVoice').checked = settings.claudeVoiceEnabled ?? false;
|
||||
// Reset key visibility to hidden
|
||||
const keyInput = document.getElementById('voiceDeepgramKey');
|
||||
keyInput.type = 'password';
|
||||
document.getElementById('voiceKeyToggleBtn').textContent = 'Show';
|
||||
// Update provider status
|
||||
const providerName = VoiceInput.getActiveProviderName();
|
||||
const providerEl = document.getElementById('voiceProviderStatus');
|
||||
providerEl.textContent = providerName;
|
||||
providerEl.className = 'voice-provider-status' + (providerName.startsWith('Deepgram') ? ' active' : '');
|
||||
// Update provider status. The Claude row needs a fresh server probe: the
|
||||
// setting is synced, so another device may have flipped it since page load.
|
||||
this._renderVoiceProviderStatus();
|
||||
VoiceInput.refreshClaudeStatus().then(() => this._renderVoiceProviderStatus());
|
||||
|
||||
// Updates section — show current version, reset transient result/progress UI.
|
||||
this._initUpdatesSection();
|
||||
@@ -503,8 +504,11 @@ Object.assign(CodemanApp.prototype, {
|
||||
this._syncSettingsChips();
|
||||
this._syncModelCards();
|
||||
this._syncEffortSegment();
|
||||
// Back to the top of the document (one scroll, not a tab reset).
|
||||
this.switchSettingsTab('settings-terminal');
|
||||
// Back to the top of the document (one scroll, not a tab reset). Updates is
|
||||
// first now: the version this install is running, and whether a newer one is
|
||||
// waiting, are the two things worth seeing before any preference. The rest of
|
||||
// the system settings (paths, automation, remote access) tail the document.
|
||||
this.switchSettingsTab('settings-updates');
|
||||
const modal = document.getElementById('appSettingsModal');
|
||||
modal.classList.add('active');
|
||||
|
||||
@@ -687,11 +691,89 @@ Object.assign(CodemanApp.prototype, {
|
||||
document.getElementById('appSettingsJump')?.setAttribute('aria-expanded', open ? 'true' : 'false');
|
||||
},
|
||||
|
||||
/** Mirror checkbox state onto the chip labels (see _initSettingsNav). */
|
||||
/**
|
||||
* Mirror checkbox state onto the chip labels (see _initSettingsNav).
|
||||
*
|
||||
* Covers Session Options too: it shares the `set-*` surface, and its cycle-step
|
||||
* chips would otherwise depend on `:has()` alone for their checked styling.
|
||||
*/
|
||||
_syncSettingsChips() {
|
||||
document.querySelectorAll('#appSettingsModal .set-chip').forEach(chip => {
|
||||
document.querySelectorAll('#appSettingsModal .set-chip, #sessionOptionsModal .set-chip').forEach(chip => {
|
||||
chip.classList.toggle('is-on', !!chip.querySelector('input')?.checked);
|
||||
});
|
||||
this._syncLayoutPreview();
|
||||
},
|
||||
|
||||
/**
|
||||
* Redraw the Header & Panels live preview from the chips above it.
|
||||
*
|
||||
* The preview is a scale model of the app, not a second list of settings, so
|
||||
* every icon is CLONED from the chip that owns it (`.set-chip-ico`): each icon
|
||||
* has exactly ONE copy in index.html and a chip can never drift from the button
|
||||
* it previews. A chip joins the preview purely by carrying `data-preview`
|
||||
* (which slot) and `data-preview-order` (where in that slot); nothing here
|
||||
* needs to know the setting's name.
|
||||
*
|
||||
* `data-preview-text` replaces the icon with a text token for the header
|
||||
* entries that are readouts rather than buttons (plan usage, CPU, font size).
|
||||
*/
|
||||
_syncLayoutPreview() {
|
||||
const modal = document.getElementById('appSettingsModal');
|
||||
if (!modal || typeof modal.querySelectorAll !== 'function') return;
|
||||
const slots = {
|
||||
header: document.getElementById('appSettingsPreviewHeader'),
|
||||
panel: document.getElementById('appSettingsPreviewPanels'),
|
||||
toolbar: document.getElementById('appSettingsPreviewToolbar'),
|
||||
float: document.getElementById('appSettingsPreviewFloats'),
|
||||
};
|
||||
if (!slots.header) return;
|
||||
Object.values(slots).forEach(el => {
|
||||
if (el) el.innerHTML = '';
|
||||
});
|
||||
|
||||
const chips = [...modal.querySelectorAll('.set-chip[data-preview]')]
|
||||
.filter(chip => chip.querySelector('input')?.checked)
|
||||
.sort((a, b) => (Number(a.dataset.previewOrder) || 0) - (Number(b.dataset.previewOrder) || 0));
|
||||
|
||||
let shown = 0;
|
||||
for (const chip of chips) {
|
||||
const kind = chip.dataset.preview;
|
||||
const slot = slots[kind];
|
||||
if (!slot) continue;
|
||||
// The label is the chip's own text; the icon span (if any) is skipped by
|
||||
// taking the LAST span, which is always the label.
|
||||
const spans = chip.querySelectorAll('span');
|
||||
const label = (spans[spans.length - 1]?.textContent || '').trim();
|
||||
const el = document.createElement('span');
|
||||
el.title = label;
|
||||
if (kind === 'header') {
|
||||
const text = chip.dataset.previewText;
|
||||
el.className = text ? 'set-preview-chip' : 'set-preview-btn';
|
||||
if (text) el.textContent = text;
|
||||
else this._appendPreviewIcon(el, chip);
|
||||
} else {
|
||||
el.className = `set-preview-${kind}`;
|
||||
this._appendPreviewIcon(el, chip);
|
||||
const name = document.createElement('span');
|
||||
name.textContent = label;
|
||||
el.appendChild(name);
|
||||
}
|
||||
slot.appendChild(el);
|
||||
shown++;
|
||||
}
|
||||
|
||||
const empty = document.getElementById('appSettingsPreviewEmpty');
|
||||
if (empty) empty.hidden = shown > 0;
|
||||
},
|
||||
|
||||
/** Clone a chip's icon into a preview element (see _syncLayoutPreview). */
|
||||
_appendPreviewIcon(target, chip) {
|
||||
const icon = chip.querySelector('.set-chip-ico');
|
||||
if (!icon) return;
|
||||
const clone = icon.cloneNode(true);
|
||||
clone.classList.remove('set-chip-ico');
|
||||
clone.classList.add('set-preview-ico');
|
||||
target.appendChild(clone);
|
||||
},
|
||||
|
||||
/**
|
||||
@@ -860,6 +942,10 @@ Object.assign(CodemanApp.prototype, {
|
||||
section.classList.toggle('set-hit-hidden', !!q && !hasVisible);
|
||||
});
|
||||
|
||||
// The live preview sits outside any group, so it survives the sweep above;
|
||||
// a search is asking for one row, not for the scale model around it.
|
||||
doc.querySelectorAll('.set-preview').forEach(pv => pv.classList.toggle('set-hit-hidden', !!q));
|
||||
|
||||
const empty = document.getElementById('appSettingsSearchEmpty');
|
||||
if (empty) empty.hidden = !q || anyVisible;
|
||||
if (!q) doc.querySelectorAll('.set-group-advanced').forEach(g => g.classList.remove('open'));
|
||||
@@ -1830,6 +1916,37 @@ Object.assign(CodemanApp.prototype, {
|
||||
}
|
||||
},
|
||||
|
||||
/**
|
||||
* Paint both Voice status rows: which provider a mic press would use, and what
|
||||
* the server reports about its Claude login. Called on open and again once the
|
||||
* /api/voice/status probe resolves.
|
||||
*/
|
||||
_renderVoiceProviderStatus() {
|
||||
const providerEl = document.getElementById('voiceProviderStatus');
|
||||
if (providerEl) {
|
||||
const providerName = VoiceInput.getActiveProviderName();
|
||||
providerEl.textContent = providerName;
|
||||
const live = providerName.startsWith('Deepgram Nova') || providerName.startsWith('Claude (this');
|
||||
providerEl.className = 'voice-provider-status' + (live ? ' active' : '');
|
||||
}
|
||||
const claudeEl = document.getElementById('voiceClaudeStatus');
|
||||
if (!claudeEl) return;
|
||||
const status = VoiceInput._claudeStatus;
|
||||
const text = !status
|
||||
? 'Checking...'
|
||||
: status.available
|
||||
? `Ready${status.subscriptionType ? ` (${status.subscriptionType})` : ''}`
|
||||
: status.reason === 'expired'
|
||||
? 'Login expired - run a Claude session to refresh'
|
||||
: status.reason === 'no-credentials'
|
||||
? 'No Claude Code login on the server'
|
||||
: status.reason === 'malformed'
|
||||
? 'Claude credentials unreadable'
|
||||
: 'Off - enable it above';
|
||||
claudeEl.textContent = text;
|
||||
claudeEl.className = 'voice-provider-status' + (status?.available ? ' active' : '');
|
||||
},
|
||||
|
||||
async saveAppSettings() {
|
||||
// Gesture overlay is injected at page render (server-side), so a change to it
|
||||
// only takes effect on reload — remember the prior value to decide below.
|
||||
@@ -1892,6 +2009,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
// Claude Permissions settings
|
||||
agentTeamsEnabled: document.getElementById('appSettingsAgentTeams').checked,
|
||||
agentSkillEnabled: document.getElementById('appSettingsAgentSkill').checked,
|
||||
claudeVoiceEnabled: document.getElementById('appSettingsClaudeVoice').checked,
|
||||
claudeModel: document.getElementById('appSettingsClaudeModel').value,
|
||||
opusContext1mEnabled: document.getElementById('appSettingsOpusContext1m').checked,
|
||||
remoteAutoReconnect: document.getElementById('appSettingsRemoteAutoReconnect').checked,
|
||||
@@ -1931,6 +2049,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
|
||||
// Save voice settings to localStorage + include in server payload for cross-device sync
|
||||
const voiceSettings = {
|
||||
provider: document.getElementById('voiceProvider').value,
|
||||
apiKey: document.getElementById('voiceDeepgramKey').value.trim(),
|
||||
language: document.getElementById('voiceLanguage').value,
|
||||
keyterms: document.getElementById('voiceKeyterms').value.trim(),
|
||||
@@ -2110,6 +2229,10 @@ Object.assign(CodemanApp.prototype, {
|
||||
|
||||
this.closeAppSettings();
|
||||
|
||||
// Voice availability is a server-side answer, so re-probe after a save:
|
||||
// otherwise the mic keeps using the pre-save provider until the next reload.
|
||||
VoiceInput.refreshClaudeStatus();
|
||||
|
||||
// The gesture overlay is injected at page render (server reads
|
||||
// gestureControlEnabled from settings.json), so a change only takes effect on
|
||||
// reload. Reload when it actually changed — the server PUT above already
|
||||
|
||||
+1207
-195
File diff suppressed because it is too large
Load Diff
+386
-39
@@ -32,6 +32,28 @@
|
||||
// short window, only the app's synthetic tap-to-position mouse event should
|
||||
// reach xterm.
|
||||
const TOUCH_COMPAT_MOUSE_SUPPRESS_MS = 450;
|
||||
// Finger travel (px) still counted as a tap rather than a scroll. Shared by
|
||||
// the terminal's own touch handling (TAP_THRESHOLD, initTerminal) and the
|
||||
// keyboard-dismiss handler (_installMobileKeyboardDismiss), which MUST agree:
|
||||
// a gesture the terminal treats as a scroll but the dismiss handler treats as
|
||||
// a tap would close the keyboard mid-scroll and drop the composer.
|
||||
const MOBILE_KEYBOARD_DISMISS_TAP_SLOP = 8;
|
||||
// Regions where a tap must NOT dismiss the on-screen keyboard
|
||||
// (_installMobileKeyboardDismiss). Two groups: anything that is about to take
|
||||
// focus itself, and the accessory bar, which is built to be used while the
|
||||
// keyboard is open.
|
||||
const MOBILE_KEYBOARD_DISMISS_EXEMPT_SELECTOR = [
|
||||
'input',
|
||||
'textarea',
|
||||
'select',
|
||||
'button',
|
||||
'a[href]',
|
||||
'[contenteditable=""]',
|
||||
'[contenteditable="true"]',
|
||||
'[tabindex]:not([tabindex="-1"])',
|
||||
'.keyboard-accessory-bar',
|
||||
'.path-picker-overlay',
|
||||
].join(',');
|
||||
// Escape sequences occupy no terminal cells, so they must come out before a
|
||||
// captured line's WIDTH can be measured (_estimateReplayRows). Covers OSC,
|
||||
// CSI, charset designators and the short escapes tmux emits; deliberately
|
||||
@@ -53,6 +75,7 @@
|
||||
// Bound on page keys emitted from one gesture batch, mirroring the SGR tick
|
||||
// cap: a fling must not build a backlog that keeps paging after it stops.
|
||||
const PAGE_KEY_MAX_PER_BATCH = 3;
|
||||
const TUI_PROMPT_DEFAULT_ROWS_FROM_BOTTOM = 4;
|
||||
// Composer navigation keys as xterm.js encodes user keystrokes: plain and
|
||||
// modified arrows (CSI A-D, CSI 1;mA-D, SS3 A-D), Home/End (CSI H/F, SS3
|
||||
// H/F, CSI 1~/4~), Insert/Delete/PgUp/PgDn (CSI 2~/3~/5~/6~, optional
|
||||
@@ -180,6 +203,9 @@
|
||||
KEY_PAGE_DOWN,
|
||||
PAGE_KEY_SCREEN_FRACTION,
|
||||
PAGE_KEY_MAX_PER_BATCH,
|
||||
TUI_PROMPT_DEFAULT_ROWS_FROM_BOTTOM,
|
||||
MOBILE_KEYBOARD_DISMISS_EXEMPT_SELECTOR,
|
||||
MOBILE_KEYBOARD_DISMISS_TAP_SLOP,
|
||||
};
|
||||
global.CODEMAN_XTERM_THEMES = CODEMAN_XTERM_THEMES;
|
||||
global.codemanCurrentXtermTheme = currentXtermTheme;
|
||||
@@ -650,7 +676,11 @@ Object.assign(CodemanApp.prototype, {
|
||||
|
||||
let didScroll = false; // track whether touchmove fired (tap vs scroll)
|
||||
let touchStartY = 0;
|
||||
const TAP_THRESHOLD = 8; // px — ignore micro-drift to distinguish tap from scroll
|
||||
let tapStartedWithTerminalFocus = false;
|
||||
let tapStartIntentCache = null;
|
||||
// px — ignore micro-drift to distinguish tap from scroll. Shared with the
|
||||
// keyboard-dismiss handler so both classify the same gesture the same way.
|
||||
const TAP_THRESHOLD = window.CodemanTerminalInput.MOBILE_KEYBOARD_DISMISS_TAP_SLOP;
|
||||
container.addEventListener(
|
||||
'touchstart',
|
||||
(ev) => {
|
||||
@@ -662,6 +692,28 @@ Object.assign(CodemanApp.prototype, {
|
||||
pixelAccum = 0;
|
||||
isTouching = true;
|
||||
didScroll = false;
|
||||
tapStartedWithTerminalFocus = this._isMobileTerminalInputFocused();
|
||||
// Classifying scans the whole viewport with translateToString, and
|
||||
// this runs at the start of EVERY gesture including scroll drags.
|
||||
// Cache the result for the touchend of this same gesture rather than
|
||||
// recomputing it; the cache is keyed on the exact start coordinates
|
||||
// so a finger that moved re-classifies at its real position.
|
||||
const touchStartIntent = this._classifyMobileTerminalTap(touchLastX, touchLastY);
|
||||
tapStartIntentCache = { x: touchLastX, y: touchLastY, intent: touchStartIntent };
|
||||
if (touchStartIntent === 'content') {
|
||||
// Cancel xterm/browser focus before the compatibility click can
|
||||
// open the OS keyboard. Content taps are re-emitted as SGR on
|
||||
// touchend.
|
||||
//
|
||||
// 'history' is deliberately NOT included. A scrolled-up viewport
|
||||
// sends nothing, so there is no compatibility click worth
|
||||
// cancelling — and preventDefault() here, paired with touchend's
|
||||
// early return, closes both routes to focus at once. Since
|
||||
// selectSession() ends with scrollToLastNonEmptyLine(), that made
|
||||
// the keyboard unreachable after every tab switch.
|
||||
ev.preventDefault();
|
||||
this._blurMobileTerminalInput();
|
||||
}
|
||||
lastTime = 0;
|
||||
if (scrollFrame) {
|
||||
cancelAnimationFrame(scrollFrame);
|
||||
@@ -669,7 +721,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
}
|
||||
}
|
||||
},
|
||||
{ passive: true }
|
||||
{ passive: false }
|
||||
);
|
||||
|
||||
container.addEventListener(
|
||||
@@ -721,44 +773,19 @@ Object.assign(CodemanApp.prototype, {
|
||||
scrollFrame = requestAnimationFrame(scrollLoop);
|
||||
}
|
||||
if (!didScroll && this.terminal) {
|
||||
// ── Tap-to-position cursor ──────────────────────────────────
|
||||
// Synthesize a click from the real touch point so the foreground app
|
||||
// moves its cursor to the tapped cell (iOS doesn't reliably do this
|
||||
// itself under touch-action:none). CRITICAL: only when mouse tracking
|
||||
// is ON. xterm disables its local SelectionService while mouse events
|
||||
// are active, so the synthetic click is forwarded to the PTY as an SGR
|
||||
// report (cursor moves). But when tracking is OFF, that same click
|
||||
// drives xterm's LOCAL selection (detail 1/2/3 → char/word/line) — a
|
||||
// tap on CJK text would select & copy it instead of positioning. So
|
||||
// gate strictly on the live mouse-tracking mode.
|
||||
const touch = ev.changedTouches && ev.changedTouches[0];
|
||||
const mouseMode = this.terminal.modes?.mouseTrackingMode;
|
||||
const mouseTrackingOn = !!mouseMode && mouseMode !== 'none';
|
||||
if (touch) {
|
||||
this._suppressTrustedTapMouseEvents();
|
||||
}
|
||||
if (touch && mouseTrackingOn) {
|
||||
this._dispatchSyntheticTerminalClick(touch.clientX, touch.clientY);
|
||||
} else if (touch && this._sessionUsesServerMouseStrip()) {
|
||||
// The server strips mouse-tracking DECSETs from claude/codex/gemini
|
||||
// output (isAltScreenStripMode, session.ts) so the wheel keeps
|
||||
// scrolling scrollback — which leaves THIS xterm permanently at
|
||||
// mouseTrackingMode 'none' even though the TUI on the PTY side has
|
||||
// tracking ON and still understands SGR reports. Encode the report
|
||||
// ourselves and send it straight to the PTY: no DOM click is
|
||||
// dispatched, so xterm's local selection can't trigger either.
|
||||
this._sendSyntheticSgrTap(touch.clientX, touch.clientY);
|
||||
}
|
||||
this._syncMobileHelperTextareaToCursor();
|
||||
// Route subsequent typing to the right place: keep the CJK input
|
||||
// field focused when Chinese input is on, otherwise the terminal.
|
||||
const cjkInput = document.getElementById('cjkInput');
|
||||
if (cjkInput?.classList.contains('cjk-input-visible')) {
|
||||
cjkInput.focus();
|
||||
} else {
|
||||
this.terminal.focus();
|
||||
const cached =
|
||||
tapStartIntentCache &&
|
||||
tapStartIntentCache.x === touch.clientX &&
|
||||
tapStartIntentCache.y === touch.clientY
|
||||
? tapStartIntentCache.intent
|
||||
: null;
|
||||
this._handleMobileTerminalTap(touch, tapStartedWithTerminalFocus, cached);
|
||||
}
|
||||
}
|
||||
tapStartedWithTerminalFocus = false;
|
||||
},
|
||||
{ passive: true }
|
||||
);
|
||||
@@ -769,6 +796,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
isTouching = false;
|
||||
velocity = 0;
|
||||
pixelAccum = 0;
|
||||
tapStartedWithTerminalFocus = false;
|
||||
},
|
||||
{ passive: true }
|
||||
);
|
||||
@@ -784,6 +812,8 @@ Object.assign(CodemanApp.prototype, {
|
||||
// Hand-encode the SGR report for plain left-clicks on those sessions.
|
||||
container.addEventListener('click', (ev) => this._handleDesktopTerminalClick(ev));
|
||||
|
||||
this._installMobileKeyboardDismiss();
|
||||
|
||||
// Welcome message
|
||||
this.showWelcome();
|
||||
|
||||
@@ -1610,11 +1640,21 @@ Object.assign(CodemanApp.prototype, {
|
||||
return workingDir.split('/').pop() || workingDir;
|
||||
},
|
||||
|
||||
/** Normalize home prefixes to "~/" on both Linux and macOS */
|
||||
/**
|
||||
* Normalize a home prefix to "~" on both Linux (`/home/<user>`) and macOS
|
||||
* (`/Users/<user>`). The lookahead lets the home directory ITSELF match, so a
|
||||
* path that is exactly `$HOME` renders "~" instead of being left raw.
|
||||
*
|
||||
* This is the only place that pattern belongs. Two hand-rolled copies had
|
||||
* drifted, each broken on the platform its author was not using: the Run
|
||||
* menu's matched `/home/` only, so on macOS nothing was stripped and every
|
||||
* Recent Sessions row spent its first ~19 characters on an identical
|
||||
* `/Users/<user>/` prefix (#273); the case-manage list's matched `/Users/`
|
||||
* only, so no Linux path was ever abbreviated there. Route new path labels
|
||||
* through here rather than writing a third copy.
|
||||
*/
|
||||
_shortenHomePath(p) {
|
||||
return (p || '')
|
||||
.replace(/^\/home\/[^/]+\//, '~/')
|
||||
.replace(/^\/Users\/[^/]+\//, '~/');
|
||||
return (p || '').replace(/^\/(?:home|Users)\/[^/]+(?=\/|$)/, '~');
|
||||
},
|
||||
|
||||
/**
|
||||
@@ -3363,6 +3403,313 @@ Object.assign(CodemanApp.prototype, {
|
||||
} catch {}
|
||||
},
|
||||
|
||||
_isMobileTerminalInputFocused() {
|
||||
const active = document.activeElement;
|
||||
return (
|
||||
active === this.terminal?.textarea ||
|
||||
active?.classList?.contains('xterm-helper-textarea') ||
|
||||
active?.id === 'cjkInput'
|
||||
);
|
||||
},
|
||||
|
||||
/**
|
||||
* Separate terminal input from TUI-owned content on touch devices. A hidden
|
||||
* keyboard must not consume taps on expandable readbacks, tool results, or
|
||||
* decision rows; those taps belong to the foreground CLI. The visible prompt
|
||||
* row remains the deliberate keyboard target.
|
||||
*/
|
||||
_classifyMobileTerminalTap(clientX, clientY) {
|
||||
if (!this._terminalViewportAtBottom()) return 'history';
|
||||
|
||||
const pos = this._clientPointToCell(clientX, clientY);
|
||||
if (!pos || !this.terminal) return 'input';
|
||||
|
||||
const mouseMode = this.terminal.modes?.mouseTrackingMode;
|
||||
const mouseTrackingOn = !!mouseMode && mouseMode !== 'none';
|
||||
if (!mouseTrackingOn && !this._sessionUsesServerMouseStrip()) return 'input';
|
||||
|
||||
const buffer = this.terminal.buffer?.active;
|
||||
if (!buffer?.getLine) return 'input';
|
||||
|
||||
const rows = Math.max(1, this.terminal.rows || 1);
|
||||
const lines = [];
|
||||
const wrappedRows = [];
|
||||
let hasVisibleContent = false;
|
||||
for (let row = 0; row < rows; row++) {
|
||||
const line = buffer.getLine(buffer.viewportY + row);
|
||||
const text = line?.translateToString?.(true) || '';
|
||||
lines.push(text);
|
||||
wrappedRows.push(Boolean(line?.isWrapped));
|
||||
if (text.trim()) hasVisibleContent = true;
|
||||
}
|
||||
if (!hasVisibleContent) return 'input';
|
||||
|
||||
const cursorRow = Math.max(0, Math.min(rows - 1, buffer.cursorY || 0));
|
||||
const mode = this.sessions?.get(this.activeSessionId)?.mode || 'claude';
|
||||
let promptRow = -1;
|
||||
let menuSelectionVisible = false;
|
||||
|
||||
if (mode === 'opencode') {
|
||||
if (lines[cursorRow]?.includes('\u2503')) promptRow = cursorRow;
|
||||
} else {
|
||||
for (let row = rows - 1; row >= 0; row--) {
|
||||
const promptMatch = lines[row].match(/^\s*[❯›]/);
|
||||
if (!promptMatch) continue;
|
||||
const tail = lines[row].slice(promptMatch[0].length).trim();
|
||||
// A highlighted numbered choice is a menu row, not an editable prompt.
|
||||
const hasSiblingChoice = lines.some(
|
||||
(line, choiceRow) => choiceRow !== row && /^\s+\d+[.)]\s/.test(line)
|
||||
);
|
||||
if (/^\d+[.)]\s/.test(tail) && hasSiblingChoice) {
|
||||
menuSelectionVisible = true;
|
||||
break;
|
||||
}
|
||||
promptRow = row;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
const tappedRow = pos.row - 1;
|
||||
let logicalLineStart = tappedRow;
|
||||
while (logicalLineStart > 0 && wrappedRows[logicalLineStart]) logicalLineStart--;
|
||||
let logicalLineEnd = tappedRow;
|
||||
while (logicalLineEnd + 1 < rows && wrappedRows[logicalLineEnd + 1]) logicalLineEnd++;
|
||||
const tappedLine = lines.slice(logicalLineStart, logicalLineEnd + 1).join('');
|
||||
// Claude's status row is TUI-owned: tapping it opens the teammate view, so it
|
||||
// must not be treated as a keyboard target. Match the AFFORDANCE, not the
|
||||
// wording — the bullet and verb are both unstable (claude 2.1.226 prints
|
||||
// "✻ Cooked for 2m 6s", "✻ Baked for 9m 47s"; earlier builds printed
|
||||
// "• Working …"), while "esc to interrupt" / "background" are what make the
|
||||
// row actionable in the first place.
|
||||
if (mode === 'claude' && /\b(?:esc to interrupt|background)\b/i.test(tappedLine)) {
|
||||
return 'content';
|
||||
}
|
||||
if (menuSelectionVisible) return 'content';
|
||||
if (promptRow >= 0) {
|
||||
const inputEnd = cursorRow >= promptRow ? cursorRow : promptRow;
|
||||
if (tappedRow >= promptRow && tappedRow <= inputEnd) return 'input';
|
||||
} else if (
|
||||
tappedRow === cursorRow ||
|
||||
tappedRow >=
|
||||
Math.max(
|
||||
0,
|
||||
rows -
|
||||
window.CodemanTerminalInput
|
||||
.TUI_PROMPT_DEFAULT_ROWS_FROM_BOTTOM
|
||||
)
|
||||
) {
|
||||
// During redraws a CLI can temporarily omit its prompt marker or place
|
||||
// the cursor above a status footer. Keep the live cursor and a stable
|
||||
// lower-screen focus band usable without turning transcript rows above
|
||||
// that band into keyboard targets.
|
||||
return 'input';
|
||||
}
|
||||
|
||||
return 'content';
|
||||
},
|
||||
|
||||
_blurMobileTerminalInput() {
|
||||
const active = document.activeElement;
|
||||
if (
|
||||
active === this.terminal?.textarea ||
|
||||
active?.classList?.contains('xterm-helper-textarea') ||
|
||||
active?.id === 'cjkInput'
|
||||
) {
|
||||
active.blur?.();
|
||||
}
|
||||
},
|
||||
|
||||
/**
|
||||
* Tapping outside the terminal closes the on-screen keyboard.
|
||||
*
|
||||
* The terminal keeps focus on a hidden textarea, and nothing ever released it:
|
||||
* once the keyboard was up, every tap on the header, the tab strip or empty
|
||||
* page chrome left it up, covering half a phone screen with no way to dismiss
|
||||
* it but the OS back gesture.
|
||||
*
|
||||
* Deliberately narrow, because focus is not ours to steal:
|
||||
*
|
||||
* - only when the terminal input actually holds focus;
|
||||
* - never for a tap inside the terminal — those are classified and routed by
|
||||
* `_handleMobileTerminalTap`, which owns that decision;
|
||||
* - never for a tap on another control. Anything focusable or clickable is
|
||||
* about to take focus itself, and the accessory bar in particular exists to
|
||||
* be used WHILE the keyboard is open, so dismissing there would fight the
|
||||
* user. `closest()` covers taps landing on a child (an icon inside a button).
|
||||
*
|
||||
* Bound to `touchend` rather than `click`: a tap that dismisses the keyboard
|
||||
* usually is not meant to activate whatever is underneath, and touchend fires
|
||||
* before the synthesized click, so the blur lands first.
|
||||
*/
|
||||
_installMobileKeyboardDismiss() {
|
||||
if (this._mobileKeyboardDismissHandler) return;
|
||||
|
||||
// A SCROLL also ends in touchend, and dismissing there is wrong: scrolling
|
||||
// to read something while composing must not close the keyboard and lose
|
||||
// the composer. Track how far the finger travelled and only treat a
|
||||
// near-stationary gesture as a tap — the same TAP_THRESHOLD the terminal's
|
||||
// own touch handling uses, so both agree on what a tap is.
|
||||
let startX = 0;
|
||||
let startY = 0;
|
||||
let moved = false;
|
||||
this._mobileKeyboardDismissStart = (ev) => {
|
||||
if (ev.touches.length !== 1) {
|
||||
moved = true; // a multi-touch gesture is never a dismissing tap
|
||||
return;
|
||||
}
|
||||
startX = ev.touches[0].clientX;
|
||||
startY = ev.touches[0].clientY;
|
||||
moved = false;
|
||||
};
|
||||
this._mobileKeyboardDismissMove = (ev) => {
|
||||
if (moved || !ev.touches.length) return;
|
||||
const dx = ev.touches[0].clientX - startX;
|
||||
const dy = ev.touches[0].clientY - startY;
|
||||
const slop = window.CodemanTerminalInput.MOBILE_KEYBOARD_DISMISS_TAP_SLOP;
|
||||
if (Math.abs(dx) > slop || Math.abs(dy) > slop) {
|
||||
moved = true;
|
||||
}
|
||||
};
|
||||
this._mobileKeyboardDismissHandler = (ev) => {
|
||||
if (moved) return;
|
||||
if (!this._isMobileTerminalInputFocused()) return;
|
||||
const target = ev.target;
|
||||
if (!target || typeof target.closest !== 'function') return;
|
||||
if (target.closest('#terminalContainer')) return;
|
||||
if (target.closest(window.CodemanTerminalInput.MOBILE_KEYBOARD_DISMISS_EXEMPT_SELECTOR)) return;
|
||||
this._blurMobileTerminalInput();
|
||||
};
|
||||
// Passive throughout: this never calls preventDefault, so it must not make
|
||||
// the page feel less responsive to scrolling.
|
||||
document.addEventListener('touchstart', this._mobileKeyboardDismissStart, { passive: true });
|
||||
document.addEventListener('touchmove', this._mobileKeyboardDismissMove, { passive: true });
|
||||
document.addEventListener('touchend', this._mobileKeyboardDismissHandler, { passive: true });
|
||||
},
|
||||
|
||||
/**
|
||||
* Which 'content' taps should DISMISS the mobile keyboard. Expandable
|
||||
* readbacks, tool results and decision rows are TUI-owned: tapping them acts
|
||||
* on the CLI, so popping the keyboard there is wrong. An inert transcript row
|
||||
* still sends its mouse report, but must keep the keyboard reachable —
|
||||
* touchstart's preventDefault cancels the compatibility click that would
|
||||
* otherwise focus xterm, so focus has to be restored explicitly.
|
||||
*/
|
||||
_isActionableMobileTerminalTap(clientX, clientY) {
|
||||
const pos = this._clientPointToCell(clientX, clientY);
|
||||
const buffer = this.terminal?.buffer?.active;
|
||||
if (!pos || !buffer?.getLine) return false;
|
||||
|
||||
const rows = Math.max(1, this.terminal.rows || 1);
|
||||
const lines = [];
|
||||
const wrappedRows = [];
|
||||
for (let row = 0; row < rows; row++) {
|
||||
const line = buffer.getLine(buffer.viewportY + row);
|
||||
lines.push(line?.translateToString?.(true) || '');
|
||||
wrappedRows.push(Boolean(line?.isWrapped));
|
||||
}
|
||||
|
||||
const tappedRow = pos.row - 1;
|
||||
let logicalLineStart = tappedRow;
|
||||
while (logicalLineStart > 0 && wrappedRows[logicalLineStart]) logicalLineStart--;
|
||||
let logicalLineEnd = tappedRow;
|
||||
while (logicalLineEnd + 1 < rows && wrappedRows[logicalLineEnd + 1]) logicalLineEnd++;
|
||||
const tappedLine = lines.slice(logicalLineStart, logicalLineEnd + 1).join('');
|
||||
|
||||
// Match the AFFORDANCE a CLI prints, not the row's title text: an
|
||||
// expandable readback, tool result or status row advertises how to act on
|
||||
// it ("ctrl+r to expand", "tap to collapse", "esc to interrupt"). Keying on
|
||||
// titles instead would only recognise the exact strings a fixture happens
|
||||
// to use, and would let a real readback keep the keyboard open.
|
||||
//
|
||||
// The hint sits on its own row, so a readback's TITLE row — the one a
|
||||
// finger actually lands on — carries no affordance text itself. Look at the
|
||||
// adjacent row too, which is how these blocks are laid out in practice.
|
||||
// Keyed on the ACTION VERB, and deliberately not on prose verbs. A CLI hint
|
||||
// names a key or a gesture ("ctrl+r to expand", "tap to collapse",
|
||||
// "esc to interrupt"); "click here to open the file" is transcript content
|
||||
// and must keep the keyboard, so `click` and bare `here` are excluded.
|
||||
// The hint may sit mid-line — Claude's status row is
|
||||
// "✻ Cooked for 2m 6s · esc to interrupt" — so this is not anchored.
|
||||
const affordance =
|
||||
/\b(?:ctrl\+\w+|shift\+\w+|esc|enter|tab|tap)\s+to\s+(?:expand|collapse|view|open|interrupt|see)\b/i;
|
||||
const blockStart = Math.max(0, logicalLineStart - 1);
|
||||
const blockEnd = Math.min(rows - 1, logicalLineEnd + 1);
|
||||
for (let row = blockStart; row <= blockEnd; row++) {
|
||||
if (affordance.test(lines[row])) return true;
|
||||
}
|
||||
// A Claude status row ("✻ Cooked for 2m 6s · esc to interrupt") is caught by
|
||||
// the affordance above; there is deliberately no verb literal here, because
|
||||
// the verb is randomised per build.
|
||||
|
||||
const hasMenuPrompt = lines.some((line) => /^\s*[❯›]\s+\d+[.)]\s/.test(line));
|
||||
const hasMenuChoice = lines.some((line) => /^\s+\d+[.)]\s/.test(line));
|
||||
return hasMenuPrompt && hasMenuChoice;
|
||||
},
|
||||
|
||||
_focusMobileTerminalInput() {
|
||||
this._syncMobileHelperTextareaToCursor();
|
||||
const cjkInput = document.getElementById('cjkInput');
|
||||
if (cjkInput?.classList.contains('cjk-input-visible')) {
|
||||
cjkInput.focus();
|
||||
} else {
|
||||
this.terminal?.focus();
|
||||
}
|
||||
},
|
||||
|
||||
_handleMobileTerminalTap(touch, startedWithTerminalFocus, cachedIntent = null) {
|
||||
// A guard bail-out, not a classification: there is nothing to classify. It is
|
||||
// deliberately NOT 'history', which would claim the viewport was scrolled up.
|
||||
if (!touch || !this.terminal) return null;
|
||||
// touchstart already classified this exact point; reuse it rather than paying
|
||||
// a second full-viewport scan for the same gesture.
|
||||
const intent = cachedIntent ?? this._classifyMobileTerminalTap(touch.clientX, touch.clientY);
|
||||
if (intent === 'history') {
|
||||
// Scrolled up: send NO mouse report — a tap on old output must not be
|
||||
// delivered to the CLI as a click on whatever row now occupies that cell.
|
||||
// Focus is a separate question, and the answer is yes: the user tapped the
|
||||
// terminal, so let them type. Blurring here stranded activeElement on
|
||||
// <body> with no way back to the keyboard.
|
||||
this._focusMobileTerminalInput();
|
||||
return intent;
|
||||
}
|
||||
|
||||
const mouseMode = this.terminal.modes?.mouseTrackingMode;
|
||||
const mouseTrackingOn = !!mouseMode && mouseMode !== 'none';
|
||||
const shouldActivate = intent === 'content' || startedWithTerminalFocus;
|
||||
if (shouldActivate && mouseTrackingOn) {
|
||||
// xterm's mouse encoder owns live DECSET modes. The synthetic DOM click
|
||||
// follows the same path as a desktop click.
|
||||
this._dispatchSyntheticTerminalClick(touch.clientX, touch.clientY);
|
||||
} else if (shouldActivate && this._sessionUsesServerMouseStrip()) {
|
||||
// Claude/Codex/Gemini DECSETs are stripped from the browser stream, so
|
||||
// report directly to the PTY while retaining local touch scrollback.
|
||||
this._sendSyntheticSgrTap(touch.clientX, touch.clientY);
|
||||
}
|
||||
|
||||
if (intent === 'content' && this._isActionableMobileTerminalTap(touch.clientX, touch.clientY)) {
|
||||
// A synthetic xterm click can focus its helper textarea. Blur after the
|
||||
// report so collapsing a readback never opens or retains the keyboard.
|
||||
this._blurMobileTerminalInput();
|
||||
} else if (intent === 'content' && startedWithTerminalFocus) {
|
||||
// Tapping INERT transcript with the keyboard already up closes it.
|
||||
//
|
||||
// Every terminal tap re-focuses, so once the keyboard is open the only way
|
||||
// to close it is the accessory bar's dismiss chevron. Tapping the
|
||||
// transcript to get the screen back is the obvious gesture, and nothing
|
||||
// else claims it: an inert row has no action to trigger, so by this point
|
||||
// the tap has already done its only other job (the mouse report above).
|
||||
//
|
||||
// Scoped to 'content' ON PURPOSE. The prompt row ('input') keeps
|
||||
// focus-then-position, so a second tap there still places the caret —
|
||||
// pinned by "keeps the first prompt tap focus-only so it cannot activate a
|
||||
// CLI row". Toggling there would trade away real capability.
|
||||
this._blurMobileTerminalInput();
|
||||
} else {
|
||||
this._focusMobileTerminalInput();
|
||||
}
|
||||
return intent;
|
||||
},
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
// Synthetic tap → mouse report
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
|
||||
+421
-13
@@ -1,7 +1,13 @@
|
||||
/**
|
||||
* @fileoverview Voice input with Deepgram Nova-3 (primary) and Web Speech API (fallback).
|
||||
* @fileoverview Voice input with three providers: Claude (this server's Claude Code
|
||||
* login), Deepgram Nova-3, and the Web Speech API.
|
||||
*
|
||||
* Defines two singleton objects:
|
||||
* Defines three singleton objects:
|
||||
*
|
||||
* - ClaudeVoiceProvider — Dictation through Codeman's own `/ws/voice/stream`, which
|
||||
* relays to the speech-to-text service Claude Code's `/voice` mode uses. No API key:
|
||||
* the server holds the OAuth token, the browser only sends PCM16 @16 kHz (AudioWorklet,
|
||||
* since MediaRecorder cannot emit raw PCM) and receives text. See docs/claude-voice-plan.md.
|
||||
*
|
||||
* - DeepgramProvider — Direct browser-to-Deepgram WebSocket connection for speech-to-text.
|
||||
* Captures audio via MediaRecorder, streams chunks every 250ms, handles KeepAlive pings,
|
||||
@@ -14,6 +20,7 @@
|
||||
* Includes a temporary green Send button that replaces the settings gear icon after voice input.
|
||||
* Web Speech API has auto-retry (up to 2x) for premature onend and iOS Safari stability check.
|
||||
*
|
||||
* @globals {object} ClaudeVoiceProvider
|
||||
* @globals {object} DeepgramProvider
|
||||
* @globals {object} VoiceInput
|
||||
*
|
||||
@@ -22,9 +29,13 @@
|
||||
* @loadorder 3 of 15 — loaded after mobile-handlers.js, before notification-manager.js
|
||||
*/
|
||||
|
||||
// Codeman — Voice input with Deepgram Nova-3 and Web Speech API fallback
|
||||
// Codeman — Voice input with Claude, Deepgram Nova-3 and Web Speech API
|
||||
// Loaded after mobile-handlers.js, before app.js
|
||||
|
||||
/** Dev vocabulary sent to the recognizer as a hint. Shared by every provider and the settings form. */
|
||||
const DEFAULT_VOICE_KEYTERMS =
|
||||
'refactor, endpoint, middleware, callback, async, regex, TypeScript, npm, API, deploy, config, linter, env, webhook, schema, CLI, JSON, CSS, DOM, SSE, backend, frontend, localhost, dependencies, repository, merge, rebase, diff, commit, com';
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
// Voice Input (Deepgram Nova-3 + Web Speech API fallback)
|
||||
// ═══════════════════════════════════════════════════════════════
|
||||
@@ -245,7 +256,282 @@ const DeepgramProvider = {
|
||||
};
|
||||
|
||||
/**
|
||||
* VoiceInput - Speech-to-text with Deepgram Nova-3 (primary) and Web Speech API (fallback).
|
||||
* ClaudeVoiceProvider - Speech-to-text through this Codeman server's Claude Code
|
||||
* login, i.e. the same service the CLI's own `/voice` mode uses. No API key.
|
||||
*
|
||||
* Audio goes browser -> Codeman -> Anthropic: the OAuth token never leaves the
|
||||
* server, so the browser only ever sends PCM and receives text
|
||||
* (docs/claude-voice-plan.md).
|
||||
*
|
||||
* ⚠️ The upstream endpoint is opened as linear16 / 16 kHz / mono, so capture MUST
|
||||
* be raw PCM at that rate. MediaRecorder cannot emit raw PCM (container formats
|
||||
* only), which is why this path uses an AudioWorklet rather than reusing
|
||||
* DeepgramProvider's recorder. The AudioContext is constructed at 16000 Hz so the
|
||||
* browser does the resampling.
|
||||
*
|
||||
* ⚠️ Transcript frames carry the WHOLE running transcript, not deltas. Callers
|
||||
* must replace, never concatenate.
|
||||
*/
|
||||
const ClaudeVoiceProvider = {
|
||||
_ws: null,
|
||||
_stream: null,
|
||||
_audioContext: null,
|
||||
_workletNode: null,
|
||||
_sourceNode: null,
|
||||
_scriptNode: null,
|
||||
_silenceTimeout: null,
|
||||
_onResult: null,
|
||||
_onError: null,
|
||||
_onEnd: null,
|
||||
_finalized: false,
|
||||
|
||||
/** How long without any transcript before the recording gives up on its own. */
|
||||
SILENCE_MS: 6000,
|
||||
|
||||
/**
|
||||
* Start streaming.
|
||||
* @param {object} opts - { language, keyterms[], onResult(text, isFinal), onError(msg), onEnd(), onStream(stream) }
|
||||
*/
|
||||
async start(opts) {
|
||||
this._onResult = opts.onResult;
|
||||
this._onError = opts.onError;
|
||||
this._onEnd = opts.onEnd;
|
||||
this._finalized = false;
|
||||
|
||||
if (!navigator.mediaDevices?.getUserMedia) {
|
||||
this._onError?.('Microphone requires a secure context (HTTPS). Use --https flag or access via localhost.');
|
||||
this._cleanup();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
this._stream = await navigator.mediaDevices.getUserMedia({
|
||||
audio: { noiseSuppression: true, echoCancellation: true, autoGainControl: true }
|
||||
});
|
||||
} catch (err) {
|
||||
const msg = err.name === 'NotAllowedError'
|
||||
? 'Microphone access denied. Check browser settings.'
|
||||
: 'Microphone error: ' + err.message;
|
||||
this._onError?.(msg);
|
||||
this._cleanup();
|
||||
return;
|
||||
}
|
||||
opts.onStream?.(this._stream);
|
||||
|
||||
const params = new URLSearchParams();
|
||||
if (opts.language) params.set('language', opts.language);
|
||||
if (opts.keyterms?.length) params.set('keyterms', opts.keyterms.join(','));
|
||||
const proto = location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
try {
|
||||
this._ws = new WebSocket(`${proto}//${location.host}/ws/voice/stream?${params}`);
|
||||
} catch (err) {
|
||||
this._onError?.('Failed to open voice stream: ' + err.message);
|
||||
this._cleanup();
|
||||
return;
|
||||
}
|
||||
this._ws.binaryType = 'arraybuffer';
|
||||
|
||||
this._ws.onopen = () => {
|
||||
// Capture starts only once the socket is up: PCM buffered before that would
|
||||
// be the oldest audio, and dropping it keeps the transcript aligned with what
|
||||
// the user hears themselves saying.
|
||||
this._startCapture().catch((err) => {
|
||||
this._onError?.('Microphone capture failed: ' + err.message);
|
||||
this.stop();
|
||||
});
|
||||
this._resetSilenceTimeout();
|
||||
};
|
||||
|
||||
this._ws.onmessage = (event) => {
|
||||
let msg;
|
||||
try {
|
||||
msg = JSON.parse(event.data);
|
||||
} catch (_e) {
|
||||
return;
|
||||
}
|
||||
if (msg.t === 'transcript' && msg.text) {
|
||||
this._resetSilenceTimeout();
|
||||
this._onResult?.(msg.text, msg.final === true);
|
||||
} else if (msg.t === 'error') {
|
||||
this._onError?.(msg.message || 'Voice transcription failed');
|
||||
}
|
||||
};
|
||||
|
||||
this._ws.onerror = () => {
|
||||
// onclose carries the actionable detail (close code); nothing useful here.
|
||||
};
|
||||
|
||||
this._ws.onclose = (event) => {
|
||||
if (event.code === 4004) {
|
||||
this._onError?.(this._unavailableMessage(event.reason));
|
||||
} else if (event.code === 4008) {
|
||||
this._onError?.('Too many voice streams are already running on this server.');
|
||||
} else if (event.code === 4003) {
|
||||
this._onError?.('Voice stream refused (origin not allowed).');
|
||||
} else if (event.code !== 1000 && !this._finalized) {
|
||||
this._onError?.('Voice stream closed: ' + (event.reason || `code ${event.code}`));
|
||||
}
|
||||
this._stopCapture();
|
||||
const onEnd = this._onEnd;
|
||||
this._onEnd = null;
|
||||
onEnd?.();
|
||||
};
|
||||
},
|
||||
|
||||
/** Map the server's close reason onto something a user can act on. */
|
||||
_unavailableMessage(reason) {
|
||||
if (reason === 'expired') return 'Claude login expired. Run a Claude session to refresh it, then try again.';
|
||||
if (reason === 'disabled') return 'Claude voice is off. Enable it in Settings > Voice.';
|
||||
return 'No Claude Code login found on the server. Sign in with `claude` there, or use Deepgram.';
|
||||
},
|
||||
|
||||
/** Wire mic -> 16 kHz PCM16 frames -> WebSocket. */
|
||||
async _startCapture() {
|
||||
const Ctx = window.AudioContext || window.webkitAudioContext;
|
||||
// Ask for 16 kHz directly so the browser resamples; Safari may hand back its
|
||||
// own rate, which _pcmFromFloat32 then downsamples to match.
|
||||
this._audioContext = new Ctx({ sampleRate: 16000 });
|
||||
if (this._audioContext.state === 'suspended') await this._audioContext.resume();
|
||||
this._sourceNode = this._audioContext.createMediaStreamSource(this._stream);
|
||||
|
||||
if (this._audioContext.audioWorklet) {
|
||||
await this._audioContext.audioWorklet.addModule(this._workletUrl());
|
||||
this._workletNode = new AudioWorkletNode(this._audioContext, 'pcm-frame-processor');
|
||||
this._workletNode.port.onmessage = (event) => this._sendAudio(event.data);
|
||||
this._sourceNode.connect(this._workletNode);
|
||||
// A worklet with no destination is not pulled in some engines; a zero-gain
|
||||
// sink keeps the graph running without echoing the mic to the speakers.
|
||||
const sink = this._audioContext.createGain();
|
||||
sink.gain.value = 0;
|
||||
this._workletNode.connect(sink).connect(this._audioContext.destination);
|
||||
return;
|
||||
}
|
||||
|
||||
// Fallback for engines without AudioWorklet (older Safari): deprecated, but
|
||||
// it is this or no dictation at all there.
|
||||
this._scriptNode = this._audioContext.createScriptProcessor(4096, 1, 1);
|
||||
this._scriptNode.onaudioprocess = (event) => {
|
||||
this._sendAudio(this._pcmFromFloat32(event.inputBuffer.getChannelData(0), this._audioContext.sampleRate));
|
||||
};
|
||||
this._sourceNode.connect(this._scriptNode);
|
||||
this._scriptNode.connect(this._audioContext.destination);
|
||||
},
|
||||
|
||||
/**
|
||||
* Worklet URL carrying this page's cache-bust token.
|
||||
*
|
||||
* ⚠️ Static assets are served `immutable` for a year, and `cacheBustAssets`
|
||||
* only rewrites `.js` refs in `<script>`/`<link>` tags — a URL built here in JS
|
||||
* is invisible to it. So the token is borrowed from voice-input.js's own script
|
||||
* tag, which the server DID rewrite. Consequence: **edit the worklet and this
|
||||
* file together**, or the browser keeps serving the old worklet.
|
||||
*/
|
||||
_workletUrl() {
|
||||
const src = document.querySelector('script[src*="voice-input.js"]')?.getAttribute('src') || '';
|
||||
const q = src.indexOf('?');
|
||||
return 'voice-pcm-worklet.js' + (q === -1 ? '' : src.slice(q));
|
||||
},
|
||||
|
||||
/** Float32 [-1,1] at any rate -> Int16 PCM at 16 kHz (nearest-neighbour decimation). */
|
||||
_pcmFromFloat32(input, sampleRate) {
|
||||
const ratio = sampleRate / 16000;
|
||||
const outLength = Math.floor(input.length / ratio);
|
||||
const out = new Int16Array(outLength);
|
||||
for (let i = 0; i < outLength; i++) {
|
||||
const sample = Math.max(-1, Math.min(1, input[Math.floor(i * ratio)]));
|
||||
out[i] = sample < 0 ? sample * 0x8000 : sample * 0x7fff;
|
||||
}
|
||||
return out.buffer;
|
||||
},
|
||||
|
||||
_sendAudio(arrayBuffer) {
|
||||
if (this._finalized) return;
|
||||
if (this._ws?.readyState !== WebSocket.OPEN) return;
|
||||
try {
|
||||
this._ws.send(arrayBuffer);
|
||||
} catch (_e) {
|
||||
/* socket died mid-frame */
|
||||
}
|
||||
},
|
||||
|
||||
_resetSilenceTimeout() {
|
||||
clearTimeout(this._silenceTimeout);
|
||||
this._silenceTimeout = setTimeout(() => this.stop(), this.SILENCE_MS);
|
||||
},
|
||||
|
||||
/**
|
||||
* Ask for the final transcript and let the server close the socket. Capture stops
|
||||
* immediately, but the WebSocket stays open: the last (and usually best) transcript
|
||||
* arrives AFTER the audio does, so closing here would throw away the utterance.
|
||||
*/
|
||||
stop() {
|
||||
clearTimeout(this._silenceTimeout);
|
||||
this._silenceTimeout = null;
|
||||
if (this._finalized) return;
|
||||
this._finalized = true;
|
||||
this._stopCapture();
|
||||
if (this._ws?.readyState === WebSocket.OPEN) {
|
||||
try {
|
||||
this._ws.send(JSON.stringify({ t: 'finalize' }));
|
||||
} catch (_e) {
|
||||
/* ignore */
|
||||
}
|
||||
} else {
|
||||
const onEnd = this._onEnd;
|
||||
this._onEnd = null;
|
||||
onEnd?.();
|
||||
}
|
||||
},
|
||||
|
||||
/** Tear down the audio graph and release the mic. Idempotent. */
|
||||
_stopCapture() {
|
||||
if (this._workletNode) {
|
||||
this._workletNode.port.onmessage = null;
|
||||
try { this._workletNode.disconnect(); } catch (_e) { /* ignore */ }
|
||||
this._workletNode = null;
|
||||
}
|
||||
if (this._scriptNode) {
|
||||
this._scriptNode.onaudioprocess = null;
|
||||
try { this._scriptNode.disconnect(); } catch (_e) { /* ignore */ }
|
||||
this._scriptNode = null;
|
||||
}
|
||||
if (this._sourceNode) {
|
||||
try { this._sourceNode.disconnect(); } catch (_e) { /* ignore */ }
|
||||
this._sourceNode = null;
|
||||
}
|
||||
if (this._audioContext) {
|
||||
try { this._audioContext.close(); } catch (_e) { /* ignore */ }
|
||||
this._audioContext = null;
|
||||
}
|
||||
if (this._stream) {
|
||||
this._stream.getTracks().forEach(t => t.stop());
|
||||
this._stream = null;
|
||||
}
|
||||
},
|
||||
|
||||
/** Hard stop: drop the socket without waiting for a final transcript. */
|
||||
_cleanup() {
|
||||
this._finalized = true;
|
||||
clearTimeout(this._silenceTimeout);
|
||||
this._silenceTimeout = null;
|
||||
this._stopCapture();
|
||||
if (this._ws) {
|
||||
this._ws.onclose = null;
|
||||
this._ws.onmessage = null;
|
||||
this._ws.onerror = null;
|
||||
if (this._ws.readyState === WebSocket.OPEN) {
|
||||
try { this._ws.close(1000); } catch (_e) { /* ignore */ }
|
||||
}
|
||||
this._ws = null;
|
||||
}
|
||||
this._onResult = null;
|
||||
this._onError = null;
|
||||
this._onEnd = null;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* VoiceInput - Speech-to-text with Claude (this server's Claude Code login),
|
||||
* Deepgram Nova-3, or the Web Speech API.
|
||||
* Toggle mode: tap mic to start, tap again to stop. Auto-stops after silence.
|
||||
* Shows interim transcription in a floating preview overlay.
|
||||
* Inserts final text into the active session (user presses Enter to submit).
|
||||
@@ -273,6 +559,29 @@ const VoiceInput = {
|
||||
this._initRecognition();
|
||||
// Always show buttons — if unsupported, toggle() shows a toast
|
||||
this._showButtons();
|
||||
// Probe the server's Claude voice availability in the background. `auto`
|
||||
// resolution reads the cached answer, so the first mic press does not wait
|
||||
// on a round trip; a miss just falls through to the next provider.
|
||||
this.refreshClaudeStatus();
|
||||
},
|
||||
|
||||
/** Last /api/voice/status answer, or null before the first probe resolves. */
|
||||
_claudeStatus: null,
|
||||
|
||||
/**
|
||||
* Re-probe whether this server can transcribe with its Claude Code login.
|
||||
* Called at init and whenever App Settings opens (the setting is server-side,
|
||||
* so another device could have flipped it).
|
||||
*/
|
||||
async refreshClaudeStatus() {
|
||||
try {
|
||||
const res = await fetch('/api/voice/status');
|
||||
const json = await res.json();
|
||||
this._claudeStatus = json?.success ? json.data : { available: false, reason: 'disabled' };
|
||||
} catch (_e) {
|
||||
this._claudeStatus = { available: false, reason: 'disabled' };
|
||||
}
|
||||
return this._claudeStatus;
|
||||
},
|
||||
|
||||
// --- Deepgram config (localStorage only, never sent to server) ---
|
||||
@@ -294,11 +603,37 @@ const VoiceInput = {
|
||||
return !!(cfg.apiKey && cfg.apiKey.trim());
|
||||
},
|
||||
|
||||
_claudeAvailable() {
|
||||
return this._claudeStatus?.available === true;
|
||||
},
|
||||
|
||||
/**
|
||||
* Which provider a press of the mic would use.
|
||||
*
|
||||
* An explicit pick always wins, even when it cannot run — the resulting error
|
||||
* ("Claude voice is off", "no Deepgram key") is more useful than silently
|
||||
* transcribing somewhere the user did not choose. `auto` prefers Claude because
|
||||
* it needs no key and no per-word billing, then the configured Deepgram key,
|
||||
* then the browser's own engine.
|
||||
*/
|
||||
_resolveProvider() {
|
||||
const pinned = this._getDeepgramConfig().provider;
|
||||
if (pinned === 'claude' || pinned === 'deepgram' || pinned === 'webspeech') return pinned;
|
||||
if (this._claudeAvailable()) return 'claude';
|
||||
if (this._shouldUseDeepgram()) return 'deepgram';
|
||||
return 'webspeech';
|
||||
},
|
||||
|
||||
/** Get the active provider name for display */
|
||||
getActiveProviderName() {
|
||||
if (this._shouldUseDeepgram()) return 'Deepgram Nova-3';
|
||||
if (this.supported) return 'Web Speech API';
|
||||
return 'None';
|
||||
switch (this._resolveProvider()) {
|
||||
case 'claude':
|
||||
return this._claudeAvailable() ? 'Claude (this server’s login)' : 'Claude (unavailable)';
|
||||
case 'deepgram':
|
||||
return this._shouldUseDeepgram() ? 'Deepgram Nova-3' : 'Deepgram (no API key)';
|
||||
default:
|
||||
return this.supported ? 'Web Speech API' : 'None';
|
||||
}
|
||||
},
|
||||
|
||||
/** Try to create a SpeechRecognition instance */
|
||||
@@ -334,13 +669,81 @@ const VoiceInput = {
|
||||
}
|
||||
this._retryCount = 0;
|
||||
|
||||
if (this._shouldUseDeepgram()) {
|
||||
const provider = this._resolveProvider();
|
||||
if (provider === 'claude') {
|
||||
this._startClaude();
|
||||
} else if (provider === 'deepgram') {
|
||||
this._startDeepgram();
|
||||
} else {
|
||||
this._startWebSpeech();
|
||||
}
|
||||
},
|
||||
|
||||
_startClaude() {
|
||||
if (!this._claudeAvailable()) {
|
||||
const reason = this._claudeStatus?.reason;
|
||||
app.showToast(
|
||||
reason === 'expired'
|
||||
? 'Claude login expired on the server. Run a Claude session to refresh it.'
|
||||
: reason === 'no-credentials'
|
||||
? 'No Claude Code login found on the server. Sign in there with `claude`.'
|
||||
: 'Claude voice is off. Enable it in Settings > Voice.',
|
||||
'warning'
|
||||
);
|
||||
// Re-probe so a setting flipped on another device is picked up by the next press.
|
||||
this.refreshClaudeStatus();
|
||||
return;
|
||||
}
|
||||
|
||||
const cfg = this._getDeepgramConfig();
|
||||
this.isRecording = true;
|
||||
this._activeProvider = 'claude';
|
||||
this._accumulatedFinal = '';
|
||||
this._lastTranscript = '';
|
||||
this._hasReceivedResult = false;
|
||||
this._recordingStartedAt = Date.now();
|
||||
this._updateButtons('recording');
|
||||
this._showPreview('Listening...', 'claude');
|
||||
this._startDurationTimer();
|
||||
|
||||
const keyterms = (cfg.keyterms || DEFAULT_VOICE_KEYTERMS)
|
||||
.split(',').map(t => t.trim()).filter(Boolean);
|
||||
|
||||
ClaudeVoiceProvider.start({
|
||||
// The upstream endpoint wants a bare language tag; the Deepgram picker's
|
||||
// 'en-US' style narrows to its base, and 'multi' means auto-detect.
|
||||
language: (cfg.language || 'en-US').split('-')[0],
|
||||
keyterms,
|
||||
onStream: (stream) => this._startLevelMeter(stream),
|
||||
onResult: (text, isFinal) => {
|
||||
if (!this.isRecording) return;
|
||||
this._hasReceivedResult = true;
|
||||
// Each frame is the WHOLE running transcript, so replace rather than append.
|
||||
this._accumulatedFinal = text;
|
||||
if (isFinal) {
|
||||
this._hidePreview();
|
||||
this._insertText(text);
|
||||
this.stop();
|
||||
} else {
|
||||
this._showPreview(text, 'claude');
|
||||
}
|
||||
},
|
||||
onError: (msg) => {
|
||||
const wasRecording = this.isRecording;
|
||||
this.stop();
|
||||
if (wasRecording) app.showToast(msg, 'error');
|
||||
},
|
||||
onEnd: () => {
|
||||
if (this.isRecording) {
|
||||
if (this._accumulatedFinal) this._insertText(this._accumulatedFinal);
|
||||
this.stop();
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (navigator.vibrate) navigator.vibrate(50);
|
||||
},
|
||||
|
||||
_startDeepgram() {
|
||||
const cfg = this._getDeepgramConfig();
|
||||
this.isRecording = true;
|
||||
@@ -353,7 +756,7 @@ const VoiceInput = {
|
||||
this._showPreview('Listening...', 'deepgram');
|
||||
this._startDurationTimer();
|
||||
|
||||
const keyterms = (cfg.keyterms || 'refactor, endpoint, middleware, callback, async, regex, TypeScript, npm, API, deploy, config, linter, env, webhook, schema, CLI, JSON, CSS, DOM, SSE, backend, frontend, localhost, dependencies, repository, merge, rebase, diff, commit, com')
|
||||
const keyterms = (cfg.keyterms || DEFAULT_VOICE_KEYTERMS)
|
||||
.split(',').map(t => t.trim()).filter(Boolean);
|
||||
|
||||
DeepgramProvider.start({
|
||||
@@ -452,7 +855,10 @@ const VoiceInput = {
|
||||
this._updateButtons('idle');
|
||||
this._hidePreview();
|
||||
|
||||
if (this._activeProvider === 'deepgram') {
|
||||
if (this._activeProvider === 'claude') {
|
||||
// Finalize, don't hang up: the last transcript arrives after the audio does.
|
||||
ClaudeVoiceProvider.stop();
|
||||
} else if (this._activeProvider === 'deepgram') {
|
||||
DeepgramProvider.stop();
|
||||
} else if (this._activeProvider === 'webspeech') {
|
||||
try {
|
||||
@@ -803,11 +1209,12 @@ const VoiceInput = {
|
||||
timerEl.textContent = '0:00';
|
||||
indicator.appendChild(timerEl);
|
||||
this.previewEl.appendChild(indicator);
|
||||
// Provider badge for Deepgram
|
||||
if (provider === 'deepgram') {
|
||||
// Provider badge (Web Speech gets none — it is the fallback, not a choice)
|
||||
const badgeText = provider === 'deepgram' ? 'DG' : provider === 'claude' ? 'CLAUDE' : '';
|
||||
if (badgeText) {
|
||||
const badge = document.createElement('span');
|
||||
badge.className = 'voice-preview-badge';
|
||||
badge.textContent = 'DG';
|
||||
badge.textContent = badgeText;
|
||||
this.previewEl.appendChild(badge);
|
||||
this.previewEl.appendChild(document.createTextNode(' '));
|
||||
}
|
||||
@@ -861,6 +1268,7 @@ const VoiceInput = {
|
||||
if (this.isRecording) this.stop();
|
||||
this._hideVoiceSendBtn();
|
||||
DeepgramProvider._cleanup();
|
||||
ClaudeVoiceProvider._cleanup();
|
||||
this.recognition = null;
|
||||
this._activeProvider = null;
|
||||
this._stopDurationTimer();
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* @fileoverview AudioWorklet that turns microphone audio into the PCM frames the
|
||||
* Claude voice endpoint expects.
|
||||
*
|
||||
* The endpoint is opened as `encoding=linear16, sample_rate=16000, channels=1`,
|
||||
* i.e. raw signed 16-bit little-endian mono. MediaRecorder cannot produce that
|
||||
* (it only emits container formats — webm/opus, mp4), which is why the Deepgram
|
||||
* path's capture code cannot be reused here: Deepgram sniffs the container,
|
||||
* Anthropic's endpoint does not.
|
||||
*
|
||||
* Sample rate is handled by the AudioContext, constructed at 16000 Hz so the
|
||||
* browser resamples the mic for us. This processor only converts Float32 [-1,1]
|
||||
* to Int16 and batches, because a raw 128-sample render quantum is a ~4 ms
|
||||
* WebSocket frame — 250 frames a second of pure overhead.
|
||||
*
|
||||
* Loaded via `audioWorklet.addModule()` from voice-input.js. Runs on the audio
|
||||
* thread: no DOM, no globals from the page.
|
||||
*
|
||||
* ⚠️ Edit this file and voice-input.js together. Static assets are served
|
||||
* `immutable` for a year and this one is fetched from JS, so it inherits its
|
||||
* cache-bust token from voice-input.js's script tag (see `_workletUrl()`); a
|
||||
* change here alone would keep serving the old copy to every returning browser.
|
||||
*/
|
||||
|
||||
/** ~256 ms at 16 kHz. Big enough to keep frame overhead down, small enough that interim transcripts stay live. */
|
||||
const FRAME_SAMPLES = 4096;
|
||||
|
||||
class PcmFrameProcessor extends AudioWorkletProcessor {
|
||||
constructor() {
|
||||
super();
|
||||
this._buffer = new Int16Array(FRAME_SAMPLES);
|
||||
this._offset = 0;
|
||||
}
|
||||
|
||||
process(inputs) {
|
||||
const channel = inputs[0]?.[0];
|
||||
// No input yet (mic still warming) — keep the processor alive.
|
||||
if (!channel) return true;
|
||||
|
||||
for (let i = 0; i < channel.length; i++) {
|
||||
// Clamp before scaling: values slightly outside [-1,1] are legal in Web Audio
|
||||
// and would wrap around to the opposite sign as Int16, which sounds like a click.
|
||||
const sample = Math.max(-1, Math.min(1, channel[i]));
|
||||
this._buffer[this._offset++] = sample < 0 ? sample * 0x8000 : sample * 0x7fff;
|
||||
|
||||
if (this._offset === FRAME_SAMPLES) {
|
||||
// Transfer a copy: the worklet keeps reusing its own buffer.
|
||||
const frame = new Int16Array(this._buffer);
|
||||
this.port.postMessage(frame.buffer, [frame.buffer]);
|
||||
this._offset = 0;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
registerProcessor('pcm-frame-processor', PcmFrameProcessor);
|
||||
@@ -24,4 +24,5 @@ export { registerSearchRoutes } from './search-routes.js';
|
||||
export { registerMeRoutes } from './me-routes.js';
|
||||
export { registerAdminRoutes } from './admin-routes.js';
|
||||
export { registerWsRoutes } from './ws-routes.js';
|
||||
export { registerVoiceRoutes } from './voice-routes.js';
|
||||
export { registerWebviewRoutes, tryWebviewRefererFallback } from './webview-routes.js';
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
/**
|
||||
* @fileoverview Claude voice dictation routes.
|
||||
*
|
||||
* - `GET /api/voice/status` — can this server transcribe? (settings gate + credential state)
|
||||
* - `GET /ws/voice/stream` — one dictation: PCM16 audio up, transcripts down
|
||||
*
|
||||
* Design and the upstream protocol: `docs/claude-voice-plan.md`. The relay itself
|
||||
* lives in `../voice-stream.ts`; this file is the auth, gating and lifetime shell
|
||||
* around it.
|
||||
*
|
||||
* ⚠️ `/api/voice/status` reports STATE, never the token: `{ available, reason,
|
||||
* subscriptionType?, expiresAt? }`. The Claude OAuth access token stays inside the
|
||||
* server process — the browser sends audio and receives text, nothing else.
|
||||
*
|
||||
* ⚠️ The WebSocket carries the same upgrade guard as `/ws/sessions/:id/terminal`
|
||||
* (allowed Host + same-site Origin, on top of the global auth hook that already ran
|
||||
* on the handshake). Without it a cross-site page could open a dictation stream on
|
||||
* the user's credentials and bill their subscription.
|
||||
*
|
||||
* ⚠️ The feature is OFF unless `claudeVoiceEnabled` is set: turning it on spends the
|
||||
* server owner's Claude subscription on transcription for anyone who can reach the
|
||||
* UI, which is a decision for the operator rather than a default.
|
||||
*/
|
||||
|
||||
import { createRequire } from 'module';
|
||||
import { FastifyInstance } from 'fastify';
|
||||
import type { WebSocket } from 'ws';
|
||||
import { ApiErrorCode, createErrorResponse } from '../../types.js';
|
||||
import { isAllowedRequestHost, isAllowedRequestOrigin, type HostPolicy } from '../network-auth-policy.js';
|
||||
import { readClaudeOAuthCredentials } from '../../claude-credentials.js';
|
||||
import { VoiceStreamRelay } from '../voice-stream.js';
|
||||
import { MAX_AUDIO_FRAME_BYTES, MAX_CONCURRENT_STREAMS } from '../../config/voice.js';
|
||||
import type { ConfigPort } from '../ports/index.js';
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const { version: APP_VERSION } = require('../../../package.json') as { version: string };
|
||||
|
||||
/** Why voice is unavailable, in a form the frontend can branch on. */
|
||||
export type VoiceUnavailableReason = 'disabled' | 'no-credentials' | 'expired' | 'malformed';
|
||||
|
||||
export interface VoiceStatus {
|
||||
available: boolean;
|
||||
reason?: VoiceUnavailableReason;
|
||||
/** Display-only ('max', 'pro'); present when the credential store reported one. */
|
||||
subscriptionType?: string;
|
||||
expiresAt?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the server's dictation readiness. Split out and exported so the status
|
||||
* endpoint and the WebSocket upgrade cannot drift apart: the socket must never
|
||||
* accept a stream the status endpoint calls unavailable.
|
||||
*/
|
||||
export async function resolveVoiceStatus(enabled: boolean): Promise<VoiceStatus> {
|
||||
if (!enabled) return { available: false, reason: 'disabled' };
|
||||
const creds = await readClaudeOAuthCredentials();
|
||||
switch (creds.status) {
|
||||
case 'ok':
|
||||
return { available: true, subscriptionType: creds.subscriptionType, expiresAt: creds.expiresAt };
|
||||
case 'expired':
|
||||
return { available: false, reason: 'expired', expiresAt: creds.expiresAt };
|
||||
case 'malformed':
|
||||
return { available: false, reason: 'malformed' };
|
||||
default:
|
||||
return { available: false, reason: 'no-credentials' };
|
||||
}
|
||||
}
|
||||
|
||||
/** Live relays, server-wide. Dictation is human-paced, so the cap is small. */
|
||||
let activeStreams = 0;
|
||||
|
||||
/** Test seam: the cap is process-wide state, so suites must be able to reset it. */
|
||||
export function _resetVoiceStreamCountForTesting(): void {
|
||||
activeStreams = 0;
|
||||
}
|
||||
|
||||
/** Split a comma-separated keyterms query value into terms. */
|
||||
function parseKeyterms(raw: unknown): string[] {
|
||||
if (typeof raw !== 'string' || !raw) return [];
|
||||
return raw
|
||||
.split(',')
|
||||
.map((t) => t.trim())
|
||||
.filter(Boolean)
|
||||
.slice(0, 100);
|
||||
}
|
||||
|
||||
export function registerVoiceRoutes(app: FastifyInstance, ctx: ConfigPort, getHostPolicy: () => HostPolicy): void {
|
||||
app.get('/api/voice/status', async (_req, reply) => {
|
||||
try {
|
||||
return { success: true, data: await resolveVoiceStatus(await ctx.getClaudeVoiceEnabled()) };
|
||||
} catch {
|
||||
reply.code(500);
|
||||
return createErrorResponse(ApiErrorCode.INTERNAL_ERROR, 'Failed to read voice status');
|
||||
}
|
||||
});
|
||||
|
||||
app.get<{ Querystring: { language?: string; keyterms?: string } }>(
|
||||
'/ws/voice/stream',
|
||||
{ websocket: true },
|
||||
async (socket: WebSocket, req) => {
|
||||
// Cross-site upgrade guard first: this socket spends the operator's Claude
|
||||
// subscription, so it must be reachable only from Codeman's own origin.
|
||||
const policy = getHostPolicy();
|
||||
if (!isAllowedRequestHost(req.headers.host, policy) || !isAllowedRequestOrigin(req.headers.origin, policy)) {
|
||||
socket.close(4003, 'Forbidden');
|
||||
return;
|
||||
}
|
||||
|
||||
const status = await resolveVoiceStatus(await ctx.getClaudeVoiceEnabled());
|
||||
if (!status.available) {
|
||||
socket.close(4004, status.reason ?? 'unavailable');
|
||||
return;
|
||||
}
|
||||
// Re-read rather than trusting resolveVoiceStatus's discarded token: the
|
||||
// status helper deliberately never returns it.
|
||||
const creds = await readClaudeOAuthCredentials();
|
||||
if (creds.status !== 'ok' || !creds.accessToken) {
|
||||
socket.close(4004, 'no-credentials');
|
||||
return;
|
||||
}
|
||||
|
||||
if (activeStreams >= MAX_CONCURRENT_STREAMS) {
|
||||
socket.close(4008, 'Too many voice streams');
|
||||
return;
|
||||
}
|
||||
activeStreams++;
|
||||
|
||||
let released = false;
|
||||
const release = () => {
|
||||
if (released) return;
|
||||
released = true;
|
||||
activeStreams--;
|
||||
};
|
||||
|
||||
const send = (payload: Record<string, unknown>) => {
|
||||
if (socket.readyState !== 1) return;
|
||||
try {
|
||||
socket.send(JSON.stringify(payload));
|
||||
} catch {
|
||||
/* client vanished mid-write */
|
||||
}
|
||||
};
|
||||
|
||||
const relay = new VoiceStreamRelay({
|
||||
accessToken: creds.accessToken,
|
||||
appVersion: APP_VERSION,
|
||||
language: req.query.language,
|
||||
keyterms: parseKeyterms(req.query.keyterms),
|
||||
onReady: () => send({ t: 'ready' }),
|
||||
onTranscript: (text, final) => send({ t: 'transcript', text, final }),
|
||||
onError: (message) => send({ t: 'error', message }),
|
||||
onClose: () => {
|
||||
release();
|
||||
send({ t: 'closed' });
|
||||
if (socket.readyState === 1) {
|
||||
try {
|
||||
socket.close(1000, 'Voice stream ended');
|
||||
} catch {
|
||||
/* already closing */
|
||||
}
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
// Handlers are attached synchronously before any further await
|
||||
// (@fastify/websocket drops messages that arrive before they exist).
|
||||
socket.on('message', (raw: Buffer, isBinary: boolean) => {
|
||||
if (isBinary) {
|
||||
if (raw.length === 0 || raw.length > MAX_AUDIO_FRAME_BYTES) return;
|
||||
relay.sendAudio(raw);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const msg = JSON.parse(String(raw)) as { t?: string };
|
||||
if (msg.t === 'finalize') relay.finalize();
|
||||
else if (msg.t === 'stop') relay.close();
|
||||
} catch {
|
||||
/* non-JSON control frame — ignore */
|
||||
}
|
||||
});
|
||||
|
||||
socket.on('close', () => {
|
||||
relay.close();
|
||||
release();
|
||||
});
|
||||
socket.on('error', () => {
|
||||
relay.close();
|
||||
release();
|
||||
});
|
||||
|
||||
relay.connect();
|
||||
}
|
||||
);
|
||||
}
|
||||
@@ -857,6 +857,16 @@ export const SettingsUpdateSchema = z
|
||||
* add-only at create; a marker keeps user-authored copies untouched.
|
||||
*/
|
||||
agentSkillEnabled: z.boolean().optional(),
|
||||
/**
|
||||
* Let browser dictation transcribe through this machine's Claude Code login,
|
||||
* the same speech-to-text service the CLI's own `/voice` mode uses
|
||||
* (docs/claude-voice-plan.md). SYNCED, default OFF: enabling it spends the
|
||||
* operator's Claude subscription on transcription for anyone who can reach
|
||||
* the UI, and routes microphone audio to Anthropic rather than to whichever
|
||||
* provider was configured before. The Deepgram and Web Speech paths are
|
||||
* untouched by this flag.
|
||||
*/
|
||||
claudeVoiceEnabled: z.boolean().optional(),
|
||||
/**
|
||||
* Approvals Inbox (header bell + drawer, phone overview answer buttons,
|
||||
* push Approve/Deny action buttons). SYNCED, default OFF (opt-in): even
|
||||
@@ -970,6 +980,8 @@ export const SettingsUpdateSchema = z
|
||||
// Voice settings (cross-device sync)
|
||||
voiceSettings: z
|
||||
.object({
|
||||
/** 'auto' | 'claude' | 'deepgram' | 'webspeech'. Unknown values fall back to auto client-side. */
|
||||
provider: z.string().max(20).optional(),
|
||||
apiKey: z.string().max(200).optional(),
|
||||
language: z.string().max(20).optional(),
|
||||
keyterms: z.string().max(500).optional(),
|
||||
|
||||
@@ -166,6 +166,7 @@ import {
|
||||
registerMeRoutes,
|
||||
registerAdminRoutes,
|
||||
registerWsRoutes,
|
||||
registerVoiceRoutes,
|
||||
registerWebviewRoutes,
|
||||
tryWebviewRefererFallback,
|
||||
} from './routes/index.js';
|
||||
@@ -635,6 +636,7 @@ export class WebServer extends EventEmitter {
|
||||
getClaudeModeConfig: this.getClaudeModeConfig.bind(this),
|
||||
getTerminalHistoryConfig: this.getTerminalHistoryConfig.bind(this),
|
||||
getAgentSkillEnabled: this.getAgentSkillEnabled.bind(this),
|
||||
getClaudeVoiceEnabled: this.getClaudeVoiceEnabled.bind(this),
|
||||
getDefaultClaudeMdPath: this.getDefaultClaudeMdPath.bind(this),
|
||||
getLightState: this.getLightState.bind(this),
|
||||
getLightSessionsState: this.getLightSessionsState.bind(this),
|
||||
@@ -982,6 +984,7 @@ export class WebServer extends EventEmitter {
|
||||
registerCronRoutes(this.app, { ...ctx, cron: this.cronService });
|
||||
|
||||
registerWsRoutes(this.app, ctx, () => this.getHostPolicy());
|
||||
registerVoiceRoutes(this.app, ctx, () => this.getHostPolicy());
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1704,6 +1707,15 @@ export class WebServer extends EventEmitter {
|
||||
return settings.agentSkillEnabled === true;
|
||||
}
|
||||
|
||||
// Whether browser dictation may use this machine's Claude Code credentials
|
||||
// (synced `claudeVoiceEnabled` setting, default OFF; docs/claude-voice-plan.md).
|
||||
// OFF by default because turning it on spends the operator's Claude subscription
|
||||
// on transcription for anyone who can reach the UI.
|
||||
private async getClaudeVoiceEnabled(): Promise<boolean> {
|
||||
const settings = await this.readSettings();
|
||||
return settings.claudeVoiceEnabled === true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read My Mind predictor model (docs/readmymind-plan.md): `readMyMindModel`
|
||||
* setting, defaulting to the AI-checker opus model. Prediction quality is
|
||||
|
||||
@@ -0,0 +1,300 @@
|
||||
/**
|
||||
* @fileoverview Upstream half of Claude voice dictation: one browser recording
|
||||
* relayed to the speech-to-text service Claude Code's own `/voice` mode uses.
|
||||
*
|
||||
* The browser cannot talk to that service directly — it would need the Claude
|
||||
* OAuth bearer token in page JavaScript, and the endpoint is not CORS-open — so
|
||||
* Codeman sits in the middle and is the only thing that ever holds the token.
|
||||
* See `docs/claude-voice-plan.md` for the protocol table this implements.
|
||||
*
|
||||
* Wire contract (mirrors the CLI's `connectVoiceStream`):
|
||||
* - Query pins the audio format: linear16 PCM, 16 kHz, mono. The browser worklet
|
||||
* produces exactly that; a mismatch transcribes as silence or noise, never an error.
|
||||
* - `{"type":"KeepAlive"}` on open and every 8s, or upstream drops the socket
|
||||
* between utterances.
|
||||
* - Audio frames go up as raw binary.
|
||||
* - Downstream, `TranscriptText`/`TranscriptInterim` carry the RUNNING transcript
|
||||
* (each frame supersedes the previous one — they are not deltas to concatenate),
|
||||
* and `TranscriptEndpoint` promotes the pending interim to final.
|
||||
* - `{"type":"CloseStream"}` finalizes; the endpoint frame that follows is the
|
||||
* last transcript, so `finalize()` waits briefly for it rather than closing.
|
||||
*
|
||||
* The pure builders at the top are unit-tested; `VoiceStreamRelay` owns the socket,
|
||||
* the keepalive timer and the lifetime cap.
|
||||
*/
|
||||
|
||||
import WebSocket from 'ws';
|
||||
import {
|
||||
AUDIO_CHANNELS,
|
||||
AUDIO_SAMPLE_RATE,
|
||||
FINALIZE_TIMEOUT_MS,
|
||||
KEEPALIVE_INTERVAL_MS,
|
||||
MAX_KEYTERMS_HEADER_CHARS,
|
||||
MAX_STREAM_MS,
|
||||
VOICE_STREAM_PATH,
|
||||
voiceStreamBase,
|
||||
} from '../config/voice.js';
|
||||
|
||||
const KEEPALIVE_FRAME = '{"type":"KeepAlive"}';
|
||||
const CLOSE_STREAM_FRAME = '{"type":"CloseStream"}';
|
||||
|
||||
export interface VoiceStreamParams {
|
||||
/** BCP-47-ish language hint. Anything unusable falls back to 'en'. */
|
||||
language?: string;
|
||||
/** Domain vocabulary sent as a recognition hint. */
|
||||
keyterms?: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Collapse keyterms into the single ASCII header value upstream accepts.
|
||||
*
|
||||
* Commas separate terms, so a comma INSIDE a term would silently split it; it is
|
||||
* replaced with a space rather than dropped. Non-ASCII is stripped because the
|
||||
* value travels as an HTTP header, where anything outside the visible ASCII range
|
||||
* is not portable. Deduped and truncated on a term boundary so a long list degrades
|
||||
* to a shorter list instead of a mangled final term.
|
||||
*/
|
||||
export function sanitizeKeyterms(terms: string[]): string {
|
||||
const seen = new Set<string>();
|
||||
const out: string[] = [];
|
||||
let length = 0;
|
||||
for (const term of terms) {
|
||||
const cleaned = term
|
||||
.replace(/,/g, ' ')
|
||||
.replace(/[^\x20-\x7E]/g, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
if (!cleaned || seen.has(cleaned)) continue;
|
||||
const cost = cleaned.length + (out.length > 0 ? 1 : 0);
|
||||
if (length + cost > MAX_KEYTERMS_HEADER_CHARS) break;
|
||||
seen.add(cleaned);
|
||||
out.push(cleaned);
|
||||
length += cost;
|
||||
}
|
||||
return out.join(',');
|
||||
}
|
||||
|
||||
/** Normalize a language hint to what the endpoint expects, defaulting to English. */
|
||||
export function normalizeVoiceLanguage(language: string | undefined): string {
|
||||
const trimmed = (language ?? '').trim();
|
||||
if (!trimmed || !/^[a-zA-Z]{2,3}(-[a-zA-Z0-9]{2,8})?$|^multi$/.test(trimmed)) return 'en';
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
/** Full upstream URL with the audio format pinned. */
|
||||
export function buildVoiceStreamUrl(params: VoiceStreamParams = {}, env: NodeJS.ProcessEnv = process.env): string {
|
||||
const query = new URLSearchParams({
|
||||
encoding: 'linear16',
|
||||
sample_rate: String(AUDIO_SAMPLE_RATE),
|
||||
channels: String(AUDIO_CHANNELS),
|
||||
endpointing_ms: '300',
|
||||
utterance_end_ms: '1000',
|
||||
language: normalizeVoiceLanguage(params.language),
|
||||
use_conversation_engine: 'true',
|
||||
stt_provider: 'deepgram-nova3',
|
||||
});
|
||||
return `${voiceStreamBase(env)}${VOICE_STREAM_PATH}?${query.toString()}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Upstream headers. Codeman identifies itself honestly (it is not the CLI), which
|
||||
* the endpoint accepts; the bearer token is the only thing that authenticates.
|
||||
*/
|
||||
export function buildVoiceStreamHeaders(
|
||||
accessToken: string,
|
||||
appVersion: string,
|
||||
keyterms: string[] = []
|
||||
): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
Authorization: `Bearer ${accessToken}`,
|
||||
'User-Agent': `codeman/${appVersion} (voice-bridge)`,
|
||||
'x-app': 'codeman',
|
||||
'anthropic-client-platform': 'codeman_web',
|
||||
};
|
||||
const sanitized = sanitizeKeyterms(keyterms);
|
||||
if (sanitized) headers['x-config-keyterms'] = sanitized;
|
||||
return headers;
|
||||
}
|
||||
|
||||
export interface VoiceStreamRelayOptions extends VoiceStreamParams {
|
||||
accessToken: string;
|
||||
appVersion: string;
|
||||
/** Called once the upstream socket is open and audio may flow. */
|
||||
onReady: () => void;
|
||||
/** Running transcript. `final` marks the utterance as complete. */
|
||||
onTranscript: (text: string, final: boolean) => void;
|
||||
/** Human-readable failure. The relay is dead (or dying) by the time this fires. */
|
||||
onError: (message: string) => void;
|
||||
/** Terminal: the relay released its socket and timers. Fires exactly once. */
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* One dictation, upstream. Owns exactly one WebSocket and dies with it: every
|
||||
* exit path (error, upstream close, lifetime cap, caller close) funnels through
|
||||
* `_teardown()`, which fires `onClose` once and clears both timers.
|
||||
*/
|
||||
export class VoiceStreamRelay {
|
||||
private ws: WebSocket | null = null;
|
||||
private keepAlive: ReturnType<typeof setInterval> | null = null;
|
||||
private lifetimeTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
private finalizeTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
private closed = false;
|
||||
private finalizing = false;
|
||||
/** Latest interim, held so a close/finalize can promote it to final. */
|
||||
private pendingTranscript = '';
|
||||
|
||||
constructor(private readonly opts: VoiceStreamRelayOptions) {}
|
||||
|
||||
/** Open the upstream socket. Safe to call once; a second call is a no-op. */
|
||||
connect(): void {
|
||||
if (this.ws || this.closed) return;
|
||||
const url = buildVoiceStreamUrl({ language: this.opts.language, keyterms: this.opts.keyterms });
|
||||
const ws = new WebSocket(url, {
|
||||
headers: buildVoiceStreamHeaders(this.opts.accessToken, this.opts.appVersion, this.opts.keyterms ?? []),
|
||||
});
|
||||
this.ws = ws;
|
||||
|
||||
ws.on('open', () => {
|
||||
// Ping immediately: the gap between upgrade and the browser's first audio
|
||||
// frame is long enough (mic permission, worklet boot) for upstream to drop us.
|
||||
this.safeSend(KEEPALIVE_FRAME);
|
||||
this.keepAlive = setInterval(() => this.safeSend(KEEPALIVE_FRAME), KEEPALIVE_INTERVAL_MS);
|
||||
this.lifetimeTimer = setTimeout(() => {
|
||||
this.opts.onError('Voice stream reached its maximum length');
|
||||
this.close();
|
||||
}, MAX_STREAM_MS);
|
||||
this.opts.onReady();
|
||||
});
|
||||
|
||||
ws.on('message', (raw) => this.handleMessage(String(raw)));
|
||||
|
||||
// An upgrade rejection never reaches 'open', so its status is the only signal
|
||||
// that the token was refused rather than the network being down.
|
||||
ws.on('unexpected-response', (_req, res) => {
|
||||
const status = res.statusCode ?? 0;
|
||||
res.resume();
|
||||
this.opts.onError(
|
||||
status === 401 || status === 403
|
||||
? 'Claude rejected the voice credentials. Run a Claude session to refresh your login.'
|
||||
: `Voice service refused the connection (HTTP ${status})`
|
||||
);
|
||||
this.teardown();
|
||||
});
|
||||
|
||||
ws.on('error', (err: Error) => {
|
||||
if (this.closed) return;
|
||||
this.opts.onError(`Voice stream error: ${err.message}`);
|
||||
});
|
||||
|
||||
ws.on('close', () => {
|
||||
this.promotePending();
|
||||
this.teardown();
|
||||
});
|
||||
}
|
||||
|
||||
/** Relay one raw PCM16 frame upstream. Dropped after finalize, as upstream ignores it. */
|
||||
sendAudio(chunk: Buffer): void {
|
||||
if (this.finalizing || this.closed) return;
|
||||
if (this.ws?.readyState !== WebSocket.OPEN) return;
|
||||
this.ws.send(chunk);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ask upstream for the final transcript. The endpoint frame usually follows
|
||||
* within a few hundred ms; the timer is the backstop so a silent upstream still
|
||||
* yields whatever interim we already have instead of hanging the caller.
|
||||
*/
|
||||
finalize(): void {
|
||||
if (this.finalizing || this.closed) return;
|
||||
this.finalizing = true;
|
||||
if (this.ws?.readyState !== WebSocket.OPEN) {
|
||||
this.promotePending();
|
||||
this.close();
|
||||
return;
|
||||
}
|
||||
this.safeSend(CLOSE_STREAM_FRAME);
|
||||
this.finalizeTimer = setTimeout(() => {
|
||||
this.promotePending();
|
||||
this.close();
|
||||
}, FINALIZE_TIMEOUT_MS);
|
||||
}
|
||||
|
||||
/** Terminal shutdown. Idempotent. */
|
||||
close(): void {
|
||||
if (this.closed) return;
|
||||
const ws = this.ws;
|
||||
this.teardown();
|
||||
if (ws && (ws.readyState === WebSocket.OPEN || ws.readyState === WebSocket.CONNECTING)) {
|
||||
try {
|
||||
ws.close();
|
||||
} catch {
|
||||
/* already closing */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private handleMessage(raw: string): void {
|
||||
let msg: { type?: string; data?: string; description?: string; error_code?: string; message?: string };
|
||||
try {
|
||||
msg = JSON.parse(raw);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
switch (msg.type) {
|
||||
case 'TranscriptText':
|
||||
case 'TranscriptInterim': {
|
||||
// Each frame is the whole running transcript, not a delta.
|
||||
if (typeof msg.data === 'string' && msg.data) {
|
||||
this.pendingTranscript = msg.data;
|
||||
this.opts.onTranscript(msg.data, false);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'TranscriptEndpoint': {
|
||||
this.promotePending();
|
||||
if (this.finalizing) this.close();
|
||||
break;
|
||||
}
|
||||
case 'TranscriptError': {
|
||||
this.opts.onError(msg.description || msg.error_code || 'Transcription failed');
|
||||
break;
|
||||
}
|
||||
case 'error': {
|
||||
this.opts.onError(msg.message || 'Voice service error');
|
||||
break;
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
/** Emit the held interim as final, exactly once per utterance. */
|
||||
private promotePending(): void {
|
||||
if (!this.pendingTranscript) return;
|
||||
const text = this.pendingTranscript;
|
||||
this.pendingTranscript = '';
|
||||
this.opts.onTranscript(text, true);
|
||||
}
|
||||
|
||||
private safeSend(frame: string): void {
|
||||
if (this.ws?.readyState !== WebSocket.OPEN) return;
|
||||
try {
|
||||
this.ws.send(frame);
|
||||
} catch {
|
||||
/* socket died between the check and the send */
|
||||
}
|
||||
}
|
||||
|
||||
private teardown(): void {
|
||||
if (this.closed) return;
|
||||
this.closed = true;
|
||||
if (this.keepAlive) clearInterval(this.keepAlive);
|
||||
if (this.lifetimeTimer) clearTimeout(this.lifetimeTimer);
|
||||
if (this.finalizeTimer) clearTimeout(this.finalizeTimer);
|
||||
this.keepAlive = null;
|
||||
this.lifetimeTimer = null;
|
||||
this.finalizeTimer = null;
|
||||
this.opts.onClose();
|
||||
}
|
||||
}
|
||||
+10
-5
@@ -135,11 +135,16 @@ describe('admin panel modal', () => {
|
||||
|
||||
describe('index.html wiring', () => {
|
||||
it('loads admin-ui.js after settings-ui.js and before session-ui.js', () => {
|
||||
const settings = INDEX_HTML.indexOf('settings-ui.js');
|
||||
const admin = INDEX_HTML.indexOf('admin-ui.js');
|
||||
const session = INDEX_HTML.indexOf('session-ui.js');
|
||||
expect(admin).toBeGreaterThan(settings);
|
||||
expect(session).toBeGreaterThan(admin);
|
||||
// Match the SCRIPT TAG, not the bare filename: modal markup earlier in the
|
||||
// document cites these modules in comments ("session-ui.js: openSessionOptions"),
|
||||
// and a bare indexOf finds the comment instead of the load order.
|
||||
const at = (file: string) => {
|
||||
const i = INDEX_HTML.indexOf(`src="${file}"`);
|
||||
expect(i, `no <script src="${file}"> in index.html`).toBeGreaterThan(-1);
|
||||
return i;
|
||||
};
|
||||
expect(at('admin-ui.js')).toBeGreaterThan(at('settings-ui.js'));
|
||||
expect(at('session-ui.js')).toBeGreaterThan(at('admin-ui.js'));
|
||||
});
|
||||
|
||||
it('ships the header Admin Panel button hidden by default', () => {
|
||||
|
||||
@@ -59,8 +59,28 @@ describe('App Settings modal structure', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('opens on Terminal & Input, so Local Echo is the first thing in reach', () => {
|
||||
expect(settingsUi).toContain("this.switchSettingsTab('settings-terminal')");
|
||||
it('opens on Updates: the version and the updater above everything else', () => {
|
||||
expect(settingsUi).toContain("this.switchSettingsTab('settings-updates')");
|
||||
const modal = settingsModal();
|
||||
const order = [...modal.matchAll(/<section class="set-section" id="([a-z-]+)"/g)].map((m) => m[1]);
|
||||
// Rail and document must agree, or scroll-spy paints the wrong entry.
|
||||
const rail = [...modal.matchAll(/data-section="([a-z-]+)"/g)].map((m) => m[1]);
|
||||
expect(rail.slice(0, 3)).toEqual(['settings-updates', 'settings-terminal', 'settings-layout']);
|
||||
expect(order.slice(0, 3)).toEqual(['settings-updates', 'settings-terminal', 'settings-layout']);
|
||||
// Updates carries ONLY the version and the update action; the rest of the
|
||||
// system settings tail the document under System, out of the way.
|
||||
const updates = modal.match(/id="settings-updates"([\s\S]*?)<\/section>/)?.[1] ?? '';
|
||||
expect(updates).toContain('id="updateCurrentVersion"');
|
||||
expect(updates).toContain('id="updateCheckBtn"');
|
||||
expect(updates).not.toContain('id="appSettingsClaudeMdPath"');
|
||||
expect(rail[rail.length - 1]).toBe('settings-system');
|
||||
expect(order[order.length - 1]).toBe('settings-system');
|
||||
const system = modal.match(/id="settings-system"([\s\S]*?)<\/section>/)?.[1] ?? '';
|
||||
expect(system).toContain('id="appSettingsClaudeMdPath"');
|
||||
expect(system).toContain('id="appSettingsTunnelEnabled"');
|
||||
});
|
||||
|
||||
it('keeps Local Echo the first row of the second section', () => {
|
||||
const terminal = settingsModal().match(/id="settings-terminal"([\s\S]*?)<\/section>/);
|
||||
const localEcho = terminal?.[1].indexOf('appSettingsLocalEcho') ?? -1;
|
||||
const cjk = terminal?.[1].indexOf('appSettingsCjkInput') ?? -1;
|
||||
@@ -68,6 +88,33 @@ describe('App Settings modal structure', () => {
|
||||
expect(localEcho).toBeLessThan(cjk);
|
||||
});
|
||||
|
||||
it('gives every previewed chip an icon to clone, and a slot that exists', () => {
|
||||
// _syncLayoutPreview clones `.set-chip-ico` out of the chip, so a chip that
|
||||
// opts into the preview without an icon renders as an empty button, and one
|
||||
// pointing at a slot id that does not exist renders as nothing at all.
|
||||
const layout = settingsModal().match(/id="settings-layout"([\s\S]*?)<\/section>/)?.[1] ?? '';
|
||||
const chips = [...layout.matchAll(/<label class="set-chip"([^>]*)>([\s\S]*?)<\/label>/g)];
|
||||
const previewed = chips.filter(([, attrs]) => attrs.includes('data-preview='));
|
||||
expect(previewed.length).toBeGreaterThanOrEqual(15);
|
||||
for (const [, attrs, body] of previewed) {
|
||||
const kind = attrs.match(/data-preview="([a-z]+)"/)?.[1];
|
||||
expect(['header', 'panel', 'toolbar', 'float']).toContain(kind);
|
||||
expect(attrs, `chip ${body} needs a preview order`).toMatch(/data-preview-order="\d+"/);
|
||||
// A text token replaces the icon for readouts (plan usage, CPU, font size).
|
||||
const hasIcon = body.includes('class="set-chip-ico') || attrs.includes('data-preview-text=');
|
||||
expect(hasIcon, `chip ${body} has nothing to render in the preview`).toBe(true);
|
||||
}
|
||||
for (const id of [
|
||||
'appSettingsPreviewHeader',
|
||||
'appSettingsPreviewPanels',
|
||||
'appSettingsPreviewToolbar',
|
||||
'appSettingsPreviewFloats',
|
||||
]) {
|
||||
expect(layout).toContain(`id="${id}"`);
|
||||
expect(settingsUi).toContain(`'${id}'`);
|
||||
}
|
||||
});
|
||||
|
||||
it('models: keeps the 1M variants as select options behind the context switch', () => {
|
||||
const modal = settingsModal();
|
||||
const select = modal.match(/id="appSettingsClaudeModel"([\s\S]*?)<\/select>/)?.[1] ?? '';
|
||||
@@ -80,15 +127,15 @@ describe('App Settings modal structure', () => {
|
||||
expect(modal).toContain('id="appSettingsOpusContext1m"');
|
||||
});
|
||||
|
||||
it('never hides sections behind .modal-tab-content (that class means display:none)', () => {
|
||||
it('has retired the modal-tab chrome everywhere, not just here', () => {
|
||||
// Session Options and Add Case moved onto this same `set-*` surface, so the
|
||||
// old tab classes have no users left. A reappearance means a modal drifted
|
||||
// back off the shared surface (or the dead CSS was resurrected).
|
||||
expect(settingsModal()).not.toContain('modal-tab-content');
|
||||
});
|
||||
|
||||
it('leaves the shared modal tab classes to the other modals', () => {
|
||||
// #sessionOptionsModal and #createCaseModal still use .modal-tabs; the
|
||||
// settings rail must not restyle them out from under those.
|
||||
expect(settingsModal()).not.toContain('class="modal-tabs"');
|
||||
expect(html).toContain('<div class="modal-tabs">');
|
||||
expect(html).not.toContain('class="modal-tabs"');
|
||||
expect(html).not.toContain('modal-tab-btn');
|
||||
const css = readFileSync(resolve(publicDir, 'styles.css'), 'utf8');
|
||||
expect(css).not.toContain('.modal-tab-btn {');
|
||||
});
|
||||
|
||||
it('exposes the rail hooks admin-ui.js injects the Users section into', () => {
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
/**
|
||||
* Claude Code credential parsing.
|
||||
*
|
||||
* The voice relay authenticates with the token this parser returns, so every
|
||||
* degraded store (absent, truncated, hand-edited, expired) must resolve to a
|
||||
* status the caller can act on rather than a throw or a silently empty token.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { parseClaudeCredentials, claudeCredentialsPath } from '../src/claude-credentials.js';
|
||||
|
||||
const NOW = 1_800_000_000_000;
|
||||
|
||||
function store(overrides: Record<string, unknown> = {}): string {
|
||||
return JSON.stringify({
|
||||
claudeAiOauth: {
|
||||
accessToken: 'sk-ant-oat01-test',
|
||||
refreshToken: 'sk-ant-ort01-test',
|
||||
expiresAt: NOW + 3_600_000,
|
||||
subscriptionType: 'max',
|
||||
...overrides,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
describe('parseClaudeCredentials', () => {
|
||||
it('returns the token and display metadata for a live store', () => {
|
||||
const result = parseClaudeCredentials(store(), NOW);
|
||||
expect(result.status).toBe('ok');
|
||||
expect(result.accessToken).toBe('sk-ant-oat01-test');
|
||||
expect(result.subscriptionType).toBe('max');
|
||||
expect(result.expiresAt).toBe(NOW + 3_600_000);
|
||||
});
|
||||
|
||||
it('reports an elapsed token as expired and withholds it', () => {
|
||||
const result = parseClaudeCredentials(store({ expiresAt: NOW - 1000 }), NOW);
|
||||
expect(result.status).toBe('expired');
|
||||
expect(result.accessToken).toBeUndefined();
|
||||
});
|
||||
|
||||
it('treats a token expiring within the skew as already expired', () => {
|
||||
// A token with 30s left would die mid-dictation; refusing up front turns a
|
||||
// confusing mid-utterance disconnect into a clear "refresh your login".
|
||||
expect(parseClaudeCredentials(store({ expiresAt: NOW + 30_000 }), NOW).status).toBe('expired');
|
||||
});
|
||||
|
||||
it('accepts a store with no expiry at all', () => {
|
||||
const raw = JSON.stringify({ claudeAiOauth: { accessToken: 'sk-ant-oat01-test' } });
|
||||
expect(parseClaudeCredentials(raw, NOW).status).toBe('ok');
|
||||
});
|
||||
|
||||
it.each([
|
||||
['not json at all', 'malformed'],
|
||||
['{}', 'malformed'],
|
||||
['null', 'malformed'],
|
||||
['[]', 'malformed'],
|
||||
['{"claudeAiOauth":null}', 'malformed'],
|
||||
['{"claudeAiOauth":{}}', 'malformed'],
|
||||
['{"claudeAiOauth":{"accessToken":""}}', 'malformed'],
|
||||
['{"claudeAiOauth":{"accessToken":" "}}', 'malformed'],
|
||||
['{"claudeAiOauth":{"accessToken":123}}', 'malformed'],
|
||||
])('reports %s as malformed instead of throwing', (raw, expected) => {
|
||||
expect(parseClaudeCredentials(raw, NOW).status).toBe(expected);
|
||||
});
|
||||
|
||||
it('trims whitespace around a token written by hand', () => {
|
||||
const raw = JSON.stringify({ claudeAiOauth: { accessToken: ' sk-ant-oat01-test\n' } });
|
||||
expect(parseClaudeCredentials(raw, NOW).accessToken).toBe('sk-ant-oat01-test');
|
||||
});
|
||||
});
|
||||
|
||||
describe('claudeCredentialsPath', () => {
|
||||
it('honors CLAUDE_CONFIG_DIR like the CLI does', () => {
|
||||
expect(claudeCredentialsPath({ CLAUDE_CONFIG_DIR: '/tmp/alt-claude' })).toBe('/tmp/alt-claude/.credentials.json');
|
||||
});
|
||||
|
||||
it('falls back to ~/.claude when the override is blank', () => {
|
||||
expect(claudeCredentialsPath({ CLAUDE_CONFIG_DIR: ' ' })).toMatch(/\.claude\/\.credentials\.json$/);
|
||||
});
|
||||
|
||||
it('falls back to ~/.claude when unset', () => {
|
||||
expect(claudeCredentialsPath({})).toMatch(/\.claude\/\.credentials\.json$/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,166 @@
|
||||
/**
|
||||
* @fileoverview Issue #273 and its mirror image: abbreviating `$HOME` in path labels.
|
||||
*
|
||||
* The rule ("show `~/project` rather than `/home/<user>/project`") had three
|
||||
* implementations in the frontend, and two of them were platform-specific in
|
||||
* opposite directions, so each looked correct to whoever wrote it:
|
||||
*
|
||||
* - the Run menu's Recent Sessions rows matched `/home/<user>/` only, so on
|
||||
* macOS nothing was stripped, every row spent its first ~19 characters on an
|
||||
* identical `/Users/<user>/` prefix, and the left-to-right ellipsis removed
|
||||
* the tail that identifies the row (#273),
|
||||
* - the case-manage list matched `/Users/<user>` only, so on a Linux host no
|
||||
* case path was ever abbreviated at all.
|
||||
*
|
||||
* Both now call `_shortenHomePath()`, which is pinned here for both layouts, and
|
||||
* a static guard fails if a fourth copy of the pattern appears.
|
||||
*
|
||||
* Loaded via `vm` against a stub CodemanApp with a fake DOM, same harness as
|
||||
* history-list-controls.test.ts. Port: none (no browser, no server).
|
||||
*/
|
||||
|
||||
import { readdirSync, readFileSync } from 'node:fs';
|
||||
import { resolve } from 'node:path';
|
||||
import vm from 'node:vm';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
/* eslint-disable @typescript-eslint/no-explicit-any */
|
||||
|
||||
const PUBLIC = resolve(import.meta.dirname, '../src/web/public');
|
||||
|
||||
/**
|
||||
* The container the vm's `document.getElementById` resolves for the case list.
|
||||
* Swapped per test: the closure lives in THIS realm, so the shipping code inside
|
||||
* the vm reads whatever the current test installed.
|
||||
*/
|
||||
let currentCaseList: { innerHTML: string } | null = null;
|
||||
|
||||
function loadTerminalUiPrototype(): Record<string, any> {
|
||||
const source = readFileSync(resolve(PUBLIC, 'terminal-ui.js'), 'utf8');
|
||||
const context = vm.createContext({
|
||||
console,
|
||||
CodemanApp: class CodemanApp {},
|
||||
setInterval: vi.fn(),
|
||||
clearInterval: vi.fn(),
|
||||
setTimeout,
|
||||
clearTimeout,
|
||||
requestAnimationFrame: vi.fn(),
|
||||
document: { addEventListener: vi.fn(), getElementById: () => null, createElement: () => ({}) },
|
||||
window: { addEventListener: vi.fn(), removeEventListener: vi.fn() },
|
||||
});
|
||||
vm.runInContext(`${source}\nglobalThis.__proto = CodemanApp.prototype;`, context);
|
||||
return (context as unknown as { __proto: Record<string, any> }).__proto;
|
||||
}
|
||||
|
||||
function loadSessionUiPrototype(): Record<string, any> {
|
||||
const source = readFileSync(resolve(PUBLIC, 'session-ui.js'), 'utf8');
|
||||
const context = vm.createContext({
|
||||
console,
|
||||
CodemanApp: class CodemanApp {},
|
||||
VoiceInput: {},
|
||||
escapeHtml: (t: unknown) => String(t ?? ''),
|
||||
setTimeout,
|
||||
clearTimeout,
|
||||
localStorage: { getItem: () => null, setItem: () => {} },
|
||||
document: { getElementById: (id: string) => (id === 'caseManageList' ? currentCaseList : null) },
|
||||
window: { addEventListener: vi.fn() },
|
||||
});
|
||||
vm.runInContext(`${source}\nglobalThis.__proto = CodemanApp.prototype;`, context);
|
||||
return (context as unknown as { __proto: Record<string, any> }).__proto;
|
||||
}
|
||||
|
||||
const terminalProto = loadTerminalUiPrototype();
|
||||
const sessionProto = loadSessionUiPrototype();
|
||||
const shorten = (p: unknown) => terminalProto._shortenHomePath.call(terminalProto, p);
|
||||
|
||||
describe('_shortenHomePath', () => {
|
||||
it('abbreviates the Linux home prefix', () => {
|
||||
expect(shorten('/home/arkon/default/claudeman')).toBe('~/default/claudeman');
|
||||
});
|
||||
|
||||
it('abbreviates the macOS home prefix, which the Run menu never did (#273)', () => {
|
||||
expect(shorten('/Users/jordanryan/code/facet/facet-agency-ops')).toBe('~/code/facet/facet-agency-ops');
|
||||
});
|
||||
|
||||
it('abbreviates the home directory itself, not only paths below it', () => {
|
||||
// The case-manage list's old regex had no trailing slash and did collapse
|
||||
// this to "~"; keep that, or a case whose path IS $HOME would regress.
|
||||
expect(shorten('/home/arkon')).toBe('~');
|
||||
expect(shorten('/Users/jordanryan')).toBe('~');
|
||||
});
|
||||
|
||||
it('leaves paths that only look like a home prefix alone', () => {
|
||||
expect(shorten('/homer/bob/x')).toBe('/homer/bob/x');
|
||||
expect(shorten('/Userspace/bob/x')).toBe('/Userspace/bob/x');
|
||||
expect(shorten('/home')).toBe('/home');
|
||||
expect(shorten('/mnt/d/work')).toBe('/mnt/d/work');
|
||||
expect(shorten('/opt/codeman')).toBe('/opt/codeman');
|
||||
});
|
||||
|
||||
it('replaces only the leading occurrence', () => {
|
||||
expect(shorten('/home/arkon/home/bob/x')).toBe('~/home/bob/x');
|
||||
});
|
||||
|
||||
it('tolerates empty and missing input', () => {
|
||||
expect(shorten('')).toBe('');
|
||||
expect(shorten(undefined)).toBe('');
|
||||
expect(shorten(null)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('renderCaseManageList path labels', () => {
|
||||
function render(cases: Array<{ name: string; path: string; location?: string }>): string {
|
||||
currentCaseList = { innerHTML: '' };
|
||||
const app: any = {
|
||||
cases,
|
||||
_shortenHomePath: terminalProto._shortenHomePath,
|
||||
renderCaseManageList: sessionProto.renderCaseManageList,
|
||||
};
|
||||
app.renderCaseManageList();
|
||||
const html = currentCaseList.innerHTML;
|
||||
currentCaseList = null;
|
||||
return html;
|
||||
}
|
||||
|
||||
it('abbreviates a Linux case path (the mirror of #273)', () => {
|
||||
const html = render([{ name: 'demo', path: '/home/arkon/codeman-cases/demo' }]);
|
||||
expect(html).toContain('~/codeman-cases/demo');
|
||||
expect(html).not.toContain('/home/arkon/codeman-cases/demo');
|
||||
});
|
||||
|
||||
it('still abbreviates a macOS case path', () => {
|
||||
const html = render([{ name: 'demo', path: '/Users/jordanryan/codeman-cases/demo' }]);
|
||||
expect(html).toContain('~/codeman-cases/demo');
|
||||
expect(html).not.toContain('/Users/jordanryan/codeman-cases/demo');
|
||||
});
|
||||
|
||||
it('renders the row when a case has no path at all', () => {
|
||||
const html = render([{ name: 'demo', path: '' }]);
|
||||
expect(html).toContain('demo');
|
||||
expect(html).toContain('class="case-manage-path"');
|
||||
});
|
||||
});
|
||||
|
||||
describe('single implementation of the home-prefix rule', () => {
|
||||
/** Every top-level frontend module (vendor/ and subdirs are not ours). */
|
||||
const sources = readdirSync(PUBLIC)
|
||||
.filter((name) => name.endsWith('.js'))
|
||||
.map((name) => ({ name, text: readFileSync(resolve(PUBLIC, name), 'utf8') }));
|
||||
|
||||
it('has exactly one home-prefix regex, in terminal-ui.js', () => {
|
||||
// Any regex literal anchored at a home root. Three of these had drifted
|
||||
// apart; a fourth would drift the same way.
|
||||
const pattern = /\/\^\\\/(?:\(\?:home\|Users\)|home|Users)\\\//g;
|
||||
const hits = sources.flatMap(({ name, text }) => (text.match(pattern) ?? []).map(() => name));
|
||||
expect(hits).toEqual(['terminal-ui.js']);
|
||||
});
|
||||
|
||||
it('routes both session-ui path labels through the helper', () => {
|
||||
// Deliberately counts calls rather than pinning source lines: the Run menu
|
||||
// row is being restructured in #274, and this guard should survive that as
|
||||
// long as the label still goes through the helper.
|
||||
const sessionUi = sources.find((s) => s.name === 'session-ui.js')!.text;
|
||||
const calls = sessionUi.match(/this\._shortenHomePath\(/g) ?? [];
|
||||
expect(calls.length).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
});
|
||||
@@ -624,6 +624,122 @@ describe('Virtual Keyboard', () => {
|
||||
expect(Number(styles?.zIndex)).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
it('dismisses the on-screen keyboard when a tap lands outside the terminal', async () => {
|
||||
// The terminal holds focus on a hidden textarea and nothing released it,
|
||||
// so once the keyboard was up every tap on the header or page chrome left
|
||||
// it up — covering half a phone screen with no in-app way to close it.
|
||||
//
|
||||
// Driven as a real dispatched gesture: the handler is bound to touchend on
|
||||
// document, and calling the internal helper would bypass the routing this
|
||||
// test exists to check.
|
||||
const result = await page.evaluate(async () => {
|
||||
const sampleX = (rect: DOMRect) => Math.max(2, rect.left + Math.min(6, rect.width / 2));
|
||||
const sampleY = (rect: DOMRect) => Math.max(2, rect.top + Math.min(6, rect.height / 2));
|
||||
const tap = async (el: Element, travel = 0, point?: { x: number; y: number }) => {
|
||||
const rect = el.getBoundingClientRect();
|
||||
const x = point ? point.x : sampleX(rect);
|
||||
const y = point ? point.y : sampleY(rect);
|
||||
const target = document.elementFromPoint(x, y) || el;
|
||||
const at = (cy: number) => new Touch({ identifier: 21, target, clientX: x, clientY: cy });
|
||||
target.dispatchEvent(
|
||||
new TouchEvent('touchstart', {
|
||||
touches: [at(y)],
|
||||
targetTouches: [at(y)],
|
||||
changedTouches: [at(y)],
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
})
|
||||
);
|
||||
for (const step of travel ? [travel / 3, (travel * 2) / 3, travel] : []) {
|
||||
target.dispatchEvent(
|
||||
new TouchEvent('touchmove', {
|
||||
touches: [at(y + step)],
|
||||
targetTouches: [at(y + step)],
|
||||
changedTouches: [at(y + step)],
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
})
|
||||
);
|
||||
await new Promise((resolve) => setTimeout(resolve, 15));
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 25));
|
||||
target.dispatchEvent(
|
||||
new TouchEvent('touchend', {
|
||||
touches: [],
|
||||
changedTouches: [at(y + travel)],
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
})
|
||||
);
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
return document.activeElement?.className ?? '';
|
||||
};
|
||||
|
||||
app.hideWelcome();
|
||||
app.terminal.reset();
|
||||
await new Promise<void>((resolve) => app.terminal.write('transcript\r\n\r\n> ', resolve));
|
||||
|
||||
// Inert page chrome: the keyboard must close.
|
||||
app._focusMobileTerminalInput();
|
||||
const focusedBefore = document.activeElement?.className ?? '';
|
||||
const afterOutside = await tap(document.querySelector('.logo, .header-brand, header') ?? document.body);
|
||||
|
||||
// A real control: it takes focus itself, so we must NOT interfere.
|
||||
app._focusMobileTerminalInput();
|
||||
const button = Array.from(document.querySelectorAll('button:not([disabled])')).find((candidate) => {
|
||||
const rect = candidate.getBoundingClientRect();
|
||||
if (rect.width <= 8 || rect.height <= 8) return false;
|
||||
// A rect is not enough. The welcome overlay is hidden by hideWelcome()
|
||||
// above but its buttons still MEASURE, so a rect-only pick sampled a
|
||||
// point the terminal actually owns — elementFromPoint returned
|
||||
// .xterm-screen and this case tapped the terminal instead of a
|
||||
// control, passing for the wrong reason. Require the sampled point to
|
||||
// really resolve to this button.
|
||||
const hit = document.elementFromPoint(sampleX(rect), sampleY(rect));
|
||||
return !!hit && candidate.contains(hit);
|
||||
});
|
||||
const afterButton = button ? await tap(button) : 'no-visible-button';
|
||||
|
||||
// Inside the terminal, tap classification owns the decision, so this
|
||||
// handler must keep its hands off. Aimed at the PROMPT row: that is the
|
||||
// one in-terminal tap whose outcome belongs to nobody else, since an
|
||||
// inert transcript row is claimed by the in-terminal dismiss toggle
|
||||
// (`toggles the keyboard shut on a second inert Claude transcript tap`)
|
||||
// and asserting focus there would be asserting that toggle's behaviour
|
||||
// rather than this exemption. The guard still bites: the container's own
|
||||
// touchend listener runs first and refocuses, so a missing
|
||||
// #terminalContainer exemption would blur right back over it.
|
||||
app._focusMobileTerminalInput();
|
||||
const screen = app.terminal.element?.querySelector('.xterm-screen');
|
||||
const cell = app.terminal._core?._renderService?.dimensions?.css?.cell;
|
||||
const screenRect = screen?.getBoundingClientRect();
|
||||
const promptPoint =
|
||||
screenRect && cell?.width && cell?.height
|
||||
? {
|
||||
x: screenRect.left + cell.width * 2,
|
||||
y: screenRect.top + cell.height * (app.terminal.buffer.active.cursorY + 0.5),
|
||||
}
|
||||
: undefined;
|
||||
const afterTerminal = await tap(document.querySelector('#terminalContainer')!, 0, promptPoint);
|
||||
|
||||
// A SCROLL also ends in touchend. Scrolling to read something while
|
||||
// composing must not close the keyboard and drop the composer.
|
||||
app._focusMobileTerminalInput();
|
||||
const afterScroll = await tap(document.querySelector('.logo, .header-brand, header') ?? document.body, 120);
|
||||
|
||||
return { focusedBefore, afterOutside, afterButton, afterTerminal, afterScroll };
|
||||
});
|
||||
|
||||
expect(result.focusedBefore).toContain('xterm-helper-textarea');
|
||||
// Red on master: the textarea keeps focus and the keyboard stays up.
|
||||
expect(result.afterOutside).not.toContain('xterm-helper-textarea');
|
||||
expect(result.afterButton).not.toBe('no-visible-button');
|
||||
expect(result.afterButton).toContain('xterm-helper-textarea');
|
||||
expect(result.afterTerminal).toContain('xterm-helper-textarea');
|
||||
// A scroll ends in touchend too, and must NOT close the keyboard.
|
||||
expect(result.afterScroll).toContain('xterm-helper-textarea');
|
||||
});
|
||||
|
||||
it('routes CJK textarea typing through local echo on Enter', async () => {
|
||||
await page.evaluate(() => {
|
||||
window.__sentInputs = [];
|
||||
@@ -845,6 +961,287 @@ describe('Virtual Keyboard', () => {
|
||||
expect(state.sentInputs).toEqual([]);
|
||||
});
|
||||
|
||||
it('collapses a terminal readback without focusing the hidden textarea', async () => {
|
||||
const point = await page.evaluate(async () => {
|
||||
window.__sentInputs = [];
|
||||
app.activeSessionId = 'mobile-readback-tap-test';
|
||||
app.sessions.set('mobile-readback-tap-test', {
|
||||
id: 'mobile-readback-tap-test',
|
||||
mode: 'codex',
|
||||
status: 'running',
|
||||
});
|
||||
app._sendInputAsync = (_sessionId: string, input: string) => {
|
||||
window.__sentInputs.push(input);
|
||||
};
|
||||
app.hideWelcome();
|
||||
const settings = app.loadAppSettingsFromStorage();
|
||||
settings.cjkInputEnabled = false;
|
||||
app.saveAppSettingsToStorage(settings);
|
||||
app._updateCjkInputState();
|
||||
app.terminal.reset();
|
||||
await new Promise<void>((resolve) =>
|
||||
app.terminal.write('Agent readback\r\n tap to collapse\r\n\r\n› ask', resolve)
|
||||
);
|
||||
app.terminal.focus();
|
||||
|
||||
const screen = app.terminal.element?.querySelector('.xterm-screen');
|
||||
const cell = app.terminal._core?._renderService?.dimensions?.css?.cell;
|
||||
const rect = screen?.getBoundingClientRect();
|
||||
if (!rect || !cell?.width || !cell?.height) return null;
|
||||
return {
|
||||
x: rect.left + cell.width * 2,
|
||||
y: rect.top + cell.height / 2,
|
||||
};
|
||||
});
|
||||
expect(point).not.toBeNull();
|
||||
|
||||
await page.touchscreen.tap(point!.x, point!.y);
|
||||
|
||||
const state = await page.evaluate(() => ({
|
||||
activeClass: document.activeElement?.className,
|
||||
sentInputs: window.__sentInputs,
|
||||
}));
|
||||
expect(state.activeClass).not.toContain('xterm-helper-textarea');
|
||||
expect(state.sentInputs).toHaveLength(1);
|
||||
expect(state.sentInputs[0]).toMatch(/^\x1b\[<0;\d+;1M\x1b\[<0;\d+;1m$/);
|
||||
});
|
||||
|
||||
it('toggles the keyboard shut on a second inert Claude transcript tap', async () => {
|
||||
const point = await page.evaluate(async () => {
|
||||
window.__sentInputs = [];
|
||||
app.activeSessionId = 'mobile-claude-transcript-tap-test';
|
||||
app.sessions.set('mobile-claude-transcript-tap-test', {
|
||||
id: 'mobile-claude-transcript-tap-test',
|
||||
mode: 'claude',
|
||||
cliVersion: '2.1.220',
|
||||
status: 'working',
|
||||
});
|
||||
app._sendInputAsync = (_sessionId: string, input: string) => {
|
||||
window.__sentInputs.push(input);
|
||||
};
|
||||
app.hideWelcome();
|
||||
const settings = app.loadAppSettingsFromStorage();
|
||||
settings.cjkInputEnabled = false;
|
||||
app.saveAppSettingsToStorage(settings);
|
||||
app._updateCjkInputState();
|
||||
app.terminal.reset();
|
||||
await new Promise<void>((resolve) =>
|
||||
app.terminal.write(
|
||||
'Transcript row one\r\nTranscript row two\r\nTranscript row three\r\nTranscript row four\r\nTranscript row five\r\nTranscript row six\r\nTranscript row seven\r\nTranscript row eight\r\nTranscript row nine\r\nTranscript row ten\r\n\r\n❯ ',
|
||||
resolve
|
||||
)
|
||||
);
|
||||
app.terminal.focus();
|
||||
|
||||
const screen = app.terminal.element?.querySelector('.xterm-screen');
|
||||
const cell = app.terminal._core?._renderService?.dimensions?.css?.cell;
|
||||
const rect = screen?.getBoundingClientRect();
|
||||
if (!screen || !rect || !cell?.width || !cell?.height) return null;
|
||||
const cursorRow = app.terminal.buffer.active.cursorY;
|
||||
const transcriptRow = Math.max(1, Math.floor(cursorRow / 2));
|
||||
const x = rect.left + cell.width * 2;
|
||||
const y = rect.top + cell.height * (transcriptRow + 0.5);
|
||||
return {
|
||||
x,
|
||||
y,
|
||||
intent: app._classifyMobileTerminalTap(x, y),
|
||||
activeClass: document.activeElement?.className,
|
||||
};
|
||||
});
|
||||
expect(point).toEqual(
|
||||
expect.objectContaining({
|
||||
intent: 'content',
|
||||
activeClass: expect.stringContaining('xterm-helper-textarea'),
|
||||
})
|
||||
);
|
||||
|
||||
await page.touchscreen.tap(point!.x, point!.y);
|
||||
|
||||
// The setup above leaves the terminal focused, so this tap is the SECOND
|
||||
// one on an inert row — the case that now closes the keyboard. Previously
|
||||
// it re-focused, which left the accessory bar's chevron as the only way to
|
||||
// dismiss. The prompt row is unaffected and still positions the caret.
|
||||
const activeClass = await page.evaluate(() => document.activeElement?.className);
|
||||
expect(activeClass).not.toContain('xterm-helper-textarea');
|
||||
});
|
||||
|
||||
it('prevents Claude subagent status taps from opening the hidden keyboard input', async () => {
|
||||
const point = await page.evaluate(async () => {
|
||||
window.__sentInputs = [];
|
||||
app.activeSessionId = 'mobile-claude-subagent-tap-test';
|
||||
app.sessions.set('mobile-claude-subagent-tap-test', {
|
||||
id: 'mobile-claude-subagent-tap-test',
|
||||
mode: 'claude',
|
||||
cliVersion: '2.1.220',
|
||||
status: 'working',
|
||||
});
|
||||
app._sendInputAsync = (_sessionId: string, input: string) => {
|
||||
window.__sentInputs.push(input);
|
||||
};
|
||||
app.hideWelcome();
|
||||
app.terminal.reset();
|
||||
const statusRow = Math.max(0, app.terminal.rows - 2);
|
||||
await new Promise<void>((resolve) =>
|
||||
app.terminal.write(
|
||||
`${'\r\n'.repeat(statusRow)}• Working (1m 50s • esc to interrupt) · 1 background teammate`,
|
||||
resolve
|
||||
)
|
||||
);
|
||||
app.terminal.focus();
|
||||
|
||||
const screen = app.terminal.element?.querySelector('.xterm-screen');
|
||||
const cell = app.terminal._core?._renderService?.dimensions?.css?.cell;
|
||||
const rect = screen?.getBoundingClientRect();
|
||||
if (!screen || !rect || !cell?.width || !cell?.height) return null;
|
||||
const cursorRow = app.terminal.buffer.active.cursorY;
|
||||
const x = rect.left + cell.width * 2;
|
||||
const y = rect.top + cell.height * (cursorRow + 0.5);
|
||||
return {
|
||||
x,
|
||||
y,
|
||||
intent: app._classifyMobileTerminalTap(x, y),
|
||||
cursorRow,
|
||||
screenBottom: rect.bottom,
|
||||
};
|
||||
});
|
||||
expect(point).toEqual(
|
||||
expect.objectContaining({
|
||||
intent: 'content',
|
||||
})
|
||||
);
|
||||
|
||||
const dispatch = await page.evaluate(({ x, y }) => {
|
||||
const target = document.querySelector('#terminalContainer .xterm-screen');
|
||||
if (!(target instanceof Element)) {
|
||||
return { prevented: false, insideTerminal: false, targetClass: null };
|
||||
}
|
||||
const touch = new Touch({
|
||||
identifier: 3,
|
||||
target,
|
||||
clientX: x,
|
||||
clientY: y,
|
||||
pageX: x,
|
||||
pageY: y,
|
||||
});
|
||||
const allowed = target.dispatchEvent(
|
||||
new TouchEvent('touchstart', {
|
||||
touches: [touch],
|
||||
changedTouches: [touch],
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
})
|
||||
);
|
||||
target.dispatchEvent(
|
||||
new TouchEvent('touchend', {
|
||||
touches: [],
|
||||
changedTouches: [touch],
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
})
|
||||
);
|
||||
return {
|
||||
prevented: !allowed,
|
||||
insideTerminal: Boolean(target.closest('#terminalContainer')),
|
||||
targetClass: target.className,
|
||||
};
|
||||
}, point!);
|
||||
|
||||
const state = await page.evaluate(() => ({
|
||||
activeClass: document.activeElement?.className,
|
||||
sentInputs: window.__sentInputs,
|
||||
}));
|
||||
expect(dispatch).toEqual(
|
||||
expect.objectContaining({
|
||||
prevented: true,
|
||||
insideTerminal: true,
|
||||
})
|
||||
);
|
||||
expect(state.activeClass).not.toContain('xterm-helper-textarea');
|
||||
expect(state.sentInputs).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('focuses the terminal helper textarea when the visible prompt is tapped', async () => {
|
||||
const point = await page.evaluate(async () => {
|
||||
window.__sentInputs = [];
|
||||
app.activeSessionId = 'mobile-focus-visible-input-test';
|
||||
app.sessions.set('mobile-focus-visible-input-test', {
|
||||
id: 'mobile-focus-visible-input-test',
|
||||
mode: 'codex',
|
||||
status: 'running',
|
||||
});
|
||||
app._sendInputAsync = (_sessionId: string, input: string) => {
|
||||
window.__sentInputs.push(input);
|
||||
};
|
||||
app.hideWelcome();
|
||||
const settings = app.loadAppSettingsFromStorage();
|
||||
settings.cjkInputEnabled = false;
|
||||
app.saveAppSettingsToStorage(settings);
|
||||
app._updateCjkInputState();
|
||||
app.terminal.reset();
|
||||
await new Promise<void>((resolve) =>
|
||||
app.terminal.write('Agent readback\r\n tap to collapse\r\n\r\n› ask', resolve)
|
||||
);
|
||||
(document.activeElement as HTMLElement | null)?.blur?.();
|
||||
|
||||
const screen = app.terminal.element?.querySelector('.xterm-screen');
|
||||
const cell = app.terminal._core?._renderService?.dimensions?.css?.cell;
|
||||
const rect = screen?.getBoundingClientRect();
|
||||
if (!rect || !cell?.width || !cell?.height) return null;
|
||||
return {
|
||||
x: rect.left + cell.width * 2,
|
||||
y: rect.top + cell.height * (app.terminal.buffer.active.cursorY + 0.5),
|
||||
};
|
||||
});
|
||||
expect(point).not.toBeNull();
|
||||
|
||||
await page.touchscreen.tap(point!.x, point!.y);
|
||||
|
||||
const state = await page.evaluate(() => ({
|
||||
activeClass: document.activeElement?.className,
|
||||
sentInputs: window.__sentInputs,
|
||||
}));
|
||||
expect(state.activeClass).toContain('xterm-helper-textarea');
|
||||
expect(state.sentInputs).toEqual([]);
|
||||
});
|
||||
|
||||
it('focuses the live Claude cursor when a redraw omits the prompt glyph', async () => {
|
||||
const point = await page.evaluate(async () => {
|
||||
window.__sentInputs = [];
|
||||
app.activeSessionId = 'mobile-focus-promptless-claude-test';
|
||||
app.sessions.set('mobile-focus-promptless-claude-test', {
|
||||
id: 'mobile-focus-promptless-claude-test',
|
||||
mode: 'claude',
|
||||
status: 'running',
|
||||
});
|
||||
app._sendInputAsync = (_sessionId: string, input: string) => {
|
||||
window.__sentInputs.push(input);
|
||||
};
|
||||
app.hideWelcome();
|
||||
app.terminal.reset();
|
||||
await new Promise<void>((resolve) => app.terminal.write('Claude response\r\nready for input', resolve));
|
||||
(document.activeElement as HTMLElement | null)?.blur?.();
|
||||
|
||||
const screen = app.terminal.element?.querySelector('.xterm-screen');
|
||||
const cell = app.terminal._core?._renderService?.dimensions?.css?.cell;
|
||||
const rect = screen?.getBoundingClientRect();
|
||||
if (!rect || !cell?.width || !cell?.height) return null;
|
||||
return {
|
||||
x: rect.left + cell.width * 2,
|
||||
y: rect.top + cell.height * (app.terminal.buffer.active.cursorY + 0.5),
|
||||
};
|
||||
});
|
||||
expect(point).not.toBeNull();
|
||||
|
||||
await page.touchscreen.tap(point!.x, point!.y);
|
||||
|
||||
const state = await page.evaluate(() => ({
|
||||
activeClass: document.activeElement?.className,
|
||||
sentInputs: window.__sentInputs,
|
||||
}));
|
||||
expect(state.activeClass).toContain('xterm-helper-textarea');
|
||||
expect(state.sentInputs).toEqual([]);
|
||||
});
|
||||
|
||||
it('keeps terminal touch drag available for scrollback with the visible textarea enabled', async () => {
|
||||
const calls = await page.evaluate(async () => {
|
||||
app.activeSessionId = 'mobile-touch-scroll-test';
|
||||
|
||||
@@ -15,7 +15,11 @@ import { resolveTerminalHistoryConfig } from '../../src/config/terminal-history.
|
||||
* Creates a mock context that satisfies all port interfaces.
|
||||
* Pre-populated with one session for convenience.
|
||||
*/
|
||||
export function createMockRouteContext(options?: { sessionId?: string; agentSkillEnabled?: boolean }) {
|
||||
export function createMockRouteContext(options?: {
|
||||
sessionId?: string;
|
||||
agentSkillEnabled?: boolean;
|
||||
claudeVoiceEnabled?: boolean;
|
||||
}) {
|
||||
const sessionId = options?.sessionId ?? 'test-session-1';
|
||||
const session = createMockSession(sessionId);
|
||||
const sessions = new Map<string, MockSession>();
|
||||
@@ -90,6 +94,8 @@ export function createMockRouteContext(options?: { sessionId?: string; agentSkil
|
||||
// case's .claude/skills. Overridable per test because the create-time
|
||||
// injection call sites are otherwise unreachable from a route test.
|
||||
getAgentSkillEnabled: vi.fn(async () => options?.agentSkillEnabled ?? false),
|
||||
// Default OFF mirrors the shipped setting: no test opens a voice relay by accident.
|
||||
getClaudeVoiceEnabled: vi.fn(async () => options?.claudeVoiceEnabled ?? false),
|
||||
getDefaultClaudeMdPath: vi.fn(async () => undefined),
|
||||
getLightState: vi.fn(() => ({ sessions: [], status: 'ok' })),
|
||||
getLightSessionsState: vi.fn(() => {
|
||||
|
||||
@@ -0,0 +1,302 @@
|
||||
/**
|
||||
* @fileoverview Claude voice dictation routes.
|
||||
*
|
||||
* Covers the status endpoint's gating and the full relay round trip against a
|
||||
* mock upstream (a local `ws` server speaking the Anthropic voice-stream
|
||||
* protocol, selected via CODEMAN_VOICE_STREAM_BASE). WebSocket routes need a
|
||||
* real listening server — app.inject() cannot do upgrades.
|
||||
*
|
||||
* What these pin, beyond "it works":
|
||||
* - the OAuth token never appears in an API response,
|
||||
* - the socket refuses exactly what the status endpoint calls unavailable,
|
||||
* - a cross-site upgrade cannot open a stream on the operator's subscription.
|
||||
*
|
||||
* Port: 3230 (routes), 3231 (mock upstream)
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
||||
import Fastify, { type FastifyInstance } from 'fastify';
|
||||
import fastifyWebsocket from '@fastify/websocket';
|
||||
import WebSocket, { WebSocketServer } from 'ws';
|
||||
import { mkdirSync, writeFileSync, rmSync } from 'node:fs';
|
||||
import { homedir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { createMockRouteContext, type MockRouteContext } from '../mocks/index.js';
|
||||
import { registerVoiceRoutes, _resetVoiceStreamCountForTesting } from '../../src/web/routes/voice-routes.js';
|
||||
import { MAX_CONCURRENT_STREAMS } from '../../src/config/voice.js';
|
||||
|
||||
const PORT = 3230;
|
||||
const UPSTREAM_PORT = 3231;
|
||||
const TOKEN = 'sk-ant-oat01-voice-route-test';
|
||||
|
||||
/** State captured by the mock upstream, so tests can assert what Codeman sent. */
|
||||
interface UpstreamCapture {
|
||||
headers: Record<string, string | string[] | undefined>;
|
||||
url: string;
|
||||
binaryFrames: Buffer[];
|
||||
textFrames: string[];
|
||||
socket: WebSocket | null;
|
||||
}
|
||||
|
||||
function writeCredentials(expiresAt: number | undefined): void {
|
||||
const dir = join(homedir(), '.claude');
|
||||
mkdirSync(dir, { recursive: true });
|
||||
writeFileSync(
|
||||
join(dir, '.credentials.json'),
|
||||
JSON.stringify({ claudeAiOauth: { accessToken: TOKEN, expiresAt, subscriptionType: 'max' } })
|
||||
);
|
||||
}
|
||||
|
||||
function removeCredentials(): void {
|
||||
rmSync(join(homedir(), '.claude', '.credentials.json'), { force: true });
|
||||
}
|
||||
|
||||
function waitForClose(ws: WebSocket, timeoutMs = 3000): Promise<{ code: number; reason: string }> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const timer = setTimeout(() => reject(new Error('WS close timeout')), timeoutMs);
|
||||
ws.on('close', (code, reason) => {
|
||||
clearTimeout(timer);
|
||||
resolve({ code, reason: reason.toString() });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/** Wait for the first message satisfying `match`, ignoring earlier frames. */
|
||||
function waitForMessage(
|
||||
ws: WebSocket,
|
||||
match: (msg: Record<string, unknown>) => boolean,
|
||||
timeoutMs = 3000
|
||||
): Promise<Record<string, unknown>> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const timer = setTimeout(() => reject(new Error('WS message timeout')), timeoutMs);
|
||||
const onMessage = (raw: WebSocket.RawData) => {
|
||||
let msg: Record<string, unknown>;
|
||||
try {
|
||||
msg = JSON.parse(String(raw));
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!match(msg)) return;
|
||||
clearTimeout(timer);
|
||||
ws.off('message', onMessage);
|
||||
resolve(msg);
|
||||
};
|
||||
ws.on('message', onMessage);
|
||||
});
|
||||
}
|
||||
|
||||
function waitUntil(predicate: () => boolean, timeoutMs = 3000): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
const tick = () => {
|
||||
if (predicate()) return resolve();
|
||||
if (Date.now() > deadline) return reject(new Error('condition not met in time'));
|
||||
setTimeout(tick, 10);
|
||||
};
|
||||
tick();
|
||||
});
|
||||
}
|
||||
|
||||
describe('voice-routes', () => {
|
||||
let app: FastifyInstance;
|
||||
let ctx: MockRouteContext;
|
||||
let upstream: WebSocketServer;
|
||||
let capture: UpstreamCapture;
|
||||
let voiceEnabled: boolean;
|
||||
|
||||
beforeEach(async () => {
|
||||
_resetVoiceStreamCountForTesting();
|
||||
voiceEnabled = true;
|
||||
capture = { headers: {}, url: '', binaryFrames: [], textFrames: [], socket: null };
|
||||
|
||||
upstream = new WebSocketServer({ port: UPSTREAM_PORT, host: '127.0.0.1' });
|
||||
upstream.on('connection', (socket, req) => {
|
||||
capture.headers = req.headers;
|
||||
capture.url = req.url ?? '';
|
||||
capture.socket = socket;
|
||||
socket.on('message', (raw, isBinary) => {
|
||||
if (isBinary) capture.binaryFrames.push(Buffer.from(raw as Buffer));
|
||||
else capture.textFrames.push(String(raw));
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve) => upstream.once('listening', resolve));
|
||||
process.env.CODEMAN_VOICE_STREAM_BASE = `ws://127.0.0.1:${UPSTREAM_PORT}`;
|
||||
|
||||
writeCredentials(Date.now() + 3_600_000);
|
||||
|
||||
app = Fastify({ logger: false });
|
||||
await app.register(fastifyWebsocket);
|
||||
ctx = createMockRouteContext();
|
||||
ctx.getClaudeVoiceEnabled = (async () => voiceEnabled) as typeof ctx.getClaudeVoiceEnabled;
|
||||
registerVoiceRoutes(app, ctx as never, () => ({ bindHost: '127.0.0.1', allowedHosts: [], tunnelHost: null }));
|
||||
await app.listen({ port: PORT, host: '127.0.0.1' });
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
delete process.env.CODEMAN_VOICE_STREAM_BASE;
|
||||
removeCredentials();
|
||||
await app.close();
|
||||
await new Promise<void>((resolve) => upstream.close(() => resolve()));
|
||||
});
|
||||
|
||||
describe('GET /api/voice/status', () => {
|
||||
it('reports available with display metadata when enabled and signed in', async () => {
|
||||
const res = await app.inject({ method: 'GET', url: '/api/voice/status' });
|
||||
expect(res.statusCode).toBe(200);
|
||||
expect(res.json().data).toMatchObject({ available: true, subscriptionType: 'max' });
|
||||
});
|
||||
|
||||
it('never returns the access token', async () => {
|
||||
const res = await app.inject({ method: 'GET', url: '/api/voice/status' });
|
||||
expect(res.body).not.toContain(TOKEN);
|
||||
expect(res.body).not.toContain('sk-ant');
|
||||
});
|
||||
|
||||
it('reports disabled when the setting is off, without touching credentials', async () => {
|
||||
voiceEnabled = false;
|
||||
const res = await app.inject({ method: 'GET', url: '/api/voice/status' });
|
||||
expect(res.json().data).toEqual({ available: false, reason: 'disabled' });
|
||||
});
|
||||
|
||||
it('reports no-credentials when nothing is signed in', async () => {
|
||||
removeCredentials();
|
||||
const res = await app.inject({ method: 'GET', url: '/api/voice/status' });
|
||||
expect(res.json().data).toEqual({ available: false, reason: 'no-credentials' });
|
||||
});
|
||||
|
||||
it('reports expired separately, so the UI can say how to fix it', async () => {
|
||||
writeCredentials(Date.now() - 1000);
|
||||
const res = await app.inject({ method: 'GET', url: '/api/voice/status' });
|
||||
expect(res.json().data.available).toBe(false);
|
||||
expect(res.json().data.reason).toBe('expired');
|
||||
});
|
||||
});
|
||||
|
||||
describe('GET /ws/voice/stream', () => {
|
||||
it('relays audio up and transcripts down, finalizing on request', async () => {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream?language=en&keyterms=tmux,respawn`);
|
||||
const ready = waitForMessage(ws, (m) => m.t === 'ready');
|
||||
await new Promise((resolve) => ws.once('open', resolve));
|
||||
await ready;
|
||||
|
||||
ws.send(Buffer.alloc(3200));
|
||||
await waitUntil(() => capture.binaryFrames.length > 0);
|
||||
expect(capture.binaryFrames[0].length).toBe(3200);
|
||||
|
||||
const interim = waitForMessage(ws, (m) => m.t === 'transcript' && m.final === false);
|
||||
capture.socket!.send(JSON.stringify({ type: 'TranscriptText', data: 'run the type check' }));
|
||||
expect((await interim).text).toBe('run the type check');
|
||||
|
||||
ws.send(JSON.stringify({ t: 'finalize' }));
|
||||
await waitUntil(() => capture.textFrames.some((f) => f.includes('CloseStream')));
|
||||
|
||||
const final = waitForMessage(ws, (m) => m.t === 'transcript' && m.final === true);
|
||||
capture.socket!.send(JSON.stringify({ type: 'TranscriptEndpoint' }));
|
||||
expect((await final).text).toBe('run the type check');
|
||||
|
||||
const { code } = await waitForClose(ws);
|
||||
expect(code).toBe(1000);
|
||||
});
|
||||
|
||||
it('authenticates upstream with the bearer token and forwards keyterms', async () => {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream?keyterms=tmux,respawn`);
|
||||
await waitForMessage(ws, (m) => m.t === 'ready');
|
||||
expect(capture.headers.authorization).toBe(`Bearer ${TOKEN}`);
|
||||
expect(capture.headers['x-config-keyterms']).toBe('tmux,respawn');
|
||||
expect(capture.url).toContain('encoding=linear16');
|
||||
expect(capture.url).toContain('sample_rate=16000');
|
||||
ws.close();
|
||||
});
|
||||
|
||||
it('pings upstream immediately so the idle gap before first audio cannot drop it', async () => {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
await waitForMessage(ws, (m) => m.t === 'ready');
|
||||
await waitUntil(() => capture.textFrames.some((f) => f.includes('KeepAlive')));
|
||||
ws.close();
|
||||
});
|
||||
|
||||
it('surfaces an upstream transcription error to the browser', async () => {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
await waitForMessage(ws, (m) => m.t === 'ready');
|
||||
const err = waitForMessage(ws, (m) => m.t === 'error');
|
||||
capture.socket!.send(JSON.stringify({ type: 'TranscriptError', description: 'no audio' }));
|
||||
expect((await err).message).toBe('no audio');
|
||||
ws.close();
|
||||
});
|
||||
|
||||
it('drops an oversized audio frame instead of relaying it', async () => {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
await waitForMessage(ws, (m) => m.t === 'ready');
|
||||
|
||||
ws.send(Buffer.alloc(200_000));
|
||||
ws.send(Buffer.alloc(1600));
|
||||
await waitUntil(() => capture.binaryFrames.length > 0);
|
||||
// The legal frame arrived; the oversized one was never forwarded.
|
||||
expect(capture.binaryFrames.every((f) => f.length === 1600)).toBe(true);
|
||||
ws.close();
|
||||
});
|
||||
|
||||
it('closes 4004 with the reason when the setting is off', async () => {
|
||||
voiceEnabled = false;
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
const { code, reason } = await waitForClose(ws);
|
||||
expect(code).toBe(4004);
|
||||
expect(reason).toBe('disabled');
|
||||
});
|
||||
|
||||
it('closes 4004 when no Claude login exists on the server', async () => {
|
||||
removeCredentials();
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
const { code, reason } = await waitForClose(ws);
|
||||
expect(code).toBe(4004);
|
||||
expect(reason).toBe('no-credentials');
|
||||
});
|
||||
|
||||
it('closes 4004 rather than streaming on an expired login', async () => {
|
||||
writeCredentials(Date.now() - 1000);
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
const { code, reason } = await waitForClose(ws);
|
||||
expect(code).toBe(4004);
|
||||
expect(reason).toBe('expired');
|
||||
});
|
||||
|
||||
it('refuses a cross-site upgrade (a foreign page must not spend the subscription)', async () => {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`, {
|
||||
headers: { origin: 'https://evil.example' },
|
||||
});
|
||||
const { code } = await waitForClose(ws);
|
||||
expect(code).toBe(4003);
|
||||
expect(capture.socket).toBeNull();
|
||||
});
|
||||
|
||||
it('caps concurrent streams', async () => {
|
||||
const open: WebSocket[] = [];
|
||||
for (let i = 0; i < MAX_CONCURRENT_STREAMS; i++) {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
await waitForMessage(ws, (m) => m.t === 'ready');
|
||||
open.push(ws);
|
||||
}
|
||||
const extra = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
const { code, reason } = await waitForClose(extra);
|
||||
expect(code).toBe(4008);
|
||||
expect(reason).toBe('Too many voice streams');
|
||||
for (const ws of open) ws.close();
|
||||
});
|
||||
|
||||
it('frees a stream slot when the browser hangs up', async () => {
|
||||
const first = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
await waitForMessage(first, (m) => m.t === 'ready');
|
||||
first.close();
|
||||
await waitForClose(first);
|
||||
|
||||
// The slot is reusable: MAX_CONCURRENT more streams must still be admitted.
|
||||
const reopened: WebSocket[] = [];
|
||||
for (let i = 0; i < MAX_CONCURRENT_STREAMS; i++) {
|
||||
const ws = new WebSocket(`ws://127.0.0.1:${PORT}/ws/voice/stream`);
|
||||
await waitForMessage(ws, (m) => m.t === 'ready');
|
||||
reopened.push(ws);
|
||||
}
|
||||
for (const ws of reopened) ws.close();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -760,6 +760,11 @@ describe('case selector refresh', () => {
|
||||
];
|
||||
app.showToast = vi.fn();
|
||||
|
||||
// deleteCase re-renders the case-manage list, whose path label goes through
|
||||
// _shortenHomePath. That method lives in terminal-ui.js, which this harness
|
||||
// does not load (the real app always has it: load order 7 before 12).
|
||||
app._shortenHomePath = (p: string) => p;
|
||||
|
||||
await app.deleteCase('deleted-case');
|
||||
|
||||
expect(quickStartCase.blur).toHaveBeenCalled();
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
/**
|
||||
* Session Options structural guard.
|
||||
*
|
||||
* The modal shares the `set-*` settings surface with App Settings, but its rail
|
||||
* is a real switcher: switchOptionsTab shows one `.set-section` and hides the
|
||||
* rest. Like App Settings, its load/save path is `getElementById` by a fixed set
|
||||
* of ids, so dropping or renaming an element in the markup fails silently — the
|
||||
* option just stops loading, or stops being written back.
|
||||
*
|
||||
* These tests read the REAL session-ui.js and index.html and pin that contract.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { resolve } from 'node:path';
|
||||
|
||||
const publicDir = resolve(import.meta.dirname, '../src/web/public');
|
||||
const html = readFileSync(resolve(publicDir, 'index.html'), 'utf8');
|
||||
const sessionUi = readFileSync(resolve(publicDir, 'session-ui.js'), 'utf8');
|
||||
|
||||
/** The Session Options markup, so assertions can't be satisfied elsewhere. */
|
||||
function optionsModal(): string {
|
||||
const start = html.indexOf('<div class="modal" id="sessionOptionsModal">');
|
||||
expect(start).toBeGreaterThan(-1);
|
||||
const end = html.indexOf('<!-- Close Session Confirmation Modal -->', start);
|
||||
expect(end).toBeGreaterThan(start);
|
||||
return html.slice(start, end);
|
||||
}
|
||||
|
||||
/** Body of a session-ui.js method, by name. */
|
||||
function methodBody(signature: string): string {
|
||||
const start = sessionUi.indexOf(`\n ${signature} {`);
|
||||
expect(start, `${signature} not found in session-ui.js`).toBeGreaterThan(-1);
|
||||
return sessionUi.slice(start, sessionUi.indexOf('\n },', start));
|
||||
}
|
||||
|
||||
const TABS = ['respawn', 'context', 'ralph', 'summary'];
|
||||
|
||||
describe('Session Options modal structure', () => {
|
||||
it('keeps every element openSessionOptions and switchOptionsTab touch by id', () => {
|
||||
const modal = optionsModal();
|
||||
const ids = new Set<string>();
|
||||
for (const sig of ['openSessionOptions(sessionId)', 'switchOptionsTab(tabName)', 'getRalphConfig()']) {
|
||||
for (const m of methodBody(sig).matchAll(/getElementById\('([A-Za-z0-9_-]+)'\)/g)) ids.add(m[1]);
|
||||
}
|
||||
// openSessionOptions also drives elements outside this modal (tabs, toasts);
|
||||
// only the ones it expects to find in here are this file's contract.
|
||||
const outside = new Set(['sessionOptionsDoc']);
|
||||
const missing = [...ids].filter((id) => !outside.has(id) && !modal.includes(`id="${id}"`));
|
||||
expect(missing).toEqual([]);
|
||||
expect(modal).toContain('id="sessionOptionsDoc"');
|
||||
});
|
||||
|
||||
it('pairs each rail entry with exactly one section, in the same order', () => {
|
||||
const modal = optionsModal();
|
||||
const rail = [...modal.matchAll(/class="set-rail-item[^"]*" data-tab="([a-z]+)"/g)].map((m) => m[1]);
|
||||
expect(rail).toEqual(TABS);
|
||||
for (const tab of TABS) {
|
||||
const hits = modal.split(`id="${tab}-tab"`).length - 1;
|
||||
expect(hits, `section ${tab}-tab should exist exactly once`).toBe(1);
|
||||
}
|
||||
// switchOptionsTab queries the rail by THIS class; `.modal-tab-btn` here
|
||||
// would silently stop the active marker from moving.
|
||||
expect(methodBody('switchOptionsTab(tabName)')).toContain("'#sessionOptionsModal .set-rail-item'");
|
||||
expect(methodBody('openSessionOptions(sessionId)')).toContain('.set-rail-item[data-tab="ralph"]');
|
||||
});
|
||||
|
||||
it('opens with exactly one section visible, the rest hidden', () => {
|
||||
const modal = optionsModal();
|
||||
const visible = TABS.filter((t) => modal.includes(`<section class="set-section" id="${t}-tab"`));
|
||||
expect(visible).toEqual(['respawn']);
|
||||
for (const t of TABS.filter((t) => t !== 'respawn')) {
|
||||
expect(modal).toContain(`<section class="set-section hidden" id="${t}-tab"`);
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps the Claude-only rail entries marked, so external CLIs lose them', () => {
|
||||
const modal = optionsModal();
|
||||
for (const tab of ['respawn', 'ralph']) {
|
||||
const entry = modal.match(new RegExp(`<button[^>]*data-tab="${tab}"[^>]*>`))?.[0] ?? '';
|
||||
expect(entry, `${tab} rail entry`).toContain('data-claude-only');
|
||||
}
|
||||
expect(modal.match(/<button[^>]*data-tab="context"[^>]*>/)?.[0]).not.toContain('data-claude-only');
|
||||
});
|
||||
|
||||
it('uses the shared settings surface rather than the modal-tab chrome', () => {
|
||||
const modal = optionsModal();
|
||||
expect(modal).toContain('class="modal-content modal-lg set-shell"');
|
||||
expect(modal).toContain('class="set-body"');
|
||||
expect(modal).not.toContain('class="modal-tabs"');
|
||||
expect(modal).not.toContain('modal-tab-btn');
|
||||
expect(modal).not.toContain('modal-tab-content');
|
||||
// The `set-*` rules are shared by both modals through one :is() scope.
|
||||
const css = readFileSync(resolve(publicDir, 'styles.css'), 'utf8');
|
||||
expect(css).toContain(':is(#appSettingsModal, #sessionOptionsModal, #createCaseModal) .set-row {');
|
||||
expect(css).toContain(':is(#sessionOptionsModal, #createCaseModal) .set-section.hidden {');
|
||||
});
|
||||
});
|
||||
@@ -6,8 +6,17 @@ import { describe, expect, it, vi } from 'vitest';
|
||||
function loadTerminalUiHarness() {
|
||||
const CodemanApp = function CodemanApp(this: any) {};
|
||||
let now = 1_000;
|
||||
let keyboardVisible = false;
|
||||
let activeElement: unknown = null;
|
||||
const context = vm.createContext({
|
||||
window: {},
|
||||
document: {
|
||||
body: { classList: { contains: () => false } },
|
||||
get activeElement() {
|
||||
return activeElement;
|
||||
},
|
||||
getElementById: () => null,
|
||||
},
|
||||
CodemanApp,
|
||||
console: { warn: vi.fn(), log: vi.fn() },
|
||||
_crashDiag: { log: vi.fn() },
|
||||
@@ -25,6 +34,11 @@ function loadTerminalUiHarness() {
|
||||
MobileDetection: {
|
||||
isTouchDevice: () => true,
|
||||
},
|
||||
KeyboardHandler: {
|
||||
get keyboardVisible() {
|
||||
return keyboardVisible;
|
||||
},
|
||||
},
|
||||
DEC_SYNC_STRIP_RE: /\x1b\[\?2026[hl]/g,
|
||||
TERMINAL_CHUNK_SIZE: 32 * 1024,
|
||||
});
|
||||
@@ -38,6 +52,12 @@ function loadTerminalUiHarness() {
|
||||
setNow: (value: number) => {
|
||||
now = value;
|
||||
},
|
||||
setKeyboardVisible: (visible: boolean) => {
|
||||
keyboardVisible = visible;
|
||||
},
|
||||
setActiveElement: (element: unknown) => {
|
||||
activeElement = element;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -55,7 +75,167 @@ function createElementHarness() {
|
||||
};
|
||||
}
|
||||
|
||||
function createTerminalGrid(lines: string[], cursorY: number, wrappedRows = new Set<number>()) {
|
||||
const textarea = {
|
||||
classList: { contains: (name: string) => name === 'xterm-helper-textarea' },
|
||||
blur: vi.fn(),
|
||||
};
|
||||
return {
|
||||
cols: 80,
|
||||
rows: lines.length,
|
||||
modes: { mouseTrackingMode: 'none' },
|
||||
buffer: {
|
||||
active: {
|
||||
viewportY: 0,
|
||||
baseY: 0,
|
||||
cursorY,
|
||||
getLine: (row: number) =>
|
||||
row >= 0 && row < lines.length
|
||||
? { isWrapped: wrappedRows.has(row), translateToString: () => lines[row] }
|
||||
: undefined,
|
||||
},
|
||||
},
|
||||
element: {
|
||||
querySelector: (selector: string) =>
|
||||
selector === '.xterm-screen' ? { getBoundingClientRect: () => ({ left: 0, top: 0 }) } : null,
|
||||
},
|
||||
_core: { _renderService: { dimensions: { css: { cell: { width: 8, height: 16 } } } } },
|
||||
textarea,
|
||||
focus: vi.fn(),
|
||||
};
|
||||
}
|
||||
|
||||
describe('terminal touch tap mouse guard', () => {
|
||||
it('recognizes focus only when a terminal input owns the active element', () => {
|
||||
const { app, setActiveElement } = loadTerminalUiHarness();
|
||||
const textarea = { classList: { contains: () => true } };
|
||||
app.terminal = { textarea };
|
||||
|
||||
setActiveElement(null);
|
||||
expect(app._isMobileTerminalInputFocused()).toBe(false);
|
||||
|
||||
setActiveElement(textarea);
|
||||
expect(app._isMobileTerminalInputFocused()).toBe(true);
|
||||
});
|
||||
|
||||
it('routes a readback row to the TUI while keeping the prompt row as keyboard input', () => {
|
||||
const { app } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'codex' }]]);
|
||||
app.terminal = createTerminalGrid(
|
||||
['Agent readback mentions › inline', ' tap to collapse', '', '', '› ask', 'gpt-5 · Context 80% left'],
|
||||
4
|
||||
);
|
||||
|
||||
expect(app._classifyMobileTerminalTap(9, 1)).toBe('content'); // inline marker is not a prompt
|
||||
expect(app._classifyMobileTerminalTap(9, 17)).toBe('content'); // row 2: readback
|
||||
expect(app._classifyMobileTerminalTap(9, 65)).toBe('input'); // row 5: prompt
|
||||
expect(app._classifyMobileTerminalTap(9, 81)).toBe('content'); // row 6: status
|
||||
});
|
||||
|
||||
it('classifies Claude background-agent status as content rather than keyboard input', () => {
|
||||
const { app } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'claude', cliVersion: '2.1.220' }]]);
|
||||
app.terminal = createTerminalGrid(
|
||||
['', '', '', '• Working (1m 50s • esc to ', 'interrupt) · 1 background teammate', ''],
|
||||
4,
|
||||
new Set([4])
|
||||
);
|
||||
|
||||
expect(app._classifyMobileTerminalTap(9, 65)).toBe('content');
|
||||
});
|
||||
|
||||
it('keeps the live cursor focusable when Claude temporarily omits its prompt glyph', () => {
|
||||
const { app } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'claude' }]]);
|
||||
app.terminal = createTerminalGrid(['Prior response', '', 'ready for input', '', 'status footer', ''], 2);
|
||||
|
||||
expect(app._classifyMobileTerminalTap(9, 33)).toBe('input');
|
||||
expect(app._classifyMobileTerminalTap(9, 1)).toBe('content');
|
||||
});
|
||||
|
||||
it('treats a highlighted numbered choice as TUI content, not an input prompt', () => {
|
||||
const { app } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'claude' }]]);
|
||||
app.terminal = createTerminalGrid(['Would you like to proceed?', '', '❯ 1. Yes', ' 2. No', '', ''], 2);
|
||||
|
||||
expect(app._classifyMobileTerminalTap(9, 33)).toBe('content');
|
||||
expect(app._classifyMobileTerminalTap(9, 49)).toBe('content');
|
||||
});
|
||||
|
||||
it('collapses TUI readback content without opening or retaining the keyboard', () => {
|
||||
const { app, setActiveElement } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'codex' }]]);
|
||||
app.terminal = createTerminalGrid(
|
||||
['Agent readback', ' tap to collapse', '', '', '› ask', 'gpt-5 · Context 80% left'],
|
||||
4
|
||||
);
|
||||
app._sendInputAsync = vi.fn();
|
||||
setActiveElement(app.terminal.textarea);
|
||||
|
||||
expect(app._handleMobileTerminalTap({ clientX: 9, clientY: 17 }, true)).toBe('content');
|
||||
expect(app._sendInputAsync).toHaveBeenCalledWith('sess-1', '\x1b[<0;2;2M\x1b[<0;2;2m');
|
||||
expect(app.terminal.textarea.blur).toHaveBeenCalledOnce();
|
||||
expect(app.terminal.focus).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('keeps the first prompt tap focus-only so it cannot activate a CLI row', () => {
|
||||
const { app, setActiveElement } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'codex' }]]);
|
||||
app.terminal = createTerminalGrid(
|
||||
['Agent readback', ' tap to collapse', '', '', '› ask', 'gpt-5 · Context 80% left'],
|
||||
4
|
||||
);
|
||||
app._sendInputAsync = vi.fn();
|
||||
setActiveElement(null);
|
||||
|
||||
expect(app._handleMobileTerminalTap({ clientX: 9, clientY: 65 }, false)).toBe('input');
|
||||
expect(app._sendInputAsync).not.toHaveBeenCalled();
|
||||
expect(app.terminal.focus).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('closes the keyboard on a second tap of INERT transcript content', () => {
|
||||
const { app, setActiveElement } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'claude' }]]);
|
||||
app.terminal = createTerminalGrid(['transcript line', '', '', '', '❯ ', ''], 4);
|
||||
app._sendInputAsync = vi.fn();
|
||||
|
||||
// Keyboard DOWN: the tap opens it.
|
||||
setActiveElement(null);
|
||||
expect(app._handleMobileTerminalTap({ clientX: 9, clientY: 1 }, false)).toBe('content');
|
||||
expect(app.terminal.focus).toHaveBeenCalledOnce();
|
||||
expect(app.terminal.textarea.blur).not.toHaveBeenCalled();
|
||||
|
||||
// Keyboard UP on the same inert row: the tap closes it.
|
||||
app.terminal.focus.mockClear();
|
||||
setActiveElement(app.terminal.textarea);
|
||||
expect(app._handleMobileTerminalTap({ clientX: 9, clientY: 1 }, true)).toBe('content');
|
||||
expect(app.terminal.textarea.blur).toHaveBeenCalledOnce();
|
||||
expect(app.terminal.focus).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('keeps the prompt row focusing rather than toggling, so the caret can still be placed', () => {
|
||||
// The toggle is scoped to 'content' on purpose: a second tap on the PROMPT
|
||||
// must still position the cursor. This is the guarantee that makes the
|
||||
// change safe to make, so it is pinned separately.
|
||||
const { app, setActiveElement } = loadTerminalUiHarness();
|
||||
app.activeSessionId = 'sess-1';
|
||||
app.sessions = new Map([['sess-1', { mode: 'claude' }]]);
|
||||
app.terminal = createTerminalGrid(['transcript line', '', '', '', '❯ ask', ''], 4);
|
||||
app._sendInputAsync = vi.fn();
|
||||
|
||||
setActiveElement(app.terminal.textarea);
|
||||
expect(app._handleMobileTerminalTap({ clientX: 9, clientY: 65 }, true)).toBe('input');
|
||||
expect(app.terminal.textarea.blur).not.toHaveBeenCalled();
|
||||
expect(app.terminal.focus).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('suppresses browser trusted compatibility mouse events during the tap window', () => {
|
||||
const { app } = loadTerminalUiHarness();
|
||||
const { element, dispatch } = createElementHarness();
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
/**
|
||||
* Voice stream request building.
|
||||
*
|
||||
* The audio format lives in the query string, so a drift between these params
|
||||
* and the browser worklet does not fail loudly — it transcribes as noise. These
|
||||
* tests pin the contract, and pin that the bearer token never leaks into a URL
|
||||
* (which would land it in proxy logs).
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import {
|
||||
buildVoiceStreamHeaders,
|
||||
buildVoiceStreamUrl,
|
||||
normalizeVoiceLanguage,
|
||||
sanitizeKeyterms,
|
||||
} from '../src/web/voice-stream.js';
|
||||
import { MAX_KEYTERMS_HEADER_CHARS } from '../src/config/voice.js';
|
||||
|
||||
describe('buildVoiceStreamUrl', () => {
|
||||
it('pins linear16 / 16 kHz / mono, matching the browser worklet', () => {
|
||||
const url = new URL(buildVoiceStreamUrl({}, {}));
|
||||
expect(url.protocol).toBe('wss:');
|
||||
expect(url.host).toBe('api.anthropic.com');
|
||||
expect(url.pathname).toBe('/api/ws/speech_to_text/voice_stream');
|
||||
expect(url.searchParams.get('encoding')).toBe('linear16');
|
||||
expect(url.searchParams.get('sample_rate')).toBe('16000');
|
||||
expect(url.searchParams.get('channels')).toBe('1');
|
||||
expect(url.searchParams.get('stt_provider')).toBe('deepgram-nova3');
|
||||
});
|
||||
|
||||
it('carries the language hint', () => {
|
||||
expect(new URL(buildVoiceStreamUrl({ language: 'de' }, {})).searchParams.get('language')).toBe('de');
|
||||
});
|
||||
|
||||
it('accepts a ws:// or wss:// base override for tests and gateways', () => {
|
||||
const url = buildVoiceStreamUrl({}, { CODEMAN_VOICE_STREAM_BASE: 'ws://127.0.0.1:3199/' });
|
||||
expect(url.startsWith('ws://127.0.0.1:3199/api/ws/speech_to_text/voice_stream?')).toBe(true);
|
||||
});
|
||||
|
||||
it('ignores a non-websocket override rather than building a broken URL', () => {
|
||||
expect(buildVoiceStreamUrl({}, { CODEMAN_VOICE_STREAM_BASE: 'https://evil.example' })).toContain(
|
||||
'wss://api.anthropic.com'
|
||||
);
|
||||
});
|
||||
|
||||
it('never puts credentials in the URL', () => {
|
||||
expect(buildVoiceStreamUrl({ language: 'en' }, {})).not.toMatch(/token|Bearer|sk-ant/i);
|
||||
});
|
||||
});
|
||||
|
||||
describe('normalizeVoiceLanguage', () => {
|
||||
it.each([
|
||||
['en', 'en'],
|
||||
['en-US', 'en-US'],
|
||||
['multi', 'multi'],
|
||||
['', 'en'],
|
||||
[undefined, 'en'],
|
||||
['not a language', 'en'],
|
||||
['../../etc/passwd', 'en'],
|
||||
])('normalizes %s to %s', (input, expected) => {
|
||||
expect(normalizeVoiceLanguage(input as string | undefined)).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
describe('sanitizeKeyterms', () => {
|
||||
it('joins terms with commas', () => {
|
||||
expect(sanitizeKeyterms(['tmux', 'respawn'])).toBe('tmux,respawn');
|
||||
});
|
||||
|
||||
it('replaces an inner comma with a space so one term cannot become two', () => {
|
||||
expect(sanitizeKeyterms(['hello, world'])).toBe('hello world');
|
||||
});
|
||||
|
||||
it('drops non-ASCII, which is not portable in a header value', () => {
|
||||
expect(sanitizeKeyterms(['café', 'naïve'])).toBe('caf,nave');
|
||||
});
|
||||
|
||||
it('strips CR/LF so a term cannot inject a header', () => {
|
||||
const result = sanitizeKeyterms(['ok\r\nX-Evil: 1']);
|
||||
expect(result).not.toContain('\r');
|
||||
expect(result).not.toContain('\n');
|
||||
expect(result).toBe('okX-Evil: 1');
|
||||
});
|
||||
|
||||
it('dedupes and skips empties', () => {
|
||||
expect(sanitizeKeyterms(['a', 'a', '', ' ', 'b'])).toBe('a,b');
|
||||
});
|
||||
|
||||
it('truncates on a term boundary rather than mangling the last term', () => {
|
||||
const terms = Array.from({ length: 500 }, (_, i) => `term${i}`);
|
||||
const result = sanitizeKeyterms(terms);
|
||||
expect(result.length).toBeLessThanOrEqual(MAX_KEYTERMS_HEADER_CHARS);
|
||||
for (const term of result.split(',')) expect(term).toMatch(/^term\d+$/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildVoiceStreamHeaders', () => {
|
||||
it('sends the bearer token and identifies Codeman honestly', () => {
|
||||
const headers = buildVoiceStreamHeaders('sk-ant-oat01-test', '1.2.3');
|
||||
expect(headers.Authorization).toBe('Bearer sk-ant-oat01-test');
|
||||
expect(headers['User-Agent']).toBe('codeman/1.2.3 (voice-bridge)');
|
||||
expect(headers['x-app']).toBe('codeman');
|
||||
});
|
||||
|
||||
it('omits the keyterms header when nothing survives sanitizing', () => {
|
||||
expect(buildVoiceStreamHeaders('t', '1.0.0', ['', ' '])).not.toHaveProperty('x-config-keyterms');
|
||||
});
|
||||
|
||||
it('includes sanitized keyterms when present', () => {
|
||||
expect(buildVoiceStreamHeaders('t', '1.0.0', ['tmux', 'respawn'])['x-config-keyterms']).toBe('tmux,respawn');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user