The mic button previously needed a Deepgram API key, or fell back to the
browser's Web Speech engine. It can now transcribe through the same
speech-to-text service Claude Code's own /voice mode uses, so anyone signed
in to Claude Code on the server gets dictation with no third-party account.
Claude Code's voice mode cannot be driven directly: it opens the HOST's
microphone (sox/arecord), and the CLI runs in a headless tmux pane while the
human is in a browser somewhere else. So capture stays in the browser and only
the transcription backend is borrowed.
Audio goes browser -> Codeman -> Anthropic. The OAuth token never reaches the
page: the browser sends PCM16 (16 kHz mono, produced by an AudioWorklet since
MediaRecorder cannot emit raw PCM) and receives text.
- GET /api/voice/status reports readiness and never the token
- GET /ws/voice/stream relays one dictation, with the same Host/Origin upgrade
guard as the terminal socket, plus caps on concurrency, stream length and
frame size
- credentials are read-only: Codeman never refreshes them, since a refresh
rotates the refresh token and could sign the user out of their own CLI
- claudeVoiceEnabled (synced, default OFF) gates the whole server side
- voiceSettings.provider picks auto/claude/deepgram/webspeech; auto prefers
Claude, then a configured Deepgram key, then the browser
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The feature as pitched in docs/readmymind-plan.md: pressing the header
brain button predicts the prompt you were about to type, from the case's
intent profile plus everything the session already knows.
Backend:
- readmymind-context.ts: pure budgeted context assembler (9 ranked
sources: pending approval dialog, user goals, last assistant turn tail,
recent prompts, tool activity, git workspace signals, away context,
sibling sessions, rethink state; 30 KB budget, whole-section drop from
the bottom of the ranking, trust tiers stated in the prompt)
- readmymind-collectors.ts: transcript tail reader (the live watcher
keeps only a 500-char snippet) and git signal collection (execFile,
2s timeout, skipped for remote-SSH cases)
- readmymind-predictor.ts: one-shot claude -p in a throwaway tmux
session, opus by default (readMyMindModel setting), strict JSON
contract with 1-3 suggestions (continue / verify / redirect), newline
stripping, 90s timeout; mutable singleton so route tests can stub it
- POST /api/sessions/:id/readmymind: claude-mode only (400), one
prediction in flight per session (409 CONFLICT), rethink body
{ steer, rejected }; ownership via findSessionOrFail
Frontend:
- readmymind-ui.js (loadorder 11.3): header brain button, marker-hidden
until readMyMindEnabled is ON, desktop only (phone key is phase 3);
modal with editable suggestion + rationale and Send / Insert /
Rethink / Dismiss; suggestion text rendered via value/textContent only
and nothing ever auto-sends
- App Settings -> Panels checkbox for readMyMindEnabled; en + zh-CN
strings
Verified end to end against a live isolated instance: transcript
capture, a real opus prediction grounded in the stated goals, rethink
steering, the 409, and the browser modal incl. Insert leaving the text
unsubmitted on the composer. 41 new unit/route tests; full test:ci
sweep green (4680 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docs/readmymind.md covers phase 1 as a user guide: how to enable the synced
readMyMindEnabled setting via the API (no UI checkbox until phase 2), exactly
what is and is not captured, the hooks dependency (Docker bridge / remote-SSH
caveats), storage and wipe paths, curl examples for the three endpoints, the
agent-skill ground rules, and a troubleshooting table. Cross-linked from the
CLAUDE.md Key Patterns entry and the api-reference section.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-case profiles of user intent (docs/readmymind-plan.md): user-stated goals
plus the user's recently submitted prompts, captured from the Claude session
transcript behind the new synced readMyMindEnabled setting (default OFF).
- intent-store.ts: keyed by owner + realpath(workingDir), FIFO/size caps,
consecutive-dupe collapse, atomic 0600 writes to ~/.codeman/intents.json
- transcript-watcher.ts: new transcript:user_prompt event for typed user turns
(tool_result-only entries stay silent); capture wiring in server.ts is
claude-only and gated on the setting per event
- readmymind-routes.ts: GET/PUT/DELETE /api/sessions/:id/intent, ownership
via findSessionOrFail, strict Zod schema
- agent skill: SKILL.md recipe + endpoints.md rows so agents can read and
record intent (PUT replaces: read + merge; never delete unprompted)
- groundwork for the phase-2 predictor button; nothing is ever auto-sent
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Permission dialogs, AskUserQuestion questions and idle prompts from every
session now land in a server-side inbox (web/approval-inbox.ts, one item per
session, claude-mode only) and are answerable in place: a header bell + drawer
on desktop, inline answer strips on the phone overview's NEEDS YOU rows, and
working push Approve/Deny buttons (previously dead ends, now answered straight
from sw.js with no tab open). Pending alerts survive reloads because the
frontend seeds from GET /api/approvals on init.
Answering sends the digit / Esc / prompt text through the existing tmux input
path; option digits are accepted only when they match options parsed from the
captured pane frame, and the answer path re-captures the pane first so a
dialog that already left the screen refuses with 409 instead of typing into
the composer. New elicitation_complete / elicitation_response hook matchers
resolve question items the moment they are answered in the terminal;
refreshStaleCodemanHooks heals existing cases.
Verified end-to-end against a live claude session: a real AskUserQuestion
dialog parsed into 5 option buttons and was answered from the drawer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Point 1 of the v1.0 lock-in: commit to a stable HTTP API (the cleanest, fullest form).
Core (centralized):
- Every JSON /api response now uses ONE envelope via a Fastify preSerialization hook (src/web/server.ts): success -> { success:true, data:<payload> }; error -> { success:false, error, errorCode } with a conventional HTTP status. Non-JSON routes (file-raw, tail-file SSE, download, screenshots, /q redirect, WS) are skipped.
- Error-code -> HTTP status is a single source of truth (httpStatusForErrorCode in src/types/api.ts): 400/401/404/409/422/429/500. Expanded ApiErrorCode (added UNAUTHORIZED, CONFLICT, RATE_LIMITED). Errors are no longer HTTP 200.
- Versioned alias: /api/v1/* rewrites to /api/* (rewriteApiV1Url), so external clients pin to a stable surface while the bundled UI keeps using /api/*.
- Handlers stripped of manual 'success:true' (50 across 14 route files) so they return bare payloads the hook wraps uniformly; fixed the mux DELETE {success:<bool>} envelope collision (-> {killed}).
Frontend (48 call sites across 10 files):
- _apiJson() auto-unwraps { success:true, data } -> data (null on error), so most bare-shape readers are transparent. Raw-fetch sites relocate payload reads under .data; success/res.ok/error checks unchanged.
Docs: new docs/api-reference.md (envelope, status table, error codes, /api/v1, SSE); versioning-policy.md flipped — the HTTP/SSE API is now part of the stable, SemVer-covered surface.
Verification: full unit/route suite green (2680 passed) incl. ~166 updated assertions across 24 test files; typecheck/lint/format/frontend-syntax clean; a headless-chromium smoke loaded the migrated UI and drove the panels with 0 console/page errors; /api/status and /api/v1/status confirmed returning the uniform envelope live.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codeman maintainer