From a5dd4fff21e0dbc511492d71965e2c9a506fd435 Mon Sep 17 00:00:00 2001 From: arkon Date: Wed, 25 Feb 2026 00:14:52 +0100 Subject: [PATCH] chore: bump version to 0.1632 --- CLAUDE.md | 2 +- docs/voice-input-plan.md | 442 ++++++++++++++++++ package.json | 2 +- screenshots-echo-diag/01-initial.png | Bin 0 -> 63471 bytes .../02-prompt-with-overlay.png | Bin 0 -> 7511 bytes .../03-full-with-overlay.png | Bin 0 -> 65337 bytes .../04-prompt-canvas-only.png | Bin 0 -> 9592 bytes .../05-vertical-comparison.png | Bin 0 -> 72625 bytes .../06-baseline-comparison.png | Bin 0 -> 62102 bytes screenshots-echo-diag/07-full-comparison.png | Bin 0 -> 97622 bytes .../08-zoomed-comparison.png | Bin 0 -> 18377 bytes src/web/public/app.js | 2 + 12 files changed, 446 insertions(+), 2 deletions(-) create mode 100644 docs/voice-input-plan.md create mode 100644 screenshots-echo-diag/01-initial.png create mode 100644 screenshots-echo-diag/02-prompt-with-overlay.png create mode 100644 screenshots-echo-diag/03-full-with-overlay.png create mode 100644 screenshots-echo-diag/04-prompt-canvas-only.png create mode 100644 screenshots-echo-diag/05-vertical-comparison.png create mode 100644 screenshots-echo-diag/06-baseline-comparison.png create mode 100644 screenshots-echo-diag/07-full-comparison.png create mode 100644 screenshots-echo-diag/08-zoomed-comparison.png diff --git a/CLAUDE.md b/CLAUDE.md index b1d33ae4..4ca68732 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -35,7 +35,7 @@ When user says "COM": 1. Increment version in BOTH `package.json` AND `CLAUDE.md` (verify they match with `grep version package.json && grep Version CLAUDE.md`) 2. Run: `git add -A && git commit -m "chore: bump version to X.XXXX" && git push && npm run build && systemctl --user restart claudeman-web` -**Version**: 0.1631 (must match `package.json`) +**Version**: 0.1632 (must match `package.json`) ## Project Overview diff --git a/docs/voice-input-plan.md b/docs/voice-input-plan.md new file mode 100644 index 00000000..ccbfc444 --- /dev/null +++ b/docs/voice-input-plan.md @@ -0,0 +1,442 @@ +# Voice Input Implementation Plan + +## Executive Summary + +Add a microphone button to Claudeman (desktop + mobile) that uses the **Web Speech API** for real-time speech-to-text. Users tap the mic, speak their prompt, see live interim transcription, review the text, and press Enter to send. Zero server cost, sub-200ms perceived latency, ~90% browser coverage (Chrome + Safari). + +--- + +## Architecture Decision: Web Speech API (Primary, No Fallback for MVP) + +### Why Web Speech API +- **Free** — no API keys, no server-side processing, no cost per minute +- **Fast** — interim results in ~150ms, feels real-time +- **Simple** — ~80 lines of JS, no server changes needed for MVP +- **Good coverage** — Chrome (desktop + Android) + Safari (desktop + iOS) = ~90% of users + +### Why NOT a server-side fallback (for MVP) +- Firefox has <5% desktop share, even less on mobile +- Edge's Web Speech API is broken despite being Chromium (events don't fire) +- Adding Whisper/Deepgram requires API keys, server endpoints, audio streaming — significant complexity +- **Decision**: Hide the mic button on unsupported browsers. Add server fallback in Phase 2 if demand exists. + +### Browser Support Matrix + +| Browser | Support | Notes | +|---------|---------|-------| +| Chrome Desktop | YES | `webkitSpeechRecognition`, sends to Google servers | +| Chrome Android | YES | Same as desktop | +| Safari Desktop | YES | `webkitSpeechRecognition`, requires Siri enabled | +| Safari iOS | YES | Shows system permission modal | +| Firefox | NO | API exists behind flag but disabled by default | +| Edge | NO | Events don't fire despite API present | + +### Microphone Permissions +- HTTPS required, but **localhost is exempt** (Claudeman default) +- Claudeman with `--https` also works +- Chrome: persistent permission per domain (ask once) +- Safari: re-asks per page reload (less persistent, no workaround) +- Can pre-request via `getUserMedia()` on first interaction to avoid delay later + +--- + +## UX Design + +### Interaction Model: Toggle Mode +- **Tap mic** → start recording (button turns red, pulses) +- **Tap again** → stop recording (text inserted into terminal) +- Auto-stop after **5 seconds of silence** as safety net +- Single-utterance mode (`continuous: false`) — perfect for prompt dictation + +Why toggle over push-to-talk: Terminal prompts are short, toggle is simpler, works one-handed on mobile, impossible to accidentally leave recording on with the auto-stop safety net. + +### Visual Feedback + +**While recording:** +1. Mic button turns red with CSS pulsing animation (1.5s breathing cycle) +2. Interim text appears in a small overlay near the terminal bottom, styled in dim/italic to indicate "draft" +3. Text updates in real-time as user speaks (~150ms updates) + +**On completion:** +1. Final text inserted at terminal prompt via `sendInput()` +2. User reviews text, presses Enter when ready +3. Button returns to idle state (gray/outlined) + +### Text Flow +``` +User speaks → SpeechRecognition interim result → Show in preview overlay + → Replace preview on each update + → SpeechRecognition final result → Insert text via sendInput() + → Clear preview overlay + → User presses Enter to submit +``` + +**Critical: NO auto-submit.** User must press Enter. This is a terminal where wrong commands could be destructive. + +### Error States + +| Error | Behavior | +|-------|----------| +| Browser unsupported | Hide mic button entirely (feature detection) | +| No mic permission | Show toast: "Microphone access needed" with retry link | +| Permission denied | Show toast: "Mic blocked. Check browser settings" | +| No speech detected (5s) | Auto-stop, show brief "No speech detected" toast | +| Network error | Show toast: "Voice input requires internet" | +| Accidental tap | Tap again immediately to cancel; ignore <0.5s recordings | + +--- + +## Implementation Details + +### Phase 1: MVP (Single PR) + +#### Files to Modify + +| File | Changes | +|------|---------| +| `src/web/public/app.js` | `VoiceInput` class, keyboard shortcut, integration with `KeyboardAccessoryBar` | +| `src/web/public/index.html` | Desktop mic button in `toolbar-right` | +| `src/web/public/styles.css` | Desktop voice button styles, recording animation | +| `src/web/public/mobile.css` | Mobile voice button in accessory bar, recording animation | + +**No server-side changes needed.** Voice runs entirely in the browser. + +#### 1. VoiceInput Class (`app.js`) + +New singleton class (~80 lines) managing the Web Speech API lifecycle: + +```javascript +class VoiceInput { + constructor() { + this.recognition = null; + this.isRecording = false; + this.supported = !!(window.SpeechRecognition || window.webkitSpeechRecognition); + this.silenceTimeout = null; + this.previewEl = null; + + if (this.supported) { + const SR = window.SpeechRecognition || window.webkitSpeechRecognition; + this.recognition = new SR(); + this.recognition.continuous = false; + this.recognition.interimResults = true; + this.recognition.lang = 'en-US'; + this.recognition.maxAlternatives = 1; + + this.recognition.onresult = (e) => this._onResult(e); + this.recognition.onerror = (e) => this._onError(e); + this.recognition.onend = () => this._onEnd(); + } + } + + toggle() { this.isRecording ? this.stop() : this.start(); } + + start() { + if (!this.supported || this.isRecording) return; + this.isRecording = true; + this._updateButtons('recording'); + this._showPreview(''); + this.recognition.start(); + // Auto-stop after 5s silence + this._resetSilenceTimeout(); + } + + stop() { + if (!this.isRecording) return; + this.isRecording = false; + clearTimeout(this.silenceTimeout); + this._updateButtons('idle'); + this.recognition.stop(); + } + + _onResult(event) { + this._resetSilenceTimeout(); + let interim = '', final = ''; + for (let i = event.resultIndex; i < event.results.length; i++) { + const transcript = event.results[i][0].transcript; + if (event.results[i].isFinal) { + final += transcript; + } else { + interim += transcript; + } + } + + if (final) { + this._hidePreview(); + this._insertText(final); + this.stop(); + } else { + this._showPreview(interim); + // iOS workaround: isFinal is always false + this._iosStabilityCheck(interim); + } + } + + _onError(event) { + this.stop(); + if (event.error === 'not-allowed') { + app.showToast('Microphone access denied. Check browser settings.', 'error'); + } else if (event.error === 'no-speech') { + // Silent fail — auto-stop is enough + } else if (event.error === 'network') { + app.showToast('Voice input requires internet connection.', 'error'); + } + } + + _onEnd() { + if (this.isRecording) this.stop(); // Cleanup if ended unexpectedly + } + + _insertText(text) { + if (!app.activeSessionId || !text.trim()) return; + app.sendInput(text.trim()); + // Don't add \r — let user review and press Enter + } + + _resetSilenceTimeout() { + clearTimeout(this.silenceTimeout); + this.silenceTimeout = setTimeout(() => this.stop(), 5000); + } + + // iOS Safari: isFinal is always false. Detect stability. + _lastTranscript = ''; + _stabilityTimer = null; + _iosStabilityCheck(transcript) { + if (transcript !== this._lastTranscript) { + this._lastTranscript = transcript; + clearTimeout(this._stabilityTimer); + this._stabilityTimer = setTimeout(() => { + this._hidePreview(); + this._insertText(transcript); + this.stop(); + }, 750); + } + } + + _showPreview(text) { /* Update DOM overlay with interim text */ } + _hidePreview() { /* Remove DOM overlay */ } + _updateButtons(state) { /* Toggle .recording class on mic buttons */ } +} +``` + +#### 2. Mobile Button (KeyboardAccessoryBar) + +**Insert in HTML template** (after `/compact` button, before `paste`): +```html + +``` + +**Add to handleAction switch:** +```javascript +case 'voice': + if (typeof voiceInput !== 'undefined') voiceInput.toggle(); + break; +``` + +**Conditionally show:** Only render the button if `VoiceInput.supported` is true. + +#### 3. Desktop Button (index.html) + +**Insert in `toolbar-right`:** +```html +
+ +
+``` + +Show via JS on init: `if (voiceInput.supported) voiceInputBtn.style.display = '';` + +#### 4. Keyboard Shortcut + +**Ctrl+Shift+V** (in `setupEventListeners` keydown handler): +```javascript +if ((e.ctrlKey || e.metaKey) && e.shiftKey && e.key === 'V') { + e.preventDefault(); + if (typeof voiceInput !== 'undefined') voiceInput.toggle(); +} +``` + +Why Ctrl+Shift+V: Ctrl+V is paste (sacred), but Ctrl+Shift+V ("paste without formatting") has no meaning in a terminal context. V for Voice is memorable. + +#### 5. CSS Styles + +**styles.css (desktop):** +```css +.btn-voice.recording { + background: rgba(239, 68, 68, 0.15); + border-color: rgba(239, 68, 68, 0.4); + color: #ef4444; + animation: voice-pulse 1.5s ease-in-out infinite; +} + +@keyframes voice-pulse { + 0%, 100% { transform: scale(1); opacity: 1; } + 50% { transform: scale(1.08); opacity: 0.8; } +} + +.voice-preview { + position: fixed; + bottom: 48px; + left: 50%; + transform: translateX(-50%); + background: rgba(0, 0, 0, 0.85); + color: rgba(255, 255, 255, 0.6); + font-style: italic; + padding: 6px 16px; + border-radius: 8px; + font-size: 0.85rem; + max-width: 80%; + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; + z-index: 100; + pointer-events: none; +} +``` + +**mobile.css:** +```css +.accessory-btn-voice.recording { + background: rgba(239, 68, 68, 0.2); + border-color: rgba(239, 68, 68, 0.4); + color: #ef4444; + animation: voice-pulse 1.5s ease-in-out infinite; +} +``` + +#### 6. Interim Text Preview Overlay + +A small floating `
` showing live transcription: +```javascript +_showPreview(text) { + if (!this.previewEl) { + this.previewEl = document.createElement('div'); + this.previewEl.className = 'voice-preview'; + document.body.appendChild(this.previewEl); + } + this.previewEl.textContent = text || 'Listening...'; + this.previewEl.style.display = ''; +} + +_hidePreview() { + if (this.previewEl) { + this.previewEl.style.display = 'none'; + } +} +``` + +#### 7. Cleanup (Memory Leak Prevention) + +Following Claudeman's cleanup patterns: +- Store `VoiceInput` instance as `window.voiceInput` singleton +- On SSE reconnect (`handleInit()`): stop any active recording, reset state +- Remove preview element from DOM in cleanup +- Clear all timeouts (silenceTimeout, stabilityTimer) + +#### 8. Accessibility + +```html +