chore: bump version to 0.1632

2026-09-30 20:49:41 +02:00 · 2026-02-25 00:14:52 +01:00
parent e4c8c7a645
commit a5dd4fff21
12 changed files with 446 additions and 2 deletions
@@ -0,0 +1,442 @@
+# Voice Input Implementation Plan
+
+## Executive Summary
+
+Add a microphone button to Claudeman (desktop + mobile) that uses the **Web Speech API** for real-time speech-to-text. Users tap the mic, speak their prompt, see live interim transcription, review the text, and press Enter to send. Zero server cost, sub-200ms perceived latency, ~90% browser coverage (Chrome + Safari).
+
+---
+
+## Architecture Decision: Web Speech API (Primary, No Fallback for MVP)
+
+### Why Web Speech API
+- **Free** — no API keys, no server-side processing, no cost per minute
+- **Fast** — interim results in ~150ms, feels real-time
+- **Simple** — ~80 lines of JS, no server changes needed for MVP
+- **Good coverage** — Chrome (desktop + Android) + Safari (desktop + iOS) = ~90% of users
+
+### Why NOT a server-side fallback (for MVP)
+- Firefox has <5% desktop share, even less on mobile
+- Edge's Web Speech API is broken despite being Chromium (events don't fire)
+- Adding Whisper/Deepgram requires API keys, server endpoints, audio streaming — significant complexity
+- **Decision**: Hide the mic button on unsupported browsers. Add server fallback in Phase 2 if demand exists.
+
+### Browser Support Matrix
+
+| Browser | Support | Notes |
+|---------|---------|-------|
+| Chrome Desktop | YES | `webkitSpeechRecognition`, sends to Google servers |
+| Chrome Android | YES | Same as desktop |
+| Safari Desktop | YES | `webkitSpeechRecognition`, requires Siri enabled |
+| Safari iOS | YES | Shows system permission modal |
+| Firefox | NO | API exists behind flag but disabled by default |
+| Edge | NO | Events don't fire despite API present |
+
+### Microphone Permissions
+- HTTPS required, but **localhost is exempt** (Claudeman default)
+- Claudeman with `--https` also works
+- Chrome: persistent permission per domain (ask once)
+- Safari: re-asks per page reload (less persistent, no workaround)
+- Can pre-request via `getUserMedia()` on first interaction to avoid delay later
+
+---
+
+## UX Design
+
+### Interaction Model: Toggle Mode
+- **Tap mic** → start recording (button turns red, pulses)
+- **Tap again** → stop recording (text inserted into terminal)
+- Auto-stop after **5 seconds of silence** as safety net
+- Single-utterance mode (`continuous: false`) — perfect for prompt dictation
+
+Why toggle over push-to-talk: Terminal prompts are short, toggle is simpler, works one-handed on mobile, impossible to accidentally leave recording on with the auto-stop safety net.
+
+### Visual Feedback
+
+**While recording:**
+1. Mic button turns red with CSS pulsing animation (1.5s breathing cycle)
+2. Interim text appears in a small overlay near the terminal bottom, styled in dim/italic to indicate "draft"
+3. Text updates in real-time as user speaks (~150ms updates)
+
+**On completion:**
+1. Final text inserted at terminal prompt via `sendInput()`
+2. User reviews text, presses Enter when ready
+3. Button returns to idle state (gray/outlined)
+
+### Text Flow
+```
+User speaks → SpeechRecognition interim result → Show in preview overlay
+                                               → Replace preview on each update
+           → SpeechRecognition final result    → Insert text via sendInput()
+                                               → Clear preview overlay
+                                               → User presses Enter to submit
+```
+
+**Critical: NO auto-submit.** User must press Enter. This is a terminal where wrong commands could be destructive.
+
+### Error States
+
+| Error | Behavior |
+|-------|----------|
+| Browser unsupported | Hide mic button entirely (feature detection) |
+| No mic permission | Show toast: "Microphone access needed" with retry link |
+| Permission denied | Show toast: "Mic blocked. Check browser settings" |
+| No speech detected (5s) | Auto-stop, show brief "No speech detected" toast |
+| Network error | Show toast: "Voice input requires internet" |
+| Accidental tap | Tap again immediately to cancel; ignore <0.5s recordings |
+
+---
+
+## Implementation Details
+
+### Phase 1: MVP (Single PR)
+
+#### Files to Modify
+
+| File | Changes |
+|------|---------|
+| `src/web/public/app.js` | `VoiceInput` class, keyboard shortcut, integration with `KeyboardAccessoryBar` |
+| `src/web/public/index.html` | Desktop mic button in `toolbar-right` |
+| `src/web/public/styles.css` | Desktop voice button styles, recording animation |
+| `src/web/public/mobile.css` | Mobile voice button in accessory bar, recording animation |
+
+**No server-side changes needed.** Voice runs entirely in the browser.
+
+#### 1. VoiceInput Class (`app.js`)
+
+New singleton class (~80 lines) managing the Web Speech API lifecycle:
+
+```javascript
+class VoiceInput {
+  constructor() {
+    this.recognition = null;
+    this.isRecording = false;
+    this.supported = !!(window.SpeechRecognition || window.webkitSpeechRecognition);
+    this.silenceTimeout = null;
+    this.previewEl = null;
+
+    if (this.supported) {
+      const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
+      this.recognition = new SR();
+      this.recognition.continuous = false;
+      this.recognition.interimResults = true;
+      this.recognition.lang = 'en-US';
+      this.recognition.maxAlternatives = 1;
+
+      this.recognition.onresult = (e) => this._onResult(e);
+      this.recognition.onerror = (e) => this._onError(e);
+      this.recognition.onend = () => this._onEnd();
+    }
+  }
+
+  toggle() { this.isRecording ? this.stop() : this.start(); }
+
+  start() {
+    if (!this.supported || this.isRecording) return;
+    this.isRecording = true;
+    this._updateButtons('recording');
+    this._showPreview('');
+    this.recognition.start();
+    // Auto-stop after 5s silence
+    this._resetSilenceTimeout();
+  }
+
+  stop() {
+    if (!this.isRecording) return;
+    this.isRecording = false;
+    clearTimeout(this.silenceTimeout);
+    this._updateButtons('idle');
+    this.recognition.stop();
+  }
+
+  _onResult(event) {
+    this._resetSilenceTimeout();
+    let interim = '', final = '';
+    for (let i = event.resultIndex; i < event.results.length; i++) {
+      const transcript = event.results[i][0].transcript;
+      if (event.results[i].isFinal) {
+        final += transcript;
+      } else {
+        interim += transcript;
+      }
+    }
+
+    if (final) {
+      this._hidePreview();
+      this._insertText(final);
+      this.stop();
+    } else {
+      this._showPreview(interim);
+      // iOS workaround: isFinal is always false
+      this._iosStabilityCheck(interim);
+    }
+  }
+
+  _onError(event) {
+    this.stop();
+    if (event.error === 'not-allowed') {
+      app.showToast('Microphone access denied. Check browser settings.', 'error');
+    } else if (event.error === 'no-speech') {
+      // Silent fail — auto-stop is enough
+    } else if (event.error === 'network') {
+      app.showToast('Voice input requires internet connection.', 'error');
+    }
+  }
+
+  _onEnd() {
+    if (this.isRecording) this.stop(); // Cleanup if ended unexpectedly
+  }
+
+  _insertText(text) {
+    if (!app.activeSessionId || !text.trim()) return;
+    app.sendInput(text.trim());
+    // Don't add \r — let user review and press Enter
+  }
+
+  _resetSilenceTimeout() {
+    clearTimeout(this.silenceTimeout);
+    this.silenceTimeout = setTimeout(() => this.stop(), 5000);
+  }
+
+  // iOS Safari: isFinal is always false. Detect stability.
+  _lastTranscript = '';
+  _stabilityTimer = null;
+  _iosStabilityCheck(transcript) {
+    if (transcript !== this._lastTranscript) {
+      this._lastTranscript = transcript;
+      clearTimeout(this._stabilityTimer);
+      this._stabilityTimer = setTimeout(() => {
+        this._hidePreview();
+        this._insertText(transcript);
+        this.stop();
+      }, 750);
+    }
+  }
+
+  _showPreview(text) { /* Update DOM overlay with interim text */ }
+  _hidePreview() { /* Remove DOM overlay */ }
+  _updateButtons(state) { /* Toggle .recording class on mic buttons */ }
+}
+```
+
+#### 2. Mobile Button (KeyboardAccessoryBar)
+
+**Insert in HTML template** (after `/compact` button, before `paste`):
+```html
+<button class="accessory-btn accessory-btn-voice" data-action="voice" title="Voice input">
+  <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
+    <path d="M12 1a3 3 0 0 0-3 3v8a3 3 0 0 0 6 0V4a3 3 0 0 0-3-3z"/>
+    <path d="M19 10v2a7 7 0 0 1-14 0v-2"/>
+    <line x1="12" y1="19" x2="12" y2="23"/>
+    <line x1="8" y1="23" x2="16" y2="23"/>
+  </svg>
+</button>
+```
+
+**Add to handleAction switch:**
+```javascript
+case 'voice':
+  if (typeof voiceInput !== 'undefined') voiceInput.toggle();
+  break;
+```
+
+**Conditionally show:** Only render the button if `VoiceInput.supported` is true.
+
+#### 3. Desktop Button (index.html)
+
+**Insert in `toolbar-right`:**
+```html
+<div class="toolbar-right">
+  <button class="btn-toolbar btn-voice" id="voiceInputBtn" onclick="app.toggleVoiceInput()"
+          title="Voice input (Ctrl+Shift+V)" style="display: none;">
+    <svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
+      <path d="M12 1a3 3 0 0 0-3 3v8a3 3 0 0 0 6 0V4a3 3 0 0 0-3-3z"/>
+      <path d="M19 10v2a7 7 0 0 1-14 0v-2"/>
+      <line x1="12" y1="19" x2="12" y2="23"/>
+      <line x1="8" y1="23" x2="16" y2="23"/>
+    </svg>
+  </button>
+</div>
+```
+
+Show via JS on init: `if (voiceInput.supported) voiceInputBtn.style.display = '';`
+
+#### 4. Keyboard Shortcut
+
+**Ctrl+Shift+V** (in `setupEventListeners` keydown handler):
+```javascript
+if ((e.ctrlKey || e.metaKey) && e.shiftKey && e.key === 'V') {
+  e.preventDefault();
+  if (typeof voiceInput !== 'undefined') voiceInput.toggle();
+}
+```
+
+Why Ctrl+Shift+V: Ctrl+V is paste (sacred), but Ctrl+Shift+V ("paste without formatting") has no meaning in a terminal context. V for Voice is memorable.
+
+#### 5. CSS Styles
+
+**styles.css (desktop):**
+```css
+.btn-voice.recording {
+  background: rgba(239, 68, 68, 0.15);
+  border-color: rgba(239, 68, 68, 0.4);
+  color: #ef4444;
+  animation: voice-pulse 1.5s ease-in-out infinite;
+}
+
+@keyframes voice-pulse {
+  0%, 100% { transform: scale(1); opacity: 1; }
+  50% { transform: scale(1.08); opacity: 0.8; }
+}
+
+.voice-preview {
+  position: fixed;
+  bottom: 48px;
+  left: 50%;
+  transform: translateX(-50%);
+  background: rgba(0, 0, 0, 0.85);
+  color: rgba(255, 255, 255, 0.6);
+  font-style: italic;
+  padding: 6px 16px;
+  border-radius: 8px;
+  font-size: 0.85rem;
+  max-width: 80%;
+  white-space: nowrap;
+  overflow: hidden;
+  text-overflow: ellipsis;
+  z-index: 100;
+  pointer-events: none;
+}
+```
+
+**mobile.css:**
+```css
+.accessory-btn-voice.recording {
+  background: rgba(239, 68, 68, 0.2);
+  border-color: rgba(239, 68, 68, 0.4);
+  color: #ef4444;
+  animation: voice-pulse 1.5s ease-in-out infinite;
+}
+```
+
+#### 6. Interim Text Preview Overlay
+
+A small floating `<div>` showing live transcription:
+```javascript
+_showPreview(text) {
+  if (!this.previewEl) {
+    this.previewEl = document.createElement('div');
+    this.previewEl.className = 'voice-preview';
+    document.body.appendChild(this.previewEl);
+  }
+  this.previewEl.textContent = text || 'Listening...';
+  this.previewEl.style.display = '';
+}
+
+_hidePreview() {
+  if (this.previewEl) {
+    this.previewEl.style.display = 'none';
+  }
+}
+```
+
+#### 7. Cleanup (Memory Leak Prevention)
+
+Following Claudeman's cleanup patterns:
+- Store `VoiceInput` instance as `window.voiceInput` singleton
+- On SSE reconnect (`handleInit()`): stop any active recording, reset state
+- Remove preview element from DOM in cleanup
+- Clear all timeouts (silenceTimeout, stabilityTimer)
+
+#### 8. Accessibility
+
+```html
+<button role="button" aria-label="Start voice input" aria-pressed="false">
+```
+- Toggle `aria-pressed` and `aria-label` between "Start/Stop voice input"
+- Use `aria-live="polite"` region for interim text preview
+- Mic button focusable and activatable via Enter/Space
+
+---
+
+### Phase 2: Enhancements (Future)
+
+1. **Server-side Whisper fallback** — For Firefox/Edge users, add `/api/transcribe` endpoint that accepts audio blobs and forwards to OpenAI Whisper API. Requires `OPENAI_API_KEY` in env.
+2. **Language selector** — Settings dropdown to change `recognition.lang` (es-ES, de-DE, fr-FR, etc.)
+3. **Waveform visualization** — Small canvas with `AnalyserNode` bars near mic button
+4. **Haptic feedback** — `navigator.vibrate(50)` on start, `navigator.vibrate([30,50,30])` on stop (mobile)
+5. **Configurable auto-stop timeout** — Settings slider: 3-10 seconds
+6. **Voice command shortcuts** — "clear", "compact", "new session" recognized as commands
+7. **Recording duration indicator** — Small timer next to mic button
+
+---
+
+## Testing Plan
+
+### Manual Testing Matrix
+
+| Scenario | Chrome Desktop | Safari Desktop | Chrome Android | Safari iOS |
+|----------|---------------|----------------|----------------|------------|
+| Mic button visible | | | | |
+| Click starts recording | | | | |
+| Pulse animation plays | | | | |
+| Interim text appears | | | | |
+| Final text inserted | | | | |
+| Click stops recording | | | | |
+| Auto-stop on silence | | | | |
+| Permission denied error | | | | |
+| No speech timeout | | | | |
+| Ctrl+Shift+V shortcut | | | N/A | N/A |
+| Button hidden on Firefox | N/A | N/A | N/A | N/A |
+
+### Automated Tests (Playwright)
+
+```javascript
+// In mobile-test/ or a new test file
+test('voice button visible on Chrome', async ({ page }) => {
+  await page.goto('http://localhost:3000', { waitUntil: 'domcontentloaded' });
+  await page.waitForTimeout(2000);
+  const voiceBtn = page.locator('#voiceInputBtn');
+  // Chrome supports SpeechRecognition, button should be visible
+  await expect(voiceBtn).toBeVisible();
+});
+
+test('voice button hidden on Firefox', async ({ page, browserName }) => {
+  test.skip(browserName !== 'firefox');
+  await page.goto('http://localhost:3000', { waitUntil: 'domcontentloaded' });
+  const voiceBtn = page.locator('#voiceInputBtn');
+  await expect(voiceBtn).not.toBeVisible();
+});
+```
+
+Note: Actual SpeechRecognition can't be tested in headless Playwright (no mic). Tests verify button visibility and state toggling only.
+
+---
+
+## Implementation Sequence
+
+1. Add `VoiceInput` class to `app.js` (~80 lines)
+2. Add desktop button to `index.html` `toolbar-right`
+3. Add mobile button to `KeyboardAccessoryBar` template in `app.js`
+4. Add CSS styles to `styles.css` and `mobile.css`
+5. Wire keyboard shortcut `Ctrl+Shift+V` in `setupEventListeners`
+6. Add feature detection: hide button on unsupported browsers
+7. Add preview overlay for interim results
+8. Add cleanup in `handleInit()` for SSE reconnect
+9. Test on Chrome desktop + Android + Safari iOS
+10. COM
+
+**Estimated scope**: ~200 lines of JS, ~40 lines of CSS, ~5 lines of HTML. Single PR, no server changes.
+
+---
+
+## Key Risks & Mitigations
+
+| Risk | Likelihood | Mitigation |
+|------|-----------|------------|
+| iOS `isFinal` always false | High | Stability timer workaround (750ms) |
+| Chrome auto-stops after silence | Medium | Silence timeout handles this gracefully |
+| Safari re-asks mic permission per reload | Medium | Acceptable UX; no workaround exists |
+| Edge events don't fire | High | Feature detection hides button |
+| User on Firefox feels excluded | Low | <5% share; Phase 2 adds Whisper fallback |
+| Noisy environment → bad transcription | Low | Web Speech API handles noise internally; not our problem |
+| 60-second Chrome limit | Low | Prompts are short; auto-stop at 5s silence covers this |