# Performance Analysis & Optimization Opportunities **Date**: 2026-03-07 **Scope**: Full-stack performance analysis — backend PTY handling, SSE broadcasting, frontend terminal rendering, local echo overlay, DOM updates, config/scaling limits. **Constraint**: All recommendations preserve existing functionality including local echo, backpressure, anti-flicker pipeline, and mobile support. --- ## Executive Summary The codebase is already well-optimized in critical paths. The multi-layer backpressure system, adaptive terminal batching, DEC 2026 sync markers, and incremental state serialization are strong. The main opportunities are in **reducing unnecessary work** (SSE filtering, DOM rebuilds, lazy terminal init) rather than algorithmic changes. **Top 5 high-impact opportunities:** | # | Optimization | Impact | Risk | Effort | |---|-------------|--------|------|--------| | 1 | Session-scoped SSE subscriptions | Bandwidth -60-80%, CPU -40% | Medium | Medium | | 2 | Lazy xterm.js for minimized subagent windows | Memory -3.5MB at 50 agents | Low | Low | | 3 | Targeted badge update (skip full tab rebuild) | Eliminates O(n) reflow on badge change | Low | Low | | 4 | Conditional SSE padding (tunnel-only, terminal-only) | Bandwidth -70% when tunneled | Low | Low | | 5 | Canvas renderer on mobile | GPU pressure reduction, battery savings | Low | Low | --- ## 1. SSE Broadcasting ### Current State - **92 event types** broadcast to all connected clients (max 100) - Single `JSON.stringify()` per event, shared across all clients (efficient) - **No per-client filtering** — every client receives every event regardless of which session they're viewing - 8KB padding appended to **every** event when tunnel is active (forces Cloudflare proxy flush) - Backpressure: clients marked as backpressured if `reply.raw.write()` returns false; recovery via `session:needsRefresh` ### Bottlenecks **B1: No session-scoped SSE subscriptions** (`server.ts:1986`) - Client viewing session A still receives all events for sessions B through T - With 20 active sessions, ~95% of terminal events are irrelevant to any given client - Cost: wasted bandwidth, CPU for JSON parsing, and event handler dispatch on client **B2: Unconditional 8KB padding** (`server.ts:1977`) - Every event gets 8KB comment padding when tunnel is active - A `task:updated` event (~200 bytes payload) becomes ~8.2KB - High-frequency events like `session:terminal` need the padding; low-frequency events like `session:created` don't ### Recommendations **R1: Session-scoped SSE subscriptions** (High impact) - Add `?sessions=id1,id2` query param to `/api/events` SSE endpoint - Server filters events by session ID before broadcasting - Client subscribes to active session + "global" events (session lifecycle, system) - Re-subscribes on tab switch (or subscribe to all with client-side filter as fallback) - **Savings**: ~80% bandwidth reduction for single-session viewers; ~60% for multi-session dashboards **R2: Tiered SSE padding** (Medium impact) - Only pad `session:terminal` events and SSE heartbeats (the two that need proxy flush) - Skip padding for low-frequency structural events (`session:created`, `task:updated`, etc.) - **Savings**: ~70% padding overhead reduction; terminal events already large enough to flush --- ## 2. Terminal Rendering ### Current State (Well-Optimized) - **6-layer anti-flicker pipeline**: Server batching (adaptive 16-50ms) → DEC 2026 sync wrap → single JSON serialize → client rAF batching → sync segment parser → chunked buffer loading (32KB/frame) - **64KB/frame write budget** with DEC 2026 sync-segment awareness (prevents 141KB single-frame freezes) - **3-layer backpressure**: SSE cap (128KB queued → drop + refresh), frame budget (64KB/frame), chunked restore (32KB/frame) - WebGL renderer enabled by default with canvas fallback on context loss - Typical latency: 16-32ms; worst case: ~115ms (50ms server batch + 50ms sync wait + 16ms rAF) ### Bottlenecks **B3: WebGL on mobile** (`app.js:627-637`) - Mobile GPUs are weaker; WebGL context loss more likely on low-end devices - Canvas renderer is sufficient for mobile (typically 1 session, smaller viewport) **B4: Static scrollback for all sessions** (`app.js:572`) - Default 5000 lines scrollback for all sessions regardless of activity level - Heavy output sessions (build logs, test runners) accumulate large scroll buffers **B5: No addon lazy loading** - FitAddon, Unicode11Addon, and WebGLAddon all loaded at terminal init - Unicode11Addon only needed for CJK content; WebGLAddon is large ### Recommendations **R3: Force canvas renderer on mobile** (Low risk) - Detect `MobileDetection.isMobile()` and skip WebGL addon loading - Reduces GPU memory pressure, prevents context loss crashes - Mobile typically has 1-2 sessions — canvas performance is more than adequate **R4: Dynamic scrollback based on session activity** (Low risk) - Active sessions (working state): 5000 lines (current default) - Inactive/idle sessions: reduce to 2000 lines - Restore on session select (fetch from server buffer) - **Savings**: ~60% scrollback memory for idle sessions **R5: Lazy-load Unicode11Addon** (Low risk) - Only load when CJK content is detected in terminal output - Detection: check for characters in CJK Unicode ranges during ANSI stripping (already iterating) - Most sessions never need it --- ## 3. DOM & Session Tab Rendering ### Current State - Session tabs use **intelligent incremental updates** with debounced 100ms rendering - Incremental path: only updates changed properties (classes, textContent, badges) when session list is stable - Full rebuild path: triggered when sessions added/removed **or badge count changes** - Subagent windows: per-window xterm.js instances, even when minimized ### Bottlenecks **B6: Badge count change triggers full tab rebuild** (`app.js:3207-3209`) - A single subagent badge increment on one tab triggers `_fullRenderSessionTabs()` — rebuilds entire sidebar HTML via `innerHTML =` - With 20 sessions, this is an O(n) reflow for a single badge number change - Badge changes are frequent during active subagent work **B7: Minimized subagent windows retain xterm.js instances** (`subagent-windows.js`) - 50 subagent windows × ~75KB per xterm.js instance = ~3.75MB DOM memory - Minimized windows are invisible but their terminals remain in DOM - xterm.js instances continue processing resize events even when hidden **B8: `backdrop-filter: blur()` on overlays** (`styles.css:2246-2247, 3098`) - Forces new stacking context, disables browser compositing optimizations - 50-100ms layout thrashing on modal open/close - Only 2 uses, but they're on frequently toggled overlays ### Recommendations **R6: Targeted badge update without full rebuild** (Low risk) - When badge count changes but session list is stable, update only the badge `` textContent - Keep incremental path for badge changes; only use full rebuild for structural changes (add/remove sessions) - **Savings**: Eliminates O(n) reflow per badge change; reduces to O(1) targeted update **R7: Lazy xterm.js initialization for subagent windows** (Medium impact) - Only create xterm.js Terminal instance when window is restored/maximized - On minimize: serialize terminal buffer, dispose Terminal instance, keep buffer in memory - On restore: create new Terminal, write buffer back - **Savings**: ~3.5MB DOM reduction at 50 minimized agents; eliminates hidden resize processing - **Trade-off**: ~200-500ms restore delay (buffer write), mitigated by chunked loading **R8: Replace `backdrop-filter: blur()` with `background: rgba()`** (Low risk) - Use semi-transparent background instead of blur effect - Or use `will-change: transform` hint if blur is kept - **Savings**: Eliminates forced recomposition layer; 50-100ms faster overlay open --- ## 4. Backend PTY & State Management ### Current State (Excellent) - **BufferAccumulator**: Array-based chunking with lazy join on read — avoids O(n) string concatenation - **ANSI stripping**: Throttled at 150ms intervals with lazy evaluation (not per-chunk) - **State persistence**: 500ms debounce + incremental JSON caching per session (only dirty sessions re-serialized) - **Expensive parsers**: Throttled to 150ms window, accumulated data capped at 64KB - **Memory**: All buffers have hard limits (2MB terminal, 1MB text, 1000 messages, 64KB line buffer) ### Bottlenecks **B9: Pending clean data cap at 64KB** (`session.ts:1097-1133`) - Between 150ms processing windows, raw PTY data accumulates in `_pendingCleanData` - Capped at 64KB — excess data rolls off (old data discarded) - During heavy output (large build logs), this means parsers may miss content - Acceptable trade-off for performance, but worth documenting **B10: `LRUMap.delete()` is O(n) worst case** (`utils/lru-map.ts:137-138`) - When deleting the newest entry, iterates all keys to find new newest - Rare in practice (delete is uncommon; set/get are hot paths) - Could matter during mass cleanup of 500 agents ### Recommendations **R9: Consider adaptive pending data cap** (Low priority) - During idle detection (critical to get right), increase cap to 128KB - During active working state, keep at 64KB (parsers less critical) - **Benefit**: More accurate idle detection during heavy output **R10: Track second-newest in LRUMap** (Low priority) - Maintain a `_secondNewestKey` alongside `_newestKey` - On delete of newest, promote second-newest without iteration - Only matters at scale (500+ agents with frequent eviction) --- ## 5. Local Echo & Input Path ### Current State (Well-Designed) - **DOM overlay approach** — `` elements in `.xterm-screen` at z-index 7, completely independent of `terminal.write()` - **Render caching**: `_lastRenderKey` includes text, position, column offsets — skips redundant re-renders - **Input flow**: Char accumulation → Enter triggers flush → 80ms delay before `\r` (ensures text reaches PTY first) - **Tab completion**: Baseline snapshot → detect buffer change → 300ms fallback timer - **CJK support**: Per-character width detection with `terminal.unicode.getStringCellWidth()` preferred, manual fallback - **Prompt detection**: Bottom-up line scan, O(rows) — cached position, column-lock prevents jitter ### Bottlenecks **B11: tmux send-keys latency** (~50-100ms per input) - Each `writeViaMux()` spawns a child process (`tmux send-keys`) - Text and Enter sent separately with 50ms delay between - For rapid typing: characters batch before Enter, so overhead is per-command not per-keystroke - **Acceptable trade-off** for session persistence (tmux survives server restarts) **B12: 80ms delay between text flush and Enter** (`app.js:872-875`) - Intentional: ensures text reaches PTY before Enter, preventing Ink from processing empty input - Adds 80ms to perceived Enter-to-response latency - Could potentially be reduced with acknowledgment-based approach **B13: Scroll listener on terminal viewport** (`zerolag-input-addon.ts:139`) - 50ms debounced re-render on scroll — acceptable but fires frequently during heavy output - Overlay hidden when scrolled up (correct behavior), shown when at bottom ### Recommendations **R11: Reduce Enter delay from 80ms to 50ms** (Low risk, test carefully) - The tmux `send-keys` already has 50ms internal delay - Combined with network latency, 80ms client-side may be excessive - Test with Ink-heavy sessions (Claude Code's status bar) — if text arrives before Enter at 50ms, reduce - **Savings**: 30ms perceived latency reduction per command **R12: Batch tmux send-keys via stdin pipe** (Medium effort, high impact for rapid input) - Instead of spawning `tmux send-keys` per input, maintain a persistent connection - Use `tmux -C` (control mode) for programmatic interaction without child process spawning - **Savings**: Eliminate ~50-100ms process spawn overhead per input - **Risk**: Control mode has different semantics; needs careful testing with session persistence **R13: Skip overlay re-render during heavy output scroll** (Low risk) - When terminal is receiving >10KB/s output, hide overlay entirely (user isn't typing during heavy output) - Re-show overlay after 500ms of output silence - **Savings**: Eliminates unnecessary DOM overlay re-renders during build logs / test output --- ## 6. Polling & File Watchers ### Current State - **SubagentWatcher**: 1s base poll, full scan throttled to every 5s, fs.watch() on known directories - **TranscriptWatcher**: 1 per session, fs.watch() primary with 1s poll fallback - **ImageWatcher**: chokidar per session with 100ms stability poll, burst limit 20/10s - **TeamWatcher**: chokidar primary with 30s poll fallback, LRU caches (50 teams, 200 tasks) - **RalphTracker**: Todo cleanup every 5 minutes ### Scaling Profile (20 sessions) | Component | Instances | Frequency | Total ops/sec | |-----------|-----------|-----------|---------------| | SubagentWatcher | 1 (global) | Full scan every 5s | 0.2/s | | TranscriptWatcher | 20 | 1s poll (fallback) | 20/s max | | ImageWatcher | 20 | 100ms poll (during writes only) | 200/s burst | | TeamWatcher | 1 (global) | 30s poll (fallback) | 0.03/s | | SSE heartbeat | 1 (global) | 15s | 0.07/s | | SSE dead client check | 1 (global) | 30s | 0.03/s | | Mux stats collection | 1 (global) | 2s | 0.5/s | | **Total steady-state** | | | **~21/s** | ### Recommendations **R14: Increase TranscriptWatcher poll interval to 2s** (Low risk) - Transcript changes are infrequent (new messages every few seconds at most) - fs.watch() is the primary mechanism; polling is fallback - **Savings**: Halves fallback filesystem checks (20/s → 10/s for 20 sessions) **R15: Share chokidar instances for co-located session directories** (Medium effort) - Sessions in the same parent directory could share a single chokidar watcher with depth:3 - Common case: multiple sessions in `~/projects/foo/` — one watcher covers all - **Savings**: Reduce chokidar instances from 20 to ~5-10 for typical workloads --- ## 7. Frontend Asset Delivery ### Current State - **app.js**: 12,027 lines (source) → esbuild minified → gzip/brotli compressed (~30-40KB gzipped) - **Static caching**: `maxAge: '1y'` via `@fastify/static` - **Service worker**: Push notification handler only — no asset caching - **No code splitting**: Single monolithic app.js bundle ### Bottlenecks **B14: No cache-busting mechanism** - `maxAge: '1y'` means browsers cache aggressively - After deployment, users need `Ctrl+Shift+R` to see updates - No content hash in filenames or ETags for automatic invalidation **B15: Monolithic app.js** - All 12K lines loaded on initial page load regardless of which features are used - Ralph wizard, plan orchestrator UI, team management — all loaded upfront - Mobile loads the same bundle as desktop ### Recommendations **R16: Add content hash to asset filenames** (Medium impact) - Build step: rename `app.js` → `app.[hash].js` - Generate a manifest or inject hash into HTML template - Keep `maxAge: '1y'` — cache invalidation happens via filename change - **Savings**: Eliminates stale cache issues after deployment; removes need for manual hard refresh **R17: Code-split app.js into core + feature modules** (High effort, medium impact) - Core (~4K lines): terminal, SSE, session management, tabs, input handling - Deferred (~8K lines): Ralph wizard, plan UI, team management, subagent windows, image viewer - Load deferred modules on first use via dynamic `import()` or lazy `