Bring the Ark0N/codeman-gesture-control repo in-tree as the codeman-gesture-control
workspace package so the hand-tracking overlay can be developed in the Codeman repo.
New npm run build:gesture bundles src/codeman/entry.ts into the served
gesture-codeman.js; scripts/build.mjs reruns it on every production build.
Source formatted to Codeman's prettier style.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The self-hosted gesture assets (~27MB of wasm runtime + gesture_recognizer.task)
were committed to the repo. Replace that with scripts/fetch-gesture-assets.mjs,
which downloads them into src/web/public/gesture/ — idempotent (skips existing)
and non-fatal (the overlay is opt-in via CODEMAN_GESTURE=1, so a fetch failure
only warns). Wired into:
- postinstall.js (dev: populates src/web/public/gesture for `npm run dev`)
- build.mjs (before `cp -r src/web/public dist/web/`, so prod/dist gets them)
The files are already covered by the bare `public` .gitignore rule, so they
stay untracked. The overlay bundle (gesture-codeman.js) remains committed — it's
built from a separate repo and is small. Pin @mediapipe wasm to 0.10.21 to match
the bundled tasks-vision API.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Updates the opt-in gesture overlay (still gated by CODEMAN_GESTURE=1):
- Bundle (gesture-codeman.js) rebuilt from Ark0N/codeman-gesture-control:
- Detach now calls window.app.detachSession(id) directly (the on-tab pop-out
hook) instead of a separate /session/:id window.open reimplementation.
- Pinch a session tab → ghost follows your hand → pull out to undock.
- Pinch the Run (#runBtn → app.run()) or Run Shell (.btn-shell →
app.runShell()) toolbar button to fire it; drift cancels the tap.
- Camera shows fullscreen-dimmed by default (⛶ toggles a corner preview).
- Robust start-error reporting; GPU→CPU MediaPipe delegate fallback.
- Self-hosted MediaPipe (no CDN): serves the wasm runtime + gesture_recognizer
.task from /gesture/ so a browser content-blocker can't break startup. The
overlay points wasmBase/modelUrl there. (~27MB of assets; could later be a
build/postinstall fetch instead of committed blobs.)
- server.ts: cache-bust the injected bundle URL with its mtime (?v=), since
static is served with a 1-year cache — a redeploy is now never stale.
format:check / lint scope (src/**/*.ts) clean; server.ts typechecks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Loads a hand-tracking overlay into the dashboard that detaches a session by
pinch-grabbing its tab and pulling it out — driving the existing
app.detachSession(id) hook. Bundle (src/web/public/gesture/gesture-codeman.js)
is built from the codeman-gesture-control project's src/codeman/entry.ts
(esbuild, MediaPipe included) and served same-origin.
OFF by default — guarded entirely by CODEMAN_GESTURE=1:
- server.ts: injects the module script into the dashboard HTML only (not solo
/session/:id popups, which have no tab strip).
- auth.ts: widens CSP only under the flag — adds 'wasm-unsafe-eval' (MediaPipe
WASM) and the pinned MediaPipe CDNs (cdn.jsdelivr.net wasm, storage.googleapis.com
model) to connect-src, plus worker-src 'self' blob:. Production CSP is unchanged
when the flag is off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
