mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-10 17:29:41 +02:00
feat(tiles): move tiles by hand, and detach a tile into its own window
Gesture control now moves tiles: with the tile grid open, pinch anywhere on a tile and carry it onto another tile (they swap) or an empty cell (it moves there); a session tab carried onto the grid joins it there. The overlay asks tile-grid.js what is under the hand (tileAtPoint, tileDropTargetAt) and drops through the mouse drag's own paths (dropSessionOnTile / dropSessionOnSlot, so _reorderTiles stays the one move path). Detach Tiles (App Settings > Appearance, per device, OFF by default, desktop only): - a tile's header let go outside the browser window opens the session as a pop-out (detachSession, which now takes an optional size and screen position), the tile's size, near the drop point, after a 120 ms settle; - dropped on another Codeman window's tiles it moves there: that window docks it through the target's own drop and posts 'tile-adopted' on the window channel, and only the window it was dragged from lets go of its copy (a last tile goes to the welcome screen, never onto that session); - a pop-out's title drags back onto any grid: the session is taken back, the pop-out is asked to close and is released (stops answering roll-calls; one a script cannot close says where the session went), and its stale 'detached' answers are ignored for 2 s; - a tile's menu offers "Open in a new window", and a hand that lets a tile go outside the grid pops it out too. With the setting off nothing changes. No server change. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -841,13 +841,15 @@ Tests: `test/tab-rail-search.test.ts` (gate) and `test/tab-rail-search.browser.t
|
||||
|
||||
⚠️ **Persistence and joining.** `codeman:tile-grid` (localStorage, per browser, never sent to the server) holds `{ v: 1, open, ids, count, focused, zoomed, colFr, rowFr }`, session ids and layout only, never content, written on every change (move, divider pointer-up, add, remove, count pick, focus, zoom); its `ids` are the CELLS, `null` for an empty one, and `sanitizeTileGridState` returns them as `cells` (a dropped id a hole, never a shift) beside the packed `ids` every list consumer wants, plus `freed` (cells whose session went away since) and `count`. ⚠️ `count` is how many tiles the user's own last change left: `removeTile(..., { gone: true })` (the `_onSessionDeleted` wrapper, `_reconcileTileGrid`, `_onTileExit`, and app.js `_markDetached` for a session popped out while the grid is open) does NOT lower it, so the next activation refills that cell, while a removal by hand does; it is derived (the sessions the cells name) for a value written before it, and the format stays `v: 1` so an older build still reads a newer value. A restore keeps the cells when their shape matches the current one, else `reformTileCells` (positions kept when all fit, else packed). ⚠️ Every close keeps the grid as `open: false` (`closeTileGrid({ keepStored: true })` from every caller: the toggle, a non-tiled pick, `leaveTiles`, Home, the width gate, the last tile leaving, "Open group as tiles", kill-all); `_closeStoredTileGrid` (a `#session=` link on load) flips `open` on the RAW stored value, so a gone id still frees its cell; `_openStoredTileGrid` holds `_persistTileGrid` (`_tilePersistHold`) until the grid is back, so `openTileGrid`'s packed intermediate layout is never written over it. Only when none of its sessions survive does activation rank from scratch. ⚠️ A RELOAD's fill (`_restoreTileGrid` inside `handleInit`) ranks on status and stamps only: pending approvals arrive later through the async `seedApprovals`, so needs-input cannot rank for that one fill, and a late seed never re-forms the restored grid (holding the fill back would open fewer tiles, possibly another shape, and move the user's tiles a moment after the reload; pinned in `test/tile-grid-restore.test.ts`). A solo window never reads or writes it, an automatic zoom is not stored. Sessions created by THIS tab's Run join the open grid (`_joinTileGridFromRun`, called from session-ui.js `_ensureCreatedSessionVisible`; a wrapper from tile-grid.js would be overwritten by session-ui.js's later `Object.assign`); sessions created elsewhere arrive only by `session:created` and never join. ⚠️ A tile that joins that way connects BEFORE Run starts its pane: its resize reaches a session with no PTY, which `Session.resize()` only records (`_lastDesktopDims`) while the spawn uses a fixed 120x40, and Run's own resize step measures the parked main terminal (`display: none`, so `proposeDimensions()` is NaN and the step is skipped). Measured live: a 97x17 tile over a 120x40 pane. So `_renderTileChrome` remembers each tile's last-seen pid and calls `tile.paneStarted()` when it appears or changes (keyed on the sessions map, so a `handleInit` after an SSE drop counts too); `paneStarted()` forgets the sent size and sends it, and a hidden tile (a zoomed neighbour) sends nothing but keeps the size forgotten, so its next `fit()` sends it. A plain `fit({ force: true })` would lose that case. Tests: `test/tile-grid-*.test.ts` over the shared vm harness `test/mocks/tile-grid-vm.ts`.
|
||||
|
||||
⚠️ **Detach Tiles** (`tileDetachEnabled`, per-device, default OFF: in `displayKeys`, stripped from the settings PUT, not in `SettingsUpdateSchema`; App Settings → Appearance, its row shown on desktop only). A tile leaves the grid for a window of its own, and comes back into any Codeman window's grid, by dragging, with nothing new on the server: a pop-out is another client of the same session (`detachSession`), and a tile is a `TerminalTile` in whichever window has it. (1) A header let go OUTSIDE the window opens it with `detachSession(id, placement)`, the tile's size (at least 480x320), its header near the drop point, after `TILE_TEAR_OFF_SETTLE_MS` (120 ms) that a claim from another window cancels. "Outside" is the dragend's own point off the viewport with `dropEffect: 'none'`; where a browser reports no point (0, 0) it is the page's record of a `dragleave` at the viewport's edge with nothing entered, cleared by any later `dragover`. A drop that any target took (`dropEffect: 'move'`, in this window or another) never opens one, a drop inside on nothing changes nothing, and with the setting on a lone or zoomed tile drags too (out only: no target here takes it). (2) A header dropped on ANOTHER Codeman window's tile or empty cell moves it there: that window's `_acceptTabDrops` takes a drag that is none of its own and carries `application/x-codeman-tile` (`_isForeignTileDrag`; the id is only readable on drop), docks it through the target's own drop (`_dockForeignTile` → `dropSessionOnTile` / `dropSessionOnSlot`, so `_reorderTiles` stays the one move path) and posts `tile-adopted` on the `codeman-windows` channel. Only the window the tile was dragged FROM acts on that (its drag still settling, or ended within `TILE_ADOPT_WINDOW_MS`): the tile goes with `focus: false`, and a last tile closes the grid onto the welcome screen, never onto that session, since two windows sizing one PTY garble it. (3) A pop-out's title (`#soloSessionTitle`, `_installSoloTileHandle`) carries the same type, so it drags back onto any grid: docking a popped-out session `_redock`s it first (a detached session never mounts), posts `close-request`, and ignores that pop-out's `detached` answers for `TILE_DOCK_GRACE_MS` (`_tileDockedRecently`), which would otherwise take the new tile straight back off. A pop-out that gets `close-request` is RELEASED (`_releaseSoloWindow`): it posts `redocked` at once, stops answering roll-calls, then closes; one a script may not close (opened by typing its URL) says where the session went instead of staying a pop-out every dashboard would mark detached again. A tile's ⋯ menu offers "Open in a new window" with the setting on (`detachTile`, placed over the tile): the keyboard's way to the drag. ⚠️ Chrome keeps a script-opened window on the opener's screen unless the page holds the Window Management permission, so a drop on another monitor opens at this screen's edge (drag the window across from there), and Wayland ignores the position. The gesture overlay reaches all of it through `tileAtPoint` / `canGrabTile` / `tileDropTargetAt` / `dropOnTileTarget` / `isOverTileGrid` / `detachTileAtPoint`, never move logic of its own; its pop-out has no click behind it, so a popup blocker refuses it unless the site's pop-ups are allowed (detachSession's toast says so, and the tile stays). Tests: `test/tile-grid-detach.test.ts`.
|
||||
|
||||
### Gesture control: the setting
|
||||
|
||||
**Gesture control** (the camera hand-tracking overlay) is **opt-in, default OFF**, under App Settings → Terminal & Input → Scrolling & rendering (`gestureControlEnabled`). `CODEMAN_GESTURE=1` makes the feature _available_ on the instance (CSP widening + `/gesture/` assets) and sets `window.__codemanGestureAvailable` (the Input section only shows when set); the overlay bundle is injected by `renderIndexHtml` **only when the setting is enabled**, so that method is `async` and reads `settings.json` via `readSettings(true)` — the `true` forces a **fresh** read (bypassing the 2s `_settingsCache`), because a post-save reload happens within that TTL and the cached value would otherwise render the pre-toggle state. Toggling the setting reloads the page (the bundle is render-injected).
|
||||
|
||||
### Gesture control: the source package
|
||||
|
||||
**Gesture-control source lives in-repo** at `packages/gesture-control/` (workspace package `codeman-gesture-control`, was the standalone `Ark0N/codeman-gesture-control` repo). The transport-agnostic core is `src/gesture/*` (MediaPipe GestureRecognizer → One-Euro-filtered cursor → pinch state machine); `src/codeman/entry.ts` is the Codeman _consumer_ that maps grab/drag/drop onto real `.session-tab`/toolbar buttons and is the bundle entry. **Edit there, then run `npm run build:gesture`** (`scripts/build-gesture-bundle.mjs` → esbuild bundles `entry.ts`, MediaPipe JS included, into `src/web/public/gesture/gesture-codeman.js`) and **commit the regenerated bundle** — the committed bundle is what dev/`tsx` serves (no bundler at runtime), and `scripts/build.mjs` reruns the same step so prod always reflects current source. The MediaPipe **wasm + model** are NOT bundled — loaded at runtime from same-origin `/gesture/wasm` + `/gesture/gesture_recognizer.task`, fetched by `scripts/fetch-gesture-assets.mjs` (gitignored, see Gotchas). `entry.ts` mounts `window.__codemanGesture = new GestureBridge()` idempotently at module-eval. A standalone vite playground (`npm run dev` in the package — fake tabs, no Codeman) lets you iterate on gesture _feel_ in isolation. ⚠️ Keep `MP_VERSION` in `fetch-gesture-assets.mjs` in sync with `@mediapipe/tasks-vision` in `packages/gesture-control/package.json`.
|
||||
**Gesture-control source lives in-repo** at `packages/gesture-control/` (workspace package `codeman-gesture-control`, was the standalone `Ark0N/codeman-gesture-control` repo). The transport-agnostic core is `src/gesture/*` (MediaPipe GestureRecognizer → One-Euro-filtered cursor → pinch state machine); `src/codeman/entry.ts` is the Codeman _consumer_ that maps grab/drag/drop onto real `.session-tab`/toolbar buttons and is the bundle entry. **Edit there, then run `npm run build:gesture`** (`scripts/build-gesture-bundle.mjs` → esbuild bundles `entry.ts`, MediaPipe JS included, into `src/web/public/gesture/gesture-codeman.js`) and **commit the regenerated bundle** — the committed bundle is what dev/`tsx` serves (no bundler at runtime), and `scripts/build.mjs` reruns the same step so prod always reflects current source. The MediaPipe **wasm + model** are NOT bundled — loaded at runtime from same-origin `/gesture/wasm` + `/gesture/gesture_recognizer.task`, fetched by `scripts/fetch-gesture-assets.mjs` (gitignored, see Gotchas). `entry.ts` mounts `window.__codemanGesture = new GestureBridge()` idempotently at module-eval. A standalone vite playground (`npm run dev` in the package — fake tabs, no Codeman) lets you iterate on gesture _feel_ in isolation. ⚠️ Keep `MP_VERSION` in `fetch-gesture-assets.mjs` in sync with `@mediapipe/tasks-vision` in `packages/gesture-control/package.json`. The tile grid's verbs (a tile pinched anywhere on it and carried onto another tile or an empty cell; a tab carried onto the grid joins it there; with Detach Tiles a tile let go outside the grid pops out) ask tile-grid.js's gesture helpers (see Tile grid, Detach Tiles) and never move tiles themselves, so the hand and the mouse cannot disagree about a drop.
|
||||
|
||||
### Theme skins
|
||||
|
||||
|
||||
Reference in New Issue
Block a user