feat(files): play video and audio from outside the workspace too

A clip an agent wrote inside the workspace played with a working scrub bar,
while the same file in /tmp was refused as an unsupported type. The workspace
preview classified media with its own inline extension sets and the attachment
allowlist had no media at all, so the two paths disagreed about what a video is.

- VIDEO_ATTACHMENT_EXTENSIONS and AUDIO_ATTACHMENT_EXTENSIONS now live in
  attachment-registry.ts and are imported by file-content's classification, so
  both paths answer the same. mp4/webm/mov/m4v/ogv and
  mp3/wav/ogg/oga/m4a/aac/flac/opus join the attachment allowlist.
- Real MIME types for those extensions. Without one the raw route falls back to
  application/octet-stream, which a <video> refuses to decode: the player
  renders and then does nothing.
- getAttachmentType() gained the video and audio members of
  AttachmentDetectedType. Attachment cards have no per-type CSS and their
  thumbnail falls back to the type label, since the thumbnailer has no media
  branch and answers 204 rather than spawning a converter.
- The preview overlay's by-id branch renders <video>/<audio> with the same
  markup as the workspace branch, playsinline included. Serving was already
  range-aware, so seeking works.

The image-watcher keeps its own narrow detection list (png/pdf/docx/pptx), so
this does not start popping cards for every video an agent writes. Text types
that are not md or txt (.json, .log, code files) remain out of the allowlist by
choice and still report what is previewable instead.

Verified on an isolated instance: an external mp4 and mp3 both play, seek, and
report the right duration, matching the in-workspace clip exactly, and a click
on an external mp4 in the terminal opens the player with no attachment card.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer
2026-08-16 17:35:43 +02:00
parent 4e2c1b9989
commit cbc54fc98d
7 changed files with 151 additions and 6 deletions
+1 -1
View File
@@ -230,7 +230,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
**Attachments** (live external document references; all wiring in `file-routes.ts`): a **registry** maps a stable `attachmentId` to a realpath-resolved, extension-allowlisted absolute path, so browser requests never carry arbitrary absolute paths. ⚠️ The **magic-link scanner** (`codeman://attach?...` in terminal output) is **prompt-injectable**, so its scan path is force-confined to the session workspace; a hostile prompt could otherwise exfiltrate arbitrary host files over SSE. The security gate is an extension **allowlist**, not a blocklist. `document-conversion-limiter.ts` caps converter spawns globally: without it, N large docs detected at once fork N multi-minute processes, which is a resource-exhaustion vector. → [architecture-invariants#attachments](docs/architecture-invariants.md#attachments)
**File-path links (terminal + chat)**: a path an agent prints is clickable on BOTH surfaces and opens the file-preview overlay. ⚠️ ONE pattern (`FILE_PATH_LINK_PATTERN` / `absoluteFilePathPattern()` in constants.js) feeds the xterm link provider AND the response viewer's `_linkifyFilePaths()`; a fresh instance per call, since `lastIndex` is per-object state. The chat linkifier walks TEXT NODES with DOM APIs (the source is model output; never rebuild sanitized markup as a string) and skips subtrees already inside an `<a>`. ⚠️ **An out-of-workspace path is served through the ATTACHMENT routes, not the file routes** — `file-content`/`file-raw` are workspace-confined and 404 exactly the paths agents print most (a `/tmp` capture, Claude's scratchpad), so `openFilePreview()` registers such a path via `POST /api/sessions/:id/attachments` with **`notify: false`** (suppresses only the `attachment:detected` broadcast — same guard, same routes; without it every click also popped a card announcing the file already on screen) and renders by id. The click is an explicit action on the explicit, Origin-guarded route, which is what distinguishes it from the force-confined magic-link scanner; the extension allowlist is unchanged. → [architecture-invariants#file-path-links-terminal--response-viewer](docs/architecture-invariants.md#file-path-links-terminal--response-viewer)
**File-path links (terminal + chat)**: a path an agent prints is clickable on BOTH surfaces and opens the file-preview overlay. ⚠️ ONE pattern (`FILE_PATH_LINK_PATTERN` / `absoluteFilePathPattern()` in constants.js) feeds the xterm link provider AND the response viewer's `_linkifyFilePaths()`; a fresh instance per call, since `lastIndex` is per-object state. The chat linkifier walks TEXT NODES with DOM APIs (the source is model output; never rebuild sanitized markup as a string) and skips subtrees already inside an `<a>`. ⚠️ **An out-of-workspace path is served through the ATTACHMENT routes, not the file routes** — `file-content`/`file-raw` are workspace-confined and 404 exactly the paths agents print most (a `/tmp` capture, Claude's scratchpad), so `openFilePreview()` registers such a path via `POST /api/sessions/:id/attachments` with **`notify: false`** (suppresses only the `attachment:detected` broadcast — same guard, same routes; without it every click also popped a card announcing the file already on screen) and renders by id. The click is an explicit action on the explicit, Origin-guarded route, which is what distinguishes it from the force-confined magic-link scanner. ⚠️ **Media extensions are single-sourced** (`VIDEO_ATTACHMENT_EXTENSIONS`/`AUDIO_ATTACHMENT_EXTENSIONS` in `attachment-registry.ts`, imported by `file-content`'s classification) so a clip plays the same in or out of the workspace; a player needs all THREE of allowlist + a real `MIME_TYPES` entry (octet-stream renders a dead player) + the range-aware body. The image-watcher keeps its own narrow detection list, so this does not card every video an agent writes. → [architecture-invariants#file-path-links-terminal--response-viewer](docs/architecture-invariants.md#file-path-links-terminal--response-viewer)
**Filesystem path picker** (Link Existing "Browse" + the mobile keyboard's `📁 Path` key): lazy one-directory browsing via `GET /api/filesystem/browse`, with `GET /api/filesystem/preview` for the tapped file. Inserts the path **without** Enter, so the prompt is never submitted; the sibling `⌫ All` key clears only the unsent prompt and must never send the agent's `/clear`. ⚠️ This is a **second file-serving surface and inherits neither the attachment confinement nor its ownership scoping** — it allowlists Home, `CASES_DIR`, `/mnt/d` and `CODEMAN_FILE_PICKER_ROOTS`, blocks sensitive trees, and rejects symlink escapes **after** `realpath`. ⚠️ The optional `sessionId` is an ownership boundary that must be `canAccessOwned`-checked by hand (it does not go through `findSessionOrFail`), and in multi-user mode a non-admin gets only their own `userSpacePath` as a root: per-user spaces live INSIDE `homedir()`, so a `Home` root exposes every other user's workspace. Previews go through the same global conversion limiter, and Markdown/TXT/JSON are served as inert `text/plain`. → [architecture-invariants#filesystem-path-picker](docs/architecture-invariants.md#filesystem-path-picker)
+3 -1
View File
@@ -130,7 +130,9 @@ A file path an agent prints is a link on both surfaces it can appear on, and cli
**The chat linkifier walks text nodes.** `_linkifyFilePaths()` builds anchors with `createElement`/`textContent` on the rendered subtree, never by rebuilding sanitized markup as a string — the source is model output. Subtrees already inside an `<a>` are skipped (marked autolinks URLs; a nested anchor would swallow the click), and the anchor's text is the path verbatim so "copy code" still yields what the agent printed. `test/response-viewer-file-links.test.ts` pins both properties.
**Out-of-workspace paths go through the attachment routes, not the file routes.** `file-content`/`file-raw` resolve against `workingDir` and 404 anything that escapes it, which is correct and unchanged — but the paths agents most often print (a `/tmp` capture, Claude's own scratchpad, another checkout) are exactly that, so clicking one used to report "File not found" for a file sitting on disk. `openFilePreview()` now detects the case (`_isExternalPreviewPath`, a string compare for ROUTING only; the real decision stays server-side) and registers the path via `POST /api/sessions/:id/attachments` first, rendering by id. ⚠️ That registration passes `notify: false`, which suppresses ONLY the `attachment:detected` broadcast — the guard, the registry entry and the by-id routes are identical either way. Without it every click also popped an attachment card announcing the file already filling the screen. ⚠️ The click is an explicit user action on the **explicit, Origin-guarded** registration route, which is why it may cross the workspace boundary at all; the passive magic-link scanner stays force-confined, and nothing here widens `SUPPORTED_ATTACHMENT_EXTENSIONS` (an out-of-workspace `.svg`/`.bmp` is refused with a message naming what IS previewable).
**Out-of-workspace paths go through the attachment routes, not the file routes.** `file-content`/`file-raw` resolve against `workingDir` and 404 anything that escapes it, which is correct and unchanged — but the paths agents most often print (a `/tmp` capture, Claude's own scratchpad, another checkout) are exactly that, so clicking one used to report "File not found" for a file sitting on disk. `openFilePreview()` now detects the case (`_isExternalPreviewPath`, a string compare for ROUTING only; the real decision stays server-side) and registers the path via `POST /api/sessions/:id/attachments` first, rendering by id. ⚠️ That registration passes `notify: false`, which suppresses ONLY the `attachment:detected` broadcast — the guard, the registry entry and the by-id routes are identical either way. Without it every click also popped an attachment card announcing the file already filling the screen. ⚠️ The click is an explicit user action on the **explicit, Origin-guarded** registration route, which is why it may cross the workspace boundary at all; the passive magic-link scanner stays force-confined. A type outside `SUPPORTED_ATTACHMENT_EXTENSIONS` (`.svg`, `.bmp`, `.json`, `.log`, code files) is refused with a message naming what IS previewable, rather than the registry's own policy term.
**Media is single-sourced across the two preview paths.** `VIDEO_ATTACHMENT_EXTENSIONS` / `AUDIO_ATTACHMENT_EXTENSIONS` live in `attachment-registry.ts` and are imported by `file-content`'s media classification, so a clip plays identically whether it is in the workspace or reached by id from outside it. They diverged first: the workspace path had its own inline sets and the registry allowlist had no media at all, so a video an agent wrote to `/tmp` was refused as an unsupported type while the same file inside the repo played. ⚠️ Three things have to line up for a player rather than a dead frame: the extension in the allowlist, a **real MIME entry** in `MIME_TYPES` (a `<video>` refuses to decode `application/octet-stream`, which presents as a player that renders and then does nothing), and the range-aware body (`serveRawFile` → `sendFileBody`) that makes the scrub bar work. `getAttachmentType()` returns the `video`/`audio` members of `AttachmentDetectedType` for them; the attachment card has no per-type CSS and its thumbnail falls back to the type label, since `generateFirstPageThumbnail` has no media branch and answers 204. ⚠️ The image-watcher keeps its OWN narrow detection list (`png/pdf/docx/pptx`), so this does not start popping cards for every video an agent writes.
⚠️ **The preview overlay must outrank the panel that launched it.** `.file-preview-overlay` sits at `z-index: 5100`, above the response viewer (5000) and its backdrop (4999); at its historical 2000 a path clicked in the chat opened the overlay *behind* the chat, which reads as a dead link. It stays below the toast/picker band (10000+) so a "Saved" toast still lands on top.
+24
View File
@@ -14,6 +14,26 @@ import { isBlockedAttachmentPath, loadAttachmentGuardConfig } from './config/att
import { validateSessionFilePath } from './web/route-helpers.js';
import type { AttachmentDetectedEvent, AttachmentDetectedType } from './types.js';
/**
* Playable media extensions, single-sourced here because the WORKSPACE preview
* (`file-content`'s media classification) and the out-of-workspace attachment
* path must agree on what plays. They diverged once: a video an agent wrote
* inside the workspace played with a working scrub bar, while the same file in
* `/tmp` was refused as an unsupported type, which reads as a bug rather than a
* boundary. Serving is range-aware in both, which is what makes seeking work.
*/
export const VIDEO_ATTACHMENT_EXTENSIONS: ReadonlySet<string> = new Set(['mp4', 'webm', 'mov', 'm4v', 'ogv']);
export const AUDIO_ATTACHMENT_EXTENSIONS: ReadonlySet<string> = new Set([
'mp3',
'wav',
'ogg',
'oga',
'm4a',
'aac',
'flac',
'opus',
]);
const SUPPORTED_ATTACHMENT_EXTENSIONS = new Set([
'png',
'jpg',
@@ -25,6 +45,8 @@ const SUPPORTED_ATTACHMENT_EXTENSIONS = new Set([
'pptx',
'md',
'txt',
...VIDEO_ATTACHMENT_EXTENSIONS,
...AUDIO_ATTACHMENT_EXTENSIONS,
]);
export type AttachmentSource = 'detected' | 'external';
@@ -108,6 +130,8 @@ export function isSupportedAttachmentExtension(extension: string): boolean {
export function getAttachmentType(extension: string): AttachmentDetectedType {
const normalized = extension.toLowerCase().replace(/^\./, '');
if (['png', 'jpg', 'jpeg', 'gif', 'webp'].includes(normalized)) return 'image';
if (VIDEO_ATTACHMENT_EXTENSIONS.has(normalized)) return 'video';
if (AUDIO_ATTACHMENT_EXTENSIONS.has(normalized)) return 'audio';
if (normalized === 'pdf') return 'pdf';
if (normalized === 'pptx') return 'presentation';
if (normalized === 'md') return 'markdown';
+9 -1
View File
@@ -63,7 +63,15 @@ export interface ImageDetectedEvent {
size: number;
}
export type AttachmentDetectedType = 'image' | 'pdf' | 'document' | 'presentation' | 'markdown' | 'text';
export type AttachmentDetectedType =
| 'image'
| 'video'
| 'audio'
| 'pdf'
| 'document'
| 'presentation'
| 'markdown'
| 'text';
/**
* Event emitted when a new previewable attachment file is detected in a session's
+11 -1
View File
@@ -3284,7 +3284,7 @@ Object.assign(CodemanApp.prototype, {
if (/unsupported/i.test(reason)) {
const ext = (filePath.split('.').pop() || '').toLowerCase();
return {
error: `Cannot preview .${ext} from outside the session workspace (images, PDF, Office documents, Markdown and text only).`,
error: `Cannot preview .${ext} from outside the session workspace (images, video, audio, PDF, Office documents, Markdown and text only).`,
};
}
return { error: reason };
@@ -3341,12 +3341,22 @@ Object.assign(CodemanApp.prototype, {
if (attachmentId) {
const base = `/api/sessions/${sessionId}/attachments/${encodeURIComponent(attachmentId)}`;
const IMAGE_EXTS = new Set(['png', 'jpg', 'jpeg', 'gif', 'webp', 'bmp', 'svg']);
const VIDEO_EXTS = new Set(['mp4', 'webm', 'mov', 'm4v', 'ogv']);
const AUDIO_EXTS = new Set(['mp3', 'wav', 'ogg', 'oga', 'm4a', 'aac', 'flac', 'opus']);
// Size when we just registered the file ourselves, so a path opened from a
// link reads like a workspace preview instead of a bare "PNG". History
// cards arrive with an id and no size and keep the short form.
footerEl.textContent = externalSize ? `${this.formatFileSize(externalSize)} • ${ext}` : ext.toUpperCase();
if (IMAGE_EXTS.has(ext)) {
bodyEl.innerHTML = `<img src="${escapeHtml(`${base}/raw`)}" alt="${escapeHtml(filePath)}">`;
} else if (VIDEO_EXTS.has(ext)) {
// Same markup as the workspace branch below, including playsinline: iOS
// otherwise hijacks playback into its own fullscreen player, which
// leaves this overlay behind it with no way back but its close button.
// The attachment raw route is range-aware, so the scrub bar works.
bodyEl.innerHTML = `<video src="${escapeHtml(`${base}/raw`)}" controls autoplay playsinline preload="metadata"></video>`;
} else if (AUDIO_EXTS.has(ext)) {
bodyEl.innerHTML = `<audio src="${escapeHtml(`${base}/raw`)}" controls autoplay preload="metadata"></audio>`;
} else if (ext === 'pdf') {
bodyEl.innerHTML = `<iframe src="${escapeHtml(`${base}/raw`)}" title="${escapeHtml(filePath)}"></iframe>`;
} else if (ext === 'docx' || ext === 'pptx') {
+22 -2
View File
@@ -23,12 +23,14 @@ import type {
import { ApiErrorCode, createErrorResponse, getErrorMessage } from '../../types.js';
import { fileStreamManager } from '../../file-stream-manager.js';
import {
AUDIO_ATTACHMENT_EXTENSIONS,
AttachmentRegistrationError,
attachmentRecordToEvent,
attachmentRegistry,
buildFileThumbnailRoute,
isSupportedAttachmentExtension,
registerExternalAttachment,
VIDEO_ATTACHMENT_EXTENSIONS,
type AttachmentRecord,
} from '../../attachment-registry.js';
import { generateFirstPageThumbnail } from '../../document-thumbnailer.js';
@@ -67,6 +69,22 @@ const MIME_TYPES: Record<string, string> = {
webp: 'image/webp',
ico: 'image/x-icon',
bmp: 'image/bmp',
// Media needs a real type, not the octet-stream fallback: a <video>/<audio>
// element refuses to decode an unknown type, so a missing entry here presents
// as a player that renders and then does nothing.
mp4: 'video/mp4',
webm: 'video/webm',
mov: 'video/quicktime',
m4v: 'video/x-m4v',
ogv: 'video/ogg',
mp3: 'audio/mpeg',
wav: 'audio/wav',
ogg: 'audio/ogg',
oga: 'audio/ogg',
m4a: 'audio/mp4',
aac: 'audio/aac',
flac: 'audio/flac',
opus: 'audio/opus',
pdf: 'application/pdf',
docx: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
pptx: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
@@ -1099,8 +1117,10 @@ export function registerFileRoutes(app: FastifyInstance, ctx: SessionPort & Even
// so the file viewer can open the same files.
const ext = filePath.split('.').pop()?.toLowerCase() || '';
const imageExts = new Set(['png', 'jpg', 'jpeg', 'gif', 'webp', 'svg', 'bmp', 'ico']);
const videoExts = new Set(['mp4', 'webm', 'mov', 'm4v', 'ogv']);
const audioExts = new Set(['mp3', 'wav', 'ogg', 'oga', 'm4a', 'aac', 'flac', 'opus']);
// Shared with the attachment registry so a video plays the same whether it
// sits in the workspace or is reached by id from outside it.
const videoExts = VIDEO_ATTACHMENT_EXTENSIONS;
const audioExts = AUDIO_ATTACHMENT_EXTENSIONS;
const otherBinaryExts = new Set([
'pdf',
'zip',
@@ -357,6 +357,87 @@ describe('file-routes attachment path guard (COD-53)', () => {
});
});
// ===== Media (click-to-preview parity with the workspace preview) =====
// A video an agent writes inside the workspace plays with a working scrub
// bar; the same file in /tmp used to be refused as an unsupported type. Both
// now go through the same extension sets, and the raw route has to answer
// with a real media Content-Type and a range, or the player renders and then
// does nothing.
describe('media attachments', () => {
it('registers a video and serves it as seekable video/mp4', async () => {
const content = Buffer.from('MP4DATA-0123456789');
mockedStat.mockResolvedValue({ size: content.length, isFile: () => true, mtimeMs: 5 } as never);
mockedCreateReadStream.mockReturnValue(Readable.from([content.subarray(4, 10)]) as never);
const res = await harness.app.inject({
method: 'POST',
url: `/api/sessions/${harness.ctx._sessionId}/attachments`,
payload: { path: '/tmp/captures/demo.mp4', notify: false },
});
expect(res.statusCode).toBe(200);
const body = JSON.parse(res.body);
expect(body.data.attachmentType).toBe('video');
const rawRes = await harness.app.inject({
method: 'GET',
url: `/api/sessions/${harness.ctx._sessionId}/attachments/${body.data.attachmentId}/raw`,
headers: { range: 'bytes=4-9' },
});
expect(rawRes.statusCode).toBe(206);
expect(rawRes.headers['content-type']).toBe('video/mp4');
expect(rawRes.headers['content-range']).toBe(`bytes 4-9/${content.length}`);
expect(rawRes.headers['accept-ranges']).toBe('bytes');
});
it('registers audio with an audio type and its real MIME', async () => {
const content = Buffer.from('ID3AUDIO');
mockedStat.mockResolvedValue({ size: content.length, isFile: () => true, mtimeMs: 5 } as never);
mockedCreateReadStream.mockReturnValue(Readable.from([content]) as never);
const res = await harness.app.inject({
method: 'POST',
url: `/api/sessions/${harness.ctx._sessionId}/attachments`,
payload: { path: '/tmp/captures/take.mp3', notify: false },
});
expect(res.statusCode).toBe(200);
const body = JSON.parse(res.body);
expect(body.data.attachmentType).toBe('audio');
const rawRes = await harness.app.inject({
method: 'GET',
url: `/api/sessions/${harness.ctx._sessionId}/attachments/${body.data.attachmentId}/raw`,
});
expect(rawRes.statusCode).toBe(200);
expect(rawRes.headers['content-type']).toBe('audio/mpeg');
});
it('answers no thumbnail for media instead of spawning a converter', async () => {
// generateFirstPageThumbnail has no media branch; the card falls back to
// its type label. This pins that the route reports that cleanly.
const res = await harness.app.inject({
method: 'POST',
url: `/api/sessions/${harness.ctx._sessionId}/attachments`,
payload: { path: '/tmp/captures/clip.webm', notify: false },
});
const { attachmentId } = JSON.parse(res.body).data;
const thumbRes = await harness.app.inject({
method: 'GET',
url: `/api/sessions/${harness.ctx._sessionId}/attachments/${attachmentId}/thumbnail`,
});
expect(thumbRes.statusCode).toBe(204);
});
it('still refuses media in a blocked tree', async () => {
const res = await harness.app.inject({
method: 'POST',
url: `/api/sessions/${harness.ctx._sessionId}/attachments`,
payload: { path: '/root/private/recording.mp4', notify: false },
});
expect(res.statusCode).toBe(403);
});
});
// ===== Quiet registration (click-to-preview) =====
// The file-preview overlay registers a clicked out-of-workspace path to mint
// an id it can render by. It is already putting the file on screen, so the