BitSquare/Codeman
2
0
Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-09-30 12:39:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

chore: bump version to 0.1652

Browse Source
This commit is contained in:
arkon
2026-02-27 00:34:38 +01:00
parent c50ccbf8a8
commit 5f1f66bf60
68 changed files with 9990 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/remotion-best-practices/SKILL.md
+61
View File
@@ -0,0 +1,61 @@
---
name: remotion-best-practices
description: Best practices for Remotion - Video creation in React
metadata:
tags: remotion, video, react, animation, composition
---
## When to use
Use this skills whenever you are dealing with Remotion code to obtain the domain-specific knowledge.
## Captions
When dealing with captions or subtitles, load the [./rules/subtitles.md](./rules/subtitles.md) file for more information.
## Using FFmpeg
For some video operations, such as trimming videos or detecting silence, FFmpeg should be used. Load the [./rules/ffmpeg.md](./rules/ffmpeg.md) file for more information.
## Audio visualization
When needing to visualize audio (spectrum bars, waveforms, bass-reactive effects), load the [./rules/audio-visualization.md](./rules/audio-visualization.md) file for more information.
## Sound effects
When needing to use sound effects, load the [./rules/sound-effects.md](./rules/sound-effects.md) file for more information.
## How to use
Read individual rule files for detailed explanations and code examples:
- [rules/3d.md](rules/3d.md) - 3D content in Remotion using Three.js and React Three Fiber
- [rules/animations.md](rules/animations.md) - Fundamental animation skills for Remotion
- [rules/assets.md](rules/assets.md) - Importing images, videos, audio, and fonts into Remotion
- [rules/audio.md](rules/audio.md) - Using audio and sound in Remotion - importing, trimming, volume, speed, pitch
- [rules/calculate-metadata.md](rules/calculate-metadata.md) - Dynamically set composition duration, dimensions, and props
- [rules/can-decode.md](rules/can-decode.md) - Check if a video can be decoded by the browser using Mediabunny
- [rules/charts.md](rules/charts.md) - Chart and data visualization patterns for Remotion (bar, pie, line, stock charts)
- [rules/compositions.md](rules/compositions.md) - Defining compositions, stills, folders, default props and dynamic metadata
- [rules/extract-frames.md](rules/extract-frames.md) - Extract frames from videos at specific timestamps using Mediabunny
- [rules/fonts.md](rules/fonts.md) - Loading Google Fonts and local fonts in Remotion
- [rules/get-audio-duration.md](rules/get-audio-duration.md) - Getting the duration of an audio file in seconds with Mediabunny
- [rules/get-video-dimensions.md](rules/get-video-dimensions.md) - Getting the width and height of a video file with Mediabunny
- [rules/get-video-duration.md](rules/get-video-duration.md) - Getting the duration of a video file in seconds with Mediabunny
- [rules/gifs.md](rules/gifs.md) - Displaying GIFs synchronized with Remotion's timeline
- [rules/images.md](rules/images.md) - Embedding images in Remotion using the Img component
- [rules/light-leaks.md](rules/light-leaks.md) - Light leak overlay effects using @remotion/light-leaks
- [rules/lottie.md](rules/lottie.md) - Embedding Lottie animations in Remotion
- [rules/measuring-dom-nodes.md](rules/measuring-dom-nodes.md) - Measuring DOM element dimensions in Remotion
- [rules/measuring-text.md](rules/measuring-text.md) - Measuring text dimensions, fitting text to containers, and checking overflow
- [rules/sequencing.md](rules/sequencing.md) - Sequencing patterns for Remotion - delay, trim, limit duration of items
- [rules/tailwind.md](rules/tailwind.md) - Using TailwindCSS in Remotion
- [rules/text-animations.md](rules/text-animations.md) - Typography and text animation patterns for Remotion
- [rules/timing.md](rules/timing.md) - Interpolation curves in Remotion - linear, easing, spring animations
- [rules/transitions.md](rules/transitions.md) - Scene transition patterns for Remotion
- [rules/transparent-videos.md](rules/transparent-videos.md) - Rendering out a video with transparency
- [rules/trimming.md](rules/trimming.md) - Trimming patterns for Remotion - cut the beginning or end of animations
- [rules/videos.md](rules/videos.md) - Embedding videos in Remotion - trimming, volume, speed, looping, pitch
- [rules/parameters.md](rules/parameters.md) - Make a video parametrizable by adding a Zod schema
- [rules/maps.md](rules/maps.md) - Add a map using Mapbox and animate it
- [rules/voiceover.md](rules/voiceover.md) - Adding AI-generated voiceover to Remotion compositions using ElevenLabs TTS
.agents/skills/remotion-best-practices/rules/3d.md
+86
View File
@@ -0,0 +1,86 @@
---
name: 3d
description: 3D content in Remotion using Three.js and React Three Fiber.
metadata:
tags: 3d, three, threejs
---
# Using Three.js and React Three Fiber in Remotion
Follow React Three Fiber and Three.js best practices.
Only the following Remotion-specific rules need to be followed:
## Prerequisites
First, the `@remotion/three` package needs to be installed.
If it is not, use the following command:
```bash
npx remotion add @remotion/three # If project uses npm
bunx remotion add @remotion/three # If project uses bun
yarn remotion add @remotion/three # If project uses yarn
pnpm exec remotion add @remotion/three # If project uses pnpm
```
## Using ThreeCanvas
You MUST wrap 3D content in `<ThreeCanvas>` and include proper lighting.
`<ThreeCanvas>` MUST have a `width` and `height` prop.
```tsx
import { ThreeCanvas } from "@remotion/three";
import { useVideoConfig } from "remotion";
const { width, height } = useVideoConfig();
<ThreeCanvas width={width} height={height}>
<ambientLight intensity={0.4} />
<directionalLight position={[5, 5, 5]} intensity={0.8} />
<mesh>
<sphereGeometry args={[1, 32, 32]} />
<meshStandardMaterial color="red" />
</mesh>
</ThreeCanvas>;
```
## No animations not driven by `useCurrentFrame()`
Shaders, models etc MUST NOT animate by themselves.
No animations are allowed unless they are driven by `useCurrentFrame()`.
Otherwise, it will cause flickering during rendering.
Using `useFrame()` from `@react-three/fiber` is forbidden.
## Animate using `useCurrentFrame()`
Use `useCurrentFrame()` to perform animations.
```tsx
const frame = useCurrentFrame();
const rotationY = frame * 0.02;
<mesh rotation={[0, rotationY, 0]}>
<boxGeometry args={[2, 2, 2]} />
<meshStandardMaterial color="#4a9eff" />
</mesh>;
```
## Using `<Sequence>` inside `<ThreeCanvas>`
The `layout` prop of any `<Sequence>` inside a `<ThreeCanvas>` must be set to `none`.
```tsx
import { Sequence } from "remotion";
import { ThreeCanvas } from "@remotion/three";
const { width, height } = useVideoConfig();
<ThreeCanvas width={width} height={height}>
<Sequence layout="none">
<mesh>
<boxGeometry args={[2, 2, 2]} />
<meshStandardMaterial color="#4a9eff" />
</mesh>
</Sequence>
</ThreeCanvas>;
```
.agents/skills/remotion-best-practices/rules/animations.md
+27
View File
@@ -0,0 +1,27 @@
---
name: animations
description: Fundamental animation skills for Remotion
metadata:
tags: animations, transitions, frames, useCurrentFrame
---
All animations MUST be driven by the `useCurrentFrame()` hook.
Write animations in seconds and multiply them by the `fps` value from `useVideoConfig()`.
```tsx
import { useCurrentFrame } from "remotion";
export const FadeIn = () => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const opacity = interpolate(frame, [0, 2 * fps], [0, 1], {
extrapolateRight: "clamp",
});
return <div style={{ opacity }}>Hello World!</div>;
};
```
CSS transitions or animations are FORBIDDEN - they will not render correctly.
Tailwind animation class names are FORBIDDEN - they will not render correctly.
.agents/skills/remotion-best-practices/rules/assets.md
+78
View File
@@ -0,0 +1,78 @@
---
name: assets
description: Importing images, videos, audio, and fonts into Remotion
metadata:
tags: assets, staticFile, images, fonts, public
---
# Importing assets in Remotion
## The public folder
Place assets in the `public/` folder at your project root.
## Using staticFile()
You MUST use `staticFile()` to reference files from the `public/` folder:
```tsx
import { Img, staticFile } from "remotion";
export const MyComposition = () => {
return <Img src={staticFile("logo.png")} />;
};
```
The function returns an encoded URL that works correctly when deploying to subdirectories.
## Using with components
**Images:**
```tsx
import { Img, staticFile } from "remotion";
<Img src={staticFile("photo.png")} />;
```
**Videos:**
```tsx
import { Video } from "@remotion/media";
import { staticFile } from "remotion";
<Video src={staticFile("clip.mp4")} />;
```
**Audio:**
```tsx
import { Audio } from "@remotion/media";
import { staticFile } from "remotion";
<Audio src={staticFile("music.mp3")} />;
```
**Fonts:**
```tsx
import { staticFile } from "remotion";
const fontFamily = new FontFace("MyFont", `url(${staticFile("font.woff2")})`);
await fontFamily.load();
document.fonts.add(fontFamily);
```
## Remote URLs
Remote URLs can be used directly without `staticFile()`:
```tsx
<Img src="https://example.com/image.png" />
<Video src="https://remotion.media/video.mp4" />
```
## Important notes
- Remotion components (`<Img>`, `<Video>`, `<Audio>`) ensure assets are fully loaded before rendering
- Special characters in filenames (`#`, `?`, `&`) are automatically encoded
.agents/skills/remotion-best-practices/rules/assets/charts-bar-chart.tsx
+173
View File
@@ -0,0 +1,173 @@
import {loadFont} from '@remotion/google-fonts/Inter';
import {AbsoluteFill, spring, useCurrentFrame, useVideoConfig} from 'remotion';
const {fontFamily} = loadFont();
const COLOR_BAR = '#D4AF37';
const COLOR_TEXT = '#ffffff';
const COLOR_MUTED = '#888888';
const COLOR_BG = '#0a0a0a';
const COLOR_AXIS = '#333333';
// Ideal composition size: 1280x720
const Title: React.FC<{children: React.ReactNode}> = ({children}) => (
<div style={{textAlign: 'center', marginBottom: 40}}>
<div style={{color: COLOR_TEXT, fontSize: 48, fontWeight: 600}}>
{children}
</div>
</div>
);
const YAxis: React.FC<{steps: number[]; height: number}> = ({
steps,
height,
}) => (
<div
style={{
display: 'flex',
flexDirection: 'column',
justifyContent: 'space-between',
height,
paddingRight: 16,
}}
>
{steps
.slice()
.reverse()
.map((step) => (
<div
key={step}
style={{
color: COLOR_MUTED,
fontSize: 20,
textAlign: 'right',
}}
>
{step.toLocaleString()}
</div>
))}
</div>
);
const Bar: React.FC<{
height: number;
progress: number;
}> = ({height, progress}) => (
<div
style={{
flex: 1,
display: 'flex',
flexDirection: 'column',
justifyContent: 'flex-end',
}}
>
<div
style={{
width: '100%',
height,
backgroundColor: COLOR_BAR,
borderRadius: '8px 8px 0 0',
opacity: progress,
}}
/>
</div>
);
const XAxis: React.FC<{
children: React.ReactNode;
labels: string[];
height: number;
}> = ({children, labels, height}) => (
<div style={{flex: 1, display: 'flex', flexDirection: 'column'}}>
<div
style={{
display: 'flex',
alignItems: 'flex-end',
gap: 16,
height,
borderLeft: `2px solid ${COLOR_AXIS}`,
borderBottom: `2px solid ${COLOR_AXIS}`,
paddingLeft: 16,
}}
>
{children}
</div>
<div
style={{
display: 'flex',
gap: 16,
paddingLeft: 16,
marginTop: 12,
}}
>
{labels.map((label) => (
<div
key={label}
style={{
flex: 1,
textAlign: 'center',
color: COLOR_MUTED,
fontSize: 20,
}}
>
{label}
</div>
))}
</div>
</div>
);
export const MyAnimation = () => {
const frame = useCurrentFrame();
const {fps, height} = useVideoConfig();
const data = [
{month: 'Jan', price: 2039},
{month: 'Mar', price: 2160},
{month: 'May', price: 2327},
{month: 'Jul', price: 2426},
{month: 'Sep', price: 2634},
{month: 'Nov', price: 2672},
];
const minPrice = 2000;
const maxPrice = 2800;
const priceRange = maxPrice - minPrice;
const chartHeight = height - 280;
const yAxisSteps = [2000, 2400, 2800];
return (
<AbsoluteFill
style={{
backgroundColor: COLOR_BG,
padding: 60,
display: 'flex',
flexDirection: 'column',
fontFamily,
}}
>
<Title>Gold Price 2024</Title>
<div style={{display: 'flex', flex: 1}}>
<YAxis steps={yAxisSteps} height={chartHeight} />
<XAxis height={chartHeight} labels={data.map((d) => d.month)}>
{data.map((item, i) => {
const progress = spring({
frame: frame - i * 5 - 10,
fps,
config: {damping: 18, stiffness: 80},
});
const barHeight =
((item.price - minPrice) / priceRange) * chartHeight * progress;
return (
<Bar key={item.month} height={barHeight} progress={progress} />
);
})}
</XAxis>
</div>
</AbsoluteFill>
);
};
.agents/skills/remotion-best-practices/rules/assets/text-animations-typewriter.tsx
+100
View File
@@ -0,0 +1,100 @@
import {
AbsoluteFill,
interpolate,
useCurrentFrame,
useVideoConfig,
} from 'remotion';
const COLOR_BG = '#ffffff';
const COLOR_TEXT = '#000000';
const FULL_TEXT = 'From prompt to motion graphics. This is Remotion.';
const PAUSE_AFTER = 'From prompt to motion graphics.';
const FONT_SIZE = 72;
const FONT_WEIGHT = 700;
const CHAR_FRAMES = 2;
const CURSOR_BLINK_FRAMES = 16;
const PAUSE_SECONDS = 1;
// Ideal composition size: 1280x720
const getTypedText = ({
frame,
fullText,
pauseAfter,
charFrames,
pauseFrames,
}: {
frame: number;
fullText: string;
pauseAfter: string;
charFrames: number;
pauseFrames: number;
}): string => {
const pauseIndex = fullText.indexOf(pauseAfter);
const preLen =
pauseIndex >= 0 ? pauseIndex + pauseAfter.length : fullText.length;
let typedChars = 0;
if (frame < preLen * charFrames) {
typedChars = Math.floor(frame / charFrames);
} else if (frame < preLen * charFrames + pauseFrames) {
typedChars = preLen;
} else {
const postPhase = frame - preLen * charFrames - pauseFrames;
typedChars = Math.min(
fullText.length,
preLen + Math.floor(postPhase / charFrames),
);
}
return fullText.slice(0, typedChars);
};
const Cursor: React.FC<{
frame: number;
blinkFrames: number;
symbol?: string;
}> = ({frame, blinkFrames, symbol = '\u258C'}) => {
const opacity = interpolate(
frame % blinkFrames,
[0, blinkFrames / 2, blinkFrames],
[1, 0, 1],
{extrapolateLeft: 'clamp', extrapolateRight: 'clamp'},
);
return <span style={{opacity}}>{symbol}</span>;
};
export const MyAnimation = () => {
const frame = useCurrentFrame();
const {fps} = useVideoConfig();
const pauseFrames = Math.round(fps * PAUSE_SECONDS);
const typedText = getTypedText({
frame,
fullText: FULL_TEXT,
pauseAfter: PAUSE_AFTER,
charFrames: CHAR_FRAMES,
pauseFrames,
});
return (
<AbsoluteFill
style={{
backgroundColor: COLOR_BG,
}}
>
<div
style={{
color: COLOR_TEXT,
fontSize: FONT_SIZE,
fontWeight: FONT_WEIGHT,
fontFamily: 'sans-serif',
}}
>
<span>{typedText}</span>
<Cursor frame={frame} blinkFrames={CURSOR_BLINK_FRAMES} />
</div>
</AbsoluteFill>
);
};
.agents/skills/remotion-best-practices/rules/assets/text-animations-word-highlight.tsx
+103
View File
@@ -0,0 +1,103 @@
import {loadFont} from '@remotion/google-fonts/Inter';
import React from 'react';
import {AbsoluteFill, spring, useCurrentFrame, useVideoConfig} from 'remotion';
/*
* Highlight a word in a sentence with a spring-animated wipe effect.
*/
// Ideal composition size: 1280x720
const COLOR_BG = '#ffffff';
const COLOR_TEXT = '#000000';
const COLOR_HIGHLIGHT = '#A7C7E7';
const FULL_TEXT = 'This is Remotion.';
const HIGHLIGHT_WORD = 'Remotion';
const FONT_SIZE = 72;
const FONT_WEIGHT = 700;
const HIGHLIGHT_START_FRAME = 30;
const HIGHLIGHT_WIPE_DURATION = 18;
const {fontFamily} = loadFont();
const Highlight: React.FC<{
word: string;
color: string;
delay: number;
durationInFrames: number;
}> = ({word, color, delay, durationInFrames}) => {
const frame = useCurrentFrame();
const {fps} = useVideoConfig();
const highlightProgress = spring({
fps,
frame,
config: {damping: 200},
delay,
durationInFrames,
});
const scaleX = Math.max(0, Math.min(1, highlightProgress));
return (
<span style={{position: 'relative', display: 'inline-block'}}>
<span
style={{
position: 'absolute',
left: 0,
right: 0,
top: '50%',
height: '1.05em',
transform: `translateY(-50%) scaleX(${scaleX})`,
transformOrigin: 'left center',
backgroundColor: color,
borderRadius: '0.18em',
zIndex: 0,
}}
/>
<span style={{position: 'relative', zIndex: 1}}>{word}</span>
</span>
);
};
export const MyAnimation = () => {
const highlightIndex = FULL_TEXT.indexOf(HIGHLIGHT_WORD);
const hasHighlight = highlightIndex >= 0;
const preText = hasHighlight ? FULL_TEXT.slice(0, highlightIndex) : FULL_TEXT;
const postText = hasHighlight
? FULL_TEXT.slice(highlightIndex + HIGHLIGHT_WORD.length)
: '';
return (
<AbsoluteFill
style={{
backgroundColor: COLOR_BG,
alignItems: 'center',
justifyContent: 'center',
fontFamily,
}}
>
<div
style={{
color: COLOR_TEXT,
fontSize: FONT_SIZE,
fontWeight: FONT_WEIGHT,
}}
>
{hasHighlight ? (
<>
<span>{preText}</span>
<Highlight
word={HIGHLIGHT_WORD}
color={COLOR_HIGHLIGHT}
delay={HIGHLIGHT_START_FRAME}
durationInFrames={HIGHLIGHT_WIPE_DURATION}
/>
<span>{postText}</span>
</>
) : (
<span>{FULL_TEXT}</span>
)}
</div>
</AbsoluteFill>
);
};
.agents/skills/remotion-best-practices/rules/audio-visualization.md
+198
View File
@@ -0,0 +1,198 @@
---
name: audio-visualization
description: Audio visualization patterns - spectrum bars, waveforms, bass-reactive effects
metadata:
tags: audio, visualization, spectrum, waveform, bass, music, audiogram, frequency
---
# Audio Visualization in Remotion
## Prerequisites
```bash
npx remotion add @remotion/media-utils
```
## Loading Audio Data
Use `useWindowedAudioData()` (https://www.remotion.dev/docs/use-windowed-audio-data) to load audio data:
```tsx
import { useWindowedAudioData } from "@remotion/media-utils";
import { staticFile, useCurrentFrame, useVideoConfig } from "remotion";
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const { audioData, dataOffsetInSeconds } = useWindowedAudioData({
src: staticFile("podcast.wav"),
frame,
fps,
windowInSeconds: 30,
});
```
## Spectrum Bar Visualization
Use `visualizeAudio()` (https://www.remotion.dev/docs/visualize-audio) to get frequency data for bar charts:
```tsx
import { useWindowedAudioData, visualizeAudio } from "@remotion/media-utils";
import { staticFile, useCurrentFrame, useVideoConfig } from "remotion";
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const { audioData, dataOffsetInSeconds } = useWindowedAudioData({
src: staticFile("music.mp3"),
frame,
fps,
windowInSeconds: 30,
});
if (!audioData) {
return null;
}
const frequencies = visualizeAudio({
fps,
frame,
audioData,
numberOfSamples: 256,
optimizeFor: "speed",
dataOffsetInSeconds,
});
return (
<div style={{ display: "flex", alignItems: "flex-end", height: 200 }}>
{frequencies.map((v, i) => (
<div
key={i}
style={{
flex: 1,
height: `${v * 100}%`,
backgroundColor: "#0b84f3",
margin: "0 1px",
}}
/>
))}
</div>
);
```
- `numberOfSamples` must be power of 2 (32, 64, 128, 256, 512, 1024)
- Values range 0-1; left of array = bass, right = highs
- Use `optimizeFor: "speed"` for Lambda or high sample counts
**Important:** When passing `audioData` to child components, also pass the `frame` from the parent. Do not call `useCurrentFrame()` in each child - this causes discontinuous visualization when children are inside `<Sequence>` with offsets.
## Waveform Visualization
Use `visualizeAudioWaveform()` (https://www.remotion.dev/docs/media-utils/visualize-audio-waveform) with `createSmoothSvgPath()` (https://www.remotion.dev/docs/media-utils/create-smooth-svg-path) for oscilloscope-style displays:
```tsx
import {
createSmoothSvgPath,
useWindowedAudioData,
visualizeAudioWaveform,
} from "@remotion/media-utils";
import { staticFile, useCurrentFrame, useVideoConfig } from "remotion";
const frame = useCurrentFrame();
const { width, fps } = useVideoConfig();
const HEIGHT = 200;
const { audioData, dataOffsetInSeconds } = useWindowedAudioData({
src: staticFile("voice.wav"),
frame,
fps,
windowInSeconds: 30,
});
if (!audioData) {
return null;
}
const waveform = visualizeAudioWaveform({
fps,
frame,
audioData,
numberOfSamples: 256,
windowInSeconds: 0.5,
dataOffsetInSeconds,
});
const path = createSmoothSvgPath({
points: waveform.map((y, i) => ({
x: (i / (waveform.length - 1)) * width,
y: HEIGHT / 2 + (y * HEIGHT) / 2,
})),
});
return (
<svg width={width} height={HEIGHT}>
<path d={path} fill="none" stroke="#0b84f3" strokeWidth={2} />
</svg>
);
```
## Bass-Reactive Effects
Extract low frequencies for beat-reactive animations:
```tsx
const frequencies = visualizeAudio({
fps,
frame,
audioData,
numberOfSamples: 128,
optimizeFor: "speed",
dataOffsetInSeconds,
});
const lowFrequencies = frequencies.slice(0, 32);
const bassIntensity =
lowFrequencies.reduce((sum, v) => sum + v, 0) / lowFrequencies.length;
const scale = 1 + bassIntensity * 0.5;
const opacity = Math.min(0.6, bassIntensity * 0.8);
```
## Volume-Based Waveform
Use `getWaveformPortion()` (https://www.remotion.dev/docs/get-waveform-portion) when you need simplified volume data instead of frequency spectrum:
```tsx
import { getWaveformPortion } from "@remotion/media-utils";
import { useCurrentFrame, useVideoConfig } from "remotion";
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentTimeInSeconds = frame / fps;
const waveform = getWaveformPortion({
audioData,
startTimeInSeconds: currentTimeInSeconds,
durationInSeconds: 5,
numberOfSamples: 50,
});
// Returns array of { index, amplitude } objects (amplitude: 0-1)
waveform.map((bar) => (
<div key={bar.index} style={{ height: bar.amplitude * 100 }} />
));
```
## Postprocessing
Low frequencies naturally dominate. Apply logarithmic scaling for visual balance:
```tsx
const minDb = -100;
const maxDb = -30;
const scaled = frequencies.map((value) => {
const db = 20 * Math.log10(value);
return (db - minDb) / (maxDb - minDb);
});
```
.agents/skills/remotion-best-practices/rules/audio.md
+169
View File
@@ -0,0 +1,169 @@
---
name: audio
description: Using audio and sound in Remotion - importing, trimming, volume, speed, pitch
metadata:
tags: audio, media, trim, volume, speed, loop, pitch, mute, sound, sfx
---
# Using audio in Remotion
## Prerequisites
First, the @remotion/media package needs to be installed.
If it is not installed, use the following command:
```bash
npx remotion add @remotion/media
```
## Importing Audio
Use `<Audio>` from `@remotion/media` to add audio to your composition.
```tsx
import { Audio } from "@remotion/media";
import { staticFile } from "remotion";
export const MyComposition = () => {
return <Audio src={staticFile("audio.mp3")} />;
};
```
Remote URLs are also supported:
```tsx
<Audio src="https://remotion.media/audio.mp3" />
```
By default, audio plays from the start, at full volume and full length.
Multiple audio tracks can be layered by adding multiple `<Audio>` components.
## Trimming
Use `trimBefore` and `trimAfter` to remove portions of the audio. Values are in frames.
```tsx
const { fps } = useVideoConfig();
return (
<Audio
src={staticFile("audio.mp3")}
trimBefore={2 * fps} // Skip the first 2 seconds
trimAfter={10 * fps} // End at the 10 second mark
/>
);
```
The audio still starts playing at the beginning of the composition - only the specified portion is played.
## Delaying
Wrap the audio in a `<Sequence>` to delay when it starts:
```tsx
import { Sequence, staticFile } from "remotion";
import { Audio } from "@remotion/media";
const { fps } = useVideoConfig();
return (
<Sequence from={1 * fps}>
<Audio src={staticFile("audio.mp3")} />
</Sequence>
);
```
The audio will start playing after 1 second.
## Volume
Set a static volume (0 to 1):
```tsx
<Audio src={staticFile("audio.mp3")} volume={0.5} />
```
Or use a callback for dynamic volume based on the current frame:
```tsx
import { interpolate } from "remotion";
const { fps } = useVideoConfig();
return (
<Audio
src={staticFile("audio.mp3")}
volume={(f) =>
interpolate(f, [0, 1 * fps], [0, 1], { extrapolateRight: "clamp" })
}
/>
);
```
The value of `f` starts at 0 when the audio begins to play, not the composition frame.
## Muting
Use `muted` to silence the audio. It can be set dynamically:
```tsx
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
return (
<Audio
src={staticFile("audio.mp3")}
muted={frame >= 2 * fps && frame <= 4 * fps} // Mute between 2s and 4s
/>
);
```
## Speed
Use `playbackRate` to change the playback speed:
```tsx
<Audio src={staticFile("audio.mp3")} playbackRate={2} /> {/* 2x speed */}
<Audio src={staticFile("audio.mp3")} playbackRate={0.5} /> {/* Half speed */}
```
Reverse playback is not supported.
## Looping
Use `loop` to loop the audio indefinitely:
```tsx
<Audio src={staticFile("audio.mp3")} loop />
```
Use `loopVolumeCurveBehavior` to control how the frame count behaves when looping:
- `"repeat"`: Frame count resets to 0 each loop (default)
- `"extend"`: Frame count continues incrementing
```tsx
<Audio
src={staticFile("audio.mp3")}
loop
loopVolumeCurveBehavior="extend"
volume={(f) => interpolate(f, [0, 300], [1, 0])} // Fade out over multiple loops
/>
```
## Pitch
Use `toneFrequency` to adjust the pitch without affecting speed. Values range from 0.01 to 2:
```tsx
<Audio
src={staticFile("audio.mp3")}
toneFrequency={1.5} // Higher pitch
/>
<Audio
src={staticFile("audio.mp3")}
toneFrequency={0.8} // Lower pitch
/>
```
Pitch shifting only works during server-side rendering, not in the Remotion Studio preview or in the `<Player />`.
.agents/skills/remotion-best-practices/rules/calculate-metadata.md
+134
View File
@@ -0,0 +1,134 @@
---
name: calculate-metadata
description: Dynamically set composition duration, dimensions, and props
metadata:
tags: calculateMetadata, duration, dimensions, props, dynamic
---
# Using calculateMetadata
Use `calculateMetadata` on a `<Composition>` to dynamically set duration, dimensions, and transform props before rendering.
```tsx
<Composition
id="MyComp"
component={MyComponent}
durationInFrames={300}
fps={30}
width={1920}
height={1080}
defaultProps={{ videoSrc: "https://remotion.media/video.mp4" }}
calculateMetadata={calculateMetadata}
/>
```
## Setting duration based on a video
Use the [`getVideoDuration`](./get-video-duration.md) and [`getVideoDimensions`](./get-video-dimensions.md) skills to get the video duration and dimensions:
```tsx
import { CalculateMetadataFunction } from "remotion";
import { getVideoDuration } from "./get-video-duration";
const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
}) => {
const durationInSeconds = await getVideoDuration(props.videoSrc);
return {
durationInFrames: Math.ceil(durationInSeconds * 30),
};
};
```
## Matching dimensions of a video
Use the [`getVideoDimensions`](./get-video-dimensions.md) skill to get the video dimensions:
```tsx
import { CalculateMetadataFunction } from "remotion";
import { getVideoDuration } from "./get-video-duration";
import { getVideoDimensions } from "./get-video-dimensions";
const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
}) => {
const dimensions = await getVideoDimensions(props.videoSrc);
return {
width: dimensions.width,
height: dimensions.height,
};
};
```
## Setting duration based on multiple videos
```tsx
const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
}) => {
const metadataPromises = props.videos.map((video) =>
getVideoDuration(video.src),
);
const allMetadata = await Promise.all(metadataPromises);
const totalDuration = allMetadata.reduce(
(sum, durationInSeconds) => sum + durationInSeconds,
0,
);
return {
durationInFrames: Math.ceil(totalDuration * 30),
};
};
```
## Setting a default outName
Set the default output filename based on props:
```tsx
const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
}) => {
return {
defaultOutName: `video-${props.id}.mp4`,
};
};
```
## Transforming props
Fetch data or transform props before rendering:
```tsx
const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
abortSignal,
}) => {
const response = await fetch(props.dataUrl, { signal: abortSignal });
const data = await response.json();
return {
props: {
...props,
fetchedData: data,
},
};
};
```
The `abortSignal` cancels stale requests when props change in the Studio.
## Return value
All fields are optional. Returned values override the `<Composition>` props:
- `durationInFrames`: Number of frames
- `width`: Composition width in pixels
- `height`: Composition height in pixels
- `fps`: Frames per second
- `props`: Transformed props passed to the component
- `defaultOutName`: Default output filename
- `defaultCodec`: Default codec for rendering
.agents/skills/remotion-best-practices/rules/can-decode.md
+75
View File
@@ -0,0 +1,75 @@
---
name: can-decode
description: Check if a video can be decoded by the browser using Mediabunny
metadata:
tags: decode, validation, video, audio, compatibility, browser
---
# Checking if a video can be decoded
Use Mediabunny to check if a video can be decoded by the browser before attempting to play it.
## The `canDecode()` function
This function can be copy-pasted into any project.
```tsx
import { Input, ALL_FORMATS, UrlSource } from "mediabunny";
export const canDecode = async (src: string) => {
const input = new Input({
formats: ALL_FORMATS,
source: new UrlSource(src, {
getRetryDelay: () => null,
}),
});
try {
await input.getFormat();
} catch {
return false;
}
const videoTrack = await input.getPrimaryVideoTrack();
if (videoTrack && !(await videoTrack.canDecode())) {
return false;
}
const audioTrack = await input.getPrimaryAudioTrack();
if (audioTrack && !(await audioTrack.canDecode())) {
return false;
}
return true;
};
```
## Usage
```tsx
const src = "https://remotion.media/video.mp4";
const isDecodable = await canDecode(src);
if (isDecodable) {
console.log("Video can be decoded");
} else {
console.log("Video cannot be decoded by this browser");
}
```
## Using with Blob
For file uploads or drag-and-drop, use `BlobSource`:
```tsx
import { Input, ALL_FORMATS, BlobSource } from "mediabunny";
export const canDecodeBlob = async (blob: Blob) => {
const input = new Input({
formats: ALL_FORMATS,
source: new BlobSource(blob),
});
// Same validation logic as above
};
```
.agents/skills/remotion-best-practices/rules/charts.md
+120
View File
@@ -0,0 +1,120 @@
---
name: charts
description: Chart and data visualization patterns for Remotion. Use when creating bar charts, pie charts, line charts, stock graphs, or any data-driven animations.
metadata:
tags: charts, data, visualization, bar-chart, pie-chart, line-chart, stock-chart, svg-paths, graphs
---
# Charts in Remotion
Create charts using React code - HTML, SVG, and D3.js are all supported.
Disable all animations from third party libraries - they cause flickering.
Drive all animations from `useCurrentFrame()`.
## Bar Chart
```tsx
const STAGGER_DELAY = 5;
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const bars = data.map((item, i) => {
const height = spring({
frame,
fps,
delay: i * STAGGER_DELAY,
config: { damping: 200 },
});
return <div style={{ height: height * item.value }} />;
});
```
## Pie Chart
Animate segments using stroke-dashoffset, starting from 12 o'clock:
```tsx
const progress = interpolate(frame, [0, 100], [0, 1]);
const circumference = 2 * Math.PI * radius;
const segmentLength = (value / total) * circumference;
const offset = interpolate(progress, [0, 1], [segmentLength, 0]);
<circle
r={radius}
cx={center}
cy={center}
fill="none"
stroke={color}
strokeWidth={strokeWidth}
strokeDasharray={`${segmentLength} ${circumference}`}
strokeDashoffset={offset}
transform={`rotate(-90 ${center} ${center})`}
/>;
```
## Line Chart / Path Animation
Use `@remotion/paths` for animating SVG paths (line charts, stock graphs, signatures).
Install: `npx remotion add @remotion/paths`
Docs: https://remotion.dev/docs/paths.md
### Convert data points to SVG path
```tsx
type Point = { x: number; y: number };
const generateLinePath = (points: Point[]): string => {
if (points.length < 2) return "";
return points.map((p, i) => `${i === 0 ? "M" : "L"} ${p.x} ${p.y}`).join(" ");
};
```
### Draw path with animation
```tsx
import { evolvePath } from "@remotion/paths";
const path = "M 100 200 L 200 150 L 300 180 L 400 100";
const progress = interpolate(frame, [0, 2 * fps], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
easing: Easing.out(Easing.quad),
});
const { strokeDasharray, strokeDashoffset } = evolvePath(progress, path);
<path
d={path}
fill="none"
stroke="#FF3232"
strokeWidth={4}
strokeDasharray={strokeDasharray}
strokeDashoffset={strokeDashoffset}
/>;
```
### Follow path with marker/arrow
```tsx
import {
getLength,
getPointAtLength,
getTangentAtLength,
} from "@remotion/paths";
const pathLength = getLength(path);
const point = getPointAtLength(path, progress * pathLength);
const tangent = getTangentAtLength(path, progress * pathLength);
const angle = Math.atan2(tangent.y, tangent.x);
<g
style={{
transform: `translate(${point.x}px, ${point.y}px) rotate(${angle}rad)`,
transformOrigin: "0 0",
}}
>
<polygon points="0,0 -20,-10 -20,10" fill="#FF3232" />
</g>;
```
.agents/skills/remotion-best-practices/rules/compositions.md
+154
View File
@@ -0,0 +1,154 @@
---
name: compositions
description: Defining compositions, stills, folders, default props and dynamic metadata
metadata:
tags: composition, still, folder, props, metadata
---
A `<Composition>` defines the component, width, height, fps and duration of a renderable video.
It normally is placed in the `src/Root.tsx` file.
```tsx
import { Composition } from "remotion";
import { MyComposition } from "./MyComposition";
export const RemotionRoot = () => {
return (
<Composition
id="MyComposition"
component={MyComposition}
durationInFrames={100}
fps={30}
width={1080}
height={1080}
/>
);
};
```
## Default Props
Pass `defaultProps` to provide initial values for your component.
Values must be JSON-serializable (`Date`, `Map`, `Set`, and `staticFile()` are supported).
```tsx
import { Composition } from "remotion";
import { MyComposition, MyCompositionProps } from "./MyComposition";
export const RemotionRoot = () => {
return (
<Composition
id="MyComposition"
component={MyComposition}
durationInFrames={100}
fps={30}
width={1080}
height={1080}
defaultProps={
{
title: "Hello World",
color: "#ff0000",
} satisfies MyCompositionProps
}
/>
);
};
```
Use `type` declarations for props rather than `interface` to ensure `defaultProps` type safety.
## Folders
Use `<Folder>` to organize compositions in the sidebar.
Folder names can only contain letters, numbers, and hyphens.
```tsx
import { Composition, Folder } from "remotion";
export const RemotionRoot = () => {
return (
<>
<Folder name="Marketing">
<Composition id="Promo" /* ... */ />
<Composition id="Ad" /* ... */ />
</Folder>
<Folder name="Social">
<Folder name="Instagram">
<Composition id="Story" /* ... */ />
<Composition id="Reel" /* ... */ />
</Folder>
</Folder>
</>
);
};
```
## Stills
Use `<Still>` for single-frame images. It does not require `durationInFrames` or `fps`.
```tsx
import { Still } from "remotion";
import { Thumbnail } from "./Thumbnail";
export const RemotionRoot = () => {
return (
<Still id="Thumbnail" component={Thumbnail} width={1280} height={720} />
);
};
```
## Calculate Metadata
Use `calculateMetadata` to make dimensions, duration, or props dynamic based on data.
```tsx
import { Composition, CalculateMetadataFunction } from "remotion";
import { MyComposition, MyCompositionProps } from "./MyComposition";
const calculateMetadata: CalculateMetadataFunction<
MyCompositionProps
> = async ({ props, abortSignal }) => {
const data = await fetch(`https://api.example.com/video/${props.videoId}`, {
signal: abortSignal,
}).then((res) => res.json());
return {
durationInFrames: Math.ceil(data.duration * 30),
props: {
...props,
videoUrl: data.url,
},
};
};
export const RemotionRoot = () => {
return (
<Composition
id="MyComposition"
component={MyComposition}
durationInFrames={100} // Placeholder, will be overridden
fps={30}
width={1080}
height={1080}
defaultProps={{ videoId: "abc123" }}
calculateMetadata={calculateMetadata}
/>
);
};
```
The function can return `props`, `durationInFrames`, `width`, `height`, `fps`, and codec-related defaults. It runs once before rendering begins.
## Nesting compositions within another
To add a composition within another composition, you can use the `<Sequence>` component with a `width` and `height` prop to specify the size of the composition.
```tsx
<AbsoluteFill>
<Sequence width={COMPOSITION_WIDTH} height={COMPOSITION_HEIGHT}>
<CompositionComponent />
</Sequence>
</AbsoluteFill>
```
.agents/skills/remotion-best-practices/rules/display-captions.md
+184
View File
@@ -0,0 +1,184 @@
---
name: display-captions
description: Displaying captions in Remotion with TikTok-style pages and word highlighting
metadata:
tags: captions, subtitles, display, tiktok, highlight
---
# Displaying captions in Remotion
This guide explains how to display captions in Remotion, assuming you already have captions in the [`Caption`](https://www.remotion.dev/docs/captions/caption) format.
## Prerequisites
Read [Transcribing audio](transcribe-captions.md) for how to generate captions.
First, the [`@remotion/captions`](https://www.remotion.dev/docs/captions) package needs to be installed.
If it is not installed, use the following command:
```bash
npx remotion add @remotion/captions
```
## Fetching captions
First, fetch your captions JSON file. Use [`useDelayRender()`](https://www.remotion.dev/docs/use-delay-render) to hold the render until the captions are loaded:
```tsx
import { useState, useEffect, useCallback } from "react";
import { AbsoluteFill, staticFile, useDelayRender } from "remotion";
import type { Caption } from "@remotion/captions";
export const MyComponent: React.FC = () => {
const [captions, setCaptions] = useState<Caption[] | null>(null);
const { delayRender, continueRender, cancelRender } = useDelayRender();
const [handle] = useState(() => delayRender());
const fetchCaptions = useCallback(async () => {
try {
// Assuming captions.json is in the public/ folder.
const response = await fetch(staticFile("captions123.json"));
const data = await response.json();
setCaptions(data);
continueRender(handle);
} catch (e) {
cancelRender(e);
}
}, [continueRender, cancelRender, handle]);
useEffect(() => {
fetchCaptions();
}, [fetchCaptions]);
if (!captions) {
return null;
}
return <AbsoluteFill>{/* Render captions here */}</AbsoluteFill>;
};
```
## Creating pages
Use `createTikTokStyleCaptions()` to group captions into pages. The `combineTokensWithinMilliseconds` option controls how many words appear at once:
```tsx
import { useMemo } from "react";
import { createTikTokStyleCaptions } from "@remotion/captions";
import type { Caption } from "@remotion/captions";
// How often captions should switch (in milliseconds)
// Higher values = more words per page
// Lower values = fewer words (more word-by-word)
const SWITCH_CAPTIONS_EVERY_MS = 1200;
const { pages } = useMemo(() => {
return createTikTokStyleCaptions({
captions,
combineTokensWithinMilliseconds: SWITCH_CAPTIONS_EVERY_MS,
});
}, [captions]);
```
## Rendering with Sequences
Map over the pages and render each one in a `<Sequence>`. Calculate the start frame and duration from the page timing:
```tsx
import { Sequence, useVideoConfig, AbsoluteFill } from "remotion";
import type { TikTokPage } from "@remotion/captions";
const CaptionedContent: React.FC = () => {
const { fps } = useVideoConfig();
return (
<AbsoluteFill>
{pages.map((page, index) => {
const nextPage = pages[index + 1] ?? null;
const startFrame = (page.startMs / 1000) * fps;
const endFrame = Math.min(
nextPage ? (nextPage.startMs / 1000) * fps : Infinity,
startFrame + (SWITCH_CAPTIONS_EVERY_MS / 1000) * fps,
);
const durationInFrames = endFrame - startFrame;
if (durationInFrames <= 0) {
return null;
}
return (
<Sequence
key={index}
from={startFrame}
durationInFrames={durationInFrames}
>
<CaptionPage page={page} />
</Sequence>
);
})}
</AbsoluteFill>
);
};
```
## White-space preservation
The captions are whitespace sensitive. You should include spaces in the `text` field before each word. Use `whiteSpace: "pre"` to preserve the whitespace in the captions.
## Separate component for captions
Put captioning logic in a separate component.
Make a new file for it.
## Word highlighting
A caption page contains `tokens` which you can use to highlight the currently spoken word:
```tsx
import { AbsoluteFill, useCurrentFrame, useVideoConfig } from "remotion";
import type { TikTokPage } from "@remotion/captions";
const HIGHLIGHT_COLOR = "#39E508";
const CaptionPage: React.FC<{ page: TikTokPage }> = ({ page }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
// Current time relative to the start of the sequence
const currentTimeMs = (frame / fps) * 1000;
// Convert to absolute time by adding the page start
const absoluteTimeMs = page.startMs + currentTimeMs;
return (
<AbsoluteFill style={{ justifyContent: "center", alignItems: "center" }}>
<div style={{ fontSize: 80, fontWeight: "bold", whiteSpace: "pre" }}>
{page.tokens.map((token) => {
const isActive =
token.fromMs <= absoluteTimeMs && token.toMs > absoluteTimeMs;
return (
<span
key={token.fromMs}
style={{ color: isActive ? HIGHLIGHT_COLOR : "white" }}
>
{token.text}
</span>
);
})}
</div>
</AbsoluteFill>
);
};
```
## Display captions alongside video content
By default, put the captions alongside the video content, so the captions are in sync.
For each video, make a new captions JSON file.
```tsx
<AbsoluteFill>
<Video src={staticFile("video.mp4")} />
<CaptionPage page={page} />
</AbsoluteFill>
```
.agents/skills/remotion-best-practices/rules/extract-frames.md
+229
View File
@@ -0,0 +1,229 @@
---
name: extract-frames
description: Extract frames from videos at specific timestamps using Mediabunny
metadata:
tags: frames, extract, video, thumbnail, filmstrip, canvas
---
# Extracting frames from videos
Use Mediabunny to extract frames from videos at specific timestamps. This is useful for generating thumbnails, filmstrips, or processing individual frames.
## The `extractFrames()` function
This function can be copy-pasted into any project.
```tsx
import {
ALL_FORMATS,
Input,
UrlSource,
VideoSample,
VideoSampleSink,
} from "mediabunny";
type Options = {
track: { width: number; height: number };
container: string;
durationInSeconds: number | null;
};
export type ExtractFramesTimestampsInSecondsFn = (
options: Options,
) => Promise<number[]> | number[];
export type ExtractFramesProps = {
src: string;
timestampsInSeconds: number[] | ExtractFramesTimestampsInSecondsFn;
onVideoSample: (sample: VideoSample) => void;
signal?: AbortSignal;
};
export async function extractFrames({
src,
timestampsInSeconds,
onVideoSample,
signal,
}: ExtractFramesProps): Promise<void> {
using input = new Input({
formats: ALL_FORMATS,
source: new UrlSource(src),
});
const [durationInSeconds, format, videoTrack] = await Promise.all([
input.computeDuration(),
input.getFormat(),
input.getPrimaryVideoTrack(),
]);
if (!videoTrack) {
throw new Error("No video track found in the input");
}
if (signal?.aborted) {
throw new Error("Aborted");
}
const timestamps =
typeof timestampsInSeconds === "function"
? await timestampsInSeconds({
track: {
width: videoTrack.displayWidth,
height: videoTrack.displayHeight,
},
container: format.name,
durationInSeconds,
})
: timestampsInSeconds;
if (timestamps.length === 0) {
return;
}
if (signal?.aborted) {
throw new Error("Aborted");
}
const sink = new VideoSampleSink(videoTrack);
for await (using videoSample of sink.samplesAtTimestamps(timestamps)) {
if (signal?.aborted) {
break;
}
if (!videoSample) {
continue;
}
onVideoSample(videoSample);
}
}
```
## Basic usage
Extract frames at specific timestamps:
```tsx
await extractFrames({
src: "https://remotion.media/video.mp4",
timestampsInSeconds: [0, 1, 2, 3, 4],
onVideoSample: (sample) => {
const canvas = document.createElement("canvas");
canvas.width = sample.displayWidth;
canvas.height = sample.displayHeight;
const ctx = canvas.getContext("2d");
sample.draw(ctx!, 0, 0);
},
});
```
## Creating a filmstrip
Use a callback function to dynamically calculate timestamps based on video metadata:
```tsx
const canvasWidth = 500;
const canvasHeight = 80;
const fromSeconds = 0;
const toSeconds = 10;
await extractFrames({
src: "https://remotion.media/video.mp4",
timestampsInSeconds: async ({ track, durationInSeconds }) => {
const aspectRatio = track.width / track.height;
const amountOfFramesFit = Math.ceil(
canvasWidth / (canvasHeight * aspectRatio),
);
const segmentDuration = toSeconds - fromSeconds;
const timestamps: number[] = [];
for (let i = 0; i < amountOfFramesFit; i++) {
timestamps.push(
fromSeconds + (segmentDuration / amountOfFramesFit) * (i + 0.5),
);
}
return timestamps;
},
onVideoSample: (sample) => {
console.log(`Frame at ${sample.timestamp}s`);
const canvas = document.createElement("canvas");
canvas.width = sample.displayWidth;
canvas.height = sample.displayHeight;
const ctx = canvas.getContext("2d");
sample.draw(ctx!, 0, 0);
},
});
```
## Cancellation with AbortSignal
Cancel frame extraction after a timeout:
```tsx
const controller = new AbortController();
setTimeout(() => controller.abort(), 5000);
try {
await extractFrames({
src: "https://remotion.media/video.mp4",
timestampsInSeconds: [0, 1, 2, 3, 4],
onVideoSample: (sample) => {
using frame = sample;
const canvas = document.createElement("canvas");
canvas.width = frame.displayWidth;
canvas.height = frame.displayHeight;
const ctx = canvas.getContext("2d");
frame.draw(ctx!, 0, 0);
},
signal: controller.signal,
});
console.log("Frame extraction complete!");
} catch (error) {
console.error("Frame extraction was aborted or failed:", error);
}
```
## Timeout with Promise.race
```tsx
const controller = new AbortController();
const timeoutPromise = new Promise<never>((_, reject) => {
const timeoutId = setTimeout(() => {
controller.abort();
reject(new Error("Frame extraction timed out after 10 seconds"));
}, 10000);
controller.signal.addEventListener("abort", () => clearTimeout(timeoutId), {
once: true,
});
});
try {
await Promise.race([
extractFrames({
src: "https://remotion.media/video.mp4",
timestampsInSeconds: [0, 1, 2, 3, 4],
onVideoSample: (sample) => {
using frame = sample;
const canvas = document.createElement("canvas");
canvas.width = frame.displayWidth;
canvas.height = frame.displayHeight;
const ctx = canvas.getContext("2d");
frame.draw(ctx!, 0, 0);
},
signal: controller.signal,
}),
timeoutPromise,
]);
console.log("Frame extraction complete!");
} catch (error) {
console.error("Frame extraction was aborted or failed:", error);
}
```
.agents/skills/remotion-best-practices/rules/ffmpeg.md
+38
View File
@@ -0,0 +1,38 @@
---
name: ffmpeg
description: Using FFmpeg and FFprobe in Remotion
metadata:
tags: ffmpeg, ffprobe, video, trimming
---
## FFmpeg in Remotion
`ffmpeg` and `ffprobe` do not need to be installed. They are available via the `bunx remotion ffmpeg` and `bunx remotion ffprobe`:
```bash
bunx remotion ffmpeg -i input.mp4 output.mp3
bunx remotion ffprobe input.mp4
```
### Trimming videos
You have 2 options for trimming videos:
1. Use the FFMpeg command line. You MUST re-encode the video to avoid frozen frames at the start of the video.
```bash
# Re-encodes from the exact frame
bunx remotion ffmpeg -ss 00:00:05 -i public/input.mp4 -to 00:00:10 -c:v libx264 -c:a aac public/output.mp4
```
2. Use the `trimBefore` and `trimAfter` props of the `<Video>` component. The benefit is that this is non-destructive and you can change the trim at any time.
```tsx
import { Video } from "@remotion/media";
<Video
src={staticFile("video.mp4")}
trimBefore={5 * fps}
trimAfter={10 * fps}
/>;
```
.agents/skills/remotion-best-practices/rules/fonts.md
+152
View File
@@ -0,0 +1,152 @@
---
name: fonts
description: Loading Google Fonts and local fonts in Remotion
metadata:
tags: fonts, google-fonts, typography, text
---
# Using fonts in Remotion
## Google Fonts with @remotion/google-fonts
The recommended way to use Google Fonts. It's type-safe and automatically blocks rendering until the font is ready.
### Prerequisites
First, the @remotion/google-fonts package needs to be installed.
If it is not installed, use the following command:
```bash
npx remotion add @remotion/google-fonts # If project uses npm
bunx remotion add @remotion/google-fonts # If project uses bun
yarn remotion add @remotion/google-fonts # If project uses yarn
pnpm exec remotion add @remotion/google-fonts # If project uses pnpm
```
```tsx
import { loadFont } from "@remotion/google-fonts/Lobster";
const { fontFamily } = loadFont();
export const MyComposition = () => {
return <div style={{ fontFamily }}>Hello World</div>;
};
```
Preferrably, specify only needed weights and subsets to reduce file size:
```tsx
import { loadFont } from "@remotion/google-fonts/Roboto";
const { fontFamily } = loadFont("normal", {
weights: ["400", "700"],
subsets: ["latin"],
});
```
### Waiting for font to load
Use `waitUntilDone()` if you need to know when the font is ready:
```tsx
import { loadFont } from "@remotion/google-fonts/Lobster";
const { fontFamily, waitUntilDone } = loadFont();
await waitUntilDone();
```
## Local fonts with @remotion/fonts
For local font files, use the `@remotion/fonts` package.
### Prerequisites
First, install @remotion/fonts:
```bash
npx remotion add @remotion/fonts # If project uses npm
bunx remotion add @remotion/fonts # If project uses bun
yarn remotion add @remotion/fonts # If project uses yarn
pnpm exec remotion add @remotion/fonts # If project uses pnpm
```
### Loading a local font
Place your font file in the `public/` folder and use `loadFont()`:
```tsx
import { loadFont } from "@remotion/fonts";
import { staticFile } from "remotion";
await loadFont({
family: "MyFont",
url: staticFile("MyFont-Regular.woff2"),
});
export const MyComposition = () => {
return <div style={{ fontFamily: "MyFont" }}>Hello World</div>;
};
```
### Loading multiple weights
Load each weight separately with the same family name:
```tsx
import { loadFont } from "@remotion/fonts";
import { staticFile } from "remotion";
await Promise.all([
loadFont({
family: "Inter",
url: staticFile("Inter-Regular.woff2"),
weight: "400",
}),
loadFont({
family: "Inter",
url: staticFile("Inter-Bold.woff2"),
weight: "700",
}),
]);
```
### Available options
```tsx
loadFont({
family: "MyFont", // Required: name to use in CSS
url: staticFile("font.woff2"), // Required: font file URL
format: "woff2", // Optional: auto-detected from extension
weight: "400", // Optional: font weight
style: "normal", // Optional: normal or italic
display: "block", // Optional: font-display behavior
});
```
## Using in components
Call `loadFont()` at the top level of your component or in a separate file that's imported early:
```tsx
import { loadFont } from "@remotion/google-fonts/Montserrat";
const { fontFamily } = loadFont("normal", {
weights: ["400", "700"],
subsets: ["latin"],
});
export const Title: React.FC<{ text: string }> = ({ text }) => {
return (
<h1
style={{
fontFamily,
fontSize: 80,
fontWeight: "bold",
}}
>
{text}
</h1>
);
};
```
.agents/skills/remotion-best-practices/rules/get-audio-duration.md
+58
View File
@@ -0,0 +1,58 @@
---
name: get-audio-duration
description: Getting the duration of an audio file in seconds with Mediabunny
metadata:
tags: duration, audio, length, time, seconds, mp3, wav
---
# Getting audio duration with Mediabunny
Mediabunny can extract the duration of an audio file. It works in browser, Node.js, and Bun environments.
## Getting audio duration
```tsx title="get-audio-duration.ts"
import { Input, ALL_FORMATS, UrlSource } from "mediabunny";
export const getAudioDuration = async (src: string) => {
const input = new Input({
formats: ALL_FORMATS,
source: new UrlSource(src, {
getRetryDelay: () => null,
}),
});
const durationInSeconds = await input.computeDuration();
return durationInSeconds;
};
```
## Usage
```tsx
const duration = await getAudioDuration("https://remotion.media/audio.mp3");
console.log(duration); // e.g. 180.5 (seconds)
```
## Using with staticFile in Remotion
Make sure to wrap the file path in `staticFile()`:
```tsx
import { staticFile } from "remotion";
const duration = await getAudioDuration(staticFile("audio.mp3"));
```
## In Node.js and Bun
Use `FileSource` instead of `UrlSource`:
```tsx
import { Input, ALL_FORMATS, FileSource } from "mediabunny";
const input = new Input({
formats: ALL_FORMATS,
source: new FileSource(file), // File object from input or drag-drop
});
```
.agents/skills/remotion-best-practices/rules/get-video-dimensions.md
+68
View File
@@ -0,0 +1,68 @@
---
name: get-video-dimensions
description: Getting the width and height of a video file with Mediabunny
metadata:
tags: dimensions, width, height, resolution, size, video
---
# Getting video dimensions with Mediabunny
Mediabunny can extract the width and height of a video file. It works in browser, Node.js, and Bun environments.
## Getting video dimensions
```tsx
import { Input, ALL_FORMATS, UrlSource } from "mediabunny";
export const getVideoDimensions = async (src: string) => {
const input = new Input({
formats: ALL_FORMATS,
source: new UrlSource(src, {
getRetryDelay: () => null,
}),
});
const videoTrack = await input.getPrimaryVideoTrack();
if (!videoTrack) {
throw new Error("No video track found");
}
return {
width: videoTrack.displayWidth,
height: videoTrack.displayHeight,
};
};
```
## Usage
```tsx
const dimensions = await getVideoDimensions("https://remotion.media/video.mp4");
console.log(dimensions.width); // e.g. 1920
console.log(dimensions.height); // e.g. 1080
```
## Using with local files
For local files, use `FileSource` instead of `UrlSource`:
```tsx
import { Input, ALL_FORMATS, FileSource } from "mediabunny";
const input = new Input({
formats: ALL_FORMATS,
source: new FileSource(file), // File object from input or drag-drop
});
const videoTrack = await input.getPrimaryVideoTrack();
const width = videoTrack.displayWidth;
const height = videoTrack.displayHeight;
```
## Using with staticFile in Remotion
```tsx
import { staticFile } from "remotion";
const dimensions = await getVideoDimensions(staticFile("video.mp4"));
```
.agents/skills/remotion-best-practices/rules/get-video-duration.md
+60
View File
@@ -0,0 +1,60 @@
---
name: get-video-duration
description: Getting the duration of a video file in seconds with Mediabunny
metadata:
tags: duration, video, length, time, seconds
---
# Getting video duration with Mediabunny
Mediabunny can extract the duration of a video file. It works in browser, Node.js, and Bun environments.
## Getting video duration
```tsx
import { Input, ALL_FORMATS, UrlSource } from "mediabunny";
export const getVideoDuration = async (src: string) => {
const input = new Input({
formats: ALL_FORMATS,
source: new UrlSource(src, {
getRetryDelay: () => null,
}),
});
const durationInSeconds = await input.computeDuration();
return durationInSeconds;
};
```
## Usage
```tsx
const duration = await getVideoDuration("https://remotion.media/video.mp4");
console.log(duration); // e.g. 10.5 (seconds)
```
## Video files from the public/ directory
Make sure to wrap the file path in `staticFile()`:
```tsx
import { staticFile } from "remotion";
const duration = await getVideoDuration(staticFile("video.mp4"));
```
## In Node.js and Bun
Use `FileSource` instead of `UrlSource`:
```tsx
import { Input, ALL_FORMATS, FileSource } from "mediabunny";
const input = new Input({
formats: ALL_FORMATS,
source: new FileSource(file), // File object from input or drag-drop
});
const durationInSeconds = await input.computeDuration();
```
.agents/skills/remotion-best-practices/rules/gifs.md
+141
View File
@@ -0,0 +1,141 @@
---
name: gif
description: Displaying GIFs, APNG, AVIF and WebP in Remotion
metadata:
tags: gif, animation, images, animated, apng, avif, webp
---
# Using Animated images in Remotion
## Basic usage
Use `<AnimatedImage>` to display a GIF, APNG, AVIF or WebP image synchronized with Remotion's timeline:
```tsx
import { AnimatedImage, staticFile } from "remotion";
export const MyComposition = () => {
return (
<AnimatedImage src={staticFile("animation.gif")} width={500} height={500} />
);
};
```
Remote URLs are also supported (must have CORS enabled):
```tsx
<AnimatedImage
src="https://example.com/animation.gif"
width={500}
height={500}
/>
```
## Sizing and fit
Control how the image fills its container with the `fit` prop:
```tsx
// Stretch to fill (default)
<AnimatedImage src={staticFile("animation.gif")} width={500} height={300} fit="fill" />
// Maintain aspect ratio, fit inside container
<AnimatedImage src={staticFile("animation.gif")} width={500} height={300} fit="contain" />
// Fill container, crop if needed
<AnimatedImage src={staticFile("animation.gif")} width={500} height={300} fit="cover" />
```
## Playback speed
Use `playbackRate` to control the animation speed:
```tsx
<AnimatedImage src={staticFile("animation.gif")} width={500} height={500} playbackRate={2} /> {/* 2x speed */}
<AnimatedImage src={staticFile("animation.gif")} width={500} height={500} playbackRate={0.5} /> {/* Half speed */}
```
## Looping behavior
Control what happens when the animation finishes:
```tsx
// Loop indefinitely (default)
<AnimatedImage src={staticFile("animation.gif")} width={500} height={500} loopBehavior="loop" />
// Play once, show final frame
<AnimatedImage src={staticFile("animation.gif")} width={500} height={500} loopBehavior="pause-after-finish" />
// Play once, then clear canvas
<AnimatedImage src={staticFile("animation.gif")} width={500} height={500} loopBehavior="clear-after-finish" />
```
## Styling
Use the `style` prop for additional CSS (use `width` and `height` props for sizing):
```tsx
<AnimatedImage
src={staticFile("animation.gif")}
width={500}
height={500}
style={{
borderRadius: 20,
position: "absolute",
top: 100,
left: 50,
}}
/>
```
## Getting GIF duration
Use `getGifDurationInSeconds()` from `@remotion/gif` to get the duration of a GIF.
```bash
npx remotion add @remotion/gif
```
```tsx
import { getGifDurationInSeconds } from "@remotion/gif";
import { staticFile } from "remotion";
const duration = await getGifDurationInSeconds(staticFile("animation.gif"));
console.log(duration); // e.g. 2.5
```
This is useful for setting the composition duration to match the GIF:
```tsx
import { getGifDurationInSeconds } from "@remotion/gif";
import { staticFile, CalculateMetadataFunction } from "remotion";
const calculateMetadata: CalculateMetadataFunction = async () => {
const duration = await getGifDurationInSeconds(staticFile("animation.gif"));
return {
durationInFrames: Math.ceil(duration * 30),
};
};
```
## Alternative
If `<AnimatedImage>` does not work (only supported in Chrome and Firefox), you can use `<Gif>` from `@remotion/gif` instead.
```bash
npx remotion add @remotion/gif # If project uses npm
bunx remotion add @remotion/gif # If project uses bun
yarn remotion add @remotion/gif # If project uses yarn
pnpm exec remotion add @remotion/gif # If project uses pnpm
```
```tsx
import { Gif } from "@remotion/gif";
import { staticFile } from "remotion";
export const MyComposition = () => {
return <Gif src={staticFile("animation.gif")} width={500} height={500} />;
};
```
The `<Gif>` component has the same props as `<AnimatedImage>` but only supports GIF files.
.agents/skills/remotion-best-practices/rules/images.md
+134
View File
@@ -0,0 +1,134 @@
---
name: images
description: Embedding images in Remotion using the <Img> component
metadata:
tags: images, img, staticFile, png, jpg, svg, webp
---
# Using images in Remotion
## The `<Img>` component
Always use the `<Img>` component from `remotion` to display images:
```tsx
import { Img, staticFile } from "remotion";
export const MyComposition = () => {
return <Img src={staticFile("photo.png")} />;
};
```
## Important restrictions
**You MUST use the `<Img>` component from `remotion`.** Do not use:
- Native HTML `<img>` elements
- Next.js `<Image>` component
- CSS `background-image`
The `<Img>` component ensures images are fully loaded before rendering, preventing flickering and blank frames during video export.
## Local images with staticFile()
Place images in the `public/` folder and use `staticFile()` to reference them:
```
my-video/
├─ public/
│ ├─ logo.png
│ ├─ avatar.jpg
│ └─ icon.svg
├─ src/
├─ package.json
```
```tsx
import { Img, staticFile } from "remotion";
<Img src={staticFile("logo.png")} />;
```
## Remote images
Remote URLs can be used directly without `staticFile()`:
```tsx
<Img src="https://example.com/image.png" />
```
Ensure remote images have CORS enabled.
For animated GIFs, use the `<Gif>` component from `@remotion/gif` instead.
## Sizing and positioning
Use the `style` prop to control size and position:
```tsx
<Img
src={staticFile("photo.png")}
style={{
width: 500,
height: 300,
position: "absolute",
top: 100,
left: 50,
objectFit: "cover",
}}
/>
```
## Dynamic image paths
Use template literals for dynamic file references:
```tsx
import { Img, staticFile, useCurrentFrame } from "remotion";
const frame = useCurrentFrame();
// Image sequence
<Img src={staticFile(`frames/frame${frame}.png`)} />
// Selecting based on props
<Img src={staticFile(`avatars/${props.userId}.png`)} />
// Conditional images
<Img src={staticFile(`icons/${isActive ? "active" : "inactive"}.svg`)} />
```
This pattern is useful for:
- Image sequences (frame-by-frame animations)
- User-specific avatars or profile images
- Theme-based icons
- State-dependent graphics
## Getting image dimensions
Use `getImageDimensions()` to get the dimensions of an image:
```tsx
import { getImageDimensions, staticFile } from "remotion";
const { width, height } = await getImageDimensions(staticFile("photo.png"));
```
This is useful for calculating aspect ratios or sizing compositions:
```tsx
import {
getImageDimensions,
staticFile,
CalculateMetadataFunction,
} from "remotion";
const calculateMetadata: CalculateMetadataFunction = async () => {
const { width, height } = await getImageDimensions(staticFile("photo.png"));
return {
width,
height,
};
};
```
.agents/skills/remotion-best-practices/rules/import-srt-captions.md
+69
View File
@@ -0,0 +1,69 @@
---
name: import-srt-captions
description: Importing .srt subtitle files into Remotion using @remotion/captions
metadata:
tags: captions, subtitles, srt, import, parse
---
# Importing .srt subtitles into Remotion
If you have an existing `.srt` subtitle file, you can import it into Remotion using `parseSrt()` from `@remotion/captions`.
If you don't have a .srt file, read [Transcribing audio](transcribe-captions.md) for how to generate captions instead.
## Prerequisites
First, the @remotion/captions package needs to be installed.
If it is not installed, use the following command:
```bash
npx remotion add @remotion/captions # If project uses npm
bunx remotion add @remotion/captions # If project uses bun
yarn remotion add @remotion/captions # If project uses yarn
pnpm exec remotion add @remotion/captions # If project uses pnpm
```
## Reading an .srt file
Use `staticFile()` to reference an `.srt` file in your `public` folder, then fetch and parse it:
```tsx
import { useState, useEffect, useCallback } from "react";
import { AbsoluteFill, staticFile, useDelayRender } from "remotion";
import { parseSrt } from "@remotion/captions";
import type { Caption } from "@remotion/captions";
export const MyComponent: React.FC = () => {
const [captions, setCaptions] = useState<Caption[] | null>(null);
const { delayRender, continueRender, cancelRender } = useDelayRender();
const [handle] = useState(() => delayRender());
const fetchCaptions = useCallback(async () => {
try {
const response = await fetch(staticFile("subtitles.srt"));
const text = await response.text();
const { captions: parsed } = parseSrt({ input: text });
setCaptions(parsed);
continueRender(handle);
} catch (e) {
cancelRender(e);
}
}, [continueRender, cancelRender, handle]);
useEffect(() => {
fetchCaptions();
}, [fetchCaptions]);
if (!captions) {
return null;
}
return <AbsoluteFill>{/* Use captions here */}</AbsoluteFill>;
};
```
Remote URLs are also supported - you can `fetch()` a remote file via URL instead of using `staticFile()`.
## Using imported captions
Once parsed, the captions are in the `Caption` format and can be used with all `@remotion/captions` utilities.
.agents/skills/remotion-best-practices/rules/light-leaks.md
+73
View File
@@ -0,0 +1,73 @@
---
name: light-leaks
description: Light leak overlay effects for Remotion using @remotion/light-leaks.
metadata:
tags: light-leaks, overlays, effects, transitions
---
## Light Leaks
This only works from Remotion 4.0.415 and up. Use `npx remotion versions` to check your Remotion version and `npx remotion upgrade` to upgrade your Remotion version.
`<LightLeak>` from `@remotion/light-leaks` renders a WebGL-based light leak effect. It reveals during the first half of its duration and retracts during the second half.
Typically used inside a `<TransitionSeries.Overlay>` to play over the cut point between two scenes. See the **transitions** rule for `<TransitionSeries>` and overlay usage.
## Prerequisites
```bash
npx remotion add @remotion/light-leaks
```
## Basic usage with TransitionSeries
```tsx
import { TransitionSeries } from "@remotion/transitions";
import { LightLeak } from "@remotion/light-leaks";
<TransitionSeries>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneA />
</TransitionSeries.Sequence>
<TransitionSeries.Overlay durationInFrames={30}>
<LightLeak />
</TransitionSeries.Overlay>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneB />
</TransitionSeries.Sequence>
</TransitionSeries>;
```
## Props
- `durationInFrames?` — defaults to the parent sequence/composition duration. The effect reveals during the first half and retracts during the second half.
- `seed?` — determines the shape of the light leak pattern. Different seeds produce different patterns. Default: `0`.
- `hueShift?` — rotates the hue in degrees (`0`–`360`). Default: `0` (yellow-to-orange). `120` = green, `240` = blue.
## Customizing the look
```tsx
import { LightLeak } from "@remotion/light-leaks";
// Blue-tinted light leak with a different pattern
<LightLeak seed={5} hueShift={240} />;
// Green-tinted light leak
<LightLeak seed={2} hueShift={120} />;
```
## Standalone usage
`<LightLeak>` can also be used outside of `<TransitionSeries>`, for example as a decorative overlay in any composition:
```tsx
import { AbsoluteFill } from "remotion";
import { LightLeak } from "@remotion/light-leaks";
const MyComp: React.FC = () => (
<AbsoluteFill>
<MyContent />
<LightLeak durationInFrames={60} seed={3} />
</AbsoluteFill>
);
```
.agents/skills/remotion-best-practices/rules/lottie.md
+70
View File
@@ -0,0 +1,70 @@
---
name: lottie
description: Embedding Lottie animations in Remotion.
metadata:
category: Animation
---
# Using Lottie Animations in Remotion
## Prerequisites
First, the @remotion/lottie package needs to be installed.
If it is not, use the following command:
```bash
npx remotion add @remotion/lottie # If project uses npm
bunx remotion add @remotion/lottie # If project uses bun
yarn remotion add @remotion/lottie # If project uses yarn
pnpm exec remotion add @remotion/lottie # If project uses pnpm
```
## Displaying a Lottie file
To import a Lottie animation:
- Fetch the Lottie asset
- Wrap the loading process in `delayRender()` and `continueRender()`
- Save the animation data in a state
- Render the Lottie animation using the `Lottie` component from the `@remotion/lottie` package
```tsx
import { Lottie, LottieAnimationData } from "@remotion/lottie";
import { useEffect, useState } from "react";
import { cancelRender, continueRender, delayRender } from "remotion";
export const MyAnimation = () => {
const [handle] = useState(() => delayRender("Loading Lottie animation"));
const [animationData, setAnimationData] =
useState<LottieAnimationData | null>(null);
useEffect(() => {
fetch("https://assets4.lottiefiles.com/packages/lf20_zyquagfl.json")
.then((data) => data.json())
.then((json) => {
setAnimationData(json);
continueRender(handle);
})
.catch((err) => {
cancelRender(err);
});
}, [handle]);
if (!animationData) {
return null;
}
return <Lottie animationData={animationData} />;
};
```
## Styling and animating
Lottie supports the `style` prop to allow styles and animations:
```tsx
return (
<Lottie animationData={animationData} style={{ width: 400, height: 400 }} />
);
```
.agents/skills/remotion-best-practices/rules/maps.md
+412
View File
@@ -0,0 +1,412 @@
---
name: maps
description: Make map animations with Mapbox
metadata:
tags: map, map animation, mapbox
---
Maps can be added to a Remotion video with Mapbox.
The [Mapbox documentation](https://docs.mapbox.com/mapbox-gl-js/api/) has the API reference.
## Prerequisites
Mapbox and `@turf/turf` need to be installed.
Search the project for lockfiles and run the correct command depending on the package manager:
If `package-lock.json` is found, use the following command:
```bash
npm i mapbox-gl @turf/turf @types/mapbox-gl
```
If `bun.lock` is found, use the following command:
```bash
bun i mapbox-gl @turf/turf @types/mapbox-gl
```
If `yarn.lock` is found, use the following command:
```bash
yarn add mapbox-gl @turf/turf @types/mapbox-gl
```
If `pnpm-lock.yaml` is found, use the following command:
```bash
pnpm i mapbox-gl @turf/turf @types/mapbox-gl
```
The user needs to create a free Mapbox account and create an access token by visiting https://console.mapbox.com/account/access-tokens/.
The mapbox token needs to be added to the `.env` file:
```txt title=".env"
REMOTION_MAPBOX_TOKEN==pk.your-mapbox-access-token
```
## Adding a map
Here is a basic example of a map in Remotion.
```tsx
import { useEffect, useMemo, useRef, useState } from "react";
import { AbsoluteFill, useDelayRender, useVideoConfig } from "remotion";
import mapboxgl, { Map } from "mapbox-gl";
export const lineCoordinates = [
[6.56158447265625, 46.059891147620725],
[6.5691375732421875, 46.05679376154153],
[6.5842437744140625, 46.05059898938315],
[6.594886779785156, 46.04702502069337],
[6.601066589355469, 46.0460718554722],
[6.6089630126953125, 46.0365370783104],
[6.6185760498046875, 46.018420689207964],
];
mapboxgl.accessToken = process.env.REMOTION_MAPBOX_TOKEN as string;
export const MyComposition = () => {
const ref = useRef<HTMLDivElement>(null);
const { delayRender, continueRender } = useDelayRender();
const { width, height } = useVideoConfig();
const [handle] = useState(() => delayRender("Loading map..."));
const [map, setMap] = useState<Map | null>(null);
useEffect(() => {
const _map = new Map({
container: ref.current!,
zoom: 11.53,
center: [6.5615, 46.0598],
pitch: 65,
bearing: 0,
style: "⁠mapbox://styles/mapbox/standard",
interactive: false,
fadeDuration: 0,
});
_map.on("style.load", () => {
// Hide all features from the Mapbox Standard style
const hideFeatures = [
"showRoadsAndTransit",
"showRoads",
"showTransit",
"showPedestrianRoads",
"showRoadLabels",
"showTransitLabels",
"showPlaceLabels",
"showPointOfInterestLabels",
"showPointsOfInterest",
"showAdminBoundaries",
"showLandmarkIcons",
"showLandmarkIconLabels",
"show3dObjects",
"show3dBuildings",
"show3dTrees",
"show3dLandmarks",
"show3dFacades",
];
for (const feature of hideFeatures) {
_map.setConfigProperty("basemap", feature, false);
}
_map.setConfigProperty("basemap", "colorTrunks", "rgba(0, 0, 0, 0)");
_map.addSource("trace", {
type: "geojson",
data: {
type: "Feature",
properties: {},
geometry: {
type: "LineString",
coordinates: lineCoordinates,
},
},
});
_map.addLayer({
type: "line",
source: "trace",
id: "line",
paint: {
"line-color": "black",
"line-width": 5,
},
layout: {
"line-cap": "round",
"line-join": "round",
},
});
});
_map.on("load", () => {
continueRender(handle);
setMap(_map);
});
}, [handle, lineCoordinates]);
const style: React.CSSProperties = useMemo(
() => ({ width, height, position: "absolute" }),
[width, height],
);
return <AbsoluteFill ref={ref} style={style} />;
};
```
The following is important in Remotion:
- Animations must be driven by `useCurrentFrame()` and animations that Mapbox brings itself should be disabled. For example, the `fadeDuration` prop should be set to `0`, `interactive` should be set to `false`, etc.
- Loading the map should be delayed using `useDelayRender()` and the map should be set to `null` until it is loaded.
- The element containing the ref MUST have an explicit width and height and `position: "absolute"`.
- Do not add a `_map.remove();` cleanup function.
## Drawing lines
Unless I request it, do not add a glow effect to the lines.
Unless I request it, do not add additional points to the lines.
## Map style
By default, use the `mapbox://styles/mapbox/standard` style.
Hide the labels from the base map style.
Unless I request otherwise, remove all features from the Mapbox Standard style.
```tsx
// Hide all features from the Mapbox Standard style
const hideFeatures = [
"showRoadsAndTransit",
"showRoads",
"showTransit",
"showPedestrianRoads",
"showRoadLabels",
"showTransitLabels",
"showPlaceLabels",
"showPointOfInterestLabels",
"showPointsOfInterest",
"showAdminBoundaries",
"showLandmarkIcons",
"showLandmarkIconLabels",
"show3dObjects",
"show3dBuildings",
"show3dTrees",
"show3dLandmarks",
"show3dFacades",
];
for (const feature of hideFeatures) {
_map.setConfigProperty("basemap", feature, false);
}
_map.setConfigProperty("basemap", "colorMotorways", "transparent");
_map.setConfigProperty("basemap", "colorRoads", "transparent");
_map.setConfigProperty("basemap", "colorTrunks", "transparent");
```
## Animating the camera
You can animate the camera along the line by adding a `useEffect` hook that updates the camera position based on the current frame.
Unless I ask for it, do not jump between camera angles.
```tsx
import * as turf from "@turf/turf";
import { interpolate } from "remotion";
import { Easing } from "remotion";
import { useCurrentFrame, useVideoConfig, useDelayRender } from "remotion";
const animationDuration = 20;
const cameraAltitude = 4000;
```
```tsx
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const { delayRender, continueRender } = useDelayRender();
useEffect(() => {
if (!map) {
return;
}
const handle = delayRender("Moving point...");
const routeDistance = turf.length(turf.lineString(lineCoordinates));
const progress = interpolate(
frame / fps,
[0.00001, animationDuration],
[0, 1],
{
easing: Easing.inOut(Easing.sin),
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
},
);
const camera = map.getFreeCameraOptions();
const alongRoute = turf.along(
turf.lineString(lineCoordinates),
routeDistance * progress,
).geometry.coordinates;
camera.lookAtPoint({
lng: alongRoute[0],
lat: alongRoute[1],
});
map.setFreeCameraOptions(camera);
map.once("idle", () => continueRender(handle));
}, [lineCoordinates, fps, frame, handle, map]);
```
Notes:
IMPORTANT: Keep the camera by default so north is up.
IMPORTANT: For multi-step animations, set all properties at all stages (zoom, position, line progress) to prevent jumps. Override initial values.
- The progress is clamped to a minimum value to avoid the line being empty, which can lead to turf errors
- See [Timing](./timing.md) for more options for timing.
- Consider the dimensions of the composition and make the lines thick enough and the label font size large enough to be legible for when the composition is scaled down.
## Animating lines
### Straight lines (linear interpolation)
To animate a line that appears straight on the map, use linear interpolation between coordinates. Do NOT use turf's `lineSliceAlong` or `along` functions, as they use geodesic (great circle) calculations which appear curved on a Mercator projection.
```tsx
const frame = useCurrentFrame();
const { durationInFrames } = useVideoConfig();
useEffect(() => {
if (!map) return;
const animationHandle = delayRender("Animating line...");
const progress = interpolate(frame, [0, durationInFrames - 1], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
easing: Easing.inOut(Easing.cubic),
});
// Linear interpolation for a straight line on the map
const start = lineCoordinates[0];
const end = lineCoordinates[1];
const currentLng = start[0] + (end[0] - start[0]) * progress;
const currentLat = start[1] + (end[1] - start[1]) * progress;
const lineData: GeoJSON.Feature<GeoJSON.LineString> = {
type: "Feature",
properties: {},
geometry: {
type: "LineString",
coordinates: [start, [currentLng, currentLat]],
},
};
const source = map.getSource("trace") as mapboxgl.GeoJSONSource;
if (source) {
source.setData(lineData);
}
map.once("idle", () => continueRender(animationHandle));
}, [frame, map, durationInFrames]);
```
### Curved lines (geodesic/great circle)
To animate a line that follows the geodesic (great circle) path between two points, use turf's `lineSliceAlong`. This is useful for showing flight paths or the actual shortest distance on Earth.
```tsx
import * as turf from "@turf/turf";
const routeLine = turf.lineString(lineCoordinates);
const routeDistance = turf.length(routeLine);
const currentDistance = Math.max(0.001, routeDistance * progress);
const slicedLine = turf.lineSliceAlong(routeLine, 0, currentDistance);
const source = map.getSource("route") as mapboxgl.GeoJSONSource;
if (source) {
source.setData(slicedLine);
}
```
## Markers
Add labels, and markers where appropriate.
```tsx
_map.addSource("markers", {
type: "geojson",
data: {
type: "FeatureCollection",
features: [
{
type: "Feature",
properties: { name: "Point 1" },
geometry: { type: "Point", coordinates: [-118.2437, 34.0522] },
},
],
},
});
_map.addLayer({
id: "city-markers",
type: "circle",
source: "markers",
paint: {
"circle-radius": 40,
"circle-color": "#FF4444",
"circle-stroke-width": 4,
"circle-stroke-color": "#FFFFFF",
},
});
_map.addLayer({
id: "labels",
type: "symbol",
source: "markers",
layout: {
"text-field": ["get", "name"],
"text-font": ["DIN Pro Bold", "Arial Unicode MS Bold"],
"text-size": 50,
"text-offset": [0, 0.5],
"text-anchor": "top",
},
paint: {
"text-color": "#FFFFFF",
"text-halo-color": "#000000",
"text-halo-width": 2,
},
});
```
Make sure they are big enough. Check the composition dimensions and scale the labels accordingly.
For a composition size of 1920x1080, the label font size should be at least 40px.
IMPORTANT: Keep the `text-offset` small enough so it is close to the marker. Consider the marker circle radius. For a circle radius of 40, this is a good offset:
```tsx
"text-offset": [0, 0.5],
```
## 3D buildings
To enable 3D buildings, use the following code:
```tsx
_map.setConfigProperty("basemap", "show3dObjects", true);
_map.setConfigProperty("basemap", "show3dLandmarks", true);
_map.setConfigProperty("basemap", "show3dBuildings", true);
```
## Rendering
When rendering a map animation, make sure to render with the following flags:
```
npx remotion render --gl=angle --concurrency=1
```
.agents/skills/remotion-best-practices/rules/measuring-dom-nodes.md
+34
View File
@@ -0,0 +1,34 @@
---
name: measuring-dom-nodes
description: Measuring DOM element dimensions in Remotion
metadata:
tags: measure, layout, dimensions, getBoundingClientRect, scale
---
# Measuring DOM nodes in Remotion
Remotion applies a `scale()` transform to the video container, which affects values from `getBoundingClientRect()`. Use `useCurrentScale()` to get correct measurements.
## Measuring element dimensions
```tsx
import { useCurrentScale } from "remotion";
import { useRef, useEffect, useState } from "react";
export const MyComponent = () => {
const ref = useRef<HTMLDivElement>(null);
const scale = useCurrentScale();
const [dimensions, setDimensions] = useState({ width: 0, height: 0 });
useEffect(() => {
if (!ref.current) return;
const rect = ref.current.getBoundingClientRect();
setDimensions({
width: rect.width / scale,
height: rect.height / scale,
});
}, [scale]);
return <div ref={ref}>Content to measure</div>;
};
```
.agents/skills/remotion-best-practices/rules/measuring-text.md
+140
View File
@@ -0,0 +1,140 @@
---
name: measuring-text
description: Measuring text dimensions, fitting text to containers, and checking overflow
metadata:
tags: measure, text, layout, dimensions, fitText, fillTextBox
---
# Measuring text in Remotion
## Prerequisites
Install @remotion/layout-utils if it is not already installed:
```bash
npx remotion add @remotion/layout-utils
```
## Measuring text dimensions
Use `measureText()` to calculate the width and height of text:
```tsx
import { measureText } from "@remotion/layout-utils";
const { width, height } = measureText({
text: "Hello World",
fontFamily: "Arial",
fontSize: 32,
fontWeight: "bold",
});
```
Results are cached - duplicate calls return the cached result.
## Fitting text to a width
Use `fitText()` to find the optimal font size for a container:
```tsx
import { fitText } from "@remotion/layout-utils";
const { fontSize } = fitText({
text: "Hello World",
withinWidth: 600,
fontFamily: "Inter",
fontWeight: "bold",
});
return (
<div
style={{
fontSize: Math.min(fontSize, 80), // Cap at 80px
fontFamily: "Inter",
fontWeight: "bold",
}}
>
Hello World
</div>
);
```
## Checking text overflow
Use `fillTextBox()` to check if text exceeds a box:
```tsx
import { fillTextBox } from "@remotion/layout-utils";
const box = fillTextBox({ maxBoxWidth: 400, maxLines: 3 });
const words = ["Hello", "World", "This", "is", "a", "test"];
for (const word of words) {
const { exceedsBox } = box.add({
text: word + " ",
fontFamily: "Arial",
fontSize: 24,
});
if (exceedsBox) {
// Text would overflow, handle accordingly
break;
}
}
```
## Best practices
**Load fonts first:** Only call measurement functions after fonts are loaded.
```tsx
import { loadFont } from "@remotion/google-fonts/Inter";
const { fontFamily, waitUntilDone } = loadFont("normal", {
weights: ["400"],
subsets: ["latin"],
});
waitUntilDone().then(() => {
// Now safe to measure
const { width } = measureText({
text: "Hello",
fontFamily,
fontSize: 32,
});
});
```
**Use validateFontIsLoaded:** Catch font loading issues early:
```tsx
measureText({
text: "Hello",
fontFamily: "MyCustomFont",
fontSize: 32,
validateFontIsLoaded: true, // Throws if font not loaded
});
```
**Match font properties:** Use the same properties for measurement and rendering:
```tsx
const fontStyle = {
fontFamily: "Inter",
fontSize: 32,
fontWeight: "bold" as const,
letterSpacing: "0.5px",
};
const { width } = measureText({
text: "Hello",
...fontStyle,
});
return <div style={fontStyle}>Hello</div>;
```
**Avoid padding and border:** Use `outline` instead of `border` to prevent layout differences:
```tsx
<div style={{ outline: "2px solid red" }}>Text</div>
```
.agents/skills/remotion-best-practices/rules/parameters.md
+109
View File
@@ -0,0 +1,109 @@
---
name: parameters
description: Make a video parametrizable by adding a Zod schema
metadata:
tags: parameters, zod, schema
---
To make a video parametrizable, a Zod schema can be added to a composition.
First, `zod` must be installed .
Search the project for lockfiles and run the correct command depending on the package manager:
If `package-lock.json` is found, use the following command:
```bash
npm i zod
```
If `bun.lockb` is found, use the following command:
```bash
bun i zod
```
If `yarn.lock` is found, use the following command:
```bash
yarn add zod
```
If `pnpm-lock.yaml` is found, use the following command:
```bash
pnpm i zod
```
Then, a Zod schema can be defined alongside the component:
```tsx title="src/MyComposition.tsx"
import { z } from "zod";
export const MyCompositionSchema = z.object({
title: z.string(),
});
const MyComponent: React.FC<z.infer<typeof MyCompositionSchema>> = () => {
return (
<div>
<h1>{props.title}</h1>
</div>
);
};
```
In the root file, the schema can be passed to the composition:
```tsx title="src/Root.tsx"
import { Composition } from "remotion";
import { MycComponent, MyCompositionSchema } from "./MyComposition";
export const RemotionRoot = () => {
return (
<Composition
id="MyComposition"
component={MyComponent}
durationInFrames={100}
fps={30}
width={1080}
height={1080}
defaultProps={{ title: "Hello World" }}
schema={MyCompositionSchema}
/>
);
};
```
Now, the user can edit the parameter visually in the sidebar.
All schemas that are supported by Zod are supported by Remotion.
Remotion requires that the top-level type is a z.object(), because the collection of props of a React component is always an object.
## Color picker
For adding a color picker, use `zColor()` from `@remotion/zod-types`.
If it is not installed, use the following command:
```bash
npx remotion add @remotion/zod-types # If project uses npm
bunx remotion add @remotion/zod-types # If project uses bun
yarn remotion add @remotion/zod-types # If project uses yarn
pnpm exec remotion add @remotion/zod-types # If project uses pnpm
```
Then import `zColor` from `@remotion/zod-types`:
```tsx
import { zColor } from "@remotion/zod-types";
```
Then use it in the schema:
```tsx
export const MyCompositionSchema = z.object({
color: zColor(),
});
```
.agents/skills/remotion-best-practices/rules/sequencing.md
+118
View File
@@ -0,0 +1,118 @@
---
name: sequencing
description: Sequencing patterns for Remotion - delay, trim, limit duration of items
metadata:
tags: sequence, series, timing, delay, trim
---
Use `<Sequence>` to delay when an element appears in the timeline.
```tsx
import { Sequence } from "remotion";
const {fps} = useVideoConfig();
<Sequence from={1 * fps} durationInFrames={2 * fps} premountFor={1 * fps}>
<Title />
</Sequence>
<Sequence from={2 * fps} durationInFrames={2 * fps} premountFor={1 * fps}>
<Subtitle />
</Sequence>
```
This will by default wrap the component in an absolute fill element.
If the items should not be wrapped, use the `layout` prop:
```tsx
<Sequence layout="none">
<Title />
</Sequence>
```
## Premounting
This loads the component in the timeline before it is actually played.
Always premount any `<Sequence>`!
```tsx
<Sequence premountFor={1 * fps}>
<Title />
</Sequence>
```
## Series
Use `<Series>` when elements should play one after another without overlap.
```tsx
import { Series } from "remotion";
<Series>
<Series.Sequence durationInFrames={45}>
<Intro />
</Series.Sequence>
<Series.Sequence durationInFrames={60}>
<MainContent />
</Series.Sequence>
<Series.Sequence durationInFrames={30}>
<Outro />
</Series.Sequence>
</Series>;
```
Same as with `<Sequence>`, the items will be wrapped in an absolute fill element by default when using `<Series.Sequence>`, unless the `layout` prop is set to `none`.
### Series with overlaps
Use negative offset for overlapping sequences:
```tsx
<Series>
<Series.Sequence durationInFrames={60}>
<SceneA />
</Series.Sequence>
<Series.Sequence offset={-15} durationInFrames={60}>
{/* Starts 15 frames before SceneA ends */}
<SceneB />
</Series.Sequence>
</Series>
```
## Frame References Inside Sequences
Inside a Sequence, `useCurrentFrame()` returns the local frame (starting from 0):
```tsx
<Sequence from={60} durationInFrames={30}>
<MyComponent />
{/* Inside MyComponent, useCurrentFrame() returns 0-29, not 60-89 */}
</Sequence>
```
## Nested Sequences
Sequences can be nested for complex timing:
```tsx
<Sequence from={0} durationInFrames={120}>
<Background />
<Sequence from={15} durationInFrames={90} layout="none">
<Title />
</Sequence>
<Sequence from={45} durationInFrames={60} layout="none">
<Subtitle />
</Sequence>
</Sequence>
```
## Nesting compositions within another
To add a composition within another composition, you can use the `<Sequence>` component with a `width` and `height` prop to specify the size of the composition.
```tsx
<AbsoluteFill>
<Sequence width={COMPOSITION_WIDTH} height={COMPOSITION_HEIGHT}>
<CompositionComponent />
</Sequence>
</AbsoluteFill>
```
.agents/skills/remotion-best-practices/rules/sfx.md
+26
View File
@@ -0,0 +1,26 @@
---
name: sfx
description: Including sound effects
metadata:
tags: sfx, sound, effect, audio
---
To include a sound effect, use the `<Audio>` tag:
```tsx
import { Audio } from "@remotion/sfx";
<Audio src={"https://remotion.media/whoosh.wav"} />;
```
The following sound effects are available:
- `https://remotion.media/whoosh.wav`
- `https://remotion.media/whip.wav`
- `https://remotion.media/page-turn.wav`
- `https://remotion.media/switch.wav`
- `https://remotion.media/mouse-click.wav`
- `https://remotion.media/shutter-modern.wav`
- `https://remotion.media/shutter-old.wav`
For more sound effects, search the internet. A good resource is https://github.com/kapishdima/soundcn/tree/main/assets.
.agents/skills/remotion-best-practices/rules/subtitles.md
+36
View File
@@ -0,0 +1,36 @@
---
name: subtitles
description: subtitles and caption rules
metadata:
tags: subtitles, captions, remotion, json
---
All captions must be processed in JSON. The captions must use the `Caption` type which is the following:
```ts
import type { Caption } from "@remotion/captions";
```
This is the definition:
```ts
type Caption = {
text: string;
startMs: number;
endMs: number;
timestampMs: number | null;
confidence: number | null;
};
```
## Generating captions
To transcribe video and audio files to generate captions, load the [./transcribe-captions.md](./transcribe-captions.md) file for more instructions.
## Displaying captions
To display captions in your video, load the [./display-captions.md](./display-captions.md) file for more instructions.
## Importing captions
To import captions from a .srt file, load the [./import-srt-captions.md](./import-srt-captions.md) file for more instructions.
.agents/skills/remotion-best-practices/rules/tailwind.md
+11
View File
@@ -0,0 +1,11 @@
---
name: tailwind
description: Using TailwindCSS in Remotion.
metadata:
---
You can and should use TailwindCSS in Remotion, if TailwindCSS is installed in the project.
Don't use `transition-*` or `animate-*` classes - always animate using the `useCurrentFrame()` hook.
Tailwind must be installed and enabled first in a Remotion project - fetch https://www.remotion.dev/docs/tailwind using WebFetch for instructions.
.agents/skills/remotion-best-practices/rules/text-animations.md
+20
View File
@@ -0,0 +1,20 @@
---
name: text-animations
description: Typography and text animation patterns for Remotion.
metadata:
tags: typography, text, typewriter, highlighter ken
---
## Text animations
Based on `useCurrentFrame()`, reduce the string character by character to create a typewriter effect.
## Typewriter Effect
See [Typewriter](assets/text-animations-typewriter.tsx) for an advanced example with a blinking cursor and a pause after the first sentence.
Always use string slicing for typewriter effects. Never use per-character opacity.
## Word Highlighting
See [Word Highlight](assets/text-animations-word-highlight.tsx) for an example for how a word highlight is animated, like with a highlighter pen.
.agents/skills/remotion-best-practices/rules/timing.md
+179
View File
@@ -0,0 +1,179 @@
---
name: timing
description: Interpolation curves in Remotion - linear, easing, spring animations
metadata:
tags: spring, bounce, easing, interpolation
---
A simple linear interpolation is done using the `interpolate` function.
```ts title="Going from 0 to 1 over 100 frames"
import { interpolate } from "remotion";
const opacity = interpolate(frame, [0, 100], [0, 1]);
```
By default, the values are not clamped, so the value can go outside the range [0, 1].
Here is how they can be clamped:
```ts title="Going from 0 to 1 over 100 frames with extrapolation"
const opacity = interpolate(frame, [0, 100], [0, 1], {
extrapolateRight: "clamp",
extrapolateLeft: "clamp",
});
```
## Spring animations
Spring animations have a more natural motion.
They go from 0 to 1 over time.
```ts title="Spring animation from 0 to 1 over 100 frames"
import { spring, useCurrentFrame, useVideoConfig } from "remotion";
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const scale = spring({
frame,
fps,
});
```
### Physical properties
The default configuration is: `mass: 1, damping: 10, stiffness: 100`.
This leads to the animation having a bit of bounce before it settles.
The config can be overwritten like this:
```ts
const scale = spring({
frame,
fps,
config: { damping: 200 },
});
```
The recommended configuration for a natural motion without a bounce is: `{ damping: 200 }`.
Here are some common configurations:
```tsx
const smooth = { damping: 200 }; // Smooth, no bounce (subtle reveals)
const snappy = { damping: 20, stiffness: 200 }; // Snappy, minimal bounce (UI elements)
const bouncy = { damping: 8 }; // Bouncy entrance (playful animations)
const heavy = { damping: 15, stiffness: 80, mass: 2 }; // Heavy, slow, small bounce
```
### Delay
The animation starts immediately by default.
Use the `delay` parameter to delay the animation by a number of frames.
```tsx
const entrance = spring({
frame: frame - ENTRANCE_DELAY,
fps,
delay: 20,
});
```
### Duration
A `spring()` has a natural duration based on the physical properties.
To stretch the animation to a specific duration, use the `durationInFrames` parameter.
```tsx
const spring = spring({
frame,
fps,
durationInFrames: 40,
});
```
### Combining spring() with interpolate()
Map spring output (0-1) to custom ranges:
```tsx
const springProgress = spring({
frame,
fps,
});
// Map to rotation
const rotation = interpolate(springProgress, [0, 1], [0, 360]);
<div style={{ rotate: rotation + "deg" }} />;
```
### Adding springs
Springs return just numbers, so math can be performed:
```tsx
const frame = useCurrentFrame();
const { fps, durationInFrames } = useVideoConfig();
const inAnimation = spring({
frame,
fps,
});
const outAnimation = spring({
frame,
fps,
durationInFrames: 1 * fps,
delay: durationInFrames - 1 * fps,
});
const scale = inAnimation - outAnimation;
```
## Easing
Easing can be added to the `interpolate` function:
```ts
import { interpolate, Easing } from "remotion";
const value1 = interpolate(frame, [0, 100], [0, 1], {
easing: Easing.inOut(Easing.quad),
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
```
The default easing is `Easing.linear`.
There are various other convexities:
- `Easing.in` for starting slow and accelerating
- `Easing.out` for starting fast and slowing down
- `Easing.inOut`
and curves (sorted from most linear to most curved):
- `Easing.quad`
- `Easing.sin`
- `Easing.exp`
- `Easing.circle`
Convexities and curves need be combined for an easing function:
```ts
const value1 = interpolate(frame, [0, 100], [0, 1], {
easing: Easing.inOut(Easing.quad),
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
```
Cubic bezier curves are also supported:
```ts
const value1 = interpolate(frame, [0, 100], [0, 1], {
easing: Easing.bezier(0.8, 0.22, 0.96, 0.65),
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
```
.agents/skills/remotion-best-practices/rules/transcribe-captions.md
+70
View File
@@ -0,0 +1,70 @@
---
name: transcribe-captions
description: Transcribing audio to generate captions in Remotion
metadata:
tags: captions, transcribe, whisper, audio, speech-to-text
---
# Transcribing audio
To transcribe audio to generate captions in Remotion, you can use the [`transcribe()`](https://www.remotion.dev/docs/install-whisper-cpp/transcribe) function from the [`@remotion/install-whisper-cpp`](https://www.remotion.dev/docs/install-whisper-cpp) package.
## Prerequisites
First, the @remotion/install-whisper-cpp package needs to be installed.
If it is not installed, use the following command:
```bash
npx remotion add @remotion/install-whisper-cpp
```
## Transcribing
Make a Node.js script to download Whisper.cpp and a model, and transcribe the audio.
```ts
import path from "path";
import {
downloadWhisperModel,
installWhisperCpp,
transcribe,
toCaptions,
} from "@remotion/install-whisper-cpp";
import fs from "fs";
const to = path.join(process.cwd(), "whisper.cpp");
await installWhisperCpp({
to,
version: "1.5.5",
});
await downloadWhisperModel({
model: "medium.en",
folder: to,
});
// Convert the audio to a 16KHz wav file first if needed:
// import {execSync} from 'child_process';
// execSync('ffmpeg -i /path/to/audio.mp4 -ar 16000 /path/to/audio.wav -y');
const whisperCppOutput = await transcribe({
model: "medium.en",
whisperPath: to,
whisperCppVersion: "1.5.5",
inputPath: "/path/to/audio123.wav",
tokenLevelTimestamps: true,
});
// Optional: Apply our recommended postprocessing
const { captions } = toCaptions({
whisperCppOutput,
});
// Write it to the public/ folder so it can be fetched from Remotion
fs.writeFileSync("captions123.json", JSON.stringify(captions, null, 2));
```
Transcribe each clip individually and create multiple JSON files.
See [Displaying captions](display-captions.md) for how to display the captions in Remotion.
.agents/skills/remotion-best-practices/rules/transitions.md
+197
View File
@@ -0,0 +1,197 @@
---
name: transitions
description: Scene transitions and overlays for Remotion using TransitionSeries.
metadata:
tags: transitions, overlays, fade, slide, wipe, scenes
---
## TransitionSeries
`<TransitionSeries>` arranges scenes and supports two ways to enhance the cut point between them:
- **Transitions** (`<TransitionSeries.Transition>`) — crossfade, slide, wipe, etc. between two scenes. Shortens the timeline because both scenes play simultaneously during the transition.
- **Overlays** (`<TransitionSeries.Overlay>`) — render an effect (e.g. a light leak) on top of the cut point without shortening the timeline.
Children are absolutely positioned.
## Prerequisites
```bash
npx remotion add @remotion/transitions
```
## Transition example
```tsx
import { TransitionSeries, linearTiming } from "@remotion/transitions";
import { fade } from "@remotion/transitions/fade";
<TransitionSeries>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneA />
</TransitionSeries.Sequence>
<TransitionSeries.Transition
presentation={fade()}
timing={linearTiming({ durationInFrames: 15 })}
/>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneB />
</TransitionSeries.Sequence>
</TransitionSeries>;
```
## Overlay example
Any React component can be used as an overlay. For a ready-made effect, see the **light-leaks** rule.
```tsx
import { TransitionSeries } from "@remotion/transitions";
import { LightLeak } from "@remotion/light-leaks";
<TransitionSeries>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneA />
</TransitionSeries.Sequence>
<TransitionSeries.Overlay durationInFrames={20}>
<LightLeak />
</TransitionSeries.Overlay>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneB />
</TransitionSeries.Sequence>
</TransitionSeries>;
```
## Mixing transitions and overlays
Transitions and overlays can coexist in the same `<TransitionSeries>`, but an overlay cannot be adjacent to a transition or another overlay.
```tsx
import { TransitionSeries, linearTiming } from "@remotion/transitions";
import { fade } from "@remotion/transitions/fade";
import { LightLeak } from "@remotion/light-leaks";
<TransitionSeries>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneA />
</TransitionSeries.Sequence>
<TransitionSeries.Overlay durationInFrames={30}>
<LightLeak />
</TransitionSeries.Overlay>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneB />
</TransitionSeries.Sequence>
<TransitionSeries.Transition
presentation={fade()}
timing={linearTiming({ durationInFrames: 15 })}
/>
<TransitionSeries.Sequence durationInFrames={60}>
<SceneC />
</TransitionSeries.Sequence>
</TransitionSeries>;
```
## Transition props
`<TransitionSeries.Transition>` requires:
- `presentation` — the visual effect (e.g. `fade()`, `slide()`, `wipe()`).
- `timing` — controls speed and easing (e.g. `linearTiming()`, `springTiming()`).
## Overlay props
`<TransitionSeries.Overlay>` accepts:
- `durationInFrames` — how long the overlay is visible (positive integer).
- `offset?` — shifts the overlay relative to the cut point center. Positive = later, negative = earlier. Default: `0`.
## Available transition types
Import transitions from their respective modules:
```tsx
import { fade } from "@remotion/transitions/fade";
import { slide } from "@remotion/transitions/slide";
import { wipe } from "@remotion/transitions/wipe";
import { flip } from "@remotion/transitions/flip";
import { clockWipe } from "@remotion/transitions/clock-wipe";
```
## Slide transition with direction
```tsx
import { slide } from "@remotion/transitions/slide";
<TransitionSeries.Transition
presentation={slide({ direction: "from-left" })}
timing={linearTiming({ durationInFrames: 20 })}
/>;
```
Directions: `"from-left"`, `"from-right"`, `"from-top"`, `"from-bottom"`
## Timing options
```tsx
import { linearTiming, springTiming } from "@remotion/transitions";
// Linear timing - constant speed
linearTiming({ durationInFrames: 20 });
// Spring timing - organic motion
springTiming({ config: { damping: 200 }, durationInFrames: 25 });
```
## Duration calculation
Transitions overlap adjacent scenes, so the total composition length is **shorter** than the sum of all sequence durations. Overlays do **not** affect the total duration.
For example, with two 60-frame sequences and a 15-frame transition:
- Without transitions: `60 + 60 = 120` frames
- With transition: `60 + 60 - 15 = 105` frames
Adding an overlay between two other sequences does not change the total.
### Getting the duration of a transition
Use the `getDurationInFrames()` method on the timing object:
```tsx
import { linearTiming, springTiming } from "@remotion/transitions";
const linearDuration = linearTiming({
durationInFrames: 20,
}).getDurationInFrames({ fps: 30 });
// Returns 20
const springDuration = springTiming({
config: { damping: 200 },
}).getDurationInFrames({ fps: 30 });
// Returns calculated duration based on spring physics
```
For `springTiming` without an explicit `durationInFrames`, the duration depends on `fps` because it calculates when the spring animation settles.
### Calculating total composition duration
```tsx
import { linearTiming } from "@remotion/transitions";
const scene1Duration = 60;
const scene2Duration = 60;
const scene3Duration = 60;
const timing1 = linearTiming({ durationInFrames: 15 });
const timing2 = linearTiming({ durationInFrames: 20 });
const transition1Duration = timing1.getDurationInFrames({ fps: 30 });
const transition2Duration = timing2.getDurationInFrames({ fps: 30 });
const totalDuration =
scene1Duration +
scene2Duration +
scene3Duration -
transition1Duration -
transition2Duration;
// 60 + 60 + 60 - 15 - 20 = 145 frames
```
.agents/skills/remotion-best-practices/rules/transparent-videos.md
+106
View File
@@ -0,0 +1,106 @@
---
name: transparent-videos
description: Rendering transparent videos in Remotion
metadata:
tags: transparent, alpha, codec, vp9, prores, webm
---
# Rendering Transparent Videos
Remotion can render transparent videos in two ways: as a ProRes video or as a WebM video.
## Transparent ProRes
Ideal for when importing into video editing software.
**CLI:**
```bash
npx remotion render --image-format=png --pixel-format=yuva444p10le --codec=prores --prores-profile=4444 MyComp out.mov
```
**Default in Studio** (restart Studio after changing):
```ts
// remotion.config.ts
import { Config } from "@remotion/cli/config";
Config.setVideoImageFormat("png");
Config.setPixelFormat("yuva444p10le");
Config.setCodec("prores");
Config.setProResProfile("4444");
```
**Setting it as the default export settings for a composition** (using `calculateMetadata`):
```tsx
import { CalculateMetadataFunction } from "remotion";
const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
}) => {
return {
defaultCodec: "prores",
defaultVideoImageFormat: "png",
defaultPixelFormat: "yuva444p10le",
defaultProResProfile: "4444",
};
};
<Composition
id="my-video"
component={MyVideo}
durationInFrames={150}
fps={30}
width={1920}
height={1080}
calculateMetadata={calculateMetadata}
/>;
```
## Transparent WebM (VP9)
Ideal for when playing in a browser.
**CLI:**
```bash
npx remotion render --image-format=png --pixel-format=yuva420p --codec=vp9 MyComp out.webm
```
**Default in Studio** (restart Studio after changing):
```ts
// remotion.config.ts
import { Config } from "@remotion/cli/config";
Config.setVideoImageFormat("png");
Config.setPixelFormat("yuva420p");
Config.setCodec("vp9");
```
**Setting it as the default export settings for a composition** (using `calculateMetadata`):
```tsx
import { CalculateMetadataFunction } from "remotion";
const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
}) => {
return {
defaultCodec: "vp8",
defaultVideoImageFormat: "png",
defaultPixelFormat: "yuva420p",
};
};
<Composition
id="my-video"
component={MyVideo}
durationInFrames={150}
fps={30}
width={1920}
height={1080}
calculateMetadata={calculateMetadata}
/>;
```
.agents/skills/remotion-best-practices/rules/trimming.md
+51
View File
@@ -0,0 +1,51 @@
---
name: trimming
description: Trimming patterns for Remotion - cut the beginning or end of animations
metadata:
tags: sequence, trim, clip, cut, offset
---
Use `<Sequence>` with a negative `from` value to trim the start of an animation.
## Trim the Beginning
A negative `from` value shifts time backwards, making the animation start partway through:
```tsx
import { Sequence, useVideoConfig } from "remotion";
const fps = useVideoConfig();
<Sequence from={-0.5 * fps}>
<MyAnimation />
</Sequence>;
```
The animation appears 15 frames into its progress - the first 15 frames are trimmed off.
Inside `<MyAnimation>`, `useCurrentFrame()` starts at 15 instead of 0.
## Trim the End
Use `durationInFrames` to unmount content after a specified duration:
```tsx
<Sequence durationInFrames={1.5 * fps}>
<MyAnimation />
</Sequence>
```
The animation plays for 45 frames, then the component unmounts.
## Trim and Delay
Nest sequences to both trim the beginning and delay when it appears:
```tsx
<Sequence from={30}>
<Sequence from={-15}>
<MyAnimation />
</Sequence>
</Sequence>
```
The inner sequence trims 15 frames from the start, and the outer sequence delays the result by 30 frames.
.agents/skills/remotion-best-practices/rules/videos.md
+171
View File
@@ -0,0 +1,171 @@
---
name: videos
description: Embedding videos in Remotion - trimming, volume, speed, looping, pitch
metadata:
tags: video, media, trim, volume, speed, loop, pitch
---
# Using videos in Remotion
## Prerequisites
First, the @remotion/media package needs to be installed.
If it is not, use the following command:
```bash
npx remotion add @remotion/media # If project uses npm
bunx remotion add @remotion/media # If project uses bun
yarn remotion add @remotion/media # If project uses yarn
pnpm exec remotion add @remotion/media # If project uses pnpm
```
Use `<Video>` from `@remotion/media` to embed videos into your composition.
```tsx
import { Video } from "@remotion/media";
import { staticFile } from "remotion";
export const MyComposition = () => {
return <Video src={staticFile("video.mp4")} />;
};
```
Remote URLs are also supported:
```tsx
<Video src="https://remotion.media/video.mp4" />
```
## Trimming
Use `trimBefore` and `trimAfter` to remove portions of the video. Values are in seconds.
```tsx
const { fps } = useVideoConfig();
return (
<Video
src={staticFile("video.mp4")}
trimBefore={2 * fps} // Skip the first 2 seconds
trimAfter={10 * fps} // End at the 10 second mark
/>
);
```
## Delaying
Wrap the video in a `<Sequence>` to delay when it appears:
```tsx
import { Sequence, staticFile } from "remotion";
import { Video } from "@remotion/media";
const { fps } = useVideoConfig();
return (
<Sequence from={1 * fps}>
<Video src={staticFile("video.mp4")} />
</Sequence>
);
```
The video will appear after 1 second.
## Sizing and Position
Use the `style` prop to control size and position:
```tsx
<Video
src={staticFile("video.mp4")}
style={{
width: 500,
height: 300,
position: "absolute",
top: 100,
left: 50,
objectFit: "cover",
}}
/>
```
## Volume
Set a static volume (0 to 1):
```tsx
<Video src={staticFile("video.mp4")} volume={0.5} />
```
Or use a callback for dynamic volume based on the current frame:
```tsx
import { interpolate } from "remotion";
const { fps } = useVideoConfig();
return (
<Video
src={staticFile("video.mp4")}
volume={(f) =>
interpolate(f, [0, 1 * fps], [0, 1], { extrapolateRight: "clamp" })
}
/>
);
```
Use `muted` to silence the video entirely:
```tsx
<Video src={staticFile("video.mp4")} muted />
```
## Speed
Use `playbackRate` to change the playback speed:
```tsx
<Video src={staticFile("video.mp4")} playbackRate={2} /> {/* 2x speed */}
<Video src={staticFile("video.mp4")} playbackRate={0.5} /> {/* Half speed */}
```
Reverse playback is not supported.
## Looping
Use `loop` to loop the video indefinitely:
```tsx
<Video src={staticFile("video.mp4")} loop />
```
Use `loopVolumeCurveBehavior` to control how the frame count behaves when looping:
- `"repeat"`: Frame count resets to 0 each loop (for `volume` callback)
- `"extend"`: Frame count continues incrementing
```tsx
<Video
src={staticFile("video.mp4")}
loop
loopVolumeCurveBehavior="extend"
volume={(f) => interpolate(f, [0, 300], [1, 0])} // Fade out over multiple loops
/>
```
## Pitch
Use `toneFrequency` to adjust the pitch without affecting speed. Values range from 0.01 to 2:
```tsx
<Video
src={staticFile("video.mp4")}
toneFrequency={1.5} // Higher pitch
/>
<Video
src={staticFile("video.mp4")}
toneFrequency={0.8} // Lower pitch
/>
```
Pitch shifting only works during server-side rendering, not in the Remotion Studio preview or in the `<Player />`.
.agents/skills/remotion-best-practices/rules/voiceover.md
+99
View File
@@ -0,0 +1,99 @@
---
name: voiceover
description: Adding AI-generated voiceover to Remotion compositions using ElevenLabs TTS
metadata:
tags: voiceover, audio, elevenlabs, tts, speech, calculateMetadata, dynamic duration
---
# Adding AI voiceover to a Remotion composition
Use ElevenLabs TTS to generate speech audio per scene, then use [`calculateMetadata`](./calculate-metadata) to dynamically size the composition to match the audio.
## Prerequisites
An **ElevenLabs API key** is required (`ELEVENLABS_API_KEY` environment variable).
**MUST** ask the user for their ElevenLabs API key if `ELEVENLABS_API_KEY` is not set. **MUST NOT** fall back to other TTS tools.
Ensure the environment variable is available when running the generation script:
```bash
node --strip-types generate-voiceover.ts
```
## Generating audio with ElevenLabs
Create a script that reads the config, calls the ElevenLabs API for each scene, and writes MP3 files to the `public/` directory so Remotion can access them via `staticFile()`.
The core API call for a single scene:
```ts title="generate-voiceover.ts"
const response = await fetch(
`https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`,
{
method: "POST",
headers: {
"xi-api-key": process.env.ELEVENLABS_API_KEY!,
"Content-Type": "application/json",
Accept: "audio/mpeg",
},
body: JSON.stringify({
text: "Welcome to the show.",
model_id: "eleven_multilingual_v2",
voice_settings: {
stability: 0.5,
similarity_boost: 0.75,
style: 0.3,
},
}),
},
);
const audioBuffer = Buffer.from(await response.arrayBuffer());
writeFileSync(`public/voiceover/${compositionId}/${scene.id}.mp3`, audioBuffer);
```
## Dynamic composition duration with calculateMetadata
Use [`calculateMetadata`](./calculate-metadata.md) to measure the [audio durations](./get-audio-duration.md) and set the composition length accordingly.
```tsx
import { CalculateMetadataFunction, staticFile } from "remotion";
import { getAudioDuration } from "./get-audio-duration";
const FPS = 30;
const SCENE_AUDIO_FILES = [
"voiceover/my-comp/scene-01-intro.mp3",
"voiceover/my-comp/scene-02-main.mp3",
"voiceover/my-comp/scene-03-outro.mp3",
];
export const calculateMetadata: CalculateMetadataFunction<Props> = async ({
props,
}) => {
const durations = await Promise.all(
SCENE_AUDIO_FILES.map((file) => getAudioDuration(staticFile(file))),
);
const sceneDurations = durations.map((durationInSeconds) => {
return durationInSeconds * FPS;
});
return {
durationInFrames: Math.ceil(sceneDurations.reduce((sum, d) => sum + d, 0)),
};
};
```
The computed `sceneDurations` are passed into the component via a `voiceover` prop so the component knows how long each scene should be.
If the composition uses [`<TransitionSeries>`](./transitions.md), subtract the overlap from total duration: [./transitions.md#calculating-total-composition-duration](./transitions.md#calculating-total-composition-duration)
## Rendering audio in the component
See [audio.md](./audio.md) for more information on how to render audio in the component.
## Delaying audio start
See [audio.md#delaying](./audio.md#delaying) for more information on how to delay the audio start.
CLAUDE.md
+68 -6
View File
@@ -35,13 +35,13 @@ When user says "COM":
1. Increment version in BOTH `package.json` AND `CLAUDE.md` (verify they match with `grep version package.json && grep Version CLAUDE.md`)
2. Run: `git add -A && git commit -m "chore: bump version to X.XXXX" && git push && npm run build && systemctl --user restart codeman-web`
**Version**: 0.1651 (must match `package.json`)
**Version**: 0.1652 (must match `package.json`)
## Project Overview
Codeman is a Claude Code session manager with web interface and autonomous Ralph Loop. Spawns Claude CLI via PTY, streams via SSE, supports respawn cycling for 24+ hour autonomous runs.
**Tech Stack**: TypeScript (ES2022/NodeNext, strict mode), Node.js, Fastify, node-pty, xterm.js
**Tech Stack**: TypeScript (ES2022/NodeNext, strict mode), Node.js, Fastify, node-pty, xterm.js. Supports both Claude Code and OpenCode AI CLIs via pluggable CLI resolvers.
**TypeScript Strictness** (see `tsconfig.json`): `noUnusedLocals`, `noUnusedParameters`, `noImplicitReturns`, `noImplicitOverride`, `noFallthroughCasesInSwitch`, `allowUnreachableCode: false`, `allowUnusedLabels: false`. Note: `src/tui` is excluded from compilation (legacy/deprecated code path).
@@ -128,10 +128,10 @@ journalctl --user -u codeman-web -f
| `src/prompts/*.ts` | Agent prompts (research-agent, planner) |
| `src/templates/claude-md.ts` | CLAUDE.md generation for new cases |
| `src/cli.ts` | Command-line interface handlers |
| `src/web/server.ts` | Fastify REST API + SSE at `/api/events` (~99 routes) |
| `src/web/server.ts` | Fastify REST API + SSE at `/api/events` (~101 routes) |
| `src/web/schemas.ts` | Zod v4 validation schemas with path/env security allowlists |
| `src/web/public/app.js` | Frontend: xterm.js, tab management, subagent windows, mobile support (~17K lines) |
| `src/types.ts` | All TypeScript interfaces (~100 types, ~1400 lines) |
| `src/types.ts` | All TypeScript interfaces (~70 type/interface/enum defs, ~1450 lines) |
**Large files** (>50KB): `app.js`, `ralph-tracker.ts`, `respawn-controller.ts`, `session.ts`, `subagent-watcher.ts` — these contain complex state machines; read `docs/respawn-state-machine.md` before modifying.
@@ -163,6 +163,7 @@ journalctl --user -u codeman-web -f
| `token-validation.ts` | Token count parsing and validation |
| `regex-patterns.ts` | Shared regex patterns for parsing |
| `type-safety.ts` | `assertNever()` for exhaustive switch/case type checking |
| `opencode-cli-resolver.ts` | Resolve OpenCode CLI binary across install paths |
### Data Flow
@@ -191,9 +192,20 @@ journalctl --user -u codeman-web -f
**Subagent-session correlation**: Session parses Task tool output via `BashToolParser` → `SubagentWatcher` discovers new agent → calls `session.findTaskDescriptionNear()` to match description for window title.
### Frontend Files
| File | Purpose |
|------|---------|
| `src/web/public/index.html` | HTML entry point with inline critical CSS and async vendor loading |
| `src/web/public/app.js` | Core UI: xterm.js, tab management, subagent windows, mobile support (~17.5K lines) |
| `src/web/public/styles.css` | Main styling (dark theme, layout, components) |
| `src/web/public/mobile.css` | Responsive overrides for screens <1024px (loaded conditionally via `media` attribute) |
| `src/web/public/upload.html` | Screenshot upload page served at `/upload.html` |
| `src/web/public/vendor/` | Self-hosted xterm.js + addons (eliminates CDN latency) |
### Frontend Architecture (`app.js`)
The frontend is a single 16K-line vanilla JS file with these key systems:
The frontend is a single ~17.5K-line vanilla JS file with these key systems:
| System | Key Classes/Functions | Purpose |
|--------|----------------------|---------|
@@ -215,6 +227,9 @@ The frontend is a single 16K-line vanilla JS file with these key systems:
### Security
- **HTTP Basic Auth**: Optional via `CODEMAN_USERNAME`/`CODEMAN_PASSWORD` env vars
- **Session cookies**: After Basic Auth, a 24h session cookie (`codeman_session`) is issued so credentials aren't re-sent on every request. Active sessions auto-extend. SSE works via same-origin cookie (`EventSource` can't send custom headers).
- **Rate limiting**: 10 failed auth attempts per IP triggers 429 rejection (15-minute decay window). Manual `StaleExpirationMap` counter — no `@fastify/rate-limit` needed.
- **Hook bypass**: `/api/hook-event` POST is exempt from auth — Claude Code hooks curl this from localhost and can't present credentials. Safe: validated by `HookEventSchema`, only triggers broadcasts.
- **CORS**: Restricted to localhost only
- **Security headers**: X-Content-Type-Options, X-Frame-Options, CSP; HSTS if HTTPS
- **Path validation** (`schemas.ts`): Strict allowlist regex, no shell metacharacters, no traversal, must be absolute
@@ -239,7 +254,7 @@ The frontend is a single 16K-line vanilla JS file with these key systems:
### API Route Categories
~99 routes in `server.ts:buildServer()`. Key groups:
~101 routes in `server.ts:buildServer()`. Key groups:
| Group | Prefix | Count | Key endpoints |
|-------|--------|-------|---------------|
@@ -473,3 +488,50 @@ Run `npx vitest run test/memory-leak-prevention.test.ts` to verify patterns.
**Modifying mobile behavior**: Mobile singletons (`MobileDetection`, `KeyboardHandler`, `SwipeHandler`, `KeyboardAccessoryBar`) all have `init()`/`cleanup()` lifecycle. KeyboardHandler uses `visualViewport` API for iOS keyboard detection (100px threshold for address bar drift). All mobile handlers are re-initialized after SSE reconnect to prevent stale closures.
**Adding a file watcher**: Use `ImageWatcher` as a template pattern — chokidar with `awaitWriteFinish`, burst throttling (max 20/10s), debouncing (200ms), and auto-ignore of `node_modules/.git/dist/`.
## Tunnel Setup (Remote Access)
Access Codeman from mobile/remote devices via Cloudflare quick tunnel.
```
Browser → Cloudflare Edge (HTTPS) → cloudflared → localhost:3000
```
**Prerequisites**: `cloudflared` installed (`cloudflared --version`), `CODEMAN_PASSWORD` set in environment.
### Quick Start
```bash
# Via CLI
./scripts/tunnel.sh start # Start tunnel, prints public URL
./scripts/tunnel.sh url # Show current URL
./scripts/tunnel.sh stop # Stop tunnel
# Via web UI: Settings → Tunnel → Toggle On
```
### systemd Service (Persistent)
```bash
# Install and enable
cp scripts/codeman-tunnel.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now codeman-tunnel
# Check logs
journalctl --user -u codeman-tunnel -f
```
### Auth Flow
1. First request → browser shows Basic Auth prompt (username: `admin` or `CODEMAN_USERNAME`)
2. On success → server issues `codeman_session` HttpOnly cookie (24h TTL, auto-extends on activity)
3. Subsequent requests → cookie authenticates silently (no more prompts)
4. SSE works automatically — `EventSource` sends same-origin cookies
5. 10 failed attempts per IP → 429 rate limit (15-minute decay)
### Security Requirements
- **Always set `CODEMAN_PASSWORD`** before exposing via tunnel — without it, anyone with the URL has full access
- Session cookies are `Secure` when using `--https` flag; through Cloudflare tunnel without `--https`, cookies are non-Secure but traffic is still encrypted end-to-end via Cloudflare
- `/api/hook-event` bypasses auth (localhost-only Claude Code hooks need unauthenticated access)
docs/images/subagent-demo-opt.gif
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.3 MiB

out/codeman-demo.mp4
BIN
View File
Binary file not shown.
package-lock.json
Generated
+2884 -39
View File
File diff suppressed because it is too large Load Diff
package.json
+7 -1
View File
@@ -1,6 +1,6 @@
{
"name": "codeman",
"version": "0.1651",
"version": "0.1652",
"description": "The missing control plane for AI coding agents - run 20 autonomous agents with real-time monitoring and session persistence",
"type": "module",
"main": "dist/index.js",
@@ -38,12 +38,17 @@
"license": "MIT",
"dependencies": {
"@fastify/compress": "^8.3.1",
"@fastify/cookie": "^11.0.2",
"@fastify/static": "^8.0.0",
"@remotion/cli": "4.0.429",
"@remotion/transitions": "4.0.429",
"chalk": "^5.3.0",
"chokidar": "^3.6.0",
"commander": "^12.1.0",
"fastify": "^5.1.0",
"node-pty": "^1.1.0",
"qrcode": "^1.5.4",
"remotion": "4.0.429",
"uuid": "^10.0.0",
"xterm": "^5.3.0",
"xterm-addon-fit": "^0.8.0",
@@ -54,6 +59,7 @@
"devDependencies": {
"@types/node": "^20.19.33",
"@types/pngjs": "^6.0.5",
"@types/react": "^19.2.14",
"@types/uuid": "^10.0.0",
"@vitest/coverage-v8": "^4.0.18",
"agent-browser": "^0.6.0",
remotion/Root.tsx
+16
View File
@@ -0,0 +1,16 @@
import React from 'react';
import { Composition } from 'remotion';
import { CodemanDemo, TOTAL_FRAMES } from './compositions/CodemanDemo';
export const RemotionRoot: React.FC = () => {
return (
<Composition
id="CodemanDemo"
component={CodemanDemo}
durationInFrames={TOTAL_FRAMES}
fps={30}
width={1920}
height={1080}
/>
);
};
remotion/components/ClickCursor.tsx
+108
View File
@@ -0,0 +1,108 @@
import React from 'react';
import {
interpolate,
spring,
useCurrentFrame,
useVideoConfig,
} from 'remotion';
type ClickCursorProps = {
/** X position to move to */
x: number;
/** Y position to move to */
y: number;
/** Frame at which cursor starts moving (local frame) */
moveStart: number;
/** Frame at which click happens */
clickAt: number;
/** Starting X (defaults to center-right) */
fromX?: number;
/** Starting Y */
fromY?: number;
};
export const ClickCursor: React.FC<ClickCursorProps> = ({
x,
y,
moveStart,
clickAt,
fromX = 960,
fromY = 400,
}) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
// Move animation (spring-based)
const moveProgress = spring({
frame: Math.max(0, frame - moveStart),
fps,
config: { damping: 200 },
durationInFrames: Math.max(1, clickAt - moveStart),
});
const cursorX = interpolate(moveProgress, [0, 1], [fromX, x]);
const cursorY = interpolate(moveProgress, [0, 1], [fromY, y]);
// Click pulse
const clickProgress =
frame >= clickAt
? interpolate(frame - clickAt, [0, 8], [0, 1], {
extrapolateRight: 'clamp',
})
: 0;
const clickScale = interpolate(clickProgress, [0, 0.5, 1], [0, 1.2, 0], {
extrapolateRight: 'clamp',
});
const clickOpacity = interpolate(clickProgress, [0, 0.5, 1], [0, 0.6, 0], {
extrapolateRight: 'clamp',
});
// Hide cursor before it starts moving
const cursorOpacity =
frame < moveStart
? 0
: interpolate(frame - moveStart, [0, 5], [0, 1], {
extrapolateRight: 'clamp',
});
return (
<div
style={{
position: 'absolute',
left: cursorX,
top: cursorY,
opacity: cursorOpacity,
zIndex: 100,
pointerEvents: 'none',
}}
>
{/* Click pulse ring */}
<div
style={{
position: 'absolute',
width: 30,
height: 30,
borderRadius: '50%',
border: '2px solid rgba(255,255,255,0.5)',
transform: `translate(-50%, -50%) scale(${clickScale})`,
opacity: clickOpacity,
}}
/>
{/* Cursor arrow */}
<svg
width="20"
height="24"
viewBox="0 0 20 24"
fill="none"
style={{ filter: 'drop-shadow(1px 2px 3px rgba(0,0,0,0.5))' }}
>
<path
d="M2 2L2 20L7 15L12 22L15 20L10 13L17 13L2 2Z"
fill="white"
stroke="black"
strokeWidth="1.5"
/>
</svg>
</div>
);
};
remotion/components/PhoneFrame.tsx
+85
View File
@@ -0,0 +1,85 @@
import React from 'react';
import { spring, useCurrentFrame, useVideoConfig } from 'remotion';
import { colors } from '../lib/theme';
type PhoneFrameProps = {
children: React.ReactNode;
/** Width of the phone viewport */
width?: number;
/** Height of the phone viewport */
height?: number;
};
export const PhoneFrame: React.FC<PhoneFrameProps> = ({
children,
width = 375,
height = 812,
}) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
// Scale-in animation
const scale = spring({
frame,
fps,
config: { damping: 15, stiffness: 80 },
});
const bezelW = width + 24;
const bezelH = height + 24;
return (
<div
style={{
display: 'flex',
justifyContent: 'center',
alignItems: 'center',
width: '100%',
height: '100%',
transform: `scale(${scale})`,
}}
>
<div
style={{
width: bezelW,
height: bezelH,
borderRadius: 44,
background: '#1a1a1a',
border: '2px solid #333',
padding: 12,
position: 'relative',
boxShadow: '0 20px 60px rgba(0,0,0,0.6)',
}}
>
{/* Notch / Dynamic Island */}
<div
style={{
position: 'absolute',
top: 10,
left: '50%',
transform: 'translateX(-50%)',
width: 120,
height: 28,
borderRadius: 14,
background: '#000',
zIndex: 20,
}}
/>
{/* Screen content */}
<div
style={{
width,
height,
borderRadius: 32,
overflow: 'hidden',
background: colors.bg.dark,
position: 'relative',
}}
>
{children}
</div>
</div>
</div>
);
};
remotion/components/TitleCard.tsx
+118
View File
@@ -0,0 +1,118 @@
import React from 'react';
import {
AbsoluteFill,
interpolate,
useCurrentFrame,
useVideoConfig,
} from 'remotion';
import { colors, fonts } from '../lib/theme';
type TitleCardProps = {
text: string;
subtitle: string;
showUrl?: boolean;
};
export const TitleCard: React.FC<TitleCardProps> = ({
text,
subtitle,
showUrl = false,
}) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
// Logo fade + scale in
const logoOpacity = interpolate(frame, [0, 0.5 * fps], [0, 1], {
extrapolateRight: 'clamp',
});
const logoScale = interpolate(frame, [0, 0.5 * fps], [0.8, 1], {
extrapolateRight: 'clamp',
});
// Subtitle fades in slightly after logo
const subtitleOpacity = interpolate(
frame,
[0.3 * fps, 0.8 * fps],
[0, 1],
{ extrapolateLeft: 'clamp', extrapolateRight: 'clamp' },
);
// URL fades in last
const urlOpacity = showUrl
? interpolate(frame, [0.6 * fps, 1.1 * fps], [0, 1], {
extrapolateLeft: 'clamp',
extrapolateRight: 'clamp',
})
: 0;
return (
<AbsoluteFill
style={{
backgroundColor: colors.bg.dark,
justifyContent: 'center',
alignItems: 'center',
}}
>
{/* Lightning bolt icon */}
<div
style={{
opacity: logoOpacity,
transform: `scale(${logoScale})`,
display: 'flex',
flexDirection: 'column',
alignItems: 'center',
gap: 16,
}}
>
<svg width="64" height="64" viewBox="0 0 32 32">
<defs>
<linearGradient id="g" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" stopColor="#60a5fa" />
<stop offset="100%" stopColor="#3b82f6" />
</linearGradient>
</defs>
<rect width="32" height="32" rx="6" fill="#0a0a0a" />
<path d="M18 4L8 18h6l-2 10 10-14h-6z" fill="url(#g)" />
</svg>
<div
style={{
fontFamily: fonts.ui,
fontSize: 72,
fontWeight: 700,
color: colors.accent.blue,
letterSpacing: -1,
}}
>
{text}
</div>
</div>
<div
style={{
opacity: subtitleOpacity,
fontFamily: fonts.ui,
fontSize: 28,
color: colors.text.dim,
marginTop: 12,
}}
>
{subtitle}
</div>
{showUrl && (
<div
style={{
opacity: urlOpacity,
fontFamily: fonts.mono,
fontSize: 18,
color: colors.text.muted,
marginTop: 24,
}}
>
github.com/Ark0N/Codeman
</div>
)}
</AbsoluteFill>
);
};
remotion/compositions/CodemanDemo.tsx
+485
View File
@@ -0,0 +1,485 @@
import React from 'react';
import {
AbsoluteFill,
Img,
interpolate,
Sequence,
spring,
staticFile,
useCurrentFrame,
useVideoConfig,
} from 'remotion';
import { colors } from '../lib/theme';
import { TitleCard } from '../components/TitleCard';
import { PhoneFrame } from '../components/PhoneFrame';
import { ClickCursor } from '../components/ClickCursor';
// ─── Scene timing (frames @ 30fps) ───
const TITLE_START = 0;
const TITLE_DUR = 45;
const DESKTOP_START = 45;
const WELCOME_DUR = 75; // 2.5s — show welcome screen
const CLAUDE_DUR = 90; // 3s — single Claude tab
const BOTH_OPENCODE_DUR = 90; // 3s — both tabs, OpenCode active
const TAB_SWITCH_DUR = 120; // 4s — tab switching animation
const DESKTOP_DUR = WELCOME_DUR + CLAUDE_DUR + BOTH_OPENCODE_DUR + TAB_SWITCH_DUR; // 375
const TRANSITION_START = DESKTOP_START + DESKTOP_DUR; // 420
const TRANSITION_DUR = 60;
const MOBILE_START = TRANSITION_START + TRANSITION_DUR; // 480
const MOBILE_DUR = 240;
const OUTRO_START = MOBILE_START + MOBILE_DUR; // 720
const OUTRO_DUR = 75;
export const TOTAL_FRAMES = OUTRO_START + OUTRO_DUR; // 795
// ─── Screenshot paths ───
const SCREENSHOTS = {
desktopWelcome: staticFile('desktop-welcome.png'),
desktopClaude: staticFile('desktop-claude.png'),
desktopBothClaude: staticFile('desktop-both-claude.png'),
desktopBothOpencode: staticFile('desktop-both-opencode.png'),
mobileClaude: staticFile('mobile-claude.png'),
mobileOpencode: staticFile('mobile-opencode.png'),
};
// ─── Desktop scene ───
const DesktopDemo: React.FC = () => {
const frame = useCurrentFrame();
// Key frames within the desktop scene (local frames)
const claudeClickFrame = WELCOME_DUR - 15; // cursor clicks near end of welcome
const claudeStart = WELCOME_DUR;
const bothStart = WELCOME_DUR + CLAUDE_DUR; // both tabs appear
const switchBase = bothStart + BOTH_OPENCODE_DUR;
const switch1Frame = switchBase + 30; // switch to Claude
const switch2Frame = switchBase + 75; // switch back to OpenCode
// Determine which screenshot to show and cross-fade
// Phase 1: Welcome (0 → WELCOME_DUR)
// Phase 2: Claude single tab (WELCOME_DUR → bothStart)
// Phase 3: Both tabs, OpenCode active (bothStart → switchBase)
// Phase 4: Tab switching — OpenCode→Claude→OpenCode (switchBase → end)
// Cross-fade durations
const FADE = 10;
// Welcome → Claude cross-fade
const welcomeOpacity =
frame < claudeStart - FADE
? 1
: frame < claudeStart
? interpolate(frame, [claudeStart - FADE, claudeStart], [1, 0], {
extrapolateRight: 'clamp',
})
: 0;
const claudeSoloOpacity =
frame < claudeStart
? 0
: frame < claudeStart + FADE
? interpolate(frame, [claudeStart, claudeStart + FADE], [0, 1], {
extrapolateRight: 'clamp',
})
: frame < bothStart - FADE
? 1
: frame < bothStart
? interpolate(frame, [bothStart - FADE, bothStart], [1, 0], {
extrapolateRight: 'clamp',
})
: 0;
// Both tabs — determine which one is active
let bothOpenCodeOpacity = 0;
let bothClaudeOpacity = 0;
if (frame >= bothStart) {
// Fade in both-opencode initially
if (frame < bothStart + FADE) {
bothOpenCodeOpacity = interpolate(
frame,
[bothStart, bothStart + FADE],
[0, 1],
{ extrapolateRight: 'clamp' },
);
} else if (frame < switch1Frame) {
// Showing OpenCode
bothOpenCodeOpacity = 1;
bothClaudeOpacity = 0;
} else if (frame < switch1Frame + FADE) {
// Cross-fade OpenCode → Claude
const t = interpolate(
frame,
[switch1Frame, switch1Frame + FADE],
[0, 1],
{ extrapolateRight: 'clamp' },
);
bothOpenCodeOpacity = 1 - t;
bothClaudeOpacity = t;
} else if (frame < switch2Frame) {
// Showing Claude
bothOpenCodeOpacity = 0;
bothClaudeOpacity = 1;
} else if (frame < switch2Frame + FADE) {
// Cross-fade Claude → OpenCode
const t = interpolate(
frame,
[switch2Frame, switch2Frame + FADE],
[0, 1],
{ extrapolateRight: 'clamp' },
);
bothOpenCodeOpacity = t;
bothClaudeOpacity = 1 - t;
} else {
// Back to OpenCode
bothOpenCodeOpacity = 1;
bothClaudeOpacity = 0;
}
}
// Welcome button positions for cursor (measured from actual screenshot)
const welcomeBtnClaudeX = 651;
const welcomeBtnClaudeY = 434;
const welcomeBtnOpenCodeX = 803;
const welcomeBtnOpenCodeY = 434;
return (
<AbsoluteFill style={{ background: colors.bg.dark }}>
{/* Welcome screenshot */}
{welcomeOpacity > 0 && (
<AbsoluteFill style={{ opacity: welcomeOpacity }}>
<Img
src={SCREENSHOTS.desktopWelcome}
style={{ width: 1920, height: 1080 }}
/>
</AbsoluteFill>
)}
{/* Single Claude tab screenshot */}
{claudeSoloOpacity > 0 && (
<AbsoluteFill style={{ opacity: claudeSoloOpacity }}>
<Img
src={SCREENSHOTS.desktopClaude}
style={{ width: 1920, height: 1080 }}
/>
</AbsoluteFill>
)}
{/* Both tabs — OpenCode active */}
{bothOpenCodeOpacity > 0 && (
<AbsoluteFill style={{ opacity: bothOpenCodeOpacity }}>
<Img
src={SCREENSHOTS.desktopBothOpencode}
style={{ width: 1920, height: 1080 }}
/>
</AbsoluteFill>
)}
{/* Both tabs — Claude active */}
{bothClaudeOpacity > 0 && (
<AbsoluteFill style={{ opacity: bothClaudeOpacity }}>
<Img
src={SCREENSHOTS.desktopBothClaude}
style={{ width: 1920, height: 1080 }}
/>
</AbsoluteFill>
)}
{/* Cursor: click Claude Code button on welcome screen */}
<Sequence
from={claudeClickFrame - 20}
durationInFrames={35}
layout="none"
>
<ClickCursor
fromX={960}
fromY={300}
x={welcomeBtnClaudeX}
y={welcomeBtnClaudeY}
moveStart={0}
clickAt={20}
/>
</Sequence>
{/* Cursor: click to add OpenCode tab */}
<Sequence
from={bothStart - 20}
durationInFrames={35}
layout="none"
>
<ClickCursor
fromX={welcomeBtnClaudeX}
fromY={welcomeBtnClaudeY}
x={welcomeBtnOpenCodeX}
y={welcomeBtnOpenCodeY}
moveStart={0}
clickAt={20}
/>
</Sequence>
{/* Cursor: switch to Claude tab (first tab in header) */}
<Sequence
from={switch1Frame - 15}
durationInFrames={20}
layout="none"
>
<ClickCursor
fromX={500}
fromY={300}
x={128}
y={17}
moveStart={0}
clickAt={15}
/>
</Sequence>
{/* Cursor: switch back to OpenCode tab (second tab in header) */}
<Sequence
from={switch2Frame - 15}
durationInFrames={20}
layout="none"
>
<ClickCursor
fromX={128}
fromY={17}
x={245}
y={17}
moveStart={0}
clickAt={15}
/>
</Sequence>
</AbsoluteFill>
);
};
// ─── Device transition ───
const DeviceTransition: React.FC = () => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
// Desktop shrinks down
const shrinkProgress = spring({
frame,
fps,
config: { damping: 200 },
durationInFrames: 45,
});
const desktopScale = interpolate(shrinkProgress, [0, 1], [1, 0.35]);
const desktopOpacity = interpolate(shrinkProgress, [0, 1], [1, 0], {
extrapolateRight: 'clamp',
});
// Phone frame fades in
const phoneOpacity = interpolate(frame, [20, 40], [0, 1], {
extrapolateLeft: 'clamp',
extrapolateRight: 'clamp',
});
const phoneScale = interpolate(frame, [20, 50], [0.8, 1], {
extrapolateLeft: 'clamp',
extrapolateRight: 'clamp',
});
return (
<AbsoluteFill
style={{
background: colors.bg.dark,
justifyContent: 'center',
alignItems: 'center',
}}
>
{/* Shrinking desktop screenshot */}
<div
style={{
position: 'absolute',
width: 1920,
height: 1080,
transform: `scale(${desktopScale})`,
opacity: desktopOpacity,
}}
>
<Img
src={SCREENSHOTS.desktopBothClaude}
style={{ width: 1920, height: 1080 }}
/>
</div>
{/* Phone frame appearing */}
<div
style={{
opacity: phoneOpacity,
transform: `scale(${phoneScale})`,
}}
>
<PhoneFrame width={375} height={700}>
<Img
src={SCREENSHOTS.mobileClaude}
style={{ width: 375, height: 700, objectFit: 'cover' }}
/>
</PhoneFrame>
</div>
</AbsoluteFill>
);
};
// ─── Mobile scene ───
const MobileDemo: React.FC = () => {
const frame = useCurrentFrame();
// Mobile: Claude for 90 frames, then swipe to OpenCode
const swipeFrame = 90;
const swipeDuration = 20;
const isOpenCode = frame >= swipeFrame + swipeDuration;
const isSwiping =
frame >= swipeFrame && frame < swipeFrame + swipeDuration;
// Slide offset
const slideProgress = isSwiping
? interpolate(frame - swipeFrame, [0, swipeDuration], [0, 1], {
extrapolateRight: 'clamp',
})
: isOpenCode
? 1
: 0;
const slideX = interpolate(slideProgress, [0, 1], [0, -375]);
return (
<PhoneFrame width={375} height={700}>
<div
style={{
width: 375,
height: 700,
overflow: 'hidden',
position: 'relative',
}}
>
{/* Sliding container with both screenshots */}
<div
style={{
display: 'flex',
width: 750,
height: 700,
transform: `translateX(${slideX}px)`,
}}
>
<Img
src={SCREENSHOTS.mobileClaude}
style={{ width: 375, height: 700, flexShrink: 0, objectFit: 'cover' }}
/>
<Img
src={SCREENSHOTS.mobileOpencode}
style={{ width: 375, height: 700, flexShrink: 0, objectFit: 'cover' }}
/>
</div>
{/* Swipe indicator */}
{isSwiping && (
<SwipeIndicator
frame={frame - swipeFrame}
duration={swipeDuration}
/>
)}
</div>
</PhoneFrame>
);
};
// Swipe gesture indicator
const SwipeIndicator: React.FC<{ frame: number; duration: number }> = ({
frame,
duration,
}) => {
const progress = interpolate(frame, [0, duration], [0, 1], {
extrapolateRight: 'clamp',
});
const x = interpolate(progress, [0, 1], [300, 75]);
const opacity = interpolate(
progress,
[0, 0.2, 0.8, 1],
[0, 0.7, 0.7, 0],
{ extrapolateRight: 'clamp' },
);
return (
<div
style={{
position: 'absolute',
bottom: 60,
left: x,
width: 40,
height: 40,
borderRadius: '50%',
background: 'rgba(255,255,255,0.15)',
border: '2px solid rgba(255,255,255,0.3)',
opacity,
zIndex: 20,
}}
/>
);
};
// ─── Main composition ───
export const CodemanDemo: React.FC = () => {
return (
<AbsoluteFill style={{ background: colors.bg.dark }}>
{/* Scene 1: Title */}
<Sequence durationInFrames={TITLE_DUR} premountFor={0}>
<TitleCard text="Codeman" subtitle="AI Session Manager" />
</Sequence>
{/* Scene 2-5: Desktop flow (cross-fading screenshots) */}
<Sequence
from={DESKTOP_START}
durationInFrames={DESKTOP_DUR}
premountFor={15}
>
<DesktopDemo />
</Sequence>
{/* Scene 6: Desktop → Mobile transition */}
<Sequence
from={TRANSITION_START}
durationInFrames={TRANSITION_DUR}
premountFor={15}
>
<DeviceTransition />
</Sequence>
{/* Scene 7-9: Mobile flow (swipe between Claude/OpenCode) */}
<Sequence
from={MOBILE_START}
durationInFrames={MOBILE_DUR}
premountFor={15}
>
<AbsoluteFill
style={{
background: colors.bg.dark,
justifyContent: 'center',
alignItems: 'center',
}}
>
<MobileDemo />
</AbsoluteFill>
</Sequence>
{/* Scene 10: Outro */}
<Sequence
from={OUTRO_START}
durationInFrames={OUTRO_DUR}
premountFor={15}
>
<TitleCard
text="Codeman"
subtitle="Manage any AI coding tool"
showUrl
/>
</Sequence>
</AbsoluteFill>
);
};
remotion/index.ts
+4
View File
@@ -0,0 +1,4 @@
import { registerRoot } from 'remotion';
import { RemotionRoot } from './Root';
registerRoot(RemotionRoot);
remotion/lib/theme.ts
+42
View File
@@ -0,0 +1,42 @@
// Design tokens extracted from Codeman's styles.css and index.html
export const colors = {
bg: {
dark: '#0a0a0a',
card: '#141414',
input: '#1a1a1a',
hover: '#222',
terminal: '#0d0d0d',
},
border: {
default: '#2a2a2a',
light: '#333',
},
text: {
primary: '#eee',
dim: '#888',
muted: '#555',
},
accent: {
blue: '#3b82f6',
blueHover: '#60a5fa',
green: '#22c55e',
yellow: '#eab308',
red: '#ef4444',
purple: '#a855f7',
orange: '#f97316',
},
} as const;
export const layout = {
headerHeight: 36,
toolbarHeight: 40,
tabHeight: 28,
borderRadius: 4,
} as const;
export const fonts = {
ui: "-apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif",
mono: "'SF Mono', 'Cascadia Code', 'Fira Code', 'JetBrains Mono', Menlo, monospace",
} as const;
remotion/public/desktop-both-claude.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 41 KiB

remotion/public/desktop-both-opencode.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 37 KiB

remotion/public/desktop-claude.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 39 KiB

remotion/public/desktop-welcome.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 57 KiB

remotion/public/mobile-claude.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 21 KiB

remotion/public/mobile-opencode.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 22 KiB

scripts/capture-video-screenshots.mjs
+731
View File
@@ -0,0 +1,731 @@
#!/usr/bin/env node
/**
* capture-video-screenshots.mjs
*
* Captures real Codeman UI screenshots for the Remotion demo video.
* Uses Playwright + page.route() mock injection (same pattern as capture-readme-screenshots.mjs).
* No real Claude CLI or server needed — all API responses are mocked.
*
* Usage: node scripts/capture-video-screenshots.mjs
* Port: 3198 (static file server)
* Output: remotion/public/ (6 PNGs)
*/
import { chromium } from 'playwright';
import { createServer } from 'http';
import { readFileSync, existsSync } from 'fs';
import { join, extname } from 'path';
import { fileURLToPath } from 'url';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
const PROJECT_ROOT = join(__dirname, '..');
const PUBLIC_DIR = join(PROJECT_ROOT, 'src', 'web', 'public');
const OUTPUT_DIR = join(PROJECT_ROOT, 'remotion', 'public');
const PORT = 3198;
const DESKTOP_VIEWPORT = { width: 1920, height: 1080 };
const MOBILE_VIEWPORT = { width: 390, height: 844 };
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
// ─── MIME Types ──────────────────────────────────────────────────────────────
const MIME_TYPES = {
'.html': 'text/html',
'.js': 'text/javascript',
'.css': 'text/css',
'.json': 'application/json',
'.png': 'image/png',
'.svg': 'image/svg+xml',
'.ico': 'image/x-icon',
'.woff2': 'font/woff2',
'.woff': 'font/woff',
'.ttf': 'font/ttf',
};
// ─── Static File Server ──────────────────────────────────────────────────────
function startStaticServer() {
return new Promise((resolve) => {
const server = createServer((req, res) => {
let urlPath = req.url.split('?')[0];
if (urlPath === '/') urlPath = '/index.html';
const filePath = join(PUBLIC_DIR, urlPath);
if (!existsSync(filePath)) {
res.writeHead(404);
res.end('Not Found');
return;
}
try {
const data = readFileSync(filePath);
const ext = extname(filePath);
res.writeHead(200, {
'Content-Type': MIME_TYPES[ext] || 'application/octet-stream',
'Cache-Control': 'no-cache',
});
res.end(data);
} catch {
res.writeHead(500);
res.end('Internal Server Error');
}
});
server.listen(PORT, () => {
console.log(`Static server on http://localhost:${PORT}`);
resolve(server);
});
});
}
// ─── Mock Data ───────────────────────────────────────────────────────────────
function makeSession(id, name, mode, status, extra = {}) {
return {
id,
pid: status === 'idle' ? null : 12345 + Math.floor(Math.random() * 10000),
status,
workingDir: '/home/user/my-project',
currentTaskId: null,
createdAt: Date.now() - 3600000,
lastActivityAt: Date.now() - (status === 'idle' ? 60000 : 5000),
name,
mode,
autoClearEnabled: false,
autoClearThreshold: 140000,
autoCompactEnabled: false,
autoCompactThreshold: 110000,
autoCompactPrompt: '',
imageWatcherEnabled: false,
totalCost: mode === 'claude' ? 0.12 : 0,
inputTokens: mode === 'claude' ? 18000 : 0,
outputTokens: mode === 'claude' ? 11500 : 0,
ralphEnabled: false,
niceEnabled: false,
niceValue: 10,
color: 'default',
flickerFilterEnabled: false,
cliVersion: '2.1.61',
cliModel: mode === 'claude' ? 'Opus 4.6' : 'GPT-4o',
cliAccountType: mode === 'claude' ? 'Claude Max' : 'API Key',
cliLatestVersion: '2.1.61',
messageCount: 15,
isWorking: status === 'busy',
lastPromptTime: Date.now() - 30000,
bufferStats: {
terminalBufferSize: 4096,
textOutputSize: 2048,
messageCount: 15,
maxTerminalBuffer: 2097152,
maxTextOutput: 1048576,
maxMessages: 1000,
},
taskStats: { total: 0, running: 0, completed: 0, failed: 0 },
taskTree: [],
tokens: {
input: mode === 'claude' ? 18000 : 0,
output: mode === 'claude' ? 11500 : 0,
total: mode === 'claude' ? 29500 : 0,
},
autoClear: { enabled: false, threshold: 140000 },
nice: { enabled: false, niceValue: 10 },
ralphLoop: null,
ralphTodos: [],
ralphTodoStats: { total: 0, completed: 0, percentComplete: 0 },
respawnEnabled: false,
respawnConfig: null,
respawn: null,
claudeSessionId: `claude-${id}`,
...extra,
};
}
function buildInitPayload(sessions) {
const respawnStatus = {};
for (const s of sessions) {
respawnStatus[s.id] = {
state: 'idle',
cycleCount: 0,
lastActivityTime: Date.now(),
timeSinceActivity: 0,
promptDetected: false,
workingDetected: false,
detection: {},
config: null,
};
}
return {
version: '0.1651',
sessions,
scheduledRuns: [],
respawnStatus,
globalStats: {
totalInputTokens: 145000,
totalOutputTokens: 87000,
totalCost: 1.82,
totalSessionsCreated: 12,
firstRecordedAt: Date.now() - 86400000,
lastUpdatedAt: Date.now(),
},
subagents: [],
timestamp: Date.now(),
};
}
// ─── Terminal Content (ANSI) ─────────────────────────────────────────────────
const RST = '\x1b[0m';
const RED = '\x1b[31m';
const GRN = '\x1b[32m';
const YEL = '\x1b[33m';
const MAG = '\x1b[35m';
const GRY = '\x1b[90m';
const BOLD = '\x1b[1m';
const DIM = '\x1b[2m';
// Claude Code init banner
const TERMINAL_CLAUDE = [
'',
` ${RED}████${RST} ${BOLD}Claude Code${RST} v2.1.61`,
` ${RED}████${RST} ${GRY}Opus 4.6${RST} ${DIM}·${RST} ${GRY}Claude Max${RST}`,
` ${RED}██${RST} ${RED}██${RST} ${GRY}~/my-project${RST}`,
'',
'',
`${GRY}❯ Try "how do I log an error?"${RST}`,
'',
'',
`${YEL}»»${RST} ${BOLD}bypass permissions on${RST} ${GRY}(shift+tab to cycle)${RST}`,
` ${BOLD}0 tokens${RST}`,
` ${GRY}current: 2.1.61${RST} ${DIM}·${RST} ${GRY}latest: 2.1.61${RST}`,
].join('\r\n');
// OpenCode init banner
const TERMINAL_OPENCODE = [
'',
` ${GRN}████${RST} ${BOLD}OpenCode${RST} v0.3.12`,
` ${GRN}████${RST} ${GRY}GPT-4o${RST} ${DIM}·${RST} ${GRY}API Key${RST}`,
` ${GRN}██${RST} ${GRN}██${RST} ${GRY}~/my-project${RST}`,
'',
'',
`${GRY}❯ Try "explain this codebase"${RST}`,
'',
'',
`${YEL}»»${RST} ${BOLD}bypass permissions on${RST}`,
` ${BOLD}0 tokens${RST}`,
].join('\r\n');
// ─── Route Interceptors ──────────────────────────────────────────────────────
async function setupRoutes(page, initPayload, terminalContent) {
// Block SSE to prevent reconnection loops
await page.route('**/api/events', async (route) => {
await route.abort();
});
// Terminal buffer endpoint
await page.route('**/api/sessions/*/terminal**', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
terminalBuffer: terminalContent,
status: 'busy',
fullSize: terminalContent.length,
truncated: false,
}),
});
});
// Mux sessions
await page.route('**/api/mux-sessions/**', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ success: true }),
});
});
await page.route('**/api/mux-sessions', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ sessions: [], muxAvailable: true }),
});
});
// Settings
await page.route('**/api/settings', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
showSubagents: false,
subagentTrackingEnabled: false,
subagentActiveTabOnly: false,
showMonitor: false,
}),
});
});
// Subagent window states
await page.route('**/api/subagent-window-states', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({}),
});
});
// Subagent parents
await page.route('**/api/subagent-parents', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({}),
});
});
// Session-specific subagents
await page.route('**/api/sessions/*/subagents', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ success: true, data: [] }),
});
});
// Interactive attach
await page.route('**/api/sessions/*/interactive', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ success: true }),
});
});
// Resize
await page.route('**/api/sessions/*/resize', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ success: true }),
});
});
// Catch-all API
await page.route('**/api/**', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ success: true }),
});
});
}
/**
* Inject mock state and render the app.
*/
async function injectState(page, initPayload, terminalContent, activeSessionId) {
await page.waitForFunction(() => window.app && window.app.terminal, { timeout: 15000 });
await sleep(2000);
// Cancel SSE fallback timer and inject state
await page.evaluate((payload) => {
const app = window.app;
if (app._initFallbackTimer) {
clearTimeout(app._initFallbackTimer);
app._initFallbackTimer = null;
}
app.handleInit(payload);
}, initPayload);
await sleep(3000);
// Select the target session
if (activeSessionId) {
await page.evaluate((sid) => {
window.app.activeSessionId = null;
window.app.selectSession(sid);
}, activeSessionId);
await sleep(4000);
}
// Write terminal content directly as backup
if (terminalContent && activeSessionId) {
await page.evaluate((content) => {
const app = window.app;
if (app.terminal) {
const buf = app.terminal.buffer?.active;
const hasContent = buf && buf.length > 2 && buf.getLine(1)?.translateToString().trim();
if (!hasContent) {
app.terminal.clear();
app.terminal.reset();
app.terminal.write(content);
app.terminal.scrollToBottom();
}
}
}, terminalContent);
await sleep(3000);
}
// Clean up UI artifacts
await page.evaluate(() => {
const app = window.app;
if (app) {
if (app.sseReconnectTimeout) clearTimeout(app.sseReconnectTimeout);
if (app.eventSource) { app.eventSource.close(); app.eventSource = null; }
app._connectionStatus = 'connected';
app._updateConnectionIndicator = () => {};
app.connectSSE = () => {};
app.setConnectionStatus = () => {};
}
// Remove connection indicator
const indicator = document.getElementById('connectionIndicator');
if (indicator) indicator.remove();
// Hide respawn banner
const respawnBanner = document.getElementById('respawnBanner');
if (respawnBanner) respawnBanner.style.display = 'none';
// Mock CPU/MEM stats
const statsEl = document.getElementById('headerSystemStats');
if (statsEl) {
statsEl.innerHTML = `
<span class="stat-label">CPU</span>
<div class="stat-bar"><div class="stat-bar-fill cpu-bar" style="width: 18%"></div></div>
<span class="stat-value">18%</span>
<span class="stat-label">MEM</span>
<div class="stat-bar"><div class="stat-bar-fill mem-bar" style="width: 42%"></div></div>
<span class="stat-value">13.7G</span>
`;
}
// Hide subagents panel entirely
const subagentsPanel = document.getElementById('subagentsPanel');
if (subagentsPanel) subagentsPanel.style.display = 'none';
// Close and hide monitor panel
const monitorPanel = document.getElementById('monitorPanel');
if (monitorPanel) {
monitorPanel.classList.remove('open');
monitorPanel.style.display = 'none';
}
// Hide monitor toggle button in toolbar
const monitorBtn = document.getElementById('monitorToggle');
if (monitorBtn) monitorBtn.style.display = 'none';
// Update token display
if (app && app.activeSessionId) {
const session = app.sessions.get(app.activeSessionId);
if (session) {
const tokens = (session.tokens?.total || 0);
const tokenEl = document.getElementById('headerTokens');
if (tokenEl) {
const formatted = tokens >= 1000 ? `${(tokens / 1000).toFixed(1)}k` : String(tokens);
tokenEl.innerHTML = `<span class="token-icon">⊙</span> <span class="token-count">${formatted} tokens</span>`;
}
}
}
});
// Final settle — let xterm.js WebGL renderer, fonts, and layout fully stabilize
await sleep(4000);
}
// ─── Screenshot Scenarios ────────────────────────────────────────────────────
// 1. Desktop welcome — no sessions, welcome overlay visible
async function captureDesktopWelcome(browser) {
console.log('\n1/6 Capturing desktop-welcome.png...');
const context = await browser.newContext({
viewport: DESKTOP_VIEWPORT,
deviceScaleFactor: 1,
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
try {
const initPayload = buildInitPayload([]);
await setupRoutes(page, initPayload, '');
await page.goto(`http://localhost:${PORT}`, { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => window.app && window.app.terminal, { timeout: 15000 });
await sleep(2000);
// Inject empty state — welcome overlay should show automatically
await page.evaluate((payload) => {
const app = window.app;
if (app._initFallbackTimer) {
clearTimeout(app._initFallbackTimer);
app._initFallbackTimer = null;
}
app.handleInit(payload);
}, initPayload);
await sleep(3000);
// Clean up UI
await page.evaluate(() => {
const app = window.app;
if (app) {
if (app.sseReconnectTimeout) clearTimeout(app.sseReconnectTimeout);
if (app.eventSource) { app.eventSource.close(); app.eventSource = null; }
app._connectionStatus = 'connected';
app._updateConnectionIndicator = () => {};
app.connectSSE = () => {};
app.setConnectionStatus = () => {};
}
const indicator = document.getElementById('connectionIndicator');
if (indicator) indicator.remove();
const respawnBanner = document.getElementById('respawnBanner');
if (respawnBanner) respawnBanner.style.display = 'none';
// Mock stats
const statsEl = document.getElementById('headerSystemStats');
if (statsEl) {
statsEl.innerHTML = `
<span class="stat-label">CPU</span>
<div class="stat-bar"><div class="stat-bar-fill cpu-bar" style="width: 18%"></div></div>
<span class="stat-value">18%</span>
<span class="stat-label">MEM</span>
<div class="stat-bar"><div class="stat-bar-fill mem-bar" style="width: 42%"></div></div>
<span class="stat-value">13.7G</span>
`;
}
// Hide monitor and subagent panels
const subagentsPanel = document.getElementById('subagentsPanel');
if (subagentsPanel) subagentsPanel.style.display = 'none';
const monitorPanel = document.getElementById('monitorPanel');
if (monitorPanel) {
monitorPanel.classList.remove('open');
monitorPanel.style.display = 'none';
}
const monitorBtn = document.getElementById('monitorToggle');
if (monitorBtn) monitorBtn.style.display = 'none';
});
await sleep(3000);
await page.screenshot({
path: join(OUTPUT_DIR, 'desktop-welcome.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-welcome.png');
} finally {
await context.close();
}
}
// 2. Desktop with single Claude tab active
async function captureDesktopClaude(browser) {
console.log('\n2/6 Capturing desktop-claude.png...');
const claudeSession = makeSession('sess-claude-1', 'w1-my-project', 'claude', 'busy');
const initPayload = buildInitPayload([claudeSession]);
const context = await browser.newContext({
viewport: DESKTOP_VIEWPORT,
deviceScaleFactor: 1,
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
try {
await setupRoutes(page, initPayload, TERMINAL_CLAUDE);
await page.goto(`http://localhost:${PORT}`, { waitUntil: 'domcontentloaded' });
await injectState(page, initPayload, TERMINAL_CLAUDE, 'sess-claude-1');
await page.screenshot({
path: join(OUTPUT_DIR, 'desktop-claude.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-claude.png');
} finally {
await context.close();
}
}
// 3. Desktop with 2 tabs (Claude + OpenCode), Claude active
async function captureDesktopBothClaude(browser) {
console.log('\n3/6 Capturing desktop-both-claude.png...');
const claudeSession = makeSession('sess-claude-2', 'w1-my-project', 'claude', 'busy');
const opencodeSession = makeSession('sess-opencode-2', 'w2-my-project', 'opencode', 'busy');
const initPayload = buildInitPayload([claudeSession, opencodeSession]);
const context = await browser.newContext({
viewport: DESKTOP_VIEWPORT,
deviceScaleFactor: 1,
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
try {
await setupRoutes(page, initPayload, TERMINAL_CLAUDE);
await page.goto(`http://localhost:${PORT}`, { waitUntil: 'domcontentloaded' });
await injectState(page, initPayload, TERMINAL_CLAUDE, 'sess-claude-2');
await page.screenshot({
path: join(OUTPUT_DIR, 'desktop-both-claude.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-both-claude.png');
} finally {
await context.close();
}
}
// 4. Desktop with 2 tabs (Claude + OpenCode), OpenCode active
async function captureDesktopBothOpencode(browser) {
console.log('\n4/6 Capturing desktop-both-opencode.png...');
const claudeSession = makeSession('sess-claude-3', 'w1-my-project', 'claude', 'busy');
const opencodeSession = makeSession('sess-opencode-3', 'w2-my-project', 'opencode', 'busy');
const initPayload = buildInitPayload([claudeSession, opencodeSession]);
const context = await browser.newContext({
viewport: DESKTOP_VIEWPORT,
deviceScaleFactor: 1,
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
try {
await setupRoutes(page, initPayload, TERMINAL_OPENCODE);
await page.goto(`http://localhost:${PORT}`, { waitUntil: 'domcontentloaded' });
await injectState(page, initPayload, TERMINAL_OPENCODE, 'sess-opencode-3');
await page.screenshot({
path: join(OUTPUT_DIR, 'desktop-both-opencode.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/desktop-both-opencode.png');
} finally {
await context.close();
}
}
// 5. Mobile with Claude tab
async function captureMobileClaude(browser) {
console.log('\n5/6 Capturing mobile-claude.png...');
const claudeSession = makeSession('sess-claude-m1', 'w1-my-project', 'claude', 'busy');
const initPayload = buildInitPayload([claudeSession]);
const context = await browser.newContext({
viewport: MOBILE_VIEWPORT,
deviceScaleFactor: 2,
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
try {
await setupRoutes(page, initPayload, TERMINAL_CLAUDE);
await page.goto(`http://localhost:${PORT}`, { waitUntil: 'domcontentloaded' });
await injectState(page, initPayload, TERMINAL_CLAUDE, 'sess-claude-m1');
await page.screenshot({
path: join(OUTPUT_DIR, 'mobile-claude.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/mobile-claude.png');
} finally {
await context.close();
}
}
// 6. Mobile with OpenCode tab
async function captureMobileOpencode(browser) {
console.log('\n6/6 Capturing mobile-opencode.png...');
const opencodeSession = makeSession('sess-opencode-m1', 'w1-my-project', 'opencode', 'busy');
const initPayload = buildInitPayload([opencodeSession]);
const context = await browser.newContext({
viewport: MOBILE_VIEWPORT,
deviceScaleFactor: 2,
});
const page = await context.newPage();
page.setDefaultTimeout(30000);
try {
await setupRoutes(page, initPayload, TERMINAL_OPENCODE);
await page.goto(`http://localhost:${PORT}`, { waitUntil: 'domcontentloaded' });
await injectState(page, initPayload, TERMINAL_OPENCODE, 'sess-opencode-m1');
await page.screenshot({
path: join(OUTPUT_DIR, 'mobile-opencode.png'),
fullPage: false,
});
console.log(' Saved: remotion/public/mobile-opencode.png');
} finally {
await context.close();
}
}
// ─── Main ────────────────────────────────────────────────────────────────────
async function main() {
console.log('='.repeat(60));
console.log('Codeman Video Screenshot Capture');
console.log('='.repeat(60));
console.log(`Port: ${PORT}`);
console.log(`Desktop: ${DESKTOP_VIEWPORT.width}x${DESKTOP_VIEWPORT.height}`);
console.log(`Mobile: ${MOBILE_VIEWPORT.width}x${MOBILE_VIEWPORT.height} @2x`);
console.log(`Output: ${OUTPUT_DIR}`);
console.log('');
const server = await startStaticServer();
let browser;
try {
browser = await chromium.launch({
headless: true,
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-dev-shm-usage',
'--disable-gpu',
],
});
await captureDesktopWelcome(browser);
await captureDesktopClaude(browser);
await captureDesktopBothClaude(browser);
await captureDesktopBothOpencode(browser);
await captureMobileClaude(browser);
await captureMobileOpencode(browser);
console.log('\n' + '='.repeat(60));
console.log('All 6 screenshots captured!');
console.log('='.repeat(60));
console.log('\nOutput files:');
console.log(' remotion/public/desktop-welcome.png');
console.log(' remotion/public/desktop-claude.png');
console.log(' remotion/public/desktop-both-claude.png');
console.log(' remotion/public/desktop-both-opencode.png');
console.log(' remotion/public/mobile-claude.png');
console.log(' remotion/public/mobile-opencode.png');
} catch (err) {
console.error('\nFatal error:', err.message);
console.error(err.stack);
process.exitCode = 1;
} finally {
if (browser) await browser.close().catch(() => {});
server.close();
console.log('\nDone.');
}
}
process.on('SIGINT', () => {
console.log('\nInterrupted.');
process.exit(1);
});
main();
scripts/codeman-tunnel.service
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=Codeman Cloudflare Tunnel
After=network-online.target codeman-web.service
Wants=network-online.target
[Service]
Type=simple
ExecStart=/bin/bash -c 'exec %h/.local/bin/cloudflared tunnel --url https://localhost:3000 --no-tls-verify 2>&1 | tee %h/.codeman/tunnel.log'
Restart=always
RestartSec=5
KillMode=process
# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=codeman-tunnel
[Install]
WantedBy=default.target
scripts/tunnel.sh
Executable
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Quick Cloudflare Tunnel for Codeman
# Usage: ./scripts/tunnel.sh [start|stop|status|url]
set -euo pipefail
SERVICE="codeman-tunnel"
case "${1:-start}" in
start)
if ! systemctl --user is-active "$SERVICE" &>/dev/null; then
# Install service if not already
if ! systemctl --user cat "$SERVICE" &>/dev/null 2>&1; then
cp "$(dirname "$0")/codeman-tunnel.service" "$HOME/.config/systemd/user/"
systemctl --user daemon-reload
fi
systemctl --user start "$SERVICE"
echo "Tunnel starting... waiting for URL"
sleep 6
fi
# Extract the tunnel URL from journal
URL=$(grep -oP 'https://[a-z0-9-]+\.trycloudflare\.com' "$HOME/.codeman/tunnel.log" 2>/dev/null | tail -1)
if [ -n "$URL" ]; then
echo "$URL"
else
echo "URL not ready yet, try: $0 url"
fi
;;
stop)
systemctl --user stop "$SERVICE"
echo "Tunnel stopped"
;;
status)
systemctl --user status "$SERVICE" --no-pager 2>&1 | head -10
echo ""
echo "URL:"
grep -oP 'https://[a-z0-9-]+\.trycloudflare\.com' "$HOME/.codeman/tunnel.log" 2>/dev/null | tail -1
;;
url)
grep -oP 'https://[a-z0-9-]+\.trycloudflare\.com' "$HOME/.codeman/tunnel.log" 2>/dev/null | tail -1
;;
*)
echo "Usage: $0 [start|stop|status|url]"
exit 1
;;
esac
skills-lock.json
+10
View File
@@ -0,0 +1,10 @@
{
"version": 1,
"skills": {
"remotion-best-practices": {
"source": "remotion-dev/skills",
"sourceType": "github",
"computedHash": "9851afb52e1b892c43b2af973bda1776ce01fedfdfe704e0cfd70069a3a9c300"
}
}
}
src/tunnel-manager.ts
+248
View File
@@ -0,0 +1,248 @@
/**
* @fileoverview Cloudflare Tunnel Manager
*
* Manages a cloudflared child process for remote access to Codeman.
* Spawns `cloudflared tunnel --url` as a child process and parses
* the trycloudflare.com URL from its stderr output.
*
* Follows the same lifecycle pattern as ImageWatcher/SubagentWatcher:
* extends EventEmitter, start()/stop(), emits typed events.
*/
import { EventEmitter } from 'node:events';
import { spawn, type ChildProcess } from 'node:child_process';
import { existsSync } from 'node:fs';
import { join } from 'node:path';
import { homedir } from 'node:os';
// ========== Types ==========
export interface TunnelStatus {
running: boolean;
url: string | null;
}
// ========== Constants ==========
/** Regex to extract the trycloudflare.com URL from cloudflared output */
const TUNNEL_URL_REGEX = /https:\/\/[a-z0-9-]+\.trycloudflare\.com/;
/** Max time to wait for URL before considering it a timeout (ms) */
const URL_TIMEOUT_MS = 30_000;
/** Restart delay after unexpected exit (ms) */
const RESTART_DELAY_MS = 5_000;
// ========== TunnelManager Class ==========
export class TunnelManager extends EventEmitter {
private process: ChildProcess | null = null;
private url: string | null = null;
private cloudflaredPath: string | null = null;
private urlTimeoutTimer: NodeJS.Timeout | null = null;
private restartTimer: NodeJS.Timeout | null = null;
private stopping = false;
private localPort = 3000;
private useHttps = false;
/**
* Resolve cloudflared binary path.
* Checks ~/.local/bin first, then falls back to PATH.
*/
private resolveCloudflared(): string | null {
if (this.cloudflaredPath) return this.cloudflaredPath;
// Check ~/.local/bin first (common user install location)
const localBin = join(homedir(), '.local', 'bin', 'cloudflared');
if (existsSync(localBin)) {
this.cloudflaredPath = localBin;
return localBin;
}
// Check /usr/local/bin
const usrLocalBin = '/usr/local/bin/cloudflared';
if (existsSync(usrLocalBin)) {
this.cloudflaredPath = usrLocalBin;
return usrLocalBin;
}
// Fall back to PATH
this.cloudflaredPath = 'cloudflared';
return 'cloudflared';
}
/**
* Start the cloudflared tunnel process.
*/
start(localPort: number, https: boolean): void {
if (this.process) {
return; // Already running
}
this.stopping = false;
this.localPort = localPort;
this.useHttps = https;
const binary = this.resolveCloudflared();
if (!binary) {
this.emit('error', 'cloudflared not found. Install from https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/');
return;
}
const protocol = https ? 'https' : 'http';
const args = ['tunnel', '--url', `${protocol}://localhost:${localPort}`];
if (https) {
args.push('--no-tls-verify');
}
console.log(`[TunnelManager] Starting: ${binary} ${args.join(' ')}`);
try {
this.process = spawn(binary, args, {
stdio: ['ignore', 'pipe', 'pipe'],
detached: false,
});
} catch (err) {
this.emit('error', `Failed to spawn cloudflared: ${err instanceof Error ? err.message : String(err)}`);
return;
}
this.emit('progress', { message: 'Spawning cloudflared process...' });
// Parse stdout/stderr for the URL, then detach once found
const handleOutput = (data: Buffer) => {
const line = data.toString().trim();
if (!line) return;
// Emit progress for interesting cloudflared log lines
if (!this.url) {
if (/connector.*registered/i.test(line)) {
this.emit('progress', { message: 'Tunnel connector registered' });
} else if (/connection.*registered/i.test(line)) {
this.emit('progress', { message: 'Connection registered with Cloudflare edge' });
} else if (/route.*propagating/i.test(line) || /ingress/i.test(line)) {
this.emit('progress', { message: 'Propagating route to Cloudflare edge...' });
} else if (/Starting tunnel/i.test(line) || /initial.*connection/i.test(line)) {
this.emit('progress', { message: 'Establishing tunnel connection...' });
} else if (/Registered tunnel connection/i.test(line)) {
this.emit('progress', { message: 'Tunnel connection registered' });
}
}
const match = line.match(TUNNEL_URL_REGEX);
if (match && !this.url) {
this.url = match[0];
console.log(`[TunnelManager] Tunnel URL: ${this.url}`);
if (this.urlTimeoutTimer) {
clearTimeout(this.urlTimeoutTimer);
this.urlTimeoutTimer = null;
}
// Detach listeners — no need to parse further output
this.process?.stdout?.off('data', handleOutput);
this.process?.stderr?.off('data', handleOutput);
this.emit('started', { url: this.url });
}
};
this.process.stdout?.on('data', handleOutput);
this.process.stderr?.on('data', handleOutput);
this.process.on('error', (err) => {
console.error(`[TunnelManager] Process error:`, err.message);
this.cleanup();
this.emit('error', `cloudflared error: ${err.message}`);
});
this.process.on('exit', (code, signal) => {
console.log(`[TunnelManager] Process exited (code=${code}, signal=${signal})`);
const wasRunning = this.url !== null;
this.cleanup();
if (!this.stopping) {
// Unexpected exit — attempt restart
this.emit('error', `cloudflared exited unexpectedly (code=${code})`);
if (wasRunning) {
console.log(`[TunnelManager] Scheduling restart in ${RESTART_DELAY_MS}ms`);
this.restartTimer = setTimeout(() => {
this.restartTimer = null;
if (!this.stopping && !this.process) {
this.start(this.localPort, this.useHttps);
}
}, RESTART_DELAY_MS);
}
} else {
this.emit('stopped', {});
}
});
// Set URL timeout
this.urlTimeoutTimer = setTimeout(() => {
this.urlTimeoutTimer = null;
if (!this.url && this.process) {
this.emit('error', 'Timed out waiting for tunnel URL');
}
}, URL_TIMEOUT_MS);
}
/**
* Stop the cloudflared tunnel process.
*/
stop(): void {
this.stopping = true;
if (this.restartTimer) {
clearTimeout(this.restartTimer);
this.restartTimer = null;
}
if (this.urlTimeoutTimer) {
clearTimeout(this.urlTimeoutTimer);
this.urlTimeoutTimer = null;
}
if (this.process) {
console.log(`[TunnelManager] Stopping tunnel (PID ${this.process.pid})`);
this.process.kill('SIGTERM');
// Force kill after 5s if still alive
const pid = this.process.pid;
const forceTimer = setTimeout(() => {
try {
if (pid) process.kill(pid, 'SIGKILL');
} catch {
// Process already gone
}
}, 5000);
this.process.once('exit', () => clearTimeout(forceTimer));
} else {
this.cleanup();
this.emit('stopped', {});
}
}
/**
* Clean up internal state after process exit.
*/
private cleanup(): void {
this.process = null;
this.url = null;
if (this.urlTimeoutTimer) {
clearTimeout(this.urlTimeoutTimer);
this.urlTimeoutTimer = null;
}
}
isRunning(): boolean {
return this.process !== null;
}
getUrl(): string | null {
return this.url;
}
getStatus(): TunnelStatus {
return {
running: this.process !== null,
url: this.url,
};
}
}
src/web/public/app.js
+289
View File
@@ -3466,6 +3466,7 @@ class CodemanApp {
const overlay = document.getElementById('welcomeOverlay');
if (overlay) {
overlay.classList.add('visible');
this.loadTunnelStatus();
}
}
@@ -3474,6 +3475,12 @@ class CodemanApp {
if (overlay) {
overlay.classList.remove('visible');
}
// Collapse expanded QR when leaving welcome screen
const qrWrap = document.getElementById('welcomeQr');
if (qrWrap) {
clearTimeout(this._welcomeQrShrinkTimer);
qrWrap.classList.remove('expanded');
}
}
/**
@@ -4894,6 +4901,57 @@ class CodemanApp {
this.openImagePopup(data);
});
// ========== Tunnel Events ==========
addListener('tunnel:started', (e) => {
const data = JSON.parse(e.data);
console.log('[Tunnel] Started:', data.url);
this._dismissTunnelConnecting();
this._updateTunnelUrlDisplay(data.url);
const welcomeVisible = document.getElementById('welcomeOverlay')?.classList.contains('visible');
if (welcomeVisible) {
// On welcome screen: QR appears inline, expanded first
this._updateWelcomeTunnelBtn(true, data.url, true);
this.showToast(`Tunnel active`, 'success');
} else {
// Not on welcome screen: popup QR overlay
this._updateWelcomeTunnelBtn(true, data.url);
this.showToast(`Tunnel active: ${data.url}`, 'success');
this.showTunnelQR();
}
});
addListener('tunnel:stopped', () => {
console.log('[Tunnel] Stopped');
this._dismissTunnelConnecting();
this._updateTunnelUrlDisplay(null);
this._updateWelcomeTunnelBtn(false);
this.closeTunnelQR();
});
addListener('tunnel:progress', (e) => {
const data = JSON.parse(e.data);
console.log('[Tunnel] Progress:', data.message);
const toast = document.getElementById('tunnelConnectingToast');
if (toast) {
toast.innerHTML = `<span class="tunnel-spinner"></span> ${data.message}`;
}
// Also update button text if on welcome screen
const btn = document.getElementById('welcomeTunnelBtn');
if (btn?.classList.contains('connecting')) {
btn.innerHTML = `<span class="tunnel-spinner"></span> ${data.message}`;
}
});
addListener('tunnel:error', (e) => {
const data = JSON.parse(e.data);
console.warn('[Tunnel] Error:', data.message);
this._dismissTunnelConnecting();
this.showToast(`Tunnel error: ${data.message}`, 'error');
const btn = document.getElementById('welcomeTunnelBtn');
if (btn) { btn.disabled = false; btn.classList.remove('connecting'); }
});
// Plan subagent visibility events (show Opus agents during plan generation)
addListener('plan:subagent', (e) => {
const data = JSON.parse(e.data);
@@ -11389,6 +11447,8 @@ class CodemanApp {
document.getElementById('appSettingsSubagentTracking').checked = settings.subagentTrackingEnabled ?? defaults.subagentTrackingEnabled ?? true;
document.getElementById('appSettingsSubagentActiveTabOnly').checked = settings.subagentActiveTabOnly ?? defaults.subagentActiveTabOnly ?? true;
document.getElementById('appSettingsImageWatcherEnabled').checked = settings.imageWatcherEnabled ?? defaults.imageWatcherEnabled ?? false;
document.getElementById('appSettingsTunnelEnabled').checked = settings.tunnelEnabled ?? false;
this.loadTunnelStatus();
document.getElementById('appSettingsLocalEcho').checked = settings.localEchoEnabled ?? MobileDetection.isTouchDevice();
document.getElementById('appSettingsTabTwoRows').checked = settings.tabTwoRows ?? defaults.tabTwoRows ?? false;
// Claude CLI settings
@@ -11515,6 +11575,229 @@ class CodemanApp {
}
}
async loadTunnelStatus() {
try {
const res = await fetch('/api/tunnel/status');
const status = await res.json();
const active = status.running && status.url;
this._updateTunnelUrlDisplay(active ? status.url : null);
this._updateWelcomeTunnelBtn(!!active, active ? status.url : null);
} catch {
this._updateTunnelUrlDisplay(null);
this._updateWelcomeTunnelBtn(false);
}
}
_updateTunnelUrlDisplay(url) {
const row = document.getElementById('tunnelUrlRow');
const display = document.getElementById('tunnelUrlDisplay');
if (!row || !display) return;
if (url) {
row.style.display = '';
display.textContent = url;
display.onclick = () => {
navigator.clipboard.writeText(url).then(() => {
this.showToast('Tunnel URL copied', 'success');
});
};
} else {
row.style.display = 'none';
display.textContent = '';
display.onclick = null;
}
}
showTunnelQR() {
// Close existing popup if open
this.closeTunnelQR();
const overlay = document.createElement('div');
overlay.id = 'tunnelQrOverlay';
overlay.style.cssText = 'position:fixed;inset:0;background:rgba(0,0,0,0.7);z-index:5000;display:flex;align-items:center;justify-content:center;cursor:pointer';
overlay.onclick = (e) => { if (e.target === overlay) this.closeTunnelQR(); };
const card = document.createElement('div');
card.style.cssText = 'background:var(--bg-card);border:1px solid var(--border);border-radius:12px;padding:24px;text-align:center;max-width:340px;width:90vw;box-shadow:var(--shadow-lg);cursor:default';
card.innerHTML = `
<div style="font-size:14px;font-weight:600;color:var(--text-primary);margin-bottom:16px">Scan to connect</div>
<div id="tunnelQrContainer" style="background:#fff;border-radius:8px;padding:16px;display:inline-block">
<div style="color:#666;font-size:12px">Loading...</div>
</div>
<div id="tunnelQrUrl" style="margin-top:12px;font-family:monospace;font-size:11px;color:var(--text-muted);word-break:break-all;cursor:pointer" title="Click to copy"></div>
<button onclick="app.closeTunnelQR()" style="margin-top:16px;padding:6px 20px;background:var(--bg-elevated);border:1px solid var(--border);border-radius:6px;color:var(--text-primary);cursor:pointer;font-size:13px">Close</button>
`;
overlay.appendChild(card);
document.body.appendChild(overlay);
// Fetch QR SVG from server
fetch('/api/tunnel/qr')
.then(res => {
if (!res.ok) throw new Error('Tunnel not running');
return res.json();
})
.then(data => {
const container = document.getElementById('tunnelQrContainer');
if (container && data.svg) container.innerHTML = data.svg;
})
.catch(() => {
const container = document.getElementById('tunnelQrContainer');
if (container) container.innerHTML = '<div style="color:#c00;font-size:12px;padding:20px">Tunnel not active</div>';
});
// Fetch URL for display
fetch('/api/tunnel/status')
.then(r => r.json())
.then(status => {
const urlEl = document.getElementById('tunnelQrUrl');
if (urlEl && status.url) {
urlEl.textContent = status.url;
urlEl.onclick = () => {
navigator.clipboard.writeText(status.url).then(() => {
this.showToast('Tunnel URL copied', 'success');
});
};
}
})
.catch(() => {});
// Close on Escape
this._tunnelQrEscHandler = (e) => { if (e.key === 'Escape') this.closeTunnelQR(); };
document.addEventListener('keydown', this._tunnelQrEscHandler);
}
closeTunnelQR() {
const overlay = document.getElementById('tunnelQrOverlay');
if (overlay) overlay.remove();
if (this._tunnelQrEscHandler) {
document.removeEventListener('keydown', this._tunnelQrEscHandler);
this._tunnelQrEscHandler = null;
}
}
async toggleTunnelFromWelcome() {
const btn = document.getElementById('welcomeTunnelBtn');
if (!btn) return;
const isActive = btn.classList.contains('active');
btn.disabled = true;
try {
const res = await fetch('/api/settings');
const current = res.ok ? await res.json() : {};
const newEnabled = !isActive;
current.tunnelEnabled = newEnabled;
await fetch('/api/settings', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(current),
});
if (newEnabled) {
this._showTunnelConnecting();
} else {
this._dismissTunnelConnecting();
this.showToast('Tunnel stopped', 'info');
this._updateWelcomeTunnelBtn(false);
btn.disabled = false;
}
} catch (err) {
this._dismissTunnelConnecting();
this.showToast('Failed to toggle tunnel', 'error');
btn.disabled = false;
}
}
_showTunnelConnecting() {
const btn = document.getElementById('welcomeTunnelBtn');
if (btn) {
btn.classList.add('connecting');
btn.innerHTML = `
<span class="tunnel-spinner"></span>
Connecting...`;
}
// Persistent toast with spinner
this._dismissTunnelConnecting();
const toast = document.createElement('div');
toast.className = 'toast toast-info show';
toast.id = 'tunnelConnectingToast';
toast.innerHTML = '<span class="tunnel-spinner"></span> Cloudflare Tunnel connecting...';
toast.style.pointerEvents = 'auto';
if (!this._toastContainer) {
this._toastContainer = document.querySelector('.toast-container');
if (!this._toastContainer) {
this._toastContainer = document.createElement('div');
this._toastContainer.className = 'toast-container';
document.body.appendChild(this._toastContainer);
}
}
this._toastContainer.appendChild(toast);
}
_dismissTunnelConnecting() {
const toast = document.getElementById('tunnelConnectingToast');
if (toast) {
toast.classList.remove('show');
setTimeout(() => toast.remove(), 200);
}
const btn = document.getElementById('welcomeTunnelBtn');
if (btn) btn.classList.remove('connecting');
}
_updateWelcomeTunnelBtn(active, url, firstAppear = false) {
const btn = document.getElementById('welcomeTunnelBtn');
if (btn) {
btn.disabled = false;
if (active) {
btn.classList.remove('connecting');
btn.classList.add('active');
btn.innerHTML = `
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg>
Tunnel Active`;
} else {
btn.classList.remove('active', 'connecting');
btn.innerHTML = `
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg>
Cloudflare Tunnel`;
}
}
// Update welcome QR code
const qrWrap = document.getElementById('welcomeQr');
const qrInner = document.getElementById('welcomeQrInner');
const qrUrl = document.getElementById('welcomeQrUrl');
if (!qrWrap || !qrInner) return;
if (active) {
qrWrap.classList.add('visible');
// First appear: start expanded, auto-shrink after 8s
if (firstAppear) {
qrWrap.classList.add('expanded');
clearTimeout(this._welcomeQrShrinkTimer);
this._welcomeQrShrinkTimer = setTimeout(() => {
qrWrap.classList.remove('expanded');
}, 8000);
}
if (url) {
qrUrl.textContent = url;
qrUrl.title = 'Click QR to enlarge';
}
fetch('/api/tunnel/qr')
.then(r => { if (!r.ok) throw new Error(); return r.json(); })
.then(data => { if (data.svg) qrInner.innerHTML = data.svg; })
.catch(() => { qrInner.innerHTML = '<div style="color:#999;font-size:11px;padding:20px">QR unavailable</div>'; });
} else {
clearTimeout(this._welcomeQrShrinkTimer);
qrWrap.classList.remove('visible', 'expanded');
qrInner.innerHTML = '';
if (qrUrl) qrUrl.textContent = '';
}
}
toggleWelcomeQrSize() {
const qrWrap = document.getElementById('welcomeQr');
if (qrWrap) {
clearTimeout(this._welcomeQrShrinkTimer);
qrWrap.classList.toggle('expanded');
}
}
toggleDeepgramKeyVisibility() {
const input = document.getElementById('voiceDeepgramKey');
const btn = document.getElementById('voiceKeyToggleBtn');
@@ -11642,6 +11925,7 @@ class CodemanApp {
subagentTrackingEnabled: document.getElementById('appSettingsSubagentTracking').checked,
subagentActiveTabOnly: document.getElementById('appSettingsSubagentActiveTabOnly').checked,
imageWatcherEnabled: document.getElementById('appSettingsImageWatcherEnabled').checked,
tunnelEnabled: document.getElementById('appSettingsTunnelEnabled').checked,
localEchoEnabled: document.getElementById('appSettingsLocalEcho').checked,
tabTwoRows: document.getElementById('appSettingsTabTwoRows').checked,
// Claude CLI settings
@@ -11760,6 +12044,11 @@ class CodemanApp {
await this.saveModelConfigFromSettings();
this.showToast('Settings saved', 'success');
// Show tunnel-specific feedback if toggled on
if (settings.tunnelEnabled) {
this.showToast('Tunnel starting — QR code will appear when ready...', 'info');
}
} catch (err) {
// Server save failed but localStorage succeeded
this.showToast('Settings saved locally', 'warning');
src/web/public/index.html
+28 -4
View File
@@ -238,11 +238,19 @@
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><polygon points="5 3 19 12 5 21 5 3"/></svg>
Run Claude Code
</button>
<button class="welcome-btn welcome-btn-tunnel" id="welcomeTunnelBtn" onclick="app.toggleTunnelFromWelcome()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg>
Cloudflare Tunnel
</button>
<button class="welcome-btn welcome-btn-opencode" onclick="app.setRunMode('opencode'); app.runOpenCode()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><polygon points="5 3 19 12 5 21 5 3"/></svg>
Run OpenCode
</button>
</div>
<div class="welcome-qr" id="welcomeQr" onclick="app.toggleWelcomeQrSize()">
<div class="welcome-qr-inner" id="welcomeQrInner"></div>
<div class="welcome-qr-url" id="welcomeQrUrl"></div>
</div>
<p class="welcome-hint">Or press <kbd>Ctrl</kbd>+<kbd>Enter</kbd> to start</p>
</div>
</div>
@@ -379,10 +387,6 @@
</div>
<div class="toolbar-center">
<span class="version-display" id="versionDisplay" title="Codeman version">v0.0.0</span>
</div>
<div class="toolbar-right">
<button class="btn-toolbar btn-sm btn-voice" id="voiceInputBtn" onclick="VoiceInput.toggle()"
title="Voice input (Ctrl+Shift+V)" aria-label="Start voice input" aria-pressed="false">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
@@ -393,6 +397,10 @@
</svg>
</button>
</div>
<div class="toolbar-right">
<span class="version-display" id="versionDisplay" title="Codeman version">v0.0.0</span>
</div>
</footer>
<!-- Help Modal -->
@@ -934,6 +942,22 @@
</label>
</div>
<!-- Remote Access Section -->
<div class="settings-section-header">Remote Access</div>
<div class="settings-item" title="Expose Codeman via Cloudflare Tunnel for remote access">
<span class="settings-item-label">Cloudflare Tunnel</span>
<label class="switch switch-sm">
<input type="checkbox" id="appSettingsTunnelEnabled">
<span class="slider"></span>
</label>
</div>
<div class="settings-item" id="tunnelUrlRow" style="display:none">
<span class="settings-item-label">Tunnel URL</span>
<div style="display:flex; align-items:center; gap:8px">
<span id="tunnelUrlDisplay" class="settings-item-value" style="cursor:pointer; text-decoration:underline; font-family:monospace; font-size:12px" title="Click to copy"></span>
<button class="btn-icon-sm" id="tunnelQrBtn" onclick="app.showTunnelQR()" title="Show QR code" style="font-size:16px; padding:2px 6px; background:none; border:1px solid var(--border); border-radius:4px; cursor:pointer; color:var(--text-secondary)">⊞</button>
</div>
</div>
</div>
</div>
src/web/public/styles.css
+104 -6
View File
@@ -1685,11 +1685,11 @@ body {
}
.welcome-title {
font-size: 1.75rem;
font-weight: 600;
font-size: 3.5rem;
font-weight: 700;
color: var(--accent-hover);
margin-bottom: 1rem;
letter-spacing: -0.02em;
letter-spacing: -0.03em;
}
.welcome-desc {
@@ -1703,6 +1703,7 @@ body {
display: flex;
gap: 1rem;
justify-content: center;
flex-wrap: wrap;
margin-top: 2rem;
margin-bottom: 1.5rem;
}
@@ -1744,6 +1745,52 @@ body {
color: #a7f3d0;
}
.welcome-btn-tunnel {
background: linear-gradient(135deg, #2a1a3e 0%, #6d28d9 100%);
border-color: #7c3aed;
color: #ddd6fe;
}
.welcome-btn-tunnel:hover {
background: linear-gradient(135deg, #6d28d9 0%, #7c3aed 100%);
box-shadow: 0 0 12px rgba(124, 58, 237, 0.4);
color: #fff;
}
.welcome-btn-tunnel.active {
background: linear-gradient(135deg, #059669 0%, #10b981 100%);
border-color: #34d399;
color: #ecfdf5;
}
.welcome-btn-tunnel.active:hover {
background: linear-gradient(135deg, #10b981 0%, #34d399 100%);
box-shadow: 0 0 12px rgba(52, 211, 153, 0.4);
color: #fff;
}
.welcome-btn-tunnel.connecting {
background: linear-gradient(135deg, #1e1e3a 0%, #4338ca 100%);
border-color: #6366f1;
color: #c7d2fe;
cursor: wait;
}
.tunnel-spinner {
display: inline-block;
width: 16px;
height: 16px;
border: 2px solid rgba(255,255,255,0.25);
border-top-color: currentColor;
border-radius: 50%;
animation: tunnel-spin 0.8s linear infinite;
flex-shrink: 0;
}
@keyframes tunnel-spin {
to { transform: rotate(360deg); }
}
.welcome-btn-ralph {
background: linear-gradient(135deg, #3d3415 0%, #4a3f1a 100%);
border-color: #6b5a1e;
@@ -1757,6 +1804,50 @@ body {
color: #fcd34d;
}
.welcome-qr {
display: none;
flex-direction: column;
align-items: center;
margin-top: 1.5rem;
margin-bottom: 0.5rem;
cursor: pointer;
transition: transform 0.2s ease;
}
.welcome-qr.visible {
display: flex;
}
.welcome-qr-inner {
background: #fff;
border-radius: 10px;
padding: 12px;
width: 140px;
height: 140px;
transition: width 0.25s ease, height 0.25s ease, padding 0.25s ease;
}
.welcome-qr-inner svg {
width: 100%;
height: 100%;
}
.welcome-qr.expanded .welcome-qr-inner {
width: 280px;
height: 280px;
padding: 16px;
}
.welcome-qr-url {
margin-top: 8px;
font-family: monospace;
font-size: 11px;
color: var(--text-muted);
word-break: break-all;
max-width: 320px;
text-align: center;
}
.welcome-hint {
color: var(--text-muted);
font-size: 0.8rem;
@@ -1804,13 +1895,14 @@ body {
.version-display {
font-family: 'SF Mono', Monaco, monospace;
font-size: 0.7rem;
font-size: 0.6rem;
color: var(--text-muted);
padding: 0.2rem 0.5rem;
padding: 0.15rem 0.4rem;
background: rgba(255, 255, 255, 0.05);
border-radius: 3px;
cursor: default;
user-select: none;
opacity: 0.7;
}
.toolbar-group {
@@ -2081,11 +2173,17 @@ body {
border-color: var(--red);
}
/* Voice Input Button */
/* Voice Input Button — desktop only (mobile uses .btn-voice-mobile) */
.btn-toolbar.btn-voice {
padding: 0.3rem 0.5rem;
}
@media (max-width: 1023px) {
.toolbar-center .btn-toolbar.btn-voice {
display: none !important;
}
}
.btn-toolbar.btn-voice:hover {
color: var(--text);
border-color: rgba(59, 130, 246, 0.4);
src/web/server.ts
+166 -6
View File
@@ -12,12 +12,14 @@
import Fastify, { FastifyInstance, FastifyReply } from 'fastify';
import fastifyCompress from '@fastify/compress';
import fastifyCookie from '@fastify/cookie';
import fastifyStatic from '@fastify/static';
import { join, dirname, resolve, relative, isAbsolute } from 'node:path';
import { fileURLToPath } from 'node:url';
import { existsSync, statSync, mkdirSync, writeFileSync, readdirSync, readFileSync, rmSync } from 'node:fs';
import fs from 'node:fs/promises';
import { execSync } from 'node:child_process';
import { randomBytes } from 'node:crypto';
import { homedir, totalmem, freemem, loadavg, cpus } from 'node:os';
import { EventEmitter } from 'node:events';
import { Session, ClaudeMessage, type BackgroundTask, type RalphTrackerState, type RalphTodoItem, type ActiveBashTool } from '../session.js';
@@ -34,6 +36,7 @@ import { subagentWatcher, type SubagentInfo, type SubagentToolCall, type Subagen
import { imageWatcher } from '../image-watcher.js';
import { TranscriptWatcher } from '../transcript-watcher.js';
import { TeamWatcher } from '../team-watcher.js';
import { TunnelManager } from '../tunnel-manager.js';
import { v4 as uuidv4 } from 'uuid';
import { createRequire } from 'node:module';
import { RunSummaryTracker } from '../run-summary.js';
@@ -89,6 +92,7 @@ import {
SubagentParentMapSchema,
InteractiveRespawnSchema,
RespawnEnableSchema,
isValidWorkingDir,
} from './schemas.js';
import { StaleExpirationMap } from '../utils/index.js';
import { MAX_CONCURRENT_SESSIONS } from '../config/map-limits.js';
@@ -141,6 +145,16 @@ const MAX_SESSION_NAME_LENGTH = 128;
const MAX_HOOK_DATA_SIZE = 8 * 1024;
// Maximum screenshot upload size (10MB)
const MAX_SCREENSHOT_SIZE = 10 * 1024 * 1024;
// Auth session cookie TTL (24h — matches autonomous run length)
const AUTH_SESSION_TTL_MS = 24 * 60 * 60 * 1000;
// Auth session cookie name
const AUTH_COOKIE_NAME = 'codeman_session';
// Max concurrent auth sessions
const MAX_AUTH_SESSIONS = 100;
// Max failed auth attempts per IP before rate-limiting
const AUTH_FAILURE_MAX = 10;
// Failed auth attempt tracking window (15 minutes)
const AUTH_FAILURE_WINDOW_MS = 15 * 60 * 1000;
// Screenshots directory
const SCREENSHOTS_DIR = join(homedir(), '.codeman', 'screenshots');
// Stats collection interval (2 seconds)
@@ -408,6 +422,9 @@ export class WebServer extends EventEmitter {
detected: (event: ImageDetectedEvent) => void;
error: (error: Error, sessionId?: string) => void;
} | null = null;
private tunnelManager: TunnelManager = new TunnelManager();
private authSessions: StaleExpirationMap<string, string> | null = null;
private authFailures: StaleExpirationMap<string, number> | null = null;
private teamWatcher: TeamWatcher = new TeamWatcher();
private teamWatcherHandlers: {
teamCreated: (config: unknown) => void;
@@ -457,6 +474,20 @@ export class WebServer extends EventEmitter {
// Set up team watcher listeners
this.setupTeamWatcherListeners();
// Set up tunnel manager listeners
this.tunnelManager.on('started', (data: { url: string }) => {
this.broadcast('tunnel:started', data);
});
this.tunnelManager.on('stopped', () => {
this.broadcast('tunnel:stopped', {});
});
this.tunnelManager.on('error', (message: string) => {
this.broadcast('tunnel:error', { message });
});
this.tunnelManager.on('progress', (data: { message: string }) => {
this.broadcast('tunnel:progress', data);
});
}
/**
@@ -582,17 +613,82 @@ export class WebServer extends EventEmitter {
threshold: 1024,
});
// Optional HTTP Basic Auth (set CODEMAN_PASSWORD env var to enable)
// Cookie plugin (needed for auth session tokens)
await this.app.register(fastifyCookie);
// Optional HTTP Basic Auth with session cookies and rate limiting
const authPassword = process.env.CODEMAN_PASSWORD;
if (authPassword) {
const authUsername = process.env.CODEMAN_USERNAME || 'admin';
const expectedHeader = 'Basic ' + Buffer.from(`${authUsername}:${authPassword}`).toString('base64');
// Session token store — active sessions extend TTL on access
this.authSessions = new StaleExpirationMap<string, string>({
ttlMs: AUTH_SESSION_TTL_MS,
refreshOnGet: true,
});
// Failure counter per IP — decay naturally after 15 minutes
this.authFailures = new StaleExpirationMap<string, number>({
ttlMs: AUTH_FAILURE_WINDOW_MS,
refreshOnGet: false,
});
this.app.addHook('onRequest', (req, reply, done) => {
const auth = req.headers.authorization;
if (auth === expectedHeader) {
// Hook events come from local Claude Code hooks (curl from localhost) — no auth headers available.
// Safe: validated by HookEventSchema, only triggers broadcasts.
if (req.url === '/api/hook-event' && req.method === 'POST') {
done();
return;
}
const clientIp = req.ip;
// Rate limit: reject if too many failed attempts from this IP
const failures = this.authFailures!.get(clientIp) ?? 0;
if (failures >= AUTH_FAILURE_MAX) {
reply.code(429).send('Too Many Requests — try again later');
return;
}
// Check session cookie first (avoids re-sending credentials on every request)
const sessionToken = req.cookies[AUTH_COOKIE_NAME];
if (sessionToken && this.authSessions!.has(sessionToken)) {
done();
return;
}
// Check Basic Auth header
const auth = req.headers.authorization;
if (auth === expectedHeader) {
// Issue session token cookie so browser doesn't need to re-send credentials
const token = randomBytes(32).toString('hex');
// Evict oldest if at capacity (prevent unbounded growth)
if (this.authSessions!.size >= MAX_AUTH_SESSIONS) {
const oldestKey = this.authSessions!.keys().next().value;
if (oldestKey !== undefined) this.authSessions!.delete(oldestKey);
}
this.authSessions!.set(token, clientIp);
// Reset failure count on successful auth
this.authFailures!.delete(clientIp);
reply.setCookie(AUTH_COOKIE_NAME, token, {
httpOnly: true,
secure: this.https,
sameSite: 'lax',
maxAge: AUTH_SESSION_TTL_MS / 1000, // seconds
path: '/',
});
done();
return;
}
// Auth failed — track failure count
this.authFailures!.set(clientIp, failures + 1);
reply.header('WWW-Authenticate', 'Basic realm="Codeman"');
reply.code(401).send('Unauthorized');
});
@@ -668,6 +764,23 @@ export class WebServer extends EventEmitter {
// API Routes
this.app.get('/api/status', async () => this.getLightState());
this.app.get('/api/tunnel/status', async () => this.tunnelManager.getStatus());
this.app.get('/api/tunnel/qr', async (_req, reply) => {
const url = this.tunnelManager.getUrl();
if (!url) {
return reply.code(404).send(createErrorResponse(ApiErrorCode.NOT_FOUND, 'Tunnel not running'));
}
try {
const QRCode = require('qrcode');
const svg: string = await QRCode.toString(url, { type: 'svg', margin: 2, width: 256 });
// Return as data URI to avoid Fastify compress issues with SVG content-type
return { svg };
} catch (err) {
return reply.code(500).send(createErrorResponse(ApiErrorCode.OPERATION_FAILED, getErrorMessage(err)));
}
});
// OpenCode CLI availability check
this.app.get('/api/opencode/status', async () => {
const { isOpenCodeAvailable, resolveOpenCodeDir } = await import('../utils/opencode-cli-resolver.js');
@@ -3426,6 +3539,18 @@ NOW: Generate the implementation plan for the task above. Think step by step.`;
console.log('Image watcher stopped via settings change');
}
// Handle tunnel toggle dynamically
if ('tunnelEnabled' in settings) {
const tunnelEnabled = settings.tunnelEnabled as boolean;
if (tunnelEnabled && !this.tunnelManager.isRunning()) {
this.tunnelManager.start(this.port, this.https);
console.log('Tunnel started via settings change');
} else if (!tunnelEnabled && this.tunnelManager.isRunning()) {
this.tunnelManager.stop();
console.log('Tunnel stopped via settings change');
}
}
return { success: true };
} catch (err) {
return createErrorResponse(ApiErrorCode.OPERATION_FAILED, getErrorMessage(err));
@@ -3744,10 +3869,10 @@ NOW: Generate the implementation plan for the task above. Think step by step.`;
}
}
// Start transcript watching if transcript_path is provided
// Start transcript watching if transcript_path is provided and safe
if (data && 'transcript_path' in data) {
const transcriptPath = String(data.transcript_path);
if (transcriptPath) {
if (transcriptPath && isValidWorkingDir(transcriptPath)) {
this.startTranscriptWatcher(sessionId, transcriptPath);
}
}
@@ -5329,6 +5454,12 @@ NOW: Generate the implementation plan for the task above. Think step by step.`;
console.log('Image watcher disabled by user settings');
}
// Start Cloudflare tunnel if enabled in settings
if (await this.isTunnelEnabled()) {
this.tunnelManager.start(this.port, this.https);
console.log('Cloudflare tunnel starting on boot (enabled in settings)');
}
// Start team watcher for agent team awareness (always on — lightweight polling)
this.teamWatcher.start();
console.log('Team watcher started - monitoring ~/.claude/teams/ for agent team activity');
@@ -5371,6 +5502,23 @@ NOW: Generate the implementation plan for the task above. Think step by step.`;
return false; // Default disabled (matches UI default)
}
/**
* Check if Cloudflare tunnel is enabled in settings (default: false)
*/
private async isTunnelEnabled(): Promise<boolean> {
const settingsPath = join(homedir(), '.codeman', 'settings.json');
try {
const content = await fs.readFile(settingsPath, 'utf-8');
const settings = JSON.parse(content);
return settings.tunnelEnabled ?? false;
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') {
console.error('Failed to read tunnel setting:', err);
}
}
return false;
}
private async restoreMuxSessions(): Promise<void> {
try {
// Reconcile mux sessions to find which ones are still alive (also discovers unknown ones)
@@ -5694,6 +5842,10 @@ NOW: Generate the implementation plan for the task above. Think step by step.`;
// Stop team watcher
this.teamWatcher.stop();
// Stop tunnel
this.tunnelManager.stop();
this.tunnelManager.removeAllListeners();
// Destroy file stream manager (clears cleanup timer and kills remaining tail processes)
fileStreamManager.destroy();
@@ -5715,8 +5867,16 @@ NOW: Generate the implementation plan for the task above. Think step by step.`;
this.transcriptWatchers.clear();
this.sessionListenerRefs.clear();
this.scheduledRuns.clear();
// Dispose StaleExpirationMap (stops internal cleanup timer)
// Dispose StaleExpirationMaps (stops internal cleanup timers)
this.lastTerminalEventTime.dispose();
if (this.authSessions) {
this.authSessions.dispose();
this.authSessions = null;
}
if (this.authFailures) {
this.authFailures.dispose();
this.authFailures = null;
}
this.activePlanOrchestrators.clear();
this.cleaningUp.clear();