This Reel was never opened in a video editor. Every cut, zoom and sound is code. I write one HTML file, every word I say gets a timestamp, Chrome renders every frame and ffmpeg glues it together. Then I tell Claude Code what to change, and it edits the file.
Below is a small working version you can run today: one word list, one HTML storyboard, one render script and one ffmpeg command. I ran every file on my Mac before posting.
#What you need
- Node.js 18 or newer
- ffmpeg (on a Mac:
brew install ffmpeg) - Python 3.10 or newer for WhisperX
- A short clip of you talking, saved as
clip.mp4
Put everything in one folder:
my-reel/
clip.mp4 your video
words.js word timestamps (step 1)
reel.html the storyboard (step 2)
render.mjs the renderer (step 3)#1. Give every word a timestamp (WhisperX)
pip install whisperx
whisperx clip.mp4 --model small --language en --compute_type int8 --output_format json --output_dir .This writes clip.json with a start and end time for every single word. --compute_type int8 runs it fine on a laptop CPU. Use --model large-v3 for the best accuracy (slower), and change --language if you don't speak English in the clip.
Turn it into the small word list the storyboard reads:
node -e 'const d=require("./clip.json");const w=d.word_segments.filter(x=>x.start!=null).map(x=>({w:x.word,s:x.start,e:x.end}));require("fs").writeFileSync("words.js","window.WORDS = "+JSON.stringify(w)+";\n")'The result looks like this:
window.WORDS = [{"w":"capacity","s":0.312,"e":0.935},{"w":"for","s":1.457,"e":1.759}, ...];It's a .js file instead of .json on purpose: Chrome blocks fetch() on pages opened from your disk, but a script tag just works. Whisper sometimes squeezes numbers like "300,000" into odd timings. If a caption drifts, fix the time by hand in words.js.
#2. The storyboard: one HTML file
Save as reel.html. It has a 9:16 stage (1080 x 1920), your clip as the background, captions that appear word by word, one keyword in blue and one punch-in zoom on a spoken word. Animations use GSAP from a CDN.
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@700;800&family=Instrument+Serif:ital@1&display=swap" rel="stylesheet">
<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.13.0/gsap.min.js"></script>
<script src="words.js"></script>
<style>
html, body { margin: 0; background: #000; }
#stage { position: relative; width: 1080px; height: 1920px; overflow: hidden; background: #f4f6fb; }
#bg { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: cover; transform-origin: 50% 40%; }
#cap {
position: absolute; left: 60px; right: 60px; top: 1280px; text-align: center;
font: 800 92px/1.1 Inter, Arial, sans-serif; color: #111; letter-spacing: -1px;
}
#cap span { display: inline-block; background: #fff; padding: 6px 22px; margin: 6px; border-radius: 22px;
box-shadow: 0 10px 30px rgba(0,0,0,.18); }
#cap span.key { color: #1F4BFF; font: italic 400 112px/1 "Instrument Serif", serif; }
</style>
</head>
<body>
<div id="stage">
<video id="bg" src="clip.mp4" muted playsinline preload="auto"></video>
<div id="cap"></div>
</div>
<script>
// ---------- settings you edit ----------
const KEYWORDS = ["machine"]; // shown in blue italic serif
const GROUP = 3; // caption shows up to 3 words at a time
// ---------------------------------------
const words = window.WORDS; // [{w, s, e}] from WhisperX
const tl = gsap.timeline({ paused: true });
const clean = (x) => x.toLowerCase().replace(/[^a-z0-9]/g, "");
// W("word", n): start time of the n-th time you say "word" (n = 1 is the first time)
function W(word, n = 1) {
const hits = words.filter((x) => clean(x.w) === clean(word));
if (!hits[n - 1]) throw new Error(`word "${word}" #${n} not found`);
return hits[n - 1].s;
}
// captions: one word appears per spoken word, in groups of GROUP words.
// Pure function of t, so any frame can be rendered in any order.
const cap = document.getElementById("cap");
function caption(t) {
let i = -1;
while (i + 1 < words.length && words[i + 1].s <= t) i++;
if (i < 0 || t > words[i].e + 0.6) { cap.innerHTML = ""; return; } // hide during pauses
const start = i - (i % GROUP);
cap.innerHTML = words.slice(start, i + 1)
.map((y) => `<span class="${KEYWORDS.includes(clean(y.w)) ? "key" : ""}">${y.w.replace(/[.,!?]$/, "")}</span>`)
.join(" ");
}
// ---------- storyboard: every animation sits on a spoken word ----------
// punch-in zoom when I say "machine", back out 1.2 s later
tl.fromTo("#bg", { scale: 1 }, { scale: 1.25, duration: 0.12, ease: "power2.out" }, W("machine"));
tl.to("#bg", { scale: 1, duration: 0.3, ease: "power2.inOut" }, W("machine") + 1.2);
// ------------------------------------------------------------------------
const video = document.getElementById("bg");
// the renderer calls this once per frame
window.seek = async (t) => {
tl.seek(t, false);
caption(t);
if (Math.abs(video.currentTime - t) > 0.001) {
await new Promise((r) => { video.addEventListener("seeked", r, { once: true }); video.currentTime = t; });
}
};
window.ready = (async () => {
await document.fonts.ready;
if (video.readyState < 2) await new Promise((r) => video.addEventListener("loadeddata", r, { once: true }));
window.DURATION = video.duration; // the Reel is as long as your clip
tl.set({}, {}, window.DURATION);
await window.seek(0);
return window.DURATION;
})();
</script>
</body>
</html>Before you run it: replace "machine" (in KEYWORDS and in the two zoom lines) with a word you actually say in your clip. If the word isn't in words.js, the page tells you in the console.
How it works:
W("4K")returns the second you say "4K".W("4K", 2)is the second time you say it. Every animation is placed on a word, so if you re-record, the edit moves with you.- The GSAP timeline is paused. Nothing plays by itself.
window.seek(t)jumps the timeline, the captions and the video to exactly secondt, so every frame is the same every time you render. - To preview, open
reel.htmlin Chrome, open the console and typeseek(2.8).
#3. Render every frame with Chrome (Puppeteer)
npm init -y
npm i puppeteerSave as render.mjs:
// Render reel.html frame by frame with headless Chrome.
// Usage: node render.mjs (all frames, 4K)
// SCALE=1 MAX=30 node render.mjs (quick test: 30 frames at 1080x1920)
import puppeteer from "puppeteer";
import fs from "node:fs";
import path from "node:path";
const FPS = Number(process.env.FPS || 30);
const SCALE = Number(process.env.SCALE || 2); // 2 = 2160x3840 (4K)
const MAX = Number(process.env.MAX || Infinity); // limit frames for a test
const page_url = "file://" + path.resolve("reel.html");
fs.mkdirSync("frames", { recursive: true });
const browser = await puppeteer.launch({
headless: true,
executablePath: process.env.CHROME, // optional: use your own Chrome
args: ["--allow-file-access-from-files", "--autoplay-policy=no-user-gesture-required", "--hide-scrollbars"],
});
const page = await browser.newPage();
page.on("pageerror", (e) => console.error("page error:", e.message));
await page.setViewport({ width: 1080, height: 1920, deviceScaleFactor: SCALE });
await page.goto(page_url, { waitUntil: "networkidle0" });
const duration = await page.evaluate(() => window.ready);
const stage = await page.$("#stage");
const total = Math.min(Math.round(duration * FPS), MAX);
for (let i = 0; i < total; i++) {
await page.evaluate((t) => window.seek(t), i / FPS);
await stage.screenshot({ path: `frames/${String(i + 1).padStart(5, "0")}.png` });
if (i % 30 === 0) console.log(`frame ${i + 1}/${total}`);
}
console.log(`done: ${total} frames, ${duration.toFixed(2)} s at ${FPS} fps`);
await browser.close();Run it:
SCALE=1 MAX=30 node render.mjs # quick test: first second at 1080x1920
node render.mjs # the full Reel in 4K (2160x3840)The trick for 4K: the page is laid out at 1080 x 1920, and deviceScaleFactor: 2 makes Chrome draw every pixel twice as sharp. Text and shapes stay crisp, no upscaling.
If the background stays black, your Chrome build can't decode the clip. Render with your installed Chrome instead (Mac path shown):
CHROME="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" node render.mjs#4. Glue frames and sound with ffmpeg
ffmpeg -framerate 30 -i frames/%05d.png -i clip.mp4 \
-map 0:v -map 1:a -c:v libx264 -preset slow -crf 16 -pix_fmt yuv420p \
-c:a aac -b:a 192k -shortest -movflags +faststart reel.mp4-map 0:v -map 1:a: picture from your frames, sound from your original clip.-crf 16: very high quality. Instagram re-encodes anyway, so give it a clean source.-pix_fmt yuv420p: plays everywhere, including iPhones and Instagram.-movflags +faststart: the video starts playing before it is fully downloaded.
That's the whole pipeline. reel.mp4 is ready to upload.
#5. Optional: cut yourself out with Apple Vision
In my Reels my head pops out above the cards. On a Mac, Apple's Vision framework does this for free: VNGeneratePersonSegmentationRequest takes a frame and returns a mask of where the person is (use qualityLevel = .accurate for video). I export every frame of the clip with ffmpeg, run a short Swift script that saves one mask per frame, and draw the cut-out on a canvas above the background cards in the HTML. No green screen, no cloud, about a third of a second per frame on an M-series Mac.
#6. Edit by voice with Claude Code
Open the folder in Claude Code (cd my-reel && claude) and talk to it. These are the kind of prompts I use, copy and adapt:
Zoom in on my face when I say "4K" and zoom back out one second later. Anchor it with W() in reel.html, like the existing zoom.Add a short pop sound on every word in KEYWORDS. Write the sound times from words.js into sfx.json and extend the ffmpeg command so the pops are mixed quietly under my voice with adelay and amix.When I say "GitHub", slide screenshot.png in as a white card with rounded corners and a soft shadow in the top half of the stage. Hide it again when I say "anyway".Every time I say a number, show it as a big #1F4BFF counter that counts up from zero and lands on the number exactly when I finish saying it.Check reel.html against words.js: list every W() call whose word I never say, and fix caption words that Whisper misheard. Then render frames 80 to 100 at SCALE=1 and show me the frame where the zoom starts.Why this works so well: the storyboard is plain text, and every animation is tied to a word. Claude Code can read both, so "when I say X" is something it can actually do.
#7. The open-source alternative: HyperFrames
HeyGen open-sourced the same idea as a full framework: HyperFrames (Apache 2.0). Their tagline: "Write HTML. Render video. Built for agents." Use it if you want a bigger toolkit than this starter. Use the starter above if you want to understand every line.
Made by @sangeeths.ai: real AI engineering from Switzerland. Share this kit: sangeeths.ai/reel · All free kits: sangeeths.ai