How to edit videos with Claude Code and Whisper

01How it works

Editing a talking video is mostly deciding what stays and what goes. And that decision is about text: which sentence opens, where the argument closes, which take was better, where the pauses are. That's why AI can do the job without "watching" the video:

1. Whisper transcribes word by word, with the exact second each word starts and ends. It's OpenAI's free speech-to-text model, and it runs on your own computer.
2. Claude Code reads the transcript and makes the editor's calls: which parts become clips, what the title says, which take to drop, what Whisper misheard.
3. ffmpeg does the work: cuts at the chosen seconds, removes the pauses, builds the vertical layout, draws the title and captions and normalizes the volume.
4. Claude checks the result by looking at frames of the finished video before handing it to you.

The video-editing skill bundles these four steps: the scripts do the mechanical part and SKILL.md teaches the agent the editing criteria.

02What you need

• Claude Code (Claude Pro or Max plan) or Codex (ChatGPT Plus or Pro). If you don't have one yet, step 2 of the ad accounts guide covers the install.
• ffmpeg and Python 3. On a Mac with Homebrew: brew install ffmpeg python. On Windows, use WSL (Ubuntu) and follow the Linux commands: sudo apt install ffmpeg python3 python3-venv.
• yt-dlp (optional), to download YouTube videos: brew install yt-dlp.
• About 2 GB free for the Whisper model, which downloads itself on the first transcription.

03Install the skill

Download the .zip and, from the folder it was saved to, run:

cd ~/Downloads
mkdir -p ~/.claude/skills ~/.agents/skills
unzip -o video-editing.zip -d ~/.claude/skills
unzip -o video-editing.zip -d ~/.agents/skills
bash ~/.claude/skills/video-editing/scripts/setup.sh

setup.sh checks ffmpeg and creates a Python environment in ~/.video-editing with Whisper and Pillow (which draws the title and captions). You only do this once. If the agent was already open, restart it so it sees the skill.

Then create one folder per video. Media is heavy: if the folder is a Git repository, keep video and audio out of it.

mkdir -p ~/videos/2026-10-clips/raw && cd ~/videos/2026-10-clips
printf '*.mp4\n*.mov\n*.wav\n*.m4a\n' > .gitignore

04Clips from a long video

This is the highest-leverage use: a long video you already recorded becomes a week of Reels. Open Claude Code in the video's folder and ask:

Use the video-editing skill. Download https://www.youtube.com/watch?v=YOUR_VIDEO
and make 30 to 80 second clips of the moments where I say something important
and finish a full line of reasoning. Layout: horizontal video in the middle,
black above and below, a catchy title above and captions below.

Here's what happens, in order:

1. Download with yt-dlp into raw/source.mp4. If YouTube answers with a 403 error, the agent uses your browser's cookies (--cookies-from-browser chrome), which fixes it.
2. Transcription of the whole video, once, into data/words.json (every word with start, end and confidence) and data/transcript.txt (readable, with timestamps).
3. Picking the moments. Claude reads the transcript end to end and returns a list: time range, title and why that part works on its own. You approve, drop or ask for others.
4. Framing. It grabs a frame, looks at where your face is and adjusts the crop to center it.
5. Dry cut + layout (video.py cut): removes every pause longer than 0.4 s, builds the 1080×1920 video and normalizes the audio to −14 LUFS, the loudness Instagram and TikTok expect.
6. Caption proofreading and render (video.py render): title and captions burned into the video, delivered to out/ with a contact sheet for checking.

My 14-minute video gave 9 clips, from 37 to 75 seconds. Together they lost 35 seconds of pauses alone, from almost nothing in one clip to 17 seconds in another where I stopped to think. Rendering takes under a minute per clip.

Every decision lives in one file, project.json. To change a title or extend a clip, you ask in plain English and the agent edits it there and renders again:

{
  "source": "raw/source.mp4",
  "layout": "letterbox",
  "crop_x": 136,
  "clips": [
    {"id": "05-trust-yourself", "start": 560.40, "end": 633.20,
     "title": "Why you don't trust yourself (and how to fix it)",
     "why": "Problem, solution and result in the same stretch."}
  ]
}

05How to pick the moments

This is what separates a good clip from a random excerpt, and it's what SKILL.md teaches the agent. A good clip:

• Opens on a hook that works without context: a claim, a number, a question. It never starts on "and", "so", "because".
• Closes a full argument: claim, explanation or example, conclusion. If the payoff comes 20 seconds later, the clip stretches to it or gets dropped.
• Runs 30 to 80 seconds. Shorter rarely finishes an idea; longer loses people halfway.
• Starts and ends on a word boundary, never mid-syllable. words.json gives the exact time.

The title is the hook in writing: concrete, in the video's language, up to two lines, and the clip has to deliver what it promises. A few from my video: "There are only 2 ways your brain builds beliefs", "Trauma is just a bad belief", "Your life is your fault. That's good news."

06Title, captions and layout

For clips I use the "letterbox" layout, the standard on many podcast and self-improvement pages: the 16:9 video in the middle, enlarged about 20% to fill the width, black above and below. I measured the positions on a reference Reel and locked them into the skill:

• Title: the system font (SF Pro on a Mac), semibold, white, split into two lines of similar width. If it doesn't fit, the font shrinks on its own.
• Captions: 2 to 3 words at a time, white, no effects, right under the video. They break on punctuation and pauses, so the end of one sentence never shares the screen with the start of the next.
• No second transcription: when the script removes the pauses, it recomputes every word's timing on the cut video. Cuts land exactly on the source's frames, so captions don't drift even with 50 cuts.

To change the look, ask the agent: size, position, outline and font (any .ttf) are fields in project.json. Like the change? Say "make it the default".

07Proofread the transcript

Whisper gets almost everything right, and the danger is exactly the mistake that sounds right. With an accent, "can't" becomes "can" and the sentence now says the opposite. SKILL.md tells the agent to read every caption for meaning, not only the low-confidence words, and to list everything it fixed by inference.

Fixes live in project.json as "wrong → right" pairs, so they apply to every later render:

"fix": [["Claude Coat", "Claude Code"], ["6 .30", "6:30"]]

• Names and brands come out phonetic. Pass the main ones to the transcription with --prompt "Claude Code, Whisper, Shopify" and Whisper gets far fewer wrong.
• Numbers break into pieces ("6.30" becomes "6" and ".30"). fix handles it.
• A stray word at the start: if the clip begins right after an "and" or "because", remove it from the captions or start the clip a little later.

08Editing from scratch

The other case: you recorded yourself talking to the camera, with pauses, "ums", repeated sentences and bad takes, and you want the finished Reel. Same skill, with the vertical layout:

Use the video-editing skill on raw/take.mov. Dry cut: remove pauses, filler
words and repeated takes (keep the last complete one). Punch-in on every cut,
captions, the title "I raised my prices and lost zero clients" for the first
3 seconds and raw/music.m4a quietly underneath.

1. Repeated takes: the agent reads the transcript looking for the same sentence said twice, sentences abandoned halfway and "wait, let me say that again". The last complete take stays, unless an earlier one is clearly better. The others go into remove.
2. Filler words: Whisper usually cleans "um" and "uh" on its own. The ones that show up go into fillers and are cut from the whole video. Only list words that never carry meaning.
3. Pauses: removed the same way as in clips, by volume.
4. Punch-in: on every cut, the framing alternates between normal and 12% tighter. It's the classic trick that keeps a jump cut from looking like a jump.
5. Horizontal footage becomes vertical, cropped around your face. Footage shot upright is used as is.
6. Music loops under the voice and dips on its own while you talk (ducking). Everything comes out at −14 LUFS.

{
  "source": "raw/take.mov",
  "layout": "vertical",
  "punch_in": 1.12,
  "title_seconds": 3,
  "fillers": ["um", "uh"],
  "music": "raw/music.m4a",
  "music_db": -22,
  "clips": [
    {"id": "reel-pricing", "start": 1.2, "end": 94.0,
     "title": "I raised my prices and lost zero clients",
     "remove": [[12.4, 18.9], [40.0, 43.2]]}
  ]
}

Recorded one long take covering several topics? Each topic becomes an item in clips, and you get one Reel per item.

09Check before posting

Every render writes, next to the video, a sheet with 6 frames spread across the clip. The agent opens that image and checks that the title is whole, the face is centered and the captions don't cover anything important. When in doubt about a moment, it grabs that exact frame and looks.

• Safe area: Instagram and TikTok cover roughly the top 220 px and the bottom 380 px with buttons and the post caption. The skill's layout already stays clear of them.
• Versions: nothing is overwritten. Each new render becomes -v2, -v3, and you compare.
• Posting is up to you. The skill delivers the files to out/ and stops there. I AirDrop them to my phone and post from the app.

10What to ask next

With the skill installed, a few requests that work well:

Give me 5 more clips from this same video, only from moments we haven't used yet.

Redo clip 03 starting one sentence earlier and with a shorter title.

Write the post caption and 3 hook options for each clip in out/.

Transcribe this podcast and tell me which parts are most likely to go viral, and why.

11FAQ

Can I make AI video clips for free?

Yes. Whisper, ffmpeg and the video-editing skill are free and run on your computer, with no watermark and no minute limits. The only cost is the agent subscription: Claude Pro or Max (Claude Code) or ChatGPT Plus or Pro (Codex).

Can Claude edit videos?

Yes, without watching the whole video. Whisper transcribes word by word with the exact timing of each word, Claude Code reads the transcript and makes the editor's decisions (segments, title, takes, pauses), ffmpeg cuts and assembles, and Claude checks frames of the finished video before handing it over.

Does it work with YouTube videos?

Yes. The skill downloads the video with yt-dlp. If YouTube returns a 403 error, the agent uses your browser cookies (--cookies-from-browser chrome).

Does it work in languages other than English?

Yes. Whisper transcribes dozens of languages, Portuguese included. For names and brands, pass the main ones with --prompt and Whisper gets them wrong far less often.

Do I need a Mac?

No. On Apple silicon Macs (M1 and later) the skill uses mlx-whisper, which transcribes a 14-minute video in one or two minutes. On Windows (through WSL) and Linux it uses faster-whisper, which works the same, just slower.

What is the ideal clip length for Reels and TikTok?

30 to 80 seconds. A clip opens on a hook that works without context and closes a full line of reasoning. Shorter rarely completes an idea; longer loses people halfway.