Video-use Tutorial: Complete AI Video Workflow from Script to Final Cut

In 2026, AI video splits into two very different paths: one uses Sora, Veo, and similar models to generate footage from text; the other uses Agents to cut existing material into a finished piece. The second path is more practical for talking heads, tutorials, interviews, and product launch reels—you shoot the footage, and AI handles filler removal, rhythm alignment, subtitle burn-in, color grading, and export.

Video-use takes the second path: an open-source Agent video editing skill pack from the browser-use team. Drop raw takes into a folder, tell Claude Code to "cut this into a launch reel," and wait for edit/final.mp4. No timeline GUI—the editing logic lives in helpers/ scripts and SKILL.md rules.

This guide walks you through the full workflow from script to final cut: concepts → install → hands-on → Cloud Mac deployment → cost and risk.

1. Background: why Agent editing matters

The most common pain point for indie developers and content teams isn't "can't shoot"—it's rough cut time: removing "um/uh," merging multiple takes, aligning subtitles, unifying color, and checking cuts for audio pops. Even a Premiere power user needs 1–2 hours for a 10-minute talking head.

Traditional automation (macros, scripts) fails because rules are brittle—change the content type and they break. Video-use delegates editing decisions to an LLM, but constrains it with structured text + on-demand vision so the model doesn't guess at frames blindly. The design mirrors browser-use giving Agents structured DOM instead of screenshots.

2. Core concepts: two reading layers and the pipeline

Video-use's core idea: the LLM never "watches" video—it "reads" it. Two information layers enable word-boundary-precision editing.

Layer 1 — audio transcription (always loaded)

Each source file is sent once to ElevenLabs Scribe, producing:

• Word-level timestamps: start and end for every word
• Speaker diarization: distinguish S0, S1 in multi-guest interviews
• Audio events: markers like (laughter), (applause), (sigh)

All takes are packed into a single ~12KB takes_packed.md—the Agent's primary reading view, formatted like:

## C0103  (duration: 43.0s, 8 phrases)
  [002.52-005.36] S0 Ninety percent of what a web agent does is completely wasted.
  [006.08-006.74] S0 We fixed this.

Compare that to a naive approach: 30,000 frames × 1,500 tokens = 45 million tokens of noise. The text surface cost is essentially negligible.

Layer 2 — visual composite (on demand)

When a cut point is ambiguous—should this pause stay, which take to pick, is the subtitle cropped—the Agent calls timeline_view to generate a PNG composite of filmstrip + waveform + word labels. Called only at decision points, not for full-frame analysis.

Five-step pipeline

Transcribe ──> Pack ──> LLM Reasons ──> EDL ──> Render ──> Self-Eval
                                                              │
                                                              └─ issue? fix + re-render (max 3)

The flow follows Ask → Confirm → Execute → Self-Eval → Persist: propose a strategy and wait for confirmation before executing; after render, self-evaluate at every cut boundary and auto-fix + re-render if needed (up to 3 rounds).

3. Install and environment setup

Video-use works with Claude Code, Codex, Hermes, OpenClaw, or any Agent that can run Shell commands. Below uses macOS + Claude Code as the example.

Option A: one-shot setup prompt (recommended)

Paste the following into an Agent session. It reads install.md, clones the repo, installs dependencies, registers the skill, and asks for your ElevenLabs API key:

Set up https://github.com/browser-use/video-use for me.

Read install.md first to install this repo, wire up ffmpeg, register the skill with whichever agent you're running under, and set up the ElevenLabs API key — ask me to paste it when you need it. Then read SKILL.md for daily usage, and always read helpers/ because that's where the editing scripts live. After install, don't transcribe anything on your own — just tell me it's ready and wait for me to drop footage into a folder.

Option B: manual install

# 1. Clone and symlink into Agent skills directory
git clone https://github.com/browser-use/video-use ~/Developer/video-use
ln -sfn ~/Developer/video-use ~/.claude/skills/video-use        # Claude Code
# ln -sfn ~/Developer/video-use ~/.codex/skills/video-use     # Codex

# 2. Install dependencies
cd ~/Developer/video-use
uv sync                         # or: pip install -e .
brew install ffmpeg             # required
brew install yt-dlp             # optional, for downloading online footage

# 3. Configure ElevenLabs API key
cp .env.example .env
# Edit .env: ELEVENLABS_API_KEY=sk_...

4. Hands-on: from footage to final.mp4

4.1 Inventory footage and confirm strategy

Put raw takes (.mp4, .mov, etc.) in one directory, e.g. ~/Videos/launch-takes/. Start the Agent from that directory:

cd ~/Videos/launch-takes
claude    # or codex, openclaw, etc.

In the session, say something like:

Cut this footage into a product launch reel—remove filler and pauses, add 2-word UPPERCASE subtitles, warm cinematic color grade.

The Agent will:

1. Inventory source files: count, total duration, speakers
2. Propose an editing strategy (filler removal, take selection logic, subtitle style, color preset)
3. Wait for your confirmation before executing—this is a hard rule

4.2 Transcribe and build takes_packed

After you confirm the strategy, the Agent calls Scribe on each source file, generates word-level transcripts, and packs them into takes_packed.md. It can now "read" the full talking-head content and plan the EDL (Edit Decision List).

You can intervene mid-stream, for example:

• "Keep the 18–22 second segment in C0103—that's the key demo"
• "For guest S1, keep only Q&A—cut all opening small talk"
• "At laughter markers, hold 0.5 seconds before cutting"

4.3 EDL and FFmpeg render

The Agent generates an EDL from the transcript and drives FFmpeg via scripts in helpers/:

• Cut filler: umm, uh, false starts, dead air between takes
• 30ms audio fade in/out: at every cut to avoid pops
• Auto color grade: warm cinematic, neutral punch, or custom ffmpeg chain
• Burn-in subtitles: default 2-word UPPERCASE chunks, fully customizable style

All output lands in <videos_dir>/edit/—the skill directory stays clean. The final file is usually edit/final.mp4.

4.4 Self-eval loop

After rendering, the Agent runs timeline_view at each cut boundary to inspect the finished piece, catching:

• Visual jumps (head position shifts, background flicker)
• Audio pops or insufficient fade-in
• Cropped or overlapping subtitles

Issues trigger EDL adjustments and re-render, up to 3 rounds. The preview you see has already passed self-eval—the biggest UX difference from traditional "export and discover a bad cut."

5. Advanced capabilities overview

Motion graphics overlay is a 2026 highlight: each animation is generated by an independent sub-Agent in parallel (HyperFrames, Remotion, Manim, or PIL). The main Agent only orchestrates the timeline and compositing—ideal for inserting diagrams, data cards, or brand intros in tutorials.

6. Cloud Mac and Apple Silicon deployment scenarios

Video-use is essentially "Python + FFmpeg + Agent Shell"—naturally suited to macOS cloud nodes, especially for these team profiles:

Scenario 1: remote batch footage processing

The shoot team captures locally and uploads to cloud storage; Claude Code SSHs into a Cloud Mac and runs the full pipeline on the node. M4 FFmpeg hardware acceleration typically finishes a 10-minute 4K talking head in minutes. Your local laptop only receives the final.mp4 notification.

Scenario 2: parallel with iOS dev pipelines

Many indie developers run Xcode and edit launch reels on the same Mac. Offload Video-use to a dedicated Cloud Mac node so FFmpeg doesn't saturate CPU and stall the Simulator. One Agent can serve both "build TestFlight" and "cut App demo video" task queues.

Scenario 3: always-on editing bot

Pair with Browser Use Box or a self-hosted VPS + OpenClaw: drop a footage link in Telegram → node auto-transcribes and edits → returns the finished piece. Suits weekly talking-head shows, course clip batches, and other fixed-output workflows.

7. Cost, performance, and risk

Cost estimate (10-minute single-speaker talking head, 3 takes)

Performance notes

• Transcription: network I/O bound; source bitrate matters less
• Render: M4 shines on 4K multi-track + color chains; 1080p single-track talking heads are fast
• Motion graphics: Remotion / Manim sub-Agents in parallel are the slowest optional step

Risks and limitations

• Not magic: complex multi-cam narrative and fine-tuned montage rhythm still need human final review
• Scribe quality: heavy accents and loud background music hurt word boundaries—manual correction may be needed
• API dependency: ElevenLabs outage blocks transcription; cache takes_packed.md ahead of time
• Subjective taste: 12 hard rules ensure technical correctness; color grade and pacing "look good" still need human sign-off

FAQ

What is Video-use, and how does it differ from traditional editing software?

An open-source Agent video editing skill pack: footage goes in a folder, you describe needs in conversation, the Agent reads transcripts + on-demand vision, FFmpeg outputs the final piece. Unlike manual NLE timelines, decisions are made by the LLM per rules—you confirm strategy.

What dependencies are required?

Python, FFmpeg, ElevenLabs API key, and a Shell-capable Agent (Claude Code / Codex / OpenClaw, etc.). Motion graphics optionally use Remotion, Manim, or HyperFrames.

Why doesn't the LLM just "watch" the video?

Frame-by-frame cost is extreme. Video-use uses a ~12KB transcript as the primary view, generating timeline PNGs only at ambiguous cut points—the same idea as browser-use using DOM instead of screenshots.

Is it a good fit for Cloud Mac?

Absolutely. Transcription and rendering are compute tasks; a remote Agent over SSH can batch-process unattended, solving the problem of FFmpeg competing with local Xcode builds.

How much does a single edit cost?

~$2–5 for a 10-minute talking head (transcription + Agent tokens) + cloud node time. Best for weekly shows, tutorial clip batches, and other recurring workflows.

Summary

Video-use pulls AI video back from "generate fantasy footage" to "efficiently cut real material": word-level transcription drives edits, on-demand vision aids decisions, FFmpeg guarantees output quality, and the self-eval loop reduces rework. For iOS developers, indie creators, and small teams, it's a reproducible, extensible, cloud-ready Agent workflow.

Three steps to get started:

1. Install the skill + FFmpeg + ElevenLabs key
2. Drop footage in a folder; confirm strategy before execution
3. Offload heavy work to Cloud Mac; collect final.mp4 locally

For Agent tool selection, see 2026 AI coding tools ranking; for OpenClaw remote orchestration, see OpenClaw remote Mac install guide.

Related reading

• Best AI Coding Tools in 2026 Ranked: Claude, Cursor, GitHub Copilot, Codex, Gemini
• OpenClaw Remote Mac Install & SSH/VNC Troubleshooting FAQ
• Personal AI Agent Architecture Triad
• Essential Open-Source Terminal Tools for Remote Mac Development

Features and pricing per the official video-use repository and ElevenLabs website. Last updated: August 6, 2026.