AI Video Editing — Highlights & Clips

AI Highlight Reels & the Edit DSL

Turn a long talking video into a highlight reel. The CLI transcribes the video with Gemini, asks it to plan an edit, and renders the result locally with FFmpeg. The AI's plan is written as a .vedl edit script — a small, human-readable video-edit DSL (cuts, transitions, text/image overlays, color grading, a ducked music bed, loudness normalization, fades) that you can read, tweak by hand, and re-render without another AI call.

# One shot: transcribe → AI picks the highlights → render
5am media highlight talk.mp4 --brief "a 60-second teaser" --target-duration 60 --output teaser.mp4

# The transcript, .vedl, and raw model ops are always written next to the
# output for hand-tuning and debugging; tweak reel.vedl and re-render:
5am media highlight talk.mp4 --output reel.mp4
5am media edit render reel.vedl --output reel2.mp4

# Composable pieces
5am media transcribe talk.mp4                    # → talk.transcript.json (or --format srt|vtt)
5am media edit generate --transcript talk.transcript.json --source talk.mp4 -o talk.vedl
5am media edit render talk.vedl --output cut.mp4 --aspect 9:16

# Layer a ducked music bed under the edit (the `media mix` pipeline)
5am media highlight talk.mp4 --music bed.mp3 --music-style documentary --output reel.mp4

# Export the edit as a Video Editor project for touch-up in the browser
5am media edit export talk.vedl --new-album "Edit sources" --project-name "Launch teaser"

A .vedl script looks like this (every line optional except source):

source "talk.mp4"
cut 01:12 01:48.5   # the origin story
cut 04:03 04:41     # key insight
transition xfade 0.5
text "The Origin Story" atsrc 01:12 for 3 pos=bottom size=56
overlay "logo.png" pos=top-right scale=0.2 opacity=0.8
grade exposure=0.3 saturation=1.15
grade hsl reds hue=5 sat=10
music "bed.mp3" style=podcast
normalize target=-16
fade in 0.5
fade out 1

Commands:

• 5am media transcribe <video-or-audio>: Timestamped, speaker-labeled transcript via Gemini. --format json|srt|vtt (SRT/VTT burn in via media visualize --subtitles), --language hint, -o output path.
• 5am media edit generate: Transcript → .vedl edit plan. --transcript (json/srt/vtt) and --source (the video) are required; steer with --brief, --target-duration (seconds), --max-segments; add a bed with --music <file> / --music-style. The model's reasoning is preserved as # comments on each cut.
• 5am media edit render <edit.vedl>: Execute a script with FFmpeg. --output required; --aspect/--width/--height override the canvas (default: source dimensions); --font for title text; --upload-album/--new-upload-album to push the result into your library.
• 5am media edit export <edit.vedl>: Create a Video Editor project from the script — cuts, crossfades, titles, and compatible grading — for hand-polish in the browser. Uploads the referenced media into an album (--album/--new-album); requires login.
• 5am media highlight <video>: The one-shot wrapper (transcribe → generate → render). The transcript, .vedl, and raw model response are always written next to the output; transcriptions are cached locally so re-runs skip the paid call (--force-transcribe to refresh). --export-project <name> also creates the Video Editor project.

Steering the edit (highlight and edit generate):

• --brief: Your editorial instruction to the AI, in plain English — the single biggest lever on what the edit becomes. It rides along with the transcript when the model picks which segments to keep, so it can steer topic ("focus on the pricing discussion"), format ("a 60-second teaser that ends on the call to action"), audience ("a recap for people who already attended"), or selection style ("only the Q&A answers, skip the presentation"). Default: "an engaging highlight reel". The brief is recorded in the generated .vedl's header comment, so you can always see what instruction produced a given edit.
• --target-duration: Rough output length in seconds (default 120). The edit is trimmed toward it (±25%), dropping the weakest segments first — a hard bound the brief can't talk its way past.
• --max-segments: Cap on the number of kept ranges (default 12), for when you want a few long takes rather than many quick cuts.

DSL notes (the complete language — every op, defaults, time formats, and gotchas — lives in the VEDL Language Reference):

• cut keeps ranges (chronological); delete removes them — one style per script. text ... atsrc and overlay ... fromsrc anchor to source time and are remapped through the kept segments; at/from are output-timeline time.
• music stages and ducks the bed exactly like media mix — same presets (podcast, documentary, promo, ambient) and measurement passes; level=/duck= override. normalize is measured gain staging to a LUFS target plus a true-peak limiter.
• Rendering needs a local FFmpeg (4.4+; per-band grade hsl needs 5.0+ and degrades with a warning on older builds). Transcription and generation need a Gemini API key (https://5am.app/settings#keys); rendering a hand-written script needs no account. Free/unauthenticated renders carry a small "Powered by 5AM" watermark; 5am login removes it.

AI Short Clips & the Clip Maker App

Cut a long video into standalone short clips — Shorts/Reels/TikTok-shaped, with burned-in subtitles and an AI title on each. One cached transcription, one Gemini call that picks the strongest self-contained moments (a hook, a complete thought, a payoff — strongest first), then a fast local render per clip. Portrait and square output fill the frame with a center crop by default; --fit contain letterboxes instead.

For footage that's more about what's on screen than what's said — a sports rally, a pet doing something, a reaction, b-roll with little or no speech — add --visual. Instead of the transcript, a low-resolution (360p) copy of the video is analyzed by Gemini, which returns the moments as time ranges; there's no transcript and no burned-in subtitles, but each clip still gets an AI title and the same editable .vedl. This also happens automatically as a fallback: if a source has no audio, or the speech is too sparse to build clips from, clips switches to visual selection on its own instead of failing.

# Five 9:16 clips with subtitles → talk-clips/
5am media clips talk.mp4

# Steer it: what to capture, how many, how long, which look
5am media clips talk.mp4 --count 3 --max-clip-len 30 \
  --brief "the product demo moments" --style bold

# Keep the original shape, skip subtitles and titles
5am media clips talk.mp4 --aspect source --style none --no-titles

# Pick clips from the FRAMES, not the speech (no transcript, no subtitles)
5am media clips rally.mp4 --visual --brief "the best rallies"

# Upload the results straight into an album
5am media clips talk.mp4 --new-upload-album "Shorts"

Every clip writes three artifacts next to the render: clip-NN.mp4, clip-NN.srt (its subtitles), and an editable clip-NN.vedl — the same edit language as highlight reels (full spec: VEDL Language Reference), so you can nudge a cut point or retitle a clip and re-render with 5am media edit render, no AI call needed. manifest.json and transcript.json round out the set.

• --count (1–10, default 5), --max-clip-len (seconds, snapped to sentence boundaries), --brief (what the clips should capture), --aspect source|9:16|1:1|16:9, --fit cover|contain.
• --style clean|bold|minimal|none: burned-in subtitle looks. Both subtitles and the clip title are burned via libass, so RTL (Farsi/Arabic) and CJK scripts shape and reorder correctly and a covering font is found automatically — no --font needed for titles (install the Noto families if a script shows as boxes).
• --visual: choose clips from the video's frames instead of its speech — a 360p copy is analyzed by Gemini and the moments come back as time ranges. No transcription and no burned-in subtitles (so libass isn't needed); best for footage with little or no speech. Kicks in automatically when a source has no audio or its speech is too sparse to build clips from.
• Requires a Gemini API key and a local FFmpeg with libass (the subtitles filter) unless --style none or --visual. Transcriptions are cached, so re-runs with a different brief or count skip the paid call.

Clip Maker in the browser — 5am studio

The Clip Maker web app is the same pipeline with a visual workflow: pick a video — from your computer or straight from your 5AM library — watch clips appear, edit the transcript or any clip's script in place, then publish the keepers to your library or a Video Editor project. It talks to a companion server on YOUR machine, so local videos never upload (library picks are downloaded once to your machine and cached) — that's what keeps it free.

5am login
5am studio          # then open 5am.app/clip-maker
5am studio --dir ~/Videos --port 5200

Clips vs. highlight — which one?

Both are transcript-driven by default (clips can also select by frames — see --visual), share the same cached transcription (running both on one video pays for one transcript), and render through the same VEDL engine — but they answer different questions. highlight answers "summarize this video": one composed reel that compresses the whole thing. clips answers "mine this video for posts": independent moments, each strong enough to stand alone in a feed.

Use highlight for a teaser or recap; use clips (or Clip Maker) when the destination is Shorts, Reels, or TikTok.

Podcast & Video Tools

Convert audio files into engaging videos with animated waveforms or full AI-generated b-roll.

# Render a waveform video
5am media visualize episode.wav --cover cover.jpg --output episode.mp4

# Auto-generate video b-roll for an entire podcast
python3 scripts/podcast_to_video.py -i episode.wav -s episode.srt -a 9:16

Command Arguments for visualize:

• <audio>: Path to the local input audio file.
• --output: The destination path for the rendered MP4 video.
• --cover: An image file (.png, .jpg, .webp) to use as the video background.
• --subtitles: Path to a .srt or .vtt file to burn timestamp-synced captions into the video.
• --aspect: Aspect ratio shortcut (e.g., 16:9 for 1280x720 landscape, 9:16 for 1080x1920 portrait/reels, 1:1 for 1080x1080 square).
• --width & --height: Explicit pixel dimensions (overrides --aspect).
• --style: Waveform visualization style (e.g., showwaves, showfreqs, showcqt, showspectrum).

Arguments for podcast_to_video.py:

• -i: Path to the input audio file.
• -s: Path to the subtitle or transcript file (.srt).
• -a: Aspect ratio for the generated b-roll (16:9, 9:16, 1:1).

Automatic Music Ducking (media mix)

Mix a voice or dialogue track over background music like a broadcast mixer: the music is staged at a bed level under the voice, ducked just enough to keep every word on top, and ducked deeper in the speech band (250 Hz–4 kHz) — an EQ pocket carved right where the voice lives. Requires a local FFmpeg (4.4+).

# Podcast defaults: bed 18 dB under the voice, tight moves, -16 LUFS output
5am media mix --voice episode.wav --music bed.mp3 --output mix.wav

# Pick a style and output format (wav, mp3, m4a, flac)
5am media mix --voice narration.wav --music score.mp3 --style documentary --output cut.m4a

Command Arguments for mix:

• --voice: The dialogue/voice audio file (required).
• --music: The background-music audio file (required). Shorter music is looped and faded out at the end; the output is as long as the voice track.
• --output: Destination audio file — the extension picks the format (.wav, .mp3, .m4a, .flac).
• --style: Mix preset — podcast (default), documentary, promo, or ambient.
• Fine-tuning overrides: --bed-level, --duck-depth, --pocket-depth (0 disables the EQ pocket), --ratio, --attack, --release, --target-lufs, --true-peak, --fade-out.

Free/unauthenticated runs append a short spoken "Powered by 5AM" tag at the very end of the mix — log in (free) to remove it.

Memory Collage Video

Turn a set of photos and/or videos into a montage ("memory collage") video, with an optional AI-generated Lyria background-music track, and write it to a local MP4. Stills get a slow Ken Burns pan/zoom and are cross-faded together; videos are normalized to the canvas and joined inline. Like visualize and concat, this requires a local FFmpeg binary.

# Photos from a folder, generated Lyria music, vertical (Reels/Stories)
5am media collage --folder ./photos --output memory.mp4 --music "warm nostalgic piano" --aspect 9:16

# Explicit ordered files; supply your own track instead of generating one
5am media collage img1.jpg img2.jpg clip.mp4 --output memory.mp4 --music-file track.mp3

# Source from a remote 5AM album (downloaded first), then upload the result
5am media collage --album "Trip 2026" --output trip.mp4 --music "upbeat indie" --upload-album "Memories"

# Tune pacing, or disable motion for a hard-cut slideshow
5am media collage ./photos/*.jpg --output m.mp4 --seconds-per-photo 3 --transition 0.5
5am media collage ./photos/*.jpg --output m.mp4 --no-kenburns --transition 0

# Fit the whole photo in (letterbox) instead of cropping — for faces near the edges
5am media collage ./photos/*.jpg --output m.mp4 --aspect 1:1 --fit

# Keep the fill crop, but use AI to center it on each photo's main subject
5am media collage ./photos/*.jpg --output m.mp4 --aspect 1:1 --smart-crop

Command Arguments for collage:

• [paths...] / --folder / --album: The source media — pass exactly one. Positional paths or a local --folder, or a remote 5AM --album (ID or name, downloaded first). Stills: .jpg/.jpeg/.png/.webp; videos: .mp4/.mov/.m4v (HEIC is skipped — convert it first with 5am media convert).
• --output: The destination path for the rendered MP4.
• --music: A Lyria prompt for the generated background track (e.g. "warm nostalgic piano").
• --music-file: Use an existing audio file as the track instead of generating one (mutually exclusive with --music).
• --aspect: Aspect ratio shortcut (16:9 default, 1:1, 9:16). Overridden by explicit --width/--height.
• --seconds-per-photo: How long each still is held on screen (default 4).
• --transition: Crossfade duration in seconds between consecutive items (default 0.5; 0 = hard cuts). Crossfades apply to all-stills sets; sets that mix in a video use hard cuts.
• --no-kenburns: Disable the Ken Burns pan/zoom on stills (static hold instead).
• --fit: Fit each item fully inside the canvas (letterbox/pillarbox) instead of filling and cropping it. By default items fill the frame and the overflow is cropped (biased toward the top so faces near the top survive); use --fit when subjects sit near the edges, or pick an --aspect closer to your sources.
• --smart-crop: Keep the fill crop, but use AI (Gemini Vision) to detect each still's main subject and center the crop on it so faces/subjects aren't cut off. Stills only; requires a Gemini API key. Detected bounding boxes — and failures — are cached locally (~/.5am/collage-subjects.json, override with --smart-crop-cache) so re-runs are instant and a photo that consistently fails detection isn't retried every run (deterministic failures are recorded once; transient ones retry a few times). Detection failures fall back to the default crop. Ignored with --fit.
• --smart-crop-cache: Path to the JSON file caching --smart-crop subject boxes (default ~/.5am/collage-subjects.json).
• --upload-album / --new-upload-album: After rendering, upload the finished video into an existing album, or create/reuse one by name.