Shoot all day. Let the machine edit.

The idea — two folders and a machine between them

The problem this solves: you shoot content all day, then lose the night to editing — or worse, the footage never gets cut at all. Editing is the bottleneck between shooting and posting.

The fix is a mental model so simple it fits in one sentence: an inbox folder for raw material, an outbox folder for finished work, and a pipeline that runs whenever the inbox gets a new day.

inbox/2026-07-11/            →   outbox/2026-07-11/
  clip001.mp4                       daily-cut.mp4        # the long cut
  clip002.mp4                       mid-1.mp4, mid-2.mp4 # 3-6 min chapters
  interview.wav                     shorts/short-N.mp4   # ~12 per hour of footage, + 9:16
  photos/*.jpg                      thumbnail.png
  music/*.mp3                       tweet.txt            # draft — a human always posts
                                    report.json          # why it made every choice

Everything else — transcription, moment selection, cutting, captions, music — is just what happens between those two folders. One hour of footage in, roughly a dozen 30-second shorts, two or three mid-length videos, and one long daily cut out.

The toolchain — what each tool actually does

Three free tools do all the work. Learning what they are is worth more than any subscription clipper, because they're the same tools every "AI video editor" SaaS is wrapping and charging you monthly for.

FFmpeg — the knife

ffmpeg is the command-line tool that reads, cuts, converts, and writes basically every audio/video format that exists. Every trim, every concat, every caption burn, every 9:16 crop, every music mix in this pipeline is one ffmpeg command. It has been free and open source for 25 years.

Whisper — the ears

Whisper is a speech-to-text model that runs on your own machine — no cloud, no per-minute billing. It turns the day's audio into a timestamped transcript, down to individual words. That transcript is the backbone of everything: it's how the machine knows what was said and exactly when.

Python + Pillow — the hands

A few hundred lines of readable Python decide what to cut and drive ffmpeg. Pillow (Python's imaging library) draws the caption graphics and the thumbnail. No frameworks, no build step — you can read the entire codebase in one sitting, and you should.

How the pipeline thinks — the six stages

1. Ingest — know what you have

ffprobe (ffmpeg's inspector) examines every file and classifies it: videos, voice recordings, music (anything in a music/ subfolder), and photos. Files are ordered by their real shoot time from camera metadata, not filename — so the day reassembles in the order you lived it.

2. Transcribe — turn sound into data

Whisper transcribes every clip with word-level timestamps. From here on, the pipeline doesn't work with video — it works with text that knows where it lives in the video. That's the core trick of all automated editing.

3. Select — find the moments

Every transcript segment gets a score: how densely the person is talking (excitement), whether it hits your configured keywords (for us: "wallet", "first time", "no way"), and punctuation energy. Top-scoring moments expand into 21-35 second windows for shorts; the best material becomes chapters and the daily cut.

4. Edit — cut, caption, mix

FFmpeg cuts the selected ranges, normalizes loudness to the broadcast standard (EBU R128, −16 LUFS), concatenates, and burns captions. Shorts get kinetic captions — word-by-word, all-caps, active word highlighted — because that style measurably holds attention (numbers in Lesson 4). If you dropped music in the folder, it's mixed under the speech with sidechain ducking: the music automatically dips whenever someone talks, exactly what a human editor does by hand.

The daily cut opens with a cold-open hook — the single strongest 8 seconds of the day — before going chronological. Shorts also get rendered as 1080×1920 verticals with captions in the platform-safe center zone.

5. Thumbnail — the first frame

The frame at the hook moment, with your title and brand strip drawn on. On X the first frame does the thumbnail's job, so the hook frame and the thumbnail are the same decision.

6. Draft the post — never send it

The strongest transcript line becomes the tweet's hook, your standing receipts get appended, and the draft lands in tweet.txt. Optionally a local AI rewrites it against a voice guide. It is never posted automatically.

The retention playbook — the numbers that matter

The pipeline's defaults aren't vibes — they come from sourced research on what holds viewers (full document with every citation: docs/RESEARCH.md). The short version:

For street-interview / IRL content specifically: lead with the reaction, not the setup. The genre-standard structure is answer-first — show the "no way, this is real?" moment, then let the context unfold.

Set it up — from zero to first cut

Step 1 — install the tools (macOS; Linux equivalents in the README):

brew install ffmpeg fswatch
pip install openai-whisper pillow

Step 2 — get the code and make your folders:

git clone https://github.com/TheCryptoAnarchist/based-content-pipeline
cd based-content-pipeline
cp config.example.toml config.toml

Step 3 — edit config.toml. The knobs that matter most:

• folders.inbox / outbox — your two folders.
• select.keywords — the words that mark a moment as interesting for your content. This is where the pipeline becomes yours: a cooking channel wants "taste", "crispy", "oh wow"; we want "wallet", "first time", "Phantom".
• select.shorts_per_hour, mid_cuts, max_cut_minutes — your output volume.
• thumbnail.title / brand, tweet.receipts — your identity on every output.

Step 4 — first run. Start with a dry run to see the cut plan without rendering:

python3 pipeline.py ~/Movies/inbox/2026-07-11 --dry-run   # prints the plan
python3 pipeline.py ~/Movies/inbox/2026-07-11             # renders everything

Step 5 — automate it. ./watch.sh watches the inbox and runs the pipeline automatically when a new day-folder finishes copying. Drop the card, walk away. First Whisper run downloads the model (~500MB, one time); a day of footage takes roughly real-time-or-faster to transcribe on Apple Silicon.

Learn from every run — the receipts

Every run writes two files that turn the pipeline into a teacher:

• report.json — what it found, what it scored highest, which moments became which outputs, and how long each stage took. If a cut surprises you, this file says why it happened.
• ffmpeg_commands.log — every ffmpeg command it executed, verbatim. Copy any line, change a value, run it yourself. This is how you go from "user of a tool" to "person who can edit video from the command line" — on your own footage, solving your own real problem.

That's the transparency philosophy applied to software: don't trust the pipeline, verify what it ran.

Limits — what this doesn't do yet

• It has no taste. Moment selection is a scoring formula. It reliably finds high-energy speech; it can't yet tell a great story beat from a loud one.
• Cuts don't snap to camera movement. Scene detection is planned; today a cut can land mid-pan.
• It doesn't know who's talking. Speaker diarization is planned — useful for interviews.
• The endgame is a learned model: log which clips actually perform, then train selection on receipts instead of heuristics — the same philosophy as the trading model. Watch the build log; this page and the defaults update as it improves.

Make it learn — the machine learning plan

The v0 selector is a formula. The endgame is a model that learns your taste and what your audience actually watches — and the discipline comes straight from our trading desk, where we learned it the expensive way: the model comes second, the labels come first. We once wired a classifier into a strategy before we had good labels; it confidently filtered the wrong things and got killed within a week. Never again.

The flywheel (live now)

• Log everything. Every run writes candidates.jsonl — every moment the pipeline considered, its features (speech density, keyword hits, energy, duration, position in the day), and whether it got picked. Rejected candidates matter as much as selected ones: a model can't learn "no" without seeing what "no" looked like.
• Rate the outputs. python3 ml/feedback.py rate <day> — one keystroke per short: keep or kill. Two minutes per day of footage. This human judgment is the ground truth.
• Add reality. When a short actually gets posted, ml/feedback.py perf records its views and likes — so eventually the model optimizes for what performs, not just what felt right.

The honesty gates (non-negotiable)

• No training below 300 labels. Small-sample models flatter themselves. The trainer literally refuses.
• Leave-one-day-out validation. The model is always tested on a day it never saw — the only score that predicts real-world behavior.
• It must beat the heuristic to deploy. ml/train_selector.py only writes the model file if its held-out AUC beats the formula it would replace. If it can't, the formula stays. No deploy without receipts.

Once deployed, the loop keeps turning: the model picks, you rate, it retrains, the picks get better. Later feature upgrades — audio energy, laughter detection, visual scene features — slot into the same flywheel without changing the discipline.