Why AI clips feel slightly off (and how we decide where to cut)

Build-in-public notes from Katto, an AI video clipper built by one person.

There is a small thing that separates a clip that feels professional from one that feels "automated": where it starts and where it ends.

A great clip opens on the hook and lands on the payoff. A weak one starts mid-thought, or runs one sentence too far. You hear the punchline, and then the speaker adds "and so anyway, moving on" right before the cut. That tiny overshoot is enough to keep a viewer's thumb scrolling.

It turns out this is one of the harder problems in automated clipping, and almost nobody talks about it. So here are some honest notes on it.

Most tools cut on signals that don't understand speech

The common way to "find the best moments" is to score a video on signals that are cheap to compute: volume peaks, scene changes, face detection, silence gaps. Those tell you something is happening, but not what is being said.

So the cut lands on a pause or a scene change, not on the end of a thought. And silence rarely lines up with meaning: people pause in the middle of a sentence, then barrel straight through the actual payoff without taking a breath. Cut on silence and you will routinely stop a beat too early, or a clause too late.

That is the root of the "slightly off" feeling. The clip is technically clean and semantically broken.

The principle: meaning decides where, not the audio

The shift that moved quality more than anything else was to stop treating the cut as an audio problem and start treating it as a comprehension problem.

Instead of asking "where is there a pause?", the right question is "where does a self-contained story begin and end?": an opening that creates curiosity, and a payoff that resolves it. That is a judgment about meaning, and it is the part a good human editor does instinctively: this arc starts here, it lands there.

Audio signals still matter, but for the opposite job. Once the meaningful boundary is chosen, they are what make the edge sound clean, snapping it to a natural break instead of chopping a syllable in half.

So the mental model is simple: meaning decides where the clip is; sound decides how the edge feels. Most tools only ever do the second half, which is why their clips feel automated even when the audio is clean.

The honest part: the edges are hard

I am not going to tell you this is solved. It isn't, fully.

Even with the right beat chosen, the last stretch, trimming a clip so it ends exactly on the payoff and not half a sentence later, is genuinely difficult, because human speech doesn't arrive in tidy sentences. It is the kind of detail you only notice when it is wrong, which is exactly why it is worth obsessing over. There are still edge cases I am actively refining, and anyone who tells you their AI cuts are flawless is selling you something.

But framing it as "meaning first, edges second" is the thing that made clips stop feeling robotic. That is the whole game with this kind of product: the invisible details are what you are actually paying for.

Katto turns long videos into short clips. Try it. This is the sort of problem I chip away at in public.