Product Video Audio: Check It With Earshot and AI

You made a demo video with AI and it looks ready. Then you watch it again: the click lands after the button, the music buries the voice. With Earshot and an AI agent you can get a list of audio issues to fix, each with its exact moment, before you publish.

An honest note first: we have not installed or tested Earshot, or listened to its output. This piece draws on Alek's articles and the project's documentation. It's a workflow, not a review and not a promise of savings.

What It Is

Earshot, by Leon Lin, is an open-source tool that gives coding agents "ears". It turns sound into descriptions, measurements and pictures an agent can read. According to the README it runs locally on your CPU, with no GPU, and works from the command line, from Python or through MCP (a connection that lets a compatible agent call its functions).

It isn't a point-and-click video editor. Whoever produces your videos installs and connects it. You just need to know what to ask for.

Why It Matters

An AI agent can edit a video, but it can't hear it. So audio problems show up at the last minute, found by you or, worse, by a client.

Earshot gives you a review with specific notes: which event, at what moment, with what problem. "At 0:12 the click doesn't match the action" takes a minute to fix. "The audio feels off" doesn't.

The functions that matter here, according to the README:

• timeline: sound events over time, with levels, labels, sections and alignment with picture cuts.
• sheet: one image with video frames, labelled events, a spectrogram and levels.
• critique: compares the video with a profile built from seven professional launch films and returns a verdict with timestamped suggestions.
• forge: the agent writes a sound "recipe" in code, Earshot renders and scores variants, the agent rewrites the recipe.

How To Do It

1. Get the setup done. The README requires uv and ffmpeg, with ffprobe available. The first time you use the listening features it downloads two models, about 1.1 GB. For repeated analysis use the MCP server: it keeps the models loaded, while each command-line call reloads them.
2. Ask for a map of the sound. Run timeline and sheet on the video. You get events over time and one image to look at with whoever edits.
3. Treat critique as questions, not a grade. The profile describes a launch-film style, not the right way to make every video. A quiet demo doesn't need an effect at every screen change.
4. Fix one thing. Move an effect, lower a level or cut an unnecessary sound. Then analyse the updated file and compare it with the previous version.
5. Listen to the whole thing, on a phone speaker too. Can you follow what's being shown? Does the music leave room for the voice? Does the ending support the message? You answer these, not the score.

If whoever works on the video has an agent connected to Earshot, hand them this prompt:

If you commission the video, add these lines to the brief:

The brief should also say who sets up the tools, who may handle the files and who signs off on the edit. Nobody needs to become a sound engineer.

Advanced step: change the sound, not just move it. With forge the agent can create a new effect from a recipe in code. Earshot also ships a bank of effects stored as code, rendered when needed.

For a small team, though, the place to start is reviewing what you already have. One more effect won't rescue a confusing demo. First decide which actions deserve a sound cue and where silence works better.

Small Test

Take a demo video you already have, ideally a short one. With the setup ready, run only timeline and ask for the three most suspicious moments, with timestamps. Jump to those seconds and listen on headphones and on a phone.

It worked if at least one note is a real problem you hadn't noticed, and it's precise enough to fix without asking anything else.

Watch Out For

• It doesn't hear words. The documentation says Earshot detects voice but doesn't understand speech. This "listening transcript" is not a transcript of what's said: copy, product names and pronunciation need a separate check.
• Measurements, not taste. The project's documented limits separate measurement from judgement. Levels and timing are one thing; emotion and brand fit are another. In a busy mix, event labels are clues, not facts.
• The llm function sends audio out. It's optional: it uploads audio to Gemini, with your own key and explicit consent. It's not part of the local workflow. With confidential footage, don't turn it on unless that transfer has been approved.
• Licences. The code is MIT (see the LICENSE file); the README lists Apache-2.0 for the CLAP model and BSD-3-Clause for AST. None of this lets you use any music or recording you like: check every asset, including when you turn it into a sound recipe.
• The tests are the project's own. The validation notes are not independent Studio Futuro tests and don't prove higher sales or lower costs.

Finishing the render isn't finishing the video. The same method (clear rules, one fix at a time, your own final check) works for design too: see how to redesign an app with AI without breaking the code.

Sources: Alek's articles on Medium and Substack, checked against Earshot's documentation.

In short: A video isn't finished when the render ends: have the agent list the audio issues to fix, each with its exact moment, then listen yourself before you approve.