이게 불편했어요

에이전트가 출시 영상이나 제품 데모를 렌더하는 코드는 쓰는데, 거기 붙은 소리는 듣지 못해요. 타격음이 컷보다 80 ms 늦게 들어와도, 키 누르는 소리가 싸구려처럼 들려도, 음악이 엉뚱한 데서 커져도 에이전트는 알 수가 없어요. 결국 사람이 매번 직접 듣고 어디가 이상한지 말로 설명해 줘야 했어요.

그래서 이렇게 풀어요

earshot은 소리를 에이전트가 읽을 수 있는 증거로 바꿔요. 무슨 소리처럼 들리는지는 말로, 원하는 설명에 얼마나 맞는지는 점수로, 화면 컷과 얼마나 어긋나는지는 시간으로 알려 줘요. 들을 수 있으니 직접 만들 수도 있어요. 에이전트가 소리를 물리 합성 레시피 코드로 쓰면 earshot이 후보를 렌더해 듣고 평가하고, 에이전트는 그 평을 읽고 레시피를 고쳐 다시 돌려요.

earshot describe enter.ogg0.56 s short transient (attack 3 ms, decays in 190 ms). Sounds most like: a single mechanical keyboard key press(z +5.4), a soft key release click (z +3.6) ... [clear reading]. peak -0.1 dBFS, brightness 1.5 kHz ...earshot match key.ogg "a deep thocky mechanical keyboard press" --alts "a clicky blue switch keyboard" "a mouse click"-> strong (z +3.7; would rank #2 among 322 vocabulary descriptions) ...earshot find keys.ogg "a single mechanical key press" --top 40:03.94-0:04.19 (0.24 s stroke, z +4.1 strong); 0:10.60-0:10.86 ...earshot critique film.mp4VERDICT: close - 21 of 23 scored metrics in range ... FIX FIRST: 1. [major] nothing reaches body level until 0:01.01 ...

안에서는 이렇게 돌아가요

1. 1

   Claude Code → earshot MCP 서버

   describe enter.ogg 호출MCP 서버는 모델을 한 번 올려 두고, 도구 호출을 하나씩 차례로 받아요.earshot/mcp_server.py
2. 2

   earshot MCP 서버 → CLAP · AST 모델

   소리를 어휘 설명들과 비교CLAP이 소리를 vocab.json의 설명들과 견줘 z 점수를 매기고, AST가 AudioSet 라벨을 붙여요.earshot/describe.py
3. 3

   CLAP · AST 모델 → Claude Code

   '키보드 키 하나 누르는 소리 (z +5.4)'길이, 어택, 감쇠 같은 측정값과 함께 한 문단의 말로 돌아와요.earshot/describe.py
4. 4

   Claude Code → impact.py 레시피

   물리 합성 레시피를 코드로녹음 없이 접촉음, 공기, 막의 울림, 서브, 잔향을 층층이 계산하는 render()를 써요.earshot/forge/recipes/impact.py
5. 5

   impact.py 레시피 → forge 비평가

   후보 렌더 → 측정으로 먼저 거르기1/3옥타브 균형과 엔벨로프로 모델 없이 많이 거르고, 상위만 CLAP이 들어요.earshot/forge/search.py
6. 6

   forge 비평가 → Claude Code

   'decay +2.4 outside the refs'비평가의 말을 읽고 에이전트가 레시피를 고쳐 다시 돌려요.earshot/forge/critic.py
7. 7

   forge 비평가 → WAV · report.md

   WAV, report.md, compare.png, audition.wav고른 소리와 평, 참고 녹음 옆에 놓은 스펙트로그램, 번갈아 듣는 파일이 남아요.earshot/forge/__init__.py

나한테 맞을까요

What was hard

An agent can write the code that renders a launch film or a product demo, but it cannot hear the result. A hit that lands 80 ms after the cut, a key press that sounds cheap, music that swells in the wrong place: the agent has no way to notice. So a person has to listen every time and describe in words what is off.

What it does about it

earshot turns sound into evidence an agent can read: what a file sounds like in words, how well it fits a description as a score, and how far it sits from a picture cut in time. Hearing also makes making possible. The agent writes a sound as a physical-synthesis recipe in code, earshot renders candidates and judges them, and the agent reads the critique, rewrites the recipe and runs it again.

earshot describe enter.ogg0.56 s short transient (attack 3 ms, decays in 190 ms). Sounds most like: a single mechanical keyboard key press(z +5.4), a soft key release click (z +3.6) ... [clear reading]. peak -0.1 dBFS, brightness 1.5 kHz ...earshot match key.ogg "a deep thocky mechanical keyboard press" --alts "a clicky blue switch keyboard" "a mouse click"-> strong (z +3.7; would rank #2 among 322 vocabulary descriptions) ...earshot find keys.ogg "a single mechanical key press" --top 40:03.94-0:04.19 (0.24 s stroke, z +4.1 strong); 0:10.60-0:10.86 ...earshot critique film.mp4VERDICT: close - 21 of 23 scored metrics in range ... FIX FIRST: 1. [major] nothing reaches body level until 0:01.01 ...

How it runs inside

1. 1

   Claude Code → earshot MCP server

   calls describe enter.oggThe MCP server keeps the models loaded and takes one tool call at a time.earshot/mcp_server.py
2. 2

   earshot MCP server → CLAP and AST models

   the sound against the vocabularyCLAP scores the sound against vocab.json's descriptions as z values; AST adds AudioSet labels.earshot/describe.py
3. 3

   CLAP and AST models → Claude Code

   'a single mechanical keyboard key press (z +5.4)'It comes back as one paragraph, with measurements such as length, attack and decay.earshot/describe.py
4. 4

   Claude Code → impact.py recipe

   a physical-synthesis recipe in codeA render() that computes contact, air shock, membrane, sub and hall in layers, with no recording.earshot/forge/recipes/impact.py
5. 5

   impact.py recipe → forge critic

   renders, pre-screened by measurement1/3-octave balance and envelope rank many renders without models; only the best are heard by CLAP.earshot/forge/search.py
6. 6

   forge critic → Claude Code

   'decay +2.4 outside the refs'The agent reads the critic's words, rewrites the recipe and runs it again.earshot/forge/critic.py
7. 7

   forge critic → WAV and report.md

   WAV, report.md, compare.png, audition.wavThe picks, their critique, spectrograms next to the references and a file to listen in pairs.earshot/forge/__init__.py

Is it for you