Aaron Helwig

Software engineer building AI products
Personal projects

Heard

Live transcripts an agent can follow

I built Heard so an agent can follow what is being said around my Mac while it is still being said. It records my microphone and the Mac’s own output as two separate tracks, transcribes both in real time, and writes final, timestamped utterances to a local file.

Two tracks: your microphone and the Mac’s output. Music is excluded and never captured. Only finished sentences reach memory.jsonl:

  1. 10:00:04, remote: Can we ship the onboarding fix today?
  2. 10:00:09, you: Yes, once the tests pass.
  3. 10:00:15, remote: Great. Aaron, can you write the release notes?
Your microphone and the Mac’s own audio stay separate; an excluded app is never captured.

Small on purpose

Start, pause, stop and follow are the whole interface. Only finished sentences enter the record; the words still changing while someone speaks stay out of it. Stopping finishes any pending speech before closing the recordings, and a dropped connection shows up as an explicit gap rather than silence.

Some apps should never be heard at all. Excluded apps are removed from the system capture itself, and Heard rebuilds the capture when one launches or quits. If that refresh fails, system audio stops being delivered and Heard says so, rather than carrying on with an uncertain filter.

What can follow it

Heard stops at the transcript. One “heard follow” process streams new lines as they are written, so an agent can wait quietly for minutes and still react the moment something matters. Paired with my other tools, that becomes a meeting assistant that messages me when my name comes up, a hands-free helper that collects instructions spoken minutes apart, or a companion that answers out loud while we watch the same video.

  1. A meeting you are half-listening to: Let’s split the launch work. Design goes to Mia. And the API part to Aaron. heard follow notices your name came up. Messages, Codex: You were just given the API work in the launch call.
  2. Instructions, minutes apart: Rename the deck to Q4 review. 4 minutes later And send it to Sam. heard follow notices two instructions, one task. Finder, Sent to Sam: Q4 review.key, renamed and attached.
  3. Watching a video together: …we run the ensemble fifty times… Wait, what’s an ensemble here? heard follow notices a question for me. Spoken reply: Fifty slightly different forecasts, compared to see how certain the storm track is.
Three things an agent can build on a transcript it follows.
Personal projects