Content

Audio Transcription & Diarization with AI Agents

Use an AI agent to turn meeting, podcast and interview audio into text and automatically separate speakers — one chain all the way to a structured summary, with no manual typing or tool-switching.

Audio Transcription & Diarization with AI Agents

Tool names in this guide are for reference only and may have changed or been renamed. Please use the latest tools in the tool marketplace when implementing.

Key takeaways

  • Turn audio into text with speaker separation in one request
  • Transcribe and diarize in one chain, straight to a structured summary
  • Pick plain transcription vs transcription + diarization by use case

Tools used

Why audio-to-text is such a chore

An hour-long meeting, a podcast episode, a recorded interview — turning any of these into searchable, quotable text usually means manual transcription: rewinding, working out who's speaking, then hand-assembling notes. It's slow, and in multi-person conversations the "who said which line" is exactly what gets muddled. The resulting transcript can't tell speakers apart, so downstream summaries, follow-ups and action items all suffer.

The real difficulty has two layers. First, the transcription itself has to be accurate and keep up with casual speech and jargon. Second, diarization — marking "this part is speaker A, that part is speaker B." Chain those two together and hand them to an AI agent connected to speech tools, and "listen for an hour, tidy up for half a day" collapses into a single-sentence operation.

The workflow in four steps

  1. Give the agent the audio: upload the file or pass a link.
  2. Transcribe to text: convert speech to text with a transcription tool.
  3. Separate speakers: label who said what, producing a speaker-tagged transcript.
  4. Produce a structured summary: have the agent turn the transcript into a summary, decisions and action items.

How to do each step

Step 1: Hand the audio to the agent

Give the agent the recording file or an accessible audio link, and say what you want — just a transcript, or speaker-labelled notes.

use ae to transcribe this recording from example.com/audio.mp3 and tell me who said what

Step 2: Transcribe to text

For everyday transcription, use a general speech-to-text tool (such as ae_whisper) to turn speech into continuous text. It suits single-speaker dictation, narration and anything that doesn't need speakers separated — fast, with broad language coverage.

use ae to transcribe the audio with ae_whisper

Step 3: Separate the speakers

For multi-person meetings, interviews and roundtables where "who said what" matters, use a transcription tool with speaker diarization (such as ae_deepgram_post_listen_transcribe). It segments the text by speaker as it transcribes, outputting a transcript tagged like "Speaker 1 / Speaker 2" so you skip the manual identification afterwards.

use ae to transcribe and diarize with ae_deepgram_post_listen_transcribe

Step 4: Produce a structured summary

With a speaker-tagged transcript in hand, let the agent continue straight into downstream work: group points by speaker, extract decisions and action items, write a summary. The whole chain runs in one conversation — no shuttling text between tools.

use ae to summarise the diarized transcript into decisions and action items

Plain transcription vs transcription + diarization, by use case:

Use case Recommended approach Output
Single-speaker dictation / narration / captions Plain transcription (e.g. ae_whisper) Continuous transcript
Multi-person meeting / interview / podcast Transcription + diarization (e.g. ae_deepgram_post_listen_transcribe) Speaker-tagged transcript
Minutes / summary / action items Hand the diarized text to the agent for downstream tidy-up Structured summary

How to get started

  1. Sign up — new users get 1,000 free credits.
  2. Connect in-app from an agent like WorkBuddy or Trae by searching for "AgentEarth", or paste a one-line MCP config into Claude Code, Cursor or VSCode.
  3. Give the agent the audio and say in natural language whether you want a transcript or speaker-labelled notes.

FAQ

Does it support speaker diarization? Yes. Use a transcription tool with diarization (such as ae_deepgram_post_listen_transcribe); it segments text by speaker during transcription and outputs a transcript tagged "Speaker 1 / Speaker 2."

Which languages are supported? General speech-to-text tools (such as ae_whisper) cover a broad range of languages and are especially stable for major ones like English; check the chosen tool's notes for specific language support.

Is there a length limit? Different tools have their own limits on how long a single audio clip can be. For longer recordings, split into segments before transcribing, or follow the tool's guidance, for more reliable results.

Browse the tools · Access guide