Channel diarization

Channel diarization transcribes each channel in your audio separately and merges the results into a single transcript. This gives perfect speaker separation at the channel level and cleaner results when speakers overlap.

Use it when each speaker is recorded on their own channel, such as a call recording with the agent and caller on separate channels. To separate speakers who share a channel, use speaker diarization instead.

Availability

Channel diarization is available for pre-recorded and streaming transcription with the Standard, Enhanced, and Melia 1 models. It is not available for agent STT. See Feature availability.

Enable channel diarization

Set diarization to channel, and optionally name each channel with channel_diarization_labels.

{
  "type": "transcription",
  "transcription_config": {
    "model": "enhanced",
    "language": "en",
    "diarization": "channel",
    "channel_diarization_labels": ["Agent", "Caller"]
  }
}

If no labels are provided, default names such as Channel 1 are used. The number of labels should match the number of channels in your audio; any extra labels are ignored.

Pre-recorded channel diarization supports up to 100 separate input files.

Provide labels for each channel in the transcription config of the StartRecognition message:

{
  "type": "transcription",
  "transcription_config": {
    "model": "enhanced",
    "language": "en",
    "diarization": "channel",
    "channel_diarization_labels": ["New_York", "Shanghai"]
  }
}

The RecognitionStarted message lists all the channels you requested:

{
  "message": "RecognitionStarted",
  ...
  "channels": ["New_York", "Shanghai"]
}

Streaming requires you to send audio per channel and to close channels explicitly. See Audio channels.

Transcript output

For pre-recorded transcription, each word includes a channel property identifying which channel it came from:

"results": [
  {
    "type": "word",
    "end_time": 1.8,
    "start_time": 1.45,
    "channel": "Agent",
    "alternatives": [
      {
        "language": "en",
        "content": "Hello",
        "confidence": 0.76
      }
    ]
  }
]

For streaming, transcripts are returned independently for each channel, with the channel property on the message:

{
  "message": "AddTranscript",
  "channel": "New_York",
  ...
  "results": [
    {
      "type": "word",
      "start_time": 1.45,
      "end_time": 1.8,
      "alternatives": [{
        "language": "en",
        "content": "Hello,",
        "confidence": 0.98,
      }]
    },
  ]
}

In streaming, the channel property is returned for AddTranscript and AddPartialTranscript messages only. The translation add-on does not currently include this property. To request it, contact Support.

Channel and speaker diarization

Channel and speaker diarization combines both modes, splitting transcripts per channel while also separating individual speakers within each channel. Use it when multiple speakers are present across multiple channels.

This mode is available for streaming transcription only. Set diarization to channel_and_speaker, and send audio per channel as described in Audio channels.

Transcripts are returned in the same way as channel diarization, but with individual speakers identified:

{
  "message": "AddTranscript",
  "channel": "New_York",
  "results": [
    {
      "alternatives": [{
        "content": "Hello",
        "confidence": 0.98,
        "speaker": "S1"
      }]
    },
    ...
    {
      "alternatives": [{
        "content": "Hi",
        "confidence": 0.98,
        "speaker": "S2"
      }]
    },
  ]
}

With channel_and_speaker diarization, speaker labeling is specific to each channel even when the labels are the same. S1 on channel 1 is not necessarily the same person as S1 on channel 2.