For AI agents: a documentation index is available at /llms.txt. Markdown versions of all pages can be requested by appending `.md` to the URL, or by setting the `Accept` header to `text/markdown`.
Skip to main content
Speech to TextFeatures

Mixed-language transcription

Mixed-language transcription uses one unified multilingual model to identify each language in the audio and switch between languages mid-sentence, returning a single continuous transcript. You do not select a language.

This is a different capability from a bilingual language pack, where you choose a fixed set of languages in advance.

Availability​

Mixed-language transcription requires the Melia 1 model. It is available for pre-recorded transcription, and for streaming transcription in Preview. See Feature availability.

Streaming with Melia 1 is available on SaaS on Cloud for evaluation and feedback. It is not production-ready and not ready to scale.

Enable mixed-language transcription​

Set model to melia-1 and language to multi.

{
  "type": "transcription",
  "transcription_config": {
    "model": "melia-1",
    "language": "multi"
  }
}

Melia 1 does not support the auto language value, which returns an error. Set language to multi.

Melia 1 has no language pack selection. For the model itself, see Models.

Language hints​

Melia 1 detects every language it hears automatically, so language hints are optional. Hints tell the model which languages to expect in the audio, biasing detection toward them. They are most useful for short clips, audio with heavy accents, or recordings where two languages sound similar, where they make language labeling more reliable.

Language hints are available for pre-recorded transcription.

Provide hints as a list of supported languages to guide detection without restricting it. This config hints that the audio contains English and Arabic:

{
  "type": "transcription",
  "transcription_config": {
    "model": "melia-1",
    "language": "multi",
    "language_hints": ["en", "ar"]
  }
}

The model can still detect and label a language you did not hint, and it labels only the languages it actually hears.

Per-word language labels​

For a Melia 1 job, the language property on each word reflects the language detected for that word, so it can change across the transcript. For Enhanced and Standard jobs, which transcribe one selected language, the same language is reported for every word.

This example shows two words in different languages within one transcript:

{
  "results": [
    {
      "alternatives": [
        { "content": "Hello", "confidence": 0.98, "language": "en" }
      ],
      "start_time": 0.20,
      "end_time": 0.52,
      "type": "word"
    },
    {
      "alternatives": [
        { "content": "مرحبا", "confidence": 0.95, "language": "ar" }
      ],
      "start_time": 0.60,
      "end_time": 1.04,
      "type": "word"
    }
  ]
}

For multilingual transcripts, language_pack_info reports the word delimiter and writing direction per language rather than for a single language pack:

{
  "metadata": {
    "language_pack_info": {
      "per_language_word_delimiters": {
        "en": " ",
        "ar": " "
      },
      "per_language_writing_direction": {
        "en": "left-to-right",
        "ar": "right-to-left"
      }
    }
  }
}

per_language_word_delimiters gives the word delimiter for each language in the transcript, and per_language_writing_direction gives its writing direction.

The LanguageInfo message​

In a streaming session, the server sends a LanguageInfo message the first time it detects a new language code in the audio. It arrives once per language code, immediately before that language first appears in an AddPartialTranscript or AddTranscript message.

Use it to set up how you process and display the transcript for that language, such as word spacing and text direction for right-to-left scripts.

{
  "message": "LanguageInfo",
  "language": "ar",
  "word_delimiter": " ",
  "writing_direction": "right-to-left",
  "partial": true
}
FieldTypeDescription
languageStringThe detected language code.
word_delimiterStringThe delimiter to use between words when you reconstruct the transcript for this language.
writing_directionStringEither left-to-right or right-to-left.
partialBooleanOptional. Present and true when a partial transcript triggered the message.

Handling LanguageInfo is not yet available in the SDKs. Read it from the WebSocket connection directly.

Transcribe live audio with Melia 1​

These examples transcribe your microphone against the Preview endpoint and print each final transcript. Replace YOUR_API_KEY_HERE with your API key.

Install the SDK and an audio input library:

pip install speechmatics-rt pyaudio
#!/usr/bin/env python3
import asyncio
from speechmatics.rt import (
    AudioEncoding, AudioFormat, AuthenticationError,
    Microphone, ServerMessageType, TranscriptResult,
    TranscriptionConfig, AsyncClient,
)

API_KEY = "YOUR_API_KEY_HERE"  # Replace with your actual API key

# Set up config and format for transcription
audio_format = AudioFormat(
    encoding=AudioEncoding.PCM_S16LE,
    sample_rate=16000,
    chunk_size=4096,
)
config = TranscriptionConfig(
    model="melia-1",
    language="multi",
)

async def main():

    # Set up microphone
    mic = Microphone(
        sample_rate=audio_format.sample_rate,
        chunk_size=audio_format.chunk_size
    )
    if not mic.start():
        print("Mic not started — please install PyAudio")
        return

    try:
        async with AsyncClient(api_key=API_KEY, url="wss://preview.rt.speechmatics.com/v2") as client:
            # Handle ADD_TRANSCRIPT message
            @client.on(ServerMessageType.ADD_TRANSCRIPT)
            def handle_finals(msg):
                if final := TranscriptResult.from_message(msg).metadata.transcript:
                    print(f"[Final]: {final}")

            try:
                # Begin transcribing
                await client.start_session(
                    transcription_config=config,
                    audio_format=audio_format
                )
                while True:
                    await client.send_audio(
                        await mic.read(
                            chunk_size=audio_format.chunk_size
                        )
                    )
            except KeyboardInterrupt:
                pass
            finally:
                mic.stop()

    except AuthenticationError as e:
        print(f"Auth error: {e}")

if __name__ == "__main__":
    asyncio.run(main())

Press Ctrl+C to stop.

The JavaScript SDK does not list melia-1 in its Model type, so the example suppresses the type error on that line.