Mist v3 WebSocket API

Connect to wss://mist.api.rime.ai/ws to send text and receive Mist v3 audio on one persistent connection. Mist uses the same WebSocket v1 protocol as Coda: the same authentication, messages, events, audio formats, errors, and limits. The Coda WebSocket reference documents them. The JSON and Protobuf quickstarts also work on Mist when you change the URL and the voice.

Differences from Coda

	Coda	Mist v3
Endpoint	`wss://coda.api.rime.ai/ws`	`wss://mist.api.rime.ai/ws`
Voices	Coda voices	Mist v3 voices. The voice must speak the request’s language. Rime refuses any other voice with `invalid_input`.
Languages	Listed in `ready.languages`	`de`, `en`, `es`, `es-MX`, and `fr`, as `ready.languages` lists. Three-letter codes such as `eng` also work.
Default `splitStrategy`	`SPLIT_STRATEGY_NONE`	`SPLIT_STRATEGY_SENTENCE`
Model options	`codaParameters`	`mistParameters`
Word timestamps	Returned for `de`, `en`, `es`, `fr`, `it`, and `pt` when you ask for them. Each span is a word as you sent it.	Returned for English when you ask for them. Each span is a word as Mist spoke it.

To stream LLM tokens, set splitStrategy to SPLIT_STRATEGY_ACCUMULATE_SENTENCE, as on Coda. The server then joins the tokens into sentences before Mist speaks them. See stream text as it becomes available.

Mist parameters

Put Mist’s options in start.mistParameters, or in config.defaults.mistParameters to apply them to every context. As with codaParameters, a mistParameters object in start replaces the whole default object. Mist ignores the HTTP API’s top-level names, such as pauseBetweenBrackets and inlineSpeedAlpha, on this protocol, so a misplaced option has no effect and causes no error.

Field	Type	Default	Behavior
`pauseBetweenBrackets`	Boolean	`false`	Reads a number in angle brackets as a pause in milliseconds, for example `Your order <500> shipped.` Works in every language. See custom pauses.
`phonemizeBetweenBrackets`	Boolean	`false`	Reads text in curly braces as phonemes, for example `{f R aI d eI}`. English only. See custom pronunciation.
`inlineTimeScaleFactors`	Number array	None	Speed factors for the words in square brackets, in order. Above `1` is slower, and below `1` is faster. English only.

This start message uses all three. Mist pauses for 500 ms after “order”, says “Tuesday” at half speed, and pronounces “Friday” from its phonemes:

{
  "contextId": "turn-1",
  "start": {
    "speaker": "cove",
    "language": "en",
    "text": "Your order <500> shipped on [Tuesday] and arrives {f R aI d eI}.",
    "audioParameters": { "audioFormat": "audio/wav" },
    "mistParameters": {
      "pauseBetweenBrackets": true,
      "phonemizeBetweenBrackets": true,
      "inlineTimeScaleFactors": [2.0]
    }
  }
}

The HTTP API takes the same speed factors as a comma-separated string named inlineSpeedAlpha.

Word timestamps

Set timestamps.enable to true in start to get the time of each spoken word. The times arrive in the done event, after the last audio. Mist returns them for English.

{
  "contextId": "turn-1",
  "done": {
    "timestamps": {
      "status": {},
      "spans": [
        { "text": "Your", "start": "0.139319727s", "end": "0.255419501s" },
        { "text": "order", "start": "0.313469387s", "end": "0.534058956s" },
        { "text": "shipped", "start": "1.091337868s", "end": "1.288707482s" }
      ]
    }
  }
}

Each span is one word as Mist spoke it, after text normalization, so numbers appear spelled out. start and end are offsets from the start of the audio, in seconds. An empty status means that every word was aligned. Otherwise, status has a code and a message, and spans is empty. The audio is unaffected either way.

Use Mist with LiveKit

Set model="mistv3" in rime.TTS, as in the Coda LiveKit integration. The other plugin settings work the same way:

from livekit.plugins import rime

tts = rime.TTS(
    websocket_url="wss://mist.api.rime.ai/ws",
    model="mistv3",
    speaker="cove",
    lang="en",
)

The plugin sends pauseBetweenBrackets and phonemizeBetweenBrackets when you set pause_between_brackets and phonemize_between_brackets. To use inlineTimeScaleFactors or splitStrategy, use a direct client.

Overview

Coda API reference

Mist v3 API reference

Speech-to-text API reference

API Metadata

Other APIs

Mist v3 WebSocket API

Differences from Coda

Mist parameters

Word timestamps

Use Mist with LiveKit

​Differences from Coda

​Mist parameters

​Word timestamps

​Use Mist with LiveKit

Differences from Coda

Mist parameters

Word timestamps

Use Mist with LiveKit