Coda WebSocket API

Connect to wss://coda.api.rime.ai/ws to send text and receive audio on one persistent connection. Each synthesis has a contextId. JSON and Protobuf support the same parameters, audio formats, and lifecycle. Either encoding can carry word timestamps. For runnable clients, use the JSON quickstart, the Protobuf quickstart, or the LiveKit integration.

Connect and authenticate

Your server sends Authorization: Bearer YOUR_API_KEY in the WebSocket upgrade request. The API accepts credentials only in this header. Create a key on the API Tokens page. Keep it on your server, because anyone who loads browser code can read a key embedded in it. Browser applications must connect through your backend. Select one subprotocol in the upgrade request and check the negotiated value:

`Sec-WebSocket-Protocol`	Server response format
`rime.v1.json`	JSON text messages with base64 audio
`rime.v1.binary`	Protobuf binary messages with audio bytes inside the envelope
No subprotocol	JSON text messages

Wait for ready, and check that ready.protocol is 1 before you send start. The same event lists the supported language tags in ready.languages, and the deployment default in ready.defaultLanguage when one is set. ready means that the engine is available. It does not mean that the connection is authenticated.

Client messages

Each message contains one payload field and an optional contextId. Do not send a type discriminator or a payload wrapper. JSON uses camelCase names. Generated Python Protobuf objects use snake_case, such as audio_parameters.sampling_rate.

Payload	Type	Purpose
`config`	Object	Set connection defaults once, before the first `start`.
`start`	Synthesis object	Open a context with complete or streaming text input.
`text`	String	Append text to streaming input. See stream text.
`end`	Empty object	Finish streaming input. Audio continues until `done`.
`cancel`	Empty object	Cancel a context and receive `cancelled`.

Use the same contextId for start, text, end, and cancel in one turn. A repeated end, or an end for a complete-text or unknown context, has no effect. This message appends a sentence to an open streaming context:

{
  "contextId": "turn-1",
  "text": "Hello. How can I help you? "
}

Run a synthesis context

The value of start.text selects the input mode:

Input mode	`start.text`	Later input	Completion
Complete text	Contains all text	Do not append `text`	Read until `done`. No `end` required.
Streaming text	Omitted, empty, or whitespace only	Send `text` messages	Send `end`, then read until `done`

This sequence streams two sentences. The client can send text while it receives audio:

Complete text in one request

This start supplies the complete utterance. The server finishes it without an end message:

{
  "contextId": "turn-1",
  "start": {
    "speaker": "lyra",
    "language": "en",
    "text": "Hello. How can I help you?",
    "audioParameters": {"audioFormat": "audio/wav"}
  }
}

Stream text as it becomes available

Set splitStrategy on the streaming start:

`splitStrategy`	Send
`SPLIT_STRATEGY_ACCUMULATE_SENTENCE`	LLM tokens as they arrive. The server joins them and passes complete sentences to Coda.
`SPLIT_STRATEGY_NONE`	Complete sentences. Each message is spoken on its own.
`SPLIT_STRATEGY_SENTENCE`	Complete sentences. The server also splits each message into sentences.

With SPLIT_STRATEGY_ACCUMULATE_SENTENCE, send tokens unchanged, spaces included. "Hello" followed by "world." is spoken as Helloworld. With SPLIT_STRATEGY_NONE, the Coda default, a message that ends mid-sentence is spoken as a fragment. Send end only at the end of the turn. A pause in your input needs no message.

Identify and reuse contexts

Use a new contextId for every turn: the server answers a cancel for a finished context with cancelled. It also applies a late cancel to any open context with the same ID. With a new ID per turn, a late cancel or cancelled cannot end the next turn, and you can reject stale audio.
Default ID: an omitted or empty ID selects default, which the server returns as "contextId":"default".
Route by ID: responses from concurrent contexts can arrive between each other. Route them by contextId from one receive loop, with a separate decoder and playback queue for each context. The engine supports up to 16 open contexts.
Log the request ID: log started.requestId with your contextId for error reports. A request can fail or be cancelled before started, so handle terminal events as soon as you send start.

Stop speech on interruption

Stop local playback and discard the turn’s queued audio.
Send cancel with that turn’s contextId.
Keep reading until cancelled, or done if synthesis completed first. A context-scoped error also ends the run. Discard audio for the interrupted turn that was already in transit.

This message cancels only turn-1:

{
  "contextId": "turn-1",
  "cancel": {}
}

You can send cancel more than once, and an unknown or finished context still gets cancelled. Cancelling a context leaves the connection open, and closing the socket cancels all of its active contexts.

Connection configuration

Send at most one config message before the first start. The server does not acknowledge it.

`config` field	Type	Behavior
`defaults`	Synthesis object	Defaults for later `start` messages. Its `text` field is ignored.
`authorization`	String	Direct engine connections only. On `coda.api.rime.ai`, it gets an `invalid_input` error, so send the key in the upgrade header. See direct engine connections.

Defaults merge at the top level. start.audioParameters replaces the whole default object, so include every audio setting that you want to keep. The same rule applies to codaParameters. This configuration uses header authentication and sets PCM output for later turns:

{
  "config": {
    "defaults": {
      "speaker": "lyra",
      "language": "en",
      "audioParameters": {
        "audioFormat": "audio/pcm",
        "samplingRate": 24000
      }
    }
  }
}

If a later start.audioParameters contains only {"samplingRate":8000}, it discards the default audioFormat, and the context fails with invalid_input. Include "audioFormat":"audio/pcm" to keep PCM.

Synthesis parameters

Set these fields in start or config.defaults. The host selects Coda, so there is no modelId field.

Field	Type	Default	Use
`speaker`	String	Deployment-specific	Set a Coda voice, such as `lyra`, explicitly.
`language`	String	Deployment default	BCP 47 tag, such as `en`. Supported tags appear in `ready.languages`.
`text`	String	Empty	Select the input mode. Ignored in `config.defaults`.
`audioParameters`	Object	See below	Audio format, sampling rate, and speed.
`splitStrategy`	Enum	Deployment default	When the server passes streamed text to Coda. See stream text.
`codaParameters`	Object	See below	Streaming text lookahead.
`timestamps`	Object	None	Set `{"enable": true}` for word timestamps.

Coda does not accept temperature, topP, topK, repetitionPenalty, maxTokens, or seed. The server can ignore unknown fields, so a successful request does not prove that an unsupported setting took effect.

Audio parameters

`audioParameters` field	Type	Default	Behavior
`audioFormat`	String	None, required	Use a MIME type from the table below. A missing or empty value fails the context with `invalid_input`.
`samplingRate`	Integer	`24000` Hz	Positive rate supported by your decoder or player. Set `8000` for 8 kHz telephony.
`timeScaleFactor`	Number	`1.0`	Below `1` is faster. Above `1` is slower. The engine clamps values to `0.4` through `2.5`.

Both protocols support these audio formats. Decode the message envelope first: JSON carries base64 audio, and Protobuf carries audio bytes.

MIME type	Encoded content	Client handling
`audio/pcm`	Raw mono, signed 16-bit little-endian PCM	Use the requested sampling rate. There is no file header.
`audio/wav`	WAV with 16-bit PCM	Feed chunks to one streaming WAV decoder.
`audio/mpeg`	MP3	Feed chunks to one MP3 decoder, or append them to a file.
`audio/ogg;codecs=opus`	Opus in Ogg	Use an Ogg/Opus decoder.
`audio/webm;codecs=opus`	Opus in WebM	Use a WebM/Opus decoder.
`audio/pcmu`	G.711 mu-law	Decode one byte per sample at the requested rate. Set `samplingRate: 8000` for 8 kHz.

Message boundaries are transport chunks, not separate files or guaranteed decoder frames. Keep one decoder alive for each context. Raw PCM consumers must keep any incomplete sample between chunks.

Text lookahead

Set codaParameters.textLookaheadTokens to control streaming text lookahead:

`codaParameters` field	Type	Default	Behavior
`textLookaheadTokens`	Non-negative integer	`0`	More leading text tokens can improve the start of an utterance, but they delay first audio. The engine caps lookahead at the available tokens and ignores it for complete-text input.

Server events

Event	Scope	Meaning
`ready`	Connection	Engine ready. Check protocol version and supported languages.
`started`	Context	Synthesis accepted. Save `requestId` for error reports.
`audio`	Context	Audio chunk: base64 in JSON, bytes in Protobuf.
`done`	Context	All audio delivered, with word timestamps when you asked for them. Finish decoding. Playback can still have queued audio.
`cancelled`	Context	Context cancelled. Discard remaining audio and release its state.
`error`	Context or connection	See completion and errors.

These examples show separate server messages. The language list is illustrative, and error.requestId is optional:

{
  "ready": {
    "protocol": 1,
    "languages": ["en"],
    "defaultLanguage": "en"
  }
}

{
  "contextId": "turn-1",
  "started": {"requestId": "request-123"}
}

{
  "contextId": "turn-1",
  "audio": "AQID"
}

{
  "contextId": "turn-1",
  "done": {}
}

{
  "contextId": "turn-1",
  "cancelled": {}
}

{
  "contextId": "turn-1",
  "error": {
    "kind": "invalid_input",
    "message": "Invalid request",
    "requestId": "request-123"
  }
}

Proto3 JSON omits default-valued fields. Test payload presence with "done" in event, not truthiness, because the empty object {} is falsy in Python. For Python Protobuf, use response.WhichOneof("payload"). Allow unknown fields for compatible schema additions.

Handle completion and errors

An active context finishes with one terminal event: done, cancelled, or a context-scoped error. No more audio belongs to that run after its terminal event.

Failure	What happens	Client action
`error` with a non-empty `contextId`	That context ends.	Stop that context. Log `kind`, `message`, and any `requestId`.
`error` with no `contextId`, or an empty one	A connection-level error. Existing contexts can remain active.	Handle the error for the connection.
A duplicate `start`, text outside an open streaming input, or too many open contexts	A connection-level error. Existing contexts continue.	Correct the message order or reduce open contexts.
A malformed envelope or failed authentication	The server can close the socket.	Fix the message or the credentials, then reconnect.
The socket closes before a terminal event	Unfinished contexts are interrupted, even if some audio arrived.	Treat them as incomplete. Reconnect and wait for `ready` before starting new work.

There is no resume operation. Retrying text whose audio already played can repeat speech, so choose retries at a turn or sentence boundary.

Error kinds

error.kind is a string, not an HTTP status code. Authentication can also fail with HTTP 401 or 503 during the upgrade.

Kind	Action
`invalid_input`	Correct fields, text, or message order before retrying. An unknown voice also reports `invalid_input`.
`unauthenticated`	Check the API key or deployment credentials.
`permission_denied`	Check access to the requested service.
`not_found`	Check the requested resource.
`resource_exhausted`	The connection already has 16 open contexts. Wait for one to finish, or cancel one.
`unavailable`	The server cannot take the request right now, for example at capacity or while restarting. Retry with backoff if the application can repeat the turn safely.
`timeout`	Check timeouts and service health before retrying.
`internal`	Record the request ID and report the failure.

Close code	Cause
`1007`	An undecodable envelope. The server also sends `invalid_input`.
`1008`	Invalid or conflicting message credentials on a direct engine connection

Word timestamps

Set timestamps.enable to true in start to get the time of each word Coda speaks. The times arrive in the done event, after the last audio. Asking for them does not change how the audio streams. Coda returns them for de, en, es, fr, it, and pt. This start asks for the timestamps of a complete-text context:

{
  "contextId": "turn-1",
  "start": {
    "speaker": "lyra",
    "language": "en",
    "text": "Dr. Smith paid $42.50 on March 3rd, 2024 at 5:30 PM.",
    "audioParameters": {"audioFormat": "audio/wav"},
    "timestamps": {"enable": true}
  }
}

Its done event has one span for each word of the text:

{
  "contextId": "turn-1",
  "done": {
    "timestamps": {
      "status": {},
      "spans": [
        {"text": "Dr.", "start": "0.080s", "end": "0.320s"},
        {"text": "Smith", "start": "0.320s", "end": "0.720s"},
        {"text": "paid", "start": "0.720s", "end": "0.880s"},
        {"text": "$42.50", "start": "0.880s", "end": "2.640s"},
        {"text": "on", "start": "2.640s", "end": "2.800s"},
        {"text": "March", "start": "2.800s", "end": "3.413333333s"},
        {"text": "3rd,", "start": "3.413333333s", "end": "4.026666666s"},
        {"text": "2024", "start": "4.026666666s", "end": "4.640s"},
        {"text": "at", "start": "4.640s", "end": "4.800s"},
        {"text": "5:30", "start": "4.880s", "end": "5.360s"},
        {"text": "PM.", "start": "5.360s", "end": "5.840s"}
      ]
    }
  }
}

Each span is a word of your text as you sent it, split at whitespace, so $42.50 keeps its spelling although Coda says “forty-two dollars and fifty cents.” start and end are offsets from the start of the context’s audio as you receive it, in seconds, with timeScaleFactor applied.

Unspoken words: a word Coda does not speak, such as punctuation on its own, has no span.
Streaming input: spans follow the text as the server joins it. "Hello" followed by "world." gives one span, Helloworld.

An empty status means that Coda aligned every word. Otherwise, status has a code, one of the gRPC status codes, and a message that says why, and spans is empty. The audio is unaffected either way. A context in a language Coda does not align, such as ja, or with more than 180 seconds of audio gets no spans.

Limits and connection health

Engine limit	Value
Incoming message or frame	64 KiB, including the envelope
Open contexts per connection	16
Context ID	128 UTF-8 bytes
Server ping interval	30 seconds
Read inactivity timeout	75 seconds, reset by client pongs

A hosted gateway can apply additional limits. Keep the connection healthy with these practices:

Answer pings: use a WebSocket library that answers ping frames, because the pongs reset the read inactivity timeout.
Keep one reader: keep one receive loop active for the whole connection.
Respect backpressure: await each send.
Bound the audio queue: a bounded queue limits memory use when the player is slow.

Advanced connection options

Subprotocol negotiation

If you offer both supported subprotocols, the server selects the first in your list. An offer with only unknown values gets no selected subprotocol, which can cause the client to reject the handshake. The server decodes incoming messages by their WebSocket type, text or binary. The selected subprotocol fixes the response encoding for the connection. Use the same encoding in both directions unless you need a mixed client.

Direct engine connections

These details apply when you connect to a Coda engine directly, for example in an on-premises deployment:

Credentials: a client that cannot send upgrade headers can send config.authorization in its first message, with the value the engine’s caller authentication expects. Send it within 10 seconds. If you send credentials in both the upgrade headers and config, the engine closes the connection. This does not make an API key safe to put in a browser.
License authentication: deployments that use license authentication accept config.license. This license field holds the license JSON as a string.
Audio aliases: the engine also accepts audio/mp3, audio/ogg, audio/webm, audio/x-mulaw, and audio/l16. The audio/l16 alias returns little-endian PCM, so use audio/pcm to avoid confusion with network-byte-order L16. Unknown formats fall back to WAV, and an empty format is invalid. LiveKit accepts only the six canonical MIME types above.

Overview

Coda API reference

Mist v3 API reference

Speech-to-text API reference

API Metadata

Other APIs

Coda WebSocket API

Connect and authenticate

Client messages

Run a synthesis context

Complete text in one request

Stream text as it becomes available

Identify and reuse contexts

Stop speech on interruption

Connection configuration

Synthesis parameters

Audio parameters

Text lookahead

Server events

Handle completion and errors

Error kinds

Word timestamps

Limits and connection health

Advanced connection options

Subprotocol negotiation

Direct engine connections

​Connect and authenticate

​Client messages

​Run a synthesis context

​Complete text in one request

​Stream text as it becomes available

​Identify and reuse contexts

​Stop speech on interruption

​Connection configuration

​Synthesis parameters

​Audio parameters

​Text lookahead

​Server events

​Handle completion and errors

​Error kinds

​Word timestamps

​Limits and connection health

​Advanced connection options

​Subprotocol negotiation

​Direct engine connections

Connect and authenticate

Client messages

Run a synthesis context

Complete text in one request

Stream text as it becomes available

Identify and reuse contexts

Stop speech on interruption

Connection configuration

Synthesis parameters

Audio parameters

Text lookahead

Server events

Handle completion and errors

Error kinds

Word timestamps

Limits and connection health

Advanced connection options

Subprotocol negotiation

Direct engine connections