> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Coda WebSocket API

> Stream Coda speech over one persistent WebSocket with JSON or Protobuf messages. This reference covers the connection, messages, parameters, audio formats, events, errors, word timestamps, and limits.

Connect to `wss://coda.api.rime.ai/ws` to send text and receive audio on one persistent connection. Each synthesis has a `contextId`. JSON and Protobuf support the same parameters, audio formats, and lifecycle. Either encoding can carry [word timestamps](#word-timestamps).

For runnable clients, use the [JSON quickstart](/api-reference/coda/websocket-v1-json), the [Protobuf quickstart](/api-reference/coda/websocket-v1-protobuf), or the [LiveKit integration](/api-reference/coda/websocket-v1-livekit).

## Connect and authenticate

Your server sends `Authorization: Bearer YOUR_API_KEY` in the WebSocket upgrade request. The API accepts credentials only in this header. Create a key on the [API Tokens page](https://app.rime.ai/tokens). Keep it on your server, because anyone who loads browser code can read a key embedded in it. Browser applications must connect through your backend.

Select one subprotocol in the upgrade request and check the negotiated value:

| `Sec-WebSocket-Protocol` | Server response format |
| - | - |
| `rime.v1.json` | JSON text messages with base64 audio |
| `rime.v1.binary` | Protobuf binary messages with audio bytes inside the envelope |
| No subprotocol | JSON text messages |

Wait for `ready`, and check that `ready.protocol` is `1` before you send `start`. The same event lists the supported language tags in `ready.languages`, and the deployment default in `ready.defaultLanguage` when one is set. `ready` means that the engine is available. It does not mean that the connection is authenticated.

## Client messages

Each message contains one payload field and an optional `contextId`. Do not send a `type` discriminator or a `payload` wrapper. JSON uses camelCase names. Generated Python Protobuf objects use snake\_case, such as `audio_parameters.sampling_rate`.

| Payload | Type | Purpose |
| - | - | - |
| `config` | Object | Set connection defaults once, before the first `start`. |
| `start` | Synthesis object | Open a context with complete or streaming text input. |
| `text` | String | Append text to streaming input. See [stream text](#stream-text-as-it-becomes-available). |
| `end` | Empty object | Finish streaming input. Audio continues until `done`. |
| `cancel` | Empty object | Cancel a context and receive `cancelled`. |

Use the same `contextId` for `start`, `text`, `end`, and `cancel` in one turn. A repeated `end`, or an `end` for a complete-text or unknown context, has no effect.

This message appends a sentence to an open streaming context:

```json theme={null}
{
  "contextId": "turn-1",
  "text": "Hello. How can I help you? "
}
```

## Run a synthesis context

The value of `start.text` selects the input mode:

| Input mode | `start.text` | Later input | Completion |
| - | - | - | - |
| Complete text | Contains all text | Do not append `text` | Read until `done`. No `end` required. |
| Streaming text | Omitted, empty, or whitespace only | Send `text` messages | Send `end`, then read until `done` |

This sequence streams two sentences. The client can send text while it receives audio:

```mermaid theme={null}
sequenceDiagram
    participant Client
    participant Coda
    Client->>Coda: Upgrade with authorization and subprotocol
    Coda-->>Client: ready
    Client->>Coda: start with empty text and contextId
    Coda-->>Client: started with requestId
    Client->>Coda: text, complete sentence
    Coda-->>Client: audio
    Client->>Coda: text, next sentence
    Coda-->>Client: audio
    Client->>Coda: end
    Coda-->>Client: remaining audio
    Coda-->>Client: done
    Note over Client,Coda: Connection remains open for the next context
```

### Complete text in one request

This `start` supplies the complete utterance. The server finishes it without an `end` message:

```json theme={null}
{
  "contextId": "turn-1",
  "start": {
    "speaker": "lyra",
    "language": "en",
    "text": "Hello. How can I help you?",
    "audioParameters": {"audioFormat": "audio/wav"}
  }
}
```

### Stream text as it becomes available

Set `splitStrategy` on the streaming `start`:

| `splitStrategy` | Send |
| - | - |
| `SPLIT_STRATEGY_ACCUMULATE_SENTENCE` | LLM tokens as they arrive. The server joins them and passes complete sentences to Coda. |
| `SPLIT_STRATEGY_NONE` | Complete sentences. Each message is spoken on its own. |
| `SPLIT_STRATEGY_SENTENCE` | Complete sentences. The server also splits each message into sentences. |

With `SPLIT_STRATEGY_ACCUMULATE_SENTENCE`, send tokens unchanged, spaces included. `"Hello"` followed by `"world."` is spoken as `Helloworld.` With `SPLIT_STRATEGY_NONE`, the Coda default, a message that ends mid-sentence is spoken as a fragment.

Send `end` only at the end of the turn. A pause in your input needs no message.

### Identify and reuse contexts

* **Use a new `contextId` for every turn:** the server answers a `cancel` for a finished context with `cancelled`. It also applies a late `cancel` to any open context with the same ID. With a new ID per turn, a late `cancel` or `cancelled` cannot end the next turn, and you can reject stale audio.
* **Default ID:** an omitted or empty ID selects `default`, which the server returns as `"contextId":"default"`.
* **Route by ID:** responses from concurrent contexts can arrive between each other. Route them by `contextId` from one receive loop, with a separate decoder and playback queue for each context. The engine supports [up to 16 open contexts](#limits-and-connection-health).
* **Log the request ID:** log `started.requestId` with your `contextId` for error reports. A request can fail or be cancelled before `started`, so handle terminal events as soon as you send `start`.

### Stop speech on interruption

1. Stop local playback and discard the turn's queued audio.
2. Send `cancel` with that turn's `contextId`.
3. Keep reading until `cancelled`, or `done` if synthesis completed first. A context-scoped `error` also ends the run. Discard audio for the interrupted turn that was already in transit.

This message cancels only `turn-1`:

```json theme={null}
{
  "contextId": "turn-1",
  "cancel": {}
}
```

You can send `cancel` more than once, and an unknown or finished context still gets `cancelled`. Cancelling a context leaves the connection open, and closing the socket cancels all of its active contexts.

## Connection configuration

Send at most one `config` message before the first `start`. The server does not acknowledge it.

| `config` field | Type | Behavior |
| - | - | - |
| `defaults` | Synthesis object | Defaults for later `start` messages. Its `text` field is ignored. |
| `authorization` | String | Direct engine connections only. On `coda.api.rime.ai`, it gets an `invalid_input` error, so send the key in the upgrade header. See [direct engine connections](#direct-engine-connections). |

Defaults merge at the top level. `start.audioParameters` replaces the whole default object, so include every audio setting that you want to keep. The same rule applies to `codaParameters`.

This configuration uses header authentication and sets PCM output for later turns:

```json theme={null}
{
  "config": {
    "defaults": {
      "speaker": "lyra",
      "language": "en",
      "audioParameters": {
        "audioFormat": "audio/pcm",
        "samplingRate": 24000
      }
    }
  }
}
```

If a later `start.audioParameters` contains only `{"samplingRate":8000}`, it discards the default `audioFormat`, and the context fails with `invalid_input`. Include `"audioFormat":"audio/pcm"` to keep PCM.

## Synthesis parameters

Set these fields in `start` or `config.defaults`. The host selects Coda, so there is no `modelId` field.

| Field | Type | Default | Use |
| - | - | - | - |
| `speaker` | String | Deployment-specific | Set a [Coda voice](/docs/voices-coda), such as `lyra`, explicitly. |
| `language` | String | Deployment default | BCP 47 tag, such as `en`. Supported tags appear in `ready.languages`. |
| `text` | String | Empty | Select the [input mode](#run-a-synthesis-context). Ignored in `config.defaults`. |
| `audioParameters` | Object | See below | Audio format, sampling rate, and speed. |
| `splitStrategy` | Enum | Deployment default | When the server passes streamed text to Coda. See [stream text](#stream-text-as-it-becomes-available). |
| `codaParameters` | Object | See below | Streaming text lookahead. |
| `timestamps` | Object | None | Set `{"enable": true}` for [word timestamps](#word-timestamps). |

Coda does not accept `temperature`, `topP`, `topK`, `repetitionPenalty`, `maxTokens`, or `seed`. The server can ignore unknown fields, so a successful request does not prove that an unsupported setting took effect.

### Audio parameters

| `audioParameters` field | Type | Default | Behavior |
| - | - | - | - |
| `audioFormat` | String | None, required | Use a MIME type from the table below. A missing or empty value fails the context with `invalid_input`. |
| `samplingRate` | Integer | `24000` Hz | Positive rate supported by your decoder or player. Set `8000` for 8 kHz telephony. |
| `timeScaleFactor` | Number | `1.0` | Below `1` is faster. Above `1` is slower. The engine clamps values to `0.4` through `2.5`. |

Both protocols support these audio formats. Decode the message envelope first: JSON carries base64 audio, and Protobuf carries audio bytes.

| MIME type | Encoded content | Client handling |
| - | - | - |
| `audio/pcm` | Raw mono, signed 16-bit little-endian PCM | Use the requested sampling rate. There is no file header. |
| `audio/wav` | WAV with 16-bit PCM | Feed chunks to one streaming WAV decoder. |
| `audio/mpeg` | MP3 | Feed chunks to one MP3 decoder, or append them to a file. |
| `audio/ogg;codecs=opus` | Opus in Ogg | Use an Ogg/Opus decoder. |
| `audio/webm;codecs=opus` | Opus in WebM | Use a WebM/Opus decoder. |
| `audio/pcmu` | G.711 mu-law | Decode one byte per sample at the requested rate. Set `samplingRate: 8000` for 8 kHz. |

Message boundaries are transport chunks, not separate files or guaranteed decoder frames. Keep one decoder alive for each context. Raw PCM consumers must keep any incomplete sample between chunks.

### Text lookahead

Set `codaParameters.textLookaheadTokens` to control streaming text lookahead:

| `codaParameters` field | Type | Default | Behavior |
| - | - | - | - |
| `textLookaheadTokens` | Non-negative integer | `0` | More leading text tokens can improve the start of an utterance, but they delay first audio. The engine caps lookahead at the available tokens and ignores it for complete-text input. |

## Server events

| Event | Scope | Meaning |
| - | - | - |
| `ready` | Connection | Engine ready. Check protocol version and supported languages. |
| `started` | Context | Synthesis accepted. Save `requestId` for error reports. |
| `audio` | Context | Audio chunk: base64 in JSON, bytes in Protobuf. |
| `done` | Context | All audio delivered, with [word timestamps](#word-timestamps) when you asked for them. Finish decoding. Playback can still have queued audio. |
| `cancelled` | Context | Context cancelled. Discard remaining audio and release its state. |
| `error` | Context or connection | See [completion and errors](#handle-completion-and-errors). |

These examples show separate server messages. The language list is illustrative, and `error.requestId` is optional:

```json ready theme={null}
{
  "ready": {
    "protocol": 1,
    "languages": ["en"],
    "defaultLanguage": "en"
  }
}
```

```json started theme={null}
{
  "contextId": "turn-1",
  "started": {"requestId": "request-123"}
}
```

```json audio theme={null}
{
  "contextId": "turn-1",
  "audio": "AQID"
}
```

```json done theme={null}
{
  "contextId": "turn-1",
  "done": {}
}
```

```json cancelled theme={null}
{
  "contextId": "turn-1",
  "cancelled": {}
}
```

```json error theme={null}
{
  "contextId": "turn-1",
  "error": {
    "kind": "invalid_input",
    "message": "Invalid request",
    "requestId": "request-123"
  }
}
```

Proto3 JSON omits default-valued fields. Test payload presence with `"done" in event`, not truthiness, because the empty object `{}` is falsy in Python. For Python Protobuf, use `response.WhichOneof("payload")`. Allow unknown fields for compatible schema additions.

### Handle completion and errors

An active context finishes with one terminal event: `done`, `cancelled`, or a context-scoped `error`. No more audio belongs to that run after its terminal event.

| Failure | What happens | Client action |
| - | - | - |
| `error` with a non-empty `contextId` | That context ends. | Stop that context. Log `kind`, `message`, and any `requestId`. |
| `error` with no `contextId`, or an empty one | A connection-level error. Existing contexts can remain active. | Handle the error for the connection. |
| A duplicate `start`, text outside an open streaming input, or too many open contexts | A connection-level error. Existing contexts continue. | Correct the message order or reduce open contexts. |
| A malformed envelope or failed authentication | The server can close the socket. | Fix the message or the credentials, then reconnect. |
| The socket closes before a terminal event | Unfinished contexts are interrupted, even if some audio arrived. | Treat them as incomplete. Reconnect and wait for `ready` before starting new work. |

There is no resume operation. Retrying text whose audio already played can repeat speech, so choose retries at a turn or sentence boundary.

### Error kinds

`error.kind` is a string, not an HTTP status code. Authentication can also fail with HTTP `401` or `503` during the upgrade.

| Kind | Action |
| - | - |
| `invalid_input` | Correct fields, text, or message order before retrying. An unknown voice also reports `invalid_input`. |
| `unauthenticated` | Check the API key or deployment credentials. |
| `permission_denied` | Check access to the requested service. |
| `not_found` | Check the requested resource. |
| `resource_exhausted` | The connection already has 16 open contexts. Wait for one to finish, or cancel one. |
| `unavailable` | The server cannot take the request right now, for example at capacity or while restarting. Retry with backoff if the application can repeat the turn safely. |
| `timeout` | Check timeouts and service health before retrying. |
| `internal` | Record the request ID and report the failure. |

| Close code | Cause |
| - | - |
| `1007` | An undecodable envelope. The server also sends `invalid_input`. |
| `1008` | Invalid or conflicting message credentials on a direct engine connection |

## Word timestamps

Set `timestamps.enable` to `true` in `start` to get the time of each word Coda speaks. The times arrive in the `done` event, after the last audio. Asking for them does not change how the audio streams. Coda returns them for `de`, `en`, `es`, `fr`, `it`, and `pt`.

This `start` asks for the timestamps of a complete-text context:

```json theme={null}
{
  "contextId": "turn-1",
  "start": {
    "speaker": "lyra",
    "language": "en",
    "text": "Dr. Smith paid $42.50 on March 3rd, 2024 at 5:30 PM.",
    "audioParameters": {"audioFormat": "audio/wav"},
    "timestamps": {"enable": true}
  }
}
```

Its `done` event has one span for each word of the text:

```json theme={null}
{
  "contextId": "turn-1",
  "done": {
    "timestamps": {
      "status": {},
      "spans": [
        {"text": "Dr.", "start": "0.080s", "end": "0.320s"},
        {"text": "Smith", "start": "0.320s", "end": "0.720s"},
        {"text": "paid", "start": "0.720s", "end": "0.880s"},
        {"text": "$42.50", "start": "0.880s", "end": "2.640s"},
        {"text": "on", "start": "2.640s", "end": "2.800s"},
        {"text": "March", "start": "2.800s", "end": "3.413333333s"},
        {"text": "3rd,", "start": "3.413333333s", "end": "4.026666666s"},
        {"text": "2024", "start": "4.026666666s", "end": "4.640s"},
        {"text": "at", "start": "4.640s", "end": "4.800s"},
        {"text": "5:30", "start": "4.880s", "end": "5.360s"},
        {"text": "PM.", "start": "5.360s", "end": "5.840s"}
      ]
    }
  }
}
```

Each span is a word of your text as you sent it, split at whitespace, so `$42.50` keeps its spelling although Coda says "forty-two dollars and fifty cents." `start` and `end` are offsets from the start of the context's audio as you receive it, in seconds, with `timeScaleFactor` applied.

* **Unspoken words:** a word Coda does not speak, such as punctuation on its own, has no span.
* **Streaming input:** spans follow the text as the server joins it. `"Hello"` followed by `"world."` gives one span, `Helloworld.`

An empty `status` means that Coda aligned every word. Otherwise, `status` has a `code`, one of the [gRPC status codes](https://grpc.io/docs/guides/status-codes/#the-full-list-of-status-codes), and a `message` that says why, and `spans` is empty. The audio is unaffected either way. A context in a language Coda does not align, such as `ja`, or with more than 180 seconds of audio gets no spans.

## Limits and connection health

| Engine limit | Value |
| - | - |
| Incoming message or frame | 64 KiB, including the envelope |
| Open contexts per connection | 16 |
| Context ID | 128 UTF-8 bytes |
| Server ping interval | 30 seconds |
| Read inactivity timeout | 75 seconds, reset by client pongs |

A hosted gateway can apply additional limits. Keep the connection healthy with these practices:

* **Answer pings:** use a WebSocket library that answers ping frames, because the pongs reset the read inactivity timeout.
* **Keep one reader:** keep one receive loop active for the whole connection.
* **Respect backpressure:** await each send.
* **Bound the audio queue:** a bounded queue limits memory use when the player is slow.

## Advanced connection options

### Subprotocol negotiation

If you offer both supported subprotocols, the server selects the first in your list. An offer with only unknown values gets no selected subprotocol, which can cause the client to reject the handshake.

The server decodes incoming messages by their WebSocket type, text or binary. The selected subprotocol fixes the response encoding for the connection. Use the same encoding in both directions unless you need a mixed client.

### Direct engine connections

These details apply when you connect to a Coda engine directly, for example in an on-premises deployment:

* **Credentials:** a client that cannot send upgrade headers can send `config.authorization` in its first message, with the value the engine's [caller authentication](/docs/on-prem/protocols#caller-authentication) expects. Send it within 10 seconds. If you send credentials in both the upgrade headers and `config`, the engine closes the connection. This does not make an API key safe to put in a browser.
* **License authentication:** deployments that use license authentication accept `config.license`. This `license` field holds the license JSON as a string.
* **Audio aliases:** the engine also accepts `audio/mp3`, `audio/ogg`, `audio/webm`, `audio/x-mulaw`, and `audio/l16`. The `audio/l16` alias returns little-endian PCM, so use `audio/pcm` to avoid confusion with network-byte-order L16. Unknown formats fall back to WAV, and an empty format is invalid. LiveKit accepts only the six canonical MIME types above.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.