> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming WebSocket

> Stream audio to Rime speech-to-text over a WebSocket, without the SDK. This reference covers the connection, the messages, the start parameters, the limits, and long recordings, plus a complete Python client.

Connect to `wss://stt.api.rime.ai/ws` from your backend, stream audio, and read the transcript as it arrives. If your backend is in Python, Node.js, Go, or Rust, use the [Rime SDK](/docs/stt/quickstart) instead, because it connects over gRPC and needs less code. The exception is a [long recording](#transcribe-a-long-recording), which needs this API's append fragments. For other languages, read [Choose how to connect](/docs/speech-to-text#choose-how-to-connect) first. That section helps you choose between this endpoint and gRPC, and between the two output contracts.

| Item | Value |
| - | - |
| URL | `wss://stt.api.rime.ai/ws`. Unlike Rime's text-to-speech WebSockets, this URL takes no query parameters, not even the API key. |
| API key | Put your [API key](/docs/api-authentication) in a header of the upgrade request: `Authorization: Bearer YOUR_API_KEY`. A browser's `WebSocket` API cannot set this header. |
| Session | Each connection is one session, and each session transcribes one utterance. |
| Audio | Headerless signed 16-bit little-endian PCM at 16 kHz, mono |

## Session at a glance

```mermaid theme={null}
sequenceDiagram
    participant C as Your backend
    participant R as Rime STT
    C->>R: WebSocket upgrade with Authorization header
    C->>R: start (outputContract, language)
    R-->>C: started (requestId, accepted)
    loop While you capture audio
        C->>R: audio (base64 PCM16)
        R-->>C: hypothesis or delta
    end
    C->>R: end
    R-->>C: remaining hypothesis or delta events
    R-->>C: done (text, language, revision)
    Note over C,R: error or cancelled can come instead of done
    R-->>C: Close frame
```

| Direction | Message | When |
| - | - | - |
| Send | [`start`](#start) | One time, as the first message, within 5 seconds of the upgrade |
| Receive | [`started`](#started) | After the server accepts `start`, before any transcript event |
| Send | [`audio`](#audio) | Many times, while you capture audio |
| Receive | [`hypothesis`](#hypothesis) | Revised hypotheses contract only. Each one has the whole transcript so far. |
| Receive | [`delta`](#delta) | Append fragments contract only. Each one has the text of the next window of audio. |
| Send | [`end`](#end) | One time, when the utterance is over |
| Receive | [`done`](#done) | One time, after `end`, with the final transcript |
| Send | [`cancel`](#cancel) | Instead of `end`, or after it, to stop the utterance |
| Receive | [`cancelled`](#cancelled) | After `cancel` |
| Receive | [`error`](#error) | Instead of `done`, when the utterance fails |

## Guarantees

| Topic | Guarantee |
| - | - |
| Order | `started` arrives before any `hypothesis`, `delta`, or `done`. |
| Revisions | Under revised hypotheses, `revision` starts at 1. It increases by 1 each time the text changes. The server does not send the same text again. |
| Final text | Under revised hypotheses, `done.text` equals the last `hypothesis.text`, and `done.revision` equals its revision. Under append fragments, `done.text` equals every `delta.text`, joined in arrival order, and `done.revision` is 0. |
| No speech | An utterance with no recognizable speech can end with no `hypothesis` or `delta`. Its `done.text` is empty, and its revision is 0. |
| Deltas | The server never revises a `delta`. The server trims the text of each window. Every `delta` after the first starts with `\n`. |
| Language | `started.accepted.language.tag` and `done.language.tag` contain the language, `en` or `es`. `source` is always `LANGUAGE_SOURCE_SELECTED`. |
| Request ID | Every `error` event has a `requestId`. |
| Terminal event | A session ends with at most one terminal event: `done`, `error`, or `cancelled`. The server tries one time, with a time limit, to send this event and then a Close frame. If the connection drops or your client stops reading, the session can end with neither. Treat a session without a terminal event as incomplete. |

## Connection

### Response encoding

Set the response encoding in the `Sec-WebSocket-Protocol` header of the upgrade request:

| Subprotocol | Response encoding |
| - | - |
| `rime.v1.json` | JSON text frames |
| `rime.v1.binary` | Protobuf binary frames |
| Both | The first one that you offer. The server echoes it back. |
| No subprotocol offered | JSON text frames |

The response encoding stays the same for the whole connection. The subprotocol sets only the response encoding, because the server decodes each client message by its frame type. For example, you can send JSON text frames and receive protobuf. Both encodings carry the `SpeechWebSocketRequest` and `SpeechWebSocketResponse` envelopes, which the [speech-to-text schema](/api-reference/stt/grpc#schema) defines. JSON uses the canonical proto3 mapping:

| Protobuf value | JSON representation |
| - | - |
| Field name | camelCase |
| `bytes` | Base64 string |
| Enum | Symbolic name |
| `uint64` | Decimal string |
| Field at its default value | Omitted |

### Upgrade responses

The server answers the upgrade request with one of these HTTP statuses. A refusal is a plain-text HTTP response, and it arrives before any event:

| Status | Meaning |
| - | - |
| `101 Switching Protocols` | The upgrade succeeded. Its `x-request-id` header contains the request ID. |
| `400` | The request is not a WebSocket upgrade. |
| `401` | The `Authorization` header is missing, or Rime does not accept the key. The response has a `WWW-Authenticate: Bearer` header and an empty body. |
| `404` | The server has no route for the path, for example `/stt`. Use `/ws`. |
| `503` | The service is not ready, is shutting down, or has no WebSocket capacity. |

## Messages

### Send

#### Start

`start` opens the utterance. `outputContract` and `language` are required. `mode` and `contextTerms` are optional, and they work the same way under both output contracts. [Start parameters](#start-parameters) describes each field.

| Rule | When you break it |
| - | - |
| Send `start` one time only, as the first message. | A second `start` fails with `invalid_input`. |
| Send `start` within 5 seconds of the upgrade. | The server sends an `error` with kind `timeout`, and closes the connection. |

```typescript theme={null}
type StartMessage = {
  start: {
    outputContract:
      | "STREAMING_OUTPUT_CONTRACT_REVISED_HYPOTHESES"
      | "STREAMING_OUTPUT_CONTRACT_APPEND_FRAGMENTS",
    language: string,           // BCP 47 tag of the spoken language: "en" or "es"
    mode?: "TRANSCRIPTION_MODE_WRITTEN" | "TRANSCRIPTION_MODE_VERBATIM",
    contextTerms?: string[],
  }
}
```

This start message asks for English with revised hypotheses. It adds recognition hints for an airline, a fare name, and a caller's surname:

```json theme={null}
{
  "start": {
    "outputContract": "STREAMING_OUTPUT_CONTRACT_REVISED_HYPOTHESES",
    "language": "en",
    "mode": "TRANSCRIPTION_MODE_WRITTEN",
    "contextTerms": ["Northwind Air", "FlexPlus", "Okonkwo"]
  }
}
```

This minimal start message selects append fragments for Spanish. It uses the default written mode, and the server confirms `es` in `started`:

```json theme={null}
{
  "start": {
    "outputContract": "STREAMING_OUTPUT_CONTRACT_APPEND_FRAGMENTS",
    "language": "es"
  }
}
```

#### Audio

Each `audio` message carries one chunk of audio, and it must meet these requirements:

| Requirement | Value |
| - | - |
| Encoding | Base64 in the JSON `audio` field. In binary frames, put the audio in the `bytes` field of the protobuf envelope. Never send bare PCM frames. |
| Decoded chunk | Not empty, with an even byte count. It fits in 64 KiB. |
| Encoded message | Fits in 96 KiB |
| Order | After `start`. `audio` before `start` fails with `invalid_input`, and no `started` follows. |

All of the decoded audio for the utterance must fit in 16 MiB. Within that limit, your client sets the chunk size by how many bytes it puts in each `audio` message. It sets the pace by when it sends each message. 100 ms of audio is 3,200 bytes, and you can send a recording faster than real time. [Limits](#limits) lists the error kind for each limit.

```typescript theme={null}
type AudioMessage = {
  audio: string, // base64
}
```

This JSON shows the shape of the message. The example shortens the base64 value:

```json theme={null}
{ "audio": "AAAAAAEA//8CAP7/AwD9/wQA/P8FAPv/BgD6/w..." }
```

#### End

`end` ends the audio and commits the utterance. After you send it, keep reading:

| After `end` | What happens |
| - | - |
| The utterance succeeds | The server sends the remaining transcript events, then `done` and a Close frame. |
| The utterance fails, and the connection is still writable | The server sends `error` instead of `done`. |
| You send audio, or a second `end` | The message fails with `invalid_input`. |

```json theme={null}
{ "end": {} }
```

#### Cancel

`cancel` stops the utterance and the work that Rime does on it. You can send it before or after `end`:

| Case | What you receive |
| - | - |
| The connection is still writable | `cancelled`, then a Close frame |
| The server already produced `delta` events | Those `delta` events can arrive before `cancelled`. |
| You send `cancel` before `start` | `cancelled`, without `started` |
| You close the connection instead of sending `cancel` | No acknowledgement. The work also stops. |

```json theme={null}
{ "cancel": {} }
```

### Receive

#### Started

`started` confirms that the server accepted your `start`.

```typescript theme={null}
type StartedEvent = {
  started: {
    requestId: string,
    accepted: {
      outputContract:
        | "STREAMING_OUTPUT_CONTRACT_REVISED_HYPOTHESES"
        | "STREAMING_OUTPUT_CONTRACT_APPEND_FRAGMENTS",
      language: { tag: string, source: "LANGUAGE_SOURCE_SELECTED" },
    },
  }
}
```

| Field | Meaning |
| - | - |
| `requestId` | The request ID. Keep it, because you need it when you report a failure. |
| `accepted.outputContract` | Echoes your contract |
| `accepted.language` | Confirms the language |

The session for the flight recording used English and revised hypotheses. It started with this event:

```json theme={null}
{
  "started": {
    "requestId": "fbd8066c-c67d-4529-80a3-75c4a288132d",
    "accepted": {
      "outputContract": "STREAMING_OUTPUT_CONTRACT_REVISED_HYPOTHESES",
      "language": { "tag": "en", "source": "LANGUAGE_SOURCE_SELECTED" }
    }
  }
}
```

#### Hypothesis

Under revised hypotheses, each `hypothesis` is a partial transcript.

```typescript theme={null}
type HypothesisEvent = {
  hypothesis: {
    text?: string,    // omitted when the transcript is empty
    revision: string, // uint64 as a decimal string
  }
}
```

| Field | Meaning |
| - | - |
| `text` | The complete transcript so far. Replace all of the previous text with it. |
| `revision` | The revision number of this text. JSON encodes it as a decimal string. |

This is the second of the four hypotheses in that session:

```json theme={null}
{"hypothesis":{"text":"Hi! I'm calling about flight 247 to Denver.","revision":"2"}}
```

#### Delta

Under append fragments, the server transcribes each five-second window of audio one time, on its own. It sends the text of the window as a `delta`, and a window with no recognized text produces no delta.

```typescript theme={null}
type DeltaEvent = {
  delta: {
    text?: string, // omitted when empty
  }
}
```

| Field | Meaning |
| - | - |
| `text` | The text of one window. Append it exactly as you receive it. The first delta has no separator at the start. Every later delta starts with a newline, `\n`, that separates it from the window before. |

Under append fragments, the same recording produced two deltas. The second delta starts with the newline separator:

```json theme={null}
{"delta":{"text":"Hi! I'm calling about flight 247 to Denver."}}
{"delta":{"text":"\nCan I move it to Friday?"}}
```

The `done.text` of that session was the two deltas joined in order: `"Hi! I'm calling about flight 247 to Denver.\nCan I move it to Friday?"`. Append fragments also suit a recording that is too long for a revised-hypotheses session, because the model's context does not grow during the session. [Transcribe a long recording](#transcribe-a-long-recording) has the steps and the limits that still apply.

#### Done

A successful utterance ends with one `done` event, then a Close frame. Under both contracts, `language` confirms the language.

| Contract | `text` | `revision` |
| - | - | - |
| Revised hypotheses | Equals the last hypothesis | Equals the revision of the last hypothesis |
| Append fragments | Every delta, joined in arrival order | 0. JSON omits it. |
| No recognized speech, under either contract | Empty. JSON omits it. | 0. JSON omits it. |

```typescript theme={null}
type DoneEvent = {
  done: {
    text?: string,     // omitted when empty
    revision?: string, // omitted when 0
    language: {
      tag: string,
      source: "LANGUAGE_SOURCE_SELECTED",
    },
  }
}
```

This is the `done` event for the flight recording, under revised hypotheses:

```json theme={null}
{
  "done": {
    "text": "Hi! I'm calling about flight 247 to Denver. Can I, move it to Friday?",
    "language": { "tag": "en", "source": "LANGUAGE_SOURCE_SELECTED" },
    "revision": "4"
  }
}
```

#### Error

`error` ends a failed utterance. If the connection is still writable, a Close frame follows.

```typescript theme={null}
type ErrorEvent = {
  error: {
    kind: "invalid_input" | "resource_exhausted" | "timeout" | "unavailable" | "cancelled" | "internal",
    message: string,
    requestId: string,
  }
}
```

| Field | Meaning |
| - | - |
| `kind` | The error category. Branch on it, and retry only the kinds that [Errors and limits](/docs/stt/errors#error-categories) marks as worth a retry. |
| `message` | A description of the error. Do not branch on it, because the text can change. |
| `requestId` | The request ID of the session |

A `start` message without `outputContract` fails with `invalid_input`:

```json theme={null}
{
  "error": {
    "kind": "invalid_input",
    "message": "streaming output contract must be explicitly selected",
    "requestId": "26c49cfc-ef9a-495a-9984-3db3fa159516"
  }
}
```

#### Cancelled

`cancelled` confirms your `cancel`. If the connection is still writable, a Close frame follows.

```json theme={null}
{ "cancelled": {} }
```

## Start parameters

**outputContract** `string` *required*

Set one of these two values. There is no default, and an omitted, unspecified, or unknown value fails with `invalid_input`. [Compare the two contracts](/docs/speech-to-text#choose-how-to-connect) to choose between them:

| Value | Updates | Use it when |
| - | - | - |
| `STREAMING_OUTPUT_CONTRACT_REVISED_HYPOTHESES` | The whole transcript so far, in each `hypothesis` | Your consumer can replace text. |
| `STREAMING_OUTPUT_CONTRACT_APPEND_FRAGMENTS` | One `delta` for each window of audio | Your consumer can only append. |

**language** `string` *required*

The spoken [language](/docs/stt/recognition#language): `en` or `es`. The service does not detect the language. An omitted tag, `und`, an unsupported language, or a malformed tag fails with `invalid_input`.

**mode** `string` (default: TRANSCRIPTION\_MODE\_WRITTEN)

Set one of these values. The mode guides the model, but it does not guarantee exact output.

| Value | Asks for | Example |
| - | - | - |
| `TRANSCRIPTION_MODE_WRITTEN` | Numbers as digits, and compact codes | Spoken "flight two four seven" becomes `flight 247`. |
| `TRANSCRIPTION_MODE_VERBATIM` | [Word-for-word mode](/docs/stt/recognition#word-for-word-mode): every filler, repetition, and false start, and numbers as spoken words | "uh flight two four seven" stays as spoken. |

**contextTerms** `string[]`

Recognition hints, such as `["Northwind Air", "FlexPlus", "Okonkwo"]`. The terms make the model more likely to recognize them, but they do not force a match. The [context-term rules](/docs/stt/recognition#context-terms) apply, and a value that breaks one fails with `invalid_input`:

| Rule | Requirement |
| - | - |
| Each term | Not empty, and not only whitespace |
| The value | An array, not a single string |
| The terms, joined with `, ` | Fit in 1,024 UTF-8 bytes |

## Limits

| Limit | Value | Error kind |
| - | - | - |
| Time to send `start` after connecting | 5 seconds | `timeout` |
| Time without audio | 15 seconds. Silence samples count as audio. WebSocket pings do not. | `timeout` |
| Total time per stream, from connecting to the final transcript | 120 seconds | `timeout` |
| Audio per utterance, decoded | 16 MiB | `resource_exhausted` |
| Audio per message, decoded | 64 KiB, non-empty, with an even byte count | `resource_exhausted` over 64 KiB. `invalid_input` if empty or odd |
| WebSocket message, encoded | 96 KiB | `resource_exhausted` |
| Context terms, joined with `, ` | 1,024 UTF-8 bytes | `invalid_input` |
| Transcript per utterance | 1 MiB | `resource_exhausted` |
| Model context | Depends on the audio, context terms, and transcript so far | `resource_exhausted` |

The [idle timer and model context](/docs/stt/errors#limits) follow some more rules.

## Transcribe a long recording

Some recordings are too long for one revised-hypotheses stream. Send them through the direct API with append fragments. Rime recognizes each five-second window on its own, so the model's context does not fill up as the recording continues. For now, only the direct API can do this, because the SDK receives only revised hypotheses.

1. Open a WebSocket stream, or a gRPC `TranscribeStreaming` call, with `STREAMING_OUTPUT_CONTRACT_APPEND_FRAGMENTS`.
2. Send the audio as fast as the connection accepts it, not at the speed of the recording. The 120-second limit is wall-clock time from connecting to `done`, not the length of the audio.
3. Keep each stream within the [16 MiB audio limit](/docs/stt/errors#limits), which is about 8.7 minutes of 16 kHz mono PCM16. Send a longer file in parts, with one stream for each part. Join the `done.text` values of the parts in order, with a newline between them, because a newline also separates deltas.

## Complete Python client

This client streams a 16 kHz mono PCM16 WAV file over the WebSocket API. It opens the file with Python's `wave` module, and it sends the samples without the header, as fast as it reads them. As the hypotheses arrive, the client replaces the console line with each one. At the end, it prints the final `done` result. The client uses revised hypotheses, which suit a short file. For a [long recording](#transcribe-a-long-recording), use append fragments.

Before you run the client:

1. Download the [sample recording](/audio/stt/flight.wav).
2. Set `RIME_API_KEY` in the environment of your backend.
3. Install `websockets` 14 or later with `pip install "websockets>=14"`. The client uses the `additional_headers` argument of that version.

To use your own recording, convert it to a WAV file in the same format:

1. Install [FFmpeg](https://ffmpeg.org/download.html).
2. Convert the recording. This command reads `input.m4a`, or any other format that FFmpeg can decode. It writes `sample.wav` as 16 kHz mono PCM16, and it keeps the WAV header.

   ```bash theme={null}
   ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le sample.wav
   ```

   The conversion in the quickstart writes headerless PCM instead, which `wave.open()` rejects.

> **transcribe\_ws.py**
>
> ```python transcribe_ws.py theme={null}
> import asyncio
> import base64
> import json
> import os
> import sys
> import wave
>
> import websockets
>
> URL = "wss://stt.api.rime.ai/ws"
> API_KEY = os.environ["RIME_API_KEY"]
> CHUNK_BYTES = 3_200  # 100 ms of PCM16 mono at 16 kHz
>
>
> async def transcribe(path):
>     with wave.open(path, "rb") as source:
>         if (source.getnchannels(), source.getsampwidth(), source.getframerate()) != (1, 2, 16000):
>             raise ValueError("expected 16 kHz mono 16-bit PCM")
>         pcm = source.readframes(source.getnframes())
>
>     headers = {"Authorization": f"Bearer {API_KEY}"}
>     async with websockets.connect(URL, additional_headers=headers) as socket:
>         await socket.send(json.dumps({
>             "start": {
>                 "outputContract": "STREAMING_OUTPUT_CONTRACT_REVISED_HYPOTHESES",
>                 "language": "en",
>             }
>         }))
>         started = json.loads(await socket.recv())
>         if "started" not in started:
>             raise RuntimeError(started)
>         print("Request:", started["started"]["requestId"])
>
>         async def send_audio():
>             for offset in range(0, len(pcm), CHUNK_BYTES):
>                 chunk = pcm[offset:offset + CHUNK_BYTES]
>                 await socket.send(json.dumps({"audio": base64.b64encode(chunk).decode()}))
>             await socket.send(json.dumps({"end": {}}))
>
>         async def receive():
>             async for frame in socket:
>                 event = json.loads(frame)
>                 if "hypothesis" in event:
>                     print("\r\x1b[K" + event["hypothesis"].get("text", ""), end="", flush=True)
>                 elif "done" in event:
>                     print()
>                     return event["done"]
>                 elif "error" in event:
>                     raise RuntimeError(f'{event["error"]["kind"]}: {event["error"]["message"]}')
>                 elif "cancelled" in event:
>                     raise RuntimeError("cancelled")
>             raise RuntimeError("connection closed before done")
>
>         _, done = await asyncio.gather(send_audio(), receive())
>         return done
>
>
> print(asyncio.run(transcribe(sys.argv[1])))
> ```

Run the client with the file name, for example `python transcribe_ws.py flight.wav`. A successful run prints the request ID, then the latest hypothesis on one line, and then the `done` result. A recording with no speech can return `done` with no `text`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.