Speech-to-Text gRPC API

Connect to rime.SpeechToText at stt.api.rime.ai:443 from your backend, with gRPC over TLS. Send your API key in the metadata of each call, as authorization: Bearer YOUR_API_KEY. Write the key name in lowercase, because gRPC metadata keys are lowercase. For example, Python’s grpcio rejects Authorization.

Method	You send	Rime returns
`Transcribe`	One complete WAV file, up to 16 MiB	One transcript
`TranscribeStreaming`	One utterance of headerless 16 kHz mono PCM16	Transcript updates, then `done`

The Rime SDK calls TranscribeStreaming for you in Python, Node.js, Go, and Rust. To call a method yourself, generate a client from the schema with the protobuf tools for your language.

Transcribe

Transcribe takes a short recording as one complete WAV file and returns one transcript. The request carries the file in audio, and the language of the recording in options:

Field	Value
`audio`	One complete WAV file, up to 16 MiB. Include the RIFF header, because headerless PCM is not a WAV file, and it fails. Use signed 16-bit little-endian PCM at 16 kHz, mono. Other WAV encodings and sample rates can work, but Rime guarantees only this format.
`options.language`	Required. `en` or `es`. See Language.
`options.mode`	Optional. `TRANSCRIPTION_MODE_WRITTEN` (default) or `TRANSCRIPTION_MODE_VERBATIM`. See Word-for-word mode.
`options.context_terms`	Optional recognition hints. Joined with `,` , they must fit in 1,024 UTF-8 bytes. See Context terms.

These commands need grpcurl and the RIME_API_KEY variable. They download the schema and the sample flight recording, transcribe the recording as English, and print the response as JSON. They pass the request through stdin with -d @, which avoids the shell’s limit on argument length for base64 audio.

curl -fsSLO https://raw.githubusercontent.com/rimelabs/rime-api/main/schema/rime/speech_to_text.proto
curl -fsSLO https://docs.rime.ai/audio/stt/flight.wav

{
  printf '{"options": {"language": "en", "mode": "TRANSCRIPTION_MODE_WRITTEN"}, "audio": "'
  base64 < flight.wav | tr -d '\n'
  printf '"}'
} | grpcurl \
  -H "authorization: Bearer $RIME_API_KEY" \
  -proto speech_to_text.proto \
  -d @ \
  stt.api.rime.ai:443 rime.SpeechToText/Transcribe

For the flight recording, Transcribe returned:

{
  "text": "Hi! I'm calling about flight 247 to Denver. Can I, move it to Friday?",
  "language": {
    "tag": "en",
    "source": "LANGUAGE_SOURCE_SELECTED"
  }
}

Field	Value
`text`	The transcript. It is empty when the recording has no recognizable speech.
`language`	Confirms the language.
`x-request-id` response header	The request ID

The 16 MiB limit is only the transport limit, and a file runs out of model context long before it reaches 16 MiB. The recording and your context terms must fit in the context together. In practice, Transcribe holds about as much speech as a revised-hypotheses stream (limits). Rime never returns a shortened transcript in place of an error:

Failure	Status
The file does not fit in 16 MiB or in the model’s context	`RESOURCE_EXHAUSTED`
Transcription does not finish within 120 seconds	`DEADLINE_EXCEEDED`
The recognition backend rejects the file	`INVALID_ARGUMENT` or `UNAVAILABLE`

For a longer recording, stream it faster than real time with append fragments over the WebSocket API. TranscribeStreaming accepts the same contract, and Transcribe a long recording has the steps.

TranscribeStreaming

Each call transcribes one utterance. Send and read messages in this order:

Step	Message	Details
1. Send	`config`	Send it first, and only one time. Set `output_contract` and `language`. You can also set `mode` and `context_terms`.
2. Read	`accepted`	The first response. It confirms the output contract and the language. The request ID arrives in the response headers. You can send audio before `accepted` arrives, and that audio waits until the server accepts `config`.
3. Send	`audio`	Headerless signed 16-bit little-endian PCM at 16 kHz, mono. Each payload is non-empty, has an even byte count, and fits in 64 KiB. The utterance can have at most 16 MiB of audio in total.
3. Read	`hypothesis` or `delta`	Read transcript updates while you send audio.
4. Send	Half-close	When the utterance ends, half-close the request stream. Stop sending, but keep the response stream open.
5. Read	`done`, then an OK status	Read the remaining updates and `done`, and then make sure that the call ends with an OK status. A stream that ends without `done` and an OK status is incomplete.

Your client sets the chunk size by how many bytes it puts in each audio payload, up to 64 KiB. It sets the pace by when it sends each payload. 100 ms of audio is 3,200 bytes, and you can send a recording faster than real time. If you cancel the call, or if either side drops, the backend work stops. A request out of order, a request with no payload, or an invalid setting fails with INVALID_ARGUMENT, as Status codes lists. Set output_contract to one of these values. There is no default. Choose by whether your consumer can replace text that it already has (compare them).

Output contract	Handle updates	Final result
`STREAMING_OUTPUT_CONTRACT_REVISED_HYPOTHESES`	Replace the previous text with each `hypothesis`. `revision` starts at 1 and increases by 1 each time the text changes.	`done.text` and `done.revision` match the last hypothesis. An utterance with no recognized speech can end with empty text at revision 0 and no hypothesis.
`STREAMING_OUTPUT_CONTRACT_APPEND_FRAGMENTS`	Append each `delta` exactly as received. Rime transcribes each five-second audio window one time, on its own. Every delta after the first starts with `\n`.	`done.text` equals every delta joined in order. `done.revision` is 0.

Both contracts return the language in done.language, with source set to LANGUAGE_SOURCE_SELECTED. The server applies these deadlines:

Deadline	Value
From the start of the call to `config`	5 seconds
Time without audio	15 seconds. Silence samples count as audio. Keepalive pings do not.
The whole call, including recognition after half-close	120 seconds

The limits table has every other limit.

Status codes

Branch on the status code, not on the message text, and retry a status only when Errors and limits says to retry it. Keep the request ID when you report a failure. The server returns x-request-id in the response headers of a successful call, and in the trailers of most failed calls.

Status	Causes
`UNAUTHENTICATED`	The `authorization` metadata is missing, or Rime does not accept the key. The response has no request ID.
`INVALID_ARGUMENT`	A missing or unknown output contract. A missing, unsupported, or malformed language. An invalid mode or context term. Messages out of order, such as audio or a half-close before `config`, or a second `config`. A request with no payload. Empty audio, or an empty or odd-length audio payload. A file that the recognition backend rejects.
`RESOURCE_EXHAUSTED`	An audio payload over 64 KiB, or an utterance or file over 16 MiB. Audio that does not fit the model’s context. A transcript over 1 MiB. The service has no capacity for the call.
`DEADLINE_EXCEEDED`	The start, idle, or total deadline passed.
`UNAVAILABLE`	The service is not ready, or it is shutting down. The recognition backend failed. Recognition ended without a final transcript.
`CANCELLED`	The call was cancelled.
`INTERNAL`	An unexpected server error.

Schema

The wire contract for both methods is speech_to_text.proto, in the public rimelabs/rime-api repository. The same file defines the protobuf envelopes of the WebSocket endpoint. These packages contain generated message definitions for this schema, starting with version 0.2.0:

Language	Package
Python	`rime-api`, with the messages. Generate the client stubs from the schema, for example with `grpcio-tools`.
npm	`@rimelabs/api`, with the messages. Load the schema with `@grpc/proto-loader`, and create the client with `@grpc/grpc-js`.
Go	The module `github.com/rimelabs/rime-api/go`, with the messages and the client stubs
Rust	The crate `rimelabs-api`, with the messages. Its `grpc` feature adds the tonic client stubs.

Overview

Coda API reference

Mist v3 API reference

Speech-to-text API reference

API Metadata

Other APIs

gRPC

Transcribe

TranscribeStreaming

Status codes

Schema

​Transcribe

​TranscribeStreaming

​Status codes

​Schema

Transcribe

TranscribeStreaming

Status codes

Schema