Skip to main content
All requests require authentication with a bearer token in the Authorization header: Authorization: Bearer YOUR_API_KEY. See API authentication for how to create a key.

Overview

In addition to a plaintext websocket implementation, Rime also has an implementation that sends and receives events as JSON objects. Like the other implementation, all synthesis arguments are provided as query parameters when establishing the connection. The WebSocket API buffers inputs up to one of the following punctuation characters: ., ?, !. This is most pertinent for the initial messages sent to the API, as synthesis won’t begin until there are sufficient tokens to generate audio with natural prosody. After the first synthesis of any given utterance, typically enough time has elapsed that subsequent audio contains multiple clauses, and the buffering becomes largely invisible.

Messages

Send

Text

This is the most common message, which contains text for synthesis. schema:
examples:
Context IDs can be provided, which will be attached to subsequent messages that the server sends back to the client. Rime will not maintain multiple simultaneous context ids. The events will contain the most recent context ID at the time that audio was requested. In the above examples, even if both messages are received by the server before it sends any audio, the audio response for the first sentence will be tagged with contextId: null, and the audio for the second will be tagged with its UUID.

Clear

Your client can clear out the accumulated buffer, which is useful in the case of interruptions.

Flush

This forces whatever buffer exists, if any, to be synthesized, and the generated audio to be sent over.

EOS

At times, your client would like to generate audio for whatever remains in the buffer, and then have the connection immediately closed.

Receive

Chunk

The most common event will be the audio chunk.
The audio will be a base64 encoded chunk of audio bytes in the audio format specified when the connection was established. If you provided any context id when sending the relevant text, it’ll be included here.

Timestamps

Word-level timestamps are emitted alongside the audio chunks so the client can tell exactly which words have been spoken at any point. This is especially useful for handling interruptions: when the user starts talking over the output, you can map the playback position back to the last word that was actually heard.
Timestamps are emitted only when lang is en/eng or es/spa, or when lang is omitted. Requests in any other language receive chunk and done events with no timestamps event and no error. Do not block playback while waiting for a timestamps event.
The three arrays inside word_timestamps are the same length and index-aligned: for a given index i, words[i] is spoken from start[i] to end[i]. Times are in seconds, measured from the beginning of the audio for the current synthesis. If a context id was attached to the text that produced this audio, it is included on the event. Example payload:

Done

After the last audio chunk for a synthesis batch has been sent, the server emits a done event. This signals that the current synthesis is fully complete. If the client sends more text and triggers further synthesis, another done will follow. The eos operation always emits a final done before the server closes the connection.
On /ws2, the done event carries an extra boolean done: true field in addition to type: "done". On /ws3 the event contains only type and contextId. Match on type === "done" to handle both.
When exactly done fires depends on the segment setting. See Segmentation and behavior settings for full details.

Error

In the event of a malformed or unexpected input, the server will immediately respond with an error message. The server will not close the connection, and will still accept subsequent well-formed messages. It’s up to the client to decide if it wants to close upon receiving an error.

Variable parameters

string
required
Must be one of the voices listed in our documentation.
string
required
The text you’d like spoken. Character limit per request is 1,000 via the API and in the dashboard UI.
string
Set to mistv2.
string
One of mp3, mulaw, or pcm
string
default:"eng"
If provided, the language must match the language spoken by the provided speaker. This can be checked in our voices documentation.
bool
default:"false"
When set to true, adds pauses between words enclosed in angle brackets. The number inside the brackets specifies the pause duration in milliseconds.
Example: “Hi. <200> I’d love to have a conversation with you.” adds a 200ms pause between the first and second sentences.
bool
default:"false"
When set to true, you can specify the phonemes for a word enclosed in curly brackets.
Example: “{h’El.o} World” will pronounce “Hello” as expected. Learn more about custom pronunciation.
string
Comma-separated list of speed values applied to words in square brackets. Values < 1.0 speed up speech, > 1.0 slow it down. Example: “This is [slow] and [fast]”, use “3, 0.5” to make “slow” slower and “fast” faster.
int
The value, if provided, must be between 4000 and 44100. Default: 22050
float
default:"1.0"
Adjusts the speed of speech. Lower than 1.0 is faster and higher than 1.0 is slower.Note: this is the legacy Mist v2 convention. Newer models (Coda, Arcana, Mist v3) invert it; for those, higher than 1.0 is faster.
bool
default:"false"
mist/mistv2 only. Skips text normalization of the input text prior to synthesizing audio. This will reduce latency at the cost of some possible mispronunciation of digits and abbreviations.
string
default:"bySentence"
Controls how text is segmented for synthesis. Available options:
  • “immediate” - Synthesizes text immediately without waiting for complete sentences
  • “never” - Never segments the text, waits for explicit flush or EOS
  • “bySentence” (default) - Waits for complete sentences before synthesis
Note: For backward compatibility, setting immediate=true in query params is equivalent to segment=immediate. If a null value is provided, it will default to “bySentence”.