Low-latency TTS for real-time voice agents

Latency at a glance

Rime’s cloud API delivers sub-200ms end-to-end latency under typical conditions. Coda reaches sub-100ms model latency on the GPU engine when self-hosted or on-prem, while Mist v3 has the lowest measured time to first audio. Via the cloud API, the total time you measure also includes network round-trip; typically 25–50ms from most of the continental US when you pick the closest regional endpoint. Contact us to optimize further.

Real-time performance benchmarks

These numbers were measured on a single Lambda H100 SXM machine with an H100 SXM5 GPU, 26 vCPUs, 221 GiB RAM, Ubuntu 24.04, and NVIDIA driver 595.58.03.

Metric	Coda	Arcana v3	Mist v3
TTFA (P50) @ 1 Concurrency	96 ms	118 ms	37 ms
TTFA (P90) @ 1 Concurrency	98 ms	119 ms	56 ms
TTFA (P50) @ 12 Concurrency	150 ms	168 ms	37 ms
TTFA (P90) @ 12 Concurrency	181 ms	182 ms	56 ms
RTF (P99)	0.33	0.29	0.004

TTFA = Time-to-First-Audio. RTF = Real-Time Factor (synthesis time ÷ audio duration; values below 1.0 mean synthesis is faster than playback).

Reduce latency

Choose the model for the constraint. Use mistv3 for the lowest cloud time to first audio. Choose Coda when voice quality is the priority and sub-100ms engine latency meets the target.
Consume the response as a stream. Start playback from the first bytes instead of buffering the complete response.
Use the nearest regional endpoint. Route East Coast traffic to US East and West Coast traffic to US West.
Request only the audio you need. Smaller payloads cross the network faster. Telephony applications should request 8kHz audio directly.
Skip normalization only for already normalized text. noTextNormalization removes preprocessing, but it is safe only when the input has no digits, abbreviations, or ambiguous punctuation.

What affects latency

End-to-end latency combines server processing, response size, and network distance.

Factors that affect API response time

Server processing: Model inference, text preprocessing, and audio postprocessing all contribute to the first response byte.
Payload size: Larger audio responses take longer to transmit.
Network distance: A longer route between the application and Rime increases round-trip time.

Stream the response

Streaming reduces time to first playback because the client can consume the first chunk before Rime finishes generating the complete response. Buffering the whole body delays playback until synthesis and transfer are complete.

Chart comparing total response time with time to first byte for a streaming PCM request — Response time and TTFB for a streaming PCM request

The chart uses this request body:

curl -X POST https://users.rime.ai/v1/rime-tts \
  -H 'Accept: audio/L16' \
  -H 'Authorization: Bearer $RIME_API_KEY' \
  -d '{"text": "I love the book that she gave me. And I was so happy that she gave it to me.", "speaker": "cove",  "modelId": "mistv3"}'

To actually consume a streaming response in Python, iterate over the response body as it arrives instead of buffering the whole thing:

Python

import requests

with requests.post(
    "https://users.rime.ai/v1/rime-tts",
    headers={
        "Authorization": f"Bearer {RIME_API_KEY}",
        "Content-Type": "application/json",
        "Accept": "audio/mpeg",
    },
    json={
        "speaker": "astra",
        "text": "Hello from Coda.",
        "modelId": "coda",
        "lang": "en",
    },
    stream=True,
) as response:
    response.raise_for_status()
    with open("hello.mp3", "wb") as f:
        for chunk in response.iter_content(chunk_size=4096):
            f.write(chunk)

Rime’s models produce PCM audio by default, and conversion and downsampling must take place for sampling rates other than 22kHz. For low-bandwidth applications, the smaller payload usually outweighs the added server processing.

Network latency by region

Rime serves the API from multiple regions so you can route requests to the data center closest to your application. Picking the closest endpoint typically shaves tens of milliseconds off round-trip time. These are rough round-trip times (RTT), the back-and-forth network delay between your application and Rime, from major US metros to each region:

From metro	To US East	To US West
East Coast (NYC, DC, Boston, Atlanta)	5–25 ms	60–85 ms
Midwest / South (Chicago, Dallas, Denver)	25–55 ms	35–65 ms
West Coast (SF, LA, Seattle)	60–85 ms	5–25 ms

Rules of thumb:

Same region (your app and the Rime endpoint in the same AWS region): typically 1–10 ms.
Coast-to-coast: ~60 ms is the physical floor, set by the speed of light in fiber across ~2,500 miles. 60–85 ms is normal.
Above 90 ms between major US metros usually points to a suboptimal network route, not Rime.

For voice AI: if you serve users nationwide from a single region, far-coast users will see 60–85 ms of network round-trip on top of Rime’s inference time. To stay comfortably under 200 ms end-to-end, route East-coast users to US East and West-coast users to US West. For the full endpoint URLs and routing guidance, see Regional endpoints. To measure TTFB against each endpoint from your own machine, run rime speedtest.

Overview

Quickstart

Voice agents

Voices & models

Customizing speech

Streaming & performance

Integrations

On-prem

Account & updates

Low-latency TTS for real-time voice agents

Latency at a glance

Real-time performance benchmarks

Reduce latency

What affects latency

Factors that affect API response time

Stream the response

Network latency by region

​Latency at a glance

​Real-time performance benchmarks

​Reduce latency

​What affects latency

​Factors that affect API response time

​Stream the response

​Network latency by region

Latency at a glance

Real-time performance benchmarks

Reduce latency

What affects latency

Factors that affect API response time

Stream the response

Network latency by region