> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Performance tuning

> Tune supported Coda on-prem deployments for concurrency without exceeding latency or real-time-factor targets.

Choose your service-level target before you raise concurrency. The standalone engine limits accepted synthesis across HTTP, gRPC, and WebSocket. Once it reaches capacity, it rejects new work. If you need requests to wait or want a rate limit per caller, add those controls in your infrastructure. See [Load balancing](/docs/on-prem/load-balancing#max-concurrency) for the admission limit.

## What to measure

* **Initial latency**, or time to first frame or byte (TTFF/TTFB), measures the time from sending a request until the first frame arrives. Lower is better.
* **Real-time factor (RTF)** is processing time divided by stream duration. RTF must remain at or below 1 for real-time delivery. Lower is better.
* **Concurrency** is the number of simultaneous requests the service can handle while meeting the target. Higher is better.

Latency and RTF usually rise with concurrency. Queue or reject requests above the measured limit instead of allowing overload to degrade every active stream. The engine reports initial latency as `rime.engine.initial_latency`, and [Metrics](/docs/on-prem/metrics) covers the other series.

## Coda

For Coda, `GENERATOR_MAX_BATCH` and `INFERENCE_CONCURRENCY_CAPACITY` are the tuning controls, and their defaults work for most deployments. `INFERENCE_CONCURRENCY_CAPACITY` (default `128`) sets the generator's admission limit and sizes its KV cache, so raising it costs GPU memory.

| Variable | Default |
| :- | :- |
| `GENERATOR_MAX_BATCH` | `64` |
| `INFERENCE_CONCURRENCY_CAPACITY` | `128` |

Increase `GENERATOR_MAX_BATCH` to `128` when you serve higher traffic or run on more modern GPUs. A larger batch supports higher concurrency at the cost of longer container startup time. Leave the other controls at their defaults.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.