Build a Prism agent with the SDK

In this quickstart, you’ll run a Prism voice agent on your computer, give it your own instructions, and talk to it through your microphone and speakers. You’ll set up two parts. The server is your agent, built with our SDK. A page in your browser acts as the caller’s phone. It captures your voice, plays the replies, and cancels the echo of Prism’s voice, so laptop speakers work and you can interrupt Prism at any time.

What you’ll need

A Rime API key. If you don’t have one yet, create one.
uv for Python, or Node.js 22 or later.
A browser and a microphone.

Set up with your coding agent

To build this agent backend and test page in your project, paste this prompt into Claude Code, Codex, or Cursor:

Prompt for your coding agent

Build the backend of a voice agent on Rime Prism, a speech-to-speech
voice-agent API, and a test page that lets me talk to the agent in my
browser, through my microphone.

Read these pages first, and follow them over anything you remember:
- https://docs.rime.ai/docs/prism/quickstart.md
- https://docs.rime.ai/docs/prism/live-conversations.md
- https://docs.rime.ai/api-reference/sdk/prism.md

Requirements:
- A server in Python (the latest rimelabs-sdk) or Node.js (the latest
  @rimelabs/sdk) reads RIME_API_KEY from the environment. The key never
  reaches the browser.
- The server uses the SDK's conversation runner: run_conversation in
  Python, or runConversation in Node.js. The browser connection is its
  source and playback adapter.
- The server connects to wss://prism.api.rime.ai/v1/realtime with my
  instructions and the voice aurora.
- The browser captures the microphone with echo cancellation on, and sends
  it to the server the whole time, also while Prism speaks.
- The browser plays Prism's 24 kHz mono PCM16 replies in order.
- When Prism reports that the caller started speaking, stop playback at
  once, drop the queued audio, and report how much of the reply played.
- The page shows the caller's transcript and Prism's reply text.

When it runs, tell me the command that starts it and the address to open.

When it’s done, your coding agent tells you the command to run and the address to open.

Set up by hand

To run our starter instead, follow these steps.

Get the starter

Clone the starter from the rime-sdk repository, at a tag that holds both the Python and Node.js servers:

git clone --depth 1 --branch python-v0.1.0-alpha.20 https://github.com/rimelabs/rime-sdk.git
cd rime-sdk/examples/prism-web

Inside, you’ll find one page and a server for each language:

File	What it does
`public/`	The test page: your instructions, the voice, the microphone, playback, and the transcript
`server.py`	Your agent in Python, with `rimelabs-sdk`
`server.mjs`	Your agent in Node.js, with `@rimelabs/sdk`
`check.py`, `check.mjs`	A test without a microphone, for servers and CI
`question.wav`	The recorded question that the check sends

For Go or Rust, see the Prism SDK reference.

Set your API key

Set your key in the terminal where you’ll start the server:

macOS and Linux
Windows PowerShell

export RIME_API_KEY="YOUR_API_KEY"

$env:RIME_API_KEY = "YOUR_API_KEY"

Start the server

Python
Node.js

Start the server with uv. The first run installs the SDK.

uv run server.py

Install the packages, and then start the server:

npm install
node server.mjs

The server prints the page’s address:

Open http://localhost:3000 and press Start.

Talk to your agent

Open http://localhost:3000 in your browser. The instructions box starts with a made-up cafe, so you can start talking right away:

Keep the instructions, or paste your own system prompt.
Choose a voice. Prism has four: aurora, wawona, brando, and astra.
Press Start, and allow the microphone when the browser asks.
Ask “What’s on the menu?” in English.
While Prism lists the menu, ask “What time do you close?” Prism stops and answers the new question.

You should hear each answer, and the page shows a transcript like this one:

Connecting to Prism...
Ready. Start talking.
You: Hi there, what's on the menu?
Prism: We have a latte for $5, a cold brew for $4, and a blueberry muffin for $3.
You: Sorry, what time do you close?
Prism: We close at 4 p.m. every day.

The wording changes from run to run, but the prices and closing time come from your instructions, so they’ll stay the same. While you try it:

To change the voice or instructions: press Stop, make the change, and press Start. Each start is a new conversation.
If the page shows an error: look up its code in Errors and limits. If the server stops right away and asks for RIME_API_KEY, set it in the terminal that runs the server.
If port 3000 is in use: start the server with PORT=3001, and open http://localhost:3001.

How the starter works

The server is the part you keep and build on: Your whole agent is three parts: one connect call, the SDK’s conversation runner, and two audio adapters. Here are the first two, simplified from the talk function in server.py and server.mjs:

Python
Node.js

async with client.realtime.connect(voice=voice, instructions=instructions) as session:
    await run_conversation(session, source=browser, playback=browser, on_event=show)

const session = await client.realtime.connect({ voice, instructions });
await runConversation(session, { source: browser, playback: browser, onEvent });

The SDK connects to wss://prism.api.rime.ai/v1/realtime. Then the runner handles the work that every Prism conversation needs:

It sends the microphone audio the whole time, even while Prism speaks, so Prism can hear an interruption.
It plays each reply in order.
It stops playback at once when the caller talks over a reply.
It reports how much of each reply the caller heard, because Prism keeps only that part.

The two adapters are yours: source brings in the caller’s audio, and playback plays each reply and measures how much the caller heard. In the starter, one Browser object is both adapters, passing audio to and from the page, and the event callback shows the transcript. Swap the adapters, and the same agent can answer phone calls or run inside your own app.

Connect real callers

When you’re ready, real callers can reach your agent in one of three ways:

Your own application: write a source and a playback adapter for its audio, and keep the rest of the server. Live conversations explains each step.
A LiveKit room: the LiveKit quickstart builds the agent with LiveKit Agents, and the Rime plugin connects to Prism for you.
A phone line: the phone audio goes through your server, and only the adapters change. They convert the audio for Prism and play the replies back, as Phone audio in Live conversations explains.

Test without a microphone

A server, a CI job, or a coding agent’s sandbox has no microphone, so the starter’s check sends a recorded question instead:

Python
Node.js

uv run check.py

node check.mjs

Here’s the output from one run:

Caller started speaking
Caller stopped speaking
Prism: We close at 4 p.m.
Saved 1.6 seconds of audio to reply.wav

The recording, question.wav, is a caller asking when the cafe closes today. The check sends it at a microphone’s pace, followed by silence, so Prism can find the end of the turn. If Prism says 4 p.m., the connection, turn detection, and instructions all worked, and you can hear the answer in reply.wav.

Use your own recording

The check reads a 16 kHz mono WAV file. To ask your own question:

Record your question in English.
Convert the recording with FFmpeg. This command reads recording.mp3 and writes recording.wav:
ffmpeg -i recording.mp3 -ac 1 -ar 16000 -c:a pcm_s16le recording.wav
Run the check with your file: uv run check.py recording.wav or node check.mjs recording.wav. The check’s instructions cover only the cafe’s hours, so change them for other questions.

Record the question without long pauses: Prism finds the end of the turn by itself, so a long pause can end the turn early. The check then stops after the first part, or fails with Reply cancelled: turn_detected.

Use the API without the SDK

To use a plain WebSocket client instead, for a backend in any language, follow the WebSocket quickstart.

Next steps

Now that your agent answers, you can:

Write your own system prompt, starting with rules for speech.
Give Prism tools, so it can ask your backend for data it doesn’t have, such as Saturday’s opening hours.
Choose the voice that suits your callers from the four that Prism offers.
Make Prism speak first with a nudge, or send it typed text.

Get started

Build with Rime

Integrations

Account & updates

Build a Prism agent with the SDK

What you’ll need

Set up with your coding agent

Set up by hand

How the starter works

Connect real callers

Test without a microphone

Use your own recording

Use the API without the SDK

Next steps

​What you’ll need

​Set up with your coding agent

​Set up by hand

​How the starter works

​Connect real callers

​Test without a microphone

​Use your own recording

​Use the API without the SDK

​Next steps

What you’ll need

Set up with your coding agent

Set up by hand

How the starter works

Connect real callers

Test without a microphone

Use your own recording

Use the API without the SDK

Next steps