Skip to main content
In this quickstart, you’ll build a Pipecat voice agent with Prism as its model, and talk to it from your browser. Your agent answers questions about a made-up cafe, and runs a function in your code to look up its hours. Prism replaces a pipeline agent’s speech-to-text, LLM, and text-to-speech services: it listens, finds the end of each turn, and replies out loud. Pipecat carries the audio between your browser and the agent, and runs your tools.

What you’ll need

  • A Rime API key. If you don’t have one, create one. The key stays on your Python server.
  • uv, which installs Python for the project if needed. This guide uses Python 3.12. The Rime SDK needs Python 3.11 or later.
  • A browser with microphone access. The browser connects to your local server through Pipecat’s SmallWebRTC transport.

Build the agent

1

Create the project

Create a Python project:
Install the packages, pinned to the versions tested for this guide. Prism support comes from a preview build in Rime’s Pipecat fork because the upstream integration has not merged. The rime-realtime extra adds the Rime SDK, and webrtc and runner add the browser transport and local server:
2

Set your API key

Add .env.local to the project’s .gitignore before you save your key:
.gitignore
The Python process reads your Rime API key from .env.local. Create that file in the project folder:
.env.local
Keep this file out of version control. The local browser client needs no Rime credentials.
3

Save the agent

Save this script as bot.py. It opens one Prism conversation for each browser connection and registers get_opening_hours as a tool. Only the tool result contains the cafe’s hours.
bot.py
The Rime aggregator pair records speech transcripts without requesting a second reply. Use llm.create_context_aggregator(context) for this service. Prism detects turns, so the pipeline has no separate STT, TTS, or local VAD service.When Prism requests the tool, Pipecat runs the function. result_callback sends the result back so Prism can continue its reply. Closing the worker also closes the service’s Prism session and SDK client.
4

Start the agent

Run the server with WebRTC enabled:
Keep the process running. Open the local browser client. Wait for the client status to show INITIALIZED, then select Connect and allow microphone access.
5

Talk to the agent

After the greeting, ask, “What time do you close today?” The terminal should log get_opening_hours called, and Prism should answer with 4 p.m. The hours are not in the instructions, so the tool result supplies the answer.Ask another question, then speak while Prism replies. The agent should stop playback and answer your new turn. Ask one more question to check that the conversation continues.

How Prism changes a Pipecat agent

Prism runs the whole conversation, so a few Pipecat features work differently than in a pipeline agent:

Update tools and hand off a conversation

Use LLMUpdateSettingsFrame to change the agent’s instructions and available tools during a call. A handoff can keep the same Prism session and conversation history while changing the agent’s task. For example, after answering an opening-hours question, your application can switch the cafe agent to menu help. Add this import to bot.py:
Inside bot, after you create worker, add a menu tool and a handoff function. The update replaces get_opening_hours with get_menu and changes the instructions in the same settings request:
Call await hand_off_to_menu() from your application’s routing code after the current tool round finishes. Pipecat registers the new function handler and sends the settings to Prism. Queueing the frame does not confirm acceptance: use the Rime service’s on_session_updated event to observe acknowledged settings, and handle Pipecat error frames if an update fails. After the update, ask, “What’s on the menu and how much does it cost?” Check that the terminal logs get_menu called and the reply gives the tool’s prices. This checks that the new tool ran, as well as that Prism accepted the settings. You can also send LLMSetToolsFrame(tools=[get_menu]) to change only the tools. Omitted settings keep their current values. Replacing tools removes fillers for tools that are no longer present unless you supply a new tool_fillers map. This handoff leaves the conversation history in place. Use message frames when you also need to edit history. History changes and settings changes are separate operations, and a rejected edit does not roll back Pipecat’s local context. After a partial edit or an unknown outcome, start a new service instance with known valid, completed history.

Troubleshoot the connection

  • The import or rime-realtime extra is missing: check that the project uses the pinned fork commit from the installation command. A published Pipecat version may not include this preview.
  • The browser cannot connect: keep the Python process running and use port 7860. Allow microphone access and check the browser’s selected audio devices.
  • The agent stays silent: check the terminal for an authentication or Prism error. Keep the initial greeting context and the Rime aggregator pair from the example. Errors and limits explains provider errors.
  • The next turn takes longer after skipped output: Prism uses its playback estimate to admit later requests. Muted or discarded audio can leave that estimate ahead of actual playback.

Next steps

Now that your agent answers, you can: