> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Build a Pipecat voice agent with Prism

> Build a Python Pipecat agent with Prism and one function tool, then talk to it from your browser.

In this quickstart, you'll build a Pipecat voice agent with Prism as its model, and talk to it from your browser. Your agent answers questions about a made-up cafe, and runs a function in your code to look up its hours.

Prism replaces a pipeline agent's speech-to-text, LLM, and text-to-speech services: it listens, finds the end of each turn, and replies out loud. Pipecat carries the audio between your browser and the agent, and runs your tools.

## What you'll need

* A Rime API key. If you don't have one, [create one](https://docs.rime.ai/docs/api-authentication.md). The key stays on your Python server.
* [uv](https://docs.astral.sh/uv/getting-started/installation/), which installs Python for the project if needed. This guide uses Python 3.12. The Rime SDK needs Python 3.11 or later.
* A browser with microphone access. The browser connects to your local server through Pipecat's SmallWebRTC transport.

## Build the agent

**1. Create the project**

Create a Python project:

```bash theme={null}
uv init --python 3.12 prism-pipecat-agent
cd prism-pipecat-agent
```

Install the packages, pinned to the versions tested for this guide. Prism support comes from a preview build in Rime's Pipecat fork because the [upstream integration](https://github.com/pipecat-ai/pipecat/pull/6370) has not merged. The `rime-realtime` extra adds the Rime SDK, and `webrtc` and `runner` add the browser transport and local server:

```bash theme={null}
uv add \
  "pipecat-ai[rime-realtime,webrtc,runner] @ git+https://github.com/rimelabs/pipecat.git@4a07f204212b1b2200c90a4fd4873c4f45da55af" \
  "rimelabs-sdk==0.1.0a24" \
  "python-dotenv>=1.1"
```

**2. Set your API key**

Add `.env.local` to the project's `.gitignore` before you save your key:

```gitignore .gitignore theme={null}
.env.local
```

The Python process reads your Rime API key from `.env.local`. Create that file in the project folder:

```dotenv .env.local theme={null}
RIME_API_KEY=YOUR_RIME_API_KEY
```

Keep this file out of version control. The local browser client needs no Rime credentials.

**3. Save the agent**

Save this script as `bot.py`. It opens one Prism conversation for each browser connection and registers `get_opening_hours` as a tool. Only the tool result contains the cafe's hours.

```python bot.py theme={null}
from dotenv import load_dotenv
from loguru import logger

from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.services.llm_service import FunctionCallParams
from pipecat.services.rime.realtime.llm import RimeRealtimeLLMService
from pipecat.transports.base_transport import TransportParams
from pipecat.workers.runner import WorkerRunner

load_dotenv(".env.local")


async def bot(runner_args: RunnerArguments):
    transport = await create_transport(
        runner_args,
        {"webrtc": lambda: TransportParams(
            audio_in_enabled=True, audio_out_enabled=True
        )},
    )

    async def get_opening_hours(params: FunctionCallParams):
        """Look up Juniper Cafe's opening hours for today."""
        logger.info("get_opening_hours called")
        await params.result_callback(
            "Juniper Cafe is open from 8 a.m. to 4 p.m. today."
        )

    llm = RimeRealtimeLLMService(
        settings=RimeRealtimeLLMService.Settings(
            system_instruction=(
                "You answer the phone for Juniper Cafe. Answer in one or two "
                "short sentences. For opening hours, use get_opening_hours."
            ),
            voice="astra",
            tools=[get_opening_hours],
        ),
    )
    context = LLMContext([
        {"role": "system", "content": "Greet the caller and ask how you can help."}
    ])
    user, assistant = llm.create_context_aggregator(context)
    pipeline = Pipeline([
        transport.input(), user, llm, transport.output(), assistant
    ])
    worker = PipelineWorker(
        pipeline, idle_timeout_secs=runner_args.pipeline_idle_timeout_secs
    )
    runner = WorkerRunner(handle_sigint=runner_args.handle_sigint)
    await runner.add_workers(worker)

    @worker.rtvi.event_handler("on_client_ready")
    async def on_client_ready(rtvi):
        await worker.queue_frame(LLMRunFrame())

    @transport.event_handler("on_client_disconnected")
    async def on_disconnected(transport, client):
        await runner.cancel()

    await runner.run()


if __name__ == "__main__":
    from pipecat.runner.run import main

    main()
```

The Rime aggregator pair records speech transcripts without requesting a second reply. Use `llm.create_context_aggregator(context)` for this service. Prism detects turns, so the pipeline has no separate STT, TTS, or local VAD service.

When Prism requests the tool, Pipecat runs the function. `result_callback` sends the result back so Prism can continue its reply. Closing the worker also closes the service's Prism session and SDK client.

**4. Start the agent**

Run the server with WebRTC enabled:

```bash theme={null}
uv run bot.py --transport webrtc --port 7860
```

Keep the process running. Open [the local browser client](http://localhost:7860/client/). Wait for the client status to show **INITIALIZED**, then select **Connect** and allow microphone access.

**5. Talk to the agent**

After the greeting, ask, "What time do you close today?" The terminal should log `get_opening_hours called`, and Prism should answer with 4 p.m. The hours are not in the instructions, so the tool result supplies the answer.

Ask another question, then speak while Prism replies. The agent should stop playback and answer your new turn. Ask one more question to check that the conversation continues.

## How Prism changes a Pipecat agent

Prism runs the whole conversation, so a few Pipecat features work differently than in a pipeline agent:

| Feature | In this agent |
| - | - |
| Audio | SmallWebRTC carries audio between the browser and Pipecat. Prism handles recognition and spoken replies. |
| Context | The Rime aggregator pair maintains Pipecat's local record. Use message frames for application history edits. |
| Tools | Supply functions in `Settings.tools` and return each result through `result_callback`. Automatic tool selection is supported. Forced or disabled tool selection is unsupported. |
| Settings | Change instructions, voice, tools, fillers, truncation, or reply labels with `LLMUpdateSettingsFrame`. See [Update tools and hand off a conversation](#update-tools-and-hand-off-a-conversation). Settings and history edits are separate operations, so a handoff that changes both isn't atomic. |
| Interruptions | Pipecat stops queued output and reports an estimated position for the latest emitted audio. It does not measure device playback or send playback-completion receipts. |
| Recovery | There is no automatic reconnect or replay. After a partial history edit or an unknown outcome, create a new service instance with known valid, completed history. |
| Language and voices | English, with `aurora`, `wawona`, `brando`, or `astra`. See [Choose a voice](https://docs.rime.ai/docs/prism/sessions.md#choose-a-voice). |

## Update tools and hand off a conversation

Use `LLMUpdateSettingsFrame` to change the agent's instructions and available tools during a call. A handoff can keep the same Prism session and conversation history while changing the agent's task.

For example, after answering an opening-hours question, your application can switch the cafe agent to menu help. Add this import to `bot.py`:

```python theme={null}
from pipecat.frames.frames import LLMUpdateSettingsFrame
```

Inside `bot`, after you create `worker`, add a menu tool and a handoff function. The update replaces `get_opening_hours` with `get_menu` and changes the instructions in the same settings request:

```python theme={null}
async def get_menu(params: FunctionCallParams):
    """Look up Juniper Cafe's current menu and prices."""
    logger.info("get_menu called")
    await params.result_callback(
        "Today's menu is a latte for $5 and a blueberry muffin for $3."
    )


async def hand_off_to_menu():
    await worker.queue_frame(
        LLMUpdateSettingsFrame(
            delta=RimeRealtimeLLMService.Settings(
                system_instruction=(
                    "You help callers choose from Juniper Cafe's menu. "
                    "Use get_menu for menu items and prices. "
                    "Answer in one or two short sentences."
                ),
                tools=[get_menu],
            ),
        )
    )
```

Call `await hand_off_to_menu()` from your application's routing code after the current tool round finishes. Pipecat registers the new function handler and sends the settings to Prism. Queueing the frame does not confirm acceptance: use the Rime service's `on_session_updated` event to observe acknowledged settings, and handle Pipecat error frames if an update fails.

After the update, ask, "What's on the menu and how much does it cost?" Check that the terminal logs `get_menu called` and the reply gives the tool's prices. This checks that the new tool ran, as well as that Prism accepted the settings.

You can also send `LLMSetToolsFrame(tools=[get_menu])` to change only the tools. Omitted settings keep their current values. Replacing tools removes fillers for tools that are no longer present unless you supply a new `tool_fillers` map.

This handoff leaves the conversation history in place. Use message frames when you also need to edit history. History changes and settings changes are separate operations, and a rejected edit does not roll back Pipecat's local context. After a partial edit or an unknown outcome, start a new service instance with known valid, completed history.

## Troubleshoot the connection

* **The import or `rime-realtime` extra is missing:** check that the project uses the pinned fork commit from the installation command. A published Pipecat version may not include this preview.
* **The browser cannot connect:** keep the Python process running and use port 7860. Allow microphone access and check the browser's selected audio devices.
* **The agent stays silent:** check the terminal for an authentication or Prism error. Keep the initial greeting context and the Rime aggregator pair from the example. [Errors and limits](https://docs.rime.ai/docs/prism/errors.md) explains provider errors.
* **The next turn takes longer after skipped output:** Prism uses its playback estimate to admit later requests. Muted or discarded audio can leave that estimate ahead of actual playback.

## Next steps

Now that your agent answers, you can:

* [Give Prism tools and fillers](https://docs.rime.ai/docs/prism/tools.md), so it can look up data and speak while a function runs.
* [Write instructions for a phone call](https://docs.rime.ai/docs/prism/sessions.md#write-instructions-for-a-phone-call), starting with rules for speech.
* [Choose a voice](https://docs.rime.ai/docs/prism/sessions.md#choose-a-voice) that suits your callers.
* [Move an OpenAI Realtime agent](https://docs.rime.ai/docs/prism/migrate-from-openai.md#move-a-pipecat-agent) on Pipecat to Prism.
* [Build a pipeline agent](https://docs.rime.ai/docs/pipecat.md) with your choice of speech-to-text and LLM, where Rime gives the agent its voice.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.