Skip to main content
Tools let Prism reach into your systems during a call, to look something up, such as Saturday’s opening hours, or to do something, such as booking a table. Prism decides when to call a tool, but it never runs your code, and it never picks the conversation back up by itself. Your backend runs the tool, submits the result, and then asks Prism to continue. The calls from one reply, their results, and that continuation make one tool round. Forget the continuation, and the caller is left in silence. The snippets build on Manage a session, which defines stream_caller_audio and, in Node.js, background. get_opening_hours (getOpeningHours in Node.js) stands for your own lookup. The rime-sdk repository has complete programs:

Define tools when the session starts

List your tools when you connect. To change them during a call, send an update, as Configure a session shows. Each tool has a name, a description that tells Prism what the tool does, and parameters, a JSON Schema object for its arguments. A tool can also set its phrase.
Prism decides when to call a tool. Your application cannot force a call, but it can stop calls to a tool by removing it with an update. If the instructions only name a tool, Prism can answer from its own knowledge. So give the instructions a direct rule for each tool, such as “Call get_opening_hours before you answer a question about opening hours.” Write your first prompt in the prompting guide has more.

Run each call and continue the reply

When Prism needs a tool, the reply sends a tool.call event for each call, and then the reply ends. The SDK parses the arguments from JSON. In the round in the diagram, Prism calls get_opening_hours with {"day": "Saturday"}, and the lookup returns 9 a.m. to 2 p.m. Your backend then follows these steps:
  1. Run each call.
  2. Submit the result of each call.
  3. Wait until the reply that made the calls ends and every result is in.
  4. Continue that reply.
Prism then uses the results in a new reply, whose response.started event has cause "tools".
run_tools (runTools in Node.js) handles only tool events. In a live call, add its two branches to play_replies, the events loop in Live conversations that plays replies. Run each tool on its own task, and do not await a tool inside the events loop. While the loop waits, it reads no events, and the SDK ends the session when 256 events wait unread. When the round cannot continue, for example because the caller spoke first or a wait ran out, continue_reply (continueReply) raises an error. finish_round drops the round, and the session stays open. Tool calls in the SDK reference lists each error. Handle three more things in your tool code:
  • Results: submit_tool_result (submitToolResult) records a result, but it does not start a reply. The result is a string, so encode a structured result as JSON. Each result stays in the conversation and takes up room in the context of later replies, so return only what Prism needs.
  • Arguments: check the arguments before you use them. The model writes the arguments, and the schema does not guarantee them.
  • Unhandled exceptions: an exception that escapes run_tool or finish_round ends the call. In Python, it stops run_tools, and the session closes. In Node.js, background closes the session.

What the caller hears while a tool runs

What the caller hears depends on the kind of turn: The reply text holds the phrase and any other words next to the calls, such as “Let me try that again.” So captions from text.delta events and logs from text.done show the phrase too. A reply to tool results that only makes tool calls has empty text. When the lookup is slow, the caller hears silence after the phrase until you continue, unless you speak while it runs.

From 0:11 to 0:35 of the Prism demo call. Prism says "One moment." while it looks up the card, and then reads the card that the lookup found. Then it asks before it freezes the card.

Set the phrase for a tool

A tool’s prism_filler setting decides the phrase that the caller hears while the tool runs, on every turn except a reply to tool results. The setting needs rimelabs-sdk 0.1.0a20 or @rimelabs/sdk 0.1.0-alpha.20. When a reply calls several tools, the first call’s tool decides the phrase.
For no phrase, set PrismFiller(type="none") in Python or { type: "none" } in Node.js. The SDK refuses a blank line or a line over 500 characters with RimeInputError.

When a tool fails

Return the failure as the tool result, and continue the reply as usual. Prism then tells the caller. For example, when get_opening_hours raises an exception, run_tool returns {"error": "get_opening_hours failed"}. Prism says that it cannot get Saturday’s opening hours, and suggests that the caller try again later or contact the cafe. Always submit a result, even for a failed call. Without a result, the caller hears nothing after the phrase until the result wait runs out. Also make each tool safe to run once, because Prism does not deduplicate requests. A call with an unknown outcome, such as a timed-out lookup, may still complete, so never run it again automatically. Tool calls in Errors and limits lists the error codes of a tool round and what to do about each one.

Tools in LiveKit Agents

In a LiveKit voice agent with Prism, each @function_tool method on your Agent becomes a Prism tool. LiveKit runs the round for you.

Call the API directly

For the WebSocket API, Tools in the WebSocket reference maps each step of a tool round to its event. Its complete example runs one round.

Next

Long-running tasks

Speak while a slow tool runs, and hand long tasks to your own agent.

Typed turns and nudges

Make Prism speak first, or report a late tool result with a nudge.