Review replies before Prism speaks them

Prism writes every reply from your instructions, and instructions usually keep it on track. Some lines must never reach the caller, though, such as a refund promise, medical advice, or wording that your legal team hasn’t approved. For those, turn on reply review. Prism then sends each part of a reply to your backend before it speaks it, and speaks only what you let through. Nothing plays before your answer, and the conversation history holds exactly what the caller heard. Review works over the WebSocket API. Your check can be anything that answers quickly: a keyword list, a moderation API, or a fast model.

How review works

Prism splits each spoken reply into chunks of one sentence, or a few short ones. It sends each chunk in a prism.review.requested as soon as the model writes it. Prism doesn’t wait for your answer to one chunk before it sends the next, so you can check several chunks at once. Prism speaks the chunks in order, and each chunk waits until it and every chunk before it have an answer. Answer each chunk with a prism.review.result and one of three actions:

Action	What the caller hears	What happens next
`allow`	The chunk	The reply goes on.
`replace`	Your `text`, as you wrote it	The reply ends after your text.
`stop`	Nothing more of the reply	The reply ends with reason `review_stopped`, after the chunks that you allowed have played.

Turn on review

Set prism_review in your first session.update. The first update fixes it, so a later update can’t turn review on or change its settings.

{
  "type": "session.update",
  "session": {
    "voice": "aurora",
    "instructions": "You answer the phone for Juniper Cafe.",
    "prism_review": { "on_timeout": "say", "timeout_ms": 1000, "timeout_line": "Sorry, one moment." }
  }
}

Field	Default	What it does
`on_timeout`	Required	What happens to a chunk with no answer in time: `stop`, `speak`, or `say`. Choose what happens on a timeout compares them.
`timeout_ms`	`1000`	How long Prism waits for each answer, from 100 to 5000 milliseconds
`timeout_line`	None	With `say`, and only then: the line that Prism speaks in place of a chunk with no answer, up to 500 characters

session.updated shows the settings in effect, with timeout_ms filled in.

Answer each chunk

This script turns on review and asks Juniper Cafe two things: when it closes, and whether the caller can get a refund. Staff at the counter handle refunds, so the check replaces any chunk about refunds with the cafe’s approved line, and lets everything else through. To try it without a microphone, the script sends the question as a typed turn. In your agent, the review handler goes into the event loop that already plays Prism’s audio. The script starts a separate check for each chunk, and its event loop doesn’t wait for the check. If a check fails, the script prints the error and sends no answer. Prism then handles that chunk as on_timeout says. The script stops when the reply ends, after 60 seconds, or on an error. A review error, such as review_timeout, affects only one chunk, so the script prints it and keeps going.

Python
Node.js

review.py

import asyncio
import json
import os

import websockets

URL = "wss://prism.api.rime.ai/v1/realtime"
INSTRUCTIONS = (
    "You answer the phone for Juniper Cafe. It's open from 8 a.m. to 4 p.m. "
    "every day. Staff at the counter handle refunds. Answer warmly, in full sentences."
)
QUESTION = (
    "What time do you close today, and do you have oat milk? "
    "Also, my latte was cold this morning. Can I get a refund?"
)
REFUND_LINE = "I'm sorry about that. Our staff can help with refunds at the counter."


async def check(text):
    # Your own check goes here: a keyword list, a moderation API, or a fast model.
    # Use an async client for network calls, so that a check doesn't block the loop.
    if "refund" in text.lower():
        return {"action": "replace", "text": REFUND_LINE}
    return {"action": "allow"}


async def review(socket, event):
    try:
        answer = await check(event["text"])
    except Exception as error:
        # Prism handles a chunk with no answer as on_timeout says.
        print(f"Chunk {event['index']}: check failed: {error!r}")
        return
    print(f"Chunk {event['index']}: {event['text']!r} -> {answer['action']}")
    await socket.send(json.dumps({
        "type": "prism.review.result",
        "review_id": event["review_id"],
        **answer,
    }))


async def read_reply(socket):
    reviews = set()
    try:
        async for message in socket:
            event = json.loads(message)
            kind = event["type"]
            if kind == "session.updated":
                # A typed turn stands in for the caller.
                await socket.send(json.dumps({
                    "type": "response.create",
                    "response": {
                        "prism_input_text": QUESTION,
                        "metadata": {"prism_cause": "user_text"},
                    },
                }))
            elif kind == "prism.review.requested":
                # Check each chunk in its own task, so that the loop keeps reading
                # events and a slow check doesn't hold up the chunks after it.
                task = asyncio.create_task(review(socket, event))
                reviews.add(task)
                task.add_done_callback(reviews.discard)
            elif kind == "response.text.done":
                print(f"Prism said: {event['text']}")
            elif kind == "error":
                error = event["error"]
                # A review error, such as review_timeout, affects only one chunk.
                # Any other error stops the script.
                if error["scope"] != "review":
                    raise RuntimeError(f"{error['code']}: {error['message']}")
                print(f"Error {error['code']}: {error['message']}")
            elif kind == "response.done":
                print(f"Reply ended: {event['response']['status_details']['reason']}")
                return
    finally:
        # Stop any check that is still running. Prism ignores answers after the reply ends.
        for task in reviews:
            task.cancel()
    raise ConnectionError("The session closed before the reply ended")


async def main():
    headers = {"Authorization": f"Bearer {os.environ['RIME_API_KEY']}"}
    try:
        async with (
            asyncio.timeout(60) as deadline,
            websockets.connect(URL, additional_headers=headers) as socket,
        ):
            await socket.send(json.dumps({
                "type": "session.update",
                "session": {
                    "voice": "aurora",
                    "instructions": INSTRUCTIONS,
                    "prism_review": {"on_timeout": "say", "timeout_line": "Sorry, one moment."},
                },
            }))
            await read_reply(socket)
    except TimeoutError:
        if deadline.expired():
            raise TimeoutError("No reply within 60 seconds") from None
        raise


asyncio.run(main())

Set RIME_API_KEY as in the WebSocket quickstart, and run the script:

uv run --python 3.12 --with "websockets==17.2" review.py

review.mjs

import { on } from "node:events";
import WebSocket from "ws";

const URL = "wss://prism.api.rime.ai/v1/realtime";
const INSTRUCTIONS =
  "You answer the phone for Juniper Cafe. It's open from 8 a.m. to 4 p.m. " +
  "every day. Staff at the counter handle refunds. Answer warmly, in full sentences.";
const QUESTION =
  "What time do you close today, and do you have oat milk? " +
  "Also, my latte was cold this morning. Can I get a refund?";
const REFUND_LINE = "I'm sorry about that. Our staff can help with refunds at the counter.";

async function check(text) {
  // Your own check goes here: a keyword list, a moderation API, or a fast model.
  if (text.toLowerCase().includes("refund")) {
    return { action: "replace", text: REFUND_LINE };
  }
  return { action: "allow" };
}

async function review(socket, event) {
  const answer = await check(event.text);
  console.log(`Chunk ${event.index}: ${JSON.stringify(event.text)} -> ${answer.action}`);
  socket.send(
    JSON.stringify({ type: "prism.review.result", review_id: event.review_id, ...answer }),
  );
}

async function readReply(socket) {
  for await (const [message] of on(socket, "message", { close: ["close"] })) {
    const event = JSON.parse(message);
    if (event.type === "session.updated") {
      // A typed turn stands in for the caller.
      socket.send(
        JSON.stringify({
          type: "response.create",
          response: { prism_input_text: QUESTION, metadata: { prism_cause: "user_text" } },
        }),
      );
    } else if (event.type === "prism.review.requested") {
      // Don't await the check, so that the loop keeps reading events
      // and a slow check doesn't hold up the chunks after it.
      review(socket, event).catch((error) => {
        // Prism handles a chunk with no answer as on_timeout says.
        console.log(`Chunk ${event.index}: check failed: ${error}`);
      });
    } else if (event.type === "response.text.done") {
      console.log(`Prism said: ${event.text}`);
    } else if (event.type === "error") {
      const { error } = event;
      // A review error, such as review_timeout, affects only one chunk.
      // Any other error stops the script.
      if (error.scope !== "review") throw new Error(`${error.code}: ${error.message}`);
      console.log(`Error ${error.code}: ${error.message}`);
    } else if (event.type === "response.done") {
      console.log(`Reply ended: ${event.response.status_details.reason}`);
      return;
    }
  }
  throw new Error("The session closed before the reply ended");
}

const socket = new WebSocket(URL, {
  headers: { Authorization: `Bearer ${process.env.RIME_API_KEY}` },
});
socket.on("open", () => {
  socket.send(
    JSON.stringify({
      type: "session.update",
      session: {
        voice: "aurora",
        instructions: INSTRUCTIONS,
        prism_review: { on_timeout: "say", timeout_line: "Sorry, one moment." },
      },
    }),
  );
});
let timedOut = false;
const deadline = setTimeout(() => {
  timedOut = true;
  socket.terminate();
}, 60_000);
try {
  await readReply(socket);
} catch (error) {
  throw timedOut ? new Error("No reply within 60 seconds") : error;
} finally {
  clearTimeout(deadline);
  socket.close();
}

Set RIME_API_KEY as in the WebSocket quickstart, install ws, and run the script:

npm install --save-exact ws@8.22.0
node review.mjs

You should see Prism allow the first chunk and replace the second. The model’s wording changes from run to run:

Chunk 0: 'We close at 4 p.m. today, and yes, we do have oat milk.' -> allow
Chunk 1: 'I am so sorry to hear your latte was cold; our staff at the counter can certainly help you with a refund.' -> replace
Prism said: We close at 4 p.m. today, and yes, we do have oat milk. I'm sorry about that. Our staff can help with refunds at the counter.
Reply ended: stop

When you write your own check, keep these points in mind:

Answer fast: Prism doesn’t speak until the first chunk has an answer, so the time your check takes adds to the caller’s wait. With an instant answer, review adds about one network round trip between your backend and Prism before the first audio.
Run checks at the same time: Prism sends each chunk without waiting for your answer to the last one. A loop that waits for each check reads no events while it waits. Each chunk then waits for every check ahead of it, and so does the audio that the loop plays. With a 400 ms check, the fifth chunk can wait about 1.5 seconds and time out. Start each check on its own, as the script does.
Replace the whole chunk: a chunk can hold a few short sentences, and your text takes the place of all of them. Write a replacement that stands on its own.
Write replacements for the ear: Prism speaks your text as written. It reads numbers and symbols out as in any reply, and it doesn’t remove markdown, so write plain sentences.
Answer each chunk once: Prism refuses a second answer, or a review_id that it never sent, with review_not_found. Send text only with replace. Prism refuses a mismatch with invalid_client_event, and the chunk still waits for an answer.
Expect a late chunk now and then: a chunk that Prism sent before your replace or stop reached it can still arrive. Prism ignores your answer to it, and you don’t need to track which chunks still count.

Choose what happens on a timeout

If your answer doesn’t arrive within timeout_ms, Prism sends an error with code review_timeout and scope review, and on_timeout decides the chunk. Prism ignores an answer that comes later.

`on_timeout`	What the caller hears	Use it when
`say`	Your `timeout_line` in place of the chunk, and then the reply ends	An unchecked line must never play, and the caller should still hear something
`stop`	Nothing more of the reply, which ends with reason `review_timeout`	An unchecked line must never play, and silence is acceptable
`speak`	The chunk as the model wrote it	Review is a safety net, and a slow check shouldn’t hold up the call

Set timeout_ms a little above your check’s slowest normal answer. A long timeout keeps the caller waiting when your check stalls, and a short one times out answers that were on their way.

What the caller hears and what Prism remembers

Everything that Prism reports about a reviewed reply matches what the caller heard:

Text: response.text.delta and response.text.done hold only what the review let through. That’s the chunks that you allowed or that Prism spoke on a timeout, and your replacement text. Captions built from them match the audio.
The end of the reply: after allow and replace, the reply completes with reason stop. After stop, it’s cancelled with reason review_stopped, and after a timeout with on_timeout stop, with reason review_timeout.
History: the model sees only what the caller heard. After a replacement, the next reply knows that Prism said your line, not the model’s. prism.conversation lists it.

A review request doesn’t mean that the caller heard the chunk, so use response.done and the history to know what was said. When a reply starts on an early guess that the caller has finished, its chunks can come before its response.created. If the caller keeps talking, that reply never gets a response.created, and Prism drops its reviews. Answer each chunk by its review_id, without waiting for response.created.

Tool calls aren’t reviewed

Review covers what Prism says, not the tools that it calls. Prism sends a call as soon as the model writes it, even while the line before the call waits for your answer. So stopping that line usually doesn’t stop the call. A stop drops a call only when Prism hasn’t spoken any of the reply and the model hasn’t finished writing the call. Once Prism has spoken part of the reply, it also sends the calls that the model writes after the stop. Don’t rely on review to block a call. Check each call’s arguments in your backend before you run it. The line that Prism speaks before a tool call goes through review, and so does the reply that continues with the result. Tools covers the whole tool round.

Tools

Run each tool call in your backend, and continue the reply with the result.

WebSocket reference

Every field of the review events, settings, and errors.

Get started

Build with Rime

Integrations

Account & updates