client.realtime.connect opens one Prism conversation and returns a session. With the session, your application streams the caller’s audio and reads the session events on a separate task. It also plays the reply audio, reports how much of each reply the caller heard, and runs tools when Prism asks. The conversation runner can do the audio part of that work for you.
Prism is in the SDK for each of the four languages, from these releases:
Some features require newer SDK releases. Request IDs, reply labels,
input_item_ids, retryable, and update need Go v0.1.0-alpha.12 or Rust 0.1.0-alpha.10. History truncation, the methods that read and edit the history, and the ErrRealtime and ErrorKind::Realtime kinds need Go v0.1.0-alpha.13 or Rust 0.1.0-alpha.11. The resource-limit error for a 429 refusal, and the request ID of a refused connection, need Go v0.1.0-alpha.14 or Rust 0.1.0-alpha.12. In Go and Rust, the first Prism release has every other feature on this page. In Python and Node.js, releases before 0.1.0a14 and 0.1.0-alpha.15 use draft event names and cannot connect, and these features need a later release:
Most tables on this page give the Python and Node.js names. Go and Rust gives the names in those languages. Install the SDK from the SDK overview. For walkthroughs and complete programs, see the SDK guides and examples.
Open a session
Each example below opens a session with a voice and instructions. It starts your capture loop and reads every event until the session closes.client: an open client.stream_microphoneandstreamMicrophone: your capture loop, which calls the audio method in Send audio.handle: your event handling, which must return quickly.ctx: in Go, the context that owns the session. When it ends, the session closes.
Run a conversation
run_conversation in Python and runConversation in Node.js run a whole conversation with your audio adapters. The runner sends the caller’s audio, plays each reply in order, and handles interruptions. It also reports how much of each reply the caller heard. When the conversation ends, it stops playback and closes the session. The SDK quickstart uses it with a page in the browser.
ConversationRunner fields, and the context of Run ends the run. In Rust, ConversationRunner::new takes the session and the two adapters, on_event sets the callback, and stop is a CancellationToken that ends the run.
Your adapters implement these methods. In Node.js,
read, stop, and completion also take an AbortSignal. In Go, they take a context.Context, and in Rust, a CancellationToken. Go and Rust use the same method names, with a capital first letter in Go:
Four rules for the runner and its adapters:
- Play replies one at a time, in the order that the runner opens them. Prism assumes that an interrupted reply leaves later replies in the same turn unheard.
- Measure playback from the audio output clock. Audio that you sent or queued has not played yet.
- During a run, the runner is the only reader of the session events and the only source of playback reports. Tool results and typed messages still use the session methods, such as
submit_tool_result(submitToolResult). - The runner does not open audio devices, run your tools, or close your adapters.
Guides and examples
The Rime SDK repository keeps the Prism guide and runnable examples for each language next to the SDK source. Every example connects towss://prism.api.rime.ai/v1/realtime by default. To connect to another Prism deployment, set PRISM_URL for the Python, Node.js, and Go examples. The Rust examples always use the default endpoint.
To talk to Prism through your microphone with your own instructions, follow the Prism SDK quickstart. Live conversations, Typed turns and nudges, and Tools show these methods in short examples.
Connection and credentials
Connect options
In both languages, connect returns after Prism accepts the session settings, and the session is then ready for audio. Connect takes these options:
Inside connect, the SDK sends one
session.update with text and audio output, PCM16 input, your interruption and turn timing settings, and your tools. It also sends the voice and instructions when you set them. If Prism refuses the settings, connect raises RimeRealtimeError, and the SDK closes the connection.
During a conversation, use update to change instructions, tools, voice, interruption, or history truncation. Use set_response_metadata (setResponseMetadata) to change reply labels. Eagerness and the end-of-turn timeout remain fixed. Session configuration lists the fields Prism accepts and which settings it can change.
Tool definitions
In Python,ToolDefinition is a frozen dataclass from rimelabs_sdk.realtime with keyword-only fields. In Node.js, it is a plain object, and the ToolDefinition interface from @rimelabs/sdk gives its type.
A tool list that does not meet these conditions raises
RimeInputError before the SDK connects. This definition requires a day argument for an opening-hours lookup:
Session info
session.info holds the settings that Prism applied. It is available as soon as connect returns.
Session methods
Each method is async, and it returns after Prism acknowledges the request, unless the table says something different. In Node.js, every method exceptclose also takes an optional signal in its last argument. The signal is an AbortSignal that cancels the method. requestReply takes it in the same options object as instruction and toolCall, and createItem in the same object as previousItemId.
The session also has two properties:
session.events and session.info.
Go and Rust
The Go and Rust SDKs follow the same session model as Python and Node.js, with the same methods, events, references, and runner. They also enforce the same limits on audio, text, queued events, and waiting operations. The names and the cancellation model are different.- Go
- Rust
Import the module as
rime "github.com/rimelabs/rime-sdk/go". Call client.Realtime.Connect(ctx, rime.RealtimeConnectOptions{...}). It returns (*rime.RealtimeSession, error) after Prism accepts the session settings. The context that you pass to Connect owns the session, so the session closes when that context ends. Every other method takes its own context, which cancels only that call.rime.PrismFiller sets the phrase with Type, Text, and Instructions. Type is "auto", "text", or "none", as Set the phrase for a tool describes.Error names in Go and Rust
Go and Rust use error kinds in place of the error classes on this page:
A Prism refusal has the realtime kind, and the session stays usable, as after
RimeRealtimeError.
Update settings during a conversation
update changes the instructions, tools, voice, interruption setting, or history truncation without a new connection. It returns the new SessionInfo after Prism confirms the change.
Send audio
Send the caller’s audio all the time, from connect until you close the session, including silence and caller audio while Prism speaks. Prism finds interruptions and the end of each turn in this audio, and it sends no acknowledgment for audio. Stream audio for the whole call explains what happens when the audio stops and when a conversation is idle. In Python,AudioChunk and PCMFormat are frozen dataclasses from rimelabs_sdk.realtime. In Node.js, a chunk is a plain object.
Send a frame captured as 48 kHz stereo PCM16:
Typed turns, nudges, and history
These methods make Prism reply without new speech, or add history without a reply. Typed turns and nudges shows them in a conversation.Read and edit the history
get_history, create_item, and delete_item read and change the stored conversation. Truncation may omit older turns from the next prompt. They never start a reply or run a tool. Use them to add a note for the model during a call, to remove an item that the agent should no longer use, or to start a new session from an earlier call.
This example adds a note for the model, reads the history, and deletes the note:
items holds the note, with the ID that Prism gave it. After the delete, get_history no longer lists the note, and the next reply no longer sees it.
History items
create_item takes one of three item types, and get_history returns the same types. In Python, they are frozen dataclasses from rimelabs_sdk.realtime with keyword-only fields. In Node.js, an item is a plain object whose type gives its kind, and the HistoryItem type from @rimelabs/sdk describes it. Node.js uses the same fields in camelCase, such as callId.
id is optional when you create an item. It is your own ID for the item, from 1 to 32 characters. Without it, Prism gives the item an ID.
Add a past tool call first, and then its result. Prism shows the call to the model, and get_history lists it, only after the result arrives. These calls add a lookup of Monday’s hours from an earlier call:
submit_tool_result.
Start from an earlier call
A new session has no memory of an earlier session, and the SDK does not reconnect. To go on with a call in a new session, read the history withget_history while the call runs, and keep the items. Then add them to the new session in order, before you stream the caller’s audio:
What to know about the history
Go and Rust have the same methods and item types, with these names:
Tool calls
When Prism needs your backend to run a tool, the reply sends atool.call event for each call and then ends. Run each tool on a separate task and submit its result so the events loop can keep reading. Call continue_reply for the parent reply after it ends. The method waits for every result before requesting the continuation. In this code, run_tool and runTool stand for your code, and payload is the payload of the event.
The continuation is a new reply. Its
response.started event has cause "tools", with the parent reply in parent. continue_reply raises these errors:
The SDK checks only what it has seen, so Prism can still refuse a continuation that passes the SDK checks. A typed turn or a nudge also ends a waiting tool round, as the caller’s speech does. Timeouts and late results explains the two Prism waits in
session.info, and Errors and limits explains each tool error code.
Playback reports
A playback report tells Prism what the caller actually heard, by the clock of your player. Prism uses the reports to decide what it remembers of an interrupted reply and whether its next reply acknowledges the interruption. It also uses them to know when a finished reply stops playing. Playback reports are optional. Without them, Prism estimates playback from the start of playback, by its own clock, as Report when a reply finishes playing explains.
When the caller interrupts, stop playback, drop the reply audio that your player has queued, and then report
PlaybackInterrupted. Stop playback when the caller talks over a reply shows this order in code.
Stop a reply or clear input
A reply can finish before the cancel gets to Prism, so read the
status of its response.ended event instead of assuming "cancelled". A cancel is also not an interruption, so audio that Prism already sent plays until your player stops it.
Events
session.events is an async iterator of SessionEvent values. Read it on a separate task for the whole session, and read the events as fast as they arrive. Only one reader can use it at a time, and a second reader raises RimeInputError. The SDK queues at most 256 unread events, and a full queue ends the session with RimeResourceLimitError. The session stays open when your loop ends.
Branch on
payload.kind in either language. In Python, each kind is also a frozen dataclass from rimelabs_sdk.realtime, which you can match with isinstance. In Node.js, payload is a union of interfaces from @rimelabs/sdk, and a switch on kind narrows it.
Event types
The table gives the Python names of the fields. Node.js uses the same fields in camelCase, such asitemId, audioStartMs, and inputItemId. Go and Rust gives the names in those languages. An optional field without a value is None in Python and null in Node.js.
The
cause of response.started is one of these values:
The
end_reason of speech.stopped is one of these values. It is None or null when Prism does not publish it.
The
status of response.ended is "completed", "cancelled", "failed", or "unknown". reason is the reason from Prism, such as "stop", "turn_detected" when the caller interrupted, or "client_cancelled". The response.done reference lists every status and reason. For a failed reply, the error event comes just before its response.ended.
Event order
The Python and Node.js SDKs deliver events in the order Prism sends them. The diagram shows the events of one spoken turn and its reply, and the table after it gives the full order for each part.References
References identify the objects that you act on. The SDK creates them, and you pass them back without changes.
A reference belongs to the session that created it, and using it in a different session raises
RimeInputError. The SDK also keeps only the last 128 replies, so a reference to an older reply raises RimeInputError too. content_index is always 0.
Faults
Theerror field of an error event is a RealtimeFault. The fault of a RimeRealtimeError is also a RealtimeFault:
A refused request raises
RimeRealtimeError from the method that sent it, and the same fault also arrives as an error event. An error with scope "session" does not arrive as an event. Instead, it ends the session, and the events iterator raises it as RimeRealtimeError.
Events the SDK doesn’t deliver
The Python and Node.js SDKs skip the Prism events that they do not model, so these events never get tosession.events:
Timeouts and limits
RealtimeTimeouts sets the three deadlines of the SDK. In Python, it is a frozen dataclass from rimelabs_sdk.realtime with keyword-only fields. In Node.js, it is a plain object, and all its fields are optional. Each value is a positive, finite number of seconds, and any other value raises RimeInputError.
No SDK deadline limits a whole conversation, or a reply after it starts. Set a deadline that fits your application, and close the session when it passes. Errors and limits gives the Prism limits, such as the 55-second turn and the idle close. The Python and Node.js SDKs also enforce these limits:
Cancel and clean up
Closing a session ends its conversation. Closing the client closes all its sessions. The SDK stops only its own work, so stop your capture loop and your player yourself, and drop the queued audio.
In Go, cancel the context of a call to cancel it, and
client.Close() closes every session of the client. In Rust, drop the future of a call to cancel it, and client.close().await closes every session of the client.
You can safely repeat both close methods. After a close, the events iterator delivers the events that are already queued, and then it ends normally. Other methods raise RimeInputError. In Go and Rust, they return rime.ErrCancelled or ErrorKind::Cancelled instead. In Python, a client and its sessions belong to the process and event loop that created them. Any other process or event loop that uses them also gets RimeInputError.
When you cancel one operation, the session continues to run, with one exception for send_audio. Cancellation stops the wait, but it does not take back what the SDK already sent:
Errors
Every SDK failure is one of the error classes below, and all of them inherit fromRimeError. Go and Rust use error kinds, which Error names in Go and Rust maps to these classes. RealtimeAdmissionTimeout also inherits from RimeTimeoutError. Branch on the class, and for RimeRealtimeError, also on fault.code. Never branch on the message text.
Connection refusals explains each HTTP status that Prism can return before the WebSocket opens. The error message of a refused connection gives the status and Prism’s reason, when Prism sends one, and the SDK doesn’t retry the connection. Log the fault code and correlation ID when Prism refuses a typed turn:

