Events & Callbacks
Patter fires async callbacks at key moments in the call lifecycle. Use them to log calls, update CRMs, trigger workflows, or control conversation flow. All callbacks are async functions. They are passed as parameters toserve().
Available Callbacks
| Callback | Trigger |
|---|---|
on_call_start | A call connects |
on_call_end | A call ends |
on_transcript | Each utterance is transcribed |
on_message | User message received (pipeline mode) |
on_metrics | After each conversation turn (real-time cost/latency) |
on_call_start
Fires when a call connects. Use it to log call starts, initialize state, or fetch customer data.Event Fields
| Field | Type | Description |
|---|---|---|
call_id | str | Unique identifier for this call. |
caller | str | The caller’s phone number (E.164). |
callee | str | The callee’s phone number (E.164). |
direction | str | "inbound" or "outbound". |
custom_params | dict | Custom parameters passed with the call (if any). |
on_call_end
Fires when a call ends. Use it to save transcripts, calculate duration, or trigger post-call workflows.Event Fields
| Field | Type | Description |
|---|---|---|
call_id | str | Unique identifier for this call. |
caller | str | The caller’s phone number (E.164). |
callee | str | The callee’s phone number (E.164). |
ended_at | float | Unix timestamp when the call ended (e.g. 1710489601.234). |
transcript | list[dict] | Full conversation transcript. Each entry has role ("user" or "assistant") and text. |
metrics | CallMetrics | None | Call metrics with cost and latency breakdowns. None if metrics collection failed. See Metrics & Cost Tracking. |
recording_path | str | None | Path to the local stereo WAV. Only present when local_recording was enabled on serve(); None if finalizing the file failed. See Local Recording. |
on_transcript
Fires each time an utterance is transcribed during the call. Use it for real-time logging, sentiment analysis, or live dashboards.Event Fields
| Field | Type | Description |
|---|---|---|
role | str | "user" or "assistant". |
text | str | The transcribed text. |
call_id | str | Unique identifier for this call. |
history | list[dict] | Conversation history so far. Each entry has role, text, and timestamp. |
on_message
Fires when a user message is received in pipeline mode. Your callback processes the message and returns the agent’s response as a string, which is then synthesized to speech.on_message is only used in pipeline mode (when you pass stt= / tts= instead of engine=). In engine mode (OpenAI Realtime, ElevenLabs ConvAI) the engine handles responses directly.Event Fields
| Field | Type | Description |
|---|---|---|
text | str | The user’s transcribed message. |
call_id | str | Unique identifier for this call. |
caller | str | The caller’s phone number. |
callee | str | The callee’s phone number. |
history | list[dict] | Conversation history. Each entry has role, text, and timestamp. |
Return Value
Return astr with the agent’s response. This text is sent to the TTS provider and played back to the caller.
Speech-edge events
For turn-taking, TTFT measurement, and barge-in / interrupt observability, set the speech-edge callbacks directly on thePatter instance. They proxy to a per-process SpeechEvents dispatcher and fire from any in-flight call.
| Attribute | Fires |
|---|---|
on_user_speech_started | Raw VAD positive edge (caller begins speaking). |
on_user_speech_ended | Raw VAD trailing edge (caller stops speaking). |
on_user_speech_eos | Committed end-of-utterance — anchor TTFT here. |
on_agent_speech_started | First wire-time agent audio chunk — turn-start marker for the caller. |
on_agent_speech_ended | Last agent audio chunk. Payload includes interrupted flag for barge-in. |
on_llm_token | First LLM token of the turn — TTFT marker. |
on_audio_out | First TTS audio bytes produced — TTS warmup signal. |
Tool events via on_transcript
Tool invocations (including the built-in transfer_call and end_call) surface through the same on_transcript callback you pass to phone.serve(...). Filter on role == "tool" to handle them:
| Key | Type | Notes |
|---|---|---|
role | "tool" | Always "tool" for tool events. |
tool_name | str | The tool that was dispatched. |
tool_args | dict | Arguments emitted by the LLM. |
tool_result | str | None | Result returned by the tool handler (truncated for log readability). |
call_id | str | The active call ID. |
text | str | Pre-formatted “tool_name(args) → result” string. |
Pipeline Hooks
PipelineHooks lets you intercept data at each stage of the pipeline mode STT → LLM → TTS pipeline. Pass an instance via phone.agent(hooks=...). Hooks may be sync or async; if a hook throws, the error is logged and the original value passes through unchanged (fail-open).
after_llm — 3-tier API
after_llm accepts either a dict with on_chunk / on_sentence / on_response keys, or any object exposing those attributes (dataclass, custom class, Protocol implementation).
| Tier | Sync/Async | Latency budget | When it runs | Return semantics |
|---|---|---|---|---|
on_chunk(chunk: str) -> str | sync | ~0 ms | Per LLM token chunk, before sentence aggregation | Return new string. Use for cheap text rewrites. |
on_sentence(sentence: str, ctx: HookContext) -> str | None | async | 50–300 ms | Per complete sentence, between chunker and TTS | Return new sentence, None to keep original, or "" to drop the sentence. |
on_response(text: str, ctx: HookContext) -> str | None | async | 500 ms–2 s | Once at end of LLM stream, blocks streaming TTS | Return new text, or None to keep original. |
on_chunk for fast string ops, on_sentence for per-sentence I/O (PII redaction, translation), on_response only when you need the whole response (JSON-schema validation, full-context moderation).
Migration: legacy after_llm callable
The legacy single-callable form is still supported for backward compatibility but is deprecated:
on_response slot and emits a one-shot PatterDeprecationWarning on first use. Migrate to the 3-tier dict to silence the warning and unlock the lower-latency on_chunk / on_sentence tiers.
HookContext
Hooks that take a ctx argument receive a frozen HookContext dataclass:
PipelineHooks also exposes before_stt / after_stt and before_tts / after_tts for audio-stage interception. See the API Reference for the full signature.
Conversation History
All callbacks that includehistory receive it as a list of dictionaries:
time.time()), not ISO-8601 strings.
Complete Example
Speech-Edge Events (Turn-Taking)
The callbacks above describe the transcript-level lifecycle of a call. For turn-taking instrumentation — barge-in, end-of-utterance, time-to-first-token, TTS warmup vs. wire-time — Patter exposes seven additional async callbacks plus a read-onlyconversation_state snapshot directly on the Patter instance.
These events expose the canonical voice-agent metric set (user/agent state transitions, turn boundaries, TTFT, audio first-byte) and align with OpenAI Realtime (input_audio_buffer.speech_started/_stopped/_committed) so downstream metrics work without translation.
Every callback defaults to
None. Existing code that does not register any speech-edge callback sees exactly the previous behaviour and zero overhead. The state machine is updated regardless of whether callbacks are registered, so conversation_state is always usable.The seven events
| Event | Fires on | Signal |
|---|---|---|
on_user_speech_started | VAD positive edge of inbound audio | Raw VAD start — not end-of-utterance. Use for cross-talk detection. |
on_user_speech_ended | VAD trailing edge | Raw VAD stop — not committed EOU. Use for talk-ratio. |
on_user_speech_eos | Committed end-of-utterance | Canonical “user finished” signal. Anchor eos_to_first_token_ms here. |
on_agent_speech_started | First wire-time chunk of the agent turn | What the user actually hears (distinct from TTS warmup). Anchor barge-in latency here. |
on_agent_speech_ended | Last wire chunk of the agent turn | Payload includes interrupted: bool. True = barge-in cancelled the turn. |
on_llm_token | First LLM token of the turn | TTFT marker. Idempotent — fires once per turn. |
on_audio_out | First TTS audio chunk produced | TTS warmup arrival (distinct from wire-time). Idempotent — fires once per turn. |
Payload signature matrix
State machine
conversation_state returns a snapshot {"user": <user_state>, "agent": <agent_state>} you can read at any time:
| Side | States | Initial | Set by |
|---|---|---|---|
user | listening · speaking · thinking · away | listening | on_user_speech_started → speaking, on_user_speech_ended / on_user_speech_eos → listening |
agent | initializing · idle · listening · thinking · speaking | initializing | call accepted → idle, EOU committed → thinking, on_agent_speech_started → speaking, on_agent_speech_ended → idle |
turn_idx counter (also exposed on the dispatcher) increments on every committed EOU. The agent_speech_*, llm_token, and audio_out payloads all carry the current turn_idx so a per-turn metric can correlate them.
Sequence for a normal turn
Sequence for a barged-in turn
Full example — wire all seven callbacks
Barge-in detection
The cleanest way to detect a barge-in is to inspecton_agent_speech_ended.interrupted:
on_user_speech_started with the next on_agent_speech_ended({"interrupted": True}):
Wiring
The realtime stream handler firesuser_speech_started/_ended/_eos and agent_speech_started/_ended automatically on the OpenAI Realtime + Twilio/Telnyx path — no extra setup required.
on_llm_token and on_audio_out are exposed on the dispatcher (phone.speech_events) so custom adapters and pipeline-mode integrations can call them. If you are building a custom provider, call phone.speech_events.fire_llm_first_token(...) on your first streamed chunk and phone.speech_events.fire_audio_out(...) on your first synthesized audio buffer; both are idempotent within a turn.
Public exports
| Export | Type | Use |
|---|---|---|
SpeechEvents | class | The dispatcher. One instance per Patter (auto-created). |
SpeechEventCallback | type alias | Callable[[dict], Awaitable[None] | None]. |
ConversationStateSnapshot | dict shape | {"user": <user_state>, "agent": <agent_state>}. |
UserState | str literal | "listening" | "speaking" | "thinking" | "away". |
AgentState | str literal | "initializing" | "idle" | "listening" | "thinking" | "speaking". |
EouTrigger | str literal | "vad_silence" | "semantic_turn_detector" | "manual_commit". |
OpenTelemetry attach contract
Every speech-edge event also records a span event on the active call span whenPATTER_OTEL_ENABLED=1 and the optional opentelemetry peer dep is installed. When OTel is missing or disabled, the OTel branch is a zero-cost no-op — there is no overhead and no failure.
| Callback | Span event name | Selected attributes |
|---|---|---|
on_user_speech_started | patter.event.user_speech_started | patter.audio.offset_ms, patter.vad.confidence |
on_user_speech_ended | patter.event.user_speech_ended | patter.speech.duration_ms |
on_user_speech_eos | patter.event.user_speech_eos | patter.eos.trigger, patter.eos.trailing_silence_ms |
on_agent_speech_started | patter.event.agent_speech_started | patter.turn.idx, patter.tts.provider, patter.engine |
on_agent_speech_ended | patter.event.agent_speech_ended | patter.turn.idx, patter.speech.duration_ms, patter.turn.interrupted |
on_llm_token | patter.event.llm_first_token | gen_ai.request.model, gen_ai.provider.name (per OTel GenAI semconv), patter.turn.idx |
on_audio_out | patter.event.tts_first_audio | patter.turn.idx, patter.tts.provider |
Callback safety
Observer exceptions are caught and logged, never propagated to the live call. A misbehaving callback cannot crash the call or break audio. Errors are logged at WARNING level under thegetpatter.events logger with the offending span event name for easy correlation.
Design notes
on_user_speech_endedvs.on_user_speech_eos: surfaced as separate events because they are two different signals.silence_gap_ms_maxwants the EOU;cross_talk_pctwants the raw VAD edge.on_agent_speech_startedvs.on_audio_out:on_audio_outis when TTS bytes arrive in the buffer (warmup metric).on_agent_speech_startedis when those bytes hit the carrier wire — what the user actually hears. Subtract the two to measure carrier-side jitter.- Idempotency:
on_llm_tokenandon_audio_outfire at most once per turn. The guard is reset onon_user_speech_eosso the next turn re-arms cleanly.

