ElevenLabs ConvAI
ElevenLabsConvAI is a Patter engine — an end-to-end speech-to-speech runtime that handles STT, LLM, and TTS in a single managed WebSocket. Instead of wiring stt=, llm=, and tts= separately (pipeline mode), you point Patter at an agent you already configured in the ElevenLabs Conversational AI dashboard and Patter bridges the carrier media stream straight to it.
This is a different mental model from the pipeline-mode TTS providers — even though both use ElevenLabs:
If you don’t yet know which mode fits, start with pipeline mode — it gives you Patter’s full feature set (custom tools, guardrails, dashboard, metrics). ConvAI is the right pick when your team treats the agent as a deployable artifact configured outside code.
Install
ElevenLabsConvAI ships in the core getpatter package — no extras needed:
Quickstart
agent_id (and an API key) to connect to it.
Engine constructor
The engine config object is intentionally tiny — most settings live on the ElevenLabs side:Lower-level adapter
When you need the full ConvAI feature set — first-message overrides, language pinning, signed-URL auth, audio-format negotiation — drop down toElevenLabsConvAIAdapter:
Telephony optimisation
Bothfor_twilio / forTwilio and for_telnyx / forTelnyx negotiate ulaw_8000 for both output_audio_format and input_audio_format. This matches Twilio’s μ-law @ 8 kHz wire format and Telnyx’s PCMU bidirectional default — the SDK detects this and skips the 8 kHz ↔ 16 kHz resamples and the PCM ↔ μ-law transcodes. Saves ~30–80 ms first-byte plus per-frame CPU on every turn.
If your Telnyx profile is pinned to L16/16000 instead, use the bare constructor (defaults to PCM16).
Signed-URL auth
When you cannot ship the API key to the WebSocket layer (browser bridges, untrusted runners), passuse_signed_url=True and Patter will fetch a short-lived signed URL from /v1/convai/conversation/get-signed-url instead:
Events
The adapter normalises ConvAI server messages into a unified event stream so the rest of Patter (metrics, dashboard, hooks) treats it like any other engine:response_done is emitted by a silence watcher because ConvAI does not send an explicit “turn complete” frame.
Limitations
- No client-side
system_promptinjection. The agent’s prompt is owned by the ElevenLabs dashboard. Patter passessystem_promptto the adapter as afirst_messageoverride only — to change behaviour, edit the agent in the ConvAI UI. - No client-side tools. ConvAI tools are configured in the ElevenLabs dashboard. Patter’s
tools=[...]is ignored when anengine=is set. model_idoverride is informational. The Python and TypeScript SDKs preserve themodel_idfield for introspection, but ConvAI does not currently honour a client-side model override — the model is whatever the agent was configured with.
Pricing
ElevenLabs ConvAI is billed by ElevenLabs per-minute (premium tiers add features like RAG and analytics). Patter does not currently meter ConvAI minutes — the engine cost shows up on your ElevenLabs invoice. Carrier minutes (Twilio / Telnyx) are still tracked as normal inCallMetrics.
See also
- Engines overview — engine vs pipeline modes.
ElevenLabsTTS— pipeline-mode TTS using ElevenLabs voices (without a managed agent).ElevenLabsWebSocketTTS— opt-in low-latency WebSocket variant ofElevenLabsTTS.OpenAIRealtime— the other end-to-end engine.

