ElevenLabs ConvAI
ElevenLabsConvAI is a Patter engine — an end-to-end speech-to-speech runtime that handles STT, LLM, and TTS in a single managed WebSocket. Instead of wiring stt, llm, and tts separately (pipeline mode), you point Patter at an agent you already configured in the ElevenLabs Conversational AI dashboard and Patter bridges the carrier media stream straight to it.
This is a different mental model from the pipeline-mode TTS providers — even though both use ElevenLabs:
If you don’t yet know which mode fits, start with pipeline mode — it gives you Patter’s full feature set (custom tools, guardrails, dashboard, metrics). ConvAI is the right pick when your team treats the agent as a deployable artifact configured outside code.
Install
ElevenLabsConvAI ships in the core getpatter package — no extras needed:
Quickstart
agentId (and an API key) to connect to it.
Engine constructor
The engine config is intentionally tiny — most settings live on the ElevenLabs side:Lower-level adapter
When you need the full ConvAI feature set — first-message overrides, language pinning, signed-URL auth, audio-format negotiation — drop down toElevenLabsConvAIAdapter:
Telephony optimisation
BothforTwilio and forTelnyx negotiate ulaw_8000 for both outputAudioFormat and inputAudioFormat. This matches Twilio’s μ-law @ 8 kHz wire format and Telnyx’s PCMU bidirectional default — the SDK detects this and skips the 8 kHz ↔ 16 kHz resamples and the PCM ↔ μ-law transcodes. Saves ~30–80 ms first-byte plus per-frame CPU on every turn.
If your Telnyx profile is pinned to L16/16000 instead, use the bare constructor (defaults to PCM16).
Signed-URL auth
When you cannot ship the API key to the WebSocket layer (browser bridges, untrusted runners), passuseSignedUrl: true and the adapter will fetch a short-lived signed URL from /v1/convai/conversation/get-signed-url instead:
Events
The adapter normalises ConvAI server messages into a unified event stream so the rest of Patter (metrics, dashboard, hooks) treats it like any other engine:response_done is emitted by a silence watcher because ConvAI does not send an explicit “turn complete” frame.
Limitations
- No client-side
systemPromptinjection. The agent’s prompt is owned by the ElevenLabs dashboard. Patter passessystemPromptto the adapter as afirstMessageoverride only — to change behaviour, edit the agent in the ConvAI UI. - No client-side tools. ConvAI tools are configured in the ElevenLabs dashboard. Patter’s
toolsarray is ignored when anengineis set. modelIdoverride is informational. The TypeScript and Python SDKs preserve themodelIdfield for introspection, but ConvAI does not currently honour a client-side model override.
Pricing
ElevenLabs ConvAI is billed by ElevenLabs per-minute (premium tiers add features like RAG and analytics). Patter does not currently meter ConvAI minutes — the engine cost shows up on your ElevenLabs invoice. Carrier minutes (Twilio / Telnyx) are still tracked as normal inCallMetrics.
See also
- Engines overview — engine vs pipeline modes.
ElevenLabsTTS— pipeline-mode TTS using ElevenLabs voices (without a managed agent).ElevenLabsWebSocketTTS— opt-in low-latency WebSocket variant ofElevenLabsTTS.OpenAIRealtime— the other end-to-end engine.

