Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.getpatter.com/llms.txt

Use this file to discover all available pages before exploring further.

​
ElevenLabs ConvAI

ElevenLabsConvAI is a Patter engine — an end-to-end speech-to-speech runtime that handles STT, LLM, and TTS in a single managed WebSocket. Instead of wiring stt=, llm=, and tts= separately (pipeline mode), you point Patter at an agent you already configured in the ElevenLabs Conversational AI dashboard and Patter bridges the carrier media stream straight to it. This is a different mental model from the pipeline-mode TTS providers — even though both use ElevenLabs:
ModeWhen to useConfig surface
Pipeline with ElevenLabsTTSYou want to keep STT, LLM, prompts, and tools in your code; ElevenLabs is just the voice.stt=, llm=, tts=, system_prompt=, tools=
Engine with ElevenLabsConvAIYou want a managed agent (prompt, tools, voice, knowledge base) authored in ElevenLabs’s UI.engine=, agent_id=
If you don’t yet know which mode fits, start with pipeline mode — it gives you Patter’s full feature set (custom tools, guardrails, dashboard, metrics). ConvAI is the right pick when your team treats the agent as a deployable artifact configured outside code.

​
Install

ElevenLabsConvAI ships in the core getpatter package — no extras needed: 
pip install getpatter

​
Quickstart

import asyncio
from getpatter import Patter, Twilio, ElevenLabsConvAI

phone = Patter(carrier=Twilio(), phone_number="+15550001234")    # TWILIO_* from env

agent = phone.agent(
    engine=ElevenLabsConvAI(agent_id="agent_abc123"),            # ELEVENLABS_API_KEY from env
    system_prompt="You are a warm and friendly concierge.",      # used as override
)

asyncio.run(phone.serve(agent))
The agent is created in the ElevenLabs ConvAI dashboard. Patter only needs the agent_id (and an API key) to connect to it.

​
Engine constructor

The engine config object is intentionally tiny — most settings live on the ElevenLabs side:
ParameterTypeDefaultDescription
api_keystr""ElevenLabs API key. Reads from ELEVENLABS_API_KEY when empty.
agent_idstr""ElevenLabs agent ID. Reads from ELEVENLABS_AGENT_ID when empty.
voicestr""Optional override for the agent’s default voice ID.
from getpatter.engines import elevenlabs as elevenlabs_engine

engine = elevenlabs_engine.ConvAI()                                # reads env
engine = elevenlabs_engine.ConvAI(agent_id="agent_abc123",
                                  voice="EXAVITQu4vr4xnSDxMaL")

​
Lower-level adapter

When you need the full ConvAI feature set — first-message overrides, language pinning, signed-URL auth, audio-format negotiation — drop down to ElevenLabsConvAIAdapter: 
from getpatter.providers.elevenlabs_convai import ElevenLabsConvAIAdapter

# Twilio-native μ-law @ 8 kHz — skips every transcoding hop.
adapter = ElevenLabsConvAIAdapter.for_twilio(
    api_key="...",
    agent_id="agent_abc123",
    voice_id="EXAVITQu4vr4xnSDxMaL",
    model_id="eleven_flash_v2_5",
    language="it",
    first_message="Ciao!",
)

# Telnyx — same μ-law optimisation when streaming_bidirectional_codec=PCMU.
adapter = ElevenLabsConvAIAdapter.for_telnyx(api_key="...", agent_id="agent_abc123")

# Or the bare constructor (defaults to PCM16 16 kHz — best for non-telephony).
adapter = ElevenLabsConvAIAdapter(api_key="...", agent_id="agent_abc123")

​
Telephony optimisation

Both for_twilio / forTwilio and for_telnyx / forTelnyx negotiate ulaw_8000 for both output_audio_format and input_audio_format. This matches Twilio’s μ-law @ 8 kHz wire format and Telnyx’s PCMU bidirectional default — the SDK detects this and skips the 8 kHz ↔ 16 kHz resamples and the PCM ↔ μ-law transcodes. Saves ~30–80 ms first-byte plus per-frame CPU on every turn. If your Telnyx profile is pinned to L16/16000 instead, use the bare constructor (defaults to PCM16).

​
Signed-URL auth

When you cannot ship the API key to the WebSocket layer (browser bridges, untrusted runners), pass use_signed_url=True and Patter will fetch a short-lived signed URL from /v1/convai/conversation/get-signed-url instead: 
adapter = ElevenLabsConvAIAdapter(
    api_key="API_KEY",
    agent_id="agent_abc123",
    use_signed_url=True,
)

​
Events

The adapter normalises ConvAI server messages into a unified event stream so the rest of Patter (metrics, dashboard, hooks) treats it like any other engine:
EventPayloadMeaning
audiobytesAgent audio chunk to forward to the caller.
transcript_inputstrWhat the user said.
transcript_outputstrWhat the agent said (full turn text).
response_start{"text": str}Agent began speaking (turn start).
response_done{}Agent turn finished — emitted on silence (~500 ms) or interruption.
interruptionNoneUser interrupted the agent.
errorstrServer-side error text.
response_done is emitted by a silence watcher because ConvAI does not send an explicit “turn complete” frame.

​
Limitations

  • No client-side system_prompt injection. The agent’s prompt is owned by the ElevenLabs dashboard. Patter passes system_prompt to the adapter as a first_message override only — to change behaviour, edit the agent in the ConvAI UI.
  • No client-side tools. ConvAI tools are configured in the ElevenLabs dashboard. Patter’s tools=[...] is ignored when an engine= is set.
  • model_id override is informational. The Python and TypeScript SDKs preserve the model_id field for introspection, but ConvAI does not currently honour a client-side model override — the model is whatever the agent was configured with.

​
Pricing

ElevenLabs ConvAI is billed by ElevenLabs per-minute (premium tiers add features like RAG and analytics). Patter does not currently meter ConvAI minutes — the engine cost shows up on your ElevenLabs invoice. Carrier minutes (Twilio / Telnyx) are still tracked as normal in CallMetrics.

​
See also