Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.getpatter.com/llms.txt

Use this file to discover all available pages before exploring further.

​
ElevenLabs ConvAI

ElevenLabsConvAI is a Patter engine — an end-to-end speech-to-speech runtime that handles STT, LLM, and TTS in a single managed WebSocket. Instead of wiring stt, llm, and tts separately (pipeline mode), you point Patter at an agent you already configured in the ElevenLabs Conversational AI dashboard and Patter bridges the carrier media stream straight to it. This is a different mental model from the pipeline-mode TTS providers — even though both use ElevenLabs:
ModeWhen to useConfig surface
Pipeline with ElevenLabsTTSYou want to keep STT, LLM, prompts, and tools in your code; ElevenLabs is just the voice.stt, llm, tts, systemPrompt, tools
Engine with ElevenLabsConvAIYou want a managed agent (prompt, tools, voice, knowledge base) authored in ElevenLabs’s UI.engine, agentId
If you don’t yet know which mode fits, start with pipeline mode — it gives you Patter’s full feature set (custom tools, guardrails, dashboard, metrics). ConvAI is the right pick when your team treats the agent as a deployable artifact configured outside code.

​
Install

ElevenLabsConvAI ships in the core getpatter package — no extras needed: 
npm install getpatter

​
Quickstart

// npx tsx example.ts
import { Patter, Twilio, ElevenLabsConvAI } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
  engine: new ElevenLabsConvAI({ agentId: "agent_abc123" }),  // ELEVENLABS_API_KEY from env
  systemPrompt: "You are a warm and friendly concierge.",     // used as override
});

await phone.serve({ agent });
The agent is created in the ElevenLabs ConvAI dashboard. Patter only needs the agentId (and an API key) to connect to it.

​
Engine constructor

The engine config is intentionally tiny — most settings live on the ElevenLabs side:
ParameterTypeDefaultDescription
apiKeystring—ElevenLabs API key. Reads from ELEVENLABS_API_KEY when omitted.
agentIdstring—ElevenLabs agent ID. Reads from ELEVENLABS_AGENT_ID when omitted.
voicestring—Optional override for the agent’s default voice ID.

​
Lower-level adapter

When you need the full ConvAI feature set — first-message overrides, language pinning, signed-URL auth, audio-format negotiation — drop down to ElevenLabsConvAIAdapter: 
import { ElevenLabsConvAIAdapter } from "getpatter";

// Twilio-native μ-law @ 8 kHz — skips every transcoding hop.
const adapter = ElevenLabsConvAIAdapter.forTwilio("API_KEY", "agent_abc123", {
  voiceId: "EXAVITQu4vr4xnSDxMaL",
  modelId: "eleven_flash_v2_5",
  language: "it",
  firstMessage: "Ciao!",
});

// Telnyx — same μ-law optimisation when streaming_bidirectional_codec=PCMU.
const adapterTelnyx = ElevenLabsConvAIAdapter.forTelnyx("API_KEY", "agent_abc123");

// Bare constructor — defaults to PCM16 16 kHz (best for non-telephony).
const adapterBare = new ElevenLabsConvAIAdapter({
  apiKey: "API_KEY",
  agentId: "agent_abc123",
});

​
Telephony optimisation

Both forTwilio and forTelnyx negotiate ulaw_8000 for both outputAudioFormat and inputAudioFormat. This matches Twilio’s μ-law @ 8 kHz wire format and Telnyx’s PCMU bidirectional default — the SDK detects this and skips the 8 kHz ↔ 16 kHz resamples and the PCM ↔ μ-law transcodes. Saves ~30–80 ms first-byte plus per-frame CPU on every turn. If your Telnyx profile is pinned to L16/16000 instead, use the bare constructor (defaults to PCM16).

​
Signed-URL auth

When you cannot ship the API key to the WebSocket layer (browser bridges, untrusted runners), pass useSignedUrl: true and the adapter will fetch a short-lived signed URL from /v1/convai/conversation/get-signed-url instead: 
const adapter = new ElevenLabsConvAIAdapter({
  apiKey: "API_KEY",
  agentId: "agent_abc123",
  useSignedUrl: true,
});

​
Events

The adapter normalises ConvAI server messages into a unified event stream so the rest of Patter (metrics, dashboard, hooks) treats it like any other engine:
EventPayloadMeaning
audioBufferAgent audio chunk to forward to the caller.
transcript_inputstringWhat the user said.
transcript_outputstringWhat the agent said (full turn text).
response_start{ text: string }Agent began speaking (turn start).
response_donenullAgent turn finished — emitted on silence (~500 ms) or interruption.
interruptionnullUser interrupted the agent.
errorstringServer-side error text.
close{ code, reason }WebSocket closed.
response_done is emitted by a silence watcher because ConvAI does not send an explicit “turn complete” frame. 
adapter.onEvent((type, data) => {
  if (type === "audio") forwardToCaller(data as Buffer);
  if (type === "transcript_output") console.log("Agent said:", data);
});

​
Limitations

  • No client-side systemPrompt injection. The agent’s prompt is owned by the ElevenLabs dashboard. Patter passes systemPrompt to the adapter as a firstMessage override only — to change behaviour, edit the agent in the ConvAI UI.
  • No client-side tools. ConvAI tools are configured in the ElevenLabs dashboard. Patter’s tools array is ignored when an engine is set.
  • modelId override is informational. The TypeScript and Python SDKs preserve the modelId field for introspection, but ConvAI does not currently honour a client-side model override.

​
Pricing

ElevenLabs ConvAI is billed by ElevenLabs per-minute (premium tiers add features like RAG and analytics). Patter does not currently meter ConvAI minutes — the engine cost shows up on your ElevenLabs invoice. Carrier minutes (Twilio / Telnyx) are still tracked as normal in CallMetrics.

​
See also