> ## Documentation Index > Fetch the complete documentation index at: https://docs.getpatter.com/llms.txt > Use this file to discover all available pages before exploring further. # Agents > Define your agent's voice, model, tools, and behavior for phone calls. # Agents An agent configuration defines the personality, capabilities, and behavior of your AI voice assistant. Use `phone.agent()` to validate your agent config before connecting it to a phone number. ## Basic Agent The simplest form leans on env-var fallback and a default engine (`OpenAIRealtime`): ```typescript theme={null} import { Patter, Twilio } from "getpatter"; const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" }); const agent = phone.agent({ systemPrompt: "You are a helpful customer support agent for Acme Corp.", firstMessage: "Hello! How can I help you today?", }); // defaults to engine: OpenAIRealtime, reads OPENAI_API_KEY ``` To pick the engine explicitly (flat imports): ```typescript theme={null} import { Patter, Twilio, OpenAIRealtime } from "getpatter"; const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" }); const agent = phone.agent({ engine: new OpenAIRealtime({ voice: "nova" }), systemPrompt: "You are a helpful customer support agent for Acme Corp.", }); ``` Pipeline mode (pick STT, LLM, TTS independently): ```typescript theme={null} import { Patter, Twilio, DeepgramSTT, AnthropicLLM, ElevenLabsTTS } from "getpatter"; const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" }); const agent = phone.agent({ stt: new DeepgramSTT({ endpointingMs: 80 }), // DEEPGRAM_API_KEY from env llm: new AnthropicLLM(), // ANTHROPIC_API_KEY from env tts: new ElevenLabsTTS({ voiceId: "rachel" }), // ELEVENLABS_API_KEY from env systemPrompt: "You are a helpful assistant.", firstMessage: "Hi!", }); ``` Available LLM providers: `OpenAILLM`, `AnthropicLLM`, `GroqLLM`, `CerebrasLLM`, `GoogleLLM`. Tool calling works across all five. See [LLM](/typescript-sdk/llm) for the full reference. For fully custom logic (multi-model routing, local models), drop `llm` and pass an `onMessage` callback to `serve()` instead — `llm` and `onMessage` are mutually exclusive. ## AgentOptions | Parameter | Type | Required | Default | Description | | ---------------------- | -------------------------------------------------------- | -------- | ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `systemPrompt` | `string` | Yes | — | Instructions that define the agent's persona and behavior. | | `engine` | `OpenAIRealtime \| OpenAIRealtime2 \| ElevenLabsConvAI` | No | defaults to `OpenAIRealtime` when none provided and pipeline pieces are absent | End-to-end engine. See [Engines](/typescript-sdk/engines). Omit for pipeline mode. | | `stt` | `STTProvider` | No | — | STT instance for pipeline mode (`new DeepgramSTT()`, `new CartesiaSTT()`, ...). | | `llm` | `LLMProvider` | No | — | LLM instance for pipeline mode (`new AnthropicLLM()`, `new GroqLLM()`, ...). Mutually exclusive with `onMessage` on `serve()`. Ignored when `engine` is set. See [LLM](/typescript-sdk/llm). | | `tts` | `TTSProvider` | No | — | TTS instance for pipeline mode (`new ElevenLabsTTS()`, `new RimeTTS()`, ...). | | `voice` | `string` | No | Provider default | Voice ID. Usually inferred from the engine or TTS instance. | | `model` | `string` | No | `"gpt-4o-mini-realtime-preview"` | Model ID for OpenAI Realtime. Usually inferred from the engine. | | `language` | `string` | No | `"en"` | BCP-47 language code. | | `firstMessage` | `string` | No | — | Greeting spoken when the call connects. | | `tools` | `ToolDefinition[]` | No | — | Function calling tools. See [Tools](/typescript-sdk/tools). | | `variables` | `Record` | No | — | Dynamic variables for `{placeholder}` substitution in `systemPrompt`. | | `guardrails` | `Guardrail[]` | No | — | Output guardrails. See [Guardrails](/typescript-sdk/guardrails). | | `hooks` | `PipelineHooks` | No | — | Pipeline hooks for intercepting STT/TTS processing. | | `textTransforms` | `((text: string) => string)[]` | No | — | Text transformation functions for LLM responses. | | `vad` | `VADProvider` | No | — | Voice activity detection provider. | | `audioFilter` | `AudioFilter` | No | — | Audio preprocessing filter. | | `backgroundAudio` | `BackgroundAudioPlayer` | No | — | Background audio player. | | `bargeInThresholdMs` | `number` | No | `300` | Barge-in hang-over window (ms). Set to `0` to disable. | | `aggressiveFirstFlush` | `boolean` | No | `false` | Opt-in low-latency mode: emits the first clause on soft punctuation (`,`, em-dash) once the buffer reaches ≥40 chars. Saves 200–500 ms TTFA. Hard-disabled when `language="it"` (Italian punctuation patterns are incompatible). | | `disablePhonePreamble` | `boolean` | No | `false` | When `false` (default), Patter prepends a phone-friendly preamble to `systemPrompt` that instructs the LLM to avoid markdown, emojis, bullet lists, and code blocks; spell out numbers and dates; and keep replies short. Set to `true` to ship `systemPrompt` verbatim. | | `prewarmFirstMessage` | `boolean` | No | `false` (pipeline mode only) | Pre-render `firstMessage` to TTS audio bytes during the ringing window and stream the cached buffer the instant the call connects, eliminating the 200–700 ms TTS first-byte latency on the greeting. Realtime / ConvAI engines silently ignore the flag (with a `WARN` log) — only pipeline mode consumes the cache. Trade-off: pays for the greeting's TTS even when the call rings out unanswered (\~$0.001–$0.005 per ring). Opt in explicitly for inbound calls and low-noise deployments: `prewarmFirstMessage: true`. | | `provider` | `'openai_realtime' \| 'elevenlabs_convai' \| 'pipeline'` | No | derived | Provider mode. Normally derived from `engine` / `stt` + `tts`. Pass `'pipeline'` explicitly when building a pipeline-mode agent without an engine instance. | ## Validation Rules The `phone.agent()` method validates: * **Engine / pipeline**: exactly one of `engine`, `(stt` + `tts)` must resolve correctly. * **Tools**: must be an array. Each tool requires a `name` field and either a `webhookUrl` or a `handler`. * **Variables**: must be a plain object (not an array). ## Dynamic Variables Use `{placeholder}` syntax in your system prompt. Variables are replaced at call time: ```typescript theme={null} const agent = phone.agent({ systemPrompt: "You are a support agent for {company}. The caller's name is {caller_name}.", variables: { company: "Acme Corp", caller_name: "John", }, }); ``` Variables can also be overridden per-call when making outbound calls. See [Features](/typescript-sdk/features). ## System Tools Two system tools are automatically injected into every agent: * **`transfer_call`** — Transfers the call to a specified phone number (E.164 format). * **`end_call`** — Ends the current call with an optional reason. You do not need to define these in your `tools` array. The AI model can invoke them based on conversation context. ## Voice Activity Detection (VAD) Pipeline-mode agents can plug a VAD provider into the `vad` option to gate STT around real speech and drive barge-in detection. The SDK ships **Silero VAD** (an ONNX model, \~1 MB) with a telephony-tuned factory: ```typescript theme={null} import { SileroVAD, DeepgramSTT, AnthropicLLM, ElevenLabsTTS } from "getpatter"; // Recommended for any phone-call deployment. const vad = await SileroVAD.forPhoneCall(); const agent = phone.agent({ stt: new DeepgramSTT(), llm: new AnthropicLLM(), tts: new ElevenLabsTTS({ voiceId: "rachel" }), systemPrompt: "You are a helpful assistant.", vad, }); ``` `SileroVAD.forPhoneCall(options?)` is identical to `SileroVAD.load(...)` but pins `sampleRate` to 16 000 Hz — the only sample rate Patter's pipeline-mode audio bus uses (8 kHz mulaw from Twilio is upsampled to 16 kHz PCM before reaching the VAD). All other parameters use the upstream `snakers4/silero-vad` defaults: | Field | Default | Upstream equivalent | | ----------------------- | -------- | -------------------------------------------------------- | | `activationThreshold` | `0.8` | `threshold` (tuned for telephony, not studio) | | `deactivationThreshold` | `0.65` | `neg_threshold = threshold − 0.15` (tuned for telephony) | | `minSpeechDuration` | `0.25` s | `min_speech_duration_ms = 250` | | `minSilenceDuration` | `0.1` s | `min_silence_duration_ms = 100` | | `prefixPaddingDuration` | `0.03` s | `speech_pad_ms = 30` | Override per call site rather than as a global default. A common tweak: deployments that experience truncation on natural pauses raise `minSilenceDuration` to 0.5–1.0 s: ```typescript theme={null} const vad = await SileroVAD.forPhoneCall({ minSilenceDuration: 0.5 }); ``` `SileroVAD.forPhoneCall()` returns a `Promise` — `await` it once at process startup before constructing your agent. The underlying ONNX session is reused across calls. ## Engine vs Pipeline Mode ```typescript theme={null} import { OpenAIRealtime, ElevenLabsConvAI, DeepgramSTT, AnthropicLLM, ElevenLabsTTS } from "getpatter"; // OpenAI Realtime — end-to-end const agent = phone.agent({ engine: new OpenAIRealtime(), systemPrompt: "...", }); // ElevenLabs Conversational AI — natural voices const agent = phone.agent({ engine: new ElevenLabsConvAI({ agentId: "agent_abc123" }), systemPrompt: "...", }); // Pipeline — pick STT, LLM, TTS independently const agent = phone.agent({ stt: new DeepgramSTT(), llm: new AnthropicLLM(), tts: new ElevenLabsTTS({ voiceId: "rachel" }), systemPrompt: "...", }); ``` See [LLM](/typescript-sdk/llm) for a deeper comparison.