> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getpatter.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Engines

> End-to-end speech-to-speech runtimes (OpenAI Realtime, ElevenLabs ConvAI).

# Engines

An **engine** is an end-to-end speech-to-speech runtime. Pass an engine instance to `phone.agent({ engine })` and Patter wires the audio stream straight through to the provider — no separate STT or TTS is needed.

Patter ships several engine classes:

* [`OpenAIRealtime`](#openairealtime) — OpenAI's Realtime API (beta endpoint)
* [`OpenAIRealtime2`](#openairealtime2) — OpenAI's GA Realtime API (targets `gpt-realtime-2`)
* [`ElevenLabsConvAI`](#elevenlabsconvai) — ElevenLabs Conversational AI
* [`XaiRealtime`](#xairealtime) — xAI Grok Voice Agent (OpenAI-GA-compatible)

Each class is imported by name from the package barrel: `import { OpenAIRealtime, OpenAIRealtime2, ElevenLabsConvAI, XaiRealtime } from "getpatter"`.

If you need full control over STT, LLM, and TTS independently, use [pipeline mode](/typescript-sdk/llm#pipeline-mode) instead and omit `engine`.

## OpenAIRealtime

OpenAI's Realtime API — the lowest-latency option.

```typescript theme={null}
// npx tsx example.ts
import { Patter, Twilio, OpenAIRealtime } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
  engine: new OpenAIRealtime({ voice: "alloy" }),           // OPENAI_API_KEY from env
  systemPrompt: "You are a helpful assistant.",
  firstMessage: "Hello!",
});

await phone.serve({ agent });
```

<Note>
  **Telephony audio.** Over Twilio/Telnyx the `OpenAIRealtime` engine routes
  through the same GA-compatible adapter as [`OpenAIRealtime2`](#openairealtime2):
  it negotiates PCM-16-LE @ 24 kHz with OpenAI and transcodes to/from the
  carrier's mulaw 8 kHz internally. Current OpenAI Realtime models return PCM16 @
  24 kHz regardless of a legacy `g711_ulaw` request, so Patter standardises on PCM
  and converts on the carrier leg — you don't configure any of this.
</Note>

| Parameter                      | Type                                       | Default                          | Description                                                                                                                                                     |
| ------------------------------ | ------------------------------------------ | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `apiKey`                       | `string`                                   | —                                | OpenAI API key. Reads from `OPENAI_API_KEY` when omitted.                                                                                                       |
| `voice`                        | `string`                                   | `"alloy"`                        | One of `"alloy"`, `"ash"`, `"ballad"`, `"coral"`, `"echo"`, `"sage"`, `"shimmer"`, `"verse"`.                                                                   |
| `model`                        | `string`                                   | `"gpt-4o-mini-realtime-preview"` | OpenAI Realtime model ID. See [supported models](/typescript-sdk/providers/openai-realtime#models).                                                             |
| `reasoningEffort`              | `"minimal" \| "low" \| "medium" \| "high"` | —                                | Reasoning-effort tier for `gpt-realtime-2`. When omitted, the field is not sent and the server default applies. OpenAI recommends `"low"` for production voice. |
| `inputAudioTranscriptionModel` | `string`                                   | —                                | Override for the Realtime session's `input_audio_transcription.model`. Omit to keep `whisper-1`.                                                                |

### Supported model identifiers

The `model` option accepts any OpenAI Realtime model ID. Common values:

| Model                            | Notes                                                                                                                                                        |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `"gpt-4o-mini-realtime-preview"` | Engine marker default. Earlier preview line — cheap and low-latency.                                                                                         |
| `"gpt-realtime-mini"`            | GA mini — recommended cheap default for new deployments.                                                                                                     |
| `"gpt-realtime"`                 | GA realtime model (Aug 2025).                                                                                                                                |
| `"gpt-realtime-2"`               | Most-capable: stronger instruction following, configurable `reasoningEffort`, 128K context. Use [`OpenAIRealtime2`](#openairealtime2) for the GA wire shape. |
| `"gpt-4o-realtime-preview"`      | Earlier preview line; \~10x the per-token cost of mini.                                                                                                      |

Pricing is auto-resolved per model — see [Metrics](/typescript-sdk/metrics). For `reasoningEffort`, transcription model, and the full configuration surface, see [OpenAI Realtime — full reference](/typescript-sdk/providers/openai-realtime).

## OpenAIRealtime2

OpenAI's **GA Realtime API** — separate engine marker because the GA endpoint speaks a different `session.update` wire shape (`output_modalities`, nested `audio.{input,output}` blocks, `session.type = "realtime"`) and rejects the legacy beta header. Targets `gpt-realtime-2` by default and routes through `OpenAIRealtime2Adapter`, which also handles bidirectional mulaw 8 kHz ↔ PCM 24 kHz transcoding (the GA audio engine silently drops mulaw frames).

```typescript theme={null}
// npx tsx example.ts
import { Patter, Twilio, OpenAIRealtime2 } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
  engine: new OpenAIRealtime2({ reasoningEffort: "low" }),  // OPENAI_API_KEY from env
  systemPrompt: "You are a helpful assistant.",
  firstMessage: "Hello!",
});

await phone.serve({ agent });
```

| Parameter                      | Type                                       | Default            | Description                                                                                                         |
| ------------------------------ | ------------------------------------------ | ------------------ | ------------------------------------------------------------------------------------------------------------------- |
| `apiKey`                       | `string`                                   | —                  | OpenAI API key. Reads from `OPENAI_API_KEY` when omitted.                                                           |
| `voice`                        | `string`                                   | `"alloy"`          | Voice preset.                                                                                                       |
| `model`                        | `string`                                   | `"gpt-realtime-2"` | GA Realtime model.                                                                                                  |
| `reasoningEffort`              | `"minimal" \| "low" \| "medium" \| "high"` | —                  | When omitted, the field is not sent and the server default applies. OpenAI recommends `"low"` for production voice. |
| `inputAudioTranscriptionModel` | `string`                                   | —                  | Override for `audio.input.transcription.model`. Omit to keep `whisper-1`.                                           |

<Note>
  The GA adapter pins `turn_detection.create_response: false` and `interrupt_response: false` in the `session.update` payload. Patter owns response creation (`response.create`) and barge-in cancellation explicitly so the hallucination filter and barge-in pipeline can decide per turn rather than letting the server VAD auto-trigger. See [`OpenAIRealtime2` — full reference](/typescript-sdk/providers/openai-realtime-2).
</Note>

## ElevenLabsConvAI

ElevenLabs Conversational AI — premium voice quality using a managed agent configured in the ElevenLabs dashboard.

```typescript theme={null}
// npx tsx example.ts
import { Patter, Twilio, ElevenLabsConvAI } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
  engine: new ElevenLabsConvAI({ agentId: "agent_abc123" }),   // ELEVENLABS_API_KEY from env
  systemPrompt: "You are a warm and friendly concierge.",
});

await phone.serve({ agent });
```

| Parameter | Type     | Default | Description                                                                                     |
| --------- | -------- | ------- | ----------------------------------------------------------------------------------------------- |
| `apiKey`  | `string` | —       | ElevenLabs API key. Reads from `ELEVENLABS_API_KEY` when omitted.                               |
| `agentId` | `string` | —       | ElevenLabs agent ID (from the ConvAI dashboard). Reads from `ELEVENLABS_AGENT_ID` when omitted. |
| `voice`   | `string` | —       | Optional override for the agent's default voice ID.                                             |

## XaiRealtime

xAI's **Grok Voice Agent** — an OpenAI-Realtime-GA-compatible speech-to-speech engine with on-by-default reasoning and server-side tools (`web_search`, `x_search`, `mcp`, `file_search`).

```typescript theme={null}
// npx tsx example.ts
import { Patter, Twilio, XaiRealtime } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
  engine: new XaiRealtime({ voice: "eve" }),                   // XAI_API_KEY from env
  systemPrompt: "You are a friendly receptionist.",
  firstMessage: "Hello! How can I help?",
});

await phone.serve({ agent });
```

| Parameter         | Type               | Default               | Description                                                                              |
| ----------------- | ------------------ | --------------------- | ---------------------------------------------------------------------------------------- |
| `apiKey`          | `string`           | —                     | xAI API key. Reads from `XAI_API_KEY` when omitted.                                      |
| `model`           | `string`           | `"grok-voice-latest"` | Grok voice model. `grok-voice-latest` tracks the flagship (`grok-voice-think-fast-1.0`). |
| `voice`           | `string`           | `"eve"`               | Built-in or custom voice ID.                                                             |
| `reasoningEffort` | `"high" \| "none"` | —                     | `"high"` (xAI server default) enables reasoning; `"none"` disables it for lower latency. |

For the full session-option surface — VAD tuning, language hint, keyterms, pronunciation replacements, session resumption, and server-side tools — see [xAI Realtime — full reference](/typescript-sdk/providers/xai-realtime).

<Note>
  **Beta** — spec-validated, not yet live-call-validated.
</Note>

## What's Next

<CardGroup cols={2}>
  <Card title="LLM" icon="brain" href="/typescript-sdk/llm">Compare engine mode with pipeline mode.</Card>
  <Card title="STT" icon="microphone" href="/typescript-sdk/stt">STT for pipeline mode.</Card>
  <Card title="TTS" icon="volume" href="/typescript-sdk/tts">TTS for pipeline mode.</Card>
</CardGroup>