xAI STT

XaiSTT streams raw audio to xAI’s real-time Speech-to-Text WebSocket (wss://api.x.ai/v1/stt) and yields Patter transcript events. Configuration is carried entirely in the URL query string — there is no setup message — and audio is sent as raw binary frames (no base64). The adapter waits for the server’s transcript.created event before it declares the connection ready, then maps xAI’s is_final / speech_final flags onto Patter’s interim / chunk-final / utterance-final semantics, the same two-flag scheme Deepgram uses. The module also exports a one-shot batch helper — xaiTranscribe (TypeScript) / xai_transcribe (Python) — that wraps the REST endpoint (POST https://api.x.ai/v1/stt) for file or URL transcription with word timestamps.

Beta. The adapter is validated against the xAI STT API spec; it has not yet been exercised against a live phone call.

Install

npm install getpatter

pip install "getpatter[xai]"

The TypeScript adapter uses the ws package and the platform fetch — no vendor SDK. The Python adapter uses aiohttp (pulled in by the [xai] extra).

Authentication

export XAI_API_KEY="<your-xai-api-key>"

Both the streaming and batch endpoints authenticate with an Authorization: Bearer <key> header. XAI_API_KEY is read automatically when apiKey / api_key is omitted.

Usage

Use the namespaced import (getpatter/stt/xai) or the flat re-export (XaiSTT). Both auto-resolve XAI_API_KEY from the environment when apiKey is omitted.

// Namespaced import (pipeline mode)
import * as xai from "getpatter/stt/xai";

const stt = new xai.STT();                                 // reads XAI_API_KEY
const stt2 = new xai.STT({ apiKey: "...", language: "en", smartTurn: 0.7 });

// Flat alias (equivalent)
import { XaiSTT } from "getpatter";

const stt3 = new XaiSTT();

# Namespaced import (pipeline mode)
from getpatter.stt import xai

stt = xai.STT()                                            # reads XAI_API_KEY
stt = xai.STT(api_key="...", language="en", smart_turn=0.7)

# Flat alias (equivalent)
from getpatter import XaiSTT

stt = XaiSTT()

Plug it into an agent:

// npx tsx example.ts
import { Patter, Twilio, XaiSTT, XaiTTS } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
  stt: new XaiSTT({ smartTurn: 0.7 }),                     // XAI_API_KEY from env
  tts: XaiTTS.forTwilio({ voice: "eve" }),
  systemPrompt: "You are a helpful assistant.",
});

await phone.serve({ agent });

import asyncio
from getpatter import Patter, Twilio, XaiSTT, XaiTTS

phone = Patter(carrier=Twilio(), phone_number="+15550001234")

agent = phone.agent(
    stt=XaiSTT(smart_turn=0.7),                            # XAI_API_KEY from env
    tts=XaiTTS.for_twilio(voice="eve"),
    system_prompt="You are a helpful assistant.",
)

asyncio.run(phone.serve(agent))

Streaming transcript semantics

xAI emits transcript.partial events whose is_final / speech_final flags map onto the transcript states the pipeline reacts to:

`is_final`	`speech_final`	Patter meaning	Emitted when
`false`	`false`	Interim — a live partial for barge-in	only with `interimResults` on (~every 500 ms)
`true`	`false`	Chunk final — a locked segment (~3 s)	continuous long speech
`true`	`true`	Utterance final — end of turn	silence / Smart Turn / `finalize()`

A trailing transcript.done (after close() sends audio.done) surfaces any remaining text as a final. error frames are logged and do not close the connection. The adapter exposes finalize() (Python finalize()), which sends {"type": "finalize"} to force the current utterance to speech_final immediately. The pipeline’s VAD speech-end fast-path duck-types this — without it every turn waits out the full endpointing delay.

Smart Turn end-of-turn detection

Set smartTurn / smart_turn to a confidence threshold in [0.0, 1.0] to enable xAI’s Smart Turn ML end-of-turn model instead of relying on silence-based endpointing alone. The threshold trades responsiveness against interruption safety:

Threshold	Behaviour
Low (~0.1–0.3)	Ends the turn eagerly on weak end-of-turn signals — snappiest, but more likely to cut a speaker off mid-thought.
Medium (~0.4–0.6)	Balanced default for most conversational agents.
High (~0.7–0.9)	Waits for a strong end-of-turn signal — safest against premature cut-offs, slightly higher latency.

Pair it with smartTurnTimeoutMs / smart_turn_timeout_ms ([1, 5000]) to force speech_final after that much silence even when the model is undecided.

const stt = new XaiSTT({ smartTurn: 0.6, smartTurnTimeoutMs: 2000 });

stt = XaiSTT(smart_turn=0.6, smart_turn_timeout_ms=2000)

Keyterms and diarization

Bias transcription toward domain vocabulary (product names, proper nouns) with up to 100 key terms (≤50 chars each), and turn on speaker diarization so each word carries a speaker index:

const stt = new XaiSTT({
  keyterms: ["Patter", "getpatter", "Grok"],
  diarize: true,
});

stt = XaiSTT(
    keyterms=["Patter", "getpatter", "Grok"],
    diarize=True,
)

Languages

Pass a language code to language to enable server-side text formatting (Inverse Text Normalization) for that language. xAI supports 25 codes for the streaming endpoint:

Code	Language	Code	Language	Code	Language
`ar`	Arabic	`id`	Indonesian	`ro`	Romanian
`cs`	Czech	`it`	Italian	`ru`	Russian
`da`	Danish	`ja`	Japanese	`es`	Spanish
`nl`	Dutch	`ko`	Korean	`sv`	Swedish
`en`	English	`mk`	Macedonian	`th`	Thai
`fil`	Filipino	`ms`	Malay	`tr`	Turkish
`fr`	French	`fa`	Persian	`vi`	Vietnamese
`de`	German	`pl`	Polish
`hi`	Hindi	`pt`	Portuguese

Encodings and sample rate

encoding accepts pcm (default), mulaw, or alaw. The default pcm @ sampleRate=16000 matches what Patter’s pipeline feeds every STT stage (the carrier bridge decodes inbound μ-law to linear PCM16 @ 16 kHz on both Twilio and Telnyx), so XaiSTT drops in without transcoding. Set mulaw / alaw @ 8000 only when feeding raw carrier audio directly.

Rates

xAI bills STT by audio duration. Patter meters the streaming rate on the pipeline (provider key xai):

Path	Rate
Streaming (WebSocket)	$0.20 / hr ($ 0.003333 / min)
Batch (REST / `xaiTranscribe`)	$0.10 / hr

Override the metered rate per-project via new Patter({ pricing: { xai: { price: ... } } }). See Metrics for the full rate table.

Options

TypeScript	Python	Default	Notes
`apiKey`	`api_key`	—	Reads `XAI_API_KEY` when omitted.
`language`	`language`	—	BCP-47 code enabling server-side text formatting. Auto when unset.
`encoding`	`encoding`	`"pcm"`	`pcm` / `mulaw` / `alaw`.
`sampleRate`	`sample_rate`	`16000`	Input sample rate (Hz).
`interimResults`	`interim_results`	`true`	Emit `is_final=false` partials (~every 500 ms).
`endpointingMs`	`endpointing_ms`	—	Silence (ms, `[0, 5000]`) before an utterance is final. Unset keeps the xAI default (10); `0` = any VAD boundary.
`smartTurn`	`smart_turn`	—	Smart Turn end-of-turn confidence threshold `[0.0, 1.0]`. Unset leaves it off.
`smartTurnTimeoutMs`	`smart_turn_timeout_ms`	—	Max silence (ms, `[1, 5000]`) before forcing `speech_final` when Smart Turn is on.
`keyterms`	`keyterms`	`[]`	Up to 100 bias terms (≤50 chars each).
`diarize`	`diarize`	`false`	Speaker diarization — words gain a `speaker` index.
`fillerWords`	`filler_words`	`false`	Include “uh” / “um” in the transcript.
`baseUrl`	`base_url`	xAI STT WS endpoint	Override for proxying or tests.

Batch transcription

For a one-shot file or URL (rather than a live stream), use the batch helper. Pass either raw audio bytes or a url for xAI to download server-side. Container formats (WAV/MP3/OGG/Opus/FLAC/AAC/MP4/M4A/MKV) are auto-detected; for raw headerless audio pass audioFormat (pcm / mulaw / alaw) and sampleRate. Set format: true (with language) for Inverse Text Normalization (“one hundred dollars” → “$100”).

import { readFile } from "node:fs/promises";
import { xaiTranscribe } from "getpatter";

// From bytes
const audio = await readFile("call.wav");
const result = await xaiTranscribe({ audio, language: "en", format: true });

// Or from a URL, with word timestamps + diarization
const fromUrl = await xaiTranscribe({
  url: "https://example.com/recording.mp3",
  diarize: true,
});

console.log(result.text, result.duration);
for (const word of result.words) {
  console.log(word.text, word.start, word.end, word.speaker);
}

from getpatter import xai_transcribe

# From bytes
with open("call.wav", "rb") as f:
    audio = f.read()
result = await xai_transcribe(audio, language="en", format=True)

# Or from a URL, with word timestamps + diarization
from_url = await xai_transcribe(
    url="https://example.com/recording.mp3",
    diarize=True,
)

print(result.text, result.duration)
for word in result.words:
    print(word.text, word.start, word.end, word.speaker)

The helper returns a parsed result — XaiTranscription { text, language, duration, words } — where each XaiWord carries text, start, end, and (when diarize is on) speaker.

Low-level usage

The pipeline-mode wrapper adds XAI_API_KEY resolution on top of the underlying provider. To use the provider directly, import it and pass the API key as the first argument:

import { XaiSTT as LowLevelSTT } from "getpatter/providers/xai-stt";

const stt = new LowLevelSTT("<api-key>", { language: "en", smartTurn: 0.7 });
await stt.connect();
stt.onTranscript((t) => console.log(t.text, t.isFinal, t.speechFinal));

What’s Next

STT

All STT providers side by side.

xAI TTS

Grok text-to-speech.

xAI Realtime

Grok Voice Agent engine.

Metrics

Cost tracking and rates.

Get Started

Setting up Patter

Observability

Integrations

Development

xAI STT

xAI STT

Install

Authentication

Usage

Streaming transcript semantics

Smart Turn end-of-turn detection

Keyterms and diarization

Languages

Encodings and sample rate

Rates

Options

Batch transcription

Low-level usage

What’s Next

STT

xAI TTS

xAI Realtime

Metrics

​xAI STT

​Install

​Authentication

​Usage

​Streaming transcript semantics

​Smart Turn end-of-turn detection

​Keyterms and diarization

​Languages

​Encodings and sample rate

​Rates

​Options

​Batch transcription

​Low-level usage

​What’s Next

STT

xAI TTS

xAI Realtime

Metrics

xAI STT

Install

Authentication

Usage

Streaming transcript semantics

Smart Turn end-of-turn detection

Keyterms and diarization

Languages

Encodings and sample rate

Rates

Options

Batch transcription

Low-level usage

What’s Next