Patter is the inbound front door; OpenClaw is the brain
The clearest way to think about this integration: Patter owns the carrier leg and the voice, OpenClaw owns the reasoning and the business tools. Patter terminates a real phone number, talks to the caller with low-latency speech, and only reaches back into a single scoped OpenClaw agent when it needs real data or an action. This split exists for a concrete reason. OpenClaw’s native voice-call plugin is built for a personal assistant, not a public line: its voiceinboundPolicy is allowlist-only
and disabled by default — unknown callers are rejected unless their number is pre-listed.
That is the right default for a private bot and the wrong one for an after-hours business
line that anyone might call. Patter has no such restriction: it answers the carrier leg
for any caller and verifies the carrier signature (X-Twilio-Signature for Twilio,
Ed25519 for Telnyx) at the public boundary. The caller reaches the receptionist agent over
the trusted operator-side gateway path instead. If today you forward calls around
OpenClaw’s allowlist with carrier tricks, that hack goes away.
There are three directions, and they compose:
- Direction A — OpenClaw places calls via Patter. Connect the
patter-mcpserver so your OpenClaw agent can dial, wait for a call to finish, and read back the transcript as MCP tools. Blocking-until-the-call-ends is the desired semantics here. - Direction B — Patter consults OpenClaw mid-call. Point Patter’s
consultfeature at a small adapter so an in-call Patter agent can escalate a hard turn to one specific OpenClaw agent over HTTP. This call must stay fast — it runs while the caller waits on the line. - Direction C — OpenClaw is the in-call LLM (voice shell). Patter answers the
carrier leg and drives speech, and routes every turn to one scoped OpenClaw agent as
the LLM, using the new
OpenClawLLMpipeline provider. No adapter; the caller is talking to OpenClaw the whole time. Stand it up in one command with thepatter openclawwizard — an inbound receptionist or an outbound dialer, both driven by the same scoped agent.
Which direction for what. Direction A (
make_call / call_third_party with
wait: true) is OpenClaw-initiated outbound — it correctly blocks for minutes while
the call runs, so raise the MCP timeout to cover the whole call. Direction B
(consult) is in-call escalation — the caller is on the line, so keep it fast and
push slow work down (see Surviving long tool calls).
Don’t hold a consult HTTP request open for 60 s of silence; don’t expect a
make_call to return in 5 s.Prerequisites
Patter’s MCP server runs locally — it is not published to npm or PyPI. Clone and build it fromPatterAI/patter-mcp:
OPENAI_API_KEY, TWILIO_ACCOUNT_SID,
TWILIO_AUTH_TOKEN, TELNYX_API_KEY, …) are read from the server’s environment.
See the patter-mcp README for full setup.
For the voice-shell direction (Direction C) you only need the SDK — no MCP server:
OpenClaw as the primary LLM (voice shell)
In Direction C, OpenClaw is not consulted on demand — it is the in-call LLM. Patter owns the carrier leg and speech (STT, turn-taking, barge-in, TTS) and routes every turn to one scoped OpenClaw agent through the newOpenClawLLM pipeline provider. The caller is
talking to OpenClaw the whole call. Use this when most turns need the agent’s full
reasoning; use Direction B instead when a
local in-call agent handles ordinary turns and only escalates the hard ones.
OpenClawLLM aligns byte-for-byte with the shipped consult OpenClaw preset
(ConsultConfig.openclaw): same gateway base url
(http://127.0.0.1:18789/v1), same OPENCLAW_API_KEY env var, same
model="openclaw/<agent>" agent-target convention and id charset rule, and the same
x-openclaw-session-key session header. The only difference is the timeout default: the
LLM provider defaults to 120 s (the runtime is the per-turn brain) where the consult
preset defaults to a phone-safe 30 s (the runtime is an on-demand escalation).
Zero-config CLI (patter openclaw)
The fastest path is the patter openclaw wizard. It scaffolds a runnable
openclaw-phone-agent project, enables the gateway’s OpenAI-compatible endpoint
(disabled by default — backed up first), and, for inbound, wires the Twilio
webhook. Because OpenClaw is multi-agent, the wizard lists the agents your
gateway serves and warns if you point the phone at the default / master
agent. The same scoped agent powers both directions — Patter answers inbound
(app.py → phone.serve) and dials outbound (dialer.py → phone.call):
patter openclaw doctor runs preflight across the gateway (is the endpoint
enabled? which agents are served? is your --agent one of them — and not the
default/master?), the Patter providers, the carrier, and a day-1 security
posture (loopback bind, a strong OPENCLAW_API_KEY, a scoped agent, plus
tool-policy and AI-disclosure / data-residency reminders). patter openclaw test
adds a real /v1/chat/completions turn; patter openclaw agents just lists the
roster.
The scaffold ships deploy/ units (systemd + launchd) so app.py / dialer.py
run 24/7 under the OS service supervisor, alongside the OpenClaw gateway
daemon and a stable named tunnel — see
Production deployment for the always-on box
model. The rest of this section is what the wizard automates, if you prefer to
wire it by hand.
Enable the gateway and pick the agent
Enable OpenClaw’s OpenAI-compatible endpoint (it is disabled by default) in~/.openclaw/openclaw.json, and bind the gateway to loopback with a strong auth credential
— exactly as in Direction B:
OpenClawLLM at one explicit, least-privileged receptionist agent — never
the default / master agent. The agent id is validated and mapped to
model="openclaw/<agent>"; an already-namespaced id (openclaw/x, openclaw:x,
agent:x) is passed through unchanged, identical to the consult preset.
Python example
OpenClawLLM(agent="receptionist") reads the operator-grade bearer from OPENCLAW_API_KEY
(never logged), targets http://127.0.0.1:18789/v1, and sends the call id as both the
OpenAI user field (patter-call-<call_id>) and the x-openclaw-session-key header — one
OpenClaw session per phone call. Override any default explicitly:
TypeScript example
Generic OpenAI-compatible runtimes (Ollama / vLLM / LM Studio).
OpenClawLLM is a
thin preset over the generic OpenAICompatibleLLM provider, which drives any
OpenAI-compatible chat endpoint. To put a local model server on the phone instead of
OpenClaw, use it directly:base_url and model are required; timeout defaults to 60 s. Keyless gateways
send no Authorization header. For a remote endpoint that needs a key, pass api_key or
set api_key_env to the env var holding it. See the Hermes page
for the full voice-shell walkthrough (carrier setup, security notes, production
deployment) — it applies to any of these runtimes.Direction A — OpenClaw places calls via Patter
OpenClaw reads MCP servers from~/.openclaw/openclaw.json (JSON5) under
mcp.servers.<name>. Add a patter server using either transport below.
Option 1 — Streamable HTTP (server already running)
This connects to the locally-runningpatter-mcp server from the prerequisites (npm start).
Option 2 — stdio (OpenClaw launches the server)
Let OpenClaw spawn the server itself over stdio, pointing at your local build (patter-mcp/dist/index.js from the clone above). The --stdio flag switches it
from the default HTTP transport to stdio.
The seven tools
| Tool | What it does |
|---|---|
make_call | Place an outbound call. With wait: true it blocks until the call ends and returns the outcome plus transcript. |
call_third_party | Bring a third party onto an existing call. |
get_calls | List recent calls and their status. |
get_transcript | Fetch the transcript for a call. |
end_call | Hang up an in-progress call. |
get_metrics | Read latency / token / cost metrics. |
configure_inbound | Configure how inbound calls are answered. |
toolFilter.include examples above narrow this to the five tools the phone path
actually uses. get_metrics and configure_inbound are admin-ish surfaces — expose them
only to an operator agent, not the voice path.
Two gotchas
Example: place a call and read the result
Once the server is connected and allowlisted, ask your OpenClaw agent:“Call the clinic at +15555550100 and ask whether they have any openings this Friday afternoon. Wait for the call to finish and tell me what they said.”The agent invokes
make_call with wait: true. Patter dials, runs the
conversation, and the tool returns the call outcome and full transcript in one
response, which the agent summarizes back to you:
Direction B — Patter consults OpenClaw mid-call
In this direction Patter conducts the call (speech-to-text, the voice agent, text-to-speech, and the carrier) and OpenClaw stays off the per-turn path. When the in-call agent hits a turn it can’t answer, Patter’sconsult tool POSTs the request to
an HTTP endpoint you host, and speaks the reply. See the consult feature pages for the
Python SDK and the TypeScript SDK.
This is the architecturally correct shape for an external brain on a voice call, and it
is the inverse of the path that fails — see
Why this beats the custom-LLM path.
OpenClaw exposes an OpenAI-compatible HTTP API at POST /v1/chat/completions. It is
disabled by default; enable it in ~/.openclaw/openclaw.json:
model field is overloaded to name the agent target, not
a provider model — this is how you route the call to one specific agent (next section).
Native target (recommended): no adapter
You don’t need to host an adapter. Patter’sconsult speaks OpenClaw’s
/v1/chat/completions endpoint natively — point it at one scoped agent in a single
line:
model="openclaw/receptionist" on the local gateway, sends the call id as
the OpenAI user field + the x-openclaw-session-key header (one OpenClaw session per
call), reads the operator-grade bearer from OPENCLAW_API_KEY (never logged), auto-enables
loopback for the co-located gateway, and attaches a default “let me check” reassurance
filler. Full reference + overridable defaults:
Python ·
TypeScript.
Notify OpenClaw at call end. Wire openclaw_post_call_notifier("receptionist")
(Python) / openclawPostCallNotifier('receptionist') (TypeScript) on
serve(on_call_end=...) to POST the finished call’s record (caller, dialed line, duration,
transcript) to the same agent and session, so the brain keeps the record and can follow up.
The rest of this section explains the agent-target routing and security model the preset
applies for you, plus a hand-written adapter — the escape hatch when you need a custom
request/response mapping (e.g. a non-OpenClaw back office).
Route the call to a SPECIFIC OpenClaw agent (security isolation)
The single most important configuration choice in this integration: the voice path must target one explicit, least-privileged receptionist agent — never the default/master agent. On/v1/chat/completions, the model field selects which OpenClaw agent answers:
"openclaw"or"openclaw/default"resolves to whatever the default agent is — which can be your master / financial / admin agent, and can drift between environments. Do not use this from the voice path."openclaw/<receptionistAgentId>"(commonly also accepted as"openclaw:<id>"/"agent:<id>") targets one named agent. This is what you want. The exact agent-target syntax is OpenClaw-version-specific — confirm it against the OpenClaw docs before going live.
Carry the agent id and per-call context into the adapter
Patter’s consult tool already sends{ request, call_id, caller, callee } on every
escalation. Use them: forward the agent target, give the call one stable OpenClaw session,
and pass the caller id so the receptionist can prefetch the customer record.
timeout_s to the real worst-case duration of
the work the consult triggers — see Surviving long tool calls:
Deploying with a tunnel? When you expose this stack to the internet, expose only the
carrier webhook path — keep the OpenClaw gateway and the consult adapter on loopback, and
secure the dashboard. See Production Deployment for
named-tunnel ingress rules and dashboard hardening.
Surviving long tool calls
This is the part that breaks most voice + agent integrations: a tool / lookup that takes 30–60 s (a calendar reschedule, a customer-record fetch, a short browser-automation step) runs while the caller is on the line. Done wrong, the line goes silent and the call drops. Done right, the agent says “let me check, one moment”, keeps talking naturally, and speaks the result when it returns.The carrier won’t drop you — your timeouts will
Twilio and Telnyx media-stream WebSockets have no idle timeout: inbound audio frames keep flowing even during silence, and the stream only closes when the call ends. So a call that “dies during a long tool” is never the carrier — it’s a timeout somewhere above it giving up. The fix is therefore two parts: (1) never go silent, and (2) raise every timeout in the chain so the shortest link is longer than the work.Acknowledge, then keep the conversation alive
Speak a filler the instant the long turn starts, then speak the result on return. Patter gives you two complementary, Realtime-mode primitives for this (today the consult / long-tool reassurance path is Realtime-only; in Pipeline mode the LLM generates the filler itself and there is no separate injection point yet):- Reassurance — a single filler line the agent speaks if a tool hasn’t returned within a grace window (default ~1.5 s), cancelled automatically if the tool returns earlier.
- Streaming progress — write a tool handler as an async generator and
yield{ "progress": "..." }updates; each is spoken inline so the caller hears live status (“still checking…”) instead of dead air on a long lookup.
On OpenAI Realtime, always trigger a response after a tool result. After you inject
a tool / consult result, the model does not automatically speak — the SDK sends the
follow-up response trigger for you on the built-in tool path, but if you build a custom
handler that defers a result, make sure the result is delivered through the SDK’s tool
return so the agent actually speaks the answer instead of going mute.
The timeout ladder
The call’s tolerance for a long turn is the shortest link in this chain. Raise all of them together — a generous Patter timeout is wasted if OpenClaw gives up at 20 s:| Layer | Where | Set it to |
|---|---|---|
| Patter consult | ConsultConfig.timeout_s / timeoutMs (default 30 s) | 75–90 s — above the worst-case OpenClaw turn |
| Consult adapter HTTP | your adapter’s HTTP client (the FastAPI example defaults to 30 s) | match the consult timeout (90 s) |
| OpenClaw gateway | /v1/chat/completions request timeout | raised to cover the turn |
| OpenClaw MCP (Direction A) | mcp.servers.patter.timeout (seconds) | 600 s for blocking make_call |
consult path, 75–90 s is the sweet spot: long enough for a 60 s
browser-automation step plus margin, short enough that a truly stuck turn fails gracefully
(the consult tool returns a spoken “I wasn’t able to reach the system right now” rather
than hanging the call).
Per-tool timeouts on custom tools (TypeScript)
If you escalate via a custom@tool / defineTool rather than consult, the generic
tool-execution timeout defaults to 10 s — far too short for a 30–60 s job, and the
literal cause of “the call died on the tool call”. In TypeScript, raise it per tool with
timeoutMs (clamped to a 300 s ceiling) and pair it with reassurance:
consult (its timeout_s / timeoutMs is the
equivalent knob and it already carries headers and the call context). When you only need to
trigger work and the caller doesn’t have to wait for the result, return quickly from the
consult and let the receptionist offload the slow part to an OpenClaw sub-agent, so the
HTTP request itself stays short.
Stay under provider ceilings — chunk, don’t block silently
Across the industry, voice platforms cap tool execution and recommend the same two moves: acknowledge before the tool runs, and chunk a long wait into spoken intervals rather than one silent block. The numbers vary (a 10 s generic default here, a 20 s default there, a 120 s ceiling elsewhere), but the rule is constant: set the ceiling above the work, and never let the line go silent for more than a few seconds. Patter’s reassurance + progress-streaming primitives are how you honour the second half of that rule; the timeout ladder above is the first half.Noise on speakerphone
Contractors call from job sites and offices on speakerphone, and tiny transients — a mouse move, a phone shifting on a desk — can false-trigger turn detection and cut the agent off mid-sentence. Noise robustness is three independent layers, tuned in order:- Clean the input signal before the VAD sees it.
- Raise the turn-start threshold so transients don’t count as the caller speaking.
- Make barge-in less twitchy so a brief noise doesn’t abort the agent’s playback.
Realtime mode (recommended for low-latency English)
On the OpenAI Realtime engine, set far-field noise reduction and tune turn detection. In TypeScript these are first-class on the engine marker (and on the agent options):barge_in_threshold_ms,
echo_cancellation). The first-class far_field / threshold engine-marker fields are
the recommended preset for speakerphone callers; they ship with the noise-reduction /
turn-detection work on the TypeScript engine marker — confirm your installed getpatter
exposes noiseReduction / turnDetection before relying on them, and fall back to the
universal levers below (which work in every release) otherwise.
Pipeline mode and the universal levers
Two levers work in both SDKs regardless of mode and are your front line for the mouse-move cut-off:barge_in_threshold_ms(bargeInThresholdMs) — minimum sustained voice before caller audio counts as a barge-in. Default 300 ms; raise to ~500 ms on a noisy line so a brief transient doesn’t interrupt the agent. See Barge-in sensitivity.echo_cancellation(echoCancellation) — pipeline mode only; subtracts the agent’s own TTS bleed from the mic feed before VAD/STT see it. Strongly recommended for speakerphone and dev-tunnel deployments where the agent can otherwise hear itself. See Echo cancellation.
server_vad preset — see the Silero VAD provider docs.
Reference architecture: after-hours receptionist for a contractor
This is the end-to-end shape for a contractor’s after-hours line — anyone can call, the agent survives long lookups, and the voice path only ever touches one scoped agent.- A homeowner calls the contractor’s published number after hours. Patter answers — no allowlist, any caller reaches the receptionist.
- The caller asks to reschedule Tuesday’s roof inspection. The agent says “Let me check the calendar for you, one moment.”
- Patter’s
consultPOSTs to the local adapter, which targetsmodel: "openclaw/receptionist-roofing-ca"withuser = call_idand the caller id. - The receptionist agent looks up the appointment in D-Tools Cloud (prefetched on the caller-ID match), reschedules via the D-Tools API — offloading the slower browser step to a scoped sub-agent so the consult returns promptly — and returns a short confirmation.
- The adapter trims the reply; Patter speaks “You’re all set — your roof inspection is moved to Thursday at 10 a.m.” The voice path never touched the master / financial agent.
Production deployment
Each client buys an always-on Mac Mini that stays powered in their office (or in your server room); you remote in to set it up. This hardware model gives you the isolation the security model needs for free: one client = one machine = one OS user = one OpenClaw gateway = one credential set. On that box you run, all co-located:- The Patter SDK serving the agent on the Twilio DID.
- The OpenClaw gateway, bound to loopback with
/v1/chat/completionsenabled and a strong auth credential — never exposed to the public internet. - Patter’s
consultpointed at the gateway over loopback —ConsultConfig.openclaw(...)auto-enablesallow_loopback(or a hand-written adapter on loopback for a custom mapping). - The receptionist agent (and the master agent, if any) as separate top-level
agents.list[]entries.
*.trycloudflare.com hostname that rotates on every restart/reboot — which breaks a
carrier webhook pinned to a fixed URL. For an always-on deployment, run a named
Cloudflare Tunnel with a stable per-client hostname (or your own production webhook_url
behind a reverse proxy) and run cloudflared as a service (launchd) so it survives reboots
and reconnects on drop. Pass it to Patter via the static-tunnel / webhook_url option, not
the default quick tunnel.
Everything else (gateway, consult, MCP) stays private on loopback / tailnet. Per-client
acceptance gate before going live: confirm the receptionist’s effective tool list can
reach the calendar / customer-DB tools and cannot reach exec / financial tools, confirm
the gateway is bound to loopback, confirm the dashboard is disabled or token-protected, and
run one full long-tool-call end to end.
Why this beats the custom-LLM path
If you previously tried wiring an external agent into a hosted voice platform as its “Custom LLM” — pointing the platform at an OpenAI-compatible/v1/chat/completions
endpoint that is the brain — you likely hit big delays and dropped calls on anything
that did a 30–60 s tool call. That topology puts the brain on the critical
conversational path of every turn: the platform waits for the brain to finish reasoning
and run its tools before it can say anything, and there is no clean “acknowledge now,
finish the 60 s job, then continue” primitive. A long browser-automation step inside that
turn starves the stream and the call dies.
Patter inverts that:
- The brain is off the per-turn path. Patter’s local voice model owns turn-taking and
TTS and answers ordinary turns at low latency; OpenClaw is consulted on demand only,
via
consult(Direction B) or reached as a tool (Direction A). The 30–60 s lookup never blocks every turn — only the one turn that needs it, and that turn is covered by reassurance + the timeout ladder so the line stays alive. - Anyone can call. Patter terminates the carrier leg for any caller and verifies the
carrier signature at the boundary. OpenClaw’s native voice plugin can’t front arbitrary
inbound (voice
inboundPolicyisallowlist-only, disabled by default), so this removes the allowlist wall and the call-forwarding hack entirely.
What’s next
- Consult feature: Python · TypeScript.
- Long-running tools: reassurance + streaming progress — Python tools · TypeScript tools.
- Production deployment: securing the tunnel and dashboard.
- Concepts: read the Concepts page to pick a mode (Realtime / Pipeline) for the in-call agent.
- Get a phone number: Twilio console or Telnyx portal.
Need help?
- Patter SDK source: https://github.com/PatterAI/Patter
- Patter MCP server: https://github.com/PatterAI/patter-mcp
- Patter community (GitHub Discussions): https://github.com/PatterAI/Patter/discussions
- OpenClaw docs: https://docs.openclaw.ai

