Groq LLM
GroqLLM plugs Groq’s OpenAI-compatible Chat Completions API at https://api.groq.com/openai/v1 into Patter’s pipeline mode. Groq’s LPU inference engine serves Llama models at very high throughput with low time-to-first-token, making it a strong pick when latency matters more than long-context reasoning.
The provider is a thin wrapper around the OpenAI Chat Completions client with a Groq-specific base URL — every OpenAI sampling option (responseFormat, parallelToolCalls, toolChoice, seed, topP, frequencyPenalty, presencePenalty, stop, temperature, maxTokens) is forwarded to chat.completions.create automatically.
Install
Usage
The namespaced import (
import * as groq from "getpatter/llm/groq" / from getpatter.llm import groq) auto-resolves the API key from GROQ_API_KEY and exposes a uniform LLM class — the same pattern Patter uses for STT and TTS namespaces.Supported models
Pricing in USD per 1M tokens. Availability depends on account tier — Groq’s free tier rate-limits more aggressively than the paid plans.
Models without listed rates are available on the API but aren’t yet pinned to a
LLM_PRICING entry — pass pricing overrides if your dashboard needs cost figures for them.
Environment variables
Options
Notes
- Groq returns the standard OpenAI Chat Completions stream shape, so tool calls, JSON mode, and seeded sampling all work without provider-specific code.
- Time-to-first-token on Groq’s LPU is typically < 200 ms for the 70B model and < 100 ms for the 8B model — well below most TTS startup latency.
- Long-context calls (32K+) use Mixtral; everything else fits comfortably in the Llama 3.3 context.

