Groq

Groq file-backed chat batches in Batchwork with OpenAI-compatible result files and the unified typed job lifecycle.

Groq uses an OpenAI-compatible file-backed chat batch. Batchwork builds JSONL, uploads it, creates the batch, then streams output and error files through the shared normalized result model.

At a glance

Example

from batchwork import BatchRequest, Batchwork

async with Batchwork() as client:
    job = await client.batch(
        model="groq/llama-3.3-70b-versatile",
        requests=[
            BatchRequest(custom_id="a", prompt="Summarize the first document."),
            BatchRequest(custom_id="b", prompt="Summarize the second document."),
        ],
    )

Lifecycle

1. Requests become OpenAI-shaped JSONL lines.
2. The serializer initially produces /openai/v1/chat/completions; the Groq adapter normalizes the batch endpoint to /v1/chat/completions.
3. Batchwork uploads through /files and creates /batches with a 24-hour completion window and optional metadata.
4. poll() retrieves the batch object.
5. results() streams output and error files.
6. cancel() calls the provider cancel route.

Options and translation

Groq-specific options include parallelToolCalls, reasoningFormat, reasoningEffort, and serviceTier. Assistant ReasoningPart content is emitted as Groq’s reasoning field. Common messages, function tools, tool choice, and request settings use the OpenAI-compatible chat shape. Unsupported arbitrary Groq options are not passed through.

Exact CLI text provider-option keys are parallelToolCalls, reasoningEffort, reasoningFormat, serviceTier, and user. Only chat-completions is a valid explicit --endpoint. Unknown keys fail locally; parallelToolCalls must be boolean and the remaining keys are strings. Canonical top_k is unsupported. Batch metadata is supported and forwarded. Package hard bounds are 50,000 requests, 20 MiB per serialized request, and 200 MiB aggregate, subject to lower Groq/model limits.

Audio transcription and translation

from batchwork import BatchTranscriptionRequest

job = await client.batch_transcriptions(
    model="groq/whisper-large-v3",
    requests=[
        BatchTranscriptionRequest(
            custom_id="call-1",
            audio_url="https://example.com/call-1.mp3",
            timestamp_granularities=["segment"],
        )
    ],
)

Groq audio bodies carry url; timestamp_granularities also sets response_format: "verbose_json". audio_url must be a hosted URL the provider can fetch. batch_translations works identically minus language (English output only). See Transcription and Translation.

Image input

Groq text batches can consume supported image inputs. Other file media types are rejected during serialization. “Image generation: No” means Groq is not exposed through batch_images(); it does not mean vision input is unavailable.

Results

Text, usage, and item errors are normalized through the OpenAI-compatible result parser. Provider-specific fields remain in result.response. Batch metadata is forwarded when creating the provider batch.

Groq does not support embeddings or image-generation output through Batchwork.

Official references

• Groq Batch API
• Groq vision

See Jobs, Results, and Text and messages.