Batch text

Use Batch V4 when a provider supports durable, asynchronous text generation. The returned TextBatchReference is persistable, so status and result reads can resume in another process without keeping the original request alive.

Anthropic Messages Batch, OpenAI Responses Batch, xAI Responses Batch, Google Generative AI Batch, and Gateway Batch V4 implement the shared adapter. Choose a batch-capable model from any of these providers:

let anthropic = try AIProviders.anthropic()
let model = try anthropic.messages("claude-sonnet-4-5")

let started = try await AI.startTextBatch(
    model: model,
    requests: [
        TextBatchRequest(
            id: "summary-1",
            request: LanguageModelRequest(messages: [
                .user("Summarize the release notes.")
            ])
        )
    ]
)

// Persist started.batch.reference if another process will resume the batch.
let status = try await AI.getBatchStatus(
    model: model,
    batch: started.batch.reference
)

OpenAI Responses uses the same facade and reference/result types:

let openAI = try AIProviders.openAI()
let openAIModel = try openAI.batchLanguageModel("gpt-5.6")

let started = try await AI.startTextBatch(
    model: openAIModel,
    requests: [
        TextBatchRequest(
            id: "summary-1",
            request: LanguageModelRequest(messages: [
                .user("Summarize the release notes.")
            ])
        )
    ]
)

xAI Responses exposes the same durable batch lifecycle:

let xAI = try AIProviders.xAI()
let xAIModel = try xAI.batchLanguageModel("grok-4")

let started = try await AI.startTextBatch(
    model: xAIModel,
    requests: [
        TextBatchRequest(
            id: "summary-1",
            request: LanguageModelRequest(messages: [
                .user("Summarize the release notes.")
            ])
        )
    ]
)

Google Generative AI selects inline requests for small batches and uploads a JSONL input file when the encoded request body crosses the inline limit:

let google = try AIProviders.google()
let googleModel = google.batchLanguageModel("gemini-3.8-flash")

let started = try await AI.startTextBatch(
    model: googleModel,
    requests: [
        TextBatchRequest(
            id: "summary-1",
            request: LanguageModelRequest(messages: [
                .user("Summarize the release notes.")
            ])
        )
    ]
)

Gateway language models also expose the shared batch facade:

let gateway = try AIProviders.gateway()
let gatewayModel = try gateway.languageModel("openai/gpt-5.6")

let started = try await AI.startTextBatch(
    model: gatewayModel,
    requests: [
        TextBatchRequest(
            id: "summary-1",
            request: LanguageModelRequest(messages: [
                .user("Summarize the release notes.")
            ])
        )
    ]
)

Starting a batch is not retried automatically because it may create billable work. Status and results setup use the normal retry policy. A caller may pass a stable idempotencyKey when the provider or gateway honors one.

AI.startTextBatch also accepts shared tools and toolChoice overlays. They are prepared with each request so batch providers receive the same function-tool content as unary generation.

Pass webhookURL to request a completion callback. Gateway and Google forward the URL through their native callback fields; direct Anthropic, OpenAI, and xAI adapters return an unsupported warning so the caller can fall back to status polling without silently assuming webhook delivery.

Results

getBatchResults is an async stream of independent terminal items. One failed, cancelled, or expired request does not terminate results for the other IDs:

let results = try AI.getBatchResults(
    model: model,
    batch: started.batch.reference
)

for try await item in results {
    switch item {
    case let .succeeded(id, generation):
        print(id, generation.text)
        print(generation.content)
        print(generation.rawFinishReason ?? "unknown")
        print(generation.providerMetadata)
    case let .failed(id, error, _):
        print(id, error.message)
    case let .cancelled(id, _, _), let .expired(id, _, _):
        print(id, "did not complete")
    }
}

Stopping result consumption cancels the active provider stream. Batch references are validated against provider and model identity before status or result I/O, preventing accidental cross-model reads. Provider-native request counts are normalized before they reach the shared status type. Start results also expose providerMetadata; OpenAI, xAI, and file-backed Google batches use it for uploaded input-file identifiers and expiry timestamps. OpenAI rejects a malformed outer result envelope before yielding any item, Google requires every result row to carry a nonempty request key, and xAI preserves the final assistant choice’s raw finish reason.

Result content is authoritative and can contain text, reasoning, files, sources, tool calls, and tool results. The text property remains a convenience projection. Google merges empty-text thought signatures onto the associated reasoning or tool part so they survive later model-message replay.