Why We Released Laravel AI Batch

Batch APIs are a good fit for a lot of AI work. Classification, enrichment, evaluations, and other background jobs do not always need an answer immediately. If they can wait up to 24 hours, OpenAI, Anthropic, and OpenRouter offer those requests at roughly half the per-token price.

The problem was that laravel/ai did not have a way to use those APIs. It also did not expose the request body it would send, which makes building a separate batch integration surprisingly easy to get wrong. A second implementation can drift from the SDK whenever prompts, tools, structured output, or provider options change.

That is why we released Laravel AI Batch.

There was already a package called refinephp/laravel-ai-batch, which helped prove that this was a useful problem to solve. We decided to release our own implementation because we wanted broader provider support and results that come back as the same response objects used by laravel/ai, rather than provider-specific payloads that each application has to map itself.

The package resolves an existing agent into the exact request the SDK would have sent, submits those requests to a provider’s batch API, and turns the results back into the same AgentResponse and StructuredAgentResponse objects as the synchronous path. It also handles storing request context, polling, partial failures, and testing with fakes.

The difference is easiest to see in code. A normal laravel/ai prompt is immediate:

$response = GenerateSummary::make()->prompt($prompt);

With refinephp/laravel-ai-batch, the agent and prompt are added through its batch facade:

$batch = AiBatch::forProvider(Lab::OpenAI)
    ->agent(GenerateSummary::class)
    ->add('summary-1', prompt: $prompt)
    ->submit();

summary-1 is an application-defined, unique custom batch ID for correlating the result with the original input. It can be any meaningful string, such as a post ID, invoice number, or database key. These custom batch IDs are only the package’s wrapper around the custom_id values required by the provider’s batch API specification; they give the application a convenient name while preserving the provider’s correlation mechanism.

With our package, the agent can resolve its request first and submit it through Batch. The result can then be read as the same response type as a normal prompt:

$request = (new GenerateSummary)->resolve($prompt, provider: Lab::OpenAI);

$batch = Batch::of(['summary-1' => $request])->submit();
$results = $batch->results();

The first call is synchronous. The other two use a provider batch API. In both batch examples, the custom batch ID becomes the provider-required custom_id used to match each result to its original request. The important difference in the last example is that request resolution and result parsing stay inside the Laravel AI gateway path, instead of requiring application code to map provider JSON back into Laravel AI responses.

The important design choice is that it does not replace or duplicate the SDK’s request and response mapping. It works through the SDK’s own gateway code instead. That keeps a batch request aligned with a normal request.

Installation is just:

composer require pietervanleuven/laravel-ai-batch

The package currently supports OpenAI, Anthropic, and OpenRouter. Other providers can be added through the same gateway contract when their batch APIs make sense.

Read the documentation and examples on GitHub.