batch API

An API for sending many requests to be processed asynchronously, usually within 24 hours, at about half the normal price.

In more detail

Batching suits work with no user waiting, such as nightly classification, evals or backfills. Anthropic's Message Batches and OpenAI's Batch API both discount by half, and the discount stacks with prompt caching.

Related terms

Taught in

References

Certification courses