prompt caching

Reusing a provider's stored work on an identical prompt prefix, so repeated calls cost less and start answering sooner.

In more detail

Anthropic charges 1.25 times the input price to write a five-minute cache entry and a tenth of it to read one; OpenAI caches long prefixes automatically. Only an exact prefix matches, so put tools, instructions and reference text first and per-request details such as the time, user and question last.

Related terms

Taught in

References

Certification courses