streaming
Sending a response in pieces as they become available instead of building the whole thing first.
In more detail
Streaming lowers peak memory and lets the client start early, but it does not reduce total work. The client needs a framing rule to know where each record ends, and the contract must say what a broken connection means for a partial result.
Example
In the browser: Streaming lowers the time until the first useful output and keeps memory use flat, because no one needs to hold the whole payload at once.Related terms
- newline-delimited JSON — A streaming format in which every line is one complete JSON document, so each record can be parsed as soon as it arrives.
- framing — The rule that marks where one message or record ends and the next begins in a stream of bytes.
- Server-Sent Events — A standard way for a server to keep an HTTP response open and push a stream of text events to the client, in one direction only.
- backpressure — A slow consumer signalling a fast producer to pause, so data does not pile up in memory faster than it can be handled.
- ReadableStream — A browser object that delivers data in chunks as it arrives, such as a fetch response.body, instead of all at once at the end.
- chunk — One piece of a larger piece of data delivered by a stream.
Taught in
- The Request And The Response — AI Engineering: LLM Apps, RAG & Agents
- Streaming Responses — AI Engineering: LLM Apps, RAG & Agents
- Cost, Latency And Caching — AI Engineering: LLM Apps, RAG & Agents
- What MCP Is — AI Engineering: LLM Apps, RAG & Agents
- Remote Servers And Trust — AI Engineering: LLM Apps, RAG & Agents
- Tracing And Monitoring — AI Engineering: LLM Apps, RAG & Agents
- Large Payloads And Streaming — Modern API Architecture: REST, GraphQL & gRPC
- Server-Sent Events — Modern API Architecture: REST, GraphQL & gRPC
- When Each Wins: REST, GraphQL, gRPC And SOAP — Modern API Architecture: REST, GraphQL & gRPC
- The Fetch API: Streaming Responses, HTTP Caching And Keepalive — Frontend Architecture & Web Platform