chunk (retrieval)
A passage cut from a document for indexing, and the unit that retrieval returns and the prompt includes.
In more detail
Split along the document's own headings, keep tables and lists whole, pack blocks up to a token budget and overlap neighbours a little. Each chunk should make sense alone, which is why many pipelines prepend the heading path or a short context.
Related terms
- document ingestion — The offline stage of RAG that splits documents into chunks, embeds them and stores them with metadata in a search index.
- contextual retrieval — Anthropic's technique of having a model write a short note placing each chunk in its document, and prepending it before embedding and keyword indexing.
- heading path — The chain of headings above a chunk, such as "Billing > Refunds > Annual plans", prepended to it so it makes sense on its own.
Taught in
- What An Embedding Is — AI Engineering: LLM Apps, RAG & Agents
- Embedding Models — AI Engineering: LLM Apps, RAG & Agents
- Vector Indexes And Databases — AI Engineering: LLM Apps, RAG & Agents
- Measuring Retrieval — AI Engineering: LLM Apps, RAG & Agents
- How RAG Works And When To Use It — AI Engineering: LLM Apps, RAG & Agents
- Chunking And Ingestion — AI Engineering: LLM Apps, RAG & Agents
- Retrieval That Finds The Right Thing — AI Engineering: LLM Apps, RAG & Agents
- Grounded Answers With Citations — AI Engineering: LLM Apps, RAG & Agents
- Evaluating RAG And Agents — AI Engineering: LLM Apps, RAG & Agents
- Capstone: The Tasks Assistant — AI Engineering: LLM Apps, RAG & Agents