retrieval-augmented generation
Answering a question by first retrieving relevant passages from your own documents and adding them to the prompt, so the model answers from them.
In more detail
Lewis and colleagues named the approach in 2020. It has three stages, built and tested apart: ingestion splits, embeds and stores documents ahead of time; retrieval finds the best chunks per question; generation answers only from them, with citations. It suits large, changing or permissioned knowledge that will not fit or should not sit in every prompt.
Related terms
- document ingestion — The offline stage of RAG that splits documents into chunks, embeds them and stores them with metadata in a search index.
- chunk (retrieval) — A passage cut from a document for indexing, and the unit that retrieval returns and the prompt includes.
- grounding — Making a model answer only from supplied sources, cite them, and refuse when they do not contain the answer.
- fine-tuning — Further training an existing model on your own examples to change its behaviour, style or format.
Taught in
- Measuring Retrieval — AI Engineering: LLM Apps, RAG & Agents
- How RAG Works And When To Use It — AI Engineering: LLM Apps, RAG & Agents
- Grounded Answers With Citations — AI Engineering: LLM Apps, RAG & Agents
- Evaluating RAG And Agents — AI Engineering: LLM Apps, RAG & Agents
References
- Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
- Anthropic: Introducing Contextual Retrieval