leaderboard
A public ranking of models by their scores on shared benchmark tasks.
In more detail
Leaderboards such as MTEB are a useful shortlist, but the tasks rarely match yours and popular benchmarks can leak into training data. Use them to choose candidates, then decide with an eval set built from your own data.
Related terms
- MTEB — The Massive Text Embedding Benchmark, a public set of tasks and a Hugging Face leaderboard for comparing embedding models.
- AI eval — A repeatable test of an AI feature: a set of inputs, a way to grade each output, and a score you track across changes.
- benchmark — A repeatable measurement of how long code takes, or how much it uses, under a fixed workload, or more broadly a standard point of comparison such as an industry average.
Taught in
- Choosing A Model — AI Engineering: LLM Apps, RAG & Agents
- Embedding Models — AI Engineering: LLM Apps, RAG & Agents
- Eval-Driven Development — AI Engineering: LLM Apps, RAG & Agents
References
- Hugging Face: MTEB leaderboard
- Zheng et al. (2023): Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena