MLOps
The practices for deploying, monitoring and updating machine learning models in production, much as DevOps does for software.
In more detail
For applications built on hosted language models, the LLMOps side is mostly versioned prompts, pinned model snapshots, evals in CI, tracing and cost tracking rather than training pipelines. The aim is the same: changes reach users only after automated checks, and production is watched.
Related terms
- AI eval — A repeatable test of an AI feature: a set of inputs, a way to grade each output, and a score you track across changes.
- observability — How well you can understand what a system is doing, and why, from the data it emits: mainly logs, metrics and traces.
- model snapshot (LLM) — A fixed, dated version of a hosted model, as opposed to an alias that the provider moves to newer versions.
Taught in
- What An AI Engineer Does — AI Engineering: LLM Apps, RAG & Agents
References
- roadmap.sh: AI Engineer roadmap
- Google Cloud: MLOps, continuous delivery and automation pipelines in machine learning