Ollama
An open-source tool that downloads open-weight models and serves them on your own machine through a local API.
In more detail
Ollama exposes an OpenAI-compatible endpoint on localhost, so code written for a hosted API can call a local model by changing the base address. It suits offline work, private data and cheap experiments, at the cost of smaller models and your own hardware.
Related terms
- open-weight model — A model whose trained weights, the learned numbers that define it, are published so anyone can download and run it under its licence.
- LM Studio — A desktop application for downloading and running open-weight models locally, with a built-in chat and a local server.
- OpenAI-compatible API — An API that accepts the same request and reply shapes as OpenAI's, so existing client code and libraries work against another model server.
Taught in
- Choosing A Model — AI Engineering: LLM Apps, RAG & Agents
- The Request And The Response — AI Engineering: LLM Apps, RAG & Agents