neural network
A model built from layers of simple numeric units whose connection weights are learned from data.
In more detail
Modern language models are transformer neural networks: each layer uses attention to weigh every earlier token when computing the next representation. Training sets billions of weights, and those weights are what an open-weight release publishes.
Related terms
- large language model — A neural network trained on huge amounts of text that generates text by predicting one token at a time from everything before it.
Taught in
- From Prompts To Context — AI Engineering: LLM Apps, RAG & Agents
- Keyword And Hybrid Search — AI Engineering: LLM Apps, RAG & Agents
References
- Vaswani et al. (2017): Attention Is All You Need
- Google Machine Learning Crash Course: Neural networks