multimodal model
A model that accepts or produces more than one kind of data, such as text, images and audio.
In more detail
Images and audio travel as content blocks in the same request as text and are priced in tokens too. Check generated media with an independent signal, because the model cannot reliably grade its own pictures or speech.
Related terms
- image input — Sending images to a multimodal model as content blocks, which it reads and bills as tokens set by the image size.
- speech to text — Converting spoken audio into text, often with a timestamp for each word.
- text to speech — Generating spoken audio from text with a voice model.
Taught in
- Images, Audio And Speech — AI Engineering: LLM Apps, RAG & Agents