GenAI span
A trace span for one model call, named after the operation and model, such as "chat claude-sonnet", carrying the GenAI convention attributes.
In more detail
Record the operation, provider and requested model on every call; add the response id, finish reasons and token usage on success, and an error type on failure. Exported spans give latency percentiles, cost per model and error rates, and feedback can be linked by response id.
Related terms
- GenAI semantic conventions — OpenTelemetry's standard attribute names, spans, metrics and events for generative AI calls, such as gen_ai.operation.name and gen_ai.usage.input_tokens.
- span — One timed operation within a trace, such as handling a request or running a query, with its start, duration and attributes.
- trace — A record of one request’s whole journey through a system, made of nested spans that show where the time went.
- time to first token — The delay between sending a request and receiving the first token of the reply, the latency a streaming user feels first.
Taught in
- Tracing And Monitoring — AI Engineering: LLM Apps, RAG & Agents
- Capstone: The Tasks Assistant — AI Engineering: LLM Apps, RAG & Agents
References
- OpenTelemetry GenAI conventions: Semantic conventions for generative client AI spans
- OpenTelemetry GenAI conventions: Metrics