Tracing AI Workflows

Course overview

Tracing AI workflows: instrument model calls (provider, model, tokens, latency), trace RAG retrieval and reranking steps, capture tool calls and agent loops, and apply redaction to protect privacy in traces.

Instrument and log model calls with provider, model version, token usage and latency metadata.

Trace RAG pipeline steps—retrieval and reranking—to establish provenance and relevance diagnostics.

Capture tool invocations and agent loops to detect unwanted behavior or runaway interactions.

Redaction techniques and best practices to protect privacy and limit leakage of sensitive data in traces.