Title: Week 15 Resources
Course: AIINFRA 301: Production RAG & LLMOps — Observability and Evaluation > Section: Week 15 — Production Monitoring: Drift Detection, Feedback Loops, Cost, and Latency

📚 Resources

- Evidently AI for drift detection and LLM monitoring: https://github.com/evidentlyai/evidently
- 2026 production monitoring stack pattern: https://valuestreamai.com/blog/ai-monitoring-in-production-guide-2026
- Evidently descriptors for LLM output monitoring: https://www.evidentlyai.com/
- Evidently: monitor embedding drift in production: https://learn.evidentlyai.com/ml-observability-course/module-3-ml-monitoring-for-unstructured-data/monitoring-embeddings-drift
- Embedding/prompt drift: detect distribution shift via centroid cosine distance: https://insightfinder.com/blog/hidden-cost-llm-drift-detection/
- Track cost and latency with token-level and percentile metrics: https://www.braintrust.dev/articles/what-is-llm-monitoring

Week 16 · Capstone Project & Course Review

Notion ID: 392c08fd-0278-8112-bd10-cb39bf5b8e81 Notion URL: https://app.notion.com/p/392c08fd02788112bd10cb39bf5b8e81