Title: Week 12 Resources
Course: AIINFRA 303: Capstone Project > Section: Week 12 — Observability, Evaluation, and Cost Instrumentation

📚 Resources

- Langfuse for LLM observability, evaluation, and cost tracking: https://langfuse.com/
- OpenTelemetry for vendor-neutral LLM tracing: https://www.firecrawl.dev/blog/best-llm-observability-tools
- An Introduction to Observability for LLM-based Applications Using OpenTelemetry: https://opentelemetry.io/blog/2024/llm-observability/
- ml-monitoring reference repo (Jeremy Jordan): https://github.com/jeremyjordan/ml-monitoring
- OpenCost documentation: https://opencost.io/docs/
- Beginner's Guide to Evaluating RAG Pipelines Using RAGAS: https://www.analyticsvidhya.com/blog/2024/05/a-beginners-guide-to-evaluating-rag-pipelines-using-ragas/

Week 13 · Load Testing, Right-Sizing, and Performance Tuning

Notion ID: 392c08fd-0278-8110-ab33-d5c62595fc05 Notion URL: https://app.notion.com/p/392c08fd02788110ab33d5c62595fc05