Title: Quiz 9 — RAG Evaluation Foundations with RAGAS
Course: AIINFRA 301: Production RAG & LLMOps — Observability and Evaluation > Section: Week 09 — RAG Evaluation Foundations with RAGAS

Q: RAGAS uses an LLM judge to score RAG outputs, which reduces the need for human annotators to manually score every example.
- True
- False
A: True

Q: Which two RAGAS metrics evaluate the retriever component specifically?
- Faithfulness and Answer Relevancy
- Context Precision and Context Recall
- Context Precision and Faithfulness
- Answer Relevancy and Context Recall
A: Context Precision and Context Recall

Q: A RAG system scores very high on Faithfulness. What can you correctly conclude?
- The answer is factually correct
- The answer's claims can be inferred from the retrieved context, regardless of whether that context is true
- The retrieved context is definitely up to date
- The answer directly addresses the user's question
A: The answer's claims can be inferred from the retrieved context, regardless of whether that context is true

Q: In one or two sentences, explain how RAGAS computes Answer Relevancy — what does it generate and compare?
Feedback: Sample/expected answer: It generates several hypothetical questions from the answer, embeds them, and averages their cosine similarity back to the original question; higher similarity means the answer stayed on-topic.

Q: Which of the four core RAGAS metrics require a reference (ground-truth) answer to compute?
- Faithfulness and Answer Relevancy
- Context Precision and Context Recall
- All four metrics
- None of the four metrics
A: Context Precision and Context Recall

Q: Context Precision and Context Recall are essentially the same measurement of retrieval quality, so you only need to check one of them.
- True
- False
A: False

Q: A teammate says, "Our Answer Relevancy score is 0.9, so our RAG system must be giving factually correct answers." What is wrong with this reasoning?
Feedback: Sample/expected answer: Answer Relevancy only measures how directly the answer addresses the question (on-topic-ness); it does not evaluate whether the answer's claims are factually accurate.

Q: Which command reflects a valid way to start using RAGAS, per this week's quickstart flow?
- `npm install ragas`
- `pip install ragas` or `uvx ragas quickstart rag_eval`
- `docker pull ragas/eval`
- `ragas --init-no-llm`
A: `pip install ragas` or `uvx ragas quickstart rag_eval`