Title: Quiz 12 — Responsible AI, Bias, Fairness, and Model Cards
Course: AIINFRA 302: AI Security, Guardrails & Governance > Section: Week 12 — Responsible AI, Bias, Fairness, and Model Cards

Q: A Model Card, as defined by Mitchell et al. (2019), primarily requires which of the following?
- A single headline accuracy score
- Disaggregated evaluation across subgroups, intended/out-of-scope uses, and ethical considerations
- Only the model's training hyperparameters
- A list of competitor models for comparison
A: Disaggregated evaluation across subgroups, intended/out-of-scope uses, and ethical considerations

Q: Removing a protected attribute like race or gender from a model's input features is generally sufficient to make the model fair.
- True
- False
A: False

Q: What is the ideal value of the disparate impact ratio, and what threshold is commonly used as a legal/regulatory rule of thumb?
- Ideal = 0; threshold \~0.5
- Ideal = 1.0; threshold \~0.8
- Ideal = 100; threshold \~50
- Ideal = -1; threshold \~0
A: Ideal = 1.0; threshold \~0.8

Q: Demographic parity and equalized odds can be satisfied simultaneously by any well-trained model.
- True
- False
A: False

Q: What is the main argument behind "Datasheets for Datasets" (Gebru et al.), and why does it matter for fairness?
Feedback: Sample/expected answer: Every dataset should ship documentation of its motivation, composition, collection process, and known biases, because bias often originates in the data itself — making dataset documentation a governance artifact as important as model cards.

Q: Which open-source toolkit is known for offering 70+ fairness metrics and 9 mitigation algorithms spanning pre-processing, in-processing, and post-processing stages?
- Giskard
- Aequitas
- AI Fairness 360 (AIF360)
- Fairlearn dashboard only
A: AI Fairness 360 (AIF360)

Q: What does RAGET, a component of the Giskard toolkit, do?
- Encrypts model weights before deployment
- Auto-generates test questions to evaluate a RAG pipeline's retrieval accuracy
- Removes protected attributes automatically from training data
- Calculates GPU cost per inference
A: Auto-generates test questions to evaluate a RAG pipeline's retrieval accuracy

Q: Why is "disaggregated evaluation" (per-subgroup performance) considered more important than aggregate accuracy alone when assessing fairness?
Feedback: Sample/expected answer: Aggregate accuracy can hide large performance gaps between subgroups (e.g., strong performance on one demographic and poor performance on another); disaggregated evaluation breaks results out per group so hidden fairness gaps become visible instead of averaged away.