Title: Quiz 2 — Embeddings and Vector Representations for Retrieval
Course: AIINFRA 301: Production RAG & LLMOps — Observability and Evaluation > Section: Week 02 — Embeddings and Vector Representations for Retrieval

Q: What is the key structural difference between a dense embedding vector and a sparse keyword vector (e.g., TF-IDF)?
- Dense vectors are always longer than sparse vectors
- Sparse vectors have mostly zero values tied to specific vocabulary terms, while dense vectors pack continuous values across every dimension to represent meaning
- Dense vectors can only represent single words, not full sentences
- There is no meaningful structural difference
A: Sparse vectors have mostly zero values tied to specific vocabulary terms, while dense vectors pack continuous values across every dimension to represent meaning

Q: In the standard indexing pipeline, it is acceptable to embed the query at search time using a different embedding model than the one used to embed the documents, as long as both models are high quality.
- True
- False
A: False

Q: Why is cosine similarity the standard metric for retrieval instead of, say, Euclidean distance?
- It is always more accurate than any other metric
- It requires no computation
- It is scale-invariant — it measures only the angle between vectors and ignores magnitude
- It only works with sparse vectors
A: It is scale-invariant — it measures only the angle between vectors and ignores magnitude

Q: Name one specific reason a high cosine similarity score should NOT be treated as an objective, model-independent measure of semantic similarity.
Feedback: Sample/expected answer: Sample acceptable answers: cosine similarity depends on the specific model's training/geometry, and embedding anisotropy can make even unrelated texts show artificially inflated similarity scores (per arXiv:2504.16318).

Q: According to the current MTEB leaderboard landscape discussed this week, which model leads the average score ranking?
- OpenAI text-embedding-3-large
- Cohere embed-v4
- Gemini Embedding 001
- BGE-M3
A: Gemini Embedding 001

Q: MTEB v1 and MTEB v2 scores can be directly compared and averaged together when ranking models.
- True
- False
A: False

Q: What does Matryoshka representation learning allow you to do with a model like OpenAI's text-embedding-3-large?
- Increase the embedding dimensionality beyond the model's trained size for free
- Truncate the embedding (e.g., from 3072 to 256 dimensions) via the `dimensions` parameter, losing only a small amount of quality while cutting storage and latency
- Convert dense embeddings into sparse keyword vectors
- Skip the query-time embedding step entirely
A: Truncate the embedding (e.g., from 3072 to 256 dimensions) via the `dimensions` parameter, losing only a small amount of quality while cutting storage and latency

Q: A colleague says, "The top-ranked model on the MTEB leaderboard is automatically the best choice for our production RAG system." What is the main flaw in this reasoning, and what should the team do instead?
Feedback: Sample/expected answer: Sample acceptable answer: MTEB largely reflects single-language, general-domain retrieval performance and may not match the team's actual domain, language mix, or document lengths; the team should shortlist candidates from the leaderboard but validate them against their own documents/queries (as practiced in this week's lab) before committing.