Question 1

In RAG retrieval evaluation, what does recall@k measure?

Accepted Answer

Correct answer: Fraction of relevant chunks appearing in the top-k results. Recall@k = (relevant chunks found in top-k) / (total relevant chunks that exist), so it measures coverage of the full relevant set (b). Fraction of top-k that are relevant is precision@k, not recall@k (a is wrong). Checking only the top-1 chunk describes hit@1, a different metric (c is wrong). The rank of the first relevant chunk is the basis of MRR, not recall@k (d is wrong).

Question 2

In RAG retrieval evaluation, what does precision@k measure?

Accepted Answer

Correct answer: Fraction of the k retrieved chunks that are relevant. Precision@k = (relevant chunks among the retrieved k) / k, so it measures how 'clean' the retrieved set is (a). Coverage of all relevant chunks in the top-k describes recall@k, not precision@k (b is wrong). Reciprocal rank of the first relevant chunk is MRR's building block (c is wrong). A binary yes/no on whether any relevant chunk appears describes hit@k (d is wrong).

Question 3

A vector search returns this ranked list of chunk IDs for a query: What is recall@2 (using only the top 2 retrieved items)?

Accepted Answer

Correct answer: 0.33. Top-2 = ["d3","d1"]. Only d1 is in the relevant set, so hits = 1. recall@2 = hits / |relevant| = 1/3 ≈ 0.33 (a). d2 only appears at rank 4, outside the top 2, so 2/3 overcounts (b is wrong). 0.50 divides by k instead of by the relevant-set size, which is the precision@k formula, not recall (c is wrong). d5 never appears in the retrieved list at all, so recall never reaches 1.00 (d is wrong).

Question 4

Using the same setup: What is precision@2?

Accepted Answer

Correct answer: 0.50. Top-2 = ["d3","d1"]; 1 of these 2 items is relevant, so precision@2 = 1/2 = 0.50 (b). Dividing by 3 (the relevant-set size) instead of by k=2 gives recall@2, not precision (a is wrong). Precision isn't inherently 1.00 just because a chunk found so far is correct — it's still 1 relevant out of 2 retrieved (c is wrong). Dividing by 5 mixes in items outside the k=2 window being evaluated (d is wrong).

Question 5

Same retrieved list and relevant set: What is hit@1 (1 if at least one relevant chunk is in the top 1, else 0)?

Accepted Answer

Correct answer: 0. Top-1 = ["d3"]. d3 is not in {"d1","d2","d5"}, so hit@1 = 0 (d). hit@k is a binary indicator per query, not a fraction, so it can't equal 0.33 (b is wrong). hit@1 only looks at the top-1 item, not the whole ranking, so appearing later doesn't count (c is wrong), and there's no notion of a 'document family' bonus in this metric (a is wrong).

Question 6

How does hit@k fundamentally differ from recall@k?

Accepted Answer

Correct answer: hit@k is binary per query; recall@k is a coverage fraction. hit@k answers 'was at least one relevant chunk retrieved?' (0 or 1), while recall@k answers 'what fraction of all relevant chunks was retrieved?' — a continuous coverage measure (c). Neither metric is systematically larger than the other; it depends on the data (a is wrong). Both metrics require the same labeled relevant set to compute (b is wrong). Both are retrieval-stage metrics computed from the ranked list alone, before any generation happens (d is wrong).

AI Engineer Rag Evaluation Interview Questions

Sample questions

By specialization

By seniority

Other fields