Sample questions
Rag Generation ContextDifficulty 1
In a RAG pipeline, retrieval selects candidate chunks, but a separate step decides how those already-retrieved chunks are ordered, filtered, and formatted before they reach the model. What is this step usually called?
- aContext assembly✓
- bRetrieval re-scoring
- cQuery expansion
- dIndex building
Explanation:Context assembly is the step between retrieval and generation that builds the actual prompt: it decides chunk order, filtering, and formatting. Retrieval re-scoring re-ranks candidates during search; query expansion rewrites the user's query for the retriever; index building happens before any query is asked, when embeddings are stored.
Rag Generation ContextDifficulty 1
import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
instruction = (
"Answer the user's question using only the information in the "
"context below. If the context does not contain the answer, "
"say you don't know."
)
print(len(enc.encode(instruction)))
What does this print?
Explanation:Encoding the instruction string with the tokenizer above yields 29 tokens (verified by running the snippet). This fixed instruction cost must be subtracted from the abstract context budget before any retrieved chunks are added.
Rag Generation ContextDifficulty 2
The system instruction above costs 29 tokens, the user's query costs 10 tokens, and three retrieved chunks cost 26, 28, and 28 tokens respectively (all counted with the same tokenizer). If the abstract context budget for this request is 120 tokens, what happens when all three chunks are added?
- aEverything fits, with 1 token to spare
- bEverything fits exactly at 120 tokens
- cIt does not fit — total is 121 tokens, 1 over budget✓
- dIt does not fit — total is 132 tokens, 12 over budget
Explanation:29 + 10 + 26 + 28 + 28 = 121, which is one token over the 120-token budget. At least one chunk (or the query/instruction) must be trimmed or a chunk dropped before this fits.
Rag Generation ContextDifficulty 2
Given the same fixed overhead (instruction 29 + query 10 = 39 tokens) and a budget of 120, chunks are added one at a time in order (26, 28, 28 tokens) and a chunk is only kept if the running total stays within budget. How many of the three chunks end up included, and what is the final running total?
- a3 chunks, 121 tokens
- b1 chunk, 65 tokens
- c0 chunks, 39 tokens
- d2 chunks, 93 tokens✓
Explanation:39 + 26 = 65 (fits), 65 + 28 = 93 (fits), 93 + 28 = 121 (exceeds 120, so this chunk is dropped and the loop stops). So 2 chunks are included with a final running total of 93 tokens.
Rag Generation ContextDifficulty 1
A context-assembly loop is close to the token budget, and the next candidate chunk would only partially fit. What is standard practice?
- aDrop the chunk entirely, keep only whole chunks✓
- bTruncate the chunk mid-sentence to fill the remaining budget
- cSummarize the chunk on the fly before inserting it
- dSplit the remaining budget evenly across all chunks
Explanation:Truncating a chunk mid-sentence risks cutting off exactly the fact that was needed and inserting a nonsensical fragment. The standard approach in basic context assembly is whole-chunk-or-nothing inclusion: a chunk that doesn't fully fit is skipped rather than cut.
Rag Generation ContextDifficulty 2
A RAG system assembles 7 retrieved chunks into the prompt, all of which are correctly relevant. The single most important chunk sits at position 4 (the middle). The model's answer misses the fact from that chunk. What is most likely happening?
- aThe chunk's embedding was corrupted during retrieval
- bPosition in a long context reduced attention to that chunk✓
- cThe chunk exceeded a per-chunk token limit and was silently dropped
- dRetrieval failed to find the chunk at all
Explanation:This is the "lost in the middle" effect: retrieval and assembly both worked correctly (the chunk is present in the prompt), but its position in the middle of a long context makes it less likely to be used in the answer — an attention/positional issue, not a retrieval failure.