Technical interview guide

Advanced RAG & Retrieval Quality

The retrieval-quality techniques that separate a working RAG demo from a production system that reliably surfaces the right context.

Read: 45 min
Practice MCQs: 25
Interview QA: 25
Edition: v4
Editorial status: Reviewed
Relevant for: AI EngineerAgentic AI EngineerForward Deployed Engineer (FDE)

Scope: DPR, ColBERT, SPLADE, BEIR, MTEB, HyDE, Lost in the Middle, RAGAS, HNSW, FAISS, RAG, Model Cards, NIST AI RMF, and OWASP Prompt Injection references reviewed 2026-09-04.

Interview QA

Treat each question like a live interview question: answer out loud first (structure, assumptions, tradeoffs), then open the model answer to spot gaps and rehearse a tighter follow-up.

Curated: 2026-10-05 · Written: 2026-10-05 · Reviewed: 2026-10-03

QA-1

How do you select, calculate, and trade off retrieval-stage metrics like Hit Rate, Recall@k, MRR, and NDCG against downstream generation quality metrics?

QA-2

Build a gold set for retrieval evaluation.

QA-3

Design structure-aware chunking and retrieval.

QA-4

Control duplicate evidence in retrieval.

QA-5

Implement and evaluate hybrid candidate generation.

QA-6

Test authorization-aware retrieval.

QA-7

Build governed multi-query expansion.

QA-8

Evaluate HyDE for a zero-shot domain.

QA-9

Design multi-hop retrieval and evidence assembly.

QA-10

Add and validate a neural reranker.

QA-11

Evaluate a contextual-compression stage.

QA-12

Calibrate retrieval confidence and abstention.

QA-13

Plan a zero-downtime embedding migration.

QA-14

Benchmark an HNSW or compressed index.

QA-15

Design freshness and deletion propagation.

QA-16

Test RAG permission revocation.

QA-17

Build claim-level citation evaluation.

QA-18

Design conflict-aware retrieval and answering.

QA-19

Evaluate context ordering and lost-in-the-middle risk.

QA-20

Create a layered RAG scorecard.

QA-21

Calibrate an automated RAG evaluator.

QA-22

Design cache safety for advanced RAG.

QA-23

Design RAG failure and degradation behavior.

QA-24

Roll out a retrieval-quality change.

QA-25