Technical interview guide

Model Evaluation Metrics

Picking the right metric — accuracy, precision/recall, F1, ROC-AUC — for the problem and its class balance.

Read: 26 min
Practice MCQs: 25
Interview QA: 25
Edition: v6
Editorial status: Reviewed
Relevant for: AI EngineerMLOps EngineerMachine Learning Engineer

Scope: scikit-learn stable model-evaluation, calibration, and cross-validation guidance and Google ML Crash Course classification, fairness, and monitoring guidance accessed 2026-08-31..

Interview QA

Treat each question like a live interview question: answer out loud first (structure, assumptions, tradeoffs), then open the model answer to spot gaps and rehearse a tighter follow-up.

Curated: 2026-10-05 · Written: 2026-09-10 · Reviewed: 2026-10-03

QA-1

How do you choose evaluation metrics for a new ML product?

QA-2

Explain precision and recall to a nontechnical stakeholder.

QA-3

When is accuracy an appropriate primary metric?

QA-4

Compare ROC AUC and precision-recall evaluation.

QA-5

How do you select a classification threshold?

QA-6

What does probability calibration mean, and how do you evaluate it?

QA-7

Compare log loss and Brier loss.

QA-8

Explain micro, macro, and weighted averaging for multiclass metrics.

QA-9

How do you evaluate a regression model?

QA-10

Compare MAE, MSE, and RMSE.

QA-11

Explain R-squared and its common misinterpretations.

QA-12

How do you evaluate quantile predictions and prediction intervals?

QA-13

How do you evaluate a ranking or retrieval model?

QA-14

How do you evaluate multilabel classification?

QA-15

How should uncertainty around model metrics be estimated?

QA-16

How do you evaluate metrics across demographic or operational slices?

QA-17

How do you compare two models fairly?

QA-18

How do you avoid metric leakage and test-set overuse?

QA-19

How do prevalence changes affect classification metrics?

QA-20

What should a model evaluation report contain?

QA-21

How do you evaluate a model when labels are delayed?

QA-22

How does selective feedback bias production evaluation?

QA-23

How should offline metrics be connected to business outcomes?

QA-24

A production model's aggregate AUC is stable but complaints rise. How do you investigate?

QA-25