Model Evaluation Metrics

Picking the right metric — accuracy, precision/recall, F1, ROC-AUC — for the problem and its class balance.

What it is

Different metrics answer different questions, and picking the wrong one hides exactly the failure mode that matters for the problem — this is one of the most common practical mistakes in applied ML.

Key points

Accuracy is misleading on imbalanced classes — a 99%-accurate model can be worthless if the positive class is 1% of the data.
Precision (of predicted positives, how many are correct) vs. recall (of actual positives, how many were found) trade off against each other; F1 is their harmonic mean.
ROC-AUC measures ranking quality across all thresholds; PR-AUC is more informative than ROC-AUC on heavily imbalanced data.
For regression: MAE is robust to outliers, RMSE penalizes large errors more, R² measures variance explained.