Glossary Entry

ROC AUC

The area under the receiver operating characteristic curve, equal to the probability that a randomly chosen positive example is scored above a randomly chosen negative one.

Metrics Evaluation

Also called: AUC, ROC-AUC, AUROC, area under the ROC curve

Seed source: scikit-learn documentation

ROC AUC measures pure ranking ability: sweep the classification threshold from strict to lenient, trace the true-positive rate against the false-positive rate, and take the area under that curve. A perfect ranker scores 1.0, a coin flip scores 0.5, and the number is threshold-free, which makes it a convenient single summary when the operating point has not been chosen yet.

Its blind spot is heavy class imbalance. Because the false-positive rate divides by the (huge) negative class, a model can post a flattering AUC while burying the rare positives under false alarms; the precision-recall view, whose denominator is the model’s own positive predictions, is the more honest lens there. AUC also says nothing about calibration: two models with identical AUC can assign wildly different probabilities.