Many models produce scores that rank examples well but are not honest probabilities: random forests rarely output values near 0 or 1, and boosted trees or class-rebalanced models systematically distort them. Calibration fits a small monotone mapping from raw score to probability on held-out data, classically either Platt scaling (a logistic curve) or isotonic regression (a flexible stepwise fit that needs more data to avoid overfitting).
It matters whenever the probability itself feeds a decision: expected-cost thresholds, ranking across heterogeneous segments, or any metric like log loss that scores the probabilities directly. Calibration cannot fix ranking ability, so it leaves AUC essentially untouched while sometimes improving log loss dramatically.