Glossary Entry

Log Loss

A metric for probabilistic classifiers that averages the negative log of the probability assigned to the true class, punishing confident wrong predictions severely.

Metrics Evaluation

Also called: logarithmic loss, logloss, cross-entropy loss, binary cross-entropy

Seed source: scikit-learn documentation

Log loss (the same quantity deep learning calls cross-entropy loss) scores a probabilistic prediction by how much probability it placed on what actually happened: predict 0.9 for an event that occurs and you pay little, predict 0.01 for it and you pay a lot, with the penalty growing without bound as confidence in the wrong answer approaches certainty.

That unbounded penalty is the defining personality trait. A model can rank examples perfectly and still post a terrible log loss if its probabilities are overconfident or miscalibrated, which is why probability calibration often improves log loss dramatically without changing the model’s decisions at all. Its optimal constant baseline is the base rate of each class, a useful sanity check before any modelling starts.