Glossary Entry

Generalized Linear Model

A model family where a weighted sum of features predicts the mean of an exponential-family distribution through a link function; linear and logistic regression are its best-known members.

Models Statistics

Also called: generalized linear models, GLM, GLMs

Seed source: Nelder & Wedderburn, Generalized Linear Models (1972)

A generalized linear model is three choices bundled together: a distribution describing how outcomes scatter (the random component), a linear predictor summing weighted features (the systematic component), and a link function connecting the sum to the distribution’s mean. Filling the slots with Gaussian and identity gives linear regression; Bernoulli and logit gives logistic regression; Poisson and log gives count regression.

The framing earns its keep in practice: it explains why the sigmoid exists (it is the inverse of the Bernoulli’s natural parameter), why least squares and cross-entropy are both maximum likelihood in disguise, and why one fitting algorithm, iteratively reweighted least squares, covers the whole family.