A distribution belongs to the exponential family if its density can be written as a base measure times the exponential of the natural parameter multiplied by a sufficient statistic, minus a normalizing term. The Gaussian, Bernoulli, Poisson, gamma, and multinomial distributions all qualify.
The form matters because the natural parameter tells you how a linear model should attach to the distribution: for the Bernoulli it turns out to be the log-odds, which is why logistic regression uses the sigmoid, and for the Gaussian it is the mean itself, which is why linear regression needs no transformation at all.