Power is a promise about sensitivity: an experiment with 80% power against a 2% lift will, four times out of five, reach statistical significance when the true lift is 2%. Run the same test underpowered and true effects routinely come back “not significant,” which teams then misread as evidence of no effect.
Power is set by three levers: the effect size you care about, the metric’s variance, and the sample size. The planning identity for a two-arm test, roughly sixteen times the variance over the squared effect per arm, makes the economics vivid: halving the detectable effect quadruples the required sample. Power analysis before launch, not after, is what separates experiments that can answer their question from theater.