Glossary Entry

Randomized Controlled Trial

An experiment that assigns treatment by chance, making treatment independent of potential outcomes so a simple difference in means is the causal effect.

Statistics Decision Making

Also called: RCT, A/B test, A/B testing, randomized experiment

Seed source: Trustworthy Online Controlled Experiments (Kohavi, Tang and Xu)

Randomization is the gold standard not through sophistication but through deletion: assigning treatment by coin flip severs every link between who gets treated and what would have happened to them, which zeroes out both selection bias and heterogeneity bias in the naive comparison. The treated-vs-control difference in means then estimates the average treatment effect directly.

In industry the RCT is the A/B test, with variants for awkward settings: switchback experiments randomize region-time buckets when marketplace units interfere with each other, and geo experiments randomize whole regions when user-level splits leak. When randomizing the treatment itself is impossible, randomizing an encouragement toward it preserves a foothold for causal analysis via instrumental variables.