Glossary Entry

Counterfactual Evaluation

Estimating how a system would have performed under decisions that were not actually taken.

Decision Making Data

Also called: counterfactual acceptance, counterfactual prices, counterfactual probabilities

Counterfactual evaluation asks what would have happened under an alternative action. This is hard because historical data only records the action that was actually taken and the outcome that followed.

In recommendation, pricing, and policy systems, counterfactual evaluation is often needed before a full online experiment is available. The key risk is optimism: evaluating a model using assumptions that make its own decisions look better than they really are.