Experimental design is the systematic planning of investigations so that the resulting data can support valid, efficient, and unbiased inferences about cause and effect. It specifies the treatments, controls, randomisation, replication, and blocking that isolate the effect of manipulated variables from confounders and noise. Grounded in the scientific method and statistical theory, it governs how hypotheses are tested, how sample sizes and power are determined, and how variability is controlled, and it underpins disciplined experimentation from laboratory trials to large-scale online A/B tests.
Overview
- Good design decides what is varied, what is held constant, and how observations are allocated before data are gathered.
- Randomisation balances unknown confounders across treatment groups, while blocking removes known nuisance variation.
- Replication and adequate power ensure effects can be detected and estimated precisely.
- The same principles scale from controlled lab experiments to online experimentation at internet scale.
Mechanisms
- Randomised assignment of units to treatment and control conditions.
- Blocking and stratification to control identifiable sources of variability.
- Factorial structures to study multiple factors and their interactions efficiently.
- Power and sample-size analysis to set the scale of the study.
Applications
- Clinical and laboratory trials testing interventions.
- A-B Testing of product and interface changes.
- Machine-learning ablations and benchmark comparisons.
- Agricultural, industrial, and social-science field experiments.