The fundamental statistical measure of dispersion, defined as the expected squared deviation of a random variable from its mean, Var(X) = E[(X − E[X])²]. Its square root is the standard deviation; its multivariate generalisation is the covariance matrix; and it parameterises the spread of the Gaussian distribution. Variance governs the reliability of sample-based estimates, decomposes prediction error in the bias-variance trade-off, and is the quantity that variance-reduction techniques in Monte Carlo estimation and stochastic optimisation exist to control.
Semantic Classification
Content
Definition
Variance measures how widely the values of a random variable scatter around their mean. For a random variable X with expectation μ = E[X], the variance is Var(X) = E[(X − μ)²], equivalently E[X²] − μ². Because deviations are squared, variance is expressed in squared units of the underlying quantity; its square root, the standard deviation σ, restores the original units and is the more interpretable companion measure. A variance of zero means the variable is almost surely constant; larger variance means greater dispersion, with squaring giving outlying values disproportionate weight.
Variance is woven through every layer of probability and Statistics. It is the second central moment of any Probability Distribution, and one of the two parameters that completely specify the Gaussian Distribution N(μ, σ²), where it directly sets the width of the bell curve. In several dimensions, the diagonal of the Covariance Matrix holds the variances of each component while the off-diagonal entries hold covariances — so variance is the one-dimensional shadow of the full second-moment structure that principal component analysis diagonalises when it seeks directions of maximal variance.
Its inferential importance flows from Random Sampling: the mean of n independent draws has variance σ²/n, so estimate uncertainty shrinks with sample size at the familiar 1/√n rate — the quantitative content of the law of large numbers and the scaling behind the central limit theorem, confidence intervals and standard errors. In machine learning the bias-variance decomposition splits expected prediction error into bias², variance and irreducible noise, framing regularisation and ensembling (bagging exists precisely to cut variance) as points on a trade-off; and in Monte Carlo methods and stochastic optimisation, estimator variance determines convergence speed, motivating control variates, importance sampling and reparameterised gradients.
Technical Details
-
Sample estimators: the unbiased sample variance s² = Σ(xᵢ − x̄)²/(n − 1) applies Bessel’s correction; dividing by n gives the maximum-likelihood estimate for Gaussian data. Numerically stable streaming computation uses Welford’s algorithm rather than the catastrophic-cancellation-prone E[X²] − μ² form.
-
Algebra: Var(aX + b) = a²Var(X); for independent (or merely uncorrelated) variables, Var(X + Y) = Var(X) + Var(Y); in general Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y).
-
Decompositions: the law of total variance Var(X) = E[Var(X|Y)] + Var(E[X|Y]) splits dispersion into within-group and between-group parts — the identity underlying ANOVA and hierarchical modelling.
-
Tail control: Chebyshev’s inequality bounds P(|X − μ| ≥ kσ) ≤ 1/k² using variance alone, no distributional assumptions required.
-
Caveats: variance is not robust — a single outlier can dominate s² — so heavy-tailed data call for robust spread measures (interquartile range, median absolute deviation); some distributions (e.g. Cauchy) possess no finite variance at all.
Current Landscape
-
Variance’s most-cited application in machine learning remains the bias–variance decomposition: expected squared prediction error decomposes exactly into bias², variance, and irreducible noise σ² (Wikipedia; MIT OCW 15.097; Stanford CS229), with the variance term measuring sensitivity of the fitted model to fluctuations in the training set (overfitting).
-
The decomposition frames standard regularisation and ensembling as variance control — bagging exists specifically to reduce variance — and bootstrap resampling is the practical estimator of a learner’s bias and variance (CMU 10-601 notes).
-
For streaming/large-scale computation, Welford’s online algorithm is the numerically stable standard, avoiding the catastrophic cancellation of the naïve E[X²] − μ² form; the unbiased sample variance applies Bessel’s (n−1) correction.
-
Variance is the one-dimensional case of the covariance matrix that principal component analysis diagonalises to find directions of maximal variance — a formulation that continues to underpin dimensionality reduction and Gaussian modelling.
Sources:
-
https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff
-
https://cs229.stanford.edu/summer2019/BiasVarianceAnalysis.pdf
-
https://plato.stanford.edu/entries/bounded-rationality/bias-variance-decomp.html