Importance sampling is a variance-reduction technique in Monte Carlo estimation that draws samples from a proposal distribution that concentrates probability mass in regions contributing most to the quantity being estimated, then corrects for the distributional mismatch using importance weights. It is foundational to Bayesian inference, reinforcement learning, path-tracing renderers, and off-policy evaluation. Importance sampling allows tractable estimation of expectations under distributions that are difficult or impossible to sample from directly. Poor choice of proposal distribution can however lead to high-variance or even infinite-variance estimators, necessitating careful design.
Content
- Importance sampling addresses a fundamental challenge in probabilistic computation: how to estimate the expectation of a function under a distribution p(x) when sampling directly from p is expensive or infeasible. The technique samples instead from a proposal q(x) and multiplies each sample’s contribution by the importance weight w(x) = p(x)/q(x), yielding an unbiased estimator of the target expectation so long as q assigns non-zero probability wherever p is non-zero.
- In rendering, importance sampling directs light-path samples towards high-radiance regions—bright light sources, specular lobes—dramatically reducing per-pixel noise compared with uniform hemisphere sampling. Multiple importance sampling (MIS) combines several proposal strategies using balance heuristics to minimise variance further. These techniques are integral to production path tracers used in film and game engine global illumination.
- In machine learning, importance sampling appears in off-policy reinforcement learning (estimating the value of a policy from data collected under a different behaviour policy), in variational inference (the importance-weighted autoencoder tightens the evidence lower bound by averaging multiple importance-weighted samples), and in curriculum learning (prioritised experience replay re-weights stored transitions by temporal-difference error magnitude).
- Practical challenges include weight degeneracy—when the proposal poorly matches the target, a handful of samples receive enormous weights whilst the rest contribute negligibly, collapsing effective sample size. Capping or normalising weights (self-normalised importance sampling) reduces variance at the cost of bias. Choosing good proposals remains an active research area, with learned proposal networks offering adaptive solutions.