A latent variable model is a statistical model that explains observed data in terms of unobserved (latent) variables, which capture hidden structure such as cluster membership, low-dimensional factors, or underlying states. By positing latent causes, these models compactly represent complex distributions and support tasks like density estimation, dimensionality reduction, and generation. Inference recovers distributions over the latent variables given the observations.
Overview
- Latent variable models assume that observations are generated by sampling hidden variables and then producing data conditioned on them. Discrete latent variables yield mixture models such as the Gaussian mixture model, while continuous latent variables underlie factor analysis, probabilistic principal component analysis, and modern variational autoencoders. Because the latent variables are unobserved, fitting relies on marginalisation and iterative inference such as expectation-maximisation or variational methods, which alternately estimate latent distributions and update model parameters.
Mechanisms
- Hidden variables capture unobserved structure behind the data.
- Discrete latents give mixture models; continuous latents give factor models.
- Marginalising over latents defines the observed-data likelihood.
- Expectation-maximisation and variational inference fit the parameters.
- Latent spaces provide compressed, often interpretable representations.
Applications
- Clustering and density estimation.
- Dimensionality reduction and feature learning.
- Generative modelling of images and text.
- Topic discovery in document collections.
- Modelling sequential and time-series data.