A Gaussian mixture model is a probabilistic model that represents a population as a weighted combination of several Gaussian distributions, each describing a latent subpopulation or cluster. Its parameters — the mixing weights, means, and covariance matrices — are typically estimated by the expectation-maximisation algorithm, which iteratively assigns soft responsibilities to data points and updates the component parameters. As a generative latent-variable model, it supports soft clustering, density estimation, and probabilistic classification.
Overview
- A GMM assumes each observation was generated by first selecting a latent component according to mixing weights, then sampling from that component’s Gaussian. The model therefore captures multimodal and elliptical structure that a single Gaussian cannot.
- Fitting proceeds by expectation-maximisation: the E-step computes posterior responsibilities of each component for each point, and the M-step re-estimates weights, means, and covariances given those responsibilities.
- Because assignments are probabilistic rather than hard, a GMM provides a richer, uncertainty-aware view of cluster membership than centroid-based methods.
- Model selection — choosing the number of components — is commonly guided by information criteria such as BIC or by cross-validated likelihood.
Key aspects
- Soft assignment via posterior responsibilities rather than hard partitions.
- Flexible covariance structures (spherical, diagonal, or full) trading parameters for expressiveness.
- Sensitivity to initialisation and the risk of converging to local optima.
- A generative formulation that allows sampling and likelihood evaluation.
Applications
- Soft clustering of customer segments and behavioural cohorts.
- Density Estimation for anomaly detection and novelty scoring.
- Speaker and acoustic modelling in classical speech systems.
- Background modelling and image segmentation in computer vision.