Singular value decomposition (SVD) is a matrix factorisation that expresses any real or complex matrix as the product of two orthogonal (or unitary) matrices and a diagonal matrix of non-negative singular values. It generalises eigenvalue decomposition to arbitrary, non-square matrices and reveals the rank, range, and dominant directions of variation in data. SVD is foundational across machine learning and numerical linear algebra, underpinning dimensionality reduction, low-rank approximation, recommender systems, and the principal component analysis used to compress and denoise high-dimensional data.
Overview
- SVD decomposes a matrix into left singular vectors, singular values, and right singular vectors, where the singular values quantify how much each orthogonal direction contributes to the data. Truncating to the largest singular values yields the best low-rank approximation in the least-squares sense (the Eckart-Young theorem), which is the mathematical core of dimensionality reduction and lossy compression. Because it is numerically stable and broadly applicable, SVD is a workhorse of scientific computing and the basis of techniques from latent semantic analysis to model compression.
Mechanisms
- Factorisation into orthogonal U, diagonal sigma, and orthogonal V
- Singular values ordering directions by variance captured
- Optimal low-rank approximation by truncation (Eckart-Young)
- Numerically stable computation via iterative algorithms
- Generalisation of eigendecomposition to non-square matrices
Applications
- Principal component analysis and data compression
- Recommender systems and collaborative filtering
- Latent semantic analysis in text retrieval
- Noise reduction and signal processing
- Low-rank model compression and adaptation
Provenance
- This class was materialised to resolve inbound references from existing classes in the knowledge graph.