A sparse representation expresses a signal or data point as a linear combination of a small number of basis elements drawn from a larger, often overcomplete dictionary, with most coefficients zero. Sparsity yields compact, interpretable and noise-robust encodings, and underpins compressed sensing, dictionary learning and feature selection. It contrasts with dense distributed representations such as neural embeddings, and is recovered using L1 regularisation or greedy pursuit algorithms.
Overview
- A sparse representation expresses a signal or data point as a linear combination of a small number of basis elements drawn from a larger dictionary, with most coefficients zero.
- Sparsity yields compact, interpretable and often noise-robust encodings, and connects to compressed sensing and feature selection.
- It contrasts with dense distributed representations such as neural embeddings.
Key aspects
- Few non-zero coefficients over an overcomplete dictionary.
- L1 regularisation and greedy pursuit algorithms for inference.
- Dictionary learning to adapt bases to the data.
- Improved interpretability and storage efficiency.
Applications
- Image and signal denoising and reconstruction.
- Compressed sensing in medical imaging acquisition.
- Feature extraction for interpretable machine learning.