A phase transition is a qualitative, often abrupt, change in the macroscopic behaviour of a system as a control parameter crosses a critical threshold. Originating in statistical physics to describe transformations such as freezing or magnetisation, the concept is now applied to complex and learning systems where a small change in scale, data, or connectivity produces a discontinuous jump in capability. It is closely associated with emergence and critical phenomena.
Overview
- Phase transitions mark points where the collective behaviour of many interacting components reorganises qualitatively.
- Near a critical point, systems often exhibit large fluctuations, long-range correlations, and power-law behaviour.
- In machine learning the term describes sharp jumps in capability as model scale, data, or training compute increases.
Key aspects
- An order parameter quantifies the degree of order and changes character across the transition.
- Critical points separate distinct phases and can be first-order (discontinuous) or continuous.
- Control parameters such as temperature, density, or model scale drive the system across the boundary.
- Emergent, system-level properties appear that are not evident in individual components.
Applications
- Explaining emergent abilities of large language models as scale crosses thresholds.
- Modelling percolation, connectivity, and resilience in networks and infrastructure.
- Analysing tipping points in collective and self-organising systems.