A stochastic process in which the probability of each future state depends only on the current state and not on the sequence of preceding states (the Markov property), enabling tractable analysis of steady-state distributions, mixing times, and long-run behaviour.
Semantic Classification
Content
- A Markov chain models a system that transitions between a set of states according to fixed probabilities, satisfying the memoryless or Markov property. Long-run behaviour is captured by stationary distributions, which describe the proportion of time spent in each state.
- Markov chains underpin sampling algorithms, the analysis of random walks, and sequential decision-making frameworks such as Markov decision processes.