An existential risk is any risk that could permanently and drastically curtail humanity’s long-run potential — including human extinction, irreversible civilisational collapse, or permanent totalitarian lock-in — as distinct from severe but recoverable catastrophes. The concept grounds a research agenda that prioritises preventing outcomes from which recovery is impossible, because their badness is unbounded by the loss of all future generations.
Content
- The term “existential risk” was introduced into academic discourse by Nick Bostrom in his 2002 paper “Existential Risks: Analysing Human Extinction Scenarios and Related Hazards.” Bostrom distinguished existential from merely catastrophic risks by emphasising finality: a pandemic that kills half of humanity is catastrophic but recoverable; a pandemic that eliminates humanity entirely is existential. The field drew on earlier nuclear-era discussions of species-level risk but provided a systematic taxonomy including extinction, permanent totalitarianism, and “flawed realisation” scenarios where civilisation persists but its potential is permanently diminished.
- Existential risk analysis combines probability estimation with moral philosophy. Because future generations are vastly more numerous than present ones, even a tiny reduction in extinction probability carries enormous expected value under many ethical frameworks. Risk sources are typically grouped into four categories: naturally occurring risks (asteroid impacts, supervolcanic eruptions), engineered risks (engineered pandemics, nuclear war), risks from misaligned AI, and socio-political lock-in. Each category demands different mitigation strategies ranging from asteroid deflection programmes to biosafety regulation to AI alignment research.
- The field matters because existential risks are uniquely irreversible: there is no recovery from human extinction, no “undo” for civilisational lock-in. This asymmetry justifies extraordinary levels of caution even at low probabilities. Existential risk research has influenced government policy through organisations such as the UK’s Centre for Long-Term Resilience and the US Office of Science and Technology Policy. It shapes biosecurity legislation, AI safety summits, and pandemic preparedness frameworks.
- In 2024–2025, existential risk studies is a recognised academic sub-field at Cambridge (Centre for the Study of Existential Risk), Oxford (now reorganised after FHI’s closure), and MIT. The closure of FHI in 2024 was a significant institutional shift, with researchers dispersing to new institutes and industry safety teams. The Bletchley Declaration (2023) and subsequent Seoul and Paris AI Safety Summits brought existential AI risk into formal intergovernmental discourse. Biosecurity remains a co-equal concern following COVID-19, with dual-use biotechnology regulation intensifying globally.