P(doom) — read "P of doom" — is shorthand for someone's subjective probability that AI causes existential or near-existential catastrophe at some point in the future. It became a widely used term in AI policy and research circles around 2022–2023 and remains a contested but pervasive piece of vocabulary in 2026.
Cited public estimates have ranged enormously — from below 1% (many academic AI researchers) to over 50% (some long-tenured AI safety researchers). Geoffrey Hinton, Yoshua Bengio, Anthropic's Dario Amodei and OpenAI's Sam Altman have all made public statements assigning meaningful probability to civilisational-scale AI risk; Yann LeCun, Andrew Ng and many others have publicly argued the probability is small.
The methodological problems with P(doom) as a concept:
- No agreed definition of "doom" — extinction, civilisational collapse, permanent loss of meaningful human autonomy, or just "very bad outcomes" all get rolled up under the same label.
- Subjective probabilities for novel events are hard — there is no base rate for "first time a species creates something smarter than itself".
- Conditional on definitions — most thoughtful people assign different probabilities depending on whether you specify a timeline, a mechanism and a definition of doom.
- Sensitive to incentives — researchers at safety-focused orgs may anchor higher; commercial AI executives may anchor lower; both face structural reasons to do so.
What people actually mean when they cite P(doom) in 2026:
- A signal of safety-orientation — high P(doom) cites are often about communicating that someone takes the risk seriously, regardless of the specific number.
- A negotiating position — in policy debates, claims about P(doom) shape acceptable risk tradeoffs.
- A psychological anchor — those working on AI safety often rationalise their work with reference to a specific number.
- A lazy summary — sometimes P(doom) is used as a conversational shortcut where careful analysis of specific risks would be more useful.
The risk taxonomy that more careful 2026 discussions use instead:
- Misuse risks — bad actors using AI for cyberattacks, bioweapon design, mass disinformation, surveillance.
- Accident risks — well-intentioned deployments going wrong (autonomous systems failing, alignment problems manifesting, prompt injection cascading).
- Structural risks — concentration of power, economic disruption without safety net, erosion of democratic institutions, loss of meaningful human work.
- Long-run alignment risks — ASI-grade systems pursuing goals incompatible with human values.
For a US engineering team in 2026, P(doom) is mostly cultural context rather than product input. Knowing the term exists, knowing why it shows up in policy debates and Twitter discourse, and knowing that thoughtful people disagree wildly on it — all useful when participating in the broader AI conversation. Pinning your engineering decisions to a specific number is generally a category error; pinning them to specific operational risks (data leaks, prompt injection, regulatory exposure, customer harm) is the productive approach.
