Last update: Saturday 6/29/24
Wikipedia defines the value p(doom) as "the probability of catastrophic outcomes (or "doom") as a result of artificial intelligence. The exact outcomes in question differ from one prediction to another, but generally allude to the existential risk from artificial general intelligence."
Many tech-savvy folk assign small, but non-zero p(doom) values to the possibility that our pursuit of AGI might result in the extermination of the human race by an AGI that developed goals that were not aligned with the needs of humanity.