Death with dignity

A framing for working on AI risk when you believe success is very unlikely: aim to improve humanity’s odds of survival on a logarithmic scale, rather than measuring yourself against the near-certain outcome of failure.

Origin

Eliezer Yudkowsky introduced it in MIRI announces new “Death With Dignity” strategy, posted on LessWrong on 2022-04-02 (dated April 1 in some time zones). The post says humanity is unlikely to solve AI alignment in time, and that doubling survival chances “from 0% to 0%” is demotivating. Its answer: “the measuring units of dignity are over humanity’s log odds of survival”. Doubling a tiny chance earns one bit of dignity, even if the outcome is still probably death.

Because it was posted on April Fools’ Day, readers asked whether it was a joke. Rob Bensinger later clarified that only the title and one answer were jokes; the views and framing were sincere.

Two rules that come with it

• Do not condition on false hope. Do not adopt a plan because “either it works or we are doomed anyway”. Surviving worlds are more likely to be ones where people faced reality and stayed ready for a lucky surprise.
• Do not do unethical things for the cause. Lying to the public or violating ordinary ethics “to save the world” almost never works and destroys trust. In the post’s words: “My ethics aren’t for sale at the price point of failure.”

The second rule is often overlooked. The post presents dignity partly as a guard against desperate, harmful schemes by people who believe the end is near.

Criticisms

• Demotivating. The most upvoted reply rejected the framing as giving up. Defenders answered that people differ: some need a frame that keeps them honest about the odds, others one that keeps them motivated.
• Mathematically odd. Jessica Taylor noted that maximizing expected log-odds differs from maximizing the probability of survival and can rank gambles strangely.
• Overconfident. Others accepted that alignment is hard but questioned near-zero survival estimates, given how many more people could work on the problem.

Connections

The framing accompanies the pessimism Yudkowsky set out two months later in AGI Ruin - A List of Lethalities. How much it applies depends on How hard is alignment: under more moderate estimates, ordinary expected-value reasoning and dignity point in the same direction.