A lot of people in and around frontier AI say they assign a meaningful probability to AI causing human extinction. I find this quite hard to take seriously.
To be crystal clear, I think there are plenty of reasons to take AI risk seriously without believing that humanity is about to be wiped out, i.e.
- It is already creating serious cybersecurity problems
- It could displace huge numbers of workers
- It could concentrate power in very uncomfortable ways
- It could destabilize industries, institutions, and politics
What I have much more trouble with are extraordinary claims that there is a meaningful chance that everyone is dead in 10 or 20 years, or that we are building something fundamentally beyond human control.
Which leads me to a much simple question: what would we expect those beliefs to look like in practice?
I don't intend this to read as a gotcha.
Extraordinary beliefs should produce some extraordinary behavior.
And if there isn’t much extraordinary behavior, that’s meaningful information to have.
I genuinely wonder whether such folks are changing their behavior in clear ways. That would change how I think.
| Question |
What it’s intended to reveal |
| What percentage of your net worth have you spent trying to reduce AI risk? |
Whether the belief is strong enough to materially change how you allocate your own money. |
| What political donations have you made to candidates who would slow frontier AI development or data-center expansion? |
Whether concern translates into concrete political spending when there are opportunities to impose real constraints. |
| What long-term financial or life plan have you changed because you think there may not be a long term? |
Whether AI risk actually changes your planning horizon. |
| Have you told your family or close friends to materially change their financial or life plans because of AI risk? What did you recommend? |
Whether you apply the belief consistently when advising people you care about. |
| What is the largest personal sacrifice you have made because of your belief about AI risk? |
Whether the belief has imposed any meaningful personal cost. |
| Have you donated your equity grant to efforts aimed at slowing AI development or reducing AI risk? |
Whether you are willing to give up direct financial upside from the activity you believe may be dangerous. |
| What power, money, or influence have you voluntarily given up because accumulating it conflicted with your safety principles? |
Whether safety beliefs ever place a limit on self-interested accumulation. |
| What action would be bad for your career but good for AI safety, and have you ever taken it? |
Whether AI safety actually wins when it comes into direct conflict with career incentives. |
|
|
.jpg)
