In partial defence of p(doom)
The things people should talk about instead are: How does the world survive? How likely are different things to happen in the future, maybe given that other things happen? etc.But most people are not aspiring rationalists, and have never heard of any of our arguments, and are not aware of the levels of worry of various people in the field.Communicating the importance of paying attention to the arguments by honestly answering the question "so, how likely do you think AI will be to kill everyone" is useful.Asking someone for the probability they'd assign to AI causing humanity's extinction is useful, too, to figure out how familiar they are with the topic. Their answer also allows asking an open-ended "why?" and getting a more detailed explanation of the view of whoever you're talking to.It is also useful to talk about, e.g., Geoffrey Hinton's stated beliefs about the probability of extinction, as a reason to pay a lot of attention to the actual arguments: it is not common that a godfather of a field regrets his life's work and thinks the consequences of it have >50% chance of killing everyone on the planet."What do you believe" is a good starter for a conversation about "why do you believe that". "A Nobel-winning scientist believes in a high chance o