Followup to This Failing Earth, Our society lacks good self-preservation mechanisms, Is short term planning in humans due to a short life or due to bias?

I don't mean that deciding to exterminate life is rational. But if, as a society of rational agents, we each maximize our expected utility, this may inevitably lead to our exterminating life, or at least intelligent life.

Ed Regis reports on p 216 of “Great Mambo Chicken and the TransHuman Condition,” (Penguin Books, London, 1992):

Edward Teller had thought about it, the chance that the atomic explosion would light up the surrounding air and that this conflagration would then propagate itself around the world. Some of the bomb makers had even calculated the numerical odds of this actually happening, coming up with the figure of three chances in a million they’d incinerate the Earth. Nevertheless, they went ahead and exploded the bomb.

Was this a bad decision? Well, consider the expected value to the people involved. Without the bomb, there was a much, much greater than 3/1,000,000 chance that either a) they would be killed in the war, or b) they would be ruled by Nazis or the Japanese. The loss to them if they ignited the atmosphere would be another 30 or so years of life. The loss to them if they lost the war and/or were killed by their enemies would also be another 30 or so years of life. The loss in being conquered would also be large. Easy decision, really.

Suppose that, once a century, some party in a conflict chooses to use some technique to help win the conflict that has a p=3/1,000,000 chance of eliminating life as we know it. Then our expected survival time is 100 times the sum from n=1 to infinity of np(1-p)^n-1. If I've done my math right, that's ≈ 33,777,000 years.

This supposition seems reasonable to me. There is a balance between offensive and defensive capability that shifts as technology develops. If technology keeps changing, it is inevitable that, much of the time, a technology will provide the ability to destroy all life before the counter-technology to defend against it has been developed. In the near future, biological weapons will be more able to wipe out life than we are able to defend against them. We may then develop the ability to defend against biological attacks; we may then be safe...