This is conditional on the probability of the ingredients of life turning into life not being extremely low.
But that is precisely the whole point of the argument: if it had indeed been extremely unlikely, then we would expect to be living in a much older universe, one that would have had much more time to meet the necessary conditions!
An interesting section in the appendices, a criticism of Ajeya Cotra’s “Forecasting Transformative AI with Biological Anchors”:
... (read more)If you do a sensitivity analysis on the most important variable (how much Moore's law will improve FLOPS/$), the output behavior doesn't make any sense, e.g., Moore's law running out of steam after "conventional" improvements give us a 144x improvement would give us a 34% chance of transformative AI (TAI) by 2100, a 144*6x increase gives a 52% chance, and a 144*600x increase gives a 66% chance (and with the predicted 60000x improve
Won't the rogue agents just use it as data storage space anyway?