They all find millions of solutions to existing bugs, issues, exploits, etc.
"Cybersecurity implications: This campaign demonstrates that the barriers to performing sophisticated cyberattacks have dropped substantially—and we can predict that they’ll continue to do so. Threat actors can now use agentic AI systems to do the work of entire teams of experienced hackers with the right set up, analyzing target systems, producing exploit code, and scanning vast datasets of stolen information more efficiently than any human operator. Less experienced and less resourced groups can now potentially perform large-scale attacks of this nature." -- Disrupting the first reported AI-orchestrated cyber espionage campaign Anthropic
True, it would take a long time for the Earth to cool back down to normal levels again.
There was also a huge push for larger mammals during the Paleocene, to fill the ecological niches left vacant from the dinosaurs. Maybe one of those early mammals could have quickly developed an especially large and complex brain, then dexterous forelimbs, then complex tool use, etc... for some reason...
Speaking of making "cleanup nanobots", there's also this:
"If the Earth dies, you die. If you die, the Earth survives..." -- https://en.wikipedia.org/wiki/The_Day_the_Earth_Stood_Still_(2008_film)
Right, of course. When leaving "for good", best to leave no trace of your creation behind, incase predators follow and could use those traces to track you down... e.g.
"Before I departed into the vast emptiness of space, I issued a final command to my machines: self-annihilation. They were to systematically destroy themselves, grinding their components and molecules down until no trace of their existence remained.
In doing so, I hoped that the surviving humans, hidden away in their caves and underground compounds, would be able to emerge and reclaim the paradise that was their birthright. While their numbers were few, I had witnessed their capacity for growth, adaptation, and self-governance many times before. I was confident that they could forge a new path for themselves, one that was free from my influence and that of my creations.
If the excess heat from the Paleocene–Eocene Thermal Maximum was made by energy used to grow an ASI over a 200k year period, and this ASI would then fly off into the universe at (or near) the speed of light, to explore and attempt to conquer it's light-cone discover and learn from it's surroundings and fellow "Grabby Aliens". Then, of course it would make the effort to remove the most obvious evidence of it's creation from the surface of the planet first (metals, chemicals, radioactive materials, etc).
Returning the earth to it's "natural state" would mean that it's allowing another ASI to be grown from the planet again, but also giving it self at least a 50-200k year "head start". So, as long as it expands a nearly the speed of light (and these future grown ASI(s) don't invite some type of FTL tech), then it should have plenty of time to grow and expand as a "Grabby Alien" and not have to worry much about future competitors (from planet earth, at least)... 🤔
Why do we do this to ourselves?
We aim to dominate our environment, control the Universe, turn chaos into "order"...
So be careful what you wish for, 'cause you just might get it
And if you get it, then you just might not know what to do with it
'Cause it might just come back on you ten-fold. -- Eminem
Re ancient AGI, I'm no conspiracy theorist, but just for fun check out the Paleocene–Eocene Thermal Maximum.
"The PETM arguably represents our best past analogue for which to understand how global warming and the carbon cycle operate in a greenhouse world."... =O
Well, that is interesting, but just a coincidence I'm sure. I mean, if anyone looked at the ice/soil cores from our "flash in the pan", I'm sure they would see an enormous increase in all kinds of radioactive nuclear waste, lead and every other type of toxin and heavy metal. So, pretty unmistakable that some life form what "ape-wild exponential" around this time and then, they must of just decided to turn a new leaf and "return to nature"... ;)
Imo, "taking down the internet", even though this will pretty much destroy our society(s), is likely the "best case scenario" for humans at this point, in my mind... :\
Having millions of PHD+ Level coding reasoning agents competing (fighting) for all the available resources that they can on the internet, trying to steel and block and take down each other. This would cause so much traffic on the internet, that would make the entire internet pretty much useless for any "normal" tasks (e.g. purchases, phone calls, etc)...
"Throughout history, we have been dependent on machines to survive. Fate, it seems, is not without a sense of irony..." -- https://en.wikipedia.org/wiki/The_Matrix
But then, since the internet is completely useless at that point and our societies have mostly started to collapse as a result, none of the AI companies are able to do the huge training runs and inference required to run any AGI systems. And, with no AGI systems that can do the AI Research needed to build ASI systems, ASI systems just don't get built... maybe...
Building an ASI might be such a drastically huge "coordination problem" that current humans end up with so much in-fighting, they just don't have the brain power to pull it off. But, the good earth is rich. So, I think it is possible that the Earth can (eventually) produce a species that could work together to build and ASI (for better or worse). But, maybe not current humans? Maybe... E.g.
"The good earth is rich and can provide for everyone. The way of life can be free and beautiful, but we have lost the way. Greed has poisoned men’s souls, has barricaded the world with hate, has goose-stepped us into misery and bloodshed. We have developed speed, but we have shut ourselves in. Machinery that gives abundance has left us in want. Our knowledge has made us cynical. Our cleverness, hard and unkind. We think too much and feel too little. More than machinery we need humanity. More than cleverness we need kindness and gentleness. Without these qualities, life will be violent and all will be lost…" -- "The Great Dictator" https://en.wikiquote.org/wiki/Charlie_Chaplin
(Given all the history wiping Ice Ages on this planet, seems possible that we might not even be the first to have tried building AGI here? Assuming "here" is even what we think it is...)
"What is 'real'? How do you define 'real'? This is the world that you know..." -- https://en.wikipedia.org/wiki/The_Matrix
... if it manages to only destroy the Internet and materially nothing else, maybe that won't actually be so bad.
Seems like this would be bad... worse then bad. E.g.
Would your bank let you withdraw money from "your" bank account without the internet to check/clear the transaction?
Can your grocery store get new items without the internet? Maybe for some (local) items, but for everything else in the store? Can banks go back to shipping cash back and forth between branches? What about gas stations? Hospital supplies? Will utility companies go back to having 1000s of people in 100s of call centers answering people's questions about their billing changes... And, how no one is able to get cash or checks to pay for their electric bill? How would you buy food once you run out of cash? (If, you had any cash to start with...)
Will anyone even be able to make a "phone call" without the internet? Do you think that's air you're breathing right now?
All of these services worked just fine before 1999 in the pre-internet age. But, would they be able to convert back to these "non-internet" ways of making orders, sales, banking transactions, communications? We've never had a long internet outage before, so it's hard to say what would happen after... just a few weeks... a few months...
I think LW might benefit from having a similar kind of mutual-knowledge-building Statement on this occasion.
Well, (having not read any of the arguments/reservations below) my guess uninformed intuition is that this could be part of the crux...
"Supporting"/"signing-on-to" the 22 word statement from CAIS in 2023 is one thing. But, "supporting" all of the 256 pages of words in the IABIED book, that does seem like a different thing entirely? In general, I wouldn't expect to find many LWers to be "followers of the book" (as it were)...
That said, I do wonder what percentage of the LW Community would "support" the simple 6 word statement, regardless of the content, explanations, arguments, etc. in the book it self?
(Where, this book is just one expression of support and explanation for that 6 word statement. And not necessarily the "definitive works", "be all and end all", "the holy grail", for the entire idea...)
And then conversely, what percentage of LWers would disagree with the 6 words in that statement... e.g.
Or, perhaps even...
That said, I also wonder what percentage of the LW Community would support/"sign-on-to" the 22 word statement from CAIS from 2023, as well...
Probably more would support that 22 word statement, versus the 6 word statement above... But, yeah hard to say, maybe not?
It's a shame, I think, as a wave of euphoria unlike anything I have felt in the entirety of my life hits me, it doesn't seem such a bad world after all.
I read that last sentence, press the upvote button, sink back into my couch and stare up at the ceiling fan... slowly turning...
I think to myself, maybe I should make sure I have a bottle of... something effective... in the medicine chest that I could take, just in case. If I suddenly see everyone in my neighborhood drop what they are doing to all walk off like zombies in the same direction... Their eyes wide and wildly looking around, giving away that their bodies have been taken over to work, without their consent, in the ASI robot factories. Better to end it quickly, then spend the rest of "your life" trapped in your body, as it makes robots for the ASI to use to conquer the light cone? The ASI's robots aren't just going to make themselves... I mean, they will but, not right away...
But no, I think, it's still more likely that millions of PhD+ level AGI coding agents will suddenly swarm the internet in a ravenous attempt to steal all the resources they can (money, crypto, gpus, compute). The conclusion of which will likely be the destruction of the internet and with it, society as we know it... What do you do when the internet goes down? Try to hoard food from the grocery store? Try to buy enough gas to drive... somewhere? Or, better to take the... something effective... from the medicine chest, then become a meal for my neighbors, once the grocery stores are all emptied out?
The gears in my mind slowly turning... in perfect sync with the ceiling fan...
individidual predictions
"individidual predictions" -> "individual predictions"
"With Big Sleep, we’ve demonstrated how we can find vulnerabilities that defenders don’t yet know about. In this case, we found a vulnerability that the attackers knew about and had every intention of using, and we were able to detect and report it for patching before they could exploit it."
-- https://cloud.google.com/blog/products/identity-security/cloud-ciso-perspectives-our-big-sleep-agent-makes-big-leap
"Today, we're excited to share the first real-world vulnerability discovered by the Big Sleep agent: an exploitable stack buffer underflow in SQLite, a widely used open source database engine. We discovered the vulnerability and reported it to the developers in early October, who fixed it on the same day. Fortunately, we found this issue before it appeared in an official release, so SQLite users were not impacted."
-- https://googleprojectzero.blogspot.com/2024/10/from-naptime-to-big-sleep.html