All services not running behind AWS, GCP or Azure will be banned from access to the newly branded "Internet 2.0", as they are proven vulnerable to attack from any newer "PhD+ level reasoning/coding ai agent".
See also:
"Critical infrastructure systems often suffer from "patch lag," resulting in software remaining unpatched for extended periods, sometimes years or decades. In many cases, patches cannot be applied in a timely manner because systems must operate without interruption, the software remains outdated because its developer went out of business, or interoperability constraints require specific legacy software." -- Superintelligence Strategy by Dan Hendrycks, Eric Schmidt, Alexandr Wang Mar 2025 https://www.nationalsecurity.ai/chapter/ai-is-pivotal-for-nationaZ<l-security, https://arxiv.org/abs/2503.05628
"Partner with critical national infrastructure companies (e.g. power utilities) to patch vulnerabilities" -- "An Approach to Technical AGI Safety and Security" Google DeepMind Team, Apr 2025 https://arxiv.org/html/2504.01849v1#S5
Humans often lack respect or compassion for other animals that they deem intellectually inferior -- e.g. arguing that because those other animals lack cognitive capabilities we have, they shouldn't be considered morally relevant.
Yes, and... "Would be interesting to see this research continue in animals. E.g. Provide evidence that they've made a "150 IQ" mouse or dog. What would a dog that's 50% smarter than the average dog behave like? or 500% smarter? Would a dog that's 10000% smarter than the average dog be able to learn, understand and "speak" in human languages?" -- From this comment
Interesting analysis, though personally, I am still not convinced that companies should be able to unilaterally (and irreversibly) change/update the human genome. But, it would be interesting to see this research continue in animals. E.g.
Provide evidence that they've made a "150 IQ" mouse or dog. What would a dog that's 50% smarter than the average dog behave like? or 500% smarter? Would a dog that's 10000% smarter than the average dog be able to learn, understand and "speak" in human languages?
Create 100s generations of these "gene updated" mice, dogs, cows, etc. as evidence that there are no "unexpected side effects", etc. Doing these types of "experiments" on humans, without providing long (long) term studies of other mammals seems to be... unwise/unethical?
Humanity has collectively decided to roll the dice on creating digital gods we don’t understand and may not be able to control instead of waiting a few decades for the super geniuses to grow up.
But yeah, given this along with the long (long) term studies mentioned above, the whole topic does seem to be (likely) moot...
You can’t just threaten the life and livelihood of 8 billion people and not expect pushback.
"Can't" seems pretty strong here, as apparently you can... at least, so far...
Definitely "shouldn't" though...
Decommission of Legacy Networks and Applications
See also:
"Critical infrastructure systems often suffer from "patch lag," resulting in software remaining unpatched for extended periods, sometimes years or decades. In many cases, patches cannot be applied in a timely manner because systems must operate without interruption, the software remains outdated because its developer went out of business, or interoperability constraints require specific legacy software." -- Superintelligence Strategy by Dan Hendrycks, Eric Schmidt, Alexandr Wang Mar 2025 https://www.nationalsecurity.ai/chapter/ai-is-pivotal-for-nationaZ<l-security, https://arxiv.org/abs/2503.05628
"Partner with critical national infrastructure companies (e.g. power utilities) to patch vulnerabilities" -- "An Approach to Technical AGI Safety and Security" Google DeepMind Team, Apr 2025 https://arxiv.org/html/2504.01849v1#S5
Bank and government mainframe software (including all custom Cobol, Pascal, Fortran code, etc.)
See also:
"Critical infrastructure systems often suffer from "patch lag," resulting in software remaining unpatched for extended periods, sometimes years or decades. In many cases, patches cannot be applied in a timely manner because systems must operate without interruption, the software remains outdated because its developer went out of business, or interoperability constraints require specific legacy software." -- Superintelligence Strategy by Dan Hendrycks, Eric Schmidt, Alexandr Wang https://www.nationalsecurity.ai/chapter/ai-is-pivotal-for-national-security
if we do well about nipping small failures in the bud, we may not get any medium-sized warning shots at all
This would be the scariest outcome: No warning, straight shot to fast self-improving, waaay above human level intelligence. And, (big shocker) turns out humans aren't that intelligent, and also super slow... 😬
"The cost of success is... embarrassment." 😳
Thank you for taking the time to write/post this and run the related Workshop!
imho, we need more people to really think deeply about how these things could plausibly play out over the next few years or so. And, actually spend the time to share (at least their mainline expectations) as well!
So, this is me taking my own advice to spend the time to layout my "mainline expectations".
An Alternate History of the Future, 2025-2040
This article was informed by my intuitions/expectations with regard to these recent quotes:
As, it's not clear to me how millions of "PhD+ reasoning/coding AI agents" can sustainably exist in the same internet as the worlds existing software stacks, which are (currently) very vulnerable to being attacked and exploited by these advanced AI agents. Patching all the software on the internet does seem possible, but not before these PhD+ AI agents are released for public use?
This post is quite short/non-specific, but I think it has some useful predictions/"load bearing claims", so very critique-able?
(I have admittedly haven't read through many other "vignette" examples, yet (somewhat purposefully, as to not bias my own intuitions, too much.))
I encourage anyone to please do read, comment, critique, challenge, etc. Thank you!
Thank you for running this Workshop!
"imho, we need more people to really think deeply about how these things could plausibly play out over the next few years or so. And, actually spend the time to share (at least their mainline expectations) as well!" -- Comment
So, this is me taking my own advice to spend the time to layout my "mainline expectations".
An Alternate History of the Future, 2025-2040
This article was informed by my intuitions/expectations with regard to these recent quotes:
As, it's not clear to me how millions of "PhD+ reasoning/coding AI agents" can sustainably exist in the same internet as the worlds existing software stacks, which are (currently) very vulnerable to being attacked and exploited by these advanced AI agents. Patching all the software on the internet does seem possible, but not before these PhD+ AI agents are released for public use?
This post is quite short/non-specific, but I think it has some useful predictions/"load bearing claims", so very critique-able?
(I have admittedly haven't read through many other "vignette" examples, yet (somewhat purposefully, as to not bias my own intuitions, too much.))
I encourage anyone to please do read, comment, critique, challenge, etc. Thank you!
"How I used o3 to find CVE-2025-37899, a remote zeroday vulnerability in the Linux kernel’s SMB implementation" -- https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899-a-remote-zeroday-vulnerability-in-the-linux-kernels-smb-implementation/