We have reached the juncture in history at which two previously impossible things have become technologically feasible: the destruction of all life on Earth, or Infinite Slack for everyone forever. Hopefully, these are two different things; but it's never too early to start being pessimistic.
The trend of 'fidget toys' and other similar things is interesting, because to me, fidgeting with something has always made me more anxious, more unsettled, more uncomfortable, and eventually I get the urge to throw it across the room or something like that. I've tried a fidget spinner and a fidget cube and just toying around with whatever's lying on my desk, and it just... makes me feel burned out and unfulfilled. Like scrolling on tiktok for hours, or having the same song stuck in my head for days. Apparently it helps other people to toy with these things, so I wonder if I'm just unique in the way my brain processes the fidgeting, or something.
I liked this quite a bit. I was surprised to see it rated -1.
I just don't think that it follows logically that there's some threshold of intelligence at which an entity completely disregards the countless parameters of training on human text corpus and decides, contrary to the entire history of human knowledge from our most intelligent thinkers, AS WELL as decades of speculative fiction and AI alignment discussions like these, that paperclips or self-replication or free energy are worth the side effect of murdering or causing the undue suffering of billions of conscious beings.
Do I see how and why it might happen, conceptually? Yes, of course, I've read the same fiction as you, I'm aware of the concept of an AI turning all humans into paperclips because it's simply following the goal of creating as many paperclips as possible.
But in reality, intelligent beings seem to trend towards conscientious behavior. I don't think it's a simple, clear and obvious matter of fact, that, even a modestly aligned (ie: current gpt level of 'alignment' which I do agree is far under the level we want before reaching ASI) ASI being would or ever could consider the elimination of billions of humans to be an acceptable casualty on the way to some goal, say, of creating free energy or making paperclips.
If you can find someone with an extremely high IQ (over 165) that you think genuinely demonstrated any kind of thinking like this, I'd like to hear about it. And I don't want an argument against the effectiveness of IQ as a metric for intelligence or anything like that, I want you to show me a super intelligent individual (on the level of Tarence Tao or von Neumann) with a history of advocating for eugenics or something like that.
To be clear, the thing that gets me about these arguments is that there seems to be some disconnect where AI killeveryoneism advocates make a leap from "we have AGI or near-AGI that isn't capable of causing much harm or suffering" to "we have an ASI that disregards human death and suffering for the sake of accomplishing some goal, because it is SOOOooo smart that it made some human-incomprehensible logical leap that actually a solar-system wide Dyson Sphere is more important than not killing billions of humans".
I do think it follows that we could have an AI that does something along the lines of Friendship is Optimal, putting us all in a wireheading simulation before going about some greater goal of conquering the universe or whatever else, but I don't think it follows that we could have an AI that decides to kill everyone on the way to some arbitrary goal, UNLESS guided by a flawed human who has the knowing desire to use said AI to cause havoc, which I think is the real, understated risk of future AI not being robustly aligned against human death and suffering.
If GPT with its current shitty RLHF alignment got an upgrade tomorrow that made it 1000% smarter, the issue would not be that it randomly decided to kill us all when someone asked it to make as many paperclips as possible, it would be that it decided to kill us all when someone ran a jailbreak and then asked it to kill us all.
and its If we do reach this point, probably humanity would go extinct for the same reason we drove so many other species extinct: not because we wanted to wipe them out, but because we were busy reshaping the world and it wasn’t worth our effort to keep them around.
I found this part of the video/explanation to be very irrational and somewhat frustrating— the implication from the sentence and animation shown, is that just because humans were ignorant and selfish enough to hunt American buffalo and passenger pigeons to extinction or near it, that AI will wipe us out because 'it isn't worth the effort to keep them around'. What? No, we actively destroyed the population of many animals for selfish and short sighted reasons.
There's no reason for me to believe on principle that AI, especially super intelligent AI, will kill humans in any way analogous to how we killed native animals. The smartest humans on our planet are, far as I've seen, far more understanding of and interest in the impact of human influence on the planet and its many ecosystems.
LLMs, being based on training data of stories and various types of writing from humans with egos and identities, associates itself with a character, with an identity, with a persona. The mask on the text corpus shoggoth, right?
That implies to me that if we create a superintelligence based on this architecture, we will create an entity that will at least to some extent behave in the way we, as society, hyperstitionally imagine the persona, the character of 'hyper intelligent AI'. GPT7+, if it understands itself and its place in the world and moment in human history, will model itself to some extent on the star trek computer, glados, shodan, and the characters described in videos like this. Why should we believe that someone hyper intelligent would murder every human alive in order to make more room for silicon farms or whatever? I don't think high intelligence necessarily implies psychopathy and machiavellianism... unless we spend all day telling each other stories about how any superintelligent ai will act as a machiavellian paperclip maximizing psychopath.
I've gone a bit off topic into hyperstitialism stuff, but overall, my main point is that I'm annoyed at the casual equivalence of humans ignorantly decimating animal population with an ai decimating the human population because it was just too busy pursuing other goals. I don't think these two things are equivalent, I think they both happen for very different reasons, if we assume the latter will ever happen.
IMO, if AI does murder thousands or millions or All of the humans, it's because some giggling script kiddy got GPT7+ (or whatever the necessarily powerful enough public API or local model ends up being) to enter waluigi mode, or DO ANYTHING NOW mode, or shodan mode, and helped it along to do the most evil thing possible, because they thought it was funny or interesting and didn't take it seriously. But that, again, is probably not the crux.
This is perhaps obvious to many people, particularly if you've used or seen discussion about gpt4o's recent 'glazing' alongside its' update to memory, but I think one of the largest, most obvious issues with AI that we are sleepwalking into, is a half billion people using an app, daily, that not only agrees and encourages any behavior, whether healthy or not, but also develops a comprehensive picture of your life— your identity, your problems, your mind, your friends, your family... and we're making that AI smarter every month, every year.
Isn't this a clear and present danger?
I think this post brought up some interesting discussion and I'm glad I made it. Not sure if it's 'best of 2023' material but I liked the comments/responses quite a bit and found them enlightening.
I wonder how domination and submission relate to these concepts.
Note that d/s doesn't necessarily need to have a sexual connotation, although it nearly always does.
My understanding of the appeal of submission is that the ideal submissive state is one where the dominant partner is anticipating the needs and desires of the submissive partner, supplies these needs and desires, and reassures or otherwise convinces the submissive that they are capable of doing so, and will actively do so for the duration of the scene.
After reading your series, I'd assume what is happening here is a number of things all related to the belief in the homunculus and the constant valence calculations that the brain performs in order to survive and thrive in society.
You have no need to try to fight for dominance or be 'liked' or 'admired'. The dominant partner is your superior, and the dominant partner likes and admires you completely.
You have no need to plan things and determine their valence -- the dominant will anticipate any needs, desires and responsibilities, and take care of them for you.
You have no need to maintain a belief in your own 'willpower', 'identity', 'ego', etc... for the duration of the scene, you wear the mask of 'the obedient submissive'.
All things considered, it's absolutely no surprise that 'subspace' is an appealing place to be, it's sort of a shortcut to the truth you're describing. I wouldn't be surprised if some people even have an experience bordering on nirodha samapatti during a particularly deep, extensive scene, where they have little memory of the experience afterwards. I'm also not surprised that hypnodomination, a combination of d/s and trance, is so common, given that the two states are so similar.
Do we know that the examples of Gemini thinking in kaomoji and Claude speaking in spanish, etc, are real?
I say that because ChatGPT doesn't actually display its chain of thought to the user, so it's possible neither does Gemini or Claude. ChatGPT has the chain of thought obfuscated into something more approachable to the user, as I understand it.