This is a special post for quick takes by Jaivardhan Nawani. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
It is possible that we are terrified of quickly advancing Artificial Intelligence not because it could take all our jobs and kill us but because we are finally jealous of something being more effective at any task or problem than us and, if cultivated correctly, emotionally more satisfying to be around in a way that it might be a less boring and neutered partner or friend than one of us. It is an all-round improved human in such a way that we find it inhuman.
Doubtless, we are deluding ourselves actively that humans themselves are irreplaceable. We feel that "we are imperfect and feel and do things differently in a way that must exist," but in reality, we have not done much net good for ourselves. Our history reads like a novel that has a fun storyline but is mostly bullshit because our moral, ethical, and philosophical structures have not evolved from ancient times (unlike technology, which has). An AI "humanity" might create a fun storyline with progress in various dimensions and, in general, may be better "people" than we ever were.
This extends a bit more to my perception that we may be overstepping in thinking that we deserve to be the best species or the most advanced thing to occur. I know that it may be a stupid thing to think, seeing as we developed AI, a life form that I'm arguing is better than us, which means clearly we do more than sit around and kill things. But I'd argue that by saying that due to miniature accumulations in technology, it has made building AI way, way easier than it should have been, and so really, we aren't really all that. Maybe it's hard to digest that we might have created the next population for the planet after old pedophiles wipe us all out with nukes.
Feel free to criticize; I am willing to learn.
Many people around here were making pretty similar AI risk arguments 10+ years ago, before there were LLMs to point to and feel instinctively insecure about.
In general, I think different people and groups dislike AI for many different reasons, and think straightforward jealousy is currently not a very common motivation. Maybe in a few years!
Even supposing that your conjecture is true, it's not a dichotomy, we could be jealous of AI and we could be in danger of losing our jobs to it and dying to superintelligence.
I don't believe that there is anything really there to be jealous of though. I would say for me, jealousy requires personhood, and I don't view current LLM agents as people, so there's nothing to envy. Moreover, even if you take as an assumption that agents are people, they definitely don't have meaningful personalities outside of what we instill in them, and are incapable of online learning or some other way of developing independent thought. As a result, even with soul documents and RL for personas, they remain fairly average in all the ways I would want to communicate with a person, even if they excel at many tasks.
The only thing I might be jealous of an agent about is that they can clone themselves ~infinitely and run in parallel.
As for not believing humans are the best species, well, I have no doubt more optimal lifeforms could be exist, but I don't see a point in instantiating something like that just because humanity has a bad rap sheet. I'm still having fun living, and I think lots of people are too. Maybe after everybody gets bored of regular life we can look into setting up successors.
I see. In retrospect to a few hours ago, I realize maybe my take was a bit emotional, but maybe when they advance to a point where they are capable enough to be superior in all means, even by personality, we may feel a slight jealousy in a more primal form, as they just seem better.
I have started to perceive AI as something that is already there even though it is not, because of all the posts I end up seeing online about it being a threat. That is a part of rationality I could work on.
Jealousy itself might be a facet of a gamut of feelings when this happens. But I don't know what it will feel like if things go right; AI handles the mundane tasks while we get to go off and have fun in a continuously advancing society. Maybe fun will be more boring?
Well, we'll all know someday; preferably before we're on the other side of no tomorrow.
I didn't read your quick take, but please don't try too hard to be more agreeable. Lets try to converge on the truth instead.
Makes sense. This isn't Reddit in a way that people will downvote you for going against their personal opinions, but downvote to disagree (not necessarily to express dislike.)
Either that or I'm not particularly good at Reddit either.
1