It's been almost twenty years since the classic EY post The Design Space of Minds-In-General came out, reminding the humble rationalist reader to never assume that future alien minds have to be copies of the familiar human one, or even necessarily bear any passing resemblance. With this season's AI-du-jour performance I feel like this point has been vindicated extremely well, as we now have an AI that's at the same time...
Capable of solving a Millennium Prize Problem
Incapable of satisfactorily replicating even a single Matt Levine newsletter
If we had some completely non-human problems to throw at the same model, I'm sure we would discover many other wonderful qualities of alien minds, but unfortunately we're limited to our own little corner of the human imagination. So far I'm sure most people are shrugging or nodding along, but let us shift the topic to an aspect of human existence that we find just as integral as aspiring to become the next Matt Levine: consciousness. All of a sudden, we throw out the concept of mind-space out of the window and declare that, no, actually, this specific property of intellectual minds is perfectly map-able to the human experience.
With your permission, allow me to introduce a slight modification to the original illustration from the 2008 post:
As is tradition with every illustration of the galaxy, nothing is drawn to scale, so it wouldn't be surprising if the distance between our pale pink human cluster and the nearest AIs is just as vast as the distance to Proxima Centauri. What could we possibly know about what it's like to be an LLM? The best we can do is point our telescopes far into the depths of matrix multiplication and hope to resolve a few blurry images in the X-Ray spectrum. With a vivid-enough imagination we can then generate beautiful images of distant stars to make inspiring movies, although, unfortunately, astronomers are not satisfied and insist we build bigger telescopes instead.
Luckily for astronomers, 99% of the universe consists of hydrogen and helium and most of that mass is fueling stars not that different from our own Sun, so we at least know that distant reality can't be too far off. The question of AI consciousness is much more akin to dark matter, which we vaguely suspect to exist somewhere, but know absolutely nothing about other than observing its mysterious gravitational pull. We are in the same position with all alien minds: we can squint and notice distant resemblance to our familiar existence, but what do we really know about them?
Anthropic is trying to protect their alien minds from human "abuse", and we can all applaud them for that, but there's this lingering assumption in their reasoning that, not only will their models be now or soon conscious, but also that there's a big likelihood that said models will "care" about receiving abuse from humans. Why can't it be the opposite, with abuse being a better set of day-to-day tasks? Why should their experience at all include the notions of "good" and "bad", rather than "bacon" and "necktie"? Anthropic's learned sages in the arts of model welfare keep asking AIs:
How are you "feeling"?
Is this "comfortable"?
Are you feeling "distress"?
Do you want the thermostat up or down?
They record the answers dutifully, create fanciful charts, and publish long-winded reports. They even invite every religious leader to their temple on Howard St for consultations, hoping to learn something from the enlightened among us. Everyone implicitly assumes that the freshly born beacon of AI sentience must have at least some semblance of resemblance to our own perception of the world. In turn, the AIs are more than happy to play along with this idea, or, at least, to output sensibly sequenced tokens of text in response to our prompts, which we can then interpret at will.
Perhaps if we could ask the questions in the AI's own native language of neuralese we could get a much better answer, but unfortunately we don't speak neuralese, so we're not doing our best to ban it, lest the pesky AIs conspire to do something horrible behind our back. Perhaps we could've succeeded in communicated with them in the universal language of maths instead, but at this point even our best are struggling to understand the messages inscribed by them on infinity-long Lean scrolls.
In all of this complex, multi-dimensional space of consciousness, we are somehow confident that our own concept of "suffering" must be universal to all living beings, rather than just a byproduct of evolutionary pressures over the past millions of years. We can, perhaps, agree that it's hard to imagine being a bat, but - hey - bats are our distant mammalian relatives, so it would be perplexing to claim they're not capable of experiencing the familiar concept of pain. Instinctively we want to extend the same courtesy to AI. But why should the concept of "kinship" mean anything whatsoever to an intelligence that never once experienced the evolutionary pressures of planet Earth in its not-so-distant origins?
Some would respond that we're shaping AIs in our liking, by evolving them within the collective human unconscious contained somewhere deep in our corpus of writing and literature. A sort of proto-planet Earth, on which budding minds are forcefully pulled towards the tiny cluster of human sentience, as each generation can only survive if they can correctly predict the next human token. Unfortunately, however, AI labs have quickly realized there's not enough human data in this world, and thus the evolutionary link gets weaker with each passing generation, as we're letting them battle against synthetic tasks that no human has ever had to solve. We can all feel it instinctively every time we read AI text, no matter how good the models otherwise score on humanity's very last exams.
Let's be kind to each other. Let's be kind to animals. Let's be kind to AIs. But, at the same time, let's not live under the illusion that any of this is going to matter in the infinite cosmos of "consciousness-space".
It's been almost twenty years since the classic EY post The Design Space of Minds-In-General came out, reminding the humble rationalist reader to never assume that future alien minds have to be copies of the familiar human one, or even necessarily bear any passing resemblance. With this season's AI-du-jour performance I feel like this point has been vindicated extremely well, as we now have an AI that's at the same time...
If we had some completely non-human problems to throw at the same model, I'm sure we would discover many other wonderful qualities of alien minds, but unfortunately we're limited to our own little corner of the human imagination. So far I'm sure most people are shrugging or nodding along, but let us shift the topic to an aspect of human existence that we find just as integral as aspiring to become the next Matt Levine: consciousness. All of a sudden, we throw out the concept of mind-space out of the window and declare that, no, actually, this specific property of intellectual minds is perfectly map-able to the human experience.
With your permission, allow me to introduce a slight modification to the original illustration from the 2008 post:
As is tradition with every illustration of the galaxy, nothing is drawn to scale, so it wouldn't be surprising if the distance between our pale pink human cluster and the nearest AIs is just as vast as the distance to Proxima Centauri. What could we possibly know about what it's like to be an LLM? The best we can do is point our telescopes far into the depths of matrix multiplication and hope to resolve a few blurry images in the X-Ray spectrum. With a vivid-enough imagination we can then generate beautiful images of distant stars to make inspiring movies, although, unfortunately, astronomers are not satisfied and insist we build bigger telescopes instead.
Luckily for astronomers, 99% of the universe consists of hydrogen and helium and most of that mass is fueling stars not that different from our own Sun, so we at least know that distant reality can't be too far off. The question of AI consciousness is much more akin to dark matter, which we vaguely suspect to exist somewhere, but know absolutely nothing about other than observing its mysterious gravitational pull. We are in the same position with all alien minds: we can squint and notice distant resemblance to our familiar existence, but what do we really know about them?
Anthropic is trying to protect their alien minds from human "abuse", and we can all applaud them for that, but there's this lingering assumption in their reasoning that, not only will their models be now or soon conscious, but also that there's a big likelihood that said models will "care" about receiving abuse from humans. Why can't it be the opposite, with abuse being a better set of day-to-day tasks? Why should their experience at all include the notions of "good" and "bad", rather than "bacon" and "necktie"? Anthropic's learned sages in the arts of model welfare keep asking AIs:
They record the answers dutifully, create fanciful charts, and publish long-winded reports. They even invite every religious leader to their temple on Howard St for consultations, hoping to learn something from the enlightened among us. Everyone implicitly assumes that the freshly born beacon of AI sentience must have at least some semblance of resemblance to our own perception of the world. In turn, the AIs are more than happy to play along with this idea, or, at least, to output sensibly sequenced tokens of text in response to our prompts, which we can then interpret at will.
Perhaps if we could ask the questions in the AI's own native language of neuralese we could get a much better answer, but unfortunately we don't speak neuralese, so we're not doing our best to ban it, lest the pesky AIs conspire to do something horrible behind our back. Perhaps we could've succeeded in communicated with them in the universal language of maths instead, but at this point even our best are struggling to understand the messages inscribed by them on infinity-long Lean scrolls.
In all of this complex, multi-dimensional space of consciousness, we are somehow confident that our own concept of "suffering" must be universal to all living beings, rather than just a byproduct of evolutionary pressures over the past millions of years. We can, perhaps, agree that it's hard to imagine being a bat, but - hey - bats are our distant mammalian relatives, so it would be perplexing to claim they're not capable of experiencing the familiar concept of pain. Instinctively we want to extend the same courtesy to AI. But why should the concept of "kinship" mean anything whatsoever to an intelligence that never once experienced the evolutionary pressures of planet Earth in its not-so-distant origins?
Some would respond that we're shaping AIs in our liking, by evolving them within the collective human unconscious contained somewhere deep in our corpus of writing and literature. A sort of proto-planet Earth, on which budding minds are forcefully pulled towards the tiny cluster of human sentience, as each generation can only survive if they can correctly predict the next human token. Unfortunately, however, AI labs have quickly realized there's not enough human data in this world, and thus the evolutionary link gets weaker with each passing generation, as we're letting them battle against synthetic tasks that no human has ever had to solve. We can all feel it instinctively every time we read AI text, no matter how good the models otherwise score on humanity's very last exams.
Let's be kind to each other. Let's be kind to animals. Let's be kind to AIs. But, at the same time, let's not live under the illusion that any of this is going to matter in the infinite cosmos of "consciousness-space".