so, like, if I was looking for places that would break upward, I would be like "universal translators that finally work"
The story of DeepL might be of relevance.
DeepL has created a translator that is noticeably better than Google Translate. Their translations are often near-flawless.
The interesting thing is: DeepL is a small company that has OOMs less compute, data, and researchers than Google.
Their small team has beaten Google (!), at the Google's own game (!), by the means of algorithmic innovation.
As a wise man pointed out:
The fate of the universe is a decision yet to be made, one which we will intelligently consider when the time is right
Our current understanding of physics is so ridiculously incomplete, it is safe to assume that every single law suggestion of physics eventually will be modified or discarded, in the same way as the theories of phlogiston, life force, and luminiferous aether were discarded.
After we gain a sufficient understanding of physics, we will see if the heat death is still a threat, and if yes, what tech we should build to prevent it.
To be practically immortal, an entity must possess the following qualities:
If we consider only biological entities, the supercolony of ants in Southern Europe is the closest to being immortal. It consists of billions of semi-autonomous nodes distributed across a 6 000 km stretch of land. Of course, it still doesn't possess all the 4 qualities, and thus is still mortal.
Although humans consist of individual nodes too, the described path to biological immortality was closed for them a long time ago. There is no way for the natural selection to take a human and gradually convert him into something like an ant colony.
And that's the main reason why humans are not immortal (yet): our monkey bodies are not designed for immortality, and cannot be redesigned for that by the natural selection.
On the other hand, mind uploading could give us all the qualities of immortality, as listed above.
An entity with the described qualities cannot be outcompeted by your "Horde of Death", as the entity can simply sit and wait until the Horde becomes extinct, which is inevitable for all biological species.
Thus, immortality is the final aromorphosis. Nothing can outcompete immortality.
In general, efficiency at the level of logic gates doesn't translate into the efficiency at the CPU level.
For example, imagine you're tasked to correctly identify the faces of your classmates from 1 billion photos of random human faces. If you fail to identify a face, you must re-do the job.
Your neurons are perfectly efficient. You have a highly optimized face-recognition circuitry.
Yet you'll consume more energy on the task than, say, Apple M1 CPU:
Even if the human brain is efficient on the level of neural circuits, it is unlikely to be the most efficient vessel for a general intelligence.
In general, high-level biological designs are a crappy mess, mostly made of kludgy bugfixes to previous dirty hacks, which were made to fix other kludgy bugfixes (an example).
And the newer is the design, the crappier it is. For example, compare:
With the exception of a few molecular-level designs, I expect that human engineers can produce much more efficient solutions than the natural selection, in some cases - orders of magnitude more efficient.
I consider naming particular years to be a cognitively harmful sort of activity; I have refrained from trying to translate my brain's native intuitions about this into probabilities, for fear that my verbalized probabilities will be stupider than my intuitions if I try to put weight on them. What feelings I do have, I worry may be unwise to voice; AGI timelines, in my own experience, are not great for one's mental health, and I worry that other people seem to have weaker immune systems than even my own.
The following metaphor helped me to understand the Eliezer's point:
Imagine you're forced to play the game of Russian roulette with the following rules:
How should you allocate your resources between the following two options?
It is clear that in this game, the option A is almost useless.
(but not entirely useless, as your escape plans might depend on the timeline).
A possible way to scale it: "collaborative fanfic dungeons":
One way they could do that, is by pitting the model against modified versions of itself, like they did in OpenAI Five (for Dota).
From the minimizing-X-risk perspective, it might be the worst possible way to train AIs.
As Jeff Clune (Uber AI) put it:
[O]ne can imagine that some ways of configuring AI-GAs (i.e. ways of incentivizing progress) that would make AI-GAs more likely to succeed in producing general AI also make their value systems more dangerous. For example, some researchers might try to replicate a basic principle of Darwinian evolution: that it is ‘red in tooth and claw.’
If a researcher tried to catalyze the creation of an AI-GA by creating conditions similar to those on Earth, the results might be similar. We might thus produce an AI with human vices, such as violence, hatred, jealousy, deception, cunning, or worse, simply because those attributes make an AI more likely to survive and succeed in a particular type of competitive simulated world. Note that one might create such an unsavory AI unintentionally by not realizing that the incentive structure they defined encourages such behavior.
Additionally, if you train a language model to outsmart millions of increasingly more intelligent copies of itself, you might end up with the perfect AI-box escape artist.
I agree. Additionally, the life expectancy of elephants is significantly higher than of paleolithic humans (1, 2). Thus, individual elephants have much more time to learn stuff.
In humans, technological progress is not a given. Across different populations, it seems to be determined by the local culture, and not by neurobiological differences. For example, the ancestors of Wernher von Braun have left their technological local minimum thousands of years later than Egyptians or Chinese. And the ancestors of Sergei Korolev lived their primitive lives well into the 8th century C.E. If a Han dynasty scholar had visited the Germanic and Slavic tribes, he would've described them as hopeless barbarians, perhaps even as inherently predisposed to barbarism.
Maybe if we give elephants more time, they will overcome their biological limitations (limited speech, limited "hand", fewer neurons in neocortex etc), and will escape the local minimum. But maybe not.
Jeff Hawkins provided a rather interesting argument on the topic:
The scaling of the human brain has happened too fast to implement any deep changes in how the circuitry works. The entire scaling process was mostly done by the favorite trick of biological evolution: copy and paste existing units (in this case - cortical columns).
Jeff argues that there is no change in the basic algorithm between earlier primates and humans. It's the same reference-frames processing algo distributed across columns. The main difference is, humans have much more columns.
I've found his arguments convincing for two reasons:
The failure of large dinosaurs to quickly scale is a measuring instrument that detects how their algorithms scaled with more compute
Are we sure about the low intelligence of dinosaurs?
Judging by the living dinos (e.g. crows), they are able to pack a chimp-like intelligence into a 0.016 kg brain.
And some of the dinos have had x60 more of it (e.g. the brain of Tyrannosaurus rex weighted about 1 kg, which is comparable to Homo erectus).
And some of the dinos have had a surprisingly large encephalization quotient, combined with bipedalism, gripping hands, forward-facing eyes, omnivorism, nest building, parental care, and living in groups (e.g. troodontids).
Maybe it was not an asteroid after all...
(Very unlikely, of course. But I find the idea rather amusing)
why aren't elephants GI?
As Herculano-Houzel called it, the human brain is a remarkable, yet not extraordinary, scaled-up primate brain. It seems that our main advantage in hardware is quantitative: more cortical columns to process more reference frames to predict more stuff.
And the primate brain is mostly the same as of other mammals (which shouldn't be surprising, as the source code is mostly the same).
And the intelligence of mammals seems to be rather general. It allows them to solve a highly diverse set of cognitive tasks, including the task of learning to navigate at the Level 5 autonomy in novel environments (which is still too hard for the most general of our AIs).
One may ask: why aren't elephants making rockets and computers yet?
But one may ask the same question about any uncontacted human tribe.
Thus, it seems to me that the "elephants are not GI" part of the argument is incorrect. Elephants (and also chimps, dolphins etc) seem to possess a rather general but computationally capped intelligence.