Wiki Contributions


Shulman and Yudkowsky on AI progress

so, like, if I was looking for places that would break upward, I would be like "universal translators that finally work"

The story of DeepL might be of relevance. 

DeepL has created a translator that is noticeably better than Google Translate. Their translations are often near-flawless. 

The interesting thing is: DeepL is a small company that has OOMs less compute, data, and researchers than Google. 

Their small team has beaten Google (!), at the Google's own game (!), by the means of algorithmic innovation.

Second-order selection against the immortal

As a wise man pointed out:

The fate of the universe is a decision yet to be made, one which we will intelligently consider when the time is right

Our current understanding of physics is so ridiculously incomplete, it is safe to assume that every single law suggestion of physics eventually will be modified or discarded, in the same way as the theories of phlogiston, life force, and luminiferous aether were discarded. 

After we gain a sufficient understanding of physics, we will see if the heat death is still a threat, and if yes, what tech we should build to prevent it.

Second-order selection against the immortal

To be practically immortal, an entity must possess the following qualities:

  • a) it must be distributed across many semi-autonomous nodes; killing some nodes should not cause the death of the entity itself
  • b) the nodes must be distributed across vast distances; no natural disaster should be able to kill all the nodes
  • c) the entity should be able to create its own backups, hidden in many remote places; no system-level sickness of the whole entity should stop it from being restored from backups
  • d) the entity should be able to rapidly self-improve itself, to win against intelligent adversaries

If we consider only biological entities, the supercolony of ants in Southern Europe is the closest to being immortal. It consists of billions of semi-autonomous nodes distributed across a 6 000 km stretch of land. Of course, it still doesn't possess all the 4 qualities, and thus is still mortal. 

Although humans consist of individual nodes too, the described path to biological immortality was closed for them a long time ago. There is no way for the natural selection to take a human and gradually convert him into something like an ant colony. 

And that's the main reason why humans are not immortal (yet): our monkey bodies are not designed for immortality, and cannot be redesigned for that by the natural selection. 

On the other hand, mind uploading could give us all the qualities of immortality, as listed above. 

An entity with the described qualities cannot be outcompeted by your "Horde of Death", as the entity can simply sit and wait until the Horde becomes extinct, which is inevitable for all biological species. 

Thus, immortality is the final aromorphosis. Nothing can outcompete immortality.

Biology-Inspired AGI Timelines: The Trick That Never Works

In general, efficiency at the level of logic gates doesn't translate into the efficiency at the CPU level. 

For example, imagine you're tasked to correctly identify the faces of your classmates from 1 billion photos of random human faces. If you fail to identify a face, you must re-do the job.

Your neurons are perfectly efficient. You have a highly optimized face-recognition circuitry.

Yet you'll consume more energy on the task than, say, Apple M1 CPU:

  • you'll waste at least 30% of your time on sleep
  • your highly optimized faces-recognition circuitry is still rather inefficient
  • you'll make mistakes, forcing you to re-do the job
  • you can't hold your attention long enough to complete such a task, even if your life depends on it

Even if the human brain is efficient on the level of neural circuits, it is unlikely to be the most efficient vessel for a general intelligence. 

In general, high-level biological designs are a crappy mess, mostly made of kludgy bugfixes to previous dirty hacks, which were made to fix other kludgy bugfixes (an example). 

And the newer is the design, the crappier it is. For example, compare:

  • the almost perfect DNA replication (optimized for ~10^9 years)
  • the faulty and biased human brain (optimized for ~10^5 years)

With the exception of a few molecular-level designs, I expect that human engineers can produce much more efficient solutions than the natural selection, in some cases -  orders of magnitude more efficient.

Biology-Inspired AGI Timelines: The Trick That Never Works

I consider naming particular years to be a cognitively harmful sort of activity; I have refrained from trying to translate my brain's native intuitions about this into probabilities, for fear that my verbalized probabilities will be stupider than my intuitions if I try to put weight on them.  What feelings I do have, I worry may be unwise to voice; AGI timelines, in my own experience, are not great for one's mental health, and I worry that other people seem to have weaker immune systems than even my own.  

The following metaphor helped me to understand the Eliezer's point:

Imagine you're forced to play the game of Russian roulette with the following rules:

  • every year on the day of Thanksgiving, you must put a revolver muzzle against your head and pull the trigger
  • the number of rounds in the revolver is a convoluted probabilistic function of various technological and societal factors (like the total knowledge in the field of AI, the number of TPUs owned by Google, etc).

How should you allocate your resources between the following two options? 

  • Option A: try to calculate the year of your death, by estimating the values for the technological and societal factors
  • Option B: try to escape the game.

It is clear that in this game, the option A is almost useless.

(but not entirely useless, as your escape plans might depend on the timeline).

Visible Thoughts Project and Bounty Announcement

A possible way to scale it: "collaborative fanfic dungeons":

  • a publicly accessible website where users can
    • write dungeon runs
    • write new steps to the existing runs
    • rate the runs / steps (perhaps with separate ratings for thoughts, actions etc)
    • only selected users can rate (initially - only the admins, then - top users etc)
  • could be as technically simple as a wiki (at least in the first iterations)
    • could go way beyond that. E.g.:
      • automatic generation of playable text adventures
      • play as the DM with real people
  • the target audience: fanfic writers / readers
    • (it's much easier to write runs in well known fictional worlds. e.g. HP)
  • the user earns money if their work is good
Christiano, Cotra, and Yudkowsky on AI progress

One way they could do that, is by pitting the model against modified versions of itself, like they did in OpenAI Five (for Dota). 

From the minimizing-X-risk perspective, it might be the worst possible way to train AIs.

As Jeff Clune (Uber AI) put it:

[O]ne can imagine that some ways of configuring AI-GAs (i.e. ways of incentivizing progress) that would make AI-GAs more likely to succeed in producing general AI also make their value systems more dangerous. For example, some researchers might try to replicate a basic principle of Darwinian evolution: that it is ‘red in tooth and claw.’

If a researcher tried to catalyze the creation of an AI-GA by creating conditions similar to those on Earth, the results might be similar. We might thus produce an AI with human vices, such as violence, hatred, jealousy, deception, cunning, or worse, simply because those attributes make an AI more likely to survive and succeed in a particular type of competitive simulated world. Note that one might create such an unsavory AI unintentionally by not realizing that the incentive structure they defined encourages such behavior.

Additionally, if you train a language model to outsmart millions of increasingly more intelligent copies of itself, you might end up with the perfect AI-box escape artist.  

Christiano, Cotra, and Yudkowsky on AI progress

I agree. Additionally, the life expectancy of elephants is significantly higher than of paleolithic humans (1, 2). Thus, individual elephants have much more time to learn stuff.  

In humans, technological progress is not a given. Across different populations, it seems to be determined by the local culture, and not by neurobiological differences. For example, the ancestors of Wernher von Braun have left their technological local minimum thousands of years later than Egyptians or Chinese. And the ancestors of Sergei Korolev lived their primitive lives well into the 8th century C.E. If a Han dynasty scholar had visited the Germanic and Slavic tribes, he would've described them as hopeless barbarians, perhaps even as inherently predisposed to barbarism. 

Maybe if we give elephants more time, they will overcome their biological limitations (limited speech, limited "hand", fewer neurons in neocortex etc), and will escape the local minimum. But maybe not. 

Christiano, Cotra, and Yudkowsky on AI progress

Jeff Hawkins provided a rather interesting argument on the topic: 

The scaling of the human brain has happened too fast to implement any deep changes in how the circuitry works. The entire scaling process was mostly done by the favorite trick of biological evolution: copy and paste existing units (in this case - cortical columns). 

Jeff argues that there is no change in the basic algorithm between earlier primates and humans. It's the same reference-frames processing algo distributed across columns. The main difference is, humans have much more columns.

I've found his arguments convincing for two reasons: 

  • his neurobiological arguments are surprisingly good (to the point of being surprisingly obvious in hindsight)
  • It's the same "just add more layers" trick we reinvented in ML

The failure of large dinosaurs to quickly scale is a measuring instrument that detects how their algorithms scaled with more compute

Are we sure about the low intelligence of dinosaurs? 

Judging by the living dinos (e.g. crows), they are able to pack a chimp-like intelligence into a 0.016 kg brain. 

And some of the dinos have had x60 more of it (e.g. the brain of Tyrannosaurus rex weighted about 1 kg, which is comparable to Homo erectus).

And some of the dinos have had a surprisingly large encephalization quotient, combined with bipedalism, gripping hands, forward-facing eyes, omnivorism, nest building, parental care, and living in groups (e.g. troodontids). 

Maybe it was not an asteroid after all...

(Very unlikely, of course. But I find the idea rather amusing)

Christiano, Cotra, and Yudkowsky on AI progress

why aren't elephants GI?

As Herculano-Houzel called it, the human brain is a remarkable, yet not extraordinary, scaled-up primate brain. It seems that our main advantage in hardware is quantitative: more cortical columns to process more reference frames to predict more stuff. 

And the primate brain is mostly the same as of other mammals (which shouldn't be surprising, as the source code is mostly the same).

And the intelligence of mammals seems to be rather general. It allows them to solve a highly diverse set of cognitive tasks, including the task of learning to navigate at the Level 5 autonomy in novel environments (which is still too hard for the most general of our AIs). 

One may ask: why aren't elephants making rockets and computers yet?

But one may ask the same question about any uncontacted human tribe.

Thus, it seems to me that the "elephants are not GI" part of the argument is incorrect. Elephants (and also chimps, dolphins etc) seem to possess a rather general but computationally capped intelligence. 

Load More