All of teradimich's Comments + Replies

I can provide several links. And you choose those that are suitable. If suitable. The problem is that I retained not the most complete justifications, but the most ... certain and brief. I will try not to repeat those that are already in the answers here.

Ben Goertzel

Jürgen Schmidhuber

Peter J.Bentley

Richard Loosemore

Jaron Lanier and Neil Gershenfeld

Magnus Vinding and his list

Tobias Baumann

Brian Tomasik
 

Maybe Abram Demski? But he changed his mind, probably.
Well, Stuart Russell. But this is a book. I can quote.

I do think that I’m an optimist. I think the

My fault. I should just copy separate quotes and links here.

Glad you understood me. Sorry for my english!
Of course, the following examples themselves do not prove the opportunity to solve the entire problem of AGI alignment! But it seems to me that this direction is interesting and strongly underestimated. Well, someone smarter than me can look at this idea and say that it is bullshit, at least.

Partly this is a source of intuition for me, that the creation of aligned superintellect is possible. And maybe not even as hard as it seems.
We have many examples of creatures that follow the goals of someone more stupid. An... (read more)

It seems to me that the brains of many animals can be aligned with the goals of someone much more stupid themselves.
People and pets. Parasites and animals. Even ants and fungus.
Perhaps the connection that we would like to have with superintellence, is observed on a much smaller scale.

I apologize for the stupid question. But…

Do we have more chances to survive in the world, which is closer to Orwell's '1984'?
It seems to me that we are moving towards more global surveillance and control. China's regime in 2021 may seem extremely liberal for an observer in 2040.

I guess I missed the term gray goo. I apologize for this and for my bad English.
Is it possible to replace it on the 'using nanotechnologies to attain a decisive strategic advantage'?
I mean the discussion of the prospects for nanotechnologies on SL4 20+ years ago. This is especially:

My current estimate, as of right now, is that humanity has no more than a 30% chance of making it, probably less. The most realistic estimate for a seed AI transcendence is 2020; nanowar, before 2015.

I understand that since then the views of EY have changed in many ways. But I a... (read more)

Nanosystems are definitely possible, if you doubt that read Drexler’s Nanosystems and perhaps Engines of Creation and think about physics. 

Is there something like the result of a survey of experts about the feasibility of drexlerian nanotechnology? Are there any consensus among specialists about the possibility of a gray goo scenario?

Drexler and Yudkowsky both extremely overestimated the impact of molecular nanotechnology in the past.

I do not know the opinions of experts on this issue. And I lack competence for such conclusions, sorry.

AlexNet was the first publication that leveraged graphical processing units (GPUs) for the training run

Do you mean the first of the data points on the chart? The GPU was used for DL long before AlexNet. References: [1], [2], [3], [4], [5].

The most realistic estimate for a seed AI transcendence is 2020; nanowar, before 2015. The most optimistic estimate for project Elisson would be 2006; the earliest nanowar, 2003.

But this is 1999, yes.

Probably that:

When we didn’t have enough information to directly count FLOPs, we looked GPU training time and total number of GPUs used and assumed a utilization efficiency (usually 0.33)

This can be useful:

We trained the league using three main agents (one for each StarCraft race), three main exploiter agents (one for each race), and six league exploiter agents (two for each race). Each agent was trained using 32 third-generation tensor processing units (TPUs) over 44 days

Perhaps my large collection of quotes about the impact of AI on the future of humanity here will be helpful.

Then it is worth considering the majority of experts from the FHI to be extreme optimists, the same 20%? I really tried to find all the publicly available forecasts of experts and those who were confident that AI would lead to the extinction of humanity, there were very few among them. But I have no reason not to believe you or Luke Muehlhauser who introduced AI safety experts as even more confident pessimists: ’Many of them are, roughly speaking, 65%-85% confident that machine superintelligence will lead to human extinction’ . The reason may... (read more)

What about this and this? Here, some researchers at the FHI give other probabilities.

I meant the results of such polls: https://www.thatsmags.com/china/post/15129/happy-planet-index-china-is-72nd-happiest-country-in-the-world. Well, it doesn’t matter.
I think that I could sleep better if everyone recognized the reduction of existential risks in a less free world.

I’m not sure that I can trust news sources that are interested in outlining China.
In any case, this does not seem to stop the Chinese people from feeling happier than the US people.
I cited this date just to contrast with your forecast. My intuition is more likely to point to AI in the 2050-2060 years.
And yes, I expect that in 2050 it will be possible to monitor the behavior of each person in countries 24/7. I can’t say that it makes me happy, but I think that the vast majority will put up with this. I don't believe in a liberal democratic utopia, but the end of the world seems unlikely to me.

Just wondering. Why are some so often convinced that the victory of China in the AGI race will lead to the end of humanity? The Chinese strategy seems to me much more focused on long terms.
The most prominent experts give a 50% chance of AI in 2099 (https://spectrum.ieee.org/automaton/robotics/artificial-intelligence/book-review-architects-of-intelligence). And I can expect that the world in 80 years will be significantly different from the present. Well, you can call this a totalitarian hell, but I think that the probability of an existential disaster in t... (read more)

How about paying attention to discontinuous progress in tasks that are related to DL? It is very easy to track with https://paperswithcode.com/sota . And https://sotabench.com/ is showing diminishing returns.

(I apologize in advance for my English). Well, only the fifth column shows an expert’s assessment of the impact of AI on humanity. Therefore, any other percentages can be quickly skipped. It took me a few seconds to examine 1/10 of the table through Ctrl+F, so it would not take long to fully study the table by such a principle. Unfortunately, I can't think of anything better.

It may be useful.

’Actually, the people Tim is talking about here are often more pessimistic about societal outcomes than Tim is suggesting. Many of them are, roughly speaking, 65%-85% confident that machine superintelligence will lead to human extinction, and that it’s only in a small minority of possible worlds that humanity rises to the challenge and gets a machine superintelligence robustly aligned with humane values.’ — Luke Muehlhauser, https://lukemuehlhauser.com/a-reply-to-wait-but-why-on-machine-superintelligence/

’... (read more)

I have collected a huge number of quotes from various experts about AGI. About the timing of AGI, about the possibility of a quick takeoff of AGI and its impact on humanity. Perhaps this will be useful to you.

https://docs.google.com/spreadsheets/d/19edstyZBkWu26PoB5LpmZR3iVKCrFENcjruTj7zCe5k/edit?fbclid=IwAR1_Lnqjv1IIgRUmGIs1McvSLs8g34IhAIb9ykST2VbxOs8d7golsBD1NUM#gid=1448563947

Then AI will have to become really smarter than very large groups of people who will try to control the world. And people by that time will surely be ready more than now. I am sure that the laws of physics allow the quick destruction of humanity, but it seems to me that without a swarm of self-reproducing nanorobots, the probability of our survival after the creation of the first AGI exceeds 50%.

It seems that this option leaves more chances for the victory for humanity than the gray goo scenario. And even if we screw up for the first time, it can be fixed. Of course, this does not eliminate the need for AI alignment efforts anyway.

Is AI Foom possible if even the godlike superintelligence cannot create gray goo? Some doubt that nanobots so quickly reproducing are possible. Without this, the ability for AI to quickly take over the world in the coming years will be significantly reduced.

It seems Russell does not agree with what is considered an LW consensus. From ’Architects of Intelligence The truth about AI from the people building it’:

When [the first AGI is created], it’s not going to be a single finishing line that we cross. It’s going to be along several dimensions.

[...]

I do think that I’m an optimist. I think there’s a long way to go. We are just scratching the surface of this control problem, but the first scratching seems to be productive, and so I’m reasonably optimistic