We founded Anthropic because we believe the impact of AI might be comparable to that of the industrial and scientific revolutions, but we aren’t confident it will go well. And we also believe this level of impact could start to arrive soon – perhaps in the coming decade. This view...
Summary DeepMind’s recent paper, A Generalist Agent, catalyzed a wave of discourse regarding the speed at which current artificial intelligence systems are improving and the risks posed by these increasingly advanced systems. We aim to make Gato accessible to non-technical folks by: (i) providing a non-technical summary, and (ii) discussing...
We founded Anthropic because we believe the impact of AI might be comparable to that of the industrial and scientific revolutions, but we aren’t confident it will go well. And we also believe this level of impact could start to arrive soon – perhaps in the coming decade.
This view may sound implausible or grandiose, and there are good reasons to be skeptical of it. For one thing, almost everyone who has said “the thing we’re working on might be one of the biggest developments in history” has been wrong, often laughably so. Nevertheless, we believe there is enough evidence to seriously prepare for a world where rapid AI progress leads to transformative... (read 6567 more words →)
Lots of good information - for people considering joining a nonprofit board and reading this, be sure to ask about whether the org carries directors and officers (often referred to as D&O) insurance - otherwise you could be personally liable for problems that happen!
Interesting post, and I generally agree.
One note - you appear to be quoting David Chapman, not Yudkowsky. The Twitter post you linked to was written by Chapman. It's also not exactly what the tweet says. Can you maybe update to reflect that it's a Chapman quote, or directly link to where Yudkowsky said this? Apologies if I'm missing something obvious in the link.
We were surprised to find a decrease in publications on the arXiv in recent years, but identified the cause for the decrease as spurious and fixed the issue in the published dataset (details in Fig. 4).
I'd be interested in hearing more about how the decrease was determined to be spurious; I looked at Fig. 4 but am not understanding how that decision was made based on the figure, if that was the intention.
This is great, thanks so much for pulling this together (and for linking to our Gato explainer!)
It just so happens I'm working with a group of people through the Cambridge EA Technical AI alignment curriculum, and this idea of IDA is what week 5 is all about - lots of further reading for those who want.
One prompt in the weekly curriculum asks whether there are any tasks that cannot easily be broken down in the way described above, and therefore might not be useful for IDA. One thing I can thing of offhand is large leaps in scientific understanding. For example, if you took 20 physicists and gave them the problems of the day, it's not clear that they ever would have come up with Einstein's theory of relativity. Given that problem, I wonder what the implications are for trying to use IDA to create AGI - does this mean there are certain types of tasks that a IDA-based AGI will not be so good at?
Thanks for reaching out! At the moment it's only here and on the EA forum; we may be publishing it elsewhere but that's still in the works. If you want to PM me with your request I'll see what I can do!
DeepMind’s recent paper, A Generalist Agent, catalyzed a wave of discourse regarding the speed at which current artificial intelligence systems are improving and the risks posed by these increasingly advanced systems. We aim to make Gato accessible to non-technical folks by: (i) providing a non-technical summary, and (ii) discussing the relevant implications related to existential risk and AI policy.
DeepMind has just introduced its new agent, Gato: the most general machine learning (ML) model to date. If you’re familiar with arguments for the potential risks posed by advanced AI systems, you’ll know the term general carries strong implications. Today’s ML systems are advancing quickly; however, even the best systems we see are narrow in the tasks... (read 1654 more words →)