Thank you for the reply. This has been an important takeaway from this post: There are significant groups (or at least informal networks) doing meaningful work that don't congregate primarily on LW or Twitter. As I said on another comment - that is encouraging! I wish this was more explicit knowledge within LW - it might give things more of a sense of hope around here.
The first question that comes to mind: Is there any sense of direction on policy proposals that might actually have a chance of getting somewhere? Something like: "Regulating card production" has momentum or anything like that?
Are policy proposals floating around even the kind that would not-kill-everyone? or is it more "Mundane Utility" type stuff, to steal the Zvi term.
This is fantastic information, thank you for taking the time.
One of my big takeaways from all of the comments on this post is a big update to my understanding of the "AI Risk" community and that LW was not actually the epicenter and there were significant efforts being made elsewhere that didn't necessarily circle back to LW.
That is very encouraging actually!
The other big update is what you say: There were just so few people with the time and ability to work on these things.
Someone else said similar about the basement possibility, which I did not know.
Interesting questions raised though: Even if it wasn't clear until GPT, wouldn't that still have left something like 2-3 years?
Granted that is not 10-20 years.
It seems we all, collectively, did not update nearly enough on ChatGPT-2?
Point taken! This I just plain did not know and I will update based on that.
It does not make sense to focus on public policy if basement guy is the primary actor.
The UK funding is far and away the biggest win to date, no doubt.
And all this is despite the immediately-preceding public relations catastrophe of FTX!
Do you feel that FTX/EA is that closely tied in the public mind and was a major setback for AI alignment? That is not my model at all.
We all know they are inextricably tied, but I suspect if you were to ask they very people in those same polls if they knew that SBF supported AI risk research they wouldn't know or care.
I've added an Edit to the post to include that right up front.
I asked this of another commenter, but I will ask you too:
Do you feel it is accurate to say that many or most people working on this (including and especially Eliezer) at the time considered nuts and bolts alignment work to be the only worthwhile path? Given what info was available at the time.
And that widescale public persuasion / overton window / policy making was not likely to matter as the most scenarios were Foom based?
You reminded me of that famous tweet:
Sci-Fi Author: In my book I invented the Torment Nexus as a cautionary tale
Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus
But more seriously, I think this is a real point that has not been explored enough in alignment circles.
I have encountered a large number of people - in fact probably almost all people I discuss AI with - who I would call "normal people". Just regular, moderately intelligent people going about their lives for which "don't invent a God-Like AI" is so obvious it is almost a truism.
It is just patently obvious based on... (read more)
I'm starting to draw a couple of conclusions for myself from this thread as I get a better understanding of the history.
Do you feel it is accurate to say that many or most people working on this (including and especially Eliezer) at the time considered nuts and bolts alignment work to be the only worthwhile path? Given what info was available at the time.
And that widescale public persuasion / overton window / policy making was not likely to matter as the most scenarios were Foom based?
It is pretty interesting that the previous discussion in all these years kind've zoomed in on only that.
Maybe someone more experienced than me will do a post-mortem of why it did not work out like that at all and we seem not to have seen that coming or even given it meaningful probability.
Our epistemic rationality has probably gotten way ahead of our instrumental rationality
-Scott Alexander
This is a question post:
Why was the AI Alignment community so unprepared for engaging with the wider world when the moment finally came?
EDIT Based on comment feedback: This is a genuine question about why something that is so obvious now with hindsight bias, was not clear back then and understand why not. Not an attempt to cast blame on any person or group.
I have been a LW reader for at least 10 years, but I confess that until the last ~1.5 years I mostly watched the AI alignment conversation float by. I knew of the work, but... (read 487 more words →)
Excellent post thank you for sharing. My comment is a bit of a hijack, but your related post linked at the top that led to this one doesn't seem to have a way to comment so I thought I'd ask here.
In that post you outline your problems grappling with the editorial decisions of the Penguin Great Ideas series (in addition to the misogyny itself).
Is there a reason you chose the Penguin series instead of the Great Books of the Western World curriculum?
My impression is that list is much less editorialized than the Penguin list and may at least solve your problem of "why did this 21st century white guy decide to cherry pick so much misogynistic content out of this vast corpus".
At least with the Britannic list, it was team of 20th century white guys applying a semi-objective inclusion matrix and they didn't (to my knowledge) trim or edit any of the individual works.