Running Lightcone Infrastructure, which runs LessWrong and Lighthaven.space. You can reach me at habryka@lesswrong.com.
(I have signed no contracts or agreements whose existence I cannot mention, which I am mentioning here as a canary)
My sense is political staffers and politicians aren't that great at predicting their future epistemic states this way, and so you won't get great answers for this question. I do think it's a really important one to model!
I think the core argument is "if you want to slow down, or somehow impose restrictions on AI research and deployment, you need some way of defining thresholds. Also, most policymaker's cruxes appear to be that AI will not be a big deal, but if they thought it was going to be a big deal they would totally want to regulate it much more. Therefore, having policy proposals that can use future eval results as a triggering mechanism is politically more feasible, and also, epistemically helpful since it allows people who do think it will be a big deal to establish a track record".
I find these arguments reasonably compelling, FWIW.
Thank you!
For lightcone's contact details I asked on LW intercom. Feels rude to put someone's phone number here, so if you're doing the same as me, I'm not gonna save you that step.
Reasonable prior, but my phone number is already publicly visible at the top of this post, so feel free to share further.
making sure there are really high standards for safety and that there isn't going to be danger what these AIs are doing
Ah yes, a great description of Anthropic's safety actions. I don't think anyone serious at Anthropic believes that they "made sure there isn't going to be danger from these AIs are doing". Indeed, many (most?) of their safety people assign double-digits probabilities to catastrophic outcomes from advanced AI system.
I do think this was a predictable quite bad consequence of Dario's essay (as well as his other essays which heavily downplay or completely omit any discussion of risks). My guess is it will majorly contribute to reckless racing while giving people a false impression of how good we are doing on actually making things safe.
Yep, when the fundraising post went live, i.e. November 29th.
Those were mostly already in-flight, so not counterfactual (and also the fundraising post still has the donation link at the top), but I do expect at least some effect!
Oops, you're right, fixed. That was just an accident.
It's the last 6 hours of the fundraiser and we have met our $2M goal! This was roughly the "we will continue existing and not go bankrupt" threshold, which was the most important one to hit.
Thank you so much to everyone who made it happen. I really did not expect that we would end up being able to raise this much funding without large donations from major philanthropists, and I am extremely grateful to have so much support from such a large community.
Let's make the last few hours in the fundraiser count, and then me and the Lightcone team will buckle down and make sure all of these donations were worth it.
I didn't watch the whole video (it's 2 hours long), but like I remember the process by which the books were written was described in the book. Mostly that they were written as a result of taped conversations between Feynman and a friend of his. This makes it pretty unlikely the books say anything that Feynman didn't actually say (since the conversations were taped), and also is pretty transparent in the book. It also seems kind of more likely to be true than an autobiography, since the impressions have filtered through at least one other person.
(I am not a huge fan of this post, but I think it's reasonable for people to care about how society orients towards x-risk and AI concerns, and as such to actively want to not screen off evidence, and take responsibility for what people affiliated with you say on the internet. So I don't think this is great advice.
I am actively subscribed to lots of people who I expect to say wrong and dumb things, because it's important to me that I correct people and avoid misunderstandings, especially when someone might mistake my opinion for the opinion of the people saying dumb stuff)