This feels worth trying to me
I do know that I want my own children to stay off social media, and minimize their ownership and use of smart phones, for as long as they possibly can. And that I intend to spend quite a lot of my available points, if needed, to fight for this. And that if I was running a school I’d do my best to shut the phones down during school hours.
(...)
We can also help this along by improving alternatives to phone use. If children aren’t allowed to go places without adults knowing, or worse adults driving them and coming along and watching them, what do you think they are going to do all day? What choices do they have?
I'm not certain whether my intuition should be trusted here, since this is definitely the kind of thing my brain would form a habit of rationalizing about. But my guess is that I would've been way worse off without phones/social media/stuff. I didn't really have any great alternatives to socializing on the internet—the only people I ever interacted with in person were devout Christians.
So I tentatively think it might be better to really focus on the improving alternatives part first? I'm sure I would've been much better off if I had good in-person friends, but I don't think not having access to social media would have really helped with that, it'd just have meant I wouldn't have any good friends at all.
(I would expect Zvi in particular has good enough parenting skills to not run into that. But I know a lot of people with terrible parents who think they can fix the problem just by monitoring their children's access to technology, which seems terrible for them to me? So I worry about how good it is as general advice.)
I don't think you understand what mathematicians mean by the word "complete." It means that all theorems which can be stated in the system can also be proven in the system (or something similar).
A (late) section of Project Lawful argues that there would likely be acausal coordination to avoid pessimizing the utility function (of anyone you are coordinating with), as well as perhaps to actively prevent utility function pessimization.
this already seems pretty good imo
I don't think security mindset means "look for flaws." That's ordinary paranoia. Security mindset is something closer to "you better have a really good reason to believe that there aren't any flaws whatsoever." My model is something like "A hard part of developing an alignment plan is figuring out how to ensure there aren't any flaws, and coming up with flawed clever schemes isn't very useful for that. Once we know how to make robust systems, it'll be more clear to us whether we should go for melting GPUs or simulating researchers or whatnot."
That said, I have a lot of respect for the idea that coming up with clever schemes is potentially more dignified than shooting everything down, even if clever schemes are unlikely to help much. I respect carado a lot for doing the brainstorming.
I mostly expect by the time we know how to make a seed superintelligence and give it a particular utility function... well, first of all the world has probably already ended, but second of all I would expect progress on corrigibility and such to have been made and probably to present better avenues.
If Omega handed me aligned-AI-part-2.exe, I'm not quite sure how I would use it to save the world? I think probably trying to just work on the utility function outside of a simulation is better, but if you are really running out of time then sure, I guess you could try to get it to simulate humans until they figure it out. I'm not very convinced that referring to a thing a person would have done in a hypothetical scenario is a robust method of getting that to happen, though?
I have a pretty strong heuristic that clever schemes like this one are pretty doomed. The proposal seems to lack security mindset, as Eliezer would put it.
The most immediate/simple concrete objection I have is that no one has any idea how to create aligned-AI-part-2.exe? I don't think figuring out what we'd do if we knew how to make a program like that is really the difficult part here.
CDT gives into blackmail (such as the basilisk), whereas timeless decision theories do not.
I think abuse issues in rationalist communities are worth discussing, but I don't think people who have been excluded from the community for years are a very productive place to begin such a discussion.