AprilSR

Wiki Contributions

Comments

AI systems, such as Large Language Models (LLMs), are trained on human data and designed by human engineers. It's impossible for them to exceed the bounds of human knowledge and expertise, as they're inherently limited by the information they've been exposed to.

Maybe, on current algorithms, LLMs run into a plateau around the level of human expertise. That does seem plausible. But it is not because being trained on human data necessarily limits you to human level!

Accurately predicting human text is much harder than just writing stuff on the internet. If GPT were to perfect the skill it is being trained on, it would have to be much smarter than a human!

Even if shut down in particular isn't something we want it to be indifferent to, I think being able to make an agent indifferent to something is very plausibly useful for designing it to be corrigible?

I don't think you can have particularly high confidence one way or the other without just thinking about AI in enough detail to have an understanding of the different ways that AI development could end up shaking out in. There isn't a royal road.

Both the "doom is disjunctive" and "AI is just like other technologies" arguments really need a lot more elaboration to be convincing, but—personally I find the argument that AI is different from other technologies pretty obvious and I have a hard time imagining what the counterargument would be.

I can imagine a world where LLMs tend to fall into local maxima where they get really good at imitation or simulation, and then they plateau (perhaps only until their developers figure out what adjustments need to be made). But I don't have a good enough model of LLMs to be very sure whether that will happen or not.

I really like the "when you don't have a good detailed model you need to figure out what space you should have the maximum entropy distribution over" framing

I think abuse issues in rationalist communities are worth discussing, but I don't think people who have been excluded from the community for years are a very productive place to begin such a discussion.

This feels worth trying to me

I do know that I want my own children to stay off social media, and minimize their ownership and use of smart phones, for as long as they possibly can. And that I intend to spend quite a lot of my available points, if needed, to fight for this. And that if I was running a school I’d do my best to shut the phones down during school hours.

(...)

We can also help this along by improving alternatives to phone use. If children aren’t allowed to go places without adults knowing, or worse adults driving them and coming along and watching them, what do you think they are going to do all day? What choices do they have?


I'm not certain whether my intuition should be trusted here, since this is definitely the kind of thing my brain would form a habit of rationalizing about. But my guess is that I would've been way worse off without phones/social media/stuff. I didn't really have any great alternatives to socializing on the internet—the only people I ever interacted with in person were devout Christians.

So I tentatively think it might be better to really focus on the improving alternatives part first? I'm sure I would've been much better off if I had good in-person friends, but I don't think not having access to social media would have really helped with that, it'd just have meant I wouldn't have any good friends at all.

(I would expect Zvi in particular has good enough parenting skills to not run into that. But I know a lot of people with terrible parents who think they can fix the problem just by monitoring their children's access to technology, which seems terrible for them to me? So I worry about how good it is as general advice.)

I don't think you understand what mathematicians mean by the word "complete." It means that all theorems which can be stated in the system can also be proven in the system (or something similar).

A (late) section of Project Lawful argues that there would likely be acausal coordination to avoid pessimizing the utility function (of anyone you are coordinating with), as well as perhaps to actively prevent utility function pessimization.

Load More