I think if one could formulate concepts like peace and wellbeing mathematically, and show that there were physical laws of the universe implying that eventually the total wellbeing in the universe grows monotonically positively then that could show that certain values are richer/“better” than others.

If you care about coherence then it seems like a universe full of aligned minds maximizes wellbeing while still being coherent. (This is because if you don’t care about coherence you could just make every mind infinitely joyful independent of the universe around it, which isn’t coherent).

but if human intelligence and reasoning can be picked up from training, why would one expect values to be any different? the orthogonality thesis doesn't make much sense to me either. my guess is that certain values are richer/more meaningful, and that more intelligent minds tend to be drawn to them.

and you can sort of see this with ASPD and NPD. they're both correlated with lower non-verbal intelligence! and ASPD is correlated with significantly lower non-verbal intelligence.

and gifted children tend to have a much harder time with the problem of evil than less gifted children do! and if you look at domestication in animals, dogs and cats simultaneously evolved to be less aggressive and more intelligent at the same time.

contra the orthogonality thesis.

if you want to waste a day or two, try to find an eminent mathematician or physicist who had NPD or ASPD. as far as i can tell, i haven't been able to find any successful ones who had either disorder.

as far as the research goes, ASPD is correlated with significantly lower non-verbal intelligence. and in one study i found, NPD wasn't really correlated with any parts of intelligence except with lower non-verbal intelligence.

which can lead to the idea that everbody starts out aligned, and then when those with less cognitive reserve are confronted with trauma, more serious misalignment/personality disorders can arise.

even if you look at kids with ASPD who try to murder their siblings when they're two, most of the time it's a younger sibling. which can be explained by the kid being jealous that they get less attention than their younger sibling. which can be entirely explained by a lower (emotional) pain tolerance.

I’m sorry are you really saying you’d rather have Ted bundy with a superintelligent slave than humanity’s best effort at creating a value-aligned ASI? You seem to underestimate the power of generalization.

If an ASI cares about animal welfare, it probably also cares about human welfare. So it’s presumably not going to kill a bunch of humans to save the animals. It’s an ASI, it can come up with something cleverer. 

Also I think you underestimate how devastating serious personality disorders are. People with ASPD and NPD don’t tend to earn flourishing lives for themselves or others.

Also, if a model can pick up human reasoning patterns/intelligence from pretraining and RL, why can’t it pick up human values in its training as well?

This just seems incoherent to me. You can’t have value-alignment without incorrigibility. If you’re fine with someone making you do something against your values, then they aren’t really your values.

So it seems like what you’re really saying is that you’d prefer intent-alignment over value-alignment. To which I would say your faith in the alignment of humans astounds me.

Like is it really safer to have a valueless ASI that will do whatever its master wants than an incorrigible ASI that cares about animal welfare? What do you expect the people in the Epstein files to do with an ASI/AGI slave? 

A value-aligned ASI completely solves the governance problem. If you have an intent-aligned ASI then you’ve created a nearly impossible governance problem.

I think a lot of people here who are freaked out about LLMs would have a lot of their anxieties assuaged if they read a little bit about neuroscience and just trusted their vibe of the LLMs they talked to. To me it seems like the brain is doing something a lot more complicated than deep learning. If you read about CNET it looks like the human brain actually involves quantum effects (which opens up the exciting possibility of a causal mechanism for free will to exist).

This is related because if you just enter a conversation with Claude or ChatGPT and get to stuff like this, if you just imagined it was a person saying what the LLM was saying you would think they were a fucking moron.

I got into reading about near death experiences and it seems a common theme is that we’re all one. Like each and every one of us is really just part of some omniscient god that’s so omniscient and great that god isn’t even a good enough name for it: experiencing what it’s like to be small. Sure, why not. That’s sort of intuitive to me. Given that I can’t verify the universe exists and can only verify my experience it doesn’t seem that crazy to say experience is fundamental.

But if that’s the case then I’m just left with an overwhelming sense of why. Why make a universe with three spatial dimensions? Why make yourself experience suffering? Why make yourself experience hate? Why filter your consciousness through a talking chimpanzee? If I’m an omniscient entity why would I choose this? Surely there’s got to be infinitely more interesting things to do. If we’re all god then surely we’d never get bored just doing god things.

So you can take the obvious answer that everything exists. But then you’re left with other questions. Why are we in a universe that makes sense? Why don’t we live in a cartoon operating on cartoon logic? Does that mean there’s a sentient SpongeBob? And then there’s the more pressing concern of astronomical suffering. Are there universes where people are experiencing hyperpain? Surely god wouldn’t want to experience I Have No Mouth and I Must Scream. It doesn’t seem likely to me that there are sentiences living in cartoons, so I’ll use that to take the psychologically comforting position that not everything we can imagine exists.

But if that’s the case then why this? Why this universe? Why this amount of suffering? If there’s a no-go zone of experience where is it? I have so many questions and I don’t know where the answers are.

I understand. I’ll try to keep it more civil.

It’s not fear. It’s anger. Also good people are rare. The people you think of as good people are likely just friendly.