LESSWRONG
LW

Orthogonality ThesisSite Meta
Personal Blog

3

[ Question ]

What would a post that argues against the Orthogonality Thesis that LessWrong users approve of look like?

by Thoth Hermes
3rd Jun 2023
1 min read
A
1
3

3

Orthogonality ThesisSite Meta
Personal Blog

3

What would a post that argues against the Orthogonality Thesis that LessWrong users approve of look like?
13Charlie Steiner
9Shmi
1green_leaf
New Answer
New Comment

1 Answers sorted by
top scoring

Charlie Steiner

Jun 03, 2023

137

There are plenty of good posts that contradict a "strict" orthogonality thesis by showing correlation between capabilities and various values-related properties (scaling laws / inverse scaling laws).

What really gets you downvoted is the claim that super-intelligent AI cannot want things that are bad for humanity, or even agitating that we should give that idea serious weight.

What also gets you downvoted is the in-between claim that all the scaling laws tend towards superhuman morality and everything will work out fine, no need to be worried or spend lots of hours working.

How to make a successful piece in the latter categories? Simple - just be right, for communcable reasons. Simple, but maybe not possible.

Add Comment
2 comments, sorted by
top scoring
Click to highlight new comments since: Today at 5:25 AM
[-]Shmi2y98

Thoughtfully engaging with the existing body of literature might help. Show that you understand the claims, the counter-claims, the arguments for and against. Show that your argument is novel and interesting, not something that has been already put forward and critiqued numerous times. Basically, whatever makes a good scientific paper.

Reply
[-]green_leaf2y12

It would bring on an enormous amount of new evidence, since the position of the orthogonality thesis is so strong (rather than arguing from some vague and visibly false philosophical assumptions).

Reply
Moderation Log
Curated and popular this week
A
1
2

This question is motivated by the following reasons:

Not many pieces exist that argue against the Orthogonality Thesis (on LessWrong, or anywhere, to my knowledge). Of those that do, none have received positive feedback.

Commenters on those pieces have stated that it is not, in principle, impossible that they would up-vote such a piece, only that none thus far have met or exceeded their standards for what they would consider to be a successful attempt (even if they were not ultimately persuaded by the arguments).

What attributes would a "successful attempt" (even one that does not persuade you to disbelieve the Orthogonality Thesis) have?