All of naimenz's Comments + Replies

Can we expect more value from AI alignment than from an ASI with the goal of running alternate trajectories of our universe?

Is your suggestion to run this system as a source of value, simulating lives for their own sake rather than to improve the quality of life of sentient beings in our universe? Our history (and present) aren't exactly utopian, and I don't see any real reason to believe that slight variations on it would lead to anything happier.

I think we can expect to achieve a lot more than that from a properly aligned AGI. There is so much suffering that could be alleviated right now with proper coordination, as a lower bound on how much better it could be than just effectively running copies of our timeline but at lower resolution.

1Maxime Riché1yI am thinking about if we should reasonably expect to produce better result by trying to align an AGI with our value than by simulating a lot of alternate universes. I am not saying that this is net-negative or net-positive. It seems to me that the expected value of both cases may be identical. Also by history, I also meant the future, not only the past and present. (I edited the question to replace "histories" by "trajectories")
Our take on CHAI’s research agenda in under 1500 words

Is the "Going Beyond Agents" section part of CHAI's research agenda, or your take on further challenges from an embedded agency perspective?

2alexflint1yThat part is my take on further challenges from an embedded agency perspective.
Rationality: An Introduction

In the example with Bob, surely the odds of Bob having a crush on you after winking (2:1) should be higher than a random person winking at you (given as 10:1), as we already have reason to suspect that Bob is more likely to have a crush on you than some random person not part of the six.

1jeronimo1962yWhat Liliet B said. Low priors will screw with you even after a "definitive" experiment. You might also want to take a look at this: https://www.lesswrong.com/posts/XTXWPQSEgoMkAupKt/an-intuitive-explanation-of-bayes-s-theorem [https://www.lesswrong.com/posts/XTXWPQSEgoMkAupKt/an-intuitive-explanation-of-bayes-s-theorem]
2Liliet B2yNot if you consider that the 1:5 figure constrains that ONLY one person among the six has a crush on you. If you learn for a fact one does, you'll also immediately know the others all don't. Which is not true for a random selection of students - you could randomly pick six that all have a crush on you. Bob belongs to a group in which you know for a fact five people DON'T have a crush on you. So you have evidence lowering Bob's odds relative to a random winker. Either that, or it doesn't matter how many actually have a crush on you, you're looking for the specific one you have definite evidence about. For thisy, a random winker is not qualified to enter the comparison at all - if they're not one of the six, they're not the person you're looking for. So Bob might have a crush on you AND not be the person you're looking for, although his odds are higher than those of the other five you don't have any evidence about. That's the interpretations that make the math not wrong, anyway. If you only know that "at least one of them has a crush on me" and more than one could potentially satisfy your search criteria, the 1:5 figure is not the right odds.