I'm an admin of this site; I work full-time on trying to help people on LessWrong refine the art of human rationality.
Longer bio: www.lesswrong.com/posts/aG74jJkiPccqdkK3c/the-lesswrong-team-page-under-construction#Ben_Pace___Benito
Posts
Wiki Contributions
Comments
"will the first animals that take over the world be able to solve the Riemann hypothesis", and the answer is no because humans intelligence, while general, is still pointed more at civilisation-building-style tasks than mathematics.
Pardon the semantics, but I think the question you want to use here is "will the first animals that take over the world have already solved the Riemann hypothesis". IMO humans do have the ability ("can") to solve the Riemann hypothesis, and the point you're making is just about the ordering in which we've done things.
This post feels like a fantasy description of a better society, one that I would internally label "wish-fulfilment". And yet it is history! So it makes me more hopeful about the world. And thus I find it beautiful.
Just for record-keeping, here is the OWID global death tracker (from google), with the vertical line at the point when the comment was written.
Brief review: I think this post represents a realization many people around here have made, and says it clearly. I think it's fine to keep it as a record that people used to be blasé about the ease of secrecy, and later learned that it was much more complex than they thought. I think I'm at +1.
My quick two-line review is something like: this post (and its sequel) is an artifact from someone with an interesting perspective on the world looking at the whole problem and trying to communicate their practical perspective. I don't really share this perspective, but it is looking at enough of the real things, and differently enough to the other perspectives I hear, that I am personally glad to have engaged with it. +4.
Most people have not put tens of thousands of deliberate hours of practice into their writing skills so do not have the clarity to be able to say what they think shortly, and this lack of skill is typically why their writing is long. Eliezer has worked hard to be able to write clearly, and also to build a rare skill of being able to expose more of the cognition behind a thought as he writes longer, which is in many important domains more valuable to do than just stating the output of the cognition.
I'm saying: Eliezer's has built the skill to say his thoughts precisely and clearly; but he has also built the next-level skill of being able to expose the cognition behind a thought, and this is the sort of valuable length that he hopes to have in his writing.
"Search versus design" explores the basic way we build and trust systems in the world. A few notes:
- My favorite part is the definitions about an abstraction layer being an artifact combined with a helpful story about it. It helps me see the world as a series of abstraction layers. We're not actually close to true reality, we are very much living within abstraction layers — the simple stories we are able to tell about the artefacts we build. A world built by AIs will be far less comprehensible than the world we live in today. (Much more like biology is, except made by something that is much smarter and faster than us instead of stupider and slower.)
- The post puts in the time to bring into the conversation a lot of other work that attempts to help build simple stories about the AI artefacts that we are building, which I appreciate.
- The post is pretty simply written, for me, and I understand all the examples and arguments.
- It also attempts to (briefly) describe a novel direction of future work for solving the problem of building untrustworthy systems with selection, and that's exciting.
For looking at the alignment problem clearly and with a subtly different frame than other discussions, one that resonates for me, and that points to new frames for a solution, I am voting this post +9.
Indeed the books are not yet in-stock in Amazon UK.
I am also quite interested to read this sequence.
(I agree with this short post, and enjoyed reading it.)