Sorted by New

Wiki Contributions


Ngo and Yudkowsky on AI capability gains

I think I might actually be happy to take e.g. the Bellman equation, a fundamental equation in RL, as a basic expression of consistent utilities and thereby claim value iteration, Q-learning, and deep Q-learning all as predictions/applications of utility theory. Certainly this seems fair if you claim applications of the central limit theorem for probability theory.

To expand a bit, the Bellman equation only expresses a certain consistency condition among utilities. The expected utility of this state must equal its immediate utility plus the best expected utility among each possible next state I may choose. Start with some random utilities assigned to states, gradually update them to be consistent, and you get optimal behavior. Huge parts of RL are centered around this equation, including e.g. DeepMind using DQNs to crack Atari games.

I understand Eliezer's frustration in answering this question. The response to "What predictions/applications does utility theory have?" in regards to intelligent behavior is, essentially, "Everything and nothing."

Orlando, FL – ACX Meetups Everywhere 2021

Hey all, organizer here. I don't know if you'll automatically get notified of this message, but I don't have emails for everyone. I just wanted to give some parking info. You can get a daily parking pass here: You can get a virtual pass and they'll check by plate. It's $3. I'd recommend parking in Garage A or I. Hope to see everyone there!

MIRI location optimization (and related topics) discussion

From what I understand, it's difficult enough to get an abortion as it is. Clinics are rather rare, insurance doesn't always cover it, there may be mandatory waiting periods and counseling, etc. I don't think it would be impossible to still get one, but the added inconvenience is not trivial.  At minimum, a big increase in travel time and probable insurance complications. But if someone here knows more than me, I'd very much like to hear it.

MIRI location optimization (and related topics) discussion

I'd like to note that Texas is passing strong restrictions on abortion. They've passed a "heartbeat bill" banning abortions after six weeks, and it seems likely that they'll pass a trigger bill outlawing abortion almost entirely, contingent on the Supreme Court overturning Roe v Wade. 

I'm not a Supreme Court expert, but I know people who are sincerely worried about Roe v Wade being undone. This would be a pretty big deal breaker for my fiancée (and by extension myself). From what I read, the Supreme Court will make a Roe v Wade ruling in the middle of 2022.

Does this factor into your considerations? I feel like this would be a pretty big deal for the rationalist community at large.

TAISU - Technical AI Safety Unconference

I think you need to fix the days listed on the application form, they say August 17th - 20th.

Micro feedback loops and learning

I don't have a wonderful example of this insta-feedback (which definitely sounds ideal for learning), but I've gotten annoyed lately with any math book that doesn't have exercises. Some of the books on MIRI's Research Guide list are like this, and it really boggles my mind how anyone could learn math from a book when they don't have anything to practice with. So I'm getting more selective.

Even some books with exercises are super hard, and really don't have any kind of walkthrough process. AI: A Modern Approach is a critically-acclaimed textbook, but has little to no build up in the difficulty of its exercises, and little to help you if you get lost. Right now I'm reading How to Prove It, which is *super* good. The whole book is one big walkthrough on mathematical proof, down to how to organize your scratch work. It has tons of exercises with varying difficulty, with some answers and hints. It's much better feedback, and is helping me a lot more, although the material is comparatively simple.

A Rationalist's Guide to...

Well, I tend to throw them onto my general to-read list, so I'm not entirely sure. A few I remember are Godel, Escher, Bach, Judgment Under Uncertainty: Heuristics and Biases, Influence: Science and Practice, The End of Time, QED: The Strange Theory of Light and Matter, The Feynman Lectures, Surely You're Joking Mr. Feynman, Probability Theory: The Logic of Science, Probabilistic Inference in Intelligent Systems, and Player of Games. There's a longer list here, but it's marked as outdated.

A Rationalist's Guide to...

Sounds awesome! A meatspace group would be great, I'm sure. One of my issues with self-study is having nobody to go to when I have questions or don't understand something. Having an empirical goal can also tell you if you've succeeded or failed in your attempt to learn the art.

A Rationalist's Guide to...

I definitely agree that there's a bigger issue, but I think this could be a good small-scale test. Can we apply or own individual rationality to pick up skills relevant to us and distinguish between good and bad practices? Are we able to coordinate as a community to distinguish between good and bad science? Rationality should in theory be able to work on big problems, but we're never going to be able to craft the perfect art without being able to test it on smaller problems first and hone the skill.

So yeah. I think a guide putting together good resources and also including practical advice in the posts and comments could be useful. Something like this could be the start of answering Eliezer's questions "how do we test which schools of rationality work" and "how do we fight akrasia". That second question might be easier once we've seen the skills work in practice. Maybe I should make a guide first to get the ball rolling, but I'm not sure I know a topic in-depth enough to craft one just yet.