All Posts

Sorted by Magic (New & Upvoted)

Monday, December 2nd 2019
Mon, Dec 2nd 2019

Shortform [Beta]
53Buck2mo[I'm not sure how good this is, it was interesting to me to think about, idk if it's useful, I wrote it quickly.] Over the last year, I internalized Bayes' Theorem much more than I previously had; this led me to noticing that when I applied it in my life it tended to have counterintuitive results; after thinking about it for a while, I concluded that my intuitions were right and I was using Bayes wrong. (I'm going to call Bayes' Theorem "Bayes" from now on.) Before I can tell you about that, I need to make sure you're thinking about Bayes in terms of ratios rather than fractions. Bayes is enormously easier to understand and use when described in terms of ratios. For example: Suppose that 1% of women have a particular type of breast cancer, and a mammogram is 20 times more likely to return a positive result if you do have breast cancer, and you want to know the probability that you have breast cancer if you got that positive result. The prior probability ratio is 1:99, and the likelihood ratio is 20:1, so the posterior probability is 1∗20:99∗1 = 20:99, so you have probability of 20/(20+99) of having breast cancer. I think that this is absurdly easier than using the fraction formulation. I think that teaching the fraction formulation is the single biggest didactic mistake that I am aware of in any field. -------------------------------------------------------------------------------- Anyway, a year or so ago I got into the habit of calculating things using Bayes whenever they came up in my life, and I quickly noticed that Bayes seemed surprisingly aggressive to me. For example, the first time I went to the Hot Tubs of Berkeley, a hot tub rental place near my house, I saw a friend of mine there. I wondered how regularly he went there. Consider the hypotheses of "he goes here three times a week" and "he goes here once a month". The likelihood ratio is about 12x in favor of the former hypothesis. So if I previously was ten to one against the three-times-a-week hyp
29Ben Pace2moGood posts you might want to nominate in the 2018 Review I'm on track to nominate around 30 posts from 2018, which is a lot. Here is a list of about 30 further posts I looked at that I think were pretty good but didn't make my top list, in the hopes that others who did get value out of the posts will nominate their favourites. Each post has a note I wrote down for myself about the post. * Reasons compute may not drive AI capabilities growth [] * I don’t know if it’s good, but I’d like it to be reviewed to find out. * The Principled-Intelligence Hypothesis [] * Very interesting hypothesis generation. Unless it’s clearly falsified, I’d like to see it get built on. * Will AI See Sudden Progress? [] DONE * I think this post should be considered paired with Paul’s almost-identical post. It’s all exactly one conversation. * Personal Relationships with Goodness [] * This felt like a clear analysis of an idea and coming up with some hypotheses. I don’t think the hypotheses really captures what’s going on, and most of the frames here seem like they’ve caused a lot of people to do a lot of hurt to themselves, but it seemed like progress in that conversation. * Are ethical asymmetries from property rights? [] * Again, another very interesting hypothesis. * Incorrect Hypotheses Point to Correct Observations []
15ozziegooen2moI think one idea I'm excited about is the idea that predictions can be made of prediction accuracy. This seems pretty useful to me. EXAMPLE Say there's a forecaster Sophia who's making a bunch of predictions for pay. She uses her predictions to make a meta-prediction of her total prediction-score on a log-loss scoring function (on all predictions except her meta-predictions). She says that she's 90% sure that her total loss score will be between -5 and -12. The problem is that you probably don't think you can trust Sophia unless she has a lot of experience making similar forecasts. This is somewhat solved if you have a forecaster that you trust that can make a prediction based on Sophia's seeming ability and honesty. The naive thing would be for that forecaster to predict their own distribution of the log-loss of Sophia, but there's perhaps a simpler solution. If Sophia's provided loss distribution is correct, that would mean that she's calibrated in this dimension (basically, this is very similar to general forecast calibration). The trusted forecaster could forecast the adjustment made to her term, instead of forecasting the same distribution. Generally this would be in the direction of adding expected loss, as Sophia probably had more of an incentive to be overconfident (which would result in a low expected score from her) than underconfident. This could perhaps make sense as a percentage modifier (-30% points), a mean modifier (-3 to -8 points), or something else. External clients would probably learn not to trust Sophia's provided expected error directly, but instead the "adjusted" forecast. This can be quite useful. Now, if Sophia wants to try to "cheat the system" and claim that she's found new data that decreases her estimated error, the trusted forecaster will pay attention and modify their adjustment accordingly. Sophia will then need to provide solid evidence that she really believes her work and is really calibrated for the trusted forecaster to bud
5Naryan Wong2moThree Activities We Might Run at Our Next Rationality+ Retreat There is lots of context about who 'we' are, why I called it 'Rationality+', and what the 'retreat' is, but for the moment I'd just like to toss the activity ideas out into the community to see what happens. Happy to answer questions on any of the context or activities in the comments. 1. Tracking - The ability to follow how the components of a group (you, others, the environment) interact to co-create a group experience. The ability to observe how the group experience in turn affects the components. Like systems thinking for groups, in a felt-sense kind of way. a) Activity starts off with a meditation on one's own subjective experience - observing how body sensations, emotions, and thoughts arise and dissipate. Watch with equanimity if possible. b) Next - pair up, and explore how you might understand your partner's subjective experience in the same way that you just observed your own. Try observing their body language, your own mirror neurons, asking questions, making predictions/experiments. You can take turns or play with simultaneous exploration. c) Each pair finds another pair to make a group of four. Taking turns speaking, see if you can collectively discover a 'group subjective truth' that feels true to all of you, and observe how it changes through time. What happens when you align on a living group narrative? Are you able to co-create a direction or meaning for this group? It's possible that this activity could extend into a 1hr+ exploration with groups actually doing things together, while maintaining cohesion. 2. Meaning-making - The ability to find/create subjective relevance in things. Weaving 'random things that happen' into a more salient 'narrative of things that are important'. I speculate that this helps with learning, memory, and allows you to shape your attention to a 'thing'. By learning to 'meaning-make', one could frame experiences in a way that lets you instrumentally get mo
1alenglander2moSomething I've been thinking about recently. I've been reading several discussions surrounding potential risks from AI, especially the essays and interviews on AI Impacts. A lot of these discussions seem to me to center on trying to extrapolate from known data, or to analyze whether AI is or is not analogous to various historical transitions. But it seems to me that trying to reason based on historical precedent or extrapolated data is only one way of looking at these issues. The other way seems to be more like what Bostrom did in Superintelligence, which seems more like reasoning based on theoretical models of how AI works, what could go wrong, how the world would likely react, etc. It seems to me that the more you go with the historical analogies / extrapolated data approach, the more skeptical you'll be of claims from people claiming that AI risk is a huge problem. And conversely, the more you go with the reasoning from theoretical models approach, the more concerned you'll be. I'd probably put Robin Hanson somewhere close to the extreme end of the extrapolated data approach, and I'd put Eliezer Yudkowsky and Nick Bostrom close to the extreme end of the theoretical models approach. AI Impacts seems to fall closer to Hanson on this spectrum. Of course, there's no real hard line between the two approaches. Reasoning from historical precedent and extrapolated data necessarily requires some theoretical modeling, and vice versa. But I still think the basic distinction holds value. If this is right, then the question is how much weight should we put on each type of reasoning, and why? Thoughts?