Power vs Precision

[-]Scott Garrabrant4y230

I claim that conversations about good fuzzy abstract clusters are both difficult, and especially in the comparative advantage of LessWrong. I claim that LW was basically founded on analogy, specifically the analogy between human cognition and agent foundations/AI. The rest of the world is not very precise, and that is a point for LW having a precision comparative advantage, but the rest of the world seems to also be lacking abstraction skill.

The claim that precision requires purity leads to the conclusion that you have to choose between precision and abstraction, and it also leads to the conclusion that precision is fragile, and thus that you should expect a priori for there to less of it naturally, but it is otherwise symmetric between precision and abstraction. It is only saying that we should pick a lane, not which lane it should be.

I claim that when I look at science academia, I see a bunch of precision directed in inefficient ways, together with a general failure to generalize across pre-defined academic fields, and even more of a failure to generalize out of the field of "science" and into the real world.

Warning: I am mostly conflating abstraction with analogy with lack of precision above, and maybe the point is that we need precise analogies.

[-]ozziegooen4y130

Minor point:
You write,
> I totally agree that there are advantages to fuzzy clusters as well; what I actually think is that we should level up both specificity-superpowers and vagueness-superpowers.

My impression is that vagueness is generally considered undesirable in philosophy, where it has a fairly particular meaning.

https://en.wikipedia.org/wiki/Vagueness
https://plato.stanford.edu/entries/vagueness/

If you are going to use "vagueness-superpowers" to mean, "our ability working on big things that happen to be vague", you might want to include ambiguity, to be more complete.

My guess is that generality would be a better word here (or abstraction, but I think that's a bit different). My impression is that typically one could build abilities for both generalizations and abstractions. Hopefully the results of this work will not come with much vagueness.

https://www.lesswrong.com/posts/vPHduFch5W6dqoMMe/two-definitions-of-generalization

[-]gjm4y60

Tangential: I followed the sunscreen link, expecting to find something at the other end that in-some-technical-sense-supported the claim in question. In fact it did the reverse, citing the claim as a false thing some people say that has no truth to it.

It seems to me that the example would be a better one if the claim in question were technically true (despite being misleading to readers who aren't thinking very carefully). For instance, if it had said not "wearing sunscreen actually tends to increase risk of skin cancer" (which is just plain false) but "people who wear sunscreen more often are actually more likely to get skin cancer" (which may be true but doesn't imply the former, as explained at the far end of that link" it would be better.

[-]abramdemski4y*20

I think I endorse it as written.

First, the article endorses a correlational link between sunscreen and skin cancer:

Q. Is there evidence that sunscreen actually causes skin cancer?
A. No. These conclusions come incorrectly from studies where individuals who used sunscreen had a higher risk of skin cancer, including melanoma. This false association was made because the individuals who used sunscreen were the same ones who were traveling to sunnier climates and sunbathing. In other words, it was the high amounts of sun exposure, not the sunscreen, that elevated their risk of skin cancer.

They appear to deny a causal link, but I think they're really only denying that all else being equal adding sunscreen increases cancer risk; because later in the article, they affirm that numerous studies have found a causal link:

Many studies have demonstrated that individuals who use sunscreen tend to stay out in the sun for a longer period of time, and thus may actually increase their risk of skin cancer.

Thus, there is an actual causal link between wearing sunscreen and increased cancer risk (although the "may" complicates matters; I'm not sure whether it's just thrown in to weaken a conclusion they don't want to reinforce in readers' minds, vs really reflecting an unknown state of underlying knowledge). It's just that the link goes thru human behavior, which scientists don't always refer to as causal, since people are used to thinking of human behavior as free well.

OTOH, I do think that it would be a better example if it wasn't one that tended to confuse the reader, so I'm going to change it as you suggest.

[-]gjm4y40

That last bit (which, I confess, I hadn't actually noticed before) doesn't say that "wearing sunscreen actually tends to increase risk of skin cancer".

All it says is "may actually increase their risk". That says to me that they haven't actually done the calculations.
It's not even clear that they mean more than "staying out in the sun for longer may increase the risk" (i.e., not making any suggestion that the increase might outweigh the benefits of wearing sunscreen), though if that's all they mean then I'm not sure why they said "may".

They also don't actually cite any of those "many studies" which means that without doing a lot of digging (a) we can't tell how large the effect is and assess whether it might outweigh the benefits of sunscreen, and (b) we can't tell whether those studies actually distinguished between "people who were going to stay out for longer anyway tend to wear sunscreen" and "a given person on a given occasion will stay out longer if you make them wear sunscreen".

But I don't think your interpretation passes the smell test. If whoever wrote that page really believed that the overall effect of wearing sunscreen was to increase the risk of skin cancer (via making you stay out in the sun for longer), would they have said "We recommend sunscreen for skin cancer prevention"? Wouldn't the overall tone of the page be different? (At present they give a firm "no" to "Is there evidence that sunscreen actually causes skin cancer", they say "there are excellent studies that sunscreen protects against all three of the most common skin cancers", right up at the top in extra-large type there's "Don't let myths deter you from using it", they say "We recommend sunscreen for skin cancer prevention", etc., etc., etc.)

Again, that page doesn't say what "many studies" they have in mind. I had a quick look to see what I could find. The first thing I found was "Sunscreen use and duration of sun exposure: a double-blind, randomized trial" (Autier et al, JNCI, 1999). They gave participants either SPF10 or SPF30 sunscreen at random; neither experimenters nor participants knew which; they found that (presumably because they stayed out until they felt they were starting to burn) people with SPF30 sunscreen stayed out for about 25% longer. The two groups had similar overall amounts of sunburn or reddening. The study doesn't make any attempt to decide whether sunscreen use is likely to tend to produce more or less skin cancer. (The obvious guess after reading it is that effectively there's a control system that holds the level of sunburn roughly constant, and that maybe that also holds the level of skin cancer roughly constant.)

The second thing I found was "Sunscreen use and melanoma risk among young Australian adults" (Watts et al, JAMA Dermatology, 2018). This was just associational; they looked at sunscreen use in childhood and lifetime sunscreen use, among subjects who had had a melanoma before age 40 and in a sample of others (including one sibling for each subject with melanoma, a curious design decision). They found that melanoma patients had made less use of sunscreen in childhood and over their lifetimes. There are obvious caveats about self-reported data.

The third thing I found was "Prevalence of sun protection use and sunburn and association of demographic and behavioral characteristics with sunburn among US adults" (Holman et al, JAMA Dermatology, 2018). Sunscreen use was among the things they found associated with more sunburn. Again, obvious caveats about self-reported data, and about the possibility that people who were going to go out in the sun more anyway are more likely to use sunscreen. As with the first study mentioned above, no assessment of overall impact on skin cancer risk.

One reference in that last one goes to another article by Autier et al, this one a review rather than a study in its own right. It seems to be a fairly consistent finding that when people are deliberately exposing themselves to the sun for tanning or similar purposes, they stay out for something like 25% longer if they're using sunscreen; it also seems to be the case that when they aren't doing that (they just happen to be out in the sun but that isn't a goal) they don't stay out for longer when using sunscreen. Again, no attempt at figuring out the overall impact on skin cancer risk.

After reading all that, I think that:

Wearing sunscreen does on balance lead to spending more time in the sun, at least for people who are spending time in the sun because they want to spend time in the sun.
It doesn't look as if it's known whether making someone wear sunscreen rather than not on a given occasion (assuming that they are seeking out time-in-the-sun) makes them more or less likely to get skin cancers overall. (The Australian study suggests less likely; the US study suggests more likely; the double-blinded study suggests not much difference; the review doesn't do anything to resolve this uncertainty.)
If I were giving advice on sunscreen use I would say: "Using sunscreen reduces the damage your skin will take from a given amount of sun exposure. If you use sunscreen you will probably spend longer in the sun, and maybe enough longer to undo the good the sunscreen does. So use sunscreen but make a conscious decision not to spend longer out in the sun than you would have without it.".
The evidence I've seen doesn't justify saying "wearing sunscreen tends to increase the risk of skin cancer". (But also doesn't justify saying "wearing sunscreen tends to reduce the risk of skin cancer" with much confidence, at least not without caveats.)

I think your modified wording is better, and wonder whether it might be improved further by replacing "correlated" with "associated" which is still technically correct (or might be? the Australian study above seems to disagree) and sounds more like "sunscreen is bad for you".

[-]abramdemski4y40

That last bit (which, I confess, I hadn't actually noticed before) doesn't say that "wearing sunscreen actually tends to increase risk of skin cancer".

I agree. I think I read into it a bit on my first reading, when I was composing the post. But I still found the interpretation probable when I reflected on what the authors might have meant.

In any case, I concede based on what you could find that it's less probable (than the alternatives). The interviewee probably didn't have any positive information to the effect that getting someone to wear sunscreen causes them to stay out in the sun sufficiently longer for the skin cancer risk to actually go up on net. So my initial wording appears to be unsupported by the data, as you originally claimed.

But I don't think your interpretation passes the smell test. If whoever wrote that page really believed that the overall effect of wearing sunscreen was to increase the risk of skin cancer (via making you stay out in the sun for longer), would they have said "We recommend sunscreen for skin cancer prevention"?

It's pretty plausible on my social model.

To use an analogy: telling people to eat less is not a very good weight loss intervention, at all. (I think. I haven't done my research on this.) More importantly, I don't think people think it is. However, people do it all the time, because it is true that people would lose weight if they ate less.

My hypothesis: when giving advice, people tend to talk about ideal behavior rather than realistic consequences.

More evidence: when I give someone advice for how to cope with a behavior problem rather than fixing it, I often get pushback like "I should just fix it", which seems to be offered as an actual argument against my advice. For example, if someone habitually stopped at McDonald's on the way home from work, I might suggest driving a different way (avoiding that McDonald's), so that the McDonald's temptation doesn't kick in when they drive past it. I might get a response like "but I should just not give in to temptation". Now, that response is sometimes valid (like if the only other way to drive is significantly longer), but I think I've gotten responses like that when there's no other reason except the person wants to see themselves as a virtuous person and any plan which accounts for their un-virtue is like admitting defeat.

So, if someone believes "putting on sunscreen has a statistical tendency to make people stay out in the sun longer, which is net negative wrt skin cancer" but also believes "all else being equal, putting on sunscreen is net positive wrt skin cancer", I expect them to give advice based on the latter rather than the former, because when giving advice they tend to model the other person as virtuous enough to overcome the temptation to stay out in the sunlight longer. Anything else might even be seen as insulting to the listener.

(Unless they are the sort of person who geeks out about behavioral economics and such, in which case I expect the opposite.)

I think your modified wording is better, and wonder whether it might be improved further by replacing "correlated" with "associated" which is still technically correct (or might be? the Australian study above seems to disagree) and sounds more like "sunscreen is bad for you".

I was really tempted to say "associated", too, but it's vague! The whole point of the example is to say something precise which is typically interpreted more loosely. Conflating correlation with causation is a pretty classic example of that, so, it seems good. "Associated" would still serve as an example of saying something that the precise person knows is true, but which a less precise person will read as implying something different. High-precision people might end up in this situation, and could still complain "I didn't say you shouldn't wear sunscreen" when misinterpreted. But it's a slightly worse example because it doesn't make the precise person sound like a precise person, so it's not overtly illustrating the thing.

[-]gjm4y20

I'm not sure what distinction you're making when you say someone might believe both of

putting on sunscreen has a statistical tendency to make people stay out in the sun longer, which is net negative wrt skin cancer
all else being equal, putting on sunscreen is net positive wrt skin cancer

If the first means only that being out in the sun longer is negative then of course it's easy to believe both of those, but then "net negative" is entirely the wrong term and no one would describe the situation by saying anything like "wearing sunscreen actually tends to increase risk of skin cancer".

If the first means that the benefit of wearing sunscreen and the harm of staying in the sun longer combine to make something net negative, then "net negative" is a good term for that and "tends to increase risk" is fine, but then I don't understand how that doesn't flatly contradict the second proposition.

What am I missing?

[-]abramdemski4y40

Sorry, here's another attempt to convey the distinction:

Possible belief #1 (first bullet point):

If we perform the causal intervention of getting someone to put on sunscreen, then (on average) that person will stay out in the sun longer; so much so that the overall incidence of skin cancer would be higher in a randomly selected group which we perform that intervention on, in comparison to a non-intervened group (despite any opposing beneficial effect of sunscreen itself).

I believe this is the same as the second interpretation you offer (the one which is consistent with use of the term "net").

Possible belief #2 (second bullet point):

If we perform the same causal intervention as in #1, but also hold fixed the time spend in the sun, then the average incidence of skin cancer would be reduced.

This doesn't flatly contradict the first bullet point, because it's possible sunscreen is helpful when we keep the amount of sun exposure fixed, but that the behavior changes of those with sunscreen changes the overall story.

[-]gjm4y20

OK, yes: I agree that that is a possible distinction and that someone could believe both those things. And, duh, if I'd read what you wrote more carefully then I would have understood that that was what you meant. ("... because when giving advice they tend to model the other person as virtuous enough to overcome the temptation to stay out in the sunlight longer.") My apologies.

[-]johnswentworth4y40

So, to first approximation, the precision of a team of people can only be a little higher than the precision of its least precise member.
Power, on the other hand, is not like this. The power of a group is basically the sum of the power of its members.

This suggests a prediction: in practice, precision will be a limiting factor a lot more often than power, because it's a lot easier to scale up power than precision.

Precision is a more scarce resource, and typically more valuable on the margin.

[-]abramdemski4y40

...Which is like predicting that humans have compute overhang, since (mental) power is compute, and (mental) precision is algorithms. Although, the hardware/software analogy additionally suggests that good algorithms are transmissible (which needn't be true from the more general hypothesis). If that's true, you'd expect it to be less of a bottleneck, since the most precise people could convey their more precise ways of thinking to the rest of the group.

[-]Chris_Leong4y20

Power vs. precision seems to be related to the central divide between those who align more with post-rationality and those who align more with rationality.

[-]abramdemski4y20

How so? Like postrats want power?

[-]Chris_Leong4y20

They want to successfully operate in the world (which can roughly be translated to power) more than they want to be right (that is, have strictly correct beliefs).

[-]abramdemski4y20

I guess that makes sense. I was just not sure whether you meant that, vs the reverse (my model of postrats is loose enough that I could see either interpretation).

[-]Richard_Kennaway4y20

I don't know which postrats you have in mind here, but the obvious (rat) question is, are they succeeding? And do they want to be right about at least that question?

LESSWRONG
LW

LESSWRONG
LW

48

48

48

Physical Power vs Precision

Generalizing

Working With Others