jimmy

Comments

Covid 9/24: Until Morale Improves

Hypothetically, what would the right response be if you noticed that one of the main vaccine trials has really terrible blinding (e.g. participants are talking about how to tell whether you get the placebo in the waiting room)?

It seems like it would really mess up the data, probably resulting in the people who got the the vaccine taking extra risk and leading the study to understate the effectiveness.  Ideally, "tell the researchers" would be the obvious right answer, but are there perverse incentives at play that make the best response something else?

If I didn’t have people thanking me every week for doing these, it would be difficult to keep going.

Thanks Zvi. The effort is definitely appreciated.

Covid 9/10: Vitamin D
There were 50 patients in the treatment group. None were admitted to the ICU. There were 26 patients in the control group. Half of them, 13 out of 26, were admitted to the ICU. So 13/26 vs. 0/50.

That's not what the paper says

Of 50 patients treated with calcifediol, one required admission to the ICU (2%),

The conclusions still hold, of course.

Do you vote based on what you think total karma should be?

Adjusting in the other direction seems useful as well. If someone Strong Upvotes ten times less frequently than average I would want to see their strong upvote as worth somewhat more.

Do you vote based on what you think total karma should be?
Answer by jimmyAug 24, 202026

Voting based on current karma is a good thing.

Without that, a post that is unanimously barely worth upvoting will get an absurd amount of upvotes while another post which is recognized as earth shatteringly important by 50% will fail to stand out. Voting based on current karma gives you a measure of the *magnitude* of people's like for a comment as well as the direction, and you don't want to throw that information out.

If everyone votes based on what they think the total karma should be, then a post's karma reflects [a weighted average of opinions on what the post's total karma should be] rather than [a weighted average of opinions on the post].

This isn't true.

If people vote based on what the karma should be, the final value you get is the median of what people think the karma should be -- i.e. a median of people's opinion of the post. If you force people to ignore the current karma, you don't actually get a weighted average of opinions on the post because there's very little flexibility in how strongly you upvote a post. In order to get that magnitude signal back, you'd have to dilute your voting with dither, and while that will no doubt happen to some extent (people might be too lazy to upvote slightly-good posts, but will make sure to upvote great ones), you will get an overestimate of the value of slightly-good posts.

This is bad, because the great posts hold a disproportionate share of the value, and we very much want them to rise to the top and stand out above the rest.

Highlights from the Blackmail Debate (Robin Hanson vs Zvi Mowshowitz)
You are very much in the minority if you want to abolish norms in general.

There's a parallel here with the fifth amendment's protection from self incrimination making it harder to enforce laws and laws being good on average. This isn't paradoxical because the fifth amendment doesn't make it equally difficult to enforce all laws. Actions that harm other people tend to have other ways of leaving evidence that can be used to convict. If you murder someone, the body is proof that someone has been harmed and the DNA in your van points towards you being the culprit. If you steal someone's bike, you don't have to confess in order to be caught with the stolen bike. On the other hand, things that stay in the privacy of your own home with consenting adults are *much* harder to acquire evidence for if you aren't allowed to force people to testify against themselves. They're also much less likely to be things that actually need to be sought out and punished.

If it were the case that one coherent agent were picking all the rules with good intent, then it wouldn't make sense to create rules that make enforcement of other rules harder. There isn't one coherent agent picking all the rules and intent isn't always good, so it's important to fight for meta rules that make it selectively hard to enforce any bad rules that get through.

You can try to argue that preventing blackmail isn't selective *enough* (or that it selects in the wrong direction), but you can't just equate blackmail with "norm enforcement [applied evenly across the board]".

What counts as defection?
I actually don't think this is a problem for the use case I have in mind. I'm not trying to solve the comparison problem. This work formalizes: "given a utility weighting, what is defection?". I don't make any claim as to what is "fair" / where that weighting should come from. I suppose in the EGTA example, you'd want to make sure eg reward functions are identical.

This strikes me as a particularly large limitation. If you don't have any way of creating meaningful weightings of utility between agents then you can't get anything meaningful out. If you're allowed to play with that free parameter then you can simply say "I'm not a utility monster, this genuinely impacts me more than you [because I said so!]" and your actual outcomes aren't constrained at all.

Defection doesn't always have to do with the Pareto frontier - look at PD, for example. (C,C), (C,D), (D,C) are usually all Pareto optimal. 

That's why I talk about "in the larger game" and use scare quotes on "defection". I think the word has to many different connotations and needs to be unpacked a bit.

The dictionary definition, for example, is:

A lack: a failure; especially, failure in the performance of duty or obligation.
n.The act of abandoning a person or a cause to which one is bound by allegiance or duty, or to which one has attached himself; a falling away; apostasy; backsliding.
n.Act of abandoning a person or cause to which one is bound by allegiance or duty, or to which one has attached himself; desertion; failure in duty; a falling away; apostasy; backsliding.

This all fits what I was talking about, and the fact that the options in prisoners dilemma are traditionally labeled "Cooperate" and "Defect" doesn't mean they fit the definition. It smuggles in these connotations when they do not necessarily apply.

The idea of using tit for tat to encourage cooperation requires determining what ones "duty" is and what "failing" this duty is, and "doesn't maximize total utility" does not actually work as a definition for this purpose because you still have to figure out how to do that scaling.

Using the Pareto frontier allows you to distinguish between cooperative and non-cooperative behavior without having to make assumptions/claims about whose preferences are more "valid". This is really important for any real world application, because you don't actually get those scalings on a silver platter, and therefore need a way to distinguish between "cooperative" and "selfishly destructive" behavior as separate from "trying to claim a higher weight to one's own utility".

What counts as defection?

As others have mentioned, there's an interpersonal utility comparison problem. In general, it is hard to determine how to weight utility between people. If I want to trade with you but you're not home, I can leave some amount of potatoes for you and take some amount of your milk. At what ratio of potatoes to milk am I "cooperating" with you, and at what level am I a thieving defector? If there's a market down the street that allows us to trade things for money then it's easy to do these comparisons and do Coasian payments as necessary to coordinate on maximizing the size of the pie. If we're on a deserted island together it's harder. Trying to drive a hard bargain and ask for more milk for my potatoes is a qualitatively different thing when there's no agreed upon metric you can use to say that I'm trying to "take more than I give".


Here is an interesting and hilarious experiment about how people play an iterated asymmetric prisoner's dilemma. The reason it wasn't more pure cooperation is that due to the asymmetry there was a disagreement between the players about what was "fair". AA thought JW should let him hit "D" some fraction of the time to equalize the payouts, and JW thought that "C/C" was the right answer to coordinate towards. If you read their comments, it's clear that AA thinks he's cooperating in the larger game, and that his "D" aren't anti-social at all. He's just trying to get a "fair" price for his potatoes, and he's mistaken about what that is. JW, on the other hand, is explicitly trying use his Ds to coax A into cooperation. This conflict is better understood as a disagreement over where on the Pareto frontier ("at which price") to trade than it is about whether it's better to cooperate with each other or defect.

In real life problems, it's usually not so obvious what options are properly thought of as "C" or "D", and when trying to play "tit for tat with forgiveness" we have to be able to figure out what actually counts as a tit to tat. To do so, we need to look at the extent to which the person is trying to cooperate vs trying to get away with shirking their duty to cooperate. In this case, AA was trying to cooperate, and so if JW could have talked to him and explained why C/C was the right cooperative solution, he might have been able to save the lossy Ds. If AA had just said "I think I can get away with stealing more value by hitting D while he cooperates", no amount of explaining what the right concept of cooperation looks like will fix that, so defecting as punishment is needed.

In general, the way to determine whether someone is "trying to cooperate" vs "trying to defect" is to look at how they see the payoff matrix, and figure out whether they're putting in effort to stay on the Pareto frontier or to go below it. If their choice shows that they are being diligent to give you as much as possible without giving up more themselves, then they may be trying to drive a hard bargain, but at least you can tell that they're trying to bargain. If their chosen move is conspicuously below (their perception of) the Pareto frontier, then you can know that they're either not-even-trying, or they're trying to make it clear that they're willing to harm themselves in order to harm you too.

In games like real life versions of "stag hunt", you don't want to punish people for not going stag hunting when it's obvious that no one else is going either and they're the one expending effort to rally people to coordinate in the first place. But when someone would have been capable of nearly assuring cooperation if they did their part and took an acceptable risk when it looked like it was going to work, then it makes sense to describe them as "defecting" when they're the one that doesn't show up to hunt the stag because they're off chasing rabbits.

"Deliberately sub-Pareto move" I think is a pretty good description of the kind of "defection" that means you're being tatted, and "negligently sub-Pareto" is a good description of the kind of tit to tat.

Noise on the Channel

To the extent that the underlying structure doesn't matter and can't be used, I agree that technically non-random "noise" behaves similarly and that this can be a reasonable use of the term. My objection to the term "noise" as a description of conversational landmines isn't just that they're "technically not completely random", but that the information content is actually important and relevant. In other words, it's not noise, it's signal.

The "landmines" are part of how their values are actually encoded. It's part of the belief structure you're looking to interact with in the first place. They're just little pockets of care which haven't yet been integrated in a smooth and stable way with everything else. Or to continue the metaphor, it's not "scary dangerous explosives to try to avoid", it's "inherently interesting stores of unstable potential energy which can be mined for energetic fuel". If someone is touchy around the subject you want to talk about, that is the interesting thing itself. What is in here that they haven't even finished explaining to themselves, and why is it so important to them that they can't even contain themselves if you try to blow past it?

It doesn't even require slow and cautious approach if you shift your focus appropriately. I've had good results starting a conversation with a complete stranger who was clearly insecure about her looks by telling her that she should make sure her makeup doesn't come off because she's probably ugly if she's that concerned about it. Not only did she not explode at me, she decided to throw the fuse away and give me a high bandwidth and low noise channel to share my perspective on her little dilemma, and then took my advice and did the thing her insecurity had been stopping her from doing.

The point is that you only run into problems with landmines as noise if you mistake landmines for noise. If your response to the potential of landmines is "Gah! Why does that unimportant noise have to get in the way of what I want to do!? I wonder if I can get away with ignoring them and marching straight ahead", then yeah, you'll probably get blowed up if you don't hold back. On the other hand, if your response is closer to "Ooh! Interesting landmine you got here! What happens if I poke it? Does it go off, or does the ensuing self reflection cause it to just dissolve away?", then you get to have engaging and worthwhile high bandwidth low noise conversations immediately, and you will more quickly get what you came for.

Noise on the Channel

I think it's worth making a distinction between "noise" and "low bandwidth channel". Your first examples of "a literal noisy room" or "people getting distracted by shiny objects passing by" fit the idea of "noise" well. Your last two examples of "inferential distance" and "land mines" don't, IMO.

"Noise" is when the useful information is getting crowded out by random information in the channel, but land mines aren't random. If you tell someone their idea is stupid and then you can't continue telling them why because they're flipping out at you, that's not a random occurrence. Even if such things aren't trivially predictable in more subtle cases, it's still a predictable possibility and you can generally feel out when such things are safe to say or when you must tread a bit more carefully.

The "trying to squeeze my ideas through a straw" metaphor seems much more fitting than "struggling to pick the signal out of the noise floor" metaphor, and I would focus instead on deliberately broadening the straw until you can just chuck whatever's on your mind down that hallway without having to focus any of your attention on the limitations of the channel.

There's a lot to say on this topic, but I think one of the more important bits is that you can often get the same sense of "low noise conversation" if you pivot from focusing on ideas which are too big for the straw to focusing on the straw itself, and how its limitations might be relaxed. This means giving up on trying to communicate the object level thing for a moment, but it wasn't going to fit anyway so you just focus on what is impeding communication and work to efficiently communicate about *that*. This is essentially "forging relationships" so that you have the ability to communicate usefully in the future. Sometimes this can be time consuming, but sometimes knowing how to carry oneself with the right aura of respectability and emotional safety does wonders for the "inferential distance" and "conversational landmines" issues right off the bat.

When the problem is inferential distance, the question comes down to what extent it makes sense to trust someone to have something worth listening to over several inferences. If our reasonings differ several layers deep then offering superficial arguments and counterarguments is a waste of time because we both know that we can both do that without even being right. When we can recognize that our conversation partner might actually be right about even some background assumptions that we disagree on, then all of a sudden the idea of listening to them describe their world view and looking for ways that it could be true becomes a lot more compelling. Similarly, when you can credibly convey that you've thought things through and are likely to have something worth listening to, they will find themselves much more interested in listening to you intently with an expectation of learning something.

When the problem is "land mines", the question becomes whether the topic is one where there's too much sensitivity to allow for nonviolent communication and whether supercritical escalation to "violent" threats (in the NonViolent Communication sense) will necessarily displace invitations to cooperate. Some of the important questions here are "Am I okay enough to stay open and not lash out when they are violent at me?" and the same thing reflected towards the person you're talking to. When you can realize "No, if they snap at me I'm not going to have an easy time absorbing that" you can know to pivot to something else (perhaps building the strength necessary for dealing with such things), but when you can notice that you can brush it off and respond only to the "invitation to cooperate" bit, then you have a great way of demonstrating for them that these things are actually safe to talk about because you're not trying to hurt them, and it's even safe to lash out unnecessarily before they recognize that it's safe. Similarly, if you can sincerely and without hint of condescension ask the person whether they're okay or whether they'd like you to back off a bit, often that space can be enough for them to decide "Actually, yeah. I can play this way. Now that I think about it, its clear that you're not out to get me".

There's a lot more to be said about how to do these things exactly and how to balance between pushing on the straw to grow and relaxing so that it can rebuild, but the first point is that it can be done intentionally and systematically, and that doing so can save you from the frustration of inefficient communication and replace it with efficient communication on the topic of how to communicate efficiently over a wider channel that is more useful for everything you might want to communicate.

Fight the Power

In general, if you're careful to avoid giving unsolicited opinions you can avoid most of these problems even with rigid ideologues. You wouldn't inform a random stranger that they're ugly just because it's true, and if you find yourself expressing or wishing to express ideas which people don't want to hear from you, it's worth reflecting on why that is and what you are looking to get out of saying it.

Load More