philh

Wiki Contributions

Comments

Some questions we should distinguish here:

  1. How confident are you that the thing you're looking into here is going to pay off? (Whatever that means - give you something that lets you reliably improve people's lives more than existing methods?) I get the sense that you're reasonably confident, but I don't think you explicitly say. Gordon's comment talks about "high EV and low probability", so I guess Gordon got a different sense from your writing than I did.

  2. How confident should you be that it's going to pay off? (It is well known that people are sometimes poorly calibrated.)

  3. How confident should other people be that it's going to pay off? You might have illegible evidence that it's a fruitful field of study, in which case it seems correct for you to be more confident than others. (Analogy: if you're selling a used car that you know is not a lemon, it seems correct for you to be more confident of this than your buyer, because distinguishing sellers of lemons from non-lemons is difficult.) But also, I think a lot of people think they have illegible evidence for something when what they actually have is wishful thinking or something. I feel like that's especially the case in this field of study.

    In any case, I think most people should have low confidence that what you're looking into should pay off; and I think you should probably agree with me on this, even if you think you personally should have high confidence.

  4. Should you be looking into this thing? My instinctive reaction is "why not, seems harmless, worst case scenario is you waste your time and your time is yours to waste". But actually I agree with what I think Said is saying, that a common result of people looking into this sort of thing is to go crazy, and this is not actually harmless in the same way that doing drugs is not harmless. So I dunno. Not endorsed, but maybe my thinking here is: "if you are not worried about going crazy and taking precautions to avoid, you should not be looking into this; but I'm not a cop, this is not community consensus, so I might judge you for doing things I think are dangerous but I'm not going to additionally judge you for doing things that are widely known to be dangerous."

  5. Should you be teaching this thing? Under what conditions? I think my feeling is that the conditions should probably not rule out teaching it entirely, but same caveat as with (4), you should be cautious about yourself and your students going crazy. Also I think you should expect skepticism from your students per (2), and I think you should explicitly endorse it. Like, your reaction to "is this backed by scientific studies?" gives me a vibe of "I don't care and I don't think you should care", and I think that's a bad vibe. I want vibe closer to "your skepticism is great here, totally warranted; but I'm explicitly asking you to set it aside for a bit".

This feels vague and not very well pinned down, but:

I think a large part of it is the difference between "how positively do people feel about you" (popularity) and "how confidently can people predict what you'll do" (reputation). Of course both of these also depend on "what people/organizations are we even talking about here".

So when NIMBYs don't want to fight you, that feels like a combination of

  • The NIMBYs themselves probably quite like you
  • The NIMBYs know lots of other people like you, and don't want to be seen fighting someone popular

both of which are because you're popular with the general public, and I can imagine you being popular with the general public even if you're incompetent at your relations with other organizations.

But when your dealings with government get fast-tracked, that's probably also related to people at city hall thinking well of you. But it also seems to me that a lot of it is "we know what happens when these people get to do the thing they're asking to do". And I can imagine that being the case even if the general public basically doesn't know you, or doesn't like you.

("Reputation" doesn't feel like quite the word for that, because I'm more imagining it being a "you have a history of working with us" thing than a "we heard about you from someone else you worked with" thing.)

Relatedly, I'd be curious about the mechanism behind Samaritans keeping its reputation for however long. (I get a sense that the org is probably between 15 and 50 years old? Not sure where I get that from.) It's probably been through a bunch of CEOs, or whatever equivalent it has, in that time. Those CEOs probably weren't selected on the basis of "who will pick the best successor to themselves". Why has no one decided "we can help people better like this, even if that means breaking some (implicit?) promises we've made" and then oops, no one really trusts them any more?

So some questions I'd be interested in, I guess:

  • Have they had any major fuck ups?
  • If so, did that cost them reputationally? How did they regain trust?
  • If not, how did they avoid them? Luck? Tending to hire the sorts of people who don't gamble with reputation? (Which might be easier because that sort of person will instead play the power game in a for-profit company?) Just not being old enough yet for that to be a serious concern?
  • How common is it for a well-reputed for-profit company to lose that reputation? /r/BuyItForLife will tell you several brands that used to be good and then outsourced manufacturing to China and now mainly or exclusively produce expensive branded shit. (Caution: I have no specific reason to doubt these stories, but it is reddit.) But how often does that sort of thing actually happen and how quickly (if at all) does their reputation disintegrate afterwards? Perhaps Nintendo and Disney count as companies that haven't done anything like this? (e: and when this does happen, does it tend to be an error? An unforced error? Like, can't maintain a reputation for producing quality product if you're not producing quality product because you're bankrupt because people weren't willing to pay what it cost.)

(Acknowledging that this is super reductive about the concept of reputation.)

I know it sucks for nerds to hear that reputation (popularity) is important

I'm not sure we should be thinking of these as the same thing. For example I'd say this is reputation:

Samaritans has a much better, easier time at city hall compared to newer organizations, because of a decades-long productive relationship where we were really helpful with issues surrounding unemployment and homelessness. Permits get back to us really fast, applications get waved through with tedious steps bypassed, and fees are frequently waived. And it made sense that this was happening! Cities also deal with budget and staffing issues, why waste more time and effort than necessary on someone who you *know *knows the proper procedure and will ethically follow it to the letter?

and this is popularity:

the NIMBYs were reluctant to come out against us

And they probably have overlapping causes and effects, but they're not the same.

I think most of what you describe feels more like it comes from reputation than popularity, and also I feel like EA as a whole simply can't ever have that kind of reputation. Individual EA orgs can, but with no barrier to entry, it will never be the case that someone can reliably say "this organization calls itself an EA organization, so I can be confident it will execute competently and clean up after itself". But also, if EA as a whole is unpopular, that's also going to cause problems for well-reputed EA orgs.

The scenario very nicely pinpoint a first problem. Either you allow the parents to do something to their child that seems horrible (actively bringing additional suffering onto their child), or you concede that you aren’t enabling free, voluntary changes to the human genome, it is only voluntary if they want the result you want.

Does epilogenics significantly change this picture? It's already the case that

  • There are things parents are allowed to do to their child, that seem horrible to some people.
  • There are things parents are not allowed to do to their child, because they seem horrible to some people.
  • Sometimes things move from one category to another.

People disagree on what is and isn't horrible. But it's rare, I think, for parents to choose things for their child on the basis of "this is what I think will cause them to suffer most", even in cases where that would be legal.

There's a difficult balancing act between "we as a society think this thing is bad and we're going to forbid it" and "we as a society acknowledge that things are complicated and simply banning things that seem bad isn't always a good idea, so we're not going to ban this thing even though we think it's bad".

But I don't think there's a way to avoid this balancing act; if we try to avoid it by forbidding the technology, then that's just coming down very hard on one side of it.

The key thing I will never accept is giving parents agency over their kids’ genome.

Do you oppose the existing efforts to prevent Tay-Sachs disease?

In the first 30 years of testing, from 1969 through 1998, more than 1.3 million persons were tested, and 48,864 carriers were identified. In at-risk families, among couples where both husband and wife were carriers, more than 3000 pregnancies were monitored by amniocentesis or chorionic villus sampling. Out of 604 monitored pregnancies where there was a prenatal diagnosis of Tay–Sachs disease, 583 pregnancies were terminated. Of the 21 pregnancies that were not terminated, 20 of the infants went on to develop classic infantile Tay–Sachs disease, and the 21st case progressed later to adult-onset Tay–Sachs disease. In more than 2500 pregnancies, at-risk families were assured that their children would not be affected by Tay–Sachs disease.

I'm a fan of this.

I'd say that "optimizing expected money is the only thing Bob cares about" is an example, not an assumption or conclusion. If you want to argue that agents should care about ergodicity regardless of their utility function, then you need to argue that to the agent whose utility function is linear in money (and has no other terms, which I assumed but didn't state in the previous comment).

Such an agent is indifferent between a certainty of dollars, and a near-certainty of dollars with a chance of dollars. That's simply what it means to have that utility function. If you think this agent, in the current hypothetical scenario, should bet Kelly to get ergodicity, then I think you just aren't taking seriously what it means to have a utility function that's linear in money.

In the limit as the number of bets goes to infinity

I spoke about limits and infinity in my conversation with Ben, my guess is it's not worth me rehashing what I said there. Though I will add that I could make someone whose utility is log in money - i.e. someone who'd normally bet Kelly - behave similarly.

Not with quite the same setup. But I can offer them a sequence of bets such that with near-certainty ( as ), they'd eventually end up with $0.01 and then stop betting because they'll under no circumstances risk going down to $0.

These bets can't be of the form "payout is some fixed multiple of your stake and you get to choose your stake", but I think it would work if I do "payout is exponential in your stake". Or I could just say "minimum stake is your entire bankroll minus $0.01" - if I offer high enough payouts each time, they'll take these bets, over and over, until they're down to their last cent. Each time they'd prefer a smaller bet for less money, but if I'm not offering that they'd rather take the bet I am offering than not bet at all.

Also,

It’s weird to me that something of measure 0 probability can swamp the entirety of the rest of the probability.

The Dirac delta has this property too, and IIUC it's a fairly standard tool.

Here were talking something that's weird in a different way, and perhaps weird in a way that's harder to deal with. But again I think that's more because of infinity than because of utility functions that are linear in money.

I don’t find the extremely high returns in the infinitesimally probable regions to make up for that. I’d like a principled way of expressing that which doesn’t rely on having a specific type of utility function

This sounds impossible to me? Like, if we're talking about agents with a utility function, then either that function is such that extremely high returns make up for extremely low probabilities, or it's such that they don't. If they do, there's no argument you can make that this agent is mistaken, they simply value things differently than you. If you want to argue that the high returns aren't worth the low probability, you're going to need to make assumptions about their utility function.

I admit that I don't know what ergodicity is (and I bounce off the wiki page). But if I put myself in the shoes of Bob whose utility function is linear in money... my anticipation is that he just doesn't care. Like, you explain what ergodicity to him, and point out that the process he's following is non-ergodic. And he replies that yes, that's true; but on the other hand, the process he's following does optimize his expected money, which is the only thing he cares about. And there's no ergodic process that maximizes his expected money. So he's just going to keep on optimizing for the thing he cares about, thanks, and if you want to give up some expected money in exchange for ergodicity, that's your right.

(e: posted overlapping with Oscar_Cunningham)

If you're trying to maximize expected money at the end of a fixed number of rounds, you do that by betting everything on every round (and, yes, almost certainly going bankrupt).

If that's not what you're trying to do, the optimal strategy is probably something else. But "how do we maximize expected money?" seems to be the question Gwern's post is exploring. It's just that with the $250 cap, maximizing expected money seems like a good idea (because you can almost always get close to $250), and with no cap, maximizing expected money seems like a terrible idea (because it gives you a 10^-67 chance of $10^92).

You don't do Kelly because it's good at maximizing expected money. You do it (when you do it) because you're trying to do something other than maximize expected money.

Nod. I think we basically agree at this point. Certainly I don't intend to claim that optimal policy and optimal actions always coincide (I have more thoughts on that but don't want to get into them).

Load More