If we had a reasonable sized cohort of psych experts with an average IQ of 140+ maybe this would work. Unfortunately, the sorting processes that run on our society have not sorted enough intellectual capital into those fields for this to be practical, even if the crystalized knowledge they provide might be useful.

If I understand correctly, Eliezer believes that coordination is human-level hard, but not ASI-level hard. Those competing firms, made up of ASI-intelligent agents, would quite easily be able to coordinate to take resources from humans, instead of trading with humans, once it was in fact the case that doing so would be better for the ASI firms.

Mechanically, if I understand the Functional Decision Theory claim, the idea is that when you can expose your own decision process to a counter-party, and they can do the same, then both of you can simply run the decision process which produces the best outcome while using the other party's process as an input to yours. You can verify, looking at their decision function, that if you cooperate, they will as well, and they are looking for that same mechanistic assurance in your decision function. Both parties have a fully selfish incentive to run these kinds of mutually transparent decision functions, because doing so lets you hop to stable equilibria like "defect against the humans but not each other" with ease. If I have the details wrong here, someone please correct me.

I'd also contend this is the primary crux of the disagreement. If coordination between ASI-agents and firms were proven to be as difficult for them as it is for humans, I suspect Eliezer would be far more optimistic.

“Twitter, I mean X” in 2045 had me dying. So did the California red tape part. Awesome job.

Agreed, and we have well-meaning conservatives like Ben Shapiro taking marriage and monogamy as almost an unfalsifiable good, and then using it as a starting point for their entire political philosophy. Maybe he's right, but I wish realistic post-birth-control norms were actually part of the overton window.

I think much of the difficulty comes from the fact that past systems-of-the-world, the unexamined normalcies that functioned only in silence, were de facto authored by evolution, so they were haphazard patchworks that worked, but without object level understanding, which makes them particualrly tricky to replace.

Take ancient fermentation rituals. They very well may break down when a child asks "why do we have to bury the fish?" or "why bury it this long?" or "why do we have to let it sit for 4 months now?"

The elder doesn't know the answer. All he knows is that, if you don't follow these steps, people who eat the fish get sick and die.

Going from that system, to the better system involving chemistry and the germ theory of disease, is a massive step, and unfortunately, there's a large gap between realizing your elder has no idea what's going on, and figuring out the chemical process behind fermentation.

We're suffering in this gap right now, unwinding many similar unexamaned normalcies that were authored by cultural evolution, which we don't yet have comprehensive models to replace. Marriage and monogomy are a good example. After birth control, that system started quickly collapsing when the first child asked "why" and no adults could give a good answer, yet we still don't have anything close to a unified theory of human mating, relationships, and child-rearing that's better.

The tough part of building new neutral common sense is that, in so many cases right now, we don't actually know the right answer, all we know is that the old cultural wisdom is at best incomplete, and at worst totally anachronistic.

Social media was essentially like 3 billion kids asking "why" all at once, about a mostly-still-patchwork system mostly authored by evolution, so it's no real surprise it fell apart. Despite those whys being justified, replacing something evolution has created with a gears level comprehensive model is really, really hard.

Worthy task, of course, but I think fundamentally more complex than the manner by which the original systems were created.

I'm single because my current location is amazing in every way except its density of young, childless people. I have met somewhere between 300-1000 people over the last 4 years, and of those, 8 have been women between the ages of 20-30.

Most are 40+ parents, the rest are those parents' kids. The sad truth (I think), is that I simply have to move if I want to date.

Same. It would take incredible effort to find one person I reasonably connect with each year. 

So much of this is just location. I've met 100s of people over the last few years. Nearly all either over 40 with kids, or those kids. I've connected with many, maybe 10%, on a pretty good level. That doesn't help with dating at all.

I just really, really don't want it to be the case that he only answer is: move to NY, SF, or Seattle, becuase I really like it here.

As someone who's gambled professionally, I believe the (Chesterton's) fence around betting for normies exists because most bets are essentially scams, which is why I'm entirely okay knocking it down for LWers. Let me elaborate.

Probability is complicated and abstract. Not only that, human intuition is really bad at it. Nearly all "bets" throughout our modern history have not been the kind of skin-in-the-game prediction competition we're praising on lesswrong - they've been predatory. One person who understands probability using emotional and logical minipulation to take someone else's money, who doesn't.

Society protects people with taboos. "Betting is icky" is a meme that can easily spread, and will quickly reproduce, becuase it's adaptive in this betting environment. [Dissertation about Bayesian reasoning, calibration, and the Kelley Criterion] is NOT a meme that can easily spread, because it's far too complex and long, and thus it will not reproduce (even though it is also adaptive).

Or at least, it can't spread in the normie population, but it CAN on LessWrong, which is why, on LessWrong, most bets are not scams. They are, in fact, what the scammers falsly proclaimed their own bets to be - friendly competitions wherein two people who disagree about the future both put skin in the game.

The sportsbooks and casinos we have today are predators. From their celebrity endorsements, to the way they form their commercials, to their messaging around winning (and especially parlays), they effectively lie about what they're selling while trying to create addicts. I've engaged with many people across the betting experience spectrum (from other winners, to big losers, to smart people, who were small losers, and realized they needed to quit), and it's pretty clear to me that "betting = icky" is a reasonable idea, even today The fence around it is not Chesterton's, though. It's there to help regular people avoid a certain species of predator gunning for their capital.

We can safely knock it down on here.

Shout out to Karl Marx for correctly identifying many issues with what he called capitalism, but providing a compeltely inaccurate hypothesis as to the root cause of those issues.