I'm sorry if I'm just being too much of a dodo to perceive the mystery, but your scenario seems easily accounted for. You can use a Bayesian network to infer causality if and only if you have valid data to fill it with. Of course wearing large pants does not cause one not to exercise, but no real set of data would indicate that it did. Am I missing something?
EDIT: shortly after writing this, I read up on faithfulness and Milton Friedman's thermostat, so the "if and only if" part of my comment isn't quite accurate. Still, the pants size scenario doesn't seem like one of these exceptional cases.
Statements of the sort "we shouldn't balance the risks and opportunities of X" are substantive only where X is closely related to a fundamental principle or a terminal goal. Since nobody really wants superhuman AGI for its own sake (in fact, it's just the opposite: it's the ultimate instrumental goal), "we should balance the risks and opportunities of AGI" is an applause light.
Agreed. Zvi's proposition also simply doesn't align with first-world people's motivations, as far as I can tell. In short, first-worlders have a lot of other interesting ways that they can use their time.
The notion that money isn't important or that "knowledge is the real wealth" wasn't intended to be a universal law; it's only applicable in cases where money is sufficiently abundant (as the title says). The scenarios you list do not meet that condition, so they are not situations that the OP intended to address.
With an additional decade of political battles to scrutinize, I see this sort of thing playing out with things like immigration policy, and possibly COVID policy, too.
From what I can gather, there are plenty of Republicans who would be willing to make a one-time amnesty concession in exchange for securing the border. However, Republican politicians are aware that if they give any ground on amnesty in this particular case, then Democratic politicians are very likely to 1) drag their feet on the securing-the-border part of the deal, and then 2) cite the previous amnesty policy as precedent for future amnesty policies in the court of public opinion.
Doesn't this depend on whether one is referring to fluid intelligence or crystal intelligence? Human babies may have the same crystal intelligence as adult pigs, but they have much higher fluid intelligence.
I think what happened here is that the vegetarian failed to realize that the component of intelligence that people find morally significant is fluid, not crystal, and then he equivocated between the two. EY realized what was going on, even if subconsciously, which is why he trolled the vegetarian instead of disputing his premise. Finally, Fallible failed to pick up on the distinction entirely by assuming that "intelligence" always refers to fluid intelligence.
It's useful until the jester gains a reputation as someone whose views shouldn't be taken seriously, at which point the jester's dissent may begin to have the opposite effect.