I'll accept time-sensitive stuff as a valid counterargument to my claim, as well as e.g. things moving beyond the observable universe.
But I don't see how the existence of the moons of Neptune works as a counterargument. The whole point is that you do something laborious to gain/accumulate/generate new knowledge (like send a space probe). And then to verify/confirm said knowledge, you don't have to send a new space probe because you can use a gazillion other cheaper methods to confirm the knowledge instead (like by pointing telescopes at the moons, or by using your improved knowledge of physical law to predict their positions, etc. etc.).
If the claim is just "producing the exact same kind of evidence (space probe pictures) can require the same cost", then I don't exactly disagree, I just don't see how that's at all relevant. The AI context here is that we have a superhuman mind that can generate knowledge we can't (the space probe or its pictures), and the question is whether it can convert that knowledge into a form we'd have a much easier time understanding. In that situation, why would it matter that we can't build a second space probe?
This is not what you're asking for, but are you aware of Pantheon (2022) (Wikipedia, LW thread)? It's a short animated TV series (16 episodes over 2 seasons, canceled / cut short) about mind uploads and related topics. It features several of the things you want, but also some weird stuff like superhero-esque fights between uploads. And while the ending of the final episode is quite bombastically sci-fi, it also makes it very clear that the series was cut short.
Comment half in jest, half serious: if the problem is lack of fluency in Obscure Language, then might this be a case where people today would be better-served by LLM nurses and doctors, rather than human ones?
I asked Claude Code to make a Slither Link puzzle game, providing it an extensive design doc about stuff like difficulty curve or UI but none about the basic game rules, and it failed to get puzzle generation to work (IIRC initially it wouldn't even generate closed loops or something) and then got stuck trying to fix it, continually giving me new versions that never worked nor even looked like they got any closer to the goal. To be clear, that doesn't meant that Claude Code can't complete this particular task, it just means that I couldn't get it to Just Work™.
To play devil's advocate, I don't see why preventive war would be "insane". If you're the first nuclear power, and you can prevent your potential rivals from acquiring their own nukes, then that makes you an unassailable hegemon. With the benefit of hindsight, a clever arguer (not meant as a compliment) could even claim that this strategy isn't evil but actually morally required because, if it indeed prevents others from obtaining nukes, then this prevents an entire source of future x-risk from MAD and the Cold War. Not to mention unpreventable human rights abuses by future nuclear powers like North Korea.
To be clear, I'm not advocating for this alternate history; most importantly from a strategic perspective, it's not at all clear that the US could've kept the technology for itself no matter how aggressive it acted. Also it would've been evil, and I can't imagine there would've been enough political will by the US public post-1945 to pursue such a war directly after World War II, so it would've eventually failed for that reason anyway.
Have donated $1000.
Sure, but I meant that you still need the votes of those other people, too. And the fewer votes you have, the more compromises make it into the final bill.
Agreed. I can't entirely appreciate what is lost in this story, not being personally interested in romance, but if given the choice between this future vs. the ones I consider likely, I'd choose this one in a heartbeat.
I'm confused about this model. You need A) leaders to suggest/champion good legislation, and then B) enough legislators to actually pass said legislation, no? So what's the point of having A without B? I suppose in your model, bad B's worsen the good legislation suggested by the A's, but I don't see an in-principle way to resolve that problem in a majoritarian legislature rather than a one-person dictatorship. How do you go from having great A's but no B's, to getting useful legislation signed into law?
While the framing of treating lack of social grace as a virtue captures something true, it's too incomplete and imo can't support its strong conclusion. The way I would put it is that you have correctly observed that, whatever the benefits of social grace are, it comes at a cost, and sometimes this cost is not worth paying. So in a discussion, if you decline to pay the cost of social grace, you can afford to buy other virtues instead.[1]
For example, it is socially graceful not to tell the Emperor Who Wears No Clothes that he wears no clothes. Whereas someone who lacks social grace is more likely to tell the emperor the truth.
But first of all, I disagree with the frame that lack of social grace is itself a virtue. In the case of the emperor, for example, the virtues are rather legibility and non-deception, traded off against whichever virtues the socially graceful response would've gotten.
And secondly, often the virtues you can buy with social grace are worth far more than whatever you could gain by declining to be socially graceful. For example, when discussing politics with someone of an opposing ideology, you could decline to be socially graceful and tell your interlocutor to their face that you hate them and everything they stand for. This would be virtuously legible and non-deceptive, at the cost of immediately ending the conversation and thus forfeiting any chance of e.g. gains from trade, coming to a compromise, etc.
One way I've seen this cost manifest on LW is that some authors complain that there's a style of commenting here that makes it unenjoyable to post here as an author. As a result, those authors are incentivized to post less, or to post elsewhere.[2]
And as a final aside, I'm skeptical of treating Feynman as socially graceless. Maybe he was less deferential towards authority figures, but if he had told nothing but the truth to all the authority figures (who likely included some naked emperors) throughout his life, his career would've presumably ended long before he could've gotten his Nobel Prize. And b), IIRC the man's physics lectures are just really fun to watch, and I'm pretty confident that a sufficiently socially graceless person would not make for a good teacher. For example, it is socially graceful not to belittle fledgling students as intellectual inferiors, even though they in some ways are just that.
Related: I wrote this comment and this follow-up where I wished that Brevity was considered a rationalist virtue. Because if there's no counterbalancing virtue to trade off against other virtues like legibility and truth-seeking, then supposedly virtuous discussions are incentivized to become arbitrarily long.
The moderation log of users banned by other users is a decent proxy for the question of which authors have considered which commenters to be too costly to interact with, whether due to lack of social grace of something else.