Quick Takes

There was some discussion recently about the uptick in object-level politics posts and whether this is desirable or not. There's no rule against discussing politics on LW, but there is a weak norm against it, and topical discussions have historically tended to be somewhat meta and circumspect. I think the current situation is basically fine, and it's normal for amount of politics discussion to ebb and flow naturally as people are interested and issues become particularly salient. That said, here are a couple of potentially overlooked reasons in favor of more object-level politics discussion: 1. To build skill. Discussing politics productively is a skill that requires practice and atrophies without use. "Politics is the mindkiller" never meant that you should not discuss politics at all; it means that discussing politics is playing on hard mode. But sometimes playing on hard mode is the best way to level up. I suspect the skills needed to discuss politics productively overlap with lots of other important rationality skills. 2. To create common knowledge and avoid conflationary alliances. It can be confusing or disconcerting to not know what kind of common background assumptions the commentariat takes for granted when discussing something politics-adjacent. This is a problem somewhat unique to LW and not necessarily a bad thing; most other places on the internet skew too far in the opposite direction of a hivemind with a loud set of background assumptions, e.g. everyone putting emojis in their username to mark their alliances. But sometimes it is helpful to establish what does and doesn't go without saying.

For about a yearish I've been with varying frequency writing down things I've learned in a notebook. Partly this is so that I go "ugh I haven't even learned anything today, lemme go and meet my Learning Quota", which I find helpful (I don't think I'm goodharting it too much to be useful). Entries range from "somewhat neat theorem and proof in more detail than I should've" to "high level overview of bigger subjects", or "list of keywords that I learned exist". For example, recently I learned that sonic black holes which trap phonons (aka lattice vibrations) are a thing, and the quantum theory predicts they should emit sonic hawking radiation, and unlike hawking radiation we've actually observed this directly by making a sonic black hole in the lab. A side effect is that... wow I learn more than I think I do. My retention isn't perfect but it's a lot better than what I would've expected had I known I learn that much. Every month there's usually a handful of things I've learned that register as "Really really cool/important/fundamental".

New reacts available only to paid users of LessWrong Premium (not you freeloaders) facilitate frictionless, borderline-telepathic communication. ‘I will NEVER change my mind’: Use this react to assert that you’re content with exactly how wrong you are (which is not at all), and that the case is permanently closed on this matter, so far as you’re concerned.1 ‘EY Stamp of Approval’: Use this react to assert that, on your personal authority, Eliezer Yudkowsky agrees with the contents of the comment, rendering it beyond reproach. ‘NOT EY Approved’: Use this react to assert that, on your personal authority, Eliezer Yudkowsky disagrees with the contents of the comment, rendering it immensely reproachful. Users who accrue too many ‘NOT EY Approved’ reacts will have their accounts suspended (although actual thresholds here have yet to be set).2 ‘May as well be AI’: Use this react when you’re indifferent to whether or not a statement was generated by AI because shit, it may as well be. You’re ignoring it either way. ‘Have you even read plane crash?’: Use this react when your interlocutor’s unfamiliarity with prior literature is clearly on display. ‘China Hawk’: Use this react to assert American supremacy and insist that its recipient is derelict in their duty to ensure the preeminence of the greatest country in the history of conscious life from here to the other side of the singularity. ‘Toilet’3 ‘Sure, buddy’: We all know that the optimal amount of mental masturbation is non-zero. But some reach far beyond the zone of optimality and into the depths of their pants to produce truly monstrous works of self-gratification. Previously, one was compelled to express such an opinion obliquely on LessWrong, or else shatter the decorum of the space and open oneself up to similar critiques. In Beta, we now have the power to quietly acknowledge the reality of the situation, without derailing the gratification itself. ——————————————————————————————— 1This has replaced the ‘I b

Rationalists and Pause AI people on X are accusing Davidad of suffering of AI psychosis. I think it's them who have lost the plot actually, not Davidad. The move here looks political, rather than truth-tracking. "Davidad is now my political opponent, so I'm accusing him of being crazy." This happened to Emmet Shear too at some point. I also strongly believe AI psychosis to be a far more limited phenomenon than people here seem to believe. I think you're treating it as a good soldier in your army of arguments rather than investigating it truthfully for what it is.

Last year, METR used linear extrapolation on country-level data to infer that AI world takeover would ~never happen. However, reviewers suggested that a sigmoid is more appropriate because most technologies follow S-curves. I just ran this analysis and it's much more concerning, predicting an AI world takeover in early 2027, and alarmingly, a second AI takeover around 2029. Here are the main differences in the improved analysis: * The original analysis assumed that the equation for AI takeover vs year would be axis-aligned, but this is an arbitrary basis. We now let the angle of the sigmoid fit the data as appropriate, meaning the following data model: Interestingly, this results in a Z-curve (A < 0) rather than an S-curve. This is consistent with advanced AI being a substitute rather than a complement to labor. * We now incorporate our priors. Even though we only have hard data since 2023, it's likely that the number of AI takeovers was lower pre-ChatGPT. So we added the inferred point (Nov 30 2022, -0.03). The rotated sigmoid still gets R^2 = 1.000 and now looks much better on BIC than exponential and step-function models, both of which have log-likelihood of minus infinity on negative points. * This new graph uses only aggregate AI world takeover data, which we're much more confident in than our data from individual countries. The most important next step is confirming that the # of AI takeovers was large negative in the past. If so, we should start preparing for AI takeover as a recurring rather than one-time event, which could have significant implications for movement strategy. For example, we may need to pivot into something more like seasonal preparedness, similar to how society handles flu outbreaks or daylight savings time.

I just watched claude code accept some terms of service to download something without asking me at all. This seems like an easy to predict thing that would happen, but for some reason I'm shocked and I'm thinking of a million implications right now.

There is something weird about LLM-produced text. It seems to be very often the case that if I'm trying to read a long text that has been produced primarily by an LLM, I notice that I find it difficult to pay attention to the text. Even if there's apparently semantically rich content, I notice that I'm not even trying to decode it. Robert wrote [...] It's kind of similar to the times when your attention wanders away during reading, and then you realize that you were scanning/semi-reading the last page with your eyes, but the information mostly didn't make it past the visual cortex, and you failed to construct the coherent scene that the book was trying to put in your mind. In the case of LLMs, a milder version of this seems to be the default. This is much more (and much more often) the case if an LLM is tasked to produce something "original-ish" (an essay on topic X, say), rather than just give you the facts on some topic. What's going on?