This is a special post for quick takes by Rattengift. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
In 2011, Phil Goetz wrote the post "What we're losing," which bemoaned the shift from applied rationality to theoretical rationality on the site.
Regardless of the era we're in, I believe LessWrong has gone too far in the other direction: To some loose approximation, every post is now about AI. Even assuming it's what you care about more than anything, even coming from a prior of "the highest-EV move I can make is to write and interact with posts on x-risk," you should heavily consider the benefits (both social and practical) of diversifying your intake and output.
Having norms around posting non-AI-related writing would be a good thing even for your ability to judge a person's output around topics you care about more, like AI safety.
When the entire discussion is indexed around one topic (or a narrow range of topics), it becomes very easy to adjust the way you're speaking and thinking to "fit" properly within the distribution of speakers around you. This allows for bad actors and poor thinkers to have an easier time hiding themselves within a group without revealing themselves as such. This is, obviously, bad: A poor thinker operating in good faith should have the opportunity to be corrected, and a community is far better off when it's able to be mindful of which people are acting in bad faith.
This is sort of a plea: Please, write some silly fiction. Write posts about getting better at thinking. Write some posts to help people get better at taking more general action! There's more juice to be squeezed out of the lemon of rationality, and when you're feeling anxious and can't stop staring at the end of the world, lemon juice will at least compliment the mood you're in.
2
1I hope you write more often in the future. Annoyingly Principled People, and what befalls them has actually changed the way I'm approaching life in a pretty substantial way, by managing to convey a specific phenomenon in a way that felt very meaningful to me, and by causing someone in that thread to write a specific comment that did the same. Remains to be seen what the outcome will be to the changes I have made due to the post, but it has certainly changed things for me.
Something that would also be worth considering is a second-chance pool for the frontpage, consisting of posts from a long time ago. Occasionally resurfacing something from a long time ago to a lot of people at once is potentially very high-value; one could say that the feed is a variant of this, but the "lot of people at once" aspect is what I think is the most important piece of a second-chance pool. The feed doesn't directly incentivize writing comments (since there's significantly reduced visibility to old posts), but comments are a huge part of the value of a LessWrong post.
What would be the most efficient way to "uplift" to competency a relatively-intelligent person whose education in mathematics stopped prior to high school?
1Math Academy ($49/mo) is great for training the fundamentals; one of the developers has a manifesto.
1Math is one of those subjects where everything builds on itself, step-by-step. And skipping prerequisites for something makes it much harder to learn. I would start by figuring out the last math that you know fairly well (multiplication tables? fractions? pre-algebra?), and then figuring out what math you would like to know.
The three big pieces of high school math in the US are algebra, geometry and (depending on the school) calculus. It's possible to dislike algebra but really enjoy geometry.
My biggest advice for math is to do the problem sets! If you don't, it will sneak up on you and you'll start losing track. Try looking at new ideas from multiple angles, and ask yourself how it works. Play around with it. The textbook and the problem sets are guides, and the real work happens in your head.
My best physical analogy is getting strong. You gotta lift the weights and do the reps, and that takes time. This gets a lot of clever people in trouble, because they try to push too far, too fast, and then they get lost. If that happens, go back study the previous step some more.
1Competency at what? Concretely, what specific thing is this person trying to do and finding that they are unable to do because they lack the math background?
Lots of things: Their job is in something they aren't really suited for given the educational background they have. Failing to understand computer science textbooks because things that the authors see as relatively basic just aren't there; striking out during the quantier parts of interviews at even relatively routine things (like basic mental arithmetic or trigonometry problems or "understanding what a logarithm is"); not having a real understanding of or capability to understand information theory. Just sort of a systemic failure to have been taught anything of any substance.
They also just seem to want to learn math, for no particular reason at all. They say it feels like they're colorblind in a world where color is intensely useful for understanding and survival; they feel longing whenever their mathematician friends talk about the things that they're interested in, but there's just a general and fairly comprehensive gap between them.
Have them need it for work. Lots of trades benefit from the ability to do basic engineering calculations, and the most important factor in learning is motivation. The second factor is tutoring, and LLMs can give you the basics these days, especially for something as standard as high school math.
Writing this just for my own future reference to be able to figure out how calibrated I was in retrospect:
38%* unemployment in 2029 is unhinged, and there's no realistic path to hitting that target in the next two and a third years, even if models get good enough to automate all work within that time period (which I also think seems unlikely).
This will likely backfire in the same way 2027 did, PR-wise, because they do not make it clear enough that this is not, whatsoever, a median prediction. It will likely be taken by the wider public as, "Look at these AI researchers convinced of their doomsday cult's wisdom trying to boost valuations pre-IPO."
* Edited due to mistyping a number.
2nd edit to stop more Linchings: They aren't talking about prime-age labor force participation rate in the doc. This comment was written under the assumption that they were, because total LFPR is basically useless as a statistic.
1
1Unless I'm missing something, I don't think they're suggesting 48% unemployment by 2029. It was initially confusing for me too, but they're using employment rate as defined (from clicking on it):
"Percentage of Americans who have jobs. As machine labor gets cheaper than people at more and more tasks, wages fall and workers leave paid work.
This is notably different from the complement of the unemployment rate[1]. For example, Percentage of Americans who have jobs includes people not actively searching for jobs too like stay at home parents, and the 62% (not 52%) number seems to be around the current employment rate.
Also, the authors can clarify but the pre-2029 predictions seem to be around their median timelines. You mention AI 2027 too, but I believe Daniel Kokotajlo's median TAI timelines were actually 2027-2028 (and then they slightly lengthened afterwards).
- ^
As in, it's not Employment Rate = (100% - Unemployment Rate)
1(In addition, those timelines indeed don't make sense, or at least so far no one has been willing to explain how they make sense.)
2LFPR in the US is currently 61.5% so to "drop" to 62% in 2029 you need approximately a -0.5% change.
1Well, yes. It was, as Vivek already pointed out, based on a misunderstanding of what was meant by a number.
I was assuming it was based around prime-age LFPR; it wasn't. I still think the end conclusion in my comment is more or less true, however.
Fwiw when people talk about the "unemployment rate", both colloquially and in political discussions, they're usually referring to U-3, which is unemployed people who sought work some time in the last 4 weeks. It is currently at 4.2%.
This makes a lot of sense as a short-term/political category, but makes less sense if we're envisioning long-term changes.
I dunno if prime-age LFPR instead of LFPR would make Plan A easier to digest (and would tentatively bet against). An obvious problem with that is that the numbers will be closer to U-3 so misunderstanding the number would be more common.
Regardless I think the exact metric doesn't matter as much as long as it's well-defined, the distribution of changes matter more.
Everyone on this site knows Ray Kurzweil. Unfortunately, I think almost no one on this site knows the thing Kurzweil was responsible for that was most important: The K2000 synthesizer.
It had an incredibly impressive synthesis engine, and an interface so complicated that the "video training manual" was over an hour long!
The sounds that come out of the thing are absolutely transcendent, and if you were into music in the 90s or 2000s that wasn't rock music, you probably have enjoyed it in something or another.
If you work on AI, please consider hurrying to one outcome or another so that Kurzweil can get back to doing what I care about (making beautiful and innovative ROMplers) prior to his eventual appointment with a liquid nitrogen vat. The new models simply are not the same, and the space could use his undivided attention once more.
I was gifted a (broken) K2600 in 2017, and am thrilled to learn of this new source of narrative continuity in my life. I’ve chalked it up to a simple name collision for the past near-decade.
That said: I don’t take the K2000 to be an especially complicated or lovely-sounding instrument among those in its reference class (MS2000, Jupiter, Virus, Yamaha SY/ES). And, indeed, it hasn’t stood the test of time in terms of a proliferation of emulations, etc. Still, definitely a canonical instrument and excellent factoid (for me especially!).
1The thing about that comparison is that the K2000 is a ROMpler (and there were releases a few years later that made this workflow much easier), so the sounds you can get out of it depend pretty heavily on what you give to it, and how innovatively you crush those sounds.
Regardless of your views on the (excellent!) K2000, please work on bringing the AI thing to a speedy close in one way or another (convincing Eliezier to become an accelerationist and to write accelerationist blog posts is perhaps the most simple route available to you as someone who works at MIRI) so there can be another good Kurzweil synthesizer, please. This would meaningfully improve my life, and maybe lead to a new Postal Service record, and therefore has a higher EV than anything else MIRI could be doing right now.
What's the best strategy for undertaking and executing on a very long goal, with a sort of blurry and ambiguous path, because you're not really far enough along to know what the process of getting there looks like?
Say the goal of "I want to have done some novel research by the time I'm 30," or, "I want to be skilled enough at drawing that I can successfully and quickly translate what I see in my head to canvas;" these are the sorts of goals you see people have often, but it seems unclear to me how to go about properly planning and executing on one of these.
Here's a post from 80,000 hours about choosing a career. The principles of trying stuff out to see if you'll enjoy it or have the aptitude, and to see what is even possible or how stuff works, applies to basically everything.
Contrary to the common school of thought I believe you should define your goals based on what immediate actions you can take. For your research goal, I would begin thinking about - what kind of research can you start tomorrow that points in novel directions? The answer is vastly different depending on if today you are a mycelium hobbyist to a python programmer who has a GPU powerful enough to run local LLM models or have a Zoom Recorder and a personal relationship with the last known speaker of a dying language.
For the drawing one I'm really confused - do you want to paint onto canvas or draw (onto paper, or into a digital tablet)? Do you want to achieve photorealism or, for example, are you more interested in gestural accuracy? In either case the immediate action is probably, research which fine art books are well regarded and loaded with exercises, buy it, once its in your hands - one by one work through all the exercises.
The prevailing wisdom is that this kind of goal-setting is restrictive because, simply put, you don't know what affordances or opportunities are available to you until you set out to accomplish a goal. However, in my personal experience, there are no opportunities or affordances that emerge... or if there are, they only exist in proximity to very clear plans and sequences of actions to accomplish the goal based on concrete assumptions.
Therefore I advise- find the most immediate action you can take, with your current resources, knowledge, and even social networks that appears to align with the goal. One way to do that is to work backwards by defining what conditions would narrowly define success for the goal.
Think for a bit, then just try getting started. The best way to get more info is to just go and try it out.
Depending on what counts as "novel research", smart highschoolers frequently manage to prove new combinatorics theorems with the help of mentors.
For drawing, my current plan is to get through the Drawing on the Right Side of the Brain workbook as recommended by Raemon. I don't expect to particularly enjoy it, so have been putting it off. Though, I do have a free span of time currently...
If you make any cool drawings (or bad ones!), feel free to show me them! I have basically an infinite appetite for seeing people learn how to draw; that's a really good plan!
1Are there any good posts in the archive about how to become more mentally resilient? I assumed it was something that I would get better at over time with maturity and experience, but every failure seems to hit a bit harder as time goes on.
Epistemic status: not sure.
People comparing Mackenzie Scott's charitable donations to EA charities to dunk on them for "being ineffective" on a health basis seem to be arguing in pretty bad faith.
A large portion of those donations weren't health-focused at all, but rather cause-focused. This doesn't seem more out of line to me than (say) EA focus on x-risk non-profits: Both have nebulous return in a quantifiable sense, but both do (for better or worse) make noticeable impacts in the world on a long enough timescale.
Sometimes a person cares about a thing that isn't about saving as many immediate-now lives as possible, and this seems... kind of critical? When EA was initially pitched to me, part of that pitch was, "More flashy stuff will likely get funded without you, so you can aim to spend as effectively as possible without worrying about the things you personally care about going underfunded."
Things like the YMCA, LGBT advocacy groups, jazz arts funding (all taken from the Yield site) - I don't feel like any of these are bad things to fund. If charitable spending was only malaria nets and chemoprevention, I feel like it'd be a worse world.
I'd go as far as to say that the current giving strategy Scott is undertaking is probably optimal for building soft power in the United States, though I don't think she's actually trying to at the moment.
Soft power breeds hard power in time. Who will society look more fondly on in twenty years, the people who allocated all of their funds to abstract figures overseas and constantly talked of how virtuous they were for it, or the person who grabbed the drowning child out of the lake, got them a good education, and then left them alone?
And who has more capability to be effective or damaging, the person who chooses who gets to control USAID, or the person who has $26bn to throw around?
I don't actually know the sum and substance of her charitable donations; I haven't paid that much attention, and I'm a pretty strong believer in malaria nets, so I'm weighted toward "probably pretty suboptimal," especially in the immediate-now.
It just seems wrong to me to go after her in the specific way a lot of EAs are, though; yeah, she's giving ineffectively. Giving most effectively in a cause-blind manner wasn't her goal. But now a lot of the things you and people with capital care about domestically are taken care of for a while, and you can make more persuasive arguments for reallocating a bunch of funding to be genuinely optimal.
Not everyone is going to be a strict, Singer-style utilitarian; the world would be a lot worse if they were. Very few people are solely spending money on shrimp, even though from a utilitarian standpoint, it's way more effective to improve the lives of shrimp than to save human lives, or to give to Lighthaven. You pick issues to care about, and you care about them; if nobody cares about the local problems, it seems likely that there'll be no one to care for the shrimp after long enough.
What would YOUR version of dath ilan be like? Do you think it would be better or worse than Yudkowsky's? What would it get right that his doesn't? What would it probably end up worse on?
There should be a religion where the afterlife starts by appearing in your median world -- presumably a kind of heaven for good people, and a kind of hell for bad people, but with no external judgment: you are judging yourself by how much you love or hate your median world.
(Originally I wanted to say that the entire afterlife should be your median world, but that is to harsh. Meeting different people is sometimes nice, and I also don't like the idea of losing your friends forever. But everyone should get a hard lesson on what is it like to have yourself as a neighbor.)
Did you mean the reader's median world, not the reader's idea[1] of Yudkowsky's MW? I suspect that the MW of most readers merely wouldn't stumble into various forms of incompetence like kids failing to learn to read. Instead, the erroneous practice would be tested, found not to outperform the baseline, and be rejected.
A worst-case scenario is that the MW is an incoherent concept due to reasons like evolution pushing the average personality away from that of the reader[2] or due to the fact that the reader would be able to think through only a few implications of a wholesale shift of personalities (e.g. if Yudkowsky's description of dath ilan has the line related to murders and resurrections from a frozen brain, but not an explanation for how murderers emerged in the first place) or biased to only think through desirable implications.
- ^
For example, if a counterfactual Yudkowsky didn't end up revealing his BDSM preferences, then they wouldn't influence the reader's views on dath ilan.
- ^
Or reasons like the reader's personality having been changed by having been a war participant and the governments being far more reluctant to start a new war, causing next generations not to receive such an experience at all.
I did mean the reader's median world! The reader's idea of Yudkowsky's MW would be too polar between participants; it would be less fun.
Good thoughts!
Let's say I want to illusory truth effect myself into caring about AI safety, or increasing chatbot capabilities, at least enough to have more interesting conversations about either topic. Where would you start? Where are the good reads? Have any of you written any good papers lately?
Plan S is pretty obviously the optimal outcome out of the choices presented, but the 2040 authors fall short of sketching what an ideal Plan S would look like. I would go as far as to say that something even further than Plan S would be ideal, but this is too far outside of the Overton Window, so probably not worth getting into on this forum.
It also is very clear that the authors speak to chatbots too much, based on the writing style used throughout. Example:
"We are sympathetic to Plan S and think that it might be better than Plan A. However, we recommend Plan A instead. Here’s our thinking:"
Note: This is not accusing them of using LLMs to generate this document. I am simply stating that they have chatbot-colonized patterns of speech.
I think {companies, people, organizations} should be way faster to reject. I think little is less comfortable or ideal than the waiting period, and it benefits almost nobody. Think about how much suffering would be eliminated by rejection happening significantly faster!
People at EA companies should push for instant rejection after a bad interview as a low-friction, trivial-to-implement, extremely effective intervention against suffering.
I have been revisiting the 2023 post "AI Timelines" today. I would be interested in seeing what the participants within would offer as estimates at the present time.
From my own cursory estimation, it seems like Kokotajlo was wildly miscalibrated on many or most fronts, while Cotra and Erdil's predictions in 2023 seem fairly calibrated, leaning heavily in favor of Erdil (who seems in retrospect to be remarkably well-calibrated).
Scattershot takeaways:
Kokotajlo correctly assumed governments would completely fail to slow down timelines.
Erdil's predictions are interesting; he speaks like a gambler (in a very positive way); very precise, taking into account many systemic factors, while not allowing any one to dominate. Conservative revenue estimate in median prediction scenario for labs in 2030 (which seems reasonable given how historically unprecedented the scale of the current economic bubble has been), with seemingly-reasonable predictions across the board otherwise.
Cotra quote that I thought was fairly solid:
Yeah, I just think the way we get our OAI-engineer-replacing-thingie is going to be radically different cognitively than human OAI-engineers, in that it will have coding instincts honed through ancestral memory the way grizzly bears have salmon-catching instincts baked into them through their ancestral memory. For example, if you give it a body, I don't think it'd learn super quickly to catch antelope in the savannah, the way a baby human caveperson could learn to code if you transported them to today.
I think the ultimate conclusion I have is that habryka needs to do more interviews. He's good at them.
1@Daniel Kokotajlo's most recent views are expressed in Q1 2026 Timelines Update. Maybe he will release a new update?
Edited to add: Why do you believe that the predictions of Cotra and Erdil are mostly correct? Erdil's prediction which struck me was the following:
Erdil's misprediction
My median world looks something like this: we keep scaling compute until we hit training runs at a size of 1e28 to 1e30 FLOP in maybe 5 to 10 years, and after that scaling becomes increasingly difficult because of us running up against supply constraints. Software progress continues but slows down along with compute scaling. However, the overall economic impact of AI continues to grow: we have individual AI labs in 10 years that might be doing on the order of e.g. $30B/yr in revenue.
We also get more impressive capabilities: maybe AI systems can get gold on the IMO in five years, we get more reliable image generation, GPT-N can handle more complicated kinds of coding tasks without making mistakes, stuff like that. So in 10 years AI systems are just pretty valuable economically, but I expect the AI industry to look more like today's tech industry - valuable but not economically transformative.
The IMO gold was achieved in July-August 2025 and IIRC the revenue was reached in 2026. Half of 1e27 FLOP was reached by Grok 4, 1E27 is likely reached with Mythos Preview, I expect 1e28 FLOP to be reached in 2027 and to bring the goddamned supercoders (or did it partially happen with Anthropic's AARs? Then why did Anthropic's ECI keep scaling linearly over time for all models except for Mythos?)
For what it's worth, I think my qualitative predictions in this essay were good, but because I was consistently putting 50% chance on the current path of AI scaling hitting limits, my "median world" looks less impressive than you might expect. I think I flagged this in the conversation - this "median" world I'm describing is basically "the current paradigm works, but barely".
I think the world we're actually in is more like my 75th percentile at that time (or, equivalently, my median conditioning on the current paradigm continuing to work well). So I think Daniel's predictions were actually better here, because he didn't have this hedge. I don't know how you can read that post and come away thinking I was better at the specific numerical forecasts that have resolved thus far.
There's a more interesting question of whether I was right ex ante or not, and I think given what we knew at the time my predictions weren't unreasonable. But it's hard to litigate a difference of 1 bit of evidence (which is all that a 50% hedge amounts to) between two forecasts in a domain like this.
Today in concerning news about people who should know better:
The current socioeconomic moment is just so.... stupid. Meritocracy and just-world fallacy have always been fake, but what exists in their place has rarely been rendered as transparently as in modern times. The bright side is that these people clearly have too much money, and they are not so bright that you will find it hard to relieve them of that burden, if you set your mind to it. The downside is that these people are not competent in their evil, which is, strangely and counterintuitively, so much worse than the alternative.
Sure, your LLM might be able to run Photoshop, solve math problems, demonstrate that billions of people have no taste, convince children to commit suicide, cause psychosis in a surprising percentage of people I know personally, and write really bad code. But can it write a good glowfic? Checkmate.