Quick Takes — LessWrong

LESSWRONG
LW

I'm about to go to sleep but I am a bit confused about Epstein stuff.

In 2019, @Rob Bensinger said that "Epstein had previously approached us in 2016 looking for organizations to donate to, and we decided against pursuing the option;"

Looking at the Justice Dept releases (which I assume are real) (eg https://www.justice.gov/epstein/files/DataSet%209/EFTA00814059.pdf)

That doesn't feel a super accurate description. It seems like there was a discussion with Epstein after it was clear he had been involved in pretty bad behaviour. (In 2008 he pleaded guilty... (read more)

Showing 3 of 10 replies (Click to show all)

-1Ben Pace9d

shrug I think sharing the info is good & fine. I think you have some responsibility for the hypotheses you privilege. Having an algorithm where you spam ppl with low-quality moral accusations just because a different social scene is generating them, is kind of attention-wasting.

Nathan Young8m20

Where do you think I was "spam"ming?

3Matrice Jacobine10d

Surely if you're around those parts you should know that billionaire philanthropy is generally ineffective and not focused on effective interventions in global health and development. Epstein was primarily known as a philanthropist focused on academic and nonprofit scientific research, hence the high amount of famous/Ivy League scientists in his social circle. According to Eliezer's account he didn't understand SIAI's beliefs on alignment.

1a3orn's Shortform

1a3orn10h190

I've heard many say that "neuralese" is superior to CoT and will inevitably supplant it. The usual justification is that the bandwidth of neuralese is going to be higher, which will make it better. But (1) bandwidth might not be better in this case; it isn't in all cases and (2) there are other factors that could theoretically operate against this, even if this is true.

Has anyone cleanly made the case for why neuralese is better or asymptotically technically inevitable, at length / clearly?

Showing 3 of 11 replies (Click to show all)

Jude Stiel9m10

(In agreement): Neuralese is ~equivalent to wrapping your model as a DEQ with the residual stream shifted by one on every pass as far as I can tell, and it's not obvious to me that this is the relevant One Weird Trick. The neural network already has a way to shuttle around vast amounts of cryptic high-dimensional data: the neural network part of the neural network.
It seems much more likely to me that the relevant axis of scaling is something like a byte-latent transformer with larger and larger patches.

21a3orn14m

I appreciate the reference, although I found this article + discussion pretty underwhelming; it's part of what's motivating my question. For instance, not all forms of unintelligibility in CoT's are necessarily evidence of a drive-to-compression. But the article takes for granted that the weirdness we see in chains-of-thought are evidence towards this; it views various forms of weird text that I'd see as evidence for screwed up training systems or spandrells of the training process and just assumes they are "thinking" driven into non-human-legible vocabulary. The guy didn't particularly consider other hypotheses for what he was seeing. And similarly he discusses "redundancy" in human languages, and immediately assumes machines would want it to go away, while not... thinking of why it's there, and whether it would stick around for machines potentially. This isn't anything like a full refutation of him, tbc, I'm just giving my impression of it at a high level. By my takeaway is that if this is the best discussion than I don't think anyone's actually tried to work out the reasoning around this carefully, even if neuralese is actually inevitable.

21a3orn34m

Yeah looks like it's vectors as some kind of an autoencoder between different text models at first glance, not using it as an intermediate state to assist thinking in a single text model? Or something; the application list is underwhelming

Viliam's Shortform

Viliam27m20

If Hitler is our default example for "someone who killed (caused to die) a lot of people on purpose", who could be our default example for "someone who killed (caused to die) a lot of people by negligence"?

Disclaimer: I am not implying that Hitler killed more people that Stalin or Mao, etc. I am just saying that he is a convenient example; if you mention him, people know what you meant. I am looking for similarly convenient examples of criminal negligence.

(Inspired by this thread, but didn't want to go off topic.)

SorenJ's Shortform

SorenJ2h10

Suppose we had a functionally infinite amount of high quality RL-/post-training environments, organized well by “difficultly,” and a functionally infinite amount of high quality data that could be used for pretraining (caveat: from what I understand, the distinction between these may be blurring.)

In that case, what pace would one expect for model releases from AI labs in the short term to be? I ask because I see the argument made that AI could help speed up AI development in the near to medium term. But it seems like the main limiting factor is just the am... (read more)

Joanna's Shortform

Joanna10h370

Hi, I am Joanna. I did a design work trial for Lesswrong that ends tonight! As part of that, I designed a new profile page. If you don't like it, I won't be around to fix it unless they hire me. But, the team would surely care if you have comments! (And I would too.)

Showing 3 of 63 replies (Click to show all)

Simon Lermen2h20

I liked the design when i saw it today, but also would like aggregate statistics like comments count/ post count/ recent activity. perhaps even something like github showing a calendar with activity for each commit. It would also be good to retain a bio with a self description and optionally urls to websites or social media accounts.

2Viliam4h

Maybe an AI should automatically read the article and generate an image based on it. At least that would give more information than a random (albeit beautiful) color thing. Not sure whether this is a good idea, but it seems like an interesting experiment. Perhaps you could try this with some selection of high-karma (so the readers are more likely to remember them) recent articles that did not contain a picture, and post the results? If the prompt "draw something that is associated with the text of the article" is not sufficient, maybe it could include some nudges, such as "check whether the article contains math? is about AI? is fiction?" and then maybe make some specific recommendations (e.g. use a specific artistic style for fiction).

5Viliam4h

Yeah, if I am looking for a post with a specific name, or just "I think I wrote a post on this topic", then seeing lots of article names is more useful than seeing only a few previews. Perhaps add a checkbox "show previews"?

sarahconstantin's Shortform

sarahconstantin1d50

links 2/11/26: https://roamresearch.com/#/app/srcpublic/page/02-11-2026

https://www.clinicaltrialsabundance.blog/p/the-moderna-rtf-and-the-cost-of-regulatory Ruxandra Tesloianu on why FDA changing its mind to reject trial protocols it previously approved is harmful for biomedical progress
https://www.drmaciver.com/2015/04/some-empirically-derived-testing-principles/ D.R. Maciver advice on software testing
https://notebook.ldeming.com/garden/index.html Laura Deming on how, if longevity ever really works, it would be creating a new sort of creature

... (read more)

Viliam4h20

if longevity ever really works, it would be creating a new sort of creature with a mindset very different from a "normal"-lifespan human

Yeah. It's like, if you imagine a world where everyone dies at the age of 25, it would seem obvious that they would be missing some psychological things, both on the individual level, and on the social level. So the same should be true in the opposite direction.

Though I wonder how much we can extrapolate from the existing data, or whether something unexpected would happen. Not all things are linear. (For example, male crim... (read more)

John Coleman's Shortform

John Coleman1d30

The music industry has already passed the threshold for digital AGI, but seems to be experiencing disruption dramatically less than AI song generation capabilities might suggest. Anyone can create a song of expert level quality in under 10s with an automatically generated prompt (or write one themselves if they want). The Chainsmokers took about 8 hours in total to make the single Roses, which has over 1.3bn streams as of 02/10/2026 and I believe is on the quicker end of writing time for pop songs, although I am not sure reliable data exists for this anywh... (read more)

Viliam4h31

the bottleneck of music productivity is in capturing attention

Yeah. I think the same applies to capitalism in general; these days it is relatively simply to produce many things, the problem is that thousand competitors can do the same thing, so the key is to convince the customer to buy your product instead of their mostly identical products.

Now I am not saying that good songs are identical, but they are in some sense fungible. Like, if you have a favorite song, it is easy to imagine a parallel universe where it doesn't exist, and something else is your fa... (read more)

Thane Ruthenis's Shortform

Thane Ruthenis2d5310

Model to track: You get 80% of the current max value LLMs could provide you from standard-issue chat models and any decent out-of-the-box coding agent, both prompted the obvious way. Trying to get the remaining 20% that are locked behind figuring out agent swarms, optimizing your prompts, setting up ad-hoc continuous-memory setups, doing comparative analyses of different frontier models' performance on your tasks, inventing new galaxy-brained workflows, writing custom software, et cetera, would not be worth it: it would take too long for too little payoff.... (read more)

Showing 3 of 14 replies (Click to show all)

Viliam5h20

My guess is the average person on LW should be spending around 10 hours a week trying to figure out how to automate themselves or other parts of their job using LLMs.

Yeah. I am nowhere near doing this systematically, but I noticed that whatever I am doing, it makes sense to ask "could I use an LLM to help me with this?" That includes even things like reading Reddit -- now the LLM could read it for me, and just give me a summary. (I haven't tried this yet.)

It is even worth revisiting the old (pre-LLM) question of "could I automate this using a shell/Python ... (read more)

7Raemon2d

I lean towards this, despite being a guy currently heavily invested in AI tools. Cluster of things that all seem true to me: * I am 100% addicted to vibecoding in a straightforward "this is fun and dopamine inducing way", which is making it hard to reason about. * As fun hobbies go, it does seem fine, all else equal * You're broadly right about the 80%/20% rule, and microoptimizations not being worth it. But: * There are some infrastructure that probably will make sense to have as the AIs scale up, which AI companies either won't provide, or, you probably don't want to trust. (This is a bet, I'm not that confident) * Learning to wield AI is going to be important (for at least many people. I think it's more straightforwardly important for software engineers than theoretic researchers). * Some of that infrastructure is stuff that AI can basically one shot. I think the skill/habit to cultivate here is "check if it 1 or 2 shots it. If not, bail."

5SatvikBeri2d

I think this is incorrect, but that agent swarms etc. are mostly not helpful, and that the large productivity boosts are specific to domains or situations. Two from my side: Claude Code got much better once I got it successfully working with a REPL (which made the feedback loop much faster, let me inspect the outputs etc.) and once I wrote up a fair bit of documentation on how to use our custom framework. Edit: I forgot that not everyone works in software. I am much less confident that this applies in other domains today.

Linda Linsefors's Shortform

Linda Linsefors3d*100

Even now and then I meet someone who tries to argue that if I don't agree with them this is because I'm not open mided enough. Is there a term for this?

Epistemically I'm not convinced buy this type of arugment, but socialy it feels like I'm beeing shamed, and I hate it.

I also find it hard to call out this type of behaviur when it happens, even when I can tell exactly what is going on. I think it I had a name for this behaviour it would be easier? Not sure though?

Edit to add:

I've now got some more time to figure out what I want and don't want out of this th... (read more)

Showing 3 of 13 replies (Click to show all)

Viliam5h20

A reply I got in a similar situation, paraphrased: "well, it's you who identifies as a rationalist, you hypocrite."

In other words, as if by calling myself a rationalist (not in that specific debate, just generally something I said in a different context that my opponent knows about) means that I accept an asymmetrical burden.

2Viliam5h

Yeah. The opponent's move is establishing "you can change your mind and adopt my opinion" as the criterion to measure your open-mindedness. If you can switch to their position, you are allowed to keep calling yourself open-minded; if you don't, you lose that right. (The rational answer would be: yes, you are capable of adopting any position, including theirs, when there is a good reason to do so. Open-mindedness does not mean adopting random positions for no good reason, or simply because someone calls you a chicken if you don't.) So the essence of the move is "I decide what is the true (costly) signal of the trait you claim to have". And the choice of the signal is obviously self-serving. I mean, in theory, they could have asked you to demonstrate your open-mindedness by adoption a position they don't agree with; that would be an equally valid proof of your ability to change your mind. But of course there is no incentive for them to do so. Which shows that evaluating your open-mindedness impartially was never the true goal here. Another thing is that there is a difference between "being open-minded" and "signaling open-mindedness". Just because you are capable of adopting various kinds of positions, doesn't mean that you should. To consider options X, Y, Z, and afterwards decide to stay with the original X, can be perfectly open-minded, even if from outside it may be difficult to distinguish from "the person did not consider Y and Z at all". (It's like when a gifted child solves a mathematical problem too fast, and the teacher accuses them of merely guessing the answer. There is a difference between doing the work, and demonstrating to other people that you did the work.) Shortly: * open-mindedness does not mean "doing what you want me to do"; that's called social pressure * it is perfectly open-minded to consider a hypothesis... and then reject it

4cubefox2d

A more transparent term would be psychologizing: See also Ayn Rand on this topic: (Lots more ranting about psychologizers. Schopenhauer energy.)

adamzerner's Shortform

Adam Zerner3d60

Toughness is a topic I spent some time thinking about today. The way I think about it is that toughness is one's ability to push through difficulty.

Imagine that Alice is able to sit in an ice bath for 6 minutes and Bob is only able to sit in the ice bath for 2 minutes. Is Alice tougher than Bob? Not necessarily. Maybe Alice takes lots of ice baths and the level of discomfort is only like a 4/10 for here whereas for Bob it's like an 8/10. I think when talking about toughness you want to avoid comparing apples to oranges.

I suspect that toughness depends on t... (read more)

Showing 3 of 4 replies (Click to show all)

Viliam6h20

I feel like (2) is usually more appropriate than (1) [...] It usually makes sense to make the thing easier than to improve your ability to push past it.

I agree with this as a long-term strategy for dealing with repetitive problems. And I share the suspicion that "tough" people often have it easier (which may or may not be a result of their previous actions).

But sometimes life throws an unexpected thing on you, and then you roll for "toughness" or fail. (Though maybe you can also prepare for the unexpected by practicing many different things, thereby increasing the chance that some skill you have will be relevant for the new situation.)

2Adam Zerner2d

Yeah I agree that we probably don't have a way to tease apart the toughness from the pain tolerance. And I guess I agree that for "many cases" you care about the outcome, not the process, so you just care that Alice is better at sitting in ice baths, not why she is better. But I also have a feeling that toughness matters. If Alice is good at sitting in ice baths because she happens to be very insensitive to cold temperatures, that isn't something that is predictive of life success. But if Alice is good at sitting in ice baths because she is tough in some sort of generalizable way, that seems important because that seems predictive of her being able to handle other difficult situations well. It sounds like you agree with this but that you are skeptical that being able to tough your way through ice baths is predictive of being able to tough your way through other adversities? If so, I have a different intuition but I'm not sure how to make the reasons for my intuition legible.

2Dagon2d

I'm sympathetic to the model - in many cases it seems that there is a generalizable trait of "toughness" (a few decades ago they called it "grit" or "determination", now it rhymes with "agentic"). It's tempting to simplify things to that level. But I'm also skeptical of my own desire to believe that, and I don't actually think it's true often enough to count on it. When I press myself on edge cases or most specific data->prediction proposals, it loses a lot of appeal. There are clearly some people who fare better than others across many domains. Exactly which traits cause this, and how ingrained and unchanging those traits are, remains quite difficult to pin down. Personally, I think it's 50% luck, 50% genes, 50% early environment, and 50% current environment. Yes, success is overdetermined :)

Szeth's Shortform

Szeth3d10

JD Vance very possibly knows who Zvi Mowshowitz is. Vance used to play the Magic the Gathering deck "Yawgmoth's Bargain" which was largely designed by Zvi. Not sure what, if any, implications this has but certainly an interesting piece of trivia.

Showing 3 of 6 replies (Click to show all)

2Szeth2d

I would put "Doesn't know Zvi, played magic" at number two. I think you might be overestimating either how well known Zvi is or how likely one would be to know Zvi conditional on playing magic in the late nineties/early 2000s.

6Linch2d

base rates, man, base rates.

Viliam6h20

Yeah, with all due respect, more people play Magic than know Zvi. (Outside of LessWrong.)

Chara#2736's Shortform

Character#27364d0-2

I know many members and influential figures of this are atheists; regardless, does anyone think it would be a good idea to take a rationalist approach to religious scripture? If anything, doing so might introduce greater numbers of the religious to rationalism. Plus, it doesn't seem like anyone here has done so before; all the posts regarding religion have been criticizing it from the outside rather than explaining things within the religious framework. Even if you do not believe in said religious framework, doing so may increase your knowledge on other cultures, provide an interesting exercise in reasoning, and most importantly, be useful in winning arguments with those who do.

Viliam6h20

I would find that difficult, because in my understanding, religion requires some sins against rationality: motivated thinking, privileging a hypothesis, writing your bottom line first...

I mean, most religions assume that a book written a few millennia ago, when people knew practically nothing about the world, because the scientific method didn't exist, and "a high-status person made it up" was a likely source of any statement that couldn't be immediately verified... that this book contains the true answers to the secrets of the universe and beyond. The obv... (read more)

5Mitchell_Porter3d

There is a very long history of interaction between scripture and reason, both collaborative and adversarial. But I assume you are thinking mostly of Less Wrong rationalism, rather than earlier rationalist movements. Let's think for a moment of where Less Wrong rationalism came from. What are its sources? It's a mix of transhumanism and secular science, with a bit of metaphysics to furnish tentative answers to questions like, what is consciousness, or what is existence. It is very secular and materialist in its approach to ontology and epistemology. But because of its transhumanism, its notion of the possible, and the destiny of the world, can remind people of religion, with its heavens and hells and its superintelligences reaching out acausally across the multiverse. The Sequences also tackle ethics and the meaning of life. Now while this may be an outgrowth of atheism and humanism which has come to acquire features also seen in religion, one thing that distinguishes it from scriptural religions, is that there is no divine revelation. The Sequences are just a message from Eliezer, not a message from God. The rationalist attitude here towards the world's scriptures is going to be that they are purely a product of human brains. Whether they are regarded as poetry, pragmatism, or delusion, there are no divine minds involved in their production. That is going to be a major "crux" of disagreement between rationalists and adherents of scriptural religion (a disagreement that was central to many earlier interactions between scripture and reason). And it raises the question, what kind of interaction are you looking for? Would it be to retain some of the premises of scriptural belief, but employ rationalist epistemology within that framework? Or it could be to find rationalist interpretations of religious parables and aphorisms, which are notoriously susceptible to reinterpretation. Or, you could be looking for people who are both rationalists and theists. Bentham's

Kongo Landwalker's Shortform

Kongo Landwalker7h10

According to ChatGPT 5.2 Jackson Kernion "is likely the same person" as Foreign Man in a Foreign Land.

I have used the same instance of a chat for many different topics, from music theory to scrabble ttg, from spiders to Lean. Apparently, the chat wanted to see some connection in that mess of random questions and linked long-long ago mentioned Nebula subscription to the new question.

Richard Ngo's Shortform

Richard_Ngo2d575

A response to someone asking about my criticisms of EA (crossposted from twitter):

EA started off with global health and ended up pivoting hard to AI safety, AI governance, etc. You can think of this as “we started with one cause area and we found another using the same core assumptions” but another way to think about it is “the worldview which generated ‘work on global health’ was wrong about some crucial things”, and the ideology hasn’t been adequately refactored to take those things out.

Some of them include:

Focusing on helping the worst-off instead of em

... (read more)

Showing 3 of 21 replies (Click to show all)

3Eli Tyre14h

Strong disagree. We could have done those things, but the rationality movement didn't have enough motive force or coordination capacity to do much, beyond AI safety.

2Richard_Ngo8h

Yes, because it funneled all of its best and brightest into AI safety? We might be evaluating the hypothetical at different points. I'm thinking of the movement coalescing around the sequences except the message underlying the sequence is "you should solve ageing" rather than "you should solve alignment".

Eli Tyre8h20

Maybe I'm missing something. Why are you comparing to the that hypothetical world?

Steve Byrnes’s Shortform

Steven Byrnes13h240

FYI, if anyone read my post The nature of LLM algorithmic progress last week, it’s now a heavily-revised version 2.

Bistrofan's Shortform

Bistrofan1d73

A lot of the rationalist discourse around birthrates don't seem to square away with AGI predictions.

Like the most negative predictions of AGI destroying humanity in the next century or two leaves birthrates completely negligent as an issue. The positive predictions with AGI leave a high possibility of robotic child rearing and artificial wombs (when considering the amount of progress even us puny humans have already made) in the next century or two which also makes natural birthrates irrelevant because we could just make and raise more humans without the n... (read more)

Showing 3 of 13 replies (Click to show all)

Eli Tyre14h42

Yes and I have a major example, one of the leading CEOs in the AI industry. He believes that AI will be more intelligent than all humans currently alive by 2030 while also saying birthrates are a top priority for all countries

Why pin this one (notably crazy-seeming) guy's take on "A lot of the rationalist discourse". He doesn't identify as a rationalist or post on LessWrong. And the rationalist discourse has long thought that his impact models about AI were bad and wrong (eg that founding OpenAI makes the situation dramatically worse, not better).

2Eli Tyre14h

Who are you talking about? It seems to me that the the people who are majorly concerned about AGI destroying humanity are almost entirely disjoint from the people majorly concerned about falling fertility leading to a population collapse? I definitely believe that there's some overlap, but not like more than 5% of either group.

3Zack_M_Davis14h

People try not to be too explicit about this in public, but the real concern is about dysgenics and human capital, not mere underpopulation. Having ten billion people in 2084 may not be good if they're statistically less of the people needed to deal with the problems of ten billion people in 2084.

MP's Shortform

MP14h30

Someone, somewhere, should put some effort and answer the following question: is the cost of AI tasks increasing exponentially that at some point, LLMs coding at a certain time horizon will be more expensive than humans?

leogao's Shortform

leogao5d70

are the Cambridge Brain Science cognitive tests actually reliable and relatively immune to practice effects? I want to have some mostly repeatable measurement of my own cognitive abilities over time, for health tracking reasons, but it's unclear to me how reliable it is

Sheikh Abdur Raheem Ali16h*10

I was thinking of trying out Sustained Attention to Response Task (SART) with response feedback (SART 2). I'm not sure how it compares to Dual n-Back FAQ · Gwern.net.

omegastick's Shortform

omegastick17h00

My wife, who uses LLMs pretty much all day, says that Claude Opus 4.6 feels more 'mature' than 4.5.

Anthropomorphizing models is dangerous, but it's always a bit of a delight when I notice us talking about software using human personality traits. It clearly points at coherent concepts, pieces of software now have distinct 'personalities' that compare naturally to those of humans.

Anyway, I use LLMs a fair bit too, and tend to agree with my wife's assessment. Anecdotally, it is more cautious, more likely to catch itself going on a tangent, and tends towards more of a neutral stance than 4.5.

Does this match your experience?

Wei Dai's Shortform

Wei Dai10h340

My 6 years as a trader / active investor

The Dilbert Afterlife by Scott Alexander, Jan 16, 2026:

Michael Jordan was the world’s best basketball player, and insisted on testing himself against baseball, where he failed. Herbert Hoover was one of the world’s best businessmen, and insisted on testing himself against politics, where he crashed and burned. We’re all inmates in prisons of different names. Most of us accept it and get on with our lives. Adams couldn’t stop rattling the bars.

The EMH Aten't Dead by Richard Meadows, May 15, 2020:

Which only leav

... (read more)