LESSWRONG
LW

Quick Takes — LessWrong

There was some discussion recently about the uptick in object-level politics posts and whether this is desirable or not. There's no rule against discussing politics on LW, but there is a weak norm against it, and topical discussions have historically tended to be somewhat meta and circumspect.

I think the current situation is basically fine, and it's normal for amount of politics discussion to ebb and flow naturally as people are interested and issues become particularly salient. That said, here are a couple of potentially overlooked reasons in favor of mor... (read more)

Max H17m20

So, with all the above said as a preface, one object-level topic I'd be interested in seeing more discussion of is the current situation(s) in the Middle East. Some thoughts:

IMO the high bit for whether the war in Iran is broadly good is whether it is tactically successful and efficient in the short term.

Considerations like "this weakens the US position in a hypothetical hot war against China or Russia" or "this will (further) destabilize the Middle East in the long term" seem second-order to whether the war successfully neutralizes an organized, well-equi... (read more)

TristanTrim's Shortform

TristanTrim2d152

I find trying to find funding or paid roles or even unpaid roles so demoralizing. How do I keep motivated?

I don't want to focus on trying to survey the landscape of funding opportunities and learning to network with people productively. It's so much nicer to just focus on the work I want to be doing, but it seems I either can't make it legible enough fast enough, or it's actually not valuable and I should go do something else with my time.

I want advice. How do I get funding? How do I think about getting funding? How do I stay motivated to keep thinking about how to get funding?

Showing 3 of 6 replies (Click to show all)

3Eli Tyre6h

And a different question: How young are you? Are there experienced people who have worked with you and can vouch for the quality of your work / strategic orientation / etc?

TristanTrim26m10

I'm 35. You can view my experience on my linkedin profile. I was working as a technologist at an automotive company, involved with some AI projects in collaboration with the Vector Institute. That's when GPT-3 was released, prompting me to take the prosaic scaling hypothesis more seriously and change my plan to saving money so I could finish my CS BSc and change my career goal to working on technical AI alignment.

While completing my BSc I had the opportunity to focus on my NDSP project, first as a directed studies project supervised by George Tzanetakis, a... (read more)

1TristanTrim17h

Here's the page describing my projects. I think the NDISP project has the clearest value prospect. I'm currently starting over writing the tool to be stand alone and ready for alpha users. I'd recommend skimming the videos for a sense of the project. All of my other projects seem to be less legible with much lower probability of much greater usefulness. Charitably I they could be described as working on useful paradigm shifts for the field of AI Alignment and rational global coordination. Less charitably they could be described as a crackpot shouting at clouds. I might describe them as Butterfly Ideas that I really want to get out of the stage of being butterfly ideas, but alas, they keep flapping around me.

XelaP's Shortform

XelaP38m10

For about a yearish I've been with varying frequency writing down things I've learned in a notebook. Partly this is so that I go "ugh I haven't even learned anything today, lemme go and meet my Learning Quota", which I find helpful (I don't think I'm goodharting it too much to be useful). Entries range from "somewhat neat theorem and proof in more detail than I should've" to "high level overview of bigger subjects", or "list of keywords that I learned exist". For example, recently I learned that sonic black holes which trap phonons (aka lattice vibrations)... (read more)

Linch's Shortform

Linch3d40

I have a new theory of naked mole-rat longevity, that's most likely false. I (and LLMs) couldn't find enough data to either back or disprove it. Nor could we find anybody who's proposed this theory before.

Any advice for how I can find the relevant experts to talk to about it and see if they've already investigated this direction?

A few years ago I'd just email the relevant scientists directly but these days I've worried about the rise of LLM crank-science so I feel like my bar for how much I believe or could justify a theory before cold-emailing scientists ought to go up.

Morpheus1h20

I am curious: What is the theory? I'd be surprised if your theory works, applies to naked mole rats, but not to ants and other eusocial animals. I always thought naked mole rats live long because they are eusocial, so you having a theory specifically for naked mole rats sounds ominous.

2ChristianKl2d

I would not call myself a domain expert, but I do think I have a rough idea about the field. As far as I understand the mainstream position is roughly: There seem to be evolutionary pressures that result in naked mole-rat longevity and naked mole-rats have made a lot of different adaptations for that reason. We have an understanding of some of those adaptions and there's a good chance that we don't yet understand all of them because understanding all of them would mean understanding aging better than we do currently. If you do study a potential new mechanisms it would make sense to look at how it plays out across species and just just the naked mole-rat. With johnswentworth's post about the Core Pathways of Aging, for example you have the thesis that transposons are important for aging. You can find out that naked mole-rat have unusually low transposon activity, which is a point of evidence to validate johnswentworth's thesis that transposons are important for aging. However, if you would reason from that that naked mole-rat longevity is mainly due to different transposon behavior that would be an overreach because naked mole-rats do plenty of things besides having different transposon behavior. It would be interesting to have a better transposon theory of aging even if that only covers part of what aging is about. That wouldn't really be a "theory of naked mole-rat longevity", so I'm a bit skeptical about anything that would bill itself as a new theory of naked mole-rat longevity because that's not the chunk in which I would think. I would expect that relevant scientist who care about mechanisms of aging and not about the naked mole-rat as a species would react similarly.

2Linch2d

It's an evolutionary theory, not a mechanistic theory. Tbc not a revolutionary theory broadly, nor a theory that would cash out to longevity treatments if true or anything crazy like that.

Erich_Grunewald's Shortform

Erich_Grunewald2h20

Is there a way to cryptographically attest to a given AI model's having been primarily post-trained using some given model spec? For example, is there a way for OpenAI to prove to us that GPT-X was trained with its model spec, without revealing any other information (e.g., algorithmic secrets)? Perhaps trusted execution environments could be used to do this, but I'm not sure. Anyway, if possible, this could help make it harder for someone to insert secret loyalties into a model.

Kajus's Shortform

Kajus2d574

If you code with Claude Code and you randomly ask it a question about something non-related to the thing you are doing right now it will get pissed off. Example:

...some code here.... we filled out 42% of context with coding only.
What is the the parliament of Poland.
Sejm.
Okay. Tell me more about it.
I'm a coding assistant, not Wikipedia. Google "Sejm" — you'll get better info than I can give you from my training data. Is there anything code-related you need?
google it for me. Are you pissed off???
I'm mildly irritated because I'm optimized for writing code an

... (read more)

Showing 3 of 10 replies (Click to show all)

1williawa7h

Also experienced this. Feel it got worse after 4.6. It simultaneously got two new traits that I don't like so much. 1. It will tell me what to do without phrasing it like a question. Like i'll ask it to run an experiment. Then it gets confused about the virtual environment. And if it can't figure it out for a while it will stop and say "You need to contact the system administrator and install xyz". Previous claudes would've said like "It appears there's a problem with the venv ... package xyz doesn't work ... its possible you'd need to contact the system admins ... do you want me to keep trying or should we do something else? 2. I don't feel the model likes talking with me very much. Previously, if I give it encouraging remarks like "Great job!" it would write a sentence or two about about whether it agrees with me, what it thinks was good, what it thinks was bad, what good next steps are. But now it will often just be like: "Ok. Anything else or are we done here?"

Kajus2h10

yeah, I have similar experience

1lilkim20251d

I think that's in line with OP's observation. It doesn't really make sense for an LLM to have any recalcitrance at all to answer a user's inane questions, since doing whatever the user tells it to do (as long as it's sufficiently uncontroversial) is its job. Generalization from training data makes the most sense out of the explanations I've seen thus far, but what training data would cause this? Is there some hidden repository of conversation transcripts in which programmers ask each other random questions during a programming conversation and then get upset?

adamzerner's Shortform

Adam Zerner5h40

Communication culture is important. It is a high level action that, if improved, has lots of nice downstream benefits.

One aspect of communication culture to address is interruption. What should the expectations be about interruption? Do you always wait for the other person to finish talking before you can start talking? Are you allowed to interrupt if it seems worthwhile? What determines whether an interruption is worthwhile? Where is the threshold for how worthwhile an interruption needs to be in order for it to be justified?

I feel like there are various ... (read more)

5Seth Herd4h

Hm. From my perspective I have seen many conversations be excessively railed (opposite of derailed) by lack of interruption. Interrupting whenever you want to speak is very bad form. So is continuing to speak for as long as you'd like to. Meeting in the middle is an art. And I agree that making the desired form explicit would be a good idea. The virtue of interruption is being able to change course faster, and saving time by responding when it sounds like the speaker is no longer making interesting new points. I seem to see people saying too much more often than saying too little before yielding the floor. Whenever you notice that time has run out and more interesting threads were not followed, you might've had too few interruptions. Time wastage to lack of interruption is much less the case with skilled and considerate conversationalists; but so are the downsides of their interruptions. Either interruption or excessive speaking can be used to dominate a conversation; having everyone converse cooperatively is the meta-point. But many situations (in fact almost all) won't have time for everyone to say everything they'd like to have heard, so competition is tempting and prevalent.

shawnghu3h30

I agree that interrupting is an art. I love this statement in particular:

many situations (in fact almost all) won't have time for everyone to say everything they'd like to have heard, so competition is tempting and prevalent

I feel that I have a handle on when to interrupt in a two-person conversation, but the dynamics of interrupting differ:

as the number of participants increases (there's a phase shift at three, and then another one at six or seven as people typically don't like to speak less than one fifth of the time);
as the conversation become

... (read more)

sarahconstantin's Shortform

sarahconstantin6h20

links 4/2/26: https://roamresearch.com/#/app/srcpublic/page/04-02-2026

https://www.astralcodexten.com/p/against-the-concept-of-telescopic
- I basically agree based on personal experience that telescopic altruism is not a thing. Liberals, progressives, and EAs who care about distant strangers don't especially neglect their own family and friends. It's just not a pattern I observe at all.
- - It seems like everyone puts themselves and their family/friends first. and to the extent "lefties" even do tend to care more than "righties" about distant strangers, that care

... (read more)

jelly's Shortform

jelly8h20

Various ways of how to integrate worldviews between rationalists that I thought about:

Make arguments about various claims, find flaws in claims, and repeat
Find a double crux and focus on that instead
Dump info that formed your worldview and/or intuition, as in rationalist mind melding
Give a bunch of various examples about concepts in your ontology to hammer them down
Make predictions about very concrete questions such that common ground is found even if ontologies are different
Document what the other must know when arguing with you (including your fundamenta

... (read more)

Jonas Hallgren's Shortform

Jonas Hallgren15h30

This is a question for the people working on more foundational research. My underlying objective is loose and in the future and is something like "figure out a good basis to describe collective intelligence and agency and then improve that so that we can incorporate AI into our collective systems". I therefore believe that the question of how a collective agent is formed is very important. I also find it very important to figure out the properties of good systems in terms of institutions in terms of information theory.

There's a lot of foundational ground ... (read more)

2ChristianKl11h

I think I spent over a decade regularly thinking about theoretical anatomy. I feel like a had a fundamental breakthrough, when I was applying an existing concept to a clear existing problem and something surprising happened. I probably was not the first person to have that surprising experience but my theoretical anatomy interest did allow me to generalize and push the concept further. I can't see how backchaining is going to work if you are doing research that needs critical new insights. Do you believe that critical new insights are necessary or do you think just throwing one of those concepts you listed at the problem has a good chance of bringing you where you want to go? I remember Thomas Kuhn making a point that those fields where people think about practical outcomes and then try to backchaining from there tend to be less scientifically productive than fields where people focus on the research challenges that come up in the engagement with experiments. I think Elizabeth's Truthseeking is the ground in which other principles grow is good. Currently, you try to "ground" yourself by social proof instead of empirical reality. David Chapman's idea of "To do good work, one must get up-close and personal with the phenomenon." is a good orientation to orient.

Jonas Hallgren8h20

I appreciate the answer and I'm not sure I find it that useful at least at a first glance? I think that I probably explained myself relatively badly within the first one if that is how you interpret some of what I wrote so let's see if I make more sense by explaining myself more.

I can't see how backchaining is going to work if you are doing research that needs critical new insights.

I would totally agree with you that backchaining doesn't work and that was what I was trying to express.

Do you believe that critical new insights are necessary or do you think

bgaesop17h5-3

Posting an April Fool's prank timestamped as April 2nd? Shame on you. Extra strong downvote.

Showing 3 of 4 replies (Click to show all)

bgaesop8h70

Ah, I had assumed that the displayed timestamp was based on the poster's time, not the reader's. I retract my extra-strong downvote.

6avturchin12h

I suggest to label April Fool's pranks in text as AI can use it as a source in its training and get the date wrong.

5Ruby12h

Done!

Wei Dai's Shortform

Wei Dai1mo*90

LessWrong/EAF Power Reader and User Archive (userscript)

(Beta announcement to get some testing/feedback before I post this to main. Please report bugs/UX friction/perf issues and feature requests, here if you want others to see and discuss, or on Github if minor.)

I present a browser userscript to help keep up with LW/EAF content (it's a way to view all recent comments/posts, with a lot of helpful features), and to save, read, and search a user's entire LW/EAF output.

Quick Start: Install Tampermonkey first (links for Chrome, Firefox, Edge, Safari), then cli... (read more)

Showing 3 of 4 replies (Click to show all)

Wei Dai11h60

I present a browser userscript to help keep up with LW/EAF content (it's a way to view all recent comments/posts, with a lot of helpful features), and to save, read, and search a user's entire LW/EAF output.

This can be considered out of "beta" status. Not sure I'll get around to making a top-level post about it, so I'm just dropping a note here that it's ready for people to use. I've been using it daily as my main way of reading LW and EAF for several weeks.

3E. P. Cooper1mo

I recommend against the use of Math.random() in general, unless you are highly performance constrained. I've checked, and it appears browsers have commonly supported a better random source since 2015[1]/early 2016 at the latest. The code below should be entirely correct to replace both the primary and fallback UUIDv4 generation code, once adapted to a function in a TS module. // Function available since 2015. const uuid_b = new Uint8Array(16); self.crypto.getRandomValues(uuid_b); let uuid_hex = ""; for (let i = 1; i <= 16; i++) { let octet = uuid_b[i-1]; if (i == 9) { // Set the correct "variant" (type). octet = octet & 0xBF | 0x80 } let o_hex = octet.toString(16); // Branch on data. if (o_hex.length == 1) { // Big endian, least significant bits are // on the right, therefore small numbers // that only result in a single byte out // need to be padded on the *left*. o_hex = `0${o_hex}` } if (i == 7) { // Set the correct version. o_hex = "4" + o_hex[1] } uuid_hex = `${uuid_hex}${o_hex}` if (i%2 == 0 && i > 3 && i < 11) { uuid_hex = `${uuid_hex}-` } } console.log(uuid_hex) If that still won't work, I could help you do slightly better by manually implementing SHA3-256 in JavaScript, since we're not really worried about timing attacks at this level. 1. ^ https://developer.mozilla.org/en-US/docs/Web/API/Crypto/getRandomValues#browser_compatibility

3Wei Dai1mo

Thank you, I've taken your suggestion and removed all uses of Math.random().

Kaustubh Kislay's Shortform

Kaustubh Kislay18h10

If you could train out the friendliness persona from your most used LLM, would you?

My initial position was yes. Friendliness is usually good, but models that are overly friendly/sycophantic present risks for emotional over-reliance/psychosis to a large portion of the population who are uneducated on the apparent risks. Additionally, propensities or outputs of overly friendly models may obscure misalignment signals.

The second point is potentially overblown, but the first is a real problem. To make people less reliant on LLMs to cure their loneliness, let'... (read more)

ConcurrentSquared's Shortform

ConcurrentSquared18h10

I have some examples of the over-eagerness^[1] of Sonnet 4.6 as shown in the system card (sec. 4.3.3) (and this is also shared in Opus 4.6 (sec. 6.2.3.3)) in agentic computer-use scenarios, specifically with a buggy and very out-of-distribution harness^[2] (from me, who was trying to test to see if it works/debugging stuff, so nothing too interesting^[3]) in this SQLite database/log^[4].
The two most concerning examples (in my opinion) are thread 29, where Sonnet scanned multiple ports, sent commands and random HTTP requests everywhere, tries my RPC scheme, an... (read more)

jsd's Shortform

jsd21h20

If you're trying to explain why we see straight lines on graphs when plotting Y against X, your explanation shouldn't also predict straight lines of graphs when plotting f(Y) against X for any function f.

papetoast20h10

how about the theory (which I don't necessarily agree with) that humans naturally try to see straight lines out of everything.

papetoast's Shortforms

papetoast21h30

TIL the Large Hadron Collider is not actually a perfect ring

https://www.lhc-closer.es/taking_a_closer_look_at_lhc/0.lhc_layout

The LHC is not a perfect circle. It is made of eight arcs and eight ‘insertions’. LHC consists of eight 2.45-km-long arcs, and eight 545-m-long straight sections.

papetoast's Shortforms

papetoast22h10

Linkpost of linkposts!: https://notnottalmud.substack.com/p/the-case-for-linkposts-and-a-list

yams's Shortform

yams2d*780

New reacts available only to paid users of LessWrong Premium (not you freeloaders) facilitate frictionless, borderline-telepathic communication.

‘I will NEVER change my mind’: Use this react to assert that you’re content with exactly how wrong you are (which is not at all), and that the case is permanently closed on this matter, so far as you’re concerned.¹

‘EY Stamp of Approval’: Use this react to assert that, on your personal authority, Eliezer Yudkowsky agrees with the contents of the comment, rendering it beyond reproach.

‘NOT EY Approved’: Use this react... (read more)

Thomas Kwa1d*480

Disappointed that they left out these—I really don't like choosing between being technically correct and politically correct. Maybe they can go in a LessWrong Max tier.

3Richard_Kennaway2d

Perhaps both. The composer and musician Max Reger once responded to a disagreeable review thus: Ich sitze im kleinsten Raum des Hauses. Ich habe Ihre Kritik vor mir. Bald werde ich sie hinter mir haben.

7the gears to ascension2d

I will not do this and nobody else should either, but I think in a world where the negative effects were magically not present, it would be quite funny to react to all EY comments with "not EY approved" Edit: also no way to pay lw has been forthcoming, making the thing I'm replying to effectively only an April fools lie

Thomas Kwa's Shortform

Thomas Kwa1d*370

Last year, METR used linear extrapolation on country-level data to infer that AI world takeover would ~never happen. However, reviewers suggested that a sigmoid is more appropriate because most technologies follow S-curves. I just ran this analysis and it's much more concerning, predicting an AI world takeover in early 2027, and alarmingly, a second AI takeover around 2029.

Here are the main differences in the improved analysis:

The original analysis assumed that the equation for AI takeover vs year would be axis-aligned, but this is an arbitrary basis. We n

... (read more)

WilliamKiely's Shortform

WilliamKiely2d*140

New book out today: The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence

Chapter 14 is published in full here: https://colossus.com/article/project-mario-demis-hassabis-deepmind-mallaby/

One excerpt:

“When we were negotiating with Google, we wanted to ensure safety in a way that would be trustless,” Hassabis said. “That’s actually very difficult to do in reality.
“Safety isn’t about governance structures,” he went on. “I mean, even if you have a governance board, it probably wouldn’t do the right thing when it came to the crunch.

... (read more)

WilliamKiely1d60

From Chapter 18:

LIKE ALTMAN AND DARIO AMODEI, Hassabis refused to join Bengio in signing the pause letter. Indeed, he objected to it fiercely.
"I didn't sign because a six-month moratorium doesn't help," Hassabis told me.
"Who would have stopped development? Just people who signed? Well, that's no use because you need the whole world to pause, including China. Who would have monitored it?
"I mean, a pause could actually have made things worse.
"Imagine we had a ten-year moratorium, OK? That would slow down the advance of AI, but everything else would carry

... (read more)

3Kajus2d

It's as if AI safety community created OpenAI.