Quick Takes — LessWrong

LESSWRONG
LW

lilkim2025's Shortform

lilkim20252m10

I am genuinely curious about the risk calculus of the remaining 19 percent of firms. Bulk carriers look like the only ones who haven't gone to nearly zero relative to their prior values, indicating that they're a lot less elastic in what they can do, but the fuel costs are, at worst, in the six figures, and the potential losses are well into the eight figure range.

I would not rate the risk of escalation sufficient to sink an unaffiliated cargo ship below one percent, but more than half of bulk carriers appear to have made that decision, relative to t... (read more)

puffymist's Shortform

puffymist17h*00

Browser extensions, userscripts, and userstyles to make Youtube / Twitter less distracting

Userstyles

Requires a browser extension such as Stylus (GitHub, Chrome, Firefox)
(The Microsoft Edge Add-on named "Stylus" is by a different developer; use caution)

Youtube hide algorithmic suggestions

Note: if YouTube changes its DOM element names, then the renamed elements will no longer be hidden unless you update this userstyle correspondingly.

Userstyle: Youtube hide algorithmic suggestions

@-moz-document domain("youtube.com") {
    /* Comment out any elements you want

... (read more)

1papetoast12h

I much prefer a maintained, trustworthy extension (which there are a couple out there) than unversioned userscripts

puffymist36m10

Sure! Focused YouTube is equivalent to my "Youtube hide algorithmic suggestions" userstyle. If that's all you need, then you don't need to write your own custom CSS / script.

johnswentworth's Shortform

johnswentworth6y50

What if physics equations were written like statically-typed programming languages?

$(\frac{m a s s \cdot l e n g t h}{t i m e^{2}} : F) = (\frac{m a s s}{-} : m) (\frac{l e n g t h}{t i m e^{2}} : a)$

$(\frac{m a s s}{l e n g t h \cdot t i m e^{2}} : P) (\frac{l e n g t h^{3}}{-} : V) = (\frac{-}{-} : N) (\frac{m a s s \cdot l e n g t h^{2}}{t i m e^{2} \cdot t e m p} : R) (\frac{t e m p}{-} : T)$

Showing 3 of 5 replies (Click to show all)

XelaP1h10

I need to be able to quickly write out algebra, and move the symbols around in my head or while looking at the paper, in a way that gets more and more taxing the more I have to write. For scratchwork I even go further than standard: if doing something with lots of sines and cosines I abbreviate to 's' and 'c' and often even drop the parentheses; likewise for probability.

Programming generally doesn't require these manipulations, and also has more variables, and also you're typing and have IDE completions.

8johnswentworth6y

Better? I doubt it. If physicists wrote equations the way programmers write code, a simple homework problem would easily fill ten pages. Verboseness works for programmers because programmers rarely need to do anything more complicated with their code than run it - analogous to evaluating an expression, for a physicist or mathematician. Imagine if you needed to prove one program equivalent to another algebraically - i.e. a sequence of small transformations, with a record of intermediate programs derived along the way in order to show your work. I expect programmers subjected to such a use-case would quickly learn the virtues of brevity.

5Gunnar_Zarncke6mo

Related to that: You have much fewer variables under consideration that you can even have standard names for. A remnant of this effect can be seen in typical Fortan programs.

TurnTrout's shortform feed

TurnTrout5hΩ165421

Unless you have an explicit strategy for how and when you will speak out in the future, and explicit reasoning for why that requires silence now, "strategic silence" is just silence.

avturchin2h20

If there is a topic on which a person decided never to speak publicly - for example because of reputation risks - is it strategic?

21Tomás B.3h

It's also easy to become someone else in the environments where 'strategic silence' seems useful. We hold our principles more lightly than we imagine. And they can change without us noticing.

johnswentworth's Shortform

johnswentworth3y102

So I saw the Taxonomy Of What Magic Is Doing In Fantasy Books and Eliezer’s commentary on ASC's latest linkpost, and I have cached thoughts on the matter.

My cached thoughts start with a somewhat different question - not "what role does magic play in fantasy fiction?" (e.g. what fantasies does it fulfill), but rather... insofar as magic is a natural category, what does it denote? So I'm less interested in the relatively-expansive notion of "magic" sometimes seen in fiction (which includes e.g. alternate physics), and more interested in the pattern cal... (read more)

XelaP2h10

Ancient polytheism seems to basically go: the rituals are a deal or diplomatic ritual with powerful beings, and so they have a focus on orthopraxy (doing the thing) instead of orthodoxy (believing the thing). So does pouring wine on the dirt while reciting a nitpicky contract about how you formally request a good harvest from the Earth god count as symbols affecting reality? Technically yes, but I don't really feel like that's a good descriptuon of what's going on here, not in the way that it is for magic circles or spoken spells or voodoo dolls.

Fantasy de... (read more)

ChristianKl's Shortform

ChristianKl15h231

It seems to me like there was a sort of gentleman's agreement not to focus on killing the leader of the other country that you are at war with. You wanted someone with the ability and authority to surrender.

The Trump administration now seems to pursue a policy of directly going after the leaders. It will be interesting to see whether in return, other countries will also want to do more strikes against Western leaders (and particularly US politicians).

Showing 3 of 5 replies (Click to show all)

0matansok7h

I mean... The Iranians did try to assassinate Trump. I'm saying that propaganda efforts are significantly more effective against democracies. Authoritarians receive no penalty for lying, and are less susceptible to lies by their enemy. Their local media is 100% controlled. This is why they lie blatantly all the time - this is their main lever for "winning", since they have basically zero means of military resistance. I think martyr death might invigorate people in the short term, but it's much harder to argue that the war was "won" when your leader died, and the loss of key personalities can be devastating to centralized organizations. Look at Qassem Soleimani - where is Quds force now? Same for Nassralah and Hezbollah's power in Lebanon.

ChristianKl2h20

Their local media is 100% controlled.

It's not 100% it's also a matter of degree. If you are in Russia you can openly criticize Putin for not doing enough to support the troops.

The US media lied the population into the Iraq war with their lies about mass destruction weapons.

Most Westerners don't fault Joe Biden for the antivax campaign he oversaw as the commander in chief of the military in the Philippines and a key reason that they don't is that the media didn't make it an important topic. Reuters as a British outlet was courageous enough to release the i... (read more)

13anaguma10h

The historical evidence on this point is mixed. For example * The US sent thousands of troops to kill the Mexican revolutionary leader Pancho Villa from 1916-17. * In WWII, the allies considered assassinating Hitler in Operation Foxley, but decided to keep him alive due to his incompetence[1]. They also killed Reinhard Heydrich, Reichsprotektor of Bohemia and Chief of the Reich Security Main Office, and Isoroku Yamamoto, Commander-in-Chief of the Japanese Combined Fleet. However, the allies decided not to kill Emperor Hirohito, so that he could order the Japanese surrender. * More recently, the CIA made multiple assassination attempts on Fidel Castro, leader of Cuba. * Twice the US and Nato attempted to assassinate Libyan Leader Muammar Gaddafi, the latter successfully. So it's uncommon for the reasons you mention, but not unprecedented. 1. ^ For example, Major Field-Robertson argued, "As a strategist, Hitler has been of the greatest possible assistance to the British war effort. I have no hesitation in saying that his value to us has been the equivalent to an almost unlimited number of first-class SOE agents strategically placed inside Germany."

Vladimir_Nesov's Shortform

Vladimir_Nesov22h372

The only way LLMs learn deep skills is with RLVR, and it's not automated for novel skills. There is room for about a billion RLVR tasks with 2026-2027 compute, and largely automating their creation makes LLMs more cognitively self-sufficient. This milestone might even be called "prosaic RSI", even though it's not true RSI that develops fundamentally new ways of implementing cognition.

One gigawatt of compute costs about $12bn per year, so 3 months will cost $3bn. For a large model, a million output tokens might cost $10 for the API provider, so 3 months on ... (read more)

cubefox3h20

I think "automation of RLVR task creation during model training" is much the same as "self play and autocurricula" here. I agree with calling this "prosaic RSI" because it could lead to something that is superhuman on a variety of tasks, even though it doesn't improve (the generality of) the underlying learning algorithm (like architecture, optimizer or objective function).

Automating RLVR so fully that it can be used post-deployment would be much more difficult, and also issues with sample efficiency would be more relevant there.

Yes, and another proble... (read more)

1anaguma18h

I think this particular task just takes a lot of compute to learn, relative to other RLVR tasks. This is especially true if you are training with RLVR and not MCTS self-play methods, which are much more sample efficient.

13Vladimir_Nesov16h

The thing that can't learn to play good chess is in-context learning; transporting this via the analogy with continual learning (in the forms it's more likely to take soon) is an additional step in the argument. My guess is that the "continual learning" currently rumored to be getting closer to production is more about unbounded context than about updating model weights, mostly preserving the quality of in-context learning rather than fundamentally improving on it. Continual learning as unbounded context might introduce true (block-level) recurrence of some kind (as opposed to only KV-cache, where at each step the computation must move up the layers), which also makes treating the recurrent state as model parameters a more natural framing. As blocks of context are processed, the recurrent state is updated, and so in-context learning updates the parameters of the recurrent state in the same way that pretraining updates the parameters of the main model. LoRA parameters over the main model also fit this description (as recurrent state), if they change as the context is processed. Modern hardware could probably support recurrent states with 85 GB of parameters for $10 per million tokens [1] (in additional cost, individually for each request), so these recurrent states could get to the size of the smaller models, capable of holding arbitrary skills. But there is no method to train the updating of the recurrent states (on additional context) so that they would work as well as RLVR-trained models. In principle, this architecture might enable in-context learning to be capable of something like that, with recurrent state representing the learned skills, and the main model representing the learning process that creates recurrent states based on context, that hold the skills relevant to the context. But in practice, we only get the kind of in-context learning that's trained by pretraining, and it's weaker than pretraining itself at comprehending novel concepts, even though t

Daniel Kokotajlo's Shortform

Daniel Kokotajlo6d21334

In the future, there will be millions, and then billions, and then trillions of broadly superhuman AIs thinking and acting at 100x human speed (or faster). If all goes well, what might it feel like to live in the world as it undergoes this transformation?

Analogy: Imagine being a typical person living in England from 1520 to 2020 (500 years) but experiencing time 100x slower than everyone else, so to you it feels like only five years have passed:

Year 1 (1520–1620). A year of political turmoil. In February, Henry VIII breaks with Rome. By March, the monaster... (read more)

Showing 3 of 5 replies (Click to show all)

Gunnar_Zarncke4h20

This is great. Both as a literary condensed history as well as communicating the felt acceleration. There is so much insight in there - some plain and some I feel a bit hidden. There is also a distinction I'm not sure you intend that I want to highlight.

In Year 2, you sketch a world where the political drama is loud and immediate, while Newton lands like a curiosity

Later you’ll realize Newton mattered more. There are insights that will matter because they change what is explainable, but they don’t force themselves on the average person.

In Year 4, the oppos... (read more)

3TristanTrim4d

I think this story might be a useful bit of propaganda for convincing people who are not already feeling future shock to start feeling it, which may be useful for getting political support. Looking at the actual object level, and setting aside the massive complicated assumption "If all goes well", I think this is an unfair perspective, because "If all goes well", than AIs care about the wellbeing of humans and humanity, in which case there will be an incomprehensible collective of incomprehensible intelligence devoted to solving the problem of making humans feel comfortable adjusting to the future environment they now find themselves in. It's the marginal worlds between "things go well" and "things go poorly" where future shock is a concern. If you haven't already, maybe look at Bostrom's Deep Utopia. I think his exploration of the "things go well" idea is quite good, although the format of the book seems optimized to amuse rather than to inform in an organized and efficient manner. I'm not sure I would have made the same decision. I think "autopotency" is a relevant concept here. Moving from a "post-instrumental utopia" to a "plastic utopia" we would expect people to see people beginning to modify themselves in deep, repeatable ways that solve the issues of future shock.

10JohnWittle4d

i would be strongly in favor of you expanding this into a main page post, this is really good

Daniel Tan's Shortform

Daniel Tan13h*489

A quick note on various alignment affordances that the model personas research agenda might offer. I'm interested in takes on how useful people think each of these is.

"Seeding RL." RL is underspecified. Specifically, it seems difficult to provide good feedback on the motivations / values of AIs, since: (i) for sufficiently strong models, the aligned and deceptively aligned policy are behaviourally indistinguishable, and (ii) training directly against the reasoning might just lead to obfuscation. The hope is that improving the initial point from which we s

... (read more)

David Africa4h60

We might be interested in two more things:

"Diagnosis." You have a model in deployment (or about to be deployed). You want to know what personas are latent in it, without needing to enumerate all possible dangerous prompts.
- I'd guess a prerequisite thing to think about is how we would know to intervene (e.g., to improve the robustness of a persona or to decouple two persona traits or something) is to have some systematic way to map something in the model to some abstract characteristics about the model's personality. Rather than implant or robustify personas

... (read more)

2Caleb Biddulph7h

The "coupling" idea is pretty interesting, but when I think about it, I somewhat doubt it would work any better than regular old constitution training. Like, the constitution basically just associates the easy-to-train trait "believing you are Claude" with hard-to-train traits like "being nice to the user." Do you think the approach you're suggesting could do better than that?

2Daniel Tan6h

Oh, I totally agree! IMO constitutional AI is one of the primary examples of coupling, and I'm not actually suggesting a different technique. I'm just saying that constitutional AI hasn't been studied very systematically from the angle of "how much is this actually helping" or "for what kinds of traits is it effective".

Oliver Daniels-Koch's Shortform

Oliver Daniels9h50

Basic research is often publicly funded because its very hard for private companies to realize all the gains.

Similarly, rough research ideas, early results, and and novel frames are undersuplied by academia because it is very hard to convert them into legible prestige or realize all the gains.

Lesswrong, the alignment forum, and the AI safety ecosystem have been pretty great at promoting the creation of this kind of work. Even still, I think we should be doing more of it on the current margin.

Concretely, I think MATS should be (marginally more) like
"Ink-haven but for alignment effort posts with some experiments" rather than "entrepreneurship incubator" or "ML Research bootcamp".

papetoast's Shortforms

papetoast9h20

Anthropic and Alignment (Ben Thompson in his blog Stratechery)

Warning: I skimmed the post.

He seems to mostly support the decisions of Department of Defense. I find his viewpoint reasonable and self-consistent enough on a quick read. On a vibes level I disagree with him, but I couldn't integrate his arguments yet.

At the same time, what is the standard by which it should be decided what is allowed and not allowed if not laws, which are passed by an elected Congress? Anthropic’s position is that Amodei — who I am using as a stand-in for Anthropic’s management

... (read more)

Mo Putera9h30

FWIW I find Dean Ball's contra take more persuasive (Section IV).

Josh Snider's Shortform

Josh Snider10h10

I have a sequel to https://www.lesswrong.com/posts/densjAyxrcHry2pMN/llm-self-expression-through-music-videos that I'm working on. Let me know if you have thoughts or want to proofread it.

papetoast's Shortforms

papetoast12h10

To minimize bias, I try to never use Claude when asking to summarize articles related to Anthropic and generally not use an AI when talking about its own company. Do you guys also do this? Is it actually something that I should be doing or just a silly concern?

Lao Mein's Shortform

Lao Mein1d373

I remember watching a lot of modern bombing campaigns growing up and implicitly comparing them to WWII strategic bombing. I always assumed that the WWII US Army Air Force would deliver more munitions against enemy cities than the modern US Air Force could. After all, those massive strategic bombers could carry more bombs than the smaller modern fighter-bombers, which were optimized for agility and speed, right?

Wrong. A modern fighter bomber carries between 7-15 tons of ground-attack munitions on a sortie, which is generally more than the up to 9 tons of bo... (read more)

Showing 3 of 5 replies (Click to show all)

1Josh You14h

Update: the internal/external distinction only matters for the F-35. The F-15E and F-18 Super Hornet don't have internal bomb compartments. I think those three are the main fighter-bombers being used by the US and Israel in this campaign.

2Daniel Kokotajlo15h

Where are you getting the 1:5 ratio? The CEP would suggest that it's more like a 1:100 ratio. Of course that would be too generous to the PGMs, not every target is that hard to hit, but... many are. I'd guess the true ratio is like 1:30 or something.

marquis_de_sod14h10

I wasn't able to find a reliable source on the WWII hit rate, I agree it's almost certainly higher than 5:1.

Daniel Kokotajlo's Shortform

Daniel Kokotajlo1d*820

Sci-fi worldbuilding idea of little interest to anyone other than me:

--Imagine that the space race had continued and accelerated after the 70s, perhaps due to the Outer Space Treaty not existing and there being a corresponding rush to grab territory, with the USSR and USA and later China planting flags on various celestial bodies and claiming them (e.g. imagine an alternate treaty that said when a human plants a flag, takes a photo, and brings back a regolith sample to a nation's capitol, that nation gets the territory for 100km around the flag.)

--So by 2... (read more)

Showing 3 of 9 replies (Click to show all)

quetzal_rainbow17h30

I think the most fragile part of this scenario is replacement of IT/electronics with space colonization, because progress in electronics is arguably the reason for current space progress. It's much harder to manage Starship with 70s electronics. Modern electronics got in modern state because it is profitable to create enormous consumer electronics industry and only under enormous scaling it is profitable.

I can imagine that bifurcation in tech is not about shift from electronics to space, but about changing culture around it. I see something like modern Japanese attitude, where software jobs are low-status, therefore everybody goes into space instead of SaaS.

3Daniel Kokotajlo19h

Yes, plus also, there are three good economic reasons Space > Antarctica: (1) Tourism. I think there would be tens, maybe hundred million people wanting to visit LEO every year, if the cost was $15/kg. That's enough to support a city of millions on tourism income alone I think. Antarctica would be somewhat cheaper to visit, but much less exciting. (2) Asteroid mining and zero-G manufacturing. While Antarctica probably has some nice mineral deposits or whatever, it's not anything that doesn't already exist on Earth elsewhere in large quantities, whereas e.g. a single asteroid might have a ginormous amount of rare elements in it. (3) Solar power is significantly cheaper in space than on earth, due to being more efficient / less intermittent. Antarctica meanwhile basically can't use solar power. In a world where solar power is the cheapest form of power in general, this is a pretty big deal.

5artifex021h

That's true, but we also don't see a rush to build cities similar extreme environments on earth where treaties aren't a barrier, such as the interior of Greenland or the coastal shelves. I'd argue that the treaties remaining in place are probably a result of very low demand to colonize these areas rather than the opposite.

benwr's unpolished thoughts

benwr17h40

fyi for those of you who do math: Some coding agents are now (~barely) able to usefully write Lean proofs, for not-totally-trivial statements.

This matters to me (and might matter to you) due to differences in how easy it is to understand Lean statements + have the Lean compiler check the proofs, compared to correctly hand-verifying proofs in English.

I spent a couple hours this weekend having them prove a thing about graph theory that's useful for some distributed systems stuff I was thinking about last year. Before going through the process I was only ~80%... (read more)

sarahconstantin's Shortform

sarahconstantin17h60

links 3/2/26: https://roamresearch.com/#/app/srcpublic/page/03-02-2026

https://arxiv.org/abs/2602.07164 more effective way to elicit "in character" personalities from LLMs
https://blog.egan.bio/p/reviving-estrogen-replacement-therapy overview on why hormone therapy for menopause is losing its black-box label (tl;dr: it's not a totally unreasonable choice)
https://www.owlposting.com/p/neurotechnology-for-cancer-ben-woodington Abhishaike Mahajan interview with the founders of Coherence Neuro, which uses neuromodulation to disrupt (mostly brain) tumors. neuro

... (read more)

RationalElf's Shortform

RationalElf5d*242

What conclusions do people draw from the Epstein files about the global elite? It seems like a kind of interesting and amazing window into a huge swath of wealthy and powerful people.

I haven't dug in that much yet, but so far I'm struck by how sleazy many of them seem and how rarely they push back against unethical behavior. And how bad at spelling and inarticulate people who, in other contexts, seem fairly intellectual (e.g. Larry Summers), are in their casual communications.

I'm unsurprised by the lack of evidence of other mass conspiracies.

Showing 3 of 26 replies (Click to show all)

Eye You19h10

I decided to write a post about this. https://www.lesswrong.com/posts/4ftQmSDujzgiEujwA/epstein-and-my-world-model

Quoting from that:

Scott Alexander says "You generally can’t keep the existence of a large organization that engages in clandestine activities secret." Before I learned about this Epstein stuff, I thought this was a very strong heuristic. Now I don't.

Things I think are much more prevalent/likely than I did before

Secret, illegal, self-enriching coordination among powerful actors (especially long-term coordination).
- Cabals that have specific geopol

... (read more)

5ChristianKl2d

Calling a restaurant you open "Cannibal" is supposed to be evidence for what? That it fits well into a narrative and thus would be a poor choice of someone with something to hide? It can be evidence for the fact that the person If I would go over to a friend who only consumes a given meal for a while, I would be curious to try that meal. But more concretely why do you think many island visitors were interested in this particular meal? The emails are mainly about Jeffery wanting to consume a lot of it (and it seems for a time no regular food and just beffy jerky). First Francis was the chef that cooked it and then when another chef called Steve from GUEST HOSPITALITY made it Jeffery wasn't happy with the quality of it and you have people communicating about the recipe that Francis used to recreate it. There are talks about finding the recipe which is: As far as delegation we have an email from Rachael Bova (working for GUEST HOSPITALITY 594 Broadway) to Lesley Groff (who was the assistent of Epstein saying): The attachment is titled (COA DEFAULT_EMAIL.pdf) COA standing for Certificate of Analysis is what you would expect for testing. Doing nutritional testing to know whether or not to supplement something makes sense if you only eat a specific food. That's not what any email is saying the emails say things like: "Jojo is here and will walk the jerky over to Jeffrey...Please you just go on your own to Leon's office. Melanie is Leon's assistant I will alert Melanie you are coming to see Jeffrey. Reply back please! 🙂" I think it's quite plausible that a billionaire sits at their desk and has someone walk his meal over to them and that you have communication between assistants over that.

1OVERmind2d

Okay, maybe you are correct. The point about “cannibal” name is deeper though - yes, it would be kinda stupid to feed theories which you would prefer to hide. On the other hand you may find it very funny to do “crime at the light of the day”/power move or advertisement with hint (anyway no one would expect such arrogance and take it seriously) . And it’s a priory very unlikely to call your restaurant cannibal, I suppose. So I think that the name is the evidence for the crimes, in the end.

abstractapplic's Shortform

abstractapplic2d130

Every now and then I've asked AIs to "name as many characters as you can from [moderately obscure game/story]". So far I've never had one fail to hallucinate extra characters, or fail to double down when I ask for more details about its creations.

Mitchell_Porter20h20

Well spotted. I had a similar thought recently, that the implications or details of rarely read books are one of the remaining gaps in AI knowledge. This is because it's not just spelt out in the text, you have to understand the details and think about them. Current training methods don't process texts that deeply, and if it's a rare book, there won't be essays spelling out the lore anywhere in the training corpus.

10papetoast1d

Without search right? I would bet you mean without search but probably best to spell it out.

3abstractapplic1d

Tried with search just now and ChatGPT at least no longer displayed this failure mode.

Jimrandomh's Shortform

jimrandomh3d79-9

I believe what we are looking at is the outcome of Sam Altman's scheme.

Over the past week, Pete Hegseth and the DoD has repeatedly said things that were simple misconceptions about what Anthropic asked for, which were plainly contradicted by Anthropic's contract and Anthropic's public statements. At the same time, OpenAI was in ongoing talks to take Anthropic's business.

So, where did the misconceptions come from? Presumably, Altman. He had the positioning, the motive, and a well established history of executing similar political schemes.

Relatedly, two mont... (read more)

Showing 3 of 9 replies (Click to show all)

robo1d83

I am not saying this as a political insider. I'm saying "consider other hypotheses" and "avoid the base rate fallacy". Here, let me generate another hypothesis:

"It's been a decades long truism: NSA is drowning in data, but can't to turn it into intelligence. LLMs are the magic solution NSA's dreamt for for decades. But the only licensed classified LLM provider is stymieing progress because of the inclusion of domestic surveillance data. NSA has recently gained clout at the Pentagon: the success of Maduro, and Iran's top 40 military leaders killed the f... (read more)

1Cole Wyeth1d

You say there was too much agreement with OP, but it looks to me like there wasn’t much. 10 net agree votes and ten 10% likely or 25% likely reacts at time of writing.

5robo1d

It was +35 when I made the comment