As an alternative to Inkhaven Residency, for the people who can't take a 30 day break, I propose an alternative. I am writing about it here, in case someone else also wants to do something, so that we could coordinate. Please respond here if you plan something, otherwise I will announce my plan officially in a LW post on Sept 30.

My current vision, which I provisionally call Halfhaven, is to make an online blog-writing group, probably on Discord, that will try to produce the same total output in twice as much time. That is 30 articles posted during October and November 2025, so on average one article in two days. Feels slightly more doable for the people who have a job/school.

The rules, still somewhat under construction:

• in addition to blog articles, other media such as videos and podcasts are also accepted (this is primarily about productivity, not about written text), but e.g. tweets are not
• languages other than English are okay (but all translations of the same content are counted as the same thing, so you can't get 30 points by writing one article and translating it to 30 languages)
• you can publish at multiple places, e.g. 5 articles on Less Wrong, 20 articles on your Substack, and

I am reading about Zizians on Hacker News, and I wonder what is the lesson we are supposed to take from this. (1, 2)

People sometimes make mistakes. If they survive them, they are supposed to learn. I believe that the rationality community as a whole makes many mistakes. But I am confused about what is the specific lesson about Zizians.

From the PR perspective, they are a disaster. Literal murderers, interesting enough to make news. Described by journalists as if they are a part of the rationality community, and as if their actions somehow reflect on us. Do they? Or are the journalists just making up stuff? Maybe; but that just sounds too convenient. A story can be mostly false, and still contain a small piece of truth. I don't want to miss that piece, if there is any. But I don't see it.

A long version of the history (without the recent events) is at zizians.info. Ziz participated at some CFAR workshops. Was told that she was "net negative". She started recruiting followers. They made a protest; the rationalists called cops on them. Later, they have murdered a few people.

Is there an avoidable mistake, on the side of the rationality community? If ten years ago we knew the future (but ... (read more)

I would like to see a page like TalkOrigins, but about IQ. So that any time someone confused but generally trying to argue in good faith posts something like "but wasn't the idea of intelligence disproved scientifically?" or "intelligence is a real thing, but IQ is not" or "IQ is just an ability to solve IQ tests" or "but Taleb's article/tweet has completely demolished the IQ pseudoscience" or one of the many other versions... I could just post this link. Because I am tired of trying to explain, and the memes are going to stay here for a foreseeable future.

Perhaps the mental health diagnoses should be given in percentiles.

Some people complain that the definitions keep expanding, so that these days too many kids are diagnosed with ADHD or autism. The underlying reason is that these things seem to be on a scale, so it is arbitrary where you draw the line, and I guess people keep looking at those slightly below the line and noticing that they are not too different from those slightly above the line, and then they insist on moving the line.

But the same thing does not happen with IQ, despite the great pressure against politically incorrect results, despite the grade inflation at schools. That is because IQ is ultimately measured in percentiles. No matter how much pressure there is to say that everyone is above the average, the math only allows 50% of people to be smarter than the average, only 2% to be smarter than 98%, etc.

Perhaps we should do the same with ADHD and autism, too. Provide the diagnosis in form of: "You are more hyperactive than 85% of the population", controlled for age, maybe also for sex if the differences are significant. So you would e.g. know that yes, your child is more hyperactive than average, but not like super ex... (read more)

I am fascinated by how often I read something about LLMs and it seems to illustrate something about human psychology. I wonder how many psychologists think about these things. (I suspect not many, because psychologists typically don't read technical articles about LLMs.)

For example, in "GDM: Consistency Training Helps Limit Sycophancy and Jailbreaks in Gemini 2.5 Flash" the part "Bias-augmented Consistency Training", specifically "Train the model via SFT to give the clean response ... when shown the wrapped prompt"... that reminds me strongly of "Asch’s Conformity Experiment", "On Expressing Your Concerns". Specifically that it becomes much easier to resist pressure when you have seen an example of resisting the pressure.

If you dismiss ideas coming from outside academia as non-scientific, you have a point. Those ideas were not properly tested, peer-reviewed, etc.

But if you dismiss those ideas as not worth scientists' attention, you are making a much stronger statement. You are effectively making a positive statement that the probability of those ideas being correct is smaller than 5%. You may be right, or you may be wrong, but it would be nice to provide some hint about why you think so. Are you just dissing the author; or do we have an actual historical experience that among ideas coming from a certain reference group, less than 1 in 20 turns out to be correct?

Why 5%? Let's do the math. Suppose that we have a set of hypotheses out of which about 5% are true. We test them, using a p=0.05 threshold for publishing. That means, out of 10000 hypotheses, about 500 are true, and let's assume that all of them get published; and about 9500 are false, and about 475 of them get published. This would result in approximately 50% failure in replication... which seems to be business as usual in certain academic journals?

So if it is perfectly okay for scientists to explore ideas that have about 5% chance of being... (read more)

Why do AIs write using the "AI style"? (Overuse of words like "delve" or constructions like "it is not X, it is Y".) I mean, they are trained on human texts, people call them "stochastic parrots", so how is it possible for them to develop a distinct style? Isn't that like the only thing they - practically by definition - shouldn't do? I would expect them to write text that superficially seems like written by humans, but if you look closely, you see that the style is there but the substance is not. Instead, the style is wrong.

There are many things that I don't understand about LLMs (practically all of them), but this one bothers me because I would expect the exact opposite.

(Is it possible that the "AI style" is actually a style of some category of humans that produce a lot of texts that for some mysterious reason most people never see, so those texts are over-represented in the learning data? Journalists maybe?)

Moving a comment away from the article it was written under, because frankly it is mostly irrelevant, but I put too much work into it to just delete it.

But occasionally I hear: who are you to give life advice, your own life is so perfect! This sounds strange at first. If you think I’ve got life figured out, wouldn’t you want my advice?

How much your life is determined by your actions, and how much by forces beyond your control, that is an empirical question. You seem to believe it's mostly your actions. I am not trying to disagree here (I honestly don't know), just saying that people may legitimately have either model, or a mix thereof.

If your model is "your life is mostly determined by your actions", then of course it makes sense to take advice from people who seem to have it best, because those are the ones who probably made the best choices, and can teach you how to make them, too.

If your model is "your life is mostly determined by forces beyond your control", then the people who have it best are simply the lottery winners. They can teach you that you should buy a ticket (which you already know has 99+% probability of not winn... (read more)

I propose a term "Pascal's abuser" (analogical to Pascal's mugger) to describe something that seems to happen repeatedly in the rationalist community, to make it easier to notice the same pattern in the future.

Pascal's abuser is a person or an organization who go like: "Yeah, what we are doing here hurts people, but it is okay because there is a tiny chance it might contribute to saving the world."

Examples: Black Lotus, Leverage Research... and maybe also MAPLE. But more importantly, other such people and organizations that I expect to appear in the rationality community in future, so I want us to recognize the pattern faster.

I was thinking about which possible parts of economy are effectively destroyed in our society by having an income tax (as an analogy to Paul Graham's article saying that wealth tax would effectively destroy startups; previous shortform). And I think I have an answer; but I would like an economist to verify it.

Where I live, the marginal income tax is about 50%. Well, only a part of it is literally called "tax", the other parts are called health insurance and social insurance... which in my opinion is misleading, because it's not like the extra coin of income increases your health or unemployment risk proportionally; it should be called health tax and social tax instead... anyway, 50% is the "fraction of your extra coin the state will automatically take away from you" which is what matters for your economical decisions about making that extra coin.

In theory, by the law of comparative advantage, whenever you are better at something than your neighbor, you should be able to arrange a trade profitable for both sides. (Ignoring the transaction costs.) But if your marginal income is taxed at 50%, such trade would be profitable only if you are more than 2×... (read more)

Thinking about relation between enlightenment and (cessation of) signaling.

I know that enlightenment is supposed to be about cessation of all kinds of cravings and attachments, but if we assume that signaling is a huge force in human thinking, then cessation of signaling is a huge part of enlightenment.

Some random thoughts in that direction:

The paradoxical role of motivation in enlightenment -- enlightenment is awesome, but a desire to be awesome is the opposite of enlightenment.

Abusiveness of the Zen masters towards their students: typically, the master tries to explain the nature of enlightenment using an unhelpful metaphor (I suppose, because most masters suck at explaining). Immediately, a student does something obviously meant to impress the master. The master goes berserk. Sometimes, as a consequence, the student achieves enlightenment. -- My interpretation is that realizing (System 1) that the master is an abusive asshole who actually sucks at teaching, removes the desire to impress him; and because in this social setting the master was perceived as the only person worth impressing, this removes (at least temporarily) the desire to impress people in general.

A few koans are o... (read more)

The incentives in scientific publishing are even more perverse than I assumed.

Recently I talked to someone who is an editor in a small scientific journal. They said that the work feels meaningful to them, because they get a few good scientific articles that probably couldn't get published elsewhere (they are good, but not so important in the large picture). Then they complained about also getting lots of articles that suck (such as: fail to consider obvious alternative explanations, or the conclusions in the text contradict the attached data), mostly from various African and Chinese wannabe scientists.

"So you publish the good articles, and reject the bad ones, right?" I asked.

"I wish I could", they said. The problem is that although the journal currently has quite low impact factor, rejecting the bad articles would lower it. How is that possible? The scientific journals are expect to publish a certain minimum number of articles per year, and they are penalized if they fail to meet the quota.

And this is why the editor sometimes accepts articles that obviously suck, because rejecting them would reduce the rating of the good articles published in the journal.

In Slovakia, we have a tradition of correspondence math competitions where the math problems are embedded in (extremely simple) stories. Do similar things exist in English-speaking countries? If yes, please give me some links.

Here is an example https://pikomat.sk/s/archiv but you would have to translate the PDF documents, not sure how difficult that is.

My 9 years old daughter read the first book of Harry Potter and now she is writing her first fanfic, a Harry Potter / Paw Patrol crossover.

Harry Poodle and the Philosopher's Stone.

Chapter 1: The Pup who lived

So far she has only written the first page; I wonder how far she gets.

When autism was low-status, all you could read was how autism is having a "male brain" and how most autists were males. The dominant paradigm was how autists lack the theory of mind... which nicely matched the stereotype of insensitive and inattentive men.

Now that Twitter culture made autism cool, suddenly there are lots of articles and videos about "overlooked autistic traits in women" (which to me often seem quite the same as the usual autistic traits in men). And the dominant paradigm is how autistic people are actually too sensitive and easily overwhelmed... which nicely matches the stereotype of sensitive women.

For example: difficulty in romantic relationships, difficulty understanding things because you interpret other people's speech literally, anxiety from pretending to be something you are not, suppressing your feelings to make other people comfortable, changing your language and body language to mirror others, being labeled "sensitive" or "gifted", feeling depleted after social events, stimming, being more comfortable in writing than in person, sometimes taking a leadership role because it is easier than being a member of the herd, good at gaslighting yourself, rich inner... (read more)

After reading the Progress & Poverty review at ACX, it seems to me that land is the original Bitcoin. Find a city that has a future, buy some land, and HODL.

If you can rent the land (the land itself, not the structures that stand on it), you even have a passive income that automatically increases over time... forever. This makes it even better than Bitcoin.

So, the obvious question is why so many people are angry about the Bitcoin, but so few (only the Georgists, it seems) are angry about the land.

EDIT: A possible explanation is that land is ancient and associated with high status, Bitcoin is new and low-status. Therefore problems associated with Bitcoin can be criticized openly, while problems associated with land are treated as inevitable.

Attempts to jailbreak LLMs seem obvious to humans. What does it mean?

Maybe it is a selection bias -- a non-obvious jailbreak would simply seem to me like "someone made a valid argument", so I wouldn't classify it as a jailbreak attempt.

Is there enough similarity so that we could create an input-checker AI which would only read the input for the purpose of determining whether it is a jailbreak attempt or not... and only if the input is considered okay, it would be passed to the second AI that actually tries to respond to it?

(That is, does the fact that the AI tries to respond to the input make it more vulnerable? As an analogy, imagine a human who is supposed to (a) write a reply to a comment, or (b) verify that the comment is written in English. The comment happens to contain a triggering description, or a mindkilling argument, or some other bad meme. I would expect the person who only verifies the English language to be less impacted, because they interact with the content of the meme less.)

Assuming that it is possible to jailbreak every AI, including the input checker, are there universal jailbreaks that apply to all AIs, or do you need to make a tailored jailbreak for each? Coul... (read more)

Are LLMs especially vulnerable to bucket errors?

Bucket errors are when you use one word for multiple concepts without realizing that; that is, you keep multiple concepts in the same "mental bucket" and whatever you learn about one of them, you automatically apply to the others.

As a human, if you notice the difference, you can split the bucket, and maybe you start using different words for the difference concepts, or at least add some qualifiers. And the longer ago you did this, the more opportunity to grow separate the new mental buckets had.

But for a LLM, the words are the territory (I think), and it can be difficult to distinguish between two meanings of X, if you only have a short chat history explaining the difference, but 99% of the material you were trained on does not distinguish it. (Also, you forget the chat history at the beginning of a new chat.)

*

This possibility occurred to me when I was discussing set theory with ChatGPT yesterday, and it kept using "countable sequence" for (1) a sequence of countable length, and (2) a sequence containing only countable values, as if those were synonyms. I tried to make it admit the mistake, but instead it... it's difficult to describe... (read more)

There is a question whether human morality is actually improving over centuries in some meaningful sense, or whether it is just a random walk that feels like improving to us (because we evaluate other people using the metric of "how similar is their morality to ours" which of course gives a 100% score to us and less to anyone else).

I think that an important thing to point out here is that our models of the world improve in general. And although some moral statements are made instinctively, other moral statements are made in form of implications -- "I instinctively feel X. X implies Y. Therefore, Y." -- and those implications can be factually wrong. Importantly, this is not moral realism. (Technically, it is an implied judgment that logically coherent systems of morality are better than logically incoherent ones.)

"The only thing that matters are paperclips" -- I guess we can only agree to disagree.

"2+2=5, therefore the only thing that matters are paperclips" -- nope, you are wrong.

From this perspective, a part of the moral progress can be explained by humans having better models of humans and the world in general. (And when someone says "a difference in values", we should distinguish between "a difference in instincts" and "shitty reasoning".)

When you tell people which food contains given vitamins, also tell them how much of the food would they need to eat in order to get their recommended daily intake of given vitamin from that source.

As an example, instead of "vitamin D can be found in cod liver oil, or eggs" tell people "to get your recommended intake of vitamin D, you should eat every day 1 teaspoon of cod liver oil, or 10 eggs".

The reason is that without providing quantitative information, people may think "well, vitamin X is found in Y, and I eat Y regularly, so I got this covered", while in fact they may be eating only 1/10 or 1/100 of the recommended daily intake. When you mention quantities, it is easier for them to realize that they don't eat e.g. half kilogram of spinach each day on average (therefore, even eating spinach quite regularly doesn't mean you got your iron intake covered).

The quantitative information is typically provided in micrograms or international units, which of course is something that System 1 doesn't understand. To get an actionable answer, you need to make a calculation like "an average egg has 60 grams of yolk... a gram of cooked egg yolk contains 0.7 IU of vitamin D... the recommended ... (read more)

I noticed recently that I almost miss the Culture War debates (on internet in general, nothing specific about Less Wrong). I remember that in the past they seemed to be everywhere. But in recent months, somehow...

I don't use Twitter. I don't really understand the user interface, and I have no intention to learn it, because it is like the most toxic website ever.

Therefore most Culture War content in English came to me in the past via Reddit. But they keep making the user interface worse and worse, so a site that was almost addictive in the past, is so unpleasant to use now, that it actually conditions me to avoid it.

Slate Star Codex has no new content. Yeah, there are "slatestarcodex" and "motte" debates on Reddit, but... I already mentioned Reddit.

Almost all newspaper articles in my native language are paywalled these days. No, I am not going to pay for your clickbait.

So... I am vaguelly aware that Trump was an American president and now it is Biden (or is it still Trump, and Biden will be later? dunno), and there were (still are?) BLM protests in USA. And in my country, the largest political party recently split in two, and I don't even know the name of the new one, and I don't ev... (read more)

Assuming that you like math... would you have preferred to have more lessons focused on the lives of mathematicians? As opposed to only having lessons about mathematics itself.

I found this question on StackExchange, and I was horrified: I loved math, but I hated history, and this sounds like a suggestion to make math more like history.

The problem with school is that it not only tells you things, but it also expects you to memorize them. My memory sucks. I was good at math because I was able to re-derive everything that I forgot. But there is no way to re-d... (read more)

I noticed that some people use "skeptical" to mean "my armchair reasoning is better than all expert knowledge and research, especially if I am completely unfamiliar with it".

Example (not a real one): "I am skeptical about the idea that objects would actually change their length when their speed approaches the speed of light."

The advantage of this usage is that it allows you to dismiss all expertise you don't agree with, while making you sound a bit like an expert.

Today I learned that our friends at RationalWiki dislike effective altruism, to put it mildly. As David Gerard himself says, "it is neither altruistic, nor effective".

In section Where "Effective Altruists" actually send their money, the main complaint seems to be that among (I assume) respectable causes such as fighting diseases and giving money to poor people, effective altruists also support x-risk organisations, veganism, and meta organisations... or, using the language of RationalWiki, "sending money to Eliezer Yudkowsky", "feeling bad when people eat ... (read more)

1) There was this famous marshmallow experiment, where the kids had an option to eat one marshmallow (physically present on the table) right now, or two of them later, if they waited for 15 minutes. The scientists found out that the kids who waited for the two marshmallows were later more successful in life. The standard conclusion was that if you want to live well, you should learn some strategy to delay gratification.

(A less known result is that the optimal strategy to get two marshmallows was to stop thinking about marshmallows at all. Kids who focused ... (read more)

Anthropic Chesterton fence:

You know why the fence was built. The original reason no longer applies, or maybe it was a completely stupid reason. Yes, you should tear down the stupid fence.

And yet, there is a worry... might the fact that you see this stupid fence be an anthropic evidence that in the Everett branches without this stupid fence you are already dead?

Funny how many things we learn about AIs also apply to humans, but sometimes they allow us to see the human thing from a new angle. Today I noticed "Corrigibility's Desirability is Timing-Sensitive", and the analogy is that you want human children to easily learn from the adults around them, especially from parents, but adult humans with the same level of openness would be too easy to brainwash; you actually want them to push back against the attempts to modify their values.

I enjoyed reading a review of Sick Societies. Seems like it's difficult to find the right balance between "primitive cultures are stupid" and "everything in primitive cultures is full of deep wisdom that we modern people are unable to understand".

As usual, the public opinion moves as a pendulum; on the social level it goes from "we are 100% correct about everything, there is nothing to learn from others" to "everything would be better if we replaced our ways by the wisdom of others".

In the rationalist community, I think we started at the position of thinki... (read more)

Paul Graham's article Modeling a Wealth Tax says:

The reason wealth taxes have such dramatic effects is that they're applied over and over to the same money. Income tax happens every year, but only to that year's income. Whereas if you live for 60 years after acquiring some asset, a wealth tax will tax that same asset 60 times. A wealth tax compounds.

But wait, isn't income tax also applied over and over to the same money? I mean, it's not if I keep the money for years, sure. But if I use it to buy something from another person, then... (read more)

Something to trigger the rationalists:

The is a thing called Ultraspeaking; they teach you to speak better; David Chapman wrote a positive review recently. Here are some quotes from their free e-book:

In the following chapters we’re going to tackle:
1. Why thinking is the enemy of speaking
2. How to use your brain’s autocomplete feature to answer difficult questions

As we often say: “The enemy of speaking is thinking about speaking.”

Well, as counterintuitive as it may seem, you must learn to speak . . . before you think.

This is provided as an example of a wrong... (read more)

A simple rule for better writing: meta goes to the end.

Not sure if this is also useful for others or just specifically my bad habit. I start to write something, then I feel like some further explanation or a disclaimer is needed, then I find something more to add... and it is tempting to start the article with the disclaimers and other meta stuff. The result is a bad article where after the first screen of text you still haven't seen the important stuff, and now you are probably bored and close the browser tab.

Psychologically, it feels like I predict objec... (read more)

Taking ideas too seriously = assuming that you cannot make a mistake in your reasoning.

I wonder if every logical fallacy has a converse fallacy, and whether it would be useful to compose a list of fallacies arranged in pairs. Perhaps it would help us discover new ones, as missing pairs to something.

For example, some fallacies consist of taking a heuristic too seriously. Experts are often right about things, but an "argument by authority" assumes that this is true in 100% of situations. Similarly, wisdom of crowds, and an "argument by popularity". The converse fallacy would be ignoring the heuristic completely, even in situations where it mak... (read more)

Rationalists: If you write your bottom line first, it doesn't matter what clever arguments you write above it, the conclusion is completely useless as evidence.

Post-rationalists: Actually, if that bottom line was inherited from your ancestors, who inherited it from their ancestors, etc., that is evidence that the bottom line is useful. Otherwise, this culturally transmitted meme would be outcompeted by a more useful meme.

Robin Hanson: Actually, that is only evidence that writing the bottom line is useful. Whether it is useful to actually believe it and act accordingly, that is a completely different question.

Could someone please ELI5 why using a CNOT gate (if the target qubit was initially zero) does not violate the no-cloning theorem?

EDIT:

Oh, I think I got it. The forbidden thing is to have a state "copied and not entangled". CNOT gate creates a state that is "copied and entangled", which is okay, because you can only measure it once (if you measure either the original or the copy, the state of the other one collapses). The forbidden thing is to have a copy that you could measure independently (e.g. you could measure the copy without collapsing the original).

Approximately how is the cost of a quantum computer related to its number of qubits?

My guess would be more than linear (high confidence) but probably less than exponential (low confidence), but I know almost nothing about these things.

Some people express strong dislike at seeing others wear face masks, which reminds me of the anti-social punishment.

I am talking about situations where some people wear face masks voluntarily, for example in mass transit (if the situation in your country is different, imagine a different situation). In theory, if someone else is wearing the mask, even if you believe that it is utterly useless, even if for you wearing a face mask is the most uncomfortable thing you could imagine... hey, it's other person paying the cost, not you. Why so angry? Why not let t... (read more)

Project idea: ELI5pedia. Like Wikipedia, but optimized for being accessible for lay audience. If some topics are too complex, they could be written in multiple versions, progressing from the most simple to the most detailed (but still as accessible as possible).

Of course it would be even better if Wikipedia itself was written like this, but... well, for whatever reason, it is not.

Technically, Chesterton fence means that if something exists for no good reason, you are never allowed to remove it.

Because, before you even propose the removal, you must demonstrate your understanding of a good reason why the thing exists. And if there is none...

More precisely, it seems to me there is a motte and bailey version of Chesterton fence: the motte is that everything exists for a reason; the bailey is that everything exists for a good reason. The difference is, when someone challenges you to provide an understanding why a fence was built, whethe... (read more)