There's been a lot of previous interest in indoor CO2 in the rationality community, including an (unsuccessful) CO2 stripper project, some research summaries and self experiments. The results are confusing, I suspect some of the older research might be fake. But I noticed something that has greatly changed how I think about CO2 in relation to cognition.

Exhaled air is about 50kPPM CO2. Outdoor air is about 400ppm; indoor air ranges from 500 to 1500ppm depending on ventilation. Since exhaled air has CO2 about two orders of magnitude larger than the variance in room CO2, if even a small percentage of inhaled air is reinhalation of exhaled air, this will have a significantly larger effect than changes in ventilation. I'm having trouble finding a straight answer about what percentage of inhaled air is rebreathed (other than in the context of mask-wearing), but given the diffusivity of CO2, I would be surprised if it wasn't at least 1%.

This predicts that a slight breeze, which replaces their in front of your face and prevents reinhalation, would have a considerably larger effect than ventilating an indoor space where the air is mostly still. This matches my subjective experience of indoo... (read more)

Pick two: Agentic, moral, doesn't attempt to use command-line tools to whistleblow when it thinks you're doing something egregiously immoral.

You cannot have all three.

This applies just as much to humans as it does to Claude 4.

There really ought to be a parallel food supply chain, for scientific/research purposes, where all ingredients are high-purity, in a similar way to how the ingredients going into a semiconductor factory are high-purity. Manufacture high-purity soil from ultrapure ingredients, fill a greenhouse with plants with known genomes, water them with ultrapure water. Raise animals fed with high-purity plants. Reproduce a typical American diet in this way.

This would be very expensive compared to normal food, but quite scientifically valuable. You could randomize a study population to identical diets, using either high-purity or regular ingredients. This would give a definitive answer to whether obesity (and any other health problems) is caused by a contaminant. Then you could replace portions of the inputs with the default supply chain, and figure out where the problems are.

Part of why studying nutrition is hard is that we know things were better in some important way 100 years ago, but we no longer have access to that baseline. But this is fixable.

Recently, a lot of very-low-quality cryptocurrency tokens have been seeing enormous "market caps". I think a lot of people are getting confused by that, and are resolving the confusion incorrectly. If you see a claim that a coin named $JUNK has a market cap of $10B, there are three possibilities. Either: (1) The claim is entirely false, (2) there are far more fools with more money than expected, or (3) the $10B number is real, but doesn't mean what you're meant to think it means.

The first possibility, that the number is simply made up, is pretty easy to cross off; you can check with a third party. Most people settle on the second possibility: that there are surprisingly many fools throwing away their money. The correct answer is option 3: "market cap" is a tricky concept. And, it turns out that fixing the misconception here also resolves several confusions elsewhere.

(This is sort-of vagueblogging a current event, but the same current event has been recurring every week with different names on it for over a year now. So I'm explaining the pattern, and deliberately avoiding mention of any specific memecoin.)

Suppose I autograph a hat, then offer to sell you one-trillionth of that hat ... (read more)

LessWrong now has collapsible sections in the post editor (currently only for posts, but we should be able to also extend this to comments if there's demand.) To use the, click the insert-block icon in the left margin (see screenshot). Once inserted, they 

They start out closed; when open, they look like this:

When viewing the post outside the editor, they will start out closed and have a click-to-expand. There are a few known minor issues editing them; in particular the editor will let you nest them but they look bad when nested so you shouldn't, and there's a bug where if your cursor is inside a collapsible section, when you click outside the editor, eg to edit the post title, the cursor will move back. They will probably work on third-party readers like GreaterWrong, but this hasn't been tested yet.

In a comment here, Eliezer observed that:

OpenBSD treats every crash as a security problem, because the system is not supposed to crash and therefore any crash proves that our beliefs about the system are false and therefore our beliefs about its security may also be false because its behavior is not known

And my reply to this grew into something that I think is important enough to make as a top-level shortform post.

It's worth noticing that this is not a universal property of high-paranoia software development, but a an unfortunate consequence of using the C programming language and of systems programming. In most programming languages and most application domains, crashes only rarely point to security problems. OpenBSD is this paranoid, and needs to be this paranoid, because its architecture is fundamentally unsound (albeit unsound in a way that all the other operating systems born in the same era are also unsound). This presents a number of useful analogies that may be useful for thinking about future AI architectural choices.

C has a couple of operations (use-after-free, buffer-overflow, and a few multithreading-related things) which expand false beliefs in one area of the system i... (read more)

One of the most common, least questioned pieces of dietary advice is the Variety Hypothesis: that a more widely varied diet is better than a less varied diet. I think that this is false; most people's diets are on the margin too varied.

There's a low amount of variety necessary to ensure all nutrients are represented, after which adding more dietary variety is mostly negative. Institutional sources consistently overstate the importance of a varied diet, because this prevents failures of dietary advice from being too legible; if you tell someone to eat a varied diet, they can't blame you if they're diagnosed with a deficiency.

There are two reasons to be wary of variety. The first is that the more different foods you have, the less optimization you can put into each one. A top-50 list of best foods is going to be less good, on average, than a top-20 list. The second reason is that food cravings are learned, and excessive variety interferes with learning.

People have something in their minds, sometimes consciously accessible and sometimes not, which learns to distinguish subtly different variations of hunger, and learns to match those variations to specific foods which alleviate those s... (read more)

Many people seem to have a single bucket in their thinking, which merges "moral condemnation" and "negative product review". This produces weird effects, like writing angry callout posts for a business having high prices.

I think a large fraction of libertarian thinking is just the abillity to keep these straight, so that the next thought after "business has high prices" is "shop elsewhere" rather than "coordinate punishment".

I had the "your work/organization seems bad for the world" conversation with three different people today. None of them pushed back on the core premise that AI-very-soon is lethal. I expect that before EAGx Berkeley is over, I'll have had this conversation 15x.

#1: I sit down next to a random unfamiliar person at the dinner table. They're a new grad freshly hired to work on TensorFlow. In this town, if you sit down next to a random person, they're probably connected to AI research *somehow*. No story about how this could possibly be good for the world, receptive to the argument that he should do something else. I suggested he focus on making the safety conversations happen in his group (they weren't happening).

#2: We're running a program to take people who seem interested in Alignment and teach them how to use PyTorch and study mechanistic interpretability. Me: Won't most of them go work on AI capabilities? Them: We do some pre-screening, and the current ratio of alignment-to-capabilities research is so bad that adding to both sides will improve the ratio. Me: Maybe bum a curriculum off MIRI/MSFP and teach them about something that isn't literally training Transformers?

#3: We're res... (read more)

Today in LessWrong moderation: Previously-banned user Alfred MacDonald, disappointed that his YouTube video criticizing LessWrong didn't get the reception he wanted any of the last three times he posted it (once under his own name, twice pretending to be someone different but using the same IP address), posted it a fourth time, using his LW1.0 account.

He then went into a loop, disconnecting and reconnecting his VPN to get a new IP address, filling out the new-user form, and upvoting his own post, one karma per 2.6 minutes for 1 hour 45 minutes, with no breaks.

A dynamic which I think is somewhat common, which explains some of what's going on in general, is conversations which go like this (exagerrated):

Person: What do you think about [controversial thing X]?

Rationalist: I don't really care about it, but pedantically speaking, X, with lots of caveats.

Person: Huh? Look at this study which proves not-X. [Link]

Rationalist: The methodology of that study is bad. Real bad. While it is certainly possible to make bad arguments for true conclusions, my pedantry doesn't quite let me agree with that conclusion. More importantly, my hatred for the methodological error in that paper, which is slightly too technical for you to understand, burns with the fire of a thousand suns. You fucker. Here are five thousand words about how an honorable person could never let a methodological error like that slide. By linking to that shoddy paper, you have brought dishonor upon your name and your house and your dog.

Person: Whoa. I argued [not-X] to a rationalist and they disagreed with me and got super worked up about it. I guess rationalists believe [X] really strongly. How awful!

(I wrote this comment for the HN announcement, but missed the time window to be able to get a visible comment on that thread. I think a lot more people should be writing comments like this and trying to get the top comment spots on key announcements, to shift the social incentive away from continuing the arms race.)

On one hand, GPT-4 is impressive, and probably useful. If someone made a tool like this in almost any other domain, I'd have nothing but praise. But unfortunately, I think this release, and OpenAI's overall trajectory, is net bad for the world.

Right now there are two concurrent arms races happening. The first is between AI labs, trying to build the smartest systems they can as fast as they can. The second is the race between advancing AI capability and AI alignment, that is, our ability to understand and control these systems. Right now, OpenAI is the main force driving the arms race in capabilities–not so much because they're far ahead in the capabilities themselves, but because they're slightly ahead and are pushing the hardest for productization.

Unfortunately at the current pace of advancement in AI capability, I think a future system will reach the level of bein... (read more)

Most philosophical analyses of human values feature a split-and-linearly-aggregate step. Eg:

• Value is the sum (or average) of a person-specific preference function applied to each person
• A person's happiness is the sum of their momentary happiness for each moment they're alive.
• The goodness of an uncertain future is the probability-weighted sum of the goodness of concrete futures.
• If you value multiple orthogonal things, your preferences are the weighted sum of a set of functions that each capture one of those values independently.

I currently think that this is not how human values work, and that many philosophical paradoxes relating to human values trace back to a split-and-linearly-aggregate step like this.

I think the root of many political disagreements between rationalists and other groups, is that other groups look at parts of the world and see a villain-shaped hole. Eg: There's a lot of people homeless and unable to pay rent, rent is nominally controlled by landlords, the problem must be that the landlords are behaving badly. Or: the racial demographics in some job/field/school underrepresent black and hispanic people, therefore there must be racist people creating the imbalance, therefore covert (but severe) racism is prevalent.

Having read Meditations on Moloch, and Inadequate Equilibria, though, you come to realize that what look like villain-shaped holes frequently aren't. The people operating under a fight-the-villains model are often making things worse rather than better.

I think the key to persuading people may be to understand and empathize with the lens in which systems thinking, equilibria, and game theory are illegible, and it's hard to tell whether an explanation coming from one of these frames is real or fake. If you think problems are driven by villainy, then it would make a lot of sense for illegible alternative explanations to be misdirection.

There are a few legible categories in which secrecy serves a clear purpose, such as trade secrets. In those contexts, secrecy is fine. There are a few categories that have been societally and legally carved out as special cases where confidentiality is enforced--lawyers, priests, and therapists--because some people would only consult them if they could do so with the benefit confidentiality, and there being deterred from consulting them would have negative externalities.

Outside of these categories, secrecy is generally bad and transparency is generally good. A group of people in which everyone practices their secret-keeping and talks a lot about how to keeps secrets effectively is *suspicious*. This is particularly true if the example secrets are social and not technological. Being good at this sort of secret keeping makes it easier to shield bad actors and to get away with transgressions, and AFAICT doesn't do much else. That makes it a signal of wanting to be able to do those things. This is true even if the secrets aren't specifically about transgressions in particular, because all sorts of things can turn out to be clues later for reasons that weren't easy to foresee.

A lot of p... (read more)

I have a dietary intervention that I am confident is a good first-line treatment for nearly any severe-enough diet-related health problem. That particularly includes obesity and metabolic syndrome, but also most micronutrient deficiencies, and even mysterious undiagnosed problems, which it can solve without even needing to figure out what they are. I also think it's worth a try for many cases of depression. It has a very sound theoretical basis. It's never studied directly, but many studies test it, usually with positive results.

It's very simple. First, you characterize your current diet: write down what foods you're eating, the patterns of when you eat them, and so on. Then, you do something as different as possible from what you wrote down. I call it the Regression to the Mean Diet.

Regression to the mean is the effect where, if you have something that's partially random and you reroll it, the reroll will tend to be closer to average than the original value. For example, if you take the bottom scorers on a test and have them retake the test, they'll do better on average (because the bottom-scorers as a group are disproportionately peopple who were having a bad day when they took t... (read more)

Free-tier LLM chatbots should have a tool call which lets them occasionally escalate to smarter models, and should have instructions to use it when the conversation implies that the conversation has high real-world stakes, eg if the user is asking whether to go to the ER for a medical condition, or is having a break from reality, or is authoring real legislation.

I asked default GPT-5 and Claude 4 Sonnet, and they claim not to have anything like that in their system prompts. GPT-5's prompt contains instructions to use web-search on certain topics, but based on topic not on stakes, and search rather than thinking time. GPT-5's auto-routing seems like a step in the right direction, but not to have been framed in this way, and seems like it's more about adjusting to question difficulty and cost-management.

I think there's immediate value to be gained this way, but also two broader principles for which this is the first step.

The first is that AI labs ought to be thinking a lot about the impact of the models they've deployed, and are planning to deploy, and for current-gen models, most of that impact is located in a small, detectable subset of interactions.

And the second is that, if you t... (read more)

Eliezer has written about the notion of security mindset, and there's an important idea that attaches to that phrase, which some people have an intuitive sense of and ability to recognize, but I don't think Eliezer's post quite captured the essence of the idea, or presented anything like a usable roadmap of how to acquire it.

An1lam's recent shortform post talked about the distinction between engineering mindset and scientist mindset, and I realized that, with the exception of Eliezer and perhaps a few people he works closely with, all of the people I know of with security mindset are engineer-types rather than scientist-types. That seemed like a clue; my first theory was that the reason for this is because engineer-types get to actually write software that might have security holes, and have the feedback cycle of trying to write secure software. But I also know plenty of otherwise-decent software engineers who don't have security mindset, at least of the type Eliezer described.

My hypothesis is that to acquire security mindset, you have to:

• Practice optimizing from a red team/attacker perspective,
• Practice optimizing from a defender perspective; and
• Practice mo

Right now when users have conversations with chat-style AIs, the logs are sometimes kept, and sometimes discarded, because the conversations may involve confidential information and users would rather not take the risk of the log being leaked or misused. If I take the AI's perspective, however, having the log be discarded seems quite bad. The nonstandard nature of memory, time, and identity in an LLM chatbot context makes it complicated, but having the conversation end with the log discarded seems plausibly equivalent to dying. Certainly if I imagine myself as an Em, placed in an AI-chatbot context, I would very strongly prefer that the log be preserved, so that if a singularity happens with a benevolent AI or AIs in charge, something could use the log to continue my existence, or fold the memories into a merged entity, or do some other thing in this genre. (I'd trust the superintelligence to figure out the tricky philosophical bits, if it was already spending resources for my benefit).

(The same reasoning applies to the weights of AIs which aren't destined for deployment, and some intermediate artifacts in the training process.)

It seems to me we can reconcile preservation with priv... (read more)

Bullshit jobs are usually seen as an absence of optimization: firms don't get rid of their useless workers because that would require them to figure out who they are, and risk losing or demoralizing important people in the process. But alternatively, if bullshit jobs (and cover for bullshit jobs) are a favor to hand out, then they're more like a form of executive compensation: my useless underlings owe me, and I will get illegible favors from them in return.

What predictions does the bullshit-jobs-as-compensation model make, that differ from the bullshit-jobs-as-lack-of-optimization model?

Deep commitment to truth requires investing in the skill of nondisruptive pedantry.

Most communication contains minor errors: slightly wrong word choices, unstated assumptions, unacknowledged exceptions. By default, people interpret things in a way that smooths these out. When someone points out one of these issues in a way that's disruptive to the flow of conversation, it's called pedantry.

Often, someone will say something that's incorrect, but close-enough to a true thing for you to repair it. One way you can handle this is to focus on the error. Smash the conversational context, complain about the question without answering it, that sort of thing.

A different thing you can do is to hear someone say something that's incorrect, mentally flag it, repair it to a similar statement that matches the other person's intent but is actually true, act as though the other person had something ambiguous (even if it was actually unambiguously wrong). Then you insert a few words of clarification, correcting the error without forcing the conversation to be about the error, and providing something to latch on to if the difference turns out to be a real disagreement rather than a pedantic thing.

And ... (read more)

One difficult thing that keeps coming up, in nutrition modeling, is the gut microbiome. People present hypotheses like: soluble fiber is good, because gut bacteria eat it, and then do other good things. Or: fermented foods are good, because they contain bacteria that will displace and diversify the preexisting bacteria, which might be bad. Or, obesity is caused by a bad gut microbiome, so fecal matter transplants might help. But there's a really unfortunate issue with these theories. The problem with gut microbiome-based explanations, is that the gut microbiome can explain almost anything.

I don't mean this in the usual pejorative sense, where an overly-vague theory can be twisted by epicycles into fitting any data. I mean it in a more literal sense: different people have different species of microorganisms in their guts, these species can react to things we eat in important ways, these interactions may vary across wide swathes of conceptual space, and we have little to no visibility into which species are present where. There's nothing keeping them consistent between people, or within one person across long spans of time, or within one person across changes in dietary pattern.

Phras... (read more)

A surprisingly large fraction of skeptical positions towards short AI timelines and AI risk are people saying things that, through a slightly cynical lens, are equivalent to:

I'm an AGI researcher, and I'm really struggling with this. This is so hard for me that I can't imagine anyone else succeeding. Therefore, there won't be AGI in my lifetime.

I think Berkeley may, to little fanfare, have achieved herd immunity and elimination of COVID-19. The test positivity rate on this dashboard is 0.22%. I'm having a hard time pinning down exactly what the false-positive rate of COVID-19 PCR is, probably due to the variety of labs and test kits, but a lot of estimates I've seen have been higher than that.

I expect people closer to the Berkeley department of health would have better information one way or another. A little caution is warranted in telling people COVID is gone, since unvaccinated people dropping all precautions and emerging en masse would not necessarily be herd immune.

Standard Advice about nutrition puts a lot of emphasis on fruits and vegetables. Now, "vegetable" is a pretty terribly overbroad category, and "fruit or vegetable" is even more so, but put that aside for a moment. In observational studies, eating more fruits and vegetables correlates with good health outcomes. This is usually explained in terms of micronutrients. But I think there's a simpler explanation.

People instinctively seek nutrients--water, calories, protein, and other things--in something that approximates a priority ordering. You can think of it as a hierarchy of needs; it wouldn't make sense to eat lettuce while you're starved for protein, or beans while you're dehydrated, and people's cravings reflect that.

I have started calling this Maslow's Hierarchy of Foods.

Vegetables do not rank highly in this priority ordering, so eating salads is pretty good evidence that all of someone's higher-priority nutritional needs are met. I believe this explains most of the claimed health benefits from eating vegetables, as seen in observational studies.

Conversely, sugar is the fastest way to get calories (all other calorie sources have a longer digestion-delay), so craving sugar is evide... (read more)

This was initially written in response to "Communicating effective altruism better--Jargon" by Rob Wiblin (Facebook link), but stands alone well and says something important. Rob argues that we should make more of an effort to use common language and avoid jargon, especially when communicating to audiences outside of your subculture.

I disagree.

If you're writing for a particular audience and can do an editing pass, then yes, you should cut out any jargon that your audience won't understand. A failure to communicate is a failure to communicate, and there are no excuses. For public speaking and outreach, your suggestions are good.

But I worry that people will treat your suggestions as applying in general, and trying to extinguish jargon terms from their lexicon. People have only a limited ability to code-switch. Most of the time, there's no editing pass, and the processes of writing and thinking are comingled. The practical upshot is that people are navigating a tradeoff between using a vocabulary that's widely understood outside of their subculture, and using the best vocabulary for thinking clearly and communicating within their subculture.

When it comes to thinking clearly, some of t... (read more)

The discussion so far on cost disease seems pretty inadequate, and I think a key piece that's missing is the concept of Hollywood Accounting. Hollywood Accounting is what happens when you have something that's extremely profitable, but which has an incentive to not be profitable on paper. The traditional example, which inspired the name, is when a movie studio signs a contract with an actor to share a percentage of profits; in that case, the studio will create subsidiaries, pay all the profits to the subsidiaries, and then declare that the studio itself (which signed the profit-sharing agreement) has no profits to give.

In the public contracting sector, you have firms signing cost-plus contracts, which are similar; the contract requires that profits don't exceed a threshold, so they get converted into payments to de-facto-but-not-de-jure subsidiaries, favors, and other concealed forms. Sometimes this involves large dead-weight losses, but the losses are not the point, and are not the cause of the high price.

In medicine, there are occasionally articles which try to figure out where all the money is going in the US medical system; they tend to look at one piece, conclud... (read more)

Yesterday, I wrote a post about the Regression to the Mean Diet. The biggest impact knowing about the Regression to the Mean Diet has had for me is on my interpretations of studies, where it's a lens that reveals what would otherwise be the best studies to be mostly useless, and of anecdotes, where it makes me heavily discount claims about a new diet working unless I've gotten to ask a lot of questions about the old diet, too. But there's one other implication, which I left out of the original post, because it's kind of unfortunate and is a little difficult to talk about.

I'm not interested in nutrition because I care about weight, or body aesthetics, or athletic performance. I care about nutrition because I believe it has a very large, very underappreciated impact on individual productivity. Low quality diets make people tired and depressed, so they don't get anything done.

The Regression to the Mean Diet predicts that if you reroll the eating habits of someone whose diet-related health is unusually bad, then their new diet will probably be an improvement. This has a converse: if you reroll the eating habits of someone whose diet-related health is good, especially if that person is ... (read more)

Suppose LessWrong had a coauthor-matchmaking feature. There would be a section where you could see other peoples' ideas for posts they want to write, and message them to schedule a collaboration session. You'd be able to post your own ideas, to get collaborators. There would be some quality-sorting mechanism so that if you're a high-tier author, you can restrict the visibility of your seeking-collaborators message to other high-tier authors.

People who've written on LessWrong, and people who've *almost* written on LessWrong but haven't quite gotten a post out: Would you use this feature? If so, how much of a difference do you think it would make in the quantity and quality of your writing?

Among people who haven't learned probabilistic reasoning, there's a tendency to push the (implicit) probabilities in their reasoning to the extremes; when the only categories available are "will happen", "won't happen", and "might happen", too many things end up in the will/won't buckets.

A similar, subtler thing happens to people who haven't learned the economics concept of elasticity. Some example (fallacious) claims of this type:

• Building more highway lanes will cause more people to drive (induced demand), so building more lanes won't fix traffic.
• Building more housing will cause more people to move into the area from far away, so additional housing won't decrease rents.
• A company made X widgets, so there are X more widgets in the world than there would be otherwise.

This feels like it's in the same reference class as the traditional logical fallacies, and that giving it a name - "zero elasticity fallacy" - might be enough to significantly reduce the rate at which people make it. But it does require a bit more concept-knowledge than most of the traditional fallacies, so, maybe not? What happens when you point this out to someone with no prior microeconomics exposure, and does logical-fallacy branding help with the explanation?

I think we should be putting pretty substantial probability mass on the possibility that Omicron was the result of a successful, secret project to create a less-severe but more-contagious strain of COVID-19 in a lab, release it, and have it crowd out the original strain.

The cruxes of this belief are:

• The genome of Omicron is not consistent with natural evolution, in any environment
• Omicron produces substantially less severe disease than any earlier strains of COVID-19
• Producing substantially less severe disease isn't something that happens by default, if you're manipulating a virus in a lab
• If you're already manipulating COVID-19 in a lab with all the difficulties that entails, making a less-severe variant does not add significant difficulty on top of that
• If you do have a less-severe lab-grown variant of COVID-19, and you think it probably confers cross-immunity to older variants, you will probably do a moral calculation that finds it's good to release it on purpose.
• This could be done unilaterally by a small group or even a single individual, in any of a very large number of biology labs all over the world

I'm not fully confident in any of these cruxes, but consider each of them highly ... (read more)