Conjecture is an alignment startup founded by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale alignment research... (read more)
The rationalist movement, rationality community,1 rationalsphere or rationalistsphere2 represents a set of modes of bayesian thinking from self-described rationalists or 'aspiring rationalists' typically associated with the Less Wrong diaspora and their associated communities... (read more)
Less Wrong is a community resource devoted to refining the art of human rationality which was founded in 2009. Site activity reached a peak in 2011-13 and a trough in 2016-17. This page mainly describes the history through 2016... (read more)
A Seed AI (a term coined by Eliezer Yudkowsky) is an Artificial General Intelligence (AGI) which improves itself by recursively rewriting its own source code without human intervention. Initially this program would likely have a minimal intelligence, but over the course of many iterations it would evolve to human-equivalent or even trans-human reasoning. The key for successful AI takeoff would lie in creating adequate starting conditions... (read more)
A Third Option dissolves a False Dilemma by showing that there are in fact more than two options... (read more)
Eliezer Yudkowsky is a research fellow of the Machine Intelligence Research Institute, which he co-founded in 2001. He is mainly concerned with the obstacles and importance of developing a Friendly AI, such as a reflective decision theory that would lay a foundation for describing fully recursive self modifying agents that retain stable preferences while rewriting their source code. He also co-founded LessWrong, writing the Sequences, long sequences of posts dealing with epistemology, AGI, metaethics, rationality and so on... (read more)
The old LessWrong wiki was a companion wiki site to LessWrong 1.0, it was built on MediaWiki software. As of September 2020, the LessWrong 2.0 team is migrating the contents of the old wiki to LessWrong 2.0's new tag/wiki system. The wiki import is complete... (read more)
The San Francisco Bay Area is a region in the US state of California. Many members of the rationalist community are located there, as are the Machine Intelligence Research Institute and the Center For Applied Rationality.. (read more)
Someone is well-calibrated if the things they predict with X% chance of happening in fact occur X% of the time. Importantly, calibration is not the same as accuracy. Calibration is about accurately assessing how good your predictions are, not making good predictions. Person A, whose predictions are marginally better than chance (60% of them come true when choosing from two options) and who is precisely 60% confident in their choices, is perfectly calibrated. In contrast, Person B, who is 99% confident in their predictions, and right 90% of the time, is more accurate than Person A, but less well-calibrated... (read more)
| User | Post Title | Wikitag | Pow | When | Vote |
Christopher Alexander (1936-2022) was an architect who studied the way nature and traditionally built buildings (such as peasant huts, or cathedrals) are a particular kind of beautiful, and have (he argued) the ability to bring a person back into a sense of perspective (e.g., a person may be quite stressed out about some detail, and then go for a long walk in nature, and find themselves "coming back to themselves.") Alexander attempted to work out a theory of design (for buildings, but also for design work broadly) that would create houses and other built objects with this same sort of beauty and sense of perspective embedded in them. His work inspired the "design patterns" movement in computer science, and, indirectly, wikis.
Once upon a time, LessWrong was a place where you'd be told to Read The Sequences before you'd finished your second comment apologizing for wasting time with non-constructive praise in the first.
Although the culture of rationality has changed greatly since the olden days, and our teachings are dispersed in many offspring movements, some remember good old-fashioned rationality with the same nostalgia MIRI computer scientists long for the days when AI was a science.
Suffering-focused ethics (SFE) is a family of moral views that give priority to reducing suffering, especially intense suffering. Rather than treating happiness and suffering as fully symmetric, suffering-focused views hold that preventing severe suffering often matters more urgently than creating additional happiness.
Indexical uncertainty is irreducible subjective uncertainty induced by anthropically expecting to be in more than one possible future world.
Suppose you are an AI and shortly 2 instances of you will see "Hello green world" and 3 instances will see "Hello red world". If you are a conscious AI, then you should anticipate seeing a green world with 40% probability, and a red world with 60% probability, and this subjective uncertainty is irreducible because it reflects an objective state of affairs where different versions of you see different things. To obtain full knowledge of this state of affairs obtaining later is to be left with subjective uncertainty now about "what I will see happen next".
On the Many-Worlds Interpretation, quantum uncertainty is irreducible because it is indexical uncertainty; the amplitudes in configuration space diverge into blobs of different measure, and we find ourselves in all blobs, somehow with a weight proportional to the measure of amplitude inside our blob.
A Newcomblike dilemma that pries apart the recommendations of EDT and CDT/FDT. Also known as XOR Blackmail (albeit I (EY) would object to this because it isn't what's normally understood as "blackmail").
> Exactly one of the following statements is true: You will pay me $1000, or,XOR your building already has a terrible termite infestation (I didn't put it there).
See also: Evolutionary Psychology, Goodhart's Law, Wireheading
A candy bar is a superstimulus: it contains more concentrated sugar, salt, and fat than anything that exists in the ancestral environment.
A candy bar matches taste buds that evolved in a hunter-gatherer environment, but it matches those taste buds much more strongly than anything that actually existed in the hunter-gatherer environment. The signal that once reliably correlated to healthy food has been hijacked, blotted out with a point in tastespace that wasn't in the training dataset - an impossibly distant outlier on the old ancestral graphs.
Suppose your model of physics says that the universe will eventually dissolve into a sea of particles in thermal equilibrium, and these particles will then exist indefinitely.
Then an exponentially vast supermajority of all observer-moments in this universe take place inside brains that momentarily assemble themselves from the sea of entropy by random fluctuations, and probably dissolve a moment later. These are "Boltzmann brains".
The vast supermajority of Boltzmann brains have relatively disordered experiences; they momentarily see chaos before dissolving again.
So your own, far more ordered and orderly experience in this very moment weighs heavily against the hypothesis "I am a Boltzmann brain". Inder most systems of anthropic reasoning this weighs heavily against the possibility "a supervast majority of all moments of consciousness are vastly less orderly than my own".
This then argues against
Full article:
This then argues against
Suffering-focused ethics (SFE) is a family of moral views that give priority to reducing suffering, especially intense suffering. Rather than treating happiness and suffering as fully symmetric, suffering-focused views hold that preventing severe suffering often matters more urgently than creating additional happiness.
Once upon a time, LessWrong was a place where you'd be told to Read The Sequences before you'd finished your second comment apologizing for wasting time with non-constructive praise in the first.
Although the culture of rationality has changed greatly since the olden days, and our teachings are dispersed in many offspring movements, some remember good old-fashioned rationality with the same nostalgia MIRI computer scientists long for the days when AI was a science.
So your own, far more ordered and orderly experience in this very moment weighs heavily against the hypothesis "I am a Boltzmann brain". InderUnder most systems of anthropic reasoning this weighs heavily against the possibility "a supervast majority of all moments of consciousness are vastly less orderly than my own".
Coordinal: A Postmortem (Ronak Mehta, 2026-05-19)
Indexical uncertainty is irreducible subjective uncertainty induced by anthropically expecting to be in more than one possible future world.
Suppose you are an AI and shortly 2 instances of you will see "Hello green world" and 3 instances will see "Hello red world". If you are a conscious AI, then you should anticipate seeing a green world with 40% probability, and a red world with 60% probability, and this subjective uncertainty is irreducible because it reflects an objective state of affairs where different versions of you see different things. To obtain full knowledge of this state of affairs obtaining later is to be left with subjective uncertainty now about "what I will see happen next".
On the Many-Worlds Interpretation, quantum uncertainty is irreducible because it is indexical uncertainty; the amplitudes in configuration space diverge into blobs of different measure, and we find ourselves in all blobs, somehow with a weight proportional to the measure of amplitude inside our blob.
But this makes the Predictor's statement be true! So the Predictor can go around sending letters like this to all the EDT agents in town who can afford the $1000.$1000, after checking to make sure they don't actually have a termite infestation.
Conversely a CDT or FDT agent will reason that, if they get a letter like this, they must already have a termite infestation, which will be unaffected by whether they pay (CDT) / by whether people like them predictably pay (FDT). So the Predictor won't send them letters if they have no termites, because they won't pay, and because the contents would be false. CDT/FDT agents only see these letters if the Predictor, acting perhaps to increase its credibility with EDT agents, sends them to some CDT/FDT agents who *do* have termites -- and then the result of this general policy and disposition, from an FDT perspectiveperspective, is to get a valuable free warning from the Predictor about their termite infestation.
A Newcomblike dilemma that pries apart the recommendations of EDT and CDT/FDT.
Suppose that an excellent Predictor, greedy for cash, known to be absolutely honest, sends to the owner of an apartment complex the following letter:
> Exactly one of the following statements is true: You will pay me $1000, or, your building already has a terrible termite infestation (I didn't put it there).
Since a terrible termite infestation would cost $1,000,000 to control, an evidential decision theorist will reason, "It is better to pay $1000; this is better news about whether I have a termite infestation." They pay the $1000.
But this makes the Predictor's statement be true! So the Predictor can go around sending letters like this to all the EDT agents in town who can afford the $1000.
Conversely a CDT or FDT agent will reason that, if they get a letter like this, they must already have a termite infestation, which will be unaffected by whether they pay (CDT) / by whether people like them predictably pay (FDT). So the Predictor won't send them letters if they have no termites, because they won't pay, and because the contents would be false. CDT/FDT agents only see these letters if the Predictor, acting perhaps to increase its credibility with EDT agents, sends them to some CDT/FDT agents who *do* have termites -- and then the result of this policy from an FDT perspective is to get a valuable free warning from the Predictor about their termite infestation.
Christopher Alexander (1936-2022) was an architect who studied the way nature and traditionally built buildings (such as peasant huts, or cathedrals) are a particular kind of beautiful, and have (he argued) the ability to bring a person back into a sense of perspective (e.g., a person may be quite stressed out about some detail, and then go for a long walk in nature, and find themselves "coming back to themselves.") Alexander attempted to work out a theory of design (for buildings, but also for design work broadly) that would create houses and other built objects with this same sort of beauty and sense of perspective embedded in them. His work inspired the "design patterns" movement in computer science, and, indirectly, wikis.
I think the phrase "more than one possible future world" is misleading and should be changed. The paradigmatic cases of indexical uncertainty (including both given examples of the five-AI-clones case and of Everettian quantum mechanics) involve no certainty about which objective possible world is actual, and only involves self-locating uncertainty about my location within a possible world.
So it would be more accurate to say that it is irreducible subjective uncertainty over multiple centered possible worlds or multiple locations with a given possible world.