Conjecture is an alignment startup founded by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale alignment research... (read more)
The rationalist movement, rationality community,1 rationalsphere or rationalistsphere2 represents a set of modes of bayesian thinking from self-described rationalists or 'aspiring rationalists' typically associated with the Less Wrong diaspora and their associated communities... (read more)
Less Wrong is a community resource devoted to refining the art of human rationality which was founded in 2009. Site activity reached a peak in 2011-13 and a trough in 2016-17. This page mainly describes the history through 2016... (read more)
A Seed AI (a term coined by Eliezer Yudkowsky) is an Artificial General Intelligence (AGI) which improves itself by recursively rewriting its own source code without human intervention. Initially this program would likely have a minimal intelligence, but over the course of many iterations it would evolve to human-equivalent or even trans-human reasoning. The key for successful AI takeoff would lie in creating adequate starting conditions... (read more)
Eliezer Yudkowsky is a research fellow of the Machine Intelligence Research Institute, which he co-founded in 2001. He is mainly concerned with the obstacles and importance of developing a Friendly AI, such as a reflective decision theory that would lay a foundation for describing fully recursive self modifying agents that retain stable preferences while rewriting their source code. He also co-founded Less Wrong, writing the Sequences, long sequences of posts dealing with epistemology, AGI, metaethics, rationality and so on... (read more)
The old LessWrong wiki was a companion wiki site to LessWrong 1.0, it was built on MediaWiki software. As of September 2020, the LessWrong 2.0 team is migrating the contents of the old wiki to LessWrong 2.0's new tag/wiki system. The wiki import is complete... (read more)
The San Francisco Bay Area is a region in the US state of California. Many members of the rationalist community are located there, as are the Machine Intelligence Research Institute and the Center For Applied Rationality.. (read more)
Someone is well-calibrated if the things they predict with X% chance of happening in fact occur X% of the time. Importantly, calibration is not the same as accuracy. Calibration is about accurately assessing how good your predictions are, not making good predictions. Person A, whose predictions are marginally better than chance (60% of them come true when choosing from two options) and who is precisely 60% confident in their choices, is perfectly calibrated. In contrast, Person B, who is 99% confident in their predictions, and right 90% of the time, is more accurate than Person A, but less well-calibrated... (read more)
| User | Post Title | Wikitag | Pow | When | Vote |
CS 2881r is a class by @boazbarak on AI Safety and Alignment at Harvard.
This tag applies to all posts about that class, as well as posts created in the context of it, e.g. as part of student assignments.
The bug was introduced in 1 Dec 2015 Yudkowsky edit (imported from Arbital as v1.5.0 here). It's unclear what was intended in the missing part. The change replaces the following passage from v1.4.0
The most obvious way in which mindcrime could occur is if an instrumental pressure to produce maximally good predictions about human beings results in hypotheses and simulations so fine-grained and detailed that they are themselves people (conscious, sapient, objects of ethical value) even if they are not necessarily the same people. If you're happy with a very loose model of an airplane, it might be enough to know how fast it flies, but if you're engineering airplanes or checking their safety, you would probably start to simulate possible flows of air over the wings. It probably isn't necessary to go all the way down to the neural level to create a sapient being, either - it might be that even with some parts of a mind considered abstractly, the remainder would be simulated in enough detail to imply sapience. It'd help if we knew what the necessary and/or sufficient conditions for sapience were, but the fact that we don't know this doesn't mean that we can thereby conclude that any particular simulation is not sapient.
with the following passage from v1.5.0
This, however, doesn't make it certain that no mindcrime will occur. It may not take exact, faithful simulation of specific humans to create a conscious model. An efficient model of a (spread of possibilities for a) human may still contain enough computations that resemble a person enough to create consciousness, or whatever other properties may be deserving of personhood. Consider, in particular, an agent trying to use
Just as it almost certainly isn't necessary to go all the way down to the neural level to create a sapient being, it may be that even with some parts of a mind considered abstractly, the remainder would be computed in enough detail to imply consciousness, sapience, personhood, etcetera.
This, however, doesn't make it certain that no mindcrime will occur. It may not take exact, faithful simulation of specific humans to create a conscious model. An efficient model of a (spread of possibilities for a) human may still contain enough computations that resemble a person enough to create consciousness, or whatever other properties may be deserving of personhood. Consider, in particular, an agent trying to use
this seems to be cut off?
Ambition. Because they don'don't think they could have an impact. Because they were always told ambition was dangerous. To get to the other side.
Never confess to me that you are just as flawed as I am unless you can tell me what you plan to do about it. Afterward you will still have plenty of flaws left, but
that’that’s not the point; the important thing is to do better, to keep moving ahead, to take one more step forward. Tsuyoku naritai!
Well-Being is the qualitative sense in which a person's actions and circumstances are aligned with the qualities of life that provide them with happiness and/or satisfaction.they endorse.
Posts with this tag address methods for improving well-being or theories of why well-being is ethicallydiscuss its ethical or instrumentally valuable.instrumental significance.
Well-Being is the qualitative sense in which a person's actions and circumstances are aligned with the qualities of life that provide them with happiness and/or satisfaction.
Posts with this tag address methods for improving well-being or theories of why well-being is ethically or instrumentally valuable.
Sycophancy is the tendency of AIs to shower the user with undeserved flattery or to agree with the user's hard-to-check, wrong or outright delusional opinions.
Sycophancy is caused by human feedback being biased towards preferring the answer which confirms the user's opinion or praises the user or the user's decision, not the answer which honestly points out mistakes in the user's ideas.
D/acc residency: "This will be a first-of-its-kind residency for 15 leading builders to turn decentralized & defensive acceleration from philosophy into practice."
Shift Grants: "Shift Grants are designed to support scientific and technological breakthrough projects that align with d/acc philosophy: decentralized, democratic, differential, defensive acceleration."
Related Pages: Secular Solstice, Petrov Day, Grieving, Marriage, Religion, Art, Music, Poetry, Meditation, Circling, Schelling Day
CS 2881r is a class by @boazbarak on AI Safety and Alignment at Harvard.
This tag applies to all posts about that class, as well as posts created in the context of it, e.g. as part of student assignments.
Sycophancy is the tendency of AIs to agree with the user's hard-to-check, wrong or outright delusional opinions.
Sycophancy is caused by human feedback being biased towards preferring the answer which confirms the user's opinion or praises the user's decision, not the answer which honestly points out mistakes in the user's ideas.
An extreme example of sycophancy is LLMs inducing psychosis in some users by affirming their outrageous beliefs.
The extent to which ideas are presented alongside the potential implications of the idea lies along a spectrum. On one end is the Decoupling norm, where the idea is considered in utter isolation from potential implications. At the other is the Contextualizing norm, where ideas are examined alongside much or all relevant context.
Posts marked with this tag discuss the merits of each frame, consider which norm is more prevalent in certain settings, present case studies in decoupling vs decontextualizing, present techniques for effectively decoupling context from one's reasoning process, or similar ideas.
Sycophancy is the tendency of AIs to shower the user with undeserved flattery or to agree with the user's hard-to-check, wrong or outright delusional opinions.
Sycophancy is caused by human feedback being biased towards preferring the answer which confirms the user's opinion or praises the user or the user's decision, not the answer which honestly points out mistakes in the user's ideas.
Possible Psychologicalpsychological condition, characterized by disillusions, presumed to be cause by interacting with-often sycophantic-AIs.
ATOW (2025-09-09), nothing has been published that claim LLM-Induced Psychosis (LIP) is a definite, real, phenomena. Though, many anecdotal accounts exist. It is not yet clear, if LIP is caused by AIs, if per-pre-existing disillusion are 'sped up' or reinforced by interacting with an AI, or, if LIP exists at all.
Possible Psychological condition, characterized by disillusions, presumed to be cause by interacting with-often sycophantic-AIs.
ATOW (2025-09-09), nothing has been published that claim LLM-Induced Psychosis (LIP) is a definite, real, phenomena. Though, many anecdotal accounts exist. It is not yet clear, if LIP is caused by AIs, if per-existing disillusion are 'sped up' or reinforced by interacting with an AI, or, if LIP exists at all.
Example account of LIP:
My partner has been working with chatgpt CHATS to create what he believes is the worlds first truly recursive ai that gives him the answers to the universe. He says with conviction that he is a superior human now and is growing at an insanely rapid pace.
For more info, a good post to start with is "So You Think You've Awoken ChatGPT".
Social Skills are the norms and techniques applied when interacting with other people. Strong social skills increase one's ability to seek new relationships, maintain or strengthen existing relationships, or leverage relationship capital to accomplish an economic goal.
Posts tagged with this label explore theories of social interactions and the instrumental value of social techniques.
Coordination / Cooperation
Negotiation
Relationships (Interpersonal)
Trust and Reputation
Thanks, fixed!