AI Control in the context of AI Alignment is a category of plans that aim to ensure safety and benefit from AI systems, even if they are goal-directed and are actively trying to subvert your control measures. From The case for ensuring that powerful AIs are controlled:.. (read more)
The Open Agency Architecture ("OAA") is an AI alignment proposal by (among others) @davidad and @Eric Drexler. .. (read more)
Singluar learning theory is a theory that applies algebraic geometry to statistical learning theory, developed by Sumio Watanabe. Reference textbooks are "the grey book", Algebraic Geometry and Statistical Learning Theory, and "the green book", Mathematical Theory of Bayesian Statistics.
Archetypal Transfer Learning (ATL) is a proposal by @whitehatStoic for what is argued by the author to be a fine tuning approach that "uses archetypal data" to "embed Synthetic Archetypes". These Synthetic Archetypes are derived from patterns that models assimilate from archetypal data, such as artificial stories. The method yielded a shutdown activation rate of 57.33% in the GPT-2-XL model after fine-tuning. .. (read more)
Open Threads are informal discussion areas, where users are welcome to post comments that didn't quite feel big enough to warrant a top-level post, nor fit in other posts... (read more)
A Black Marble is a technology that by default destroys the civilization that invents it. It's one type of Existential Risk. AGI may be such an invention, but isn't the only one... (read more)
AI Evaluations focus on experimentally assessing the capabilities, safety, and alignment of advanced AI systems. These evaluations can be divided into two main categories: behavioral and understanding-based... (read more)
| User | Post Title | Wikitag | Pow | When | Vote |
Consent is a foundational concept in many practical systems of ethics (such as found in medicine).
Self-immolation is a hypothetical act that could be executed by a leading AI (capabilities) lab, involving self-destruction, including the destruction of all its resources relevant for furthering progress towards AGI (or more broadly, extremely dangerous capabilities). It would also be a signal to the world that existential/catastrophic risk from AI has been taken seriously by one of the leading AI capabilities actors.
A weaker version would involve a credibly signaled and faithfully executed pivot away from AGI progress towards safer, narrower, bounded AI systems (see Tool AI).
The idea has been independently proposed at least two times:...
The agent-structure problem is the question about whether systems that behave like agents necessarily have an internal structure that makes them agentic. This structure usually involves some search procedure.
When CDT makes its decisions, it only thinks it controls things causally downstream of its actions. UDT by contrast, is choosing as if it controls every part of reality that is logically correlated withdownstream of its actions.logical output. This allows it to acausally bargaindetermine a wide range of other facts across the multiverse.universe that are logically correlated with itself, like what is or has been reliably predicted about its present decision, or what other agents sufficiently similar to itself will choose. Son of CDT is somewhere in the middle. It acts as if it controls only the things logically correlated with its actions that are causally downstream of its moment of original creation.
If a Son of CDT agent goes on to create further agents, all of those agents will have the same magic moment. They will all care about whether or not Omega's knowledge of them is causally downstream of the moment the moment the CDT agent first wrote Son-of-CDT code.
The Machine Alignment, Transparency, and Security (MATS) Program is an independent research and educational seminar program that provides emerging researchers with mentorship, talks,talks & workshops, research support, and research support and connects themconnections with the SF Bay Area and London AI safety research communities.
The name was suggested by Ryan Grenblatt in niplav in a reply“AI companies are unlikely to Daniel Kokotajlo's shortformmake high-assurance safety cases if timelines are short”.
The name was suggested by niplav in a reply to Daniel Kokotajlo's shortform.
CDT agents don't consider the acausal impactlogical impacts of their decisionsdecision algorithms' outputs when choosing actions.actions, only the physical consequences of their physical act. Whenever a CDT agent is put in a situation where it has to make a decision, it considers multiple hypothetical worlds,hypotheticals, one for each decision it could make. In a CDT agent, the only difference between these hypothetical worlds is the decision it makes.physical act in the moment of that act, and what happens physically / causally downstream from that. This means that when CDT is faced with something trying to predict its actions, CDT imagines its decision to not have any effect on its predicted decision.
Although slavery is usually involuntary and involves coercion, there are also cases where people voluntarily enter into slavery (like(link to wikipedia!) to pay a debt or earn money due to poverty.
SlavesKant's third formulation of the categorical imperative lets you build up most of the structure of the key moral ideas from a simple rule: "treat no person as purely a means to and end, but always also as an end in themselves". Many applications of the categorical imperative require baroque derivations to loop back and be justified from this premise (treated as a generative axiom) but "slavery is forbidden" is an elementary proof. A slave is a person, turned into a tool and piece of property of another person... a literal "means" to ANY end that the owning person (or "Master") deems desirable and feasible.
Historically, slaves would be kept in bondage for life, or for a fixed period of time after which they would be grated freedom. Many historical cases of enslavement occurred as a result of breaking the law, becoming indebted, suffering a military defeat, or exploitation for cheaper labor; other forms of slavery were instituted along demographic lines such as race or sex.
Self-immolation is a hypothetical act that could be executed by a leading AI (capabilities) lab, involving self-destruction, including the destruction of all its resources relevant for furthering progress towards AGI (or more broadly, extremely dangerous capabilities). It would also be a signal to the world that existential/catastrophic risk from AI has been taken seriously by one of the leading AI capabilities actors.
A weaker version would involve a credibly signaled and faithfully executed pivot away from AGI progress towards safer, narrower, bounded AI systems (see Tool AI).
The idea has been independently proposed at least two times:
The Machine Alignment, Transparency, and Security (MATS) Program is an independent research and educational seminar program that provides emerging researchers with mentorship, talks, workshops, and workshopsresearch support and connects them with the SF Bay Area and London AI safety research communities.
Kant's third formulation of the categorical imperative lets you build up most of the structure of the key moral ideas from a simple rule: "treat no person as purely a means to andan end, but always also as an end in themselves". Many applications of the categorical imperative require baroque derivations to loop back and be justified from this premise (treated as a generative axiom) but "consent ethics" in general, and "slavery is forbidden" is anare both elementary proof.proofs from this starting point. A slave is a person, turned into a tool and piece of property of another person... a literal "means" to ANY end that the owning person (or "Master") deems desirable and feasible.
Superposition in this context means storing information in an overcompleetovercomplete basis. Comp-in-Sup is the study of how (and if) neural networks performs computation, using information stored in overcomplete basis. The reason we want to do this, is because this information could inform better feature extraction methods.
The agent-structure problem is the question about whether systems that behave like agents necessarily have an internal structure that makes them agentic. This structure usually involves some search procedure.
Consent is a foundational concept in many practical systems of ethics (such as found in medicine).
do you still think these are possible to build/define/do you know about any relevant papers?
I have been exploring building something like this as an exploration into better-controllable default developer choices for agent programming primitives, as opposed to "assistant persona + completions API for everything".