Archetypal Transfer Learning (ATL) is a proposal by @whitehatStoic for what is argued by the author to be a fine tuning approach that "uses archetypal data" to "embed Synthetic Archetypes". These Synthetic Archetypes are derived from patterns that models assimilate from archetypal data, such as artificial stories. The method yielded a shutdown activation rate of 57.33% in the GPT-2-XL model after fine-tuning. .. (read more)
AI Evaluations focus on experimentally assessing the capabilities, safety, and alignment of advanced AI systems. These evaluations can be divided into two main categories: behavioral and understanding-based... (read more)
Encultured AI is a for-profit public benefit corporation working to make AI safer and healthier for human beings... (read more)
Poor Air Quality can reduce cognitive functioning[1], lifespans[2] and the techniques to improve air quality are also useful for getting rid of aerosolized respiratory pathogens. Improving air quality can be an impactful global health intervention.[3] Many members of the LessWrong community have also put effort into improving the air quality of their own homes or offices, as an implication of instrumental rationality... (read more)
Bureaucracy.. (read more)
| User | Post Title | Wikitag | Pow | When | Vote |
Hiding information within seemingly innocuous content.
In AI, it refers to models encoding hidden messages in their outputs, potentially enabling covert communication between models or evading human oversight through being Opaque.
Embedded Agency is the problem that an understanding of the theory of rational agents must account for the fact that the agents we create (and we ourselves) are inside the world or universe we are trying to affect, and not separated from it. This is in contrast with much current basic theory of AI or Rationality (such as Solomonoff induction or Bayesianism) which implicitly supposes a separation between the agent and the-things-the-agent-has-beliefs about. In other words, agents in this universe do not have Cartesian or dualistic boundaries like much of philosophy assumes, and are instead reductionist, that is agents are made up of non-agent parts like bits and atoms. This includes the question of the Boundary of the embedded agent, i.e. the questions which parts an agent considers part of itself and how it delineates other agents (or itself) from its observations.
re "The"The Axiom of Choice is necessary to select a set from an infinite number of pairs of socks, but not an infinite number of pairs of shoes." — Bertrand Russell, Introduction to mathematical philosophy
The axiom states that a certain kind of function, called a `choice' function, always exists. It is called a choice function, because, given a collection of non-empty sets, the function 'chooses' a single element from each of the sets. It is a powerful and useful axiom, asserting the existence of useful mathematical structures (such as bases for vector spaces of arbitrary dimension, and ultraproducts). It is a generally accepted axiom, and is in wide use by mathematicians. In fact, according to Elliott Mendelson in Introduction to Mathematical Logic (1964) "The status of the Axiom of Choice has become less controversial in recent years. To most mathematicians it seems quite plausible and it has so many important applications in practically all branches of mathematics that not to accept it would seem to be a wilfulwillful hobbling of the practicing mathematician. "
This is the statmentstatement that taking the mathematical product of non-empty sets will always yield a non-empty set. Consider infinitely many sets
A suprisingly good video explanation in laymanslayman's terms by Vsauce can be found Here.
This theorem is the main counterexample used as evidence of the falsehood of the Axiom of Choice. If not taken as evidence of its falsehood, thsithis is at least used as evidence of the counter-intuitiveness of AC.
Yet, mathematicians do disagree about this axiom, and whether it applies to our world as well. Some mathematicians take a Platonic view of mathematics, in which mathematical objects such as sets actually exist in some abstract realm, and for which the axiom of choice is either true or false, but we do not know which. Others take a highly constructive view (in many cases motivated by realism and the ability of the mathematics to model the world) in which either the axiom of choice is false, or infinite sets do not exist in which case the axiom of choice is provable and hence superfluous. Others take the view that the axiom is not true or false, but merely useless, and that anything provable from it is meaningless. Many seem not to care: the axiom is convenient for the mathematics they wish to do (whose application they are not...
Let OoP()UEA
However, Wei Dai noticed a problem with UDT 1.0, as detailed in Explicit Optimization of Global Strategy (Fixing a Bug in UDT1). In some cases, UDT 1.0 fails to coordinate with itself. The fix is as follows, with π∈Π
Getting up early to get a good start on the day seems appealing the previous evening, but when the alarm rings, the relative reward of sleeping in another few minutes is larger.
We can more rigorously define dynamic inconsistency as follows:
If the agent is given the opportunity to commit to a decision early, there are cases where it strictly prefers a different choice than the one it would make in-the-moment.
In Counterfactual Mugging, we understand Omega as "making a copy" of the agent at some point in time (EG, taking a detailed scan for use in a simulation). If a CDT agent is given the opportunity to commit to a decision in Counterfactual Mugging before this point in time, then it will think of the simulation as being downstream of its decision, so it will make the same decision as UDT. If a CDT agent is asked after the copy is made, but before the coin-flip result is revealed, then the CDT agent will decide to refuse, just like it does after the coin-flip is revealed.
The analysis for EDT agents is a little bit simpler. If you ask the EDT agent to commit to a decision before the coin-flip is revealed, then it will agree to give up the money just like UDT would, since it sees the cost as smaller than the potential payout, and always thinks of its decision as strongly correlated with Omega's simulation.[6] If the EDT agent is asked after the coin-flip, then EDT refuses.
This supposes that the agent isn't doing any anthropic reasoning; specifically, they don't put any probability on the possibility that they themselves are the agent inside Omega's hypothetical simulation. If the agent reasons anthropically, CDT and EDT can reach the same conclusion as UDT.
There are several different theories of causality, meaning there is a different version of CDT for each alternative.
In contrast, CDT chooses the following:
choice(o):=argmaxa∈AEP(U|do(choice(o)=a),O=o)
The only difference between this and CDT is that the "do" operator has been wrapped around the choice of action. This notation says that the action is an "intervention" rather than a normal Bayesian conditioning. In the context of a causal Bayesian network, we cut the parents of variables with a "do" before we make inferences (again assuming Judea...
A dual operations is to ask what an f reports (eliciting latent knowledge).
Embedded Agency is the problem that an understanding of the theory of rational agents must account for the fact that the agents we create (and we ourselves) are inside the world or universe we are trying to affect, and not separated from it. This is in contrast with much current basic theory of AI or Rationality (such as Solomonoff induction or Bayesianism) which implicitly supposes a separation between the agent and the-things-the-agent-has-beliefs about. In other words, agents in this universe do not have Cartesian or dualistic boundaries like much of philosophy assumes, and are instead reductionist, that is agents are made up of non-agent parts like bits and atoms. This includes the question of the Boundary of the embedded agent, i.e. the questions which parts an agent considers part of itself and how it delineates other agents from its observations.
Hiding information within seemingly innocuous content.
In AI, it refers to models encoding hidden messages in their outputs, potentially enabling covert communication between models or evading human oversight through being Opaque.
The pointers problem is about figuring out f.Value learning is an approach to learning f from humans.
Even if we can find f for human values, we would still have to build a system that acts on it and is robust to optimization pressure (Goodhart's law). And these two can fail independently of identifying f.
Agentic Misaligment
Could the Axiom of Choice, like the Axiom of Parallel lines, have physical implications? If it is false in our universe, then perhaps the Copenhagen interpretation of quantum mechanics must be false, as it involves a notion of true randomness. The axiom of choice is the only axiom of set theory which is incompatible with a constructivist approach to mathematics in which proving that anything exists requires an algorithm for its construction, which suggests that it is impossible to prove the existence of a randomly selected set. In such a universe, even selecting a single arbitrary element from an infinite set without an algorithm might be impossible, although proving that one exists within Z F does not rely upon the Axiom of Choice. This would seem to be incompatible with a particle collapsing into a partially random position.
Example 2: Suppose a funder, worried about the prospect of advanced AIs wiping out humanity, startingstarted offering grants for "AI safety". It may then be tempting to try to write papers that you know you can finish, like a paper on robotic cars causing unemployment in the trucking industry, or a paper on who holds legal liability when a factory machine crushes a worker. These have the advantage of being much less difficult problems than those involved in making something actually smarter than you be safe.
Achieving total corrigibility everywhereeverywhere, i.e., including in the correction mechanism itself, without manipulating its correctors, and preserving correction through self-modification, via some single, general mental state in which the AI "knows that it is still under construction" or "believes that the programmers know more than it does about its own goals" is termed 'the hard problem of corrigibility'.
btw, for the wiki generally, I think the "edited by Eliezer" thing should be changed, otherwise it can be interpreted as these words largely being endorsed by or written by him