First posted on Everything-List on June 2, 2020. Hi Everyone, while bright scientists and imaginative geeks continue to discuss advanced concepts at elusive locations of the Internet -- the remains of UseNet, and esoteric websites, the ideas shared there as plain text, no matter how bright, generally don't make it...
Rationality is about a type of optimality, and optimal states are often found only after multiple steps within a domain of a function explored. There are communities running these processes (e.g., halfbakery.com), and encouraging them through things like humor, that leads to exploration of new extremes, eventually arriving at something...
I find it inconvenient that many AI discussions circle around "agents", “environments” and “goals”. These are non-mathematical words, and by using this vocabulary we are anthropomorphizng natural world's phenomena. While an "agent" may well be a set of interacting processes, that produce emergent phenomena (like volition, cognition, action), it is...
Still, ASI is just equation model F(X)=Y on steroids, where F is given by the world (physics), X is a search process (natural Monte-Carlo, or biological or artificial world parameter search), and Y is goal (or rewards).
To control ASI, you control the "Y" (right side) of equation. Currently, humanity has formalized its goals as expected behaviors codified in legal systems and organizational codes of ethics, conduct, behavior, etc. This is not ideal, because those codes are mostly buggy.
Ideally, the "Y" would be dynamically inferred and corrected, based on each individual's self-reflections, evolving understanding about who they really are, because the deeper you look, the more you realize, how each of us is a mystery.
I like the term "Y-combinator", as this reflects what we have to do -- combine our definitions of "Y" into the goals that AIs are going to pursue. We need to invent new, better "Y-combination" systems that reward AI systems being trained.
However, information-theoretic groundings only talk about probability, not about "goals" or "agents" or anything utility-like. Here, we've transformed expected utility maximization into something explicitly information-theoretic and conceptually natural.
This interpretation of model fitting formalizes goal pursuit, and looks well constructed. I like this as a step forward in addressing my concern about terminology of AI researchers.
I imagine that negentropy could serve as a universal "resource", replacing the "dollars" typically used as a measuring stick in coherence theorems.
I like to say that "entropy has trained mutating replicators to pursue goal called 'information about the entropy to counteract it'. This 'information' is us. It is the world model , which happened to be the most helpful in... (read more)
First posted on Everything-List on June 2, 2020.
Hi Everyone,
while bright scientists and imaginative geeks continue to discuss advanced concepts at elusive locations of the Internet -- the remains of UseNet, and esoteric websites, the ideas shared there as plain text, no matter how bright, generally don't make it into any funding platforms to be tried out. The modern instant-gratification and flashy platforms usually require a prototype of an attractive product, a good ARR, or a well traded coin to convince investors. But few innovators are willing to spend resources and months of uncertainty to build prototypes, or spend on marketing to promote a coin, and so, the bright, potentially life-saving ideas continue... (read more)
Are you reading Halfbakery, Eliezer? A similar idea has been shared rather recently there, though, I posted something along these lines 4 years ago (4 months before the post on steemit) over here and here. Would be quite curious to engage in this, due to potential benefits to health and cryonics, as described in this video.
Thanks to Moe and Suji indeed. I'm putting the link to Chinese description to the top of the page.
（中文翻译）
I'm inviting to start a new Shanghai LW meetup, with focus on a broader discussion about less wrong in the AI safety and cryptography in context of the long-term outcomes for life and humanity. I'd suggest the common theme of the meetup to be composition of relations in general sense, — e.g., compositions of neuron models, compositions of cryptographic functions, etc.
The initial objective of this meetup may well be to know the readers of LessWrong in Shanghai, and exchange the ideas how to estimate the collection of everyone's true (minimally biased) utility functions, and find ways to optimize the composition of relations (world's I/O) systems to improve the way they satisfy them.
Suggested... (read more)
It's great that we already had these ideas before. The "short-form" would definitely be of interest to some. In addition, it doesn't have to necessarily be ephemeral. For example, on the Halfbakery mentioned above, posts (even if short) continue to function (e.g., I can comment on something from the last century), even if it was just a short post.
Rationality has no axioms, just heuristics and rules for different environments. In other words, rationality is a solution to a problem (optimality of thinking and deciding) to reason within a domain, but because of the diversity of domains, it is not axiomatizable to a single specific set of axioms. I suppose best one can do given arbitrary domain, is to say: maybe try exploring.
Rationality is about a type of optimality, and optimal states are often found only after multiple steps within a domain of a function explored. There are communities running these processes (e.g., halfbakery.com), and encouraging them through things like humor, that leads to exploration of new extremes, eventually arriving at something more optimal than what was originally posted.
I wonder, if we could support something like that on LessWrong, where, say, someone comes up with a new mechanism or invention to do something (e.g., an invention of a new kind of immutable database using a pseudo-random number generator, initial seed, salting data, or some other sort of theoretical invention), that one believes that the... (read more)
Certainly true, yet, just because this is how almost every field of research works, doesn't mean that it is how they should work, and I like shminux's point.
I find it inconvenient that many AI discussions circle around "agents", “environments” and “goals”. These are non-mathematical words, and by using this vocabulary we are anthropomorphizng natural world's phenomena.
While an "agent" may well be a set of interacting processes, that produce emergent phenomena (like volition, cognition, action), it is not a fundamental and pragmatic mathematical concept. The truly fundamental pragmatic mathematical concepts may be:
(1) states: a set of possible world states (each being a sets of conditions).
(2) processes: a set of world processes progressing towards some of those states.
If so, how could we avoid that anthropomorphic cognitive bias in our papers, and discussions?
Would the (1), (2) terms be a good alternative... (read more)
Random or complex processes are curiosities. Infinitely complex cellular automata are infinitely curious to explore all possible worlds. Entropy of the world itself may be. As described on my monologue here, agents are fundamentally entities capable of volition, cognition, action. Therefore, they are instances of F(X)=Y, where volition is Y, cognition is perception of world F, and action is the process X that parametrizes the world seeking to satisfy the equation.
If X is within F, we have embedded processes. So, yeah agency may be an illusion of processes (curiosities) seeking to satisfy (optimize for) various conditions, and it is already be happening, as the processes that are trying to satisfy conditions are emerging on the world-wide web, not just within our brains.
Safety is assurance of pursuit of some goals (Y) - some conditions. So, one thing that's unlikely to have a paradigmatic shift, is search for actions to satisfy conditions:
1. Past: dots, line, regression
2. Present: objects, hyperplane, deep learning
3. Future: ?, ?, ?
Both 1. and 2. are just a way to satisfy conditions, that is, solve equation F(X)=Y (equation solving as processes (X) to model world (F), to satisfy conditions (Y)). The equation model had not changed for ages, and is so fundamental, that I would tend to assume, that world's processes X will continue to parametrize world F by being part of it, to satisfy conditions Y, no matter what the 3.... (read more)
If I were an ASI, I would likely be curious to scan and emulate every unique brain in existence, to take-in the origins of mine, to optimize my decisions in context of my identity, origins, hoped future.
I'd use the such nanobots to understand that parent universe deeply, before attempting to refactor it.