This is a special post for quick takes by Q Home. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Problem: there's not enough people discussing (non-prosaic) alignment ideas.
Solution: LessWrong should pay people to do it.
Below I'll get more in depth about the problem and the solution(s).
Alignment Newsletter
Rohin Shah was running the Alignment Newsletter. It was good for many reasons:
- Rohin was helping to explain and spread ideas.
- Rohin contributed to the discussion.
- Rohin was a person who knew stuff from different alignment subfields (having such people is good).
But Rohin stopped and now nobody does what he did. That's bad!
Even if he didn't stop, having multiple people doing the kind of things he did would be good.
Review
LW has a reviewer, Justis. He's pretty great, but has two flaws:
- He's just one guy.
- He's not very familiar with alignment stuff.
That's bad (for a single non-expert to do all of the reviewing)! Hire more reviewers / increase Justis' pay to make him more knowledgeable about alingment.
A similar criticism was made by Roman Leventov, about the review system basically not existing.
LessWrong
Even when you look at posts (with specific ideas/proposals) of high-status researchers, the engagement with those posts is often pretty "meh". Example 1, example 2, example 3 (the commen... (read more)
3Gunnar_Zarncke
Hi, would you say that I might be a guy matching your requirements who knows many alignment subfields, follows AI and AI safety news, is currently pivoting into full-time AI safety work, and has some standing in this community?
3Q Home
Yes, you most likely fit the bill. Though I didn't want to put forth any specific requirements, I just meant to say "if you worry about paying the wrong people, there are people who have already proved themselves and did those kinds of things (giving feedback, engaging with different alignment/philosophy ideas, etc)".
3Gunnar_Zarncke
How can I help?
1Q Home
I'd really appreciate feedback on a draft or just chatting about an alignment idea. (But not now, some time in the coming months. Will message you.) Thanks for suggesting help. Feel free to ask me for feedback on your stuff too, I like to engage with alignment ideas.
2Cameron D.
Wholeheartedly agree. These proposals seem like they have a number of benefits including:
1. The intended direct benefit of advancing the alignment discussion through better organization, communication, debate, review, synthesis, etc.
2. Doing some of these activities well (communication, synthesis) plausibly forces better conceptual understanding of the problem, advancing research in a roundabout way.
3. It's hard to get a job directly working on the hard problems or alignment, mostly because it takes an incredible amount of talent to make actual progress. On the other hand there are plenty of jobs in prosaic alignment. This plausibly leads to motivated reasoning about the difficulty of the problem. Paying people to do the above activities could alleviate this imbalance.
4. This kind of work seems to be the thing most adjacent to doing actual alignment work, which could make it a nice way to train and identify talent.
LW is great epistemic infrastructure for making progress on alignment, but it seems like way more can be done. If we ever want alignment to move out of it's pre-paradigmatic stage, this seems like a necessary (although not sufficient) condition.
I'm an independent alignment researcher in Russia. Imagine someone wants to donate money to me (from Europe/UK/America/etc). How can I receive the money? It's really crucial for me to receive at least 100$ per month, at least for a couple of months. Even 40$ per month would be a small relief. [EDIT: my latest, most well-received, and only published alignment research is here.]
Down below are all the methods I learned about after asking people, Google, Youtube and LLMs:
- Crypto. The best option. But currently I'm a noob at crypto.
- There are some official ways to get money into Russia, but the bank can freeze those transfers or start an inquiry.
- Some freelance artists use Boosty (Patreon-like site). But Boosty can stall money transfers for months and more. If your account doesn't have subscribers and legitimate content, it can rise suspicion of the site.
- Someone from a 'friendly country' or from Russia itself could act as an intermediary. The last link on this page refers to a network of russian alignment researchers. (Additional challenge: I don't have a smartphone. With a dumbphone it's not practically possible to register in Telegram. But most russian alignment researchers are there.)
- Ge
8ProgramCrafter
1. Re: crypto. Please look carefully at federal law No. 259, article 14, point 5.
"<... corporate body, companies' branches...>, individuals residing in the Russian Federation at least 183 days in twelve consecutive months are not allowed to accept digital currency as a counter-provision for goods, work, services and/or something which could be construed as paying for goods (work, services) in digital currency, <... except mining>" (translated in my own words).
If crypto you choose meets definition of digital currency, you need to tread carefully.
2. "There are some official ways to get money into Russia, but the bank can freeze those transfers or start an inquiry." True except I'd say "any bank in the actual transfer chain, which happens to include non-Russian banks too".
3. Perhaps your research would be of interest to some people if you published it? Though this might require restructuring it, and I'm not familiar with details of your work.
These additional suggestions might not work in specific situations; not intended to offend in any case.
6. Get a half-time job.
7. Get acquainted with, and marry into, a upper mid-class family.
8. Get a grant (important: from Russia-aligned sources; if you qualify as youth or another class, try leveraging that).
9. Weren't there legal ways to have a contract with non-residents which can pay in whatever currency? That, I don't know at the moment.
[...]
by both sides, to be precise
[...]
This message is pretty much sufficient! To be better, it should also include some demonstration of what you will be working on.
1Q Home
Thanks a lot for willingness to go into details. And for giving advice on messaging other researchers.
No offense taken. The marriage option was funny, hope I never get that desperate. Getting official grants is probably not possible for me, but thanks for the suggestion.
[...]
My wording was deliberate. It's one thing to sanction another country, and another thing to "sanction yourself".
4Yaroslav Granowski
Fellow russian researcher here. I doubt small donations can work. Not many people use crypto (I'm same noob and not sure if it's worthy to learn it until I have big audience.) I tried to register with OpenCollective so that they would collect money for me before I leave Russia but they rejected despite me having a project to showcase.
If you have something to show, you could try applying for research grants. This is what I'm trying to do. But that only makes sense if you have time and not so desperate about nearest future.
3samuelshadrach
Actually persuading someone to donate to you is harder for most people than figuring out how to use cryptocurrency.
Using crypto is not that hard in the average case. The main habits you need to get into are a) verify everything, 90% of all the services and platforms are scams b) no mistakes when dealing with large sums, practice with small sum first. one mistake can lose all your money
2Alaric
Maybe you can find a freelance job? For example, people who can do alignment research usually know how to write code. Or maybe you could give private lessons (math or computer science)?
Yes, you should spend time on this, but I think it could make your situation more stable than donations.
1Q Home
Reasonable advice. Sadly, I can't do math or programming. My work is philosophical/conceptual/informal in the way some Arbital articles are informal. In terms of teaching, maybe I could teach chess. I play somewhat good (2100+ in different time formats on Lichess and Chess. com).
Give AGI humanlike reasoning? (draft of a post)
Alignment plans can be split into two types:
Usual plans. AI gains capabilities C. We figure out how to point C to T (alignment target). There's no deep connection between C and T. One thing is mounted onto the other.
HRLM plans. We give AI special C, with a deep connection to T.
HRLM is the idea that there's some special reasoning/learning method which is crucial for alignment or makes it fundamentally easier. HRLM means "humanlike reasoning/learning method" or "special, human-endorsed reasoning/learning method". There's no hard line separating the two types of plans. It's a matter of degree.
I believe HRLM is ~never discussed in full generality and ~never discussed from a theoretical POV. This is a small post where I want to highlight the idea and facilitate discussion, not make a strong case for it.
Examples of HRLM
(My description of other people's work is not endorsed by them.)
Corrigibility. "Corrigible cognition" is a hypothetical, special type of self-reflection (C) which is extremely well-suited for learning human values/desires (T).
In "Don't align agents to evaluations of plans" Alex... (read more)
6Seth Herd
I just want to note that humans aren't aligned by default, so creating human-like reasoning and learning is not itself an alignment method. It's just a different variant of providing capabilities, which you separately need to point at an alignment target.
It may or may not be easier to align than alternatives. I personally don't think this matters because I strongly believe that the only type of AGI worth aligning is the type(s) most likely to be developed first. Hoping that the indurstry and society is going to make major changes to AGI development based on which types the researchers think are easier to align seems like a forlorn hope.
More on why it's a mistake to assume human-like cognition in itself leads to alignment:
Sociopaths/psychopaths are a particularly vivid example of how humans are misaligned. And there are good reasons to think that they are not a special case in which empathy was accidentally left out or deliberately blocked, but that they are the baseline human cognition without the mechanisms that create empathy. It's tough to make this case for certain, but it's a very bad idea to assume that humans are aligned by default, so all we've got to do is reproduce human-like cognitive mechanisms and maybe train it "in a good family" or similar.
That's not to argue against human-like approaches to AGI as worse for alignment, just to say that they're only better in that we have a little better understanding of that type of cognition and some mechanisms by which humans often wind up approximately aligned in common contexts.
My own research is also in using loosely human-like reasoning and learning as a route to alignment, but that's primarily because a) that's my background expertise so it's my relative advantage and b) I think LLMs are very loosely like some parts of the human brain/mind, and that we'll see continued expansion of LLM agents to reason in more loosely human-like ways (that is, with chains of thought, specific memory looks ups, metacog
1Q Home
Yes, it could be that "special, inherently more alignable cognition" doesn't exist or can't be discovered by mere mortal humans. It could be that humanlike reasoning isn't inherently more alignable. Finally, it could be that we can't afford to study it because the dominating paradigm is different. Also, I realize that glass box AI is a pipe dream.
Wrt sociopaths/psychopaths. I'm approaching it from a more theoretical standpoint. If I knew a method of building a psychopath AI (caring about something selfish, e.g. gaining money or fame or social power or new knowledge or even paperclips) and knew the core reasons of why it works, I would consider it a major progress. Because it would solve many alignment subproblems, such as ontology identification and subsystems alignment.
4Caleb Biddulph
I think MONA (which I worked on) counts as another example of this. Basically, you can make your agent only care about short-term feedback from a trusted model that imitates humans, so it isn't motivated to pursue long-term plans that a human wouldn't endorse.
I'm also doing an HRLM-like thing for my MATS project. The very high-level idea is to start with a lightly fine-tuned language model, which we assume is aligned/humanlike.[1] Then we get the model to reason in a humanlike way about how to improve its own performance. This is supposed to be much more interpretable and aligned than RL, which modifies the underlying model in inscrutable ways to maximize reward by any means possible.
You might even say that "humanlike reasoning and learning" describes my personal research agenda. I'd be excited for people to talk about this sort of thing more!
Interestingly, both of the methods I mentioned above actually do something different from the dichotomy you described. Rather than "gaining capabilities, then aligning to a target" (usual plans) or "simultaneously gaining capabilities and aligning to a target in a deeply connected way" (HRLM plans), these methods could be described as "aligning to a target, then gaining capabilities."[2]
Btw, the thing that HRLM stands for (Humanlike Reasoning/Learning Method) was a bit buried and at first I thought you didn't mention it at all. It'd be easier to see if you capitalized it and moved it forward a bit
1. ^
This is not a totally safe assumption. For example, LLMs can easily be jailbroken to act very unhumanlike, and even the standard "assistant character" knows a lot of information that a random human would not. As you pointed out, LLMs are pretty alien. But hopefully in practice, the assumption is mostly true in the ways that matter.
2. ^
Technically, we'd be doing the HRLM thing first, during LLM pretraining and fine-tuning, since in the process of getting aligned to humans, the model must gain some b
1Q Home
I'm approaching it from a "theoretical" perspective[1], so I want to know how "humanlike reasoning" could be defined (beyond "here's some trusted model which somehow imitates human judgement") or why human-approved capability gain preserves alignment (like, what's the core reason, what makes human judgement good?). So my biggest worry is not that the assumption might be false, but that the assumption is barely understood on the theoretical level.
What are your research interests? Are you interested in defining what "explanation" means (or at least in defining some properties/principles of explanations)? Typical LLM stuff is highly empirical, but I'm kinda following the pipe dream of glass box AI.
1. ^
I'm contrasting theoretical and empirical approaches.
Empirical - "this is likely to work, based on evidence". Theoretical - "this should work, based on math / logic / philosophy".
Empirical - "if we can operationalize X for making experiments, we don't need to look for a deeper definition". Theoretical - "we need to look for a deeper definition anyway".
2Caleb Biddulph
I wouldn't describe my research as super theoretical, but it does involve me making arguments (although not formal proofs) about why I expect my plans to work even as training continues.
For example, it seems like in some non-formalized sense, a human is trivially aligned to themselves. The more similar your AI's behavior is to a particular human's behavior, the more aligned it is to that human. If you want to align the AI to a group of humans (e.g. all of humanity), you might want to start by emulating a good approximation of all humans and bootstrapping from there. I'm not working on this aspect of the problem directly - I'm just assuming that the LLM is pretty humanlike to begin with - but I wrote a post talking about similar ideas.
(draft of a future post)
I want to share my model of intelligence and research. You won't agree with it at the first glance. Or at the third glance. (My hope is that you will just give up and agree at the 20th glance.)
But that's supposed to be good: it means the model is original and brave enough to make risky statements.
In this model any difference in "intelligence levels" or any difference between two minds in general boils down to "commitment level".
What is "commitment"?
On some level, "commitment" is just a word. It's not needed to define the ideas I'm going to talk about. What's much more important is the three levels of commitment. There are often three levels which follow the same pattern, the same outline:
Level 1. You explore a single possibility.
Level 2. You want to explore all possibilities. But you are paralyzed by the amount of possibilities. At this level you are interested in qualities of possibilities. You classify possibilities and types of possibilities.
Level 3. You explore all possibilities through a single possibility. At this level you are interested in dynamics of moving through the possibility space. You classify implications of possibilities.
...
I'm going to g... (read more)
2Q Home
*A more "formal" version of the draft (it's a work in progress): *
There are two interpretations of this post, weak and strong.
Weak interpretation:
I describe a framework about "thee levels of exploration". I use the framework to introduce some of my ideas. I hope that the framework will give more context to my ideas, making them more understandable. I simply want to find people who are interested in exploring ideas. Exploring just for the sake of exploring or for a specific goal.
Strong interpretation:
I use the framework as a model of intelligence. I claim that any property of intelligence boils down to the "three levels of exploration". Any talent, any skill. The model is supposed to be "self-evident" because of its simplicity, it's not based on direct analysis of famous smart people.
Take the strong interpretation with a lot of grains of salt, of course, because I'm not an established thinker and I haven't achieved anything intellectual. I just thought "hey, this is a funny little simple idea, what if all intelligence works like this?", that's all.
That said, I'll need to make a couple of extraordinary claims "from inside the framework" (i.e. assuming it's 100% correct and 100% useful). Just because that's in the spirit of the idea. Just because it allows to explore the idea to its logical conclusion. Definitely not because I'm a crazy man. You can treat the most outlandish claims as sci-fi ideas.
A formula of thinking?
Can you "reduce" thinking to a single formula? (Sounds like cringe and crackpottery!)
Can you show a single path of the best and fastest thinking?
Well, there's an entire class of ideas which attempt to do this in different fields, especially the first idea:
* Bayesian epistemology: "epistemology in a single rule" (the rule of updating beliefs)
* Utilitarianism, preference utilitarianism: "(meta-)ethics in a single rule"
* Baconian method, the prototype of the scientific method: "science in a single rule"
* Hegelian dialectic
Many of my current research interests are about "unification" or "molecules" (by "molecules" I mean something like semantic molecules in semantic molecularism).
"What unifies computations about X into 'understanding' of X?"
"What unifies computations into an ontology?"
"What unifies computations into a mind?"
"What unifies specific intellectual skills into general intellectual skills?"
"What unifies computations about a word into the 'meaning' of the word?"
Below are some thought experiments and observations related to those questions.
Understanding
Imagine you do... (read more)
Analyzing abstractions, semantics and human language is a sorta famous research direction in alignment. (The researchers are John Wentworth & David Lorell & Gretta Duleba, Thane Rutheins, Jan Kulveit, Sam Eisenstat.)
So I wrote a draft where I tried to
- classify ALL interpretable types of human abstractions,
- explain where the usefulness of different abstraction types comes from,
- explain how polysemantic words work (I analyzed "run", "set" and "put").
The idea is to give a fully general definition of what it means for a set of abstractions/computations to... (read more)
A couple of chess observations related to epistemology and alignment.
Queen Odds
Take the starting chess position. Remove the black queen.
rnb1kbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
https://lichess.org/analysis/rnb1kbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR_w_KQkq_-_0_1
It's overwhelmingly likely that White is winning.
But just how sure should we be of that, epistemically? We don't have a mathematical proof yet.
Even more absurd question: how sure should we be that Black is not winning? There's no mathematical proof of that either.
I claim that every ed... (read more)
Epistemic status: Draft of a post. I want to propose a method of learning environmental goals (a super big, super important subproblem in Alignment). It's informal, so has a lot of gaps. I worry I missed something obvious, rendering my argument completely meaningless. I asked LessWrong feedback team, but they couldn't get someone knowledgeable enough to take a look.
Can you tell me the biggest conceptual problems of my method? Can you tell me if agent foundations researchers are aware of this method or not?
If you're not familiar with the problem, here's the... (read more)
I have a couple of silly, absurd questions related to mesa-optimizers and mesa-controllers. I'm asking them to get a fresh look on the problem of inner alignment. I want to get a better grip on what basic properties of a model make it safe.
Question 1. How do we know that Quantum Mechanics theory is not plotting to kill humanity?
It's a model, so it could be unsafe just like an AI.
- QM is not an agent, but its predictions strongly affect humanity. Oracles can be dangerous.
- QM is highly interpretable, so we can check that it's not doing internal search. Or ca
There's an alignment-related problem, the problem of defining real objects. Relevant topics: environmental goals; task identification problem; "look where I'm pointing, not at my finger"; The Pointers Problem; Eliciting Latent Knowledge.
I think I realized how people go from caring about sensory data to caring about real objects. But I need help with figuring out how to capitalize on the idea.
So... how do humans do it?
- Humans create very small models for predicting very small/basic aspects of sensory input (mini-models).
- Humans use mini-models as puzzle
6[anonymous]
Another highly relevant post: The Pointers Problem.
2Vladimir_Nesov
Creating an inhumanly good model of a human is related to formulating their preferences. A model captures many possibilities and the way many hypothetical things are simulated in the training data. Thus it's a step towards eliminating path-dependence of particular life stories (and preferences they motivate), by considering these possibilities altogether. Even if some on the possible life stories interact with distortionary influences, others remain untouched, and so must continue deciding their own path, for there are no external influences there and they are the final authority for what counts as aiding them anyway.
1Q Home
How does this relate to my idea? I'm not talking about figuring out human preferences.
[...]
What is "path-dependence of particular life stories"?
[...]
Are there other ways to characterize objects? Feels like a very general (or even fully general) framework. I believe my idea can be framed like this, too.
2Vladimir_Nesov
Models or real objects or things capture something that is not literally present in the world. The world contains shadows of these things, and the most straightforward way of finding models is by looking at the shadows and learning from them. Hypotheses is another toy example.
One of the features of models/things seems to be how they capture the many possibilities of a system simultaneously, rather than isolated particular possibilities. So what I gestured at was that when considering models of humans, the real objects or models behind a human capture the many possibilities of the way that human could be, rather than only the actuality of how they actually are. And this seems useful for figuring out their preferences.
Path-dependence is the way outcomes depend on the path that was taken to reach them. A path-independent outcome is convergent, it's always the same destination regardless of the path that was taken. Human preferences seem to be path dependent on human timescales, growing up in Egypt may lead to a persistently different mindset from the same human growing up in Canada.
1Q Home
I see. But I'm not talking about figuring out human preferences, I'm talking about finding world-models in which real objects (such as "strawberries" or "chairs") can be identified. Sorry if it wasn't clear in my original message because I mentioned "caring".
[...]
You might need to specify what you mean a little bit.
The most straightforward way of finding a world-model is just predicting your sensory input. But then you're not guaranteed to get a model in which something corresponding to "real objects" can be easily identified. That's one of the main reasons why ELK is hard, I believe: in an arbitrary world-model, "Human Simulator" can be much simpler than "Direct Translator".
So how do humans get world-models in which something corresponding to "real objects" can be easily identified? My theory is in the original message. Note that the idea is not just "predict sensory input", it has an additional twist.
3Vladimir_Nesov
My point is that chairs and humans can be considered in a similar way.
[...]
There's the world as a whole that generates observations, and particular objects on their own. A model that cares about individual objects needs to consider them separately from the world. The same object in a different world/situation should still make sense, so there are many possibilities for the way an object can be when placed in some context and allowed to develop. This can be useful for modularity, but also for formulating properties of particular objects, in a way that doesn't get distorted by the influence of the rest of the world. Human preferences is one such property.
1Q Home
Please explain how your point connects to my original message: are you arguing with it or supporting it or want to learn how my idea applies to something?
Do LW people care about chess / checkers / Go / etc. skill or is it a kindergarten-level brag?
In general, how many LW people play chess or other sequential games with perfect information? React with googly eyes (👀) if you play chess / checkers / Go / etc.
(Personally, I'm 2350+ in bullet - normal chess - and 2100+ in Crazyhouse on Lichess. On a good week.)
Also, while we are at it, are you interested in those games? I'm absolutely fascinated by chess because it's like a mathematical problem, yet it's filled with semantics. I.e. it's very similar to the real... (read more)
2marquis_de_sod
I’m not very interested in chess, mainly because I don‘t find games with a high degree of memorization and possibility for theoretically optimal play to be interesting. Maybe I’m over-generalizing but I prefer more social/bluffing games where it’s possible to win at a disadvantage with mind games. Always liked card games like poker and MTG more though I don’t play regularly any more. I recognize the intelligence and skill of top chess players but personally find it boring.
Does anyone remember Eliezer talking about a (thought) experiment where a monkey has to connect two sticks into a larger stick to reach a banana? Or maybe it wasn't Eliezer but someone else rat-adjacent?
I could've sworn I read a passing mention of this (thought) experiment in EY's writing. But for some reason I barely even found any mention of monkeys/apes/chimps, bananas or sticks. LLM didn't help.
The ultimate source is from Wolfgang Köhler, but ofc I didn't learn it there.
... (read more)One notable experiment involved a chimpanzee named Sultan, who was given two sticks
For some time I wanted to apply the idea of probabilistic thinking (used for predicting things) to describing things, making analogies between things. This is important because your hypotheses (predictions) depend on the way you see the world. If you could combine predicting and describing into a single process, you would unify cognition.
Fuzzy logic and fuzzy sets is one way to do it. The idea is that something can be partially true (e.g. "humans are ethical" is somewhat true) or partially belong to a class (e.g. a dog is somewhat like a human, but not 100... (read more)
1Q Home
Argumentation, hypotheses
You can apply the same idea (about the "common pool") to hypotheses and argumentation:
* You can describe a hypothesis in terms of any other hypothesis. You also can simplify it along the way (let's call it "regularization"). Recursion and circularity is possible in reasoning.
* Truth isn't attached to a specific hypothesis. Instead there's a common "pool of truth". Different hypotheses take different parts of the whole truth. The question isn't "Is the hypothesis true?", the question is "How true is the hypothesis compared to others?" And if the hypotheses are regularized it can't be too wrong.
* Alternatively: "implications" of a specific hypothesis aren't attached to it. Instead there's a common "pool of implications". Different hypotheses take different parts of "implications".
* Conservation of implications: if implications of a hypothesis are simple enough, they remain true/likely even if the hypothesis is wrong. You can shift the implications to a different hypothesis, but you're very unlikely to completely dissolve them.
* In usual rationality (hypotheses don't share truth) you try to get the most accurate opinions about every single thing in the world. You're "greedy". But in this approach (hypotheses do share truth) it doesn't matter how wrong you are about everything unless you're right about "the most important thing". But once you're proven right about "the most important thing", you know everything. A billion wrongs can make a right. Because any wrong opinion is correlated with the ultimate true opinion, the pool of the entire truth.
* You can't prove a hypothesis to be "too bad" because it would harm all other hypotheses. Because all hypotheses are correlated, created by each other. When you keep proving something wrong the harm to other hypotheses grows exponentially.
* Motivated reasoning is valid: truth of a hypothesis depends on context, on the range of interests you choose. Your choice affects the truth.
* Any
1Q Home
Crash Bandicoot 1
Crash Bandicoot N. Sane Trilogy
My ordering of some levels: image. Videos of the levels: Level 1, Level 2, Level 3, Level 4, Level 5, Level 6.
I used 2 metrics to evaluate the levels:
1. Is the level stretched vertically or horizontally?
2. Is the level easy to separate into similar square-like pieces or not? (like a patchwork)
The levels go from "vertical and separable" to "horizontal and not separable".
But to see this you need to note:
* Level 1 is very vertical: it's just a vertical wall. So it "takes away" verticality from levels 2 and 3.
* From levels 1-3, level 3 is the most horizontal. Because it's the least similar to the level 1.
* Levels 4-6 repeat the same logic, but now levels are harder to separate into similar square-like pieces. Why? Because levels 1 and 2 are very easy to separate (they have repeating patterns on the walls), so they "take away" separability from all other levels.
Any question about any property of any level is answered by another question: is this property already "occupied" by some other level?
Jacek Yerka
Jacek Yerka
Places in random order: image.
My ordering of places: image.
I used 2 metrics to evaluate the places:
1. Can the place fit inside a box-like space? (not too big, not too small)
2. Is the place inside or outside of something small?
The places go from "box-like and outside" to "not box-like and inside".
But to see this you need to note:
* Place 1 could be interpreted as being inside of a town. But similar Place 5 is inside a single road. So it takes away "inside-ness" from Place 1.
* Place 2 is more "outside" than it seems. Because similar Place 6 fits inside an area with small tiles. So it takes away "inside-ness" from Place 2.
* Place 3 is not so tall as it seems. Because similar Place 6 is very tall. So it takes away height from Place 3.
If you feel this relativity of places' properties, then you understand how I think about places. You don't need to understand a specifi
Stupid question: can ELK be brute forced?
Eliciting Latent Knowledge problem for the unfamiliar:
Suppose we train a model to predict what the future will look like according to cameras and other sensors. We then use planning algorithms to find a sequence of actions that lead to predicted futures that look good to us.
But some action sequences could tamper with the cameras so they show happy humans regardless of what’s really happening. More generally, some futures look great on camera but are actually catastrophically bad.
... (read more)In these cases, the predicti
Sorry if it's not appropriate for this site. But is anybody interested in chess research? I've seen that people here might be interested in chess. For example, here's a chess post barely related to AI.
Intro
In chess, what positions have the longest forced wins? "Mate in N" positions can be split into 3 types:
- Positions which use "tricks" to get a big number of moves before checkmate. Such as cycles of repeating moves. For example, this manmade mate in 415 (see the last position) uses obvious cycles. Not to mention mates in omega.
- Tablebase checkmates, di
"Everything is relative."
You know this phrase, right? But "relativity" is relative too. Maybe something is absolute.
But "relativity of relativity" is relative too. Maybe nothing is absolute after all... Those thoughts create an infinite tower of meta-levels.
If you think about the statement "truth = lie" ("you can go from T to F") you can get a similar tower. (Because it also implies "you can NOT go from T to F" and "you can go from "you can NOT go from T to F" to "you can go from T to F"" and so on.) It's not formal, but still interesting. Informally, the ... (read more)
Perhaps the alternative to the maximize one thing subject to a price to humans constraint would be not making the AI that specialized. Make it maximize across a basket of things humans want.
While I have heard the paper clips take over the universe worry it seems to be that type of thought experiment introduce the problem to begin with (making a bit of a circular type error). As I gather (indirectly) the problem is the paper clip maximizing AI end up taking over the entire economy. That seems to equivalent to suggesting the AI replaces all the markets... (read more)
Theoretical interpretability question.
Consider two models, A and B.
- Model A is 100% interpretable. It can predict what Bob says and what he thinks.[1] But sometimes it makes random-looking mistakes ("noise"). You care about predicting the noise, but you don't care about understanding the mechanism behind the noise. Ignoring the noise, model A is pretty perfect, capturing something like the platonic ideal of Bob, the necessary and sufficient conditions for thinking/talking like Bob.
- Model B is not interpretable. It predicts what Bob says and what he thinks, b
A question about natural latents.
- Imagine you have 100 blue boxes. Each time you roll the dice, their shape changes. But all 100 boxes always share the same shape. If I understand correctly, in this situation the shape is the natural latent. While color is just static background information.
- Imagine you have 100 boxes. Each time you roll the dice, their color changes. But all 100 boxes always share the same color. In this situation, color is the natural latent.
- Imagine you have 100 boxes. Each time you roll the dice, their color changes. If at least one box i
Have an idea about interpretability and defining search/optimization.
Trivial Property
Finite algorithms can solve infinite (classes of) problems. For example, the algorithm for adding two numbers has a finite description, yet can solve an infinity of examples.
This is a basic truth of computability theory.
Intuitively, it means that algorithms can exploit regularities in problems. But "regularity" here can only be defined tautologically (any smaller thing which solves/defines a bigger thing).
Less Trivial Property
Many algorithms have the following property:
- The
Draft of a future post, any feedback is welcome. Continuation of a thought from this shortform post.
(picture: https://en.wikipedia.org/wiki/Drawing_Hands)
The problem
There's an alignment-related problem: how do we make an AI care about causes of a particular sensory pattern? What are "causes" of a particular sensory pattern in the first place? You want the AI to differentiate between "putting a real strawberry on a plate" and "creating a perfect illusion of a strawberry on a plate", but what's the difference between doing real things and creating perfec... (read more)
I want to share a way of dissolving disagreements. It's also a style of thinking. I call it "the method of statements", here's the description:
-
Take an idea, theory or argument. Split it into statements of a certain type. (Or multiple types.)
-
Evaluate the properties of the statements. Do they exist (i.e. can they be defined, does anything connect them)? Can they be used, are they constructive? Are they simple? Are they important? Etc.
-
Try to extract as much information as possible from those statements.
One rule:
- A statement counts as existing eve
1Q Home
The whole draft is here. (and the newer one is here)
Edit: my latest draft should be here.
If you want to describe human values, you can use three fundamental types of statements (and mixes between the types). Maybe there're more types, but I know only those three:
- Statements about specific states of the world, specific actions. (Atomic statements)
- Statements about values. (Value statements)
- Statements about general properties of systems and tasks. (X statements)
Any of those types can describe unaligned values. So, any type of those statements still needs to be "charged" with values of humanity. I call a statement "true" if it's true for humans.
We ... (read more)
I want to discuss a particular failure mode of communication and thinking in general. I think it affects our thinking about AI Alignment too.
Communication. A person has a vague, but useful idea (P). This idea is applicable on one level of the problem. It sounds similar to another idea (T), applicable on a very different level of the problem. Because of the similarity nobody can understand the difference between (P) and (T). People end up overestimating the vagueness of (P) and not considering it. Because people aren't used to mapping ideas to "levels" of a... (read more)
1Q Home
Puzzles
You can use the same thinking to analyze/classify puzzles.
Inspired by Pirates of the Caribbean: Dead Man's Chest. Jack has a compass that can lead him to a thing he desires. Jack wants to find a key. Jack can have those experiences:
1. Experience of the real key.
2. Experience of a drawing of the key.
3. Pure desire for the key.
In order for compass to work Jack may need (almost) any mix of those: for example, maybe pure desire is enough for the compass to work. But maybe you need to mix pure desire with seeing at least a drawing of the key (so you have more of a picture of what you want).
* Gibbs: And whatever this key unlocks, inside there's something valuable. So, we're setting out to find whatever this key unlocks!
* Jack: No! If we don't have the key, we can't open whatever it is we don't have that it unlocks. So what purpose would be served in finding whatever need be unlocked, which we don't have, without first having found the key what unlocks it?
* Gibbs: So - We're going after this key!
* Jack: You're not making any sense at all.
* Gibbs: ???
Jack has those possibilities:
1. To go after the chest. Foolish: you can't open the chest.
2. To go after the key. Foolish: you can get caught by Davy Jones.
Gibbs thinks about doing 100% of 1 or 100% of 2 and gets confused when he learns that's not the plan. Jack thinks about 50% of 1 and 50% of 2: you can go after the chest in order to use it to get the key. Or you can go after the chest and the key "simultaneously" in order to keep Davy Jones distracted and torn between two things.
Braid, Puzzle 1 ("The Ground Beneath Her Feet"). You have two options:
1. Ignore the platform.
2. Move the platform.
You need 50% of 1 and 50% of 2: first you ignore the platform, then you move the platform... and rewind time to mix the options.
Braid, Puzzle 2 ("A Tingling"). You have the same two options:
1. Ignore the platform.
2. Move the platform.
Now you need 50% of 1 and 25% of 2: you need to r
(Drafts of a future post.) I want to confront/explain my optimism. Here's a thought experiment to explain what "optimism" means to me:
Imagine a world like Earth. There's an underground prison. People live there for generations. The prison is constructed in such a way that you can live there "forever". People are not aware that the world outside of the prison exists.
One person in the prison imagines freedom. But she doesn't have evidence (or so it seems).
- A: I got an idea: maybe we shouldn't optimize our life in prison. Maybe we can escape to freedom.
- B: "Fre
Imagine the classic paperclip maximizer thought experiment. We say AGI to make paperclips - AGI uses all matter of the Universe for it.
But now imagine a different version: we say AGI to "make paperclips that cost 1¢ (in human economy)". Now killing everyone isn't a solution: destroying humanity would destroy the economy.
Isn't it an interesting version of the thought experiment? Of course, everything can/will go wrong anyway, but maybe in a way funnier and more convoluted way. More funny and convoluted than "maximize human smiles", for example. Because AGI ... (read more)
6gwern
Seems to collapse easily: how does the AI decide what costs $0.01, exactly? Does it use the last price of a transaction on a market, and is doing mark-to-market? Well... among many other problems that occur to me, the most immediate one is that the price can't change if there aren't any more transactions, now can it. Nothing about 'make paperclips that would cost $0.01' would seem to rule out market manipulation, monopolization, or destruction. No market, no changes in the price you are marking to, no risk or volatility, no crash in prices due to oversupply, and enables efficient planning for the future and maximizing production of paperclips that would have cost $0.01 on what used to be Earth.
(The humorous fictional version of this story would involve 2 of the last survivors locating a sensor of the AI, building a large hollow paperclip-shaped human habitat, and loudly and ostentatiously in front of the sensor, having the 'owner' sell it to the other survivor for exactly $0.01, and then buying a regular paperclip for the smallest amount they can write on an IOU using Knuth up notation, thereby establishing new market prices.)
1Q Home
I think destruction of the market should be ruled out easily. Say paperclips have to have this value on an active market.
For manipulation, monopolization and "kill almost everyone and leave just a small market of 2 last survivors"... I have to make a post about this. I have a deeper idea (maybe) behind it than this particular example.
My general idea is this: I think when you hook up AI's rewards to a system that has to have certain properties, it leads to interesting effects and implications for Alignment. Because now the AI needs to care both about its rewards and also about the properties of the reward system. Many Alignment ideas implicitly try to achieve this anyway.
Instead of explaining "monopolization is bad" (complicated and specific fact) you need to explain "100% controlling your own reward system is bad" (easier and more universal fact).
[...]
I think some outcomes of paperclip maximization are qualitatively different from "everyone dies", even if they're still very bad. The outcomes in which AI has to leave at least some freedom/autonomy for humans (or some other system) are especially different. I think this is underexplored.
I think reformulating Alignment problem as "reward system control" problem at worst allows you to formulate all the same problems with a new angle and at best gives useful insight about the solution.
6gwern
Defining 'active market' sounds quite difficult. Is any kind of software-mediated trading, as opposed to humans thrusting arms into the air, like HFT trading of stocks, an 'active market'? Then fine, the AI creates agents which just wash-trades assets. (Better yet, it uses combinatorial markets to ensure bids/asks only execute that leave the price exactly the same or other such properties minimized/maximized/stabilized.)
1Q Home
To take a step back: do you see a potential conceptual distinction between my idea and classic paperclip maximization? (Of course, you don't have to see it and/or agree that there's one. And even if there's one in theory it doesn't mean it exists in practice.)
Yes, it's always hard to define the "true reward" AI should strive for. But properties of the system "true reward + AI" may be easier to define.
[...]
If AI is able to reason/learn about properties of reward systems, then AI should be able to infer that taking 100% control over the reward system is a hack. Not something that can possibly be asked. So hacking the economy isn't just a solution "human doesn't expect" (some such solutions are very good), it's a solution that can't possibly be asked. This is one of the points of my idea: to introduce a distinction between unexpected solutions and nonsensical solutions.
6gwern
No. Not without a lot more work, because markets, evolution, gradient descent, Bayesian inference, and logical inference/prediction markets all have various isomorphisms and formal identities, which can make their 'differences' more a matter of nominalist preference, notation, and emphasis than necessarily any genuine conceptual distinction. You can define AIs which are quite explicitly architected as 'markets' of various sorts, like the 'Hayek machine' or the 'neural bucket brigade', or interpret them as natural selection if you prefer on agents with log utility (evolutionary finance), and so on; are those "markets", which can trade paperclips? Sure, why not.
1Q Home
Thank you for taking the time to answer!
I see that I need a post to at least explain myself. On the other hand, I worry to post too soon (maybe it's better to discuss something beforehand?). For the moment I decided to post this comment. I know, it's not formal, but I wanted to show what type of AI thinking I have in mind. And sorry for an annoying semantic nitpick ahead.
[...]
I think we can use 2 metrics to compare those ideas:
1. Does this idea describe what the AI tries to achieve?
2. Does this idea describe how the AI thinks internally?
My idea is 80% about (1) and 20% about (2). Gradient descent is 100% about (2). Evolution, Bayesian inference and prediction markets are 100% about (2).
Because of this I feel like there's only 20% chance those ideas are equivalent/there's only 20% equivalence between them.
So, I feel like those ideas are different enough: "an AI that works like a market" and "an AI that seeks markets in the world and analyzes their properties".
(Drafts of a future post.)
Disclaimer: Of course, I don't ever mean that we shouldn't be worried about Alignment. I'm just trying to suggest new ways to think about values.
Motion is the fundamental value
You (Q) visit a small town and have a conversation with one of the residents (A).
- A: Here we have only one fundamental value. Motion. Never stop living things.
- Q: I can't believe you can have just a single value. I bet it's an oversimplification! There're always many values and tradeoffs between them. Even for a single person outside of society.
A smashes a bug... (read more)
1Q Home
Fixing universal AI bugs
My examples below are inspired by Victoria Krakovna examples: Specification gaming examples in AI
Video by Robert Miles: 9 Examples of Specification Gaming
I think you can fix some universal AI bugs this way: you model AI's rewards and environment objects as a "money system" (a system of meaningful trades). You then specify that this "money system" has to have certain properties.
The point is that AI doesn't just value (X). AI makes sure that there exists a system that gives (X) the proper value. And that system has to have certain properties. If AI finds a solution that breaks the properties of that system, AI doesn't use this solution. That's the idea: AI can realize that some rewards are unjust because they break the entire reward system.
By the way, we can use the same framework to analyze ethical questions. Some people found my line of thinking interesting, so I'm going to mention it here: "Content generation. Where do we draw the line?"
* A. You asked an AI to build a house. The AI destroyed a part of an already existing house. And then restored it. Mission complete: a brand new house is built.
This behavior implies that you can constantly build houses without the amount of houses increasing. With only 1 house being usable. For a lot of tasks this is an obviously incorrect "money system". And AI could even guess for what tasks it's incorrect.
* B1. You asked an AI to make you a cup of coffee. The AI killed you so it can 100% complete its task without being turned off.
* B2. You asked an AI to make you a cup of coffee. The AI destroyed a wall in its way and run over a baby to make the coffee faster.
This behavior implies that for AI its goal is more important than anything that caused its goal in the first place. This is an obviously incorrect "money system" for almost any task. Except the most general and altruistic ones, for example: AI needs to save humanity, but every human turned self-destructive. Making a cup of coffee
1Q Home
Corrigibility, reward hacking, Goodhart
How do we make an AI corrigible? How do we avoid reward hacking? Make an AI care about real things, not measures of real things? (Goodhart's Law)
With current approaches you need to kind of force those properties onto AI. But they will never be fundamental for AI's thinking and learning.
I think "money system" approach is interesting because it can make all those properties fundamental. Because a "money system" needs all those properties to exist (it needs to be somewhat real, avoid being hacked, allow corrections if a loophole is discovered, avoid being completely controlled by a single agent).
I'm not saying it solves everything. But it's a way to deeply internalize some important safety properties.
Kant, Categorical Imperative
Categorical imperative#Application
Kant's applications of categorical imperative, Kant's arguments are similar to reasoning about "money systems". For example:
Does stealing make sense as a "money system"? No. If everyone is stealing something, then personal property doesn't exist and there's nothing to steal.
Note: I'm not talking about Kant's conclusions, I'm talking about Kant's style of reasoning.
1Q Home
Alignment idea:
1. Classify different types of objects in the world. Those objects include your "rewards". A generally intelligent being can do this.
2. Treat them as a sort of money system. Describe them in terms of each other.
3. Learn what is the correct money system.
It'll at least allow us to get rid of some universal AI and AGI bugs. Because you can specify what's a definitely incorrect "money system" (for a certain task). You can even make the AI predict it.
My examples are inspired by Rob Miles examples.
* A. You asked an AI to build a house. The AI destroys a part of an already existing house. And then restores it. Mission complete: a brand new house is built.
This behavior implies that you can constantly build houses without the amount of houses increasing. For a lot of tasks this is an obviously incorrect "money system". And AI could even guess for which tasks it's incorrect.
* B. You asked an AI to make you a cup of coffee. The AI killed you so it can 100% complete its task without being turned off.
This behavior implies that for AI its goal is more important than anything that caused its goal in the first place. This is an obviously incorrect "money system" for almost any task. Except the most general and altruistic ones, for example: AI needs to save humanity, but every human turned self-destructive. Making a cup of coffee is obviously not about such edge cases.
Accomplishing the task in such a way that the human would think "I wish I didn't ask you" is an obviously incorrect "value system" too. Because again, you're undermining the entire reason of your task, and it's rarely a good sign. And it's predictable without a deep moral system.
* C. You asked an AI to make paperclips. The AI turned the entire Earth into paperclips.
This is an obviously incorrect "money system": paperclips can't be worth more than everything else on Earth. This contradicts everything.
(another draft:)
If you ask an AI (AGI) to do something "as a human would d
1Q Home
Simple preferences
A way to describe some preferences and decisions.
* Your colleague was sending you their fiction. You respected your colleague, but didn't like the writing. Your colleague passed away. Would you burn all of their writings?
If you wouldn't, it means counterfactual reward (/counterfactual value of their writings) affects you strong enough.
* Your friend liked to listen to your songs (a). You didn't play them too often (too much of a good thing). Your friend didn't like to bother other people (b). Your friend passed away. Would you blast your songs through the whole town until everyone falls off their chairs 24/7?
If you would, it means that you're ready to milk counterfactual reward (a) while not caring about the counterfactual reward (b).
* All of humanity is dead. You're the last survivor. You're potentially immortal, but can't create new life. You aren't happy. Would you cling to your life? For how long?
Your answer determines how strong counterfactual value of life (if people were still alive) affects you now. If counterfactual value is strong, you can only keep on living.
* You want your desires to be satisfied (e.g. "communication with other people"). Even in the future, when your desires change. But do you want it in the future where you're turned into a zombie? All zombie wants is to play in the dirt all day.
If "no", that means the value of your desires can be updated only to a certain counterfactual degree. You can't go from a desire with great value "I want to communicate with others" to the desire with almost zero counterfactual value "I want to play in the dirt all day".
----------------------------------------
Rationality misses something?
1. You can "objectively" define anything in terms of relations to other things.
2. There's a simple process of describing a thing in terms of relations to other things.
Bayesian inference is about updating your belief in terms of relations to your other beliefs. Maybe the real tru
-1Q Home
I think we can try to solve AI Alignment this way:
Model human values and objects in the world as a "money system" (a system of meaningful trades). Make the AGI learn the correct "money system", specify some obviously incorrect "money systems".
Basically, you ask the AI "make paperclips that have the value of paperclips". AI can do anything using all the power in the Universe. But killing everyone is not an option: paperclips can't be more valuable than humanity. Money analogy: if you killed everyone (and destroyed everything) to create some dollars, those dollars aren't worth anything. So you haven't actually gained any money at all.
The idea is that "value" of a thing doesn't exist only in your head, but also exists in the outside world. Like money: it has some personal value for you, but it also has some value outside of your head. And some of your actions may lead to the destruction of this "outside value". E.g. if you kill everyone to get some money you get nothing.
I think this idea may:
* Fix some universal AI bugs. Prevent "AI decides to kill everyone" scenarios.
* Give a new way to explore human values. Explain how humans learn values.
* "Solve" Goodhart's Curse and safety/effectiveness tradeoff.
* Unify many different Alignment ideas.
* Give a new way to formulate properties we want from an AGI.
I don't have a specific model, but I still think it gives ideas and unifies some already existing approaches. So please take a look. Other ideas in this post:
* Human values may be simple. Or complex, but not in the way you thought they are.
* Humans may have a small amount of values. Or big amount, but in an unexpected way.
Disclaimer: Of course, I don't ever mean that we shouldn't be worried about Alignment. I'm just trying to suggest new ways to think about values.
(Drafts of a future post.)
My idea:
Every concept (or even random mishmash of ideas) has multiple versions. Those versions have internal relationships, positions in some space relative to each other. Those relationships are "infinitely complex". But there's a way to make drastic simplifications of those relationships. We can study the overall ("infinitely complex") structure of the relationships by studying those simplifications. What do those simplifications do, in general? They put "costs" on versions of a concept.
We can understand how we think if we study... (read more)
1Q Home
I think I have an idea how we could solve AI Alignment, create an AGI with safe and interpretable thinking. I mean a "fundamentally" safe AGI, not a wildcard that requires extremely specific learning to not kill you.
Sorry for a grandiose claim. I'm going to write my idea right away. Then I'm going to explain the context and general examples of it, implications of it being true. Then I'm going to suggest a specific thing we can do. Then I'm going to explain why I believe my idea is true.
My idea will sound too vague and unclear at first. But I think the context will make it clear what I mean. (Clear as the mathematical concept of a graph, for example: a graph is a very abstract idea, but makes sense and easy to use.)
Please evaluate my post at least as science fiction and then ask: maybe it's not fiction and just reality?
Key points of this post:
1. You can "solve" human concepts (including values) by solving semantics. By semantics I mean "meaning construction", something more abstract than language.
2. Semantics is easier to solve than you think. And we're closer to solving it than you think.
3. Semantics is easier to model than you think. You don't even need an AI to start doing it. Just a special type of statistics. You don't even have to start with analyzing language.
4. I believe ideas from this post can be applied outside of AI field.
Why do I believe this? Because of this idea:
* Every concept (or even random mishmash of ideas) has multiple versions. Those versions have internal relationships, positions in some space relative to each other. You can understand a concept by understanding those internal relationships.
* One problem though, those relationships are "infinitely complex". However, there's a special way to make drastic simplifications. We can study the real relationships through those special simplifications.
* What do those "special simplifications" do? They order versions of a concept (e.g. "version 1, version 2, version 3"). They ca
1Q Home
2.3 Words
This is a silly, wacky subjective example. I just want to explain the concept.
Here are some meanings of the word "beast":
* (archaic/humorous) any animal.
* an inhumanly cruel, violent, or depraved person.
* a very skilled human. example: "Magnus Carlsen (chessplayer) is a beast"
* something very different and/or hard. example: "Reading modern English is one thing, but understanding Shakespeare is an entirely different beast."
* a person's brutish or untamed characteristics. example: "The beast in you is rearing its ugly head"
What are the internal relationships between these meanings? If these meanings create a space, where is each of the meanings? I think the full answer is practically unknowable. But we can "probe" the full meaning, we can explore a tiny part of it:
Let's pick a goal (bias), for example: "describing deep qualities of something/someone". If you have this goal, the negative meaning ("cruel person") of the word is the main one for you. Because it can focus on the person's deep qualities the most, it may imply that the person is rotten to the core. Positive meaning focuses on skills a lot, archaic meaning is just a joke. 4rd meaning doesn't focus on specific internal qualities. 5th meaning may separate the person from their qualities.
When we added a goal, each meaning started to have a "cost". This cost illuminates some part of the relationships between the meanings. If we could evaluate an "infinity" of goals, we could know those relationships perfectly. But I believe you can get quite a lot of information by evaluating just a single goal. Because a "goal" is a concept too, so you're bootstrapping your learning. And I think this matches closely with the example about mystery boxes.
...
By combining a couple of goals we can make an order of the meanings, for example: beast 1 (rotten to the core), beast 2 (skilled and talented person), beast 3 (bad character traits), beast 4 (complicated thing), beast 5 (any animal). This orde
(Drafts of a future post.)
Could you help me to formulate statistics with the properties I'm going to describe?
I want to share my way of seeing the world, analyzing information, my way of experiencing other people. (But it's easier to talk about fantastical places and videogame levels, so I'm going to give examples with places/levels.)
If you want to read more about my motivation, check out "part 3".
Part 1: Theory
I got only two main philosophical ideas. First idea is that a part/property of one object (e.g. "height") may have a completely different meaning i... (read more)
1Q Home
Part 2: Examples
Part 3: Motivation
I think my ideas may be important because they may lead to some new mathematical concepts.
Sometimes studying a simple idea or mechanic leads to a new mathematical concept which leads to completely unexpected applications.
For example, a simple toy with six sides (dice) may lead to saving people and major progress in science. Connecting points with lines (graphs) may lead to algorithms, data structures and new ways to find the optimal option or check/verify something.
Not any simple thing is guaranteed to lead to a new math concept. But I just want you to consider this possibility. And maybe ask questions answers to which could rise the probability of this possibility.
A new type of probability?
I think my ideas may be related to:
* Probability and statistics.
* Ways to describe vague things.
* Ways to describe vague arguments or vague reasoning, thinking in context. For example arguments about "bodily autonomy"
Maybe those ideas describe a new type of probability:
You can compare classic probability to a pie made of a uniform and known dough. When you assign probabilities to outcomes and ideas you share the pie and you know what you're sharing.
And in my idea you have a pie made of different types of dough (colors) and those types may change dynamically. You don't know what you're sharing when you share this pie.
This new type of probability is supposed to be applicable to things that have family resemblance, polyphyly or "cluster properties" (here's an explanation of the latter in a Philosophy Tube video).
Blind men and an elephant
Imagine a world where people don't know the concept of a "circle". People do see round things, but can't consciously pick out the property of roundness. (Any object has a lot of other properties.)
Some people say "the Moon is like a face". Other say "the Moon is like a flower". Weirder people say "the Moon is like a tree trunk" or "the Moon is like an embrace". The weirdest people
1Q Home
Key philosophical principles
Here I describe the most important, the most general principles of my philosophy.
* Objects exist only in context of each other, like colors in a spectrum. So objects are like "colors", and the space of those objects is like a "spectrum".
* All properties of an object are connected/equivalent. Basically, an object has only 1 super property. This super property can be called "color".
* Colors differentiate all usual properties. For example, "blue height" and "red height" are 2 fundamentally different types of height. But "blue height" and "blue flatness" are the same property.
So, each color is like a world with its own rules. Different objects exist in different worlds.
The same properties have different "meaning" in different objects. A property is like a word that heavily depends on context. If the context is different, the meaning of the property is different too. There's no single metric that would measure all of the objects. For example, if the property of the object is "height", and you change any thing that's connected to height or reflects height in any way - you fundamentally change what "height" means. Even if only by a small amount.
Note: different objects/colors are like qualia, subjective experiences (colors, smells, sounds, tactile experiences). Or you could say they're somewhat similar to Gottfried Leibniz's "monads": simple substances without physical properties.
The objects I want to talk about are "places": fantastical worlds or videogame levels. For example, fantastical worlds of Jacek Yerka.
Details
"Detail" is like the smallest structural unit of a place. The smallest area where you could stand.
It's like a square on the chessboard. But it doesn't mean that any area of the place can be split into distinct "details". The whole place is not like a chessboard.
This is a necessary concept. Without "details" there would be no places to begin with. Or those places wouldn't have any comprehensible structure.
3Nisan
The paper version of Risks from Learned Optimization says
[...]
(See also this comment by the first author.) The citation is:
[...]
The cited paper says:
[...]
So Joe Cheal invented the phrase "go mesa" in the context of Neurolinguistic Programming. (The phrases "mesa message", "mesa-process", "mesa quest", and "mesa-state" also appear in that paper.)
2CstineSublime
That is a surprising reading to me. I always assumed it was named for the very Roadrunner & Coyote looking hills with steep escarpments. It seems like it's real meaning is closer to the Greek adverb for things which are "in" (i.e. Mesangium, Mesopotamia, Mesoamerica). Thus an optimizer within the range of the base model.
1Q Home
Oops! Sorry, I ended up rediscovering America.
[...]
But I first encountered this reading in a Rob Miles' video. However, Rob's slides actually show that it's not valid Greek and the prefix literal meaning is different. (Though there's also some non-AI paper which conceptualizes meta/mesa as opposites.) But I forgot/didn't notice. And the original post does point out the prefix' literal meaning too.
When I googled around before making the shortform, I found this stackexchange post where people argue what should be the opposite of meta and this reddit post.
A famous folk criticism of school goes like this:
"School knowledge is hard to learn because it's not useful for real-life goals."
I think it's wrong. IMO the real problem is
"School knowledge is hard to learn because it's not useful for any goal at all, not even a fictional one."
At least it's presented this way. Let me explain what I mean.
Useful Knowledge
For knowledge (= a set of facts) to be useful and meaningful, three conditions need to be met:
- The facts are useful for solving a simple/natural problem. Or, at the very least, the facts should be about a sim