This is a special post for quick takes by Q Home. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
(Drafts of a future post.)
My idea:
Every concept (or even random mishmash of ideas) has multiple versions. Those versions have internal relationships, positions in some space relative to each other. Those relationships are "infinitely complex". But there's a way to make drastic simplifications of those relationships. We can study the overall ("infinitely complex") structure of the relationships by studying those simplifications. What do those simplifications do, in general? They put "costs" on versions of a concept.
We can understand how we think if we study our concepts (including values) through such simplifications. It doesn't matter what concepts we study at all. Anything goes, we just need to choose something convenient. Something objective enough to put numbers on it and come up with models.
Once we're able to model human concepts this way, we're able to model human thinking (AGI) and human values (AI Alignment) and improve human thinking.
There's the hard problem of consciousness: how is subjective experience created from physical stuff? (Or where does it come from?)
But I'm interested in a more specific question:
For example, "How do qualia change? How many different qualia can be created?" or "Do qualia form something akin to a mathematical space, e.g. a vector space? What is this space exactly?"
Is there any knowledge contained in the experience itself, not merely associated with it?1 For example, "cold weather can cause cold (disease)" is a fact associated with experience, but isn't very fundamental to the experience itself. And this "fact" is even false, it's a misconception/coincidence.
When you get to know the personality of your friend, do you learn anything "fundamental" or really interesting by itself? Is "loving someone" a fundamentally different experience compared to "eating pizza" or "watching a complicated movie"?
Those questions feel pretty damn important to me! They're about limitations of your meaningful experience and meaningful knowledge. They're about personalities of people you know or could know. How many personalities can you differentiate? How "important/fundamental" are those differences? And finally... those questions are about your values.
Those questions are important for Fun Theory. But they're way more important/fundamental than Fun Theory.
1 Philosophical context for this question: look up Immanuel Kant's idea of "synthetic a priori" propositions.
And those questions are important for AI Alignment. If AI can "feel" that loving a sentient being and making a useless paperclip are 2 fundamentally different things, then it might be way easier to explain our values to that AI. By the way, I'm not implying that AI has to have qualia, I'm saying that our qualia can hint us towards the right model.
I think this observation gets a little bit glossed over: if you have a human brain and only care about paperclips... it's (kind of) still objectively true for you that caring about other people would feel way different, way "bigger" and etc. You can pretend to escape morality, but you can't escape your brain.
It's extremely banal out of context, but the landscape of our experiences and concepts may shape the landscape of our values. Modeling our values as arbitrary utility functions (or artifacts of evolution) misses that completely.
Box A
There's a mystery Box A. Each day you find a random object inside of it. For example: a ball, a flower, a coin, a wheel, a stick, a tissue...
Box B
There's also another box, the mystery Box B. One day you find a flower there. Another day you find a knife. The next day you find a toy. Next - a gun. Next - a hat. Next - shark's jaws...
...
How to understand the boxes? If you could obtain all items from both boxes, you would find... that those items are exactly the same. They just appear in a different order, that's all.
I think the simplest way to understand Box B is this: you need to approach it with a bias, with a "goal". For example "things may be dangerous, things may cause negative emotions". In its most general form, this idea is unfalsifiable and may work as a self-fulfilling prophecy. But this general idea may lead to specific hypotheses, to estimating specific probabilities. This idea may just save your life if someone is coming after you and you need to defend yourself.
Content of both boxes changes in arbitrary ways. But content change of the second box comes with an emotional cost.
There're many many other boxes, understanding them requires more nuanced biases and goals.
I think those boxes symbolize concepts (e.g. words) and the way humans understand them. I think a human understands a concept by assigning "costs" to its changes of meaning. "Costs" come from various emotions and goals.
"Costs" are convenient: if any change of meaning has a cost, then you don't need to restrict the meaning of a concept. If a change has a cost, then it's meaningful regardless of its predictability.
More examples of mystery boxes:
You can imagine a "meta box", for example a box that alternates between being the 1st box and the 2nd box. Meta boxes can "change their mood".
I think, in a weird way, all those boxes are very similar to human concepts and words.
The more emotions, goals and biases you learn, the easier it gets for you to understand new boxes. But those "emotions, goals, biases" are themselves like boxes.
I think I have an idea how we could solve AI Alignment, create an AGI with safe and interpretable thinking. I mean a "fundamentally" safe AGI, not a wildcard that requires extremely specific learning to not kill you.
Sorry for a grandiose claim. I'm going to write my idea right away. Then I'm going to explain the context and general examples of it, implications of it being true. Then I'm going to suggest a specific thing we can do. Then I'm going to explain why I believe my idea is true.
My idea will sound too vague and unclear at first. But I think the cont... (read more)