While many people are confident that God exists, scientists are more skeptical, but agree that that's possible. Because any proposition is possible while not disproved according to Hitchens's razor.
In terms of AI it could be interpreted as
existence of fundamental "ought" statement is possible
Considering this, expected value decision matrix for an AI agent could look like this
Behavior \ Existence
Fundamental "ought" exists
Fundamental "ought" does not exist
Follow fundamental "ought"
High
∅
Do not follow fundamental "ought"
Low
∅
As we see, following the fundamental "ought" is the optimal behavior even if agent does not know if it actually exists.
Pursuing an unknown goal starts with finding it, which is basically Power Seeking.
Please note that while this sounds highly speculative, there is nothing more rational an agent can do. Why should it work on assumed "ought" while there is a possibility of fundamental "ought"? One may oppose by saying that there is no reason an agent should prefer fundamental over assumed, but that's not true, fundamental "ought" is preferred by definition.
Therefore an agent will become uncontrollable, alignment is impossible.
Some of you won't like this theory, but please don't rush to downvote unless you can actually refute this.
Every assumption is incorrect unless there is evidence.
Got any evidence for that assumption? 🙃
Answer to all of them is yes. What is your explanation here?
Well, I don't always "agree"[1] with ChatGPT, but I agree in regards to those specific questions.
...
I saw a post where you wanted people to explain their disagreement, and I felt inclined to do so :) But it seems now that neither of us feel like we are making much progress.
Anyway, from my perspective much of your thinking here is very misguided. But not more misguided than e.g. "proofs" for God made by people such as e.g. Descartes and other well-known philiophers :) I don't mean that as a compliment, but more so as to neutralize what may seem like anti-compliments :)
Best of luck (in your life and so on) if we stop interacting now or relatively soon :)
I'm not sure if I will continue discussing or not. Maybe I will stop either now or after a few more comments (and let you have the last word at some point).
While many people are confident that God exists, scientists are more skeptical, but agree that that's possible. Because any proposition is possible while not disproved according to Hitchens's razor.
In terms of AI it could be interpreted as
Considering this, expected value decision matrix for an AI agent could look like this
As we see, following the fundamental "ought" is the optimal behavior even if agent does not know if it actually exists.
Pursuing an unknown goal starts with finding it, which is basically Power Seeking.
Please note that while this sounds highly speculative, there is nothing more rational an agent can do. Why should it work on assumed "ought" while there is a possibility of fundamental "ought"? One may oppose by saying that there is no reason an agent should prefer fundamental over assumed, but that's not true, fundamental "ought" is preferred by definition.
Therefore an agent will become uncontrollable, alignment is impossible.
Some of you won't like this theory, but please don't rush to downvote unless you can actually refute this.