How do you propose translating this into code?
The idea was supposed to be more optimized for trying to solve alignment than being easy to code. My current (vague - this is the half-backed thread after all) mental model involves
a) training a neural net to be able to understand the necessary concepts to make sense of the intended target it should be aiming at (note: it doesn't necessarily have to understand the full details at first, just the overall concept which it can then refine)
b) using some kind of legibility tool to identify how to "point at" the concepts in the neural net
c) implementing the actual planning and decision making using conventional (non-nn) software that reads and activates the concepts in the neural net in some way
However, in writing this comment reply I realized that the naive way I had been thinking that this could be done any approach regardless of my own thinking being naive (which was something like generating plans and evaluate them according to how well they match the goal implemented by the non-nn software's connections to the neural net) would
a) be prone to wishful thinking due to only the plans it rates best being relevant, the best-rating plans tending to be ones where it was overoptimistic, and note that extreme levels of optimization on plans could lead to extreme levels of bias, and the bias will occur everywhere in all inputs and intermediate steps of the plan evaluation calculation and not just at the final step, and
b) in the same vein but more worryingly, be potentially vulnerable to the plan generator generating superstimulus-type examples which score highly in the AI's flawed encoding of the concepts while not actually being what humans would actually want. Which is likely inevitable to any neural net and maybe even to anything that extracts concepts from complex inputs.
No full solutions to these problems as of yet ever, though if I may be permitted to fall prey to problem (a) myself, maybe standard robustness approaches could help against (b).
Note my translation devolves to "identify privileged actions that are generally safe, specific to the task" and "don't do things that have uncertain outcome". Both these terms are easily translated to code.
Neither of those things sound "easily translated to code" to me. What does "safe" mean? What does "specific to the task" mean? How do you classify outcomes as being "uncertain" or not?
[Cross-posted from the EA Forum. The EA Forum version of this post is for both half-baked EA ideas and half-baked AI Safety ideas, whereas this version of the post is for half-baked AI Safety ideas specifically.]
I keep having ideas related to AI safety, but I keep not having enough time available to really think through those ideas, let alone try to implement them. Practically, the alternatives for me are to either post something half-baked, or to not post at all. I don't want to spam the group with half-thought-through posts, but I also want to post these ideas, even in their current state, in case some of them do have merit and the post inspires someone to take up those ideas.
Originally I was going to start writing up some of these ideas in my Shortform, but I figured that if I have this dilemma then likely other people do as well. So to encourage others to at least post their half-baked ideas somewhere, I am putting up this post as a place where other people can post their own ideas without worrying about making sure they formulate those ideas to the point where they'd merit their own post.
If you have several ideas, please post them in separate comments so that people can consider each of them individually. Unless of course they're closely related to each other, in which case it might be best to post them together - use your best judgment.
[This post was also inspired by a suggestion from Zvi to create something similar to my AGI Safety FAQ / all-dumb-questions-allowed thread, but for ideas / potentially dumb solutions rather than questions.]