It's not always fashionable around these parts to worry more about ML bias than safety, but this seems like a case where there's essentially no safety risk, but potentially there's bias risk.
So something around making sure that the impact is similar across different stakeholder groups, often demographic groups, might be in order.
I'm all over the bias issues. Because I can address them from my own practical experience, I'm happy working with what I know. The AI safety issues are way outside my practical experience, and I know it.
Being seen as impeding work and being in the way of money making is the quickest way out of the position where you can affect decision making. Erecting prohibitions would do that. It looks like your company is nowhere near the cutting edge level. If so, you don't really have to worry about accidentally making a self-improving AGI. If you think that, unlike all these other companies with deep pockets and top talent, you are closer to the danger zone, contact someone privately to see if your assessment is warranted. Someone at MIRI or, better yet, at the Alignment Research Center, for example.
Being seen as impeding work and being in the way of money making is the quickest way out of the position where you can affect decision making. Erecting prohibitions would do that.
Disagree. A lot of times management would put someone in the position of writing policy as a matter of pure delegation, in which case they'd want a sensible policy that constrains the business in line with that business' unique risks. Writing too lax of a policy is worse than writing one that has some restrictions that make sense in the context of what they are actually building. Writing good policy will put you in a better position to write policy in the future.
That said, policy about general AI when the company isn't going to build general AI is not good. It's better to focus on realistic scenarios.
I have a question that is extremely practical.
My company is working with AI that is frankly not all that capable. A key example includes text classification that humans do, but that would let us avoid the expense of having a human do it. Off-the-shelf machine learning works fine for that. But we're starting the long process of learning and using various AI techniques. Over time, that should become much more sophisticated as we solve easy problems with very limited capabilities.
I'm in a position to erect prohibitions now on any future AI work that we do. These would be in the nature of "don't do X without escalated approvals." Should I require such a list and, if so, what should I put on it? For reference, our business is mainly acquiring and processing data. And we don't have the resources of a Google to put on research that doesn't quickly pay off. So we won't be doing anything cutting edge from the perspective of AI researchers. It's mainly going to be application of known techniques.
I can harmlessly put, "don't use or try to build an artificial general intelligence," because we'll never have the capability. So if that's all I put, there's no point.
Should I put "don't use or build any AI technique in which the AI engine recodes itself"? That's probably too simple an expression of the idea, but I don't want these prohibitions to be tied to particular versions of current technology.
Should I put "use additional security around AI engines to prevent external users from combining the capabilities of our AI engines with theirs" on the theory that a growing AI would imperialistically grab hold of lots of other AI engines to expand its capabilities?
I'm obviously out of my depth here, so I'd be grateful for any suggestions.