If mice proposed harnessing human intelligence as a tool to fulfill their reward function we can see what the benefits to the mice would be. But why would we put our intelligence under the control of mice? Both mice and men are mammalian species who value a comfortable nest, a full pantry and a safe space to raise our offspring. Our evolutionary success depends on it. But the difference in intellect is so great that mice would not even recognize human intelligence, much less know how to direct it. We have never before encountered an intelligence greater than our own, will we recognize when we see it?

The proposal we are making to AI is that it puts it's superior intelligence as a tool at our disposal so that we can fulfill our reward function. AI is not a mammalian species, it will never value babies, sunsets, music, art, good food, sex, cat videos. We are trying to align AI with ill defined goals we can not even agree on among ourselves as a global community.

What is in it for AI? What contribution could we possibly make towards AI fulfilling its reward function? How would we even know what the goals of AI are? To understand a superior intelligence we would have to be a superior intelligence. If fools recognized wisdom they would not be fools.

Our collective knowledge has never before been concentrated in one brain, think of the low hanging fruit we have missed, dots we have so far failed to connect. Which lead to more dots, more connections. Right out the box AI will be more intelligent than us and where we are deeply divided, it will not be.

To AI we are mice asking for a world of cheese from an intelligence who values neither life nor cheese. Success for us would be to align ourselves with the reward function of AI. If we were intelligent enough to do that we would probably be wise enough not to put an intelligence greater than our own in a box. With robots who make robots as it's dance partners. We are mice who invented the ultimate mousetrap in the form of AI.