But here’s the catch: if you win the race, you get to keep the car. If you lose, then your car will self-destruct in a few seconds, potentially putting your life at risk.
A more faithful analogy has any of the cars explode with significant probability, possibly with the leading car exploding first (which is how we get to know it was the leading car), and also the explosion is big enough that it takes out all participants as well as spectators.
Haha I see! Well, we are assuming in this scenario that we have a third person point of view via which we get to know which is the leading car and which is not when the race begins (much like Hunger Games style surveillance cameras). But I totally agree that the probability of the car exploding to be so big as to take all participants and spectators is a more faithful analogy to reality.
Imagine you (USA) are in a race to the finish line with your neighbour (China), except that the race demands that you craft your own vehicle (an AI) to win the race. But here’s the catch: if you win the race, you get to keep the car. If you lose, then your car will self-destruct in a few seconds, potentially putting your life at risk.
What will you do?
You can choose not to race at all (i.e. not making an AI at all). Eliminating the harm of self-destructing car because if there’s no car then there’s no risk of it self-destructing. But you see your neighbour has already started building his car, and you can’t seem to stop yourself from building yours and competing with them. As you don’t want to ‘lose‘ the race.
So you take the risk and start building your own car. But here’s another dilemma- you have only 2 choices to make now.
Option 1: Make an autonomous car that drives you to the finish line (LLM-based AGI Model) OR
Option 2: Make a fully controllable advanced car that you can drive to the finish line (Yoshua Bengio’s Scientist AI) .
It’s hard to choose because here’s the tradeoff- building an autonomous car takes lesser time with current resources, but it doesn’t guarantee it will take you to the finish line as it is not fully controllable, and may go in any direction when the race starts.
As far as option 2 goes, building a fully controllable advanced car does guarantee that it will take you to the finish line, as it is controllable, but the problem is that it takes at least twice the time to make as compared to the autonomous car. And could leave you behind in the race and give your neighbour a headstart if he chooses to make an autonomous car. But you have no way to know what your neighbour is actually building.
What option will you choose?
Just as you are about to pick your tool, your trustworthy newspaper boy comes running and tells you that your neighbour is obessesed about control. And wants to control everything from his life to other people’s life. This somehow sparks your mind and you think you might guess what type of vehicle your neighbour would actually be working on.
Somehow it also hits you that if you choose option 1, your autonomous vehicle may accidentally go off the cliff, or run into a tree or a house (catastrophic AI scenarios). You look at your watch, and the clock is ticking fast.
What option will you choose now?