For me, the biggest most tangible risks from AI are from those AI (agent or tool AIs) that are connected to the real world and can influence human incentives, especially those that can work at a global level.
If you have a Tool AI connected to the real world and able to influence humans through strong incentives, wouldn't this be a very high risk even if they never become full AGI?
There are actual real-life examples doing this right now, most notoriously AIs used for trading financial assets. These AIs are optimizing for profit, by playing with one of the strongest human incentives: money. Changes in financial incentives worldwide can trigger bankruptcies, bank-runs, currency... (read more)
For me, the biggest most tangible risks from AI are from those AI (agent or tool AIs) that are connected to the real world and can influence human incentives, especially those that can work at a global level.
If you have a Tool AI connected to the real world and able to influence humans through strong incentives, wouldn't this be a very high risk even if they never become full AGI?
There are actual real-life examples doing this right now, most notoriously AIs used for trading financial assets. These AIs are optimizing for profit, by playing with one of the strongest human incentives: money. Changes in financial incentives worldwide can trigger bankruptcies, bank-runs, currency... (read more)