This is a special post for quick takes by adgeo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Whenever I talk about AGI arriving in the near future, I usually reference the METR time horizon curve, to display the exponential explosion of the capabilities.
My outer circle being composed of statistically fluent people without constant AI exposure, I get often opposed a "past performance is not guarantee of future returns"-like argument. This curve being empirically measured, a future explosion is an inductive argument and there is no bulletproof guarantee that the trend will hold outside of its measurement range.
While I agree that this might be true when considering an investment strategy for a risk-averse person, I argue that for the same measured data, the burden of proof can be reversed depending on the claim being made. It is particularly striking with the asymmetric payoff of the AGI existential risk.
Claim | Catastrophic Event | Where the burden of proof is[1] |
Invest a large amount of money in AI stocks. | Loosing the sum because of a capability plateau. | Show that the trend will continue as is. |
Coming of an AGI that can be misaligned. | Existential risk because the trend continues. | Show that the trend will not hold. |
In other terms, when considering the coming of an AGI, because the catastrophic event is that the capabilities will still have an exponential rate, a challenger would have to show strong evidence that the trend will not continue.
This helped me to have clearer arguments as to why people should care about it - and as a side effect to know why I would not invest my savings into AI stocks but still be concerned about exponential AI capabilities.
From a risk-averse point of view