Andrew Rogers

Andrew Rogers's Posts

Sorted by New

Andrew Rogers's Comments

A list of good heuristics that the case for AI x-risk fails

There is an issue of definition here. Categories of scenario exist where it is unclear if they constitute an "AI takeover" even though there is recognition of a real and likely risk of some type. Almost everyone stakes out positions at binary extremes of outcome, good or bad, without much consideration for plausible quasi-equilibrium states in the middle that fall out of some risk models. For researchers working in the latter camp, it will feel a bit like a false dichotomy.

As another heuristic, the inability to arrive at a common set of elementary computational assumptions, grounded in physics, whence the AI risk models are derived is sufficient reason to be skeptical of any particular AI risk model without knowing much else.

How common is it for one entity to have a 3+ year technological lead on its nearest competitor?

An example of a 10+(!) year technology lead is computational discrete topology. Every large-scale geospatial, graph, et al analysis system is based on it — you can’t build one without it — but there is virtually no literature on how it works and a practical expression of the theory is robustly non-obvious. The same few people continue research and design every kernel for companies/governments. AGI and autonomous systems specifically drive much demand for this tech currently, since it is needed to reason about relationships/behaviors in space-time at scale.

There is no company behind this tech currently but I’ve heard rumors of one being created. It could have a strong feedback loop, not just due to tech exclusivity but because a platform-level implementation would effectively provide a consensus model of physical reality for machines.

Tangentially, I am aware of AGI research programs working from first principles that have made impressive theoretical CS advances while completely under the radar. It is difficult to determine if any have 3+ year leads on any other program though since that assessment implies global visibility.

Distinguishing between a technological lead and ineffective competition is also important. An example is database engine technology. Some proprietary databases are orders of magnitude more efficient/scalable than any open source comparable, which looks qualitative, but is widely recognized as a product of design quality rather than any technological lead. (see also: Google’s data infrastructure)