Intro

The recent invention of AI systems with human-like capabilities across multiple domains of intelligence, it often seems these days that a future where AI will radically improve our society for has gone from a sci-fi premise to a just-beyond-the-horizon reality. However, for each one of the AI ages' golden promises there is a correspondingly terrifying risk: AI will allow the rapid design of precision-targeted drugs, but will also deliver the basic tools for bio-terrorism into the hands of anyone with an internet connection. LLMs will relieve us of the burden of mindless work, but AGI may one day make all of us economically redundant. Self-driving cars will zip us around futuristic trafficless cities, but the same autonomous motion capability will enable the development of killer robodogs.

This duality between artificial intelligence's potential boons and potential risks promises at least one thing: AI will be extremely disruptive to the status quo. While the dangers of machine intelligence have been acknowledged since the Industrial Revolution are are today the subject of intensive research at LLM developers like Anthropic, there is no question that current AI safety measures are lacking. With the US government seemingly content to pursue a gung-ho approach to AGI development, it is up to us, the research community, we must evaluate our current methods for mitigating AI-caused harm and then determine which safety measures we most need to create. Developing criteria to rank future benchmarks is crucial for our ability to select which research to invest money and time in. In the authors' opinion, the AI safety community still lacks a systematic framework for assessing which new benchmarks would be most beneficial, i.e. prevent the most harm to humankind.

We present a first attempt at such a framework by extending a list of AI's risks to society and their expected harm compiled by the Centre pour la Sécurité de l’IA (CeSIA). Using this framework, we evaluate to what extent existing AI benchmarks and safety methods guard against each potential danger to decide which threats most urgently require good benchmarks, that is which benchmarks AI safety researchers should focus on to maximize their expected benefit to society. In this respect our methodology draws inspiration from Effective Altruism. While our study of benchmarks is by no mea...

Posts

Wikitag Contributions

Comments