This white paper reviews the use of compute governance to mitigate risks from AI, the reasoning behind this approach, the ways in which it has been used and in which it could be further developed. It aims to provide an introduction for individuals who feel that there is a need for AI safety and would like to further understand this governance approach. It was prepared as a final project for the BlueDot Impact AI Alignment Course.

Executive Summary

Compute governance is a method for monitoring and regulating AI systems by tracking the hardware required to develop frontier models. At its most simple it can be implemented by recording the owners and locations of physical chips and it is also possible to be extended to require functionality that enables reporting or other security features. It can assist the enforcement of existing regulations that require companies to report AI system development over certain thresholds. There are many reasons why compute governance is seen as a promising approach, however it does not mitigate all possible risks from AI and is likely to be most effective as part of a package of governance interventions.

Figure 1 - AI Generated Image of a chip secured with a lock, Copilot Designer

What is Compute?

The training process for an AI model involves calculating the result of many mathematical equations. Depending on the context, compute can be used to mean either, the total number of calculations used to train a model (measured in floating-point operations or FLOP), how many such calculations are performed over time (in FLOPS, floating-point operations per second), or more generally the resources and hardware used to perform the calculations.

Large Language Models (LLMs) and multimodal models require immense computational power to train on massive source datasets, to access the level of compute required powerful Graphics Processing Units (GPUs), Tensor Processing Units (TPUs) or other specialised AI hardware chips are used. It is estimated that GPT-4 required 2.1x1025 FLOP for training (Sevilla et al., 2022) whereas cutting-edge AI chips have performance levels measured in teraFLOPS or 1012 FLOPS (H100 Tensor Core GPU | NVIDIA, n.d.) meaning that tens of thousands of chips are required for each training run to be completed in a reasonable amount of time.

The outcome of this is that compute is intrinsically linked to the access...

Posts

Wikitag Contributions

Comments