Verification mechanisms for international AI treaties

Written By Renan Araujo

International cooperation is essential for strong AI governance. But, when we have international treaties on AI, how do we ensure they are enforced? Enter verification mechanisms.

Nuclear analogy

In the nuclear weapons sector, the ability of countries verifying whether others are complying with international agreements has been essential to limit proliferation and strengthen mutual trust. The cornerstone of nuclear verification is the International Atomic Energy Agency (IAEA), who plays a central role in conducting nuclear site inspections.

While there are disanalogies, there are also valuable lessons from nuclear arms control verification that can be translated to AI treaties, as shown by Mauricio Baker on this paper: https://arxiv.org/abs/2304.04123

Categories of verification mechanisms

Two categories of AI verification mechanisms are:

Hardware-dependent methods: these include modifying chips used in training and inference to locate them (preventing smuggling or secret data centers) and to record what they are being used for or blocking them from being used for large training runs (preventing misuse).

With these measures in place, governments can set up a registry of chip activities, monitor suspicious activities more readily, and audit data centers in case of potential breaches of international agreements.

Access-dependent methods: these include inspections of data centers (verifying compliance with hardware agreements and safety standards), chip fabs (verifying if the amount and type of chips manufactured are in tune with agreements), and developers of the most advanced and risk-prone models (besides verifying compliance, could also lower the bar for whistleblowers).

These categories were presented by Akash Wasil, Tom Reed, Jack Miller, and Peter Barnett on this paper: https://arxiv.org/pdf/2408.16074

Renan Araujo
Previous

Open roles in AI policy #8
Next

Research management: some tips and resources