The US helped the world rein in nuclear weapons. Could it do the same for AI?

The unfinished work of nuclear governance offers an important lesson for those struggling with the risk of AI now: learning to live with a dangerous technology is not the same as eliminating its threat.  | Daniel Gnap/NurPhoto/Getty Images

When Illinois Gov. JB Pritzker talks about the threat of artificial intelligence, he offers an increasingly common point of comparison: it is a danger “on the level of nuclear weapons.” 

The comparison is not far-fetched. If the international community found ways to constrain a potentially catastrophic technology once before, perhaps those arrangements can help us constrain another. 

The trouble is that while nuclear nonproliferation may be a useful framework for thinking about regulating AI, it isn’t necessarily a good one. Not because the technologies are too different, but because nuclear nonproliferation hasn’t actually freed us from the prospect of nuclear war. 

Nevertheless, the unfinished work of nuclear governance offers an important lesson for those struggling with the risk of AI now: learning to live with a dangerous technology is not the same as eliminating its threat. 

What we might be able to do, however, is eliminate uncertainty about who’s building AI, what AI is capable of, and when AI puts people at risk. The US has done exactly that with nuclear weapons by declassifying information about the size and condition of its own arsenal, encouraging other nuclear powers to do the same.

Prioritizing transparency about AI wouldn’t eliminate the dangers altogether, but it might help us move forward more collectively as we strive to manage the very real risks of this emerging technology.

What the nuclear-AI analogy gets right… and wrong

Comparisons between AI and nuclear weapons have emerged not just from politicians and commentators but from those at the very top of the frontier AI food chain. For some AI leaders, nuclear weapons also represent an ideal model for oversight.

In a 2023 essay, OpenAI’s Sam Altman, Greg Brockman, and Ilya Sutskever called for “something like an [International Atomic Energy Agency] (IAEA) for superintelligence efforts.” Their proposed international authority would “inspect systems, require audits, test for compliance with safety standards,” and restrict how the most powerful systems could be deployed.

The goal of such measures is to make the development of superintelligence safe, rather than prevent it, much as nonproliferation regimes stop short of banning nuclear weapons outright. Altman and his colleagues argue that the technology could deliver enormous benefits, while stopping its creation would be “unintuitively risky and difficult.” 

Not everyone invoking nuclear governance draws the line in the same place. In September, AI researcher Yoshua Bengio told the UN Security Council that “frontier AI should be licensed like… medicine, aviation, and nuclear energy to incentivize safe development.” He also called for a universal constraint on AI development, one more expansive than those imposed on nuclear weapons: “No one, no company and no country, should develop an AI system that could cause large-scale harm.”

The real problem with such schemes may be figuring out how they would be enforced — and by whom. Sure, there are fewer states with nuclear weapons today than there could have been if nonproliferation had not become a policy priority for the US. However, this was not the result of diplomatic prowess or good-faith restraint. In the most important cases, it was a result of coercion by the US and its allies. 

As Christopher David LaRoche and Ankit Panda, two nuclear experts at the Carnegie Endowment for International Peace, explained in Foreign Affairs: “South Korea and Taiwan gave up nascent weapons programs after intense pressure from the United States…and a threat to pull military support if they did not comply. Israeli airstrikes took out the Iraqi and Syrian programs; Libya surrendered its own under sustained U.S. and British pressure.” 

In other words, unless the US seeks to go to war to prevent countries around the world from getting hands on frontier AI, nuclear nonproliferation is not the ideal mold for AI regulation. Plus, even in the face of a robust nonproliferation regime, a few states managed to slip through the cracks — India, Israel, North Korea, and Pakistan each acquired nuclear weapons. 

Despite that, there are still fewer nuclear weapons in the world today than there were at the peak of the Cold War. But even the best-laid plans might not last forever. The arms control agreements that made such disarmament possible have all crumbled — whether due to cheating, repudiation, or expired treaties. Moreover, in addition to the increased number of states in possession of nuclear weapons, China has tripled its number of nuclear weapons just this decade.

These are the consequences of a regime built to manage a technology, rather than eliminate it. A similar frontier AI nonproliferation regime would govern who may build the most powerful systems, while leaving unaddressed the question of whether anyone should have that option in the first place.

A similar frontier AI nonproliferation regime would govern who may build the most powerful systems, while leaving unaddressed the question of whether anyone should have that option in the first place.

The greatest lesson is for the US

Today, the threat of nuclear war persists. However, nuclear nonproliferation offers us one powerful takeaway: the United States, in particular, has a tremendous role to play in shaping the future of regulating frontier AI. 

“In the nuclear domain specifically, the US historically has had a powerful norm-entrepreneurship function,” Panda told me. “What the United States does when it comes to nuclear weapons does get noticed by other countries and often gets adopted by other countries.”

This has historically involved leadership in transparency, Panda said. “The US, to its credit, is much more transparent than every other nuclear country when it comes to talking about things like command and control and what we do and don’t do.” For example, the US regularly declassifies and publishes data regarding the size of its nuclear weapons stockpile, fissile material inventories, and production facilities.

In a similar way, the US could lead the world in transparency about the frontier AI it is pursuing, the limits it might impose on that technology, and the safeguards in place to keep it under control. 

This could take shape in a few ways, beginning with the US government’s own use of AI:

  • Before deploying a powerful and risky AI system, the government could disclose what risks it tested for, the conditions under which the system is considered reliable, its known limitations, and what findings would prevent deployment. 
  • The US could explain which military decisions AI may contribute to and which require human authorization. This might, for example, involve distinguishing intelligence analysis from authority to initiate an attack. The government could also identify military applications of AI that it rules out altogether.
  • The US could publish accounts of failures or unexpected behavior, explaining whether a system was withdrawn, restricted, or retested after something went wrong.

These transparency measures should also extend to the private sector, and the government could require companies to:

  • Disclose dangerous-capability testing: This would involve requiring frontier developers to publish standardized summaries of tests for risks such as assisting cyberattacks, contributing to biological weapons development, or operating autonomously beyond intended limits.
  • Report serious incidents and near misses: Developers and deployers could be required to notify a regulator of significant safety failures, breaches, or unexpected dangerous behavior.
  • Publish the conditions for pausing development or withholding release: Companies could disclose in advance which findings would trigger additional safeguards, delay product launches, or halt further development.

As AI continues to reshape the world we live in, prioritizing transparency around the technology’s achievements — and limits — is one way to ensure that safety and security remain central.

As AI continues to reshape the world we live in, prioritizing transparency around the technology’s achievements — and limits — is one way to ensure that safety and security remain central. Panda suggested that such measures could reassure adversaries that the US is not going to implement frontier AI models in ways that might be dangerous or irresponsible. 

Source Author
Author: Source Author

Source: Vox.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *