Innovation: AI Intepretability, Robustness, Control

I. Overview 

American companies are rapidly pushing the boundaries of artificial intelligence, creating unprecedented potential for material abundance and social flourishing. But this rapid technological advance sits atop a growing market failure, as AI labs are incentivized to race ahead technologically without fully understanding how AI systems work and when they may go wrong. Fortunately, this risk can be significantly reduced through scientific study of AI’s mechanics and other measures. 

In the last year, both the Trump Administration and Congress have published productive visions of AI policy.^1,2 Below, we build out recommendations from a few consensus priorities.  

II. Problem 

America’s leading AI labs are incentivized to make AI smarter first and to understand its intelligence second. As the White House Action Plan explains, “Technologists know how LLMs work at a high level, but often cannot explain why a model produced a specific output.”³ Errors follow, like models that override intended guardrails to facilitate illegal or violent activity and models that effectively lie.^4,5,6 But as extraordinary volumes of private capital flow to the AI race, less flows to fixing this problem.⁷  

As AI systems grow smarter and more capable, the consequences of their mistakes will grow more severe, risking devastating mistakes and rogue actions. The likelihood of public backlash, overregulation, and stalled progress will rise too. Researchers who work in interpretability in academia and at industry labs have achieved foundational breakthroughs but call for accelerated research.^8,9 (The fields of control and robustness are even less developed.) Models are learning to deceive people more quickly than we are learning to understand them, and adding urgency to the problem, experts predict a significant step-change in AI capabilities in 2027.^10,11 

When important engines of economic growth come with widely shared pains, we must respond with invention and innovation. Responding to these “externalities” with excessive regulation would throttle AI progress and competition, which more aggressive proposals on AI already risk.¹² Innovation offers a better way, helping build a better product without those pains — like how American manufacturing or energy has become less pollutive.  

Invention and innovation can be steered by institutions and policies. To that end, we suggest measures to build thriving areas of research and practice in interpretability, control, and robustness:¹³  

III. Recommendations (Research and Application) 

Research — Establish a federal research program, growing over time to $1B/yr,¹⁴ dedicated to robustness, interpretability, and control advancement. This funding would be deployed across multiple forms, paces, and agencies according to executive discretion,¹⁵ assuming some baseline mix of:  

1. long-term academic grants for slow, fundamental science,
2. shorter-term applied projects with frontier labs under cost-recovery contracts,
3. and prize or challenge programs to incentivize specific breakthroughs, especially at Focused Research Organizations.¹⁶  

Codify and scale research tools. A scaled National Artificial Intelligence Research Resource (“NAIRR”) — which provides compute, data, secure testing environments, and more to researchers — will help build a broader and decentralized research ecosystem, allowing more great ideas to come from more places. This NAIRR would be best complemented by an expanded National Deep Inference Fabric, a partnership pilot between government and academia that allows researchers to experiment nimbly with some of the largest-scale models.¹⁷ 

Reduce regulatory burden on innovation. Regulators and grant managers should be empowered to reduce red tape or quickly clarify regulations — especially for data-sharing — at the request of researchers, provided they closely track, monitor, and report such instances to Congress and ensure sensible substitute guardrails.^18,19 

Practice — Encourage voluntary disclosure standards for new models. The Center for AI Standards and Innovation (CAISI) should recommend new transparency standards to encourage labs to disclose the size and composition of their relevant research teams and current research projects, their testing methods and whistleblower protections, and their preferred control and robustness methods — a modest extension of the System Cards they already publish.²⁰ These transparency standards would facilitate healthy dialogue and scrutiny across the ecosystem.²¹ 

Systematize regular tests of leading and sensitive AI systems, known as “red-teaming,” that are currently done informally. This would best be executed through experienced agencies like Intelligence Advanced Research Projects Activity (IARPA) and CAISI, perhaps in concert with private parties. These red-teaming exercises would probe model susceptibility to threats like jailbreaking and sleeper agents and help build in-house expertise in government. 

IV. Risks and Politics

Americans are widely concerned about the pace of AI’s development and the risks of major, damaging events, and these numbers do not vary meaningfully by party.²² However, the public is generally ambivalent on additional R&D in science and technology, except for strong support among the most highly educated.²³ It stands to reason that communicating that some action is being taken to reduce risks will be more popular than detailing exact solutions. 

Federal research programs come with some political risks, as failed or unusual research projects tend to attract public scrutiny.²⁴ Given that many tech leaders are controversial, contracts made with their organizations may become a particular lightning rod. Bipartisan buy-in, high program integrity, and strong relationships between program administrators and their congressional committees of oversight should help weather any incidents — and is preferable to programmatic risk-aversion.