Assessing the Risk of AI-Enabled Computer Worms
Several frontier AI companies test their systems for dual-use cyber capabilities, such as vulnerability discovery and exploit development, that might be misused by threat actors. But what do these test results imply about the overall risk from cyberattacks? Answers to this question are needed to calibrate responses to cyber-AI capability progress, particularly in light of recent models’ considerable advances.
Currently, few published risk models explain how AI cyber capabilities might cause harm. We take an initial step toward filling this gap by developing an in-depth risk model for AI helping threat actors to develop data-damaging worms similar to WannaCry and NotPetya. We identify the development of “elite exploits” that spread without user interaction, allow remote code execution with high privileges, and are effective against widely used software as the primary bottleneck to such worms.
Drawing on historical case studies, a model that decomposes risk into threat-actor capability, willingness, and resulting damages, and a pilot survey of cybersecurity experts and high-performing forecasters, we conclude that if frontier AI were to enable a quarter of moderately skilled actors to develop elite exploits, the marginal economic damage from data-damaging worms would plausibly run to billions of dollars per year. Conditional on this capability being widely available, respondents' median estimate of the probability of at least one worm attack causing $10 billion or more in 2026 roughly tripled, and median total annual expected damages rose two- to fivefold, to tens of billions of dollars.
These results provide a prima facie case that AI companies should evaluate for this capability and consider mitigations should it emerge – though experts disagreed about which release and safeguard policies would best reduce risk. All estimates carry high uncertainty given the small sample and the fragmentary underlying evidence. This work demonstrates a methodological approach for converting AI-cyber capability evaluations into risk assessments, while highlighting the continued need for better evidence and expert discussion to refine its assumptions.
