A Biosafety Agenda for AI-Bio Governance
To date, assessments of AI-related risks from the design, acquisition, or release of dangerous biological materials have largely focused on intentional misuse – such as actors using AI to design and release harmful pathogens or toxins – and on biosecurity measures aimed at preventing it. We argue that policymakers and researchers should increase focus on biosafety – the management of accidents, errors, and other inadvertent harms.
AI is likely to increase the volume and speed of biological work, with more actors carrying out more ambitious experiments. Most of this work will bring real benefits in medicine and public health. But the same expansion may create more opportunities for accidents and unauthorized access to dangerous biological materials. Renewed biosafety efforts can support this expansion while lowering both risks: laboratory accidents, which are frequent but usually lower-consequence, and some forms of misuse, which are rarer but often more severe.
The case for greater focus on biosafety rests on three observations, which are developed further in this report:
- Laboratory accidents are a persistent and well-documented source of biological harm associated with both legitimate biological research and illicit weaponization programs.
- AI is increasing both the capability to perform advanced biological work and the number of actors attempting it, which may raise the aggregate risk of accidents.
- By increasing decentralization, AI may expand the number of grey zone actors – such as under-resourced laboratories, hobbyists, or new entrants without biosafety capacity – that work with dangerous materials outside or at the margins of standard biosafety and biosecurity protocols. Their intent is benign or ambiguous rather than harmful.
Biosafety and biosecurity overlap in practice, and many mitigations serve both purposes. Biosecurity measures – such as tracking dangerous materials and controlling personnel access – primarily reduce the risk of diversion or deliberate release. At the same time, they lower accident risk by ensuring hazardous biological materials are monitored and accessed only by appropriately trained individuals. Similarly, some biosafety measures – such as safety training, near-miss reporting, and physical containment of dangerous materials – primarily reduce accident risk, while also closing some opportunities for misuse by exposing work conducted outside required protocols. These measures also make unsafe or unauthorized grey zone activity more visible, which is where biosafety most directly reinforces biosecurity. Biosafety should be understood as the foundation of a layered defense, not a substitute for targeted biosecurity measures and counterproliferation efforts aimed at the catastrophic tail of AI-bio risk.
To act on this, we propose a portfolio of biosafety-centered mitigations and highlight four priority interventions as near-term next steps:
- Require biosafety certifications or assurances as a condition for managed access to AI models with sufficiently high-risk biological capabilities and to other controlled tools that pose biorisks.
- Establish a central AI-bio incident and near-miss taxonomy and registry, building on existing biosafety reporting mechanisms (e.g. the US Federal Select Agent Program) with broader participation and coverage of AI-enabled workflows.
- Ensure that insurance, grant, and investor due diligence take into account biosafety standards.
- Establish physical containment standards and guidance for self-driving laboratories, including stop conditions under which experiments should be halted for review.
Renewed biosafety mitigations for AI-enabled advanced biological research may reduce incentives to cut corners under competitive pressure and encourage a race to the top in safety practices.
