To deal with the risks of these high-compute frontier AI systems, we must govern not only how they can be used but also how they are developed and made available to people in the first place. President Biden’s recent AI executive order serves as an important first step in this direction in the U.S. by introducing reporting requirements for advanced models, trained using more computational power than any to date. However, regulatory oversight must not stop there. There is currently more public oversight and accountability for constructing an apartment building than there is for building frontier AI models. As the stakes of this technology increase, the problems associated with this lack of oversight will become increasingly dire. Further efforts will be required to ensure models’ safety and security, addressing malicious misuse of AI, reducing bias and discrimination, boosting innovation, protecting workers, and more. And the public should have a say in such decisions.