Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
12 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
37 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Towards Responsible Governance of Biological Design Tools (2311.15936v3)

Published 27 Nov 2023 in cs.CY and cs.LG

Abstract: Recent advancements in generative machine learning have enabled rapid progress in biological design tools (BDTs) such as protein structure and sequence prediction models. The unprecedented predictive accuracy and novel design capabilities of BDTs present new and significant dual-use risks. For example, their predictive accuracy allows biological agents, whether vaccines or pathogens, to be developed more quickly, while the design capabilities could be used to discover drugs or evade DNA screening techniques. Similar to other dual-use AI systems, BDTs present a wicked problem: how can regulators uphold public safety without stifling innovation? We highlight how current regulatory proposals that are primarily tailored toward LLMs may be less effective for BDTs, which require fewer computational resources to train and are often developed in an open-source manner. We propose a range of measures to mitigate the risk that BDTs are misused, across the areas of responsible development, risk assessment, transparency, access management, cybersecurity, and investing in resilience. Implementing such measures will require close coordination between developers and governments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. 15 CFR 734.7. United States Code of Federal Regulations. URL https://www.ecfr.gov/current/title-15/subtitle-B/chapter-VII/subchapter-C/part-734/section-734.7.
  2. 15 CFR 734.8. United States Code of Federal Regulations. URL https://www.ecfr.gov/current/title-15/subtitle-B/chapter-VII/subchapter-C/part-734/section-734.8.
  3. 22 CFR 120.34. United States Code of Federal Regulations. URL https://www.ecfr.gov/current/title-22/chapter-I/subchapter-M/part-120/subpart-C/section-120.34.
  4. Frontier AI regulation: Managing emerging risks to public safety. arXiv:2307.03718, 2023.
  5. Anthropic. Frontier Threats Red Teaming for AI Safety. 2023. URL https://www.anthropic.com/index/frontier-threats-red-teaming-for-ai-safety.
  6. Apache Software Foundation. Apache license, version 2.0, section 8, 2004. URL https://www.apache.org/licenses/LICENSE-2.0.
  7. C. Arnold. Inside the nascent industry of AI-designed drugs. Nature Medicine, 2023.
  8. Accurate prediction of protein structures and interactions using a three-track neural network. Science, 373(6557):871–876, 2021.
  9. Bipartisan Commission on Biodefense. The Apollo Program for Biodefense: Winning the Race Against Biological Threats. Technical report, Bipartisan Commission on Biodefense, Washington, D.C., January 2021.
  10. Censoring chemical data to mitigate dual use risk. arXiv preprint arXiv:2304.10510, 2023.
  11. The Convergence of Artificial Intelligence and the Life Sciences: Safeguarding Technology, Rethinking Governance, and Preventing Catastrophe. 2023a.
  12. Benchtop DNA Synthesis Devices: Capabilities, Biosecurity Implications, and Governance. 2023b. URL {https://www.nti.org/news/new-report-recommends-biosecurity-safeguards-for-benchtop-dna-synthesis-devices/}.
  13. Accurate proteome-wide missense variant effect prediction with AlphaMissense. Science, page eadg7492, 2023.
  14. Supporting Open Source and Open Science in the EU AI Act. 2023. URL https://github.blog/wp-content/uploads/2023/07/Supporting-Open-Source-and-Open-Science-in-the-EU-AI-Act.pdf.
  15. Aum Shinrikyo: Insights into How Terrorists Develop Biological and Chemical Weapons. Center for a New American Security, 2011.
  16. EU Parliament. Amendments adopted by the European Parliament on 14 June 2023 on the […] Artificial Intelligence Act. 2023a. URL https://www.europarl.europa.eu/doceo/document/TA-9-2023-0236_EN.html.
  17. EU Parliament. Artificial intelligence liability directive. 2023b. URL https://www.europarl.europa.eu/RegData/etudes/BRIE/2023/739342/EPRS_BRI(2023)739342_EN.pdf.
  18. Free Software Foundation. GNU general public license, version 3.0, section 16, 2007. URL https://www.gnu.org/licenses/gpl-3.0.en.html.
  19. Predictability and surprise in large generative models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1747–1764, 2022.
  20. Study on the effect on the development of the information society of European public bodies making their own software available as open source. European Commission, 2007.
  21. Will releasing the weights of large language models grant widespread access to pandemic agents? arXiv preprint arXiv:2310.18233, 2023.
  22. Helena. Biosecurity in the Age of AI. 2023. URL https://www.helenabiosecurity.org/.
  23. Institute for Protein Design. Results from our summit on responsible AI. 2023. URL https://www.ipd.uw.edu/2023/10/responsible-ai-summit/.
  24. AI in small-molecule drug discovery: A coming wave. Nat. Rev. Drug Discov, 21:175–176, 2022.
  25. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583–589, 2021.
  26. Scaling laws for neural language models. arXiv:2001.08361, 2020.
  27. Thieves on Sesame Street! Model extraction of Bert-based APIs. arXiv:1910.12366, 2019.
  28. B. Kuhlman and P. Bradley. Advances in protein structure prediction and design. Nature Reviews Molecular Cell Biology, 20(11):681–697, 2019.
  29. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123–1130, 2023.
  30. E. J. Markey and T. Budd. Sens. Markey, Budd announce legislation to assess health security risks of AI. 2023. URL https://www.markey.senate.gov/news/press-releases/sens-markey-budd-announce-legislation-to-assess-health-security-risks-of-ai.
  31. N. Metcalfe. A short history of biological warfare. Medicine, Conflict and Survival, 18(3):271–282, 2002.
  32. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 220–229, 2019.
  33. Office for AI. A pro-innovation approach to AI regulation. 2023. URL https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach/white-paper.
  34. Strengthen oversight of risky research on pathogens. Science, 378(6625):1170–1172, 2022. doi: 10.1126/science.adf6020. URL https://www.science.org/doi/abs/10.1126/science.adf6020.
  35. Red-teaming the stable diffusion safety filter. arXiv:2210.04610, 2022.
  36. S. Rose and C. Nelson. Understanding AI-Facilitated Biological Weapon Development. The Centre for Long-Term Resilience, 2023. URL https://www.longtermresilience.org/post/report-launch-examining-risks-at-the-intersection-of-ai-and-bio.
  37. Going Viral: Bioeconomy Defense: Summary Report . 2023. URL https://www.isac.bio/post/going-viral-bioeconomy-defense-summary-report.
  38. J. B. Sandbrink. Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools. arXiv:2306.13952, 2023.
  39. M. Schreiner. GPT-4 architecture, datasets, costs and more leaked. 2023. URL https://the-decoder.com/gpt-4-architecture-datasets-costs-and-more-leaked/.
  40. Towards best practices in AGI safety and governance: A survey of expert opinion. arXiv preprint arXiv:2305.07153, 2023.
  41. Open-sourcing highly capable foundation models. 2023. URL https://www.governance.ai/research-paper/open-sourcing-highly-capable-foundation-models.
  42. Y. Shavit. What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring. arXiv:2303.11341, 2023.
  43. M. Sheehan. China’s AI Regulations and How They Get Made . 2023. URL https://carnegieendowment.org/2023/07/10/china-s-ai-regulations-and-how-they-get-made-pub-90117.
  44. T. Shevlane. Structured access: an emerging paradigm for safe AI deployment. arXiv:2201.05159, 2022.
  45. Model evaluation for extreme risks. arXiv:2305.15324, 2023.
  46. A Seismic Shift: The New U.S. Semiconductor Export Controls and the Implications for U.S. Firms, Allies, and the Innovation Ecosystem. 2023. URL https://www.csis.org/analysis/seismic-shift-new-us-semiconductor-export-controls-and-implications-us-firms-allies-and.
  47. Biosecurity in an age of open science. PLoS Biology, 20(4):e3001600, 2022.
  48. Can large language models democratize access to dual-use biotechnology? arXiv:2306.03809, 2023.
  49. I. Solaiman. The gradient of generative AI release: Methods and considerations. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 111–122, 2023.
  50. B. Soyer and A. Tettenborn. Artificial intelligence and civil liability—do we need a new regime? International Journal of Law and Information Technology, 30(4):385–397, 2022.
  51. Are current tort liability doctrines adequate for addressing injury caused by AI? AMA Journal of Ethics, 21(2):160–166, 2019.
  52. Synopsys. Open Source Security and Risk Analysis Report. 2023. URL https://www.synopsys.com/software-integrity/resources/analyst-reports/open-source-security-risk-analysis.html.
  53. Linking engineered cells to their digital twins: A version control system for strain engineering. ACS Synthetic Biology, 9(3):536–545, 2020. doi: 10.1021/acssynbio.9b00400. URL https://doi.org/10.1021/acssynbio.9b00400.
  54. UK Government. Emerging processes for frontier AI safety. 2023.
  55. Dual use of artificial-intelligence-powered drug discovery. Nature Machine Intelligence, 4(3):189–191, 2022.
  56. U.S. Department of Health & Human Services. Dual Use Research of Concern. 2021. URL https://www.phe.gov/s3/dualuse/Pages/default.aspx.
  57. AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic Acids Research, 50(D1):D439–D444, 2022.
  58. C. Watney. Where Can Federal AI R&D Funding Go the Furthest? 2023. URL https://ifp.org/where-can-federal-ai-rd-funding-go-the-furthest/.
  59. De novo design of protein structure and function with RFdiffusion. Nature, pages 1–3, 2023.
  60. Emergent abilities of large language models. arXiv:2206.07682, 2022.
  61. White House. Exec. Order No. 14110, 88 Fed. Reg. 210 (October 30, 2023)., 2023a.
  62. White House. FACT SHEET: Biden–Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI. 2023b. URL https://www.whitehouse.gov/briefing-room/statements-releases/2023/07/21/fact-sheet-biden-harris-administration-secures-voluntary-commitments-from-leading-artificial-intelligence-companies-to-manage-the-risks-posed-by-ai/.
  63. B. Williams and R. Kane. Preventing the Misuse of DNA Synthesis. 2023. URL https://progress.institute/preventing-the-misuse-of-dna-synthesis/.
  64. Understanding Open Source Technology & US Export Controls. Technical report, The Linux Foundation, 2021.
  65. Guarding Against Catastrophic Biological Risks: Preventing State Biological Weapon Development and Use by Shaping Intentions. Health Security, 2023.
  66. S. Zeitouni. Milling the F/LOSS: Export Controls, Free and Open Source Software, and the Regulatory Future of the Internet. NYUJ Legis. & Pub. Pol’y, 23:905, 2020.
Citations (2)

Summary

We haven't generated a summary for this paper yet.