Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On the Societal Impact of Open Foundation Models (2403.07918v1)

Published 27 Feb 2024 in cs.CY, cs.AI, and cs.LG
On the Societal Impact of Open Foundation Models

Abstract: Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.

PDF HTML Abstract

On the Societal Impact of Open Foundation Models

The paper "On the Societal Impact of Open Foundation Models" examines the implications of open foundation models whose weights are publicly available, such as Llama 2 and Stable Diffusion XL. The authors categorize the societal consequences into benefits and risks, attempting to strike a balance between them through a nuanced framework.

Distinctive Properties

Five distinctive properties of open foundation models are identified:

  1. Broader Access: These models are characterized by the wide availability of weights, facilitating greater inclusivity in usage.
  2. Greater Customizability: Users can modify these models for specific applications due to open access to weights.
  3. Local Adaptation and Inference: These models can run locally, circumventing the necessity to share data with external servers.
  4. Irrevocability of Access: Once released, access to model weights cannot be revoked, raising potential misuse concerns.
  5. Weaker Monitoring: Monitoring usage is challenging, especially for local instances.

Benefits

The paper articulates several benefits derived from these properties:

  • Distributing Decision-Making Power: Open models allow for a more decentralized determination of acceptable behavior, diversifying perspectives and applications.
  • Innovation and Competition: The accessibility of open models accelerates new use cases and market innovation, though this must be weighed against the potential for model fragmentation and the comparative advantages of closed models in improvement and feedback.
  • Scientific Research: Open models enhance reproducibility and inclusion in research, though the complete benefits hinge on access to data and other assets beyond weights.
  • Transparency: Open models permit greater scrutiny and accountability, particularly pertinent given historic opacity in digital technologies.
  • Mitigating Market Concentration: By enabling smaller entities to compete, open models could distribute market power more evenly.

Risks

The authors provide a framework to assess the societal risks of open foundation models, emphasizing the marginal risk:

  • Misuse Vectors: The risks are articulated across vectors such as cybersecurity, biosecurity, disinformation, and other forms of social harm.
  • Marginal Risk Framework: This framework assesses whether these models increase societal risk relative to pre-existing technologies or closed models and considers the existing defenses and their potential efficacy against newly introduced risks.

Evidence and Analysis

The paper's strength lies in its structured approach to risk assessment, although it acknowledges the current insufficiency of data in several areas. The framework aims to stimulate precise discourse on the marginal risks of open models, contrasting with pre-existing risks from other technologies.

Recommendations

The paper concludes with targeted recommendations:

  • AI Developers: Transparency and clear delineation of responsibility for safe and responsible use are critical.
  • Risk Researchers: Further empirical work is needed to elucidate the nuanced balance of risks and benefits.
  • Policymakers and Regulators: The authors suggest thoughtful regulation that considers the unique challenges posed by open models, without stifling competition or innovation.

Conclusion

This comprehensive examination provides a rational basis for ongoing debates on open foundation models, emphasizing the need for collaborative efforts among developers, researchers, and policymakers to harness their benefits while mitigating associated risks. The research implies a call to action for empirical studies to inform future AI governance and deployment strategies.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (146)
  1. Automating Image Abuse: Deepfake Bots on Telegram. Sensity, October 2020.
  2. Amos, Z. What Is FraudGPT?, August 2023. URL https://hackernoon.com/what-is-fraudgpt.
  3. Frontier AI Regulation: Managing Emerging Risks to Public Safety, November 2023. URL http://arxiv.org/abs/2307.03718. arXiv:2307.03718 [cs].
  4. Apple Support. Safely open apps on your Mac, September 2023. URL https://support.apple.com/en-us/HT202491.
  5. Identifying and Mitigating the Security Risks of Generative AI. Foundations and Trends® in Privacy and Security, 6(1):1–52, December 2023. ISSN 2474-1558, 2474-1566. doi: 10.1561/3300000041. URL https://www.nowpublishers.com/article/Details/SEC-041. Publisher: Now Publishers, Inc.
  6. Batalis, S. Can Chatbots Help You Build a Bioweapon?, November 2023. URL https://foreignpolicy.com/2023/11/05/ai-artificial-intelligence-chatbot-bioweapon-virus-bacteria-genetic-engineering/.
  7. Biderman, S. Good Work Enabled by Open Models, November 2023. URL https://docs.google.com/spreadsheets/d/1kt5jp1U50AfDGEKsqFXvtCCQ8HYvQGcya1vf_OaD8x0/edit?usp=sharing&usp=embed_facebook.
  8. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp.  2397–2430. PMLR, 2023.
  9. Into the LAIONs Den: Investigating Hate in Multimodal Datasets, November 2023. URL http://arxiv.org/abs/2311.03449. arXiv:2311.03449 [cs].
  10. Gpt-neox-20b: An open-source autoregressive language model. In Proceedings of BigScience Episode# 5–Workshop on Challenges & Perspectives in Creating Large Language Models, pp.  95–136, 2022.
  11. Letter on the leak of meta’s ai model. US Senate, 2023a. URL https://www.blumenthal.senate.gov/imo/media/doc/06062023metallamamodelleakletter.pdf.
  12. Bipartisan Framework for U.S. AI Act, September 2023b. URL https://www.blumenthal.senate.gov/imo/media/doc/09072023bipartisanaiframework.pdf.
  13. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  14. Picking on the same person: Does algorithmic monoculture lead to outcome homogenization? In Advances in Neural Information Processing Systems, 2022.
  15. Considerations for Governing Open Foundation Models, December 2023a. URL https://hai.stanford.edu/issue-brief-considerations-governing-open-foundation-models.
  16. The foundation model transparency index, 2023b.
  17. Ecosystem graphs: The social footprint of foundation models. ArXiv, abs/2303.15772, 2023c. URL https://api.semanticscholar.org/CorpusID:257771875.
  18. AI might help unleash the next pandemic, MIT study says - The Boston Globe, 2023. URL https://www.bostonglobe.com/2023/06/28/business/ai-might-help-unleash-next-pandemic-mit-study-says/.
  19. Broughton, A. Tennessee man charged in ’virtual pornography’ case, June 2009. URL https://edition.cnn.com/2009/CRIME/06/24/virtual.child.porn/index.html.
  20. The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation, February 2018. URL http://arxiv.org/abs/1802.07228. arXiv:1802.07228 [cs].
  21. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency, pp.  77–91, 2018.
  22. Cañas, J. Illegal use of artificial intelligence to create deepfakes depicting sexual content of minors. European Parliament, 2023. URL https://www.europarl.europa.eu/doceo/document/E-9-2023-002788_EN.html.
  23. Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security, July 2018. URL https://papers.ssrn.com/abstract=3213954.
  24. Chinese National Information Security Standardization Technical Committee. Basic Safety Requirements for Generative Artificial Intelligence Services (Draft for Feedback), November 2023. URL https://cset.georgetown.edu/publication/china-safety-requirements-for-generative-ai/.
  25. Tools for verifying neural models’ training data. arXiv preprint arXiv:2307.00682, 2023.
  26. Cihon, P. How to get AI regulation right for open source, July 2023. URL https://github.blog/2023-07-26-how-to-get-ai-regulation-right-for-open-source/.
  27. Cole, S. AI-Assisted Fake Porn Is Here and We’re All Fucked, December 2017. URL https://www.vice.com/en/article/gydydm/gal-gadot-fake-ai-porn.
  28. CraigMarcho. IE7 - Introducing the Phishing Filter, March 2007. URL https://techcommunity.microsoft.com/t5/ask-the-performance-team/ie7-introducing-the-phishing-filter/ba-p/372327.
  29. Supporting Open Source and Open Science in the EU AI Act. Technical report, July 2023. URL https://github.blog/wp-content/uploads/2023/07/Supporting-Open-Source-and-Open-Science-in-the-EU-AI-Act.pdf.
  30. Machine-generated text: A comprehensive survey of threat models and detection methods. IEEE Access, 11:70977–71002, 2023. doi: 10.1109/ACCESS.2023.3294090.
  31. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314, 2023.
  32. Drake, V. Threat Modeling, August 2021. URL https://owasp.org/www-community/Threat_Modeling.
  33. Executive Office of the President. Safe, secure, and trustworthy development and use of artificial intelligence. Executive Order, 10 2023. URL https://www.federalregister.gov/documents/2023/10/30/2023-24110/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence. Federal Register Vol. 88, No. 210 (October 30, 2023).
  34. Flavio Ceolin. Two buffer overflow vulnerabilities in Zephyr USB code, 2023. URL https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-4vgv-5r6q-r6xh.
  35. OPTQ: Accurate Quantization for Generative Pre-trained Transformers. 2023. URL https://openreview.net/forum?id=tcbBPnfwxS.
  36. G7 Hiroshima Summit. Hiroshima Process International Guiding Principles for Advanced AI system. Technical report, October 2023. URL https://www.mofa.go.jp/files/100573471.pdf.
  37. Google. Email sender guidelines, December 2023. URL https://support.google.com/mail/answer/81126?hl=en.
  38. Will releasing the weights of future large language models grant widespread access to pandemic agents?, November 2023. URL http://arxiv.org/abs/2310.18233. arXiv:2310.18233 [cs].
  39. Moderating Model Marketplaces: Platform Governance Puzzles for AI Intermediaries, November 2023. URL http://arxiv.org/abs/2311.12573. arXiv:2311.12573 [cs].
  40. Ai regulation has its own alignment problem: The technical and institutional feasibility of disclosure, registration, licensing, and auditing. George Washington Law Review, Symposium on Legally Disruptive Emerging Technologies, 2023.
  41. Medalpaca–an open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.08247, 2023.
  42. Hao, K. Deepfake porn is ruining women’s lives. Now the law may finally ban it., February 2021. URL https://www.technologyreview.com/2021/02/12/1018222/deepfake-revenge-porn-coming-ban/.
  43. Cleaning up chatgpt takes heavy toll on human workers. The Wall Street Journal, July 2023. URL https://www.wsj.com/articles/chatgpt-openai-content-abusive-sexually-explicit-harassment-kenya-workers-on-human-workers-cf191483. Photographs by Natalia Jidovanu.
  44. Harwell, D. AI-generated child sex images spawn new nightmare for the web. Washington Post, June 2023. ISSN 0190-8286. URL https://www.washingtonpost.com/technology/2023/06/19/artificial-intelligence-child-sex-abuse-images/.
  45. Hazell, J. Large Language Models Can Be Used To Effectively Scale Spear Phishing Campaigns, May 2023. URL http://arxiv.org/abs/2305.06972. arXiv:2305.06972 [cs].
  46. Foundation models and fair use. arXiv preprint arXiv:2303.15715, 2023.
  47. Illinois General Assembly. Full Text of HB2123, July 2023. URL https://www.ilga.gov/legislation/fulltext.asp?DocName=&SessionId=112&GA=103&DocTypeId=HB&DocNum=2123&GAID=17&LegID=145586&SpecSess=&Session=.
  48. Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy. In Keet, C. M., Lee, H.-Y., and Zarrieß, S. (eds.), Proceedings of the 16th International Natural Language Generation Conference, pp.  28–53, Prague, Czechia, September 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.inlg-main.3. URL https://aclanthology.org/2023.inlg-main.3.
  49. Joshi, S. They Follow You on Instagram, Then Use Your Face To Make Deepfake Porn in This Sex Extortion Scam, September 2021. URL https://www.vice.com/en/article/z3x9yj/india-instagram-sextortion-phishing-deepfake-porn-scam.
  50. OpenAI’s policies hinder reproducible research on language models, March 2023. URL https://www.aisnakeoil.com/p/openais-policies-hinder-reproducible.
  51. Promises and pitfalls of artificial intelligence for legal applications, January 2024. URL http://arxiv.org/abs/2402.01656. arXiv:2402.01656 [cs].
  52. Kaspersky. How real is deepfake threat? Kaspersky Daily, 2023.
  53. Metasploit: The Penetration Tester’s Guide. No Starch Press, USA, 1st edition, 2011. ISBN 159327288X.
  54. A Watermark for Large Language Models. In Proceedings of the 40th International Conference on Machine Learning, pp.  17061–17084. PMLR, July 2023. URL https://proceedings.mlr.press/v202/kirchenbauer23a.html. ISSN: 2640-3498.
  55. Algorithmic monoculture and social welfare. Proceedings of the National Academy of Sciences, 118(22):e2018340118, 2021. doi: 10.1073/pnas.2018340118. URL https://www.pnas.org/doi/abs/10.1073/pnas.2018340118.
  56. Spectre Attacks: Exploiting Speculative Execution, January 2018. URL http://arxiv.org/abs/1801.01203. arXiv:1801.01203 [cs].
  57. Kocsis, E. Deepfakes, Shallowfakes, and the Need for a Private Right of Action Comments. PennState Law Review, 126(2):621–650, 2021. URL https://heinonline.org/HOL/P?h=hein.journals/dknslr126&i=633.
  58. Control Systems Cyber Security:Defense in Depth Strategies. Technical Report INL/EXT-06-11478, Idaho National Lab. (INL), Idaho Falls, ID (United States), May 2006. URL https://www.osti.gov/biblio/911553.
  59. Lakatos, S. A Revealing Picture: AI-Generated ‘Undressing’ Images Move from Niche Pornography Discussion Forums to a Scaled and Monetized Online Business. Technical report, December 2023. URL https://public-assets.graphika.com/reports/graphika-report-a-revealing-picture.pdf.
  60. Lambda. GPU Cloud - VMs for Deep Learning, February 2024. URL https://web.archive.org//web/20240226155153/https://lambdalabs.com/service/gpu-cloud#pricing.
  61. Lazar, S. Governing the algorithmic city. Tanner Lectures, 2023. URL https://write.as/sethlazar/.
  62. Bloom: A 176b-parameter open-access multilingual language model. 2022. doi: 10.48550/ARXIV.2211.05100. URL https://arxiv.org/abs/2211.05100.
  63. Do language models plagiarize? arXiv preprint arXiv:2203.07618, 2022.
  64. Talkin”bout ai generation: Copyright and the generative-ai supply chain. arXiv preprint arXiv:2309.08133, 2023.
  65. Structural similarities between language models and neural response measurements, 2023.
  66. The time is now to develop community norms for the release of foundation models. City, 2022a.
  67. The time is now to develop community norms for the release of foundation models, 2022b. URL https://crfm.stanford.edu/2022/05/17/community-norms.html.
  68. Meltdown, January 2018. URL http://arxiv.org/abs/1801.01207. arXiv:1801.01207 [cs].
  69. AI-Powered Fuzzing: Breaking the Bug Hunting Barrier, 2023. URL https://security.googleblog.com/2023/08/ai-powered-fuzzing-breaking-bug-hunting.html.
  70. Llach, L. Naked deepfake images of teenage girls shock spanish town: But is it an ai crime? EuroNews, 2023. URL https://www.euronews.com/next/2023/09/24/spanish-teens-received-deepfake-ai-nudes-of-themselves-but-is-it-a-crime.
  71. The data provenance initiative: A large scale audit of dataset licensing & attribution in ai. arXiv preprint arXiv:2310.16787, 2023a.
  72. A pretrainer’s guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity. arXiv preprint arXiv:2305.13169, 2023b.
  73. Letter to the National Institute of Standards and Technology Director Laurie Locascio from the House Committee on Science, Space, and Technology, December 2023. URL https://republicans-science.house.gov/_cache/files/8/a/8a9f893d-858a-419f-9904-52163f22be71/191E586AF744B32E6831A248CD7F4D41.2023-12-14-aisi-scientific-merit-final-signed.pdf.
  74. Maiberg, E. Civitai and OctoML Introduce Radical New Measures to Stop Abuse After 404 Media Investigation, December 2023a. URL https://www.404media.co/civitai-and-octoml-introduce-radical-new-measures-to-stop-abuse-after-404-media-investigation/.
  75. Maiberg, E. Inside the AI Porn Marketplace Where Everything and Everyone Is for Sale, August 2023b. URL https://www.404media.co/inside-the-ai-porn-marketplace-where-everything-and-everyone-is-for-sale/.
  76. Matthews, D. Scientists grapple with risk of artificial intelligence-created pandemics, 2023. URL https://sciencebusiness.net/news/ai/scientists-grapple-risk-artificial-intelligence-created-pandemics.
  77. MITRE. Adversarial threat landscape for ai systems, 2021. URL https://atlas.mitre.org/.
  78. Mortimer, S. StopNCII.org has launched, December 2021. URL https://revengepornhelpline.org.uk/news/stopncii-org-has-launched/.
  79. The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study. Technical report, RAND Corporation, January 2024. URL https://www.rand.org/pubs/research_reports/RRA2977-2.html.
  80. Mozilla. Joint Statement on AI Safety and Openness, October 2023. URL https://open.mozilla.org/letter/.
  81. Musser, M. A Cost Analysis of Generative Language Models and Influence Operations, August 2023. URL http://arxiv.org/abs/2308.03740. arXiv:2308.03740 [cs].
  82. Generative ai companies must publish transparency reports, 2023a. URL https://knightcolumbia.org/blog/generative-ai-companies-must-publish-transparency-reports.
  83. GPT-4 and professional benchmarks: the wrong answer to the wrong question, March 2023b. URL https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmarks.
  84. Model alignment protects against accidental harms, not intentional ones, December 2023. URL https://www.aisnakeoil.com/p/model-alignment-protects-against.
  85. Dual use foundation artificial intelligence models with widely available model weights. Request for Comment, 02 2024. URL https://www.ntia.gov/federal-register-notice/2024/dual-use-foundation-artificial-intelligence-models-widely-available. Federal Register.
  86. Noble, S. U. Algorithms of Oppression. New York University Press, 2018.
  87. OpenAI. Better language models and their implications, 2019a. URL https://openai.com/research/better-language-models.
  88. OpenAI. GPT-2: 1.5B release, 2019b. URL https://openai.com/research/gpt-2-1-5b-release.
  89. Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning, July 2019. URL http://arxiv.org/abs/1907.11274. arXiv:1907.11274 [cs].
  90. Does writing with language models reduce content diversity?, 2023.
  91. Deepfakes and Cheap Fakes, September 2019. URL https://datasociety.net/library/deepfakes-and-cheap-fakes/. Publisher: Data & Society Research Institute.
  92. Social Simulacra: Creating Populated Prototypes for Social Computing Systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22, pp.  1–18, New York, NY, USA, October 2022. Association for Computing Machinery. ISBN 978-1-4503-9320-1. doi: 10.1145/3526113.3545616. URL https://dl.acm.org/doi/10.1145/3526113.3545616.
  93. Can sensitive information be deleted from llms? objectives for defending against extraction attacks. arXiv preprint arXiv:2309.17410, 2023.
  94. Building an early warning system for LLM-aided biological threat creation, January 2024. URL https://openai.com/research/building-an-early-warning-system-for-llm-aided-biological-threat-creation.
  95. Perrigo, B. Openai used kenyan workers on less than 2 per hour to make chatgpt less toxic, Jan 2023. URL https://time.com/6247678/openai-chatgpt-kenya-workers/.
  96. Typhoon: Thai Large Language Models, December 2023. URL http://arxiv.org/abs/2312.13951. arXiv:2312.13951 [cs].
  97. Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023.
  98. Quintero, B. Introducing virustotal code insight: Empowering threat analysis with generative ai, apr 2023. URL https://blog.virustotal.com/2023/04/introducing-virustotal-code-insight.html.
  99. Improving language understanding by generative pre-training. Technical report, OpenAI, 2018.
  100. Language models are unsupervised multitask learners. 2019a.
  101. Language Models are Unsupervised Multitask Learners. 2019b. URL https://www.semanticscholar.org/paper/Language-Models-are-Unsupervised-Multitask-Learners-Radford-Wu/9405cc0d6169988371b2755e573cc28650d14dfe.
  102. Raffel, C. Building Machine Learning Models Like Open Source Software, February 2023. URL https://cacm.acm.org/magazines/2023/2/268952-building-machine-learning-models-like-open-source-software/fulltext.
  103. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’19, pp.  429–435, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450363242. doi: 10.1145/3306618.3314244. URL https://doi.org/10.1145/3306618.3314244.
  104. Reid, S. The Deepfake Dilemma: Reconciling Privacy and First Amendment Protections, June 2020. URL https://papers.ssrn.com/abstract=3636464.
  105. Roberts, J. J. Fake Porn Videos Are Terrorizing Women. Do We Need a Law to Stop Them?, January 2019. URL https://fortune.com/2019/01/15/deepfakes-law/.
  106. Saliba, E. Sharing deepfake pornography could soon be illegal in America, June 2023. URL https://abcnews.go.com/Politics/sharing-deepfake-pornography-illegal-america/story?id=99084399.
  107. Sandbrink, J. B. Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools, 2023.
  108. Sastry, G. Beyond “release” vs. “not release”, 2021. URL https://crfm.stanford.edu/commentary/2021/10/18/sastry.html.
  109. Satter, R. FBI says artificial intelligence being used for ’sextortion’ and harassment. Reuters, June 2023. URL https://www.reuters.com/world/us/fbi-says-artificial-intelligence-being-used-sextortion-harassment-2023-06-07/.
  110. Scott, D. Deepfake Porn Nearly Ruined My Life, February 2020. URL https://www.elle.com/uk/life-and-culture/a30748079/deepfake-porn/. Section: Life + Culture.
  111. Seaman, J. Cyber threat prediction and modelling. In Montasari, R. (ed.), Artificial Intelligence and National Security. Springer, Cham, 2022. doi: 10.1007/978-3-031-06709-9˙7.
  112. Open-sourcing highly capable foundation models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives. 2023.
  113. Service, R. F. Could chatbots help devise the next pandemic virus?, 2023. URL https://www.science.org/content/article/could-chatbots-help-devise-next-pandemic-virus.
  114. Shevlane, T. Structured access: an emerging paradigm for safe ai deployment. arXiv preprint arXiv:2201.05159, 2022.
  115. The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse? In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES ’20, pp.  173–179, New York, NY, USA, February 2020. Association for Computing Machinery. ISBN 978-1-4503-7110-0. doi: 10.1145/3375627.3375815. URL https://doi.org/10.1145/3375627.3375815.
  116. Shostack, A. Threat Modeling: Designing for Security. John Wiley & Sons, Feb 2014. ISBN 978-1-118-80999-0. The only security book to be chosen as a Dr. Dobbs Jolt Award Finalist since Bruce Schneier’s Secrets and Lies and Applied Cryptography!
  117. Siddique, H. Sharing deepfake intimate images to be criminalised in England and Wales. The Guardian, June 2023. ISSN 0261-3077. URL https://www.theguardian.com/society/2023/jun/27/sharing-deepfake-intimate-images-to-be-criminalised-in-england-and-wales.
  118. Can large language models democratize access to dual-use biotechnology?, 2023.
  119. Solaiman, I. The gradient of generative ai release: Methods and considerations. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp.  111–122, 2023.
  120. Release Strategies and the Social Impacts of Language Models, November 2019a. URL http://arxiv.org/abs/1908.09203. arXiv:1908.09203 [cs].
  121. Release strategies and the social impacts of language models. ArXiv, abs/1908.09203, 2019b.
  122. Solsman, J. E. A deepfake bot is creating nudes out of regular photos, October 2020. URL https://www.cnet.com/news/privacy/deepfake-bot-on-telegram-is-violating-women-by-forging-nudes-from-regular-pics/.
  123. Staff, A. Meltdown and Spectre: Here’s what Intel, Apple, Microsoft, others are doing about it, January 2018. URL https://arstechnica.com/gadgets/2018/01/meltdown-and-spectre-heres-what-intel-apple-microsoft-others-are-doing-about-it/.
  124. Sweeney, L. Discrimination in online ad delivery. Queue, 11(3):10:10–10:29, March 2013. ISSN 1542-7730. doi: 10.1145/2460276.2460278. URL http://doi.acm.org/10.1145/2460276.2460278.
  125. Fuzzing for Software Security. Artech House Publishers, Norwood, MA, 1st edition edition, July 2008. ISBN 978-1-59693-214-2.
  126. Online Consent Moderation, December 2020. URL https://cyber.fsi.stanford.edu/io/news/ncii-legislation-limitations.
  127. Generative ML and CSAM: Implications and Mitigations. 2023. doi: 10.25740/jv206yg3793. URL https://purl.stanford.edu/jv206yg3793.
  128. Joma: Demystifying multilayer transformers via joint dynamics of mlp and attention. arXiv preprint arXiv:2310.00535, 2023.
  129. Generative AI could revolutionize health care — but not if control is ceded to big tech. Nature, 624(7990):36–38, December 2023. doi: 10.1038/d41586-023-03803-y. URL https://www.nature.com/articles/d41586-023-03803-y. Bandiera_abtest: a Cg_type: Comment Number: 7990 Publisher: Nature Publishing Group Subject_term: Machine learning, Health care, Medical research, Technology.
  130. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a.
  131. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023b.
  132. UK AISI. Introducing the ai safety institute, 2023. URL https://www.gov.uk/government/publications/ai-safety-institute-overview/introducing-the-ai-safety-institute.
  133. UK CMA. Ai foundation models: Initial report, 2023. URL https://assets.publishing.service.gov.uk/media/65081d3aa41cc300145612c0/Full_report_.pdf.
  134. Vincent, J. Meta’s powerful AI language model has leaked online — what happens now?, March 2023. URL https://www.theverge.com/2023/3/8/23629362/meta-ai-language-model-llama-leak-online-misuse.
  135. Market concentration implications of foundation models: The invisible hand of chatgpt. The Brookings Institution, 2023. URL https://www.brookings.edu/articles/market-concentration-implications-of-foundation-models-the-invisible-hand-of-chatgpt.
  136. Directional Bias Amplification. In Proceedings of the 38th International Conference on Machine Learning, pp.  10882–10893. PMLR, July 2021. URL https://proceedings.mlr.press/v139/wang21t.html. ISSN: 2640-3498.
  137. Gpt-j-6b: A 6 billion parameter autoregressive language model, 2021.
  138. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. In The Eleventh International Conference on Learning Representations, 2022.
  139. Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, pp.  2035–2046, New York, NY, USA, June 2022. Association for Computing Machinery. ISBN 978-1-4503-9352-2. doi: 10.1145/3531146.3533779. URL https://dl.acm.org/doi/10.1145/3531146.3533779.
  140. Open (for business): Big tech, concentrated power, and the political economy of open ai. 2023.
  141. Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning, October 2023. URL http://arxiv.org/abs/2310.06694. arXiv:2310.06694 [cs].
  142. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint arXiv:2310.02949, 2023.
  143. Defending against neural fake news. In Advances in Neural Information Processing Systems (NeurIPS), pp.  9054–9065, 2019.
  144. LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention, June 2023. URL http://arxiv.org/abs/2303.16199. arXiv:2303.16199 [cs].
  145. OPT: Open pre-trained transformer language models. arXiv, 2022.
  146. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (25)
  1. Sayash Kapoor (23 papers)
  2. Rishi Bommasani (28 papers)
  3. Kevin Klyman (17 papers)
  4. Shayne Longpre (49 papers)
  5. Ashwin Ramaswami (2 papers)
  6. Peter Cihon (9 papers)
  7. Aspen Hopkins (4 papers)
  8. Kevin Bankston (2 papers)
  9. Stella Biderman (55 papers)
  10. Miranda Bogen (9 papers)
  11. Rumman Chowdhury (11 papers)
  12. Alex Engler (1 paper)
  13. Peter Henderson (67 papers)
  14. Yacine Jernite (46 papers)
  15. Seth Lazar (13 papers)
  16. Stefano Maffulli (2 papers)
  17. Alondra Nelson (4 papers)
  18. Joelle Pineau (123 papers)
  19. Aviya Skowron (8 papers)
  20. Dawn Song (229 papers)
Citations (40)
View on Semantic Scholar
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube

HackerNews

  1. The Societal Impact of Open Foundation Models (2 points, 0 comments)