Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generation (2403.03344v1)

Published 5 Mar 2024 in cs.SE and cs.AI

Abstract: The increasing use of information technology has led to a significant share of energy consumption and carbon emissions from data centers. These contributions are expected to rise with the growing demand for big data analytics, increasing digitization, and the development of large AI models. The need to address the environmental impact of software development has led to increased interest in green (sustainable) coding and claims that the use of AI models can lead to energy efficiency gains. Here, we provide an empirical study on green code and an overview of green coding practices, as well as metrics used to quantify the sustainability awareness of AI models. In this framework, we evaluate the sustainability of auto-generated code. The auto-generate codes considered in this study are produced by generative commercial AI LLMs, GitHub Copilot, OpenAI ChatGPT-3, and Amazon CodeWhisperer. Within our methodology, in order to quantify the sustainability awareness of these AI models, we propose a definition of the code's "green capacity", based on certain sustainability metrics. We compare the performance and green capacity of human-generated code and code generated by the three AI LLMs in response to easy-to-hard problem statements. Our findings shed light on the current capacity of AI models to contribute to sustainable software development.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. IEA “Tracking Clean Energy Progress 2023”, 2023 URL: https://www.iea.org/reports/tracking-clean-energy-progress-2023
  2. “Recalibrating global data center energy-use estimates” In Science 367.6481, 2020, pp. 984–986
  3. “Energy consumption of data centers worldwide How will the Internet become green?”, 2020
  4. Ireland Central Statistics Office “Data Centres Metered Electricity Consumption 2022” ISSN: 2811-5422, 2023
  5. Amodei and Hernandez, 2018 URL: https://openai.com/research/ai-and-compute
  6. Emma Strubell, Ananya Ganesh and Andrew McCallum “Energy and Policy Considerations for Deep Learning in NLP” In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics Florence, Italy: Association for Computational Linguistics, 2019, pp. 3645–3650 DOI: 10.18653/v1/P19-1355
  7. “Carbon Emissions and Large Neural Network Training” In arXiv preprint arxiv: 2104.10350, 2021
  8. “Green AI” In Communications of the ACM 63.12 Association for Computing Machinery (ACM), 2020, pp. 54–63 DOI: 10.1145/3381831
  9. Radosvet Desislavov, Fernando Martinez-Plumed and Jose Hernander-Orallo “Trends in AI inference energy consumption: beyond the performance-vs-parameter laws of deep learning” In Sustainable Computing: Informatics and Systems 38, 2023, pp. 100857 DOI: 10.1016/j.suscom.2023.100857
  10. “Sustainable ai: Environmental implications, challenges and opportunities” In Proceedings of Machine Learning and Systems 4, 2022, pp. 795–813
  11. IEA “World Energy Outlook 2022”, 2022 URL: https://www.iea.org/reports/world-energy-outlook-2022
  12. “Carbon Dependencies in Datacenter Design and Management” In ACM SIGEnergy Energy Informatics Review 3.3 Association for Computing Machinery (ACM), 2023, pp. 21–26 DOI: 10.1145/3630614.3630619
  13. Sumit Gulwani, Oleksandr Polozov and Rishabh Singh “Program Synthesis” In Foundations and Trends® in Programming Languages 4.1-2, 2017, pp. 1–119 DOI: 10.1561/2500000010
  14. “Reconciling enumerative and deductive program synthesis” In Proceedings of the 41st ACM SIGPLAN International Conference on Programming Language Design and Implementation, PLDI 2020, London, UK, June 15-20, 2020 ACM, 2020, pp. 1159–1174 DOI: 10.1145/3385412.3386027
  15. “Write, execute, assess: Program synthesis with a repl” In Advances in Neural Information Processing Systems 32, 2019
  16. Sumit Gulwani “Dimensions in Program Synthesis” In Proceedings of the 12th International ACM SIGPLAN Symposium on Principles and Practice of Declarative Programming, PPDP ’10 Hagenberg, Austria: Association for Computing Machinery, 2010, pp. 13–24 DOI: 10.1145/1836089.1836091
  17. LeetCode “The World’s Leading Online Programming Learning Platform — leetcode.com” [Accessed 04-12-2023], https://leetcode.com/, 2010
  18. “Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning” arXiv, 2020 DOI: 10.48550/ARXIV.2002.05651
  19. “Data Center Cooling using Model-predictive Control” In Proceedings of the Thirty-second Conference on Neural Information Processing Systems (NeurIPS-18), 2018, pp. 3818–3827 URL: https://papers.nips.cc/paper/7638-data-center-cooling-using-model-predictive-control
  20. “Quantifying the Carbon Emissions of Machine Learning” arXiv, 2019 DOI: 10.48550/ARXIV.1910.09700
  21. David Richard So, Quoc V. Le and Chen Liang “The Evolved Transformer”, 2019 URL: https://arxiv.org/pdf/1901.11117.pdf
  22. “Constructing Fast Network through Deconstruction of Convolution” In Advances in Neural Information Processing Systems 31 Curran Associates, Inc., 2018 URL: https://proceedings.neurips.cc/paper_files/paper/2018/file/9719a00ed0c5709d80dfef33795dcef3-Paper.pdf
  23. “Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions” In 2018 International Joint Conference on Neural Networks (IJCNN) IEEE, 2018 DOI: 10.1109/ijcnn.2018.8489312
  24. “RAPL: memory power estimation and capping” In Proceedings of the 16th ACM/IEEE international symposium on Low power electronics and design, ISLPED’10 ACM, 2010 DOI: 10.1145/1840845.1840883
  25. “MobileNetV2: Inverted Residuals and Linear Bottlenecks” arXiv, 2018 DOI: 10.48550/ARXIV.1801.04381
  26. “Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning” arXiv, 2019 DOI: 10.48550/ARXIV.1906.04585
  27. “Accelerating Reinforcement Learning through GPU Atari Emulation” In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20 Vancouver, BC, Canada: Curran Associates Inc., 2020
  28. Alfredo Canziani, Adam Paszke and Eugenio Culurciello “An Analysis of Deep Neural Network Models for Practical Applications” arXiv, 2016 DOI: 10.48550/ARXIV.1605.07678
  29. Mingxing Tan and Quoc V. Le “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks” arXiv, 2019 DOI: 10.48550/ARXIV.1905.11946
  30. “ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design” In Lecture Notes in Computer Science Springer International Publishing, 2018, pp. 122–138 DOI: 10.1007/978-3-030-01264-9˙8
  31. Radosvet Desislavov, Fernando Martínez-Plumed and José Hernández-Orallo “Trends in AI inference energy consumption: Beyond the performance-vs-parameter laws of deep learning” In Sustainable Computing: Informatics and Systems 38, 2023, pp. 100857 DOI: https://doi.org/10.1016/j.suscom.2023.100857
  32. Arnaldo Carvalho Melo and Red Hat “The New Linux ’ perf ’ Tools”, 2010 URL: https://api.semanticscholar.org/CorpusID:10296207
  33. Javier Mancebo, Félix García and Coral Calero “A process for analysing the energy efficiency of software” In Information and Software Technology 134, 2021, pp. 106560 DOI: https://doi.org/10.1016/j.infsof.2021.106560
  34. Nicola Jones “The Information Factories: Data centers are chewing up vast amounts of energy — So researchers are trying to make them more efficient” In Nature 561.163, 2019, pp. 163–165 DOI: 10.1038/d41586-018-07025-z
  35. “Evaluating the Energy Efficiency of Deep Convolutional Neural Networks on CPUs and GPUs” In 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), 2016, pp. 88–95 DOI: 10.1109/BDCloud.2016.95
  36. Emma Strubell, Ananya Ganesh and Andrew McCallum “Energy and Policy Considerations for Deep Learning in NLP” In arXiv preprint arXiv:1906.02243, 2019
  37. “Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing” In Neural Information Processing Systems, 2020
  38. “The Framework Tax: Disparities Between Inference Efficiency in Research and Deployment”, 2023
  39. “A method to estimate the energy consumption of deep neural networks” In arXiv preprint arXiv:1709.04854, 2017
  40. “Energy efficiency across programming languages: how do energy, time, and memory relate?” In Proceedings of the 10th ACM SIGPLAN international conference on software language engineering, 2017, pp. 256–267
  41. William Brown “How is the speed of code review affected by activity, usage and code quality?”, 2023 arXiv:2305.05770 [cs.SE]
  42. “Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT”, 2023 DOI: 10.48550/arXiv.2304.10778
  43. “A Dataset for Analysis of Quality Code and Toxic Comments”, 2023, pp. 559–574 DOI: 10.1007/978-3-031-24985-3˙41
  44. Burak Yetiştiren, Eray Tüzün and Işık Özsoy “Assessing the Quality of GitHub Copilot’s Code Generation”, 2022 DOI: 10.1145/3558489.3559072
  45. “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter” In arXiv preprint arXiv: 1910.01108, 2020 DOI: 10.48550/arXiv.1910.01108
  46. “Learning both Weights and Connections for Efficient Neural Networks” In NIPS 2015, 2015 DOI: 10.48550/arXiv.1506.02626
  47. Dally Han “Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding” In ICLR 2016, 2016 DOI: 10.48550/arXiv.1510.00149
  48. Shazeer Fedus “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity” In arXiv preprint arXiv:2101.03961, 2021 DOI: 10.48550/arXiv.2101.03961
  49. “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer” In arXiv preprint arXiv:1701.06538, 2017 DOI: 10.48550/arXiv.1701.06538
  50. “GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding” In arXiv preprint arXiv:2006.16668, 2020 DOI: 10.48550/arXiv.2006.16668
  51. “Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping” In arXiv preprint arXiv:2002.06305, 2020 DOI: 10.48550/arXiv.2002.06305
  52. “Measuring the Carbon Intensity of AI in Cloud Instances” In ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 1877–1894 DOI: 10.1145/3531146.3533234
  53. Luccioni, Viguier and Ligozat “ESTIMATING THE CARBON FOOTPRINT OF BLOOM, A 176B PARAMETER LANGUAGE MODEL” In Journal of Machine Learning Research, 2023, pp. 1–15
  54. Tapomoy Adhikari “Investigating the Use of Natural Language Processing for Automated Code Generation” In SSRN Electronic Journal, 2023 DOI: 10.2139/ssrn.4419501
  55. “Automated Code generation, status, quality and validity in 2022”, 2022
  56. “Green AI: Do Deep Learning Frameworks Have Different Costs?”, 2022 DOI: 10.1145/3510003.3510221
  57. Richard J. Waldinger and Richard C.T. Lee “PROW: A Step Toward Automatic Program Writing” In International Joint Conference on Artificial Intelligence, 1969
  58. “On the Synthesis of a Reactive Module” In Proceedings of the 16th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’89 Austin, Texas, USA: Association for Computing Machinery, 1989, pp. 179–190 DOI: 10.1145/75277.75293
  59. Eric Schkufza, Rahul Sharma and Alex Aiken “Stochastic superoptimization” In Architectural Support for Programming Languages and Operating Systems, ASPLOS 2013, Houston, TX, USA, March 16-20, 2013 ACM, 2013, pp. 305–316 DOI: 10.1145/2451116.2451150
  60. Henry Massalin “Superoptimizer - A Look at the Smallest Program” In Proceedings of the Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS II), Palo Alto, California, USA, October 5-8, 1987 ACM Press, 1987, pp. 122–126 DOI: 10.1145/36206.36194
  61. “A Survey of Automatic Generation of Source Code Comments: Algorithms and Techniques” In IEEE Access PP, 2019, pp. 1–1 DOI: 10.1109/ACCESS.2019.2931579
  62. “An Empirical Evaluation of GitHub Copilot’s Code Suggestions” In Proceedings of the 19th International Conference on Mining Software Repositories, MSR ’22 Pittsburgh, Pennsylvania: Association for Computing Machinery, 2022, pp. 1–5 DOI: 10.1145/3524842.3528470
  63. “Jigsaw: Large Language Models meet Program Synthesis” In 44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022 ACM, 2022, pp. 1219–1231 DOI: 10.1145/3510003.3510203
  64. Sergey Mechtaev, Jooyong Yi and Abhik Roychoudhury “Angelix: scalable multiline program patch synthesis via symbolic analysis” In Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016 ACM, 2016, pp. 691–701 DOI: 10.1145/2884781.2884807
  65. “LooPy: interactive program synthesis with control structures” In Proc. ACM Program. Lang. 5.OOPSLA, 2021, pp. 1–29 DOI: 10.1145/3485530
  66. “A systematic evaluation of large language models of code” In MAPS@PLDI 2022: 6th ACM SIGPLAN International Symposium on Machine Programming, San Diego, CA, USA, 13 June 2022 ACM, 2022, pp. 1–10 DOI: 10.1145/3520312.3534862
  67. Patricia Lago, Qing Gu and Paolo Bozzelli “A systematic literature review of green software metrics”, 2014
  68. “Green AI: Do Deep Learning Frameworks Have Different Costs?” In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22 Pittsburgh, Pennsylvania: Association for Computing Machinery, 2022, pp. 1082–1094 DOI: 10.1145/3510003.3510221
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Tina Vartziotis (2 papers)
  2. Ippolyti Dellatolas (3 papers)
  3. George Dasoulas (20 papers)
  4. Maximilian Schmidt (27 papers)
  5. Florian Schneider (30 papers)
  6. Tim Hoffmann (18 papers)
  7. Sotirios Kotsopoulos (2 papers)
  8. Michael Keckeisen (3 papers)
Citations (4)
View on Semantic Scholar