Papers
Topics
Authors
Recent
Search
2000 character limit reached

The False Dawn: Reevaluating Google's Reinforcement Learning for Chip Macro Placement

Published 16 Jun 2023 in cs.LG, cs.AI, cs.AR, and cs.CY | (2306.09633v10)

Abstract: Reinforcement learning (RL) for physical design of silicon chips in a Google 2021 Nature paper stirred controversy due to poorly documented claims that raised eyebrows and drew critical media coverage. The paper withheld critical methodology steps and most inputs needed to reproduce results. Our meta-analysis shows how two separate evaluations filled in the gaps and demonstrated that Google RL lags behind (i) human designers, (ii) a well-known algorithm (Simulated Annealing), and (iii) generally-available commercial software, while being slower; and in a 2023 open research contest, RL methods weren't in top 5. Crosschecked data indicate that the integrity of the Nature paper is substantially undermined owing to errors in conduct, analysis and reporting. Before publishing, Google rebuffed internal allegations of fraud, which still stand. We note policy implications and conclusions for chip design.

Authors (1)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. Peer Review File for the Nature paper released by the editor: https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-021-03544-w/MediaObjects/41586_2021_3544_MOESM1_ESM.pdf
  2. Circuit Training: An Open-source Framework for Generating Chip Floorplans with Distributed Deep Reinforcement Learning. https://github.com/googleresearch/circuit_training,
  3. Summer Yue, Ebrahim M. Songhori, Joe Wenjie Jiang, Toby Boyd, Anna Goldie, Azalia Mirhoseini, Sergio Guadarrama, “Scalability and Generalization of Circuit Training for Chip Floorplanning,” Proceedings of Int’l Symp. Physical Design 2022: 65-70.
  4. Sungmin Bae, Amir Yazdanbaksh, Myung-Chul Kim, Satrajit Chatterjee, Mingyu Woo, Igor. L. Markov, “Stronger Baselines for Evaluating Deep Reinforcement Learning in Chip Placement”, March, 2022.161616 [58] disclosed some authors of the Stronger Baselines paper [5]. Apparently the authorship was well-known within Google from 2021. https://statmodeling.stat.columbia.edu/wp-content/uploads/2022/05/MLcontra.pdf
  5. MacroPlacement Repo. https://github.com/TILOS-AI-Institute/MacroPlacement
  6. Chung-Kuan Cheng, Andrew B. Kahng, Sayak Kundu, Yucheng Wang, Zhiang Wang, “Assessment of Reinforcement Learning for Macro Placement”, Proceedings of Int’l Symp. Physical Design 2023: 158-166, (arXiv:2302:11014).
  7. “Extraordinary claims require extraordinary evidence” (a quote by Carl Sagan), the Wikipedia https://en.wikipedia.org/wiki/Extraordinary_claims_require_extraordinary_evidence
  8. John P.A. Ioannidis, “Why Most Published Research Findings are False,” PLoS Med. 2, e124, 2005 doi:10.1371/journal.pmed.00201244commentsonPubPeer(by:Peer1,UlrichSchimmack,MatthiasWjst,MatthewElvey), https://en.wikipedia.org/wiki/Why_Most_Published_Research_Findings_Are_False
  9. Scott Kirkpatrick, C. Daniel Gelatt, Jr., Mario P. Vecchi, “Optimization by Simulated Annealing,” Science 1983, 220(4598),pp.671-680.
  10. Chung-Kuan Cheng, Ernest S. Kuh, “Module Placement Based on Resistive Network Optimization,” IEEE Transactions on Computer-Aided Design of Integrated Circuits 3(3), pp. 218-225 (1984)
  11. Carl Sechen, Alberto Sangiovanni-Vincentelli, “The Timberwolf Placement and Routing Package,” IEEE Journal of Solid-State Circuits, SC-20(2), pp. 510-522, Apr. 1985.
  12. Saul A. Kravitz, Rob A. Rutenbar, “Placement by Simulated Annealing on a Multiprocessor,” IEEE Transactions on Computer-Aided Design of Integrated Circuits 6(4): 534-549 (1987)
  13. D.-F. Wong, Hon Wai Leong, C.L. Liu, “Simulated annealing for VLSI design,” Springer, 1988.
  14. William Swartz, Carl Sechen, “Timing Driven Placement for Large Standard Cell Circuits,” Proceedings of Design Automation Conf. 1995, pp. 211-215.
  15. Vaughn Betz and Jonathan Rose, “VPR: A new packing, placement and routing tool for FPGA research,” Field-Programmable Logic and Applications, pp. 213–222, 1997.
  16. Andrew B. Kahng, “Classical floorplanning harmful?’ Proceedings of Int’l Symp. Physical Design 2000, pp. 207-213.
  17. Andrew E. Caldwell, Andrew B. Kahng, Igor L. Markov, “Can recursive bisection alone produce routable placements?” Proceedings of Design Automation Conf. 2000, pp. 477-482.
  18. Saurabh N. Adya, Igor L. Markov. “Fixed-outline floorplanning: enabling hierarchical design,” IEEE Transactions Very Large Scale Integr. 11(6), pp. 1120-1135 (2003).
  19. Richard Goering, “IC placement benchmarks needed, researchers say,” EE Times, April 10, 2003.
  20. Enno Wein and Jacques Benkoski, “Hard macros will revolutionize SoC design,” EE Times, August 2004 https://www.eetimes.com/hard-macros-will-revolutionize-soc-design/
  21. Saurabh N. Adya, Igor L. Markov, “ICCAD04 Mixed-size Placement Benchmarks,” http://vlsicad.eecs.umich.edu/BK/ICCAD04bench
  22. Ateen Khatkhate, Chen Li, Ameya R. Agnihotri, Mehmet Can Yildiz, Satoshi Ono, Cheng-Kok Koh, Patrick H. Madden, “Recursive bisection based mixed block placement,” Proceedings of Int’l Symp. Physical Design 2004, pp. 84-89.
  23. Ameya R. Agnihotri, Satoshi Ono, Patrick H. Madden, “Recursive bisection placement: Feng Shui 5.0 implementation details,” Proceedings of Int’l Symp. Physical Design 2005, pp. 230-232.
  24. Saurabh Adya, Igor L. Markov, “Combinatorial techniques for mixed-size placement,” ACM Transactions on Design Automation of Electronic Systems 10(1): 58-90 (2005).
  25. Jarrod A. Roy, Saurabh N. Adya, David A. Papa, Igor L. Markov: “Min-cut floorplacement,” IEEE Transactions on Computer-Aided Design of Integrated Circuits 25(7), pp. 1313-1326 (2006).
  26. Jason Cong, Michalis Romesis, Joseph R. Shinnerl, Kenton Sze, Min Xie, “PEKO-MS benchmarks for mixed-size placement,” 2006 https://github.com/jshinnerl/pekoMS_2006_book
  27. Gi-Joon Nam, Jason Cong, “Modern Circuit Placement, Best Practices and Results,” Springer 2007. Chapter 2 available as arXiv:2305.16413.
  28. Charles J. Alpert, Dinesh P. Mehta, Sachin S. Sapatnekar, eds.: “Handbook of Algorithms for Physical Design Automation,” Auerbach, 2008.
  29. Jarrod A. Roy, Natarajan Viswanathan, Gi-Joon Nam, Charles J. Alpert, Igor L. Markov, “CRISP: Congestion reduction by iterated spreading during placement,” Proceedings of Int’l Conf. Computer-Aided Design 2009: 357-362.
  30. Myung-Chul Kim, Natarajan Viswanathan, Charles J. Alpert, Igor L. Markov, Shyam Ramji, “MAPLE: multilevel adaptive placement for mixed-size designs,” Proceedings of Int’l Symp. Physical Design 2012, pp. 193-200, 2011.
  31. Meng-Kai Hsu, Yao-Wen Chang: “Unified Analytical Global Placement for Large-Scale Mixed-Size Circuit Designs,” IEEE Transactions on Computer-Aided Design of Integrated Circuits 31(9): 1366-1378 (2012)
  32. Myung-Chul Kim, Dongjin Lee, Igor L. Markov, “SimPL: an algorithm for placing VLSI circuits,” Comm. ACM 56(6): 105-113 (2013)
  33. Jason Cong, Guojie Luo, Kalliopi Tsota, Bingjun Xiao, “Optimizing routability in large-scale mixed-size placement,” Proceedings of Asia and South Pacific Design Automation Conf. 2013, pp. 441-446.
  34. Myung-Chul Kim, Jin Hu, Natarajan Viswanathan: “ICCAD-2014 CAD contest in incremental timing-driven placement and benchmark suite,” Proceedings of Int’l Conf. Computer-Aided Design 2014, pp. 361-366.
  35. Myung-Chul Kim, Jin Hu, Jiajia Li, Natarajan Viswanathan, “ICCAD-2015 CAD Contest in incremental timing-driven placement and benchmark suite,” Proceedings of Int’l Conf. Computer-Aided Design 2015, pp. 921-926
  36. Natarajan Viswanathan, Shih-Hsu Huang, Rung-Bin Lin, Myung-Chul Kim, “Overview of the 2015 CAD Contest at ICCAD,” Proceedings of Int’l Conf. Computer-Aided Design 2015, pp. 910-911.
  37. Igor L. Markov, Jin Hu, Myung-Chul Kim, “Progress and Challenges in VLSI Placement Research,”Proceedings of the IEEE 103(11), pp. 1985-2003 (2015).
  38. Open Science Collaboration, “Estimating the reproducibility of psychological science,” Science 349, aac4716, 2015. doi:10.1126/science.aac4716
  39. Shih-Hsu Huang, Rung-Bin Lin, Myung-Chul Kim, Shigetoshi Nakatake, “Overview of the 2016 CAD contest at ICCAD,” Proceedings of Int’l Conf. Computer-Aided Design 2016, pp. 38.
  40. Luciano Lavagno, Igor L. Markov, Grant E. Martin, Louis K. Scheffer, eds.; “Electronic Design Automation for Integrated Circuits Handbook,” 2nd ed. (two volumes) CRC. 2016.
  41. Szu-To Chen, Yao-Wen Chang, Tung-Chieh Chen, “An integrated-spreading-based macro-refining algorithm for large-scale mixed-size circuit designs,” Proceedings of Int’l Conf. Computer-Aided Design 2017, pp. 496-503
  42. Chung-Kuan Cheng, Andrew B. Kahng, Ilgweon Kang, Lutong Wang: “RePlAce: Advancing Solution Quality and Routability Validation in Global Placement”, IEEE TCAD 38(9) (2018), pp. 1717-1730. https://github.com/mgwoo/RePlAce
  43. Nina Mazyavkina, Sergey Sviridov, Sergei Ivanov, Evgeny Burnaev, “Reinforcement Learning for Combinatorial Optimization: A Survey,” Computers and Operations Research 134, October 2021, 105400; arXiv:2003.03600
  44. Andrew B. Kahng, “Advancing Placement,” Proceedings of Int’l Symp. Physical Design 2021, pp. 15-22.
  45. Yibo Lin, Zixuan Jiang, Jiaqi Gu, Wuxi Li, Shounak Dhar, Haoxing Ren, Brucek Khailany, David Z. Pan. “DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement,” IEEE Transactions on Computer-Aided Design of Integrated Circuits 40(4), pp. 748-761 (2021).
  46. Zixuan Jiang, Ebrahim Songhori, Shen Wang, Anna Goldie, Azalia Mirhoseini, Joe Jiang, Young-Joon Lee, David Z. Pan, “Delving into Macro Placement with Reinforcement Learning,” Int’l Workshop on Machine Learning for Computer-Aided Design, 2021. arXiv:2109.02587
  47. Andrew B. Kahng, Jens Lienig, Igor L. Markov, Jin Hu, “VLSI Physical Design: from Graph Partitioning to Timing Closure”, Springer, 2nd ed, 2022. https://link.springer.com/book/10.1007/978-3-030-96415-3
  48. Zoubin Gahramani, “Google stands by this work published in Nature”, Twitter April 7, 2022, https://twitter.com/ZoubinGhahrama1/status/1512203509646741507?t=9kyvWT0gCXQ5coWCG7LvsA
  49. Daisuke Wakabayashi and Cade Metz, “Another Firing Among Google’s A.I. Brain Trust, and More Discord,” The New York Times, May 2, 2022. https://www.nytimes.com/2022/05/02/technology/google-fires-ai-researchers.html
  50. Paresh Dave, “Google faces internal battle over research on AI to speed chip design,” May 3, 2022. https://www.reuters.com/technology/google-faces-internal-battle-over-research-ai-speed-chip-design-2022-05-03/
  51. Tom Simonite, “Tension Inside Google Over a Fired AI Researcher’s Conduct,” Wired, May 31, 2022. https://www.wired.com/story/google-brain-ai-researcher-fired-tension/
  52. “Lightmatter Names Top Google TPU Engineer Richard Ho as VP of Hardware Engineering,” Business Wire, August 2021 https://www.businesswire.com/news/home/20220822005051/en/Lightmatter-Names-Top-Google-TPU-Engineer-Richard-Ho-as-VP-of-Hardware-Engineering
  53. Jeffrey Dean, “ML for Computer Systems at Google,” keynote at the SysML workshop at NeurIPS, Dec 2, 2022 https://slideslive.com/38994456/machine-learning-for-computer-systems-at-google
  54. “Satrajit Chatterjee vs. Google, Case No. 22CV398683”, first Amended Complaint (FAC), filed in the Superior Court of the Santa Clara County, February 2023. https://regmedia.co.uk/2023/03/26/satrajit_vs_google.pdf
  55. Anna Goldie, Azalia Mirhoseini, “Statement on Reinforcement Learning for Chip Design,” March 24, 2023 https://drive.google.com/file/d/1jWUw6rUDcc7fuHu_iGeVDUkBxNJjhHdd/view
  56. Katyanna Quach, “Google’s claims of super-human AI chip layout back under the microscope,” March 27, 2023. https://www.theregister.com/2023/03/27/google_ai_chip_paper_nature/
  57. Pengwen Chen, Chung-Kuan Cheng, Albert Chern, Chester Holtz, Aoxi Li, Yucheng Wang, “Placement Initialization via Sequential Subspace Optimization with Sphere Constraints,” Proceedings of Int’l Symp. Physical Design 2023, pp. 133-140.
  58. Anthony Agnesina, Puranjay Rajvanshi, Tian Yang, Geraldo Pradipta, Austin Jiao, Ben Keller, Brucek Khailany, Haoxing Ren, “AutoDMP: Automated DREAMPlace-based Macro Placement,” Proceedings of Int’l Symp. Physical Design 2023: 149-157.
  59. Gregory Goth, “More Details, but Not Enough”, Comms. of the ACM, 29 Mar 2023 https://cacm.acm.org/news/271439-more-details-but-not-enough/fulltext
  60. Samuel Moore, “Ending an Ugly Chapter in Chip Design Study tries to settle a bitter disagreement over Google’s chip design AI”, IEEE Spectrum, 4 Apr 2023. https://spectrum.ieee.org/chip-design-controversy
  61. Robert Burnson, “Fired Google AI Engineer’s Whistleblower Lawsuit Moves Ahead,” Bloomberg, July 20, 2023 https://www.bloomberg.com/news/articles/2023-07-20/fired-google-ai-engineer-s-whistleblower-lawsuit-moves-ahead
  62. Belle Lin, “In Race for AI Chips, Google DeepMind Uses AI to Design Specialized Semiconductors,” The Wall Street Journal, July 20, 2023. https://www.wsj.com/articles/in-race-for-ai-chips-google-deepmind-uses-ai-to-design-specialized-semiconductors-dcd78967
  63. Katyanna Quach, “Google fails to get AI engineer lawsuit claiming wrongful termination thrown out,” July 24, 2023 https://www.theregister.com/2023/07/24/ai_in_brief/
  64. D. van Ravenzwaaij, M. Bakker, R. Heesen, F. Romero, et al, “Perspectives on scientific error,” Royal Society Open Science, July 19, 2023 https://doi.org/10.1098/rsos.230448
  65. James Brooks, “The battle for research integrity is winnable,” Research Professional News, July 28, 2023.https://www.researchprofessionalnews.com/rr-funding-know-how-start-here-2023-the-battle-for-research-integrity-is-winnable/
  66. Jeffrey Brainard, “In some scientific papers, words expressing uncertainty have decreased,” Science, 28 July 2023. doi:10.1126/science.adk0158
  67. Andrew Jack, “ ‘Open science’ advocates warn of widespread academic fraud,” Financial Times, July 31, 2023.
  68. Judge Frederick S. Chung, Ruling in the case “Satrajit Chatterjee vs. Google, Case No. 22CV398683”, Superior Court of the Santa Clara County, August 4, 2023. https://traffic.scscourt.org/search
  69. Ivan Oransky and Adam Marcus “There’s far more scientific fraud than anyone wants to admit,” August 9, 2023, https://www.theguardian.com/commentisfree/2023/aug/09/scientific-misconduct-retraction-watch
  70. Frederik Joelving, “Nature flags doubts over Google AI study, pulls commentary,” https://retractionwatch.com/2023/09/26/nature-flags-doubts-over-google-ai-study-pulls-commentary/, September 26, 2023.
  71. Rylan Schaeffer, “Pretraining on the Test Set Is All You Need,” https://arxiv.org/abs/2309.08632 September 2023.
  72. Atanu Biswas, “Reproduce or it didn’t happen: why replicable science is better science,” The Hindu October 26, 2023.
  73. Dan Garisto, “Superconductivity scandal: the inside story of deception in a rising star’s physics lab,” March 8, 2024, Nature. https://www.nature.com/articles/d41586-024-00716-2
  74. Dan Garisto, “Exclusive: official investigation reveals how superconductivity physicist faked blockbuster results.” April 6. 2024, Nature https://www.nature.com/articles/d41586-024-00976-y
Citations (5)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up