Next-generation Co-Packaged Optics for Future Disaggregated AI Systems

Abstract: Co-packaged optics is poised to solve the interconnect bandwidth bottleneck for GPUs and AI accelerators in near future. This technology can immediately boost today's AI/ML compute power to train larger neural networks that can perform more complex tasks. More importantly, co-packaged optics unlocks new system-level opportunities to rethink our conventional supercomputing and datacenter architectures. Disaggregation of memory and compute units is one of such new paradigms that can greatly speed up AI/ML workloads by providing low-latency and high-throughput performance, while maintaining flexibility to support conventional cloud computing applications as well. This paper gives a brief overview of state-of-the-art of co-packaged optical I/O and requirements of its next generations. We also discuss ideas to exploit co-packaged optics in disaggregated AI systems and possible future directions.