Tensor Processing with Homodyne Photonic Integrated Circuits exceeds 1,000 TOPS

Abstract: High-performance computing underpins modern AI, enabling foundation models, real-time inference and perception in autonomous systems, and data-intensive scientific simulations. Recent advances in quantization techniques utilizing low-precision computation without degrading model accuracy, creates new opportunities for analog photonic computing characterized by ultra-high clock rates and low energy consumption. Here we propose and demonstrate a coherent homodyne integrated circuit capable of general matrix multiplication(GEMM) with aggregate throughput that exceeds 1,000 TOPS (tera-operations per second), enabled by massive on-chip optical fanout and parallelism. By leveraging time multiplexing, the required modulator count is reduced from O($N^2$) to O(N), allowing dense integration of record-scale 256 $\times$ 256 homodyne units (each <0.0064 $mm^2$) within a single reticle. We employ wafer-scale fabricated 64 thin-film lithium niobate (TFLN) transmitters (each over 40-GHz bandwidth with propagation loss of 0.2 dB/cm) to encode data and chip-to-chip coupled to Si/SiN computing circuits (64 channels). Our system achieves up to 7-bit computational accuracy across 8 $\times$ 8 parallel channels at record computing clockrate 120 Gbaud/s, and 6-bit statistical accuracy across 256 $\times$ 100 channels at 20-128 Gbaud/s, representing a total throughput of 1,000-6,000 TOPS. Massive parallelism amortizes the optoelectronic (OE) conversion to allow 330-TOPS/W efficiency using foundry-available packaging technology. The system throughput is benchmarked with Qwen2.5-0.5 billion parameter models that generate accurate tokens. High throughput and energy efficiency establish a near-term pathway toward light-based accelerators for large-scale training and low-latency inference from datacenters to edges, accelerating new models toward artificial general intelligence.

• The paper presents a pioneering homodyne photonic tensor processor architecture that integrates TFLN modulators with Si photonic meshes to achieve over 1,000 TOPS.
• It employs time multiplexing to drastically reduce modulator counts, enabling high throughput and low error rates in general matrix multiplication operations.
• Experimental results validate competitive AI inference for models such as ResNet and Qwen2.5, highlighting exceptional energy efficiency (~330 TOPS/W) for scalable deployment.

Tensor Processing with Homodyne Photonic Integrated Circuits: Exceeding 1,000 TOPS

Overview

This work presents a fully integrated homodyne photonic tensor processor achieving general matrix multiplication (GEMM) workloads with aggregate throughput surpassing 1,000 TOPS. The architecture leverages hybrid thin-film lithium niobate (TFLN) and silicon photonic platforms. Key advances include record-breaking channel counts, clock rates, statistical accuracy, and energy efficiency. These results demonstrate competitive AI inference using modern models such as Qwen2.5 and ResNet variants on practical datasets, emphasizing the near-term potential of photonic accelerators in AI and scientific computation.

Motivation and Context

The computational intensity of current deep neural networks—dominated by GEMM operations in LLMs and other foundation models—has created bottlenecks in energy efficiency and data movement within conventional electronic architectures. The maturity of quantization-aware training methods enables the use of lower-precision arithmetic without appreciable degradation in model quality, which synergizes with the reduced SNR constraints and enhanced density achievable with analog photonic computation.

Current photonic solutions based on free-space and wavelength multiplexing architectures face issues in integration, scaling, crosstalk, and CMOS compatibility. Past attempts at time-multiplexed and hybrid approaches either suffered from complex demultiplexing or phase instability, severely limiting practical channel counts, accuracy, and throughput.

System Architecture

The reported tensor processor utilizes a space-time multiplexed architecture. Each GEMM $Y_{N \times K} = W_{N \times M} X_{M \times K}$ is realized using $N + K$ high-speed TFLN modulators and a mesh of $N \times K$ path-matched, homodyne detector nodes fabricated on Si/SiN.

The design achieves the following:

• Modulator Count Reduction: Through time multiplexing, the number of required modulators is reduced from $\mathcal{O}(N^2)$ to $\mathcal{O}(N)$, allowing the integration of record-scale $256 \times 256$ homodyne units ($<0.0064\,\mathrm{mm}^2$ each) on a single reticle.
• Wafer-scale Integration: The modulator array (TFLN) and the photonic crossbar (Si/SiN) are coupled with $<2\,\mathrm{dB}$ loss, leveraging well-established foundry processes.
• Interference-based MACs: At waveguide crossings in the mesh, coherent interference provides direct analog computation proportional to the MAC terms, with path length matching and calibrated homodyne readout for drift-resilience.
• FPGA-based Interface: Real-time digital control and ADC are provided through an FPGA subsystem, enabling direct benchmarking against digital baselines for inference.

Experimental Results

Numerical Accuracy and Throughput

• Clock Rates: Up to $128~\mathrm{Gbaud/s}$ operation is demonstrated, with measured 6–7 bit accuracy, and considerably lower statistical error than previous state-of-the-art analog photonic systems.
• Channel Counts: The system supports up to 256 input and 256 output channels, achieving $65{,}536$ parallel operations per time step.
• Error Metrics: For $N + K$0 arrays, error deviation is as low as 1.19% at 50 GSa/s (7 bits), 1.52% at 100 GSa/s (6 bits), and 2.26% at 120 GSa/s (5.5 bits). For the large-scale $N + K$1 arrays, deviations are higher, primarily due to residual phase noise from the test setup, but are expected to be mitigated through future on-chip modulator integration.

System Efficiency

• Throughput: With 256x256 channels at 20 GSa/s, the throughput is calculated as $N + K$2 TOPS, scaling up to a demonstrated range of $N + K$3–$N + K$4 TOPS depending on channel utilization and clock rate.
• Energy Efficiency: Optical-electronic amortization yields $N + K$5 TOPS/W energy efficiency, with total system power (including drivers and I/O) estimated at 8 W for the full 256x256 configuration at 8 bits.

AI Model Inference

• News Title Classification (Transformer-based): Optical inference achieves 78% accuracy on 100 samples versus 79% for an electronic baseline on the same model.
• ResNet on CIFAR-10 and MNIST: Measured optical classification accuracy is 87%/90.88%, compared to the digital values of 89.83%/91.12%, demonstrating the viability of photonic inference for deep learning workloads across data types.
• LLM Prefill/Decode Emulation: GEMMs of up to 256x100 are executed in parallel, with error deviations maintained below 10%, directly supporting scaling toward LLM architectures.

Claims and Implications

The key claims substantiated by rigorous benchmarking are:

• Record Channel Count Integration: Largest monolithically integrated photonic tensor processor—$N + K$6 mesh—demonstrated to date.
• Record Clock Rate: Sustained operation at up to $N + K$7, over 100$N + K$8 faster than digital electronic circuits in comparable form factors.
• High-Precision Analog MACs: Statistically significant reduction in MAC error (as low as 1.19% in small arrays, $N + K$9% in large ones).
• Photonic Energy Efficiency: Demonstration of $N \times K$0 TOPS/W using commodity packaging, with paths for higher integration density.
• Competitive Task-Level AI Inference: Classification and LLM tasks show equivalent accuracy to digital implementations, validating photonic computing for end-to-end AI pipelines.

These results substantiate the claim that integrated photonics can deliver computation at or beyond $N \times K$1 TOPS with feasible energy and area scaling, while maintaining accuracy for quantized AI workloads. The architecture provides a direct path toward reducing data-movement overhead inherent in distributed digital systems, by substantially increasing both the in situ compute density and parallelism.

Broader Impact and Future Directions

The work establishes photonics as a viable platform for large-scale GEMM-based computation—a fundamental primitive underpinning deep learning, scientific simulation, and on-device AI inference. The reduction in modulator count and the demonstration of wafer-scale integration directly address prior scalability bottlenecks, advancing photonics from laboratory devices to near-commercial prototypes.

Further developments could focus on:

• Deeper On-Chip Integration: Direct integration of phase-stable, dense modulator arrays could further reduce error and enable real-time operation at peak chipset performance.
• Expanded Nonlinearity and Memory Integration: Incorporation of analog or photonic memory and nonlinear elements to natively support more of the deep learning stack.
• Datacenter and Edge Inference: Immediate impact in low-latency inference (LLMs, perception for autonomous systems), with medium-term relevance for training workloads as precision and integration improve.
• Universal AI Accelerator Adoption: As quantization and hybrid-precision methods mature, such analog photonic architectures can be further aligned to neural architectures and software frameworks to maximize utilization.
• Scaling Beyond Current LLMs: The demonstrated scalability is sufficient for model sizes with context lengths and batch sizes at or beyond current state-of-the-art LLMs, supporting prefill and decode at unprecedented speed and efficiency.

Conclusion

This research demonstrates the feasibility and practicality of large-scale, high-throughput, homodyne photonic integrated circuits for AI tensor processing. By combining scalable TFLN modulators with Si/SiN photonic meshes, the architecture achieves over 1,000 TOPS throughput, high statistical accuracy, and strong energy efficiency. AI tasks—from LLM inference to image classification—are performed near digital parity, supporting the assertion that photonic accelerators are poised for deployment in both datacenter and edge environments, and marking a substantive advancement in physical-neural hardware.