A native signed incoherent photonic matrix core on thin-film lithium niobate supporting in situ backpropagation
Abstract: Artificial intelligence workloads increasingly demand computing architectures combining high throughput, energy efficiency, and physical scalability. Here we present a signed incoherent optical matrix multiplier on thin-film lithium niobate that natively supports signed inputs and weights. We demonstrate closed-loop in situ backpropagation on a 16 X 16 photonic core, including forward computation, nonlinear operations, error propagation, and gradient computation. Measured physical outputs directly participate in optimization, thereby incorporating actual device responses and nonidealities into training. The core achieves 8-bit multiplication and 10-bit accumulation precision across 256 channels, maintains 10-bit accuracy in tiled 256 X 256 matrix computation, and further executes Transformer linear operations in a BERT-mini workload. This work presents a fundamental, scalable building block that addresses key limitations in practical optical computing, thereby opening avenues for large-scale, high-efficiency photonic neuromorphic systems.
Paper Prompts
Sign up for free to create and run prompts on this paper.
Explain it Like I'm 14
1. What is this paper about?
This paper describes a new computer chip that uses light instead of electricity to perform some of the calculations needed for artificial intelligence.
The chip is made from a material called thin-film lithium niobate. The researchers call the chip ZEN-2. Its main job is to quickly multiply numbers arranged in rows and columns, an operation called matrix multiplication.
Matrix multiplication is extremely important in AI. For example, it is used when a program recognizes pictures, understands text, or learns from examples.
The special feature of this chip is that it can work with both positive and negative numbers directly. It can also take part in the process used to train neural networks, called backpropagation.
2. What questions did the researchers ask?
The researchers wanted to find out whether a small optical chip could:
- Represent positive and negative numbers using light.
- Perform matrix multiplication accurately.
- Use its own real measured outputs during neural-network training.
- Carry out both the forward and backward parts of learning.
- Remain accurate when reused for much larger calculations.
- Perform useful AI tasks, including part of a Transformer LLM.
A major challenge is that normal light intensity cannot be negative. A light beam can be bright or dim, but it cannot have “minus brightness.” This is a problem because neural networks often use positive and negative values for their inputs, weights, errors, and learning updates.
3. How did the researchers solve the problem?
Using two colors of light
The chip uses two slightly different wavelengths, or colors, of light. Think of them as two separate colored lanes on a road.
A number is represented by comparing the amount of light in the two lanes:
- More light in the first wavelength represents a positive value.
- More light in the second wavelength represents a negative value.
- Equal amounts represent a value near zero.
In simple form:
The chip uses the same idea for both the numbers being processed and the numbers stored as neural-network weights.
It then measures the difference between two light outputs. This lets it calculate products such as:
even when either the input or the weight is negative. This is called four-quadrant multiplication, meaning it can handle all four combinations:
- positive input × positive weight,
- positive input × negative weight,
- negative input × positive weight,
- negative input × negative weight.
Using optical circuits as adjustable weights
The chip contains 256 adjustable computing elements, arranged as a 16 × 16 matrix.
These elements are built from tiny optical circuits called Mach–Zehnder interferometers, or MZIs. In everyday terms, an MZI acts like a controllable light valve. By changing an electrical signal, the researchers can change how much light passes through.
Each optical element represents a neural-network weight. The input values are sent through the chip as rapidly changing light signals, while the weights are programmed more slowly and then reused many times.
This matches how neural networks normally work:
- Inputs and intermediate results change constantly.
- Weights are reused for many examples before being updated.
Performing backpropagation
During training, a neural network makes a prediction, compares it with the correct answer, and then adjusts its weights. This adjustment process is called backpropagation.
An everyday analogy is learning to throw a ball into a basket:
- You throw the ball.
- You see how far it missed.
- You work out which part of your movement caused the mistake.
- You adjust your next throw.
The chip performs similar steps:
- It calculates the network’s output.
- It compares the output with the correct answer.
- It sends error information backward through the network.
- It calculates how each weight should change.
The phrase in situ backpropagation means that the real chip measurements are used during training. Instead of pretending that the chip is perfect, the training process learns while taking the chip’s actual errors and imperfections into account.
Some nonlinear functions and control steps were handled electronically, while the major matrix calculations were done optically. Thus, the system was a hybrid optical-electronic system.
4. What did the researchers find?
The chip performed accurate signed calculations
The researchers tested all 256 computing channels and made about 1.28 million signed multiplication measurements.
The chip achieved about 8-bit precision for individual multiplication operations. In simple terms, this means its results were accurate to roughly the level expected from an 8-bit digital calculation.
When the chip added many multiplication results together, it achieved about 10-bit precision.
The chip also remained accurate when it was reused for larger calculations. By dividing a large matrix into smaller sections, a method called matrix tiling, the researchers performed calculations as large as 256 × 256.
The accuracy remained close to 10 bits. The chip also stayed stable during 60 repeated computation cycles.
The chip successfully trained an XOR classifier
The researchers used the chip to train a small neural network on a difficult pattern called XOR.
XOR is difficult for a simple classifier because the correct answer depends on a combination of two inputs. The researchers also added noise to make the task more realistic.
After training:
- The classifier reached 94% accuracy.
- Its error decreased during training.
- The chip successfully used both positive and negative gradients, which are the signals that tell the network how to improve.
This showed that the chip was not merely performing calculations. It was participating in a complete learning process.
The chip performed Transformer calculations
The researchers also used the chip for the linear calculations inside a small BERT-style Transformer model. The model classified news articles into four groups:
- World
- Sports
- Business
- Science and technology
Because the model was much larger than the 16 × 16 chip, the researchers split its calculations into smaller pieces and processed them one after another.
The optical-electronic system achieved a classification accuracy of about 90.5%, compared with about 93.0% for a conventional electronic version.
This small drop in accuracy suggests that the chip’s calculation errors did not seriously damage the model’s performance.
The input signals were very fast
The lithium-niobate input modulators had a measured bandwidth above 50 GHz. Bandwidth describes how quickly a device can change its signal.
The researchers estimated a peak processing speed of 51.2 trillion operations per second, or 51.2 TOPS, under their testing conditions.
5. Why are these results important?
Traditional computers move data back and forth between memory and calculation units. This movement can use a lot of energy, especially when large AI models are involved.
Optical computers may help because light can:
- Travel very quickly.
- Carry many signals at once.
- Perform many calculations in parallel.
- Potentially use less energy than electronic circuits.
Earlier optical systems often had trouble with negative numbers. They sometimes needed extra hardware, duplicate circuits, or electronic assistance. This paper’s system represents positive and negative values directly using two wavelengths and differential measurement.
That makes it more suitable for neural-network training, where negative errors and negative weight updates are common.
6. What could this lead to?
The research suggests that optical chips could eventually become important parts of faster and more energy-efficient AI computers.
The chip is still small, and the demonstrated system is partly electronic. It is not yet a complete replacement for ordinary computer processors. The researchers also need to solve practical challenges such as:
- Building much larger optical arrays.
- Reducing calibration requirements.
- Connecting optical and electronic parts efficiently.
- Improving energy measurements in full systems.
- Making the technology easier and cheaper to manufacture.
The researchers suggest several ways to expand the system. For example, many different light wavelengths could be used at the same time, allowing more calculations to happen in parallel.
Overall, the paper shows that a photonic chip can represent signed numbers, perform accurate matrix calculations, and take part in real neural-network training. This is an important step toward AI hardware that uses the speed and parallel nature of light to process large models more efficiently.
Practical Applications
Immediate Applications
- Photonic inference accelerator for Transformer and neural-network workloads — Industry, software, data centers
- Deploy the demonstrated TFLN core as a hybrid photonic–electronic accelerator for linear layers in models such as compact BERT variants, classifiers, recommendation models, and multimodal networks.
- The core can execute tiled signed matrix operations for query, key, value, feed-forward, attention, and classification layers while conventional electronics handle tokenization, softmax, normalization, nonlinear functions, and control.
- A practical product could be a PCIe, optical-interposer, or co-packaged accelerator exposing matrix-multiplication primitives to frameworks such as PyTorch, ONNX Runtime, or custom inference software.
- Basis in the paper: 16 × 16 signed matrix operations were tiled to support 256 × 256 computation, with approximately 10-bit MAC precision and BERT-mini AG News accuracy of 0.905 versus 0.930 for the electronic reference.
- Dependencies and assumptions: Useful system-level gains require accounting for laser power, DAC/ADC conversion, photodetectors, memory movement, calibration, thermal control, and electronic scheduling. The reported 51.2 TOPS figure is a peak optical processing rate, not necessarily end-to-end application throughput or energy efficiency.
- High-throughput, weight-stationary matrix processing — AI hardware and edge computing
- Use the stable, low-update-rate weight path to keep a matrix programmed while streaming many activation vectors through the 50 GHz input modulators.
- This is suitable for batched inference, recommendation scoring, convolutional or fully connected layers, and repeated linear transformations in edge servers or communications equipment.
- A likely workflow would combine electronic memory for model storage with photonic weight programming and rapid optical streaming of activations.
- Basis in the paper: The architecture explicitly separates high-speed inputs from slowly updated weights, and reports more than 50 GHz electro-optic bandwidth and stable weight behavior over two hours.
- Dependencies and assumptions: The benefit depends on sufficiently large batches or repeated reuse of each weight tile. Frequent weight reprogramming, small workloads, or substantial electronic I/O may eliminate the advantage.
- Hardware-aware calibration and inference — AI deployment and software tooling
- Build calibration-aware compilers and runtime libraries that map matrix tiles to individual photonic channels, compensate wavelength-dependent transfer functions, normalize detector outputs, and track device-specific offsets.
- Because the measured precision varies across channels, software can assign sensitive matrix blocks to higher-performing channels and use device-aware quantization or error compensation.
- Basis in the paper: All 256 channels were individually calibrated, achieving average multiplication precision above 8 bits and approximately 9.87-bit precision for tiled 256 × 256 MAC operations.
- Dependencies and assumptions: Calibration must remain valid under temperature, aging, optical-power variation, packaging changes, and repeated reconfiguration. Automated recalibration and standardized compiler interfaces remain necessary for commercial deployment.
- Experimental platform for photonic neural-network research — Academia and industrial R&D
- Use the core as a research testbed for signed optical computation, analog error propagation, hardware-aware training, photonic quantization, device nonidealities, and hybrid optical–electronic neural architectures.
- Researchers can compare algorithms using the same physical device for forward propagation, transposed matrix operations, and gradient-related products rather than simulating the optical device separately.
- Basis in the paper: The system performs closed-loop in situ backpropagation, with measured physical outputs incorporated directly into the optimization loop.
- Dependencies and assumptions: Access currently appears laboratory-scale; data and code are available only upon request, and the architecture is subject to patent preparation. Reproducibility would benefit from public hardware specifications, control software, and standardized benchmarks.
- Hardware-in-the-loop training of small and moderate neural networks — Academia, adaptive systems, robotics
- Train compact classifiers or task-specific models directly against the measured behavior of the photonic core, allowing the optimizer to absorb calibration errors, detector offsets, modulation nonlinearities, and other hardware imperfections.
- Suitable near-term experiments include sensor classification, anomaly detection, simple vision models, and adaptive signal-processing networks.
- Basis in the paper: A noisy XOR network was trained for 80 epochs using on-chip forward, error, and gradient-related operations, reaching 0.940 classification accuracy and an MSE of 0.0588.
- Dependencies and assumptions: The demonstrated training task is small. Larger models require efficient weight-update mechanisms, batch accumulation, optimizer support, electronic nonlinear operations, and verification that gradient noise does not prevent convergence.
- Low-latency signal-processing and communications workloads — Telecommunications and sensing
- Apply signed matrix operations to channel equalization, beamforming, multiple-input multiple-output detection, waveform classification, radar preprocessing, and optical-communications signal processing.
- These applications can exploit rapid streaming of signed data and repeated use of fixed or slowly changing matrices.
- Basis in the paper: The device supports high-rate electro-optic input loading, parallel optical channels, and stable repeated computation over 60 cycles.
- Dependencies and assumptions: Communications use cases require integration with photonic transceivers, strict latency and bit-error-rate validation, real-time control, and operation across wider bandwidths and environmental conditions.
- Energy-aware accelerator prototyping and policy evaluation — Energy, computing infrastructure, public-sector planning
- Use measured device characteristics to evaluate photonic alternatives to electronic AI accelerators, particularly for workloads dominated by matrix multiplication and data movement.
- Data-center operators and policymakers could use such prototypes to model potential reductions in computational energy, cooling demand, and carbon intensity.
- Basis in the paper: The authors identify non-thermal electro-optic tuning, weight reuse, and reduced thermal crosstalk as potential efficiency advantages.
- Dependencies and assumptions: The paper predicts energy benefits under a stated device-load model but does not establish complete system-level energy per inference. Policy or procurement decisions should therefore require lifecycle, packaging, laser, conversion, manufacturing, and reliability measurements.
Long-Term Applications
- Large-scale photonic training accelerators — AI infrastructure and semiconductor industry
- Scale the 16 × 16 core into arrays capable of training larger CNNs, LLMs, and generative models using native signed forward and backward operations.
- A future accelerator could combine many TFLN cores, shared optical sources, electronic nonlinear units, local gradient accumulation, and high-bandwidth optical or electrical interconnects.
- Native signed computation could reduce the need for duplicated positive/negative arrays or repeated detection paths during backpropagation.
- Dependencies: Scaling requires wafer-scale fabrication, reliable optical routing, low-overhead data conversion, fast and precise weight programming, synchronization among tiles, gradient accumulation, fault tolerance, and verification of convergence for deep networks.
- Wavelength-division-multiplexed photonic tensor processors — AI hardware and optical interconnects
- Add multiple calibrated wavelength pairs to increase the number of parallel signed channels without proportionally increasing the physical footprint.
- Frequency-comb sources could support massively parallel matrix operations for large neural-network layers or optical communication workloads.
- Basis in the paper: Multiple wavelength combinations across approximately 1546.7–1553.0 nm showed calibratable transmission responses, and the authors identify frequency combs as a route to increased parallelism.
- Dependencies: This requires stable multiwavelength sources, wavelength-selective routing, channel isolation, uniform detector response, calibration at scale, management of dispersion and crosstalk, and adequate optical power.
- Deep on-chip in situ learning and continual adaptation — Robotics, autonomous systems, industrial control
- Embed photonic training or adaptation in robots, drones, autonomous vehicles, industrial machines, and sensor networks so models can update locally in response to changing environments.
- The physical device could optimize against its own measured outputs, allowing adaptation to component drift and task-specific sensor distributions.
- Dependencies: Safety-critical deployment requires bounded updates, rollback mechanisms, stable and verified optimization, protection against corrupted data, secure model updates, and substantially improved scale and reliability. The current demonstration does not establish real-time continual learning or safety guarantees.
- Photonic accelerators for edge AI and intelligent sensors — Healthcare, smart infrastructure, consumer devices
- Integrate compact TFLN matrix cores with cameras, microphones, biomedical sensors, or industrial instruments for local classification and anomaly detection with low latency.
- Potential products include optical sensor hubs, wearable signal-processing modules, and low-power embedded inference boards.
- Dependencies: Miniaturization, optical-source integration, packaging, thermal and environmental robustness, low-power converters, manufacturability, and cost must be demonstrated. The current chip is a laboratory processor rather than a complete edge device.
- Photonic scientific and engineering computing — Academia, simulation, optimization, finance
- Use tiled signed matrix operations for iterative numerical algorithms, inverse problems, optimization, model reduction, portfolio optimization, and scientific machine-learning workloads.
- Repeated-cycle stability could support matrix-heavy iterative computations beyond neural networks.
- Dependencies: Many scientific and financial algorithms require higher precision, strict numerical reproducibility, dynamic sparsity, or nonlinear operations not provided natively by the core. Error-aware algorithms and mixed-precision electronic correction would be required.
- Standardized photonic neural-network development platforms — Academia and industry
- Develop open hardware abstractions, compilers, simulators, calibration APIs, and benchmark suites for mapping arbitrary signed linear algebra onto photonic tiles.
- Such tools could allow model designers to estimate precision loss, tile-placement effects, optical power, latency, and energy before fabrication.
- Dependencies: Standard interfaces for TFLN devices, publicly available calibration data, repeatable fabrication, software support, and cross-platform benchmarks are needed. Proprietary control electronics or patent restrictions could limit interoperability.
- Large-area and wafer-scale photonic AI manufacturing — Semiconductor manufacturing and policy
- Apply the PLACE fabrication approach to build larger interferometric arrays beyond the area of a single conventional deep-ultraviolet exposure field.
- This could enable commercially scalable photonic accelerators and create new manufacturing workflows combining direct writing, chemo-mechanical processing, electronic packaging, and photonic–electronic co-design.
- Dependencies: High-volume yield, alignment accuracy, defect inspection, process uniformity, wafer-level testing, packaging cost, and compatibility with standard semiconductor supply chains remain to be established. Policy support may be relevant for fabrication infrastructure and domestic photonic-computing capacity.
- Photonic acceleration of larger language and multimodal models — Cloud computing and software platforms
- Extend the BERT-mini demonstration to larger Transformer inference and, eventually, training workloads by distributing layers and tiles across photonic cores.
- Future products could provide optical acceleration for embedding projections, attention, feed-forward layers, and classifier heads in cloud AI services.
- Dependencies: Larger models introduce memory capacity, attention bandwidth, model-parallel scheduling, nonlinear activation overhead, softmax and normalization costs, and frequent model updates. Demonstrating accuracy on large, diverse benchmarks and measuring end-to-end cost will be essential.
- Trusted and adaptive hardware-aware AI — Healthcare, finance, defense, and regulated applications
- Use in situ training to maintain performance despite device aging, manufacturing variation, or environmental changes, potentially creating self-calibrating accelerators.
- In regulated sectors, the measured hardware response could be logged as part of an auditable deployment and used to detect drift.
- Dependencies: Hardware adaptation must be explainable, reproducible, secure, and validated against adversarial or faulty inputs. Optical nonidealities may improve hardware-aware robustness in some cases but can also introduce unpredictable behavior without monitoring and certification.
Glossary
- Accumulation precision: Accuracy with which multiple products are summed in a multiply-accumulate operation. “accumulated precision exceeding 10 bits”
- Asymmetric Mach–Zehnder interferometer (MZI): An interferometer whose two optical arms have different lengths, producing wavelength-dependent routing or transmission. “16 asymmetric MZIs for routing the calibrated wavelength pairs”
- Backpropagation: A training algorithm that propagates output errors backward through a neural network to compute parameter gradients. “closed-loop in situ backpropagation”
- Bipolar encoding: Representation of signed values using separate non-negative positive and negative components. “For a bipolar input vector ”
- Chemo-mechanical polishing (CMP): A fabrication process that combines chemical etching and mechanical abrasion to shape semiconductor or photonic structures. “followed by chemo-mechanical polishing (CMP) to transfer the patterns into the TFLN layer”
- Closed-loop training: Training in which measured outputs from the physical computational system are fed back into the optimization process. “the photonic processor formed a closed-loop training system”
- Complementary transfer characteristics: Paired optical responses in which one channel’s transmission changes oppositely to the other’s. “the two optical channels exhibited complementary transfer characteristics”
- Differential detection: Measurement method that subtracts signals from two detector outputs to recover signed information. “differential detection of the two output ports yields”
- Electro-optic bandwidth: The frequency range over which an electrical signal can effectively modulate an optical signal. “3-dB electro-optic bandwidths exceeding ”
- Electro-optic modulation: Control of optical properties using an applied electrical field or voltage. “high-speed electro-optic modulation for rapid data loading”
- Effective number of bits (ENOB): A precision metric expressing the equivalent resolution of an imperfect analog or mixed-signal system in bits. “The root-mean-square error (RMSE) corresponded to an effective number of bits (ENOB) of 8.219 bits”
- Four-quadrant multiplication: Multiplication supporting every combination of positive and negative operands. “The scheme realizes four-quadrant operation through wavelength assisted differential encoding”
- Gradient computation: Calculation of derivatives of a loss function with respect to model parameters for optimization. “gradient computation to be implemented directly on the photonic chip”
- Incoherent optical computing: Optical computation based on optical intensities rather than phase-coherent interference. “incoherent optical matrix multiplier on thin-film lithium niobate”
- In situ backpropagation: Backpropagation performed using measurements from the physical computational device itself. “experimentally demonstrating closed-loop on-chip in situ backpropagation”
- In situ training: Model training carried out directly on or through the hardware that performs the computation. “in situ training experiments”
- Intensity domain: A representation or computational regime in which optical signal intensity, rather than optical phase, carries information. “signed intensity-domain photonic computing core”
- Layer normalization: A neural-network normalization technique that normalizes activations across the features of an individual sample or token. “layer normalization were performed electronically”
- Lithium niobate on insulator: A photonic platform consisting of a lithium-niobate film integrated with an insulating layer and substrate. “Long low-loss lithium niobate on insulator waveguides”
- Macro-averaged accuracy: An accuracy measure obtained by averaging performance values across classes, giving each class equal weight. “The resulting macro-averaged diagonal accuracy was 0.905”
- Matrix tiling: Decomposition of a large matrix operation into smaller suboperations that fit available hardware. “Using matrix tiling, the calibrated core was applied to matrix-vector and matrix-matrix operations”
- Mean-squared-error (MSE) loss: A loss function equal to the average squared difference between predictions and target values. “For the mean-squared-error objective”
- Multiply-accumulate (MAC): An operation that multiplies pairs of values and sums the resulting products. “The associated matrix multiply-accumulate (MAC) operations consume energy”
- Nonlinear activation: A neural-network function that transforms a linear-layer output nonlinearly. “The nonlinear operations required for training were also incorporated into the photonic processor”
- Optical carrier: A light wave at a particular wavelength used to transmit information through modulation. “The two components are independently mapped onto optical carriers at and ”
- Optical frequency comb: A set of regularly spaced optical frequencies generated from a common source. “Combined with high-performance optical frequency comb sources”
- Optical neural network (ONN): A neural network whose computations are implemented partly or wholly using optical signals and devices. “optical neural networks (ONNs) distinguish themselves”
- Pockels effect: A linear electro-optic effect in which an electric field changes a material’s refractive index. “both electrode types utilize the Pockels effect in the lithium-niobate layer”
- PReLU activation: Parametric rectified linear unit activation that retains a scaled negative branch in addition to the positive branch. “The PReLU activation function was implemented by separating positive and negative branches”
- Photonic matrix core: An integrated optical circuit designed to perform matrix operations using photonic signals. “a fully programmable thin-film lithium niobate (TFLN) signed incoherent photonic matrix core”
- Photonic neuromorphic system: A brain-inspired computing system implemented with photonic devices. “large-scale, high-efficiency photonic neuromorphic systems”
- Residual connection: A neural-network connection that adds an earlier representation directly to a later layer’s output. “residual connections were performed electronically”
- Root-mean-square error (RMSE): The square root of the mean of squared differences between measured and reference values. “The root-mean-square error (RMSE) corresponded to an effective number of bits”
- Softmax: A function that converts a vector of scores into normalized values summing to one, often interpreted as probabilities. “softmax, GELU activation, residual connections and layer normalization were performed electronically”
- Thermal crosstalk: Unintended interaction between nearby devices caused by heat generated during operation. “avoid thermal crosstalk associated with thermo-optic tuning”
- Thermo-optic tuning: Adjustment of an optical device through temperature-dependent changes in refractive index. “the lithium-niobate Pockels response is controlled by an applied electric field rather than resistive heating”
- Thin-film lithium niobate (TFLN): A thin layer of lithium niobate used for integrated photonic and electro-optic devices. “fabricated on TFLN using photolithography-assisted chemo-mechanical etching”
- Transformer encoder: A neural-network architecture that uses attention and feed-forward layers to process sequences. “a four-layer BERT-mini encoder”
- Wavelength-division multiplexing: Transmission of multiple independently encoded optical channels using different wavelengths. “wavelength-division-multiplexed extensions”
- Wavelength-assisted differential encoding: Encoding signed values as differences between signals carried at two optical wavelengths. “four-quadrant operation through wavelength assisted differential encoding”
- Weight-stationary computing: A hardware dataflow in which weights remain programmed while inputs are repeatedly streamed through the computation unit. “Under such a weight-stationary computing scheme”
- Zero energy-consumption neural network (ZEN): The name given in the paper to an earlier photonic neural-network platform. “ZEN: zero energy-consumption neural-network”