Who Needs DRAM? We Have Fiber

Abstract: The rising pressure on DRAM availability and contract pricing reflects generative AI's massive high-performance memory requirements. This pressure is heavily compounded by hyperscale data center expansion, which now consumes a significant portion of global DRAM output. In this work, we propose a new architecture: Fiber Memory, which reimagines the role of optical fiber in a hyperscale data center, deploying it as an active, recirculating delay-line memory for immutable data, such as LLM weights. We present a data-parallel optical broadcast delay-line memory architecture that accounts for fiber's physical realities. By incorporating space-division multiplexed multi-core fibers (MCFs), passive optical tap-and-amplify interfaces, co-packaged optics (CPO), and regional all-optical regeneration, our case study evaluation demonstrates that Fiber Memory can eliminate redundant weight storage across 10,000 AI accelerators and reduce weight-delivery energy by over 70% compared to traditional HBM3e configurations.

• The paper introduces Fiber Memory, a novel photonic delay-line architecture that repurposes optical fiber loops to stream LLM weights, reducing DRAM redundancy and energy consumption by over 70%.
• It leverages multi-core fiber and DWDM technologies to enable parallel in-flight weight broadcasting to large accelerator clusters with delivery bandwidth comparable to HBM3-class interfaces.
• The paper quantifies system performance, demonstrating in a Llama-3-70B case study a per-accelerator weight delivery of 3.2 TB/s and a scalable, lower-cost total cost of ownership.

Fiber Memory: Photonic Delay-Line Architecture for Large-Scale LLM Inference

Introduction

This paper introduces a system-level architectural proposal—Fiber Memory—that fundamentally reconsiders high-volume immutable memory provisioning for LLM inference in hyperscale datacenters (2607.08407). The concept leverages physical fiber delay lines using data center optical infrastructure, dense spatial and wavelength multiplexing, co-packaged photonic-electronic integration, and all-optical broadcast distribution. The primary technical motivation is to eliminate the massive redundancy and significant energy overhead of DRAM and HBM replication for static weight parameters by transforming the existing fiber fabric into a recirculating memory substrate.

Motivation and Problem Definition

Current cluster-scale LLM inference replicates model weights in high-cost HBM/DRAM at every accelerator, resulting in inefficient resource and energy utilization—especially given the volume of globally identical, immutable parameters distributed across nodes. DRAM supply constrains cluster scaling, and per-node local memory is thermally and financially expensive. The paper quantifies that an aggregate 100,000 km of datacenter fiber can, due to the bandwidth-delay product, physically buffer multiple terabytes of data simultaneously "in-flight," enough to hold trillion-parameter LLMs in quantized form.

The insight is to treat the fiber ring as a high-speed recirculating delay-line memory, as was done with mercury tubes historically—but at present-day, unprecedented speeds and capacity.

Fiber Memory: Architectural Overview

The Fiber Memory proposal leverages space-division multiplexed multi-core fiber (MCF) and wavelength-division multiplexing (WDM) for parallel broadcast, organizing the topology as a central weights server periodically injecting weights into a ring comprising bundled MCF loops. Regional splitters broadcast the signal to pods; within each pod, local distribution buses allow chassis-level concurrent access. This design minimizes aggregate optical losses and reduces per-node transceiver overhead.

Figure 1: Ring and pod topology for distributing LLM weights using wavelength-multiplexed multi-core fiber, supporting efficient optical multicast to pods.

Nodes extract a fraction of the optical weight stream using highly asymmetric tap-and-amplify interfaces: each receiver taps 1% of the signal locally, forwarding 99% downstream. Optical amplifiers (Praseodymium-doped fiber amplifiers, PDFAs) compensate loss without periodic O-E-O conversions—thus realizing scalable, low-latency, all-optical broadcast infrastructure.

Figure 2: Asymmetric tap-and-amplify receiver schematic minimizing cumulative fiber loss and allowing efficient pass-through of the primary optical signal.

Photonic integrated circuits (PICs), co-packaged with accelerator compute silicon, contain arrays of passive micro-ring resonators (MRRs), spatially demultiplexing MCF+DWDM streams and feeding them into the systolic array registers in each accelerator.

Figure 3: CPO integration with compute silicon for direct delivery of demultiplexed LLM weights to systolic array registers, minimizing buffer/interconnect overhead.

Data, Packetization, and Compute Integration

LLM inference is overwhelmingly dominated by read-only, bulk parameter accesses: model weight bandwidth dwarfs activation and KV-cache bandwidth requirements. Fiber Memory splits the memory hierarchy accordingly: activations and temporary state are stored locally (SRAM/DRAM), while all redundant global weights are streamed from the fiber in deterministic, layer-structured packets.

Each "streamed weight packet" (SWP) is pre-unrolled to align with the accelerator’s spatial compute fabrics, and includes:

• A lock-acquisition preamble for CDR
• A FEC-protected header with layer and quantization metadata
• Ordered payload matching array layout
• CRC for endpoint boundary check

  Figure 4: Streamed weight packet structure for robust high-throughput, error-resilient parameter streaming.

This approach eliminates both address translation and deep buffering: compute proceeds in lockstep with the fiber stream, with optional slack/interleaving for scheduling flexibility. Multiple redundant SWPs or insertion of timing slack between packets can mitigate desynchronization or compute micro-stalls but at the cost of reducing net fiber capacity.

Quantitative Realization: Llama-3-70B Case Study

A concrete evaluation is performed for a Llama-3-70B (70B parameter, INT8) deployment over a cluster of 10,000 accelerators. The system design targets 128 GB of recirculating fiber memory (across timing slack and packet replication), realized via 14×19-core MCF cables (totaling 266 cores, 256 active, each with 8×100 Gbps WDM channels). Aggregate fiber bandwidth is 25.6 TB/s, and total recirculation latency over the 1,000 km fiber loop is 5 ms.

Per-accelerator, this yields 3.2 TB/s weight delivery bandwidth—comparable with state-of-the-art HBM3-class memory interfaces (e.g., 3.35 TB/s in NVIDIA H100), but without local replication and at substantially reduced energy and capital expense.

Energy Efficiency and Power Analysis

The weight fetch energy for 10,000 HBM3e-equipped nodes totals 1.024 MW, assuming 4 pJ/bit. Fiber Memory's aggregate central laser, inline PDFA (loop), local PDFA, 2R regenerators, and direct-detect receiver electronics consume a total of 284.8 kW—representing a 72.1% reduction in weight-delivery power. This is exclusive of DRAM static/leakage and cooling savings, as fiber storage is passive and lasers/amplifiers can be field-serviced separately.

Such a reduction is possible because all LLM weights are broadcast in-flight rather than repeatedly read from static, distributed HBM/DRAM arrays.

Physical Realizability and Implementation Constraints

Key physical considerations addressed include:

• Attenuation and Cumulative Loss: O-band fiber loss ($\sim$0.32 dB/km), tap loss (0.043 dB/chassis), and the necessity for periodic amplification limit ring length and node scaling per region.
• ASE Noise and Regeneration: Inline PDFAs introduce OSNR degradation per segment; all-optical 2R regenerators at physically optimal pod boundaries prevent BER accumulation incompatible with standard FEC operating points.
• Chromatic Dispersion: The O-band's zero-dispersion wavelength (1312 nm, $\sim$0.1 ps/(nm·km) dispersion) enables transmission of tightly packed DWDM without ISI at 50 Gbaud over 1,000 km.
• Laser and Thermal Management: Centralized external laser banks (not co-packaged per-chip) maximize lifetime and serviceability; on-chip passive MRR tuning via micro-thermoelectric units maintains wavelength stability at pW power budget.

Error correction is handled with low-latency FEC (Reed-Solomon/LDPC); further, LLMs are resilient to weight quantization (INT8/FP16) bit flips, as long as the scaling factors and normalization statistics are protected.

Implications and Future Work

The architectural proposal has direct implications for datacenter TCO: reducing DRAM supply chain constraints, sharply lowering latent memory replication, and substantially improving energy efficiency for LLM-scale inference. It enables dynamic scale-up: storage increases linearly with additional fiber loops, not silicon memory modules. Practically, synchronization strategies, fiber-integrated system call handling, scheduler-aware packet insertion, and support for dynamic model and context size must be further explored.

Theoretically, the design points toward hybrid compute-photonic scheduling models where push-based bulk immutable parameter delivery shifts the performance bottleneck from local memory to global, pipelined optical multicast. The approach capitalizes on inherent LLM robustness to minor physical-layer errors and paves the way for minimal-overhead, dynamic accelerator provisioning.

Conclusion

The Fiber Memory architecture combines MCF and DWDM photonic broadcast, all-optical amplification/regeneration, and direct photonic-electronic packaging to realize a high-throughput, low-redundancy, scalable, and energy-efficient immutable memory fabric for LLM inference (2607.08407). The strong energy efficiency results—a 70+% reduction vs. DRAM/HBM3e for parameter delivery—highlight the technical viability and transformative potential in the evolving datacenter landscape. Future engineering should address dynamic and fine-grained scheduling, system integration, expanded error models, and cross-disciplinary photonic-electronic compiler pipelines to further realize the advantages of in-fiber memory constructs.