Interference-limited capacity at large fact loads

Characterize the capacity law and interference behavior of bounded in-cell learning when the number of injected facts reaches the regime of approximately 10^5–10^6 facts, where facts may begin to interfere with one another rather than primarily with the base model.

Background

The experiments measure absorption from 103 to 104 presented facts, with cell-space utilization below 1%. In that range, absorption follows an approximate exponent of 0.8, but the authors regard the regime as optimization-limited and far below the information-theoretic shipping-bit ceiling.

The paper has not reached the larger regime in which interference among injected facts may dominate. Consequently, extrapolations from the two measured points are explicitly characterized as conjectural rather than as an established saturation or capacity law.

References

The regime where facts begin to interfere with one another rather than with the base model---plausibly $105$--$106$ facts---is where the interesting capacity law lives, and we have not reached it; two-point extrapolations from our data are order-of-magnitude conjecture and nothing more.

— Nothing Changed but the Model: CellFill -- Bounded In-Cell Learning for Bit-Identical, Revocable Updates to Quantized LLMs  (2608.20873 - Liu et al., 21 Aug 2026) in Section Discussion and outlook, subsection “Open problems, ranked” (3)