Neural Information Causality

Abstract: Query-separated computation forces a representation to play an operational role: data are encoded before a query is known, and a later decoder can answer only through the intermediate interface. In this regime the representation functions as a message rather than merely as a feature map. We formalize this observation by embedding information causality (IC) into representation learning, obtaining a framework called neural information causality (Neural-IC). The revised formulation separates two logically distinct statements. First, every query-separated architecture induces a random-access communication experiment and obeys the embedding inequality $I_{\mathrm{N\text{-}RAC}}\le I(\vec a:H,B)$. Second, any independently certified physical capacity bound on the interface, such as a hard $m$-bit alphabet, a finite-precision register, or a power-constrained noisy channel, implies $I_{\mathrm{N\text{-}RAC}}\le C_H$. This separation avoids treating capacity as a post hoc definition and makes Neural-IC an operational diagnostic for query leakage, precision leakage, and episode-specific memory. We also provide an exact one-bit classical RAC benchmark, showing explicitly that the relevant quantum enhancement is not total information beyond the bottleneck, but fair query-conditioned access. For CHSH-type correlation layers, nested Neural-RAC protocols multiply correlation biases across depth; requiring stability of a one-bit bottleneck for arbitrary depth selects the Tsirelson threshold. We extend the analysis to asymmetric seed biases, to multi-capacity finite-depth phase diagrams, and to correlated data via a conditional information score. Controlled simulations, including straight-through binary bottlenecks and deliberately leaky ablations, verify that apparent violations are accounted for by broken query separation or undercounted capacity.

• The paper establishes a rigorous link between information causality and query-separated representation learning by introducing the Neural-IC framework.
• It defines the Neural-RAC primitive, where fixed-capacity representations diagnose capacity and query leakage in both classical and quantum architectures.
• The framework connects classical encoding with nested CHSH protocols, validating the Tsirelson bound and operational metrics for neural systems.

Neural Information Causality: Operational Embedding and Implications

Introduction

The paper "Neural Information Causality" (2605.09316) establishes a formal connection between information causality (IC)—a principle originally grounded in quantum foundations—and operational constraints in representation learning. By embedding IC into the paradigm of query-separated computation, the authors introduce Neural Information Causality (Neural-IC), which rigorously bounds the total information retrievable from a fixed-capacity representation under adversarial random-access queries. The proposed framework provides diagnostic tools for capacity estimation, query leakage assessment, and causal architecture analysis, with direct relevance to both classical and quantum-enhanced learning systems.

Figure 1: The representation of information causality and neural encoder-decoder architectures reveals their structurally equivalent causal skeleton, justifying the bottleneck-as-message formalism for query-separated computation.

Query-Separated Computation and the Neural-RAC Primitive

The core architectural assumption underlying Neural-IC is query separation: an encoder fixes a representation $H$ from data $\vec a$ before the query $b$ arrives, and the decoder must answer using $(H, b)$ only. This regime—prevalent in memory-augmented models, retrieval-augmented generation, and encoder-decoder pipelines—imposes a strict operational bottleneck, making the hidden layer an effective message rather than a mere feature map.

The paper defines the Neural Random Access Coding (Neural-RAC) task, where the goal is to retrieve any bit of a database $\vec a$ through a fixed-capacity representation when the query $b$ is revealed only at inference. The information score is the aggregate mutual information between each queried bit $a_K$ and the decoder's guess, conditioned on $b=K$.

Figure 2: The Neural-RAC primitive enforces strict query separation, ensuring the encoder's representation is the only causal conduit from data to answer—parameters and pre-established resources are fixed before the episode.

Formal Embedding and Capacity-Law Separation

Neural-IC is established as a two-step principle: (1) the embedding claim that query-separated computation induces a random-access communication experiment, with the observable Neural-RAC score bounded by $I(\vec a : H, B)$; (2) the capacity claim that an independent certificate for $H$ (finite alphabet, precision, noise, bandwidth) yields an information law $\vec a$0. Hard bottlenecks (e.g., $\vec a$1-bit registers) specialize this bound to $\vec a$2.

This separation avoids defining capacity post hoc and instead operationalizes capacity as a physical resource. The framework diagnoses oracle-memory pathologies: apparent violations (retrieving many bits from a small bottleneck) necessarily imply undercounted capacity, broken query separation, episode-specific weights, or super-quantum correlation layers.

Classical and Quantum Random Access Benchmarks

The paper rigorously compares classical and quantum protocols for one-bit random-access coding. Classical majority encoding saturates the IC bound but only for specific queries; averaged random-access success remains strictly below the Tsirelson quantum threshold.

Figure 3: Classical one-bit majority code and nested CHSH protocols yield distinct asymptotics; only the quantum (Tsirelson) protocol remains below the Neural-IC bound at large database size.

Nested CHSH Correlations and Tsirelson-Type Stability

A pivotal contribution is the analytic bridge between CHSH correlations and nested Neural-RAC queries. Isotropic CHSH cells with bias $\vec a$3 are recursively composed, yielding a success probability $\vec a$4 for depth $\vec a$5. The information score grows as $\vec a$6 for large $\vec a$7.

Figure 4: The minimal $\vec a$8 case is algebraically equivalent to a CHSH correlation cell, providing an explicit operational link between representation and quantum bias.

Crucially, $\vec a$9 leads to exponential amplification of query-conditioned access, violating Neural-IC; the Tsirelson threshold $b$0 thus emerges as a boundary for stable memory through recursive correlation layers.

Figure 5: The nested pyramid protocol composes CHSH cells to implement large-scale random access, with bias amplification manifesting as depth increases.

Quantum Realization and Operational Capacity

Quantum mechanics is shown to both satisfy the required information calculus (with von Neumann mutual information) and saturate the maximal stable isotropic bias attainable—$b$1. Quantum protocols improve fair query-symmetric access at fixed capacity while strictly obeying the Neural-IC bound, as the total information accessible from a quantum-assisted interface remains limited by the certified bottleneck.

Closed-Form and Simulation Probes

Extensive closed-form evaluations visualize the phase transition between subcritical and supercritical regimes for nested correlation-enhanced protocols. The Tsirelson boundary is numerically validated, and finite-depth experiments illustrate how violations manifest only at sufficient nesting levels. Interface accounting probes confirm that physical bottlenecks (finite bits, precision, noise) saturate or strictly bound the observable Neural-RAC score when correctly certified.

Figure 6: Closed-form information score $b$2 for the nested protocol, with all valid quantum curves remaining below the one-bit Neural-IC limit.

Figure 7: Bias scan demonstrates that supercritical seed biases are dangerous only under recursive nesting; even slight biases above Tsirelson lead to eventual violation at large depth.

Figure 8: The critical bias boundary shows convergence toward $b$3 for increasing depth, contextualizing finite-size effects in practical experiments.

Figure 9: Capacity accounting for finite-precision and noisy interfaces affirms that counted capacity constrains achievable information score—oracle behavior signals undercounted resources.

Figure 10: Finite-depth phase diagram generalizes capacity boundaries for various bottleneck sizes, confirming that asymptotic stability is lost only beyond the Tsirelson threshold.

Figure 11: Simulation-based diagnostics highlight that query leakage, precision packing, or episode-specific weights fake apparent violations—a correct accounting resolves them.

Quantum Layer Tuning and Trainability

Quantum correlation layers are parametrized (e.g., via measurement angles), allowing continuous tuning from classical to maximal quantum bias. Trainable QNN modules need not maximize bias; task-specific loss may prefer subcritical angles, with regularization or practical constraints often opting for reduced correlation strength.

Figure 12: Quantum layer sweep links measurement angle and visibility with effective isotropic bias, providing operational control over the information score.

Figure 13: Effective isotropic bias as a function of quantum layer parameters enables direct mapping from device configurations to Neural-IC performance.

Figure 14: Predicted information score tracks quantum layer tuning, illustrating controllable trade-offs between capacity and accessible bias.

Figure 15: Optimization of quantum layer angle under regularization demonstrates that loss-optimal settings are often strictly sub-Tsirelson.

Practical and Theoretical Implications

Neural-IC reframes IC as a general operational constraint for query-separated representations, linking foundational quantum limits, communication-complexity access, and learning architectures with explicit capacity budgeting. The implications are:

• Diagnostics for Learning Systems: Neural-IC can stress-test claims of memory and retrieval capacity in neural and quantum neural networks by enforcing explicit interface capacity certification and verifying causal query separation.
• Robustness in Quantum-Enhanced Pipelines: Quantum correlation layers provide physically admissible enhancements, improving fair query-conditioned access without violating bottleneck accounting.
• Capacity Law Generalization: Arbitrary real-valued representations require physical constraint modeling; capacity certificates (alphabet size, precision, noise, power) are indispensable for operational accounting.
• Oracle-Memory Preclusion: Genuine advantage through enhanced correlation must remain strictly bounded unless physical resources (or post-quantum correlations) violate IC, in which case the bottleneck collapses.

Conclusion

Neural-IC offers a rigorous, operationally testable capacity-accounting principle for query-separated computation and memory. The framework binds information causality to representation learning, necessitating explicit physical models for bottleneck capacity, and providing a diagnostic lens for both classical and quantum architectures. Quantum resources are particularly significant—they provide real advantages while respecting causal stability. Future directions include empirical evaluation on quantum hardware and refinement of capacity certificates for non-uniform and correlated databases, cementing Neural-IC's role in both foundational and applied AI research.