Systematic Linear Propagation (SLP)

• SLP is a structural principle that formalizes how first-order parameter updates in neural networks should propagate coherently across logically related queries.
• The framework employs relation algebra operators—negation, converse, and composition—to derive necessary tensor factorizations and expose propagation constraints.
• SLP reveals inherent limitations in linear composition for knowledge editing and multi-hop reasoning, explaining empirical failures such as the reversal curse and feature collapse.

Systematic Linear Propagation (SLP) is a structural principle formalizing how first-order (linearized) parameter updates in neural networks should propagate coherently to all logically related queries. The methodology rigorously analyzes the geometric and algebraic constraints imposed on network features by relation algebra operators—negation, converse, and composition—revealing both necessary tensor factorizations and fatal obstructions for certain logical propagations. The framework elucidates why empirically observed failures—such as local knowledge editing not propagating globally, the reversal curse, and multi-hop reasoning breakdown—are intrinsic consequences of these linear limitations (Chang et al., 29 Jan 2026).

1. Formalization of the Linear Propagation Assumption

At the core of SLP is the Linear Propagation Assumption (LPA): any small, first-order parameter update that improves the score of a model on a given query should immediately propagate to all logical relatives of that query. Specifically, in a differentiable model parameterized by $\theta \in \Theta$, each query $q$ is assigned a scalar score $s_\theta(q)$. In the linearized regime around $\theta_0$, the query’s feature embedding is defined as:

$$\phi_q = \nabla_\theta s_\theta(q)\big|_{\theta_0} \in \Theta$$

A small update $\Delta\theta$ then induces a score change:

$$s_{\theta_0+\Delta\theta}(q) - s_{\theta_0}(q) \approx \langle \phi_q, \Delta\theta \rangle$$

The LPA requires that every $\Delta\theta$ within the feature spans should also induce correspondingly prescribed changes in the scores of logically related queries.

2. Relation Algebra Operations and Propagation Rules

The logical structure is built on relation algebra, with entities $E$ and the universe of ordered pairs $U = E \times E$. A relation $r \subseteq U$ is a binary predicate, and queries are represented $q = (h, r, t)$, where $(h, t) \in U$ and $r$ is closed under negation and converse. The three core operations and their associated propagation constraints are:

• Negation: $\neg r := U \setminus r$, requiring

$\phi_{\neg q} = -\phi_q$

• Converse: $r^{\smile} = \{(t, h) \mid (h, t) \in r\}$, with propagated features:

$\phi_{\mathrm{rev}(q)} = \phi_q$

• Composition: $$r;s = \{(h, t) \mid \exists b \in E : (h, b) \in r \wedge (b, t) \in s\}$$. For unique witnesses, this reduces to a feature conjunction operator $F$:

$F(\phi_p, \phi_q) = \phi_{p \wedge q}$

SLP imposes kernel-stability, idempotence ($F(u,u) = u$), symmetry ($F(u,v) = F(v,u)$), and linearity on $F$, yielding a bilinear form:

$\tilde F(u, v) = u^\top B v$

with some bilinear form $B$.

3. Tensor Factorization for Negation and Converse

The propagation constraints for negation and converse lead to a tensor product factorization of features. Theorem 3.1 establishes that, under the SLP, the query feature map admits the decomposition:

$W \cong \bigoplus_i (C_i \otimes R_i)$

and for each query $(h, r, t)$,

$$\phi(h, r, t) = \bigoplus_i \sum_{k=1}^{m_i} \left( u_{i,k}(h, t) \otimes v_{i,k}(r) \right)$$

where $u_{i,k}: E \times E \rightarrow C_i$ encodes entity-pair context and $v_{i,k}: \mathcal{R} \rightarrow R_i$ encodes relation slots. Negation acts only on relation factors:

$v_{i,k}(\neg r) = -v_{i,k}(r)$

Further, the converse operation partitions each block into symmetric and antisymmetric modules. Specifically,

$$\phi_i(h, r, t) = \phi_i^+(h, r, t) + \phi_i^-(h, r, t)$$

with $\phi_i^+$ using symmetric context–relation pairs and $\phi_i^-$ using antisymmetric pairs, enforcing equivariance under entity renaming and logical operators via representation theory (Maschke's theorem for finite groups).

4. Impossibility Results for Linear Composition

A fundamental obstruction arises for composition and conjunction under purely linear propagation. For a symmetric bilinear conjunction $F$ with negation equivariance, logical idempotence ($\phi_{p \wedge p} = \phi_p$), and bilinearity imply:

$\phi_{\neg p \wedge \neg p} = \tilde F(-u, -u) = u$

But under negation equivariance:

$\phi_{\neg p \wedge \neg p} = \phi_{\neg p} = -u$

forcing $u = -u$, i.e., feature collapse to zero for all queries. This is formalized in Theorem 4.7: any attempt to implement logical conjunction via a symmetric bilinear form in the SLP regime necessarily leads to trivial (zero) features.

5. Consequences for Knowledge Editing, Reversal, and Multi-Hop Reasoning

Empirically observed failures in neural networks can thus be interpreted as direct consequences of the structure uncovered by SLP.

• Knowledge Editing: Editors employing small linear least-squares updates (such as ROME or MEMIT) cannot simultaneously propagate a new fact $p$, its negation $\neg p$, and maintain logical consistency for nontrivial conjunctions. This misalignment results in systematic propagation failure for negations or implied logical triples.
• Reversal Curse: Absent untethered converse equivariance, there is no guarantee that features for $(h, r, t)$ and $(t, r^{\smile}, h)$ are matched. Linear schemes cannot enforce direction-agnostic propagation, matching the empirical struggle of LLMs to generalize converse relations.
• Multi-Hop Reasoning: In graph structures, composition corresponds to conjunction (when unique witnesses exist). The collapse theorem implies that any attempt to propagate multi-hop relations using a single bilinear feature product must either break idempotence or collapse all features, concordant with practical failures of chain-of-thought and multi-hop propagation in one-shot updates.

6. Implications and Theoretical Significance

SLP demonstrates that demanding logically coherent linear propagation forces a strict tensor-product architecture for unary logical operations, while making binary operations (composition, conjunction) impossible in a purely linear regime. This conceptual unification explains not only specific empirical results but also provides a structural origin for observed limitations in knowledge editing and reasoning in current differentiable networks. The guiding principle, "dynamics reveals structure," ties the logical algebra, representation theory, and parameter geometry under a common framework, delineating the boundaries of what first-order adaptations in neural networks can accomplish (Chang et al., 29 Jan 2026).

7. Critical Perspective and Future Directions

A plausible implication is that overcoming the structural failures highlighted by SLP requires moving beyond linearized local updates—potentially developing architectures or algorithms that can respect the desired algebraic constraints at higher orders, or employing nonlinear compositional mechanisms. The revealed incompatibility between bilinear conjunction and negation under linear propagation may spur investigation into alternate representations, feature space regularizations, or explicit connective architectures. Further studies may refine tensor factorizations for partial or approximate logical coherence or illuminate algebraic invariants under nonlinear editing regimes.