Entity-Preserving Reversal

• Entity-preserving reversal is a data transformation approach that reverses non-entity content while keeping entities intact to maintain semantic integrity in text and graphs.
• It mitigates the reversal curse in language models by enabling bidirectional training through reverse operations and whole-entity masking strategies.
• The method extends to reversible computation and dynamic graph systems, offering robust data augmentation and preserving atomic substructures in complex structures.

Entity-preserving reversal is a family of data transformation and modeling schemes in which the order of non-entity content in a sequence (e.g., a sentence) is inverted, while explicitly preserving the internal sequential integrity of marked entities or atomic substructures. This approach has become central in overcoming the "reversal curse" in LLMs and in the mathematically rigorous study of reversible transformations on more general combinatorial objects such as graphs. Entity-preserving reversal occupies a pivotal role in bidirectional language modeling, reversible computation, and information-preserving dynamics.

1. Formal Definitions and Mechanisms

Let $x = (w_1, w_2, \ldots, w_m)$ be a sequence of tokens (words). A set of entity spans $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$ is detected, each representing a contiguous subsequence designated as an entity (typically via NER tools such as flair/ner-english-large). The sequence is then partitioned into contiguous segments $C_1, C_2, \ldots, C_K$, where each $C_j$ is either an entity chunk (an interval in $E$) or a non-entity chunk.

The entity-preserving reversal operation, $x_{\text{rev}}$, concatenates segments in reverse order, where inside non-entity chunks the order of words is reversed, while entity chunks are left intact: $$x_{\text{rev}} = \text{CONCAT}_{j=K \rightarrow 1} \left[ \begin{cases} \text{REVERSE\_NON\_ENTITY}(C_j) & \text{if } C_j \notin E \ C_j & \text{if } C_j \in E \end{cases} \right]$$ This ensures that structural and semantic information encapsulated in entities persists under reversal, preventing label fragmentation and enabling reversible mappings that do not disrupt key semantic units (Golovneva et al., 2024).

2. The Reversal Curse in Language Modeling

The "reversal curse" refers to the persistent asymmetry observed in LLMs trained with standard autoregressive or even bidirectional (diffusion) objectives: models are proficient at answering queries in the canonical (A $\rightarrow$ B) direction but fail in the logically equivalent inverse (B $\rightarrow$ A) direction. Given accuracy metrics

$$\text{Acc}_+ = P_{\text{model}}(\text{output} = B \mid "A \text{’s } r"?),\quad \text{Acc}_- = P_{\text{model}}(\text{output} = A \mid "B \text{’s } r^{-1}"?)$$

the reversal curse is quantified by a large gap $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$0 (He et al., 12 Jan 2026, Golovneva et al., 2024).

Autoregressive LLMs naturally encode asymmetry due to their left-to-right attention and loss, while even bidirectional architectures (discrete denoising/diffusion models) are susceptible via entity fragmentation, data asymmetry (prevalence of A $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$1 B facts), and the lack of explicit inverse-relation training.

3. Entity-Preserving Reverse Training in LLMs

Entity-preserving reverse training augments the corpus by applying entity-preserving reversal to each training sequence and training the model on both original and reversed data (Golovneva et al., 2024). The procedure involves:

• Entity detection (typically via NER tools) for robust entity chunking.
• Reversal of non-entity chunks and preservation of entity chunks.
• Data augmentation: if $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$2 is the dataset, training occurs over $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$3, effectively doubling exposure.

The training loss becomes: $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$4 where each term is a standard autoregressive negative-log-likelihood on the corresponding sequence direction.

Empirically, entity-preserving reverse-trained LLMs show near-perfect exact-match accuracy on symbolic reversal tasks (100% for multi-word entities), strong gains on biographical QA, and improved performance on reversal and standard language understanding benchmarks—a direct mitigation of the reversal curse (Golovneva et al., 2024).

4. Entity-Preserving Reversal in Diffusion Models

In diffusion-based LLMs (DLLMs), fragmentation arises when multi-token entities are masked and reconstructed non-atomically. This can be remediated by whole-entity masking (WEM): whenever any part of an entity is masked, the entire span is masked as a unit, and the model is trained to predict the whole entity in a single denoising step (He et al., 12 Jan 2026). Formally, for entity span $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$5, the mask $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$6 is set as: $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$7 This holistic treatment of entity reconstruction ensures that diffusion-based denoising no longer fragments entities, directly improving reverse query accuracy and reducing hallucinations.

Combined with distribution-symmetric data construction (pairing each forward triple $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$8 with its inverse $E = \{[a_1,b_1], [a_2,b_2], \ldots \}$9) and relation-enhanced data (explicitly supervising on both $C_1, C_2, \ldots, C_K$0 and $C_1, C_2, \ldots, C_K$1), the resulting models achieve high bidirectional consistency and minimal forward-reverse performance gap:

Task	Baseline Forward	DiffER Forward	Baseline Reverse	DiffER Reverse
Parent–Child (LLaDA-8B)	92.00%	97.75%	46.73%	49.83%
Company–CEO (LLaDA-8B)	83.74%	90.87%	0.29%	2.89%

Overall, the DiffER methodology systematically reduces reversal curse symptoms in DLLMs by maintaining entity atomicity and training for bidirectional logical correspondence (He et al., 12 Jan 2026).

5. Entity-Preserving Reversal in Reversible Causal Graph Dynamics

Reversal with preservation of atomic substructures extends to dynamic graph systems. In reversible causal graph dynamics (CGD), classical vertex-preserving bijective maps preclude local node creation/destruction. However, by shifting to settings where certain "memory reservoirs," "signal-passing conventions," or algebraic name structures are maintained, one can support local reversible transformations that split or merge subgraphs or entities while preserving the ability to reconstruct the original state (Arrighi et al., 2018).

Three equivalent formal frameworks are established:

1. Memory-augmented CGD: Each visible node is accompanied by an infinite tree reservoir ("invisible matter"). Node creation/removal is modeled as movement between visible and invisible status, preserving overall bijectivity.
2. Anonymous CGD: On unlabeled graphs, node birth/destruction is permissible as long as bijectivity is preserved modulo isomorphism.
3. Named-graph CGD (name algebra): Entities (nodes) are named using algebraic structures supporting merge/split operations, ensuring that inverse transformations are well-defined at the algebraic level.

These mechanisms generalize entity-preserving reversal principles beyond language, supporting information-preserving dynamic reconfiguration of combinatorial structures (Arrighi et al., 2018).

6. Evaluation, Limitations, and Future Directions

Empirical Results

• Entity-preserving reverse LLMs (1.4B parameter scale) attain 100% on symbolic multi-word entity reversal and 99% on biography QA (full-name recall), compared to 0% for standard LMs (Golovneva et al., 2024).
• DiffER improves both forward and reverse accuracy on canonical relation databases (PORE parent–child, company–CEO) (He et al., 12 Jan 2026).
• In graph dynamics, entity-preserving and name-algebraic models enable reversible, local, size-varying transformations (node split/merge), broadening the scope of reversible computation.

Limitations

• Current entity detection relies on NER, which may omit non-named-entity multi-token concepts (dates, technical terms) (Golovneva et al., 2024).
• The approach is sensitive if the target to predict is itself not fully preserved under the entity partition.
• Scaling to open-domain or higher-arity relations, and automating D_sym and D_rel construction at web-scale, remain open challenges (He et al., 12 Jan 2026).

Extensions

• Segmentation into concept chunks beyond canonical entities using unsupervised or cross-domain approaches.
• Mixing random-segment reversal to reduce reliance on external NER tools.
• Adapting the paradigm to multilingual corpora and structurally rich objects (code, tables).
• Extending entity-preserving principles to full joint pre-training and larger model scales (Golovneva et al., 2024, He et al., 12 Jan 2026).

7. Relevance and Applications

Entity-preserving reversal methods reorient data augmentation and reversible transformation to maintain the integrity of core semantic, syntactic, or combinatorial structures. In language, this directly impacts knowledge probing, bidirectional reasoning, and the reversibility of factual associations. In computation and physics, these methods underpin formal models for reversible computing, extend cellular automata to dynamic topologies, and provide a basis for discrete quantum gravity approaches through causal graph dynamics (Arrighi et al., 2018). A plausible implication is that future foundation models and dynamic systems will increasingly rely on entity-preserving reversibility to guarantee logical soundness, compositionality, and efficient reuse of information across tasks and transformations.