Representation Edit Distance (RED)

• Representation Edit Distance (RED) is a measure that quantifies the normalized editing cost required to update an agent's compressed internal representation upon encountering novel information.
• It leverages information-theoretic principles such as minimum description length and algorithmic complexity to approximate changes in representations across knowledge graphs, regression models, and neural networks.
• The method offers actionable insights into adaptation difficulty by linking compressed code-length changes to the effort needed in model restructuring, while its accuracy depends on the representation and compression techniques used.

Representation Edit Distance (RED) is a proposed measure of novelty-adaptation difficulty that quantifies how much an agent’s internal representation must be edited to restore adequate performance after encountering novelty. In the formulation introduced by Johnston, novelty is not an absolute property of the environment, but a mismatch between what an agent’s mental model predicts and what it observes; adaptation is therefore the process of modifying the internal representation used by the agent’s skill programs. RED is intended as a practical, information-theoretic proxy for that modification burden, estimated from compressed or near-optimal pre-novelty and post-novelty representations rather than from raw environmental descriptions (Alspector, 2021).

1. Definition and conceptual setting

RED is defined in the context of an agent whose mental model has two components: a representation and a prediction program. The representation may be a knowledge graph, a neural network, a regression model, or another learned structure; the prediction program reads that representation and uses it to predict or act. Within this setting, the central claim is that “the amount of editing of an effective representation … used in a set of skill programs in an agent’s mental model is a measure of difficulty for adaptation to novelty” (Alspector, 2021).

This framing makes RED explicitly agent-relative. Novelty is “in the eye of the beholder”: what counts as novel depends on the mismatch between a particular agent’s priors, experience, and current representational resources and the demands of the environment. A small environmental change can therefore induce either low or high RED depending on whether the agent’s pre-novelty representation is already close to an adequate post-novelty representation.

Conceptually, RED sits between ordinary edit distance and information distance. It resembles edit distance because it asks how many edits are needed to transform one object into another. It differs from ordinary string edit distance because the objects being compared are not raw strings over a fixed alphabet, but “effective,” “near-optimal” representations of skill programs or mental models. It also resembles algorithmic information distance because the cost of representation change is tied to compressed description length, conditional complexity, and minimum-description-length considerations rather than to fixed insertion, deletion, and substitution penalties alone (Alspector, 2021).

A common misconception is that RED is simply Levenshtein distance applied to serialized model states. The original proposal does not define RED that way. It does not introduce a standardized edit algebra with exact primitive costs; instead, it proposes a computable approximation to representational change based on compression and near-optimal representations. This suggests that RED is best understood as a framework for measuring adaptation difficulty, not as a fully axiomatized metric in the classical string-algorithmic sense.

2. Information-theoretic and MDL foundations

The paper places RED in an MDL/AIT framework in which a hypothesis $H$ is decomposed into a representation $RH$ and a prediction program $PH$, with description length

$L(H) = L(RH) + L(PH).$

Observed data $D$ is encoded with the hypothesis via $L(D \mid H)$, and the “crude” two-part MDL principle selects models minimizing

$L(H) + L(D \mid H).$

The motivating argument is that for a fixed architecture, $L(PH)$ is often approximately constant, so novelty adaptation is driven primarily by the complexity of the representation and how much that representation must change (Alspector, 2021).

The proposal is also connected to algorithmic information distance. The background notions reviewed in the paper include

$$\begin{split} AID(x,y) = max{[K(x|y),K(y|x)]} \ = K(xy) - min[K(x),K(y)], \end{split}$$

its normalized version

$NID(x,y) = \frac{AID(x,y)}{max[K(x),K(y)]},$

and the compression-based approximation

$RH$0

For semantic concepts, the paper also cites normalized web distance,

$RH$1

These quantities are not RED itself, but they motivate the idea that representational change can be approximated by changes in compressed information content rather than by purely syntactic edit counts (Alspector, 2021).

More directly, RED is positioned as an approximation to a novelty-adaptation quantity called generalization difficulty: $RH$2 As printed, this formula contains a formatting issue, but the intended meaning is explicit in the surrounding discussion: $RH$3 is a normalized conditional Kolmogorov complexity from a training-time solution to a post-novelty solution. RED is introduced as an approximation to this quantity, replacing uncomputable Kolmogorov complexity with computable compressed code lengths and representation-level edits.

The broader notation is stable across the framework. The symbols include $RH$4 for agent, $RH$5 for task, $RH$6 for curriculum or training experience, $RH$7 for performance threshold, $RH$8 for the shortest evaluation-time solution achieving $RH$9, $PH$0 for the shortest optimal training-time solution under curriculum $PH$1, $PH$2 for the initial agent, $PH$3 for Kolmogorov complexity, and $PH$4 for computable compressed code length (Alspector, 2021).

3. Formal characterization and operational estimation

The paper’s clearest direct definition of RED appears in the knowledge-graph discussion: $PH$5 This printed equation is corrupted. The accompanying prose clarifies the intended meaning: RED “represents the edits ($PH$6 in the equation, which may be $PH$7) one would have to make to the shortest training-time solution … to obtain the evaluation-time solution while using the initial agent’s capabilities without penalty on compressed, near-optimal versions of the knowledge graphs” (Alspector, 2021).

Accordingly, the intended characterization is that $PH$8 is the compressed code length of a near-optimal training-aligned representation, $PH$9 is the compressed code length of the post-novelty solution representation, RED is approximately the edit cost from $L(H) = L(RH) + L(PH).$0 to $L(H) = L(RH) + L(PH).$1, and in the simplest approximation that edit cost may be taken as $L(H) = L(RH) + L(PH).$2, normalized by $L(H) = L(RH) + L(PH).$3. The framework also introduces related quantities for priors,

$L(H) = L(RH) + L(PH).$4

and an effective adaptation difficulty expression printed as

$L(H) = L(RH) + L(PH).$5

This expression is also malformed. From context, the intended interpretation is roughly $L(H) = L(RH) + L(PH).$6, but the paper does not provide a clean formal statement, so that reading remains interpretive rather than definitive (Alspector, 2021).

The operational workflow described for estimating RED is architecture-dependent but concrete. It consists of choosing a representation formalism; reducing or compressing the representation to a near-optimal form; constructing pre-novelty and post-novelty representations and estimating their compressed lengths; constructing an intermediate $L(H) = L(RH) + L(PH).$7 object aligned as closely as possible to the post-novelty structure without charging for built-in capabilities; estimating the edit cost from $L(H) = L(RH) + L(PH).$8 to $L(H) = L(RH) + L(PH).$9; and normalizing by $D$0. In the simplest approximation, the edit cost is taken as a code-length difference such as $D$1 (Alspector, 2021).

This procedure is explicitly approximate and heuristic. The allowable edit operations are not standardized and depend on representation type. The chosen compressor or code-length estimator also matters, because Kolmogorov complexity is uncomputable. RED is therefore quantitative in spirit but architecture-dependent in practice.

4. Representation-specific interpretations

The paper gives three main representation families—knowledge graphs, regression models, and neural networks—as worked interpretations of RED. In each case, RED is assessed on compressed or near-optimal representations rather than on raw state descriptions (Alspector, 2021).

Representation	Edits or changes considered
Knowledge graph	adding entities, relations, actions, attributes, or changing semantic structure
Regression	changing the function family, changing polynomial degree, and adjusting parameters
Neural net	changing graph structure and changing encoded weights or connections

In the knowledge-graph case, a novelty appears as new concepts or relations that make prior inference incorrect. The graph before and after adaptation should be “most complete, correct, and consistent but simplest,” ideally in its most compressed semantic form. Difficulty is then the normalized code-length change from the training-time graph to the post-novelty graph. The paper notes that graph compression or even semantic distance tools such as NWD-style methods may be used in practice (Alspector, 2021).

The coronavirus example is an extended informal illustration. An early representation of the disease may include infectious disease, quarantine, hand washing, disinfection, and not touching one’s face. Aerosol transmission requires additional transmission and mitigation concepts such as masks and social distancing. Vaccine development requires much deeper refinement, including genome, protein folding, molecular structure, vaccine technology, and especially mRNA vaccine knowledge. The paper’s implication is that the transition from an initial infectious-disease graph to an aerosol-transmission graph has lower RED than the transition from an early pandemic graph to a fine-grained molecular and vaccine-development graph, because the latter requires deeper hierarchy expansion and greater increase in compressed semantic structure (Alspector, 2021).

In the regression case, RED is framed through MDL-based model selection. A third-degree polynomial may be an MDL-optimal fit before novelty. If new data still fits the same family, adaptation may require only small parameter changes and RED is low. If the new data is better explained by a sine function or another function family, the old model produces a large noise term and the representation must change more substantially. The paper suggests defining $D$2 and $D$3 by running model selection before and after novelty and comparing the resulting code lengths. It also makes an explicit posit that $D$4 can be taken as the $D$5 function before parameter optimization; the paper treats this as a heuristic, not a theorem (Alspector, 2021).

In the neural-network case, the compressed object to be edited is the network adjacency matrix plus quantized or binary-coded connections or weights. One compares compressed versions of the pre-novelty and post-novelty networks. If the structure changes, for example through dropout-induced effective architecture changes, $D$6 may use the post-training structure; if structure does not change, pre and post share architecture and only weight encodings differ. The paper also suggests estimating effective experience $D$7 by observing how much the compressed network representation changes when exposed to novel, relevant training samples (Alspector, 2021).

These cases show that RED is representation-centric rather than modality-specific. The same task may yield different RED values when encoded as a knowledge graph, a regression hypothesis, or a neural network, because the distance is defined over effective representations and their compressed edit burden rather than over a representation-invariant semantic substrate.

5. Relation to adjacent edit-distance literatures

RED is not identical to classical edit distance, but it is closely related to several neighboring research programs that compare objects through compressed, structured, or learned representations.

A direct algorithmic analogy appears in run-length encoded string edit distance. The RLE paper computes standard unit-cost edit distance directly on compressed strings in $D$8 time, where $D$9 and $L(D \mid H)$0 are the compressed lengths, using block-DP over piecewise-linear border summaries rather than decompression (Clifford et al., 2019). This is not RED in Johnston’s sense, because it does not measure novelty adaptation or internal mental-model change, but it is a concrete example of edit distance over compressed representations rather than raw uncompressed objects.

In graph similarity computation, the REDRAFT model learns graph-pair representations that emphasize “graph structural difference” after alignment. Its Different Attention mechanism uses $L(D \mid H)$1 to construct attention over graph-level embedding dimensions and predicts GED-derived similarity from the resulting fused representation (Lv et al., 2023). REDRAFT is not a formal RED metric, and it does not provide guaranteed metric properties, but it is closely aligned with the representation-centric intuition that edit-relevant discrepancy should be encoded in a latent representation rather than recovered only through explicit combinatorial matching.

For strings, learned embeddings have pursued analogous goals. CNN-ED trains a convolutional encoder so that Euclidean distance in latent space approximates Levenshtein distance and preserves neighborhood ordering, combining triplet loss with direct approximation error (Dai et al., 2020). Again, this differs from RED’s novelty-adaptation framing, but it shares the ambition of replacing raw edit computations with distances over learned representations.

On the structured-object side, tree edit distance offers a canonical exact model for hierarchical representations. RTED computes minimum-cost edit sequences between ordered labeled trees and is robust across tree shapes, giving a shape-adaptive exact algorithmic backbone for representations that can be normalized into rooted ordered trees (Pawlik et al., 2011). This provides a useful contrast with RED: RTED defines exact edit cost on an external structured object, whereas RED measures how much an agent’s internal representation of a task must be altered.

Theoretical work on edit-distance embeddings and alignment recovery provides another adjacent perspective. Saks and Seshadhri show that approximate distance estimators can be converted into approximate alignments, and that randomized embeddings and dimension-reduction maps can preserve edit structure approximately in Hamming-like or shorter representations (Charikar et al., 2018). This suggests that RED-like systems may benefit from representations that preserve not only scalar dissimilarity but also recoverable alignment structure.

Taken together, these neighboring literatures indicate that “representation-level” edit comparison can mean several distinct things: exact distance on compressed encodings, learned latent surrogates for edit cost, exact edit distance on structured symbolic objects, or approximations preserving alignments. RED differs from all of them in centering adaptation difficulty and internal model change, but it sits naturally among them.

6. Interpretation, limitations, and research status

The paper’s principal claim is straightforward: higher RED predicts harder adaptation. A task with high $L(D \mid H)$2, and by intended approximation high RED, is one where “the evaluation-time behavior needs to differ significantly from the simplest possible optimal training-time behavior in order to achieve sufficient skill.” Low RED indicates that existing representation and priors are already close to what is needed; high RED indicates substantial restructuring of the mental model and, plausibly, greater demands on relevant experience, priors, or optimization effort (Alspector, 2021).

At the same time, the proposal remains a hypothesis and framework rather than an empirically validated metric. The paper provides conceptual arguments from AIT and MDL, along with notional examples across knowledge graphs, regression, and neural networks, but it does not report an empirical benchmark showing that RED numerically predicts adaptation curves. The conclusion explicitly leaves both theoretical justification and experimental validation to future work (Alspector, 2021).

Several limitations are stated directly. RED depends strongly on the chosen representation; the paper says that defining the metric for a particular architecture “requires some thought and will depend strongly on the structure of the representation.” It depends on identifying a near-optimal representation, which is itself a modeling judgment. It depends on the chosen compression or code-length estimator. The edit-cost notion is not invariant across representation languages, so semantically equivalent skills may receive different RED values under different encodings. Some formulas in the paper are malformed, leaving parts of the formalism suggestive rather than fully precise (Alspector, 2021).

There are also important subtleties. One is the assumption that prediction-program complexity is approximately constant and can be ignored; this may be reasonable for fixed architectures but not universally. Another is the assumption that compressed code-length change tracks practical adaptation effort. Optimization difficulty, sample complexity, and wall-clock adaptation time need not coincide with description-length change. A compact representational edit could still be hard to learn in practice. The paper acknowledges this gap indirectly by positioning RED as a proxy rather than as an exact operational measure.

The broader significance of RED is therefore synthetic. It attempts to connect representation learning, compression, description length, semantic distance, and adaptation to novelty within a single measure of difficulty. In its most concise interpretation, RED is the normalized amount of representational editing needed to convert a pre-novelty, near-optimal skill representation into a post-novelty one that attains the required performance threshold. As presently formulated, it is best regarded as a representation-dependent, computable approximation to conditional algorithmic complexity and generalization difficulty, with higher RED indicating that novelty requires a larger restructuring of the agent’s mental model (Alspector, 2021).