--- title: Representation Conditioning (RECON) in AI Models url: https://www.emergentmind.com/topics/representation-conditioning-recon type: topic --- # Representation Conditioning (RECON) in AI Models Representation Conditioning (RECON) is a name used for several research frameworks in which a model conditions prediction, generation, inference, or learning dynamics on an explicitly constructed representation rather than relying solely on the original input. The representation may be a bridge-token relation, a knowledge-graph context, a discovered symmetry distribution, a neural scene map, condensed textual evidence, a residual visual-token summary, an aligned semantic feature, or a task-relevant exploration state. Across these uses, RECON denotes an architectural or methodological separation between representation construction and representation-conditioned computation. ## 1. Terminology and conceptual scope The acronym RECON is not associated with a single unified algorithm. It has been used for methods addressing relation extraction, knowledge-graph augmentation, symmetry discovery, structure-from-motion, retrieval-augmented generation, visual-token compression, diffusion generation, model-based imitation learning, long-context memory evaluation, and endogenous market information structures. Despite their different domains, these methods share a general pattern: 1. A model constructs an intermediate representation from the input, external context, or learned state. 2. The representation is designed to expose information that is difficult to use directly. 3. A downstream module conditions its prediction or generation on that representation. 4. The representation is often compressed, normalized, retrieved, projected, or otherwise structured before conditioning. The conditioning mechanism varies substantially. It may use bilinear scores and LogSumExp aggregation, graph message passing, nearest-neighbor distributions on a Lie group, masked attention, textual summarization, residual token selection, additive latent injection, concatenated semantic tokens, or a recoverability-weighted exploration objective. Consequently, “RECON” should be interpreted as a family label rather than evidence that the corresponding systems share code, benchmarks, or theoretical assumptions. A useful abstraction is: $$ \text{input or environment} \;\longrightarrow\; \text{constructed representation} \;\longrightarrow\; \text{conditioned prediction, generation, or action}. $$ The representation can be explicit and human-readable, as in condensed retrieval evidence, or latent and model-specific, as in neural scene features or diffusion semantic tokens. ## 2. Relation extraction and external-context conditioning ### 2.1 Explicit context conditioning The earliest use considered here is “Relation Extraction using Explicit Context Conditioning,” which introduces Representation Conditioning for biomedical and clinical relation extraction [1902.09271]. Standard relation-extraction systems encode a document and directly score a pair of target entities. This direct score is termed a **first-order relation**. It is often effective for intra-sentence relations but can fail when the entities are separated across sentences or connected through complex discourse structure. RECON adds **second-order relations**, also called SOR in the experiments. For target entities $i$ and $j$, every document token $k$ outside the two target entities is treated as a possible bridge: $$ i \rightarrow k \rightarrow j. $$ The model computes a direct relation score and an indirect context-conditioned score. With pairwise scores $B_{ij}$, the score through a context token is $$ C(i,j\mid k)=B_{ik}+B_{kj}. $$ Entity-level scores aggregate mention pairs and candidate bridge tokens using LogSumExp. This provides a smooth approximation to selecting the strongest mention pair or bridge, while producing less sparse gradients than hard maximum selection. The approach differs from ordinary contextualized embeddings. Context is not merely encoded implicitly into token representations; RECON explicitly constructs relations through candidate bridge tokens and performs an additional aggregation over those paths. All document tokens outside the target entities are eligible, including ordinary words and potentially noisy tokens such as stopwords. ### 2.2 Architecture and efficient computation The model is based on BRAN and jointly performs document encoding, entity recognition, first-order scoring, second-order scoring, and score combination. Its encoder is a Transformer variant with multi-head self-attention, residual connections, layer normalization, and a position-wise convolutional feed-forward module containing a kernel-width-5 convolution between kernel-width-1 convolutions. The reported configuration uses two encoder layers, eight attention heads per layer, 256 convolutional filters, dropout probability $0.3$, and word dropout probability $0.15$. Each token representation is projected into separate head and tail spaces for first-order scoring. A separate pair of head/tail MLPs is used for second-order scoring. First-order scores use a learned bilinear tensor $L\in\mathbb{R}^{d\times R\times d}$, whereas second-order scores use a separate tensor $M\in\mathbb{R}^{d\times R\times d}$. A naïve implementation would materialize $C(i,j\mid k)$ for all triples of token indices, requiring $O(N^3)$ intermediate storage. RECON avoids this by exploiting $$ \exp(B_{ik}+B_{kj})=\exp(B_{ik})\exp(B_{kj}), $$ and computing the sum over bridge tokens through batched matrix multiplication. The implementation stores pairwise scores in tensors with effective storage $O(bRN^2)$ rather than cubic context-conditioned tensors. The potentially cubic summation is performed implicitly through optimized dense matrix multiplication. The final relation score is $$ \operatorname{scores} = \operatorname{scores}^{(1)} + \alpha\,\operatorname{scores}^{(2)}. $$ The tuned value was $\alpha=0.2$ for DCN and i2b2 and $\alpha=0$ for CDR. Consequently, on CDR the system effectively reduced to BRAN because second-order evidence did not improve development-set performance. ### 2.3 Results and limitations The method was evaluated on DCN, i2b2 2010, and CDR using macro-averaged precision, recall, and F1. Macro-F1 increased from $0.712$ to $0.734$ on DCN and from $0.395$ to $0.407$ on i2b2. On CDR, macro-F1 remained $0.618$ because tuning selected $\alpha=0$. The results support the interpretation that explicit bridge-token reasoning is useful when relations are distributed over long or complex contexts, but not universally beneficial. The main limitations are the one-token bridge assumption, the use of all tokens as candidate contexts, the absence of learned composition between different relation types, and the restriction to second-order rather than higher-order relations. ### 2.4 Knowledge-graph context A separate model also named RECON uses knowledge-graph context in a graph neural network for sentential relation extraction [2009.08694]. It predicts a relation between two linked entities in a sentence while retrieving external information from Freebase or Wikidata. The model combines: - **Entity Attribute Context (EAC)**: labels, aliases, descriptions, and instance-of/type values; - **Triple Context**: one- and two-hop factual neighborhoods; - **Sentence context**: a sentence graph whose transitions are conditioned on the input sentence. The knowledge graph is represented as $$ KG=(\mathcal E,\mathcal R,\mathcal T^+), $$ and the sentence entities are linked to KG identifiers. The sentence encoder uses a bidirectional LSTM and an MLP to generate sentence-dependent transition matrices for graph propagation. Initial graph node states are supplied by the entity-attribute encoder, so the sentence graph begins with KG-derived representations rather than sentence-only states. Triple context is modeled with **KGGAT-SEP**, an extension of KBGAT that uses graph attention and keeps entity and relation embeddings in separate vector spaces. Relation-specific nonlinear transformations map entities into relation-specific spaces, avoiding the restriction of a single translation space when multiple relations, inverse patterns, or multi-hop structures occur. On Wikidata, the full RECON model reaches a micro-F1 of $87.23$ and a macro-F1 of $44.23$, compared with $82.29$ micro-F1 for GP-GNN. On NYT Freebase, it reaches $87.5$ P@10 and $74.1$ P@30. The model can infer relations implicit or absent from the sentence, such as occupation, spouse, or performer relations. Its limitations include sparse induced KG graphs, the computational cost of retrieving and processing two-hop neighborhoods, possible harm from irrelevant KG context, incomplete end-to-end optimization of EAC and triple context, and unmeasured sensitivity to KG quality. ## 3. Symmetry, geometric reconstruction, and visual conditioning ### 3.1 Symmetry discovery and canonical orientation “Robust symmetry discovery via Explicit Canonical Orientation Normalization” uses RECON for instance-dependent symmetry discovery [2505.13289]. The method addresses the mismatch between fixed equivariant groups and data whose symmetries may be partial, approximate, class-dependent, or input-dependent. RECON builds on an invariant–equivariant autoencoder. An input $x$ is decomposed into an invariant content representation $z=\eta(x)$ and an equivariant pose representation $g=\psi(x)$. A decoder reconstructs a learned canonical object, while the pose estimator maps that canonical representation back to the observed input. The central problem is gauge ambiguity. The learned canonical orientation may differ from the natural reference pose by an arbitrary group element $\Gamma_{[x]}$. Consequently, observed pose distributions are right-translated versions of the intrinsic symmetry distribution: $$ \nu_{[x]}=\mu_{[x]}\Gamma_{[x]}. $$ Two classes with identical intrinsic symmetry can therefore appear to have different pose distributions merely because their learned canonical orientations differ. RECON estimates the offset with an empirical Fréchet mean over poses from nearest neighbors in invariant space: $$ \widehat\Gamma_{[x]} = \arg\min_{y\in G} \sum_{g_i\in\psi([x])}d_G(y,g_i)^2. $$ The pose samples are then right-normalized: $$ \psi'([x]) = \{g_i\widehat\Gamma_{[x]}^{-1}\}. $$ The resulting identity-centered distribution is an estimate of the intrinsic symmetry distribution. Under the paper’s assumptions—including a Lie group, continuity of the pose estimator, approximate generative structure, and identity-centered intrinsic distributions—the normalized empirical distribution converges to the target distribution in Wasserstein distance as noise decreases and neighborhood size increases. The method uses $k=25$ invariant-space neighbors to construct pseudo-labels, then trains invariant predictors for the centering transformation and normalized symmetry parameters. It evaluates $SO(2)$ distributions on rotated MNIST and FashionMNIST and $SO(3)$ matrix-Fisher distributions on molecular conformers from GEOM. Reported out-of-distribution pose-detection AUC-ROC values are $0.92$ on MNIST, $0.80$ on FashionMNIST, and $0.75$ on GEOM. RECON does not infer an entirely unknown transformation group. The ambient group $G$ must still be selected in advance. Its contribution is to discover the distribution within that group and remove arbitrary canonical offsets. ### 3.2 Scene representation conditioning for large-scale SfM SAIL-Recon applies representation conditioning to large-scale structure-from-motion [2508.17972]. It first constructs a reusable neural scene representation from a relatively small set of anchor images and then localizes query images against that representation. Given $M$ images, $N\ll M$ are selected as anchors. The anchor pass uses a VGGT-initialized Transformer to infer a global scene representation. Unlike a representation consisting only of final-layer tokens, SAIL-Recon retains features from all 24 Transformer layers and applies token downsampling: $$ \mathcal R = \left[ \left\{ \Theta(t_j^{\prime\mathcal I}) \right\}_{j=1}^{L} \right]. $$ The resulting representation is a multi-level implicit scene map. Early layers preserve visual appearance and local features, whereas later layers encode increasingly geometric information. A query image attends to the anchor-derived representation at corresponding Transformer layers. Query tokens may attend to their own image and to scene-representation tokens, but not to other query images. The scene representation is cached as keys and values, so query images can be processed independently and sequentially or in manageable batches. The system predicts camera parameters, depth maps, and dense scene-coordinate maps. It uses 50–100 anchors in many reported large-scale experiments and can process thousands of images while peak memory is determined primarily by the anchor representation and one query batch rather than by all images jointly. On TUM-RGBD, SAIL-Recon reports an average uncalibrated absolute trajectory error of $0.051$ m. On CO3Dv2, ten anchors produce RRA@15 of $98.3$, RTA@15 of $94.0$, and mAA@30 of $88.1$, close to the corresponding full-VGGT values. On Tanks and Temples, it reports RRA@5 of $70.4$, RTA@5 of $74.7$, ATE of $0.008$, and runtime of 81 seconds in the listed feed-forward comparison. The main limitation is anchor coverage. Uniform sampling can miss visually diverse or weakly textured regions, and large viewpoint gaps between anchors and queries can degrade localization. ### 3.3 Projected representation conditioning for novel-view synthesis ReNoV uses external visual representations to condition diffusion-based novel-view synthesis [2602.12003]. It is designed for sparse, unposed image collections and combines explicit geometry with semantically and geometrically informative features. The system uses a reference U-Net and a denoising U-Net initialized from Stable Diffusion 2.1. External representations include DINOv2-L, VGGT, and Depth Anything V3-L. VGGT features are extracted from multiple layers, concatenated, and attached to predicted 3D point maps. The resulting feature point cloud is projected into the target camera. The target condition contains projected geometry, projected representation features, and a validity mask: $$ c_{\mathrm{tgt}}^d = [ \gamma(X_{\mathrm{tgt}}^\Pi); \gamma(T_{\mathrm{tgt}}^\Pi); M_{\mathrm{tgt}} ]. $$ A shallow convolutional projection module transforms this condition and adds it to the noisy target latent: $$ \widetilde z_t = z_t+\phi_{\mathrm{tgt}}(c_{\mathrm{tgt}}^d). $$ The method also uses reference-to-target spatial attention. Thus, its conditioning is hybrid: 3D lifting, feature concatenation, convolutional projection, additive latent injection, and reference-target attention. On RealEstate10K far-view synthesis, ReNoV with VGGT reports PSNR $17.49$, SSIM $0.598$, and LPIPS $0.247$. In an ablation, warped image conditioning alone gives PSNR $16.55$, point-map conditioning gives $16.93$, and point-map plus VGGT features gives $17.50$. Removing 50% of projected geometric points decreases PSNR by only $0.12$ dB in the reported robustness experiment. The approach remains dependent on external geometry quality, feature-layer selection, multi-view feature extraction, and diffusion-based completion of regions lacking projected evidence. It is evaluated on static scenes and does not address dynamic objects or nonrigid motion. ## 4. Textual, token-level, and diffusion representation conditioning ### 4.1 Condensed evidence for retrieval-augmented generation “RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation” introduces a learned textual evidence bottleneck for Search-R1-style reinforcement-learning-based RAG [2510.10448]. Retrieved documents are passed through a separately trained summarizer rather than concatenated directly into the reasoning trajectory. The summarizer is trained in two stages. First, it receives relevance pretraining on MS MARCO using a Qwen2.5-3B-Instruct model with a classification head. Second, it is distilled from GPT-4o-mini on NQ and HotpotQA using prompts emphasizing factual correctness, completeness, coverage, coherence, clarity, and logicality. During reasoning, the policy emits a search query, receives retrieved documents, and passes them through the frozen summarizer: $$ d' \gets S(d). $$ The condensed evidence is inserted into the trajectory as an information block. Retrieved evidence tokens are excluded from the PPO policy loss, while policy-generated reasoning, search, and answer tokens remain optimized. The summarizer and reasoner are not trained through a joint end-to-end objective. The summarizer is trained with supervised relevance and distillation objectives, then frozen while the policy is optimized with PPO. Relative to Search-R1, RECON increases documents retrieved per turn from three to five and maximum reasoning/search turns from three to five. It reports a 35% context-length reduction, average inference time falling from 28.8 seconds to 19.9 seconds, and a 5.2% training-speed improvement in the reported setup. Average exact match improves from $0.303$ to $0.347$ for the 3B model, a 14.5% relative improvement, and from $0.431$ to $0.444$ for the 7B model, a 3.0% relative improvement. The strongest gains occur on several multi-hop and adversarial benchmarks, although PopQA is a counterexample where RECON slightly underperforms Search-R1. Limitations include evidence omission, teacher-model bias, summarizer latency, domain-transfer uncertainty, and the absence of comprehensive ablations isolating relevance pretraining and each distillation aspect. ### 4.2 Reconstruction-guided visual-token condensation ReConV uses conditioning to compress visual-token sequences in Vision Transformers and multimodal language models [2610.17412]. It maintains a compact **Summary Bank (SMB)** that summarizes shared information across all current visual tokens. SMB tokens query the visual sequence through asymmetric cross-attention. Each visual token is assigned to one SMB slot, and a position-conditioned decoder reconstructs each token from its assigned summary token and spatial embedding: $$ \widehat v_i = g_\theta \left( [ \phi_s(s_{m_i}); \phi_p(p_i) ] \right). $$ The method computes an orthogonal reconstruction residual: $$ r_i = v_i-(v_i^\top u_i)u_i, \qquad u_i=\frac{\widehat v_i}{\|\widehat v_i\|}. $$ Tokens with large residual norms are retained because their information is not adequately captured by the shared summary. Tokens with small residuals are discarded. The representation therefore decomposes into a compact shared representation and explicit complementary residual tokens. The reconstruction module is trained with an InfoNCE objective, using hard negatives from different positions assigned to the same SMB slot and easy negatives from other images. The full objective combines the task loss with a weighted reconstruction loss. On ImageNet with a DeiT-S backbone, ReConV reports 2.88 GMACs and 80.4% top-1 accuracy, compared with 4.61 GMACs and 79.8% for the uncompressed baseline. A more aggressive configuration reports 2.42 GMACs and 79.9% accuracy. In LLaVA-style experiments, ReConV reduces visual-token counts to 32–128 while retaining substantial multimodal performance and reports a 3.07× POPE prefill speedup at 32 tokens. The method is prompt-independent, allowing a compressed image representation to be reused across multiple questions. This improves multi-turn efficiency but sacrifices prompt-specific token selection. Extreme compression degrades performance, and reconstruction residuals are only an approximation to downstream information value. ### 4.3 Semantic conditioning of diffusion-transformer layers “Representation Conditioning for Generation” applies RECON to diffusion transformers trained with flow matching [2610.20608]. It addresses a limitation of representation-alignment methods such as REPA: an intermediate diffusion representation may become aligned with a pretrained vision encoder, but later layers are not guaranteed to preserve or use that semantic information. REPA uses a projector $g_\phi$ to align an intermediate diffusion representation $h$ with frozen vision-encoder tokens. The projector is commonly discarded after training. RECON retains it as a semantic interface. During training, semantic tokens are obtained from clean-image vision-encoder features: $$ s_{\mathrm{train}} = m(\operatorname{Pool}(f_{\mathrm{ext}}(x_0))). $$ They are concatenated with diffusion patch tokens after an alignment depth and supplied to later Transformer blocks. Teacher-forced semantic conditioning is used only for $t