Embedded Reconstructability (EmRec)

• Embedded Reconstructability (EmRec) is a design pattern where recoverable information is embedded in released artifacts to support reliable reconstruction across varied domains.
• It underpins diverse applications, from 3D spatial memory in vision models and provenance in graph mining to leakage analysis in learned representations.
• EmRec enables practical advancements in system optimization, interactivity in mixed reality, and privacy protection by tightly controlling how embedded data can be exploited.

Embedded Reconstructability (EmRec) denotes a family of reconstruction-centered ideas in which recoverable information is treated as embedded in some released artifact, internal memory, query representation, embedding space, or digital surrogate. In the cited literature, the term is applied to several distinct technical settings rather than a single standardized formalism: 3D reconstruction as explicit memory for spatial reasoning in VLMs, provenance-aware output reconstructability in graph pattern mining, leakage of training examples or graph structure from learned representations, reconstruction of an object’s interactivity in mixed reality, and Bayesian reconstruction-security notions that explicitly discount what was already predictable from prior knowledge (He et al., 31 May 2026, Yousefian et al., 25 May 2026, Balle et al., 2022, Shen et al., 2022, Li et al., 14 Feb 2025, Kaplan et al., 29 May 2025).

1. Terminological scope and recurring structure

Across the cited works, EmRec is consistently tied to a question of the following form: what information is embedded in a representation, and under what conditions can that information be reconstructed, exploited, or preserved? The answer varies sharply by domain.

Setting	Embedded substrate	Reconstruction target
VLM spatial reasoning	explicit 3D spatial memory	objects, viewpoints, rendered observations
Graph pattern mining	reconstruction paths on Count nodes	original outputs of batched queries
Model and embedding privacy	model parameters or node embeddings	training points, edges, graph structure
Mixed reality memory systems	geometry, interface, embedded content	interactive digital item
Bayesian security theory	release plus prior/side information	meaningful extraction under relation $R$

In "Geo: A Query Rewrite Framework for Graph Pattern Mining" (Yousefian et al., 25 May 2026), EmRec is explicitly a provenance-aware reconstructability discipline. In "Reconstructing Training Data with Informed Adversaries" (Balle et al., 2022) and "Finding MNEMON: Reviving Memories of Node Embeddings" (Shen et al., 2022), the operative question is whether released parameters or embeddings embed enough information to support recovery of hidden data. In "InteRecon: Towards Reconstructing Interactivity of Personal Memorable Items in Mixed Reality" (Li et al., 14 Feb 2025), reconstructability concerns geometry, interfaces, and embedded content of memorable objects. In "Bayesian Perspective on Memorization and Reconstruction" (Kaplan et al., 29 May 2025), the central issue is whether an apparent reconstruction should count as genuine extraction once prior knowledge is taken into account. "Reasmory: 3D Reconstruction as Explicit Memory for VLMs Spatial Reasoning" (He et al., 31 May 2026) does not present the term EmRec as a formal name, but it is described as very close in spirit to an EmRec-like idea.

A plausible unifying implication is that EmRec is best understood not as one algorithmic recipe but as a design and analysis pattern: reconstruction-relevant information is embedded inside an intermediate or released representation, and the central technical problem is either to exploit that embedding reliably or to limit its consequences.

2. EmRec-like explicit memory in spatial reasoning

"Reasmory" (He et al., 31 May 2026) treats reconstruction not as a terminal perception output but as an internal, explicit memory substrate for spatial reasoning. The paper’s motivation is that in multi-view images and monocular videos, “relevant spatial cues are often sparse and distributed across redundant observations,” and that “adding more frames can increase visual clutter and hinder recall rather than improve spatial understanding.” Reconstruction-based Vision Foundation Models therefore provide “a natural way to aggregate such observations into explicit spatial memory, such as point clouds.”

Formally, each scene is represented as an explicit 3D spatial memory in the form of a colored point cloud. Given views $\{V_1,\dots,V_n\}$, a feed-forward reconstructor $\mathcal R$ predicts depths $\{D_i\}$, camera poses $\{T_i\}$, and intrinsics $K$, with back-projection

$$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$

The memory stores the point cloud $\mathbf X$, camera trajectories $\mathbf T$, and depth maps $\mathbf D$. It is also canonicalized by aligning “the global up direction with the $\{V_1,\dots,V_n\}$0-axis” and using “minimum-area bounding-box alignment” for wall directions.

The architecture has three stages: spatial memory construction, semantic augmentation, and structured interaction. Spatial memory is built from multi-view images or video using feed-forward models such as Pi3 or Depth Anything v3, with Flow3r for dynamic scenes. Semantic augmentation lifts category-specific segmentation proposals into 3D and merges them by cross-view agreement, under the principle that “Proposals corresponding to the same physical object should occupy consistent 3D regions across views.” The resulting object instances store “a category label, an estimated 3D center, and the set of associated 3D points.”

The structured interaction layer is the central mechanism that makes the memory usable during reasoning. Reasmory introduces a lightweight DSL, described as “a restricted subset of Python,” with memory-construction primitives build_static_memory and build_dynamic_memory; memory-query primitives query_camera_pose and query_3d_object_location; memory-transformation primitives set_viewpoint, step_camera, and turn_camera; and rendering primitives render_egocentric, render_semantic_bev, and render_rgb_bev. Reasoning is decomposed into reference viewpoint selection, viewpoint refinement, and entity comparison, abstracted as

$\{V_1,\dots,V_n\}$1

Different stages permit different primitive categories, so access to reconstructed memory is constrained by the reasoning state.

Compiler-level validation is equally central. The AST checks include “syntactic validity; valid function usage; dependency consistency; viewpoint state consistency; execution discipline; and plan consistency,” with representative rules including “Restricted Python subset,” “Whitelisted primitives,” “Define-before-use,” “Stale-query invalidation,” “No dangling motion,” and “Motion-direction alignment.” The stale-query rule is especially revealing: “Query results derived under a previous viewpoint become invalid after viewpoint-changing operations and must be recomputed.”

The empirical results show that explicit memory alone is not the whole story. On MindCube-Tiny, Reasmory improves GPT-5-mini from 58.0% to 76.0% and Gemini-3-flash from 68.7% to 86.0%. On VSI-Bench-Tiny, it improves GPT-5-mini from 49.6% to 55.8% and Gemini-3-flash from 51.6% to 65.0%. On VLM4D-Real, it improves GPT-5-mini from 65.3% to 72.7% and Gemini-3-flash from 76.0% to 82.0%. The ablation is more pointed: directly exposing spatial primitives without verification can reduce performance, whereas “adding the DSL verifier consistently gives the best performance across all benchmarks and both frontier models.” Semantic grounding on a ScanNet subset improves object grounding from mAP50 16.7 to 38.4 and mAP25 35.7 to 68.4 compared to MaskClustering; the compiler feedback loop yields GPT-5-mini pass@1 82.6%, pass@3 95.3%, and Gemini-3-flash pass@3 100%.

This supports an EmRec-like reading in which reconstruction-derived memory is useful not merely because it exists, but because access to it is constrained, validated, and state-consistent.

3. Provenance-aware reconstructability in graph pattern mining

In "Geo" (Yousefian et al., 25 May 2026), EmRec is a mechanism for preserving reconstructability during rewrite-based optimization of graph pattern mining queries. GEO represents pattern matching workloads with a language

$\{V_1,\dots,V_n\}$2

where $\{V_1,\dots,V_n\}$3 is a reconstruction path. Query semantics map a provenance identifier to a numeric result, so reconstructability is encoded directly in the query representation rather than treated as an external bookkeeping task.

The need for EmRec arises because graph mining rewrites are often many-to-many linear equivalences rather than simple one-to-one rewrites. Shared substructure may be introduced across multiple patterns, and naïve algebraic simplification can eliminate intermediate terms still needed to reconstruct the outputs for a batched workload. GEO therefore distinguishes at least two reconstructability modes: individual reconstructability, in which each input pattern must be recoverable separately, and collective reconstructability, in which only the aggregate result over the batch matters.

The formal semantics attach lightweight provenance annotations to Count nodes. The key query clauses are

$\{V_1,\dots,V_n\}$4

$\{V_1,\dots,V_n\}$5

The provenance identifier $\{V_1,\dots,V_n\}$6 serves as a sentinel or unital provenance, and reconstruction paths map provenances to multipliers. The induced algebra satisfies idempotent semiring axioms such as

$\{V_1,\dots,V_n\}$7

During equality saturation, EmRec is embedded through rewrite rules that manipulate Count and Union without discarding provenance. The built-in rules include distributivity of Count over Union,

$\{V_1,\dots,V_n\}$8

and merging counts over identical patterns,

$\{V_1,\dots,V_n\}$9

This is the mechanism that prevents a still-needed pattern from being “simplified away” when subexpressions are shared or canceled.

EmRec in GEO is paired with canonicalization. Patterns are canonicalized using a graph canonizer such as BLISS so that isomorphic patterns share the same representative. The rewrite system must respect canonicalization,

$\mathcal R$0

and the cost function must also respect canonicalization,

$\mathcal R$1

For pattern-matching queries, the paper defines a canonicalization $\mathcal R$2 and proves that it is semantics-preserving: $\mathcal R$3

The optimizer’s correctness theorem makes reconstructability part of semantic equivalence. Theorem 5.1 states: if $\mathcal R$4 respects OGPM and $\mathcal R$5, then if Algorithm 1 returns a query $\mathcal R$6, $\mathcal R$7 is a minimal cost query satisfying $\mathcal R$8. Evaluation shows that this reconstructability discipline is not merely formal overhead. GEO can achieve cost reduction of up to 99% compared to queries in prior work, up to 71% in case studies, and up to 27% improvement when using the same reconstruction path versus unique ones. On the $\mathcal R$9 example, collective reconstructability yields runtime 717 sec versus 987 sec under individual reconstructability. For approximate pattern mining, collective reconstructability gives up to 64% runtime reduction and individual reconstructability up to 62%.

In this usage, EmRec is not about leakage; it is about preserving the ability to reconstruct intended outputs while optimizing aggressively over shared substructure.

4. Leakage-oriented EmRec in learned representations

A different meaning appears in work on privacy and memorization. "Reconstructing Training Data with Informed Adversaries" (Balle et al., 2022) studies whether released model parameters embed enough information about a missing training example that an informed adversary can reconstruct it. The adversary knows the fixed dataset $\{D_i\}$0, the released model parameters $\{D_i\}$1, the training algorithm $\{D_i\}$2, and optionally side knowledge $\{D_i\}$3 about the target point.

For convex models, the reconstruction is algebraic. If the released model is the exact optimizer of an empirical-risk objective, then

$\{D_i\}$4

For generalized linear models trained by regularized maximum likelihood, the paper gives a closed-form recovery of $\{D_i\}$5 and $\{D_i\}$6 when the model has an intercept term. For non-convex models such as neural networks, the paper introduces RecoNN, a learned inverse map from model weights to the target point. The attack trains shadow models $\{D_i\}$7 and then trains a reconstructor on pairs $\{D_i\}$8.

The experimental evidence shows high-fidelity reconstruction. In the default white-box setting, the average reconstruction MSE is $\{D_i\}$9 on MNIST and $\{T_i\}$0 on CIFAR-10, below the nearest-neighbor oracle baselines $\{T_i\}$1 and $\{T_i\}$2. On MNIST, about $\{T_i\}$3 shadow models already give high-fidelity reconstructions, and around $\{T_i\}$4 are enough to outperform the nearest-neighbor oracle; on CIFAR-10, about $\{T_i\}$5 can get below the 1st-percentile oracle distance and $\{T_i\}$6 can beat the nearest-neighbor oracle. The paper also proves reconstruction-robustness guarantees from differential privacy through $\{T_i\}$7-ReRo, with bounds such as $\{T_i\}$8 for pure DP and $\{T_i\}$9 for $K$0-zCDP.

"Finding MNEMON: Reviving Memories of Node Embeddings" (Shen et al., 2022) shifts the substrate from model parameters to node embeddings. Here the attacker has only the embedding matrix

$K$1

plus background knowledge about the origin of the graph, and has no query access, no shadow model training, and no auxiliary labeled data. The attack exploits the fact that connected nodes tend to have approximate closeness in latent space. It estimates average degree $K$2, initializes a seed graph with edge probabilities

$K$3

uses Gumbel-Top-$K$4 sampling to obtain $K$5, then iteratively refines the graph with a learnable weighted cosine metric and a graph autoencoder: $K$6 The self-supervised objective is

$K$7

Recovery is evaluated by Precision, Recall, F1, Joint Degree Distribution, relative Frobenius error, relative triangle error, and relative average clustering coefficient error. For embedding size 256, the paper reports Cora + NetSMF with Precision 0.579, Recall 0.640, F1 0.608, and Actor + DeepWalk with Precision 0.687, Recall 0.435, F1 0.533. The attack is robust to substantial error in the degree estimate and remains effective for embedding sizes 64, 128, and 256. A tentative defense adds Laplace noise to embeddings,

$K$8

reducing attack success but also harming downstream utility; on Cora + GCN, link prediction precision drops from 0.881 to 0.761 at $K$9, while attack F1 drops from 0.486 to 0.088.

These two papers define an explicitly leakage-oriented notion of EmRec: the representation is “embedded” with reconstructable information, and the main question is whether recovery is feasible and how it can be mitigated.

5. Interactive reconstructability in mixed reality

"InteRecon" (Li et al., 14 Feb 2025) uses reconstructability in a different, constructive sense. The paper argues that photos, videos, and 3D scans usually preserve appearance but often fail to preserve interactivity. It introduces the Interactive Digital Item (IDI), defined as an interactive digital reconstruction of a personal memorable item that preserves not only its visual form but also its physical interactivity.

The formative study identifies three major dimensions of physical interactivity: geometry, interfaces, and embedded content. Geometry includes shape, size, texture, appearance, physical properties such as gravity, collision, and motion, and for articulated objects, joints and mechanical transformations. Interfaces include buttons, knobs, sliders, screens, directional pads, and similar tangible controls. Embedded content includes songs, photos, videos, software, stories, usage scenarios, people, rituals, and rules. These become the design goals $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$0.

InteRecon operationalizes those goals through four functions: reconstruct 3D appearance, add physical transforms, reconstruct interface, and add embedded content. Users scan an item with a mobile app on iPhone, receive an automatic bounding box, circle the object in multiple orbits, capture front, side, and bottom views, and reconstruct a 3D model locally. Physical transforms are created by segmenting the mesh manually in AR with a movable cutting plane or automatically with a spectral segmentation method inspired by Style2Fab, then mapping predefined joints. The joint templates are pivot, ball-and-socket, hinge, condyloid, plane, and saddle. For electronic devices, users add widgets drawn from knob, screen, slider, and button, positioning and scaling them to match the original device. Embedded content supports uploaded songs, videos, pictures, and texts.

The paper reports a two-session user study with 16 participants. Session One uses three micro-tasks: T1 scan a toy panda, T2 reconstruct physical transforms for a moon lamp, and T3 reconstruct the interface of a TV IDI. Session Two allows free exploration with personal items. Reported mean ratings on a 7-point scale are ease of use 5.81, learnability 5.88, helpfulness 5.55, expressiveness 6.08, and non-frustration 5.94. Qualitatively, participants reported stronger ownership and emotional connection to IDIs than to static scans, and found reconstructed motions, controls, and content to make the object feel more alive and memorable. At the same time, realistic AR segmentation could be emotionally uncomfortable when an object was split into parts in an immersive environment.

Relative to EmRec, the paper extends reconstructability from visual fidelity to what might be called embedded interactivity: the object’s actions, controls, and associated content are treated as reconstructable components of memory rather than secondary annotations.

6. Bayesian reinterpretation and conceptual boundaries

"Bayesian Perspective on Memorization and Reconstruction" (Kaplan et al., 29 May 2025) reframes reconstruction as a context-dependent notion rather than an absolute property of a release. The paper’s central claim is that one must distinguish what is genuinely learned from the release from what was already knowable from prior knowledge. This position is used to challenge the common tendency to treat all successful extraction-like attacks as “reconstruction.”

The paper begins by contrasting reconstruction with statistical memorization and membership-inference-style memorization. Its XOR warmup example shows that a release can contain large conditional mutual information $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$1 and support a strong membership test, yet still make it impossible to reconstruct much more than half the bits of any row. Reconstruction is therefore not reducible to information-theoretic dependence.

The proposed Bayesian Narcissus-style condition states that algorithm $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$2 is $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$3-BN if, for all attackers $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$4,

$$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$5

where $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$6, $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$7, $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$8, $$\mathbf{x} = T_i^{-1} \left( D_i(u,v)\cdot K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} \right).$$9, and $\mathbf X$0. The interpretation is that a release is reconstruction-safe if it does not let the attacker do much better on the actual sample $\mathbf X$1 than on a fresh sample $\mathbf X$2 from the same underlying distribution. The paper then studies Hamming reconstruction relations

$\mathbf X$3

including exact recovery at $\mathbf X$4.

Under the Tardos-Prior, the paper proves that the exact average is secure for near-exact Hamming reconstruction for every $\mathbf X$5, and gives a bi-criteria extension with a slightly inflated $\mathbf X$6. It then shows that side information about the sample changes the picture decisively: if $\mathbf X$7 reveals the first $\mathbf X$8 rows of $\mathbf X$9, the exact average lets the attacker recover the last row exactly by subtraction, so the earlier notion is insufficient. This motivates a stronger, side-information-aware definition based on conditional surprisal

$\mathbf T$0

The attacker is only credited with success if the reconstructed element remains sufficiently surprising given the side information. The paper proves that under this definition, a noisy average mechanism protected by differential privacy can be secure for exact reconstruction.

The broader conceptual consequence is a direct challenge to one of the field’s recurrent conflations. The paper argues that fingerprinting code attacks are better understood as membership inference attacks rather than reconstruction attacks. This is a substantive boundary claim: impossibility results about traceability or membership do not automatically imply impossibility of reconstruction prevention in the Bayesian sense.

Taken together with the other EmRec usages, this suggests a persistent divide between two questions. One asks how to embed reconstructable structure so that reasoning, optimization, or interaction becomes more capable. The other asks when an embedding leaks so much structure that recovery becomes a privacy or security problem. The cited literature uses the same vocabulary across both sides of that divide, but the underlying objects, guarantees, and threat models differ materially.