Activity-Driven Graph-Edit Forecasting

• The framework models human activities by forecasting explicit graph editing operations, including node deletions, insertions, and edge modifications.
• It integrates diverse formulations—egocentric, third-person, and latent generative—demonstrating robust performance and practical application in dynamic scene analysis.
• Methodologies combine operator-based decoding with latent diffusion, allowing controlled, compositional reasoning about evolving human–environment interactions.

Activity-Driven Graph-Edit Forecasting (A-GEF) denotes a family of formulations in which human activity is modeled through explicit changes to a graph-structured scene or activity state. In the egocentric formulation introduced in "Learning to Evolve Scenes: Reasoning about Human Activities with Scene Graphs" (Pistilli et al., 2 Jul 2026), the state is a spatio-temporal scene graph and the task is to forecast the final consolidated graph after an activity, conditioned on an initial graph and a textual narration. Related formulations realize the same general idea through exact graph-edit matching for activity and next active object prediction in third-person interaction videos (Manousaki et al., 2022), or through latent generative modeling that decodes future scene-graph edits without assuming fixed graph content or structure (Alliegro et al., 8 Mar 2025). Across these variants, the unifying premise is that objects, relations, and their edits constitute an explicit, compositional, and editable substrate for reasoning about human–environment interaction.

1. Formal definitions and problem scope

In the 2026 egocentric formulation, a scene graph at temporal index $i$ is a directed multi-edge graph

$G_i = (V_i, E_i, X_i, Y_i),$

where $V_i$ is the set of object nodes, $E_i \subseteq V_i \times V_i$ is the set of directed edges, $X_i \in \mathbb{R}^{|V_i|\times d}$ are node attributes, and $Y_i \in \mathbb{R}^{|E_i|\times r}$ are edge attributes. $G_t$ denotes a spatial graph at frame $t$, while $G_{t:t+T}$ denotes a consolidated spatio-temporal graph summarizing relations from $t$ to $G_i = (V_i, E_i, X_i, Y_i),$0. Each directed edge $G_i = (V_i, E_i, X_i, Y_i),$1 is associated with a predicate $G_i = (V_i, E_i, X_i, Y_i),$2, producing a triplet $G_i = (V_i, E_i, X_i, Y_i),$3, and each node carries a category in $G_i = (V_i, E_i, X_i, Y_i),$4. SG-Ego fixes $G_i = (V_i, E_i, X_i, Y_i),$5 with $G_i = (V_i, E_i, X_i, Y_i),$6 object classes and $G_i = (V_i, E_i, X_i, Y_i),$7 with $G_i = (V_i, E_i, X_i, Y_i),$8 relation classes (Pistilli et al., 2 Jul 2026).

The forecasting target in that formulation is the final consolidated graph after the activity $G_i = (V_i, E_i, X_i, Y_i),$9, given the initial graph $V_i$0 and the textual narration:

$V_i$1

Equivalently, one can define a set of edits $V_i$2 transforming $V_i$3 into $V_i$4 and model

$V_i$5

where $V_i$6 is a composition of atomic edit operations. This makes scene dynamics an explicitly editable state transition rather than an implicit latent evolution (Pistilli et al., 2 Jul 2026).

A related but distinct formulation appears in "Graphing the Future: Activity and Next Active Object Prediction using Graph-based Activity Representations" (Manousaki et al., 2022). There, the input is a third-person RGB/RGB-D video with tracked human skeletal joints and scene objects, represented as a complete, undirected, attributed spatio-temporal graph over tracked entities. The outputs are the current interaction class, the class(es) of next active object(s) (NAOs), and the time-to-next interaction. Activity prediction is evaluated at observation ratios from $V_i$7 to $V_i$8 of the activity duration in steps of $V_i$9, and NAO forecasting is evaluated at fixed time horizons before the next action segment starts: $E_i \subseteq V_i \times V_i$0 (Manousaki et al., 2022).

FORESCENE formulates the time-varying scene graph as

$E_i \subseteq V_i \times V_i$1

with node attributes consisting of object category and bounding box, and edge attributes consisting of predicate categories. Given $E_i \subseteq V_i \times V_i$2, graph-edit forecasting seeks $E_i \subseteq V_i \times V_i$3 that transforms $E_i \subseteq V_i \times V_i$4 into $E_i \subseteq V_i \times V_i$5 through node additions, removals, attribute changes, edge additions, removals, and predicate changes. In that framework, variable graph cardinality is handled by a permutation-invariant encoder and a DETR-style decoder with Hungarian matching, a special empty class $E_i \subseteq V_i \times V_i$6, and connectivity thresholding for edges (Alliegro et al., 8 Mar 2025).

2. Graph states, edit operators, and edit distance

The explicit edit semantics of A-GEF are most fully spelled out in the 2026 formulation. A graph edit function $E_i \subseteq V_i \times V_i$7 is defined through primitive operations: node deletion, node insertion, node replacement, edge deletion, and edge insertion. In practice, the edit space used by GLEN is restricted to node deletions and insertions, with replacements realized as delete-and-insert, and edge deletions and insertions with multi-label relation classification per node pair. Closed ontologies enforce a consistent schema, directed multi-edges permit multiple predicates between a node pair, and consolidated graphs preserve all relations appearing in the time window (Pistilli et al., 2 Jul 2026).

In the 2022 graph-edit-distance formulation, edit operators appear through the general Graph Edit Distance definition

$E_i \subseteq V_i \times V_i$8

where $E_i \subseteq V_i \times V_i$9 is a sequence of edit operations transforming $X_i \in \mathbb{R}^{|V_i|\times d}$0 to $X_i \in \mathbb{R}^{|V_i|\times d}$1. The specific instantiation is Bipartite Graph Edit Distance (BP-GED), solved via a complete bipartite graph and the Kuhn–Munkres (Hungarian) algorithm. Node insertions and deletions have constant cost $X_i \in \mathbb{R}^{|V_i|\times d}$2, node substitution depends on cross-graph dissimilarity $X_i \in \mathbb{R}^{|V_i|\times d}$3, edge insertions and deletions have constant cost $X_i \in \mathbb{R}^{|V_i|\times d}$4, and edge substitution depends on intra-graph dissimilarities. The final dissimilarity is normalized by the number of matched object pairs $X_i \in \mathbb{R}^{|V_i|\times d}$5:

$X_i \in \mathbb{R}^{|V_i|\times d}$6

This turns edit distance into a forecasting substrate through nearest-neighbor retrieval of a complete reference activity graph from a partially observed test graph (Manousaki et al., 2022).

The same paper defines graph weights by combining motion and semantics:

$X_i \in \mathbb{R}^{|V_i|\times d}$7

where $X_i \in \mathbb{R}^{|V_i|\times d}$8 controls the trade-off between motion and semantics, semantic dissimilarity is derived from WordNet Wu–Palmer similarity, and motion dissimilarity is derived from Segregational Soft-DTW (SSDTW). This gives graph-edit matching a concrete interpretation as an activity-aware comparison of entity trajectories and semantics (Manousaki et al., 2022).

FORESCENE realizes graph edits implicitly rather than by enumerating symbolic operators. Objects appear when decoder queries are assigned non-$X_i \in \mathbb{R}^{|V_i|\times d}$9 classes, disappear when they are mapped to $Y_i \in \mathbb{R}^{|E_i|\times r}$0, relations turn on and off through a learned connectivity matrix, and predicates change through categorical prediction. This suggests a distinction within A-GEF between explicit operator-based decoding and latent generative decoding: both forecast node and edge edits, but they expose different levels of edit-level controllability (Alliegro et al., 8 Mar 2025).

Formulation	Graph substrate	Edit mechanism
GLEN / SG-Ego (Pistilli et al., 2 Jul 2026)	Directed multi-edge spatio-temporal scene graphs	Explicit node deletions/insertions and edge deletions/insertions with multi-label relation classification
GTF (Manousaki et al., 2022)	Complete undirected attributed graphs of joints and objects	BP-GED with node/edge insertions, deletions, and substitutions
FORESCENE (Alliegro et al., 8 Mar 2025)	Time-varying scene graphs with categories, boxes, and predicates	Latent decoding via empty-class assignment, connectivity gating, and predicate prediction

3. Representation construction and datasets

The principal large-scale dataset for the 2026 formulation is SG-Ego, which comprises $Y_i \in \mathbb{R}^{|E_i|\times r}$1M spatio-temporal scene graphs extracted from Ego4D videos, using closed-set vocabularies with $Y_i \in \mathbb{R}^{|E_i|\times r}$2 and $Y_i \in \mathbb{R}^{|E_i|\times r}$3. Its construction proceeds in three stages. First, frames are sampled at $Y_i \in \mathbb{R}^{|E_i|\times r}$4 fps, and Qwen3.5-9B directly captions $Y_i \in \mathbb{R}^{|E_i|\times r}$5 triplets per frame, with malformed or duplicated triplets filtered. Second, GroundingDINO links triplet text spans to bounding boxes, instance suffixes associate subject and object detections, and heuristics filter invalid spatial relations and duplicates, producing a spatial graph $Y_i \in \mathbb{R}^{|E_i|\times r}$6. Third, a consolidation function $Y_i \in \mathbb{R}^{|E_i|\times r}$7 merges $Y_i \in \mathbb{R}^{|E_i|\times r}$8 into $Y_i \in \mathbb{R}^{|E_i|\times r}$9 using masks propagated by SAM2, DINOv2 features, and Hungarian matching with $G_t$0 to associate tracked and detected objects; relations and unmatched nodes are merged sequentially. The stated effect is to resolve temporal fragmentation, occlusions, and caption omissions by canonicalizing entities across time and aggregating all relations observed in the window (Pistilli et al., 2 Jul 2026).

SG-Ego is partitioned into task-specific splits. SG-Ego Align contains approximately $G_t$1M consolidated graphs from $G_t$2 unique videos, with average per-graph statistics of $G_t$3 nodes and $G_t$4 edges. SG-Ego Edit, used for A-GEF, contains $G_t$5k training and $G_t$6k validation triplets $G_t$7, sampled to mirror Ego4D scenario distribution. In the training split, start graphs have $G_t$8 nodes and $G_t$9 edges, while final consolidated graphs have $t$0 nodes and $t$1 edges; in validation, final consolidated graphs have $t$2 nodes and $t$3 edges (Pistilli et al., 2 Jul 2026).

The 2022 graph-edit-distance formulation is evaluated on MSR Daily Activities and CAD-120. MSR Daily Activities contains $t$4 activity classes, with subjects performing activities twice, and provides $t$5D human joints. CAD-120 contains $t$6 complex activities with annotations for activity and sub-activity labels, object labels, affordances, and temporal segmentation. The representation in that work uses upper-body joints and visible objects as graph nodes, with object labels and centroid positions, and dataset-specific preprocessing based on YOLOv4 detections for MSR and ground-truth object labels and $t$7D centroids for CAD-120 (Manousaki et al., 2022).

FORESCENE uses Action Genome, with $t$8 object categories and $t$9 relationship categories across attention, spatial, and contacting relations. It evaluates both GAGS, where observed portions use ground-truth boxes and categories, and PGAGS, where observed portions use ground-truth boxes but categories predicted by Faster R-CNN. It also defines an Object Distribution Shift benchmark through the Jaccard distance between object sets in the last observed and next future frame, with MID and HARD regimes (Alliegro et al., 8 Mar 2025).

4. Model families and forecasting mechanisms

GLEN, the Graph-Language Edit Network, consists of a Graph Encoder $G_{t:t+T}$0, a Text Encoder $G_{t:t+T}$1, and a Graph Edit Model $G_{t:t+T}$2. Its graph encoder uses $G_{t:t+T}$3 TripletGCN layers. Node updates are edge-aware:

$G_{t:t+T}$4

with analogous updates for edge features. Mean pooling over node and edge embeddings produces a graph embedding $G_{t:t+T}$5, followed by a projection MLP. For matching and editing, cross-attention with activity text updates node embeddings as

$G_{t:t+T}$6

with similar conditioning for edges. The edit model augments the input graph with $G_{t:t+T}$7 learnable query nodes, constructs a fully connected augmented graph, conditions nodes and edges on the action text, and applies an edit encoder. The node head predicts $G_{t:t+T}$8, where the $G_{t:t+T}$9-th “no-object” class encodes deletion for existing nodes and insertion for query nodes is indicated by a valid object class. Edge heads predict binary deletion/existence logits and multi-label relation logits over $t$0 (Pistilli et al., 2 Jul 2026).

GLEN also supports Graph-Text Alignment through Graph-Text Alignment (GTA) and Graph-Text Matching (GTM). GTA uses contrastive InfoNCE-style losses on graph and action embeddings with mined positives and negatives, while GTM enables cross-attention and trains an MLP head for binary match/no-match prediction. The combined objective uses scalar weights over alignment and edit supervision, and training uses teacher forcing on the consolidated target: the model predicts $t$1 from $t$2 rather than autoregressing intermediate steps (Pistilli et al., 2 Jul 2026).

The 2022 GTF method is non-parametric. For each partially observed test graph $t$3, it computes $t$4 against all complete reference graphs in the training set, selects the nearest neighbor

$t$5

and uses the induced node correspondences to predict the current activity class, the next active object(s), and the time-to-next interaction. SSDTW provides temporal alignment and a “matching point” in the reference video corresponding to the current test time, allowing projection of the future engagement time of the NAO(s) (Manousaki et al., 2022).

FORESCENE follows a latent generative route. A Graph Auto-Encoder maps variable-size scene graphs to latent vectors $t$6 and reconstructs node classes, boxes, edge predicates, and connectivity. Node features combine visual RoI features and box coordinates, edge features combine node features, union-box features, and learnable semantic embeddings, and a GCN with triplet message passing produces permutation-invariant graph latents through max-pooling. A Transformer decoder with $t$7 object queries reconstructs objects and edges, using Hungarian matching to align predictions to ground truth and the empty class $t$8 to handle variable cardinality. A Latent Diffusion Model is then trained on latent sequences $t$9 with a DDPM objective

$G_i = (V_i, E_i, X_i, Y_i),$00

using sliding temporal windows in which half the tokens are conditioning latents and half are noised targets. At inference, reverse diffusion is applied window by window, and decoded latents yield node appearance/disappearance, relation activation/deactivation, and attribute changes (Alliegro et al., 8 Mar 2025).

5. Supervision, metrics, and implementation

In the 2026 A-GEF pipeline, supervision requires node matching between $G_i = (V_i, E_i, X_i, Y_i),$01 and $G_i = (V_i, E_i, X_i, Y_i),$02 via label agreement and visual feature similarity, with Hungarian assignment used to match $G_i = (V_i, E_i, X_i, Y_i),$03 query nodes to unmatched targets. Edge supervision uses matched node pairs to define multi-label relation targets over $G_i = (V_i, E_i, X_i, Y_i),$04 and binary existence targets. The edit losses are node cross-entropy, binary cross-entropy for edge deletion, and multi-label binary cross-entropy for edge relation classification:

$G_i = (V_i, E_i, X_i, Y_i),$05

The reported implementation uses frozen visual and semantic backbones, a largely frozen text encoder initialized from EgoVLP, $G_i = (V_i, E_i, X_i, Y_i),$06 with $G_i = (V_i, E_i, X_i, Y_i),$07 TripletGCN layers and $G_i = (V_i, E_i, X_i, Y_i),$08, $G_i = (V_i, E_i, X_i, Y_i),$09 with $G_i = (V_i, E_i, X_i, Y_i),$10 TripletGCN layers and $G_i = (V_i, E_i, X_i, Y_i),$11, negative sampling with $G_i = (V_i, E_i, X_i, Y_i),$12 per anchor for GTA, and $G_i = (V_i, E_i, X_i, Y_i),$13 query nodes. Training is reported on a single A100 GPU for approximately $G_i = (V_i, E_i, X_i, Y_i),$14 hours (Pistilli et al., 2 Jul 2026).

A-GEF evaluation in that work uses Triplet Recall@K. If $G_i = (V_i, E_i, X_i, Y_i),$15 is the set of ground-truth triplets in $G_i = (V_i, E_i, X_i, Y_i),$16 and $G_i = (V_i, E_i, X_i, Y_i),$17 is the top-$G_i = (V_i, E_i, X_i, Y_i),$18 predicted triplets ranked by confidence, then

$G_i = (V_i, E_i, X_i, Y_i),$19

Directed multi-edges are respected, so a match requires exact category and directed relation match. The paper also uses an entropy filter on node classifications: predicted nodes whose class posterior entropy exceeds $G_i = (V_i, E_i, X_i, Y_i),$20 are removed, and triplets involving filtered nodes are excluded from $G_i = (V_i, E_i, X_i, Y_i),$21 (Pistilli et al., 2 Jul 2026).

The 2022 GTF formulation does not learn edit costs or graph similarity weights. Its parameters are set experimentally: $G_i = (V_i, E_i, X_i, Y_i),$22, $G_i = (V_i, E_i, X_i, Y_i),$23, $G_i = (V_i, E_i, X_i, Y_i),$24, $G_i = (V_i, E_i, X_i, Y_i),$25, $G_i = (V_i, E_i, X_i, Y_i),$26, $G_i = (V_i, E_i, X_i, Y_i),$27, and $G_i = (V_i, E_i, X_i, Y_i),$28. Activity prediction is evaluated by accuracy versus observation ratio, NAO prediction is evaluated by accuracy at fixed time horizons before the next action begins, and time-to-next interaction error is the absolute offset between predicted and ground-truth engagement times normalized by video length (Manousaki et al., 2022).

FORESCENE optimizes a Graph Auto-Encoder loss

$G_i = (V_i, E_i, X_i, Y_i),$29

where the encoder loss consists of auxiliary node and edge classification heads, the decoder loss combines object, relation, and connectivity losses, and the regularization term is

$G_i = (V_i, E_i, X_i, Y_i),$30

The reported implementation uses a GAE with $G_i = (V_i, E_i, X_i, Y_i),$31 GCN layers, latent dimension $G_i = (V_i, E_i, X_i, Y_i),$32, a decoder with $G_i = (V_i, E_i, X_i, Y_i),$33 blocks, $G_i = (V_i, E_i, X_i, Y_i),$34 heads, $G_i = (V_i, E_i, X_i, Y_i),$35, and $G_i = (V_i, E_i, X_i, Y_i),$36 queries, followed by a DiT-S latent diffusion model with $G_i = (V_i, E_i, X_i, Y_i),$37 blocks, $G_i = (V_i, E_i, X_i, Y_i),$38 heads, $G_i = (V_i, E_i, X_i, Y_i),$39 diffusion steps, and window size $G_i = (V_i, E_i, X_i, Y_i),$40. Metrics include object Recall@K, Jaccard similarity over future-frame object sets, and triplet Recall@K under With Constraint and No Constraint evaluation (Alliegro et al., 8 Mar 2025).

6. Empirical results, positioning, and limitations

On the 2026 A-GEF benchmark, GLEN achieves Triplet Recall@20/50/100 of $G_i = (V_i, E_i, X_i, Y_i),$41, compared with $G_i = (V_i, E_i, X_i, Y_i),$42 for a static $G_i = (V_i, E_i, X_i, Y_i),$43 baseline and $G_i = (V_i, E_i, X_i, Y_i),$44 for a Qwen3.5-9B text baseline. On EgoMCQ, GLEN (Perception Encoder) attains $G_i = (V_i, E_i, X_i, Y_i),$45 on Inter/Intra, and on EgoCVR it reports Global Recall@1/5/10 of $G_i = (V_i, E_i, X_i, Y_i),$46 and Local Recall@1/2/3 of $G_i = (V_i, E_i, X_i, Y_i),$47. On EXPLORE-Bench (Full), GLEN achieves $G_i = (V_i, E_i, X_i, Y_i),$48 with $G_i = (V_i, E_i, X_i, Y_i),$49. Ablations report that combining GTCA and GTM gives the best balance, ensembling GTCA+GTM heads improves EgoMCQ Intra from $G_i = (V_i, E_i, X_i, Y_i),$50 to $G_i = (V_i, E_i, X_i, Y_i),$51, and increasing query nodes from $G_i = (V_i, E_i, X_i, Y_i),$52 to $G_i = (V_i, E_i, X_i, Y_i),$53 improves A-GEF from $G_i = (V_i, E_i, X_i, Y_i),$54 to $G_i = (V_i, E_i, X_i, Y_i),$55 (Pistilli et al., 2 Jul 2026).

In the 2022 graph-edit-distance setting, GTF is reported to outperform competing activity prediction methods on MSR Daily Activities and CAD-120, especially at low observation ratios. On CAD-120 NAO prediction, it achieves $G_i = (V_i, E_i, X_i, Y_i),$56 at $G_i = (V_i, E_i, X_i, Y_i),$57s before the next action and $G_i = (V_i, E_i, X_i, Y_i),$58 at $G_i = (V_i, E_i, X_i, Y_i),$59s, compared with $G_i = (V_i, E_i, X_i, Y_i),$60 and $G_i = (V_i, E_i, X_i, Y_i),$61 for RULSTM at the same horizons. Time-to-next interaction error decreases from $G_i = (V_i, E_i, X_i, Y_i),$62 at $G_i = (V_i, E_i, X_i, Y_i),$63s to $G_i = (V_i, E_i, X_i, Y_i),$64 at $G_i = (V_i, E_i, X_i, Y_i),$65s. For multiple NAO prediction on CAD-120, accuracy rises from $G_i = (V_i, E_i, X_i, Y_i),$66 at $G_i = (V_i, E_i, X_i, Y_i),$67 observation to $G_i = (V_i, E_i, X_i, Y_i),$68 at $G_i = (V_i, E_i, X_i, Y_i),$69 observation. An ablation on $G_i = (V_i, E_i, X_i, Y_i),$70 shows that motion-only and semantic-only settings are inferior, while the combined model performs best with $G_i = (V_i, E_i, X_i, Y_i),$71 across datasets (Manousaki et al., 2022).

FORESCENE reports improvements over relation-only scene graph anticipation baselines on Action Genome while addressing the more complex case of variable object sets. In GAGS at observation fraction $G_i = (V_i, E_i, X_i, Y_i),$72, it reaches No Constraint triplet Recall@10 of $G_i = (V_i, E_i, X_i, Y_i),$73 versus $G_i = (V_i, E_i, X_i, Y_i),$74 for SceneSayerSDE, With Constraint Recall@10 of $G_i = (V_i, E_i, X_i, Y_i),$75 versus $G_i = (V_i, E_i, X_i, Y_i),$76, and object $G_i = (V_i, E_i, X_i, Y_i),$77. At $G_i = (V_i, E_i, X_i, Y_i),$78, it reaches No Constraint Recall@10 of $G_i = (V_i, E_i, X_i, Y_i),$79 versus $G_i = (V_i, E_i, X_i, Y_i),$80, With Constraint Recall@10 of $G_i = (V_i, E_i, X_i, Y_i),$81 versus $G_i = (V_i, E_i, X_i, Y_i),$82, and object $G_i = (V_i, E_i, X_i, Y_i),$83. Under HARD object distribution shift in GAGS, it reports $G_i = (V_i, E_i, X_i, Y_i),$84 versus $G_i = (V_i, E_i, X_i, Y_i),$85 and No Constraint Recall@10 of $G_i = (V_i, E_i, X_i, Y_i),$86 versus $G_i = (V_i, E_i, X_i, Y_i),$87. Ablations indicate that $G_i = (V_i, E_i, X_i, Y_i),$88 diffusion steps are sufficient, $G_i = (V_i, E_i, X_i, Y_i),$89 is best while $G_i = (V_i, E_i, X_i, Y_i),$90 balances performance and efficiency, and auxiliary encoder losses improve performance slightly (Alliegro et al., 8 Mar 2025).

Two recurrent misconceptions are directly addressed by these works. One is that future scene-graph forecasting can assume fixed graph content; FORESCENE explicitly identifies fixed-object assumptions as unrealistic for long-term activities in which interacted objects may appear or disappear. Another is that future graph prediction without activity conditioning is sufficient; the 2026 A-GEF formulation argues that prior scene graph anticipation methods forecast future graphs from past visuals without conditioning on actions, which limits controllability (Alliegro et al., 8 Mar 2025, Pistilli et al., 2 Jul 2026).

The principal limitations are likewise explicit. In the 2026 formulation, errors arise from occlusions and missed captions in stage 1, grounding failures in stage 2, and tracking or matching noise in stage 3; long-horizon dependencies may exceed a single window; closed ontologies limit open-world generalization; and modeling attribute changes via delete-and-insert can momentarily disrupt edge consistency. In the 2022 formulation, sensitivity to detection and tracking errors, reliance on third-person viewpoint, dependence on WordNet-based semantics, and the difficulty of multiple NAO forecasting over long horizons are noted. In FORESCENE, detection and feature noise in the observed portion, rare relations, long-horizon drift, the fixed query upper bound $G_i = (V_i, E_i, X_i, Y_i),$91, and closed-set categories are identified as limitations. The stated future directions include dynamic ontologies, uncertainty-aware edit decoding, principled attribute update operators, recurrent multi-step edit models, hierarchical scene graphs or event graphs, open-vocabulary objects and relations, and integration with spatial prediction and control for embodied AI (Pistilli et al., 2 Jul 2026, Manousaki et al., 2022, Alliegro et al., 8 Mar 2025).