REACT++: Real-Time Scene Graph Generation

• REACT++ is a real-time scene graph generation architecture that balances high relation prediction accuracy, object detection precision, and low inference latency.
• It replaces heavy ROI Align with DAMP, integrates global context via AIFI, and utilizes CARPE for efficient prototype-space relation decoding.
• Empirical results on the PSG benchmark show a 20% speed increase and about a 10% average gain in relation accuracy while reducing overall model parameters.

REACT++ is a real-time scene graph generation (SGG) architecture that extends the REACT framework to balance three objectives that are usually in tension: high relation prediction accuracy, high object detection accuracy, and low inference latency. It builds on a decoupled two-stage detector, replaces heavier region-feature extraction with Detection-Anchored Multi-scale Pooling (DAMP), injects global scene context through an AIFI branch, and performs relation decoding in a compact prototype space using CARPE, a cross-attention relation head with explicit geometry encoding. On the reported PSG benchmark with a YOLOv8m-DTS backbone, REACT++ attains the highest inference speed among existing SGG models while improving relation prediction accuracy without sacrificing object detection performance; compared with REACT, it is reported as 20% faster with a gain of 10% in relation prediction accuracy on average (Neau et al., 6 Mar 2026).

1. Terminology and domain scope

The name REACT++ most specifically denotes “REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation”, a computer-vision model for SGG in static images (Neau et al., 6 Mar 2026). In that setting, the goal is to output graph-structured triplets of the form $$\langle \text{subject}, \text{predicate}, \text{object} \rangle$$ from detected objects and their relations.

The term should not be conflated with similarly named systems in other domains. In question answering with LLMs, Focused ReAct is described as a “ReAct++-style enhancement” that adds reiteration and early stop to the Thought–Action–Observation loop in order to reduce focus drift and action loops, but it is not the same scene-graph model family (Li et al., 2024). Likewise, REACT in wearable biosignal modeling denotes a conditioning framework for user-adaptive sEMG hand pose estimation, where a compact user embedding modulates a frozen EMG-to-pose backbone through FiLM; this is a distinct architecture, task, and modality (Xie et al., 28 May 2026).

A common source of confusion is therefore nominal rather than methodological. In current usage, REACT++ is primarily the SGG system, whereas Focused ReAct and the sEMG REACT framework belong to separate literatures with different problem formulations, data modalities, and control mechanisms.

2. Problem setting and design objectives

Scene graph generation combines OD and RelPred. The central design problem addressed by REACT++ is the real-time compromise among three competing objectives: high relation prediction accuracy, high object detection accuracy, and low inference latency (Neau et al., 6 Mar 2026).

The paper motivates the architecture against two established lines of SGG. First, two-stage methods based on Faster R-CNN + ROI Align achieve reasonable relation accuracy, but they are latency-heavy because of expensive feature extraction and pairwise fusion; they also often degrade OD performance because object classification is re-learned within the relation head. Second, one-stage DETR-family models reduce latency by avoiding dense pair enumeration, but their OD accuracy is sub-optimal in SGG benchmarks due to joint learning of objects and relations, and they still need global context and directional geometry bias to capture fine-grained predicates (Neau et al., 6 Mar 2026).

The original REACT addressed latency using a YOLO-based DTS detector and prototype learning for relations, but left three bottlenecks: slow, parameter-heavy ROI Align for subject/object feature extraction; limited global context aggregation; and symmetric prototype interactions, which make directional relations such as $$\langle \text{person}, \text{above}, \text{dog} \rangle$$ versus $$\langle \text{dog}, \text{above}, \text{person} \rangle$$ harder to learn (Neau et al., 6 Mar 2026).

REACT++ is presented as a first-principles redesign of this two-stage setting. It preserves the advantages of a decoupled detector while replacing feature alignment, context pooling, and the relation head with lighter and more expressive modules that operate in a compact prototype space. This suggests that the model’s defining claim is not a single accuracy increase in isolation, but a systems-level rebalancing of feature extraction cost, relation expressivity, and detector preservation.

3. Detection-anchored feature extraction and global context

The backbone is a Decoupled Two-Stage (DTS) design. REACT++ freezes the detector and reuses its class logits, thereby preserving OD accuracy and enabling plug-in detectors from the YOLOv8/9/10/11/12 family. In the reported setup, the stage-1 backbone is a YOLO family detector such as YOLOv8m, with a CSPNet-based backbone feeding a PAN-FPN neck over scales $P3$, $P4$, and $P5$, followed by an anchor-free detection head and Non-Maximum Suppression (NMS). Stage 1 outputs a set of $K$ detections with class probabilities, box coordinates, and indices into the corresponding FPN feature maps, together with the multi-scale feature maps $\{FPN_i\}_{i \in \{3,4,5\}}$ (Neau et al., 6 Mar 2026).

The first major replacement is Detection-Anchored Multi-scale Pooling (DAMP), which substitutes ROI Align with pooled feature vectors gathered directly from detector grid indices using a small Gaussian neighborhood. For a detection with FPN indices $(r_i, c_i)$ at level $i$, REACT++ gathers a $$\langle \text{person}, \text{above}, \text{dog} \rangle$$0 neighborhood with weights $$\langle \text{person}, \text{above}, \text{dog} \rangle$$1:

$$\langle \text{person}, \text{above}, \text{dog} \rangle$$2

Each scale is then projected to a shared dimensionality $$\langle \text{person}, \text{above}, \text{dog} \rangle$$3 and fused:

$$\langle \text{person}, \text{above}, \text{dog} \rangle$$4

This replaces $$\langle \text{person}, \text{above}, \text{dog} \rangle$$5 ROI samples with $$\langle \text{person}, \text{above}, \text{dog} \rangle$$6 gathers, and the gather stage has no learned parameters (Neau et al., 6 Mar 2026).

The paper reports that DAMP reduces feature extraction latency by up to 32% while improving F1@K by approximately +16 points over ROI Align in IndoorVG ablations: F1: 8.0 → 23.8, latency: 29.2 → 20.8 ms. A simpler DA variant is fastest but loses 1.9 F1@K relative to DAMP, whereas DAP and DAM recover some accuracy at negligible latency; DAMP is reported as the best accuracy–latency compromise (Neau et al., 6 Mar 2026).

The second major addition is AIFI, an Attention-based Intra-scale Feature Interaction block used to derive a compact global context vector $$\langle \text{person}, \text{above}, \text{dog} \rangle$$7, functioning as a scene prior such as kitchen, beach, or mountain. Empirically, AIFI yields +0.42 F1@K (+1.8%), at the cost of approximately +2.8 ms latency and about 2M parameters in the relation head (Neau et al., 6 Mar 2026).

These design choices make feature extraction explicitly detector-anchored rather than region-resampled. A plausible implication is that REACT++ treats detector outputs not only as localization hypotheses but also as indexing primitives for low-cost relation-aware feature retrieval.

4. Prototype-space relation reasoning

The core relation head is CARPE, short for Cross-Attention Rotary Prototype Embedding. REACT++ operates in a compact prototype space: instead of enumerating all subject–object pair interactions in a high-dimensional feature space, it lets subject and object features query a bank of learned predicate prototypes via cross-attention. The paper describes this as reducing compute from quadratic in the number of boxes to near-linear in the number of boxes times the number of prototypes, while increasing expressivity through attention, multi-head mixing, and geometry-aware logits (Neau et al., 6 Mar 2026).

CARPE begins from the notation

• subject visual feature $$\langle \text{person}, \text{above}, \text{dog} \rangle$$8 from DAMP,
• object visual feature $$\langle \text{person}, \text{above}, \text{dog} \rangle$$9 from DAMP,
• semantic class embeddings $$\langle \text{dog}, \text{above}, \text{person} \rangle$$0,
• predicate prototype bank $$\langle \text{dog}, \text{above}, \text{person} \rangle$$1 with $$\langle \text{dog}, \text{above}, \text{person} \rangle$$2,
• geometry encoding $$\langle \text{dog}, \text{above}, \text{person} \rangle$$3 derived from union box, centers, widths, heights, and area (Neau et al., 6 Mar 2026).

Semantic inputs are lifted using SwiGLU:

$$\langle \text{dog}, \text{above}, \text{person} \rangle$$4

with subject and object weights optionally tied as $$\langle \text{dog}, \text{above}, \text{person} \rangle$$5 to halve parameters. Predicate prototype seeds are likewise lifted from text embeddings to initialize prototypes (Neau et al., 6 Mar 2026).

To stabilize rare predicates, REACT++ maintains an EMA prototype memory:

$$\langle \text{dog}, \text{above}, \text{person} \rangle$$6

with $$\langle \text{dog}, \text{above}, \text{person} \rangle$$7. If initialized, $$\langle \text{dog}, \text{above}, \text{person} \rangle$$8 replaces the learnable $$\langle \text{dog}, \text{above}, \text{person} \rangle$$9 at inference, improving tail stability (Neau et al., 6 Mar 2026).

The geometry-aware cross-attention uses shared subject/object attention weights but different inputs and geometry biases. With multi-head attention and head dimension $P3$0, the queries, keys, and values are

• subject branch: $P3$1, $P3$2, $P3$3,
• object branch: $P3$4, with shared $P3$5 and $P3$6.

The attention operator is

$P3$7

where the per-head geometry bias $P3$8 conditions on relative position and scale. The enriched subject and object representations are

$P3$9

The associated attention weights

$P4$0

act as mixing coefficients over the prototype bank (Neau et al., 6 Mar 2026).

The final relation embedding is

$P4$1

followed by predicate logits

$P4$2

where $P4$3 applies class-dependent logit adjustments. Triplet ranking then uses the standard two-stage product of detection and predicate priors:

$P4$4

This formulation encodes asymmetry through separate subject and object queries plus geometry-conditioned biases, directly addressing the symmetry limitation identified in REACT (Neau et al., 6 Mar 2026).

5. Optimization, inference, and empirical performance

The computational motivation of REACT++ is explicit. Dense pairwise fusion in earlier two-stage SGG incurs costs of order $P4$5 plus ROI Align interpolation $P4$6. In REACT++, prototype cross-attention scales as $P4$7 per branch, or $P4$8 total, and DAMP gather cost is $P4$9 memory fetches rather than $P5$0 for ROI Align. With typical values $P5$1, $P5$2, $P5$3, and $P5$4, the arithmetic count is similar at the stated nominal point, but REACT++ avoids the quadratic blow-up in keys/values memory; when Dynamic Candidate Selection (DCS) reduces proposals to approximately 47 on PSG, the operation count drops to about $P5$5, and ROI Align is removed entirely. The paper attributes the resulting real-time behavior to this prototype-space formulation, rectangular GPU-friendly matrix sizes, and the smaller parameter count of about 35.8M total (Neau et al., 6 Mar 2026).

Training is decoupled. The detector is trained separately with YOLO recipes, while relation training freezes detector heads and optimizes a predicate loss with focal loss and logit adjustment:

$P5$6

The combined objective is

$P5$7

where optional auxiliaries include prototype EMA warm-up penalties, geometry-embedding regularization, and mild $P5$8 weight decay via AdamW. Recommended hyperparameters on PSG are AdamW, initial learning rate $P5$9, cosine annealing, and gradient accumulation 4; the experiments section also reports a stable SGD variant with learning rate $K$0 (Neau et al., 6 Mar 2026).

The inference pipeline consists of five steps. First, stage 1 runs the YOLO detector with NMS and retains class logits, confidence, and FPN indices. Second, DCS estimates the accuracy–latency curve $K$1 over retained proposal count $K$2 on a validation set and chooses

$K$3

On PSG, saturation occurs at $K$4, reducing latency by about 66.5% on average across DTS backbones with at most 1% F1@K drop. Third, DAMP extracts pooled box features and AIFI computes the global context vector. Fourth, CARPE forms all subject–object pairs among the retained detections, predicts predicates, and ranks triplets using $K$5. Fifth, the top-ranked relations are assembled into the scene graph (Neau et al., 6 Mar 2026).

The reported experiments use PSG, IndoorVG, and VG150, over PredCls, SGCls, and SGDet variants, evaluated with R@K, mR@K, F1@K, OD mAP@50, and latency in ms per image at batch size 1. On PSG with YOLOv8m-DTS, REACT++ without DCS reports mR@20/50/100 = 22.2 / 24.9 / 27.1, R@20/50/100 = 29.1 / 34.2 / 37.5, F1@K = 28.4, OD mAP@50 = 53.1, latency = 25.9 ms, and 35.8M parameters. With DCS, the paper reports the same accuracy and latency = 19.4 ms. The corresponding REACT baseline without DCS reports mR@20/50/100 = 18.3 / 20.1 / 20.9, R@20/50/100 = 27.6 / 30.9 / 32.3, F1@K = 23.9, latency = 32.5 ms, and 43.3M parameters (Neau et al., 6 Mar 2026).

The headline comparison is therefore multifaceted rather than single-metric. On PSG, the summary improvement is 20% faster (32.5 → 25.9 ms), a reported ~10% average gain in relation accuracy, and −17% parameters. The same section also notes that F1@K improves from 23.9 to 28.4, which is approximately +18.8% relative. On IndoorVG, REACT++ reports mR@K = 20.7, R@K = 28.1, F1@K = 23.9, mAP@50 = 37.2, compared with REACT at 18.0, 30.9, 22.8, and 37.2 respectively. On VG150, REACT++ reports mR@K = 13.2, R@K = 28.9, F1@K = 18.2, mAP@50 = 31.8, compared with REACT at 12.9, 27.4, 17.6, and 31.8 (Neau et al., 6 Mar 2026).

6. Limitations, trade-offs, and broader context

The paper identifies several limits. First, noisy datasets such as VG150 reduce the benefits of compact relation heads, and the lower-parameter CARPE head can be more sensitive to label noise and ambiguous classes. Second, REACT++ remains reliant on detector quality: OD accuracy drives relational performance, so poor boxes limit the gains achievable through prototype-space attention. Third, the system is formulated for static images only; the authors propose extending CARPE with temporal prototypes and motion-aware geometry biases for video SGG (Neau et al., 6 Mar 2026).

The trade-off profile is also explicit. DAMP improves both speed and F1@K relative to ROI Align, but AIFI adds a small latency cost, and DCS introduces an operating-point decision based on the empirical accuracy–latency curve. The reported YOLO12 family experiments further suggest a backbone sweet spot: YOLO12m yields F1@K ≈ 30.0 and mAP@50 ≈ 53.5 on PSG at about 28 ms, whereas YOLO12l increases latency without being strictly more accurate in F1@K (Neau et al., 6 Mar 2026). This suggests that relation learning convergence, rather than detector scale alone, constrains the practical optimum.

In application terms, the architecture is positioned as suitable for real-time SGG and for reasoning for embodied agents. The paper states that REACT++ is well-suited to onboard deployment because of its low latency and parameter count, and identifies future integration with navigation/planning and online adaptation of prototypes to domain-specific predicates as directions of interest (Neau et al., 6 Mar 2026).

Within the broader “REACT” naming landscape, REACT++ therefore occupies a specific niche: it is not a general-purpose reasoning policy, nor a personalization module for biosignals, but a compact and decoupled SGG architecture whose central contribution is an efficiency-oriented redesign of relation decoding around detector-anchored features, global scene context, and prototype-space cross-attention.