Dual Query Enhancement Network (DQEN)

• Dual Query Enhancement Network (DQEN) is a design principle that maintains two distinct query streams with asymmetric roles to handle temporal and semantic discrepancies.
• It refines target queries by aligning and fusing support queries using mechanisms like deformable attention, motion cues, and CLIP-based semantic priors.
• Empirical results on nuScenes and HICO-Det confirm that structured query enhancement outperforms naive token fusion in complex detection tasks.

Dual Query Enhancement Network (DQEN) denotes a class of architectures in which two distinct query representations are explicitly enhanced rather than treated as undifferentiated latent tokens. In the literature provided here, the term appears in two closely related but domain-specific forms: as an interpretive description of D-Align, a multi-frame LiDAR 3D detector built around a dual-query co-attention network for target and support frames (Lee et al., 2022), and as the explicit name of a DETR-based Human-Object Interaction (HOI) detector that enhances object and interaction queries with visual and semantic priors (Li et al., 26 Aug 2025). Across both uses, the central premise is that multi-source evidence is beneficial only when query semantics are structured and the discrepancy between sources is handled explicitly.

1. Terminological scope and core definition

In the available literature, DQEN is not a single canonical architecture. Instead, it names a recurring design principle: maintain two query streams or two query enhancement branches, use one to inform the other, and update them through mechanisms that are more structured than naive concatenation or purely random initialization.

Instantiation	Domain	Dual-query formulation
D-Align	Multi-frame LiDAR 3D detection	target query set (T-QS) and support query set (S-QS)
DQEN	DETR-based HOI detection	object query enhancement and interaction query enhancement

For D-Align, the motivation is temporal: increasing the number of LiDAR sweeps helps only up to a point because point cloud distributions change over time, especially for moving objects, so a detector must model inter-frame motion and feature misalignment explicitly (Lee et al., 2022). For the HOI model named DQEN, the motivation is semantic: prior DETR-based HOI methods such as QPIC, GEN-VLKT, HOICLIP, and TED-Net typically use randomly initialized queries, which the paper characterizes as producing vague query semantics, particularly for objects and interactions (Li et al., 26 Aug 2025).

This suggests that “dual query enhancement” is best understood as a methodological template rather than a task-specific module. Its recurring elements are asymmetric query roles, explicit enhancement signals, and iterative refinement.

2. D-Align as a dual-query enhancement network for multi-frame LiDAR

D-Align is a 3D object detector for point cloud sequences with three blocks: a BEV feature extractor, DUCANet (dual-query co-attention network), and a 3D detection head (Lee et al., 2022). Given a LiDAR sequence $\{P_n\}_{n=t-N+1}^{t}$, frame $t$ is the target frame and earlier frames are support frames. Each frame is encoded into multi-scale BEV feature maps,

$F_n=\{F_n^s\}_{s=1}^{S}.$

DUCANet maintains a target query set $B_t^{(l)}$ and support query sets $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$, where $l$ indexes attention layers. Initialization is

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$

At each layer, IDANet aligns each support query to the target query, IGANet fuses aligned support and target queries with gating, and the refined queries are passed to the next layer. After $L$ layers, the final target query

$\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$

is used as the enhanced BEV representation for detection.

The defining asymmetry is important. T-QS and S-QS are not used symmetrically in the final objective: support queries are aligned to the target and then fused into the target, so the target frame representation is progressively strengthened while support frames serve as auxiliary temporal evidence. That asymmetry is the basis for interpreting D-Align as a DQEN rather than as a generic temporal fusion module.

The paper’s stated motivation is that conventional detectors use points acquired over a fixed duration, whereas spatio-temporal information from point cloud sequences can further enhance performance only if temporal discrepancy is addressed explicitly. Naive stacking or recurrence is therefore insufficient in the sense described by the paper, because moving objects produce changing point distributions even after ego-motion compensation (Lee et al., 2022).

3. Motion-aware alignment, gated aggregation, and iterative refinement in D-Align

A distinctive component of D-Align is the use of temporal context features to guide alignment. The feature pyramid motion extraction network, FPMNet, computes motion cues from the current target query and a support query at each scale:

$$M_{t-k}^{(l),s} = \mathrm{conv}_{3\times3}(B_t^{(l),s} - U_{t-k}^{(l),s}).$$

These are transformed across pyramid levels and concatenated as

$t$0

yielding the temporal context feature pyramid $t$1 (Lee et al., 2022).

In IDANet, the support query set is aligned to the target query set using deformable attention. For head $t$2, scale $t$3, and reference location $t$4, the projected support embedding is

$t$5

with $t$6 and $t$7. Offset masks and attention weights are predicted from the motion context:

$t$8

$t$9

The attention output is

$F_n=\{F_n^s\}_{s=1}^{S}.$0

and the refined support query is updated by

$F_n=\{F_n^s\}_{s=1}^{S}.$1

In IGANet, aligned support queries are fused with the target query by a gated attention mechanism:

$F_n=\{F_n^s\}_{s=1}^{S}.$2

with gate

$F_n=\{F_n^s\}_{s=1}^{S}.$3

The fused outputs across support frames are then aggregated:

$F_n=\{F_n^s\}_{s=1}^{S}.$4

The model is explicitly iterative. DUCANet stacks $F_n=\{F_n^s\}_{s=1}^{S}.$5 attention layers, each implementing

$F_n=\{F_n^s\}_{s=1}^{S}.$6

On nuScenes validation, the ablation reported for the PointPillars baseline is 44.40 mAP / 58.15 NDS; adding IGANet only gives 46.17 / 59.42; adding IGANet + IDANet (no FPMNet) gives 48.19 / 60.60; and the full D-Align with FPMNet reaches 49.66 / 61.33. Increasing the number of attention layers from 1 to 2 to 3 improves performance from 46.44 / 59.55 to 48.58 / 60.72 to 49.66 / 61.33. The paper therefore attributes the gain not merely to multi-frame aggregation, but to alignment and motion-guided query refinement (Lee et al., 2022).

4. DQEN for DETR-based human-object interaction detection

The DQEN model for HOI detection is built on a single encoder + dual decoder architecture (Li et al., 26 Aug 2025). A CNN backbone extracts a feature map $F_n=\{F_n^s\}_{s=1}^{S}.$7 from input image $F_n=\{F_n^s\}_{s=1}^{S}.$8; with positional encoding $F_n=\{F_n^s\}_{s=1}^{S}.$9, a Transformer encoder produces visual memory

$B_t^{(l)}$0

This encoder output is shared by an instance decoder and an interaction decoder.

The instance decoder takes human queries

$B_t^{(l)}$1

initialized randomly and enhanced object queries

$B_t^{(l)}$2

It outputs human features

$B_t^{(l)}$3

and object features

$B_t^{(l)}$4

The interaction stream begins from a semantic interaction feature

$B_t^{(l)}$5

that is repeated $B_t^{(l)}$6 times to form

$B_t^{(l)}$7

This repeated semantic feature is combined with $B_t^{(l)}$8 and $B_t^{(l)}$9 to form interaction queries $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$0, and the interaction decoder outputs

$\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$1

A skip connection from $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$2 to $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$3 is used for final interaction or verb prediction.

The first enhancement branch is Object Query Enhancement (OQE). Encoder output $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$4 is passed through a linear classifier to produce object category scores, and the Top-$\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$5 highest-scoring encoder features are selected:

$\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$6

Object queries are then enhanced by

$\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$7

where $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$8. A crucial training detail is that $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$9 is duplicated as classifier input and gradient backpropagation is disabled through this classifier path at this stage. The ablation reported in the paper states that allowing direct backpropagation through this path lowers performance. The authors further note that OQE does not improve object detection mAP: the baseline GEN-VLKT object mAP is 34.86, “Ours w/o OQE” is 33.01, and “Ours w/ OQE” is 32.72. The interpretation offered in the paper is that the selected object-like features often emphasize the person-object interaction region, which helps HOI detection even when pure object detection accuracy decreases (Li et al., 26 Aug 2025).

5. Interaction semantic enhancement, auxiliary prediction, and scoring in HOI DQEN

The second branch, Interaction Query Enhancement (IQE), is the main contribution in the HOI paper (Li et al., 26 Aug 2025). Each HOI triplet $l$0 is converted to a prompt of the form “A photo of a person [verb-ing] a/an [object]”, creating a text set $l$1. A CLIP image encoder and text encoder then produce

$l$2

with $l$3 and $l$4. Similarity is computed as

$l$5

and top-$l$6 matching HOI labels are selected:

$l$7

The paper states that CLIP Top-8 matched labels can cover over 50% of HICO-Det ground-truth HOIs.

The model does not directly use CLIP text features as interaction queries. Instead, it introduces Interaction Semantic Fusion (ISF) to disentangle and fuse verb and object semantics. From $l$8, verb words, object words, and full HOI triplets are encoded as

$l$9

Verb and object words are fused by

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$0

and HOI triplet embeddings are projected as

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$1

A correlation matrix is computed:

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$2

where $B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$3 is learnable self-attention weights. The fused semantics are reweighted by

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$4

and the final semantic fusion is

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$5

Summing over candidate dimension and projecting yields the semantic interaction feature:

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$6

The paper also introduces an Auxiliary Prediction Unit (APU). An additional verb classifier $B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$7 takes

$B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$8

as input, producing $B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.$9. These scores are combined with the interaction prediction $L$0 from $L$1:

$L$2

The training objective includes the DETR-style matching cost,

$L$3

together with cross-entropy loss $L$4 for the object classifier and CLIP knowledge distillation loss $L$5, giving

$L$6

The paper specifies $L$7, $L$8, $L$9, $\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$0, and $\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$1.

At inference, a training-free CLIP inference option is used. From $\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$2, the top-$\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$3 normalized similarities are selected as

$\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$4

with $\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$5. For HOI category $\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$6, the final triplet score is

$\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$7

followed by triplet NMS.

6. Training regimes, empirical behavior, and interpretive significance

The two DQEN instantiations differ substantially in optimization and evaluation protocol. D-Align does not introduce a new detection loss; it is trained with the same losses as its base detector, specifically PointPillars loss for the anchor-based version and CenterPoint loss for the anchor-free version. Training uses the Adam optimizer, pretraining of the single-frame detector, and then fine-tuning of the full D-Align network. The schedule is 20 epochs of pretraining with max learning rate 0.001, followed by 40 epochs of fine-tuning with max learning rate 0.0002, with random flipping, rotation, scaling, and ground-truth box sampling as data augmentation (Lee et al., 2022).

On nuScenes, D-Align is evaluated with sequence length $\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$8 by default, using 1 target frame and 2 support frames; each frame uses 10 consecutive LiDAR samples over 0.5 s; DUCANet uses $\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}$9 attention layers; deformable attention uses $$M_{t-k}^{(l),s} = \mathrm{conv}_{3\times3}(B_t^{(l),s} - U_{t-k}^{(l),s}).$$0 heads and $$M_{t-k}^{(l),s} = \mathrm{conv}_{3\times3}(B_t^{(l),s} - U_{t-k}^{(l),s}).$$1 sampling points. The test-set result highlighted for D-Align-CP is 70.2 NDS and 64.0 mAP, with the paper further reporting a 22.5% gain over PointPillars and a 3.7% mAP gain over CenterPoint. Gains are described as especially strong on motorcycle, bicycle, and traffic cone (Lee et al., 2022).

The HOI DQEN uses ResNet-50, ViT-B/32 for CLIP, 3 + 3 encoder/decoder layers, 64 queries, feature dimension 256, CLIP embedding dimension 512, word embedding dimension 512, AdamW, and 90 epochs with learning rate $$M_{t-k}^{(l),s} = \mathrm{conv}_{3\times3}(B_t^{(l),s} - U_{t-k}^{(l),s}).$$2 reduced after 60 epochs. The paper reports about 48M parameters and about 29.02 FPS (Li et al., 26 Aug 2025).

On HICO-Det, the main result is 35.23 Full, 32.79 Rare, and 35.96 Non-Rare for DQEN without training-free CLIP, and 35.34, 33.10, and 36.01 with training-free CLIP. The paper states gains over GEN-VLKT of +1.48 mAP Full, +3.54 mAP Rare, and +0.86 mAP Non-Rare in the default setting, and +1.72 / +3.67 / +1.13 in the known-object setting. On V-COCO, DQEN reports $$M_{t-k}^{(l),s} = \mathrm{conv}_{3\times3}(B_t^{(l),s} - U_{t-k}^{(l),s}).$$3 and $$M_{t-k}^{(l),s} = \mathrm{conv}_{3\times3}(B_t^{(l),s} - U_{t-k}^{(l),s}).$$4, while also noting that improvement is smaller than on HICO-Det because V-COCO has more human-object pairs per image, making candidate selection and semantic enhancement less precise. The key HICO-Det default ablation gives 32.68 for baseline, 33.72 for +APU, 33.28 for +OQE, 34.22 for +IQE, 33.50 for +APU+OQE, 34.47 for +APU+IQE, 34.26 for +OQE+IQE, and 35.23 for all three modules (Li et al., 26 Aug 2025).

A common misconception would be to treat “dual query” as meaning simple duplication of decoder tokens. The two papers indicate a stricter meaning: in D-Align, the distinction is between target and support frame representations, with alignment preceding fusion; in HOI DQEN, the distinction is between object-aware and interaction-aware query enhancement, with visual priors and CLIP-derived semantic priors serving different roles. Another possible misconception is that DQEN implies symmetric information exchange. The available evidence points in the opposite direction: D-Align uses support frames as auxiliary evidence for target enhancement, whereas HOI DQEN enhances separate downstream streams for instance localization and interaction recognition. This suggests that the unifying concept is not symmetry, but controlled asymmetry in query design.

Taken together, these works present DQEN as a response to two failure modes of standard architectures: unmodeled temporal misalignment in multi-frame BEV detection and vague query semantics in DETR-based HOI detection. In both cases, performance gains are tied to explicit enhancement signals rather than to additional frames or additional queries alone.