---
title: Dual Query Enhancement Network (DQEN)
url: https://www.emergentmind.com/topics/dual-query-enhancement-network-dqen
type: topic
---

# Dual Query Enhancement Network (DQEN)

Dual Query Enhancement Network (DQEN) denotes a class of architectures in which two distinct query representations are explicitly enhanced rather than treated as undifferentiated latent tokens. In the literature provided here, the term appears in two closely related but domain-specific forms: as an interpretive description of D-Align, a multi-frame LiDAR 3D detector built around a dual-query co-attention network for target and support frames [2210.00087], and as the explicit name of a DETR-based Human-Object Interaction (HOI) detector that enhances object and interaction queries with visual and semantic priors [2508.18896]. Across both uses, the central premise is that multi-source evidence is beneficial only when query semantics are structured and the discrepancy between sources is handled explicitly.

## 1. Terminological scope and core definition

In the available literature, DQEN is not a single canonical architecture. Instead, it names a recurring design principle: maintain two query streams or two query enhancement branches, use one to inform the other, and update them through mechanisms that are more structured than naive concatenation or purely random initialization.

| Instantiation | Domain | Dual-query formulation |
|---|---|---|
| D-Align | Multi-frame LiDAR 3D detection | target query set (T-QS) and support query set (S-QS) |
| DQEN | DETR-based HOI detection | object query enhancement and interaction query enhancement |

For D-Align, the motivation is temporal: increasing the number of LiDAR sweeps helps only up to a point because point cloud distributions change over time, especially for moving objects, so a detector must model inter-frame motion and feature misalignment explicitly [2210.00087]. For the HOI model named DQEN, the motivation is semantic: prior DETR-based HOI methods such as QPIC, GEN-VLKT, HOICLIP, and TED-Net typically use randomly initialized queries, which the paper characterizes as producing vague query semantics, particularly for objects and interactions [2508.18896].

This suggests that “dual query enhancement” is best understood as a methodological template rather than a task-specific module. Its recurring elements are asymmetric query roles, explicit enhancement signals, and iterative refinement.

## 2. D-Align as a dual-query enhancement network for multi-frame LiDAR

D-Align is a 3D object detector for point cloud sequences with three blocks: a **BEV feature extractor**, **DUCANet** (dual-query co-attention network), and a **3D detection head** [2210.00087]. Given a LiDAR sequence $\{P_n\}_{n=t-N+1}^{t}$, frame $t$ is the target frame and earlier frames are support frames. Each frame is encoded into multi-scale BEV feature maps,
$$
F_n=\{F_n^s\}_{s=1}^{S}.
$$

DUCANet maintains a target query set $B_t^{(l)}$ and support query sets $\{U_{t-k}^{(l)}\}_{k=1}^{N-1}$, where $l$ indexes attention layers. Initialization is
$$
B_t^{(0)} = F_t,\qquad U_{t-k}^{(0)} = F_{t-k}.
$$
At each layer, IDANet aligns each support query to the target query, IGANet fuses aligned support and target queries with gating, and the refined queries are passed to the next layer. After $L$ layers, the final target query
$$
\hat{F}_t = \{ \hat{F}_t^s \}_{s=1}^{S}
$$
is used as the enhanced BEV representation for detection.

The defining asymmetry is important. T-QS and S-QS are not used symmetrically in the final objective: support queries are aligned to the target and then fused into the target, so the target frame representation is progressively strengthened while support frames serve as auxiliary temporal evidence. That asymmetry is the basis for interpreting D-Align as a DQEN rather than as a generic temporal fusion module.

The paper’s stated motivation is that conventional detectors use points acquired over a fixed duration, whereas spatio-temporal information from point cloud sequences can further enhance performance only if temporal discrepancy is addressed explicitly. Naive stacking or recurrence is therefore insufficient in the sense described by the paper, because moving objects produce changing point distributions even after ego-motion compensation [2210.00087].

## 3. Motion-aware alignment, gated aggregation, and iterative refinement in D-Align

A distinctive component of D-Align is the use of **temporal context features** to guide alignment. The feature pyramid motion extraction network, **FPMNet**, computes motion cues from the current target query and a support query at each scale:
$$
M_{t-k}^{(l),s} = \mathrm{conv}_{3\times3}(B_t^{(l),s} - U_{t-k}^{(l),s}).
$$
These are transformed across pyramid levels and concatenated as
$$
\tilde{M}_{t-k}^{(l),s'} = \mathrm{conv}_{1\times1}\left([M_{t-k}^{(l),1s'}, M_{t-k}^{(l),2s'}, \ldots, M_{t-k}^{(l),Ss'}]\right),
$$
yielding the temporal context feature pyramid $\tilde{M}_{t-k}^{(l)}=\{\tilde{M}_{t-k}^{(l),s'}\}_{s'=1}^{S}$ [2210.00087].

In **IDANet**, the support query set is aligned to the target query set using deformable attention. For head $h$, scale $s$, and reference location $p_s$, the projected support embedding is
$$
V_{t-k}^{sh}(p_s) = W_h\, U_{t-k}^{(l),s}(p_s),
$$
with $W_h \in \mathbb{R}^{D' \times D}$ and $D'=D/H$. Offset masks and attention weights are predicted from the motion context:
$$
\Delta_{t-k}^{sh}(p_s) = W'_h\, \tilde{M}_{t-k}^{(l),s}(p_s),
$$
$$
A_{t-k}^{sh}(p_s) = \mathrm{softmax}(W''_h\, \tilde{M}_{t-k}^{(l),s}(p_s)).
$$
The attention output is
$$
z^s_{t-k}(p_s)=\sum_{h=1}^{H}W'''_{h} \cdot \left(\sum_{i=1}^{S}\sum_{j=1}^{J}A_{t-k}^{shij}(p_s)\cdot V^{ih}_{t-k}\left(\psi_i(p_s)+\Delta_{t-k}^{shij}(p_s)\right)\right),
$$
and the refined support query is updated by
$$
U_{t-k}^{(l+1),s}(p_s) = \mathrm{FFN}\left(\mathrm{LN}\left(\mathrm{dropout}(z^s_{t-k}(p_s)) + U_{t-k}^{(l),s}(p_s)\right)\right).
$$

In **IGANet**, aligned support queries are fused with the target query by a gated attention mechanism:
$$
\tilde{B}_{t-k}^{(l),s} = G_{t-k}^{(l),s}\otimes B_t^{(l),s} + \left(1-G_{t-k}^{(l),s}\right)\otimes U_{t-k}^{(l+1),s},
$$
with gate
$$
G_{t-k}^{(l),s} = \sigma\left(\mathrm{conv}_{3\times3}([B_t^{(l),s}, U_{t-k}^{(l+1),s}])\right).
$$
The fused outputs across support frames are then aggregated:
$$
B_t^{(l+1),s} = \mathrm{conv}_{3\times3}\left([\tilde{B}_{t-1}^{(l),s},\ldots,\tilde{B}_{t-N+1}^{(l),s}]\right).
$$

The model is explicitly iterative. DUCANet stacks $L$ attention layers, each implementing
$$
\{U_{t-k}^{(l)}, B_t^{(l)}\} \rightarrow \{U_{t-k}^{(l+1)}, B_t^{(l+1)}\}.
$$
On nuScenes validation, the ablation reported for the PointPillars baseline is **44.40 mAP / 58.15 NDS**; adding **IGANet only** gives **46.17 / 59.42**; adding **IGANet + IDANet (no FPMNet)** gives **48.19 / 60.60**; and the full D-Align with **FPMNet** reaches **49.66 / 61.33**. Increasing the number of attention layers from 1 to 2 to 3 improves performance from **46.44 / 59.55** to **48.58 / 60.72** to **49.66 / 61.33**. The paper therefore attributes the gain not merely to multi-frame aggregation, but to alignment and motion-guided query refinement [2210.00087].

## 4. DQEN for DETR-based human-object interaction detection

The DQEN model for HOI detection is built on a **single encoder + dual decoder** architecture [2508.18896]. A CNN backbone extracts a feature map $V_f \in \mathbb{R}^{(H \times W)\times C'}$ from input image $I$; with positional encoding $P_e$, a Transformer encoder produces visual memory
$$
V_e \in \mathbb{R}^{(H \times W)\times C'}.
$$
This encoder output is shared by an **instance decoder** and an **interaction decoder**.

The instance decoder takes human queries
$$
Q_h \in \mathbb{R}^{N_q \times C'}
$$
initialized randomly and enhanced object queries
$$
\hat{Q}_o \in \mathbb{R}^{N_q \times C'}.
$$
It outputs human features
$$
V_h \in \mathbb{R}^{N_q \times C'}
$$
and object features
$$
V_o \in \mathbb{R}^{N_q \times C'}.
$$
The interaction stream begins from a semantic interaction feature
$$
Q_i \in \mathbb{R}^{1 \times C'}
$$
that is repeated $N_q$ times to form
$$
Q_r \in \mathbb{R}^{N_q \times C'}.
$$
This repeated semantic feature is combined with $V_h$ and $V_o$ to form interaction queries $Q_{inter}$, and the interaction decoder outputs
$$
V_{inter} \in \mathbb{R}^{N_q \times C'}.
$$
A skip connection from $Q_r$ to $V_{inter}$ is used for final interaction or verb prediction.

The first enhancement branch is **Object Query Enhancement (OQE)**. Encoder output $V_e$ is passed through a linear classifier to produce object category scores, and the **Top-$N$** highest-scoring encoder features are selected:
$$
V_a \in \mathbb{R}^{N \times C'}.
$$
Object queries are then enhanced by
$$
\hat{Q}_o = Q_o + V_a,
$$
where $N=N_q$. A crucial training detail is that $V_e$ is duplicated as classifier input and **gradient backpropagation is disabled** through this classifier path at this stage. The ablation reported in the paper states that allowing direct backpropagation through this path lowers performance. The authors further note that OQE does **not** improve object detection mAP: the baseline GEN-VLKT object mAP is **34.86**, “Ours w/o OQE” is **33.01**, and “Ours w/ OQE” is **32.72**. The interpretation offered in the paper is that the selected object-like features often emphasize the **person-object interaction region**, which helps HOI detection even when pure object detection accuracy decreases [2508.18896].

## 5. Interaction semantic enhancement, auxiliary prediction, and scoring in HOI DQEN

The second branch, **Interaction Query Enhancement (IQE)**, is the main contribution in the HOI paper [2508.18896]. Each HOI triplet $\langle human, verb, object \rangle$ is converted to a prompt of the form “A photo of a person [verb-ing] a/an [object]”, creating a text set $\mathcal{T}$. A CLIP image encoder and text encoder then produce
$$
V_c = ImgEnc(I), \qquad T_c = TextEnc(\mathcal{T}),
$$
with $V_c \in \mathbb{R}^{D}$ and $T_c \in \mathbb{R}^{D \times N_{hoi}}$. Similarity is computed as
$$
m_{sim} = T_c^T V_c,
$$
and top-$K$ matching HOI labels are selected:
$$
\mathcal{T}_{can} = \text{Top}K(\mathcal{T}, m_{sim}).
$$
The paper states that CLIP Top-8 matched labels can cover over **50%** of HICO-Det ground-truth HOIs.

The model does not directly use CLIP text features as interaction queries. Instead, it introduces **Interaction Semantic Fusion (ISF)** to disentangle and fuse verb and object semantics. From $\mathcal{T}_{can}$, verb words, object words, and full HOI triplets are encoded as
$$
T_{verb} \in \mathbb{R}^{K \times C_k},\quad T_{obj} \in \mathbb{R}^{K \times C_k},\quad T_{hoi} \in \mathbb{R}^{K \times C_k}.
$$
Verb and object words are fused by
$$
T_{vo} = \text{MLP}(\text{Concat}(T_{verb}, T_{obj})),
$$
and HOI triplet embeddings are projected as
$$
\hat{T}_{hoi} = \text{MLP}(T_{hoi}).
$$
A correlation matrix is computed:
$$
C = (w_a \odot T_{vo}) \otimes \hat{T}_{hoi},
$$
where $w_a$ is learnable self-attention weights. The fused semantics are reweighted by
$$
\hat{T}_{vo} = softmax(C) \cdot T_{vo},
$$
and the final semantic fusion is
$$
F = \hat{T}_{vo} \odot \hat{T}_{hoi}.
$$
Summing over candidate dimension and projecting yields the semantic interaction feature:
$$
Q_i = \text{MLP}\left(\sum_{j=1}^{K} F(j,:)\right).
$$

The paper also introduces an **Auxiliary Prediction Unit (APU)**. An additional verb classifier $CLS_{verb}$ takes
$$
Q_r + V_{inter}
$$
as input, producing $S_{verb}$. These scores are combined with the interaction prediction $S_{inter}$ from $CLS_{inter}$:
$$
S_{hoi} = S_{inter} + \alpha S_{verb}.
$$
The training objective includes the DETR-style matching cost,
$$
\mathcal{L}_{c}= \sum_{i \in (h,o)} \left(\lambda_{b}\mathcal{L}_{b}^{(i)}+\lambda_{u}\mathcal{L}_{u}^{(i)}\right)+\sum_{j \in (o,a)} \lambda_{c}^{(j)} \mathcal{L}_{c}^{(j)},
$$
together with cross-entropy loss $\mathcal{L}_{ce}$ for the object classifier and CLIP knowledge distillation loss $\mathcal{L}_{kd}$, giving
$$
\mathcal{L}_{total} = \mathcal{L}_{cost} + \mathcal{L}_{ce} + \lambda_{kd}\mathcal{L}_{kd}.
$$
The paper specifies $\lambda_{kd}=20$, $\lambda_b=2.5$, $\lambda_u=1$, $\lambda_c^o=1$, and $\lambda_c^a=1$.

At inference, a **training-free CLIP inference option** is used. From $m_{sim}$, the top-$R$ normalized similarities are selected as
$$
S_{tf} = \{ x \in m_{sim} \mid \text{Rank}(x) \leq R \},
$$
with $R=10$. For HOI category $n$, the final triplet score is
$$
S^{(n)} = S_{hoi}^{(n)} + S_o^{(m)} \odot S_o^{(m)} + S_{tf}^{(n)},
$$
followed by triplet NMS.

## 6. Training regimes, empirical behavior, and interpretive significance

The two DQEN instantiations differ substantially in optimization and evaluation protocol. D-Align does **not** introduce a new detection loss; it is trained with the same losses as its base detector, specifically PointPillars loss for the anchor-based version and CenterPoint loss for the anchor-free version. Training uses the Adam optimizer, pretraining of the single-frame detector, and then fine-tuning of the full D-Align network. The schedule is **20 epochs** of pretraining with max learning rate **0.001**, followed by **40 epochs** of fine-tuning with max learning rate **0.0002**, with random flipping, rotation, scaling, and ground-truth box sampling as data augmentation [2210.00087].

On **nuScenes**, D-Align is evaluated with sequence length $N=3$ by default, using 1 target frame and 2 support frames; each frame uses 10 consecutive LiDAR samples over 0.5 s; DUCANet uses $L=3$ attention layers; deformable attention uses $H=8$ heads and $J=4$ sampling points. The test-set result highlighted for **D-Align-CP** is **70.2 NDS** and **64.0 mAP**, with the paper further reporting a **22.5% gain over PointPillars** and a **3.7% mAP gain over CenterPoint**. Gains are described as especially strong on motorcycle, bicycle, and traffic cone [2210.00087].

The HOI DQEN uses **ResNet-50**, **ViT-B/32** for CLIP, **3 + 3** encoder/decoder layers, **64** queries, feature dimension **256**, CLIP embedding dimension **512**, word embedding dimension **512**, **AdamW**, and **90** epochs with learning rate $10^{-4}$ reduced after 60 epochs. The paper reports about **48M parameters** and about **29.02 FPS** [2508.18896].

On **HICO-Det**, the main result is **35.23** Full, **32.79** Rare, and **35.96** Non-Rare for DQEN without training-free CLIP, and **35.34**, **33.10**, and **36.01** with training-free CLIP. The paper states gains over GEN-VLKT of **+1.48 mAP Full**, **+3.54 mAP Rare**, and **+0.86 mAP Non-Rare** in the default setting, and **+1.72 / +3.67 / +1.13** in the known-object setting. On **V-COCO**, DQEN reports $AP_{role}^{S_1} = 60.88$ and $AP_{role}^{S_2} = 63.47$, while also noting that improvement is smaller than on HICO-Det because V-COCO has more human-object pairs per image, making candidate selection and semantic enhancement less precise. The key HICO-Det default ablation gives **32.68** for baseline, **33.72** for +APU, **33.28** for +OQE, **34.22** for +IQE, **33.50** for +APU+OQE, **34.47** for +APU+IQE, **34.26** for +OQE+IQE, and **35.23** for all three modules [2508.18896].

A common misconception would be to treat “dual query” as meaning simple duplication of decoder tokens. The two papers indicate a stricter meaning: in D-Align, the distinction is between target and support frame representations, with alignment preceding fusion; in HOI DQEN, the distinction is between object-aware and interaction-aware query enhancement, with visual priors and CLIP-derived semantic priors serving different roles. Another possible misconception is that DQEN implies symmetric information exchange. The available evidence points in the opposite direction: D-Align uses support frames as auxiliary evidence for target enhancement, whereas HOI DQEN enhances separate downstream streams for instance localization and interaction recognition. This suggests that the unifying concept is not symmetry, but controlled asymmetry in query design.

Taken together, these works present DQEN as a response to two failure modes of standard architectures: unmodeled temporal misalignment in multi-frame BEV detection and vague query semantics in DETR-based HOI detection. In both cases, performance gains are tied to explicit enhancement signals rather than to additional frames or additional queries alone.

Source: https://www.emergentmind.com/topics/dual-query-enhancement-network-dqen