DS-Det: Disentangled Single-Query Detector

• The paper introduces DS-Det, a transformer-based detector that separates query roles to address query ambiguity and recurrent opposing interactions.
• It employs a flexible single-query paradigm (FLET) to generate variable query counts from encoder features, enhancing adaptability per image.
• A two-stage attention-disentangled decoder, combined with SGQ and PoCoo loss, leads to improved AP and efficiency on benchmarks like COCO2017 and WiderPerson.

Searching arXiv for the DS-Det paper and closely related detector papers to anchor citations. DS-Det is a transformer-based object detector that reformulates decoder querying and decoder supervision around a “Single-Query” paradigm and an attention-disentangled decoder design. It replaces the fixed set of learnable decoder queries used in DETR-like detectors with Single-Queries generated from encoder features, enabling a flexible number of predictions per image. It further separates localization and deduplication into distinct decoder stages: a Box Locating Part trained with one-to-many assignment and Cross-Attention only, and a Deduplication Part trained with one-to-one assignment and Self-Attention only. In this formulation, DS-Det is presented as a response to two identified issues in prior query-based detectors—“Recurrent Opposing inTeractions” and “query ambiguity”—while retaining end-to-end set prediction and improving performance on COCO2017 and WiderPerson (Cao et al., 26 Jul 2025).

1. Definition and problem setting

DS-Det, expanded in the source material as “Disentangled Single-query Detection,” belongs to the DETR lineage of transformer detectors, in which object instances are predicted from decoder queries through attention-based decoding rather than anchor enumeration or non-maximum suppression pipelines. The immediate point of departure is the observation that existing decoder query types, including content query and positional query, are generally predefined with a fixed number, so their flexibility is limited (Cao et al., 26 Jul 2025).

Within that setting, DS-Det identifies two structural problems. The first is “Recurrent Opposing inTeractions” (ROT), described as an adverse interaction between Self-Attention and Cross-Attention inside conventional decoder stacks. In the formulation given for DS-Det, Cross-Attention tends to gather multiple queries around the same object, while Self-Attention tends to disperse queries. Repeating SA and CA layer by layer therefore induces an oscillatory dynamic that degrades decoder efficiency (Cao et al., 26 Jul 2025). The second is “query ambiguity,” which arises when shared-weight decoder layers are trained under both one-to-one and one-to-many label assignments, thereby violating DETR’s one-to-one matching principle (Cao et al., 26 Jul 2025). This framing situates DS-Det against fixed-query DETR-style designs such as DETR (Carion et al., 2020), Deformable DETR (Zhu et al., 2020), and DINO (Zhang et al., 2022).

A plausible implication is that DS-Det should be understood less as a minor decoder variant than as a re-specification of what a decoder query is supposed to represent. In standard DETR-like practice, a query is both a computational token and a supervisory unit. DS-Det separates these roles by redefining the query source and by assigning distinct training semantics to distinct decoder stages (Cao et al., 26 Jul 2025).

2. Single-Query paradigm and flexible query generation

The first core component of DS-Det is the Single-Query paradigm implemented by the Flexible singLe-quEry generaTion module, abbreviated FLET. Instead of maintaining a fixed set of learnable positional and content queries, FLET derives queries directly from encoder tokens. If the encoder output is written as $$\mathbf{E}=\{e_1,\dots,e_L\}\in\mathbb{R}^{L\times C}$$, then a classification head produces a foreground score for each token, and thresholding with score $S$ selects the tokens that become decoder queries (Cao et al., 26 Jul 2025).

This selected set is written in the source material as

$\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$

These vectors are the Single-Queries. Because they are encoder features associated with image locations, they implicitly carry both semantic and positional information, which is why DS-Det argues that fixed learned positional queries are not crucial (Cao et al., 26 Jul 2025).

Two additional mechanisms accompany this query generation. The first is Query Alignment across the batch. If per-image query counts differ, the batch query number is set to the maximum count in the batch, and images with fewer selected queries are padded using low-scoring encoder tokens so that tensor shapes remain aligned for batched processing (Cao et al., 26 Jul 2025). The second is Query Sampling, used especially in early training when token scores are noisy and many redundant queries would otherwise be selected (Cao et al., 26 Jul 2025).

The consequence is a detector whose number of predictions varies with image complexity rather than remaining fixed at values such as 300 or 900. For COCO val2017 images with 1–5 objects, DS-Det uses on average only about 65 queries, described in the source as 7.25% of DINO’s 900, yet achieves a reported gain of +0.9 AP (Cao et al., 26 Jul 2025). This suggests that flexible query cardinality is not merely a computational convenience but part of the representational claim of the method: simple images need not be decoded with dense query budgets, while dense images can receive more queries without discretized query-count bins.

3. Attention-disentangled decoder

The second core component is the Attention Disentangled Decoder, abbreviated ADD. Its central design principle is that Cross-Attention and Self-Attention should not be interleaved under conflicting supervisory regimes. Instead, DS-Det decomposes decoding into two stages with different objectives and different label assignments (Cao et al., 26 Jul 2025).

The first stage is the Box Locating Part, or BLP. BLP uses Cross-Attention only and is trained with one-to-many assignment. Each ground-truth box is replicated $K$ times, and each replica is matched to a distinct query, so multiple queries are explicitly permitted to localize the same object (Cao et al., 26 Jul 2025). The BLP layer is described in the source by

$\begin{split} \text{BLP}(q_d, E) = &\ \text{LN}\left\{\text{MLP}\left(\text{LN}(q_d + \text{CA}(q_d, E))\right) + \right. \ &\quad\left. \text{LN}(q_d + \text{CA}(q_d, E)) \right\}. \end{split}$

In this stage, the omission of Self-Attention is deliberate: CA is allowed to gather multiple candidate queries around objects without being counteracted by SA-driven query dispersion (Cao et al., 26 Jul 2025).

The second stage is the Deduplication Part, or DP. DP is trained with one-to-one assignment and uses Self-Attention to suppress duplicates and restore DETR-style uniqueness. It contains a CA refinement step and then a stack of Multi-head Self-Attention Blocks. In the source formulation,

$$\text{DP}_{\text{CA}}(q_d, E) = \text{LN}(q_d + \text{CA}(q_d, E)),$$

followed by iterative SA-based refinement

$$\begin{split} q_d^{m} = \text{MLP}^m\bigg( \text{LN}^m\big( q_d^{m-1} + \text{SA}^m(q_d^{m-1}) \big)\bigg). \end{split}$$

The output features are then used for final box and class prediction (Cao et al., 26 Jul 2025).

This staging is the explicit mechanism through which DS-Det claims to break both ROT and query ambiguity. In BLP, CA is paired with one-to-many assignment; in DP, SA is paired with one-to-one assignment. The same query state is therefore not simultaneously trained to behave as both a redundant candidate and a unique object representative (Cao et al., 26 Jul 2025). Relative to prior DETR variants that retain interleaved SA/CA or mix one-to-one and one-to-many supervision on shared decoder weights, this is the distinctive architectural thesis of DS-Det (Cao et al., 26 Jul 2025).

4. Training formulation, SGQ, and PoCoo loss

DS-Det complements ADD with two training devices: Stopping Gradient of Queries (SGQ) and the PoCoo classification loss. SGQ prevents gradients from the Deduplication Part from propagating back through the BLP’s query input. The stated purpose is to ensure that BLP is optimized purely for one-to-many localization while DP is optimized purely for one-to-one deduplication (Cao et al., 26 Jul 2025). In the reported ablation, removing SGQ yields AP 48.5, while enabling SGQ gives AP 50.5, a gain of +2.0 AP (Cao et al., 26 Jul 2025).

The PoCoo loss—expanded in the source as “incorPOrates Classification with IOU and BOx size”—modifies classification supervision by combining IoU-aware targets with a box-size prior that emphasizes hard samples such as small objects. Its definition is given as

$\begin{split} \text{PoCoo} = &\ \sum_{i}^{N_{\text{pos}}} \text{BCE}(p_i, t_i)\; \times \left[ \left( 1 - \sqrt{\frac{h_i}{H} \frac{w_i}{W}} \right)^\alpha + 1 \right] \ & + \sum_{j}^{N_{\text{neg}}} p_j^{2} \, \text{BCE}(p_j, 0). \end{split}$

Here $p_i$ is the predicted classification score, $t_i$ is the target label incorporating IoU information, $S$0 are matched ground-truth box dimensions, and $S$1 are image dimensions (Cao et al., 26 Jul 2025). Because the multiplicative size-prior factor is larger for small normalized areas, PoCoo increases the classification weight of small objects.

The reported ablation compares standard BCE, IA-BCE, and PoCoo. BCE yields AP 49.0 and AP$S$2 32.6; IA-BCE yields AP 50.1 and AP$S$3 33.1; PoCoo yields AP 50.5 and AP$S$4 34.3 (Cao et al., 26 Jul 2025). This establishes PoCoo as a targeted small-object training refinement rather than a generic replacement loss.

The training configuration also specifies distinct matching costs for BLP and DP. In BLP, the best configuration in the provided details uses BCE weight 0.2, L1 weight 5.0, and GIoU weight 2.0, with one-to-many replication factor $S$5 (Cao et al., 26 Jul 2025). In DP, the best reported configuration uses BCE weight 2.0, L1 5.0, and GIoU 2.0 under one-to-one Hungarian matching (Cao et al., 26 Jul 2025). This asymmetry in matching weights reinforces the larger design principle that the two decoder parts have different supervisory semantics.

5. Empirical performance and efficiency

The experimental evaluation reported for DS-Det spans COCO2017 and WiderPerson and uses five different backbone models: ResNet-50, Strip-MLP-T, Swin-T, Swin-L, and VMamba-T (Cao et al., 26 Jul 2025). The comparison baseline throughout is DINO, a strong DETR-family detector (Zhang et al., 2022).

On COCO2017 with ResNet-50 and a 12-epoch schedule, DINO reports AP 49.0, AP$S$6 32.0, AP$S$7 52.3, and AP$S$8 63.0, whereas DS-Det reports AP 50.5, AP$S$9 34.3, AP$\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$0 54.6, and AP$\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$1 64.5 (Cao et al., 26 Jul 2025). Under a longer comparison, DINO at 36 epochs reports AP 50.9 and AP$\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$2 34.6, while DS-Det at 24 epochs reports AP 51.3 and AP$\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$3 35.5, which the source interprets as evidence of faster convergence as well as improved small-object performance (Cao et al., 26 Jul 2025).

Across additional backbones, the reported 12-epoch gains remain positive. With Strip-MLP-T, DINO reports AP 51.7 and DS-Det 52.8; with Swin-T, DINO reports 51.3 and DS-Det 52.7; with Swin-L, DINO reports 56.8 and DS-Det 57.7; with VMamba-T, DINO reports 53.5 and DS-Det 54.4 (Cao et al., 26 Jul 2025). The source further compares DS-Det to DiffusionDet (Chen et al., 2022) in the context of flexible detectors: DiffusionDet with ResNet-50 at 60 epochs reports AP 46.8 and AP$\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$4 29.6, while DS-Det with ResNet-50 at 24 epochs reports AP 51.3 and AP$\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$5 35.5 (Cao et al., 26 Jul 2025).

On WiderPerson, particularly the “Hard” subset, DS-Det is reported to improve over DINO for all listed backbones. For example, with ResNet-50, DS-Det reports AP 93.24, Recall 99.57, and mMR 39.47, compared with DINO’s AP 92.75, Recall 99.08, and mMR 40.08 (Cao et al., 26 Jul 2025). The source notes especially strong behavior on dense images, including analyses for images with at least 30 persons (Cao et al., 26 Jul 2025).

Efficiency is reported both as FLOPs and decoder inference time. For ResNet-50 at 1280×800 with 900 queries for fairness, DINO is reported at 279G FLOPs and 47M parameters, while DS-Det is reported at 275G FLOPs and 48M parameters (Cao et al., 26 Jul 2025). More notably, decoder inference time is reported as 14.1 ms for DINO and 9.2 ms for DS-Det, described as a 34.8% faster decoder (Cao et al., 26 Jul 2025). Since DS-Det often uses fewer than 900 queries in actual operation, the source further suggests that practical FLOPs can be lower than this controlled comparison indicates (Cao et al., 26 Jul 2025).

6. Relation to prior detector families and interpretive significance

DS-Det is closely connected to the DETR family, but it reassigns the conceptual roles of query generation, attention, and assignment. Relative to DETR (Carion et al., 2020), Deformable DETR (Zhu et al., 2020), and DINO (Zhang et al., 2022), the immediate contrast is the abandonment of fixed learned queries in favor of encoder-derived Single-Queries (Cao et al., 26 Jul 2025). Relative to one-to-many extensions such as Hybrid-DETR, Co-DETR, or DAC-DETR, the crucial distinction in the source is that DS-Det does not merely add one-to-many supervision but disentangles one-to-many and one-to-one supervision into different decoder stages so that shared decoder weights are not simultaneously updated under contradictory query semantics (Cao et al., 26 Jul 2025).

The source also positions DS-Det against DQ-DETR, which classifies the required number of queries into discrete bins, and against DiffusionDet, which is flexible in a denoising-diffusion sense but remains architecturally and operationally different from direct set prediction (Cao et al., 26 Jul 2025). In this sense, DS-Det’s flexibility is not a matter of selecting from a small set of predefined query counts, nor of iterative denoising, but of thresholding encoder tokens into a variable-cardinality query set.

A common misconception would be to interpret DS-Det as only a query-pruning method. The source material supports a broader interpretation. FLET reduces fixed-query dependence, but the central contribution also includes ADD, SGQ, and PoCoo, which together reframe how localization, deduplication, and small-object emphasis are distributed across the detector (Cao et al., 26 Jul 2025). Another misconception would be to treat the architecture as merely removing Self-Attention to save computation. The ablation data argue against that simplification: a naïve flexible-query variant without the full DS-Det formulation drops to AP 44.1, whereas adding FLET, PoCoo, and ADD raises performance to AP 50.5 (Cao et al., 26 Jul 2025).

This suggests that the main significance of DS-Det lies in a staged decomposition of decoder behavior. Cross-Attention is assigned the role of localization under deliberate redundancy; Self-Attention is assigned the role of deduplication under uniqueness constraints. That division provides a principled answer to the dual pathologies—ROT and query ambiguity—identified by the method (Cao et al., 26 Jul 2025).

7. Ablations, limitations, and prospective directions

The component ablations reported in the source clarify which parts of DS-Det are essential. Starting from DINO-ResNet-50 at AP 49.0, removing Self-Attention and using a naïve flexible-query formulation reduces AP to 44.1. Adding FLET recovers performance to 48.7, adding PoCoo increases it to 49.3, and adding ADD yields 50.5 (Cao et al., 26 Jul 2025). This sequence indicates that flexibility alone is insufficient; the gains are tied to the coupled redesign of query generation, decoder staging, and loss.

Decoder depth allocation is also ablated. The best configuration is reported as 4 BLP layers plus 2 DP layers, while too few DP layers or too many BLP layers reduce AP (Cao et al., 26 Jul 2025). The number of Self-Attention blocks inside DP is likewise critical: $\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$6 yields AP 34.7, $\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$7 yields 50.1, and $\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$8 yields 50.5, with $\mathcal{Q}=\{q_i\mid q_i=e_k,\; s_k\ge S\}.$9 providing similar AP at higher cost (Cao et al., 26 Jul 2025). These numbers reinforce that DS-Det does not remove SA from the decoder altogether; it relocates SA to the stage where deduplication is the explicit objective.

The source also reports that reintroducing positional queries into ADD does not improve DS-Det, and the PQ-free Single-Query variant remains the best reported configuration (Cao et al., 26 Jul 2025). This is important because it narrows the claim: the method is not merely robust to PQ ablation in one setting, but is explicitly designed around the idea that encoder-derived Single-Queries already encode sufficient position-content coupling.

The limitations discussed in the source are more implicit than formal. DS-Det still inherits transformer complexity, and the backbone plus encoder remain the dominant runtime components (Cao et al., 26 Jul 2025). FLET introduces a threshold hyperparameter $K$0, which governs the accuracy–FLOP tradeoff (Cao et al., 26 Jul 2025). The method is also described as a basis for future combinations with sparse encoders or YOLO-like backbones, suggesting that its current form is one point within a broader design space rather than an endpoint (Cao et al., 26 Jul 2025).

A plausible implication is that DS-Det’s broader relevance lies in its separation of query selection, localization redundancy, and uniqueness enforcement. Those ideas are not obviously restricted to the exact DINO-style stack used in the paper. The source itself suggests future combinations with other detector systems, which indicates that “Single-Query” and “attention disentangled learning” may be interpreted as transferable principles rather than as a closed architecture (Cao et al., 26 Jul 2025).