TRACE: Backdoor Detection in Object Detection

• TRACE is a test-time backdoor detection method that evaluates transformation consistency in object detection outputs under foreground and background perturbations.
• It measures variance in detection confidence to distinguish between clean and trigger-activated inputs without accessing model internals or training data.
• Experimental results show substantial improvements in F1 and AUROC across YOLO, Faster-RCNN, and DETR, supporting its robust, universal defense approach.

TRACE, in the context of object-detection security, denotes TRAnsformation Consistency Evaluation, a test-time backdoor detection method for object detection models. It is designed to identify poisoned test samples—inputs containing triggers that activate a backdoor—without requiring model internals, gradients, or training data. The method was introduced to address the mismatch between existing backdoor defenses and the distinctive structure of object detection, where models output variable sets of objects and are vulnerable to attack effects such as ghost object emergence and vanishing object behavior. TRACE operates by applying foreground and background transformations to a test image and measuring how detection confidences vary under those transformations, yielding a combined anomaly score for black-box, universal backdoor detection (Zhang et al., 19 Mar 2025).

1. Problem setting and motivation

Backdoor attacks on object detectors poison a small subset of training samples by embedding a predefined trigger so that, at inference time, the model exhibits attacker-chosen behavior when the trigger reappears. In object detection, those behaviors include not only false positive insertion but also false negative induction, such as causing expected objects to vanish. The threat model addressed by TRACE is specifically test-time detection of poisoned samples, so that trigger-bearing inputs can be intercepted before the backdoor is activated (Zhang et al., 19 Mar 2025).

The object-detection setting introduces complications that are absent in standard image classification. A detector outputs a variable number of objects rather than a single class label, must jointly predict category and localization, and is often implemented through a dual-branch design combining classification and regression. The data further states that existing defenses, especially those developed for classification, transfer poorly because they either require white-box access or cannot handle the complexity of detection outputs. This motivates a defense that is simultaneously black-box, attack-agnostic, universal, and suitable for online use.

A central premise of TRACE is that poisoned and clean detector inputs respond differently to semantically meaningful transformations. The method therefore does not attempt to reverse-engineer the trigger itself. Instead, it treats transformation-induced stability or instability in detector confidences as an observable signature of poisoning. This suggests a shift from trigger-pattern recognition to behavioral consistency analysis under controlled input perturbations.

2. Foundational observations

TRACE is built on two empirical observations about backdoored object detectors. The first is contextual transformation consistency: when objects, including possible triggers, are placed over diverse backgrounds, clean objects show significant variation in detection confidences, whereas trigger-induced or poisoned objects show much less variance and remain confidently detected across contexts (Zhang et al., 19 Mar 2025).

The data attributes this effect to shortcut learning. Backdoor training creates a strong, context-invariant pattern-to-label mapping, while benign detection depends more strongly on scene context. TRACE exploits that asymmetry by examining whether confidence remains anomalously stable under background changes. In this framework, unusually low variance is treated as suspicious for trigger-induced detections.

The second observation is focal transformation consistency. If a small, well-recognized foreground patch—especially a natural backdoor object such as a traffic sign—is inserted at different positions in a test image, clean images tend to preserve stable detection confidence for that patch as it moves around. By contrast, poisoned images associated with vanishing or false-negative-inducing triggers exhibit a marked drop in detection confidence when the patch overlaps the trigger region. The paper characterizes this phenomenon as the “island effect”, namely an abrupt hole in an otherwise uniform confidence surface generated by scanning the patch.

These two observations are complementary. Contextual consistency is most informative when some object remains visible and can be tracked across transformed backgrounds. Focal consistency addresses the harder case in which the trigger suppresses an object so that there is little or nothing left for direct contextual analysis. This division of labor is structurally important: TRACE is not a single statistic but a paired analysis of background-induced and foreground-induced response variation.

3. Formal methodology

TRACE combines background and foreground transformations into two statistical metrics: Contextual Transformation Consistency (CTC) and Focal Transformation Consistency (FTC) (Zhang et al., 19 Mar 2025).

For an object $\mathbf{o}$ and background distribution $\mathcal{B}$, CTC is defined as the maximum deviation in detection confidence for the object’s class $\mathbf{y}$ across pairs of backgrounds:

$$\Delta_{\mathcal{B}}(\mathbb{F}_\theta, \mathbf{o}) := \underset{\delta_1, \delta_2 \sim \mathcal{B}}{\mathrm{max}} \left|\mathbb{F}_\theta(\mathbf{o} \oplus \delta_1)_{\mathbf{y}(\mathbf{o})} - \mathbb{F}_\theta(\mathbf{o} \oplus \delta_2)_{\mathbf{y}(\mathbf{o})}\right|.$$

In practice, the method uses a variance-based estimator rather than the maximum difference:

$$\Delta_{\mathcal{B}}^{\text{Var}}(\mathbb{F}_\theta, \mathbf{o}) = \text{Var}\left(\{ \mathbb{F}_\theta(\mathbf{o} \oplus \delta)_{\mathbf{y}(\mathbf{o})} \}_{\delta\sim \mathcal{U}(\mathcal{B})} \right).$$

Because images may contain multiple objects and possibly several triggers, TRACE defines image-level CTC as the minimum object-level CTC among objects passing an SSIM-based natural backdoor filter:

$$\Delta_{\mathcal{B}}^{\text{Var}}(\mathbb{F}_\theta, \mathbf{x}) = \min_{i} \left\{ \Delta_{\mathcal{B}}^{\text{Var}}(\mathbb{F}_\theta, \mathbf{o}_i) \,\bigg|\, \text{SSIM}(\mathbf{o}_i, \mathbf{o}^{\text{ref}}_{y_i}) \leq \tau \right\}.$$

Here $\mathbf{o}^{\text{ref}}_{y_i}$ is a class-matched reference “benchmark” object from public images, and $\tau$ is an SSIM threshold. The purpose of the filter is to reduce false alarms caused by natural backdoor objects, such as traffic signs, that may themselves exhibit unusually stable detection behavior.

FTC is designed for the invisible trigger case. Given an image $\mathbf{x}$, a patch $\mathbf{o}$, and a spatial domain $\mathcal{B}$0, the method constructs transformed inputs $\mathcal{B}$1 by inserting the patch at position $\mathcal{B}$2. FTC is defined as

$\mathcal{B}$3

The Laplacian term captures sharp local fluctuations associated with the island effect, while the second term captures “on-off” events as the patch covers or uncovers a trigger region. In practical estimation, TRACE avoids dense scanning and instead samples $\mathcal{B}$4 random non-overlapping positions repeated $\mathcal{B}$5 times, then computes

$\mathcal{B}$6

The final anomaly score is

$\mathcal{B}$7

where $\mathcal{B}$8. With threshold $\mathcal{B}$9,

$\mathbf{y}$0

Operationally, low CTC variance indicates likely trigger presence, whereas high FTC variance indicates likely vanishing-trigger behavior.

4. Inference procedure and implementation logic

TRACE is applied per test image through a three-stage procedure. First, the method performs contextual transformation. It prepares $\mathbf{y}$1 backgrounds, which may be public or natural images, blends each background with the test image using small opacity such as $\mathbf{y}$2, queries the detector on each blended image, and records object confidence scores. For each detected object, it computes the variance of those confidences, applies the SSIM filter, and takes the minimum remaining variance as the image-level CTC (Zhang et al., 19 Mar 2025).

Second, TRACE performs focal transformation. It chooses a robust natural backdoor object patch, samples $\mathbf{y}$3 random non-overlapping positions over $\mathbf{y}$4 repetitions, pastes the patch into the image, queries the detector, and records the detection confidence for the patched object’s class. The variance across all sampled placements is then used as FTC.

Third, it computes the final anomaly score by sigmoid-normalizing FTC and CTC and subtracting the latter from the former. The threshold $\mathbf{y}$5 may be set to $\mathbf{y}$6 in the formulation provided in the data. This yields a binary test-time decision indicating whether the image is likely poisoned.

The implementation logic emphasizes semantic transformations rather than arbitrary augmentation. For CTC, the empirical measurement uses blending of the entire input image with different backgrounds rather than pixel-level cut-and-paste of individual objects, because the latter would be slow. For FTC, Monte Carlo sampling replaces exhaustive spatial scanning. A plausible implication is that TRACE is designed to balance detection fidelity against online query cost, although the paper also explicitly lists computational overhead as a limitation due to the multiple model queries per input.

A common misconception in this area is that a backdoor detector for object detection can be obtained by directly porting classification defenses. TRACE is explicitly framed against that assumption. It addresses multi-object outputs, false-positive and false-negative attack modes, and the absence of white-box access, all of which make direct transfer inadequate in the authors’ formulation.

5. Experimental evaluation

TRACE was evaluated on YOLO-v3, Faster-RCNN, and DETR, covering one-stage, two-stage, and transformer-based detection paradigms. The experiments used MS-COCO, PASCAL VOC, and Synthetic Traffic Signs as datasets, and seven attack types: OGA, RMA, GMA, ODA for false-positive-inducing attacks, and CIB, “Detector Collapse”, UTA for false-negative-inducing attacks (Zhang et al., 19 Mar 2025).

The defense comparisons included Detector Cleanse, which is prior OD-specific, and classification-adapted black-box methods including Strip, TeCo, SCALE-UP, and FreqDetector. The reported metrics were F1 score and AUROC. The data states that TRACE always performs best in Table 2.

A concise summary of the explicitly reported benchmark figures is given below.

Setting	TRACE	Next best reported baseline
YOLO on MS-COCO, attacks average, F1	0.877	Detector Cleanse: 0.665
YOLO on MS-COCO, attacks average, AUROC	0.900	Detector Cleanse: 0.657

The paper further reports a 30%+ F1 improvement and 0.25+ AUROC advantage, with stability across models and datasets, reflected in low standard deviation. In the abstract, the authors summarize the overall result as a 30% improvement in AUROC over state-of-the-art defenses. Because both formulations appear in the provided data, the safer reading is that the paper reports substantial improvements under both summary descriptions, depending on the aggregation being referenced.

TRACE was also tested against an omniscient adaptive attacker using an adaptive loss intended to suppress CTC and FTC variance differences on poisoned inputs. The reported result is that TRACE still distinguishes poisoned samples effectively except at an extreme tradeoff where attack success is itself lost. The paper’s interpretation is that transform consistency and attack objective are fundamentally at odds. This suggests that the very invariances exploited by a successful backdoor are difficult to erase without degrading the attack’s intended effect.

6. Properties, limitations, and significance

TRACE is characterized in the paper as fully black-box: it uses only final detections or class probabilities, does not require model internals or gradients, and requires no access to training data (Zhang et al., 19 Mar 2025). It is also described as universal in the sense of being applicable to any object detection model and not being attack-specific. Public background and foreground images serve as auxiliary inputs for the transformation process.

Its strengths follow directly from this design. The method handles both ghost and vanishing attack patterns through the CTC/FTC decomposition, supports multi-object scenes, and maintains effectiveness across architectures and datasets. The experimental section presents these as the basis for state-of-the-art black-box input-level defense performance and adaptive robustness.

The listed limitations are equally specific. TRACE incurs computational overhead because it requires multiple detector queries per test image. It depends on auxiliary data, namely public background and foreground images. It also requires tuning of hyperparameters such as $\mathbf{y}$7, $\mathbf{y}$8, and $\mathbf{y}$9, although the paper states that the results are robust to reasonable choices. A further limitation is the possibility of false positives from natural backdoor objects; the SSIM filter mitigates but does not eliminate that risk.

The significance of TRACE lies in how it reframes test-time backdoor detection for object detection. Rather than focusing on model introspection or trigger reconstruction, it measures consistency structure under controlled semantic perturbations. In the terminology of the paper, it is presented as the first universal, black-box, attack-agnostic test-time backdoor detection method for object detection. A plausible implication is that it establishes a template for future defenses in structured prediction settings where outputs are multi-instance and attack effects are not reducible to single-label misclassification.