Generalization Under Scrutiny: Cross-Domain Detection Progresses, Pitfalls, and Persistent Challenges

Abstract: Object detection models trained on a source domain often exhibit significant performance degradation when deployed in unseen target domains, due to various kinds of variations, such as sensing conditions, environments and data distributions. Hence, regardless the recent breakthrough advances in deep learning-based detection technology, cross-domain object detection (CDOD) remains a critical research area. Moreover, the existing literature remains fragmented, lacking a unified perspective on the structural challenges underlying domain shift and the effectiveness of adaptation strategies. This survey provides a comprehensive and systematic analysis of CDOD. We start upon a problem formulation that highlights the multi-stage nature of object detection under domain shift. Then, we organize the existing methods through a conceptual taxonomy that categorizes approaches based on adaptation paradigms, modeling assumptions, and pipeline components. Furthermore, we analyze how domain shift propagates across detection stages and discuss why adaptation in object detection is inherently more complex than in classification. In addition, we review commonly used datasets, evaluation protocols, and benchmarking practices. Finally, we identify the key challenges and outline promising future research directions. Cohesively, this survey aims to provide a unified framework for understanding CDOD and to guide the development of more robust detection systems.

• The paper reframes cross-domain object detection as a stage-coupled problem, emphasizing intertwined challenges in proposal coverage, feature discriminativity, and calibration.
• The paper decomposes CDOD methods along six axes, revealing gaps in alignment, open-set adaptation, and the limitations of current evaluation metrics like mAP.
• The paper highlights practical pitfalls such as rare-class collapse and localization drift, and advocates for stage-wise diagnostics and causal modeling for robust deployment.

Cross-Domain Object Detection: Progress, Pitfalls, and Challenges

Introduction

This survey provides a rigorous analysis of cross-domain object detection (CDOD), contextualizing the task as a stage-coupled optimization problem characterized by the intertwined requirements of proposal coverage, feature discriminativity, and calibration under domain shift. Unlike classification, object detection models must infer both what (semantic class) and where (spatial location), making adaptation inherently more complex. The review addresses the fragmentation in the literature, where methods are often benchmark-centric and lack unified analysis, leading to ambiguity in interpreting CDOD progress.

Figure 1: The survey motivation: traditional detectors degrade under domain shift; previous surveys focus mainly on taxonomy, whereas this work synthesizes four complementary pillars—formal formulation, pipeline analysis, failure analysis, and unified design with diagnostic tools.

Theoretical Framework for Cross-Domain Object Detection

Formal Problem Definition

CDOD is defined as minimizing the expected detection risk on a target distribution while maintaining three coupled invariants:

• Proposal Coverage: Ensures that target proposals retain recall comparable to the source.
• Feature Discriminativity: The latent representation must support foreground/background and inter-class separability (e.g., preserved Fisher ratio).
• Calibration: Maintains well-aligned confidence scores (low ECE).

The challenge is the recursive dependency between detection stages: domain shift in features alters proposal distribution, adversely affecting subsequent head modules and overall recall, with downstream modules incapable of recovering missed proposals. Unlike classification, where DA theory (e.g., Ben-David's error bounds) is defined over fixed input-output mappings, in detection, the proposal distribution is endogenous and output assignment is non-decomposable.

Figure 2: Detection pipeline misalignments—feature, proposal, and proposal-to-label—propagate, inducing compounded degradation in target performance.

Probabilistic Decomposition and Failure Propagation

The survey formalizes the detection process as

$P_T(y|x) = \int P_T(y|b, x) P_T(b|x) db,$

showing that errors in $P_T(b|x)$ (proposal recall) cannot be rectified by adaptation solely at the detection head. Thus, cross-domain adaptation objectives are forcibly stage-coupled; improvements at any stage influence data distribution for the subsequent steps, often resulting in unpredictable failures if not jointly optimized.

Taxonomy and Analytical Decomposition

Six orthogonal axes are used to decompose and systematize CDOD approaches:

1. Alignment / Invariance / Robustness Paradigms: Alignment reduces inter-domain discrepancies; invariance targets causal/statistical stability; robustness-based approaches aim for deployment-readiness via domain generalization.
2. Geometry vs. Semantic Preservation: Methods may emphasize geometric consistency (e.g., regression stability, aspect ratios) or semantic transfer (category separation).
3. Implicit vs. Explicit Distribution Modeling: Implicit optimization (adversarial, consistency) dominates; explicit probabilistic modeling (prototypes, Gaussian) remains underutilized.
4. Instance- vs. Scene-Level: Foreground-centric versus global alignment, with corresponding failure modes linked to proposal quality and context sensitivity.

   Figure 3: Alignment-discriminativity tension—excessive alignment collapses task-relevant feature geometry, while moderate alignment optimizes task transfer.

   

   Figure 4: Instance-level versus scene-level adaptation, elucidating pipeline stages targeted: localized features versus global context.

5. Closed/Open/Universal Shift: Most methods assume closed-set; open/universal adaptation is underexplored, despite practical relevance for unknown or shifting categories.

   Figure 5: Progression from closed-set to universal shift—benchmarks and methods rarely address the latter, resulting in fragility under real deployment.

6. Causal/Correlational Adaptation: Causal invariance, though conceptually promising, is rarely instantiated due to challenges in model identification and intervention.

Failure Mode Analysis and Empirical Deficiency

The empirical landscape reveals several persistent failure modes:

1. Alignment-centric Fragility: Methods based on adversarial alignment consistently exhibit rare-class collapse, as majority classes dominate discriminator gradients [Chen et al., 2018; Zhu et al., 2019].

   Figure 6: Adversarial adaptation aligns marginal feature distributions, but collapses rare-class structure due to gradient dominance by common classes.

2. Self-Training Collapse: Pseudo-labeling amplifies model bias and confirmation errors, particularly when calibration degrades under target shift.

   Figure 7: Pseudo-label self-training—a teacher generates target pseudo-labels, resulting in confirmation bias and error reinforcement if calibration is poor.

3. Localization/Classification Entanglement: Alignment primarily improves classification but incurs localization drift, a trade-off that is rarely measured due to overreliance on aggregate mAP.
4. Open-set Vulnerability: Most CDOD methods misclassify target-private categories or suppress them as background, leading to negative transfer.
5. Proposal Instability and Background Over-alignment: The proposal generator, calibrated on source, suffers recall loss on target, compounding downstream head failure. Scene-level alignment overfits to dominant background statistics, erasing fragile foreground discriminativity.
6. Metric Myopia: Dominance of mAP conceals which pipeline component fails, impeding progress on composability and robust design.

Datasets and Evaluation Protocols

Major benchmarks (e.g., Cityscapes → Foggy Cityscapes, SIM10K → Cityscapes, COCO → BDD100K) are curated for specific shift types (weather, synthetic-to-real), but rarely address complex or universal domain shifts (e.g., context reconfiguration, open-label). Critically, these datasets support the current overemphasis on explicit feature alignment with controlled shift, whereas long-tail, context, and annotation biases in real deployment remain insufficiently tested.

Future Directions

The review identifies several research gaps, each with actionable open problems:

• Causal Modeling: Operationalizing interventions and causal structures for detection remains an open challenge, crucial for robust transfer beyond statistical alignment [Zhang et al., 2022].
• Foundation Model Supervision: Leveraging vision-language or foundation models as robust teachers or pseudo-labelers can mitigate feedback collapse under shift [VCR et al., 2025].
• Test-Time and Continual Adaptation: Adaptation at deployment (test-time, continual) is vastly underexplored for detection compared to classification, with open questions on which modules to adapt and how to avoid catastrophic forgetting.
• Calibration-aware Methods: None of the current leading approaches directly optimizes or preserves target domain calibration, despite its critical role in pseudo-label selection and deployment reliability.
• Prompt-driven Detection: Emerging research on promptable or instruction-tuned detection models can enable dynamic adaptation without retraining; however, interface design and effect on the detection pipeline remain open [Zhan et al., 2025].
• Stage-wise Diagnostics & Evaluation: Adoption of diagnostics beyond mAP (e.g., proposal recall, regression error, calibration curves) is necessary to understand and address stage-specific failures.

  Figure 8: Teacher-based distillation—knowledge transfer from EMA or foundation models enhances target robustness, breaking self-training feedback loops.

Conclusion

This survey reframes CDOD as a stage-coupled optimization problem, highlighting that proposal coverage, feature discriminativity, and calibration must be preserved together. The review demonstrates that most literature is concentrated in a narrow region of the design space—alignment-based, closed-set, implicit, scene- or instance-level, correlational methods—leaving significant empirical and theoretical gaps. Robust adaptation and credible progress require explicit tracking of pipeline invariants, multi-domain diagnostics, and methodological expansion into compositional, data-centric, and causally-motivated paradigms. By synthesizing taxonomy, structural analysis, and diagnostic evaluation, this work provides an actionable foundation for future CDOD research and practical deployment (2604.08230).