Day-Augmented Night Auto-Labeling

• Day-Augmented Night Auto-Labeling Engine (DAN-ALE) is an automated system that transfers rich day annotations to low-light night scenes using progressive and generative adaptation methods.
• It integrates techniques like pseudo-label fusion, cross-modal label transfer, and dual-branch networks to bridge the illumination gap between day and night data.
• Empirical results show significant mIoU improvements and cost-efficiency in annotating challenging nighttime images and videos.

A Day-Augmented Night Auto-Labeling Engine (DAN-ALE) is a class of automated annotation systems and supporting pipelines that exploit robust recognition models, richly labeled daytime imagery, and multimodal or synthetic knowledge sources to generate high-quality semantic, detection, or tracking labels for nighttime images or videos. The core aim is to mitigate, through algorithmic adaptation and cross-domain supervision, the acute annotation gap that arises in challenging low-light environments due to visibility degradation, domain shift, and the difficulty or cost of reliable human labeling.

1. Principles and Rationale

DAN-ALE leverages the fundamental observation that daytime datasets are abundant, richly labeled, and visually clear, whereas gathering high-quality night-time annotations is labor-intensive and often unreliable under adverse conditions. The engine systematically transfers knowledge from the robust source domain (day) to the target domain (night) by employing a mix of approaches, including:

• Adaptation of robust feature extractors trained on labeled daytime or thermal data
• Pseudo-labeling and progressive domain adaptation via twilight or intermediate domains
• Cross-modal label transfer exploiting physically aligned sensor modalities
• Generative models for synthesizing (e.g., through style transfer) scene variants that preserve annotation fidelity
• Domain-invariant or domain-bridging architectures that learn features robust across illumination regimes

These methods enable the facility for auto-labeling by leveraging day data directly or as a scaffold for generating pseudo-labels on otherwise weakly supervised or entirely unlabeled night data.

2. Core Methodologies

DAN-ALE instantiations employ a variety of strategies, summarized as follows:

Method Family	Day-to-Night Adaptation Mechanism	Representative Paper(s) [arXiv id]
Progressive Model Adaptation	Gradual fine-tuning via intermediate (twilight) data	(Dai et al., 2018)
Generative Style Transfer	Cycle-consistent or attention-based night synthesis	(Sun et al., 2019, Yang et al., 21 Dec 2024, Mohwald et al., 2023)
Dual-Branch/Ensemble Networks	Night-focused network with pseudo-label fusion	(Xie et al., 15 Jun 2024)
Cross-Modal Label Transfer	Direct IR→RGB label mapping from calibrated pairs	(Bouzoulas et al., 3 Jul 2025)
Bidirectional/Cross-domain Mixing	Mask-based sample mixing and domain-invariant enhancement	(Yang et al., 2021, Liu et al., 2022)
Transformer/Feature Alignment	Domain-invariant features via transformer bridging	(Ye et al., 2022)
Self-Loop and Co-Teaching	Reconstruction and cross-domain agreement strategies	(Shen et al., 2022)
Physics-Prior Invariant Layers	Color/edge invariance for zero-shot adaptation	(Lengyel et al., 2021)
GAN Edge Consistency	Edge-preserving synthetic night generation	(Mohwald et al., 2023)

A key distinguishing property is whether the labeling is achieved via pseudo-labels derived from model predictions, direct mapping from another modality, or transferred labels on synthetic images, and which form of domain adaptation is used to mitigate the illumination-induced distribution shift.

3. Progressive and Pseudo-Label-Driven Adaptation

In progressive model adaptation approaches—for example, gradual adaptation using civil, nautical, and astronomical twilight sequences (Dai et al., 2018)—the segmentation network trained on annotated day images is iteratively fine-tuned on increasingly night-like, unannotated samples. At each stage, pseudo-labels generated by the previous model are used to supervise the current model on the next, less-illuminated domain. This recursive pseudo-labeling propagates reliable knowledge across the day-night gap, as in:

$$\text{minimize}_{\phi^j} \Big\{ \sum_{i} L(\phi^j(x^i), y^i) + \lambda \sum_{k} L(\phi^j(x^{j-1}_k), \hat{y}^{j-1}_k) \Big\}$$

where each $\hat{y}^{j-1}_k$ is the pseudo-label obtained from $\phi^{j-1}$ on new domain samples $x^{j-1}_k$.

Empirical results show significant mIoU improvements on nighttime images, and effective performance even with minimal human annotation, supporting the engine’s cost-efficiency.

4. Generative Style Transfer and Label Consistency

GAN-based style transfer engines synthesize night images from day (or vice versa) to leverage day labels or robust daytime-trained models directly (Sun et al., 2019, Yang et al., 21 Dec 2024). Two key instantiations are:

• Inference-time translation: Night images are converted to synthetic day, then segmented/detected using a daytime model.
• Training-time augmentation: Labeled daytime images are translated to night via CycleGAN or Efficient Attention GAN, maintaining annotation mappings since geometry remains unchanged.

A cycle-consistency loss ensures realism and label fidelity:

$$L_{\text{cyc}}(G, F) = \mathbb{E}_{x \sim P_X}[\|F(G(x)) - x\|_{1}] + \mathbb{E}_{y \sim P_Y}[\|G(F(y)) - y\|_{1}]$$

This enables “labeling-free” augmentation, where annotations from the original domain are used on style-transferred images, circumventing the need for direct night annotation.

5. Cross-Modal and Multi-View Auto-Labeling

In environments where multiple sensor modalities are available and spatially aligned, such as datasets with perfectly registered infrared (IR) and RGB cameras (Bouzoulas et al., 3 Jul 2025), a high-confidence object detector trained on IR data can generate bounding boxes that are directly mapped onto the RGB counterpart. This cross-modal transfer is enabled by hardware-level calibration (e.g., beam splitter alignments and matched focal lengths), obviating the need for pixel-wise geometric transformations or manual labeling in low-light RGB scenarios.

Similarly, datasets with day–night paired and geometrically aligned video (e.g., via SLAM-based camera pose estimation (Wang et al., 4 Jun 2025), or repeated egocentric trajectories (Zhang et al., 7 Oct 2025)) allow propagation of 3D or semantic labels from day to night views with spatial precision. This is especially effective for static scene content, though dynamic object labelling may require auxiliary alignment strategies or verification mechanisms.

6. Composite Networks, Pseudo-Label Fusion, and Self-Supervised Learning

Several modern day-augmented engines architect composite or dual-branch networks. For example, PIG (Xie et al., 15 Jun 2024) introduces a Night-Focused Network (NFNet) which learns from a small number of labeled prompt night images and target night samples, while a main UDA branch brings semantic knowledge from abundant day labels. The selection of which branch to use for the pseudo-label of each class is determined by a domain similarity metric (e.g., LPIPS), with the fusion:

$$\hat{y}_T^{(\text{fuse})} = \mathbf{M}_{\text{fuse}} \odot \hat{y}_T^{(\text{nf})} + (\sim \mathbf{M}_{\text{fuse}}) \odot \hat{y}_T^{(\text{uda})}$$

This allows DAN-ALE systems to address class-specific day-night discrepancies, avoiding confirmation bias and improving overall pseudo-label quality for re-training.

Unsupervised/self-loop frameworks further refine labels using reconstruction or agreement signals (Shen et al., 2022). Inner and outer self-loops reconstruct the input image from the predicted semantic map and latent encoding, with the outer loop crossing domain boundaries to enforce representation alignment. Co-teaching, as in the “DNA” (Day-Night Agreement) strategy, combines offline pseudo-labelling and online agreement between day reference and night candidate via label overlap measures.

7. Applications, Impact, and Limitations

DAN-ALEs have demonstrated substantial impact across semantic segmentation (Dai et al., 2018, Tan et al., 2020, Yang et al., 2021, Xie et al., 15 Jun 2024), object detection (Yang et al., 21 Dec 2024, Kennerley et al., 2023, Bouzoulas et al., 3 Jul 2025), tracking (Ye et al., 2022), egocentric vision and VQA (Zhang et al., 7 Oct 2025), and metric learning-based retrieval (Mohwald et al., 2023). Across tasks, auto-labeled datasets consistently outperform or match the performance of baseline night-trained models, even in scenarios where human annotation is infeasible or unreliable due to poor visibility.

Common to almost all approaches is the finding that simply applying daytime-trained models or naive transfer yields significant drops in accuracy under night conditions. However, progressive adaptation, generative style transfer, exposure modeling, and cross-modal mapping reliably recover much of this gap, especially for static-scene content (e.g., roads, buildings).

Limitations persist: pseudo-labels may contain errors due to distribution shift, misalignment, or failure to accommodate dynamic scene changes. Highly dynamic objects and rare classes remain a challenge. Sophisticated uncertainty modeling, fusion strategies, or additional domain-bridging mechanisms (e.g., via physics priors (Lengyel et al., 2021)) represent active areas for further advancement.

8. Practical Implications and Future Directions

DAN-ALEs have broad applicability in automated annotation for data-scarce and hazardous conditions (autonomous driving, robotics, surveillance, egocentric navigation, infrastructure monitoring). The use of simulation environments (CARLA (Yang et al., 21 Dec 2024)), prompt-limited night knowledge injection (Xie et al., 15 Jun 2024), and self-supervised consistency set new directions for cost- and data-efficient learning.

As datasets with day-night aligned, multimodal, and spatially calibrated data proliferate (e.g., EgoNight (Zhang et al., 7 Oct 2025), Oxford Day-and-Night (Wang et al., 4 Jun 2025)), evaluation frameworks will further support robust domain-adaptive learning and long-term deployment across the day-night spectrum.

The architecture of these auto-labeling engines is expected to evolve towards greater modularity, uncertainty-aware integration, and hybrid domain bridging—potentially combining the strengths of all discussed approaches to systematically close the illumination gap and enable reliable, automatic annotation for a wide spectrum of nighttime vision applications.