Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environments

Abstract: Object detection in hazy environments is challenging because degraded objects are nearly invisible and their semantics are weakened by environmental noise, making it difficult for detectors to identify. Common approaches involve image enhancement to boost weakened semantics, but these methods are limited by the instability of enhanced modules. This paper proposes a novel solution by employing language prompts to enhance weakened semantics without image enhancement. Specifically, we design Approximation of Mutual Exclusion (AME) to provide credible weights for Cross-Entropy Loss, resulting in CLIP-guided Cross-Entropy Loss (CLIP-CE). The provided weights assess the semantic weakening of objects. Through the backpropagation of CLIP-CE, weakened semantics are enhanced, making degraded objects easier to detect. In addition, we present Fine-tuned AME (FAME) which adaptively fine-tunes the weight of AME based on the predicted confidence. The proposed FAME compensates for the imbalanced optimization in AME. Furthermore, we present HazyCOCO, a large-scale synthetic hazy dataset comprising 61258 images. Experimental results demonstrate that our method achieves state-of-the-art performance. The code and dataset will be released.

• The paper demonstrates that leveraging language prompts with CLIP-based loss effectively compensates for semantic degradation in hazy images.
• The novel methodology integrates AME and FAME weighting to reweight standard cross-entropy loss, yielding significant mAP improvements on both synthetic and real datasets.
• The approach is validated using HazyCOCO and ablation studies, showing robust performance enhancements across diverse adverse imaging conditions.

Language Prompts versus Image Enhancement: Vision-Language Losses for Robust Object Detection in Hazy Environments

Introduction

The persistent challenge of object detection in hazy environments (ODHE) is largely attributable to severe visual degradation, where objects become indistinct, and their semantic content is frequently suppressed by environmental noise. Traditional pipelines rely extensively on image enhancement—either as a preprocessing step or via jointly-trained enhancement modules—to mitigate loss of visibility and semantic richness. However, such pipelines present fundamental issues including suboptimal enhancement targeting (i.e., overemphasis on background or introduction of artifacts) and instability due to network-induced noise.

The paper "Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environments" (2604.10637) introduces a paradigm shift by leveraging vision-LLMs and textual prompts to directly compensate for semantic attenuation in degraded images. Instead of modifying the image or its low-level representations, the methodology focuses on enhancing high-level semantic reasoning within the detector via a novel cross-entropy loss, CLIP-CE, guided by CLIP similarity scores between visual regions and carefully constructed textual prompts.

Figure 1: Pipeline comparison; the proposed method can discover overlooked objects in a hazy image by leveraging only language prompts.

Methodology

Critique of Existing Enhancement Solutions

The limitations of image enhancement-based ODHE strategies are twofold:

1. Inaccurate Enhancement: Enhancement operations are spatially agnostic, frequently improving irrelevant image regions (e.g., background) while neglecting severely degraded objects. This undermines downstream detection as these objects remain poorly represented.
2. Noise Amplification: CNN-driven enhancement can engender undetectable noise, potentially increasing misclassification or omission rates (Figure 2).

   Figure 2: Failure cases of IA-YOLO; enhancement modules can result in loss or misclassification of objects due to uncontrolled semantic modification.

CLIP-Guided Cross-Entropy (CLIP-CE) Loss

The core innovation is the CLIP-guided Cross-Entropy (CLIP-CE) loss, which reweights the standard CE loss using semantic similarity signals between visual crops and textual prompts—computed via the pre-trained, frozen CLIP model. This mechanism is implemented in two stages:

• Approximation of Mutual Exclusion (AME): Each object instance is compared (via CLIP) with a positive prompt ("a photo of a {cls}") and a negative prompt ("a photo without {cls}"). The negative similarity score serves as an indicator of semantic loss for each object; higher negative similarity correlates with greater degradation. The softmax of these similarities is used as a sample-wise loss weight.
• Fine-tuned AME (FAME): Recognizing that AME-derived weights do not adapt throughout training (since CLIP is frozen), FAME introduces a lightweight adapter network. This network dynamically adjusts the AME weight using both visual features and classifier confidence, ensuring that attention on poorly represented objects is adaptively maintained as training progresses.

The final detection loss alternates between AME weighting and FAME weighting, depending on the training epoch.

Figure 3: CLIP-CE architecture. ROI features are combined with AME/FAME-based CLIP similarities for semantic-aware loss reweighting throughout the object detection pipeline.

Synthetic Dataset: HazyCOCO

To address existing dataset limitations—insufficient data and poor realism—the authors synthesize HazyCOCO, a 61,258-image dataset derived from COCO. Images are filtered for outdoor scenes, and corresponding depth maps are generated with Depth-Anything, post-processed with clamping to maintain realistic scene hierarchy. Haze is then added via the Atmospheric Scattering Model using empirically selected parameters, yielding a dataset with authentic hazy statistics superior to the widely used VOC Fog.

Figure 4: HazyCOCO generation pipeline juxtaposed with VOC Fog; left: clear image and depth, right: corresponding synthetic haze results.

Figure 5: Depth map visualization; clamping improves scene stratification and haze synthesis realism.

Figure 6: HazyCOCO images under varying haze density, parameterized by the scattering coefficient.

Experimental Results

Quantitative Benchmarks

Extensive experiments on synthetic and real datasets (HazyCOCO, RTTS, Exdark, TrashCan) validate the superiority of using CLIP-CE over image enhancement or focal loss weighting. Notably, on HazyCOCO, mAP increases from 39.67 (Focal Loss) and 42.82 (DCP) to 44.92 (CLIP-CE). On the real RTTS dataset, the improvement is pronounced (from 74.26 to 76.76). In low-light and underwater detection, the CLIP-CE approach generalizes robustly, outperforming both enhancement-based and direct weighting methods.

Figure 7: Visualization of AME vs. Focal weights. AME weights scale with object degradation levels, unlike Focal weights which may become non-informative due to overfitting.

Visual Results

Qualitative results on HazyCOCO and RTTS show that the method is able to recover and detect instances that are essentially invisible to standard detectors or enhancement-based approaches (Figure 8). Similar effects are observed in adverse non-hazy conditions (low-light and underwater; Figure 9), demonstrating strong cross-domain generalization.

Figure 8: Detection visualizations on HazyCOCO and RTTS: CLIP-CE enables detection of severely occluded/degraded objects missed by all baselines.

Figure 9: Results on low-light and underwater datasets; CLIP-CE displays improved robustness across distinct adverse imaging conditions.

Analysis and Ablations

Ablation studies confirm the progressive utility of loss components:

• AME weighting alone consistently boosts mAP, particularly for highly degraded datasets.
• FAME further improves performance, especially in domains where semantic recovery lags training progress.

Prompt engineering experiments affirm that matching the CLIP pre-training distribution (e.g., standard "a photo of a {class}") is critical for maximizing vision-language alignment and detection accuracy.

Theoretical and Practical Implications

This work highlights the capacity of foundation vision-LLMs to address domain-specific perception failures without explicit image restoration. By injecting prompt-driven semantic priors directly into the loss function, detection networks can be made robust to a suite of real-world imaging conditions (haze, darkness, turbidity) in a modality-agnostic manner.

The approach is amenable to further theoretical generalization. Potential research avenues include:

• Extension to open-vocabulary detection by broadening the set of textual prompts.
• Coupling with dynamic prompt selection based on scene context.
• Integration with more advanced transformer-based detectors.
• Application to other semantic segmentation or panoptic tasks under adverse conditions.

Conclusion

This work demonstrates that vision-language guidance, operationalized via a CLIP-based semantic weighting of cross-entropy loss, can robustly overcome the detection bottleneck in hazy and other visually adverse environments. The approach eschews unstable image domain enhancement in favor of targeting semantic attenuation at the representation level, offering both strong cross-domain performance and enhanced generalization to new adverse conditions. The release of HazyCOCO further advances the empirical study of ODHE and will underpin future innovations in semantic robustness under environmental degradation.

Reference: "Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environments" (2604.10637).