Task-Oriented Infrared Image Enhancement

• Task-oriented infrared image enhancement is a framework that optimizes IR images for downstream tasks by enhancing key structural details and preserving low-light information.
• It employs a two-step layer decomposition using joint l0-l1 gradient regularization to separate base and detail layers, ensuring clarity while suppressing noise.
• Morphological reconstruction-based saliency extraction highlights target regions effectively, resulting in measurable gains in object detection and semantic segmentation.

Task-oriented infrared image enhancement refers to algorithmic approaches that improve the perceptual and analytic utility of infrared images specifically for downstream tasks such as object detection, semantic segmentation, and scene understanding, rather than purely for visual aesthetics. Modern methods recognize that conventional enhancement (e.g., contrast maximization) may not yield optimal results for machine perception. Instead, the enhancement is directly optimized to maximize target information, suppress noise, and support high-level computer vision models in challenging conditions—including adverse weather, low-light, and for hard-to-detect objects.

1. Layer Decomposition for Infrared Image Enhancement

Infrared images commonly exhibit low contrast, especially for non-thermal targets such as bicycles or background features. The method introduces a two-step layer decomposition based on joint $l_0$-$l_1$ gradient regularization, explicitly designed for the characteristics of IR imagery.

The optimization objective for layer decomposition is: $$\min_{I_B} \Big\{ (I_{in} - I_B)^2 + \lambda_1 \|\nabla I_B\|_1 + \lambda_2 \|\nabla(I_{in} - I_B)\|_0 \Big\}$$ where:

• $I_{in}$ is the input infrared image,
• $I_B$ is the base layer,
• $I_{in} - I_B$ defines the detail layer,
• $\|\nabla \cdot\|_1$ encourages sparsity in gradients (sharp edge preservation) for the base,
• $\|\nabla \cdot\|_0$ ensures sparsity in the detail layer's gradients (keeping only prominent, structural details),
• $\lambda_1, \lambda_2$ are weighting parameters.

A second decomposition on the base layer yields finer detail extraction, ultimately generating both compressed/stretched detail and base representations. The enhanced image is assembled as: $I_{out} = \alpha I_{D'} + I_{D_1} + \beta I_{B_1'}$ with empirically determined fusion coefficients $\alpha, \beta$.

This approach enhances informative structure (edges, salient regions), while retaining crucial low-light region information that is often suppressed by standard deep learning or classical enhancement techniques. It avoids over-enhancement of bright areas and inappropriate amplification of sensor noise.

2. Morphological Reconstruction-Based Saliency Extraction

To further address the challenge of object information obscuration (especially for non-thermal targets) without introducing noise, the method incorporates a grayscale morphological reconstruction (GMR) strategy.

Two principal steps are employed:

• Noise Removal: Apply GMR with a small structuring element (e.g., $2 \times 2$) to the image, which suppresses isolated noise pixels.
• Salient Target Extraction: Employ a much larger structuring element to extract only the connected salient regions (large object/target structures) from the image.

The difference between the outcomes of large and small structuring elements is computed and partitioned:

• Positive differences yield the "bright" region saliency map,
• Negative differences yield the "dark" region saliency map.

The resulting saliency maps are linearly combined with the cleaned base image: $$f_E = f_B + \alpha_1 \cdot \frac{f_m^d}{\max(|f_m^d|)} + \alpha_2 \cdot \frac{f_m^b}{\max(f_m^b)}$$ where $f_B$ is the small-structuring-element output, $f_m^d$ and $f_m^b$ are the dark/bright saliency maps, and $\alpha_1, \alpha_2$ are adaptive weights.

This process robustly highlights salient objects and target regions, enhancing their visibility without introducing excessive noise or over-amplifying inconsequential image fluctuations.

3. Empirical Performance for Detection and Segmentation Tasks

The method’s effectiveness was evaluated on widely-used benchmarks for object detection (FLIR Clean, with YOLOv11) and semantic segmentation (SUS, with CLNet-T). Standard measures (mAP, mAP$_{50}$, mIoU) and image-level metrics (Entropy, Spatial Frequency, Average Gradient, Standard Deviation, Visual Information Fidelity) were assessed.

Key findings:

• Object detection: Average performance increased by +1.4% mAP and +0.6% mAP$_{50}$ over the baseline, with the largest gains for bicycle detection (+2.4% mAP, +4.9% mAP$_{50}$). This is significant, as bicycles are cold/low-contrast targets typically missed in infrared images.
• Semantic segmentation: mIoU reaches 0.813 (validation) and 0.804 (test), outperforming alternatives across nearly all classes, especially rare/hard classes.
• Qualitative assessment: Visualizations confirm that challenging targets, including non-thermal objects and background structures, become more discernible without the system introducing noise or washing out high-emissivity areas.

Ablation studies show that both layer decomposition and morphological saliency extraction independently improve results, and their combination provides the largest gains in both perceptual and analytic metrics.

4. Task-Driven Enhancement: Principles and Implications

Unlike generic infrared image enhancement pipelines, the proposed approach is explicitly tailored to improve the accuracy of high-level computer vision tasks (object detection, semantic segmentation), not just visual quality. Noteworthy principles include:

• "Task-oriented enhancement" optimizes contrast to facilitate machine perception, not simply to satisfy human visual preference. For instance, excessive noise reduction or global contrast stretching may reduce detection/segmentation performance, even if they yield images subjectively pleasing to the eye.
• The approach emphasizes structural and target region enhancement, especially for classes at high risk of being missed (cold targets, background features).
• Bounding the noise amplification and preserving critical information is prioritized over aggressive denoising or global histogram equalization.

This suggests a model-driven paradigm: enhancement algorithms should always be evaluated in the context of the final task, rather than solely with traditional image quality or visual assessment metrics.

5. Applicability and Broader Context

The described enhancement framework is particularly beneficial for:

• Autonomous driving and ADAS systems: It extends reliable detection to difficult conditions (fog, rain, low-light) and improves identification of vulnerable road users.
• General high-level vision tasks: Security/surveillance, UAV imaging, medical thermal imaging, and night-time robotics, anywhere that infrared imaging is used under adverse conditions.
• Resource-constrained or real-time systems: The method is model-based, making it suitable for deployment in devices with limited computational resources and without extensive annotated data for retraining.

A plausible implication is that the same layer decomposition and morphological reconstruction principles could be extended to other imaging modalities or multi-modal enhancement frameworks, provided that task-driven metrics drive the optimization.

6. Summary Table: Method Components and Contributions

Component	Purpose	Enhancement Outcome
Layer Decomposition ($l_0$-$l_1$)	Enhance details, preserve dark regions	Structural detail and target clarity
Morphological Saliency Extraction	Extract salient object information, suppress noise	Salient targets highlighted, noise controlled
Model-based Fusion	Weighted combination of base and detail/saliency layers	Balanced visibility, artifact-avoidance
Task-Oriented Evaluation	Measure on detection, segmentation, perceptual metrics	Superior real-world machine performance

7. Conclusions and Field Positioning

This task-oriented infrared image enhancement method exemplifies a trend in computational imaging: pipeline design is being increasingly integrated with and evaluated by the needs of downstream analytic models. By combining principled layer decomposition, morphological saliency extraction, and empirical evaluation directly on high-level tasks, the approach advances performance in complex, practical scenarios and sets a detailed baseline for future research in both enhancement and holistic scene understanding under challenging sensing conditions.