Text as Illumination: Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation

Abstract: Language-guided Medical Image Segmentation (LMIS) has shown great potential to improve the delineation of anatomical structures and lesions by integrating clinical textual information. Existing methods generally rely on either implicit interaction between textual and visual features or auxiliary coarse-grained supervision for cross-modal alignment. However, these methods lack explicit and fine-grained constraints to ensure semantic consistency, causing a mismatch between language and the segmentation outputs. To address this issue, we propose Text-as-Illumination Retinex Network (TIRNet), a novel Retinex-inspired framework that treats text embeddings as semantic illumination for feature modulation, thereby improving semantic consistency in LMIS. TIRNet introduces two key blocks integrated at each decoder stage: (1) the Retinex-inspired Text Modulation Block (RTMB), which employs positive and negative illumination maps to enhance text-relevant foreground features and suppress background interference; and (2) the Consistent Detail Compensation Block (CDCB), which selectively recovers high-frequency details via a consistency-gated mechanism conditioned on illumination reliability. Furthermore, we propose a Multi-Scale Illumination Supervision Loss (MSIS-Loss), comprising a Region-Grounded Contrastive Loss (RGC-Loss) that enforces cross-modal similarity to be concentrated in text-relevant foreground regions and suppressed in background regions, and a Background Suppression Loss (BS-Loss) that provides pixel-level supervision for negative illumination maps, jointly ensuring a precise cross-modal alignment at each decoder stage. Extensive experiments on the MosMedData+ and QaTa-COV19 datasets demonstrate that TIRNet achieves state-of-the-art performance in LMIS. The code is available at: https://github.com/anaanaa/TIRNet.

• The paper introduces TIRNet that leverages Retinex-inspired text modulation to enhance spatial semantic alignment between clinical text and imaging features.
• It employs a dual-block design with RTMB and CDCB, combined with a multi-scale illumination supervision loss, achieving significant gains in m-Dice and g-Dice scores.
• Experimental results on chest CT and X-ray datasets demonstrate that TIRNet improves boundary accuracy and robustness in complex segmentation tasks.

Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation

Introduction and Motivation

Language-guided medical image segmentation (LMIS) leverages textual clinical priors to enhance the delineation of anatomical structures and pathological lesions. Despite progress in both unimodal and multimodal approaches, existing techniques predominantly utilize either implicit cross-modal fusion or apply coarse-grained supervision, both of which insufficiently enforce spatially precise semantic alignment. The implict nature of such modeling strategies leads to a semantic mismatch between language and segmentation outputs, especially in cases where fine spatial detail and accurate region-level correspondence are necessary for high-precision clinical applications. The paper "Text as Illumination: Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation" (2606.27794) introduces TIRNet, a framework that applies Retinex principles to enable text-driven semantic illumination and spatially-aware contrastive learning, yielding explicit region-level cross-modal consistency that advances the state of the art in LMIS.

Figure 1: The inadequacy of both implicit interaction (LViT) and coarse supervision (TeViA) to focus on foregrounds, compared to the accurate localization provided by TIRNet.

Methodology

The proposed TIRNet adopts a U-shaped encoder-decoder architecture, with two critical architectural innovations integrated into each decoder stage: the Retinex-inspired Text Modulation Block (RTMB) and the Consistent Detail Compensation Block (CDCB). Global text features are extracted via a frozen CLIP text encoder and modulate visual features through a semantic illumination mechanism.

Figure 2: TIRNet architecture, highlighting the integration of RTMB and CDCB at each decoder stage and the RGC-Loss for maximizing cross-modal similarity on text-relevant foregrounds.

Retinex-inspired Text Modulation Block (RTMB)

Drawing on Retinex theory, RTMB computes a semantic relevance map by projecting the text embedding into the visual feature space and calculating normalized inner products at each spatial location. Two complementary illumination maps are derived:

• Positive Illumination ($\mathbf{L}^+$): Focuses on high-responding, text-relevant (foreground) locations.
• Negative Illumination ($\mathbf{L}^-$): Highlights low-relevance (background) regions, adaptively tuned via a learnable sharpness parameter.

Visual features are then explicitly amplified or suppressed at each spatial location according to these text-driven illumination maps, with learnable weights controlling foreground enhancement, background suppression, and residual preservation.

Consistent Detail Compensation Block (CDCB)

While RTMB improves region-level semantic alignment, it may attenuate high-frequency structural details essential for accurate segmentation of fine boundaries. The CDCB compensates for this by:

• Extracting high-frequency details from the encoder stream using residual computation (feature minus average pool).
• Employing a semantic consistency gate that weights detail injection according to the cross-modal consistency between base decoder features and encoder-derived detail.
• Utilizing the positive illumination map for gating, ensuring that structural details are injected only in semantically-relevant (foreground) regions.

Multi-Scale Illumination Supervision Loss (MSIS-Loss)

TIRNet's MSIS-Loss combines two synergistic supervision signals at all decoder stages:

• Region-Grounded Contrastive Loss (RGC-Loss): Enforces maximization of cross-modal similarity within text-relevant (foreground) regions while concurrently suppressing background activations, explicitly optimizing the foreground-background contrast margin.
• Background Suppression Loss (BS-Loss): Applies binary cross-entropy supervision to the negative illumination map, ensuring pixel-wise background suppression aligns with the inverse of ground-truth labels.

This dual-objective loss ensures fine-grained, stage-wise semantic consistency and optimizes for not just global but also spatially-distributed segmentation accuracy.

Experimental Results

Evaluation was conducted on two large-scale public benchmark datasets: MosMedData+ (chest CT) and QaTa-COV19 (chest X-ray), using metrics such as $\mathrm{m\text{-}Dice}$, $\mathrm{m\text{-}IoU}$, $\mathrm{g\text{-}Dice}$, and $\mathrm{g\text{-}IoU}$. TIRNet demonstrates marked improvements over prior SOTA multimodal methods including LViT and TeViA, with consistent gains in all reported metrics.

Numerical highlights include:

• On QaTa-COV19, TIRNet achieves $\mathrm{m\text{-}Dice} = 84.77\%$ and $\mathrm{g\text{-}Dice} = 91.23\%$, outperforming LViT by 1.37 and 0.60 points, and TeViA by 0.65 and 0.27 points respectively.
• On MosMedData+, TIRNet leads by 0.89 ($\mathrm{m\text{-}Dice}$) and 1.91 ($\mathrm{g\text{-}Dice}$) over LViT and TeViA.

Visual comparisons confirm superior boundary accuracy and robustness in challenging cases, with TIRNet localizing target regions and suppressing spurious activations more effectively.

Ablation and Analysis

Component ablation demonstrates that each block provides complementary improvements: the RTMB yields 4.69 percentage points gain in $\mathbf{L}^-$0 over the baseline; the CDCB supplies an additional 1.53. Incorporation of the MSIS-Loss further increases $\mathbf{L}^-$1 by 0.52. The full configuration aggregates to a 6.33-point improvement in $\mathbf{L}^-$2.

Hyperparameter sensitivity analysis confirms that optimal performance is achieved with $\mathbf{L}^-$3 and $\mathbf{L}^-$4.

Implications and Future Directions

TIRNet's architectural principle of treating text as semantic illumination realizes explicit spatial modulation of visual features, surpassing the representational limits of implicit fusion or coarse alignment. The theoretical implications are significant: Retinex-inspired illumination provides a formal method to bridge the granularity gap between global text priors and pixel-level segmentation. Practically, this enables more accurate localization of text-relevant structures, with direct applicability to AI-augmented computer-aided diagnosis systems that need to integrate radiological images and clinical text.

Future developments could extend TIRNet's semantic modulation paradigm to additional medical imaging modalities or incorporate further linguistic structure from structured reports. Moreover, the presented contrastive supervision mechanism may prove valuable for other vision-language tasks requiring dense alignment, such as referring image segmentation and multimodal explainable AI.

Conclusion

TIRNet establishes a new benchmark for language-guided medical image segmentation via Retinex-inspired text-driven illumination and spatially-grounded contrastive learning. The explicit amplification and suppression of visual regions in accordance with text semantics, combined with stage-wise multimodal alignment objectives, addresses longstanding limitations of prior LMIS approaches. The demonstrated empirical gains and principled architectural contributions position TIRNet as a strong reference model for future cross-modal medical image analysis research.