Luminance–Chromaticity Decomposition Overview

• Luminance–chromaticity decomposition is the process of separating brightness from color information, facilitating tailored processing for tasks like color inference and denoising.
• It underpins various imaging techniques by employing both multiplicative and linear models to achieve illumination invariance and perceptual alignment.
• Recent advances leverage deep learning and variational methods with dual-branch architectures to improve noise resilience, HDR rendering, and computational efficiency.

Luminance–chromaticity decomposition is a foundational strategy in color image analysis wherein an observed signal is factored into a scalar luminance or intensity component and a multidimensional chromaticity or color-component field. This split decouples the energy or “brightness” of the signal from its color information, enabling domain-specific handling for color inference, regularization, compression, denoising, and inverse problems under diverse illumination and modality constraints. Recent advances demonstrate substantial improvements in ill-posed inverse imaging, robust color restoration under non-uniform lighting, hyperspectral scene rendering, and computational efficiency, by explicitly modeling and exploiting this decomposition.

1. Mathematical Formulations and Modeling Approaches

Luminance–chromaticity decomposition takes several canonical forms:

• Multiplicative Split (Radiance/HSI): For a multichannel signal $X(u,v,\lambda)$ (e.g., hyperspectral radiance), factorization

$X(u,v,\lambda) = I(u,v) \cdot C(u,v,\lambda)$

with $$I(u,v) = N_\lambda^{-1} \sum_\lambda X(u,v,\lambda)$$ (intensity map) and $C(u,v,\lambda) = X(u,v,\lambda)/(I(u,v)+\epsilon)$ (chromaticity cube), enforcing normalization $\sum_\lambda C(u,v,\lambda) = 1$ (Wang et al., 20 Sep 2025).

• Linear Color Space Decomposition: RGB is mapped to YCbCr or YUV via a fixed linear transformation,

$$\begin{bmatrix} Y \ U \ V \end{bmatrix} = M_{RGB \to YUV} \begin{bmatrix} R \ G \ B \end{bmatrix}$$

yielding one luminance channel and two chrominance channels, with inverse for mapping back to RGB (Hu et al., 3 Dec 2024, Prativadibhayankaram et al., 2023, Shimobaba et al., 2013).

• Sphere-based Chromaticity in Color Constancy: For $v(n) \in \mathbb{R}^3_+$, define chromaticity as $\hat w = w/\|w\|_2$ on $S^2_+$ and luminance as $\|w\|_1$; normalization per-image compensates for exposure (Chakrabarti, 2015).
• Retinex-inspired Factorization: For image $I(x,y)$, model as

$I(x,y) = L(x,y) \cdot R(x,y)$

using HSV for component guidance and residual corrections for illumination normalization under colored lighting (Vasluianu et al., 4 Aug 2025).

• Brightness–Chromaticity in Variational Models: Pointwise, $u_0(x) = (u_0)_b(x)\cdot (u_0)_c(x)$ with $(u_0)_b(x) = |u_0(x)|$ (magnitude), $(u_0)_c(x) = u_0(x)/|u_0(x)|$ on $S^2$ (Ferreira et al., 2016).

This decomposition fundamentally underlies color science, including the von Kries model and modern inverse reconstruction pipelines.

2. Principles and Motivation

Key motivations for luminance–chromaticity decomposition include:

• Illumination invariance: Chromaticity is invariant under global intensity scaling; thus, factoring out luminance allows models to recover intrinsic reflectance properties independent of lighting variations (Wang et al., 20 Sep 2025, Chakrabarti, 2015, Vasluianu et al., 4 Aug 2025).
• Perceptual alignment: Human vision is more sensitive to spatial detail in luminance than in chrominance, justifying resolution reduction and regularization on chromatic channels (Shimobaba et al., 2013, Prativadibhayankaram et al., 2023).
• Algorithmic tractability: Decomposition reduces problem dimensionality, improves conditioning (e.g., of sensing matrices in compressive spectral imaging), and enables decoupled or stage-wise processing (Wang et al., 20 Sep 2025, Hu et al., 3 Dec 2024, Vasluianu et al., 4 Aug 2025).
• Physical interpretability: Split provides a meaningful prior for reconstruction—separating energy-dependent from spectral or color components (Zhang et al., 17 Nov 2025, Ferreira et al., 2016).
• Statistical modeling: Enables the definition of conditional priors $P(\text{chromaticity}|\text{luminance})$ or learning robust mappings via end-to-end architectures (Chakrabarti, 2015, Prativadibhayankaram et al., 2023).

A plausible implication is that all imaging pipelines subject to variable illumination or chromatic distortions benefit from explicit luminance–chromaticity separation.

3. Deep Learning Architectures and Algorithmic Realizations

Recent literature demonstrates a diverse set of architectures exploiting luminance–chromaticity decomposition:

• CIDNet (CASSI Hyperspectral Imaging): A dual-camera pipeline observes intensity directly, reconstructs chromaticity via a deep unfolding network with a Hybrid Spatial–Spectral Transformer (HSST), incorporating TopK spectral attention and spatial branch (window-based Swin Transformer). A degradation-aware, spatially-adaptive noise estimator outputs variance maps for anisotropic fidelity (Wang et al., 20 Sep 2025).
• ShadowHack: Division into LRNet (luminance restoration, U-Net-like, rectified outreach attention module for shadow regions) and CRNet (chromaticity regeneration with cross-attention, ConvNeXt color encoder). Processing in YCbCr allows targeted shadow and color correction, with checkpoint ensembling to expose color regeneration to minor luminance errors (Hu et al., 3 Dec 2024).
• NH-3DGS (Native-HDR 3D Gaussian Splatting): Color representation as $c = \text{Lum}\cdot\text{Chromaticity}$, with per-Gaussian luminance scalars and chromaticity SH coefficients encoding view-dependent hues, enabling balanced gradients and HDR fidelity (Zhang et al., 17 Nov 2025).
• Retinex-inspired RLN²: Parallel U-shaped branches for luminance and chromaticity residual estimation; cross-domain feature fusion attention leverages HSV guidance, Haar DWT multi-spectral features, and ConvNeXt-wide context for ambient lighting normalization (Vasluianu et al., 4 Aug 2025).
• Dual-branch Compression Models: Image compression models process $Y$ (structural/luminance) and $UV$ (color/chrominance) in separate autoencoder branches, optimizing with color difference (CIEDE2000) in the loss function (Prativadibhayankaram et al., 2023).

The typical workflow involves initial decomposition, individual processing branches (often with attentional or feature fusion mechanisms), and recombination for final image formation.

4. Applications and Empirical Outcomes

Luminance–chromaticity decomposition is deployed across domains:

Application Area	Specific Usage	Reference
Hyperspectral Imaging (CASSI)	Factorization improves lighting-invariant recovery, robust to anisotropic noise	(Wang et al., 20 Sep 2025)
Shadow/Color Restoration	Two-stage separation yields improved shadow removal and color fidelity	(Hu et al., 3 Dec 2024)
HDR 3D Scene Reconstruction	NH-3DGS enables artifact-free HDR renderings from single-capture	(Zhang et al., 17 Nov 2025)
Color Constancy	Pixelwise classifier exploits $P(\text{chromaticity}|$luminance$)$ for illuminant estimation	(Chakrabarti, 2015)
Image Compression	Dedicated branches control structure	color fidelity, better rate-distortion-color trade-off
Hologram Computation	Chroma subsampling accelerates computation with negligible sharpness loss	(Shimobaba et al., 2013)
Ambient Lighting Normalization	RLN² with HSV-driven cross-domain attention improves restoration under colored lighting	(Vasluianu et al., 4 Aug 2025)
Variational Denoising/Cartoon–Texture Split	BV–G-norm and harmonic map regularization enables edge-adaptive denoising	(Ferreira et al., 2016)

Reported metrics include significant improvements in PSNR, SSIM, and perceptual quality:

• CIDNet-9stg reaches KAIST simulation PSNR: 44.12 dB (HSIs), SSIM: 0.991; chromaticity PSNR: 35.8 dB, SSIM: 0.93 (Wang et al., 20 Sep 2025).
• NH-3DGS: PSNR +6.6 dB over vanilla 3DGS, SSIM: 0.972, LPIPS: 0.011 (Zhang et al., 17 Nov 2025).
• ShadowHack: ISTD⁺ PSNR: 36.31 dB, SSIM: 0.977, RMSE: 2.48; SRD PSNR: 35.94 dB, SSIM: 0.982 (Hu et al., 3 Dec 2024).
• RLN²-𝓛ᶠ: PSNR: 20.52 dB, SSIM: 0.746, LPIPS: 0.208 (CL3AN) (Vasluianu et al., 4 Aug 2025).
• Image compression: ΔE₀₀ (CIEDE2000) reduced by 10–15% over competing codecs (Prativadibhayankaram et al., 2023).

These empirical outcomes suggest superior robustness to lighting, shadow, and color distortions, improved computational efficiency, and enhanced perceptual fidelity.

5. Theoretical Guarantees and Variational Analysis

The variational literature formalizes the decomposition in settings with regularization and fidelity constraints:

• Meyer’s “u+v” and Sphere-valued Maps: Brightness channel handled via BV–G-norm for cartoon|texture split; chromaticity regularized on $S^2$ with edge-adaptive terms:

$$F^{reg}(u_b,u_c) = \int |∇u_b| + g(|∇u_b|)|∇u_c| + |u_c⊗∇u_b + u_b∇u_c|\, dx$$

$\Gamma$-convergence analysis demonstrates existence and compactness of minimizers as coupling penalties vanish, together with sharp edge preservation and adaptive smoothing (Ferreira et al., 2016).

• Statistical Luminance-to-Chromaticity Classifiers: Learned per-pixel priors $P(\text{chromaticity}|\text{luminance})$ provide robust global inference under uncertainty, outperforming previous deep and exemplar-based color constancy methods (Chakrabarti, 2015).

Decoupling brightness and chromaticity allows for tailored regularization—Meyer’s G-norm for oscillatory energy, spherical harmonic or low-frequency bases for chromaticity, and joint fidelity for reconstruction.

6. Computational Efficiency and Cost–Quality Trade-offs

Decomposition strategies directly reduce computational cost:

• Subsampling Chrominance: In color CGH generation, aggressive chroma subsampling (4:1:1, 8:1:1) yields up to 40% time reduction with very limited image quality degradation due to visual insensitivity to high-frequency chroma (Shimobaba et al., 2013).
• Dual-branch Compression: Luminance branch is afforded higher model capacity, while chrominance branch maintains color; per-channel latent visualizations show texture|structure distinct in Y, color blobs in UV (Prativadibhayankaram et al., 2023).
• NH-3DGS Real-time Rendering: Lum-chroma split obviates the need for high-order SH expansion, stabilizing gradients and enabling 233 fps real-time performance (Zhang et al., 17 Nov 2025).

A plausible implication is that tailored resolution, attention, and regularization across decomposed luminance/chromaticity channels nearly always yields favorable cost-quality trade-offs.

7. Limitations and Open Problems

Current decompositions face several challenges:

• High-frequency or non-smooth chromaticity: Very sharp or view-dependent spectral effects may not be captured by low-order chromatic bases (SH or similar), requiring higher expressiveness or multiplicative splits for specular tints (Zhang et al., 17 Nov 2025).
• Dynamic scene complexity: Most models assume static illumination; adapting frameworks to dynamic or deformable scenes is not fully addressed (Zhang et al., 17 Nov 2025).
• Residual artifact suppression: Methods may underflow extreme luminance or leave minor color spill artifacts in deep shadows (Vasluianu et al., 4 Aug 2025).
• Learning vs. analytic trade-offs: Fixed linear decompositions (e.g., YCbCr/YUV) may not optimally separate perceptual or task-relevant chromaticity, while learned bases could offer improved disentanglement.

Further research may focus on expanding decomposition theory to dynamic materials, adaptive or learned color spaces, and joint multimodal domains (e.g., polarization, spectral imaging), as well as rigorously quantifying the limitations under extreme scene compositions.