Multispectral Imaging Systems

• Multispectral imaging system is an optical platform that acquires spatially resolved data across discrete spectral bands for material and chemical characterization.
• It employs diverse optical strategies like multiplexed illumination, diffractive optics, and monolithic filter arrays to achieve high-speed and high-resolution imaging.
• Design trade-offs between spatial resolution, spectral fidelity, and computational demand are addressed through advanced calibration and deep learning-based reconstruction.

A multispectral imaging system (MSI) is an optical architecture that acquires spatially resolved information across multiple discrete spectral bands, extending far beyond the conventional trichromatic (RGB) paradigm. These systems are pivotal in fields where detailed spectral signatures encode material, chemical, or biochemical properties—such as biomedical diagnostics, remote sensing, industrial inspection, agriculture, and advanced scientific instrumentation. By integrating custom optics, wavelength-selective elements, structured illumination, and computational reconstruction, MSIs provide a versatile framework for characterizing scenes in both spatial and spectral domains.

1. Fundamental Architectures and Optical Encoding Strategies

Multispectral imaging systems employ diverse principles for coding, separating, and capturing spectral and spatial information. Core architectures include:

• Multiplexed Illumination and Single-Pixel Detection: Computational ghost imaging platforms, as demonstrated by Huang and Shi, have replaced traditional filter wheels or spectrometers and detector arrays with a spatial light modulator (SLM) and a single “bucket” detector. Spatial–spectral information is multiplexed within illumination patterns using orthogonal binary masks (e.g., $E_R$, $E_G$, $E_B$), with spectral information (such as red, green, blue reflectance) encoded into each projected frame. This enables full-color recovery with a compressed-sensing–style inversion (Huang et al., 2017).
• Diffractive and Refractive Optics with Direct Spectral Coding: Some MSIs employ micro-structured diffractive filters (e.g., Wang & Menon’s transmissive DF), placed near the sensor to create a direct, calibrated mapping from spectral content to spatial code on the detector. These system matrices can be inverted through regularized linear algebra, supporting single-shot video-rate multi-band imaging (Wang et al., 2017). Metalens arrays (engineered metasurfaces with strong chromatic dispersion), as in single-shot architectures, spatially separate different spectral passbands directly onto the sensor, greatly reducing backend computation and maximizing acquisition speed (Audhkhasi et al., 26 Feb 2025).
• Monolithic Multispectral Filter Arrays (MSFAs): Incorporation of high-order Fabry–Pérot or plasmonic filter arrays directly atop CMOS sensors enables snapshot architectures, providing channel-selective transmission with narrow linewidth and broad coverage (e.g., 400–1000 nm). State-of-the-art high-order MSFAs use subwavelength structure tuning and selective resonance suppression for enhanced color purity and CMOS compatibility at micrometer pixel scales (Xiang et al., 2023). Similar principles are translated to the longwave infrared (LWIR, 8–14 µm) using plasmonic Al–Ge filter arrays (Shaik et al., 2023).
• Computational & Hybrid Approaches: Systems exploiting physical encoding via defocus, chromatic aberration, or engineered scatterers (e.g., strongly scattering media creating spectrally decorrelated speckle PSFs), combined with deep learning for spectral decoding, allow snapshot MSI using standard monochrome sensors without filters (Yang et al., 24 Jan 2025, Sahoo et al., 2017).
• Single-Pixel Imaging and Frequency-Division Modulation: Fast sinusoidal spectral coders in concert with spatial modulators achieve 3D ($x$–$y$–$\lambda$) data cube acquisition using one high-speed bucket detector, with temporal or frequency modulation separating bands in the detection sequence (Bian et al., 2015, Zhu et al., 28 Aug 2025).
• Upconversion–Based MSI in the Infrared: Room-temperature, single-shot MSI in the 2–5 µm regime has been achieved via adiabatic sum-frequency conversion in chirped-polled non-linear crystals, optically mapping mid-IR content into the visible for detection with silicon imagers (Beitner et al., 28 Jan 2026).

2. System Calibration, Forward Models, and Inverse Reconstruction

Each MSI architecture relies on a precise mathematical measurement model that couples system optics, spectral elements, and detector characteristics:

• Forward Models: The physical process is generally represented as $y = A x + n$, with $y$ the measured data (e.g., sensor pixels or bucket detector signals), $A$ a system matrix or operator (formed by PSFs or modulation matrices across space and wavelength), $x$ the unknown multispectral cube, and $E_G$0 additive noise. For diffractive filter–based systems, $E_G$1 encodes the effect of spatially and spectrally variant PSFs, while for bucket-detection systems, $E_G$2 reflects the spectral-spatial mixing imposed by both spatial and spectral modulations (Huang et al., 2017, Wang et al., 2017, Bian et al., 2015).
• Reconstruction Algorithms: Inverse problems are typically posed as regularized least-squares or compressed-sensing tasks, with reconstruction strategies ranging from closed-form Tikhonov inversion and sequential Fourier demodulation (for single-pixel and frequency-multiplexed systems) to iterative TV minimization and deep convolutional neural networks (e.g., U-Net or physics-informed networks). Loss functions may combine spatial and spectral TV regularization, structural similarity, and data fidelity terms (Huang et al., 2017, Yang et al., 24 Jan 2025, Zhu et al., 28 Aug 2025, Cohen et al., 2023).
• Calibration: System response calibration entails mapping spatial locations and spectral bands through measurement of monochromatic point-like inputs or narrowband filters across the FOV (e.g., Wang & Menon’s calibration grid of PSFs per $E_G$3), accounting for variations in detector quantum efficiency, filter transmission, and optical aberrations (Wang et al., 2017, Udayanga et al., 2023).

3. Performance Metrics and Comparative Evaluation

Assessment of MSI performance encompasses spatial and spectral resolution, SNR, throughput, and acquisition speed:

• Spectral Resolution: State-of-the-art diffractive systems and MSFAs deliver 9.6 nm or finer resolution in the visible (see Table below), with spectral crosstalk below $E_G$4 dB for non-neighboring bands (Wang et al., 2017, Xiang et al., 2023).
• Spatial Fidelity: Spatial resolution is dictated by lens optics, pixel pitch, and system design (e.g., 256×256 px at ~25 μm/px for Fresnel-lens systems; <20 μm for upconversion MSI; 20 lp/mm per channel for metalens systems) (Cohen et al., 2023, Audhkhasi et al., 26 Feb 2025, Beitner et al., 28 Jan 2026).
• Acquisition Rate: Snapshot and hardware-multiplexed systems routinely achieve 15–30 fps, while sequential SLM/bucket and scanning architectures are limited by required pattern count or scanning speed (e.g., ~60 s for $E_G$5 scenes using frequency-division single-pixel imaging at 3,000 patterns) (Bian et al., 2015).
• Noise Robustness/SNR: All-photon capture into a single detector or sensor region enables high photon efficiency in single-pixel and upconversion designs, with SNRs exceeding 40–50 dB under standard illumination for modern systems (Bian et al., 2015, Beitner et al., 28 Jan 2026).

Representative Performance Table

Architecture	Bands / Range	Spectral Δλ (nm)	Spatial Pixels	Frame Rate	SNR	Reference
Diffractive Filter (Vis/NIR)	25 (430–718 nm)	~9.6	30×30 (~120 μm/px)	15 fps	13+ dB	(Wang et al., 2017)
Fabry–Pérot MSFA (CMOS)	10–20 (400–1000 nm)	13–31	down to 1 μm	Snapshot	>60% trans.	(Xiang et al., 2023)
Metalens Array	8 (475–865 nm)	18–32	512×512 (ROI)	30 fps	~25 dB/ch	(Audhkhasi et al., 26 Feb 2025)
Single-Pixel/SPI	10 (450–650 nm)	20	64×64	1/minute	High	(Bian et al., 2015)
Upconversion (Mid-IR)	5 (2–5 μm)	150–200	512×512 (16 μm/px)	Snapshot	50–100	(Beitner et al., 28 Jan 2026)
Multi-camera RGB + filters	12 (vis/NIR)	~10–20	N×(sensor px)	Config.-dep.	Config-dep.	(Matulić et al., 21 Apr 2026)

4. Application Domains and Demonstrated Use Cases

Multispectral imaging systems are deployed in a broad range of environments:

• Biomedical/Clinical: Video-rate MSI is integrated into laparoscopes (16-band, 25 Hz), enabling real-time quantitative tissue assessment during surgery (e.g., perfusion, hemoglobin state) (Ayala et al., 2021). Advanced intraoperative systems combine simultaneous reflectance and fluorescence using optimized optics and linear unmixing, achieving sub-50 ms end-to-end latency and high color fidelity (ΔE ≈ 1.6) (Pentarakis et al., 19 Sep 2025). Multispectral phase imaging with diffractive stacks supports label-free QPI across up to 16 bands (Shen et al., 2023).
• Agriculture and Food Quality: Portable dual-mode reflectance/transmittance platforms with LED panels (13 bands, 365–940 nm) enable in-field product assessment, with data-fusion “merged” modes achieving up to 99% classification accuracy in food adulteration tasks (Udayanga et al., 2023).
• Remote Sensing/Environmental Monitoring: Compact, high-resolution MSI systems with on-chip filter arrays or metalens arrays support field deployment for vegetation mapping, mineral analysis, atmospheric gas detection, and environmental surveillance (Audhkhasi et al., 26 Feb 2025, Xiang et al., 2023, Shaik et al., 2023).
• Scientific and Industrial: MSI enables modalities such as compact light-field spectral imaging with neural radiance fields (BSNeRF), which reconstruct six-dimensional datacubes in a single exposure (x, y, z, θ, φ, λ) (Huang et al., 1 Sep 2025). Upconversion MSI extends room-temperature, high-resolution imaging to mid-IR regimes vital for vibrational spectroscopy, process monitoring, and security (Beitner et al., 28 Jan 2026). Superresolution and denoising via deep learning has improved spatial and perceptual quality in LWIR MSI (Shaik et al., 2023).

5. System Trade-Offs, Limitations, and Future Extensions

Any MSI design necessitates balancing spectral, spatial, hardware, and computational resources:

• Spatial–Spectral Trade-Off: Systems constrained by fixed sensor pixel budgets can reallocate between spatial and spectral resolutions with appropriate calibration (e.g., more bands at lower spatial detail, or vice versa) (Wang et al., 2017). Diffraction and filter-array systems also trade sharpness for bandwidth.
• Temporal/Snapshot Constraints: Sequential or scanning-based approaches offer higher spectral flexibility at the cost of slower acquisition. All-optical approaches (diffractive networks, metalens arrays, upconversion) enable true snapshot or video-rate imaging up to dozens of bands (Mengu et al., 2022, Audhkhasi et al., 26 Feb 2025).
• Crosstalk and SNR: High channel count systems (e.g., 16-band diffractive networks or MSFAs) must control for spectral crosstalk. Innovations such as selective resonance suppression (Pt layers at antinodes, polysilicon absorption) and deep spectral decoupling in neural networks (BSNeRF) address these issues for higher purity and fidelity (Xiang et al., 2023, Huang et al., 1 Sep 2025).
• Computational Burden vs. Optical Complexity: Architectures emphasizing computational inversion must address noise amplification, numerical stability (minimized spectral condition number), and feasibility of real-time operation, while hardware-multiplexed and monolithic-on-chip systems shift complexity to design/fabrication (Matulić et al., 21 Apr 2026).
• Miniaturization and Integration: Custom filter arrays, diffractive optics, and chip-scale single-pixel sensors have enabled fully integrated compact systems, some suitable for mobile platforms and low-cost educational deployments (Howell et al., 2024, Zhu et al., 28 Aug 2025).

6. Advances in Computational and Deep Learning–Driven MSI

Recent MSI platforms have incorporated advanced computational methods:

• Physics-Informed Neural Networks: PINNs integrate measurement physics directly into the network loss, enabling accurate reconstruction even at low sampling rates and challenging SNRs, without requiring ground-truth datasets for training (Zhu et al., 28 Aug 2025).
• Deep Learning for Demosaicing and Superresolution: Convolutional neural networks (ResNet blocks, 3D convolutions) demosaic filter-array outputs with high spectral and spatial fidelity (Wisotzky et al., 2023, Shaik et al., 2023).
• Spectral Neural Radiance Fields (NeRF): BSNeRF achieves full spectral and plenoptic scene reconstruction by learning a continuous radiance field conditioned on wavelength, spatial, and angular coordinates (Huang et al., 1 Sep 2025).
• Snapshot Spectral Inversion via Deep Decoders: Models such as mFIN (multi-spectral Fourier Imager Network) invert chromatic-aberration–encoded snapshots directly into multichannel images, leveraging spatial–frequency adaptive modules and joint loss metrics (Yang et al., 24 Jan 2025).

7. Outlook and Emerging Directions

MSI technologies continue to evolve in fabrication, computation, and deployment:

• Expansion to New Spectral Regimes: Chirped upconversion and high-order filter array approaches push into the mid-infrared with room-temperature snapshot detection (Beitner et al., 28 Jan 2026, Xiang et al., 2023).
• Programmable and Tunable Structures: MEMS-actuated filters, phase-change materials, or electro-optically tuned layers will enable actively reconfigurable spectral bands (Xiang et al., 2023).
• On-Chip Hybridization: Monolithic integration of MSFAs with CMOS, wafer-bonded diffractive networks, or stacked metasurface optics will further miniaturize and ruggedize systems for field and mobile use (Xiang et al., 2023, Mengu et al., 2022).
• All-Optical and Low-Latency Approaches: Engineering of optical front-ends (e.g., polarization-insensitive, high-throughput metasurfaces) in conjunction with computational backend optimization remains a key focus for enabling real-time operation in challenging environments (Mengu et al., 2022, Audhkhasi et al., 26 Feb 2025).

In summary, multispectral imaging systems constitute a highly diverse class of optical–computational instruments vital to scientific, industrial, clinical, and mobile applications. They leverage a spectrum of strategies from pure optics to deep computational inversion, with advances driven by both hardware innovations (diffractive networks, high-order MSFAs, metasurfaces, upconversion) and powerful numerically stable, data-driven algorithms. The design of an MSI requires careful attention to the trade-offs in spatial, spectral, temporal, and noise-performance dimensions for its targeted application domain.