ThermalGen: RGB-to-Thermal & Energy Applications

• ThermalGen is a multifaceted term representing both a style-disentangled, flow-based RGB-to-thermal generative model and system blueprints for thermal analysis and energy conversion.
• It leverages advanced neural techniques such as KL-VAE, Scalable Interpolate Transformer, and classifier-free guidance to achieve robust cross-domain thermal image translation.
• ThermalGen systems integrate explicit physical priors with calibration and domain conditioning to optimize thermal reconstruction, energy harvesting, and power generation performance.

ThermalGen is used in recent arXiv literature in more than one sense. Its most specific usage denotes the style-disentangled, flow-based RGB-to-thermal image translation model introduced in “ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation” (Xiao et al., 29 Sep 2025). In parallel, the same label also appears as a system name in several technical blueprints for multimodal thermal reconstruction, chip-scale thermal analysis, and heat-to-electricity conversion. Taken together, these usages place ThermalGen at the intersection of thermal imaging, generative modeling, physics-informed inference, and thermal-energy systems.

1. Terminological scope

In the supplied literature, “ThermalGen” is not a single universally fixed term but a recurring research label applied to distinct technical objects.

Usage	Domain	Defining elements
ThermalGen as a named model	RGB-to-thermal image translation	Style-disentangled, adaptive flow-based model with KL-VAE thermal latents, SiT backbone, RGB conditioning, and dataset-style embeddings (Xiao et al., 29 Sep 2025)
ThermalGen as a baseline/model family	Aerial RGB-to-thermal translation	Public ThermalGen-L-2-concat checkpoint evaluated zero-shot on Midwest aerial scenes (Sherpa et al., 17 May 2026)
ThermalGen as a system blueprint	Multimodal thermal reconstruction	ThermalGaussian-based RGB–thermal 3DGS pipeline with MSMG, OMMG, and multimodal regularization (Lu et al., 2024)
ThermalGen as a system blueprint	Thermal analysis and thermal power generation	Physics-informed IC thermal mapper, rectennas, radiative-cooling TEGs, near-field TPV, thermomagnetic generators, STCs, and related harvesters (Chandra et al., 1 Dec 2025)

The most sharply defined and self-contained usage is the 2025 RGB-to-thermal generative model (Xiao et al., 29 Sep 2025). The broader system-label usages are best understood as application-specific “ThermalGen” instantiations rather than a single canonical architecture.

2. ThermalGen as a style-disentangled RGB-to-thermal generator

The 2025 model defines ThermalGen as an adaptive, style-disentangled, flow-based generative framework for RGB-to-thermal translation across satellite–aerial, aerial, and ground domains (Xiao et al., 29 Sep 2025). Its central motivation is the scarcity of synchronized, calibrated RGB–thermal pairs and the resulting difficulty of training multimodal alignment, retrieval, and homography systems at scale.

Architecturally, ThermalGen combines a thermal KL-VAE with a Scalable Interpolate Transformer (SiT). The thermal image $x_T$ is encoded to a latent $z_T$, and generation is performed in latent space through a probability-flow ODE. The forward process is

$$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$

and the learned velocity field is conditioned on RGB latents and style:

$$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$

Sampling integrates the learned ODE from terminal noise to $\hat{\mathbf{z}}_{0,T}$ and then decodes the result to $\hat{\mathbf{x}}_T$ (Xiao et al., 29 Sep 2025).

RGB conditioning is implemented in two variants. One inserts multi-head cross-attention between RGB latents and the current thermal latent, while the other concatenates $z_{\mathrm{RGB}}$ with the noised thermal latent. The reported ablations state that concatenation yields better FID than cross-attention in this task (Xiao et al., 29 Sep 2025). This is significant because it favors a simple conditioning pathway over a heavier token-interaction mechanism.

The model’s notion of “style” is dataset-specific rather than purely aesthetic. Style is defined as the RGB-to-thermal mapping induced by sensor characteristics, viewpoint regime, and environmental conditions. ThermalGen represents style through learnable embeddings $Y=\{y_0,\dots,y_n,y_{un}\}$, injected through adaLN-Zero conditioning, and uses classifier-free guidance (CFG) through an unconditional style branch (Xiao et al., 29 Sep 2025). The disentanglement is therefore operational: RGB latents carry content, while the style embedding modulates the mapping rules.

A notable technical clarification in the paper is that ThermalGen is not a normalizing flow in the RealNVP/Glow sense. It uses the probability-flow ODE perspective of diffusion and minimizes a velocity-regression objective rather than a tractable log-likelihood with invertible coupling layers (Xiao et al., 29 Sep 2025).

3. Data regime, training configuration, and benchmark behavior

ThermalGen is trained jointly across an extensive paired RGB–thermal corpus spanning satellite–aerial, aerial, and ground settings, including over ten public datasets and three new large-scale datasets: DJI-day, Bosonplus-day, and Bosonplus-night (Xiao et al., 29 Sep 2025). These new datasets contribute broader geographic coverage, multiple sensor types, and both day and night conditions.

The reported implementation uses random resize-and-crop to $256\times256$, evaluation at $256\times256$, denoising steps $z_T$0, and a style embedding dimensionality of 1024. The thermal KL-VAE is trained for 200k steps with batch size 16, AdamW, learning rate $z_T$1, and weight decay $z_T$2; the SiT generator is trained for 200k steps with batch size 64 and AdamW learning rate $z_T$3 on a single NVIDIA A100 or H100 (Xiao et al., 29 Sep 2025).

Representative benchmark results illustrate the model’s cross-domain reach.

Benchmark	ThermalGen-L/2 result	Comparison note
Bosonplus-day	FID 76.91, LPIPS 0.35, PSNR 14.66, SSIM 0.31	Lower FID than pix2pix 170.45 and pix2pixHD 157.65
NII-CU	PSNR 26.44, SSIM 0.92, FID 69.30, LPIPS 0.21	Better FID than pix2pix 168.77 and pix2pixHD 118.60
M$z_T$4FD	PSNR 23.73, SSIM 0.81, FID 35.82, LPIPS 0.14	Improves on VQGAN FID 79.21 and DiffV2IR 75.95

Beyond headline scores, the paper reports that CFG materially improves difficult domains. On Boson-night, FID improves from 161.22 at base conditioning to 116.46 with CFG scale 8.0; on FLIR, FID improves from 70.09 to 63.43 with CFG scale 4.0 (Xiao et al., 29 Sep 2025). These numbers support the claim that style control is not ancillary but central to robustness across low-contrast nighttime imagery, sensor shifts, and domain imbalance.

The 2026 aerial translation study provides an instructive external perspective. Using the public ThermalGen-L-2-concat checkpoint zero-shot on Midwest urban scenes, that study reports PSNR 7.56, SSIM 0.2444, and LPIPS 0.6317, and describes the outputs as low-contrast and washed on out-of-domain data (Sherpa et al., 17 May 2026). The same paper’s conditional U-Net, trained in-domain with weather and location metadata, reaches PSNR 14.5485, SSIM 0.8095, and LPIPS 0.1666 (Sherpa et al., 17 May 2026). This does not negate ThermalGen’s multi-domain design; it shows that the model remains sensitive to unresolved distribution shift when deployed zero-shot outside its training distribution.

4. Relations to thermal reconstruction and multimodal scene modeling

ThermalGen also appears in system-blueprint form within multimodal 3D reconstruction. The most direct example is the ThermalGaussian blueprint, which specifies a thermal 3D Gaussian Splatting pipeline capable of rendering both RGB and thermal views from aligned image pairs (Lu et al., 2024). That system calibrates RGB and thermal cameras with a heated chessboard, initializes poses through RGB-only SfM, blended images, or MSX fusion, and trains either two jointly optimized Gaussian sets (MSMG) or one unified Gaussian set (OMMG).

Its multimodal regularization uses

$z_T$5

with total loss

$z_T$6

to suppress modality-specific overfitting and reduce redundancy (Lu et al., 2024). The reported outcome is approximately 90% storage reduction compared with training each modality separately. Thermal rendering quality reaches about PSNR 25.6 dB, SSIM 0.882, and LPIPS 0.170 for OMMG, while RGB quality improves by about 1.1 dB over the RGB-only 3DGS baseline (Lu et al., 2024). In this usage, ThermalGen denotes an engineering pipeline built on ThermalGaussian rather than the flow-based translator of (Xiao et al., 29 Sep 2025).

A second adjacent reconstruction line is Thermoxels, which uses a voxel lattice with per-corner density $z_T$7, spherical harmonics coefficients $z_T$8, and temperature $z_T$9 to generate FEA-compatible 3D thermal models from paired RGB and thermal imagery (Chassaing et al., 6 Apr 2025). Unlike NeRF or Gaussian splatting, the Thermoxels representation is explicitly volumetric and can be converted into hexahedral or tetrahedral meshes for conduction simulation. The paper reports that JaxFEM conduction simulation converged in 10 steps from the reconstructed initial thermal field (Chassaing et al., 6 Apr 2025). This makes the contrast with the image-centric ThermalGen model particularly clear: one line emphasizes cross-modal synthesis, the other simulation-ready geometry.

The aerial conditional U-Net study bridges these strands by treating ThermalGen as a comparative RGB-to-thermal baseline and emphasizing metadata conditioning. Its FiLM-conditioned bottleneck injects a 15-dimensional weather/location vector, and the authors argue that environmental metadata capture thermal factors not inferable from RGB alone (Sherpa et al., 17 May 2026). That study therefore positions ThermalGen as a strong but not sufficient baseline when environmental context is omitted.

5. ThermalGen as a label for heat-to-electricity systems

In several engineering syntheses, “ThermalGen” denotes thermal-power-generation systems rather than image models. These systems span multiple transduction mechanisms.

Direct radiative rectification appears in the large-area infrared rectenna work, where an unbiased nanoantenna-coupled MOS tunnel diode produces a peak generated power density of 8 nW/cm$$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$0 at 450 °C across an optimized load (Shank et al., 2018). The same synthesis emphasizes photon-assisted tunneling, load matching at $$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$1, and a presently very low radiative-to-electrical efficiency of about $$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$2 (Shank et al., 2018).

Radiative-cooling thermoelectric generation defines another ThermalGen family. One optimization study reports that with $$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$3, $$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$4, and $$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$5, a nighttime radiative-cooling TEG can reach approximately 2.2 W/m$$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$6, with the TEG covering less than 1% of the system footprint and a 153% gain over a regular blackbody emitter (Fan et al., 2020). A related RC-TEG study finds that maximum power occurs at load ratio $$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$7, that an optimal cooler-to-leg area ratio of about 86 gives baseline power density near 19.5 mW/m$$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$8, and that favorable ambient and humidity conditions can raise power density to about 291 mW/m$$\mathbf{z}_t = \alpha_t \mathbf{z}_0 + \sigma_t \boldsymbol{\epsilon},$$9 (Zhao et al., 2020).

Conventional thermoelectric hardware also appears under the ThermalGen label. A thermoelectric power-generating heat exchanger with commercial Bi$$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$0Te$$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$1 modules produces 2 W per TEG, or 0.22 W cm$$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$2, at a fluid temperature difference of 175 °C and about 5 L min$$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$3 per channel; a larger realization reaches 200 W from 100 TEGs (Bjørk et al., 2016). At the micro scale, Bi/Sb thin-film Seebeck generators on 1 cm$$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$4 glass chips achieve $$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$5 V at $$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$6 K and about 1.2 $$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$7W after final annealing in one reported geometry (0711.3294). At the suspended-membrane extreme, planar nanoTEG networks on SiN reach roughly 0.3 $$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$8W per 1 cm$$v_\theta\big(\hat{\mathbf{z}}_{t,T}, t; \mathbf{z}_{\mathrm{RGB}}, \mathbf{y}\big).$$9 chip for an effective temperature gradient of 10 K and realize about 60% of the available air–substrate temperature difference across the membrane (Tainoff et al., 2018). The carbogel TVIP line proposes still another large-area thermoelectric route, with current p-type-only material extrapolated to about 125 mW m$\hat{\mathbf{z}}_{0,T}$0 at $\hat{\mathbf{z}}_{0,T}$1 °C and a target p-type material projected to about 5.5 W m$\hat{\mathbf{z}}_{0,T}$2 (Athar et al., 21 Dec 2025).

Near-field thermophotovoltaics constitute a higher-power radiative branch. A large-area NF-TPV device with a nominal 150 nm gap and 0.28 cm$\hat{\mathbf{z}}_{0,T}$3 active area produces 1.22 mW at 460 °C, or about 4.4 mW cm$\hat{\mathbf{z}}_{0,T}$4, which is a twenty-five-fold increase over the corresponding far-field configuration (Selvidge et al., 2024). An earlier integrated NEMS near-field TPV platform reports about 1.25 $\hat{\mathbf{z}}_{0,T}$5W cm$\hat{\mathbf{z}}_{0,T}$6 when the emitter–detector gap is tuned from about 500 nm to about 100 nm, with $\hat{\mathbf{z}}_{0,T}$7 (Bhatt et al., 2019). A solar thermoradiative–photovoltaic variant pushes the thermodynamic envelope in another direction, with a detailed-balance limiting solar conversion efficiency of 85% for fully concentrated sunlight and 45% for one sun in the equal-area case (Tervo et al., 2020).

Other ThermalGen mechanisms are non-thermoelectric. The validated digital twin of a genus-3 thermomagnetic generator predicts 0.158 V open-circuit against 0.165 V measured and 0.8 mW peak power against 0.84 mW measured, while identifying mixing losses and thermal nonuniformity as dominant inefficiencies (Izadi et al., 18 Apr 2026). Semiconductor-sensitized thermal cells generate up to about 0.2 mW from $\hat{\mathbf{z}}_{0,T}$8 devices at 40–55 °C, exhibit sustained cooling of about 1 °C under periodic discharge, and show about 5 °C cooling when four cells are integrated in parallel (Hayashida et al., 13 Dec 2025). Spin-caloritronic generation offers a different limit case: in a LaY$\hat{\mathbf{z}}_{0,T}$9Fe$\hat{\mathbf{x}}_T$0O$\hat{\mathbf{x}}_T$1/Pt bilayer, the spin Seebeck effect produces microvolt-scale signals, with fitted magnon relaxation length around 6.7 mm at 300 K (Uchida et al., 2010).

6. Recurrent design principles and open constraints

Across these disparate usages, several common principles recur. One is the systematic insertion of explicit physical structure into otherwise data-driven pipelines. ThermalGaussian adds heat-diffusion-inspired smoothing and multimodal regularization to 3DGS (Lu et al., 2024). Thermoxels hard-codes a volumetric temperature field to remain FEA-compatible (Chassaing et al., 6 Apr 2025). The IC thermal-analysis system derived from 2D-ThermAl uses a hybrid U-Net with positional encoding and a Boltzmann regularizer, attaining RMSE about 0.71 °C and running up to about 200 times faster than COMSOL on the studied 2D chip-thermal task (Chandra et al., 1 Dec 2025). ThermalGen in the RGB-to-thermal sense likewise incorporates explicit style variables rather than leaving all nuisance variation to latent entanglement (Xiao et al., 29 Sep 2025).

A second recurring principle is that calibration, alignment, and domain conditioning are often the dominant bottlenecks. ThermalGaussian requires precise RGB–thermal calibration and notes that pure thermal SfM often fails without multimodal initialization or MSX enhancement (Lu et al., 2024). Thermoxels emphasizes that cross-modal registration errors bias temperature assignment to geometry and that reflective or flat surfaces cause holes in the recovered volume (Chassaing et al., 6 Apr 2025). The aerial U-Net comparison shows that a zero-shot ThermalGen checkpoint can degrade sharply under aerial domain shift, while metadata conditioning restores much of the missing thermal structure (Sherpa et al., 17 May 2026).

A third principle is that thermal systems are frequently constrained not by first-order conversion laws but by secondary bottlenecks such as contact resistance, unused spectral power, parasitic heat flow, or transport mismatch. The rectenna is limited by coupling, barrier engineering, and impedance matching (Shank et al., 2018). NF-TPV is limited by sub-bandgap heating and series resistance even while surpassing the far-field photocurrent limit (Selvidge et al., 2024). Thermomagnetic generation is constrained by mixing-chamber losses and delayed heat propagation (Izadi et al., 18 Apr 2026). STCs show that time modulation can matter as much as peak instantaneous conversion because preventing thermal steady state is what yields sustained cooling (Hayashida et al., 13 Dec 2025).

A plausible implication is that “ThermalGen,” across subfields, names less a single formalism than a recurring design pattern: combine thermal observables or thermal gradients with explicit structural priors, then engineer the bottleneck that most strongly limits usable output. In computer vision, that bottleneck is often calibration or domain shift; in energy systems, it is commonly interface physics, thermal management, or impedance matching.