---
title: 'ThermEval-D: Thermal Imagery & Climate Diagnostics'
url: https://www.emergentmind.com/topics/thermeval-d
type: topic
---

# ThermEval-D: Thermal Imagery & Climate Diagnostics

ThermEval-D is an ambiguous designation in the arXiv literature. In "ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery," it denotes a newly collected thermal-imagery dataset that provides dense per-pixel temperature maps with semantic body-part annotations and is integrated into the broader ThermEval-B benchmark for thermal visual question answering [2602.14989]. In "A new diagnostic tool for water, energy and entropy budgets in climate models," it denotes a modular diagnostic package developed within the ESMValTool framework to compute energy, water and entropy metrics from climate-model output [1902.04669]. The two usages are unrelated in domain, methodology, and intended application, and precise citation context is therefore necessary.

## 1. Name disambiguation

The cited literature uses the same label for two distinct research artifacts.

| Usage of “ThermEval-D” | Domain | Core description |
|---|---|---|
| ThermEval-D in ThermEval-B | Thermal vision-language evaluation | The first dataset to provide dense per-pixel temperature maps with semantic body-part annotations across diverse indoor and outdoor environments |
| ThermEval-D in ESMValTool | Climate-model diagnostics | A modular diagnostic package developed within the ESMValTool framework to compute energy, water and entropy metrics from climate-model output |

The thermal-imagery ThermEval-D is embedded in a benchmark motivated by the observation that vision language models achieve strong performance on RGB imagery, but they do not generalize to thermal images [2602.14989]. The climate-model ThermEval-D is embedded in a process-oriented evaluation workflow whose primary objectives are to quantify imbalances and transports in the top-of-atmosphere, atmospheric and surface energy budgets; close the global hydrological budgets; diagnose the Lorenz Energy Cycle; and estimate the climate system’s material entropy production by two complementary routes [1902.04669].

A common misconception is to treat ThermEval-D as a single framework. The cited sources do not support that interpretation. Instead, they document homonymous artifacts serving different communities: one centered on thermal vision-language understanding, the other on non-equilibrium thermodynamics and climate-model evaluation.

## 2. ThermEval-D as a thermal-imagery dataset

Within the thermal-imagery literature, ThermEval-D is introduced as part of ThermEval-B, a structured benchmark of approximately 55,000 thermal visual question answering pairs designed to assess the foundational primitives required for thermal vision language understanding [2602.14989]. ThermEval-D itself contributes newly collected data with dense temperature supervision and semantic annotations. The stated sensor is a TOPDON TC001 Plus handheld thermal imager with native resolution 256 × 192 pixels, thermal sensitivity less than 40 mK, frame rate 25 Hz, and measurement range –20 °C to 550 °C with accuracy ±1 °C.

The data-acquisition protocol covers diverse indoor scenes such as offices, labs, and workspaces, and outdoor scenes such as parks and open grounds on a university campus. The subjects are 35 adult volunteers, age 18–47 yrs and weight 64–108 kg, with mixed skin tones, performing natural activities including standing, sitting, walking, and stair climbing. The collection procedure includes Institutional Ethics Committee approval, informed consent, anonymization, and on-site medical support. The dataset contains 1,000 thermal images, with approximately 40% outdoor and approximately 60% indoor scenes, and body-pose variation distributed between single-person scenes at approximately 70% and multi-person scenes at approximately 30% [2602.14989].

The ground truth consists of dense 256 × 192 temperature matrices aligned with false-color images. Raw sensor radiance $R(x,y)$ is internally converted by camera firmware to temperature $T(x,y)$ via factory calibration, and an empirical first-order approximation is given for reference as
$$
T(x,y) = a \cdot R(x,y) + b,
$$
where $a,b$ are per-camera calibration constants determined by blackbody reference. Semantic body-part annotations cover four regions of interest: person, forehead, nose, and chest. Three expert annotators independently drew polygon masks for each region per image, periodic consensus meetings resolved edge cases such as glasses and occlusions, and axis-aligned bounding boxes were automatically derived from polygons [2602.14989].

Inter-annotator agreement is reported as mean across all 1,000 images: bounding-box IoU 0.76, segmentation IoU 0.72, bounding-box Dice 0.86, segmentation Dice 0.84, and temperature SD within region 0.18 °C. The released structure is organized as `images/`, `temps/`, and `annotations/`, with `<img_id>.png` false-color thermal images, `<img_id>.npy` NumPy arrays of per-pixel temperature in °C, and `<img_id>.json` COCO-style JSON entries containing `image_id`, `file_name`, `width`, `height`, and annotation objects with `category_id`, `segmentation`, `bbox`, and `area`. The release is described as a Kaggle release under CC BY-NC 4.0 [2602.14989].

## 3. Benchmark role, evaluation tasks, and empirical findings

ThermEval-D supports four key tasks in ThermEval-B: T4 colorbar inference, including detection, localization, and min/max extraction; T5 thermal reasoning, including comparative and within-person ranking; T6 absolute temperature estimation, including coordinate, marker, and region; and T7 temperature estimation at varying distance, specifically 2 ft, 6 ft, and 10 ft [2602.14989]. The benchmark metrics are classification accuracy,
$$
\mathrm{Acc} = \frac{1}{N}\sum_{i=1}^{N} [\hat{y}_i = y_i],
$$
mean absolute error,
$$
\mathrm{MAE} = \frac{1}{N}\sum_{i=1}^{N} |\hat{y}_i - y_i|,
$$
root mean squared error,
$$
\mathrm{RMSE} = \sqrt{\frac{1}{N}\sum_{i=1}^{N} (\hat{y}_i - y_i)^2},
$$
and Intersection over Union for segmentation masks,
$$
\mathrm{IoU} = \frac{|\hat{M} \cap M|}{|\hat{M} \cup M|}.
$$

The reported baseline results emphasize asymmetry across task types. For T4, top models such as Qwen-VL 2.5 and Intern-VL achieve nearly 100% detection and localization accuracy, with MAE approximately 0 °C in min/max extraction, whereas some models such as LLaVA and BLIP-2 exhibit OCR failures including hallucinated scales and decimal-shift errors [2602.14989]. For T5, zero-shot open-source VLMs achieve accuracy approximately 0.35–0.55 on double-person reasoning and 0.18–0.53 on within-person reasoning, compared with human 0.84/0.54; Gemini 3 Pro reaches 0.74/0.61 but remains below full human reliability. The listed failure modes are reliance on canonical body temperature priors and ignoring subtle thermal cues.

For T6 and T7, zero-shot MAE on T6-region for the best open-source model, Intern-VL 38B, is approximately 1.51 °C, compared with human approximately 2.04 °C. Coordinate and marker tasks are harder, with MAE approximately 3–6 °C, and many models default to fixed values such as 37 °C. For T7, top closed-source models achieve MAE 0.74–1.23 °C, compared with human approximately 1.2 °C. Colormap transformations are also diagnostically important: complex maps such as Spring and Summer degrade modality awareness (T2) by up to 30 percentage points in some models. A supervised fine-tuned upper bound, Qwen-VL 2.5 SFT, achieves or exceeds human performance on most tasks, including T5 at 0.58/0.56, T6 at MAE approximately 1.0 °C, and T7 at MAE approximately 0.5–0.6 °C. The paper interprets this as evidence that latent capacity exists, but zero-shot models lack thermal-domain grounding [2602.14989].

The stated applications are night-time surveillance and perimeter security, search and rescue via aerial drones, autonomous driving under low visibility, industrial inspection, and non-contact medical screening. The stated limitations are dataset size of 1,000 images, limited demographic and environmental diversity, availability of only false-color images and no raw radiometric data, annotation scope restricted to four semantic regions, and benchmark complexity limited to foundational tasks. Future work is framed as expansion to more subjects, longer time spans, varied climates, inclusion of sensor radiance, extension to more anatomical landmarks and non-person objects, and incorporation of higher-level reasoning such as anomaly detection and temporal dynamics [2602.14989].

## 4. ThermEval-D as a climate-model diagnostic package

In climate science, ThermEval-D is described as a modular diagnostic package developed within the ESMValTool framework to compute energy, water and entropy metrics from climate-model output, including CMIP5, CMIP6, and other gridded datasets [1902.04669]. Its primary objectives are explicitly process-oriented: quantify imbalances and transports in top-of-atmosphere, atmospheric and surface energy budgets; close the global hydrological budgets; diagnose the Lorenz Energy Cycle of the atmosphere, including available potential energy reservoirs, kinetic-energy reservoirs, conversion rates and dissipation; and estimate material entropy production by indirect and direct methods. By embedding these metrics in ESMValTool v2 and later, the package is intended to allow systematic intercomparison of many models and scenarios under CMIP6 standards.

The architecture consists of four independent modules that can be switched on or off in an ESMValTool recipe or in a stand-alone bash/Makefile version. The energy-budgets and meridional enthalpy-transports module requires monthly means on a full-globe lon–lat grid for `rsdt`, `rsut`, `rlut`, `rsds`, `rsus`, `rlds`, `rlus`, `hfls`, and `hfss`, with an optional land-sea mask to compute separate ocean and land budgets. The hydrological-cycle module requires monthly means of `hfls`, `pr`, and `prsn`, with optional `prw`. The Lorenz Energy Cycle module requires daily or higher resolution three-dimensional fields on pressure levels over 1000–1 hPa for `ua`, `va`, `wap`, and `ta`, and may additionally use `tas`, `uas`, and `vas` when model output omits data below certain pressure levels. The material entropy production module uses the same radiative fields as the energy-budget module for the indirect method, and for the direct method additionally requires `hfls`, `hfss`, `pr`, `prsn`, `tas`, 2 m winds, `ps`, and near-surface humidity `hus` or `huss` [1902.04669].

The implementation section specifies ESMValTool v2.0 or later, Python 3.x, `iris`, optional `cdo`, optional `nco`, and `esmpy` for interpolation. A stand-alone version requires CDO, NCO, NCL, MATLAB or `gfortran` for the Fortran binaries, and uses a master bash script `run_ThermEvalD.sh` to call preprocessing, Fortran LEC, MATLAB MEP routines, and NCL plotting. The prescribed outputs include global 1D time series, 2D maps of energy budgets, water budgets, and MEP components, diagnostic plots such as meridional-transport profiles and LEC diagrams, and NetCDF output files following CF-1.6/CMOR tables for variables and metadata [1902.04669].

## 5. Mathematical formulations and numerical methods in the climate usage

The climate-model ThermEval-D is built around explicit conservation-law and non-equilibrium thermodynamic formulations [1902.04669]. For vertically integrated total energy, neglecting kinetic-energy storage in the atmosphere, the tendency is
$$
\frac{\partial E}{\partial t} = R_t - \nabla \cdot J_t,
$$
with
$$
R_t = S_t^\downarrow - S_t^\uparrow - L_t^\uparrow,
$$
where $J_t$ is meridional enthalpy transport. The zonal-mean decomposition into total, atmosphere, and surface/ocean sub-systems is written as
$$
\dot{\bar{E}}_t(\lambda,\phi) = R_t(\lambda,\phi) - \nabla \cdot J_t(\lambda,\phi),
$$
$$
\dot{\bar{E}}_a(\lambda,\phi) = F_a(\lambda,\phi) - \nabla \cdot J_a(\lambda,\phi),
$$
$$
\dot{\bar{E}}_s(\lambda,\phi) = F_s(\lambda,\phi) - \nabla \cdot J_o(\lambda,\phi),
$$
with
$$
F_a = R_t - F_s,
$$
and
$$
F_s = S_s^\downarrow - S_s^\uparrow + L_s^\downarrow - L_s^\uparrow - (hfls + hfss).
$$
Under steady state, northward transport is computed by integrating the zonal-mean net flux poleward of latitude $\phi$:
$$
T_x(\phi) = 2\pi a^2 \int_\phi^{90^\circ} \cos\phi' \, \langle B_x(\phi') \rangle \, d\phi',
$$
where $B_t = R_t$, $B_a = F_a$, and $B_s = F_s$. When the model has a small global imbalance $\delta$, a corrected budget
$$
\langle B_x \rangle_{\mathrm{corr}}(\phi) = \langle B_x \rangle(\phi) - \frac{\delta}{2\pi a}
$$
is used to avoid spurious cross-polar transports.

For the hydrological cycle, evaporation is derived from latent heat flux as
$$
E = \frac{H_L}{L_v},
$$
with $L_v = 2.5008\times 10^6 \,\mathrm{J/kg}$. Rainfall and snowfall are related by $P_r = P - P_s$, and the stationary mass-budget condition is $\langle E \rangle - \langle P \rangle = 0$. The latent energy budget is given as
$$
\langle R_L \rangle = \langle H_L \rangle - L_v \langle P_r \rangle - (L_v + L_f)\langle P_s \rangle,
$$
with $L_f = 3.34\times 10^5 \,\mathrm{J/kg}$ [1902.04669].

For the Lorenz Energy Cycle, the package defines zonal mean APE, eddy APE, zonal mean KE, and eddy KE reservoirs per unit area:
$$
A_z = \frac{\gamma}{2g}([T]-\{T\})^2,
$$
$$
A_e = \frac{\gamma}{2g}([T^{*2}] + [T'^2]),
$$
$$
K_z = \frac{1}{2g}([u]^2 + [v]^2),
$$
$$
K_e = \frac{1}{2g}([u^{*2}+v^{*2}] + [u'^2+v'^2]),
$$
where $\gamma$ is the static stability parameter, $[\cdot]$ denotes zonal mean, $\{\cdot\}$ global mean, prime denotes time deviation, and star denotes zonal deviation. Conversion terms $C(A_z \to A_e)$, $C(A_e \to K_e)$, $C(K_e \to K_z)$, and $C(A_z \to K_z)$ are computed by spectral decomposition in zonal-wavenumber space. Under steady state, the total mechanical work equals the KE dissipation,
$$
\langle W \rangle = \langle D \rangle = \int_V \rho \,\kappa^2 \, dV,
$$
and $D$ is obtained as the residual of the KE tendency balance [1902.04669].

Material entropy production is implemented by two complementary routes. The indirect method begins from the steady-state entropy-budget equation
$$
\int dV \, \left\langle \frac{\dot{q}_{rad}}{T} \right\rangle + \langle \dot{S}_{mat} \rangle = 0,
$$
and yields
$$
\langle \Sigma_{mat}^{ind} \rangle =
\int_A (\langle S_s + L_s \rangle)\left(\frac{1}{T_s} - \frac{1}{T_E}\right)\, dA
+ \int_A \frac{\langle S_t + L_t \rangle}{T_E}\, dA.
$$
The first term is the vertical MEP and the second the horizontal MEP. The direct method starts from
$$
\dot{\Sigma}_{mat} = \int \left(\frac{\kappa^2}{T_v} + \frac{\phi}{T_\phi}\right)\, dV,
$$
and, after the approximations stated in Sect. 3.4.1 of the paper, arrives at
$$
\dot{\Sigma}_{mat}^{dir} =
\int_A \left[
\frac{\kappa_s^2}{T_d}
- H_S\left(\frac{1}{T_s} - \frac{1}{T_{BL}}\right)
- \frac{L_v E}{T_s}
- \frac{L_v P_r}{T_C}
- \frac{(L_v+L_f)P_s}{T_C}
+ \frac{L_f P_s}{T_s}
- g\,h_{ct}\left(\frac{P_r}{T_p} + \frac{P_s}{T_p}\right)
\right] dA.
$$
Here $T_d = (T_s + T_{2m})/2$, $T_{BL}$ is the boundary-layer top temperature, $T_C$ the cloud characteristic temperature, $T_p$ the mean of $T_C$ and $T_s$, and $h_{ct}$ the cloud-top height. The kinetic-dissipation term $\kappa_s^2$ is obtained by setting $\int \kappa_s^2 dV = W$ [1902.04669].

The numerical implementation uses monthly means for energy and hydrological modules, and daily or better temporal resolution for the LEC and direct-MEP modules. Vertically, it uses pressure-level fields, with missing near-surface levels filled by interpolation from surface data, and it has been tested on standard CMIP 17-level grids over 1000–1 hPa. Horizontally, it uses Fourier series in longitude and Legendre polynomials in latitude to separate zonal versus eddy and planetary versus synoptic scales. Meridional transports are computed from zonal mean flux divergences and then integrated poleward, while kinetic-energy dissipation is computed as the residual of the KE-tendency budget. The code is parallelized over latitudinal bands and time slices for speed [1902.04669].

## 6. Validation, applications, and interpretive significance

The thermal-imagery ThermEval-D is validated primarily through benchmark behavior. Its associated ThermEval-B evaluation shows that current vision-language models can identify thermal modalities but systematically fail at temperature-grounded reasoning and estimation, while degrading under colormap transformations and often defaulting to language priors or fixed responses [2602.14989]. The stated interpretation is that thermal understanding requires dedicated evaluation beyond RGB-centric assumptions and, more specifically, thermal-aware pretraining and specialized model architectures. A plausible implication is that ThermEval-D functions less as a generic image collection than as an instrument for isolating failures in modality grounding, OCR-based scale parsing, comparative reasoning, and absolute temperature estimation.

The climate-model ThermEval-D is validated through demonstrations on a 20-year subset of the CMIP5 CanESM2 pre-industrial run and on a seven-model ensemble under `piControl`, historical, and `RCP8.5` conditions [1902.04669]. The reported analyses include spatial maps of TOA, atmosphere, and surface budgets; meridional transports; water-mass convergence; LEC diagrams; and MEP maps. The ensemble study reports biases in energy budgets, changes in meridional transports, partitioning of APE and KE, stability of LEC intensity, increases in MEP components, baroclinic efficiency decline, and irreversibility rise. The applications explicitly listed are multi-model intercomparison of climate change response, model tuning via process-oriented diagnostics, paleoclimate scenario analysis, and potentially exoplanet climate studies. Ongoing work includes sensitivity to vertical and horizontal resolution, inclusion of full 3D radiative-flux MEP, and isentropic mass-streamfunction diagnostics.

Taken together, the two usages show that “ThermEval-D” does not identify a single methodological lineage. In one case it is a dense, semantically annotated thermal dataset designed to probe temperature-grounded visual reasoning; in the other it is a unified, process-oriented suite of diagnostics for energy, water, and entropy in climate models [2602.14989] [1902.04669]. The shared naming is incidental rather than conceptual.

Source: https://www.emergentmind.com/topics/thermeval-d