---
title: 'IRisPath: Thermal–RGB Fusion for Off-road Navigation'
url: https://www.emergentmind.com/topics/irispath
type: topic
---

# IRisPath: Thermal–RGB Fusion for Off-road Navigation

IRisPath denotes a multimodal off-road navigation framework centered on thermal–RGB traversability estimation, day–night dataset release, and targetless Thermal–LiDAR–RGB extrinsic calibration for autonomous operation under dynamic light and weather conditions. In the cited literature, the term is used explicitly for a 2024 system that couples long-wave infrared sensing, visible-spectrum imagery, kinesthetic self-supervision, and costmap generation for off-road robots, with the stated aim of improving robustness against out-of-domain uncertainty in agriculture, construction, search and rescue, and defence scenarios [2412.03173].

## 1. Problem setting and scope

IRisPath is motivated by the gap between structured on-road autonomy and unstructured off-road mobility. Off-road terrain is heterogeneous, with dirt, gravel, rocks, grass, shrubs, ditches, and roots, and traversability depends not only on scene appearance but also on vehicle properties and speed. Classical semantic-segmentation and occupancy-grid pipelines can overestimate obstacles, while LiDAR-only geometric maps still fail to capture how the vehicle actually experiences terrain. Recent self-supervised traversability methods improve this by regressing ride cost from visual input and kinesthetic feedback, but they remain vulnerable to out-of-domain uncertainty under night, fog, dust, rain, and similar appearance shifts [2412.03173].

The core IRisPath response is multimodal redundancy. RGB provides detailed semantic cues in daytime, whereas long-wave infrared remains informative in darkness and is less sensitive to lighting changes. The framework therefore combines RGB and thermal imagery to produce a traversability costmap that remains useful across day and night, and supplements this with a targetless calibration pipeline so that the sensing modalities can be co-registered for fusion [2412.03173].

## 2. Dataset, sensor suite, and calibration stack

IRisPath refers both to a dataset and to a fusion model. The dataset was collected on the Copernicus all-terrain robot and includes LWIR images, RGB images, multiple LiDARs, IMU, GNSS, stereo, wheel odometry, day and night sequences, and pseudo-labels for traversability. The data were also converted to KITTI and nuScenes formats, and post-processed with FastLIO2 plus Scan-Context++ using Livox Mid360 and with LIO-SAM using Velodyne plus GNSS [2412.03173].

The sensing stack includes a FLIR Hadron thermal camera at \(640 \times 512\) on `/ir_cam/image_raw` at 24 Hz, a FLIR BlackFly S RGB camera at \(2048 \times 1536\) on `/camera/image_color` at 10 Hz, a ZED-2 stereo camera at \(1920 \times 1080\), Livox Mid360 and Livox HAP LiDARs, a Velodyne VLP-32C, an Xsense MTI-680G IMU at 350 Hz, GNSS, and wheel RPM sensing [2412.03173].

A distinctive component is the targetless extrinsic calibration method. Rather than relying on a thermal-visible checkerboard, IRisPath estimates \(^{rgb}T_{li}\) and \(^{ir}T_{li}\) independently and composes them as

$$
{}^{rgb}T_{ir} = {}^{rgb}T_{li}\,\big({}^{ir}T_{li}\big)^{-1}.
$$

The underlying optimization is stated as

$$
\arg \min_{{}^{rgb}T_{li}} \sum_{i=1}^{N} \left\| w_i - {}^{rgb}T_{li} \big(\pi^{-1}(x_{rgb,i})\big) \right\|^2,
$$

with feature matching between LiDAR intensity images and RGB or IR imagery via SuperGlue or ORB and robust fitting with RANSAC. The reported calibration accuracy is approximately \(\pm 1.7\,\text{cm}\) in translation and \(\pm 0.827^\circ\) in rotation [2412.03173].

## 3. Fusion model and costmap construction

The model, described in the detailed reconstruction as FuseIsPath, operates on co-registered RGB and IR image patches plus vehicle speed. First, the IR image is warped into the RGB frame through the RGB–IR calibration. The image is then subdivided into patches of size \(i \times i\) with stride \(s\), yielding

$$
N_{\text{patches}}=
\left\lfloor \frac{w-i}{s}+1 \right\rfloor
\times
\left\lfloor \frac{h-i}{s}+1 \right\rfloor.
$$

Vehicle speed is encoded from the planar velocity norm

$$
v = \sqrt{V_x^2 + V_y^2}
$$

using Fourier features

$$
\gamma(v)=\big[\cos(2\pi B v),\,\sin(2\pi B v)\big]^T,
$$

where \(B\) is sampled from \(\mathcal{N}(0,\sigma^2)\) [2412.03173].

Architecturally, IRisPath uses two separate ResNet-18 encoders, one for RGB and one for IR, with no weight sharing. The Fourier-encoded velocity is passed through an MLP, and the three feature streams are concatenated:

$$
f_{\text{fused}}=\text{concat}(f_{\text{RGB}}, f_{\text{IR}}, f_v).
$$

A prediction head then outputs a scalar traversability cost per patch:

$$
y_{\text{patch}}=\text{MLP}_{\text{head}}(f_{\text{fused}}).
$$

These patch-level predictions are placed back into a 2D grid and resized to form an image-aligned costmap that can be consumed by a downstream planner [2412.03173].

## 4. Self-supervised traversability target and robustness mechanisms

IRisPath is self-supervised rather than manually annotated. Its patch labels are derived from the vertical acceleration signal measured while the vehicle traverses terrain, using the power spectral density of the IMU \(z\)-axis acceleration normalized by speed:

$$
y = \frac{\text{PSD}(acc_z)}{\sqrt{V_x^2 + V_y^2} + 10}.
$$

This cost is intended to encode ride roughness and speed dependence, so that the same terrain can become more expensive at higher speed. Training is then cast as scalar regression, conceptually using mean squared error between predicted and target costs [2412.03173].

Robustness is pursued through three mechanisms. The first is modality fusion itself: RGB is informative in daytime, while thermal remains informative at night and under degraded visibility. The second is explicit speed conditioning through Fourier features, which helps model non-linear speed dependence. The third is optional test-time adaptation. The paper evaluates TENT and MoASE under synthetic fog, rain, and snow in day and night conditions. Reported costmap errors decrease relative to the source model in all six conditions, with MoASE generally best; for example, SnowNight decreases from 0.156 to 0.109 with TENT and to 0.105 with MoASE, while FogNight decreases from 0.112 to 0.110 and 0.104, respectively [2412.03173].

## 5. Empirical behavior, costmap semantics, and deployment

Qualitative evaluation emphasizes that IR-only inference remains informative at night, whereas RGB-only inference degrades sharply; fusion recovers road structure at night and produces richer costmaps in daytime by combining semantic detail from RGB with robust structure from IR [2412.03173].

The learned costmap is speed-sensitive. At low speed, grass and small roughness appear as moderate cost. At high speed, grass becomes substantially more expensive, while minor hard-terrain roughness may be penalized less severely. The reported ablations also indicate that reducing or removing Fourier features makes the costmap less structured and less consistent with speed variation [2412.03173].

The output is designed for integration into standard navigation stacks. The paper positions the costmap as a layer compatible with ROS `costmap_2d` or Nav2, after projection from image coordinates into robot coordinates using LiDAR or stereo depth. It also states that dataset and code are to be available at `https://github.com/codeck313/IRisPath`, including raw rosbags, KITTI and nuScenes formatted data, calibration files, baseline models, and evaluation scripts [2412.03173].

The principal limitations are also explicit: no formal uncertainty estimation, simple late fusion rather than attention-based or transformer fusion, a single geographic collection site, synthetic rather than real adverse-weather evaluation, and limited discussion of synchronization and latency [2412.03173].

## 6. Terminological scope and related uses

Among the cited sources, the title “IRisPath” appears explicitly in the off-road navigation paper just described [2412.03173]. The same capitalization is not stable across other domains. In digital pathology, “Iris” names a rendering engine, a file format, and a RESTful serving ecosystem rather than an IRisPath-named method [2504.15437] [2506.10009] [2508.06615]. In ocular biometrics and monocular depth estimation, several detailed summaries explicitly state that “IRisPath” does not appear in the original paper or is only an interpretive extension applied to a deformation path, pathology pathway, or priors-to-geometry schedule, rather than the formal name of the method itself [2207.08980] [2607.04344] [2208.03138] [2603.16340].

This makes IRisPath, in strict bibliographic usage, primarily an autonomous off-road traversability framework built around thermal–RGB fusion, self-supervised costmap learning, and targetless calibration. Broader uses of the string in adjacent literatures are best treated as non-canonical reinterpretations rather than as a single cross-domain technical lineage [2412.03173].

Source: https://www.emergentmind.com/topics/irispath