MARTIAN: A Rendering Framework for Aerial Mars Imagery from HiRISE Orbital Data

Abstract: Aerial navigation on Mars requires vision-based pipelines that are robust to the diverse illumination conditions and terrain morphology of the Martian surface. A key bottleneck for training and evaluating such methods is the scarcity of large-scale, annotated aerial datasets. We present MARTIAN, an open-source Blender-based rendering framework that leverages real HiRISE orbital map products to synthesize realistic aerial views of the Martian terrain under controllable lighting conditions and at varying altitudes. MARTIAN generates observations with accurate pose annotations, directly addressing the scarcity of training data for vision-based navigation on Mars. The framework has been validated through its deployment in concurrent work on map-based localization systems for Ingenuity and future Mars rotorcraft, where synthetically trained deep image matchers were successfully evaluated on real Mars imagery. MARTIAN is publicly available at: https://github.com/nasa-jpl/martian.

• The paper introduces MARTIAN, a rendering framework that synthesizes high-fidelity aerial Mars imagery from real HiRISE data using Blender 4.0.
• It employs adjustable lighting, camera parameters, and altitude settings to simulate diverse Martian conditions for training robust geo-localization algorithms.
• Validation on platforms like Ingenuity and MSH shows up to 31.8% accuracy improvement, demonstrating effective sim-to-real transfer in planetary navigation.

MARTIAN: A Rendering Framework for Aerial Mars Imagery from HiRISE Orbital Data

Motivation and Context

Precision vision-based navigation in planetary environments, particularly Mars, demands robust data pipelines capable of operating under diverse illumination and terrain variations. Recent advances in planetary aerial vehicles, such as NASA's Ingenuity helicopter and prospective Mars Science Helicopter (MSH) platforms, have underscored the critical need for autonomous geo-localization methods in GNSS-denied conditions. Orbital mapping data, notably from the HiRISE camera aboard MRO, provide high-resolution context but there exists a persistent bottleneck—an absence of large-scale, annotated aerial datasets with pose accuracy and controllable scene parameters tailored for vision-based navigation and localization. The MARTIAN framework directly targets this deficiency by synthesizing aerial Mars observations using real HiRISE map products, producing scalable, annotated datasets for learning-based perception pipelines.

Technical Architecture

MARTIAN leverages Blender 4.0 as a rendering backend, employing a Python-based interface for ingesting HiRISE Digital Terrain Models (DTMs) and ortho-projected images. The primary region of focus is Jezero Crater, site of the Mars2020 landing, utilizing DTMs at 1 m/post resolution and ortho-images at 0.25 m/pixel. Terrain modeling proceeds via a UV-mapped mesh creation, with textures precisely draped to preserve spatial fidelity via explicit metadata and georeferencing.

Scene configuration supports both perspective and orthographic imaging, with user-defined camera intrinsics and extrinsics aligned to Mars-centric ENU coordinate frames. Camera placement above terrain exploits ray-tracing with a Bounding Volume Hierarchy for efficient geometric queries, and full pose annotations ($\mathbf{R}_{WC} \vert \mathbf{t}_{WC}$) are output for each synthetic observation.

Lighting control is a core feature: Sun illumination is parametrized by irradiance, disk diameter, and orientation (EL, AZ), simulating local Martian time-dependent effects. Rendering employs the Blender Cycles engine for physically accurate light/shadow modeling, enabling direct simulation of diverse diurnal conditions.

Figure 1: Perspective rendering of Jezero Crater in MARTIAN with a simulated camera and synthetic observation.

Dataset Generation and Annotation

MARTIAN facilitates large-scale data generation for supervised training and evaluation of navigation algorithms. The platform produces orthographic grayscale maps and depth maps using combinations of Sun azimuth ($0^\circ$–$360^\circ$, $45^\circ$ increments) and elevations ($30^\circ$, $60^\circ$, $90^\circ$), as well as thousands of nadir-pointing aerial images sampled across the HiRISE DTM, spanning altitudes from 64 to 200 meters. All observations are annotated with ground truth pose and altitude, critical for downstream localization benchmarking.

Figure 2: MARTIAN renderings demonstrating orthographic maps and depth maps at varying local Martian times and lighting conditions.

Validation in Map-Based Localization Pipelines

The synthetic datasets enabled by MARTIAN have been pivotal for training and validating deep image matching pipelines, notably LoFTR and its geometry-aided variant Geo-LoFTR. In concurrent evaluations:

• Ingenuity Helicopter MbL: LoFTR models pretrained on MARTIAN were fine-tuned on limited real Ingenuity navigation images. Intermediate synthetic pretraining reduced illumination- and scale-related error, with numerical results showing an 11.2% improvement in Acc@5m compared to direct fine-tuning and up to 26.4% gains on difficult terrain. The final pipeline achieved 89.4% Acc@5m and 99.8% Acc@10m, surpassing template-matching and hand-crafted approaches and generalizing to new flights with minimal real data (2605.29647).
• MSH and Rotorcraft Localization: Geo-LoFTR, incorporating HiRISE-derived geometric context, was trained on MARTIAN and validated up to 200 m altitude under severe illumination shifts. Performance showed up to 31.8% improvement in Acc@1m compared with prior methods, with robustness across the full simulation diurnal cycle at Jezero Crater and consistent matching across scale and lighting regimes. Transfer validation on Mars2020 descent imagery (LCAM vs CTX) confirmed effective synthetic-to-real adaptation, maintaining accurate global localization over altitude ranges from 6 km to 960 m.

  Figure 3: Geo-LoFTR matches between CTX map crops and Mars2020 LCAM descent observations at multiple altitudes, demonstrating sim-to-real transfer.

Practical Implications and Limitations

MARTIAN positions itself as a critical enabler for advancing vision-based autonomous navigation in planetary exploration. The capability to generate high-fidelity, annotated aerial datasets under varied lighting and altitude conditions is indispensable for benchmarking and development of deep learning-based geo-localization, particularly as reliance on GNSS is impossible and mission operational ranges expand.

However, the framework inherits limitations from HiRISE input data: textures are captured at fixed time-of-day, restricting full decoupling of embedded illumination and shadow properties in rendered outputs. This constrains certain types of lighting invariance studies. Additionally, broader sim-to-real generalization across Mars landing sites and alternative map sources requires further empirical exploration.

Theoretical Implications and Future Directions

From a theoretical standpoint, MARTIAN enables quantitative evaluation of cross-domain transfer for deep models—simulated to real imagery, varying scale, and illumination shifts. The framework supports research into learning geometric context from DTMs, lighting invariance, and robust feature correspondence under planetary constraints. Future integration of more dynamic environmental models (e.g., atmospheric effects, multiple map layers) and multi-modal sensor simulation (thermal, multispectral) will further catalyze algorithm development for next-generation rotorcraft, landers, and autonomous surface vehicles.

Conclusion

MARTIAN represents a tailored Blender-based rendering and dataset generation framework built atop real HiRISE data, filling a critical gap for annotated aerial Mars imagery in vision-based navigation and localization research. Its utility has been empirically demonstrated in map-based localization pipelines for both Ingenuity and concept Mars rotorcraft, providing strong numerical evidence for improved deep matcher performance and synthetic-to-real transfer. The public release of MARTIAN promises to accelerate research in robust planetary perception, although additional validation and extension across broader Martian geography and sensor modalities will be essential for full operational generalization (2605.29647).