MoonAnything: A Vision Benchmark with Large-Scale Lunar Supervised Data

Abstract: Accurate perception of lunar surfaces is critical for modern lunar exploration missions. However, developing robust learning-based perception systems is hindered by the lack of datasets that provide both geometric and photometric supervision. Existing lunar datasets typically lack either geometric ground truth, photometric realism, illumination diversity, or large-scale coverage. In this paper, we introduce MoonAnything, a unified benchmark built on real lunar topography with physically-based rendering, providing the first comprehensive geometric and photometric supervision under diverse illumination with large scale. The benchmark comprises two complementary sub-datasets : i) LunarGeo provides stereo images with corresponding dense depth maps and camera calibration enabling 3D reconstruction and pose estimation; ii) LunarPhoto provides photorealistic images using a spatially-varying BRDF model, along with multi-illumination renderings under real solar configurations, enabling reflectance estimation and illumination-robust perception. Together, these datasets offer over 130K samples with comprehensive supervision. Beyond lunar applications, MoonAnything offers a unique setting and challenging testbed for algorithms under low-textured, high-contrast conditions and applies to other airless celestial bodies and could generalize beyond. We establish baselines using state-of-the-art methods and release the complete dataset along with generation tools to support community extension: https://github.com/clementinegrethen/MoonAnything.

• The paper introduces MoonAnything, a unified benchmark that combines over 130,000 samples for supervised lunar perception using both geometric and photometric data.
• It integrates two sub-datasets—LunarGeo and LunarPhoto—to support tasks like 3D reconstruction, pose estimation, and reflectance analysis under extreme illumination and low-texture conditions.
• Experimental results reveal significant reductions in reconstruction errors, validating the dataset's effectiveness in generalizing across diverse lunar terrains and mission-specific imaging scenarios.

MoonAnything: A Comprehensive Vision Benchmark for Large-Scale Supervised Lunar Data

Motivation and Problem Statement

Robust computer vision for lunar exploration is limited by the absence of large-scale, physically-realistic, and task-diverse datasets that provide both geometric and photometric supervision. Lunar surface perception demands 3D understanding and illumination-robust modeling under extreme contrast, weak texture, and substantial domain shifts from terrestrial imagery. Unlike the Earth-centric datasets fueling progress in geometric deep learning, existing lunar datasets are partitioned, either focused on geometry without realistic appearance or with photometric cues but lacking dense geometry. Several high-resolution orbital and surface datasets exist, yet none offer comprehensive annotations for end-to-end supervised learning, especially under varying illuminations.

Benchmark Construction

MoonAnything is constructed as a unified benchmark supporting both geometric and photometric perception. It is built upon real lunar topography and incorporates physically-based rendering to generate both stereo image pairs (for dense geometry) and multi-illumination photorealistic images (for material reflectance and robustness evaluation), systematically filling critical gaps in all previous datasets.

Figure 1: Overview of the MoonAnything dataset generation pipeline, highlighting integration of lunar DEMs, orbital imagery, SVBRDF, and rendering for both geometry and appearance data.

MoonAnything consists of two complementary sub-datasets:

• LunarGeo: Delivers 58,000 stereo image pairs (South Pole, Tycho regions) with dense, metrically-accurate depth maps and explicit camera calibrations, supporting 3D reconstruction, pose estimation, and stereo vision under realistic lunar descent trajectories. Scenes are rendered using Hapke BRDF and spatially-varying BRDF (SVBRDF) models for regional realism.
• LunarPhoto: Comprises 84,000 patches with real high-resolution orbital images (orthorectified LRO NAC), DEM-aligned geometry, learned SVBRDFs, and multi-illumination (9 Sun positions per sample) physically-based renderings. These enable reflectance estimation, normal estimation via photometric stereo, and intrinsic decomposition experiments.

  Figure 2: Examples from LunarGeo showing stereo pairs and their 3D scene renderings for nadir, oblique, and dynamic trajectories in both South Pole and Tycho regions.

MoonAnything exceeds precedent with over 130,000 samples and more than 750,000 rendered images, spanning two key lunar environments: the South Pole (target of upcoming Artemis missions with permanent low-angle sunlight and extreme shadowing) and Tycho crater (mid-latitude, diverse geomorphology).

Figure 3: Samples from LunarPhoto, including a real LRO NAC image, DEM crop, depth map, normal map, per-pixel SVBRDF parameters, and multi-illumination renderings.

Dataset Characteristics and Challenges

MoonAnything purposely comprises scenarios characterized by:

• Low-texture, high-contrast imagery derived from physically-based rendering.
• Challenging illumination: extreme shadow boundaries, large cast shadows, and high intra-scene contrast.
• Regional reflectance variation through per-pixel learned SVBRDF maps capturing complex lunar regolith properties, including opposition effect and anisotropy.
• Acquisition geometry mimicking actual mission profiles (nadir, oblique, dynamic) with varying baselines, altitudes, and view angles across real lunar terrain.

  Figure 4: Challenging cases including harsh shadows, raking illumination, and nearly textureless flat regions as seen in both geometric and photometric data, exemplifying scenarios difficult for conventional feature-based and model-based methods.

These properties make MoonAnything not only critical for lunar robotics but a valuable testbed for multimodal 3D vision, robust feature learning, and methods generalization to other airless planetary bodies or texture-poor domains.

Data Generation Pipeline

• Geometry: DEMs (resolution up to 1 m/pixel for Tycho) are derived from NASA’s LRO LOLA and stereo-matched imaging products, providing topographically accurate ground truth.
• Rendering: SurRender engine is used to simulate physically consistent images driven by Hapke-based reflectance (both constant and SVBRDF), spatially referencing camera and Sun geometry with selenographic accuracy.
• BRDFs: The Hapke model and its SVBRDF extension account for the photometric opposition effect and surface scattering diversity, with parameters learned from real multi-angle LRO NAC imagery.
• Camera and Illumination: Cameras are sampled according to mission-inspired descent and flyover trajectories with realistic FOV, baseline, and GSD; illumination is systematically varied via solar position sampling from the NASA SPICE toolkit, ensuring that all rendered conditions are achievable in real lunar contexts.

Experimental Baselines and Results

MoonAnything is used to fine-tune recent state-of-the-art stereo 3D reconstruction methods (MASt3R [leroy2024mast3r] and VGGT [wang2025vggt]). Quantitative evaluation on both seen (South Pole) and unseen (Tycho) regions with nadir, oblique, and dynamic trajectories reveals:

• Fine-tuned models achieve an order of magnitude reduction in reconstruction errors (accuracy, completeness, Chamfer distance) compared to initial models trained only on terrestrial or synthetic non-lunar domains.
• Notably, generalization to unseen lunar regions is strong when models are fine-tuned with only a subset of MoonAnything, validating the realism and diversity of the dataset.

The task difficulty is dominated by solar geometry, shadows, and terrain structure, which in MoonAnything are independently controllable and well-annotated.

Implications and Future Developments

MoonAnything constitutes the first large-scale, realistic, unified dataset for lunar geometric and photometric vision. Its key implications are:

• For algorithmic development: Enables fully supervised training and benchmarking of stereo, depth, reflectance, and photometric stereo methods where both geometry and reflectance are explicitly annotated under variable illumination.
• For mission design: Provides a simulation testbed enabling robust evaluation of terrain-relative navigation, hazard detection, and landing site selection under lunar-realistic constraints.
• For generalization research: Offers a benchmark for studying transfer learning, domain adaptation, and the robustness of SOTA geometric transformers and neural radiance field methods under high-contrast, low-texture, and domain-shifted conditions.
• For surface appearance modeling: Advances learning-based BRDF and intrinsic decomposition approaches for airless bodies, aligning with planetary science requirements.

Practically, the benchmark may be extended both by the authors and the community (with released code and generation tools) to novel geographic locales, sensor geometries, higher-resolution data, or multi-modal sensor combinations (e.g., fusion with event cameras or LiDAR).

Conclusion

MoonAnything (2604.00682) decisively advances lunar computer vision research by providing the first unified, large-scale, physically-real benchmark simultaneously enabling geometric and photometric supervision. It supports training and fair comparison of algorithms across stereo 3D vision, material reflectance estimation, and illumination-robust perception, directly facilitating the development of algorithms essential for future lunar and planetary exploration. Its structure and content ensure adaptability to broader computer vision challenges, with immediate application in robust perception for autonomous systems operating in extreme and under-constrained environments.