Mars-Bench: Multi-Domain Evaluation

• Mars-Bench is a multi-use benchmark designation that spans stereo reconstruction, radiation modeling, vision-language retrieval, and even dialogue evaluation.
• It employs rigorous evaluation metrics—including disparity error, IoU, mAP, and photometric reprojection—to compare methodologies across diverse Mars science tasks.
• The benchmark stresses the need for clear disambiguation in technical literature due to its common use across distinct domains such as planetary imaging and dialogue systems.

Searching arXiv for the term and associated benchmarks to ground the article in the cited literature. Mars-Bench is a reused designation in recent arXiv literature rather than a single canonical benchmark. In Mars science, the label has been attached to a stereo-reconstruction benchmark for 3D printable terrain models, a retrieval-centric benchmark for Mars geospatial discovery, a broad benchmark for evaluating foundation models on Mars imagery, and a technical report documenting an AtRIS/GEANT4 implementation for the Martian radiation environment. Outside planetary science, the closely related title “MARS-Bench” denotes a multi-turn athletic dialogue benchmark. The term therefore identifies multiple unrelated evaluation frameworks whose commonality is benchmarking, not a shared dataset, task definition, or methodology (Wang, 9 Jun 2026, Wang et al., 15 Feb 2026, Purohit et al., 28 Oct 2025, Guo et al., 2019, Yang et al., 27 May 2025).

1. Nomenclature and major usages

The principal usages of the name span stereo geometry, particle transport, multimodal retrieval, and foundation-model evaluation. The overlap in naming is substantive enough that disambiguation is necessary in technical writing, especially because several of the benchmarks are Mars-specific while one is not.

Variant in the literature	Domain	Core scope
“Benchmarking stereo reconstruction for 3D printable Martian terrain models” (Wang, 9 Jun 2026)	Stereo geometry and mesh completion	Curiosity stereo depth, geometry completion, watertight OBJ export
Mars-Bench Technical Report in the AtRIS/GEANT4 Mars study (Guo et al., 2019)	Radiation transport	Martian atmosphere/regolith parameterization, ARM folding, RAD validation
MarsRetrieval (“Mars-Bench”) (Wang et al., 15 Feb 2026)	Vision-language retrieval	Paired image-text retrieval, landform retrieval, global geo-localization
“Mars-Bench: A Benchmark for Evaluating Foundation Models for Mars Science Tasks” (Purohit et al., 28 Oct 2025)	Mars computer vision	20 datasets across classification, segmentation, and object detection
“MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation” (Yang et al., 27 May 2025)	Dialogue evaluation	Ultra Multi-turn, Interactive Multi-turn, and Cross-turn Tasks

A plausible implication is that “Mars-Bench” now functions as a namespace collision across subfields. In planetary ML papers, the term usually refers either to MarsRetrieval or to the 20-dataset foundation-model suite; in geometry and radiation contexts, it denotes entirely different benchmark constructions.

2. Stereo reconstruction benchmark for printable Martian terrain

In Wang et al., Mars-Bench denotes an end-to-end pipeline for reconstructing printable 3D models from Mars rover stereo imagery. The workflow starts from a left/right stereo pair and proceeds through disparity estimation, back-projection to a partial point cloud, geometry completion, and mesh repair plus voxelization into a watertight, printable OBJ. Stereo depth is estimated either with OpenCV’s implementation of Hirschmüller’s SGM using block size 7, 96 disparity levels, speckle filtering, three-way matching, and weighted least-squares post-filtering, or with RAFT-Stereo run for 32 updates on a Middlebury-trained checkpoint, with grayscale→RGB conversion and output flow negated to yield disparity (Wang, 9 Jun 2026).

The completion stage compares three strategies. Adaptive Alpha Shapes carve a surface from the Delaunay triangulation using

$$\alpha=\lambda_\alpha \cdot \mathrm{diag}(\mathcal{B}),\qquad \lambda_\alpha=0.02.$$

Poisson Surface Reconstruction solves an implicit surface at octree depth 7 and scale 1.2, with normals estimated and oriented toward the camera. The deterministic diffusion-fill baseline iteratively in-paints the depth grid while anchoring observed stereo depths and adding small, fixed height perturbations in unseen regions. A “hybrid” variant concatenates alpha-shape and Poisson meshes before repair.

The benchmark defines stereo metrics separately for Curiosity imagery and Middlebury. For Curiosity, the reported quantities include valid disparity coverage

$$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$

valid depth coverage, photometric reprojection error

$$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$

edge-alignment $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$, median disparity gradient, and the median valid disparity and depth. For Middlebury, the benchmark uses valid-prediction ratio, MAE, RMSE, and bad-pixel rates.

On Middlebury, RAFT-Stereo outperforms SGBM across all reported metrics: Valid rises from $0.763$ to $1.000$, MAE drops from $3.224$ px to $0.729$ px, RMSE from $9.892$ px to $3.049$ px, Bad-1 from $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$0 to $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$1, and Bad-4 from $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$2 to $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$3. On Curiosity imagery, however, the transfer is adverse on several domain-specific diagnostics. The reported table gives SGBM values of Disp. $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$4, Depth $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$5, Reproj. $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$6, Photo $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$7, and Edge $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$8, whereas RAFT gives $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$9, $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$0, $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$1, $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$2, and $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$3. Median disparity changes from $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$4 px to $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$5 px, median depth from $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$6 m to $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$7 m, and disparity gradient from $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$8 to $$E_{\mathrm{photo}}=\frac{1}{|\Omega_{\mathrm{reproj}}|}\sum_{(u,v)\in\Omega_{\mathrm{reproj}}}\left|I_L(u,v)-I_R(u-d(u,v),v)\right|,$$9. The paper interprets this as a transfer gap: benchmark accuracy on high-texture terrestrial data does not directly transfer to Martian terrain reconstruction.

The geometry-completion results sharpen the fidelity-versus-connectivity trade-off. Alpha shapes attain the lowest normalized Chamfer distance—SGBM: $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$0, RAFT: $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$1—and high visible coverage, but can fragment severely: RAFT-alpha yields approximately $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$2 connected components with largest-component fraction only $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$3, while SGBM-alpha has $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$4. Poisson reconstruction yields coherent meshes—RAFT-poisson has connected components approximately $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$5 and $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$6; SGBM-poisson has connected components approximately $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$7 and $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$8—but normalized Chamfer rises to $$A_{\mathrm{edge}}=\mathrm{corr}(\|\nabla I_L\|,\|\nabla d\|)$$9 for RAFT and $0.763$0 for SGBM, and novelty increases, including approximately $0.763$1 for RAFT. Deterministic diffusion-fill is intermediate for RAFT inputs, with Chamfer $0.763$2, Coverage $0.763$3, Novelty $0.763$4, connected components $0.763$5, and $0.763$6, but it degrades on SGBM inputs, where Chamfer becomes $0.763$7 and Coverage $0.763$8. Synthetic-ground-truth ablation preserves the same dichotomy: alpha shapes fragment at approximately $0.763$9 connected components, diffusion-fill attains the best Chamfer of $1.000$0 and coverage of $1.000$1, and Poisson remains fully connected while oversmoothing, with novelty $1.000$2.

The paper’s main methodological conclusion is that planetary stereo should not be validated only by conventional disparity error. It recommends edge alignment, photometric reprojection consistency, and metric-depth statistics; separation of directly observed geometry from inferred surfaces; and the use of synthetic Mars renderings, multi-view rover sequences, or controlled physical scans for evaluation. It also recommends clear labeling of reconstructed versus unobserved regions when preparing printable models.

3. Mars-Bench as an AtRIS/GEANT4 radiation benchmark

In the AtRIS study, “Mars-Bench” refers to a technical report documenting a reproducible implementation of the Atmospheric Radiation Interaction Simulator with GEANT4 for the Martian surface radiation environment. The planetary geometry is a sphere of radius $1.000$3, with an atmosphere extending to $1.000$4 and subdivided into $1.000$5 layers, plus a $1.000$6 thick regolith shell of density $1.000$7. Atmospheric composition and vertical profiles are taken from the Mars Climate Database “clim aveEUV” scenario at Gale Crater $1.000$8 for solar longitude $1.000$9, with surface pressure approximately $3.224$0 and surface column depth approximately $3.224$1. The regolith composition is modeled with mass fractions Si $3.224$2, O $3.224$3, and Fe $3.224$4 (Guo et al., 2019).

Four compound GEANT4 physics lists are benchmarked: QGSP_BIC_HP, QGSP_BERT_HP, FTFP_BERT_HP, and FTFP_INCLXX_HP. The preferred list is Model D, FTFP_INCLXX_HP, because it produces significantly better agreement with MSL/RAD measurements of secondary deuterons, tritons, and He³ in the $3.224$5–$3.224$6 range, owing to its treatment of spallation through the INCL++ cascade.

The formalism is matrix-based. For each primary type $3.224$7 and secondary type $3.224$8, AtRIS computes raw histograms $3.224$9, normalizes them as

$0.729$0

scales them through the geometry and solid angle factors to obtain $0.729$1, and folds them with an external spectrum $0.729$2:

$0.729$3

The reported configuration uses BON10 GCR protons and He⁴ at modulation $0.729$4, primary energies from $0.729$5 to $0.729$6 in 50 logarithmic bins, isotropic injection from the top sphere, and three secondary-counting cases: downward full hemisphere, upward full hemisphere, and RAD view cone with $0.729$7 and $0.729$8.

Quantitatively, all four models predict surface proton and He⁴ spectra within $0.729$9 of MSL/RAD measurements in the $9.892$0–$9.892$1 interval, aside from a slight underprediction of approximately $9.892$2–$9.892$3 attributed to omission of heavier primaries. Models A–C underpredict deuteron, triton, and He³ by factors of $9.892$4–$9.892$5, whereas Model D matches RAD within $9.892$6 across all three species. In the RAD view cone, the sample comparison reports proton flux $9.892$7 for RAD and $9.892$8 for Model D; deuteron $9.892$9 for RAD and $3.049$0 for Model D; triton $3.049$1 for RAD and $3.049$2 for Model D; He³ $3.049$3 for RAD and $3.049$4 for Model D; and He⁴ $3.049$5 for RAD and $3.049$6 for Model D. Downward-hemisphere and RAD-cone spectra agree within $3.049$7, while upward fluxes are $3.049$8–$3.049$9 orders of magnitude lower and are dominated by regolith albedo.

The report emphasizes reproducibility and recommended practice: use FTFP_INCLXX_HP when spallation-heavy secondaries matter, maintain at least $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$00 primaries per log-energy bin, match MCD season and latitude to the site of interest, and fold with ICRU-sphere response matrices when converting flux to dose. This is a physically grounded use of the Mars-Bench name that is unrelated to the image benchmarks.

4. MarsRetrieval (“Mars-Bench”) and retrieval-centric planetary discovery

MarsRetrieval defines “Mars-Bench” as a retrieval-centric benchmark for evaluating vision-LLMs in language-guided scientific discovery on Mars at multiple spatial scales. The benchmark contains three tasks. Paired Image–Text Retrieval uses $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$01 expert-validated image/caption pairs spanning scales from global orbital mosaics to rover-level micrometer imagery. Landform Retrieval uses $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$02 image patches organized into $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$03 geomorphic sub-classes under $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$04 major genetic classes. Global Geo-Localization evaluates retrieval against a global gallery of approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$05 million CTX tiles using five catalogued landforms: Alluvial Fans, Glacier-Like Forms, Landslides, Pitted Cones, and Yardangs. All three tasks are evaluated zero-shot, with no train/val/test split for Tasks 1 and 2, and zero-shot text→image and image→image evaluation for Task 3 (Wang et al., 15 Feb 2026).

The benchmark adopts a unified protocol. An encoder $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$06 maps image or text inputs to a $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$07-dimensional Euclidean embedding, similarity is defined as $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$08, and candidates are ranked by descending similarity. The principal metrics are Recall@K, Mean Reciprocal Rank, Median Rank, mAP for multi-positive retrieval, nDCG@10 for Task 2, and AUPRC plus Optimal F1 for point-set matching in Task 3. For geo-localization, a retrieved tile centered at $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$09 is counted as a true positive when its great-circle distance to a catalogued point is at most $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$10.

The benchmark compares contrastive dual-tower encoders and generative or MLLM-based embedding models. In the dual-tower setting, image encoder $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$11 and text encoder $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$12 are trained with an InfoNCE objective, and inference uses $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$13-normalized embeddings with cosine similarity. The reported example families for generative embeddings are E5-V, GME, and Qwen3-VL-Embedding.

Task 1 shows that encoder-based VLMs outperform MLLM-based models. For PE-Core-L-14-336, Text→Image performance is approximately R@1 $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$14, R@10 $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$15, MRR $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$16, MedR $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$17, while Image→Text performance is approximately R@1 $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$18, R@10 $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$19, MRR $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$20, MedR $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$21. Domain-specific fine-tuning through MarScope on CLIP-DFN2B improves R@10 by $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$22 for Text→Image and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$23 for Image→Text. Caption refinement yields consistent MRR gains, including $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$24 for Qwen3-VL and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$25 for Ops-MM, and scaling PE-Core from B16 to L14 to G14 improves MRR by approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$26–$$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$27 points.

Task 2 demonstrates a much larger domain-adaptation effect. PE-Core-L-14 reaches mAP approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$28, nDCG@10 approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$29, and Hits@10 approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$30. MarScope reaches mAP approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$31, nDCG@10 approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$32, and Hits@10 approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$33, a $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$34 absolute gain in mAP. Prompt-ensemble with three templates adds $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$35 mAP for MarScope, while general models saturate after approximately three prompts.

Task 3 shows marked class dependence. For text-based MarScope retrieval, AUPRC is $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$36 on Alluvial Fans, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$37 on Glacier-Like Forms, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$38 on Landslides, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$39 on Pitted Cones, and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$40 on Yardangs. Image-based AUPRC is systematically lower, including $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$41 on Pitted Cones and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$42 on Yardangs. Optimal F1@K* for MarScope text queries is $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$43 for Alluvial Fans, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$44 for Glacier-Like Forms, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$45 for Landslides, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$46 for Pitted Cones, and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$47 for Yardangs. The paper states that general VLMs achieve AUPRC below $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$48 on most classes, while DINOv3-Vit-L/16 can reach up to $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$49 AUPRC on Yardangs but lacks language alignment. The stated conclusion is that domain-specific fine-tuning is critical for generalizable geospatial discovery in planetary settings.

5. Mars-Bench as a benchmark suite for Mars foundation models

A different Mars-Bench is the benchmark suite for evaluating foundation models on Mars science tasks. It comprises $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$50 datasets spanning classification, semantic segmentation, and object detection, drawn from $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$51 orbiters, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$52 rovers, and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$53 sensors. The task split is classification $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$54, segmentation $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$55, and object detection $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$56, with standardized train/validation/test splits across datasets. Representative classification datasets include mb-atmospheric_dust_cls_edr, mb-change_cls_ctx, mb-change_cls_hirise, mb-domars16k, mb-frost_cls, mb-landmark_cls, mb-surface_cls, and mb-surface_multi_label_cls. Representative segmentation datasets include mb-boulder_seg, mb-conequest_seg, mb-crater_binary_seg, mb-crater_multi_seg, mb-mars_seg_mer, mb-mars_seg_msl, mb-mmls, and mb-s5mars. Detection datasets include mb-boulder_det, mb-conequest_det, and mb-dust_devil_det (Purohit et al., 28 Oct 2025).

The benchmark formalizes three task families: image classification, semantic segmentation, and object detection. It reports IoU for segmentation, Precision, Recall, F1, mean Average Precision for detection, mean IoU across classes for multi-class segmentation, and uses the Interquartile Mean across seeds and tasks with $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$57 bootstrap confidence intervals for aggregated reporting. Baselines include ImageNet-pre-trained models such as ResNet101, SqueezeNet1.1, InceptionV3, ViT-L/16, and SwinV2-B; segmentation architectures including U-Net, DeepLabV3+, SegFormer, and DPT; detection architectures including YOLOv1, SSD, RetinaNet, and Faster R-CNN; EO pre-trained models such as SatMAE, CROMA, and Prithvi; and vision-LLMs including GPT-4o Mini, Gemini 2.0 Flash, CLIP, SigLIP, and SmolVLM.

The baseline results show that, under feature extraction, SwinV2-B and ViT-L/16 achieve normalized F1 of approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$58 across six classification datasets, ResNet101 approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$59, and SqueezeNet1.1 approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$60. For segmentation, U-Net reaches normalized IoU approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$61, ahead of SegFormer at approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$62. For detection, YOLOv1 reaches normalized mAP approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$63, while SSD, RetinaNet, and Faster R-CNN fall in the approximate range $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$64–$$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$65. The paper further states that ViT-L/16 pre-trained on ImageNet outperforms SatMAE, CROMA, and Prithvi on four classification tasks, despite their Earth-observation relevance, and that VLMs exhibit high variance and often misinterpret specialized Martian features.

The same benchmark becomes an evaluation substrate in MOMO, which uses nine orbital downstream tasks drawn from HiRISE, CTX, and THEMIS: four image-classification benchmarks—AtmosDust, DoMars16k, Frost, Landmark—and five semantic-segmentation benchmarks—Boulder, ConeQuest, Crater Binary, Crater Multi, and MMLS (Purohit et al., 3 Apr 2026). MOMO compares scratch, ImageNet, EO foundation models, sensor-specific pre-training, joint data-merge pre-training, and a merged multi-sensor MAE. The reported consolidated table gives MOMO weighted-F1 or mIoU of $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$66 on AtmosDust, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$67 on DoMars16k, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$68 on Frost, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$69 on Landmark, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$70 on Boulder, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$71 on ConeQuest, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$72 on Crater Binary, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$73 on Crater Multi, and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$74 on MMLS. The paper states that MOMO ranks first on average with avg rank $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$75.

MOMO’s distinctive methodological contribution is Equal Validation Loss checkpoint alignment. Instead of merging terminal checkpoints, it selects epochs satisfying

$$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$76

and among aligned tuples minimizes the average distance from early-stop epochs. The ablation reports that EVL yields $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$77 mIoU over ES/LE merging on three segmentation tasks. The paper also states that gains are largest on fine-detail segmentation tasks, including $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$78 mIoU on Crater Multi and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$79 mIoU on MMLS.

Taken together, these two papers establish Mars-Bench as a standardized Mars-vision benchmark in the conventional ML sense: unified datasets, fixed splits, common metrics, reproducible baselines, and downstream evaluation for Mars-specific pre-training strategies.

6. Non-planetary reuse, related benchmarks, and terminological boundaries

The name is also used outside Mars science. “MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation” is built from play-by-play text commentary and evaluates three settings: Ultra Multi-turn, Interactive Multi-turn, and Cross-turn Tasks. Each sample is one complete game, and the benchmark contains $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$80 games with average dialogue turns $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$81 and approximately $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$82 total queries. Its task categories are Instruction Following, Context Retrieval, Information Reasoning, and Task Switching; scoring is checklist-based, with overall performance defined as $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$83. The reported leader is Gemini-2.5-Pro at $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$84 overall, while the best open-source model, DeepSeek-R1, reaches $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$85 (Yang et al., 27 May 2025).

That benchmark is conceptually unrelated to the Mars-science usages, although it shares the surface form “MARS-Bench.” Its reported mechanistic analysis attributes degradation in long complex dialogue sessions partly to attention sinks associated with special tokens, with attention to key spans falling from $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$86 for one-turn inputs to $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$87 for twenty-turn inputs.

Several nearby titles further complicate the nomenclature but are not themselves Mars-Bench. “MARS: Benchmarking the Metaphysical Reasoning Abilities of LLMs with a Multi-task Evaluation Dataset” defines three tasks—Metaphysical Event Discrimination, Metaphysical Inference Discrimination, and Metaphysical Transition Reasoning—with dataset totals of $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$88, $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$89, and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$90 entries respectively, and reports fine-tuned DeBERTa-Large at $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$91 across the three tasks (Wang et al., 2024). “Planetary Terrain Datasets and Benchmarks for Rover Path Planning” introduces MarsPlanBench rather than Mars-Bench; it is built from $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$92 HiRISE DTM tiles, yields $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$93 occupancy grids via $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$94 and $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$95 slope-threshold variants, and reports $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$96 success for Dijkstra, Theta*, and A* on $$r_d=\frac{|\Omega_d|}{HW},\qquad \Omega_d=\{(u,v):d(u,v)>0\ \text{finite}\},$$97 challenging MarsPlanBench-10 maps (Chancán et al., 24 Dec 2025).

The terminological boundary is therefore important. In planetary science, “Mars-Bench” may denote stereo reconstruction, radiation transport validation, multimodal retrieval, or Mars-vision foundation-model benchmarking. Outside planetary science, the nearly identical “MARS-Bench” names a dialogue benchmark. This suggests that precise citation by title and arXiv identifier is necessary whenever the term appears in technical discourse.