PathBench: Multi-Domain Benchmarking Suite
- PathBench is a unified platform for developing, visualizing, training, testing, and benchmarking classical and learning-based path planning algorithms in 2D/3D grid environments with ROS integration.
- PathBench extends into computational pathology and pathological speech assessment, offering leakage-free benchmarks and multilingual evaluation protocols across diverse clinical datasets.
- The framework promotes reproducible research by standardizing map generation, training pipelines, evaluation metrics, and system comparisons across multiple hardware platforms.
Searching arXiv for PathBench papers to ground the article. arxiv_search query: "PathBench" max_results: 10 arxiv_search("PathBench", max_results=10) PathBench is a name used in multiple arXiv research programs for benchmarking infrastructures, most prominently an open-source platform for classical and learning-based path planning in mobile robotics and, in later independent usages, a benchmark family in computational pathology and pathological speech assessment. The robotics PathBench was introduced as a Python-based system for developing, visualizing, training, testing, and benchmarking planners in 2D and 3D grid-world environments with Robot Operating System integration (Toma et al., 2021), and was later expanded for systematic comparison across hardware systems, map sources, and evaluation metrics (Hsueh et al., 2022). Subsequent work reused the same name for a leakage-free benchmark for pathology foundation models (Ma et al., 26 May 2025), a multilingual benchmark for pathological speech intelligibility assessment (Halpern et al., 9 Mar 2026), and PathBench-MIL, an AutoML framework for multiple instance learning in histopathology (Brussee et al., 19 Dec 2025). The term is therefore domain-dependent rather than uniquely tied to a single software artifact.
1. Terminological scope and disambiguation
The earliest PathBench usage in the material here is the robotics platform for path planning, first presented in 2021 and then elaborated in a 2022 systematic benchmarking study. From 2025 onward, the same name appears in computational pathology and speech assessment, with different datasets, metrics, and software assumptions. A common source of confusion is to treat all references to PathBench as references to the robotics system; the literature does not support that assumption.
| Variant | Domain | Defining characteristics |
|---|---|---|
| PathBench | Mobile robotics | Unified platform for developing, visualizing, training, testing, and benchmarking classical and learning-based planners in 2D and 3D, with ROS support (Hsueh et al., 2022) |
| PathBench | Computational histopathology | Leakage-free benchmark for pathology foundation models over 15,888 WSIs, 8,549 patients, and 64 tasks (Ma et al., 26 May 2025) |
| PathBench | Pathological speech assessment | Unified benchmark across six public datasets, four languages, and three evaluation protocols (Halpern et al., 9 Mar 2026) |
| PathBench-MIL | Histopathology MIL | AutoML and benchmarking framework for end-to-end WSI multiple instance learning pipelines (Brussee et al., 19 Dec 2025) |
This multiplicity of usage suggests that “PathBench” functions primarily as a naming convention for standardized evaluation ecosystems. In historical terms, however, the robotics platform remains the original and most direct referent of the name in arXiv records considered here.
2. Robotics PathBench: platform definition and system architecture
In mobile robotics, PathBench is a unified platform for developing, visualizing, training, testing, and benchmarking both classical and learning-based path planning algorithms in 2D and 3D grid-world environments. Its stated goal is reproducible, comparable experimentation across standardized maps and metrics, with support for Robot Operating System integration and real-time robot experiments (Hsueh et al., 2022).
The platform’s scope is grid environments for point-mass navigation: 2D occupancy grids and 3D voxels or point clouds. Planned extensions include continuous spaces, topological maps, and higher-dimensional motion planning. The software architecture is organized around four main modules—Simulator, Generator, Trainer, and Analyzer—connected through information flow among Environment, ML, ROS, and Evaluation. The system is implemented in Python, uses PyTorch for ML, and Panda3D for visualization (Hsueh et al., 2022).
The Simulator provides an interactive visualization and environment engine. It supports animation of algorithm execution, rendering of internal state, key-frame-based step-by-step inspection, and custom overlays for search space, fringe structure, graphs, and related internals. The map abstractions are explicit: DenseMap for static 2D grid maps, SparseMap for 3D point clouds, and RosMap for 2D occupancy grids with live SLAM updates. Generated maps and datasets are stored in an Atlas directory in .pickle and JSON formats. The earlier platform description also emphasized that maps could plug in their own physics or use third-party engines such as pymunk and OpenAI Gym, reinforcing the platform’s role as a unifying infrastructure rather than a single fixed simulator (Toma et al., 2021).
3. Data generation, training pipelines, and algorithm integration
A central feature of the robotics PathBench is that classical and learned planners are benchmarked through the same environment abstraction and analysis harness. The Generator produces synthetic maps, labels them, augments datasets, and applies procedural modifications. The generation step includes uniform random fill, block, house, and point-cloud maps with configurable parameters such as obstacle fill rate, number of obstacles, and room size ranges. In the labelling step, A* serves as ground truth for features and labels including distance to goal, direction to goal, global map, local view, and agent position. The Modification stage supports lambda transforms over maps, such as repositioning agent and goal or adding doors (Hsueh et al., 2022).
The Trainer wraps PyTorch training pipelines for ML planners. Its default procedure is data pre-processing, holdout splitting with a default 60% train, 20% validation, and 20% test partition, epoch-based training, evaluation, result display, and artifact saving. Default batch size is 50, though this is configurable. Model-specific batch_start hooks compute losses; optimizer steps are applied in the standard loop; logs are saved in EvaluationResults; and the pipeline persists state_dict, configurations, plots, and logs. The 2021 description further notes support for sequence handling via PackedDataset, sorted by sequence length, and hook points including start, epoch_start, batch_start, batch_finish, epoch_finish, and finish (Toma et al., 2021).
The supported algorithm set spans several planning traditions. Classical planners include graph-based methods such as A*, Dijkstra, and wavefront; discrete sampling-based methods such as discretized PRM, d-RRT, d-RRT*, and d-RRT-Connect; sensory-based Bug1 and Bug2; and the potential field method. Learning-based planners include Value Iteration Networks, Motion Planning Networks, Gated Path Planning Networks, Online LSTM, CAE-LSTM, Bagging LSTM, and Waypoint Planning Networks. The platform’s API makes extension explicit: a new algorithm class interacts with the Map and Agent interfaces for movement primitives and collision checks, optionally renders internals through MapDisplay, and hooks into AlgorithmRunner and BasicTesting for standardized evaluation. ML planners additionally define models and batch_start hooks and consume Generator-produced features and labels (Hsueh et al., 2022).
4. Benchmarking protocol, map corpora, and evaluation metrics
The benchmarking methodology in the 2022 study was designed to compare algorithms across five hardware systems and three broad map families: built-in synthetic PathBench maps, video-game maps, and maps drawn from real-world databases. The hardware systems were a Commodity Laptop, a Gaming Laptop, an Intel NUC10i7FNK, the Compute Canada Graham cluster, and a Raspberry Pi 4b. The 2D synthetic benchmark used 3,000 maps at , with 1,000 maps per type and randomized start-goal pairs. External evaluation used 30 city maps at and 30 video-game maps with 800–1200 cells per dimension, each with randomized trials per map. The 3D benchmark used maps, again with 1,000 maps per type. Identical pseudorandom seeds were used for sampling-based planners to ensure fair comparisons (Hsueh et al., 2022).
PathBench distinguishes simple and complex analyses. Simple Analysis averages results over samples per map type and algorithms, typically producing bar and violin plots. Complex Analysis runs algorithms over maps with randomized start-goal trials per map and averages over the full collection. A Training Dataset Analysis mode inspects dataset characteristics such as Euclidean start-goal distances, success rates, obstacle ratios, and search space (Hsueh et al., 2022).
The metrics emphasize both task completion and path quality. Success rate is defined as
Path deviation is defined relative to the A* shortest path:
0
The A* evaluation function is
1
with 2 typically Euclidean or Manhattan distance on a grid. Additional reported metrics include path length, distance left to goal when planning fails, wall-clock computation time, search space, maximum memory consumption, obstacle clearance, and smoothness as mean absolute turning angle between consecutive path segments (Hsueh et al., 2022).
These definitions matter because PathBench does not reduce path planning to a single scalar score. It treats optimality, feasibility, smoothness, clearance, resource usage, and speed as separable dimensions, which is especially important when comparing graph search, sampling-based planners, and learned policies.
5. Empirical findings, hardware sensitivity, and ROS deployment
The 2022 systematic comparison showed that classical graph-search planners remained the most reliable across map types. On 2D 3 PathBench maps, A* achieved a success rate of 99.4%, path deviation of 0.00%, and mean time of 0.106 s; Dijkstra matched the success rate and deviation but required 0.338 s; wavefront achieved 99.4% success, 0.36% path deviation, and 0.340 s. Among sampling-based methods, d-RRT-Connect achieved 99.33% success with 21.17% path deviation in 0.2458 s, while d-RRT was slower at 7.336 s and achieved 96.8% success. In the learned category, WPN reached 99.4% success with 2.78% path deviation and 0.817 s runtime, whereas VIN and GPPN showed substantially lower success rates of 18.2% and 30.3%, respectively (Hsueh et al., 2022).
The performance gap widened on large external maps. On 512 × 512 city maps, A*, Dijkstra, and wavefront each achieved 100% success; WPN also achieved 100% success but with longer compute time of 13.816 s, while VIN and GPPN had 0% success. On large video-game maps, A*, Dijkstra, and wavefront again achieved 100% success, d-RRT-Connect achieved 95.3%, and WPN maintained 100% success at 110.307 s, whereas VIN and GPPN again had 0% success. This pattern supports the paper’s main conclusion that classical planners reliably find optimal or near-optimal paths with high success rates, sampling-based methods trade optimality for speed in higher-dimensional or larger search spaces, and most end-to-end learned planners in the benchmark struggle to scale to large and complex maps unless hybridized (Hsueh et al., 2022).
The 3D results sharpen the trade-off. On 4 maps, A* achieved 100% success, path length 20.69, path deviation 0.00%, and time 0.475 s. d-RRT-Connect achieved 99.7% success with 17.56% path deviation but only 0.097 s runtime, making it the fastest reported 3D planner. The contrast is classical: optimality from graph search versus speed from sampling-based exploration (Hsueh et al., 2022).
Hardware sensitivity analysis showed that obstacle clearance, success rate, path deviation, and smoothness remained consistent across hardware for deterministic classical algorithms, while computation time improved with stronger CPUs and GPUs. ML planners benefited substantially from better GPUs such as the RTX 2080 and V100. The choice of hardware therefore most strongly affected compute time rather than task outcome metrics (Hsueh et al., 2022).
PathBench also closes the loop with real robots through ROS. RosMap converts ROS SLAM outputs such as gmapping occupancy grids into PathBench maps, supports live updates, and converts PathBench movement actions into ROS publisher-subscriber control messages. The real-world experiment demonstrated A*, d-RRT-Connect, and WPN in ROS/Gazebo interoperability, including fixed and exploration scenarios with live map updates during traversal. The planner could navigate known and unknown regions while SLAM updated the map in real time (Hsueh et al., 2022).
6. PathBench beyond robotics: pathology, speech, and benchmarking lineage
From 2025 onward, the name PathBench was independently adopted in computational pathology. In "PathBench: A comprehensive comparison benchmark for pathology foundation models towards precision oncology," PathBench denotes a clinically grounded, leakage-free benchmark for pathology foundation models, built from 15,888 whole-slide images from 8,549 patients across 10 hospitals and covering 64 diagnosis and prognosis tasks in lung, breast, gastric, colorectal, and brain cancer (Ma et al., 26 May 2025). Its central design principles are private evaluation data, formal exclusion from pretraining corpora, patient-level splits, cross-hospital external validation, repeated-seed experiments, bootstrap confidence intervals, and a live leaderboard. Evaluation is standardized through a two-stage pipeline in which a pathology foundation model extracts features and a common attention-based MIL aggregator is trained from scratch on each task. Reported aggregate rankings place Virchow2 first with an average ranking score of 5.0 and H-Optimus-1 second at 5.9, while organ-wise leaders differ, reinforcing the benchmark’s main thesis that the best model is cancer- and task-specific (Ma et al., 26 May 2025).
In speech assessment, "PathBench: Speech Intelligibility Benchmark for Automatic Pathological Speech Assessment" defines PathBench as a unified, reproducible benchmark for pathological speech intelligibility assessment using only public datasets (Halpern et al., 9 Mar 2026). It spans six datasets, four languages, and three requirement categories—reference-free, reference-text, and reference-audio—and organizes evaluation through three protocols: Matched Content, Extended, and Full. The primary metric is speaker-level correlation with each dataset’s native clinical target, rather than absolute-scale calibration. The benchmark introduces Dual-ASR Articulatory Precision, a reference-free method that achieves the highest average correlation among reference-free baselines with 5, while ArtP and NAD tie as the best overall methods at 6. The results also show that Extended significantly outperforms Matched Content overall and that sentence stimuli benefit reference-audio methods (Halpern et al., 9 Mar 2026).
PathBench-MIL extends the naming lineage within histopathology but with a different object of study. "PathBench-MIL: A Comprehensive AutoML and Benchmarking Framework for Multiple Instance Learning in Histopathology" describes an open-source framework for end-to-end benchmarking and automated optimization of MIL pipelines, covering WSI preprocessing, tiling, stain handling, feature extraction, aggregation, and survival modeling under a unified YAML configuration (Brussee et al., 19 Dec 2025). The framework builds on SlideFlow, uses Optuna with samplers such as TPE and pruners such as Hyperband, supports caching of intermediate artifacts, and exposes interpretability outputs such as attention heatmaps and tile-level scores. The paper explicitly frames PathBench-MIL as the MIL-focused component within a broader PathBench effort toward standardized, reproducible evaluation in computational pathology (Brussee et al., 19 Dec 2025).
A further, weaker form of reuse appears in autonomous navigation work that borrows PathBench analyzer concepts without adopting the full benchmark. "Hybrid DQN-TD3 Reinforcement Learning for Autonomous Navigation in Dynamic Environments" states that selected analyzer functionalities from PathBench were integrated for evaluation in ROS1 Noetic + Gazebo, but the study does not use standard PathBench scenarios, maps, protocols, or numerical PathBench-style results (He et al., 30 Oct 2025). This is a useful boundary case: the paper draws on PathBench as an evaluation vocabulary—success rate, collision rate, path efficiency, and smoothness—rather than as a complete benchmarking suite.
Taken together, these later works show that PathBench has evolved from the title of a robotics benchmarking platform into a broader label for rigorous, standardized evaluation ecosystems. The shared theme is benchmarking under controlled protocols; the implementations, datasets, and scientific questions are otherwise distinct.