PathBench-MIL: AutoML MIL Pipelines

• The paper demonstrates that PathBench-MIL automates and benchmarks complete MIL pipelines for histopathology, integrating preprocessing, feature extraction, and aggregation.
• The framework leverages a YAML-driven configuration and Optuna-powered AutoML to optimize tiling, normalization, feature extraction, and MIL hyperparameters in end-to-end workflows.
• Results include efficient pipeline execution using caching, early pruning, and interactive visualization to support model selection and reproducible experiments.

PathBench-MIL is an open-source AutoML and benchmarking framework for end-to-end multiple instance learning (MIL) in computational histopathology. It automates full MIL pipeline construction, from raw whole-slide images (WSIs) to slide-level predictions, and benchmarks complete pipelines across datasets and tasks rather than isolated components. The framework treats tiling, quality control, stain normalization, feature extraction, MIL aggregation, and training hyperparameters as an explicit search and benchmark space, with reproducible, configuration-driven experiments, integrated visualization tools, and support for classification, regression, and survival analysis (Brussee et al., 19 Dec 2025).

1. Definition, scope, and design objective

PathBench-MIL addresses the combinatorial design space of modern pathology MIL pipelines. In the framework’s formulation, tiling, quality control, stain normalization, feature extraction, aggregation architectures, and optimization hyperparameters interact in dataset- and task-dependent ways, making manual exploration expensive and difficult to reproduce. PathBench-MIL therefore treats the entire pipeline as a structured object for benchmarking and optimization rather than reducing evaluation to a single model family or a single feature extractor (Brussee et al., 19 Dec 2025).

Its stated design goals are to build full MIL pipelines automatically, benchmark complete pipelines across datasets and tasks, perform AutoML over both architectural choices and continuous hyperparameters, and provide reproducible, configuration-driven experiments with integrated visualization tools. This positioning distinguishes it from frameworks that concentrate primarily on foundation model benchmarking or on a restricted subset of the pathology workflow. The framework is publicly available at https://github.com/Sbrussee/PathBench-MIL (Brussee et al., 19 Dec 2025).

A central premise is that end-to-end evaluation matters. The framework is designed to reveal interactions across preprocessing, feature extraction, and MIL aggregation. This suggests that PathBench-MIL is intended not merely as a model zoo, but as infrastructure for systematic study of pipeline-level dependencies in computational pathology.

2. System architecture and formal MIL model

PathBench-MIL is implemented on top of SlideFlow, which provides WSI I/O, tiling, and multiprocessing. On top of this substrate, the framework adds a YAML-based configuration layer, a MIL pipeline engine that instantiates end-to-end workflows, two operational modes—Benchmarking and Optimization—and a Dash-based visualization application (Brussee et al., 19 Dec 2025).

The pipeline begins with three explicit inputs: WSIs, a CSV containing slide-level annotations, and a YAML configuration specifying pipeline components and mode. The preprocessing stage includes quality control, tiling or patch extraction, and optional stain normalization such as Macenko or Reinhard normalization. Feature extraction then produces tile-level embeddings using either standard CNNs or pathology-specific foundation or self-supervised models. These embeddings are passed to a MIL aggregation module, which outputs slide-level predictions for classification or regression tasks, or survival risk for survival tasks. Training and evaluation record task-appropriate metrics and preserve configurations and artifacts for later analysis.

The WSI-to-prediction data flow is described explicitly: the system reads a WSI, applies quality control and tissue detection, extracts tiles according to tile size and magnification, optionally applies stain normalization, generates an embedding for each tile via the selected feature extractor, and aggregates tile embeddings using a MIL model to obtain a slide embedding or prediction. Training is performed at the slide level, and only slide labels are required (Brussee et al., 19 Dec 2025).

The formal MIL representation defines slide $i$ as a bag of tile embeddings,

$$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$

with slide-level label $y^{(i)}$, which may be categorical, continuous, or survival-associated. The MIL model is decomposed into feature extraction and aggregation: $$g_{\text{feat}}(\mathcal{X}) = \{ h_1, \dots, h_n \}, \qquad \hat{y} = f_{\text{agg}}\!\left(g_{\text{feat}}(\mathcal{X})\right).$$ Training is written as minimization of a task-specific loss over training bags: $$\theta^* = \arg\min_{\theta} \sum_{i=1}^N \mathcal{L}\!\left( f_{\text{agg}}(g_{\text{feat}}(\mathcal{X}^{(i)}; \theta)),\, y^{(i)} \right).$$ The paper also presents typical attention-pooling expressions, including attention weights

$$a_i = \frac{\exp\left( w^\top \tanh(V h_i^\top) \right)} {\sum_j \exp\left( w^\top \tanh(V h_j^\top) \right)}$$

and bag representation

$z_B = \sum_i a_i h_i.$

These equations are included as examples of supported MIL families rather than as a derivation of a single canonical model (Brussee et al., 19 Dec 2025).

3. AutoML and search-space formulation

PathBench-MIL’s AutoML subsystem is built on Optuna and treats the MIL pipeline as a structured search space,

$$\mathcal{S} = \mathcal{T}_{\text{tile}} \times \mathcal{P}_{\text{norm}} \times \mathcal{M}_{\text{feat}} \times \mathcal{A}_{\text{agg}} \times \mathcal{H}_{\text{opt}},$$

where the components correspond to tiling parameters, preprocessing and stain normalization, feature extractor architectures, MIL aggregation architectures, and training hyperparameters. Each configuration $\mathbf{c} \in \mathcal{S}$ defines a full pipeline, and the optimization objective is expressed as

$$\mathbf{c}^* = \arg\min_{\mathbf{c} \in \mathcal{S}} \mathcal{L}_{val}\!\left( \text{Train}(\mathbf{c}, \mathcal{D}_{train}), \mathcal{D}_{val} \right).$$

For classification, the validation objective is described as usually “1 − AUC” or similar (Brussee et al., 19 Dec 2025).

The framework exposes multiple search strategies. Users can choose TPE, CMA-ES, or Random Search as samplers, and Hyperband or Median Pruning as pruners. Optimization is budget-constrained, with a total trial count $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$0, and underperforming trials may be terminated early based on intermediate validation metrics. An approximate speedup formula is provided for TPE combined with Hyperband: $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$1 where $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$2 is the number of configurations in exhaustive benchmarking, $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$3 is the number of trials actually executed, and $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$4 is the average fraction of a full training run completed before pruning. The paper further states that Hyperband alone can reach approximately $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$5 speedup in time-to-good-solution and that, in structured spaces, TPE plus Hyperband can reach one to two orders of magnitude (Brussee et al., 19 Dec 2025).

The Optuna trial lifecycle is explicitly staged. Each trial samples a configuration, checks whether artifacts associated with the preprocessing subconfiguration already exist, reuses cached tiles or feature bags if available, trains and evaluates the MIL model, applies pruning, and serializes the trial state for resumption after preemption or time limits on HPC clusters. This suggests that PathBench-MIL conceptualizes AutoML not as isolated hyperparameter tuning but as pipeline reuse under artifact-aware execution.

4. Preprocessing, feature extraction, and aggregator model zoo

The preprocessing layer supports configurable patch size and magnification, tissue detection to avoid background, and quality control based on tissue coverage and potential artifacts if implemented through SlideFlow. Stain normalization methods explicitly named in the framework are Macenko and Reinhard normalization (Brussee et al., 19 Dec 2025).

Feature extraction is treated as a pluggable module $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$6. Examples given include ResNet50 as a standard CNN and pathology-specific foundation or self-supervised models such as UNI and PLIP. Additional literature-referenced models include Virchow, Virchow2, H-optimus, Hibou, and HistoSSL, with the framework designed to integrate such models as they become available. For each tile image $$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$7, feature extraction yields

$$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$8

The paper notes that these vectors are usually high-dimensional, for example 512–2048 dimensions, and may optionally undergo feature normalization or dimensionality reduction such as PCA, although explicit PCA equations are not given (Brussee et al., 19 Dec 2025).

A key engineering decision is precomputation and caching. WSIs are decomposed into tiles, tile features are precomputed and cached, and slide-level bags are stored as feature bags. Subsequent AutoML trials can therefore skip tile extraction and feature computation and retrain only the MIL aggregator. The paper emphasizes that this caching is crucial for scaling to large cohorts and large search spaces (Brussee et al., 19 Dec 2025).

The model zoo includes standardized MIL aggregators such as AB-MIL, CLAM, DSMIL, transformer-based MIL, simple mean and max pooling, and Deep Sets-style models with

$$\mathcal{X}^{(i)} = \{ x^{(i)}_1, x^{(i)}_2, \dots, x^{(i)}_{n_i} \},$$9

Each architecture is implemented as a module that maps a bag of embeddings to slide-level predictions, enabling aggregator swapping in YAML and extension through a small interface such as forward(bag) (Brussee et al., 19 Dec 2025).

5. Configuration system, benchmarking workflow, and reproducibility

All experiments are defined in a single YAML file. The configuration covers dataset paths, slide and patient identifiers, labels, survival times and events, preprocessing parameters, feature extractors, MIL models, loss functions, optimizers, evaluation metrics, and the operating mode. Benchmarking mode specifies enumerated combinations of parameters, whereas Optimization mode specifies Optuna settings such as sampler, pruner, number of trials, and pruning warm-up (Brussee et al., 19 Dec 2025).

The framework is modular. Feature extractors, aggregators, loss functions, augmentations, and metrics implement standard interfaces. Adding a new component requires implementing a Python class with the expected methods, registering it in the configuration registry, and referring to it by name in YAML. This design is intended to let researchers benchmark new MIL models in a uniform environment, integrate new foundation models rapidly, and add custom evaluation metrics (Brussee et al., 19 Dec 2025).

The benchmarking setup is methodological rather than tied to a fixed built-in public dataset suite. Multiple datasets can be specified, including train, validation, and external test datasets, and supported tasks include classification, regression, deep continuous survival models, and deep discrete survival models. The paper describes metrics such as AUC for classification, regression loss for regression, and C-index for survival; it also notes likely configurable metrics such as accuracy, F1, MSE, MAE, and possibly integrated Brier score depending on chosen modules (Brussee et al., 19 Dec 2025). Because the paper does not enumerate benchmark result tables or specific public datasets, any stronger claim about default leaderboard behavior would be unwarranted. A plausible implication is that PathBench-MIL is meant to function as benchmark infrastructure rather than as a benchmark with a canonical fixed task suite.

Reproducibility is supported by config-driven experiments, explicit CSV-based data specification, caching and artifact reuse, and serialized trial states that allow interrupted studies to resume. Implementation details further identify SlideFlow as the WSI backend, Optuna as the AutoML engine, Dash as the visualization layer, CPU multiprocessing for tiling and preprocessing, and GPU parallelization via the underlying deep-learning stack (Brussee et al., 19 Dec 2025).

6. Visualization, comparative positioning, and subsequent benchmark relevance

PathBench-MIL includes an interactive Dash application for experiment analysis. Users can load benchmarking or AutoML studies, filter experiments by configuration parameters, compare performance across subsets of the search space, create plots such as AUC versus patch size or C-index versus aggregator, and export figures. The visualization layer is described primarily as support for meta-analysis of experiments and model selection rather than tile-level interpretability; tile-level visual analytics such as attention heatmaps are not detailed in the text, although the integration with SlideFlow and standard MIL models makes such extensions straightforward (Brussee et al., 19 Dec 2025).

The framework’s comparison to Patho-Bench and EVA is summarized in the following dimensions.

Capability	PathBench-MIL	Patho-Bench / EVA
Entire MIL pipeline coverage	Yes	No
Regression	Yes	No
Deep continuous survival	Yes	No
Built-in AutoML	Yes	No
Interactive visualization	Yes	No
Built-in datasets and tasks	No	Yes

More specifically, all three frameworks support patch foundation models; PathBench-MIL and Patho-Bench support slide-level foundation models, while EVA does not. Classification is supported by all three, deep discrete survival is supported by PathBench-MIL and Patho-Bench, and EVA additionally supports semantic segmentation and patch-level tasks. PathBench-MIL does not bundle datasets and is currently slide-level MIL-focused, whereas Patho-Bench supports retrieval and EVA supports segmentation-related tasks (Brussee et al., 19 Dec 2025).

Later pathology MIL papers explicitly position themselves with respect to PathBench-MIL as a benchmark environment. PA-MIL is described as a pathology multiple instance learning method with explicit phenotype modeling, language prompting, and transcriptomic guidance that, in a benchmark setting such as PathBench-MIL, would sit alongside standard attention-based and transformer-based MIL models, and its authors state that its phenotype saliency representation could be used in a PathBench-MIL context for both WSI-only and multimodal benchmarking (Yang et al., 30 Jan 2026). RAA-MIL, a weakly supervised oral cytology framework, is described as directly relevant to PathBench-MIL because it provides both a task and a methodological innovation—Region-Affinity Attention—that could be integrated into richer MIL benchmarks (Mukherjee et al., 15 Nov 2025). AGE-MIL is explicitly framed as a strong patient-level baseline for multi-WSI tasks in a PathBench-MIL-like benchmark, extending MIL from slide-level to patient-level reasoning (Niu et al., 10 Jun 2026). FOCI, meanwhile, is presented as a rationale evaluation layer that a PathBench-MIL framework could use to move beyond AUC-centric assessment toward interpretability- and audit-centric evaluation through metrics such as SRP, MSK, AUKC, and SHI (Jung et al., 12 May 2026). These references do not demonstrate adoption within PathBench-MIL itself, but they do indicate that subsequent literature treats it as a natural benchmark reference point.

7. Limitations and projected development

PathBench-MIL’s principal limitations are computational and scope-related. Large-scale searches over complex pipelines remain expensive even with caching and pruning, and exhaustive benchmarking over large spaces still requires substantial compute. The framework is currently focused on slide-level MIL and does not yet include dedicated modules for segmentation, nuclei detection, or retrieval. It also does not provide direct download integration or standardized management of public datasets; users must manually curate WSIs and labels (Brussee et al., 19 Dec 2025).

Planned extensions include direct download integration for public datasets, distributed benchmarking across multiple GPUs or nodes, multimodal modeling that combines WSIs with clinical or molecular data, ensemble modeling across feature extractors or aggregators, graph neural network modules with AutoML over graph-construction choices, and new task types such as WSI retrieval, morphological prototyping via clustering, nuclei segmentation, and semantic segmentation (Brussee et al., 19 Dec 2025). The paper states that these developments aim to evolve PathBench-MIL from a MIL-specific framework into a broader platform for pathology AI pipelines.

This trajectory suggests a broader interpretation of the framework’s significance. PathBench-MIL is not only a system for enumerating MIL variants; it is an attempt to standardize how pathology AI pipelines are specified, optimized, compared, and interrogated. Within that interpretation, its importance lies less in a single model contribution than in its formalization of the end-to-end pathology MIL workflow as a reproducible experimental object (Brussee et al., 19 Dec 2025).