SciML Framework: Reproducible Scientific ML with DLHub

• Scientific Machine Learning (SciML) frameworks are engineered systems that support reproducible, scalable, and domain-specific ML for scientific research.
• DLHub decouples model management from serving, leveraging techniques like memoization and batching to achieve low-latency and high-throughput performance.
• Its self-service repository ensures complete packaging of ML models with essential artifacts, fostering reproducibility and seamless integration in complex scientific workflows.

Scientific Machine Learning (SciML) Frameworks are engineered systems supporting the deployment, reproducibility, scalability, and performance of ML models that address scientific applications. Unlike conventional ML platforms, SciML frameworks must accommodate domain-specific requirements for model reproducibility, integration with complex scientific workflows, broad compatibility with research code and artifacts, and the efficient use of heterogeneous, often distributed, computational resources. The DLHub framework exemplifies this class of systems, providing a comprehensive solution for model repository, serving, indexing, and integration tailored to scientific use cases (Chard et al., 2018).

1. Model Repository: Publication and Reproducibility

DLHub implements a self-service model repository supporting the full lifecycle of ML models for science. This repository enables model publication, sharing, discovery, verification, and re-use, while explicitly enforcing reproducibility. Users are required to package not only trained models, but also all constituent components—such as weights, hyperparameters, associated scripts, and relevant training/test datasets. Published models are indexed using a standardized metadata schema and are discoverable via Globus Search.

This packaging approach ensures that any published result can be independently reproduced and extended by third parties, addressing a central challenge in scientific machine learning. The use of structured, searchable metadata supports community-wide practices for citation, tracking model provenance, and attribution.

2. Architecture: Scalability and Serving

DLHub adopts a multi-component architecture that decouples model management from model serving, allowing each function to scale independently. The architecture consists of:

• Management Service: Responsible for publication, discovery (using Globus Search), packaging (Dockerization of dependencies), and routing of inference tasks.
• Task Managers: Dispatched to distributed computing resources, these poll a ZeroMQ queue to retrieve inference requests and dispatch them to executable environments.
• Executors: Backends such as Parsl (for general Python models), TensorFlow Serving, and external systems like SageMaker, allow for scalable, parallel, and heterogeneous execution (deployment on Kubernetes clusters, HPC via Singularity, etc.).

This architecture enables fine-grained control over deployment and execution, supporting both high-throughput and latency-sensitive scientific inference workloads.

3. Serving Techniques: Memoization and Batching

DLHub integrates two critical serving accelerators:

• Memoization: Caches input-output pairs for servables, enabling immediate response to repeated requests and reducing redundant computation. In benchmark experiments, memoization was shown to reduce invocation latency by up to 99%, reaching around 1 ms in optimized scenarios.
• Batching: Aggregates multiple input requests for joint execution. This amortizes network and invocation overhead, yielding a roughly linear scaling of invocation time with batch size. Such batching is particularly beneficial for scientific workflows where inference is applied over large datasets or in ensemble scenarios.

Empirical comparisons demonstrate that DLHub achieves performance comparable to optimized C++ inference engines (e.g., TensorFlow Serving) in default conditions, while outperforming them by a significant margin when batching or memoization is leveraged.

Serving Framework	Arbitrary Python	Pipelines	Batching & Memoization	Distributed
DLHub	Yes	Yes	Yes	Yes
TensorFlow Serving	No	No	Yes (limited)	Partial
SageMaker	Yes (limited)	Yes	Yes	Yes
Clipper	Yes	Yes	Yes	Yes

4. Compatibility: Pythonic and Pipeline-Oriented Design

A distinguishing feature of DLHub is its support for any Python 3–compatible model or processing function. This flexibility stands in contrast to systems limited to specific ML frameworks or model formats. In addition, DLHub exposes interfaces for chaining multiple servables into end-to-end pipelines, enabling complex scientific workflows—for example, combining pre-processing, feature extraction, multi-model inference, and post-processing in a single, automated pipeline. Each component executes in isolation, but is orchestrated centrally, ensuring modular, reusable pipeline design.

This design supports the needs of scientific domains where workflows often involve heterogeneous toolchains and processing stages. For instance, in materials science, a pipeline might sequentially convert chemical formulae to structured objects, compute domain-specific features, and perform property prediction with an ML model.

5. Early Applications: Scientific Impact and Workflow Integration

DLHub's capabilities for integration and reproducibility are illustrated in several early use cases:

• Cancer research (CANDLE project): Secure, staged sharing of drug response models, with access control for test-stage models.
• Materials science: Enrichment of Materials Data Facility datasets via pipelines comprising composition parsing, featurization (matminer), and property prediction (random forest).
• Neuroanatomy: Near-real-time tomographic image segmentation, supporting high-throughput analysis of brain tissue structures.

In these workflows, DLHub's serving model provides both high throughput and fine-grained access control, while model discovery and invocation are exposed via web interfaces and APIs suitable for integration with external scientific portals and platforms.

6. Performance Metrics, Resource Accounting, and Deployment

DLHub explicitly distinguishes several performance metrics to facilitate rigorous evaluation, particularly relevant in scientific applications demanding predictable latency and throughput:

• Inference time: Time for the servable to compute the result.
• Invocation time: Overhead for dispatching and executing the task (including input serialization and outbound communication).
• Request time: End-to-end duration as measured at the user-facing API.

Experiments indicate that Python "test functions" achieve inference times below 20 ms, with the architectural overhead typically maintaining total request times under 40 ms (excluding batching/memoization). DLHub supports dynamic allocation and scaling of execution resources through Kubernetes and HPC integration, providing elasticity and cost efficiency for both batch and interactive workloads.

7. Scientific Model Lifecycle and Interoperability

DLHub promotes best practices across the scientific model lifecycle, emphasizing reproducibility, reuse, and collaboration. Published models can be referenced using persistent identifiers and cited in academic publications. The system's modular, Python-first approach and broad framework compatibility lower barriers for adoption and extension, facilitating cross-disciplinary interoperability.

By decoupling management and serving, and exposing standardized interfaces for publication, discovery, and execution, DLHub lays a practical foundation for next-generation scientific workflows that incorporate automated model invocation, provenance tracking, and seamless integration with large-scale scientific data portals.

Summary

DLHub addresses the central requirements of scientific machine learning frameworks—model reproducibility, flexible serving of arbitrary models and pipelines, scalability across heterogeneous resources, and integration with scientific cyberinfrastructure. By combining a self-service repository with a low-latency, distributed serving platform, and incorporating advanced techniques such as batching and memoization, DLHub provides a robust solution tailored to the demands of computational science. Its support for modular pipeline design, fine-grained access control, and rigorous resource accounting position it as a foundational component in the broader ecosystem of SciML research and application (Chard et al., 2018).