Two-Stage Open-Source Pipelines

• Two-stage open-source pipelines are computational workflows that split processing into an initial candidate generation stage and a subsequent refinement stage for improved efficiency.
• They integrate modular components—using languages like Python and Julia—with advanced techniques such as stochastic optimization and LLM-guided analysis for tasks including EMS deployment and fault localization.
• Practical applications demonstrate significant improvements, including reduced ambulance response times and enhanced software debugging, validated through rigorous simulation and ablation studies.

Two-stage open-source pipelines are end-to-end computational workflows built from modular, open-access components and structured into two distinct algorithmic phases. This architectural motif is increasingly prevalent in large-scale optimization, machine learning, and systems engineering contexts, with demonstrable applications in emergency services optimization and code fault localization. In such pipelines, each stage leverages the outputs of the previous stage to incrementally refine solutions, yielding both computational efficiency and analytic interpretability. Notable recent instantiations include OpenEMS for two-stage stochastic and robust ambulance deployment optimization, and FlexFL for flexible, LLM–augmented fault localization in software codebases (Ong et al., 2022, Xu et al., 2024).

1. Definition and Conceptual Foundations

A two-stage open-source pipeline comprises two architectural phases, each encapsulating a different computational objective. The first stage typically generates a reduced, tractable set of candidate solutions or scenarios by applying lightweight, interpretable analytics, statistical modeling, or heuristic search. The second stage executes a more resource-intensive or context-aware algorithmic refinement over this reduced set, often leveraging advanced optimization, simulation, or deep learning techniques.

The open-source paradigm ensures transparency, replicability, and extensibility. Standard toolchains—such as Julia+JuMP for mathematical programming, Python for data handling and simulation, and containerized agents for interfacing with LLMs—facilitate adoption and customization across research domains (Ong et al., 2022, Xu et al., 2024).

2. Mathematical and Algorithmic Structure

In formal settings, two-stage pipelines frequently originate in stochastic or robust optimization. The canonical example is the two-stage stochastic program, where first-stage "here-and-now" decisions (e.g., ambulance stationing $x_i$ in OpenEMS) are optimized acknowledging recourse actions in the second stage (vehicle routing or allocation $y_{ij}^m$, shortfall $z_j^m$) once uncertainty (demand scenarios $d^m$) is resolved. The mathematical structure is:

$$\min_{x,\,y^1,...,y^M,\,z^1,...,z^M} \;\frac{1}{M} \sum_{m=1}^M \sum_{j\in J} z_j^m$$

subject to fleet and operational constraints (Ong et al., 2022).

In FlexFL, the two-stage structure parallels this logic but operates in the discrete domain of software entities. Stage 1 employs a suite of fault localization heuristics—IRFL, SBFL, and hybrid SBIR—to prune thousands of methods in a codebase to a short candidate list (typically $m=20$). Stage 2 invokes localized LLM-driven reasoning (Agent4LR) to deeply inspect, rerank, and output the top-$k$ most likely buggy methods (Xu et al., 2024).

3. Pipeline Architectures and Modularization

A defining characteristic is modular decomposition: each pipeline stage may be implemented with distinct programming languages, libraries, and computational backends. For example, OpenEMS delineates four modules:

• Data ingestion and preprocessing (Python)
• Model building and optimization (Julia + JuMP + Gurobi)
• Post-optimization simulation (Julia or Python, discrete-event-based)
• Visualization and analysis (Python: matplotlib, seaborn, geopandas)

Data flows as serialized JSON objects, facilitating decoupling and reproducibility. FlexFL formalizes agent modularity, specifying function-call APIs and JSON/text protocols for agent interactions, and enables any open-source LLM to be substituted as the underlying reasoning engine (Ong et al., 2022, Xu et al., 2024).

4. Detailed Methodologies and Case Studies

OpenEMS: EMS Location and Routing

OpenEMS, developed with Austin-Travis County EMS, implements a two-stage stochastic and robust linear programming framework for ambulance deployment. First-stage decision variables allocate ambulances to stations; the second stage routes these resources under Monte Carlo–sampled scenarios or robust Poisson distributional envelopes (DRO-style). Key techniques include value-at-risk–based ambiguity set construction and column-and-constraint generation. The pipeline produces not only optimized allocations but calibrated, event-driven simulation results that, for Austin, yielded a reduction of mean response times from $7.677 \pm 0.443$ min (historical) to $6.210 \pm 0.037$ min (stochastic) (Ong et al., 2022).

FlexFL: Fault Localization with Open-Source LLMs

FlexFL orchestrates two distinct algorithmic stages for method-level fault localization:

1. Space Reduction: Combines non-LLM heuristics (IRFL, SBFL, SBIR) and LLM-guided searching (Agent4SR) to generate a shortlist of 20 code methods potentially containing bugs.
2. Localization Refinement: Engages LLM-based reasoning (Agent4LR) to inspect and rerank the candidate methods into a top-5 output.

Extensive evaluation on Defects4J (835 Java bugs) shows that FlexFL with Llama3-8B localized 350, 478, and 529 bugs at Top-1, Top-3, and Top-5, respectively, substantially outperforming non-LLM and proprietary-LLM baselines (Xu et al., 2024).

5. Calibration, Validation, and Ablation Analysis

Rigorous validation is integral to two-stage open-source pipelines. OpenEMS performs calibration of travel time mappings (log-linear regression between OSM-derived grid times and actual ambulance times, $R^2 \approx 0.15$) and simulates policy effects using verified discrete-event frameworks, with simulation errors post-calibration tightly controlled ($y_{ij}^m$0 s $y_{ij}^m$1 $y_{ij}^m$2 s) (Ong et al., 2022).

FlexFL ablates input combinations (trigger tests, bug reports, source/tool-calls), agent stages, and function-call protocols to demonstrate criticality of each module. Removal of key inputs or LLM stages causes Top-1 localization rates to degrade dramatically (e.g., from 350 to 45 bugs without source/tool-calls), supporting the necessity of each pipeline phase (Xu et al., 2024).

6. Extensibility and Generalization

Both exemplar systems emphasize extensibility. OpenEMS is configurable for other cities or services (e.g., fire, police) by altering grid resolution, demand models, solver backend, and agent dispatch policy. FlexFL is modular to the LLM/model layer and accommodates various open-source LLMs (Llama3-8B, Mistral-12B, Qwen2-7B) with minimal drops in localization performance. The pipelines' function-call and data APIs, together with containerized or language-agnostic execution, serve as templates for new domains (Ong et al., 2022, Xu et al., 2024).

7. Comparative Performance, Impact, and Implications

Empirical results demonstrate the operational advantage of two-stage open-source pipelines over monolithic or single-stage alternatives. In OpenEMS, stochastic optimization improved mean response times by 88.02 seconds relative to standard dispatch, an increase substantiated by confidence intervals from batched simulation. In FlexFL, the two-stage pipeline achieved 50–80% higher Top-1 localization rates compared to single-stage baselines, while outperforming proprietary LLM-based systems (e.g., GPT-3.5) both in bug coverage and data privacy preservation (Ong et al., 2022, Xu et al., 2024).

A plausible implication is that the structured decoupling of search and refinement phases, allied with open-source toolchains, offers an effective pathway for scalable, transparent, and adaptable solutions across diverse computational research challenges.