FlexCAST: Flexible Data Analysis

• FlexCAST is a framework that generalizes RECAST by preserving a functional to generate analysis, rather than a fixed function.
• It emphasizes modularity, validity, and robustness to adapt analysis workflows to changes in both data and parameters.
• The framework is demonstrated with an anomaly detection case study, enhancing signal significance through retraining and recalibration.

FlexCAST is a framework for flexible scientific data analysis and reinterpretation that generalizes RECAST by preserving the analysis design itself, rather than only a fixed analysis implementation. In the formulation introduced for FlexCAST, reinterpretation is allowed to change both the entire input data and the complete analysis parametrization, so that the preserved object is not a single analysis function but a functional that generates an analysis from data and parameters. The framework is organized around three core principles—modularity, validity, and robustness—and is motivated especially by data-driven machine-learning analyses, where changing the input data can require retraining classifiers, recalibrating thresholds, and revalidating downstream inference (Nachman et al., 15 Jul 2025).

1. Definition and scope

FlexCAST addresses a recurring limitation in scientific workflows: an analysis often contains more reusable structure than is preserved at publication time. Existing reinterpretation frameworks such as RECAST preserve an analysis implementation so that particular parts of the input data can be changed, most notably the signal model, while the analysis parametrization remains fixed. FlexCAST extends that model by allowing reinterpretation to alter the full input dataset and the full analysis parametrization, including data-dependent analysis choices (Nachman et al., 15 Jul 2025).

The framework is intended for the part of a scientific workflow that lies between detector reconstruction and the final scientific result. In the use case emphasized by the paper, this includes event selection, machine-learning classification, histogram construction, fitting, and statistical inference. The motivating examples are analyses in which a fixed preserved pipeline is inadequate: a resonance search whose optimal signal window should depend on the reinterpretation target, and data-driven machine-learning analyses whose trained models depend directly on the data being analyzed (Nachman et al., 15 Jul 2025).

A common misunderstanding is to treat FlexCAST as a mechanism for rerunning old code on new data. The paper draws a sharper distinction. What is preserved is the logic by which analysis choices are derived from data and parameters, not merely the final instantiated choices. In that sense, FlexCAST is designed for analyses whose operational form is itself data-dependent (Nachman et al., 15 Jul 2025).

2. Formalism: from fixed functions to analysis functionals

The paper formalizes the distinction between conventional reinterpretation and FlexCAST through a shift from a fixed function to a functional. In a RECAST-style setting, the analysis is represented as

$r = f(d),$

where $d$ is the input data, $f$ is a fixed analysis function, and $r$ is the resulting scientific output. In FlexCAST, the preserved object is instead

$f = F(d,p),$

where $F$ is an analysis functional and $p$ denotes analysis parameters. Reinterpretation then produces

$f_R = F(d_R, p_R), \qquad r_R = f_R(d_R).$

The essential point is that reinterpretation may generate a new instantiated analysis function, rather than merely replaying an old one on altered inputs (Nachman et al., 15 Jul 2025).

This distinction is operational rather than purely formal. The paper uses two examples to clarify it. If an analysis searches for a resonance with an optimized signal window, preserving only the final cut corresponds to preserving $f$, whereas preserving the optimization rule that determines the cut corresponds to preserving $F$. Likewise, applying a trained neural network is part of $d$0, whereas preserving the training procedure that learns the network from data belongs to $d$1 (Nachman et al., 15 Jul 2025).

The same logic appears at the workflow level. If an analysis is modularized into tasks, the full functional is written as

$d$2

Each task $d$3 is then a parametrized component of the overall design. This decomposition is not merely a software convenience; it is the structural precondition that allows reinterpretation to propagate changes through the analysis in a controlled way (Nachman et al., 15 Jul 2025).

3. Core principles: modularity, validity, and robustness

The paper summarizes the framework by the conceptual identity

$d$4

These three terms define the conditions under which a preserved analysis can be reused meaningfully rather than only executable mechanically (Nachman et al., 15 Jul 2025).

Modularity is the requirement that an analysis be decomposed into granular parametrized tasks arranged in a directed acyclic graph. Storage-based interfaces connect tasks, and outputs are represented by targets that are unique for a particular set of task parameters. This makes it possible to change data or external parameters, rerun only affected downstream tasks, and localize both recomputation and validation. The paper distinguishes internal parameters, which are fixed during design or determined dynamically inside the workflow, from external parameters, which are intentionally exposed for reinterpretation (Nachman et al., 15 Jul 2025).

Validity is the requirement that the resulting instantiated analysis remain scientifically meaningful after reinterpretation. Because FlexCAST allows data and parametrization to change, validity cannot simply be inherited from the original publication. The framework therefore associates testing tasks with production tasks. These tests return at least two outputs: a qualitative status and quantitative notes. Status is defined through a three-level convention: green for deviations consistent with $d$5, yellow for $d$6 and $d$7, and red for $d$8. The paper explicitly allows these thresholds to be grounded in mathematical expectations, p-values, validated assumptions, best practices, or heuristics when necessary (Nachman et al., 15 Jul 2025).

Robustness is defined as the extent of parameter space over which a parametrized task can operate successfully, meaning it passes its associated tests. Validity certifies a given point in parameter space; robustness enlarges the region over which such certification can be obtained. The paper describes robustness as especially important for fitting procedures and machine-learning components. Recommended robustness mechanisms include iterative fitting strategies that increase complexity only as needed, training-data augmentation, regularization, choosing the simplest sufficiently performant models, ensemble methods, and fine-tuning pretrained models during reinterpretation (Nachman et al., 15 Jul 2025).

4. Workflow architecture and execution model

The implementation discussed in the paper uses Python, the Law workflow library, Luigi-style task parameterization, Docker-based sandboxing, and GitLab CI/CD, although the authors present the architectural principles as implementation-agnostic. The workflow is represented as a task DAG with three task classes: production tasks, testing tasks, and supervision tasks. Production tasks perform the scientific computation; testing tasks validate production-task behavior at runtime; supervision tasks provide non-critical monitoring or visualization (Nachman et al., 15 Jul 2025).

Task interfaces are storage-based. Each task declares its inputs, outputs, and parameters, and the target corresponding to a task output must be unique for the parameter set that produced it. This uniqueness underlies reproducibility, caching, and selective recomputation. In the implementation described, task parameters are embedded directly in task code rather than delegated to external configuration files (Nachman et al., 15 Jul 2025).

A significant technical point is that FlexCAST testing is not treated as ordinary software testing with mocks. The framework instead emphasizes tests on actual runtime data and runtime parameter settings. This is particularly important for machine learning, where relevant checks may include data schemas, data distributions, training-loss evolution, train/validation accuracy comparison, overtraining diagnostics, numerical stability such as NaNs or infinities, bounded weights, expected neuron activation ranges, and memory or inference performance. The paper proposes a reusable “check catalogue” of such best-practice validations for different algorithm classes (Nachman et al., 15 Jul 2025).

This suggests a different preservation target from conventional workflow archival. FlexCAST preserves not only code and environments, but also the orchestration logic, the parameter semantics, and the test structure needed to determine whether a re-instantiated analysis remains acceptable.

5. Demonstration on anomaly detection with LHC-like data

The paper demonstrates FlexCAST on a weakly supervised anomaly-detection analysis using CWoLa (“Classification Without Labels”) and the public LHC Olympics R&D dataset. The target signal process is

$d$9

with benchmark masses

$f$0

Events are clustered into jets with anti-$f$1 and

$f$2

The feature set includes jet masses $f$3 and the $f$4-subjettiness ratios

$f$5

specifically $f$6 and $f$7 for the two leading jets (Nachman et al., 15 Jul 2025).

Four benchmark datasets are used: one background-only sample with about $f$8 events, and three “blackbox” datasets containing the same amount of background plus injected signal corresponding to inclusive signal-region significances of $f$9, $r$0, and $r$1. Each dataset is split 50/50 into training and application subsets (Nachman et al., 15 Jul 2025).

The production workflow contains tasks for dataset retrieval, data preparation, classifier training, classifier application, histogramming, fitting, inference, and repeated inference under a sliding signal-region scan. In the demonstrator, the signal-region width of

$r$2

is an internal parameter, and the sidebands each also have width $r$3. External parameters include the dataset, classifier input features, the number of classifiers in the ensemble, the number of hidden-layer nodes, the number of epochs, and the target background rejection efficiency (Nachman et al., 15 Jul 2025).

The histogram task constructs $r$4 histograms with nine bins total: three in each sideband and three in the signal region. The nominal histogram is the mean over the ensemble’s individual classifier histograms. The fit task uses an exponential sideband fit and improves robustness by iteratively increasing model complexity until

$r$5

or the most complex function is reached (Nachman et al., 15 Jul 2025).

The testing workflow is similarly explicit. TestClassifierBias compares training and validation cross-validation losses, with heuristic thresholds of relative difference below $r$6 for green, $r$7–$r$8 for yellow, and above $r$9 for red. TestHistogramSpread evaluates ensemble consistency using

$f = F(d,p),$0

where $f = F(d,p),$1 is the count from classifier $f = F(d,p),$2 in bin $f = F(d,p),$3, and $f = F(d,p),$4 and $f = F(d,p),$5 are the mean and standard deviation over the remaining classifiers. TestHistogramStats requires at least 10 events in every histogram bin. TestFit derives a p-value from the sideband-fit $f = F(d,p),$6 and number of degrees of freedom and maps it to the green/yellow/red status convention (Nachman et al., 15 Jul 2025).

The demonstrator’s main empirical result is that, in benchmark dataset blackbox 2, an injected inclusive signal-region significance of $f = F(d,p),$7 is enhanced to nearly $f = F(d,p),$8 after the anomaly-detection pipeline. A similar enhancement is observed for blackbox 3, whereas the background-only sample and blackbox 1 do not exhibit significant excesses. The paper also reports that across all benchmark datasets most tests are green, some are yellow, and no tests are red or failing. One explicitly quoted example is a TestFit result on benchmark 2 with green status and p-value $f = F(d,p),$9 (Nachman et al., 15 Jul 2025).

The authors present this demonstrator as, to their knowledge, the first fully reinterpretable implementation of an anomaly-detection analysis on LHC-like data. That claim is framed as a practical demonstration of the framework’s central point: a data-driven ML analysis can be preserved not as a frozen trained artifact, but as a validated, rerunnable design (Nachman et al., 15 Jul 2025).

6. Significance, limitations, and relation to prior reinterpretation frameworks

FlexCAST is presented as a direct generalization of RECAST. RECAST preserves a fixed analysis implementation and is therefore well suited to reinterpretations that modify only restricted input components, especially the signal model, while leaving selections, trained model weights, and other analysis parameters unchanged. FlexCAST trades that simplicity for broader reinterpretation scope. It supports retraining, recalibration, and parameter updates, but correspondingly requires explicit revalidation of the resulting instantiated analysis (Nachman et al., 15 Jul 2025).

This broader flexibility has practical consequences. The paper states that FlexCAST requires more computational resources than RECAST because it reruns larger portions of an analysis rather than minimally substituting an input component. It also requires more preservation discipline: task decomposition, parameter semantics, software environments, test definitions, and validity criteria all have to be preserved. Human intervention remains necessary in designing the decomposition, identifying external parameters, specifying tests, and reviewing whether the framework has been applied correctly. The paper states that collaboration-level validation should occur at the latest during analysis review (Nachman et al., 15 Jul 2025).

The framework’s current limitations are explicit. The demonstrator’s tests are illustrative rather than exhaustive; some are heuristic rather than derived from a universal statistical formalism. The anomaly-detection case study is methodologically substantial but computationally lightweight relative to full experimental workflows. The paper also notes unresolved infrastructure questions, including integration with REANA, access to Slurm or HTCondor resources, hardware acceleration, persistent or shared targets, and improved user interfaces (Nachman et al., 15 Jul 2025).

At the same time, the paper’s broader implication is clear. FlexCAST is not restricted to high-energy physics, even though its motivating example is. A plausible implication is that any scientific workflow whose effective analysis function depends materially on input data—especially through data-driven optimization or machine learning—fits the design space that FlexCAST is intended to address. In that sense, the framework shifts reinterpretation from archival replay toward validated re-instantiation of an analysis design (Nachman et al., 15 Jul 2025).