Research Execution Plan (REP)

• Research Execution Plan (REP) is a structured template that unifies experiment specification, environment setup, data management, and execution orchestration to ensure reproducibility.
• It details methods for capturing computational environments, automating workflows, and validating results across domains with clear success criteria.
• REP underpins reproducibility platforms like SciRep and RRP, enabling seamless sharing, verification, and portable execution across varied systems.

Searching arXiv for the cited papers on reproducibility platforms, execution-grounded research, and REP-related protocols to ground the article in current literature. Research Execution Plan (REP) denotes a structured plan for specifying, executing, validating, packaging, and disseminating computational research so that results can be re-executed and assessed across time, systems, and domains. In recent literature, REP is used as a cross-domain planning and operational template that binds objectives, inputs and outputs, code, data, programming languages, dependencies, databases, commands, validation criteria, and publication artifacts into a single executable workflow. In this role, REP is closely associated with reproducibility infrastructure such as SciRep and the Reproducible Research Platform, and with domain-specific execution frameworks in automated AI research, database systems, and multi-hop question answering (Costa et al., 10 Mar 2025, Cuny et al., 4 Dec 2025).

1. Conceptual foundations

A REP is grounded in the distinction between reproducibility and replicability. One formulation states: “Reproducibility is the ability to replicate the results of a study via the original methods, materials, and data, whereas replicability is the capacity to obtain the same results via new methods, materials, and data and/or conditions that are consistent with those of the original study.” In SciRep’s evaluation, “Results Validated – Reproduced” means reproducing the main results with the original artifacts, whereas independent reimplementation was not evaluated (Costa et al., 10 Mar 2025).

Within this framework, a REP is not merely a prose protocol. It is an executable specification that records how an experiment is configured, how it is run, what outputs are expected, and how those outputs are checked. This includes the computational environment, exact commands, parameterization, and, where necessary, stateful resources such as databases. The produced artifact may be a portable “capsule” or a published executable research object that can be re-executed on another machine with minimal local setup (Costa et al., 10 Mar 2025, Cuny et al., 4 Dec 2025).

The acronym is not unique across the literature. In multi-agent systems, REP can also denote the “Ripple Effect Protocol,” a coordination protocol in which agents exchange decisions and lightweight sensitivities. That usage is conceptually distinct from Research Execution Plan, though both share an emphasis on explicit protocol structure and repeatable execution (Chopra et al., 18 Oct 2025).

2. Core elements of a Research Execution Plan

Across the cited work, a REP typically begins with experiment specification. This includes objectives and hypotheses, explicit inputs and outputs, success criteria for reproducibility and replicability, and a randomness policy defining seeds and determinism controls. The success criterion for reproducibility is identical outputs across re-runs with the same inputs, while replicability is defined in terms of consistent trends or results under substituted datasets or equivalent methods, with acceptable deviation explicitly defined (Costa et al., 10 Mar 2025).

Environment capture is central. A REP records the base operating system image, language runtimes, compilers, system packages, language-specific dependencies, build scripts, Dockerfiles, and any required environment variables. In SciRep, programming languages and dependencies are inferred automatically from file extensions and conventions such as requirements.txt, pom.xml, Makefile, and import statements; generated Dockerfiles pin language runtimes and system packages to image tags such as ubuntu:20.04. Supported languages include C++, Perl, R, Java (Maven), Python, Unix Shell, and Jupyter Notebook (Costa et al., 10 Mar 2025).

Data management is treated as a first-class component. A REP records provenance, dataset versioning, checksums, access policies, directory structure inside the image, and whether data are packed inside the artifact or fetched at run time. Where privacy or licensing prevents inclusion of data, the plan records metadata and access instructions instead. For stateful resources, a REP can specify the database engine, schema, seed data, initialization scripts, and connectivity conventions. SciRep supports MongoDB, SQLite, MySQL, and PostgreSQL, and integrates database configuration during build and run (Costa et al., 10 Mar 2025).

Execution orchestration is explicit rather than implicit. Commands and parameters are captured as part of the plan, multi-step pipelines may be encoded in scripts or workflow engines, logs are preserved, and timeouts and resource expectations are documented. Determinism controls include explicit seeds for AI and statistical workloads, pinned library versions, and, where necessary, floating-point tolerances for output comparison (Costa et al., 10 Mar 2025).

In domain-specific systems, the plan may also maintain higher-level execution state. In execution-grounded automated AI research, the loop consists of idea generation, automated implementation, experiment execution, feedback, and search or model update. In REAP for multi-hop question answering, the maintained state includes a sub-task set and a facts set, and execution terminates when the plan is fully resolved or the step budget is exhausted. These examples suggest that REP can denote not only environment capture but also explicit control state for iterative research procedures (Si et al., 20 Jan 2026, Zhu et al., 13 Nov 2025).

3. Workflow, automation, and execution infrastructure

SciRep provides a concrete workflow for REP execution. A project is created, code and data are uploaded from a ZIP archive or imported from GitHub, the system detects languages and dependencies, generates a Dockerfile, builds a Docker image, runs the experiment inside a container with an exact command, captures logs and outputs, compares them against registered expected outputs, and finally packages the entire artifact as a portable capsule (Costa et al., 10 Mar 2025).

The artifact model is container-centric. Containers, specifically Docker images plus simple scripts, are the primary portability mechanism. The package can include code, data, configuration, commands, and optional database initialization assets, and can be re-executed on Windows, macOS, or Linux provided Docker is installed. SciRep abstracts container technology from the researcher and exposes an API rather than a graphical user interface in its current form (Costa et al., 10 Mar 2025).

A second execution model is provided by the Reproducible Research Platform. RRP is a Kubernetes-based web application with persistent storage, project-level builders and publishers, a minimal GUI with six tabs, and integration with openBIS ELN-LIMS for research data management. It captures software environments through REES-compliant repo2docker builds from Git repositories, mounts datasets directly from the RDMS into standardized project folders such as /openbis, and supports familiar interfaces including JupyterLab, RStudio, VS Code, and full desktop environments (Cuny et al., 4 Dec 2025).

The RRP lifecycle extends the REP concept from proposal through publication and post-publication reuse. Datasets are registered in the RDMS and receive persistent identifiers, environments are built from .binder specifications, outputs are stored in /results, and complete executable research objects can be disseminated as “player bundles,” “player scripts,” shareIDs, OCI images, and DOI-backed publications through Zenodo or institutional repositories. This architecture binds data, code, environment, and provenance in a single project model rather than treating them as separate workflows (Cuny et al., 4 Dec 2025).

4. Validation and empirical performance

Validation in a REP is based on registered expected outputs, output hashes, regression tests, or domain-specific metrics. In SciRep, expected outputs are stored within the project and compared automatically on each run. In its evaluation, 28 experiments were collected across VLDB, ICSE, climate change, medicine, artificial intelligence, and related tool case studies; 18 were runnable and appropriate for reproduction. SciRep executed 16 of 18 experiments, or 89%, and for all executed experiments the produced results matched the originals, yielding 100% reproducibility verification for the experiments it could run. The baselines were Code Ocean with 11 of 18, or 61%, RenkuLab with 9 of 18, or 50%, and Whole Tale with 1 of 18, or 6% (Costa et al., 10 Mar 2025).

The same planning logic appears in execution-grounded automated AI research, but here validation is tied to reward from automated execution. Two research problems were converted into execution environments: pre-training and post-training. Execution-guided evolutionary search found a method that significantly outperformed the GRPO baseline on post-training, with 69.4% versus 48.0%, and found a pre-training recipe that outperformed the nanoGPT baseline, reaching the target in 19.7 minutes versus 35.9 minutes, all within ten search epochs. Reinforcement learning from execution reward improved average reward but not the upper bound, and the paper reports mode collapse toward simple, easy-to-implement ideas (Si et al., 20 Jan 2026).

In REAP for multi-hop question answering, REP-style execution and evaluation are expressed through recursive sub-task planning, fact extraction, and bounded iteration. On HotpotQA, REAP achieved 59.2 CEM, 68.0 F1, and 72.4 ACC†, compared with 56.8, 63.4, and 68.4 for R1-Searcher. On 2WikiMultihopQA, REAP achieved 79.2, 79.6, and 81.6. On the out-of-domain datasets MuSiQue and Bamboogle, REAP achieved F1 scores of 38.3 and 65.2, respectively. Ablations showed degradation when replanning, verification, or clue handling was removed (Zhu et al., 13 Nov 2025).

In database systems, a REP can be used for faithful reproduction and extension of a performance-prediction study. One plan specifies an automated pipeline that generates more than 1000 parameterized TPC-H queries, executes them in PostgreSQL with EXPLAIN ANALYZE, extracts scalar, structural, and semantic features, and evaluates regressors for runtime prediction. The refined XGBoost model achieved an MSE of 0.3002 and prediction accuracy within 10% of the true runtime in 65.52% of cases, outperforming the flat-feature XGBoost baseline at 0.3607 MSE and 57.23% accuracy within 10% (Pathak et al., 7 Oct 2025).

5. Representative domains and plan variants

In cross-domain computational reproducibility, REP functions as a reusable template. The SciRep-based template covers experiment specification, environment capture, data management, database handling, execution orchestration, determinism controls, validation, packaging, and optional comparative benchmarking against alternative tools. This template is explicitly intended for domains ranging from medicine to computer science (Costa et al., 10 Mar 2025).

In automated AI research, REP becomes an execution-grounded research loop. The ideator proposes natural-language ideas, the executor synthesizes code diffs and patches baseline codebases, worker nodes run experiments, rewards are computed from execution outcomes, and the ideator is updated through evolutionary search or reinforcement learning. The executor samples 10 candidate code diffs in parallel and can perform up to 2 self-revision iterations when patching fails. The implementation operates at substantial scale, including group sizes of 256 for GRPO and 128 for nanoGPT, the latter requiring 1024 GPUs in parallel per RL batch because each idea uses 8 GPUs (Si et al., 20 Jan 2026).

In multi-hop QA, the plan is recursive and stateful. The Decomposer generates an initial plan, the Sub-task Planner selects executable sub-tasks, the Fact Extractor retrieves documents and emits structured facts, and the Synthesizer produces the final answer. The global knowledge state is represented by sub-tasks and facts, and the fulfillment level of a fact is classified as DirectAnswer, PartialClue, or Failed. The iterative budget is capped at 5 rounds in the reported experiments (Zhu et al., 13 Nov 2025).

In query optimization research, the plan emphasizes workload construction, trace collection, feature extraction, model training, and integration into the optimizer workflow. The reproduced environment uses Ubuntu 20.04, PostgreSQL 14.7, Python 3.12, and a laptop-class machine with an Intel Core i5-1135G7 and 16 GB RAM. The REP formalizes storage schemas for queries, plans, and plan nodes, as well as train-test splits, feature families, and error-analysis procedures (Pathak et al., 7 Oct 2025).

In open science infrastructure, REP is integrated with institutional data governance. RRP spans proposal, project setup, data acquisition, analysis, execution and validation, publication and archiving, and post-publication maintenance. It is explicitly project-centric and unifies research data management with executable computational environments and version-controlled code. Case studies include pyPOCQuant in biomedical imaging, a decade-old archaeology workflow in R, and a neuroscience or machine-learning workflow requiring Python 3.6 and deprecated dependencies (Cuny et al., 4 Dec 2025).

6. Limitations, ambiguities, and future directions

The principal limitations reported for REP-style systems concern hardware dependence, extensibility, metadata fidelity, and operational overhead. SciRep explicitly does not target hardware-dependent experiments in the reported work; GPU dependencies, accelerator-sensitive execution, and HPC setups are out of scope. Adding new languages or database engines requires container expertise, FAIR metadata support inside the container is limited, large images can be difficult to store and share, and Docker installation on the target machine remains an external dependency (Costa et al., 10 Mar 2025).

Execution-grounded AI research exposes a different set of constraints. The executor can fail on invalid diffs or runtime errors, rewards may be zero or NaN, and reinforcement learning from execution reward can collapse onto simple ideas. The paper reports that dynamic prompts, length rewards, and diversity rewards maintained thinking length or diversity but did not improve the upper bound in early epochs, indicating that stronger interventions such as novelty search, ensemble ideators, or hierarchical planning may be required (Si et al., 20 Jan 2026).

For REAP, the main risks are retrieval noise, planning drift, clue extraction errors, and the cost of recursive evaluation. Performance remains dependent on the quality of the underlying LLMs and retrievers, and domain transfer to highly specialized corpora may require domain-adaptive retrieval or added supervision. Nevertheless, the architecture is presented as extensible toward probabilistic sufficiency assessment, graph-based memory, and end-to-end reinforcement learning over planning and extraction modules (Zhu et al., 13 Nov 2025).

Open science platforms add governance and infrastructure considerations. RRP notes that the largest datasets and GPU or AI workloads are not yet fully addressed and that integration with HPC or GPU clusters is ongoing. Federated infrastructures, including EOSC integration, and AI-assisted interfaces are identified as future directions. A plausible implication is that mature REP practice increasingly depends on coupling reproducibility tooling with institutional RDMS infrastructure, persistent identifiers, and long-term executable packaging rather than on containerization alone (Cuny et al., 4 Dec 2025).