Seeds+Logs Artifact: Reproducibility Approach

• Seeds+Logs Artifact is a framework that combines initialization data with detailed runtime logs to enable accurate re-execution and auditability of workflows.
• It emphasizes preserving starting conditions and complete execution evidence using fixed environment parameters and structured logs for reliable reproducibility.
• Empirical evaluations show that structured workflow graphing and environment normalization markedly improve artifact reproducibility and validation success rates.

Searching arXiv for the cited software-artifact evaluation paper and closely related artifact-sharing work to ground the article. “Seeds+Logs Artifact” denotes an artifact perspective in which reproducibility depends on preserving both the initial conditions of execution and the runtime evidence needed to reconstruct, audit, and validate a workflow. In the software-engineering literature, the clearest instantiation is “Agent-Based Software Artifact Evaluation,” which defines “end-to-end artifact evaluation” as beginning after repository download and ending at badge determination, and operationalizes reproducibility as successful re-execution of the documented workflow under a normalized environment, with observed outputs aligned with paper claims and official artifact results (Wu et al., 2 Feb 2026). Across adjacent work on artifact hosting, artifact packaging, provenance, and domain-specific datasets, the same perspective recurs: code alone is insufficient when execution context, command structure, intermediate products, and logs are missing or unstable (Liu et al., 2024).

1. Conceptual scope and definitional boundaries

The term encompasses two related but distinct ideas. In reproducibility-oriented software artifacts, “seed” refers primarily to the initial corpus, execution setup, or initialization context from which a workflow begins, whereas “logs” refer to the execution traces, statuses, failures, and outputs that make the workflow inspectable and replayable. In domain-specific data work, “seed” may also be literal, as in ancient plant seed image datasets and seed-image segmentation pipelines (Xing et al., 20 Dec 2025, Vale et al., 2020).

The software-engineering definition of artifact is broad. Liu et al. adopt the formulation that an artifact is “any external materials or information provided in conjunction with a research paper via a link,” which makes the category large enough to include code, data, documentation, archived artifacts, and reproducibility packages (Liu et al., 2024). Within that broad category, the Seeds+Logs perspective isolates a narrower requirement: not merely availability, but preservation of the information necessary to rerun, inspect, and validate a workflow.

A central clarification is that the perspective is asymmetric in the current literature. “Agent-Based Software Artifact Evaluation” is strong on the “logs/evidence/stable environment” half and only indirectly informative on the “seeds” half. The paper explicitly states that it does not report that ArtifactCopilot extracts, sets, or verifies random seeds in artifacts; no seed schema is proposed; and no repeated-run methodology is used to test whether a stochastic artifact reproduces within variance bounds (Wu et al., 2 Feb 2026). This suggests that a Seeds+Logs Artifact is not yet a standardized protocol, but rather an overview of requirements distributed across reproducibility, provenance, and artifact-sharing work.

2. The seed side: initial corpus, initialization context, and fixed inputs

In the narrow reproducibility sense, the “seed” component is the specification of what the system starts from. In “Agent-Based Software Artifact Evaluation,” inputs to the automated system are either the paper PDF, from which the system extracts artifact links and repository context, or a pre-downloaded repository with confirmed availability. For each artifact, the authors create a manually validated Golden Command Set extracted from the README by two annotators with a third resolving disagreements. They also fix the execution platform: Ubuntu 22.04 LTS, NVIDIA RTX 4090, and Docker 24.0.1 in the preliminary study, and Ubuntu 22.04 with RTX 4090 in the evaluation (Wu et al., 2 Feb 2026). These are not random seeds, but they function as execution seeds in the broader sense of initialization control.

A related example appears in the SBOM-mining literature. “Software Bills of Materials in Maven Central” defines a clear seed corpus: the public Goblin Maven dependency graph dump “as of August 30, 2024,” with 14,459,139 release nodes, from which the authors select a 10% per-year stratified sample yielding 1,445,910 sampled release nodes (Gamage et al., 23 Jan 2025). The paper is therefore a good seed-data paper, but only a weak logs/provenance paper; it does not report a random seed, the explicit list of sampled release coordinates inside the paper, or crawl logs.

The provenance literature formalizes the same issue at a more abstract level. In artifact-centric BPM, the proposed logging schema must support “Incoming event <-> Artifact (Artifact initialization),” and in Hierarchical Artifact Systems the closest formal analogue to a seed is the root input valuation satisfying $\Pi$, together with default initialization of non-input variables and initially empty artifact relations [(Baradaran-Hosseini, 2014); (Deutsch et al., 2016)]. A plausible implication is that a Seeds+Logs Artifact should treat initialization as a first-class object, whether it is a repository snapshot, a sampled corpus, a root task valuation, or an explicitly preserved parameter bundle.

The literal-seed literature shows the same concern with controlled initial conditions. The APS dataset contains 8,340 images from 17 genus- or species-level seed categories excavated from 18 archaeological sites across China, and most images were acquired at fixed 1.6× magnification specifically to preserve meaningful inter-class size differences (Xing et al., 20 Dec 2025). Likewise, the ImageJ-based seed segmentation plugin was evaluated on 3,386 seed samples from 120 species, with acquisition constrained by scanner model, blue background, minimum 400 dpi, and non-overlapping seeds (Vale et al., 2020). Here the “seed” is the artifact itself, but the logic is similar: reproducible downstream analysis depends on preserving acquisition conditions rather than treating them as incidental.

3. The logs side: execution evidence, state representation, and provenance

The strongest contribution of the Seeds+Logs perspective is its insistence that logs are more than stdout and stderr. ArtifactCopilot records “detailed execution logs recording command statuses, outputs, and errors for auditability,” but it also externalizes execution state into an Artifact Evaluation Graph, $G=(V,E)$, whose nodes represent Start, Command, and Artifact, and whose edges encode sequential and artifact dependencies (Wu et al., 2 Feb 2026). Command nodes carry attributes including env and status; artifact nodes carry path and type; and the Start node contains a use_docker flag. This means the execution ledger is structured, not merely textual.

The same paper emphasizes several additional forms of runtime evidence: failure traces used by an LLM-based diagnostic module; environment state in the container environment description; resource monitoring through sustained low CPU utilization as a signal of stalled execution; and semantic compression of prior traces into a compact state summary (Wu et al., 2 Feb 2026). The underlying claim is that artifact evaluation resists automation because execution state is implicit in README prose and failures are not visible in a structured enough way for automated recovery.

This concern with provenance predates recent agent-based systems. The artifact-centric BPM logging paper argues that provenance is needed to explain artifact instance creation and evolution, and identifies correlations such as incoming event to artifact initialization, artifact to artifact association, guard to stage, and stage to milestone. Its central runtime unit is the GSM B-step: “handling of a single incoming event into a GSM system including all achieving/invalidating milestones and opening/closing stages caused by processing on that event” (Baradaran-Hosseini, 2014). In parallel, “Artifact Lifecycle Discovery” starts from raw event logs with no case notion and transforms them into one artifact-centric log per artifact type and one trace per artifact instance (Popova et al., 2013). “Verification of Hierarchical Artifact Systems” then distinguishes local runs, trees of local runs, and global runs as increasingly flattened views of artifact execution (Deutsch et al., 2016).

Taken together, these works suggest that a Seeds+Logs Artifact is best understood as a combination of initialization metadata and provenance-rich runtime state. Logs alone are not enough; nor are static manifests. What matters is a representation that links initial inputs, execution steps, artifacts consumed and produced, failure points, and final outcomes.

4. Workflow structure, environment control, and automated reproducibility

The operational core of the perspective is workflow externalization. ArtifactCopilot is a hierarchical, agent-based framework with three stages: Repository Acquisition and Task Modeling; Environment Preparation; and Execution and Evaluation. In Stage 2, the Planning Agent uses one of three adaptive container-construction strategies: Dockerfile reuse, automatic Dockerfile synthesis, and template fallback to a predefined Ubuntu 22.04 base template if image construction fails after $k=3$ attempts. In Stage 3, the Evaluation Agent traverses the AE Graph in topological order, executes commands, updates node statuses, diagnoses failures using logs and traces, retries failed commands up to five times, and produces detailed execution logs and an evaluation report (Wu et al., 2 Feb 2026).

Environment normalization is a decisive design point. The paper motivates it by observing that commands such as docker run -it silently switch execution context from host to container, creating instability for automated agents. ArtifactCopilot therefore enforces a host-centric unified scheduling model in which all commands are issued from the host and explicitly mapped to the target container through a fixed execution interface (Wu et al., 2 Feb 2026). In the ablation study, removing environment normalization reduced Badge Consistency Rate from 85.42% to 56.25%, a 29.17-point drop; removing the AE Graph reduced BCR to 41.67%; and replacing the hierarchical two-agent architecture with a single agent yielded 62.50% (Wu et al., 2 Feb 2026). These results make the workflow claim explicit: reproducibility is materially improved when execution structure and environment semantics are machine-readable.

The same lesson appears, in a more specialized form, in the flaky-job literature. FlakeRanker packages regex-based labeling, complete analysis results, and RFM prioritization into a CLI with label, analyze, rank, and run, where the main command executes (input) => label => analyze => rank => (output) on a dataset of jobs with metadata and execution logs (Aïdasso, 16 Mar 2025). This is not an environment-normalization framework, but it exemplifies the same principle: reusable artifacts depend on explicit stage boundaries, explicit inputs and outputs, and preserved intermediate products.

5. Packaging, hosting, and long-term accessibility

A Seeds+Logs Artifact is not only executed; it must also be discoverable and durable. Liu et al. show that among 2,196 papers published in ASE, FSE, ICSE, and ISSTA from 2017 to 2022, 1,487 offered artifacts, yielding an overall artifact-sharing rate of 67.7%. They also show that GitHub was dominant in 2022 at 64.2%, Zenodo rose from 0.0% in 2017 to 16.0% in 2022, and overall unavailable artifacts due to link rot reached 9.4%, with temporary drives at 32.6%, personal homepages at 11.8%, GitHub at 6.4%, and artifact service platforms such as Zenodo at 7.1% (Liu et al., 2024). The implication is direct: logs, seeds, and result bundles are only as reproducible as the platform that preserves them.

The documentation problem is equally concrete. The same study evaluates README quality using six criteria—Completeness, Structure, Usability, Example, Certificate, and Contact—and reports, for analyzed ICSE GitHub artifacts, Completeness at 49.8%, Structure at 27.9%, Usability at 60.8%, and Example at 26.2% (Liu et al., 2024). Since incomplete README files can “greatly damage the reproducibility of an artifact, even rendering the artifact completely unusable,” a Seeds+Logs Artifact requires documentation that explains not only how to run the code, but how initial inputs, logs, generated artifacts, and reported results correspond.

In information retrieval, “Artifact Sharing for Information Retrieval Research” proposes a packaging substrate that directly addresses this problem. Its framework consists of Artifact Serialization File and metadata, Artifact Hosts, and Artifact Classes. The serialization format is a compressed TAR file using LZ4 compression; metadata is stored in pt_meta.json; timestamps are removed by default; and artifact files are sorted lexically by default so that “functionally-identical artifacts are encoded to the same serialization file” (MacAvaney, 8 May 2025). The system supports multiple hosts, including HuggingFace Hub, Zenodo, Magic-Wormhole, PyTerrier Data Repository, CIFF Hub, Pyserini prebuilt-index listing, TIREx, and arbitrary URLs. Although the paper does not define a seed/log-specific schema, it explicitly supports metadata-rich packaging and immediate reuse of intermediate artifacts such as built indexes and caches.

6. Performance, limitations, and future directions

The most explicit empirical demonstration of the Seeds+Logs perspective remains ArtifactCopilot. Evaluated on 48 real artifacts from ICSE, FSE, ASE, and ISSTA, it achieves 100% Link Consistency Rate, 100% Repository Acquisition Success Rate, and 85.42% Badge Consistency Rate, versus 33.33% BCR for Claude Code and 20.83% for LLM+Scripts. Average intervention count is 0.11 for ArtifactCopilot, 5.32 for Claude Code, and 0 for LLM+Scripts; average cost per artifact is $0.091 for ArtifactCopilot, $0.078 for Claude Code, and $0.002 for LLM+Scripts; and 45 of 48 artifacts completed with zero human intervention, with the remaining three requiring input only for sudo permission requests (Wu et al., 2 Feb 2026). These numbers show that structured execution evidence, normalized environments, and explicit workflow graphs can support functional-level reproducibility at nontrivial scale.

The limitations are equally important. ArtifactCopilot does not model scientific reproducibility under stochastic variation; there is no repeated-trial protocol, no distributional acceptance criterion, and no built-in seed extraction or seed verification. The AE Graph captures artifact existence and dependencies but not data content validity. Code-level implicit dependencies remain hard to infer, cross-file configuration constraints can escape document-level parsing, and the evaluation excludes artifacts needing specialized hardware, private datasets, commercial software, or very large storage (Wu et al., 2 Feb 2026). FlakeRanker exposes a complementary limitation on the log-analysis side: the artifact paper excerpt does not disclose exact RFM formulas, clustering algorithm, hyperparameters, or random seeds (Aïdasso, 16 Mar 2025). The SBOM paper similarly releases valuable outputs but little process-log evidence (Gamage et al., 23 Jan 2025).

Future directions are already visible in the literature. “Agent-Based Software Artifact Evaluation” explicitly suggests tighter integration of code-level dependency analysis, stronger data-flow validation, and finer-grained constraint modeling in the AE Graph, and from a Seeds+Logs standpoint identifies the natural extension: explicit seed extraction and control, variance-aware output checking for stochastic artifacts, and richer persisted execution provenance that links seeds, commands, environment hashes, and outputs (Wu et al., 2 Feb 2026). A plausible implication is that a mature Seeds+Logs Artifact standard would combine three layers now studied separately: stable and archivable packaging, machine-readable workflow and environment semantics, and explicit provenance linking initialization state to runtime evidence and final claims.