Consistency of the observation-ablation runtime difference

Determine whether the runtime advantage of disabling observations on the bsg_link benchmark is consistent across repeated runs rather than attributable to run-to-run variation.

Background

The paper evaluates Agentic-IC3 with and without its derived-observation mechanism. On the bsg_link benchmark, the configuration without observations completes faster than the full system. Because the experiments use agent interactions and may exhibit run-to-run variability, the paper does not establish whether this observed runtime difference is reproducible. Repeated experiments are required to determine its consistency.

References

The run without observations is faster on bsg_link, but this difference may reflect variation between runs; repeated runs are needed to determine whether it is consistent.

— Agentic-IC3: Enabling Semantic Proof Search in IC3 Model Checking  (2609.27162 - Fan et al., 22 Sep 2026) in Section 5.2, subsection “Ablation Study of Individual Mechanisms,” paragraph “Observations”