MultiSHAP: Static Validation in Evolving Systems

• MultiSHAP is a static validation framework that assesses whether program, database, or model updates preserve specified semantic properties across all executions.
• It employs techniques like call-graph differencing, automata-theoretic typing, and FO+MOD enumeration to analyze the impact of updates and flag unsafe behavioral changes.
• Empirical evaluations demonstrate that MultiSHAP’s static change-impact analysis significantly improves fault detection compared to dynamic testing, optimizing dependency management and schema evolution.

MultiSHAP refers to a class of analyses, algorithms, or tools that support static validation under updates—the a priori, pre-execution assessment of whether program, database, or model changes preserve specified semantic properties across all possible executions. MultiSHAP systems natively support multiple specification and update paradigms (e.g., static/dynamic, syntactic/semantic, or multi-property), aiming for high sensitivity in detecting unsafe or property-violating consequences of updates. The underpinning theme is automated reasoning about possible state evolutions, allowing one to rule out or certify classes of regressions, bugs, or specification violations before deployment. This is crucial in environments where runtime revalidation is impractical, coverage is incomplete, or failures have high cost.

1. Foundational Formalisms for Static Validation under Updates

The static validation paradigm abstracts updates as transformations (syntactic edits, logical rules, graph operations, or parameter modifications) over state spaces such as code, data, or models. The essential problem is: for a given specification $\mathcal{P}$ and update $U$, decide whether for all states $S$ satisfying $\mathcal{P}$, $U(S)$ also satisfies $\mathcal{P}$.

Several foundational approaches exemplify MultiSHAP capabilities:

• Call-Graph and Diff-Based Analysis for Software Dependencies: Uppdatera constructs call graphs $G=(V, E)$ integrating the client application and all dependencies, computes AST-level diffs $D$, and uses reachability analysis to flag potentially unsafe interactions between updates and client code. Safety is certified if no client function transitively invokes an updated, semantically changed function (Hejderup et al., 2021).
• Automata-Theoretic Typing and Rewriting in XML/Data: Rewrite-based frameworks model XML documents as terms in unranked trees; updates as parameterized rewrite rules; and type invariants as hedge automata languages. The post-update closure $post^*_{R_A}(L)$ is computed to statically check whether all reachable states post-update remain well-typed. Safety and policy consistency reduce to automata inclusion or emptiness (0907.5125).
• First-Order/MOD Query Enumeration in Databases: FO+MOD dynamic evaluation utilizes Hanf normal forms to reduce any query to a finite set of local spheres and type counts. Each update modifies only bounded local neighborhoods, ensuring that property testing remains tractable and update-local (Berkholz et al., 2017).

2. Change-Impact, Coverage, and Detection Metrics

A MultiSHAP system provides quantitative and structural analyses to characterize the scope and sensitivity of static validation. Key metrics include:

Metric	Direct	Transitive
Test-suite coverage (call-site)	58%	20%
Test-suite fault detection	47%	35%
Static impact analysis detection (Uppdatera)	74%	64%

In Uppdatera (Hejderup et al., 2021), coverage is computed as the fraction of statically declared dependency call-sites exercised by the dynamic test suite, with substantial gaps for transitive dependencies. Mutation-based fault injection quantifies detection—tests detect only $47\%$ of direct and $U$0 of indirect faults, but static change impact uncovers $U$1 and $U$2, respectively.

In automata-theoretic static validation (0907.5125), completeness is measured via automata closure properties: forward and backward closure for all update sequences is decisive for type safety, and algorithms are polynomial in schema and automaton size.

For relational and deductive systems, "inconsistency-tolerant" reductions (Decker et al., 2024) guarantee that no new violations of already satisfied constraints are introduced, regardless of pre-existing inconsistencies.

3. Algorithmic Techniques and Framework Pseudocode

The core MultiSHAP workflow in code or data systems involves:

• Differencing and Change Classification: AST or rule-level differencing between versions to identify semantic changes (data- or control-flow).
• Graph Construction and Reachability: Construction of extended program/data/model graphs, call graphs (CHA), or automata, followed by reachability analysis to propagate semantic changes to clients (Hejderup et al., 2021).
• Invariant/Type Closure: Application of automata (HA/CF-HA) closure to determine the post-update type or property language, followed by intersection/inclusion testing for safety (0907.5125).
• Dynamic Maintenance: For dynamic queries (e.g., FO+MOD), precompute type counters and update them incrementally per local modification (Berkholz et al., 2017).

High-level pseudocode for change-impact analysis (Uppdatera):

$U$3

4. Empirical Evaluations and Real-World Effectiveness

Experimental benchmarks confirm that static impact analysis more than doubles fault-detection coverage over dynamic testing for both direct and transitive dependencies in Java projects (Hejderup et al., 2021). Uppdatera, deployed on 22 real-world Dependabot pull requests, identified three semantic conflicts missed by tests and detected five unused dependencies.

In XML/type-preservation, the framework achieves polynomial-time, complete, and exact validation for all standard XQuery-update and access control primitives (0907.5125). Empirical evaluation demonstrates efficiency and full coverage for schema evolution scenarios.

For query answering in bounded-degree databases, dynamic algorithms support constant-time static validation of FO+MOD properties under arbitrary tuple insertions/deletions, regardless of database size, after a 3-fold exponential precomputation in query and degree (Berkholz et al., 2017).

5. Limitations and Boundary Cases

MultiSHAP-style static validation inherits limitations from underlying abstractions:

• Static Over-Approximation: Call-graph reachability and CHA can yield false positives, especially in the presence of polymorphism or reflection (Hejderup et al., 2021).
• Coverage Gaps in Dynamic Approaches: Dynamic tests rarely exercise full dependency closure (only 20% for transitives), and may miss semantic conflicts (Hejderup et al., 2021).
• Automata Scalability: For certain combinatorially large type schemas or queries (e.g., high-arity/degree FO+MOD), preprocessing and auxiliary structure size grows triple-exponentially.
• Incompleteness Beyond Supported Patterns: Reflection, code generation, or global query containment optimizations may break the guarantees of specific static analysis frameworks (Decker et al., 2024, Hejderup et al., 2021).

6. Integration, Hybrid Systems, and Future Directions

Consensus across both code-centric and data-centric research is that hybrid approaches—static multi-criteria validation followed by targeted dynamic test selection—offer the most robust practical path forward (Hejderup et al., 2021). Recommendations include reporting both "dependency-coverage" and "change-impact" scores in automated PRs, and using static reachability to restrict regression test selection.

In the database context, frameworks now support inconsistency tolerance (Decker et al., 2024), ensuring that repeated application of MultiSHAP-style checks reduces net inconsistency even in partially corrupt states.

In all domains, broader support for richer update languages, incremental/differential analysis, and the development of explainable static validation artifacts (e.g., reporting all affected client functions, minimal counterexamples) are active areas of research.

7. Practical Impact and Tooling Implications

MultiSHAP methodologies underpin the design of modern, property-aware dependency management and schema-evolution tools. Tools embodying these principles provide formal guarantees unattainable by runtime checks alone, close coverage gaps in legacy test suites, and systematically reduce manual intervention during continuous integration and deployment. Hybrid, data-driven static validation now represents a foundational requirement for robust, safe, and efficient automation in code, data, and model evolution (Hejderup et al., 2021, 0907.5125, Decker et al., 2024, Berkholz et al., 2017).