Points-to Analysis Using MDE: A Multi-level Deduplication Engine for Repetitive Data and Operations

Abstract: Precise pointer analysis is a foundational component of many client analyses and optimizations. Scaling flow- and context-sensitive pointer analysis has been a long-standing challenge, suffering from combinatorial growth in both memory usage and runtime. Existing approaches address this primarily by reducing the amount of information tracked often, at the cost of precision and soundness. In our experience a significant proportion of this cost comes from propagation of duplicate data and low-level data structure operations being repeated a large number of times. Our measurements on SPEC benchmarks show that more than 90% of all set-union operations performed can be redundant. We present Multi-level Deduplication Engine (MDE), a mechanism that recursively augments the representation of data through de-duplication and the assignment of unique identifiers to values to eliminate redundancy. This allows MDE to trivialize many operations, and memoize operations enabling their future reuse. MDE's recursive structure allows it to represent de-duplicated values that themselves are constructed from other de-deuplicated values, capturing structural redundancy not easily possible with non-recursive techniques. We provide a full C++ implementation of MDE as a library and integrate it into an existing implementation of a flow- and context-sensitive pointer analysis. Evaluation on selected SPEC benchmarks shows a reduction up to 18.10x in peak memory usage and 8.15x in runtime. More notably, MDE exhibits an upward trend of effectiveness with the increase in benchmark size. Besides performance improvements, this work highlights the importance of representation design and suggests new opportunities for bringing efficiency to future analyses.

• The paper introduces a recursive MDE that eliminates redundancy in both data and operations in flow- and context-sensitive pointer analysis.
• It leverages multi-level deduplication and memoization to reduce runtime by up to 8.15x and peak memory usage by 18.10x on benchmarks.
• The approach scales static analysis without sacrificing precision, opening avenues for applications in other set-based and nested analyses.

Multi-level Deduplication in Flow- and Context-Sensitive Points-to Analysis

Introduction

Scaling flow- and context-sensitive pointer analysis remains a central challenge in static program analysis due to exponential growth in memory and compute requirements. Traditional approaches either sacrifice precision and soundness through over-approximation or sparsity, or incur high costs by repeatedly performing identical data operations. The work "Points-to Analysis Using MDE: A Multi-level Deduplication Engine for Repetitive Data and Operations" (2604.10445) introduces the Multi-level Deduplication Engine (MDE), a recursive, highly structured memoization system that targets the elimination of redundancy in both data and operations at all levels of pointer analysis, including within aggregate data structures.

Motivation and Problem Formulation

The crux of the MDE approach stems from empirical findings: over 90% of set-union operations and a substantial share of computational costs in traditional pointer analysis are attributable to redundant data flows and repeated operations. Standard data structures such as std::set and std::map in C++ incur $O(n)$ complexity for most operations on large aggregate data, causing combinatorially increasing resource demands with program size and context depth.

The primary observation motivating MDE is the presence of deep structural redundancy within analysis data. For instance, sets of pointees often appear identically across multiple contexts, and pointer relationships among objects can be highly repetitive across control flow points and call contexts.

Figure 1: Control-flow graph with pointer assignments.

The Multi-level Deduplication Engine (MDE): Architecture and Operational Semantics

Core Abstractions and Data Structures

MDE revolves around two fundamental concepts:

• Property: The base data element (e.g., variable or pointer).
• Property Set: Immutable, totally-ordered collections of properties. These sets are the core units subjected to deduplication and memoization.

Deduplication is achieved by registering each property set in a structure that maps unique-value hashes (in practice, pointers to uniquely stored instances) to integer identifiers. These identifiers then become the canonical references in all downstream data structures.

Operation memoization leverages hash maps from tuples of operand indices to result indices for every supported set operation (e.g., union, intersection). Lookup precedes execution, and new results are registered only on a cache miss, supporting efficient reuse.

Figure 2: Schematic diagram of the state of an individual MDE. Subcomponents concerned with performing, as an example, the union operation, are highlighted in bold.

Recursive/Nested Representation

Unlike previous single-level hash-consing mechanisms, MDE recursively deduplicates data at multiple levels: not only are sets of pointees deduplicated, but also the mappings from pointers to these sets. Each level maintains its own deduplication and memoization maps, and the parent MDE node may reference children via their unique indices, forming a nested structure.

This enables the sharing of substructure even when complete set equality is absent at higher levels, substantially increasing reuse in programs with large structurally repetitive patterns.

Figure 3: Final configuration of MDE after peforming the steps in a non-nested demo.

Figure 4: Final configuration of MDEs after peforming the steps in a nested memoization scenario.

Set Operations and Efficiency

Most data-flow operations are implemented in $O(n)$ time due to total ordering; equality, subset checks, and unions exploit early exits, hash lookups, subset memoization, and commutativity where applicable. Calls on nested structures (e.g., union of points-to maps) invoke the same memoization mechanisms recursively on their constituent sets.

Figure 5: Gradual, incremental lattice structure emerging from the usage of MDE from the union operation. Note how the red edges form a tree, but it is not the only possible tree leading to the top-most node $\{a, b, c\}$.

MDE effectively builds a join-semilattice over the domain of property sets, with efficient traversal and edge reuse due to the persistent storage and memoization maps.

Empirical Results

MDE is integrated within a flow- and context-sensitive pointer analysis based on LFCPA and VASCO, running on large-scale SPEC benchmarks. Its performance is systematically compared against:

• Baseline (std::set/std::map)
• Single-level hash-consing
• Sparse bit-vector and ZDD-based alternatives

The results reveal strong dominance in both memory and runtime for multi-level MDE, especially as program complexity scales. Notably:

• Up to 18.10x reduction in peak memory usage (sjeng benchmark)
• Up to 8.15x reduction in runtime (hmmer benchmark)
• Effectiveness increases as program size and structural redundancies grow.

Memoization hit ratios exceed 95% on most benchmarks for set operations, validating the approach's empirical premise.

Theoretical and Practical Implications

Representation Design as a First-Class Optimization

The work demonstrates that algorithmic speedups for pointer analysis are not bounded solely by sparsity or over-approximation techniques. Intelligent design of the underlying data representation—exploiting persistent, multi-level deduplication—can yield orders-of-magnitude improvements without loss of precision or soundness.

Comparison and Contrast with Related Techniques

• Hash-consing [barbar2021hash]: Prior hash-consing mechanisms typically operate at only the level of sets of pointees. MDE recursively extends this deduplication to sets-of-mappings-of-sets, intercepting redundancy that cannot be captured by single-level schemes.
• Symbolic/Sparse Approaches: BDD/ZDD methods [bddbddb, zddpta] provide symbolic compressions but lack direct, concrete representation and incur overhead in decoding summary results. MDE provides immediate access to concrete data-flow values and is compatible with bulk updates typical in analyses like LFCPA.
• Sparse Bit-Vectors (e.g., SVF [svf-paper]): These representations increase efficiency for sparse data, but do not capture cross-context redundancy in composite relationships.

Generality and Extension

The recursive model of MDE is abstract and can be instantiated on arbitrary set-based domains, allowing application to problems beyond pointer analysis, such as liveness analysis, heap reference graphs, and data-flow analyses with nested structures or multiple interdependent domains.

Figure 6: A network of generalized MDE nodes. (b), (c), (d) depict the property elements of node 1, node 3, and node 2 respectively. (e) depicts the property elements of the non-nested nodes 4 and 5, which are the ``leaves'' in the network.

Future Directions

• Representation optimization: Automated discovery of representations that expose maximum redundancy, perhaps by information-theoretic analysis of the input.
• Eviction and resource management: For analyses producing unbounded domains, integration with eviction or external memory techniques is needed.
• Generalization: Extension to arbitrary nested and intersecting set-based domains allows application to a wide range of analyses and optimizations.
• Incremental and parallel implementations: The persistent, immutable property of MDE supports parallelization and out-of-core/incremental recomputation strategies.

Conclusion

Multi-level Deduplication Engine (MDE) demonstrates that recursive memoization and structural sharing across all aggregate levels in pointer analysis data can yield substantial gains in performance and memory usage while preserving full flow- and context-sensitivity. By shifting the focus towards representation-level optimization, this approach opens avenues for scaling static analysis to large codebases without requiring unsound approximations. The versatility of MDE lays groundwork for future automated and information-theoretic approaches to efficient static program analysis.