Papers
Topics
Authors
Recent
Search
2000 character limit reached

Exactly-Mergeable Summaries

Updated 17 June 2026
  • Exactly mergeable summaries are compact data structures with a binary operator that ensures deterministic and lossless aggregation over disjoint data sets.
  • They provide strict space guarantees, enabling efficient one-pass computations in streaming, parallel, and distributed analytics frameworks.
  • Key implementations include scalar monoid summaries, top-k lists, histograms, and quantile sketches (e.g., DDSketch, UDDSketch) that ensure exact mergeability.

Exactly mergeable summaries are compact data structures or objects, together with a binary operator, enabling the precise and efficient aggregation of information from disjoint subsets of data. They guarantee that the summary over the union of disjoint sets can be reconstructed deterministically and losslessly from the summaries of its constituent parts, with bounded or constant space complexity relative to data set size. Exactly mergeable summaries underpin scalable streaming, parallel, and distributed data analytics, supporting the execution of merge and reduce computations with strong theoretical guarantees (Batagelj, 2023).

1. Formal Definition and Algebraic Structure

Let UU be a universe and AUA \subseteq U. A summary function Σ\Sigma maps subsets AA to a compact set SS (the summary or sketch domain), such that:

  • Space constraint: The data representation Σ(A)\Sigma(A) is bounded in size, i.e., O(1)O(1) or O(logA)O(\log |A|).
  • Mergeability: Given disjoint A,BUA,B \subseteq U, there exists a binary operator F:S×SSF : S \times S \to S such that:

AUA \subseteq U0

This operator AUA \subseteq U1 must be commutative and associative, so that mergeability generalizes from pairs to arbitrary finite disjoint collections.

An exactly mergeable summary is thus determined by the pair AUA \subseteq U2 with the above structural properties (Batagelj, 2023).

2. Fundamental Examples and Taxonomy

Exactly mergeable summaries include a broad spectrum of aggregation types:

  • Scalar monoid summaries: AUA \subseteq U3 or AUA \subseteq U4 equipped with AUA \subseteq U5, AUA \subseteq U6, AUA \subseteq U7 (e.g., sum, count, min, max).
  • Top-AUA \subseteq U8 summaries: Store the AUA \subseteq U9 largest (or smallest) elements. Merge via partial sort of Σ\Sigma0 elements.
  • Histograms: Fixed-length frequency vectors indexed by bins or categories, merged by component-wise addition.
  • Moments and means: E.g., Σ\Sigma1 where Σ\Sigma2. Merge via weighted averages.
  • Intervals: Σ\Sigma3. Merge via Σ\Sigma4 of endpoints.
  • Composed summaries: Direct product of mergeable summaries is itself mergeable (cf. Lemma 6.1 in (Batagelj, 2023)).

A decisive theoretical result is that the median is not exactly mergeable: equal medians among two pairs of subsets can yield different union medians, violating the merge property (Example 6.3 in (Batagelj, 2023)).

Type Representation Merge Rule
Count Σ\Sigma5 Σ\Sigma6
Sum Σ\Sigma7 Σ\Sigma8
Top-Σ\Sigma9 AA0-item list Top-AA1 of AA2
Histogram Bin counts Elementwise sum
(Count, Mean) AA3 Weighted mean

3. Algorithmic Framework and Complexity

Exactly mergeable summaries require one-pass scanning over subsets. For summary types such as count, sum, min, and max, both computation and merging are AA4; for top-AA5, AA6; for histograms, AA7 where AA8 is the number of bins or categories (Batagelj, 2023). Merge operations never revisit raw data. This property is leveraged in streaming and parallel reductions (e.g., MapReduce), where summaries from separate partitions are merged hierarchically with no loss in final summary fidelity.

For streaming quantile estimation, advanced sketches such as DDSketch (Masson et al., 2019), UDDSketch (Cafaro et al., 2021), and adaptive compactors (Domes et al., 21 Nov 2025) instantiate exactly mergeable summaries for approximate order statistics under space and error guarantees.

4. Exactly Mergeable Quantile Sketches

Parallel and distributed quantile computation critically depends on exactly mergeable sketches. Several key architectures exemplify this:

  • DDSketch: For AA9, maintains logarithmically spaced buckets covering positive real values. The insertion procedure assigns each observation to a bucket; the merge operation combines by summing per-bucket counts, followed by bucket collapses if an upper bound is exceeded (Masson et al., 2019). The quantile estimate for a rank SS0 is within multiplicative error SS1 of the empirical quantile. Merging two DDSketches (same SS2) yields a summary identical to that computed on the union stream.
  • UDDSketch: Refines DDSketch via a uniform collapsing rule to guarantee, after merging, that the output matches the one-pass sketch of the combined data. Parallel reduction is enabled by a commutative, associative merge that preserves the error and size bounds, with the merged summary accurately covering all quantiles regardless of the incoming data's distribution (Cafaro et al., 2021).
  • Adaptive Compactors: Organize the sketch as a stack of compactors with adaptive section sizes and markers, so that during merge, invariants on buffer sizes and compaction tokens are preserved. Marker and token bookkeeping guarantees that the relative error and space remain within prescribed bounds even after nontrivial merges, and simplified proofs eliminate difficult schedule/charging arguments present in earlier designs. Mergeability remains exact: merging two adaptive compactor sketches with compatible parameters yields a sketch that encodes the same distributional summary as processing the combined set (Domes et al., 21 Nov 2025).

5. Theoretical Framework and Non-Mergeability Criteria

The theoretical underpinnings require that the summary SS3 be a function of SS4 satisfying compositionality under disjoint union:

SS5

The operation SS6 is typically commutative and associative, forming a semigroup or monoid.

Lemma (Composition): The direct product of two exactly mergeable summaries is again exactly mergeable via componentwise application of their respective operators. See Lemma 6.1 in (Batagelj, 2023).

Theorem (Non-mergeability test): If there exist SS7 disjoint with SS8, but SS9, then Σ(A)\Sigma(A)0 is not exactly mergeable (Batagelj, 2023).

Median fails this criterion; mean, count, and variance do not.

6. Applications and Advantages

Exactly mergeable summaries are foundational in high-throughput, distributed, and parallel data analysis:

  • MapReduce and streaming frameworks: Partial summaries allow segment-level processing with global aggregation by iterative merging, eliminating the need for raw data shuffling.
  • Big data analytics: Support for fixed-space, high-fidelity summarization (histograms, top-Σ(A)\Sigma(A)1, frequency moments, quantile sketches).
  • Symbolic and interval data analysis: Mergeable summaries facilitate modal clustering, categorical distribution tracking, and streaming histogram maintenance.
  • Quantile and distribution estimation in network telemetry or distributed storage: DDSketch, UDDSketch, and adaptive compactors offer relative-error guarantees, speed, space efficiency, and near-linear scalability with number of streams or nodes (Domes et al., 21 Nov 2025, Cafaro et al., 2021, Masson et al., 2019).

The principal advantages are strict space guarantees, lossless merge when summaries are properly constructed and merged, and independence from the order of operations or data partitioning (Batagelj, 2023).

7. Extensions, Limitations, and Connections to Abstractive Fusion

The exactly mergeable summaries paradigm is tightly linked to algebraic aggregation theory. While mean, sum, min, max, and histograms are exactly mergeable, statistics such as the median or quantiles (without approximation) are not, owing to their lack of compositionality (Batagelj, 2023).

A closely related but structurally distinct concept is summary-candidate fusion in neural abstractive summarization, as in SummaFusion (Ravaut et al., 2022). Here, the fusion operator is learned (neural cross-attention), and the merged output is not guaranteed to encode the full union of all salient tokens from the inputs—so formal exact mergeability is absent. Instead, fusion empirically synthesizes and rescues key content from diverse inputs, but can also abstract, paraphrase, or discard information. Thus, while neural fusion methods yield superior qualitative and quantitative results in second-stage summarization, they do not provide the exact mergeability formalized in the data summarization literature.

A plausible implication is that exactly mergeable summaries remain the only family supporting strict, lossless parallel and incremental computation for complex aggregation under hard space constraints, while fusion-based neural summarization offers high-quality empirical “soft merges” without formal guarantees of completeness.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Exactly-Mergeable Summaries.