Pattern-sharing Degree (PSD) in SHARP

• Pattern-sharing Degree (PSD) is a bitmap-based abstraction that identifies sub-pattern appearances across sequential pattern workloads.
• It clusters partial matches by grouping states with identical bitmap signatures, enabling constant-time retrieval and efficient runtime shedding.
• PSD underpins best-effort processing in SHARP by reducing state overhead and achieving significant speedups while maintaining high recall under latency constraints.

Searching arXiv for the cited SHARP paper to ground the article in the source record. Pattern-sharing Degree (PSD) is an abstraction introduced in SHARP, a library for efficient best-effort matching of sequential patterns under latency bounds. In SHARP, sequential pattern matching is evaluated over a shared plan in which states correspond to sub-patterns and partial matches (PMs) represent intermediate results. PSD assigns each state a bitmap indicating in which patterns that sub-pattern appears, thereby turning cross-pattern state sharing into an explicit indexing primitive. This supports categorization and indexing of PMs at runtime, constant-time lookup of PM clusters, and overload-time selection of PMs for further processing or shedding under latency bounds (Yu et al., 7 Jul 2025).

1. Formal definition

Let

$\mathbf{P} \;=\;\{\,P_1,\dots,P_n\}$

be a fixed workload of $n$ sequential patterns. During query planning, one builds a shared evaluation plan

$G \;=\;\bigl(V,E\bigr)$

whose nodes $V$ correspond to all sub-patterns (states) of all $P_i$, and whose edges $E$ model valid one-event transitions between sub-patterns (Yu et al., 7 Jul 2025).

For each state $v\in V$, PSD is defined through a bitmap

$$b(v)\;=\;\bigl(b_1(v),b_2(v),\dots,b_n(v)\bigr)\;\in\;\{0,1\}^n$$

with components

$$b_i(v)\;=\; \begin{cases} 1, &\text{if sub-pattern }v\text{ appears in the plan of }P_i,\ 0, &\text{otherwise.} \end{cases}$$

Equivalently, interpreting the $n$-bit word as an integer,

$n$0

When a PM $n$1 currently resides at state $n$2, it inherits the state bitmap: $n$3

This definition makes PSD a compact representation of cross-pattern relevance. A state with more set bits is shared by more patterns; a state with a single set bit is pattern-specific. This suggests that PSD is not merely a label on states but a structural summary of how a shared evaluation DAG encodes workload overlap.

2. Computation over the shared evaluation DAG

PSD can be computed by a single reverse traversal of the DAG $n$4. Writing $n$5 for the children, or successor states, of $n$6, the procedure initializes

$n$7

and for each pattern root $n$8 sets

$n$9

A post-order, sink-to-source DFS then propagates bitmaps via

$G \;=\;\bigl(V,E\bigr)$0

After convergence, each state carries exactly the OR of the bitmasks of all patterns whose execution plan includes that state (Yu et al., 7 Jul 2025). The computation therefore reduces multi-pattern membership to repeated bitwise aggregation along the DAG.

The operational significance is twofold. First, PSD is established during planning rather than during event-time overload handling. Second, the propagation rule aligns with the semantics of shared sub-patterns in a DAG: a state is relevant to every pattern reachable through its successors. A plausible implication is that the abstraction is especially natural in systems already organized around automata-style or DAG-style state progression.

3. Clustering and indexing of partial matches

Once each state $G \;=\;\bigl(V,E\bigr)$1 has a bitmap $G \;=\;\bigl(V,E\bigr)$2, SHARP groups together all PMs at states sharing the same bitmap. Concretely, it maintains a map, or array, of clusters

$G \;=\;\bigl(V,E\bigr)$3

where

$G \;=\;\bigl(V,E\bigr)$4

Lookup of the entire cluster $G \;=\;\bigl(V,E\bigr)$5 is then $G \;=\;\bigl(V,E\bigr)$6 (Yu et al., 7 Jul 2025).

Within each cluster, PMs are ordered by a partial order induced by their “quality,” as defined by SHARP’s cost model. PSD thus acts as the first-level partitioning key, while quality determines prioritization inside each equivalence class of cross-pattern relevance.

This organization changes the granularity at which best-effort control can act. Instead of treating every PM as an isolated object, the system can reason over sets of PMs that have identical participation across the pattern workload. This suggests that PSD clustering is a workload-aware state index rather than a purely local state descriptor.

4. Role in best-effort processing under latency bounds

The abstract presents pattern matching as a setting in which state, meaning intermediate results, grows exponentially in the input size, while common applications rely on a large set of patterns that must be evaluated with tight latency bounds. SHARP addresses this by best-effort processing, striving for maximal recall under a latency bound, and PSD is the abstraction that exposes the state-sharing opportunities neglected by techniques that consider each pattern in isolation (Yu et al., 7 Jul 2025).

When processing event $G \;=\;\bigl(V,E\bigr)$7 would violate latency bounds $G \;=\;\bigl(V,E\bigr)$8, SHARP must shed PMs so as to restore throughput. PSD guides this through three mechanisms:

• Identifying overloaded patterns: by the OR of their bit-positions in a global overload bitmap $G \;=\;\bigl(V,E\bigr)$9.
• Selecting affected clusters: entire clusters whose PMs contribute to at least one overloaded pattern.
• Choosing PMs greedily: highest-quality PMs are chosen from these clusters until the estimated resource-consumption reduction meets

$V$0

Here $V$1 is the modeled overhead of continuing $V$2 in pattern $V$3. The exposition further states that, by processing clusters in descending order of their bitmap weight, meaning the number of patterns sharing the state, SHARP first preserves PMs that benefit many patterns at once (Yu et al., 7 Jul 2025).

The associated trade-off analysis considers the full PM set

$V$4

at time $V$5, partitioned into PSD clusters: $V$6 If a fraction $V$7 of PMs in a cluster $V$8 is dropped, the worst-case recall loss on pattern $V$9 is proportional to

$P_i$0

Because clusters with higher bit-counts $P_i$1 simultaneously reduce multiple $P_i$2 losses, PSD-aware shedding attains higher aggregate recall for the same CPU/time budget (Yu et al., 7 Jul 2025).

A common misconception would be to interpret PSD as a semantic similarity measure between patterns. The formalism instead defines exact membership of a state in pattern execution plans. Its utility comes from shared execution structure and bitmap-indexed control, not from an approximate notion of pattern resemblance.

5. Complexity characteristics

The complexity claims given for PSD in SHARP are explicit. PSD computation by one DFS over $P_i$3 is

$P_i$4

The number of non-empty clusters is bounded by

$P_i$5

The exposition notes that, in practice, $P_i$6 is in the hundreds, but not all $P_i$7 combinations arise, so clustering is linear in $P_i$8 (Yu et al., 7 Jul 2025).

Insertion of a new PM $P_i$9 into the proper cluster is $E$0. During overload-time selection, SHARP inspects at most those clusters with nonzero bit-count. Each PM pop from a max-heap is $E$1, but SHARP uses a bit-count-ordered array of heaps so that peeking is $E$2. Overall selection to shed $E$3 PMs is $E$4 (Yu et al., 7 Jul 2025).

These bounds clarify what PSD contributes algorithmically. The bitmap abstraction is not only descriptive; it defines a data layout in which the expensive decision point—overload-time triage of PMs—can be reduced to constant-time cluster access plus linear-time shedding in the number of removed PMs. A plausible implication is that PSD is most valuable when overload decisions must be made at high frequency and under tight tail-latency constraints.

6. Empirical behavior and application domains

The abstract situates SHARP in three application classes: complex event processing (CEP), OLAP, and retrieval-augmented generation (RAG). In experiments with real-world data, SHARP achieves a recall of $E$5, $E$6, and $E$7 for pattern matching in CEP, OLAP, and RAG applications, under a bound of $E$8 of the average processing latency (Yu et al., 7 Jul 2025).

The detailed exposition gives an empirical illustration with three patterns. Exhaustive evaluation generates approximately $E$9 as many PMs as single-pattern runs. With PSD enabled, the actual number of maintained PMs drops to

$v\in V$0

yielding a $v\in V$1-reduction in state and a 2–4 orders of magnitude speedup under tight latency bounds, while preserving $v\in V$2 recall (Yu et al., 7 Jul 2025).

These observations place PSD at the intersection of recall preservation, state reduction, and runtime control. The data do not claim that PSD alone determines end-to-end accuracy; rather, PSD provides the shared-state abstraction that lets SHARP organize PMs in a way that supports best-effort behavior. This suggests that the empirical gains arise from the interaction between PSD, clustering, quality-based ordering, and latency-bound-aware shedding.

7. Conceptual significance within SHARP

The summary in the exposition characterizes PSD as a compact, bitwise index over shared sub-patterns, enabling three capabilities: rapid clustering of PMs by their cross-pattern relevance, constant-time retrieval of affected states at overload time, and a principled prioritization for best-effort state reduction under latency bounds (Yu et al., 7 Jul 2025).

Within SHARP, PSD therefore serves as the bridge between shared execution planning and runtime best-effort control. It begins from a fixed workload of sequential patterns, derives a shared execution structure, projects that structure into bitmaps, and uses those bitmaps to control PM retention and shedding. The abstraction is notable because it turns state sharing—often an implicit property of shared plans—into an explicit optimization handle.

A plausible implication is that PSD is best understood as an indexing discipline for shared intermediate state. In that interpretation, its main contribution is not a new matching semantics for sequential patterns, but a way to expose cross-pattern structure so that latency-bounded systems can preserve higher aggregate recall for a given processing budget.