--- title: 'String Fingerprints: Methods & Applications' url: https://www.emergentmind.com/topics/string-fingerprints type: topic --- # String Fingerprints: Methods & Applications Searching arXiv for the cited papers and closely related work on string fingerprints. arXiv search results for: 1711.08475 string fingerprints approximate keyword matching String fingerprints are compact representations of strings or string-derived objects that preserve selected information for downstream queries. In the literature considered here, the term denotes several distinct constructions: the set of distinct characters appearing in a substring; a lightweight fixed-size block of data for approximate keyword matching using bitwise operations; a binary vector used as a lightweight secondary index for approximate `LIKE` predicates in columnar engines; and the Karp–Rabin fingerprint used to answer substring fingerprint queries on grammar-compressed strings [1301.3488] [1711.08475] [2507.10391] [1305.2777]. These constructions differ substantially in semantics, guarantees, and intended workload, but they share a common objective: replacing direct string inspection with a smaller representation that supports efficient filtering, indexing, or comparison. ## 1. Terminological scope and formal models A first source of ambiguity is that “fingerprint” is not used uniformly across stringology. In one line of work, a fingerprint of a factor $s_i \ldots s_j$ is the set of its distinct letters, $$ C_s(i,j)=\{s_k \mid i \le k \le j\}, $$ and the universe of fingerprints is $$ F=\{C_s(i,j)\mid 1\le i\le j\le n\}, $$ with the empty fingerprint also included by convention [1301.3488]. In another line of work, a fingerprint is a function $$ f:\Sigma^\ast \to \{0,1\}^m $$ mapping a string to an $m$-bit word that stores information regarding symbol occurrences using individual bits and can be compared with a constant number of bitwise operations [1711.08475]. A third formulation uses a binary vector of length $n$ over bins of an alphabet $A$, with $$ d^s_j := \max_{i=1\ldots \mathrm{len}(s)} x_{s_i,j}, $$ so that the $j$-th bit is set exactly when the string contains at least one symbol assigned to bin $j$ [2507.10391]. A fourth, algebraic notion defines the Karp–Rabin fingerprint of a string $S$ as $$ f(S)=\sum_{k=1}^{|S|} S[k]\;p^{|S|-k}\bmod M, $$ with concatenation and subtraction identities that support substring extraction [1305.2777]. These definitions imply that “string fingerprint” is best treated as a family of summarization mechanisms rather than a single method. A plausible implication is that comparisons across papers require attention to the query model: membership in a family of substring alphabets, approximate distance filtering, predicate pruning in database scans, and equality-style hashing of substrings are not interchangeable tasks. | Fingerprint notion | Representation | Primary query type | |---|---|---| | Substring alphabet fingerprint | Set of distinct characters in a substring | Compute $F$, test whether $f \in F$, report maximal locations | | Lightweight approximate-matching fingerprint | Fixed-size $m$-bit word | Prune string pairs before exact Hamming or Levenshtein verification | | Columnar `LIKE` fingerprint | Binary vector / fixed-width integer | Filter rows via `(FP & pattern_mask) = pattern_mask` | | Karp–Rabin fingerprint | Modular polynomial hash | Return fingerprint of substring $S[i,j]$ without decompression | ## 2. Fingerprints as sets of distinct substring characters For a text $s=s_1\ldots s_n$ over an ordered alphabet $\Sigma$ of size $\sigma \le n$, the fingerprint of a factor is the set of distinct letters occurring in that factor, and the central objects are the family $F$ of all such fingerprints, the set $L$ of maximal locations, and the quotient set $L_C$ obtained by grouping copies, i.e. equal substrings that realize maximal locations [1301.3488]. A pair $\langle i,j\rangle$ is a maximal location of a fingerprint $f\in F$ if $C_s(i,j)=f$, the left neighbor $s_{i-1}$ is not in $f$ when defined, and the right neighbor $s_{j+1}$ is not in $f$ when defined. The bound $|L| \le n\sigma$ is immediate from the definition. Three core problems are studied in this framework. Problem 1 is to compute $F$. Problem 2 is, given $f \subseteq \Sigma$, to answer whether $f \in F$. Problem 3 is, given $f$, to find all maximal locations of $f$ in $s$ [1301.3488]. The exact algorithms combine a suffix tree, a participation tree, and a naming procedure based on Karp–Miller–Rosenberg stacks. The participation tree is a trimmed version of suffix-tree paths, has size $O(n+|L_C|)$, and every path from its root spells a character-permutation that realizes at least one maximal location. Naming proceeds level by level in $\log \sigma$ levels, using radix sort on pairs of child names. The worst-case exact construction for Problem 1 builds the suffix tree in $O(n)$ time, the participation tree in $O(n+|L_C|)$ time, names all fingerprints in $O(\sigma + |L_C|\log \sigma)$ time, and achieves total time $O(n+|L_C|\log \sigma)$ with space $O((n+|L_C|+|F|\log \sigma)\log n)$ bits [1301.3488]. A randomized expected variant replaces suffix-tree-based naming by online randomized hashing and dynamic hash tables, with total expected time $O(n+|L|\log \sigma)$ and space $O((n+|F|\log \sigma)\log n)$ bits. A Monte Carlo approximate construction computes polynomial hashes at maximal locations, inserts them into a dynamic MC hash table, and runs in expected $O(n+|L|)$ time with space $O((n+|F|)\log n)$ bits and error probability $O(n^{-c})$ for any chosen constant $c$. For existential queries, the succinct structure uses $|F|(2\log \sigma+\log_2 e)(1+o(1))$ bits and answers a query set $f$ of size $m$ in overall time $O(m)$ expected or deterministic $O(m/\epsilon)$, with no false positives or negatives [1301.3488]. For location reporting, augmenting the structure with pointers into the suffix tree yields query time $O(|f|+\mathrm{occ})$ and space $O((n+|L_C|)\log n)$ bits. The applications stated for this notion include DNA-motif discovery, information retrieval, and natural-language processing. ## 3. Lightweight bitwise fingerprints for approximate keyword matching In approximate keyword matching, a fingerprint is a lightweight, fixed-size block of data for each string that works in a similar way to hash values but can also be used for matching with errors [1711.08475]. Let $\Sigma$ be the alphabet and let $\Sigma' \subseteq \Sigma$ be a chosen subset of tracked symbols, with fingerprint size $m$ bits, often $m=16$ or $32$. In the simplest occurrence scheme, each bit position corresponds to one symbol in $\Sigma'$, and $$ f_{\mathrm{occ}}(S)=\bigvee_{i=1..n}(1 \ll \alpha(S[i])). $$ Bit $j$ is set if and only if the $j$-th tracked symbol occurs at least once in $S$. The method can also halve the string and use $2\sigma'$ bits to record occurrence in the first and second half separately, use $b$ bits per symbol to count up to $2^b-1$ occurrences, or use $p$ bits per symbol to encode the $0$-indexed position of first occurrence, saturating at $2^p-1$. Given two fingerprints $S'_1=f(S_1)$ and $S'_2=f(S_2)$, the fingerprint distance is $$ F_D(S'_1,S'_2)=\mathrm{popcount}(S'_1 \oplus S'_2), $$ or, for position fingerprints, the number of differing $p$-bit fields [1711.08475]. The key inequality is $$ D(S_1,S_2)\ge \left\lceil \frac{F_D(S'_1,S'_2)}{2}\right\rceil $$ for $D \in \{\mathrm{Ham},\mathrm{Lev}\}$. Equivalently, if $F_D(S'_1,S'_2)>2k$, then one can conclude $D(S_1,S_2)>k$ without explicit verification. The intuition formalized in the cited theorems is that a single edit operation can change at most two bits, or two symbol-fields, in the occurrence or count fingerprint; by induction, $k$ edits change at most $2k$ bits. This filter has a one-sided guarantee. There are no false negatives in pruning: a true match within $k$ errors is never rejected. However, many non-matching pairs may not be ruled out, so false positives remain for exact verification [1711.08475]. The limitations are explicit: only symbols in $\Sigma'$ are tracked; edits involving untracked symbols may go unnoticed; and the bound may be very loose when $\sigma' \ll \sigma$ or strings are extremely repetitive. The preprocessing and query workflow is correspondingly simple. For each dictionary string $S$, compute $S'=f(S)$ in $O(n)$ time for occurrence or count fingerprints, or $O(n\cdot \sigma')$ for position fingerprints, and store $(S,S')$. Given a pattern $P$ and threshold $k$, compute $P'=f(P)$ in $O(|P|)$, evaluate $\mathrm{popcount}(P' \oplus S')$ for each candidate in $O(1)$ time using bitwise operations and a small lookup table, and verify exact distance only for candidates satisfying $\lceil D_f/2 \rceil \le k$ [1711.08475]. The space overhead is one $m$-bit fingerprint per stored string, i.e. $O(|D|\cdot m)$ extra bits. Empirically, for $k=1$ and $m=16$ bits using occurrence fingerprints with 16 common letters, the reported speedups were over $2.5$ times for the Hamming distance and over $10$ times for the Levenshtein distance on synthetic English data; approximately $1.2\times$ and $2\times$ respectively on English words; and approximately $1.5$–$1.8\times$ for Hamming and $1.7$–$1.8\times$ for Levenshtein on URLs [1711.08475]. Rejection rates at the fingerprint stage for $k=1$ were approximately $98\%$ by occurrence, $94\%$ by count, and $91\%$ by position on English words of length $9$, and approximately $71\%$ by occurrence and $80\%$ by count on URLs of length $69$. The worked examples in the source make clear that the lower bound can be weak: a pair may survive filtering even when the true Hamming distance is larger than $k$. ## 4. Workload-aware fingerprints for `LIKE` predicate pruning A more recent use of string fingerprints arises in cloud data warehouses and columnar query engines, where string columns are described as text-heavy and current processing as relying primarily on techniques like dictionary encoding and prefix-based partition pruning [2507.10391]. In this setting, a string fingerprint is a binary vector of fixed bitwidth $n$ obtained by partitioning the alphabet $A$ into bins via a binary assignment matrix $x \in \{0,1\}^{A\times n}$. For a string $s$, the fingerprint bit $d^s_j$ indicates whether $s$ contains at least one character assigned to bin $j$. To approximate a `LIKE`-substring test `s.contains(q)`, the pattern fingerprint $d^q$ is computed while ignoring wildcards, and the approximate membership test is $$ d^q \subseteq d^s, \qquad\text{i.e.}\qquad (d^s \,\&\, d^q)=d^q. $$ If this test fails, $s$ cannot contain $q$; if it passes, the full `LIKE` pattern must still be checked [2507.10391]. The operational realization in a columnar engine stores, alongside each string column $S$, an auxiliary column `FP` of fixed-width integer type such as a 16-bit integer. At load time, each row’s string is scanned once, mapped through `char2bin`, and accumulated into a bit mask using bitwise OR. At query time, a `pattern_mask` is built from the distinct characters of the pattern, and the engine evaluates ```sql SELECT … FROM table WHERE (FP & pattern_mask) = pattern_mask AND title LIKE '%P%' ``` so that the bitwise test filters rows before the expensive full `LIKE` evaluation [2507.10391]. The no-false-negative property is stated directly: if the pattern’s characters all appear in a string, then every bit set in $d^P$ will also be set in $d^S$, so the filter never rejects a true match [2507.10391]. False positives occur when the bit-mask test passes but the string does not contain the pattern as a substring. For a workload $Q$, the per-pattern false-positive rate is defined as $$ \mathrm{FPR}(q)= \frac{|\{w\in W\setminus f(q):(d^q\subseteq d^w)\}|}{|W\setminus f(q)|}, $$ and the overall expected false positive probability is $P_{FP}=E_{q\in Q}[\mathrm{FPR}(q)]$. Under the simplifying assumption that bin bits are set independently with empirical probabilities $p_j$, a pattern whose fingerprint has $k$ set bits passes on a random non-matching row with approximate probability $$ P[\mathrm{FP}\mid q]\approx \prod_{j:d^q_j=1} p_j. $$ The source uses this product-of-densities formula to explain why longer patterns and sparser bins lead to exponentially lower false-positive rates. A distinctive contribution of this framework is instance-optimized partitioning via mixed-integer programming. Binary decision variables assign each character to exactly one bin, define the string and pattern fingerprints, and introduce $\eta^{w,q}$ to indicate a correctly rejected non-match [2507.10391]. The objective maximizes $$ \sum_{q\in\mathcal Q}\sum_{w\in\mathcal W\setminus f(q)} \eta^{w,q}, $$ which is equivalent to minimizing false positives. Constraint (1) enforces a partition of characters into exactly one bin each; Constraints (2–3) define the fingerprint $d^s$ from the assignment; Constraint (4) permits $\eta^{w,q}=1$ only when at least one fingerprint bit of $q$ is missing in $w$. The source notes that a linearized version can be solved with Gurobi or CPLEX under a time limit such as $300\,\mathrm{s}$. On DuckDB v1.3.0 with the IMDb `title` column from the JOB benchmark, consisting of $2.53\,\mathrm{M}$ rows reduced to $2.37\,\mathrm{M}$ printable bytes, the reported experiment extracted 300 $k$-grams for $k=1\ldots 10$, split into 20 training patterns and 280 unseen test patterns [2507.10391]. A 16-bit fingerprint optimized on a 50-tuple sample of the first data block with a $300\,\mathrm{s}$ Gurobi time limit yielded a speedup of $1.36\times$ on the 20 training patterns and $1.26\times$ on the 280 unseen patterns during the full $2.37\,\mathrm{M}$-row scan. At 8-bit, the reported overall speedup was $1.2\times$, with a solver optimality gap under $2\%$ after $300\,\mathrm{s}$. The optimization overhead of approximately $300\,\mathrm{s}$ is stated to amortize after only 4 runs of the 300-query workload, and the additional storage is only $n$ bits per row, i.e. 2 bytes for $n=16$. ## 5. Karp–Rabin fingerprints in compressed strings The Karp–Rabin fingerprint is a type of hash value with strong algebraic properties, and in grammar-compressed strings it serves as the basis for substring fingerprint queries without decompressing any characters [1305.2777]. The paper considers a string $S$ of size $N$ compressed by a context-free grammar of size $n$, and gives the first $O(n)$-space data structures answering fingerprint queries for substrings $S[i,j]$. The two compression models are Straight-Line Programs (SLPs), which are context-free grammars in Chomsky normal form generating exactly one string, and Linear SLPs, in which the root may have an arbitrary number of children while every other nonterminal has a nonterminal left child and a terminal right child. The algebraic properties of the Karp–Rabin fingerprint enable substring extraction from prefix fingerprints. If $$ f(S)=\sum_{k=1}^{|S|}S[k]\,p^{|S|-k}\bmod M, $$ then for strings $X$ and $Y$, $$ f(X\!\cdot\!Y)=f(X)\,p^{|Y|}+f(Y), $$ and from prefix fingerprints one obtains $$ f(S[i..j])=\bigl(f(S[1..j])-f(S[1..i-1])\,p^{j-i+1}\bigr)\bmod M. $$ All such operations take $O(1)$ time when powers $p^k \bmod M$ are precomputed [1305.2777]. For general SLPs, the data structure uses a heavy-path decomposition of the parse tree and stores, for each nonterminal, the length of its expansion, the fingerprint of its full expansion, the fingerprint of a distinguished heavy-path prefix $P(v)$, and the power $p^{L(v)} \bmod M$ [1305.2777]. Prefix queries $f(S[1..i])$ are answered in $O(\log N)$ time by following the sequence of heavy paths returned by a standard random-access routine and maintaining a modular accumulator that adds and subtracts contributions of heavy-path side subtrees. Once $f(S[1..i])$ is available in $O(\log N)$ time, arbitrary substring queries follow in $O(1)$ extra time via the subtraction identity, so the whole structure uses $O(n)$ space and answers fingerprint queries in $O(\log N)$ time. For Linear SLPs, the structure stores predecessor-search support on block boundaries, prefix fingerprints at those boundaries, a dictionary tree with an $O(k)$-space level-ancestor structure, and $f(S(v))$ and $|S(v)|$ for each nonterminal [1305.2777]. A query for $f(S[1..i])$ first finds the predecessor block boundary $R(m)