Multiply-Shift Hashing

• Multiply-shift hashing is a word-arithmetic hash family that multiplies a key by a uniformly selected odd machine-word multiplier modulo a power of two and retains the most significant bits of the result, efficient for modern computer architecture.
• Its advantages include efficient machine-operation and substantial arithmetic structure, but it is distinct from affine prime-modulus hashing and iterative affine hash generation (used in some presentations of MinHash).
• The primary application is for distributing data into hash table buckets, with a key result being that an efficiently defined load maximum is $O(n^{1/3})$.

Multiply-shift hashing is a word-arithmetic hash family that multiplies a key by a uniformly selected odd machine-word multiplier modulo a power of two and retains the most significant bits of the result. For a word universe $q=2^r$ and a table of size $m=2^\ell$, its standard form is

$h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$

with output in $\{0,\ldots,2^\ell-1\}$. The oddness of $a$ makes it invertible modulo $2^r$. The family is distinguished from affine prime-modulus hashing and from iterative affine hash generation used in MinHash. Its implementation advantages arise from machine-word multiplication, implicit modular reduction through overflow, and bit truncation; its theoretical behavior reflects substantial arithmetic structure rather than full randomness.

1. Definition and arithmetic structure

Let $q=2^r$ denote the word universe, let $m=2^\ell$ be the number of buckets, and assume $0\le \ell\le r$. A multiplier $a$ is sampled uniformly from the odd residues

$m=2^\ell$0

For a key $m=2^\ell$1, the multiply-shift function is

$m=2^\ell$2

The output consists of the $m=2^\ell$3 most significant bits of the product $m=2^\ell$4. Equivalently, multiplication partitions the word universe into $m=2^\ell$5 consecutive intervals, each of width

$m=2^\ell$6

Bucket $m=2^\ell$7 contains precisely those products whose representatives lie in

$m=2^\ell$8

The multiplier is restricted to odd residues because odd elements are invertible modulo $m=2^\ell$9. This invertibility is central both to the conventional construction and to analyses of its collision and maximum-load behavior. There is no additive parameter $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$0 in the family considered in the standard theoretical formulation.

Multiply-shift hashing is therefore not a hash into residue classes modulo $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$1. It is multiplication modulo $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$2 followed by high-bit extraction. This distinction is important because the final projection has interval geometry: a bucket corresponds to a consecutive interval in $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$3, rather than to a congruence class.

2. Relation to affine and prime-modulus hashing

A related affine family over a prime modulus is

$h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$4

where $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$5 and $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$6 are independently and uniformly selected from $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$7. Here $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$8 is prime, the intermediate value is computed modulo $h_a(x)=\bigl((ax)\bmod 2^r\bigr)\gg(r-\ell),$9, and the final operation reduces modulo $\{0,\ldots,2^\ell-1\}$0. The family used in multiply-shift hashing instead has modulus $\{0,\ldots,2^\ell-1\}$1, no additive shift, and a high-bit projection.

The two constructions can be contrasted as follows:

Property	Prime-modulus affine hashing	Multiply-shift hashing
Modulus	Prime $\{0,\ldots,2^\ell-1\}$2	$\{0,\ldots,2^\ell-1\}$3
Parameters	$\{0,\ldots,2^\ell-1\}$4	Odd multiplier $\{0,\ldots,2^\ell-1\}$5
Final projection	Reduction modulo $\{0,\ldots,2^\ell-1\}$6	Retain $\{0,\ldots,2^\ell-1\}$7 high bits
Bucket geometry	Residue classes after reduction	Consecutive intervals of width $\{0,\ldots,2^\ell-1\}$8

The expression “linear hashing” is potentially misleading for the prime-modulus family because the final reduction modulo $\{0,\ldots,2^\ell-1\}$9 destroys linearity. Multiply-shift likewise does not preserve the usual finite-field or vector-space structure; its principal appeal is that multiplication modulo a power of two and bit shifting are efficient machine operations.

The affine prime-modulus family is the baseline used in Carter–Wegman-style universal hashing. In contrast, the multiply-shift family analyzed by Dietzfelbinger, Hagerup, Katajainen, and Penttonen uses a uniformly random odd multiplier and truncation of the product. The two families nevertheless admit closely related geometric analyses of heavy buckets.

3. Universal hashing and load guarantees

For a fixed set $a$0 of $a$1 distinct keys, let

$a$2

denote the maximum bucket load, equivalently the longest chain when hashing with chaining.

For multiply-shift hashing, the established upper bound applies when

$a$3

and $a$4 is arbitrary and fixed. The expectation is over the uniformly random odd multiplier $a$5. The bound is

$a$6

This result is proved for the multiply-shift family in “Linear Hashing is Awesome” (Knudsen, 2017).

The theorem is an expectation bound, not a high-probability statement. It does not assert that the family behaves as a fully random function, nor does it establish the usual pairwise collision guarantee as its principal conclusion. For $a$7, fully random hashing has maximum load typically about

$a$8

with high probability, whereas the $a$9 guarantee is substantially larger. The result instead exploits arithmetic structure to improve the generic $2^r$0 expected longest-chain bound associated with arbitrary 2-independent hashing.

The proof establishes a tail estimate of the form

$2^r$1

for $2^r$2. Summing this tail with

$2^r$3

gives the stated expectation bound. The cubic exponent arises from converting heavy-bucket events into a lower bound on the number of close pairs produced by multiplication.

4. Geometric interpretation of heavy buckets

For multiply-shift hashing, bucket $2^r$4 is exactly the interval

$2^r$5

in the word universe. Consequently, if $2^r$6 keys collide in one bucket, then the transformed set

$2^r$7

contains $2^r$8 points in an interval of length $2^r$9.

The proof studies the event $q=2^r$0. It introduces auxiliary multipliers whose canonical representatives are primes in $q=2^r$1. If many such multipliers produce heavy buckets, an interval argument implies that the original key set contains many ordered pairs whose transformed differences are small:

$q=2^r$2

For distinct keys $q=2^r$3 and $q=2^r$4, multiplication by their nonzero difference does not behave identically to a fully random mapping, but the invertibility of odd multipliers allows the probability of this close-pair event to be bounded by

$q=2^r$5

Summing over ordered pairs yields an upper bound on the expected number of close pairs. A technical interval lemma supplies a competing lower bound whenever many multipliers induce heavy buckets. Combining the two bounds produces

$q=2^r$6

The interval geometry differs slightly from that of affine hashing modulo a prime. In the prime-modulus case, collisions correspond to concentration in a residue class modulo $q=2^r$7, which can be transformed into concentration inside a short cyclic interval. In multiply-shift hashing, the high-bit projection directly identifies a heavy bucket with concentration in a consecutive interval.

5. Lower bounds and adversarial key sets

The upper bound does not imply that multiply-shift hashing has polylogarithmic maximum load on every key set. “Linear Hashing is Not That Awesome” (Zamir, 24 Aug 2026) proves that, for sufficiently large word modulus, there exist sets of keys with substantially larger expected maximum load.

Let

$q=2^r$8

and let $q=2^r$9. For fixed constants $m=2^\ell$0 and $m=2^\ell$1, define

$m=2^\ell$2

For sufficiently large $m=2^\ell$3, every power of two $m=2^\ell$4 satisfying $m=2^\ell$5, and every power of two $m=2^\ell$6 satisfying $m=2^\ell$7, there exists a set $m=2^\ell$8 of $m=2^\ell$9 keys such that

$0\le \ell\le r$0

Equivalently,

$0\le \ell\le r$1

This is an existential worst-case statement: for each permitted parameter regime, a particular key set exists. It is not a claim that ordinary or randomly selected key sets exhibit this load. The expectation is over the random odd multiplier, and a constant fraction of admissible multipliers produce load $0\le \ell\le r$2 for the constructed set. Thus the phenomenon is not attributed solely to an extremely rare multiplier.

The result establishes a super-polylogarithmic lower bound while leaving open whether the worst-case expected maximum load is polynomial in $0\le \ell\le r$3. In particular, it remains open whether there exists a constant $0\le \ell\le r$4 such that some key set satisfies

$0\le \ell\le r$5

6. Additive combinatorics and arithmetic Kakeya sets

The lower-bound construction connects maximum-load analysis to a density variant of the arithmetic Kakeya problem. For a set $0\le \ell\le r$6, a length parameter $0\le \ell\le r$7, and a nonzero difference $0\le \ell\le r$8, define

$0\le \ell\le r$9

The subset $a$0 need not be an interval or contain an arithmetic progression. This flexibility motivates the term dense arithmetic Kakeya set.

For prime-modulus affine hashing, with

$a$1

the expected maximum load is equivalent, up to constant factors and the contribution from $a$2, to the average directional richness

$a$3

The multiply-shift analogue replaces bounded modular progressions with short cyclic intervals of length at most $a$4. If $a$5 transformed points lie in such an interval, the interval intersects at most two consecutive high-bit buckets, so one bucket contains at least $a$6 points.

The lower-bound construction uses a construction of Green and Ruzsa. For the first $a$7 odd primes $a$8, let

$a$9

There is a set

$m=2^\ell$00

whose size satisfies

$m=2^\ell$01

and which contains a complete $m=2^\ell$02-term arithmetic progression for every integer difference from $m=2^\ell$03 through $m=2^\ell$04.

Difference amplification transforms these progressions into rich sets of modular differences. For multiply-shift hashing, odd differences and odd multipliers are used because only odd elements are invertible modulo $m=2^\ell$05. Appropriate parameter choices produce $m=2^\ell$06 distinct good odd multipliers. For each such multiplier, a $m=2^\ell$07-term progression is mapped into a cyclic interval shorter than one bucket width:

$m=2^\ell$08

At least $m=2^\ell$09 transformed keys consequently occupy one bucket, yielding the lower bound on expected maximum load.

7. Practical variants, MinHash, and limitations

The iterative affine construction described in “Iterative Universal Hash Function Generator for Minhashing” (Franca, 2014) is related to multiply-based universal hashing but is not the standard multiply-shift family. Its baseline is

$m=2^\ell$10

where $m=2^\ell$11 is a large prime. For MinHash, each feature is hashed and the feature attaining the minimum value is retained:

$m=2^\ell$12

The paper generates a sequence of affine functions using

$m=2^\ell$13

For a fixed feature, consecutive values satisfy

$m=2^\ell$14

Thus, after one initial multiplication, subsequent values can be generated using modular additions. The construction reduces coefficient storage from $m=2^\ell$15 values for $m=2^\ell$16 independently parameterized affine functions to $m=2^\ell$17 initial values. It is therefore an iterative Carter–Wegman-style affine generator modulo a prime, not a machine-word multiply-shift implementation.

The distinction has practical consequences. Multiply-shift uses implicit modulo-$m=2^\ell$18 arithmetic and high-bit extraction, whereas the iterative MinHash construction uses an arbitrary prime modulus and a recurrence that avoids repeated affine multiplications. The former is not a drop-in replacement when a prime-modulus affine family or a MinHash-specific recurrence is required; the latter is not a substitute for word-level multiply-shift when power-of-two bucket ranges and native machine arithmetic are the target.

Multiply-shift’s principal limitations are structural. Its hash values are correlated through multiplication by one random multiplier, and the family is not fully random. Its worst-case expected maximum load is now known to be super-polylogarithmic for suitable key sets, despite the earlier $m=2^\ell$19 upper bound. Arbitrary word-modulus arithmetic can also differ in cost from prime-modulus reduction, although multiply-shift benefits from efficient hardware multiplication, overflow semantics, and right shifts.

The current theoretical picture is therefore bounded between

$m=2^\ell$20

for worst-case expected maximum load when $m=2^\ell$21 in the relevant parameter regimes. Improving the upper bound substantially would imply progress on standard arithmetic Kakeya questions, while stronger lower bounds could follow from constructions for the weaker density-Kakeya formulation.