History-Independent Load Balancing

Abstract: We give a (strongly) history-independent two-choice balls-and-bins algorithm on $n$ bins that supports both insertions and deletions on a set of up to $m$ balls, while guaranteeing a maximum load of $m / n + O(1)$ with high probability, and achieving an expected recourse of $O(\log \log (m/n))$ per operation. To the best of our knowledge, this is the first history-independent solution to achieve nontrivial guarantees of any sort for $m/n \ge ω(1)$ and is the first fully dynamic solution (history independent or not) to achieve $O(1)$ overload with $o(m/n)$ expected recourse.

• The paper introduces a strongly history-independent two-choice allocation algorithm that maintains O(1) overload with high probability and O(log log μ) expected recourse per insertion or deletion.
• The Slice-and-Spread method repeatedly smooths bin loads using fresh hash choices, reducing the active workload across log log μ rounds while controlling cumulative imbalance and update costs.
• A randomized swapping and graph-orientation transformation converts constant average excess into constant maximum overload, while open questions include recourse lower bounds and high-probability update guarantees.

The problem and the main result

The paper studies fully dynamic two-choice load balancing: up to $m$ balls, each with two independent uniform hash choices $h_1(x), h_2(x)$ among $n$ bins, must be maintained under insertions and deletions so that (i) the overload — the amount by which the fullest bin exceeds $m/n$ — is small with high probability in $n$, and (ii) the expected recourse — the number of balls moved per operation, as a function of $\mu = m/n$ — is small. The central contribution is a strongly history-independent allocation algorithm achieving overload $O(1)$ with high probability in $n$ and expected recourse $O(\log\log\mu)$ per operation. This is claimed to be the first history-independent solution with nontrivial guarantees for $\mu \ge \omega(1)$, and the first fully dynamic solution of any kind achieving constant overload with $h_1(x), h_2(x)$0 expected recourse. By Hartline et al.'s characterization, strong history independence is equivalent to unique representability in this setting, so the algorithm is fully specified by a function from the current ball set (plus hash functions and random tape) to an allocation.

The result also improves on the best history-dependent state of the art: Dietzfelbinger–Weidling achieve overload at most 1 but with expected recourse $h_1(x), h_2(x)$1, and tombstone-based dynamization of the greedy algorithm yields amortized recourse $h_1(x), h_2(x)$2. The new bound reduces recourse doubly exponentially relative to these baselines.

History-independent greedy

As a warm-up, the paper analyzes the natural canonical-ordering construction: assign balls a total order and compute the allocation that the classical greedy algorithm would produce if balls were inserted in that order. This "HI Greedy" inherits the BCSV overload guarantee of $h_1(x), h_2(x)$3 with high probability, and the paper proves its expected recourse is $h_1(x), h_2(x)$4 via a two-world coupling argument: after the insertion point where two neighboring sets differ, exactly one bin differs in load between the worlds at all times, and each subsequent insertion perturbs the outcome only if one of its hashes hits that special bin, contributing $h_1(x), h_2(x)$5 expected recourse. Notably, this analysis is tight for $h_1(x), h_2(x)$6: an appendix proves an $h_1(x), h_2(x)$7 lower bound via "critical ties" — moments when the special bin's load exceeds another bin by exactly one, which cause divergent greedy decisions with probability $h_1(x), h_2(x)$8 each. Thus the simple canonical-greedy approach cannot beat recourse linear in load factor.

Slice and Spread

The main algorithmic contribution replaces greedy with a multi-round smoothing procedure. Balls are first placed at their first-choice hash. The algorithm runs $h_1(x), h_2(x)$9 rounds; in round $n$0 it slices each bin down to threshold $n$1 (where $n$2), evicting only balls assigned to round $n$3 (each ball is round-assigned with probability 0.01, with per-round assignment probabilities proportional to $n$4), and spreads the evicted balls to their second-choice hashes. Because each ball is sliced at most once, every spreading stage exploits fresh randomness from $n$5. The number of balls in play shrinks geometrically ($n$6), giving both the $n$7 cumulative-overload guarantee and the $n$8 recourse bound.

The analysis must control feedback loops between overfilled and underfilled bins across rounds: slicing failures (bins lacking enough round-assigned balls to reach the threshold) and spreading failures compound across rounds. Using a concentration lemma showing that random throwing leaves total imbalance $n$9 with high probability, the paper shows the per-round error $m/n$0 grows by only $m/n$1 per round, hence stays $m/n$2 through all $m/n$3 rounds. Recourse is bounded via a discrepancy argument between neighboring sets: the total discrepancy between two worlds never increases except by a constant when the differing ball itself is processed, so total recourse is $m/n$4.

At this stage the guarantee is cumulative overload $m/n$5 — i.e., average excess load per bin is constant — not maximum overload.

Reducing maximum overload to O(1)

The final step is an almost black-box transformation. Any "good pre-baking" algorithm (one placing Type-1/Type-2 balls at $m/n$6, having $m/n$7 cumulative overload with high probability) can be converted into one with maximum overload $m/n$8 at constant-factor recourse cost. Balls are randomly typed with probabilities $m/n$9, $n$0, $n$1. A Two-Phase Swapping Procedure exchanges the overloaded balls above height $n$2 for fresh balls whose hashes are provably independent and uniform (Phase 1 swaps against Type-2 balls, Phase 2 against Type-1 balls), inserting dummy balls on failure. Then the Extended Canonical Orientation procedure — Naor–Segev–Wieder's component-wise minimum-in-degree orientation, generalized to $n$3 balls by random edge partitioning — is applied to the swapped set and to carefully defined over-approximating sets $n$4 and $n$5 of failure-related balls. These sets have spoiled randomness, so the paper develops graph machinery (via Poissonization, McDiarmid's inequality, and a downsampling lemma) proving that the induced graphs still have components with $n$6 cycles and $n$7 expected size, which suffices for low-overload orientation and $n$8 expected recourse per structure. Combining the pieces yields the full theorem: overload $n$9 with high probability in $\mu = m/n$0, expected recourse $\mu = m/n$1.

A conceptual observation worth noting: history independence sidesteps reappearance dependencies — the difficulty that reinserted elements' hashes are no longer fresh — because the allocation depends only on the current set. History-dependent algorithms seeking better recourse would need nontrivial techniques to handle this issue.

Limitations and open questions

Several caveats bear directly on the results. The recourse bound is proven only in expectation over hash functions and internal randomness; the paper does not establish high-probability recourse bounds. The HI Greedy lower bound holds for $\mu = m/n$2, leaving larger regimes unaddressed. The final theorem's constants depend on fixed type probabilities and the ECO parameter, and implementations run in time proportional to recourse using $\mu = m/n$3 metadata space, though no formal time-complexity theorem is stated. The authors conjecture that $\mu = m/n$4 overload requires $\mu = m/n$5 recourse, making their upper bound optimal, but even proving any $\mu = m/n$6 recourse lower bound for history-independent solutions remains open. Whether a history-dependent algorithm can beat $\mu = m/n$7 recourse at constant overload is likewise unresolved. Finally, the related question of space-efficient bucketized cuckoo hashing with load factor $\mu = m/n$8, buckets of size $\mu = m/n$9, and $O(1)$0 expected-time updates remains open.

Conclusion

The paper establishes that strongly history-independent two-choice load balancing is compatible with near-optimal performance: constant overload with high probability and doubly logarithmic expected recourse, improving simultaneously on all prior history-independent results (which were confined to $O(1)$1) and on prior history-dependent dynamic solutions (which required $O(1)$2 recourse). The techniques — canonical-ordering constructions, multi-round slice-and-spread smoothing, and randomized type-based swapping combined with canonical graph orientations under spoiled randomness — suggest that history independence can serve as an algorithmic design principle rather than merely a privacy constraint. The optimality of the $O(1)$3 recourse bound remains conjectural.