Dynamic Weighted Fairness & Minimal Disruptions

• The paper presents a method that guarantees near-optimal weighted resource allocation with only O(log* n) disruptions in arrival-only models, ensuring proportional fairness despite dynamic job changes.
• It employs techniques such as weight bucketing and monotone rounding to derive tight theoretical bounds and scalable algorithms across single and multi-resource environments.
• It balances fairness and disruption by leveraging randomized thresholding, achieving constant expected disruptions per job in adversarial dynamic systems.

Dynamic Weighted Fairness with Minimal Disruptions refers to the challenge of maintaining approximately fair allocations of resources among jobs with dynamic arrival and departure, such that each job's share is proportional to its weight, while minimizing the number of allocation changes ("disruptions") to any individual job. This framework extends classic fair-share and weighted-fair resource management to the context where system composition evolves, often adversarially, over time. The principal goal is to approximate instantaneous fairness with as little churn in allocations as theoretically possible, an objective crucial for distributed systems, cluster schedulers, and dynamic networks (Im et al., 2020).

1. Formal Model and Definitions

Let time $t = 0,1,2,\ldots$ index the operation of a resource-sharing system. At each time, the set $N^t$ contains the currently alive jobs, originally from the universe $\{1,\ldots,n\}$. For each job $j$, a fixed positive weight $w_j > 0$ governs its target share.

The canonical weighted-fair allocation at time $t$ is

$x_j^*(t) = \frac{w_j}{\sum_{k \in N^t}w_k}$

where $x_j^*(t)$ is $j$'s instantaneous resource share. The allocation algorithm $A$ maintains allocations $x_j(t) \geq 0$ such that $\sum_{j \in N^t} x_j(t) \leq 1$. To accommodate implementation and theoretical constraints, most algorithms only require an $\alpha$-approximate guarantees:

$x_j(t) \geq \frac{1}{\alpha} x_j^*(t)$

for some constant $\alpha \geq 1$.

Disruptions are counted whenever an allocation for a job $j$ changes between consecutive time steps, formalized as $x_j(t) \neq x_j(t-1)$ for $j \in N^t \cap N^{t-1}$. The total disruption is the sum over all such events.

2. Algorithmic Results and Theoretical Bounds

The paper "Dynamic Weighted Fairness with Minimal Disruptions" (Im et al., 2020) presents tight bounds across several models:

• Arrival-only: For sequences where jobs only arrive but never depart, $O(\log^* n)$ disruptions per job (where $\log^*$ is the iterated logarithm) are both necessary and sufficient for maintaining $O(1)$-approximate proportional fairness.
• Departure-only: Symmetric results to the arrival-only case; same $O(\log^* n)$ disruption bound holds.
• General arrivals and departures: In the worst case, any deterministic $c$-approximate algorithm incurs $\Omega(n^{1+1/(4c+1)})$ total disruptions. However, if job weights are randomly permuted with respect to arrival/departure order, then a randomized algorithm sustains $O(1)$ expected disruptions per job while maintaining a 4-approximation.

For multiple resources, such as CPU, memory, and disk bandwidth, the algorithm generalizes: apply the method independently to each resource, multiplying the disruption bound by the number of dimensions.

3. Core Techniques for Minimal Disruption

The $O(\log^* n)$ upper bound is achieved via weight bucketing and monotone rounding. Jobs are grouped into exponential weight classes, turning the allocation process into one over "super-jobs." The key is to apply a slowly growing tower-type function $g$ to the inverse fair share $1/x_j^*(t)$, rounding allocations so that each job's share only changes $O(\log^* n)$ times over its lifetime.

Feasibility is maintained by ensuring that the overall allocation is always less than or equal to the resource capacity via geometric series bounds. Faithfulness is guaranteed because monotonic rounding does not reduce any job's ideal share by more than a constant factor.

For the adversarial lower bound, geometric weight sequences force any deterministic algorithm to disrupt allocations at least $O(\log^* n)$ times per job.

Randomized algorithms for general dynamic scenarios utilize random doubling thresholds: allocations are reset for jobs only when the total weight of alive jobs crosses randomly chosen thresholds, ensuring only $O(1)$ expected disruptions per event.

4. Representative Algorithms and Pseudocode

The pseudocode for the arrival-only, proportional-share allocation algorithm is:

def handle_arrival(job_j, w_j, W_t_minus_1, existing_jobs): # Step 1: Place job in its weight group R_j = W_t_minus_1 / w_j # Step 2: Compute integer level (log-star bucketing) ell_j = floor(g_inv(R_j)) # g_inv grows like log^* # Step 3: Assign allocation x_j = (1/d) * (1/g(ell_j)) # Step 4: Update existing jobs' allocations and count disruptions for k in existing_jobs: R_k = W_t_minus_1 / w_k ell_k = floor(g_inv(R_k)) if previous_ell_k != ell_k: # Disruption: update allocation x_k = (1/d) * (1/g(ell_k)) # Step 5: Update total weight W_t = W_t_minus_1 + w_j

Constant $d$ is chosen so that the sum of allocations remains less than one.

5. Extension to Distributed Systems and Scheduling

Dynamic weighted fairness with minimal disruptions is foundational for resource managers in distributed systems, cloud scheduling, and network bandwidth allocation. Its principles extend to queueing theory—e.g., MaxWeight scheduling in overloaded parallel queues (Chan et al., 2010). Here, systems maintain weighted stress ratios (workload backlogs) during overload by selecting fixed diagonal priorities $(\beta_q)$ aligned with a fair-share direction $\theta$, minimizing total backlog growth rate.

Extensions such as the User Weighted Fair Queuing (UWFQ) model in Spark (Kažemaks et al., 17 Oct 2025) manage fairness across users and jobs via virtual time, adaptive weights, and runtime partitioning, ensuring bounded deviations from ideal fairness and minimizing opportunistic rescheduling.

6. Trade-offs, Limitations, and Practical Considerations

There is an inherent trade-off between the tightness of the fairness approximation ($\alpha$) and disruption rate. Decreasing $\alpha$ (improving fairness) increases the number of necessary reallocations; for arbitrary arrival/departure sequences, the lower bound grows polynomially in $n$ as $\Omega(n^{1+1/(4\alpha+1)})$. In realistic scenarios (random weight orders, cluster settings), the randomized algorithm is preferable, achieving constant expected disruptions and low overhead.

Applications in multi-resource environments use independent allocation tracks, with the total disruption bounded multiplicatively by the number of resources.

General guidelines: apply weight bucketing, monotone rounding, and random-threshold resets for online environments with many jobs, prioritizing minimal state changes and allocation churn.

7. Broader Impact and Future Directions

The minimal disruption paradigm is increasingly relevant for cloud resource allocation, streaming data platforms, and recommender systems. Techniques such as incremental fine-tuning with restart in dynamic recommenders (Yoo et al., 2023) and Pareto-frontier optimization for risk fairness (Martinez et al., 2019) further generalize weighted fairness to domains demanding tight trade-offs between fairness, stability, and aggregate utility.

The adoption of tower-type approximations, randomized thresholding, and group-weighted fairness metrics can provide broadly applicable frameworks for other dynamic online optimization problems where state preservation and fairness tracking are critical.