Cluster Hyper-Grid Scheduling (PSTS)

• Cluster Hyper-Grid Scheduling (PSTS) is a dynamic, adaptive algorithm that maps cluster nodes onto k-dimensional hyper-grids for efficient task scheduling and load balancing.
• It employs exclusive parallel prefix-scan operations to redistribute workloads in-place, achieving near-perfect load balance while minimizing makespan.
• The algorithm optimizes scheduling cost by leveraging recursive, logarithmic-depth computation, making it scalable for heterogeneous and dynamically evolving clusters.

Cluster Hyper-Grid Scheduling (PSTS), also referred to as Positional Scan Task Scheduling, is a dynamic, nonpreemptive, and adaptive algorithm designed for efficient task scheduling and load balancing in cluster computing environments. Rooted in the divide and conquer paradigm, PSTS leverages a recursive mapping of the physical cluster onto k-dimensional hyper-grids, applying parallel prefix-scan primitives for in-place, scalable redistribution of workloads. This approach aims to equalize the task load assigned to each node proportional to its relative processing capacity, thereby minimizing makespan and enhancing throughput in heterogeneous and dynamically evolving clusters (Savvas et al., 2019).

1. Formal Model and Hyper-Grid Construction

A computational cluster is modeled as an undirected graph $G(V,E)$, with $V = \{v_1, v_2, ..., v_N\}$ representing $N$ processing nodes and $E$ the set of bidirectional network links. Each node $v_i$ is characterized by its processing power $T_i \in \mathbb{R}^+$ and normalized power share $y_i = T_i / \sum_{j=1}^N T_j$. Tasks are given as $T = \{t_1, ..., t_m\}$, each defined by its computational weight $w(t)$ and packet communication size $p(t)$. The overall work to be distributed is $W = \sum_{t \in T} w(t)$.

The k-dimensional hyper-grid, $G_k = [p_1 \times p_2 \times ... \times p_k]$, is constructed as the direct Cartesian product of linear arrays of sizes $p_1 ... p_k$. Cluster nodes are bijectively mapped to grid cells (hyper-nodes), padding with virtual nodes to account for unused grid positions if necessary. Each real node belongs to $k$ different sub-grids (one along each dimension), supporting multidimensional partitioning and balancing.

2. Scheduling Objectives and Prefix-Scan Mechanism

The principal objective is to rebalance tasks so that every node $v_i$ ultimately receives a load $W_i \approx y_i W$, minimizing the makespan $\min \max_{i=1...N} (W_i / T_i)$ given the constraints $\sum_{i=1}^N W_i = W$ and $W_i \geq 0$.

The central operation underpinning the balancing is the exclusive parallel prefix-scan (Scan), defined for an array $A = [a_0, ..., a_{n-1}]$ by

$$\text{Scan}(A)[j] = \sum_{t=0}^{j-1} a_t, \quad j = 0...n$$

Utilized on linear array topologies, Scan executes in $O(n)$ time, requiring $2(n-1)$ communication and local computation steps. This primitive facilitates efficient range-summation and partitioning of workloads along each grid dimension.

3. Recursive PSTS Algorithm and Pseudocode

At each recursion level $r$ (from $1$ to $k$), PSTS performs the following in parallel for every $r$-D sub-grid:

• Computes exclusive prefix-scans of total work and normalized power along the current dimension, yielding $S_r$ (loads) and $A_r$ (normalized powers).
• The terminal node in each sub-grid broadcasts total load and power, as well as the scan results, to its sub-grid peers.
• Each sub-grid determines its target load fraction for perfect balance: $target_{rq} = P_{rq} \cdot (\sum_{q'} W_{rq'})$.
• Sub-grids labeled as "senders" (with surplus load) compact their excess workloads using PSLB, then migrate indivisible tasks to designated "receiver" sub-grids, partitioned via scan indices.
• "Receiver" sub-grids rebalance tasks locally via 1-D PSLB. When all required migrations and intra-sub-grid rebalancing are complete, the process recurses to the next dimension.

The process results in near-perfect proportional load balancing subject to the indivisibility of tasks.

PSTS Pseudocode (abridged):

Algorithm PSTS(G_k, T): r ← 1 while r ≤ k do For each r–D sub-grid Q in parallel: Compute L_r[i] = total work in line i (i=1…p_r) Compute Y_r[i] = total normalized power in line i S_r = Scan(L_r); A_r = Scan(Y_r) W_Q = sum_i L_r[i]; P_Q = sum_i Y_r[i] Broadcast W_Q, P_Q, S_r, A_r target_r[i] = A_r[i]·W_Q if W_Q < sum(target_r): // receiver Invoke 1–D PSLB on each line else: // sender Use PSLB to pack excess Partition and migrate tasks by A_r 1–D PSLB on residual lines r ← r + 1 Return new assignments

4. Hyper-Grid Dimension Optimization and Complexity

Hyper-grid dimension $k$ critically impacts the parallelism and efficiency of PSTS. Given $N' = \prod_{j=1}^k n_j \geq N$, the per-level message and computation costs are $S_\text{comm}(d) = 2\ell p$ and $S_\text{comp}(d) = 2\ell q$ where $\ell = \sum_{j=1}^d n_j - d$, $p$ is per-message cost, and $q$ is per-local-computation cost. The total scheduling cost sums over all $k$ levels:

$$S_\text{total}(k) = \sum_{d=1}^k [2(\sum_{j=1}^d n_j - d)(p+q)]$$

Optimizing $k$ yields $n_j \approx N^{1/k}$ and the optimal choice is $k^* = \lfloor \log_2 N \rfloor$, attaining $S_\text{total}(k^*) = O(\log N \cdot (p+q))$, contrasting starkly with the $O(N \cdot (p+q))$ cost of naïve 1-D scheduling. This logarithmic depth underpins PSTS's scalability on large clusters (Savvas et al., 2019).

5. Performance, Overhead, and Crossover Thresholds

Simulations on clusters of heterogeneous nodes (Sun UltraSPARC IIi and PC) with $m=4,000$ randomly-sized tasks (work and communication) evaluated overhead scalability, makespan speedup, and balancing thresholds. Key results:

• For $d=1$, overhead grows nearly linearly with $N$ (from 50 at $N=16$ to $\approx$ 1,200 at $N=64$).
• For $d>1$, overhead is reduced to peaks of $\approx$ 200 at $N=64$ with $k=4$ or $5$.
• PSTS yields up to $20-30\%$ makespan improvement under moderate imbalance.
• The critical crossover imbalance threshold $\rho^*$ drops as dimension increases, as shown:

Number of nodes	$\rho^*$ (d=1)	$\rho^*$ (d $\geq$ log$_2$N)
16	1.20	1.02
32	1.15	1.01
64	1.10	1.00

Empirically, PSTS can be triggered for imbalances as low as $1.00 \leq \rho^* \leq 1.20$, depending on grid dimension, enabling balancing for even modest workload skews.

6. Comparative Assessment and Applicability

Cluster Hyper-Grid Scheduling exhibits several salient strengths:

• Parallel, recursive divide and conquer design achieves $O(\log N)$ communication and computation depth.
• Locality-preserving: tasks nearby in the index space remain close, reducing communication overhead for localized workloads.
• Adopts a mixed centralized/decentralized control structure, ensuring resilience to bottlenecks.
• Adaptive activation, with low imbalance thresholds, allows dynamic reaction to workload fluctuations.

Limitations include the requisite for explicit hyper-grid embedding, making irregular topologies more cumbersome (necessitating virtual node/link padding), and the assumption of independent, migratable tasks (Bag-of-Tasks model). Task precedence constraints and costly migrations for fine-grained tasks are not directly supported; threshold tuning is required to mitigate overhead in such cases.

PSTS contrasts with static heuristics (which require foreknowledge and are non-adaptive), diffusion and work-stealing methods (which mix slower, $O(N)$), and 1-D PSLB (which scales linearly and lacks parallelism). It is particularly well-suited for dynamic, heterogeneous, and scalable cluster environments where nodes may join or exit, and workloads are highly irregular (Savvas et al., 2019).