Halton Scheduler in MaskGIT

• Halton scheduler is a quasi-random token selection method that uses low-discrepancy Halton sequences to uniformly unmask tokens in MaskGIT.
• It outperforms confidence and random schedulers by achieving lower FID scores (e.g., 5.3 vs 7.5) and producing richer image details through deterministic token scheduling.
• The method simplifies implementation by eliminating dynamic confidence adjustments and hyperparameter tuning, using a fixed, precomputed unmasking order.

The Halton scheduler is a quasi-random, low-discrepancy token selection strategy introduced for inference in Masked Generative Image Transformers (MaskGIT) to improve the spatial coverage and diversity of image generation. Unlike the previously established Confidence scheduler, the Halton scheduler deterministically selects the next tokens to unmask based on the Halton sequence, offering superior uniformity at each step and reducing non-recoverable errors during parallel token sampling (Besnier et al., 21 Mar 2025).

1. Token Unmasking in MaskGIT and Limitations of Prior Schedulers

MaskGIT models an image as a grid of discrete tokens, $X = \{X^1, ..., X^n\}$. Generation proceeds via iterative unmasking: at each step $s$, a subset $X_s$ of the masked tokens is predicted and revealed, proceeding over $S$ steps until the image is fully unmasked. MaskGIT is trained for per-token cross-entropy, modeling marginals $p(X_s^i|X_{<s})$ rather than the joint $p(X_s^1, ..., X_s^{n_s} | X_{<s})$. This leads to an approximation gap, quantifiable by the mutual information:

$$MI(X_s|X_{<s}) = D_{KL}\left(p(X_s|X_{<s})\,\Vert\,\prod_{i=1}^{n_s}p(X_s^i|X_{<s})\right).$$

The decomposition (Eq. 5) shows two terms: (a) per-token entropy $\sum_{i=1}^{n_s} H(X_s^i|X_{<s})$, and (b) redundancy among tokens $-H(X_s|X_{<s})$. The Confidence scheduler minimizes (a) by selecting the most confident tokens (lowest entropy), but it spatially clusters unmasked tokens, exacerbating redundancy (b). This redundancy increases mutual information and can generate high-entropy “holes” that persist until the final sampling steps, impairing sample quality. To mitigate this, practitioners typically inject Gumbel noise into the selection process—however, the resulting hyperparameter schedule (temperature, noise decay, top-$k$ fraction) is non-trivial to tune and sensitive to sampling step count, with fidelity (FID) degrading unless parameters are meticulously re-optimized as $S$ increases [(Besnier et al., 21 Mar 2025): Sec. 3, Fig. 3, 5].

2. Halton Sequence-Based Token Selection

The Halton sequence is a well-known low-discrepancy sequence, constructed by the radical-inverse function in relatively prime bases (typically $b_1=2$, $b_2=3$ for 2D spaces). Given integer $i$, its representation in base-$b$ as $i = a_m b^m + ... + a_0$ leads to the radical-inverse:

$$\Phi_b(i) = a_0 b^{-1} + a_1 b^{-2} + ... + a_m b^{-m-1}.$$

To schedule $n = W \times H$ token positions for a 2D image grid, two Halton sequences are generated—$H_2$ and $H_3$ of length $n' \geq n$—and their pairs $$(x_i, y_i) = (\lfloor \Phi_2(i) W \rfloor, \lfloor \Phi_3(i) H \rfloor )$$ mapped to the discrete token coordinates. Duplicates are removed to yield a unique permutation of all token positions, forming the deterministic unmasking schedule. This sequence ensures that each prefix of positions uniformly covers the image, in contrast to random or confidence-based selection [(Besnier et al., 21 Mar 2025): Sec. 4, Alg. 1].

3. Theoretical Rationale for Discrepancy Reduction

Low-discrepancy sequences such as Halton ensure star-discrepancy $D^*(H^n) = O((\log N)^d / N)$ as $N \to \infty$, where $d$ is the dimension and $N$ the number of points. Each prefix of the Halton sequence thus approximates a uniform sampling of the image domain, with the fraction of positions in any axis-aligned rectangle closely matching the rectangle’s proportion of area. In MaskGIT inference, this spatial spreading minimizes both the redundancy among recently unmasked tokens and avoids late-stage high-entropy “holes,” thus better controlling the approximation gap to the true joint $p(X_S|X_{<S})$. This echoes quasi-Monte Carlo guarantees such as the Koksma-Hlawka inequality, where integration error is bounded by the product of function variation and discrepancy; by analogy, the Halton scheduler enhances joint coverage and controls error propagation in the generation process [(Besnier et al., 21 Mar 2025): Sec. 5].

4. Empirical Evaluation and Ablative Analysis

Experiments on class-to-image and text-to-image generation tasks reveal the Halton scheduler’s consistent superiority over the Confidence and Random schedulers. For instance, on ImageNet 256×256 (ViT-L, 484M parameters, 32 steps), Halton achieves FID 5.3 (Confidence: 7.5, Random: 9.7) and IS 229.2 (vs 227.2, 196.2). On ImageNet 512×512, FID improves from 8.38 (Confidence) to 6.11 (Halton). On class-to-image synthesis, Halton reaches FID 3.74 compared to 6.18 for the original MaskGIT scheduler. For zero-shot COCO text-to-image (256×256), FID is reduced from 38.9 (aMused confidence) to 28.8, with corresponding increases in CLIP-score, precision, and recall. Ablation studies demonstrate that Halton’s FID continues to decline as the step count $S$ increases (e.g., $S=12 \to 32$), unlike Confidence, which demands careful hyperparameter retuning to avoid worsening performance. Qualitative analysis (Fig. 2, 7, 8) shows that Halton leads to richer backgrounds and finer details, with uniform coverage reducing artifact accumulation [(Besnier et al., 21 Mar 2025): Tab. 2-5, Fig. 2,3,5,7,8,12, Tab. 11].

5. Hyperparameter Efficiency and Practical Implementation

A primary advantage of the Halton scheduler is hyperparameter simplicity. Whereas the Confidence scheduler mandates tuning the softmax temperature, a Gumbel noise schedule, and per-step top-$k$ selection thresholds, the Halton scheduler depends solely on the unmasking schedule $\{n_s\}$, typically chosen as a constant, linear, or square-root schedule over $S$ steps. No retraining, noise injection, or per-step confidence computations are required. The unmasking order is generated by precomputing the Halton sequence mapping and can be stored as a list of length $n$, yielding a minimal code addition (≈20 lines) and allowing drop-in replacement in existing MaskGIT implementations. Inference proceeds by unmasking the next $\{n_s\}$ tokens at each step from this precomputed order, with no modification to the model’s generative or masking logic [(Besnier et al., 21 Mar 2025): Sec. 7].

6. Comparative Summary of Scheduler Behavior

Scheduler	Position Selection	Hyperparameters	Empirical Results
Confidence	Greedy by confidence	Temp., noise, top-$k$	Prone to spatial clustering
Random	Uniform random	Step count, random seed	Higher FID, less uniformity
Halton	Deterministic (Halton)	Step count ($S$) only	Best FID, detail, diversity

The Halton scheduler’s spatial spreading prevents non-recoverable late-stage artifacts typical of confidence clustering, while eliminating the need to balance conflicting hyperparameters as required by noise-injected strategies [(Besnier et al., 21 Mar 2025): Tab. 2-5, Fig. 3, 5].

7. Implementation Workflow

The integration of the Halton scheduler into MaskGIT follows a two-stage process:

1. Offline Preprocessing:
   • Compute Halton sequences $H_2$, $H_3$ of length $n' \geq W \cdot H$ in bases 2 and 3.
   • Form coordinate pairs, discretize, deduplicate, and retain only $W \cdot H$ unique positions to obtain the unmasking order.
2. Sampling Loop:
   • For $S$ steps, unmask the next $n_s$ tokens from the fixed list; continue until all tokens are revealed.

No confidence calculation, noise application, or dynamic selection is required within the main loop. A reference pseudocode sketch mirrors that described in (Besnier et al., 21 Mar 2025) (see Sec. 7).

The Halton scheduler for MaskGIT thus provides an efficient, theoretically-grounded, and empirically validated solution for progressive image token unmasking, yielding enhanced sample quality, diversity, and implementation simplicity compared to previous scheduling techniques (Besnier et al., 21 Mar 2025).