VibeToken: Resolution-Agnostic Image Tokenizer

• VibeToken is a resolution-agnostic 1D image tokenizer that compresses images into a dynamic sequence of tokens, decoupling token count from image resolution.
• It uses adaptive patch embedding, dynamic grid positional embedding, and multi-vector quantization to support efficient autoregressive synthesis over various aspect ratios.
• VibeToken-Gen, a class-conditioned autoregressive generator built on VibeToken, achieves high-resolution image generation with a fixed latent token budget, substantially reducing computational cost.

Searching arXiv for the specified VibeToken paper to ground the article in the cited source. VibeToken is a resolution-agnostic 1D image tokenizer introduced for autoregressive image synthesis across arbitrary resolutions and aspect ratios (Patel et al., 27 Apr 2026). In the same work, VibeToken-Gen denotes the class-conditioned autoregressive generator trained on top of its discrete latents. The central design goal is to decouple token length from image resolution: instead of inheriting a 2D latent grid whose token count scales with image size, VibeToken compresses an image into a dynamic, user-controllable sequence of 32–256 tokens, with headline experiments emphasizing 64-token generation at $1024\times1024$ (Patel et al., 27 Apr 2026). The paper positions this as a way to narrow the gap between autoregressive models and diffusion models at scale while preserving out-of-the-box support for arbitrary resolutions.

1. Problem setting and definition

VibeToken is motivated by a specific bottleneck in visual autoregressive modeling: standard visual AR pipelines usually inherit tokens from 2D grid tokenizers, so the token count grows with spatial resolution as

$T = \frac{H}{f}\cdot \frac{W}{f},$

which makes transformer cost rise rapidly with image size (Patel et al., 27 Apr 2026). The autoregressive factorization used in the paper is the standard next-token form

$$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$

or, with end-of-sequence handling,

$$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$

Because transformer self-attention scales quadratically in sequence length, fixed-grid tokenizers make high-resolution AR generation computationally expensive (Patel et al., 27 Apr 2026).

The paper defines VibeToken as a 1D tokenizer intended to break that coupling. Rather than preserving a latent 2D lattice, it maps an image of arbitrary resolution and aspect ratio into a short 1D discrete sequence, then decodes that sequence back to a target canvas (Patel et al., 27 Apr 2026). In the paper’s framing, “resolution-agnostic” means that the tokenizer can encode images at different resolutions and aspect ratios, decode to arbitrary target resolutions, and support native super-resolution without retraining a separate model per canvas (Patel et al., 27 Apr 2026).

A common point of confusion is nomenclature. In the VibeVoice technical report, “VibeToken” is not the paper’s formal term; that work instead describes a continuous VAE-style acoustic tokenizer for speech (Peng et al., 26 Aug 2025). In (Patel et al., 27 Apr 2026), by contrast, VibeToken refers specifically to a 1D image tokenizer for dynamic-resolution image generation.

2. Tokenizer architecture

VibeToken is a Transformer-based 1D tokenizer built around four components named explicitly in the paper: dynamic grid positional embedding, adaptive patch embedding, adaptive decoder resolution, and dynamic-length latent tokenization (Patel et al., 27 Apr 2026). The starting point is a TiTok-style encoder-decoder formulation. For an image

$v \in \mathbb{R}^{3\times H \times W},$

with patch size $k$, the image is patchified into

$N = \frac{HW}{k^2}$

patches, projected to $\mathbb{R}^d$, concatenated with $L$ learned latent tokens, and passed through the encoder: $$x^{\mathrm{enc}} = \mathcal{E}_{\theta}(x_0)\in\mathbb{R}^{(N+L)\times d}, \qquad h=x^{\mathrm{enc}}_{N+1:N+L}\in\mathbb{R}^{L\times d}.$$ A codebook

$T = \frac{H}{f}\cdot \frac{W}{f},$0

then quantizes the latent sequence as

$T = \frac{H}{f}\cdot \frac{W}{f},$1

The decoder consumes quantized token embeddings and masked output tokens: $T = \frac{H}{f}\cdot \frac{W}{f},$2 This is the baseline 1D-tokenizer formulation that VibeToken generalizes (Patel et al., 27 Apr 2026).

The dynamic grid positional embedding addresses arbitrary-resolution inputs. The paper learns a base positional grid

$T = \frac{H}{f}\cdot \frac{W}{f},$3

with $T = \frac{H}{f}\cdot \frac{W}{f},$4, and resizes it to the current patch lattice: $T = \frac{H}{f}\cdot \frac{W}{f},$5

$T = \frac{H}{f}\cdot \frac{W}{f},$6

The paper reports that this dynamic grid embedding yields about 33% FLOPs reduction relative to learnable axial RoPE, without quality loss (Patel et al., 27 Apr 2026).

Adaptive patch embedding makes patch size variable rather than fixed. Let $T = \frac{H}{f}\cdot \frac{W}{f},$7 be the learned patch projection for the maximum patch size $T = \frac{H}{f}\cdot \frac{W}{f},$8, and $T = \frac{H}{f}\cdot \frac{W}{f},$9 a resizing operator. The paper defines

$$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$0

In practice, the model uses $$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$1 and trains over patch sizes $$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$2 (Patel et al., 27 Apr 2026). This allows the encoder to handle variable patchification schemes without learning separate projections for each case.

Adaptive decoder resolution decouples patch layout from output canvas. The decoder first predicts an intermediate image at

$$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$3

producing $$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$4, then applies a learned 2D convolutional downscaling layer to obtain the target output

$$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$5

This is the mechanism that the paper uses to support arbitrary target resolutions and native super-resolution (Patel et al., 27 Apr 2026).

Dynamic-length latent tokenization is the final architectural ingredient. Instead of fixing one sequence length, VibeToken samples a latent length

$$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$6

with the reported operating range $$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$7 (Patel et al., 27 Apr 2026). The encoder produces exactly $$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$8 latent tokens and the decoder consumes exactly $$p(x_{1:T}) = \prod_{t=1}^{T} p_{\theta}(x_t \mid x_{<t}),$$9 latent tokens, without padding. This gives a direct quality–efficiency trade-off under user control.

3. Quantization, objectives, and training regime

The final tokenizer uses multi-vector quantization. The implementation details summarized in the paper specify 8 codebooks, each of size 4096, for an effective vocabulary size of 32,768; the token latent dimension is 256, factorized into 8 sub-codes of 32 dimensions each (Patel et al., 27 Apr 2026). The paper does not reproduce the full MVQ update equations in the visible main text, but it does state that quantization uses

$$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$0

and that a reparameterization trick is applied so gradients flow to $$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$1 and $$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$2 (Patel et al., 27 Apr 2026).

The full tokenizer loss is not written out explicitly in the main text excerpt, but the supplement reports the component weights. The loss includes reconstruction, perceptual, discriminator, quantizer, and commitment terms, with reconstruction weight $$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$3, perceptual weight $$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$4, discriminator weight $$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$5, quantizer weight $$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$6, commitment cost $$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$7, and discriminator start at 300,000 steps (Patel et al., 27 Apr 2026). A structured summary consistent with the reported components is

$$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$8

with

$$p(x_{1:T}, \langle\mathrm{eos}\rangle) = \prod_{t=1}^{T+1} p_{\theta}(x_t \mid x_{<t}), \qquad x_{T+1}=\langle\mathrm{eos}\rangle.$$9

The paper itself does not spell out the exact closed forms of each component in the excerpted text (Patel et al., 27 Apr 2026).

Tokenizer training is carried out on ImageNet-1k under mixed resolutions between $v \in \mathbb{R}^{3\times H \times W},$0 and $v \in \mathbb{R}^{3\times H \times W},$1, with aspect ratios sampled from $v \in \mathbb{R}^{3\times H \times W},$2 (Patel et al., 27 Apr 2026). Patch sizes are sampled from $v \in \mathbb{R}^{3\times H \times W},$3, subject to the condition $v \in \mathbb{R}^{3\times H \times W},$4, and the input resolution $v \in \mathbb{R}^{3\times H \times W},$5 and target resolution $v \in \mathbb{R}^{3\times H \times W},$6 are sampled independently (Patel et al., 27 Apr 2026). This suggests that the model is trained not merely for same-resolution reconstruction, but for a genuinely flexible encode–decode mapping across canvases.

The paper trains two tokenizer variants: VibeToken-SL, described as a small encoder with large decoder, and VibeToken-LL, described as a large encoder with large decoder (Patel et al., 27 Apr 2026). The default downstream choice is VibeToken-LL.

4. VibeToken-Gen and autoregressive generation

VibeToken-Gen is the class-conditioned autoregressive generator trained over VibeToken latents (Patel et al., 27 Apr 2026). The AR training objective is the standard next-token likelihood, which the paper describes as maximizing log-likelihood or minimizing next-token cross-entropy. In structured form,

$v \in \mathbb{R}^{3\times H \times W},$7

where $v \in \mathbb{R}^{3\times H \times W},$8 denotes conditioning. The paper’s conditioning includes both class and target resolution: $v \in \mathbb{R}^{3\times H \times W},$9 with $k$0 (Patel et al., 27 Apr 2026). This resolution-conditioning is used to prevent stretching artifacts and to make the generator compatible with arbitrary output canvases.

Architecturally, the paper keeps the LlamaGen-style stack largely unchanged, while replacing the tokenizer with VibeToken-MVQ-LL, adding Query-Key LayerNorm for stability, and using a 4-layer residual transformer prediction head after UniTok to predict the 8 MVQ sub-codes per token (Patel et al., 27 Apr 2026). Reported model scales include GPT-B at about 90M parameters and GPT-XXL at about 1.4B parameters, with another table reporting 1.5B for GPT-XXL (Patel et al., 27 Apr 2026). The paper also notes that AR training is performed in fp32 because bf16 was unstable (Patel et al., 27 Apr 2026).

VibeToken-Gen is trained after the tokenizer, not jointly with it (Patel et al., 27 Apr 2026). The training data are again ImageNet-1k under the same mixed-resolution, mixed-aspect-ratio regime, using latent lengths sampled from $k$1, with class dropout probability 0.1 for classifier-free guidance (Patel et al., 27 Apr 2026). Training duration is reported as 300 epochs for GPT-B and 150 epochs for GPT-XXL (Patel et al., 27 Apr 2026).

At inference, generation proceeds by choosing a class $k$2, choosing a target resolution $k$3, sampling a short latent sequence autoregressively, and decoding it with the VibeToken decoder (Patel et al., 27 Apr 2026). Quantitative results generally use temperature $k$4, top-$k$5, and top-$k$6, while qualitative settings include CFG $k$7, temperature $k$8, top-$k$9, and top-$N = \frac{HW}{k^2}$0 (Patel et al., 27 Apr 2026). A notable empirical result is that 64 tokens are often sufficient for generation, and in one GPT-B ablation at $N = \frac{HW}{k^2}$1, 64 tokens outperform 128 and 256 on gFID (Patel et al., 27 Apr 2026).

5. Efficiency, reconstruction, and generation results

The paper’s principal systems claim is that VibeToken shifts AR compute from being resolution-dependent to being token-budget-dependent (Patel et al., 27 Apr 2026). For conventional 2D tokenizers, token count scales as

$N = \frac{HW}{k^2}$2

so attention cost is approximately $N = \frac{HW}{k^2}$3. For VibeToken-Gen with fixed latent length $N = \frac{HW}{k^2}$4, the AR cost is instead governed by $N = \frac{HW}{k^2}$5, which is effectively constant across resolutions when $N = \frac{HW}{k^2}$6 is held fixed (Patel et al., 27 Apr 2026).

The tokenizer-side scaling argument is analogous. The paper reports that a 2D tokenizer baseline, IBQ, grows from about 0.64T to 10.30T FLOPs from $N = \frac{HW}{k^2}$7 to $N = \frac{HW}{k^2}$8, whereas VibeToken remains around 1.04T FLOPs maximum, effectively constant (Patel et al., 27 Apr 2026). On the generator side, the abstract contrasts fixed-resolution AR and VibeToken-Gen by stating that LlamaGen reaches about 11T FLOPs at $N = \frac{HW}{k^2}$9, while VibeToken-Gen maintains 179G FLOPs, or 63.4× greater efficiency, independent of resolution (Patel et al., 27 Apr 2026).

The main reported high-resolution comparison is summarized below.

Model / comparison	Tokens at $\mathbb{R}^d$0	Reported result
VibeToken-Gen	64	179G FLOPs
Diffusion alternative (NiT)	1,024	5.87 gFID
LlamaGen	not stated here as token count	11T FLOPs

The abstract reports that VibeToken-Gen synthesizes $\mathbb{R}^d$1 images using only 64 tokens and achieves 3.94 gFID, whereas the diffusion-based alternative requires 1,024 tokens and attains 5.87 gFID (Patel et al., 27 Apr 2026). A later high-resolution results table, however, reports 3.54 gFID for VibeToken-Gen-XXL at $\mathbb{R}^d$2 against 5.87 gFID for NiT-XL (Patel et al., 27 Apr 2026). This discrepancy is present in the paper summary itself.

Latency results show the same scaling pattern. For tokenizer encode/decode, LlamaGen-Tok rises from 0.005 s at $\mathbb{R}^d$3 to 0.082 s at $\mathbb{R}^d$4, while VibeToken-LL remains at 0.017 s at both resolutions (Patel et al., 27 Apr 2026). For end-to-end generation, LlamaGen-XXL rises from 0.20 s at $\mathbb{R}^d$5 to 32.79 s at $\mathbb{R}^d$6, whereas VibeToken-Gen-XXL remains at 0.46 s at both $\mathbb{R}^d$7 and $\mathbb{R}^d$8 (Patel et al., 27 Apr 2026).

On tokenizer reconstruction, the reported VibeToken-LL rFID values are 0.40 at $\mathbb{R}^d$9, 0.51 at $L$0, 2.40 at $L$1, and 3.60 on arbitrary-resolution stress tests (Patel et al., 27 Apr 2026). On ImageNet, VibeToken-LL at $L$2 reports PSNR 25.04, SSIM 0.8194, LPIPS 0.1048, and rFID 0.40; at $L$3, PSNR 23.37, SSIM 0.7649, LPIPS 0.1867, and rFID 0.51 (Patel et al., 27 Apr 2026). The paper interprets this as strong generalization beyond the tokenizer’s $L$4 training ceiling.

The paper also reports native super-resolution. On FFHQ $L$5, VibeToken-LL gives, for $L$6 super-resolution, PSNR 24.98, SSIM 0.838, and LPIPS 0.261; for $L$7, PSNR 24.11, SSIM 0.805, and LPIPS 0.310 (Patel et al., 27 Apr 2026). The paper notes that diffusion upscalers may have higher PSNR, while VibeToken can be stronger on SSIM or LPIPS in some cases (Patel et al., 27 Apr 2026).

For arbitrary-resolution generation, VibeToken-Gen-XXL reports an average gFID of 5.63 on lower-resolution arbitrary-canvas settings, compared with 4.22 for NiT-XL, and an average gFID of 5.53 on higher resolutions from 512 to 1024, compared with 6.05 for NiT-XL (Patel et al., 27 Apr 2026). This suggests that the model is less dominant at low resolutions than at high resolutions.

6. Interpretation, trade-offs, and limitations

The paper’s central technical claim is not merely that VibeToken is a compact tokenizer, but that it makes a generalist AR image model practical across multiple resolutions and aspect ratios (Patel et al., 27 Apr 2026). This is distinct from fixed-resolution specialist systems such as LlamaGen, which may achieve lower FID at exactly $L$8 but incur rapidly increasing cost when extended to $L$9 (Patel et al., 27 Apr 2026). A plausible implication is that VibeToken shifts the main burden of scaling from the AR backbone to the tokenizer interface, making token budget rather than canvas size the dominant control variable.

The trade-off is explicit in the ablations. For reconstruction, 128 or 256 tokens improve fidelity, but for generation the paper finds that 64 tokens can be best (Patel et al., 27 Apr 2026). This suggests that later tokens encode detail valuable for reconstruction yet not necessarily beneficial for the distribution learned by the generator. The paper also notes that VibeToken-Gen, as a generalist trained under a mixed-resolution regime and limited compute budget, can trail specialist fixed-resolution models at exactly $$x^{\mathrm{enc}} = \mathcal{E}_{\theta}(x_0)\in\mathbb{R}^{(N+L)\times d}, \qquad h=x^{\mathrm{enc}}_{N+1:N+L}\in\mathbb{R}^{L\times d}.$$0 or $$x^{\mathrm{enc}} = \mathcal{E}_{\theta}(x_0)\in\mathbb{R}^{(N+L)\times d}, \qquad h=x^{\mathrm{enc}}_{N+1:N+L}\in\mathbb{R}^{L\times d}.$$1 (Patel et al., 27 Apr 2026).

Several limitations are stated directly. Experiments are limited to ImageNet-1k and class-conditional generation (Patel et al., 27 Apr 2026). The paper does not present text-to-image generation or open-vocabulary conditioning (Patel et al., 27 Apr 2026). It suggests that longer training, stronger augmentation, larger pretraining corpora, larger models, randomized orderings, scale-wise AR training, and alternative quantization schemes are natural future directions (Patel et al., 27 Apr 2026). The paper also notes that without target-resolution conditioning the decoder can exhibit stretching artifacts, and qualitative generations may show crops or truncations attributed to randomized cropping during AR training (Patel et al., 27 Apr 2026).

Within the broader literature supplied here, VibeToken belongs to a wider trend toward flexible token budgets and token-efficient multimodal modeling. TokenFLEX, for example, addresses variable visual token counts in VLMs through stochastic multi-budget training (Hu et al., 4 Apr 2025). VibeToken differs in focusing on discrete 1D image tokenization and autoregressive image generation rather than vision-language understanding (Patel et al., 27 Apr 2026). This suggests that “flexible token count” and “resolution agnosticism” are related but not identical research programs.

Taken as a whole, VibeToken is best understood as a tokenizer-centered reconfiguration of autoregressive image generation (Patel et al., 27 Apr 2026). Its significance lies in demonstrating that a short, dynamic, resolution-agnostic 1D latent sequence can support arbitrary-resolution generation, native super-resolution, and substantially flatter compute scaling than fixed-grid AR baselines.