One-D-Piece: Image Tokenizer Meets Quality-Controllable Compression (2501.10064v1)

Abstract: Current image tokenization methods require a large number of tokens to capture the information contained within images. Although the amount of information varies across images, most image tokenizers only support fixed-length tokenization, leading to inefficiency in token allocation. In this study, we introduce One-D-Piece, a discrete image tokenizer designed for variable-length tokenization, achieving quality-controllable mechanism. To enable variable compression rate, we introduce a simple but effective regularization mechanism named "Tail Token Drop" into discrete one-dimensional image tokenizers. This method encourages critical information to concentrate at the head of the token sequence, enabling support of variadic tokenization, while preserving state-of-the-art reconstruction quality. We evaluate our tokenizer across multiple reconstruction quality metrics and find that it delivers significantly better perceptual quality than existing quality-controllable compression methods, including JPEG and WebP, at smaller byte sizes. Furthermore, we assess our tokenizer on various downstream computer vision tasks, including image classification, object detection, semantic segmentation, and depth estimation, confirming its adaptability to numerous applications compared to other variable-rate methods. Our approach demonstrates the versatility of variable-length discrete image tokenization, establishing a new paradigm in both compression efficiency and reconstruction performance. Finally, we validate the effectiveness of tail token drop via detailed analysis of tokenizers.

• The paper introduces One-D-Piece with Tail Token Drop to enable variable-length tokenization for quality-controllable image compression.
• It employs a Transformer-based architecture and a two-stage training strategy to achieve efficient compression, outperforming JPEG and WebP.
• Evaluations demonstrate improved reconstruction quality and versatility in downstream tasks, highlighting both practical benefits and theoretical advances.

Analysis of "One-D-Piece: Image Tokenizer Meets Quality-Controllable Compression"

The presented research paper introduces a novel approach named One-D-Piece, which contributes to the field of image compression and tokenization. The authors propose a discrete image tokenizer that allows for variable-length tokenization, thereby facilitating quality-controllable image compression. This research addresses limitations in existing tokenizers that rely on fixed-length sequences, posing inefficiencies when encoding diverse image information.

The core innovation in this work is the Tail Token Drop regularization mechanism. This technique encourages necessary image information to be concentrated at the beginning of the token sequence, enhancing the ability to adjust token lengths and thereby controlling the image quality. The authors demonstrate that One-D-Piece achieves state-of-the-art reconstruction quality across different metrics while also providing enhanced compression efficiency compared to traditional approaches like JPEG and WebP, especially at lower byte sizes.

Methodological Contributions

1. Variable-Length Tokenization: One-D-Piece offers a flexible tokenization approach, allowing token counts to vary between 1 and 256. This flexibility sets a new standard in visual data tokenization, contrasting with the fixed-length requirements of existing models.
2. Tail Token Drop Regularization: This novel technique fundamentally enhances the capacity to adapt compression levels based on specific quality needs. The model is trained to allocate important information upfront, enabling efficient compression and improved perceptual quality without increasing reconstruction complexity.
3. Architecture and Training: The One-D-Piece employs a Transformer-based architecture similar to TiTok, but with significant adaptations to support variable token lengths. A two-stage training strategy is implemented, first learning the logits of a pre-existing tokenizer, followed by reconstructing image details with regularization in place.

Evaluation and Results

The evaluation of One-D-Piece across multiple benchmarks indicates superior performance:

• With a variable token count, One-D-Piece surpasses JPEG, JPEG 2000, and WebP in terms of Fréchet Inception Distance (FID) while achieving high-quality reconstructions even at low byte sizes.
• The model also exhibits strong adaptability in downstream tasks like image classification, object detection, and semantic segmentation. Interestingly, these results show that One-D-Piece can outperform existing algorithm-based image compression formats, even when using a significantly smaller number of tokens.
• The primary analysis underscores the model’s effectiveness, indicating that critical image information is indeed prioritized and captured early in the token sequence, as intended by the Tail Token Drop mechanism.

Discussion and Implications

The implications of this work are multifaceted. Practically, the model's ability to control tokenization quality dynamically means it could greatly benefit applications requiring different levels of image fidelity, such as real-time video processing, efficient storage, and transmission in constrained bandwidth scenarios.

Theoretically, the novel regularization method and the flexibility of the architecture propose a new paradigm, offering a potentially broader application of discrete tokenization across other domains like audio and video data. Future work might extend these concepts beyond image data, enhancing adaptive encoding approaches in AI-driven contexts.

This paper reaffirms the viability of integrating adaptive, model-driven strategies into neural network design, marrying the strengths of classical compression with modern tokenization to offer a robust solution for diverse applications in the rapidly evolving field of visual data processing.