I-Segmenter: Integer-Only ViT Segmentation

• I-Segmenter is an integer-only adaptation of a ViT segmentation model that replaces all floating-point operations with precise integer counterparts to preserve accuracy.
• It employs symmetric uniform quantization, custom operators like λ-ShiftGELU, and modifications in the decoder to ensure complete integer execution.
• The framework supports both one-shot PTQ and QAT, achieving competitive accuracy with reduced model size and faster inference on challenging benchmarks.

I-Segmenter is an integer-only adaptation of a Vision Transformer-based semantic segmentation model that builds on the Segmenter architecture and systematically replaces floating-point operations with integer-only counterparts throughout both the encoder and decoder (Sassoon et al., 12 Sep 2025). It is presented as the first fully integer-only ViT segmentation framework, motivated by the observation that ViT-based segmentation models are fragile under low precision because quantization errors accumulate across deep encoder-decoder pipelines (Sassoon et al., 12 Sep 2025). Within that design space, I-Segmenter combines symmetric uniform quantization, integer approximations for nonlinear operators, a modified decoder, and a new activation function called $\lambda$-ShiftGELU in order to preserve semantic segmentation accuracy while improving deployment efficiency on resource-constrained hardware (Sassoon et al., 12 Sep 2025).

1. Scope and problem setting

I-Segmenter addresses semantic segmentation with Vision Transformers under strict deployment constraints. The core problem is that Vision Transformers have recently achieved strong results in semantic segmentation, yet their deployment on resource-constrained devices remains limited due to their high memory footprint and computational cost (Sassoon et al., 12 Sep 2025). The paper positions quantization as an effective strategy to improve efficiency, while emphasizing that low-precision ViT segmentation is unusually brittle because errors propagate through both the ViT encoder and the mask decoder (Sassoon et al., 12 Sep 2025).

The framework is specifically an integer-only adaptation of Segmenter. In the paper’s formulation, the encoder is a standard ViT that processes non-overlapping image patches via self-attention and outputs contextualized patch embeddings, while the decoder is a Mask Transformer with class-specific learnable mask embeddings and additional Transformer blocks for segmentation mask generation (Sassoon et al., 12 Sep 2025). I-Segmenter retains that high-level structure but modifies the computational graph so that inference can be carried out without floating-point operators (Sassoon et al., 12 Sep 2025).

This design choice distinguishes I-Segmenter from segmentation systems that optimize for other constraints. For example, Inter2Former targets high-precision interactive segmentation with CPU efficiency through Dynamic Prompt Embedding, Dynamic Hybrid Attention, Hybrid Mixture of Experts, and Dynamic Local Upsampling (Huang et al., 13 Jul 2025), whereas I-Segmenter targets integer-only semantic segmentation with a ViT encoder-decoder (Sassoon et al., 12 Sep 2025).

2. Integer-only adaptation of the Segmenter architecture

The architectural premise of I-Segmenter is systematic replacement rather than partial quantization. The paper states that every step—from quantization of weights and activations to approximating nonlinear functions and modifying specific structural components—is adapted for true integer-only computation (Sassoon et al., 12 Sep 2025). The encoder remains a ViT, and the decoder remains a Transformer-based mask decoder, but both are rewritten around integer-compatible operators (Sassoon et al., 12 Sep 2025).

The quantization scheme follows I-ViT and extends it for segmentation. Symmetric uniform quantization is applied to all weights, biases, embeddings, and activations: $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$ where $F$ is the full-precision value, $m$ is the dynamic range bound, and $k$ is the bitwidth (Sassoon et al., 12 Sep 2025). The scaling factor is then approximated in dyadic form so that scaling can be implemented through integer multiplication and bit-shifting: $I \cdot S \approx (I \cdot b) \gg c$ with integer parameters $b$ and $c$ (Sassoon et al., 12 Sep 2025).

The framework quantizes all class and positional embeddings for integer-only storage and execution, and it uses INT8 activations, INT8 weights, and INT32 biases, with mixed precision as needed to avoid overflow, especially in residuals and attention (Sassoon et al., 12 Sep 2025). Linear layers such as matrix multiplication, linear projections, and convolutions are performed with quantized inputs and weights accumulated in INT32 before requantization to INT8; residual connections are performed as INT16 addition with INT32 accumulation followed by requantization (Sassoon et al., 12 Sep 2025).

A concise view of the principal substitutions is given below.

Component	FP32 Segmenter	I-Segmenter
Data type	FP32	INT8 throughout (weights/acts)
GELU	FP32 GELU	$\lambda$-ShiftGELU (integer only)
Norm	L2, LayerNorm	INT LayerNorm; L2 Norm removed
Decoder upsampling	Bilinear	Nearest Neighbour

These substitutions are not independent implementation details. Taken together, they ensure that the computational graph does not silently fall back to floating-point operations at normalization, activation, or upsampling stages (Sassoon et al., 12 Sep 2025).

3. Quantized operators and training modes

The integer-only graph requires replacements for several operations that are straightforward in floating point but awkward under discrete arithmetic. I-Segmenter therefore uses integer-only Shiftmax for softmax and I-LayerNorm for layer normalization, while replacing standard GELU with $\lambda$-ShiftGELU (Sassoon et al., 12 Sep 2025). This is significant because the fragility of ViT segmentation under low precision is not confined to matrix multiplications; it also arises from repeated nonlinear and normalization steps across deep encoder-decoder stacks (Sassoon et al., 12 Sep 2025).

The framework supports both post-training quantization and quantization-aware training. In PTQ, only scales and zero-points are updated while weights remain frozen, and calibration can be performed with a very small number of samples (Sassoon et al., 12 Sep 2025). In QAT, the model is retrained in simulated quantized mode so that weights adapt to the quantized computational pathway (Sassoon et al., 12 Sep 2025). The paper reports that PTQ can be performed with a single sample and about 1 second total time, while QAT requires hours to days and much more memory (Sassoon et al., 12 Sep 2025).

This PTQ/QAT duality matters because the paper’s broader claim is not merely that integer-only segmentation is possible, but that it can remain practical under severe calibration constraints. The strongest version of that claim appears in the one-shot PTQ setting, where I-Segmenter delivers competitive accuracy with a single calibration image (Sassoon et al., 12 Sep 2025). A plausible implication is that the framework is intended not only for carefully tuned offline deployment, but also for environments in which calibration data or engineering time is limited.

4. $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$0-ShiftGELU and activation quantization

The most distinctive operator-level contribution in I-Segmenter is $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$1-ShiftGELU. The paper motivates it by noting that standard GELU activations in transformers have long-tailed, sharply peaked distributions around zero, which are poorly matched to uniform quantization bins (Sassoon et al., 12 Sep 2025). Under low precision, this produces underrepresented small values, distorted outliers, and degraded stability in both PTQ and QAT (Sassoon et al., 12 Sep 2025).

The predecessor approximation, ShiftGELU from I-ViT, is described as too aggressive in clamping negative exponents to prevent overflow, which suppresses informative negative activations and creates a large functional gap relative to FP32 GELU (Sassoon et al., 12 Sep 2025). $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$2-ShiftGELU modifies that clamping rule by introducing a tunable scalar $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$3: $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$4 where $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$5 is the integer exponent, $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$6 is the intermediate bitwidth, and $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$7 with input scale $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$8 (Sassoon et al., 12 Sep 2025). The paper reports that $$I = \left\lfloor\frac{\text{clip}(F,-m,m)}{S}\right\rceil,\qquad S = \frac{2m}{2^k-1}$$9 is optionally set to 6 and describes this value as empirically optimal (Sassoon et al., 12 Sep 2025).

The quantitative role of this modification is explicit. Table 1 is summarized as showing up to $F$0 reduction in activation RMSE relative to baseline ShiftGELU, especially in larger models, and Figure 1 is described as showing a much closer match to FP32 GELU in both distribution and functional shape (Sassoon et al., 12 Sep 2025). The paper further states that $F$1-ShiftGELU is critical: without it, even QAT fails in large models (Sassoon et al., 12 Sep 2025).

This is the principal stabilization mechanism in the framework. While the rest of I-Segmenter establishes integer-only execution, $F$2-ShiftGELU is the component most directly aimed at making that execution numerically viable for ViT segmentation (Sassoon et al., 12 Sep 2025).

5. Decoder modifications and graph-level integer compatibility

I-Segmenter also modifies two decoder-level operations that are common in segmentation architectures but poorly suited to integer-only execution: L2 normalization and bilinear interpolation (Sassoon et al., 12 Sep 2025).

The original Segmenter applies L2 normalization before the inner product of projected patch and class embeddings. I-Segmenter removes that L2 normalization because it is expensive and nontrivial to quantize, involving square-root, inverse, and floating-point division (Sassoon et al., 12 Sep 2025). The replacement is direct normalization through symmetric quantization by range control and scaling after matrix multiplication in patch and class embedding projections (Sassoon et al., 12 Sep 2025).

The decoder’s bilinear upsampling is replaced with nearest-neighbor interpolation. The stated reason is that bilinear upsampling is FP32-based weighted interpolation, whereas nearest-neighbor interpolation can be implemented entirely with integer arithmetic and is widely supported in efficient inference engines (Sassoon et al., 12 Sep 2025). The ablation summary reports that removing L2 normalization causes a modest average loss of about $F$3 mIoU points, while replacing bilinear with nearest neighbor yields a more pronounced average loss of about $F$4 mIoU points; removing both leads to up to $F$5 mIoU points lost versus the FP32 best (Sassoon et al., 12 Sep 2025).

These changes illustrate an important property of the framework: integer-only execution is treated as a graph-level constraint rather than a local operator constraint. In other words, the paper is not satisfied with quantized attention and MLP blocks alone; it also removes or replaces small floating-point islands that would otherwise remain in the decoder (Sassoon et al., 12 Sep 2025).

6. Empirical performance, efficiency, and deployment characteristics

The paper reports that I-Segmenter achieves accuracy within a reasonable margin of its FP32 baseline, with an average gap of $F$6, while reducing model size by up to $F$7 and enabling up to $F$8 faster inference with optimized runtimes (Sassoon et al., 12 Sep 2025). On ADE20k and Cityscapes, PTQ with $F$9-ShiftGELU is summarized as retaining near-baseline accuracy with losses of at most about $m$0–$m$1 mIoU points, while QAT narrows the gap further, often to about $m$2–$m$3 mIoU (Sassoon et al., 12 Sep 2025).

The efficiency results extend beyond checkpoint size. The paper reports bit-level read/write reductions of $m$4–$m$5 in TVM, which it interprets as promising for energy and inference efficiency (Sassoon et al., 12 Sep 2025). Latency gains are said to scale with model size because quantize/dequantize overheads are amortized more effectively in larger models (Sassoon et al., 12 Sep 2025).

A noteworthy systems claim concerns backend fidelity. The paper states that only TVM enforces true integer-only computation to the kernel level, not merely for storage or operator stubs (Sassoon et al., 12 Sep 2025). This addresses a common misconception in quantization work: “integer-only” can refer either to parameter storage or to actual end-to-end execution. In I-Segmenter, the distinction is explicit, and the paper treats kernel-level integer execution as the stricter standard (Sassoon et al., 12 Sep 2025).

The one-shot PTQ result is also a central deployment claim. The paper states that calibration with a single sample uses less than 2 GB GPU memory and can achieve accuracy close to that of a full calibration set of $m$6 (Sassoon et al., 12 Sep 2025). This is presented as evidence of practicality for real-world deployment, especially when rapid adaptation is needed (Sassoon et al., 12 Sep 2025).

7. Terminological context and related uses of the name

Within the broader literature represented here, the name “I-Segmenter” is not unique. In “Iterative Utterance Segmentation for Neural Semantic Parsing,” I-Segmenter refers to a GRU-based span predictor used inside the PDE framework to select semantically meaningful spans during iterative semantic parsing (Guo et al., 2020). That module predicts start and end indices for a span and interacts with a parser that maps the span to a partial meaning representation (Guo et al., 2020). This is a different use of the term from the integer-only ViT segmentation framework of (Sassoon et al., 12 Sep 2025).

A second source of ambiguity is the distinction between semantic segmentation and interactive segmentation. “Does SAM dream of EIG? Characterizing Interactive Segmenter Performance using Expected Information Gain” studies interactive segmentation models through an EIG-based assessment procedure and argues that Oracle Dice can be insensitive or misleading for measuring prompt understanding (Chung et al., 2024). I-Segmenter, by contrast, is not an interactive segmentation model; it is a semantic segmentation framework designed around integer-only execution (Sassoon et al., 12 Sep 2025).

These distinctions are useful because the word “segmenter” spans several research traditions: semantic segmentation, interactive image segmentation, utterance segmentation for semantic parsing, and other segmentation problems. In the specific sense established by (Sassoon et al., 12 Sep 2025), I-Segmenter denotes an integer-only Vision Transformer for semantic segmentation, characterized by full integer execution, $m$7-ShiftGELU, removal of L2 normalization, replacement of bilinear interpolation with nearest-neighbor upsampling, and support for both one-shot PTQ and QAT (Sassoon et al., 12 Sep 2025).