OneComp: Open-Source Compression Framework

• OneComp is an open-source post-training compression framework that automates model inspection, mixed-precision planning, and progressive quantization for large Transformer models.
• The framework employs hardware-aware workflows and calibration strategies to optimize quantization, reducing memory footprint while preserving model accuracy.
• Its deployable pivot strategy and AutoBit mixed-precision planner enable staged refinements of a quantized model, facilitating practical deployment on commodity accelerators.

OneComp is an open-source, end-to-end post-training compression framework for foundation models, especially large Transformer-based generative models. It is defined not as a single quantization algorithm but as a unified, resource-adaptive pipeline that automates model inspection, mixed-precision planning, calibration construction, optional equivalent transformations, progressive quantization, and deployment-oriented checkpoint export. Its central architectural choice is to treat the first quantized checkpoint as a deployable pivot, so that later stages refine the same compressed model rather than replacing it with an incompatible artifact (Ichikawa et al., 30 Mar 2026).

1. Deployment setting and problem formulation

OneComp is situated in the systems problem of deploying foundation models under constraints imposed by memory footprint, memory bandwidth, latency, and hardware cost. The motivating observation is that large models frequently cannot fit on commodity accelerators at FP16 or BF16 precision, while inference for LLMs is often bandwidth-bound rather than compute-bound. In that setting, weight-only post-training quantization is attractive because it reduces model size after training without requiring access to the original training pipeline or full retraining data (Ichikawa et al., 30 Mar 2026).

The framework is motivated by the fragmentation of practical post-training quantization. The source landscape includes heterogeneous quantization families such as GPTQ, AWQ, rotation-based methods, block-wise methods, and structured low-bit formats; heterogeneous precision choices across layers and modules; calibration strategies whose quality can dominate final performance; and hardware-dependent execution regimes that determine whether a compressed representation is actually deployable. OneComp reframes that fragmentation as a pipeline orchestration problem. In the paper’s formulation, compression quality depends not only on the local quantizer but also on how precision is assigned, how calibration data are sampled, what equivalent transforms are applied before quantization, and how later refinement stages reuse the first quantized model (Ichikawa et al., 30 Mar 2026).

The scale of the deployment problem is illustrated numerically in the paper: a 70B model at 16 bits requires about 140 GB for weights, versus about 35 GB at 4 bits and about 18 GB at 2 bits. This makes precision reduction a feasibility condition for single-device deployment as much as an optimization for throughput. A plausible implication is that OneComp’s target use case is not merely research comparison among quantizers, but operational conversion of pretrained models into hardware-constrained deployment artifacts (Ichikawa et al., 30 Mar 2026).

2. Pipeline architecture and hardware-aware workflow

OneComp organizes compression as a staged workflow beginning with model inspection. The framework loads a pretrained model and identifies the model family or architecture type, the quantizable linear layers, the processing order through the model, and structural groupings such as Transformer blocks and submodules. The paper states that it can automatically recognize dense LLMs, mixture-of-experts models, and vision-LLMs, although the reported experiments focus on decoder-only LLMs (Ichikawa et al., 30 Mar 2026).

A defining feature is hardware awareness. Quantization scope scales with available GPU memory: layer-wise post-training quantization processes one linear layer at a time; block-wise post-training quantization processes one Transformer block at a time; and global post-training quantization treats the full model jointly. Hardware budget therefore determines not only whether the model can be compressed, but also how large an optimization scope the framework can afford during compression (Ichikawa et al., 30 Mar 2026).

Calibration data are treated as a first-class systems component. OneComp constructs a calibration set $\mathcal{C} = \{x_1, \ldots, x_n\}$ either from user-provided data or automatically from a corpus. Its default construction is concat_chunk: concatenate text samples, tokenize the combined text, and slice fixed-length chunks of $T$ tokens. Alternative strategies are drop_head and drop_rand. These calibration samples provide layer inputs $X_l$, Gram matrices $H_l = X_l^\top X_l$, and teacher outputs for block-wise and global distillation (Ichikawa et al., 30 Mar 2026).

Before quantization, the framework can apply equivalent transformations intended to reduce outlier concentration and improve quantizability. The paper includes SmoothQuant-like channel-wise scaling,

$$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$

with scaling factors

$$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$

and rotation-based preprocessing,

$$Y = XW = \underbrace{(XR)}_{\widehat{X}} \cdot \underbrace{(R^\top W)}_{\widehat{W}},$$

where $R^\top R = I$. Supported preprocessors include SpinQuant-style learned rotations using CayleySGD, QuIP-style random per-layer rotations in the layer-wise path, L1/L2 Sinkhorn weight balancing, and channel equalization (Ichikawa et al., 30 Mar 2026).

Workflow component	Function	Representative mechanisms
Model inspection	Derive compression plan from architecture	layer discovery, block grouping
Calibration construction	Build statistics and teacher signals	concat_chunk, drop_head, drop_rand
Preprocessing	Improve quantizability without changing function	scaling, rotations, Sinkhorn, equalization
Progressive compression	Increase optimization scope with more memory	layer-wise, block-wise, global

3. AutoBit mixed-precision planning

OneComp’s mixed-precision planner, AutoBit, assigns heterogeneous quantization configurations across layers or modules under a global resource budget. The optimization problem is

$$\min_{\{c_l \in \mathcal{C}_l\}} \sum_{l=1}^{L} \mathrm{err}(l, c_l) \quad \text{s.t.} \quad \sum_{l=1}^{L} \mathrm{cost}(l, c_l) \le C^*,$$

where $l$ indexes a layer or module, $T$0 is its candidate configuration set, each candidate specifies bit-width, numeric format, and group size, $T$1 is memory cost, and $T$2 is predicted quantization error. The planner is formulated as an ILP and solved using SCIP in OR-Tools (Ichikawa et al., 30 Mar 2026).

The key technical point is that error prediction is activation-aware. A naive proxy would use the Frobenius norm of the weight perturbation,

$T$3

but OneComp instead uses a second-order approximation,

$T$4

where $T$5 is the perturbation from a fast RTN proxy, $T$6 is the input Gram matrix, and $T$7 is output curvature. In practice only diagonal elements of $T$8 and $T$9 are used. This makes the planner sensitivity-aware rather than purely size-aware (Ichikawa et al., 30 Mar 2026).

The paper’s experiments instantiate candidate bit-widths $X_l$0 and group sizes $X_l$1. The resulting mixed-precision plan can therefore vary not only in nominal bit-width but also in grouping granularity and numeric format. This suggests that OneComp treats precision allocation as a constrained optimization layer above the individual quantizer, rather than as a fixed hyperparameter supplied externally (Ichikawa et al., 30 Mar 2026).

4. Progressive quantization stages and the deployable pivot

The most distinctive design feature of OneComp is its progressive refinement architecture. The output of the first stage is already a complete quantized checkpoint; later stages refine that checkpoint rather than generating a separate incompatible representation. The paper describes this as a monotonic compute-quality tradeoff: minimal resources produce a deployable model quickly, while additional compute improves quality on the same compressed artifact (Ichikawa et al., 30 Mar 2026).

The first stage is layer-wise compression. For a layer $X_l$2, the base reconstruction objective is

$X_l$3

where $X_l$4 is the full-precision weight, $X_l$5 is the perturbed input activation matrix from the partially quantized upstream model, and $X_l$6 is the allowed quantized format. This stage is memory efficient because only one layer and its Hessian-like statistic need to be materialized. Integrated base quantizers include GPTQ, AWQ, and RTN; OneComp then augments layer-wise PTQ with QEP for error-propagation correction, LPCD for submodule-aware joint optimization, JointQ for improved 3–4 bit GPTQ-format checkpoints, and DBF or MDBF for 1–2 bit structured compression (Ichikawa et al., 30 Mar 2026).

QEP addresses the mismatch between original activations and perturbed activations after upstream quantization. Instead of quantizing $X_l$7 against the original input, it solves

$X_l$8

With activation error $X_l$9 and propagation matrix $H_l = X_l^\top X_l$0, the corrected target is

$H_l = X_l^\top X_l$1

where $H_l = X_l^\top X_l$2 controls correction strength and $H_l = X_l^\top X_l$3 is Tikhonov regularization. Any base quantizer can then quantize $H_l = X_l^\top X_l$4 instead of $H_l = X_l^\top X_l$5 (Ichikawa et al., 30 Mar 2026).

LPCD extends the scope from a single matrix to interacting submodules. It solves a joint problem

$H_l = X_l^\top X_l$6

by cyclic relax-then-project updates. OneComp applies this to the QK module $H_l = X_l^\top X_l$7, the VO module $H_l = X_l^\top X_l$8, and the gate-up-down module $H_l = X_l^\top X_l$9. The paper reports typical convergence in 2–4 outer iterations (Ichikawa et al., 30 Mar 2026).

Block-wise refinement follows once a full quantized model exists. One form, block-wise compensation, updates the remaining FP16 parameters in a partially quantized block to absorb earlier quantization errors. A second form optimizes all quantized parameters of a Transformer block to match full-precision teacher outputs, optionally adding cosine alignment. OneComp then performs cross-block refinement on adjacent block pairs with a sliding window $$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$0 and rolls back if refinement degrades results. To optimize discrete quantized parameters, it uses a Smooth STE surrogate,

$$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$1

and Progressive STE Temperature Annealing,

$$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$2

with $$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$3 and $$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$4. Training uses Adam, cosine learning-rate decay, 10% warmup, gradient clipping at 1.0, learning rate $$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$5 for scales and zero-points, and learning rate $$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$6 for integer weights (Ichikawa et al., 30 Mar 2026).

Global refinement is described architecturally but marked as planned or under development. Its design treats full-model compression as teacher-student distillation combining token-level KL distillation, intermediate hidden-state alignment, and entropy regularization. The planned optimizer includes SAM,

$$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$7

with

$$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$8

and progressive layer unfreezing beginning near output layers. OneComp can additionally add QLoRA-style low-rank adapters,

$$Y = XW = \underbrace{(X\mathrm{diag}(\bm{s}))}_{\widetilde{X}} \cdot \underbrace{(\mathrm{diag}(\bm{s})^{-1} W)}_{\widetilde{W}},$$9

for domain adaptation or residual quantization-error recovery (Ichikawa et al., 30 Mar 2026).

5. Quantization formats, integrated methods, and empirical behavior

OneComp supports a broad span of formats. For conventional uniform quantization it uses

$$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$0

with symmetric or asymmetric quantization and per-group or per-channel configurations. It also produces GPTQ-compatible packed checkpoints, which is important because the paper treats deployability in existing low-precision inference kernels as a first-class engineering constraint (Ichikawa et al., 30 Mar 2026).

JointQ is OneComp’s dedicated method for 3–4 bit regimes. It keeps the GPTQ packed representation while jointly optimizing integer assignments and scales:

$$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$1

An optional proximity-regularized version adds

$$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$2

The paper states that this is solved by local search: propose small integer code changes, update the affected group scale jointly, accept improving moves, and stop at a local optimum. The practical significance is that JointQ improves quality while still emitting a standard GPTQ-format checkpoint (Ichikawa et al., 30 Mar 2026).

At the extreme low-bit end, OneComp includes DBF, or Double Binary Factorization,

$$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$3

where $$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$4 are binary sign matrices and the $$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$5 terms are diagonal scaling matrices. MDBF, or Multi-Envelope DBF, addresses DBF’s rank-one envelope bottleneck through

$$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$6

with envelope rank $$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$7. The paper states that MDBF support is planned for a future release, but it nevertheless reports evaluation numbers for the three-stage pipeline (Ichikawa et al., 30 Mar 2026).

The main evaluation uses Llama-3-8B, Qwen3-8B, and Qwen3-14B, with extreme low-bit experiments also covering Qwen3-0.6B, Llama-3.2-1B, TinyLlama-1.1B, Llama-2-7B, and Llama-3-8B. Calibration uses samples from C4 with maximum sequence length 2048; extreme low-bit experiments use 512 samples from WikiText-2 with the same length. Evaluation relies on WikiText-2 perplexity and zero-shot accuracy from lm-eval-harness on ARC-Challenge, ARC-Easy, PIQA, and WinoGrande (Ichikawa et al., 30 Mar 2026).

Representative results show that the planner and the progressive stages materially change outcomes.

Setting	Baseline	OneComp result
Llama-3-8B, 4.00 bpw mixed precision	Uniform: PPL 383.26, ACC 0.595	Activation-aware: PPL 7.04, ACC 0.701
Llama-3-8B, 4b/ch layer-wise PTQ	Base GPTQ: 665.94 / 0.533	QEP: 7.67 / 0.688; Submodule: 7.33 / 0.703
Llama-3-8B, 4b/128 packed format	GPTQ: 12.66 / 0.697	JointQ: 6.67 / 0.705
Llama-3-8B, 1.50 bpw extreme low-bit	DBF: 10.61 / 0.560	MDBF: 10.03 / 0.572

These numbers support several empirical claims made in the paper. First, activation-aware AutoBit can preserve near-full-precision behavior under budgets where uniform assignment collapses. Second, naive layer-wise GPTQ can fail badly in channel-wise and lower-bit regimes, whereas QEP and LPCD-style submodule correction largely recover quality. Third, JointQ improves 3–4 bit GPTQ-format checkpoints without sacrificing compatibility with standard packed inference formats. Fourth, MDBF consistently outperforms DBF on all five tested models at both 1.00 and 1.50 bpw, which the paper attributes to more expressive magnitude modeling (Ichikawa et al., 30 Mar 2026).

The framework’s progressive thesis is also directly evaluated. On Llama-3.2-1B under DBF at 1.0, 1.5, and 2.0 bpw, the paper reports a monotonic ordering: layer-wise PTQ $$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$8 block-wise PTQ $$s_j = \frac{\max(|X_j|)^\alpha}{\max(|W_j|)^{1-\alpha}},$$9 global PTQ, with gains at every bit-width and larger gains at higher bit-widths. This suggests that the “deployable pivot” is not merely a software abstraction but a quantitatively meaningful refinement path (Ichikawa et al., 30 Mar 2026).

6. Software design, limitations, and nomenclature

OneComp is presented explicitly as open-source software rather than as a research-only prototype, with repository linkage to https://github.com/FujitsuResearch/OneCompression. The software interface is two-level: a default mode that is automatic and opinionated, and an expert mode that allows manual control over preprocessing, calibration, optimization regime, and refinement strategy. Later-stage improvements are implemented through a “refiner” abstraction whose input is a quantized model and whose output is an improved quantized model. This design supports reproducibility, modularity, and the addition of new post-quantization methods as composable passes (Ichikawa et al., 30 Mar 2026).

The framework also has explicit boundaries. Global PTQ is described architecturally but planned for future release. MDBF support is likewise planned for future release. OneComp currently focuses on weight-only post-training quantization; activation quantization and KV-cache compression are outside current scope, though listed as next directions. Mixed-bit support below a threshold like 2 bits is not yet supported in current AutoBit. The experiments emphasize quality metrics such as perplexity and zero-shot accuracy more than full systems benchmarking of latency, throughput, or deployment cost. Future directions listed in the paper include activation quantization, KV-cache compression, knowledge distillation, structured pruning, a broader compression platform beyond quantization, diversity-aware calibration sampling, and expanded hardware or backend support (Ichikawa et al., 30 Mar 2026).

In nomenclature, the exact name OneComp denotes the 2026 post-training compression framework. It should be distinguished from COMP, a separate method introduced as a lightweight post-training structured pruning technique for on-device LLMs; that paper explicitly states that the term “OneComp” does not appear anywhere in its text (Xu et al., 25 Jan 2025).