CompactifAI: A Paradigm for Model Compression

• CompactifAI is a family of model compression strategies that reduce parameters, activations, token vocabularies, and intermediate features in AI systems.
• The COMPACT method exemplifies joint pruning by removing infrequent tokens and low-importance FFN channels, leading to significant memory and latency improvements in transformers.
• Other approaches under CompactifAI employ quantum-inspired tensor network factorization, low-rank activation sketches, and feature compactness for deployment efficiency and secure computation.

Searching arXiv for the cited CompactifAI-related papers to ground the article in the current literature. arxiv_search.query{"search_query":"id:(Kwek et al., 8 Sep 2025) OR id:(Tomut et al., 2024) OR id:(Shamshoum et al., 2024) OR id:(Islam et al., 2023) OR id:(Fovet et al., 7 Jul 2025)","start":0} arxiv_search.query{"search_query":"id:(Kwek et al., 8 Sep 2025)","start":0} arxiv_search.query{"search_query":"(Kwek et al., 8 Sep 2025)","start":0} arxiv_search.query{"search_query":"COMPACT Common-token Optimized Model Pruning Across Channels and Tokens","start":0} CompactifAI is an overloaded designation in the arXiv literature for methods that make AI systems more compact in parameters, activations, token vocabularies, secure-computation circuits, or transmitted intermediate representations. In the most direct usage, it denotes a quantum-inspired tensor-network compression framework for LLMs (Tomut et al., 2024). Closely related usages apply the label to COMPACT, a training-free pruning method for transformer vocabularies and FFN channels (Kwek et al., 8 Sep 2025), to compressed activations for memory-efficient LLM training (Shamshoum et al., 2024), to piece-wise polynomial activation approximations for secure MPC inference (Islam et al., 2023), and to compact feature-learning schemes in collaborative intelligence and medical imaging (Alvar et al., 2019, Dahes, 6 Dec 2025). The common theme is not a single canonical algorithm but a family of compaction strategies that target different bottlenecks in modern model deployment.

1. Terminological scope and recurring meanings

Across recent work, “CompactifAI” does not refer to a single standardized stack. Rather, it denotes several technically distinct programs of model and representation compression. One line uses the name for quantum-inspired tensor-network factorization of LLM weights (Tomut et al., 2024). Another uses it as an expository frame for COMPACT, which jointly prunes rare vocabulary entries and FFN intermediate channels while preserving the standard transformer layout (Kwek et al., 8 Sep 2025). Other usages include low-rank activation sketches for training (Shamshoum et al., 2024), piece-wise polynomial replacement of complex activation functions for MPC (Islam et al., 2023), compressibility-aware bottleneck features for collaborative intelligence (Alvar et al., 2019), and compactness-and-separability modules for mammography classification (Dahes, 6 Dec 2025).

Usage	Core mechanism	Target setting
COMPACT	rare-token pruning + common-token-weighted FFN channel pruning	transformer inference compression
CompactifAI	tensor-network factorization with quantization	LLM size, latency, and memory
CompAct	random low-rank activation sketches	memory-efficient LLM training
Compact	piece-wise polynomial AF approximation	secure MPC inference
Compressible features / GAGM+SEVector	compact feature geometry and transmission	collaborative intelligence, medical imaging

A common misconception is to treat CompactifAI as synonymous with parameter pruning alone. The literature is broader. Compactness may refer to fewer parameters, lower-rank activations, smaller vocabularies, cheaper nonlinear operators, or bottleneck features that are easier to encode and transmit. This suggests that “compactness” is being used operationally: whatever representation dominates memory, latency, communication, or secure-computation cost becomes the object of compression.

2. COMPACT: joint pruning across tokens and FFN channels

The 2025 COMPACT method formalizes transformer pruning around two parameter reservoirs: embeddings/unembeddings and FFNs. With vocabulary size $|V|$, hidden dimension $D$, layer count $L$, and FFN intermediate size $I$, the parameter counts are written as

$$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$

COMPACT assigns each token $t$ the importance score $s_v(t)=p(t)$, where $p(t)$ is its empirical frequency on a calibration corpus, and prunes the rarest $|V|-V'$ tokens. FFN channel importance is then computed under the post-pruning token distribution rather than under the full calibration distribution:

$$I_k = \sum_{i=1}^N w_i\cdot(\sigma(X_iW_1)\odot X_iW_2)_k^2,$$

with $D$0 if the corresponding token is pruned and $D$1 otherwise. The associated joint view is

$D$2

although the practical procedure simply fixes $D$3 and $D$4 and removes the least frequent tokens and lowest-scoring channels (Kwek et al., 8 Sep 2025).

Algorithmically, COMPACT consists of four steps: rare-token identification from empirical frequency, calibration forward passes that record $D$5, common-Act$D$6 channel scoring, and vocabulary/unembedding pruning. Removing a channel deletes the corresponding row in $D$7, the corresponding column in $D$8, and the corresponding column/row in the output and input projection matrices. Vocabulary pruning removes the final $D$9 rows of the embedding matrix $L$0 and the corresponding columns of the unembedding matrix $L$1; tokenizer merge rules for the pruned tokens are also deleted so that those tokens are never generated. Because all deletions align with existing matrix boundaries, no change to the transformer block or new inference kernels are required. The method is described as deployment-friendly, scale-adaptive, and training-free after a small calibration pass, and is reported as compatible with Huggingface, vLLM, and DeepSpeed (Kwek et al., 8 Sep 2025).

Empirically, COMPACT is evaluated on Qwen 2.5, LLaMA 3.2 and 3.1, and Gemma 3 families from 0.5B to 70B, over MMLU, HellaSwag, WinoGrande, ARC-C/E, PIQA, and GSM8K. At pruning ratios of approximately 10%, 20%, and 35%, it reduces parameters by up to 37% on small models and by 34–35% on larger ones; on LLaMA 8B at 35% prune, GPU memory for classification drops from 50 GB to 32 GB and throughput rises from 147 q/s to 201 q/s, while generation memory falls from 21.8 GB to 14.3 GB and throughput climbs from 81 q/s to 112 q/s. Pruning time is reported as 0:32 for LLaMA 8B and 2:17 for LLaMA 70B, with the text also stating that a 70B LLaMA 3.1 model at 35% ratio can be pruned in under three minutes on a single 80 GB A100 (Kwek et al., 8 Sep 2025).

3. Quantum-inspired tensor-network CompactifAI for LLM compression

The 2024 paper titled “CompactifAI: Extreme Compression of LLMs using Quantum-Inspired Tensor Networks” treats large weight matrices as tensors and compresses them in correlation space via Matrix Product Operator or Tensor-Train factorizations (Tomut et al., 2024). A weight matrix $L$2 is reshaped into a higher-order tensor and approximated by a chain of low-order cores with bond dimension $L$3. In the Tensor-Train notation used in a later evaluation paper,

$L$4

so that the parameter count becomes

$L$5

In the MPO presentation, the bond dimension $L$6 controls how many correlations are retained across bipartitions of the tensorized weight (Tomut et al., 2024, Fovet et al., 7 Jul 2025).

This framework explicitly contrasts “correlation space” truncation with neuron-count truncation. The bond spectrum from successive SVDs is used as a quantitative indicator of retained correlations, and the method is described as allowing a refined layer sensitivity profiling: candidate layers are compressed at different maximal bond dimensions, zero-shot or few-shot accuracy is measured, and layers that degrade sharply are marked sensitive. Reported findings on LlaMA-2 7B are that initial attention blocks are sensitive, middle to deeper blocks are robust, and the final MLP layer in each block is more sensitive than preceding sub-layers (Tomut et al., 2024).

The compression pipeline combines tensorization with “healing” retraining and quantization. The stated sequence is: select layers through sensitivity profiling, replace each selected weight by an MPO with chosen $L$7, run 1–2 epochs of healing on a downstream corpus, quantize remaining full layers, and optionally cast MPO cores to float-16. Reported benchmark results on LlaMA-2 7B give a reduction from 27.1 GB and 7 B parameters in float-32 to 4.1 GB and 2.1 B parameters for an 88% compressed float-16 MPO model, and to 2.1 GB and 2.1 B parameters for a 93% compressed mixed MPO+int-4 model. The same source reports 50% faster training, 25% faster inference, and a small 2%–3% accuracy drop (Tomut et al., 2024).

A subsequent study evaluates CompactifAI on Llama 3.1 8B using CodeCarbon and Ragas. That paper describes the method as combining Tensor-Train decomposition with 8-bit uniform quantization and magnitude pruning of TT-core entries below a threshold $L$8, and states that these steps “typically achieve $L$9 fewer parameters than the original 8 billion.” On 104 question-answer pairs, the compressed model reduces total runtime by 5.7% in a 200-token setting and by 18.2% in a 1000-token setting; total energy drops by 30.1% and 39.2%, respectively, and the linear fit for energy per token is $I$0 kWh/token versus $I$1 kWh/token. Accuracy metrics show higher ROUGE and BLEU, slightly improved semantic similarity and answer correctness, and small degradations in factual correctness and response relevancy (Fovet et al., 7 Jul 2025).

4. CompactifAI at training time and under secure computation

A different line of work uses “CompAct” for memory-efficient LLM training by compressing activations rather than parameters. The method stores a low-rank sketch of each activation tensor using random projections:

$I$2

with $I$3 in the simplified form, so that $I$4. Forward pass saves $I$5, where $I$6, and backward pass regenerates the projection from the seed and computes a compressed gradient in the low-rank space. The stated memory ratio is $I$7, implying roughly $I$8 savings in activation memory. Implementation details emphasize on-the-fly projection sampling from a 64-bit seed, fused GEMMs in forward and backward, and no extra GPU memory for storing projection matrices. Reported gains are 25–30% peak-memory savings for pretraining, approximately 50% for fine-tuning, and +2–5% compute overhead, with specific examples including 39.97 GB to 34.7 GB on a 350M LLaMA-family pretraining setup and 6.3 GB to 3.1 GB for RoBERTa-Base fine-tuning on GLUE (Shamshoum et al., 2024).

Under secure multi-party computation, “Compact” denotes piece-wise polynomial approximation of complex activation functions such as SiLU, GeLU, and Mish:

$I$9

The construction fixes $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$0, sets $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$1 for $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$2 and $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$3 for $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$4, and optimizes the cut points and polynomial degrees using an input-density-aware partitioning and a simulated-annealing-style search over $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$5. The text states that batch-normalization makes pre-activations approximately $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$6, which motivates finer subdivision near $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$7. Each evaluation needs $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$8 secure comparisons and $$N_{\text{vocab}} = 2|V|D,\qquad N_{\text{FFN}} = 3LDI.$$9 secure multiplications, and the approach introduces no new primitives beyond Add, Mul, and Compare. Experiments on MNIST, CIFAR-10, ImageNet-1K, and CelebA-Spoof report accuracy loss of at most 1% and speedups of about $t$0 to $t$1 relative to prior approaches on deeper networks (Islam et al., 2023).

These two lines show that compactification in the literature is not restricted to static inference-time model shrinkage. It also includes training-memory reduction and cryptographic circuit simplification, each targeting the dominant resource term in its own execution regime.

5. Compact feature representations in collaborative intelligence and medical imaging

In collaborative intelligence, compactness is formulated at the level of transmitted intermediate features rather than at the level of weights. A ResNet-34-based encoder maps an image $t$2 to a bottleneck feature tensor $t$3, which is quantized, entropy-coded, transmitted to the cloud, decoded, and consumed by multiple task heads for semantic segmentation, disparity estimation, and input reconstruction. The central regularizer is a compressibility loss

$t$4

where $t$5 is obtained by DPCM residual formation and a 2D DCT. The $t$6 norm of the DCT coefficients serves as a differentiable proxy for bit rate. On Cityscapes, the paper reports that the compression-friendly loss yields around 20% bitrate reduction without sacrificing performance, with BD-rate improvements such as –20.7% for segmentation under the shallower encoder variant (Alvar et al., 2019).

In mammography classification, compactness is interpreted geometrically through feature pooling, channel attention, and an explicit intra-class compactness penalty. The validated modules are Global Average and Max Pooling fusion (GAGM), which concatenates GAP and GMP outputs into a $t$7-dimensional vector, and SEVector attention,

$t$8

The corresponding Feature Smoothing Loss is

$t$9

combined with cross-entropy as $s_v(t)=p(t)$0. On a Kaggle dataset consolidating INbreast, MIAS, and DDSM, enhanced ConvNeXt Tiny and InceptionV3 backbones with GAGM and SEVector improve macro F1 and malignant false-negative rate, while FSL is reported not to yield measurable improvements under mammography classification conditions. The paper attributes the failure of FSL to class imbalance, small batch sizes, subtle textural differences, and computational overhead without gain (Dahes, 6 Dec 2025).

A plausible implication is that compactness in feature space is highly task-dependent. In collaborative intelligence it is evaluated by compressibility under codecs; in mammography it is evaluated by intra-class compactness, inter-class separability, Grad-CAM localization behavior, and false-negative reduction.

6. Search-based compression and deployment context

CompactifAI-related work sits within a broader ecosystem of automated compact-model design. ASCAI formulates per-layer compression hyperparameters as a vector $s_v(t)=p(t)$1 and uses adaptive sampling with genetic-operator-inspired selection, crossover, and mutation to optimize the FLOPs–accuracy trade-off. Its score is based on $s_v(t)=p(t)$2 divided by an accuracy-drop penalty with an exponential barrier below an accuracy threshold, and the paper reports that ASCAI outperforms rule-based and reinforcement-learning methods on CIFAR-10 and ImageNet benchmarks (Javaheripi et al., 2019). GeneCAI extends this search view with a bijective genotype–phenotype encoding of compression configurations, Pareto-oriented optimization, diversity-guided mutation, and near-linear multi-GPU scaling; it is reported to find models on a better accuracy-complexity Pareto curve than heuristic and RL baselines (Javaheripi et al., 2020).

The deployment-oriented rationale for compactification is reinforced by domain toolboxes such as EfficientBioAI, which combines pruning, int8 post-training quantization, and optional distillation in a plug-and-play workflow built around NNI, MQBench, OpenVINO, and TensorRT. Across four bioimage analysis tasks, that toolbox reports around $s_v(t)=p(t)$3–$s_v(t)=p(t)$4 inference speed-up and 30–80% energy savings, with several cases showing slight accuracy improvement after compression (Zhou et al., 2023). This broader context clarifies why CompactifAI-style methods repeatedly emphasize calibration-only procedures, structured deletions aligned with existing matrix boundaries, compatibility with existing runtimes, and direct measurement of latency, memory, energy, and accuracy.

Taken together, the literature presents CompactifAI less as a single method than as a compact-model design paradigm spanning inference compression, training-memory reduction, secure inference, communication-efficient feature transport, and compact feature geometry. The unifying principle is to identify the dominant computational or representational bottleneck—parameters, tokens, activations, nonlinearities, or transmitted features—and compress that object with minimal disruption to downstream utility.