Input Compression in Machine Learning

• Input Compression is the process of reducing an input’s size—via token truncation, quantization, or pruning—to decrease computational cost while preserving essential, task-critical information.
• It employs varied methods across modalities, such as language prompt shortening, visual-token pruning, and spectral compression in speech, to trade off between resource savings and output quality.
• Its practical impact is measured by changes in API cost, FLOPs reduction, and accuracy, highlighting that benefits depend on preserving structure critical for downstream tasks.

Input compression is the reduction of the representation presented to a model before or during processing, typically by shortening a token sequence, lowering spatial or spectral resolution, pruning visual tokens, collapsing redundant temporal states, or quantizing features. Recent work treats it as a distinct optimization axis rather than a mere preprocessing trick: in LLMs it is evaluated against realized per-item API cost and semantic fidelity (Adeyemi et al., 23 Jun 2026); in multimodal systems it is used to reduce FLOPs, prefilling latency, inference time, and token counts (Xiao et al., 15 Sep 2025, Peng et al., 4 Nov 2025); in speech, source separation, and edge inference it is used to trade sequence length, memory, and bandwidth against downstream quality (Gaido et al., 2021, Saijo et al., 9 Feb 2026, Schrödter et al., 5 Mar 2026). The literature shows that input compression is not intrinsically beneficial: its effect depends on what is compressed, how the compressed representation preserves task-critical structure, and which deployment metric is actually measured (Adeyemi et al., 23 Jun 2026, Johnson, 6 Mar 2026).

1. Formalizations and problem structure

A common formalization views input compression as an operator that maps an original input sequence to a shorter one. In ICPC, an input $x=(x_1,x_2,\dots,x_n)$ is mapped by a compression operator $C_l(\cdot)$ at compression level $l$ to a compressed sequence $x' = C_l(x)$ with $m = |x'| < n$, and the compression ratio is $r = m/n$. In Transformer settings, this changes self-attention cost from $O(n^2)$ to $O(m^2)=O(r^2 n^2)$ (Nagarajan et al., 2023).

In prompt compression for LLMs, CAVEWOMAN defines a user question as a token sequence $x = (w_1,\dots,w_n)\in V^n$, with deterministic POS tags $g(w_i)$ from spaCy, and introduces reduction functions $C_l(\cdot)$0. These range from the identity $C_l(\cdot)$1, to deletion of function-word categories, to noun/verb filtering, to noun-only truncation with $C_l(\cdot)$2. Under Condition A, the model receives $C_l(\cdot)$3 in place of $C_l(\cdot)$4 under a neutral system prompt (Adeyemi et al., 23 Jun 2026).

Other domains use different state spaces but analogous compression ratios. UniPruneBench defines visual-token retention $C_l(\cdot)$5 and pruning ratio $C_l(\cdot)$6 for multimodal LLMs (Peng et al., 4 Nov 2025). Spectral Feature Compression (SFC) compresses a frequency axis $C_l(\cdot)$7 to $C_l(\cdot)$8 bands, giving compression ratio $C_l(\cdot)$9, for example $l$0 (Saijo et al., 9 Feb 2026). STAR defines the compression ratio for streaming sequence transduction as $l$1, where $l$2 is the number of low-level frames and $l$3 the number of anchor representations (Tan et al., 2024). Trainable Bitwise Soft Quantization compresses each $l$4-bit feature to a user-defined $l$5-bit code, with compression approximately $l$6 (Schrödter et al., 5 Mar 2026).

These formulations differ in modality and objective, but they share a structural pattern: an original representation is mapped to a smaller one before the main compute bottleneck. This suggests that input compression is best understood as a front-end budget-allocation mechanism rather than a single algorithmic family.

2. Prompt-side input compression in LLMs

CAVEWOMAN makes the strongest recent case that prompt shortening and cost reduction should be treated as separate questions. Its two-channel protocol evaluates the same items under Condition A, where the prompt is filtered via $l$7 and the output is unconstrained, and Condition B, where the prompt is unfiltered but the response is constrained to a matching register. Every generation is scored on task accuracy, realized per-item cost

$l$8

and reference-text agreement using bidirectional NLI against the model’s own $l$9 generation, plus $x' = C_l(x)$0 complementary measures (Adeyemi et al., 23 Jun 2026).

The central empirical finding is that input compression is a strict lose-lose under realistic API pricing. Across all eight models and five benchmarks, input compression raises net API cost by approximately $x' = C_l(x)$1 at $x' = C_l(x)$2, reaches up to $x' = C_l(x)$3 on the worst per-dataset cell, and grows to approximately $x' = C_l(x)$4 under stronger compression as accuracy collapses. The mechanism is compensatory verbosity: input token savings are small, while output tokens increase, and $x' = C_l(x)$5, so output expansion dominates the bill (Adeyemi et al., 23 Jun 2026).

At $x' = C_l(x)$6, the mean relative cost changes over the four API models make this asymmetry explicit. GPT-4o saves $x' = C_l(x)$7 input tokens, adds $x' = C_l(x)$8 output tokens, and changes cost by $x' = C_l(x)$9; GPT-5.4 saves $m = |x'| < n$0 input tokens, adds $m = |x'| < n$1 output tokens, and increases cost by $m = |x'| < n$2; Claude Haiku 4.5 saves $m = |x'| < n$3 input tokens, adds $m = |x'| < n$4 output tokens, and increases cost by $m = |x'| < n$5; Claude Sonnet 4.6 saves $m = |x'| < n$6 input tokens, adds $m = |x'| < n$7 output tokens, and increases cost by $m = |x'| < n$8 (Adeyemi et al., 23 Jun 2026).

Quality degrades in parallel. Task accuracy and same-channel entailment both worsen monotonically with compression level $m = |x'| < n$9. Wilcoxon tests against $r = m/n$0 show significant accuracy drops already at $r = m/n$1 in $r = m/n$2 $r = m/n$3 cells with $r = m/n$4, and mean bidirectional NLI in Condition A at $r = m/n$5 falls to approximately $r = m/n$6 from $r = m/n$7. For input compression specifically, there is no dissociation phenomenon: correct answers almost always still entail the $r = m/n$8 reference (Adeyemi et al., 23 Jun 2026).

A related prompt-compression study refines the structural explanation. “Compression Method Matters” introduces instruction survival probability

$r = m/n$9

where $O(n^2)$0 indicates whether a task-critical segment survives first-$O(n^2)$1-words truncation. Under $O(n^2)$2, DeepSeek shows a $O(n^2)$3 output expansion on MBPP with $O(n^2)$4, but only $O(n^2)$5 on HumanEval with $O(n^2)$6; GPT-4o-mini is comparatively stable across benchmarks. The same work introduces the Compression Robustness Index (CRI) and argues that prompt structure, not provider identity alone, is the primary moderator of output explosion (Johnson, 6 Mar 2026). This suggests that destructive prompt compression is less about compression per se than about whether task-defining segments survive.

A contrasting result comes from AdaCoder, which compresses long programmatic VQA preprompts by preserving API scaffolding and specializing code snippets by question type. In ViperGPT, the preprompt length falls from $O(n^2)$7 tokens to $O(n^2)$8 tokens, a $O(n^2)$9 reduction, while answer accuracy improves on GQA, VQAv2, and NLVR2. The ablations show that removing question-type specialization collapses accuracy to $O(m^2)=O(r^2 n^2)$0, whereas oracle types reach $O(m^2)=O(r^2 n^2)$1 (Ukai et al., 2024). The plausible implication is that prompt compression can succeed when it is structure-aware and task-conditional rather than telegraphic.

3. Vision and multimodal input compression

In vision and multimodal systems, input compression is usually motivated by prefilling cost, quadratic attention, or latency inside large image-conditioned backbones. The methods differ sharply in whether they are training-free heuristics, architecture-preserving rescalings, or task-specific token selectors.

Setting	Compression mechanism	Reported effect
Programmatic VQA	Type-conditioned compressed preprompts in AdaCoder	$O(m^2)=O(r^2 n^2)$2 tokens; $O(m^2)=O(r^2 n^2)$3 reduction (Ukai et al., 2024)
UI code generation	ELTC + RTR in EfficientUICoder	$O(m^2)=O(r^2 n^2)$4 compression ratio; FLOPs $O(m^2)=O(r^2 n^2)$5; prefill $O(m^2)=O(r^2 n^2)$6; inference $O(m^2)=O(r^2 n^2)$7 (Xiao et al., 15 Sep 2025)
Image editing	HiLo-Token high/low-frequency token allocation	DiT speedups $O(m^2)=O(r^2 n^2)$8, $O(m^2)=O(r^2 n^2)$9, $x = (w_1,\dots,w_n)\in V^n$0 for small, medium, large masks (You et al., 11 Jun 2026)
Visual-token pruning benchmark	UniPruneBench standardized retention $x = (w_1,\dots,w_n)\in V^n$1	Random pruning is strong; OCR is most vulnerable; pruning ratio dominates degradation (Peng et al., 4 Nov 2025)
Vision backbones	Input resolution downsizing	ViT-S $x = (w_1,\dots,w_n)\in V^n$2: Top-1 $x = (w_1,\dots,w_n)\in V^n$3, FLOPs $x = (w_1,\dots,w_n)\in V^n$4 GFLOPs (Morlier et al., 1 Apr 2025)

EfficientUICoder is representative of task-structured multimodal compression. Its Element and Layout-aware Token Compression detects UI elements, builds a graph over bounding boxes with edge weights $x = (w_1,\dots,w_n)\in V^n$5 defined by minimum Euclidean perimeter distance, extracts a minimum spanning tree, and retains patch tokens whose receptive fields overlap the selected regions. Region-aware Token Refinement then swaps low-attention retained tokens for high-attention discarded ones using the final CLIP vision-layer attention. Under $x = (w_1,\dots,w_n)\in V^n$6B-level MLLMs, this produces a $x = (w_1,\dots,w_n)\in V^n$7 compression ratio without compromising webpage quality, reduces generated tokens by $x = (w_1,\dots,w_n)\in V^n$8, and lowers redundant-sample count from $x = (w_1,\dots,w_n)\in V^n$9 to $g(w_i)$0 (Xiao et al., 15 Sep 2025).

HiLo-Token addresses a different regime: DiT-based image editing under user masks. It retains all latent tokens inside a dilated mask, selects additional high-frequency tokens outside the mask using Sobel-derived spatial frequency, and represents the rest with tokens from a $g(w_i)$1 downsampled image. On A100-80GB, it reduces the DiT token count by up to $g(w_i)$2 for small masks and yields end-to-end inference speedups of $g(w_i)$3, $g(w_i)$4, and $g(w_i)$5 across small, medium, and large mask categories, with no measurable regression in key perceptual or structural-fidelity metrics (You et al., 11 Jun 2026).

A more architecture-agnostic approach is input resolution downsizing. For convnets, per-layer FLOPs scale as $g(w_i)$6, so uniform downsampling by factor $g(w_i)$7 gives an $g(w_i)$8 reduction; for ViTs, reducing token-grid resolution by $g(w_i)$9 reduces FLOPs roughly by $C_l(\cdot)$00 and memory by $C_l(\cdot)$01. On ImageNet with ViT-S, moving from $C_l(\cdot)$02 to $C_l(\cdot)$03 changes Top-1 from $C_l(\cdot)$04 to $C_l(\cdot)$05 while lowering FLOPs from $C_l(\cdot)$06 to $C_l(\cdot)$07 GFLOPs and activation memory from $C_l(\cdot)$08 to $C_l(\cdot)$09 MB (Morlier et al., 1 Apr 2025).

UniPruneBench systematizes evaluation of visual-token compression across ten algorithms, ten datasets, and three LMM families. Its main findings are that random pruning is a surprisingly strong baseline, no single method consistently dominates, OCR tasks degrade drastically under aggressive pruning, and the pruning ratio is the dominant factor governing performance degradation. On InternVL3-8B at $C_l(\cdot)$10, DivPrune reduces total time from $C_l(\cdot)$11 to $C_l(\cdot)$12 and prefill time from $C_l(\cdot)$13 to $C_l(\cdot)$14, while method time remains below $C_l(\cdot)$15 for all methods (Peng et al., 4 Nov 2025).

4. Speech, audio, vector, and edge settings

In speech and audio, input compression often arises as sequence shortening inside encoder-decoder pipelines. CTC-based compression for direct speech translation branches from encoder layer $C_l(\cdot)$16, predicts phone or transcript labels, groups consecutive frames with the same label, and replaces each group with a single vector using AVG, WEIGHTED, or SOFTMAX collapse. On MuST-C English$C_l(\cdot)$17Italian, adding phone-informed CTC compression with AVG at layer $C_l(\cdot)$18 improves BLEU from $C_l(\cdot)$19 to $C_l(\cdot)$20 and reduces RAM from $C_l(\cdot)$21 MB to $C_l(\cdot)$22 MB; on English$C_l(\cdot)$23German, BLEU rises from $C_l(\cdot)$24 to $C_l(\cdot)$25 while RAM drops from $C_l(\cdot)$26 MB to $C_l(\cdot)$27 MB (Gaido et al., 2021).

STAR compresses streaming inputs by dynamically segmenting a frame sequence with a sigmoid read-score $C_l(\cdot)$28, emitting an anchor whenever the running accumulator reaches $C_l(\cdot)$29, and using the last hidden state of the segment as its anchor representation. In ASR, STAR reports nearly lossless $C_l(\cdot)$30 compression, with WER changing from $C_l(\cdot)$31 to $C_l(\cdot)$32, and shows $C_l(\cdot)$33 memory savings already at $C_l(\cdot)$34 for long inputs. In simultaneous ASR and EN$C_l(\cdot)$35DE speech translation it obtains more favorable latency-quality trade-offs than CIF (Tan et al., 2024).

Spectral Feature Compression moves from temporal compression to frequency compression. Instead of the fixed band-split (BS) encoder, which uses $C_l(\cdot)$36 separate subband modules, SFC uses a single sequence-modeling block to compress $C_l(\cdot)$37 and symmetrically decode $C_l(\cdot)$38. The cross-attention variant, SFC-CA, injects BS-inspired positional bias while remaining input-adaptive. On MUSDB18-HQ with a small separator, BS reaches $C_l(\cdot)$39 dB and $C_l(\cdot)$40 dB, whereas SFC-CA reaches $C_l(\cdot)$41 and $C_l(\cdot)$42; the encoder-decoder parameter count drops from about $C_l(\cdot)$43 M for BS to about $C_l(\cdot)$44 M for SFC-CA (Saijo et al., 9 Feb 2026).

At the client-server boundary, Trainable Bitwise Soft Quantization compresses each scalar input feature with $C_l(\cdot)$45 thresholded bits using differentiable sigmoids during training and hard bit-packing at inference. Across six regression datasets, it reports compression factors of $C_l(\cdot)$46 to $C_l(\cdot)$47 relative to $C_l(\cdot)$48-bit input without significant performance loss, with a mean compression factor of $C_l(\cdot)$49 while maintaining statistical parity with full precision (Schrödter et al., 5 Mar 2026).

RQ-MoE addresses input-dependent vector compression rather than raw sensory input. It combines a hyper-dimensional codebook, a second-level MoE that deforms base centroids into an input-specific codebook, and dual-stream quantization that decouples instruction from quantization. It matches or exceeds QINCo in reconstruction and Recall@1 while providing $C_l(\cdot)$50 to $C_l(\cdot)$51 faster decoding; for example, at $C_l(\cdot)$52 bytes on BigANN1M it improves Recall@1 from $C_l(\cdot)$53 to $C_l(\cdot)$54 and MSE from $C_l(\cdot)$55 to $C_l(\cdot)$56 (Zhong et al., 14 May 2026).

5. Evaluation criteria, robustness, and failure modes

A central lesson of recent work is that token reduction alone is an inadequate evaluation target. CAVEWOMAN therefore measures realized cost, not nominal prompt length, and compares compressed outputs to the model’s own unconstrained baseline with bidirectional NLI (Adeyemi et al., 23 Jun 2026). “Compression Method Matters” adds instruction survival probability $C_l(\cdot)$57, CRI, Welch’s $C_l(\cdot)$58, bootstrap confidence intervals, Tobit regression for censoring, and companion NVML measurements to show that token savings can overstate joule savings (Johnson, 6 Mar 2026). UniPruneBench adds total time, prefill time, and compression-method overhead to task accuracy (Peng et al., 4 Nov 2025).

These richer metrics expose recurrent failure modes. In prompt compression, stripping function words can preserve syntactic remnants while deleting task-critical segments, causing output expansion and accuracy collapse (Adeyemi et al., 23 Jun 2026, Johnson, 6 Mar 2026). In visual-token pruning, OCR is much more fragile than general VQA, and no compression method dominates across benchmarks (Peng et al., 4 Nov 2025). In image editing, fixed heuristics such as threshold $C_l(\cdot)$59 and dilation radius $C_l(\cdot)$60 can miss smooth but semantically important regions (You et al., 11 Jun 2026). In speech translation, applying CTC compression too early saves more memory but hurts translation quality more severely (Gaido et al., 2021).

Some work turns these fragilities into adaptive inference policies. ICPC varies compression level across epochs as data augmentation and at test time uses progressive re-evaluation: samples are first inferred at heavy compression, then reprocessed at medium or full size if confidence remains below a threshold. On average, more than $C_l(\cdot)$61 of samples terminate at heavy compression and fewer than $C_l(\cdot)$62 require full size, yielding net inference speed-ups of approximately $C_l(\cdot)$63 with no accuracy loss; across nine tasks it reports training and inference speed-ups of up to $C_l(\cdot)$64 and $C_l(\cdot)$65, with accuracy gains of up to $C_l(\cdot)$66 (Nagarajan et al., 2023).

Taken together, these results indicate that robustness under input compression is benchmark-dependent and modality-dependent. A plausible implication is that compression policies should be evaluated as conditional control policies over structure, difficulty, and downstream billing, not as single scalar reductions in token count.

6. Compression theory, information-theoretic views, and broader significance

Although current ML papers use input compression primarily to control compute and latency, related work connects it to classical compression objectives and to generalization theory. In bit-optimal LZ77 parsing, the objective is to minimize

$C_l(\cdot)$67

over all parsings of a string, equivalently the shortest path in a DAG whose edges represent phrases. The resulting algorithm computes a bit-optimal parsing in $C_l(\cdot)$68 time and $C_l(\cdot)$69 space under increasing-cost codeword encodings (0802.0835). In sequential-access compression, one-pass algorithms face nearly tight memory-quality trade-offs, and one stream is insufficient for grammar-based compression or entropy-only bounds under polylogarithmic resources (0902.0133).

An information-theoretic perspective appears in work on generalization bounds. “Bounding generalization error with input compression” uses the Xu-Raginsky bound

$C_l(\cdot)$70

and, in the infinite-width limit with additive Gaussian noise, computes

$C_l(\cdot)$71

Empirically, the bound is often tight for best-performing models, detects randomized labels, reflects robustness to perturbations, and works with few training samples (Galloway et al., 2022).

These results do not collapse into a single doctrine. In classical coding, compression is about minimizing output bits of a source representation [(0802.0835); (0902.0133)]. In modern ML systems, it is often about reducing the effective size of the representation consumed by a model, or the information retained about the original input, while preserving task utility (Adeyemi et al., 23 Jun 2026, Nagarajan et al., 2023, Galloway et al., 2022). This suggests that the contemporary significance of input compression lies in its role as a unifying systems-and-representation problem: it links efficiency, fidelity, and inductive bias, but the correct objective depends on whether the bottleneck is billing, latency, memory, bandwidth, or generalization.