ProCut: Prompt Compression via Attribution Estimation

• The paper proposes a segment-level, attribution-guided prompt compression method that prunes low-utility components to improve efficiency.
• It employs perturbation-based and LLM-driven attribution estimation to quantify each segment's impact on downstream performance.
• ProCut achieves significant token reductions—up to 78%—while maintaining or slightly improving task performance across various benchmarks.

Searching arXiv for ProCut and closely related prompt compression work to ground the article. Prompt Compression via Attribution Estimation (ProCut) is a prompt-compression framework for large-scale industrial LLM systems in which prompt templates have expanded to thousands of tokens through iterative additions of task instructions, few-shot examples, and heuristic rules. It reframes prompt compression as a segment-selection problem: rather than deleting individual tokens or learning model-specific continuous prompt embeddings, it partitions a prompt into semantically meaningful units, estimates each unit’s contribution to downstream performance, and prunes low-utility components. The method is presented as LLM-agnostic, training-free, and compatible with black-box APIs, with the stated goals of reducing inference latency and serving cost, avoiding prompt dilution in long contexts, and improving prompt maintainability, auditability, and debugging (Xu et al., 4 Aug 2025).

1. Problem setting and design rationale

ProCut is motivated by a recurrent operational pattern in industrial prompting: prompt templates accumulate role descriptions, instructions, examples, chain-of-thought cues, and task-specific rules until they become bloated. The paper identifies three resulting problems. First, long prompts increase inference latency and API cost. Second, long contexts can reduce accuracy because important instructions become diluted. Third, long prompts are difficult to maintain, audit, and debug (Xu et al., 4 Aug 2025).

The framework therefore formulates compression as selection over prompt segments rather than token-level editing. Given an initial prompt template $p$, ProCut assumes that it can be partitioned into an ordered list of $M$ disjoint segments,

$p = [p_1, p_2, \dots, p_M].$

Given a dataset $\mathcal D = \{(x_i, y_i)\}$, the objective is to select a subset $\mathcal K \subseteq \{1,\dots,M\}$ of size $k$ so that the compressed prompt $p_{\mathcal K}$ maximizes the average task metric over the dataset. For each input $x_i$, the prompt is instantiated and sent to a black-box LLM, producing $\hat y_i = \text{LLM}(p(x_i))$, which is then scored by a task metric $s(y_i,\hat y_i)$ or a reference-free score $M$0 (Xu et al., 4 Aug 2025).

This segment-level formulation is central to ProCut’s position within the prompt-compression literature. The paper explicitly contrasts it with token-level methods, which can break placeholders or produce disfluent text, and with soft-compression methods, which are not easily transferable across LLMs and typically require retraining (Xu et al., 4 Aug 2025). A plausible implication is that ProCut is intended primarily for prompt templates that must remain human-readable, auditable, and directly deployable across model providers.

2. Prompt segmentation as the first stage

The first stage of ProCut is prompt template segmentation. The framework divides a prompt into semantically meaningful units and describes three segmentation strategies (Xu et al., 4 Aug 2025).

Strategy	Description	Intended setting
Pre-defined segmentation	Domain experts manually label logical blocks	Structured prompts with known components
Structure-aware segmentation	Split at sentence or paragraph boundaries	Prompts whose units are semantically coherent at those boundaries
LLM-driven segmentation	An LLM splits the prompt into at most a configurable number of units while preserving exact text and placeholders	Unstructured or model-generated prompts

The LLM-driven strategy is designed for prompts that are unstructured or produced by prompt-optimization systems such as TextGrad. Its segmentation prompt instructs the LLM to preserve the exact text, keep placeholders intact, and ensure that the resulting units can be concatenated to reconstruct the original prompt exactly (Xu et al., 4 Aug 2025). The paper specifically notes placeholders such as $M$1, $M$2, and $M$3, indicating that the method is designed for templated industrial prompts rather than only free-form prompts.

This segmentation stage is also what distinguishes ProCut from prompt compressors that operate directly on tokens. LongLLMLingua, for example, compresses by document selection and question-aware token scoring based on contrastive perplexity rather than by selecting semantically defined prompt-template units (Jiang et al., 2023). Selection-$M$4 likewise produces token preservation probabilities and keeps the top-$M$5 tokens in a single forward pass (Chung et al., 2024). ProCut instead treats the prompt as a structured composition of coherent natural-language blocks (Xu et al., 4 Aug 2025).

3. Attribution estimation and the ProCut scoring pipeline

The second stage is attribution estimation, where ProCut quantifies how much each segment contributes to downstream task performance. The paper emphasizes that this module is built from perturbation-based methods and requires only API access, preserving model agnosticism (Xu et al., 4 Aug 2025). In this formulation, each segment is treated as a binary feature, and the effect of masking or retaining segments is measured on a held-out test set using the task metric.

The paper reports four representative black-box attribution methods:

Method	Core idea	Cost characterization in the paper
Shapley values (SHAP)	Estimate each segment’s marginal contribution by averaging over many coalitions	Can require evaluation over many subsets; worst case $M$6
Leave-One-Out (LOO)	Measure the performance drop when one segment is removed	$M$7 calls
LASSO regression	Fit a sparse linear model from randomly masked prompt evaluations	Requires many LLM calls
Greedy forward selection	Incrementally add the segment that yields the largest gain	Requires repeated prompt evaluations

Because SHAP-like methods can be expensive, ProCut introduces an LLM-driven attribution estimator. This is described as one of the paper’s key contributions (Xu et al., 4 Aug 2025). The estimator uses a probe-and-test loop:

1. Ask the LLM to generate a small set of candidate masks indicating which segments to keep.
2. Evaluate each masked prompt on the training set.
3. Feed the results back to the LLM in a second prompt that asks it to rank the segments by importance.
4. Convert the ranking into attribution scores.

The appendix pseudocode is: $M$8

$M$9

$p = [p_1, p_2, \dots, p_M].$0

$p = [p_1, p_2, \dots, p_M].$1

The paper characterizes this estimator as constant-call, or at least bounded-call, relative to the number of segments, and states that it completes in fewer than $p = [p_1, p_2, \dots, p_M].$2 LLM calls (Xu et al., 4 Aug 2025). The reported latency reductions are substantial: the LLM-as-Ranker variant achieves near-SHAP quality while reducing end-to-end latency by 80% relative to SHAP, 52% relative to LOO, and 66% relative to LASSO. Attribution quality is evaluated using NDCG, where the LLM-as-Ranker (2-shot) variant remains close to the SHAP “gold standard” (Xu et al., 4 Aug 2025).

This attribution mechanism differs from earlier compression families. The 2022 work on prompt compression for controllability learns soft prompt embeddings by minimizing expected KL divergence between hard-prompt and soft-prompt continuation distributions rather than estimating which prompt segments matter most (Wingate et al., 2022). Similarly, “Compress, Then Prompt” optimizes continuous prompt embeddings $p = [p_1, p_2, \dots, p_M].$3 for compressed models with frozen weights, not segment attribution or prompt pruning (Xu et al., 2023).

4. Pruning rule, controllability of compression, and benchmark evidence

The third stage is pruning. Once ProCut has segment scores $p = [p_1, p_2, \dots, p_M].$4, it retains the top $p = [p_1, p_2, \dots, p_M].$5 segments, where $p = [p_1, p_2, \dots, p_M].$6 is a user-defined compression ratio: $p = [p_1, p_2, \dots, p_M].$7 The retained segments preserve the original order, which the paper identifies as important for prompt coherence (Xu et al., 4 Aug 2025). This yields a directly controllable compression procedure: $p = [p_1, p_2, \dots, p_M].$8 can be set according to latency, cost, or performance constraints.

The benchmark evaluation covers 12 tasks from five benchmark datasets across four categories: GSM8K for mathematical reasoning, HumanEval for code generation, SQuAD for extractive QA, and nine tasks drawn from BBH and MMLU for broader reasoning and knowledge. The specific BBH tasks are Geometry Shapes, Object Counting, Color Reasoning, Penguins, and Temporal Sequence; the MMLU tasks are College Medicine, College Math, Anatomy, and Astronomy. Metrics are Exact Match for GSM8K, BBH, and MMLU; unbiased Pass@1 for HumanEval; and F1 for SQuAD. For each task, the paper uses 20 training examples and 75–100 test examples depending on the dataset (Xu et al., 4 Aug 2025).

The initial prompts in these experiments are composed of five common segment types: role-playing, zero-shot chain-of-thought, few-shot chain-of-thought examples, question placeholder, and, where relevant, context placeholder (Xu et al., 4 Aug 2025). This design makes the attribution output interpretable as well as compressive. Across tasks, the question placeholder has by far the largest attribution. Few-shot CoT examples are important on tasks like GSM8K, BBH, and MMLU, while role-playing and zero-shot CoT frequently have small or even negative attribution (Xu et al., 4 Aug 2025). The paper interprets this as evidence that curated demonstrations and task-specific inputs matter more than generic heuristics in these settings.

ProCut is compared against Random Selection, Vanilla LLM compression, Selective Context, and a brute-force oracle that exhaustively searches all $p = [p_1, p_2, \dots, p_M].$9 subsets. Token-level methods such as LLMLingua are excluded because they can break placeholders and invalidate templates (Xu et al., 4 Aug 2025). Across compression ratios of 25%, 50%, and 75%, ProCut consistently matches or nearly matches the brute-force oracle and significantly outperforms the non-oracle baselines. The paper reports average compressed-prompt performance of 0.575–0.752 for ProCut variants at different ratios, versus 0.090–0.462 for non-oracle baselines on average, depending on the method and ratio. Table-level averages are reported as about 0.57–0.58 at 25% retention, about 0.84–0.85 at 50% retention, and about 0.83–0.85 at 75% retention. The abstract summarizes this as up to 62% better performance than alternative methods (Xu et al., 4 Aug 2025).

A common misconception is that prompt compression is necessarily destructive. ProCut’s benchmark evidence does not support that simplification. The paper reports that compressed prompts can maintain, and in some cases slightly improve, task performance relative to longer templates (Xu et al., 4 Aug 2025). This suggests that prompt compression can function not only as a cost-reduction mechanism but also as a way to remove low-utility or distracting prompt components.

5. Integration with prompt optimization and industrial deployments

ProCut is designed to integrate with prompt optimization frameworks rather than replace them. The paper highlights TextGrad as a particularly natural integration point and treats ProCut as a compression layer that can be applied after each optimization iteration (Xu et al., 4 Aug 2025). In the SQuAD RQ3 experiment, the comparison is between plain TextGrad and TextGrad + ProCut regularization. The ProCut-regularized pipeline controls prompt growth without harming quality.

After three iterations, the compressed prompts are reduced to 27%, 47%, and 66% of the token count of the TextGrad-only prompts under compression ratios of 40%, 60%, and 80%, respectively, while maintaining comparable F1 scores around 0.815, 0.819, and 0.813, close to the uncompressed baseline of 0.813 (Xu et al., 4 Aug 2025). In an appendix example, TextGrad alone produces a prompt of 2272 tokens with F1 0.803, while TextGrad plus ProCut produces a prompt of 896 tokens with F1 0.820 (Xu et al., 4 Aug 2025). The paper presents this as evidence that ProCut can regularize iterative prompt optimization and prevent runaway prompt expansion.

The framework also includes real industrial deployments. In a high-traffic intent classification prompt, ProCut achieves a 73% reduction in prompt length with no accuracy loss. In a qualification assessment prompt, an original prompt exceeding 2200 tokens is compressed to about 300 tokens, an 84% reduction, with performance slightly improved (Xu et al., 4 Aug 2025). The abstract summarizes the production outcome as 78% fewer tokens in production (Xu et al., 4 Aug 2025).

The paper additionally estimates cost savings based on GPT-4o pricing: roughly $\mathcal D = \{(x_i, y_i)\}$08K per 1M calls for qualification assessment (Xu et al., 4 Aug 2025). These figures place ProCut in a deployment-oriented lineage distinct from prompt-compression work centered primarily on model behavior under long context. LongLLMLingua, for example, emphasizes long-context latency, position bias, and question-aware content density, reporting 94.0% cost reduction on LooGLE and prompt-length speedups in 10k-token settings (Jiang et al., 2023). ProCut’s focus is instead prompt-template maintainability and selective removal of low-utility template components (Xu et al., 4 Aug 2025).

6. Relation to earlier prompt compression methods

ProCut belongs to a broader family of methods that seek compact prompts while preserving downstream behavior, but its mechanism is distinct from several earlier lines of work.

The 2022 paper “Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction” compresses prompts by learning soft prompt vectors $\mathcal D = \{(x_i, y_i)\}$1 that approximate the distribution induced by a hard prompt $\mathcal D = \{(x_i, y_i)\}$2, optimizing

$\mathcal D = \{(x_i, y_i)\}$3

That method is a direct precursor in spirit, but it does not use attribution estimation, saliency scoring, gradient-based token ranking, or token-selection heuristics; instead, it learns continuous soft prompts by distribution matching (Wingate et al., 2022).

“Compress, Then Prompt” addresses a different deployment problem: prompting compressed LLMs. It freezes the compressed model parameters $\mathcal D = \{(x_i, y_i)\}$4, prepends $\mathcal D = \{(x_i, y_i)\}$5 trainable prompt tokens with embeddings $\mathcal D = \{(x_i, y_i)\}$6, and optimizes

$\mathcal D = \{(x_i, y_i)\}$7

Its aim is to recover quality lost through pruning and quantization, and it studies transferability across datasets, tasks, and compression levels. It is not a prompt-compression method in the sense of selecting or pruning segments from an existing prompt (Xu et al., 2023).

LongLLMLingua compresses long-context prompts through a question-aware coarse-to-fine pipeline. It ranks documents and then ranks tokens using contrastive perplexity,

$\mathcal D = \{(x_i, y_i)\}$8

together with dynamic budget allocation, document reordering, and subsequence recovery (Jiang et al., 2023). This makes it closer to token-importance estimation than the soft-prompt literature, but its importance signal is derived from perplexity contrast rather than explicit attribution over prompt-template segments.

Selection-$\mathcal D = \{(x_i, y_i)\}$9 is still closer to importance estimation. It computes a token preservation probability

$\mathcal K \subseteq \{1,\dots,M\}$0

keeps the top-$\mathcal K \subseteq \{1,\dots,M\}$1 tokens, and trains self-supervised under a masked causal language modeling objective (Chung et al., 2024). The paper explicitly states that Selection-$\mathcal K \subseteq \{1,\dots,M\}$2 is not an attribution method in the classic gradient-based sense, but that it is conceptually very close to token importance estimation (Chung et al., 2024). Relative to ProCut, however, Selection-$\mathcal K \subseteq \{1,\dots,M\}$3 operates at token level, uses continual pre-training, and is not training-free.

Within this landscape, ProCut is distinguished by three properties stated directly in the paper: it is segment-level rather than token-level, attribution-guided rather than perplexity-guided or soft-prompt-based, and training-free rather than requiring prompt tuning or continual pre-training (Xu et al., 4 Aug 2025).

7. Interpretive significance and practical boundaries

ProCut’s attribution analysis provides an interpretability layer in addition to compression. By assigning scores to prompt components such as role-playing, zero-shot CoT, few-shot CoT examples, and placeholders, it gives a direct account of which prompt segments contribute to measured task success (Xu et al., 4 Aug 2025). This is presented as a practical benefit for prompt debugging in industrial settings.

The framework also defines a particular view of prompt engineering. Rather than assuming that longer prompts are inherently more robust, ProCut treats prompt quality as a sparse composition problem in which only some components are utility-bearing for a given task and dataset (Xu et al., 4 Aug 2025). This suggests a shift from prompt accretion toward prompt auditing: segments are retained because they improve the metric under black-box evaluation, not because they seem intuitively useful.

At the same time, the method’s empirical claims are tied to the specific evaluation setup described in the paper: five benchmark datasets, 12 tasks, task-specific metrics, and real industrial prompts in intent classification and qualification assessment (Xu et al., 4 Aug 2025). A plausible implication is that ProCut is best understood as an operational framework for prompt-template compression under measurable task objectives, rather than as a general theory of prompt semantics.

In synthesis, ProCut defines prompt compression as attribution-guided segment pruning. The prompt is segmented into coherent natural-language units, each unit’s contribution is estimated by perturbation-based black-box evaluation or by an LLM-driven attribution estimator, and the top-ranked units are retained in their original order. The reported outcomes are large token reductions, substantial attribution-latency savings, compatibility with prompt-optimization frameworks, and production-scale prompt simplification with maintained or slightly improved performance (Xu et al., 4 Aug 2025).