CROP: Cost-Regularized Prompt Optimization

• The paper introduces a scalarized regularization objective that improves task accuracy while reducing token consumption during inference.
• CROP encompasses approaches from scalarization and budget allocation to Pareto-front search for balancing performance with deployment costs.
• Empirical results show up to 80% token reduction with minimal accuracy loss, demonstrating practical efficiency gains in LLM reasoning.

Cost-Regularized Optimization of Prompts (CROP) denotes a family of automatic prompt optimization methods in which prompt quality is not optimized solely for task performance, but jointly with some notion of cost. In the narrow sense, the term names the method introduced in "CROP: Token-Efficient Reasoning in LLMs via Regularized Prompt Optimization" (Shah et al., 8 Apr 2026), where the regularized quantity is generated response length. In a broader technical sense, the same design principle appears across prompt optimization research whenever the search objective or search procedure explicitly accounts for prompt length, output length, inference-token consumption, latency proxies, or the budget required to evaluate candidate prompts (Zehle et al., 22 Apr 2025). The field has consequently developed along three closely related lines: scalarized regularization, hard budget allocation, and Pareto-front search over performance-cost trade-offs (Kumar et al., 3 Jun 2026, Büssing et al., 15 May 2026).

1. Conceptual scope and cost models

The central premise of CROP is that prompt optimization has at least two cost axes. One is deployment-time cost: the cost of using the final prompt during inference. Depending on the formulation, this may be approximated by prompt-token count, response-token count, or a weighted combination of input and output tokens. The other is optimization-time cost: the budget consumed while searching over prompts, typically measured in LLM calls, evaluated examples, or total input tokens during optimization. CAPO makes this distinction explicit and is described as reducible to a CROP lens in which prompt quality is optimized under cost pressure arising both from search and from deployment (Zehle et al., 22 Apr 2025).

This distinction is methodologically important because different papers operationalize cost differently. The 2026 CROP method regularizes output length, motivated by the observation that accuracy-only automatic prompt optimization for reasoning tasks tends to elicit increasingly verbose reasoning traces, thereby increasing latency and token cost (Shah et al., 8 Apr 2026). CAPO regularizes prompt length while separately reducing search-time spend through racing (Zehle et al., 22 Apr 2025). MO-CAPO extends the deployment-time objective further by defining inference cost as a weighted sum of input and output tokens, rather than prompt length alone (Büssing et al., 15 May 2026). TextReg uses a broader structural penalty in which cost is not just prompt length but representational inefficiency, defined as token length multiplied by scope narrowness (Fu et al., 20 May 2026).

A second conceptual divide concerns how cost enters optimization. In scalarized formulations, task quality and cost are collapsed into a single objective with a coefficient such as $\lambda$ or $\gamma$. In budgeted formulations, the objective remains performance-only, but search must operate within a fixed evaluation horizon. In Pareto formulations, performance and cost are retained as separate objectives and the optimizer attempts to recover a frontier of non-dominated prompts rather than a single weighted optimum (Wang et al., 7 Jan 2025, Kumar et al., 3 Jun 2026).

2. Formal objective families

The most direct scalarized formulation appears in the original CROP method, which scores prompts by

$$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$

where $L_{\text{norm}}(p)$ is generated token count normalized by the average CoT output token length, and $\lambda$ controls the accuracy-cost trade-off. The paper reports a validation grid search over $\lambda \in [0.01, 0.5]$ and uses $\lambda = 0.2$ (Shah et al., 8 Apr 2026).

CAPO instantiates a closely related scalarization for prompt length. Its empirical task objective on dataset $D$ is

$$f(p; D) = \frac{1}{n} \sum_{i=1}^n \sigma(y_i, L(p,x_i)),$$

with exact-match scoring, and its cost-aware objective is

$$g(p; D) = f(p; D) - \gamma \cdot \mathrm{rel\_token\_length}(p).$$

The paper presents this as a form of multi-objective optimization, but operationally it is an a priori scalarization rather than Pareto-front optimization (Zehle et al., 22 Apr 2025). This suggests that CAPO is a canonical scalar CROP method for prompt-input cost.

TextReg makes the regularization structure still more explicit. It defines empirical task risk

$\gamma$0

and optimizes

$\gamma$1

where representational inefficiency is

$\gamma$2

Here $\gamma$3 is capacity cost and $\gamma$4 is scope narrowness (Fu et al., 20 May 2026). Relative to simpler CROP formulations, this replaces a one-dimensional token penalty with a cost that couples prompt size to semantic specificity.

Budget-aware methods are adjacent but formally distinct. SOPL defines a finite-horizon sequential decision problem with a maximum number of evaluations $\gamma$5, a prompt selection policy $\gamma$6, and Bayesian updates to a knowledge state, but does not introduce a regularizer of the form $\gamma$7 (Wang et al., 7 Jan 2025). TRIPLE similarly assumes a hard budget $\gamma$8 on prompt-evaluation trials and frames prompt selection as fixed-budget best arm identification, again without a cost penalty in the objective (Shi et al., 2024). These methods are cost-aware in the sense of search-budget allocation, but not cost-regularized in the strict scalarized sense.

Pareto formulations retain the objectives separately. CRAFT defines task score $\gamma$9 and prompt-token cost $$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$0, with Pareto dominance

$$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$1

with at least one strict inequality, and argues that fixing a weighted sum in advance can cause "scalarization collapse" (Kumar et al., 3 Jun 2026). MO-CAPO likewise formulates prompt optimization as

$$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$2

where $$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$3 is negative empirical task performance and $$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$4 is deployment-time inference cost (Büssing et al., 15 May 2026). A plausible implication is that Pareto CROP methods generalize scalar CROP by postponing the choice of trade-off coefficient until after search.

3. Prompt representations and optimization mechanisms

The search space in CROP-style methods is typically discrete and structured. CAPO defines a prompt as

$$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$5

where $$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$6 is an instruction and $$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$7 is an ordered tuple of up to $$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$8 few-shot examples. The resulting space is

$$S(p) = \text{Accuracy}(p) - \lambda \cdot L_{\text{norm}}(p),$$9

Task descriptions are not optimized variables but side information injected into the meta-prompts used by the search operators (Zehle et al., 22 Apr 2025). MO-CAPO inherits essentially the same prompt structure, but evaluates candidates under a bi-objective criterion and an adaptive intensification schedule (Büssing et al., 15 May 2026).

The original CROP method operates differently. Rather than using evolutionary operators over instruction and exemplar tuples, it extends textual-gradient prompt optimization. A target model $L_{\text{norm}}(p)$0 produces outputs $L_{\text{norm}}(p)$1; an evaluator $L_{\text{norm}}(p)$2 generates correctness-oriented textual feedback $L_{\text{norm}}(p)$3; a second evaluator $L_{\text{norm}}(p)$4 generates brevity-oriented feedback $L_{\text{norm}}(p)$5; and an optimizer LLM $L_{\text{norm}}(p)$6 updates the prompt using the concatenated textual gradients (Shah et al., 8 Apr 2026). The regularization gradient is computed only when $L_{\text{norm}}(p)$7 exceeds a predefined threshold, and the evaluator is instructed to request only gradual reductions in length while preserving critical computational steps. This makes CROP a prompt-only, non-parametric regularization method: model weights remain unchanged, but prompt revisions are guided by both correctness and conciseness feedback.

TextReg also works in text space, but its mechanism is explicitly regularization-driven. It decomposes the textual update into a purified task gradient and a regularization gradient,

$L_{\text{norm}}(p)$8

and then selects rewrites that faithfully implement the task update while minimizing incompatibility with the regularization signal (Fu et al., 20 May 2026). Its three named components are Dual-Evidence Gradient Purification, Semantic Edit Regularization, and Regularization-Guided Prompt Update. In CROP terms, this is a discrete-text approximation to gradient-based regularized optimization.

Search-budget control introduces another layer of mechanism. CAPO uses racing: the development set is partitioned into blocks, candidate prompts are evaluated incrementally, and a prompt is eliminated when more than $L_{\text{norm}}(p)$9 other prompts are significantly better according to a paired $\lambda$0-test at level $\lambda$1 (Zehle et al., 22 Apr 2025). MO-CAPO uses intensification instead: challengers are progressively evaluated on more blocks only if they continue to look promising relative to an incumbent non-dominated set (Büssing et al., 15 May 2026). TRIPLE and SOPL address the same problem through bandit identification and knowledge-gradient allocation, respectively (Shi et al., 2024, Wang et al., 7 Jan 2025). This suggests that CROP is not only about how cost enters the objective, but also about how scarce evaluation evidence is allocated during search.

4. Empirical trade-offs and observed behavior

The original CROP paper reports large output-token reductions on reasoning benchmarks while maintaining competitive accuracy. On Gemini 2.0 Flash, GSM8K changes from 95.3 accuracy and 198.4 tokens under CoT to 93.4 accuracy and 50.0 tokens under CROP; Object Counting changes from 96.0 and 101.2 to 94.8 and 19.6; and LogiQA changes from 65.2 and 540.5 to 64.2 and 181.0 (Shah et al., 8 Apr 2026). The paper describes this as up to an 80.6% reduction in token consumption with only a nominal decline in performance. On Qwen 2.5 7B, the same pattern holds, with reductions of 82.7% on GSM8K, 78.5% on Object Counting, and 70.7% on LogiQA relative to CoT, again with moderate accuracy loss (Shah et al., 8 Apr 2026).

A qualitative result of CROP is that it discovers compressed reasoning styles rather than eliminating reasoning entirely. The learned GSM8K prompt instructs the model to answer "concisely, showing the combined essential calculations leading directly to the final numerical answer" and to "Omit intermediate labels" (Shah et al., 8 Apr 2026). The reported behavior is therefore not direct-answer suppression of reasoning, but dense symbolic reasoning with conversational filler removed.

CAPO shows a different trade-off profile. Across five datasets and three LLMs, it outperforms state-of-the-art discrete prompt optimization baselines in 11 of 15 model-dataset combinations, with improvements up to 21 percentage points in accuracy, while racing saves 44% of evaluations on average (Zehle et al., 22 Apr 2025). The most prominent example is GSM8K with Llama-3.3-70B, where CAPO reaches 73.73% test accuracy versus 51.87% for OPRO and 50.87% for EvoPromptGA (Zehle et al., 22 Apr 2025). The same paper also shows the regularization trade-off directly: removing the length penalty by setting $\lambda$2 tends to increase prompt length and can increase accuracy slightly, whereas the penalty reduces average prompt cost during search.

TextReg’s empirical contribution is oriented toward generalization rather than token efficiency per se. Across reasoning benchmarks, it reports OOD gains of up to +11.8% over TextGrad and +16.5% over REVOLVE (Fu et al., 20 May 2026). The paper argues that unregularized prompt optimization often overfits by accumulating case-specific instructions, and that explicit regularization of representational inefficiency improves transfer to harder or shifted test distributions.

MO-CAPO evaluates the Pareto alternative. It reports that the algorithm outperforms its NSGA-II-based baseline on 8 out of 12 cases in terms of noisy $\lambda$3 and often reaches competitive performance at a considerably lower optimization budget (Büssing et al., 15 May 2026). Because it returns a diverse set of non-dominated prompts, it can recover trade-offs omitted by single-objective methods. CRAFT reports a similar qualitative outcome: accuracy-only, cost-only, and weighted-sum baselines each concentrate in narrower regions of the performance-cost plane, whereas CRAFT’s retained fronts span both high-accuracy and low-cost regions (Kumar et al., 3 Jun 2026).

5. Scalarization, budget allocation, and Pareto-front search

One of the main debates within CROP research concerns whether cost should be built into the search objective as a scalar penalty or preserved as a separate objective. Scalarized methods such as CROP, CAPO, and TextReg choose a trade-off coefficient before search. This gives a single target objective and often simplifies implementation. It also aligns naturally with classical regularization,

$\lambda$4

which is the interpretive form used to explain CAPO under a CROP lens (Zehle et al., 22 Apr 2025).

Pareto methods reject that commitment. CRAFT argues that weighted-sum scalarization can recover only a narrow region of the front, especially in discrete and non-convex prompt spaces, and calls this failure mode "scalarization collapse" (Kumar et al., 3 Jun 2026). MO-CAPO makes a similar point indirectly by showing that a Pareto optimizer can discover prompt sets spanning diverse performance-cost trade-offs that are omitted by single-objective optimizers (Büssing et al., 15 May 2026). A plausible implication is that scalar CROP is most appropriate when a deployment budget or trade-off weight is already known, whereas Pareto CROP is more appropriate when downstream requirements are uncertain.

A separate but equally important debate concerns where cost enters the pipeline. TRIPLE and SOPL do not regularize prompt content or inference cost; instead, they optimize how scarce evaluation budget is spent during search (Shi et al., 2024, Wang et al., 7 Jan 2025). CPO goes further in this direction by replacing expensive online evaluation with an offline causal reward model learned using Double Machine Learning, thereby enabling query-specific prompt search with only 7 prompt-generation calls per query after a one-time offline data collection stage (Chen et al., 2 Feb 2026). These methods are not CROP in the strict regularization sense, but they address the same practical bottleneck from the evaluation architecture side rather than from the objective side.

The result is a useful taxonomy. Scalarized CROP penalizes cost directly in the objective. Budget-aware prompt optimization keeps the objective performance-centered but constrains or reallocates search effort. Pareto CROP treats performance and cost as coequal objectives and returns a frontier. The literature increasingly treats these as complementary rather than mutually exclusive design choices (Shah et al., 8 Apr 2026, Büssing et al., 15 May 2026).

6. Limitations, boundary conditions, and open directions

Current CROP formulations remain incomplete as full models of LLM economics. The original CROP paper regularizes output length but explicitly does not model input token cost, partly motivating this omission by appeal to prefix caching for static system prompts (Shah et al., 8 Apr 2026). CAPO regularizes prompt length and reduces search-time cost through racing, but does not model output token length, which the authors identify as another major cost component influenced by prompt design (Zehle et al., 22 Apr 2025). MO-CAPO broadens the cost model to weighted input and output tokens, yet still approximates deployment cost through token counts rather than direct latency, concurrency effects, cache reuse, or provider-specific billing details (Büssing et al., 15 May 2026).

The regularization signal itself can also fail. CROP notes that over-regularization, weak optimizer capacity, or very small batch size can collapse prompts toward answer-only behavior, thereby harming tasks that require intermediate reasoning (Shah et al., 8 Apr 2026). TextReg similarly treats prompt overfitting as a representation-control problem and restricts its experiments to single-turn reasoning with well-defined behavioral rules, leaving open-ended generation, multi-turn prompts, and agent instructions to future work (Fu et al., 20 May 2026). CAPO reports that racing may be too aggressive at $\lambda$5, potentially eliminating promising prompts early (Zehle et al., 22 Apr 2025).

Several forward directions recur across the literature. CAPO explicitly proposes moving from scalarization to an a posteriori multi-objective method so that users can choose among prompts with different length-performance trade-offs after search rather than before it (Zehle et al., 22 Apr 2025). MO-CAPO suggests separating input and output tokens into distinct objectives and adding further objectives such as latency, robustness, or fairness (Büssing et al., 15 May 2026). CRAFT’s central recommendation is to make the accuracy-cost trade-off a post-search decision, not a pre-search weight (Kumar et al., 3 Jun 2026). A plausible synthesis is that mature CROP systems will combine all three strands: explicit cost-sensitive objectives, adaptive search-budget allocation, and user-facing Pareto sets over multiple deployment costs.

In that broader sense, CROP has evolved from a response-length regularizer for reasoning prompts into a general research program for prompt optimization under resource constraints. Its common insight is that prompt quality is not exhausted by raw task accuracy. For deployment, the prompt is also a computational object whose structure affects search expense, inference cost, and generalization behavior, and prompt optimization methods that ignore those costs increasingly appear incomplete (Shah et al., 8 Apr 2026, Fu et al., 20 May 2026).