POLCA: Prioritized Local Context Aggregation

• POLCA is a framework for stochastic generative optimization that leverages LLMs to generate proposals based on both numerical rewards and textual feedback.
• It integrates a priority queue with UCB-style exploration, a semantic ε-Net for diversity control, and an LLM Summarizer for global contextual aggregation.
• Empirical results on benchmarks like τ-bench, HotpotQA, and KernelBench demonstrate its superior performance in noisy, sparse reward environments.

Prioritized Optimization with Local Contextual Aggregation (POLCA) is a framework for stochastic generative optimization in which a generative LLM serves as the optimizer, using numerical rewards and text feedback to search for high-performing parameterizations of complex systems such as LLM prompts, multi-turn agents, code translators, and CUDA kernel generators. The method is formulated for settings in which evaluations are noisy because feedback may be stochastic, only minibatches can be sampled, and the underlying system may itself behave stochastically. POLCA addresses these conditions by combining a priority queue for exploration–exploitation management, a semantic $\varepsilon$-Net for diversity control, and an LLM Summarizer for aggregation of historical local context. The framework is presented with a convergence guarantee under stochasticity and evaluated on $τ$-bench, HotpotQA, VeriBench, and KernelBench, where it is reported to outperform state-of-the-art baselines in both deterministic and stochastic regimes (Ren et al., 16 Mar 2026).

1. Formal problem setting

POLCA is defined over a parameterized program $P_\theta$ with parameter $\theta$ in a typically large, discrete space $\Theta$. There is an underlying distribution $\mathcal{D}$ over pairs $(\omega, x)$, where $x$ is an input and $\omega$ is side information. Running $P_\theta$ on $τ$0 produces an output $τ$1. A guide $τ$2 then provides two feedback modalities from $τ$3: a numerical score $τ$4 and textual feedback $τ$5. The score may take forms such as binary pass/fail, an LLM-judge score in $τ$6, or runtime speedup; the textual channel may contain error messages, critiques, or hints.

The target quantity is the true expected performance

$τ$7

The optimization objective is to interactively propose a sequence $τ$8 through an LLM optimizer $τ$9 so as to maximize $P_\theta$0 under a finite compute budget, where $P_\theta$1 is the returned candidate. Because each step may only evaluate a small minibatch $P_\theta$2, empirical scores are necessarily noisy. This formulation places prompt optimization, agent optimization, code translation, and low-level code generation under a common stochastic optimization abstraction.

A central feature of the formulation is that the optimizer itself may be stochastic and error-prone. This departs from classical black-box optimization assumptions in which proposal generation is externally fixed or analytically specified. Here, the search operator is itself a LLM conditioned on prior rollouts and textual context. This suggests that POLCA is intended not merely as an evaluation scheduler, but as a framework for coupling statistical decision-making with generative proposal mechanisms.

2. Core algorithmic structure

POLCA maintains a priority queue $P_\theta$3 containing candidate solutions together with empirical histories. The loop begins with initialization $P_\theta$4 and proceeds while compute budget remains. At each iteration, a minibatch $P_\theta$5 of size $P_\theta$6 is sampled, up to $P_\theta$7 programs $P_\theta$8 are selected from $P_\theta$9 by highest exploration priority $\theta$0, those candidates are evaluated on $\theta$1, and their empirical means $\theta$2 are updated. The queue is then summarized into a compact global context $\theta$3, after which the optimizer $\theta$4 is called for each explored candidate using local rollouts together with the global summary to produce raw proposals $\theta$5. A semantic $\theta$6-Net filter reduces these to $\theta$7, which are then evaluated on the same minibatch and inserted into the queue with updated statistics. The returned output is

$\theta$8

The default exploration priority is the empirical mean $\theta$9. For theory, however, POLCA uses a UCB-style priority

$\Theta$0

where $\Theta$1 bounds the variance, $\Theta$2 is the number of samples for $\Theta$3, and $\Theta$4 is the total number of samples so far. This substitution is used to guarantee systematic exploration under noise (Ren et al., 16 Mar 2026).

The algorithm combines local and global contextual signals. Locality comes from conditioning each optimizer call on the rollouts of the candidate currently being explored. Globality comes from the summary of the entire queue and its evaluation history. The phrase “Local Contextual Aggregation” refers to this combination of candidate-specific evidence and queue-level distilled experience. A plausible implication is that POLCA is designed to preserve exploitative refinement of promising candidates while avoiding myopic dependence on only the most recent rollout.

3. Diversity control via the semantic $\Theta$5-Net

A persistent difficulty in generative search is unconstrained growth of semantically redundant candidates. POLCA addresses this using an $\Theta$6-Net mechanism defined in an embedding space. Each candidate is mapped to $\Theta$7, and semantic distance is measured by

$\Theta$8

At each iteration, the union $\Theta$9 is maintained as an $\mathcal{D}$0-Net, meaning that no two stored points are within distance $\mathcal{D}$1.

The construction is a farthest-first greedy traversal over the raw proposals. Starting with $\mathcal{D}$2, the algorithm repeatedly computes, for each remaining candidate, its minimum distance to the existing queue and accepted new points:

$\mathcal{D}$3

It then selects the candidate with maximal $\mathcal{D}$4. If this maximal distance is at least $\mathcal{D}$5, the candidate is retained; otherwise the procedure stops. The resulting filtered set is inserted into the queue.

This construction guarantees the bound $\mathcal{D}$6, where $\mathcal{D}$7 is the covering number of $\mathcal{D}$8 under radius $\mathcal{D}$9 (Ren et al., 16 Mar 2026). Within the framework, the $(\omega, x)$0-Net serves both as a memory-control device and as a structural bias toward search-space coverage. The reported ablations further indicate that removing the $(\omega, x)$1-Net degrades both learning speed and final performance on $(\omega, x)$2-bench by $(\omega, x)$3–$(\omega, x)$4, and that setting $(\omega, x)$5 is worst because it permits unbounded growth of $(\omega, x)$6. The paper also reports that varying $(\omega, x)$7 in $(\omega, x)$8 shows robust performance, suggesting that the mechanism is not overly sensitive within that range.

4. Historical aggregation through the LLM Summarizer

POLCA includes an external LLM Summarizer that recomputes a compact natural-language summary of the queue and its empirical rollouts at every iteration. The input is the entire set $(\omega, x)$9. The summarizer partitions trajectories into “success” cases with $x$0 and “failure” cases with $x$1 for a threshold $x$2, samples contrastive pairs, and outputs a short summary

$x$3

that highlights patterns in successes and failures. This summary is prepended to each optimizer call as global context.

The module is explicitly not implemented as a parametric learner with persistent internal updates; instead, “no explicit parameter update is needed” because $x$4 is recomputed each cycle to reflect the evolving frontier. In practice, XML-style tags are enforced so that actionable guidance is contained within <summary>...</summary> (Ren et al., 16 Mar 2026).

This mechanism addresses a common ambiguity in LLM-based optimization workflows: whether meta-learning must be embedded in model weights or can arise through repeated context construction. POLCA adopts the latter view. The reported ablation that removing the Summarizer degrades both learning speed and final performance on $x$5-bench by $x$6–$x$7 indicates that historical context aggregation is not merely auxiliary bookkeeping but an active component of the optimization procedure.

5. Convergence analysis and theoretical claims

The theoretical analysis is given under a strict-improvement assumption. Specifically, there exist $x$8 and $x$9 such that for any $\omega$0 with $\omega$1, an optimizer call producing $\omega$2 satisfies

$\omega$3

Assuming also that each reward observation is $\omega$4-sub-Gaussian, the main result states that if POLCA runs for $\omega$5 iterations with UCB priorities, then

$\omega$6

Here $\omega$7 is the total number of samples allocated to candidate $\omega$8 and $\omega$9 is the $P_\theta$0-covering number. In the limit $P_\theta$1, the bound degenerates to $P_\theta$2 (Ren et al., 16 Mar 2026).

The proof sketch proceeds by partitioning $P_\theta$3 into intervals of width $P_\theta$4, showing that $P_\theta$5 draws per interval suffice to propose a $P_\theta$6-improvement with probability at least $P_\theta$7, and then using UCB to distinguish candidates within an interval after $P_\theta$8 additional samples. Summation over at most $P_\theta$9 intervals yields the stated bound.

A second theorem compares POLCA with naive sequential improvement. According to the reported result, using a global best-so-far baseline yields an expected $τ$00 improvement chain, whereas sequential restarting requires $τ$01 steps in the worst case. This comparison formalizes the claim that queue-based reuse of discovered high-performing candidates changes the scaling behavior of iterative generative optimization under the paper’s assumptions.

6. Benchmarks, empirical results, and computational profile

POLCA is evaluated on four benchmark families spanning prompt optimization, agent optimization, program translation, and CUDA kernel generation (Ren et al., 16 Mar 2026).

Benchmark	Setting	Reported result
$τ$02-bench	10-task retail training, sparse binary reward; held-out 105 tasks	57.5% pass@1 on train vs. 34.8% base, 55.7% GEPA, 37.3% OpenEvolve; on all 115 tasks 43.9% vs. 38.9% base
HotpotQA	100 validation tasks, binary exact-match reward	With 2,000 metric calls, POLCA outperformed beam-search, GEPA, OpenEvolve by a large margin ($τ$03 accuracy)
VeriBench	3-stage evaluation: compile $τ$04 unit tests $τ$05 LLM judge; score in $τ$06	With 50 samples per task, DSPy $τ$07, GEPA $τ$08, OpenEvolve $τ$09, POLCA $τ$10
VeriBench (compilation only)	Binary compile-success reward	POLCA reached 95.2% pass rate vs. DSPy $τ$11, GEPA $τ$12, OpenEvolve $τ$13
KernelBench	16 batched-matmul tasks, require speedup $τ$14	POLCA converged fastest, achieving $τ$15 within 10 eval steps vs. $τ$16 for baselines

The reported evaluation criteria emphasize both sample efficiency and final performance. On $τ$17-bench and HotpotQA, the setting is prompt or agent optimization under sparse or exact-match reward. On VeriBench, the evaluation pipeline is explicitly staged, moving from compile success to unit tests and then LLM judgment. On KernelBench, performance is expressed through a speedup threshold. Taken together, these tasks instantiate the paper’s claim that the same optimization framework can operate across heterogeneous feedback topologies.

The implementation notes quantify the computational profile. Each evaluation step samples $τ$18 candidates and $τ$19 data points, costing $τ$20 guide calls and $τ$21 program runs. The memory bound is $τ$22; embedding lookups and distance computations require $τ$23 per filter pass. Each iteration issues $τ$24 parallel calls to the optimizer $τ$25 and one summary call. The paper reports total LLM token usage of at most approximately $τ$26 per 100 steps on $τ$27-bench, and notes that asynchronous Evaluate and Propose operations, parallelized LLM API calls, and contrastive sampling in the Summarizer are used for efficiency. The hyperparameters are characterized operationally: $τ$28 trades off exploration versus cost, batch size $τ$29 trades off noise reduction versus latency, and $τ$30 trades off breadth versus compute per iteration.

These results support the paper’s central empirical claim that bandit-style prioritization, diversity filtering, and local-plus-global contextual prompting can jointly improve robustness and efficiency in stochastic LLM-driven optimization. A plausible implication is that POLCA is particularly well matched to domains in which evaluation is expensive, sparse, or noisy, and in which textual feedback can be exploited as a structured search signal rather than treated as incidental metadata.