AlphaOPT: Continual Learning for Optimization

• AlphaOPT is a continual-learning framework that converts informal optimization descriptions into precise mathematical formulations and executable solver programs.
• It employs a reusable experience library structured with taxonomy, condition, explanation, and example to capture and transfer modeling insights.
• The system uses a two-phase cycle of library learning and evolution, validated by solver feedback, to improve performance and manage library growth.

AlphaOPT is a continual-learning framework for converting informal optimization problem descriptions into correct mathematical formulations and executable solver programs by accumulating an explicit library of reusable modeling experience rather than retraining model weights. Introduced in "AlphaOPT: Formulating Optimization Programs with Self-Improving LLM Experience Library" (Kong et al., 21 Oct 2025), it is designed for the setting in which natural-language descriptions must be mapped into variables, objectives, constraints, domains, indices, bounds, and solver-compatible code, while correctness depends not only on syntax but also on selecting the right modeling abstraction under the right applicability conditions. Its central mechanism is a solver-verified, interpretable experience library whose entries are structured as $$\{\text{taxonomy},\ \text{condition},\ \text{explanation},\ \text{example}\}$$, and whose retrieval boundaries are continually refined through a two-phase cycle of Library Learning and Library Evolution.

1. Problem formulation and conceptual position

AlphaOPT addresses optimization-program formulation as a knowledge-transfer problem rather than as a pure text-to-code problem. The motivating claim is that LLMs fail on optimization tasks not merely because solver APIs are brittle, but because the intermediate modeling knowledge—such as when to use a fixed-charge linking pattern, when makespan requires an auxiliary variable, or when flow-conservation constraints are semantically justified—is often absent from datasets and therefore not reliably internalized by prompting or finetuning alone (Kong et al., 21 Oct 2025).

The framework is positioned between prompt engineering and finetuning. Like prompt-based systems, it uses a general LLM at inference time to read task descriptions and generate formulations and code. Unlike fixed prompting, it improves over time by storing newly discovered insights externally. Like learning-based systems, it improves with additional data, but unlike finetuning it does not update model parameters, does not require curated chain-of-thought or rationale annotations, and can operate under answer-only supervision. In that regime, if no gold program is available, the system uses solver-guided self-exploration to search for a program whose optimal objective matches the provided answer; a solver-validated program then becomes a proxy reference from which reusable insights are extracted (Kong et al., 21 Oct 2025).

The paper formalizes library construction through an optimization objective over library space $\mathcal{L}$ and task distribution $\mathcal{T}$: $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$ Here, $\mathrm{Success}(t \mid \ell)$ indicates whether library $\ell$ enables the system to produce a program achieving the correct optimal objective for task $t$, while $\Omega(\ell)$ is a complexity penalty such as library size or redundancy-adjusted size. In the appendix, the same idea is restated as

$$F(\ell) \;=\; \mathbb{E}_{t \sim \mathcal{T}_{\mathrm{train}}}\big[\, r(t \mid \ell)\,\big] \;-\; \lambda \, \Omega(\ell),$$

with $r(t,\ell)$ as a bounded reward. This formulation makes explicit that AlphaOPT seeks both improved success and controlled library growth.

2. Structured experience representation

The core data structure in AlphaOPT is an experience entry of the form

$\mathcal{L}$0

The paper treats these four fields as distinct functional components rather than as free-form memory (Kong et al., 21 Oct 2025).

The taxonomy field is a hierarchical label for indexing and retrieval. The library is organized into three top-level tracks—Domain Modeling, General Formulation, and Code Implementation—with level-1 and level-2 labels underneath them. Examples given in the paper include Domain Modeling $\mathcal{L}$1 Facility Location $\mathcal{L}$2 Fixed Charge (Big-M Linking), General Formulation $\mathcal{L}$3 Objective Specification $\mathcal{L}$4 Sum vs. Makespan Confusion, and Code Implementation $\mathcal{L}$5 Solver/API Syntax $\mathcal{L}$6 Strict Inequalities. Taxonomy supports a first retrieval stage described as quick label matching.

The condition field specifies when an insight applies. This is the principal transfer mechanism and the principal object of refinement during Library Evolution. A fixed-charge entry, for example, is described as applicable “when service or flow is allowed only if a facility is opened.” The emphasis on explicit applicability conditions is central: AlphaOPT is built around the claim that many failures arise from using a familiar modeling pattern in the wrong context rather than from not knowing the pattern at all.

The explanation field states the modeling principle. It functions both as guidance for the LLM and as a mechanism of interpretability for human inspection. The example field provides an operational demonstration, such as a formula, constraint fragment, or code snippet. The paper’s ablation results show that examples materially improve application quality beyond explanation alone (Kong et al., 21 Oct 2025).

This representation is accompanied by three design principles: the system is failure-driven, because every error is treated as a learning opportunity; locally validated, because an insight is admitted only if it solves its source task; and compact yet generalizable, because redundant insights are merged and conditions are evolved to avoid both over-specialization and over-generalization.

3. Continual cycle: Library Learning and Library Evolution

AlphaOPT operates in a continual two-phase loop. The first phase, Library Learning, adds new knowledge from individual tasks. The second phase, Library Evolution, repairs the retrieval boundary of existing knowledge using aggregate task–insight interaction history (Kong et al., 21 Oct 2025).

In Library Learning, the system begins with a natural-language optimization task, retrieves relevant insights if the library is non-empty, generates a mathematical formulation and solver code, executes the program, and checks success by comparing the optimal objective value against the ground-truth answer. If a failure occurs, two supervision modes are possible. Under gold-program supervision, the candidate formulation is compared against the reference, and discrepancies such as missing variables, incorrect objective terms, or misformulated constraints are distilled into candidate insights. Under answer-only supervision, the system performs solver-guided self-exploration, iteratively proposing executable programs, reusing prior failures as context, and continuing until a program achieves the correct objective. The transition from failure to successful repair is then mined for reusable insights.

Before storage, each candidate insight undergoes local verification: it is reapplied to the source task, and the solver must confirm that it resolves the failure. The paper states that this process is repeated over three independent trials per task to exploit stochastic failures and successes as additional signals. Verified insights are then inserted into the library, mapped to an existing taxonomy label or used to create a new one, and merged when sufficiently similar. Task solving can be parallelized over minibatches, but library updates are serialized through a centralized queue to avoid conflicting concurrent modifications.

Library Evolution operates at the population level. For each insight $\mathcal{L}$7, AlphaOPT maintains a partition

$\mathcal{L}$8

where $\mathcal{L}$9 contains tasks on which the insight was applicable and helpful, $\mathcal{T}$0 contains tasks on which it was retrieved but misleading, and $\mathcal{T}$1 contains tasks on which it was not retrieved but would have helped. The Evolver proposes condition refinements using evidence from these sets. Candidate refinements are evaluated on

$\mathcal{T}$2

with score

$\mathcal{T}$3

A refinement is accepted only if it improves $\mathcal{T}$4, and the best-improving candidate is retained. The paper describes typical refinement patterns as generalizing with equivalent phrasings, adding lexical anchors, adding explicit exclusions, and adding structural qualifiers.

4. Retrieval semantics, solver feedback, and answer-only learning

Retrieval in AlphaOPT is procedural and two-stage. First, the LLM performs quick label matching over the taxonomy dictionary to identify plausible regions of the library. Second, it evaluates each candidate insight’s condition and retains only the most applicable ones (Kong et al., 21 Oct 2025). This design distinguishes AlphaOPT from naïve similarity-based retrieval and explains why taxonomy materially affects performance.

Solver feedback is the framework’s grounding signal. Candidate programs are validated by executing them with a solver and checking whether the resulting optimal objective closely aligns with the provided ground-truth solution. Under answer-only training, this solver outcome is the only supervision signal needed to discover a correct program. Solver feedback also determines whether an extracted lesson is admitted to the library, since each insight must pass local self-verification on its source task.

The paper treats incorrect formulations and execution failures as structured sources of knowledge rather than as discarded attempts. If a gold reference exists, discrepancies are diagnosed directly. Otherwise, solver-guided self-exploration is used to discover a correct executable alternative. If the issue is not lack of knowledge but bad retrieval, the failed case is recorded as negative or unretrieved for an existing insight and is later used during Library Evolution. This distinction between missing knowledge and misaligned applicability conditions is one of AlphaOPT’s central technical claims.

A common misconception is that answer-only supervision implies weak or purely heuristic learning. In AlphaOPT, answer-only supervision is paired with executable search and solver validation, so the system still requires a concrete program that achieves the correct optimal objective before it can extract reusable experience. Another misconception is that the framework is merely a retrieval-augmented prompting scheme. The continual refinement of applicability conditions through $\mathcal{T}$5 and $\mathcal{T}$6 makes the retrieval semantics themselves objects of learning rather than fixed prompt design.

5. Empirical performance and ablations

The experiments use a cleaned aggregated dataset of 454 problem instances drawn from NLP4LP, NL4OPT, IndustryOR, and MAMO (ComplexLP), spanning LP, MILP, MINLP, IP, NLP, and other optimization types. The split is 70% training and 30% testing, stratified within each dataset, with strict separation so that the library is built only from training tasks. Out-of-distribution evaluation uses LogiOR and OptiBench. The default backbone is GPT-4o with temperature $\mathcal{T}$7, and the metric is success rate, defined by whether the generated optimal value closely matches the ground-truth solution (Kong et al., 21 Oct 2025).

The reported results show strong out-of-distribution behavior. AlphaOPT (full) achieves 51.1% on LogiOR and 91.8% on OptiBench, while AlphaOPT (self-exploration / answer-only) achieves 50.0% and 92.1%, respectively.

Setting	LogiOR	OptiBench
AlphaOPT (full)	51.1%	91.8%
AlphaOPT (self-exploration / answer-only)	50.0%	92.1%

The paper states that AlphaOPT achieves the best OOD performance reported there, and that the abstract’s 7.7% gain over the strongest baseline on OOD OptiBench when trained only on answers is consistent with the detailed comparison of 92.1% for answer-only AlphaOPT against 84.4% for ORThought on OptiBench. In-distribution results are also reported: AlphaOPT (full) reaches 83.6 on NLP4LP, 79.7 on NL4OPT, 60.0 on IndustryOR, and 85.3 on MAMO, while the self-exploration variant reaches 86.3, 79.7, 60.0, and 76.5 on those datasets, respectively (Kong et al., 21 Oct 2025).

A second empirical pattern is steady improvement with more training data and without parameter updates. With training size 100, AlphaOPT reports MicroAvg 83.24%, MacroAvg 65.80%, and library size 38. With 200 training items, these become 85.09%, 69.22%, and 103. With 300 training items, they become 85.21%, 72.12%, and 110. The rise from 65.80% to 72.12% is the basis for the paper’s summary that the system improves from 65% to 72% as training data scales from 100 to 300 items.

The appendix reports three ablations: w/o self-debug, w/o taxonomy, and w/o insight example. On LogiOR / OptiBench, the full system scores 51.08 / 91.81, while the ablations score 35.87 / 89.26, 46.74 / 90.72, and 45.65 / 91.06, respectively. The largest drop comes from removing self-debug, especially on LogiOR, while removing taxonomy degrades retrieval and removing examples reduces the LLM’s ability to operationalize retrieved principles.

6. Interpretability, failure modes, and place among adjacent methods

AlphaOPT’s library is explicitly interpretable. The learned taxonomy comprises 3 main tracks, 14 level-1 labels, and 38 level-2 labels, distributed as 52% Domain Modeling, 30% General Formulation, and 18% Code Implementation (Kong et al., 21 Oct 2025). The paper interprets this distribution as evidence that LLM weakness in optimization is concentrated more in domain-specific structural modeling than in syntax alone.

High-success categories include Code Implementation $\mathcal{T}$8 Solver/API Syntax $\mathcal{T}$9 Strict Inequality, where strict inequalities must be converted into non-strict forms such as $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$0 or $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$1 depending on context, and General Formulation $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$2 Variable Definition $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$3 Explicit Bounds, where natural upper and lower bounds improve feasibility and efficiency. The paper also highlights Sum vs. Makespan Confusion as a characteristic modeling error that is corrected by introducing an auxiliary variable for makespan. High-failure categories include Domain Modeling $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$4 Facility Location $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$5 Fixed Charge (Big-M Linking), whose principle is correct but can be harmful if applied without genuine conditional activation semantics. High-invalid categories include Resource Allocation, Production Planning, and Nonlinear Constraint Handling, where the correct principle may be retrieved but not correctly operationalized.

The paper is explicit about limitations. Retrieval remains a bottleneck; correct retrieval does not guarantee correct execution; the current library scope is limited for complex out-of-distribution logic, especially in LogiOR; dependence on solver-verifiable correctness means that noisy, ambiguous, or expensive solver feedback can degrade learning quality; and the workflow remains computationally more expensive than single-shot prompting because it includes iterative generation, execution, self-exploration, local verification, and condition-refinement replay. The theoretical convergence sketch also relies on idealized assumptions such as boundedness, sufficient exploration, and, in the appendix, equal train/test distributions for the local optimality statement.

Despite the name, AlphaOPT is not a parameter-space numerical optimizer in the sense of alpha-controlled reinforcement learning or alpha-divergence preference optimization. That distinguishes it from methods such as AM-PPO, which uses an adaptive scalar $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$6 to modulate advantages inside PPO (Sane, 21 May 2025), and APO, which interpolates between forward- and reverse-KL behavior through a scheduled Csiszár $$\max_{\ell \in \mathcal{L}} \;\; \mathbb{E}_{t \sim \mathcal{T}}\!\left[ \mathrm{Success}(t \mid \ell) \right] \;-\; \lambda \,\Omega(\ell).$$7-divergence in anchored RLHF (Zixian, 28 Dec 2025). AlphaOPT instead concerns optimization modeling: it externalizes applicability-aware formulation knowledge into a solver-grounded, continually refined experience library. A plausible implication is that its main contribution lies less in improving generic language-model reasoning and more in making the latent “when-to-apply-what” structure of optimization modeling explicit, inspectable, and correctable by both machines and humans.