Natural Language Dynamic Optimization

• Natural language-guided dynamic optimization is a computational framework where optimization problems are defined and refined using free-form text instead of traditional mathematical formulations.
• It employs iterative prompt engineering and LLM-driven candidate generation to dynamically update constraints and objectives, enhancing adaptability and efficiency.
• Applications span numerical optimization, scheduling, design tuning, and hyperparameter tuning, demonstrating versatility across various complex domains.

Natural language-guided dynamic optimization refers to the family of computational frameworks and methodologies in which optimization processes are interactively specified, manipulated, and steered via natural language (NL) instructions—often using LLMs—rather than explicit mathematical formulas or programmatic interfaces. This paradigm enables both black-box and structured optimization tasks to be dynamically specified, re-specified, and evolved through human-in-the-loop, free-form language descriptions. It finds application in numerical optimization, scheduling, multi-objective design, configuration tuning, and long-context reasoning tasks.

1. Formal Problem Definition and Mathematical Foundations

Central to natural language-guided dynamic optimization is the recasting of traditional optimization objectives and constraints into NL prompts for LLMs. Given an objective function $f(x)$ (possibly non-convex, black-box, or domain-specific), the canonical form is

$\min_{x \in \mathcal{X}_\mathrm{feas}} f(x)$

where $\mathcal{X}_\mathrm{feas} \subseteq \mathcal{X}$ is either fully specified or partially implicit in the language description. In such frameworks, the explicit mathematical constraint functions are rarely written; instead, all constraints, heuristics, and domain knowledge are encoded in natural language and supplied to the LLM as part of a composite prompt. The feasible region may thus be reconfigured in real time by editing the text description, allowing for interactive and dynamic domain modification without changing underlying code or mathematical declarations (Oktavian, 30 Dec 2025).

For time-indexed or dynamic optimization problems, NL-to-symbolic parsing pipelines synthesize temporally-extended models, mapping NL statements such as "subject to evolving capacity constraints" to forms like

$\min_{x_1,\dots,x_T} \sum_{t=1}^T c_t^\top x_t$

$$\text{s.t.}\quad A_t x_t + B_t x_{t-1} \leq b_t \quad \forall t=1,\dots,T$$

with dynamic constraints and parameters mapped automatically from NL (Zhang et al., 2024).

Formally, the LLM samples candidate solutions as $x_t \sim p_\theta(x | P_t)$, where $P_t$ is the constructed prompt at iteration $t$, comprising the static problem description, domain heuristics, and solution history.

2. Core Methodologies and Algorithms

In LLMized optimization, several algorithmic variants are used, all mediated by iterative prompt construction and LLM reasoning. The principal algorithms include:

• Optimization by Prompting (OPRO):

Iteratively constructs a prompt with problem description $D$, NL-encoded constraints, and the last $K$ solution-score pairs. The LLM samples $\min_{x \in \mathcal{X}_\mathrm{feas}} f(x)$0 candidate solutions, which are evaluated externally, and the best update is recorded (Oktavian, 30 Dec 2025).

• LLM + Evolutionary Algorithms (HLMEA):

Maintains a history of elite solutions; at each step, new candidates are proposed via the prompt, evaluated, and top-$\min_{x \in \mathcal{X}_\mathrm{feas}} f(x)$1 are retained as the next elite set.

• LLM + Simulated Annealing (HLMSA):

A state is proposed by the LLM based on the prompt; improvements are always accepted, others are accepted probabilistically according to a temperature, potentially modulated by the LLM.

• Dynamic Constraint Mapping for Robotics:

Utilizes structured models such as Dynamic Grounding Graphs (DGG), learning parametric cost and constraint mappings from NL instructions, yielding optimization problems with problem-specific cost weightings and differentiable constraints (Park et al., 2017).

• Bayesian Optimization with NL-Guided Candidate Selection:

The LLM operates as an interpreter that re-ranks candidate points (from, e.g., a qLogNEHVI batch) to align with a user's free-form NL request, effectively modulating the acquisition function (Niwa et al., 22 Aug 2025).

These methodologies enable the optimization process to be explicitly dependent on, and immediately responsive to, new or evolving NL specifications.

3. Dynamic Re-Specification, Feedback, and Adaptivity

A defining feature is the ability to dynamically alter optimization criteria, constraints, or heuristics during runtime via language alone:

• Constraint Tightening and Relaxation: Modifying language constraints such as changing "PPF<1.5" to "PPF<1.4" steers the LLM's candidate generation towards the updated feasible region on-the-fly. Adaptation is rapid and localized to only the necessary iterations, with LLMs leveraging prior solution-score pairs in adjusting proposals (Oktavian, 30 Dec 2025).
• Objective Modification: Shifting objectives (e.g., adding variance penalties) requires merely rewriting the prompt. The LLM quickly refocuses search (Oktavian, 30 Dec 2025).
• User-Driven NL Requests: Bayesian optimization with cooperative NL supports user interventions such as "Optimize for stability" or "Balance accuracy with speed," which are parsed by the LLM into re-ranked candidate selections (Niwa et al., 22 Aug 2025).
• Directional Feedback in Text-space Optimization: Explicit textual feedback—analogous to gradients—enables rapid convergence in prompt optimization and function maximization, outperforming reward-only and non-directional feedback loops (Nie et al., 2024).
• Validation and Repair: LLM-augmented pipelines include iterative validation and code repair, where failed constraints or infeasibility are automatically diagnosed, patched via NL, and corrected in subsequent iterations (Hu et al., 11 Aug 2025, Zhang et al., 2024).

Empirically, early iterations after a re-specification exhibit increased variance in candidate scores but adapt quickly. When constraints are tightened or conflicting, convergence slows but does not stall, as in-context examples guide the shift in solution space.

4. Prompt Engineering and System Architectures

All frameworks rely on carefully engineered prompts to inject domain knowledge, constraints, and history:

• Prompt Templates: Structured NL templates elaborate variables, ranges, objectives, constraints ("The decision variable x=(x₁,…,xₙ) must satisfy…"), and provide evaluated examples ("x=(x₁=3, x₂=2) → f=7.1"), further guiding in-context learning (Oktavian, 30 Dec 2025).
• Dialogue-Based Refinement: Multi-round interaction modules identify missing model components, ask clarifying questions, and incrementally emit more formal symbolic or code representations (Zhang et al., 2024).
• Retrieval-Augmented Generation (RAG): For dynamic scheduling and code synthesis, retrieved example triplets (NL + formulation + code) seed the LLM’s planning and coding agents, underpinning accurate code generation and constraint specification (Tang et al., 2024).
• Token-Level Dynamic Context Optimization: In long-context settings, NL instructions define the granularity and focus of context compression (keywords, sentences, paragraphs), and are enacted by hybrid causal-bidirectional LLMs employing token critic mechanisms that tag and select the most relevant spans for subsequent generation (Shen et al., 23 May 2025).

The system architecture thus supports both batch-mode candidate exploration and fine-grained, token-level context shaping via NL.

5. Representative Applications and Empirical Results

Natural language-guided dynamic optimization has been validated in a wide range of applications:

Task Domain	Optimization Objective / Operation	Language Role	Empirical Outcome
Convex Optimization	Minimize analytic function	Box-bounds, heuristics in NL	Converge to minimum in 2–3 iterations; 26s runtime (Oktavian, 30 Dec 2025)
Linear Programming	Maximize linear form under inequalities	Constraints/penalties in NL	Near-optimal in 2–3 iterations; self-corrects infeasibilities (Oktavian, 30 Dec 2025)
TSP (n=10)	Shortest tour (permutation)	State as permutations in NL	Best tour length drops to ~290 in 5–6 iters (Oktavian, 30 Dec 2025)
Hyperparameter Tuning	Maximize classifier accuracy	Param bounds and rules in NL	Accuracy climbs from ~0.918 to ~0.955 in 4 iters (Oktavian, 30 Dec 2025)
Nuclear Fuel Lattice	Tradeoff k_inf, PPF, safety constraints	Technical heuristics in NL	Outperforms GA; integrates expert knowledge (Oktavian, 30 Dec 2025)
Power System Scheduling	MILP for unit commitment	Domain schema and policies in NL	100% feasible, optimal schedules; robust to errors via repair loop (Hu et al., 11 Aug 2025)
Robotic Motion Planning	Trajectory generation under NL constraints	Smoothness, collision, target NL	<0.5s end-to-end; 100% success (Park et al., 2017)
Design Optimization (BO)	Multi-objective with user agency	User requests, tradeoffs in NL	HV comparable to automated BO, higher perceived agency (Niwa et al., 22 Aug 2025)
Context Compression	Long-context token selection	Granularity/compression in NL	Up to 290× compression, +40 pts accuracy, 3× speedup (Shen et al., 23 May 2025)

Across domains, LLM-guided optimization excels where constraints or objectives are difficult to formalize or revise algorithmically, and where dynamic, domain-specific adaptation is valuable.

6. Limitations, Challenges, and Directions

Several theoretical, empirical, and practical limitations are highlighted:

• Numerical Competence and Arithmetic Limitations: LLMs are not numerically precise optimizers. For low-dimensional, well-structured problems, classical solvers are faster and more reliable (Oktavian, 30 Dec 2025, Zhang et al., 2024).
• Prompt Dependence and Ambiguity: Quality of solutions hinges on precise and unambiguous NL descriptions; mis-specification or misunderstanding in NL can propagate errors (Tang et al., 2024).
• Scaling and Feasibility: Token/context window limits affect the complexity and scalability of NL-encoded models; RAG and example databases partly mitigate this (Zhang et al., 2024, Tang et al., 2024, Shen et al., 23 May 2025).
• Feedback Quality: In LLM-based optimizers, explicit directional feedback is essential for rapid convergence. In its absence, optimization is slow or stalls (Nie et al., 2024).
• Model Generalizability: Coverage limitations exist for novel domain concepts or constraint types not present in the training data or prompt templates (Hu et al., 11 Aug 2025, Zhang et al., 2024).
• Reliance on External Solvers: High-precision results and numerical guarantees require handing off to established solvers; the LLM’s role is fundamentally as a modeling, steering, or code assistant.

Potential directions include integrating retrieval-augmented or tool-augmented generation, enabling richer scenario branches and visualizations, and further coupling LLM-driven reasoning to solver diagnostics and feedback (Zhang et al., 2024, Hu et al., 11 Aug 2025).

---

The current state-of-the-art demonstrates that natural language-guided dynamic optimization enables non-specialists and experts alike to interactively define, manipulate, and steer complex optimization processes using language alone, with practical impact in domains requiring rapid, domain-evolving, or highly contextualized solution generation (Oktavian, 30 Dec 2025, Zhang et al., 2024, Hu et al., 11 Aug 2025, Tang et al., 2024, Shen et al., 23 May 2025, Niwa et al., 22 Aug 2025, Nie et al., 2024, Park et al., 2017).