Desire-Driven Objective Optimization

Updated 24 June 2026

Desire-driven objective optimization is a framework that infers user preferences to construct dynamic objectives instead of traditional static, handcrafted rules.
It integrates interactive preference elicitation, inverse reinforcement learning, and bilevel neural optimization to align optimization outcomes with true user desires.
Applications span urban map generalization, adaptive control, and multi-agent system design, empirically achieving higher fidelity in matching desired outcomes.

Desire-driven objective optimization encompasses a set of methodologies that directly encode, infer, or elicit user or agent preferences to construct, adapt, or learn optimization objectives such that resultant solutions closely match those preferences. Unlike traditional objective design—which prescribes static weighted sums or hand-crafted rules—these approaches leverage user judgments, observed behavior, or surrogate models to align the optimization targets with what is truly desired. Frameworks in this family span interactive preference elicitation, data-driven inverse reinforcement learning, bilevel neural optimization, multi-agent reasoning over verbal desiderata, and aspiration-satisficing optimal control. This article provides a systematic treatment of desire-driven objective optimization across methodologies, mathematical formulations, and empirical instantiations.

1. Foundational Principles and Problem Structure

Desire-driven objective optimization reframes the classical optimization problem by positing that the "true" objective is unknown and must be constructed, inferred, or steered by user desires or behavioral data. Key formal elements are:

Unknown Latent Utility: Actual user or agent utility $U(s)$ is not given explicitly; instead, only pairwise preferences, choices, or behaviors are accessible.
Multi-criterion Characterization: Candidate solutions are described by vectors of measurable criteria $v(s) = (v_1(s), ..., v_m(s))$ .
Surrogate Objective Construction: The optimization system aims to learn or adapt a tunable objective $f_\theta(s)$ that produces rankings matching $U$ as closely as possible.

Typical problem settings include cartographic generalization, interactive system optimization, multi-objective control, Bayesian hyperparameter tuning, and agent-based simulation. Problem data may take the form of pairwise comparisons, raw interaction trajectories, or scalarized "desirability" assignments (Taillandier et al., 2012, Li et al., 2018, Ma et al., 15 Oct 2025, Bartz-Beielstein, 30 Mar 2025, Li et al., 2020).

2. Interactive and Preference-guided Objective Learning

A prominent class of methods organizes a man–machine loop for iterative preference elicitation and objective updating. The central steps are:

Candidate Generation and Pairwise Comparison: For each problem instance, the solver generates alternative solutions. The user compares solution pairs, expressing preference ( $s_1 \succ s_2$ ), reverse ( $s_2 \succ s_1$ ), or tie.
Adaptive Elicitation Strategy: Pairs are selected systematically to maximize information for model learning, e.g., by varying one or two measures, enforcing measure consistency, or sampling randomly.
Piecewise Linear Objective Induction: The surrogate $f_\theta$ is constructed as a set of regression rules, each parametrized by local linear weights over the measures. The system recursively partitions the measure space whenever a single linear rule cannot explain the user's preferences, using classifiers like RIPPER to define regions $\phi_r(v(s))$ .
Metaheuristic Weight Optimization: Weights within each partition are optimized (e.g., via genetic algorithms) according to a global error metric: the mismatch between user-indicated and model-induced rankings.
Plug-in to Optimization Loop: The learned $f_\theta$ directly replaces the handcrafted objective within the existing search or optimization procedure (Taillandier et al., 2012).

This interaction protocol delivers objectives that match user tastes with high out-of-sample fidelity, as empirically validated in urban map generalization where a three-rule $f_\theta$ retained only 5 out of 50 ranking disagreements on unseen regions.

3. Data-driven Reward Inference and System Policy Optimization

In interactive system domains, the desire-driven paradigm proceeds via inverse reinforcement learning (IRL):

Behavioral Data Assimilation: Collect user trajectories $v(s) = (v_1(s), ..., v_m(s))$ 0 on the interactive system, each $v(s) = (v_1(s), ..., v_m(s))$ 1.
Linear Reward Model and IRL: Assume $v(s) = (v_1(s), ..., v_m(s))$ 2 with unknown $v(s) = (v_1(s), ..., v_m(s))$ 3, and maximize the (regularized) maximum-entropy likelihood

$v(s) = (v_1(s), ..., v_m(s))$ 4

with $v(s) = (v_1(s), ..., v_m(s))$ 5.

Gradient Step via Feature Matching: Update according to the difference between empirical and model feature counts.
Dual MDP Optimization: Re-cast the system MDP into a dual form where the system’s transitions are controlled so as to maximize the value realized by the (estimated optimal) user policy under $v(s) = (v_1(s), ..., v_m(s))$ 6.
Alternating Optimization: Iterate reward inference and environment optimization until user value converges. The Interactive System Optimizer (ISO) implementation achieves 100–300% value improvement in simulation, consistently outperforming fixed-objective and non-adaptive baselines (Li et al., 2018, Li et al., 2020).

This approach generalizes to continuous or high-dimensional domains via adversarial IRL or model-free policy optimization.

4. Multi-objective and Preference-satisficing Formulations

Desire-driven optimization is natural in multi-objective contexts where trade-off surfaces are prominent:

Desirability Function Scalarization: Map each objective $v(s) = (v_1(s), ..., v_m(s))$ 7 into $v(s) = (v_1(s), ..., v_m(s))$ 8 by user-specified spec-limits and shape parameters, then aggregate into a single desirability score $v(s) = (v_1(s), ..., v_m(s))$ 9. This scalarized $f_\theta(s)$ 0 is maximized directly or via surrogate-based Bayesian optimization (Bartz-Beielstein, 30 Mar 2025).
Aspiration-satisfying Optimal Control: In dynamic control, specify aspiration thresholds for secondary objectives. Optimize a primary performance criterion under the constraint that secondary Hamiltonians do not exceed prescribed levels:

$f_\theta(s)$ 1

with aspirations enforced through Hamiltonian inequalities within a sum-of-squares (SOS) iterative learning scheme (Mazouchi et al., 2020).

Bayesian Pareto Front Optimization: Embed the unknown user utility as a scalarization parameter $f_\theta(s)$ 2, elicited once from the DM over a GP-modeled Pareto front. Continue the search with the elicited $f_\theta(s)$ 3 to focus on the most desired compromise (Ungredda et al., 2021, Gharib et al., 2021).

These approaches enable systematic exploration of trade-offs and encode user “desires” via explicit bounds, target values, or elicited scalarizations.

5. End-to-End and Bilevel Objective Learning

Where the mapping from data to the optimization objective is itself learnable and differentiable, bilevel optimization architectures allow the innermost objective to be trained for alignment with observed or desired solutions:

Bilevel Construction: For each data instance, optimize $f_\theta(s)$ 4 in an inner loop, then update $f_\theta(s)$ 5 such that $f_\theta(s)$ 6 matches the ground-truth $f_\theta(s)$ 7 in an outer loop.
Neural Objective Instantiation: Model $f_\theta(s)$ 8 as a neural network, unroll inner optimization for $f_\theta(s)$ 9 steps, and use automatic differentiation or implicit function differentiation for the outer update.
Generalization: Enables the system to learn loss surfaces for tasks where hand-designed objectives are inadequate or poorly aligned with evaluation metrics. Successful applications include toy regression, projection problems, and optical flow estimation, outperforming hand-crafted losses and capturing user- or data-driven desiderata (Jeon et al., 2019).

This class of approaches blurs the distinction between "objective" and "evaluation metric," substituting the learning of the former to match the latter.

6. Language and Multi-Agentic Interpretation of Verbal Desires

Contemporary multi-agentic frameworks ingest free-form verbal preferences and automatically instantiate multi-objective optimization problems:

Natural Language Parsing and Disambiguation: Sequential agent modules analyze ambiguous instructions, clarify intent, and extract relevant entities, interaction modalities, and placement constraints.
Objective and Parameter Selection: Agents map verbal or written desires to predefined objective function libraries, assigning parameters according to context (e.g., user’s position, field-of-view constraints, interaction probabilities).
Solver Integration and Automated Validation: The instantiated multi-objective problem is solved for Pareto-optimal candidates (NSGA-III or equivalent), which are subsequently filtered and validated by a further agent (often a VLM) for maximal alignment with user intent.
Empirical Efficacy: This process enables the elimination of designer-handcrafted weights and the reduction in user adjustment, as shown in UI layout tasks where AutoOptimization achieved a mean of 2.4 adjustments versus 8.5 for manual placement, with statistically significant improvement in workload (Li et al., 13 Feb 2026).

Such pipelines generalize to any domain with parameterizable objectives and amenable to preference elicitation via human or agentic communication.

7. Integrated Cognition, Emotion Modeling, and Bounded Rationality

Recent agent frameworks incorporate emotional and cognitive modeling to simulate boundedly rational desire-driven optimization:

State and Desire Vectorization: Agents maintain vectors encoding both environmental state (income, health, rank) and PAD (pleasure, arousal, dominance) emotional states.
Emotion-to-Desire Mapping: PAD vectors are transformed into normalized desire weights via empirical or rule-based functions (e.g., softmax over affine transformations).
Composite Reward Formulation: Decision-making is performed with respect to a desire-weighted aggregate of outcome changes.
Soft Optimization via Prompt Rewriting and Policy Reweighting: Rather than altering the policy network directly, agents optimize auxiliary distributions over prompts, with action selection governed by softmax-weighted likelihood of achieving desire-weighted reward.
Rationality Constraints and Alignment: The system enforces strict limits on optimization depth, prompt length, and sampling temperature to mirror real-world bounded rationality and prevent pathologies observed in purely reward-maximizing architectures (Ma et al., 15 Oct 2025).

This structured, cognitively inspired loop enables LLM-powered agents to match human-like decision profiles in social simulations.

References

Paper Title	arXiv ID	Core Contribution
Objective Function Designing Led by User Preferences Acquisition	(Taillandier et al., 2012)	Man–machine dialog for interactive objective learning
Learning Data-Driven Objectives to Optimize Interactive Systems	(Li et al., 2018)	IRL-based inference and optimization in interactive systems
Data-driven Dynamic Multi-objective Optimal Control	(Mazouchi et al., 2020)	Aspiration-satisfying RL for MO control
Emotional Cognitive Modeling Framework with Desire-Driven ...	(Ma et al., 15 Oct 2025)	Emotion-driven desire aggregation for LLM agents
Multi-Objective Optimization and Hyperparameter Tuning ...	(Bartz-Beielstein, 30 Mar 2025)	Desirability approach for MO optimization
Neuro-Optimization: Learning Objective Functions ...	(Jeon et al., 2019)	Bilevel NN-based objective learning
Automating UI Optimization through Multi-Agentic Reasoning	(Li et al., 13 Feb 2026)	Multi-agentic verbal-to-quantifiable-MOO parsing
Differentiation of Multi-objective Data-driven Decision Pipeline	(Li et al., 2024)	Differentiable, end-to-end MO objective learning
One Step Preference Elicitation in Multi-Objective Bayesian Opt.	(Ungredda et al., 2021)	Bayesian MO with DM-in-the-loop preference update
Multi-Objective Optimization of a Path-following MPC ...	(Gharib et al., 2021)	BO-based MPC cost tuning for desire-driven trade-offs

Desire-driven objective optimization thus defines a corpus of methods spanning interactive, data-inferred, scalarization, multi-objective, and cognitively constrained schemes, unified by the direct encoding of subjective or agentic desires into the optimization process itself.