Interactive Pareto navigation for deep multi-task learning

Abstract: In multi-task learning, handling an increasing number of objectives can quickly become challenging, both in terms of the computational resources and the decision maker's capacity to choose appropriate trade-offs. A widely used approach is thus to aggregate the individual losses in a single loss function by a weighted sum. This often fails to capture either the decision maker's preferences as a result of the shape of the Pareto front, or requires multiple adjustments and computations which becomes prohibitively expensive in deep learning applications. To address these issues, we introduce a novel framework, Preference Pareto Exploration (PPE), which enforces the decision maker's preferences while accounting for the geometry of the Pareto set in an interactive exploration process. PPE is based on a predictor-corrector method that performs predictor steps tangential to the manifold of Pareto-optimal solutions, following the decision maker's preference. The subsequent corrector step results in a new trade-off reflecting this preference. To avoid explicit Hessian computations when characterizing the tangent space of the manifold, we employ a Krylov subspace method that relies solely on matrix-vector products. These products can be efficiently obtained via automatic differentiation, ensuring both efficiency and robustness throughout the optimization process. The method's functionality and performance are demonstrated using both toy problems and examples from deep learning.

• The paper introduces PPE, a novel framework that integrates interactive preference guidance with multi-objective optimization for deep learning.
• It leverages a predictor–corrector continuation approach with Krylov subspace solvers to avoid costly Hessian computations, enhancing scalability.
• Empirical evaluations on MultiMNIST and UCI Census show PPE outperforms traditional weighted sum and Pareto MTL methods in efficiency and accuracy.

Interactive Pareto Navigation for Deep Multi-Task Learning

Introduction and Problem Context

Multi-task learning (MTL) in deep neural networks often requires simultaneous optimization of multiple objective functions. Classical scalarization techniques, such as weighted sums of losses, offer limited expressiveness and struggle particularly with many-objective problems (MaOPs, $m>3$), as the Pareto front grows exponentially in complexity and dimensionality. Moreover, such approaches cannot effectively encode dynamic, interactive user preferences for trade-off exploration along the Pareto front, especially as the number of objectives increases. Existing alternatives like Multi-Gradient Descent Algorithm (MGDA) capture Pareto criticality but do not support interactive preference articulation or efficient navigation for high-dimensional problems due to computational cost (e.g., explicit Hessian calculation). The presented work addresses these deficiencies by proposing the Preference Pareto Exploration framework (PPE).

Theoretical Framework: Preference Pareto Exploration (PPE)

PPE enables interactive, preference-guided navigation of the Pareto front in deep MTL. It models the Pareto set as a differentiable manifold, leveraging first-order optimality (KKT) conditions to characterize critical points. The exploration adopts a predictor–corrector continuation perspective: predictor steps move along the manifold tangent in a user-specified preference direction; a corrector step projects the iterate onto the Pareto set via MGDA.

Computation of Tangent Directions: PPE avoids explicit Hessian computations, which are prohibitive in deep learning, by using a Krylov subspace solver with matrix-free Hessian-vector products obtained via automatic differentiation. This is critical for scalability to high-dimensional deep networks.

Preference Integration: PPE incorporates a decision maker’s (user’s) partial preference vector $\pi$ per predictor step. Preferences can target arbitrary subsets of objectives and specify desired improvement or relaxation (allowing for both prioritized minimization and acceptable increases in loss components). The preference direction is projected onto the tangent space of the Pareto manifold, yielding a feasible search vector consistent with the geometry of Pareto optimality.

Figure 1: A Pareto-optimal solution with its tangent vector (blue) on the Pareto set manifold, illustrating how PPE computes preference-guided tangent directions for exploration.

PPE Algorithm

1. Initialization: Compute an initial Pareto critical solution via MGDA or another multi-objective solver.
2. Preference Elicitation: The user provides a preference vector $\pi$ over a subset $\mathcal{I}$ of objectives.
3. Predictor Step:
   • Construct tangent vectors to the manifold using linearly independent $\beta$ vectors.
   • Determine the projected, preference-respecting search direction using efficient Krylov-based solvers.
   • Propose a new parameter vector by an update in the tangent direction (step size $\eta$).
4. Corrector Step: Project the predictor step back onto the Pareto manifold (using MGDA) to regain Pareto criticality.
5. Iteration: Repeat steps 2–4, updating preferences as desired.

This process yields a sequence of Pareto optimal solutions reflecting user-driven trade-off navigation, supporting efficient search even in large-scale deep learning contexts.

Empirical Evaluation

Benchmark Problems and Toy Examples

PPE is evaluated on classic MOP test functions, including DTLZ suites, with 3–5 objectives. Iterative preference changes demonstrate that PPE traces the Pareto front in directions aligned with evolving user preferences, producing well-spread, Pareto-optimal solutions not constrained by convexity, and supporting many-objective exploration.

Figure 2: Radar plot of 20 Pareto-optimal points and associated weights $\alpha^*$ found by PPE for 5-objective DTLZ3, visualizing coverage and preference-driven navigation.

Deep Learning Tasks: MultiMNIST and UCI Census Income

On realistic DNN-based MTL scenarios, the framework demonstrates:

• MultiMNIST: Integration of losses corresponding to digit, fashion, and Japanese character classification. PPE efficiently tracks the front across these tasks, producing a diverse portfolio of models with different trade-off characteristics as the user updates preference weights.

  Figure 3: Example from the MultiMNIST dataset, illustrating multitask image input requiring joint label prediction on three sub-tasks.

• UCI Census Income: Multi-task binary classification extends to 3 and 5 objectives (education, marital status, age, sex, ethnicity). PPE produces well-distributed Pareto-optimal classifiers, responsive to real-time trade-off adjustments by the decision maker.

  Figure 4: Radar plot of PPE solutions and corresponding optimal weights $\alpha^*$ for the 5-objective UCI Census Income training set.

  

  Figure 5: Radar plot for the 5-objective UCI Census Income test set, highlighting generalization of Pareto-optimal solutions.

Comparison to Baselines

• Weighted Sum (WS): PPE solutions strongly dominate WS in the objective space, even when equivalent or greater compute budget is allocated to WS. This demonstrates that preference-guided continuation avoids the well-known non-convexity/gap issues of scalarization for Pareto set recovery.
• Pareto Multi-Task Learning (Pareto MTL): PPE yields a more accurate and efficient traversal of the Pareto front, produces solutions better aligned to user preferences, and incurs lower compute burden compared to hypervolume-based and other alternative Pareto MTL methods. Computational times for each Pareto point post-initialization are significantly reduced compared to both WS and Pareto MTL methods.

Implications and Future Directions

Practical Implications

PPE provides a scalable, interactive toolkit for human-in-the-loop multi-objective deep learning. It enables decision makers to actively interrogate and traverse the Pareto set, choosing models that reflect situational priorities. This is particularly valuable in real-world scenarios (e.g., fairness–accuracy trade-offs, multi-domain adaptation, resource-constrained deployment) where automated, fixed-weight optimization is insufficient.

Theoretical Implications

By operationalizing predictor–corrector continuation on the Pareto manifold and integrating preference learning, this work links classic nonlinear optimization theory with state-of-the-art deep learning. Efficient, Hessian-free tangent computation closes a critical gap for practical, scalable active multi-objective model selection.

Future Directions

Extensions of PPE can target:

• Incorporation of uncertainty and model robustness objectives.
• Application in discrete optimization (e.g., neural architecture search under multiple constraints (Peitz et al., 2024)).
• Visualization and interpretability techniques to facilitate more intuitive preference articulation by non-expert users.
• Automated adaptation of preferences in response to dynamic external conditions.

Conclusion

The Preference Pareto Exploration framework advances interactive, scalable Pareto manifold navigation for multi-task deep learning. By combining differentiable multi-objective optimization, efficient Krylov subspace methods, and direct preference integration, PPE supports dynamic, human-guided exploration of trade-offs, yielding Pareto-optimal solutions with strong empirical and practical performance across benchmarks and real-world DL tasks. The decoupling of computational cost from network parameterization and the capacity for active decision making mark this as a substantive methodological contribution to MTL and multi-objective deep learning (2606.19521).