HydraOpt: Multi-Domain Optimization Workflows

• HydraOpt is a multi-domain optimization label covering techniques from LLM adapter merging to CFD tuning, hydropower scheduling, and hydraulic topology optimization.
• In the LLM context, HydraOpt merges multiple low-rank adapters by learning a shared A' and task-specific B', achieving nearly 50% storage reduction with a minimal performance drop.
• It also encompasses advanced workflows in industrial CFD and transformer-based hydropower control, highlighting its broad and practical applicability across diverse engineering challenges.

HydraOpt is a label with multiple technical referents rather than a single canonical method. In the most direct sense, it denotes the adapter-merging technique introduced in “HydraOpt: Navigating the Efficiency-Performance Trade-off of Adapter Merging,” which compresses multiple low-rank adapters for LLMs by learning one shared $A'$ and a small bank of $B'$ components (Ceritli et al., 23 Jul 2025). In several other syntheses, the same label is defined for clarity rather than used by the original papers themselves: it denotes the end-to-end optimization of the Rolls-Royce Hydra industrial CFD application with OP2, a transformer-based framework for multiobjective multireservoir hydropower operation, a simple dynamic-programming hydropower scheduler, and related optimization workflows connected to hydraulic topology optimization and to the Hydra distributed coordinate-descent algorithm (Reguly et al., 2014, Wu et al., 2023, Olofsson et al., 2023, Padmaprabhan et al., 14 Jun 2026, Richtárik et al., 2013).

1. Terminological scope

Within the cited corpus, the label spans several unrelated domains: LLM adaptation, unstructured-mesh CFD, hydropower scheduling, soft-robotic hydraulic design, and large-scale distributed convex optimization. Only one of these works uses “HydraOpt” in the paper title; several others explicitly state that the original paper does not use the term and introduce it only for clarity in an overview (Ceritli et al., 23 Jul 2025, Reguly et al., 2014, Wu et al., 2023, Olofsson et al., 2023, Richtárik et al., 2013, Padmaprabhan et al., 14 Jun 2026).

Usage	Source	Core content
Adapter merging for LLMs	(Ceritli et al., 23 Jul 2025)	Shared $A'$, $M$ learned $B'$, tunable storage–performance trade-off
Industrial CFD optimization	(Reguly et al., 2014)	OP2 conversion of Hydra, coloring, halo exchanges, CPU/GPU/MPI tuning
Multireservoir hydropower DRL	(Wu et al., 2023)	Transformer encoder–decoder, scalarized multiobjective control
Hydropower scheduling	(Olofsson et al., 2023)	Dynamic programming with forecasts and switching costs
Related Hydra-derived uses	(Richtárik et al., 2013, Padmaprabhan et al., 14 Jun 2026)	Distributed coordinate descent; hydraulic topology optimization

A common source of confusion is that these usages are not minor variants of one algorithm. They address different decision variables, objective functions, and computational substrates. This suggests that “HydraOpt” functions less as a stable technical term than as a label attached to optimization workflows involving Hydra, hydraulic systems, or both.

2. HydraOpt as adapter merging for LLMs

In its primary named sense, HydraOpt is a data-free model-merging method for low-rank adapters such as LoRA and VeRA. For a base weight matrix $W \in \mathbb{R}^{m \times n}$, the LoRA update is written as $\Delta W = (\alpha/r)AB^\top$, while the paper’s layer notation uses $\Delta W = BA$, with $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$ (Ceritli et al., 23 Jul 2025). The storage problem is immediate: storing $B'$0 task-specific adapters requires approximately $B'$1 parameters per adapted layer. HydraOpt exploits the empirical asymmetry that “A-side” parameters tend to be similar across tasks, whereas “B-side” parameters diverge and encode task-specific information.

Its central objective learns one shared $B'$2, $B'$3 learned components $B'$4, and assignment coefficients $B'$5 for tasks $B'$6: $B'$7 Here $B'$8 is a distance function, with MAE as the default; Frobenius norm, cosine similarity, and MSE are also evaluated. The temperature $B'$9 controls how close the softmax assignment becomes to one-hot. The special case $A'$0 removes the assignment vectors and uses $A'$1 directly for task $A'$2.

Training is performed per adapted weight type. The method initializes $A'$3, $A'$4, and $A'$5, then minimizes the loss with AdamW at learning rate $A'$6, batch size $A'$7, for $A'$8 epochs. After training, $A'$9 is discarded, so inference uses the same computational structure as LoRA and incurs no extra latency. Storage becomes

$M$0

with the assignment vectors absent at inference time. If $M$1, the asymptotic storage reduction approaches $M$2 when many adapters are merged.

The empirical results define the method’s position. On 5 English adapters with Llama-3.2-1B, LoRA attains an average score of $M$3 at $M$4 storage, whereas HydraOpt($M$5) attains $M$6 at $M$7 storage, a $M$8 drop with an approximately $M$9 reduction. Across 40 tasks from 5 applications and 8 languages, LoRA averages $B'$0 at $B'$1 storage and HydraOpt($B'$2) averages $B'$3 at $B'$4 storage. Across languages, HydraOpt($B'$5) reaches $B'$6 at $B'$7 storage. Across 40-way merges, HydraOpt($B'$8) reaches $B'$9 at $W \in \mathbb{R}^{m \times n}$0 storage. The abstract summarizes the broad trend as a $W \in \mathbb{R}^{m \times n}$1 storage reduction with a $W \in \mathbb{R}^{m \times n}$2–$W \in \mathbb{R}^{m \times n}$3 performance drop, while outperforming TA, TIES, DARE, and DARE-TIES at similar or slightly worse storage efficiency. The paper does not provide formal approximation bounds, and it identifies highly dissimilar adapters and excessively small $W \in \mathbb{R}^{m \times n}$4 as failure modes for the shared-$W \in \mathbb{R}^{m \times n}$5 assumption.

3. HydraOpt as OP2-based optimization of the Hydra CFD code

In the CFD literature represented here, “HydraOpt” is defined as the end-to-end optimization of the Rolls-Royce Hydra industrial CFD application using the OP2 high-level active library. Hydra is a full-scale industrial code for turbomachinery design, solves the Reynolds-Averaged Navier–Stokes equations, typically employs a 5-stage Runge–Kutta time-marching scheme with multigrid and block-Jacobi preconditioning, and operates on highly detailed unstructured meshes with sets such as nodes, edges, and faces or cells (Reguly et al., 2014). The production code contains more than 300 parallel loops, exceeds 50K lines of code, and was originally Fortran 77 with OPlus and an MPI owner-compute model on single-threaded CPUs.

OP2 supplies a high-level API centered on sets, maps, data arrays called op_dats, and op_par_loop. Access descriptors such as OP_READ, OP_WRITE, and OP_INC encode data intent, allowing OP2 to construct safe execution plans that use graph coloring for indirect updates, halo exchanges for distributed memory, and device-specific data staging or layout selection. A single OP2 source can target MPI, OpenMP, CUDA, MPI+OpenMP, and MPI+CUDA back ends.

The port revealed two major code-generation issues. First, the initial OP2-generated version was approximately $W \in \mathbb{R}^{m \times n}$6 slower than the hand-tuned OPlus code on Ruby; co-locating the user kernel and loop and enabling inlining recovered approximately $W \in \mathbb{R}^{m \times n}$7. Second, VTune profiling showed that a heavy loop spent up to $W \in \mathbb{R}^{m \times n}$8 of instructions on Fortran assumed-shape indexing overhead; introducing a wrapper to compute index mappings once per loop eliminated the per-iteration overhead and matched OPlus performance on a single node. Because OP2 owns op_dats and maps, it also enabled renumbering with Scotch Gibbs–Poole–Stockmeyer, partitioning with PTScotch or ParMetis, and AoS-to-SoA layout changes.

The optimization results were architecture-specific but systematic. On Ruby, mesh renumbering improved single-node CPU performance by approximately $W \in \mathbb{R}^{m \times n}$9, and PTScotch improved approximately $\Delta W = (\alpha/r)AB^\top$0 over RCB. On Kepler K20 GPUs, switching high-dimensional op_dats from AoS to SoA improved single-GPU performance by approximately $\Delta W = (\alpha/r)AB^\top$1, and block-size plus register tuning added approximately $\Delta W = (\alpha/r)AB^\top$2. After these changes, a single K20 GPU was approximately $\Delta W = (\alpha/r)AB^\top$3 faster than the original OPlus CPU and approximately $\Delta W = (\alpha/r)AB^\top$4 faster than OP2 MPI on CPUs; two K20 GPUs with MPI delivered approximately $\Delta W = (\alpha/r)AB^\top$5 speedup over OP2 CPU and approximately $\Delta W = (\alpha/r)AB^\top$6 over OPlus. At scale, OP2 MPI-only with PTScotch scaled well to 128 nodes on HECToR and was approximately $\Delta W = (\alpha/r)AB^\top$7 faster than OPlus at 32 nodes. The broader significance lies in developer productivity: generator-level fixes, coloring, halo formation, partitioning, and retargeting from Fermi-style shared-memory staging to Kepler cache-reliant execution were implemented in the OP2 framework rather than by rewriting all $\Delta W = (\alpha/r)AB^\top$8 loops.

4. HydraOpt as transformer-based multireservoir hydropower optimization

A separate synthesis uses HydraOpt to denote a modular framework for multiobjective optimization of multireservoir hydropower operations derived from transformer-based deep reinforcement learning. The formulation is an episodic Markov decision process over $\Delta W = (\alpha/r)AB^\top$9 discrete periods, monthly in the case study, with reservoir states including storage $\Delta W = BA$0, elevation $\Delta W = BA$1, inflow $\Delta W = BA$2, ecological target flow $\Delta W = BA$3, and operational bounds, together with residential-area demand and benefit variables (Wu et al., 2023). Actions comprise power-generation release $\Delta W = BA$4, a binary supply decision $\Delta W = BA$5, and supply flow $\Delta W = BA$6. The transition law is the reservoir mass balance

$\Delta W = BA$7

The three objectives are hydropower generation, ecological protection, and water-supply benefit. Hydropower uses

$\Delta W = BA$8

ecological deviation uses

$\Delta W = BA$9

and water-supply benefit uses

$B \in \mathbb{R}^{d \times r}$0

The reward is a weighted normalized scalarization over 171 subproblems generated by enumerating weight vectors with step $B \in \mathbb{R}^{d \times r}$1. If operational constraints are violated, the reward is set to zero.

The policy architecture uses multihead attention in an encoder and a decoder with multireservoir attention. The attention kernel is the standard scaled dot-product form,

$B \in \mathbb{R}^{d \times r}$2

A two-stage embedding strategy first constructs reservoir embeddings from $B \in \mathbb{R}^{d \times r}$3, then water-supply embeddings from $B \in \mathbb{R}^{d \times r}$4, elevation, distance, and cumulative prior supply. Decoding proceeds sequentially across time, reservoirs, and residential areas. Training uses REINFORCE with baseline, PyTorch 1.11, Adam, learning rate $B \in \mathbb{R}^{d \times r}$5 for epochs $B \in \mathbb{R}^{d \times r}$6 and $B \in \mathbb{R}^{d \times r}$7 for epochs $B \in \mathbb{R}^{d \times r}$8, batch size $B \in \mathbb{R}^{d \times r}$9, embedding size $A \in \mathbb{R}^{r \times k}$0, 8 attention heads, 5 maximum epochs, and 200 iterations per epoch.

The case study covers Lake Powell and Lake Mead, five downstream states, and monthly periods with $A \in \mathbb{R}^{r \times k}$1. Against NSGA-III, the two-stage transformer-based DRL achieves $A \in \mathbb{R}^{r \times k}$2 electricity generation, $A \in \mathbb{R}^{r \times k}$3 AAPFD, and $A \in \mathbb{R}^{r \times k}$4 water-supply revenue. Against Direct T-DRL, it achieves $A \in \mathbb{R}^{r \times k}$5 electricity generation and $A \in \mathbb{R}^{r \times k}$6 AAPFD. The synthesis also records limitations: inflows are treated as exogenous, upstream–downstream routing is not explicit, the head model $A \in \mathbb{R}^{r \times k}$7 is only denoted generically, and final statistical confidence intervals across independent runs are not reported.

5. HydraOpt-associated hydropower scheduling literatures

A hydropower-oriented use of the label denotes a simple dynamic-programming approach for plant management under forecasted inflows. The problem includes a dam with a reservoir and a continuously adjustable unit, and also a run-of-river plant with two units and negligible storage (Olofsson et al., 2023). The objective is to maximize net profit from electricity generation over a one-year horizon $A \in \mathbb{R}^{r \times k}$8 days, with constant electricity price $A \in \mathbb{R}^{r \times k}$9, running costs, and switching costs between operating modes. In the dam case, the reservoir follows

$B'$00

and the unit efficiency curve is

$B'$01

with $B'$02 and $B'$03. The approximate Bellman recursion uses a deterministic estimate $B'$04 built from a 7-day historical running average, a short-term forecast window $B'$05, and exponential mean reversion with default half-life $B'$06 days. With default parameters $B'$07 days, $B'$08, and $B'$09 days, the method achieves on average $B'$10 of the perfect-foresight optimum for 2015–2022. In the run-of-river comparison, the paper reports that the simple DP often finds the true optimum when forecasts are sufficiently rich, whereas the PDE-based optimal switching method is more stable when forecasts are shorter or weaker.

A second hydropower optimization line in the provided corpus formulates deterministic short-term scheduling for cascades with reservoirs, plants, canals, pipes, and hydraulic delays (Gardini et al., 2016). The general discrete objective is

$B'$11

with reservoir mass balance, storage bounds, terminal targets, and time-of-concentration delays represented by delay-split coefficients. Depending on whether the water-to-energy coefficient $B'$12 is treated as constant, piecewise linear, or nonlinear in head, the resulting optimization class becomes LP, MILP, or NLP. The numerical example uses $B'$13 reservoirs and $B'$14 plants over 24 hourly intervals. The LP case solved by MATLAB linprog required about $B'$15 seconds, and the NLP case solved by fmincon required about $B'$16 seconds. In both tests, no spill occurred and production shifted toward high-price hours. This suggests that, within hydropower scheduling, the HydraOpt label has been attached both to lightweight forecast-driven DP control and to deterministic cascade scheduling with standard mathematical-programming formulations.

6. Related Hydra-derived optimization usages

The Hydra distributed coordinate-descent method is another source of HydraOpt terminology in secondary descriptions. Hydra solves composite convex problems

$B'$17

with smooth convex $B'$18 and separable convex $B'$19, under a column-wise partition of coordinates across $B'$20 computing nodes (Richtárik et al., 2013). At each iteration, each node samples $B'$21 coordinates from its local partition and performs the one-dimensional proximal update

$B'$22

The safe stepsize arises from an Expected Separable Overapproximation bound with $B'$23, where $B'$24 depends on the global interaction parameter $B'$25 and $B'$26 depends on the cross-partition parameter $B'$27. In the strongly convex case, the iteration bound is

$B'$28

The large-scale experiment uses a 3 TB LASSO instance on a Cray XE6 with 128 nodes, 4 MPI processes per node, 8 OpenMP threads per process, and 4,096 cores in total. Under the ASL-FP communication regime, the objective gap is reduced by 25 orders of magnitude in under 30 minutes.

A hydraulics-oriented synthesis also maps HydraOpt onto topology optimization for a hydraulic soft fish tail. The optimization domain is a symmetric half of a PneuNet chamber discretized with $B'$29 bilinear finite elements; the structural stiffness uses SIMP with penalty $B'$30, the density filter radius is $B'$31, and projection continuation drives $B'$32 up to $B'$33 (Padmaprabhan et al., 14 Jun 2026). The hydraulic field is modeled by Darcy flow with a drainage term,

$B'$34

and nodal forces are mapped by $B'$35. The robust optimization constrains the blueprint volume fraction with $B'$36 and the eroded-design strain energy with $B'$37, and is solved with MMA for at most 400 iterations. Under identical hydraulic pressure of 3 kPa, the optimized 2D chamber achieves $B'$38 larger deformation than the rectangular counterpart. In 3D ABAQUS validation, tip displacement $B'$39 reaches $B'$40 mm at 5 kPa, $B'$41 mm at 10 kPa, $B'$42 mm at 15 kPa, and $B'$43 mm at 20 kPa. Together, these examples indicate that the HydraOpt label can denote either optimization algorithms named Hydra, or hydraulic optimization workflows that have no direct relation to the adapter-merging method.

7. Comparative significance

Across its usages, HydraOpt consistently denotes an optimization layer added on top of an existing structured model: low-rank adapters for LLMs, the Hydra CFD solver, reservoir operation equations, plant-operation mode switching, or FE-based hydraulic actuation. The underlying mathematical objects, however, differ sharply. The adapter-merging version is a data-free reconstruction problem over low-rank matrices with explicit storage formulas and accuracy–efficiency knobs $B'$44 and $B'$45 (Ceritli et al., 23 Jul 2025). The OP2-based CFD version is a performance-portability workflow over unstructured-mesh loops, where correctness depends on coloring and halo management and performance depends on layout, partitioning, and architecture-specific back ends (Reguly et al., 2014). The hydropower versions formulate either scalarized multiobjective control with transformer-based policy gradients, deterministic Bellman recursions with forecast surrogates, or deterministic LP/MILP/NLP scheduling on cascades (Wu et al., 2023, Olofsson et al., 2023, Gardini et al., 2016). The Hydra algorithm is instead a distributed optimization method for convex loss minimization (Richtárik et al., 2013), while the soft-tail workflow is a robust topology-optimization problem coupling Darcy flow and structural FE (Padmaprabhan et al., 14 Jun 2026).

A plausible implication is that any technical use of “HydraOpt” requires immediate disambiguation. Without that step, the term can refer to a 2025 adapter-merging method, a 2014 OP2 acceleration campaign for industrial CFD, a 2023 transformer-based hydropower controller, a simple DP scheduler for hydropower plants, a deterministic industrial hydropower optimizer, a 2013 distributed coordinate-descent method, or a 2026 hydraulic topology-optimization workflow. For researchers, the practical consequence is that the surrounding notation—$B'$46 for adapters, op_par_loop and op_dats for CFD, $B'$47, $B'$48, and AAPFD for hydropower, or $B'$49 and $B'$50 for soft robotics—must determine which HydraOpt is under discussion.