Residual-Controlled Douglas--Rachford Splitting for Differentiable Solver Layers

Abstract: Differentiable solver layers embed constrained optimization into end-to-end learning systems, but fixed-depth unrolling must trade off solution quality, feasibility, and computational budget. We propose Residual-Controlled Douglas--Rachford Splitting (RCDRS), a differentiable solver layer for conic linear programs. RCDRS treats an unrolled solver as a feedback-controlled dynamical system, where a causal controller adapts the relaxation and objective-drive parameters while preserving the projection-splitting structure of Douglas--Rachford splitting. Theoretically, we show that each fixed admissible block remains an averaged relaxed DRS operator and admits finite-step fixed-point residual bounds. We further analyze safeguarded time-varying rollouts as summable perturbations of a limiting averaged operator, and recover terminal primal-dual diagnostics from the final splitting state. Experiments on mixed-cone benchmarks and engineering applications show that RCDRS improves solution quality, feasibility and downstream decision performance. The code is available at https://anonymous.4open.science/r/RC-DRS-180C/.

• The paper introduces Residual-Controlled Douglas–Rachford Splitting (RCDRS), a GRU-based feedback controller that adapts relaxation and objective-drive parameters using solver residuals while preserving DRS projections and structure.
• The method provides convergence analysis for safeguarded time-varying controls and outperforms fixed and open-loop baselines, reducing SOCP objective error roughly sevenfold and winning across 36 benchmark settings.
• RCDRS improves downstream performance in decision-focused learning and energy dispatch, but its guarantees remain limited for unconstrained controls, quadratic-objective extensions, cross-depth generalization, and large-scale systems.

Motivation and problem setting

Differentiable solver layers embed constrained optimization into end-to-end learning pipelines, but fixed-depth unrolling forces a trade-off among solution quality, feasibility, and computational budget. The paper addresses this trade-off for conic linear programs (CLPs) of the form $\min_x c^\top x$ subject to $Ax=b$, $x\in\mathcal K$, where $\mathcal K$ is a Cartesian product of projection-computable closed convex cones. Fixed-depth unrolled Douglas–Rachford splitting (DRS) with static parameters cannot adapt across instances or along a trajectory: early iterations may benefit from aggressive objective progress while later iterations require feasibility recovery and terminal stabilization, and an open-loop learned schedule cannot react to observed residuals.

The proposed method, Residual-Controlled Douglas–Rachford Splitting (RCDRS), treats the unrolled solver as a finite-horizon feedback-controlled dynamical system. A causal controller observes normalized residual, action-history, objective-progress, and time-budget features from the current trajectory and adapts two transition variables: the relaxation parameter $\alpha_k\in(0,2)$ and the objective-drive parameter $\beta_k>0$. Crucially, the controller modifies only these two scalars; the affine projection, cone projection, relaxation, and dual-correction structure of DRS is preserved intact.

Method

The controlled transition replaces the fixed objective shift $-\gamma c$ in standard relaxed DRS with the adaptive normalized shift $-\beta_k \bar c$, where $\bar c = c/(\|c\|_2+\varepsilon_c)$. After a $K$-step rollout, the primal decision is $Ax=b$0, which lies in $Ax=b$1 by construction since the cone projection is applied at every iteration. The retained splitting state also yields a terminal diagnostic readout: the slack $Ax=b$2 with $Ax=b$3, and a least-squares equality multiplier $Ax=b$4.

The controller is a lightweight GRU encoder followed by a control head and an admissibility map, consuming a ten-dimensional feature vector built from log-compressed residuals ($Ax=b$5, $Ax=b$6, $Ax=b$7, $Ax=b$8), signed objective progress via $Ax=b$9, previous-action encodings, and normalized time indices. Two variants are studied: RCDRS-NoEnv maps raw outputs directly to admissible ranges, while RCDRS-Env restricts controls around a validation-selected base action through a decaying envelope $x\in\mathcal K$0 with $x\in\mathcal K$1, ensuring $x\in\mathcal K$2. Training uses a terminal-oriented self-supervised objective combining affine feasibility, splitting consistency, iterate movement, a baseline-relative objective penalty against a fixed-core reference rollout at the same depth, action smoothness, and an optional dominance term. No optimal solutions, optimal values, or KKT multipliers are used as labels; reference optima are used only for validation and evaluation.

Theoretical analysis

The operator-theoretic results are structured in three parts. First, for any fixed admissible pair $x\in\mathcal K$3, the paper shows that the transition is exactly a relaxed DRS operator for a positively rescaled but solution-equivalent CLP: the scaled program has the same primal solution set, the state variables realize the DRS shadow iteration in $x\in\mathcal K$4, and the operator $x\in\mathcal K$5 is $x\in\mathcal K$6-averaged and nonexpansive. This yields a terminal (not merely best-iterate) fixed-point residual bound of order $x\in\mathcal K$7, and the terminal movement loss used in training is shown to be a computable surrogate for this residual decay.

Second, safeguarded time-varying rollouts are analyzed as summable perturbations of the limiting averaged operator $x\in\mathcal K$8. Using local Lipschitz dependence of $x\in\mathcal K$9 on $\mathcal K$0 on bounded trajectories, the envelope converts parameter deviations into summable perturbations, giving a best-iterate bound $\mathcal K$1 and a terminal residual bound whose square is $\mathcal K$2 when $\mathcal K$3. This guarantee applies only to RCDRS-Env; RCDRS-NoEnv retains only the fixed-block properties, a restriction the authors state explicitly.

Third, the terminal primal–dual readout is consistent: the recovered slack satisfies $\mathcal K$4 and $\mathcal K$5 by the normal-cone property of the final projection residual, and any cluster point of bounded terminal readouts whose primal, dual, and gap residuals vanish is primal–dual optimal. Auxiliary results connect the training losses to distance-to-solution under a local error-bound assumption, which is stated as an assumption rather than proved.

Experimental findings

Experiments cover fixed-budget conic optimization on SOCP and mixed-cone benchmarks (nonnegative, second-order, rotated second-order, and PSD blocks), decision-focused learning (DFL), and forecast-aware energy dispatch on a six-bus network, all under a strict same-depth protocol with three seeds.

On hard SOCP instances at depth $\mathcal K$6, both RCDRS variants dominate all baselines—including validation-tuned Grid-DRS, spectral S-ADMM, Anderson-accelerated AA-DRS, learned L-PDHG, and open-loop Layerwise-DRS—across objective error and all three KKT-style residuals; RCDRS-NoEnv attains an objective error of $\mathcal K$7 versus $\mathcal K$8 for Grid-DRS, roughly a sevenfold reduction. On mixed-cone instances the variants split the best metrics between them, reflecting a trade-off between aggressive objective progress and safeguarded primal–dual behavior. An aggregate sweep over 36 family–scale–depth settings reports that the best RCDRS variant wins the positive gap in all 36 settings, with a mean gap ratio of 33.1% relative to the strongest classical baseline.

The ablations attribute most of the gain to closed-loop feedback and the objective-drive variable: replacing the GRU feedback controller with an open-loop layerwise schedule degrades mixed-cone objective error from 0.0156 to 0.1032, and $\mathcal K$9-only control captures most of the improvement over the fixed core (objective error 0.0046 vs. 0.0866 on SOCP). Removing the consistency loss substantially degrades feasibility, particularly on mixed-cone problems.

In DFL, jointly fine-tuned RCDRS achieves the lowest regret (0.0406 for RCDRS-NoEnv-J vs. 0.0511 for fixed DRS and 0.1053 for predict-then-optimize) and lowest decision distance at $\alpha_k\in(0,2)$0, with gains persisting across depths $\alpha_k\in(0,2)$1. In energy dispatch, RCDRS-Env reduces the operating-cost gap from 0.029 (Grid-DRS) to $\alpha_k\in(0,2)$2 at $\alpha_k\in(0,2)$3 and cuts load shedding roughly fourfold, while maintaining a perfect operational pass rate. These improvements carry a measurable cost: controller evaluation raises per-instance runtime from about 0.028 ms to 0.074 ms in the dispatch setting, though absolute overhead remains below 0.1 ms in the batched GPU configuration.

Limitations and open questions

The authors identify several constraints plainly. The time-varying convergence analysis covers only the enveloped variant; the unconstrained controller enjoys no rollout-level guarantee beyond fixed-block properties. The dispatch experiment extends the method to quadratic objectives via a gradient-driven shift, and the operator-theoretic guarantees are explicitly not claimed for this extension. The controller is trained for a prescribed depth and problem distribution, so cross-depth generalization and out-of-distribution robustness remain open. The implementation requires projection-computable cones and efficient affine projections, which may limit scalability to very large sparse systems. The distance-to-solution corollary additionally depends on a local error-bound assumption that is not verified for the benchmark families.

Conclusion

RCDRS demonstrates that residual-feedback control of splitting parameters can materially improve fixed-budget conic solving without abandoning the interpretable structure of DRS. The combination of solution-preserving averaged-operator semantics, summable-perturbation analysis of safeguarded rollouts, self-supervised terminal training, and consistent primal–dual diagnostics provides a coherent account of why closed-loop adaptation outperforms fixed and open-loop schedules across synthetic optimization, decision-focused learning, and operational dispatch tasks.