Hypernetwork-Conditioned Reinforcement Learning for Robust Control of Fixed-Wing Aircraft under Actuator Failures

Abstract: This paper presents a reinforcement learning-based path-following controller for a fixed-wing small uncrewed aircraft system (sUAS) that is robust to certain actuator failures. The controller is conditioned on a parameterization of actuator faults using hypernetwork-based adaptation. We consider parameter-efficient formulations based on Feature-wise Linear Modulation (FiLM) and Low-Rank Adaptation (LoRA), trained using proximal policy optimization. We demonstrate that hypernetwork-conditioned policies can improve robustness compared to standard multilayer perceptron policies. In particular, hypernetwork-conditioned policies generalize effectively to time-varying actuator failure modes not encountered during training. The approach is validated through high-fidelity simulations, using a realistic six-degree-of-freedom fixed-wing aircraft model.

• The paper introduces hypernetwork-conditioned policy adaptation via FiLM and LoRA for robust fixed-wing aircraft control under actuator failures.
• It leverages real-time fault representation to modulate policy behavior, achieving lower Mean/Max Path Errors compared to standard MLP controllers in both static and dynamic fault scenarios.
• Experimental results in high-fidelity simulations demonstrate improved stability and safety, highlighting the approach’s practical deployability on embedded systems.

Hypernetwork-Conditioned Reinforcement Learning for Robust Fixed-Wing Aircraft Control Under Actuator Failures

Problem Formulation and Motivation

Fixed-wing sUAS control faces acute robustness challenges due to actuator failures, which induce abrupt changes in the system's dynamics and demand controllers capable of deploying fundamentally distinct strategies depending on fault characteristics. Standard MLP-based RL policies, which encode all regimes with shared parameters, exhibit pronounced gradient interference and overfit phenomena, especially with time-varying, nonstationary faults. This work explores hypernetwork-based policy conditioning, introducing parameter-efficient adaptation mechanisms—FiLM and LoRA—that map explicit actuator failure information to policy adaptations. The central idea is joint end-to-end training of both the policy and the adaptation module via PPO, enabling direct, real-time modulation of control behavior according to actuator configuration.

Hypernetwork Architectures and Adaptation Strategies

Hypernetworks condition policies by generating adaptation parameters in response to actuator fault vectors ($\boldsymbol{\lambda}$), representing both binary fault status and normalized stuck deflection levels for relevant actuators. The main network for the control policy is a feedforward structure; adaptation is achieved via:

• FiLM: Applies elementwise feature-wise linear modulation (scaling and shifting) to hidden activations. The hypernetwork generates these transformations, providing a low-dimensional adaptation that alters the activations in response to actuator fault characterization.
• LoRA: Introduces low-rank updates to main weight matrices, using hypernetwork-generated rank-limited parameter vectors. The expressive capacity is managed via the adaptation rank ($n_r$), with higher ranks yielding greater robustness but increasing computational complexity.

Both approaches minimize the dimensionality of adaptation, avoiding the inefficiencies and training instability of full-weight generation.

Experimental Setup and Evaluation Protocol

The evaluation uses high-fidelity CZ-150 sUAS simulation with realistic six-DoF nonlinear dynamics, wind disturbances, actuator delays, sensor noise, and stochastic aerodynamic perturbations. The failure parameterization encompasses discrete and interpolated stuck actuator positions and onset times, enabling controlled explorations of static and dynamic faults. Policies are assessed in distribution (static failures) and out-of-distribution (flutter/oscillatory, time-varying failures), with performance quantified via Mean Path Error (MPE) and Maximum Path Error (MaxPE) for both average and worst-case episodes.

Robustness and Generalization Results

Hypernetwork-conditioned policies exhibit substantial robustness gains over baseline MLP controllers in demanding fault conditions. For static failures, FiLM + hyper-conditioned critic (HC) and LoRA (up to $n_r=64$) policies constrain worst-case static rudder errors to $\sim$22 m, whereas the MLP reaches 36.8 m. More notably, under rudder flutter—a scenario never seen during training—MLP policies fail catastrophically, exhibiting MaxPE values up to 159.9 m and high variability; hypernetwork policies maintain MaxPE $<$30 m and notably reduced standard deviation.

Figure 1: Average MaxPE for static actuator failures across architectures, highlighting comparable error profiles.

Under flutter failures, the divergence between MLP and hypernetwork architectures is pronounced. MLP generalization collapses for large, positive actuator deflections, while FiLM and LoRA architectures bound errors effectively. These results refute the claim that single-policy architectures can robustly interpolate in high-dimensional failure spaces.

Figure 2: Example rudder flutter signal used to stress-test generalization to unseen actuator failures.

Value Function Conditioning and Architectural Sensitivity

The application of hypernetwork conditioning to the value function (critic) exhibits disparate effects across adaptation mechanisms. FiLM-based architectures benefit substantially—increasing robustness by 40–50% under rudder failures—while LoRA (particularly at lower ranks) degrades when HC is used. This indicates that low-rank adaptation, when applied to both actor and critic networks, induces optimization instability and high-variance gradients. Rank selection in LoRA is also highly sensitive: performance improves with increased $n_r$, but non-monotonic instability manifests at $n_r=48$, highlighting the need for architectural and initialization tuning.

Lipschitz Regularity and Expressivity

Empirical estimation of the Lipschitz constant ($\mathcal{L}$) reveals a correlation between policy regularity and robust tracking. In LoRA-based policies, increasing $n_r$ monotonically decreases $\mathcal{L}$ and increases robustness. FiLM-based policies also benefit from higher main network expressivity, with diminished performance and increased $n_r$0 for smaller architectures. The results motivate use of spectral normalization during training to constrain policy sensitivity and enforce regularity.

Computational Profile and Practical Implications

Parameter-efficient hypernetwork conditioning maintains compute costs on par with MLP architectures (10$n_r$1–10$n_r$2 FLOPs per actor forward pass), with total policy parameters well below the threshold for practical deployment on embedded processors. Training time per PPO iteration remains dominated by simulation rollout collection; network size increases only marginally impact computational requirements. Hyper-conditioning the value function does not affect inference cost, further supporting deployability.

Qualitative Simulation Comparisons

Worst-case simulation episodes provide further validation. The MLP policy under rudder flutter exhibits excessive roll/pitch, poor altitude control, and path deviation approaching 40 m; recovery occurs only once the actuator returns to nominal operation.

Figure 3: State and control histories for a MLP WC episode under rudder flutter, compared against the FiLM + HC policy.

The FiLM + HC policy maintains tighter tracking, using aggressive aileron compensation and exploiting roll-to-yaw coupling, maintaining both stability and reduced path error under identical environmental conditions.

Figure 4: State and control histories for a FiLM + HC WC episode under rudder flutter, compared against the MLP policy.

Conclusion

Hypernetwork-conditioned reinforcement learning, implemented via FiLM and LoRA, enables robust control of fixed-wing aircraft in the presence of complex actuator failures. These policies generalize effectively to unseen, time-varying faults, outperforming conventional MLP architectures in both error magnitude and stability. Sensitivity to adaptation rank (LoRA), main network expressivity (FiLM), and critic conditioning requires careful architecture optimization. Theoretical implications suggest further exploration of spectral normalization for Lipschitz regularity. Practical deployment is fully viable given parameter efficiency and compute requirements. Future directions include flight-test validation and extension to wider classes of control conditioning signals and adaptation architectures.