Dora: RL for Satellite Network Controller Provisioning

• Dora is a reinforcement learning-based provisioning strategy for hierarchical satellite networks in SAGIN, optimizing controller placement and domain assignments.
• It employs a three-layer architecture with LEO, MEO, and senior MEO nodes and formulates controller provisioning as an MDP solved via PPO.
• Dora outperforms traditional heuristics by achieving around a 10% improvement in provisioning quality while reducing computation time by up to two orders of magnitude.

Searching arXiv for the specified paper and closely related satellite-network controller provisioning work. Dora is a reinforcement learning-based controller provisioning strategy for hierarchical domain-based satellite networks in Space-Air-Ground Integrated Networks (SAGIN). It addresses the problem of deciding how many controllers to use, where to place them, and which Low Earth Orbit (LEO) satellites each controller should manage, under time-varying topology and traffic. The method is embedded in a three-layer domain-based architecture with LEO satellites in the data plane, Medium Earth Orbit (MEO) satellites in the controller plane, and a high-performance MEO node in the senior controller plane. Dora reformulates controller provisioning as a Markov Decision Process (MDP) and uses Proximal Policy Optimization (PPO) to optimize network overhead and average response delay while reducing online computation relative to search-based baselines (Peng et al., 19 Jul 2025).

1. Problem setting and architectural context

Dora was proposed in response to the scalability limits of flat, centralized satellite network management. In large constellations, especially in LEO networks with continuously changing topology and traffic, synchronization overhead, signaling load, and control latency become prohibitive when a ground station or a single centralized controller attempts to manage the full system. Hierarchical domain-based architectures therefore emerge as a more practical alternative, but they introduce a distinct controller provisioning problem: the system must determine which satellites act as controllers, how domains are formed, how satellites are assigned to controllers, and how overhead and response delay are kept low in real time (Peng et al., 19 Jul 2025).

The architecture associated with Dora has three layers.

Layer	Main role	Satellite type
Data plane	Carries traffic and is grouped into domains	LEO
Controller plane	Domain control and intra-domain routing	Selected MEO satellites
Senior controller plane	Inter-domain management and real-time provisioning strategy generation	High-performance MEO node

The use of MEO satellites is a central architectural choice. The paper states that MEO satellites provide more computing power than LEO satellites, lower latency than GEO satellites or ground stations, and better suitability for real-time control than offloaded ground-based management. This combination is intended to deliver both scalability and low-latency control. A plausible implication is that Dora should be read not merely as an isolated RL policy, but as one component of a broader control-plane reorganization for next-generation SAGIN deployments (Peng et al., 19 Jul 2025).

2. System model and optimization objective

The network is modeled in discrete time slots $t = 0,1,2,\dots$, with topology $G_t(S,E)$. The formulation uses $S^L$ for the LEO satellite set, $S^M$ for the MEO satellite set, $H_i$ for the set of LEO nodes controlled by controller $S_i^M$, and $T_t(i,j)$, $f_t(i,j)$ for traffic volume and number of flows between LEO satellites $i$ and $j$. The excerpt presents the link-delay term using link distance $G_t(S,E)$0 and the speed of light $G_t(S,E)$1, indicating a propagation-delay model based on inter-satellite distance (Peng et al., 19 Jul 2025).

The total network overhead $G_t(S,E)$2 is decomposed into synchronization overhead, flow table update overhead, and path computation overhead: $G_t(S,E)$3 Average response delay $G_t(S,E)$4 is built from intra-domain delay $G_t(S,E)$5 and inter-domain delay $G_t(S,E)$6. The paper provides representative formulations for both terms and then averages them over the active flows. The exact excerpted LaTeX is partially corrupted, but the intended construction is explicit: controller placement and domain assignment jointly affect both control overhead and service response delay (Peng et al., 19 Jul 2025).

Dora combines these quantities into a single objective. Overhead and delay are first normalized as $G_t(S,E)$7 and $G_t(S,E)$8, and then combined into a logarithmic score $G_t(S,E)$9 weighted by $S^L$0, where $S^L$1 balances the importance of overhead versus delay. The optimization target is

$S^L$2

The paper notes that the log penalty makes the score sensitive when improvements are small, which is useful because in large systems the normalized metrics are often close to $S^L$3. This suggests that Dora is designed for a regime in which marginal improvements remain operationally meaningful even when the system is already near a constrained performance frontier (Peng et al., 19 Jul 2025).

3. MDP formulation and PPO-based provisioning policy

Dora reformulates controller provisioning as an MDP,

$S^L$4

with state, action, transition, and reward engineered around domain reassignment. The state $S^L$5 contains a normalized traffic matrix, a one-hot encoded initial domain allocation, and the current domain allocation. The policy therefore observes both current traffic conditions and the present controller-assignment context, rather than traffic alone (Peng et al., 19 Jul 2025).

The action space permits simultaneous reassignment of up to $S^L$6 LEO satellites to new MEO controllers: $S^L$7 where each $S^L$8 denotes

$S^L$9

This batch-style action design is an important technical choice. Dora does not optimize one node move at a time; it can perform a bounded multi-node reallocation in one decision step, which reduces the effective horizon of local restructuring (Peng et al., 19 Jul 2025).

The transition is deterministic: $S^M$0 for the resulting new allocation. The reward is defined as a potential difference with penalty: $S^M$1 where $S^M$2 is the discount factor, $S^M$3 is a step penalty weight, and $S^M$4 is a per-action penalty. This reward favors moves that improve the provisioning state while discouraging excessive or costly switching. In functional terms, Dora learns a policy over controller assignments rather than repeatedly solving a fresh combinatorial optimization problem online (Peng et al., 19 Jul 2025).

4. Representation, training pipeline, and computational design

Dora uses PPO, chosen in the paper for stable and efficient policy learning. Its network begins with a compact state encoder built from satellite attributes such as position, traffic load, and controller assignment, followed by a Graph Neural Network (GNN) encoder to capture topology-aware relationships. An Actor-Critic structure is then placed on top of the GNN representation, with both actor and critic implemented as MLPs. The actor outputs action probabilities, and the critic estimates state value. This architecture matches the graph-structured nature of the provisioning problem, where performance depends on connectivity and domain topology rather than only on tabular statistics (Peng et al., 19 Jul 2025).

The training loop is described as follows: initialize policy network $S^M$5 and value network $S^M$6; sample a scenario; interact with the environment for several steps; store transitions in a buffer; update the networks with PPO when the buffer is full; and periodically switch to a new scenario every $S^M$7 episodes. The paper’s algorithm summary also states that training returns a trained policy after repeated scenario sampling, interaction, storage, and PPO update. This establishes a clear separation between training and inference: training can be offloaded to powerful ground infrastructure, whereas inference reduces to a forward pass through the pretrained policy and can be deployed on satellites (Peng et al., 19 Jul 2025).

That separation is the main source of Dora’s computational efficiency. Traditional methods such as GA, K-means, and heuristic regularization algorithms repeatedly search the solution space online. Dora instead learns a mapping from state to provisioning action offline. The paper explicitly claims that this provides high provisioning quality, short computation time, and low computational resource requirements. In the satellite-network setting, where onboard compute is limited and provisioning decisions must often be made in real time, that design choice is as important as the raw objective value (Peng et al., 19 Jul 2025).

5. Experimental configuration and empirical results

The experiments were run on Ubuntu 22.04.4 LTS with an Intel Xeon Gold 6418H CPU, 30 GB RAM, and an RTX 4090 GPU. The simulation stack used Poliastro and SGP4 for satellite orbit and topology generation, NS-3 for traffic generation, and BGP and OSPF for path computation delay modeling. One main constellation setting used LEO satellites at 550 km altitude and MEO satellites at 8000 km altitude. The evaluation covered training dynamics over 15,000 episodes, sensitivity to $S^M$8, scalability from 50 to 1000 LEO satellites, and comparison against HROA and GA + K-means (Peng et al., 19 Jul 2025).

For a constellation with 500 LEO and 20 MEO satellites at $S^M$9, the network score improves from $H_i$0 to $H_i$1, corresponding to a 55.3% improvement in the objective during training. The sensitivity study shows that as $H_i$2 increases, the model places more weight on delay and the measured delay-related behavior changes accordingly, confirming that the objective can trade off overhead and delay. The scalability study reports “consistently high network grades” across 50 to 1000 LEO satellites, no major degradation at larger scales, and polynomial computation growth roughly between first- and second-order scaling with satellite count (Peng et al., 19 Jul 2025).

The benchmark comparison is the paper’s main quantitative claim.

Method	Controller provisioning quality	Computation time
Dora	About 10% better than baselines	10.26 s
GA + K-means	Lower than Dora	1105.25 s
HROA	Lower than Dora	482.27 s

From these reported timings, Dora is about $H_i$3 of GA + K-means computation time and about $H_i$4 of HROA computation time in the cited setting. The abstract and conclusion summarize the result more generally as requiring only $H_i$5 to $H_i$6 of the computation time compared with traditional algorithms, depending on the baseline and scenario. The paper therefore presents Dora as simultaneously improving provisioning quality by about 10% and reducing runtime by roughly one to two orders of magnitude (Peng et al., 19 Jul 2025).

6. Interpretation, significance, and nomenclature

The significance attributed to Dora is that it demonstrates a practical route to controller provisioning in large SAGIN deployments without expensive online search. The paper interprets the method as evidence that learning-based provisioning can outperform classical heuristic and search-based methods, that offloaded training with lightweight inference is well suited to space systems, and that controller provisioning can be treated as a learned decision policy rather than a repeated optimization routine. In that sense, Dora functions both as a specific PPO-based framework and as a proof-of-concept for real-time controller management in large-scale hierarchical satellite networks (Peng et al., 19 Jul 2025).

A common source of confusion is that the string “Dora,” “DoRA,” or “DORA” is used for unrelated research systems in other areas. In arXiv literature, it refers not only to the satellite-network controller provisioning strategy discussed here (Peng et al., 19 Jul 2025), but also to Weight-Decomposed Low-Rank Adaptation in PEFT (Liu et al., 2024), Dynamic Low-Rank Adaptation (Mao et al., 2024), an asynchronous RL system for LLM training (Hu et al., 29 Apr 2026), and other systems in robotics, dialogue, wireless networking, and hardware acceleration. Those works are unrelated in method and domain. Within satellite networking, however, Dora specifically denotes the three-layer hierarchical-domain provisioning strategy built around PPO, GNN-based state encoding, and the overhead-delay objective for SAGIN controller placement and assignment (Peng et al., 19 Jul 2025).