Density-Progressive Safety Curriculum (DPSC)

• DPSC is a training regime that systematically increases traffic density and tightens loss-of-separation criteria to safely train air traffic conflict resolution policies.
• It employs a stage-based progression with performance-driven promotion, keeping the underlying diffusion policy, dual Q-critics, and safety mask unchanged.
• Empirical results demonstrate that DPSC reduces convergence steps and significantly lowers NMAC rates, enhancing stability under high-density conditions.

Density-Progressive Safety Curriculum (DPSC) is a curriculum-learning mechanism introduced within Diffusion-AC for autonomous air-traffic Conflict Detection and Resolution (CD&R) in an en-route, 3D area-control sector. It structures training by starting in low-density traffic with relaxed Loss of Separation (LoS) criteria and then progressively increasing both traffic density and the strictness of safety evaluation, with advancement controlled by a rolling success-rate threshold. In "Diffusion-RL Based Air Traffic Conflict Detection and Resolution Method" (li et al., 2 Sep 2025), DPSC is not a separate policy architecture; it is a training regime that changes scenario density, reward triggering, and episode termination while keeping the diffusion policy, dual Q-critics, and state-dependent feasibility mask fixed.

1. Problem setting and rationale

DPSC is defined in the context of CD&R, where an aircraft agent must simultaneously satisfy an efficiency objective and a safety objective. The efficiency term $J_G$ encodes reaching a goal waypoint, minimizing deviation, avoiding boundary violations, and penalizing time, while the safety term penalizes Loss of Separation and Near Mid-Air Collisions (NMACs). The total optimization target is

$\max J_{\text{total}} = J_G + J_S,$

with safety quantified from pairwise interactions between the agent and intruder aircraft (li et al., 2 Sep 2025).

The state-dependent safety quantities are the horizontal distance

$$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$

and the flight-level difference

$$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$

The NMAC penalty is

$$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$

with $d_{\text{NMAC}} = 0.2\,\mathrm{km}$. The LoS penalty is

$$P_{\text{LoS}}(d, FL_{\text{diff}}) = \begin{cases} 0.5, & \text{if } d_{\text{NMAC}} \le d < d_{\text{LoS}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$

and in the physical environment $d_{\text{LoS}} = 10\,\mathrm{km}$.

The safety objective is

$$J_S = - \sum_{k=1}^{N} \int \Big[ P_{\text{NMAC}}(d_{\text{agent},k}(t), FL_{\text{agent},k}(t)) + P_{\text{LoS}}(d_{\text{agent},k}(t), FL_{\text{agent},k}(t)) \Big] dt.$$

The immediate motivation for DPSC is the instability of directly training an expressive diffusion-based policy under high-density traffic and full safety standards. Conventional DRL methods with unimodal policies are described as prone to “policy rigidity / unimodal bias” and “decision deadlocks,” because multiple distinct maneuvers may all be safe and efficient, yet a unimodal policy tends to commit to a single learned high-value maneuver. Diffusion-AC addresses this by modeling policy generation as a reverse denoising process in logit space over 27 discrete joint actions, but the paper states that training such a policy directly in high-density traffic is unstable and sample-inefficient. DPSC is the mechanism used to regularize that training regime.

2. Curriculum structure and formal specification

DPSC consists of 12 curriculum stages, indexed by $k \in \{1,\dots,12\}$, with stage progression determined by both traffic density and safety strictness (li et al., 2 Sep 2025). Its density schedule is

$\max J_{\text{total}} = J_G + J_S,$0

where $\max J_{\text{total}} = J_G + J_S,$1 is the number of active routes and $\max J_{\text{total}} = J_G + J_S,$2 is the number of concurrent intruder aircraft. Training therefore begins at $\max J_{\text{total}} = J_G + J_S,$3 and ends at $\max J_{\text{total}} = J_G + J_S,$4.

Its safety-threshold schedule keeps the NMAC threshold fixed while progressively tightening LoS evaluation:

$\max J_{\text{total}} = J_G + J_S,$5

and

$\max J_{\text{total}} = J_G + J_S,$6

Stage 1 therefore uses $\max J_{\text{total}} = J_G + J_S,$7 km, while stage 12 recovers the operational LoS threshold of $\max J_{\text{total}} = J_G + J_S,$8 km. The paper characterizes early stages as more forgiving because LoS violations are triggered less often in practice.

Progression is performance-based rather than time-based. For each stage $\max J_{\text{total}} = J_G + J_S,$9, a window of $$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$0 episodes is used. If $$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$1 indicates whether episode $$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$2 is a successful deconfliction episode, the rolling success rate is

$$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$3

Promotion from stage $$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$4 to stage $$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$5 occurs only when

$$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$6

Otherwise, training continues at the current stage. This design makes curriculum advancement contingent on maintaining at least $$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$7 success over the most recent 100 episodes.

A central invariant of DPSC is that it does not alter the policy architecture, the 27-dimensional joint categorical action representation, or the action feasibility mask. Only the environment density, the LoS threshold used in reward and termination, and the resulting experience distribution are changed from stage to stage.

3. Reward shaping, safety semantics, and hard-versus-soft constraints

The per-step reward retains the same decomposition throughout the curriculum (li et al., 2 Sep 2025). The efficiency component is

$$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$8

the safety component is

$$d_{\text{agent},k}(t) = \|P_{\text{agent}}(t) - P_k(t)\|$$9

and the total reward is

$$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$0

Under DPSC, the functional form of the reward is unchanged. What changes are the stage-dependent thresholds $$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$1 and $$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$2 that determine when the penalty terms become non-zero. Because $$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$3 is fixed and $$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$4 increases across stages, the effective safety pressure becomes stricter as the curriculum proceeds.

The paper distinguishes hard and soft safety mechanisms. Hard safety is implemented through a state-dependent feasibility mask $$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$5, which encodes aircraft-envelope and immutable procedural constraints. It is applied before softmax in the policy, used in critic target maximization, and used in the teacher distribution. Actions outside $$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$6 therefore receive zero probability. Soft safety is implemented through LoS and NMAC penalties in the reward and through termination conditions. DPSC belongs entirely to this soft layer: it schedules the density of conflicts and the thresholds at which soft penalties and terminations are triggered, but it does not modify the hard action mask.

The safety conditions modulated by DPSC can be summarized as

$$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$7

and

$$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$8

Early stages expose the policy to fewer aircraft and smaller LoS thresholds, which reduces the frequency of near-collision penalties and allows exploration without constant catastrophic penalties. Later stages increase both congestion and the size of the protected zone.

4. Integration with Diffusion-AC

DPSC is integrated into the agent-environment interaction loop rather than into the underlying Diffusion-Actor-Critic update rule (li et al., 2 Sep 2025). At each curriculum stage, the scenario generator is configured with $$FL_{\text{agent},k}(t) = |FL_{\text{agent}}(t) - FL_k(t)|.$$9 routes, $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$0 intruders, $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$1, and $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$2. The environment is then reset repeatedly until the success criterion over the rolling window is satisfied.

Action selection remains the Diffusion-AC procedure: sample $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$3, run reverse denoising conditioned on $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$4 to obtain $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$5, apply the feasibility mask $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$6, transform logits by softmax to obtain $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$7, pick $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$8, decode the selected discrete action into $$P_{\text{NMAC}}(d, FL_{\text{diff}}) = \begin{cases} 1, & \text{if } d < d_{\text{NMAC}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$9, and execute it in the simulator. The environment then returns $d_{\text{NMAC}} = 0.2\,\mathrm{km}$0 using the stage-$d_{\text{NMAC}} = 0.2\,\mathrm{km}$1 reward and termination rules, and the tuple is stored in replay memory.

Critic learning is unchanged. For each experience tuple $d_{\text{NMAC}} = 0.2\,\mathrm{km}$2, the target is

$d_{\text{NMAC}} = 0.2\,\mathrm{km}$3

The teacher distribution for the policy is built from the conservative critic

$d_{\text{NMAC}} = 0.2\,\mathrm{km}$4

as

$d_{\text{NMAC}} = 0.2\,\mathrm{km}$5

The diffusion denoiser is trained to reconstruct logits of $d_{\text{NMAC}} = 0.2\,\mathrm{km}$6 from noised versions. DPSC affects this process only indirectly: it changes the distribution of states, rewards, terminal flags, and thus the Q-values and teacher distributions encountered during training. The paper further notes that the temperature $d_{\text{NMAC}} = 0.2\,\mathrm{km}$7 can be tuned and, in practice, is annealed along with curriculum stage to balance diversity and exploitation.

5. Density progression and empirical effects

The curriculum’s notion of density is tied to the geometry of the simulated airspace and the number of simultaneously active aircraft (li et al., 2 Sep 2025). The base high-density configuration comprises 12 straight-line routes crossing a $d_{\text{NMAC}} = 0.2\,\mathrm{km}$8 square, with up to 3 intruders per route—one per flight level FL0/FL1/FL2—and approximately 36 intruders active at any time. Stage 12 of DPSC matches this full high-density environment.

An ablation in a high-density test with 48 intruders isolates the effect of removing DPSC:

Variant	Rates	Efficiency
Diffusion-AC	Success 94.1%, LoS 2.8%, NMAC 1.2%, Timeout 4.7%	Avg Steps 302
W/O DPSC	Success 87.6%, LoS 6.4%, NMAC 2.9%, Timeout 9.5%	Avg Steps 350

The ablation shows that removing DPSC reduces success by 6.5 percentage points, more than doubles the NMAC rate, increases the LoS rate, nearly doubles the timeout rate, and increases the number of steps required to resolve conflicts. The abstract further summarizes that DPSC reduces convergence steps by about 14% and reduces NMAC incidence by about 59% under peak traffic conditions.

These results are significant because the paper attributes the overall high-density performance of Diffusion-AC not only to multimodal action generation, but also to the training regime that lets the policy acquire that flexibility without destabilization. In the reported most challenging high-density scenarios, Diffusion-AC maintains a success rate of 94.1%. The paper also states that, in combination with multimodal diffusion policies and conservative dual-Q critics, this curriculum is a key ingredient in outperforming PPO, Safe-DQN-X, Rainbow-DQN, and TD3 in high-density CD&R.

6. Relation to curriculum learning and documented limitations

Within the paper’s own framing, DPSC is a specialization of curriculum RL and safe RL rather than a generic curriculum template (li et al., 2 Sep 2025). It differs from a generic easy-to-hard schedule by jointly increasing traffic density and tightening safety thresholds, using a hard promotion gate at $d_{\text{NMAC}} = 0.2\,\mathrm{km}$9 success, while preserving an invariant policy-safety interface through the fixed feasibility mask. In the safe RL taxonomy used by the paper, safety is enforced through a combination of hard masking, reward and termination penalties, conservative dual-Q evaluation, and staged exposure to risk.

A useful comparison can be made to "Evolutionary Curriculum Training for DRL-Based Navigation Systems" (Asselmeier et al., 2023). That work also uses performance-conditioned progression in collision-avoidance training, but its curriculum is organized around environment parameters such as Room Number, Pedestrian Policy, Pedestrian Number, and Pedestrian Speed, and it advances when PerfScore reaches $$P_{\text{LoS}}(d, FL_{\text{diff}}) = \begin{cases} 0.5, & \text{if } d_{\text{NMAC}} \le d < d_{\text{LoS}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$0. DPSC is narrower and more explicitly coupled to aviation safety semantics: its progression variable is traffic density $$P_{\text{LoS}}(d, FL_{\text{diff}}) = \begin{cases} 0.5, & \text{if } d_{\text{NMAC}} \le d < d_{\text{LoS}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$1, its safety schedule is the stagewise LoS threshold $$P_{\text{LoS}}(d, FL_{\text{diff}}) = \begin{cases} 0.5, & \text{if } d_{\text{NMAC}} \le d < d_{\text{LoS}} \text{ and } FL_{\text{diff}} = 0 \ 0, & \text{otherwise}, \end{cases}$$2, and its action mask remains constant across stages.

A second comparison arises with "Curriculum Learning for Safety Alignment" (Kumar et al., 25 May 2026), where safety training is organized by difficulty buckets, competence-based sampling, and staged reference-model updates for DPO. That work is methodologically different—preference optimization for LLMs rather than diffusion RL for aircraft conflict resolution—but it reinforces a broader pattern: safety curricula often become more effective when they are progression-based, performance-gated, and structured so that harder safety cases are introduced only after competence on easier cases is established. This suggests a cross-domain convergence in curriculum design principles, although DPSC itself is defined specifically for CD&R.

The limitations explicitly associated with DPSC derive from the underlying modeling assumptions. The scenario model uses a specific 3-level, 12-route rectangular airspace with stylized intruder generation; the agent is assumed to have perfect state observability of its own state and the 3 nearest intruders; aircraft dynamics are simplified to discrete control increments with fixed step sizes; only a single aircraft is controlled by learning; intruders follow non-adaptive rules; stage promotion assumes that success rate over 100 episodes is a sufficient indicator; and LoS progression is linear in stage index with no adaptive tuning. A plausible implication is that transferring DPSC unchanged to more realistic sector geometries, multi-agent control, or alternative operational modes such as UAM corridors would require re-specifying both the density parameterization and the safety schedule.