EOS-Bench: A Comprehensive Benchmark for Earth Observation Satellite Scheduling

Abstract: Earth observation satellite imaging scheduling is a challenging NP-hard combinatorial optimisation problem central to space mission operations. While next-generation agile Earth observation satellites (EOS) increase operational flexibility, they also significantly raise scheduling complexity. The lack of a unified, open-source benchmark makes it difficult to compare algorithms across studies. This paper introduces EOS-Bench, a comprehensive framework for systematic and reproducible evaluation of scheduling methods. By integrating high-fidelity orbital dynamics and platform constraints, EOS-Bench generates 1,390 scenarios and 13,900 benchmark instances, spanning from small-scale validation cases to large coordination problems with up to 1,000 satellites and 10,000 requests. We further propose a scenario characterisation scheme to quantify structural difficulty based on factors such as opportunity density, task flexibility, conflict intensity, and satellite congestion. A multidimensional evaluation protocol is introduced, assessing performance across five metrics: task profit, completion rate, workload balance, timeliness, and runtime. The framework is evaluated using mixed-integer programming, heuristics, meta-heuristics, and deep reinforcement learning across both agile and non-agile settings. Results show that EOS-Bench effectively distinguishes solver performance across scales and conditions, revealing trade-offs between solution quality and computational efficiency, and providing deeper insight into scenario complexity. EOS-Bench offers a unified and extensible open testbed for advancing research in Earth observation satellite scheduling. The code and data are available at https://github.com/Ethan19YQ/EOS-Bench.

• The paper presents a comprehensive benchmark framework for Earth Observation Satellite Scheduling that provides 1,390 scenarios and 13,900 instances for robust solver evaluation.
• It integrates high-fidelity orbital dynamics, stochastic instance sampling, and scenario characterization descriptors to assess algorithmic performance under varied constraints.
• Experimental results reveal that meta-heuristic and RL-based solvers balance task profit and runtime across scalable, real-world scheduling challenges.

EOS-Bench: A Rigorous Benchmark for Earth Observation Satellite Scheduling

Introduction and Motivation

EOS-Bench establishes a unified, extensible benchmarking framework for the Earth Observation Satellite Scheduling Problem (EOSSP), directly addressing enduring limitations in the EOSSP research landscape. Previous studies have been plagued by inconsistent instance modelling, private datasets, and narrow, nominal metrics—making algorithmic comparison difficult and often non-reproducible. EOS-Bench resolves these limitations by introducing a comprehensive, platform-agnostic library of 1,390 parametric scenarios and 13,900 instances spanning both agile and non-agile architectures, diverse planning horizons (1–168 hours), and constellation configurations from single satellites to mega-constellations of 1,000 assets.

By integrating high-fidelity orbital dynamics, granular platform constraints, and stochastic instance instantiation under each scenario template, EOS-Bench provides a robust experimental substrate. The framework further introduces a scenario characterisation formalism, capturing intrinsic structural difficulty via pre-optimisation descriptors (feasible opportunity density, flexibility, pairwise conflict, congestion), and supports a multi-metric solver evaluation protocol (task profit, completion rate, balance, timeliness, runtime), enabling controlled, transparent cross-method analysis.

Benchmark Framework Design

Scenario and Instance Generation

EOS-Bench formalises the benchmark domain as a structured scenario space, parametrised over fundamental axes: platform type (agile, non-agile), planning horizon, constellation configuration (including real catalog orbits and Walker–Delta synthetic constellations), mission load, target distribution, and further specific parameters. Each scenario $\sigma$ selects a unique combination of these factors; instances are realised by stochastic sampling from target pools and seeds:

Figure 1: Hierarchical organisation and generation mechanism of EOS-Bench; scenario templates are mapped to realisable instances via a stochastic generator, yielding standard and specific libraries.

This design ensures that algorithmic evaluation results are statistically meaningful and systematically reproducible across a parameter sweep of operational settings.

Satellite, Task, and Feasibility Modelling

Satellite dynamics are modelled with physically plausible orbital elements—using real platforms for small constellations and Walker–Delta symmetries for scalability—coupled with platform-specific attitude envelopes and discrete payload parameters (normalised energy/memory rates). For agility, EOS-Bench employs a piecewise linear slewing-time model consistent with state-of-the-art agile scheduling literature, ensuring realistic time-dependent coupling in transition feasibility.

Task generation supports both synthetic (uniform, clustered, hybrid) and real (capital cities) target sets, with multi-objective attributes (profit, priority, duration). The visibility module robustly links orbital geometry and platform constraints to discretised access windows, ensuring precise downstream scheduling feasibility.

All structural data are supplied to solvers in an optimisation-agnostic format, enforcing consistency and solver neutrality.

Scenario Characterisation

Beyond scale descriptors (number of satellites/tasks), EOS-Bench aggregates 10 structural indicators at both the task and satellite level—quantifying feasible opportunity, flexibility, and the spatio-temporal topology of conflicts and congestion. These scenario descriptors enable calibrated difficulty assessment and illuminate why nominally similar instances provoke divergent solver responses.

Figure 2: Merged continuous conflict windows in a representative scenario, for GAOFEN_10R; time periods with no conflicts are elided, exposing only active contention intervals and their conflict types.

Figure 3: Evolution of descriptor values (e.g., opportunity, elasticity, congestion) as constellation size scales, under fixed load and horizon—quantifying the phase transition from congestion- to routing-dominated regimes.

Figure 4: Impact of constellation Walker–Delta geometry on structural descriptors at 50/100 satellite scale; Few-Planes architectures amplify overlap and congestion, whereas Many-Planes configurations balance opportunity with lower timeline contention.

These visual and mathematical tools endow EOS-Bench with interpretive power not available in earlier nominal-only benchmarks.

Unified Evaluation Protocol

EOS-Bench standardises metrics critical for real-world mission efficacy and computational viability: task profit (TP), completion rate (TCR), workload balance (BD), timeliness (TM), and runtime (RT). Separating these dimensions, rather than collapsing into a single score, surfaces practical tradeoffs among velocity, quality, and fairness—critical for mission planning.

Comparative Algorithm Evaluation

EOS-Bench’s experiments rigorously evaluated exact optimisation (MIP), greedy construction, meta-heuristics (GA, SA, ACO), and RL (PPO) instances across small- to large-scale regimes. The results validate EOS-Bench’s discriminative capacity, showcasing:

• In the small-scale regime (e.g., 3 satellites, 100 tasks): MIP and SA/ACO achieve the optimal frontiers for TP and TCR, but with significant runtime escalation as temporal or spatial complexity grows. Greedy heuristics are computationally efficient but yield lower solution quality, especially under congested clustered-target distributions.
• At large scale (e.g., 100 satellites, 2,000 tasks): Meta-heuristics (GA/SA/ACO) dominate in solution quality despite superlinear runtime growth; RL-based solvers offer a runtime/quality balance, outperforming greedy methods but trailing elite meta-heuristics on the most complex instances.

Population-based and iterative solvers demonstrate greater robustness under structurally difficult regimes (dense clustering, heterogeneous resource configurations, low-agility platforms). Meanwhile, EOS-Bench reveals that scenario topology—not raw instance size—predicts when nominally “scalable” heuristics will collapse in the presence of increased timeline contention or opportunity sparsity.

Sensitivity and Structural Analysis

EOS-Bench supports controlled experiments isolating:

• Resource capacity: Workload and profit metrics are resource-bounded under tight budgets; heterogeneous mixes (e.g., mixed high/low capacity satellites) challenge load balancing, exposing the adaptability limits of sequential heuristics versus meta-heuristics/learning-based candidates.
• Manoeuvrability (agility): Higher angular velocities reduce transition bottlenecks and open late-insertion windows, with diminishing returns above a structural threshold dictated by task geometry.
• Constellation geometry: Few-Planes Walker–Delta designs produce excess timeline redundancy and periodic congestion spikes, while Many-Planes distributions lead to spatially balanced, less congested schedules.
• Real-world targets and ground-tracks: Completion rates exhibit concave growth against fleet size, asymptotically saturating as assets exceed spatial demand; meta-heuristics sustain quality at the cost of coordination overhead, while RL maintains constant-latency inference with stable but sub-optimal quality.

Visualisation and Interpretability

EOS-Bench provides interactive HTML5/CesiumJS-based interfaces for spatio-temporal visualisation of satellite trajectories, target demands, and executed schedules, supporting granular investigation of conflict bottlenecks and coordination patterns.

Implications and Prospective Extensions

Practical Significance

EOS-Bench's design, characterised by protocol-agnostic scenario generation and structural characterisation, enables reproducible, fair, and interpretive comparison of new algorithms, verified across a spectrum of realistic operational conditions. The agnostic data interface allows seamless integration of new solver paradigms—meta-heuristics, RL, distributed schedulers, quantum-enhanced methods—and is openly available for third-party adoption and extension.

Theoretical Trajectories

EOS-Bench’s scenario descriptors provide a blueprint for linking instance topology to algorithmic hardness, suggesting future work on predictive modelling of instance difficulty, solver selection, and adaptive benchmarking.

Future Directions

Promising research axes include: regional area/strip-target modelling for coverage scheduling; multi-payload, SAR/hyperspectral extensions with quality-aware/geometry-coupled reward functions; highly heterogeneous, reconfigurable constellations supporting orbital phasing and formation adaptation; autonomy- and uncertainty-centric mission planning with rolling-horizon and online replanning; joint imaging/downlink/gateway scheduling for end-to-end mission optimisation; and formal explainability/audit layers for interpretable scheduling traces—aligning with the increasing operational complexity of next-generation space assets.

Conclusion

EOS-Bench delivers a transformative leap in EOSSP benchmarking rigor, shifting the field from non-reproducible, nominal-case-driven studies to a multi-scenario, multi-metric, structurally-annotated environment. By exposing subtle interplays among opportunity, congestion, platform limitations, and scheduling methodology, EOS-Bench catalyses robust methodological advance and transparent evaluation standards for the remote sensing and scheduling communities.

---

• “EOS-Bench: A Comprehensive Benchmark for Earth Observation Satellite Scheduling” (2604.25782)