Collaborative Space Object Detection with Multi-Satellite Viewpoints in LEO Constellations

Abstract: With the growing number of satellites in low Earth orbit (LEO) constellations, the near-Earth space environment has become increasingly congested, making space object detection (SOD) a pressing challenge for space safety and sustainability. To mitigate collision risks and ensure the continuity of space operations, SOD systems must deliver fast and accurate detection under stringent onboard constraints. In this paper, we investigate the potential of multi-viewpoint observation fusion within a deep learning (DL) framework to enhance SOD performance. We design a practical multi-view pipeline and several input representations for feeding multi-view data into YOLO-based detectors. Our experiments show that using multi-view inputs is feasible in most cases and typically produces better results for mAP50 and mAP50-95. For example, in model YOLOv9-m, single-view compared to a three-view fused RGB setting, mAP50 increases from 0.638 to 0.732, while mAP50-95 improves from 0.227 to 0.276. Compared with the single-view setting, the best three-view grayscale configuration improves mAP50 by 36.3% and mAP50-95 by 46.5%. These findings establish multi-view fusion as a viable and effective strategy for SOD, with broad implications for space situational awareness in LEO constellation deployments.

• The paper introduces an early-fusion YOLOv9 pipeline that combines three satellite viewpoints by widening only the first convolutional layer, enabling controlled multi-view detection with minimal architectural changes.
• The experiments show that grayscale fusion delivers the strongest gains, increasing mAP50 by up to 36.3% and mAP50-95 by up to 46.5% over single-view detection while adding only modest inference overhead.
• The results indicate that multi-view collaboration improves localization through complementary geometric cues, but validation on larger real-world datasets, realistic satellite separations, and more than three views remains necessary.

Motivation and problem setting

The paper addresses space object detection (SOD) in low Earth orbit (LEO), where the growing density of satellites makes conjunction assessment and collision avoidance increasingly demanding. SOD must satisfy two conflicting requirements: high detection precision and low latency, under onboard size, weight, and power (SWaP) constraints. Because targets are typically observed at long range against dark backgrounds, they occupy few pixels, and a single observation may not capture sufficient information for reliable detection.

The central hypothesis is that multiple satellites within a cluster, observing the same target from different viewpoints, can supply complementary spatial cues that improve detection when fused into a single deep learning detector. The work builds on YOLOv9 (Wang et al., 2024), whose Programmable Gradient Information (PGI) mitigates information loss for small targets and whose Generalized Efficient Layer Aggregation Network (GELAN) offers lightweight feature aggregation—properties well suited to multi-channel fused inputs.

Method: early-fusion multi-view pipeline

The proposed design is deliberately minimal. Rather than introducing multi-branch architectures or dedicated fusion modules, the authors adopt early fusion at the input stage: $k$ views of the same scene are concatenated along the channel dimension before feature extraction. For three RGB views this yields a $H \times W \times 9$ tensor; the only architectural change is widening the first convolutional layer from 3 to 9 input channels. The backbone, neck, and detection head remain untouched, which permits controlled single-view versus multi-view comparisons.

Two input representations are studied:

• RGB fusion: three RGB views concatenated into a 9-channel feature map.
• Grayscale fusion: each view converted via standard luminance weighting ($0.299R + 0.587G + 0.114B$), then concatenated into a 3-channel tensor that fits YOLO's default input format without any modification.

A key training detail is the repeated stacking strategy. Each view in a cluster takes a turn as the primary view, whose annotations serve as supervision targets; the remaining views act as support views contributing only complementary visual information. This ensures every view's labels participate in training without requiring additional annotation effort. Interpretability is supported through Grad-CAM++ heatmaps overlaid on the primary view.

The system model assumes a satellite formation with one center satellite and support satellites within a radius $r_i$, consistent with the SOD_Clustering dataset (Hu et al., 1 Aug 2025), where each cluster comprises one center and two support satellites.

Experimental setup

Experiments run on an NVIDIA H100 GPU using PyTorch 2.2.2 and CUDA 12.1. The SOD_Clustering dataset contains only 180 images, split 80/20 train/validation, resized to $640 \times 640$, batch size 16, trained up to 1000 epochs with early stopping (patience 300). Four detector configurations are evaluated: YOLOv9-t with and without COCO pretrained weights, YOLOv9-m without pretraining, and GELAN-t without pretraining. All experiments use the three-view configuration imposed by the dataset.

Results

Three-view versus single-view RGB

Three-view fusion improves both mAP50 and mAP50-95 in most configurations:

Model	Setting	mAP50 SV→3V	mAP50-95 SV→3V
YOLOv9-t	w/o pretrain	0.686 → 0.673 (−1.9%)	0.228 → 0.263 (+15.4%)
YOLOv9-t	w/ COCO pretrain	0.668 → 0.758 (+13.5%)	0.276 → 0.321 (+16.3%)
YOLOv9-m	w/o pretrain	0.638 → 0.732 (+14.7%)	0.227 → 0.276 (+21.6%)
GELAN-t	w/o pretrain	0.709 → 0.755 (+6.5%)	0.223 → 0.266 (+19.3%)

Notably, even where mAP50 does not improve (YOLOv9-t without pretraining), mAP50-95 still gains ~15%, indicating that multi-view fusion primarily sharpens localization rather than merely increasing recall. COCO pretraining appears to help the model exploit additional channels more effectively. The largest-capacity model (YOLOv9-m) shows the biggest gains, suggesting capacity is a limiting factor for multi-view learning. Inference time increases by roughly 3–5 ms for the lightweight models but is essentially unchanged for YOLOv9-m, since its baseline computation dominates the extra first-layer cost.

Three-view versus single-view grayscale

Grayscale fusion produces the strongest relative improvements in the paper:

Model	Setting	mAP50 SV→3V	mAP50-95 SV→3V
YOLOv9-t	w/o pretrain	0.590 → 0.804 (+36.3%)	0.200 → 0.293 (+46.5%)
YOLOv9-t	w/ COCO pretrain	0.701 → 0.731 (+4.3%)	0.236 → 0.286 (+21.2%)
YOLOv9-m	w/o pretrain	0.589 → 0.762 (+29.4%)	0.191 → 0.288 (+50.8%)
GELAN-t	w/o pretrain	0.654 → 0.708 (+8.3%)	0.214 → 0.254 (+18.7%)

The best configuration (YOLOv9-t without pretraining) improves mAP50 by 36.3% and mAP50-95 by 46.5% over single-view grayscale. A striking pattern is that grayscale fusion consistently outperforms RGB fusion in relative gain—for YOLOv9-m, the mAP50-95 improvement rises from +21.6% (RGB) to +50.8% (grayscale). The authors attribute this to color reduction forcing the detector to rely on complementary shape and spatial structure across viewpoints, amplifying the benefit of multi-view information. Inference time is nearly identical between single- and three-view grayscale inputs because the channel difference affects only the first convolutional layer.

GELAN-t remains the weakest beneficiary, with its final mAP50-95 capped at 0.254, indicating that lightweight architectures cannot fully exploit multi-view cues—a practical constraint given onboard SWaP budgets.

Communication cost

Transmitting support-view images to the main satellite incurs latency modeled as propagation plus transmission delay over inter-satellite links with capacity $b = 10$ Gbps. Measured latencies fall between approximately 0.02 ms and 0.08 ms per sample across close/mid/far target-distance settings, which is negligible relative to inference time. This supports the feasibility of the collaborative scheme, though it presumes high-capacity free-space optical crosslinks typical of modern constellation hardware.

Limitations and open questions

Several constraints qualify these results. The dataset is very small (180 images) and simulated, so reported absolute mAP values may not transfer to operational conditions. The simulated center-to-support satellite distances are short (roughly 100–1800 m), which the authors acknowledge may not reflect realistic observation geometry. All experiments are restricted to exactly three views because the dataset provides no other configuration, leaving the behavior at larger view counts untested. The evaluation relies on supervised annotations of the primary view only; how performance scales when support views are unlabeled or misregistered is not examined. Finally, the comparison isolates early fusion by design—whether learned or mid-level fusion strategies could yield further gains, or whether the observed advantage persists on real onboard imagery, remains open.

Conclusion

This paper demonstrates that channel-wise early fusion of multi-satellite viewpoints into an otherwise unmodified YOLOv9 detector is a practical route to improved SOD, delivering up to 36.3% mAP50 and 46.5% mAP50-95 relative gains in the grayscale setting while adding negligible communication latency and modest inference overhead. The finding that grayscale fusion benefits more than RGB fusion, and that gains concentrate in the stricter localization metric, indicates the mechanism is complementary geometric information rather than appearance enrichment. Validation on larger, real-world datasets and beyond three views remains necessary before deployment conclusions can be drawn.