Communication-Efficient Collaborative LLM Inference over LEO Satellite Networks

Abstract: Low Earth orbit (LEO) satellites play an essential role in intelligent Earth observation by leveraging artificial intelligence models. However, limited onboard memory and excessive inference delay prevent the practical deployment of LLMs on a single satellite. In this paper, we propose a communication-efficient collaborative LLM inference scheme for LEO satellite networks. Specifically, the entire LLM is split into multiple sub-models, with each deployed on a satellite, thereby enabling collaborative LLM inference via exchanging intermediate activations between satellites. The proposed scheme also leverages the pipeline parallelism mechanism that overlaps sub-model inference with intermediate activation transmission, thereby reducing LLM inference delay. An adaptive activation compression scheme is designed to mitigate cumulative errors from multi-stage model splitting while preserving inference accuracy. Furthermore, we formulate the LLM inference delay minimization problem by jointly optimizing model splitting and compression ratios under onboard memory and inference accuracy constraints. The problem is transformed into a shortest-path search problem over a directed acyclic graph that edge weights explicitly quantify the inference delay induced by model splitting and compression strategies, which is solved via a modified A Star-based search algorithm. Extensive simulation results indicate that the proposed solution can reduce inference delay by up to 42% and communication overhead by up to 71% compared to state-of-the-art benchmarks, while maintaining the inference accuracy loss of less than 1%.

• The paper presents a novel framework that partitions large LLMs across satellites to overcome hardware constraints and enable real-time inference.
• It employs pipeline parallelism and a Gumbel-mask guided adaptive compression module to reduce delay by 42% and communication overhead by 71%.
• Experimental evaluations show minimal accuracy loss (<1%) and efficient optimization convergence, supporting robust Earth observation applications.

Communication-Efficient Collaborative LLM Inference in LEO Satellite Networks

Introduction and Context

The deployment of LLMs, such as Vision Transformers (ViTs), for intelligent Earth observation from LEO satellites is severely constrained by onboard hardware—primarily memory and computation. Prior techniques based on model compression alone (pruning, quantization, sparsification) offer only partial solutions, frequently inducing accuracy degradations incompatible with downstream requirements. Recent efforts in model splitting enable partial model deployment but amplify inter-node communication, especially in heterogeneous multi-satellite networks. This paper systematically addresses these bottlenecks by proposing a collaborative inference framework optimized for communication and computational efficiency in satellite constellations.

Figure 1: Schematic overview of collaborative LLM inference in a LEO satellite network, highlighting model splitting and inter-satellite communication.

Technical Contributions

The core components of the proposed system are as follows:

1. LLM Model Splitting: The complete LLM is partitioned at both coarse and fine-grained layer levels, resulting in sub-models deployed across a chain of computing-enabled satellites. This approach addresses the infeasibility of single-satellite deployment for large models.
2. Pipeline Parallelism: Inference is executed with overlapping computation and communication via parallel threading, maximizing satellite resource utilization and minimizing idle times as illustrated by the pipeline parallelism process.

   Figure 2: Illustration of the pipeline parallelism computation for split models across satellites.

3. Adaptive Activation Compression: To limit the bandwidth overhead from inter-satellite transmission of activations, the framework introduces a Gumbel-mask guided sparsification module paired with quantization and entropy coding. The mask is differentiable and end-to-end trainable, promoting dynamic, task-relevant selection of activations, robust against spatial and semantic variability in inputs.

Formulation and Optimization

The system jointly optimizes:

• Layer-to-satellite assignment (model splitting)
• Stage-wise activation compression ratios

The joint objective is to minimize total inference delay, subject to individual satellite memory constraints and a tight lower bound on inference accuracy. The full mixed-integer nonlinear program is recast as a shortest-path search over a DAG: nodes encode split points, edges represent assignments with corresponding compression, and edge costs are computed as the composite of computation and transmission delay.

A two-level optimization strategy is deployed:

• Outer loop: Explores assignment paths using a modified A* algorithm, evaluating feasible splits given resource constraints.
• Inner loop: Solves per-edge continuous subproblems to determine the optimal compression ratio at each inter-satellite link, constrained by accuracy and memory.

The formal complexity is $\mathcal{O}\left(\binom{L}{K-1}(N+1)^{K-1}K\right)$, where $L$ is the total number of layers, $K$ satellites, and $N$ the compression discretization.

Experimental Evaluation

Latency and Throughput

Comprehensive simulation and hardware-in-the-loop experiments using ViT models (from ViT-B to ViT-G) and the EuroSAT and RESISC45 datasets validate the system.

• Inference delay is reduced by up to 42% (relative to ground-only and single-satellite baselines) as satellite-side splitting and compression shift computation and communication bottlenecks (Figures 3–6).
• Communication overhead is reduced by up to 71%, even with increased node counts and higher split granularity (Figure 3).

  Figure 4: Inference delay as a function of input image resolution.

  

  Figure 5: Inference delay with varying S2G transmission rates.

  

  Figure 6: Inference delay under different satellite participation counts.

  

  Figure 7: Inference delay comparison across multiple deployment and optimization strategies.

  

  Figure 3: Aggregate communication overhead for each inference scheme.

Compression Efficacy and Ablation

The adaptive compression achieves $>25\times$ reduction in activation size (sparsification + quantization + entropy coding), as demonstrated in the ablation experiments, with negligible incremental accuracy loss (Figure 8).

Figure 8: Ablation study on the sequential efficacy of sparsification, quantization, and entropy coding.

Accuracy and Robustness

• Accuracy drop is always $<$1% compared to the uncompressed baseline, robust across varying compression ratios, model scales, data types, and layer split locations (Tables; Figures 9, 10).
• The proposed Gumbel-mask compression outperforms Top-k selection in terms of accuracy retention, especially for smaller ViT variants where naive token dropping is highly lossy.

  Figure 9: Accuracy evolution on EuroSAT and RESISC45 datasets during model training with different schemes.

  

  Figure 10: Validation accuracy distribution for various model splitting strategies around the uncompromised baseline.

Convergence and Optimization

• The joint optimization converges efficiently across all tested network scales (Figure 11).
• Compared to heuristic and naïve uniform splitting, the proposed strategy achieves a doubling of delay reduction for large-scale ViTs—highlighting the benefit of coordinated, resource-aware partitioning (Figure 12).

  Figure 11: Convergence plot of the optimization algorithm w.r.t. number of satellites.

  

  Figure 12: Total inference delay under various model splitting strategies.

Broader Implications and Future Directions

This methodology enables resource-constrained, real-time AI inference onboard satellite constellations, which is crucial for latency-critical tasks (e.g., disaster response, dynamic monitoring) and alleviates downlink congestion. By integrating model splitting with adaptive, layer-specific compression and pipeline scheduling, the solution bridges the hardware limitations of LEO satellites and the computational demands of LLMs.

Practical adoption will require further research on the dynamic adaptation of splits and compression in response to time-varying connectivity and workload heterogeneity, as well as robust extension to hierarchical multi-orbit (LEO–MEO–GEO) networked inference. Additionally, integration with decentralized multi-task learning and on-orbit federated updates represents an immediate follow-up.

Conclusion

The paper presents a modular, optimization-driven framework for collaborative, communication-efficient LLM inference in LEO satellite networks. The approach combines layerwise partitioning, adaptive activation compression, and pipeline parallelism within a DAG-based optimization landscape, achieving substantial reductions in delay and network usage while tightly controlling accuracy loss. These results highlight a concrete path for deploying advanced AI workloads in edge-space environments, with direct impact on future satellite AI, groundstation integration, and real-time, distributed inference systems.