Progressive Semantic Communication for Efficient Edge-Cloud Vision-Language Models

Abstract: Deploying Vision-LLMs (VLMs) on edge devices remains challenging due to their substantial computational and memory demands, which exceed the capabilities of resource-constrained embedded platforms. Conversely, fully offloading inference to the cloud is often impractical in bandwidth-limited environments, where transmitting raw visual data introduces substantial latency overhead. While recent edge-cloud collaborative architectures attempt to partition VLM workloads across devices, they typically rely on transmitting fixed-size representations, lacking adaptability to dynamic network conditions and failing to fully exploit semantic redundancy. In this paper, we propose a progressive semantic communication framework for edge-cloud VLM inference, using a Meta AutoEncoder that compresses visual tokens into adaptive, progressively refinable representations, enabling plug-and-play deployment with off-the-shelf VLMs without additional fine-tuning. This design allows flexible transmission at different information levels, providing a controllable trade-off between communication cost and semantic fidelity. We implement a full end-to-end edge-cloud system comprising an embedded NXP i.MX95 platform and a GPU server, communicating over bandwidth-constrained networks. Experimental results show that, at 1 Mbps uplink, the proposed progressive scheme significantly reduces network latency compared to full-edge and full-cloud solutions, while maintaining high semantic consistency even under high compression. The implementation code will be released upon publication at https://github.com/open-ep/ProSemComVLM.

• The paper introduces a progressive semantic communication framework using a Meta AutoEncoder that encodes visual inputs into an ordered token sequence.
• The system enables adaptive transmission by using a cloud-side semantic quality metric to balance latency and semantic fidelity.
• Empirical results show up to a 75% reduction in transmission delay with minimal semantic degradation, optimizing edge-cloud VLM performance.

Progressive Semantic Communication for Efficient Edge-Cloud Vision-LLMs

Introduction and Problem Context

The deployment of large-scale Vision-LLMs (VLMs) on resource-constrained edge devices is a central challenge in modern multimodal AI due to their substantial computational and storage requirements. While cloud-centric inference can relieve local constraints, transmitting raw visual data imposes unacceptable latency and bandwidth overhead, especially in real-time and bandwidth-limited applications. Prior edge–cloud collaborative approaches primarily rely on fixed-size representations, offering limited adaptivity to changing network conditions, and lack mechanisms for progressive or semantic-aware communication control.

Methodological Advances

This work presents a progressive semantic communication framework for edge–cloud VLMs that introduces a Meta AutoEncoder (MetaAE) capable of encoding visual inputs into progressively refinable representations. The key innovation is structuring the latent space as an ordered sequence of tokens, where initial prefixes yield coarse semantic content and subsequent token increments refine the representation. This design enables:

• Partial and progressive transmission: The system can transmit just enough latent content to satisfy semantic fidelity requirements, then incrementally refine the result upon request.
• Semantic quality-aware control: A feedback mechanism on the cloud evaluates a learned semantic quality metric, adaptively determining if additional transmission is necessary.
• Plug-and-play compatibility: The approach does not require VLM backbone modifications or fine-tuning, enabling integration with off-the-shelf models.

The systematic structure of the latent representation is illustrated in (Figure 1).

Figure 1: Progressively refinable latent representation via MetaAE, where ordered token prefixes provide monotonic reconstruction improvements, supporting fine-grained transmission/bandwidth trade-offs.

The complete operational loop is visualized in (Figure 2): the edge transmits incrementally larger latent prefixes, stopping transmission as soon as the semantic quality at the cloud reaches a task-specific threshold.

Figure 2: System overview of the proposed progressive edge–cloud semantic communication framework, enabling adaptive incremental transmission and server-side semantic quality monitoring.

Meta AutoEncoder and Training Protocol

The MetaAE consists of a transformer-based encoder and decoder. The encoder converts high-dimensional vision latents to an ordered token sequence, with chunk ordering reflecting information granularity. At each iteration, a prefix masking module selects a subset of prefix tokens, mimicking progressive transmission in deployment. The decoder reconstructs the original latent from masked inputs and includes a learnable error token for semantic quality estimation.

Prefix-masked training induces an asymmetric optimization gradient: earlier tokens are involved in reconstruction for all subsequent prefix levels, compelling them to encode broad, coarse semantic content, while later tokens focus on incrementally finer details. This approach is theoretically justified by showing that mask-based training enforces a nested, monotonic refinement property. An implementation example for prefix masking is provided in (Figure 3).

Figure 3: Example PyTorch code for prefix masking, supporting efficient simulation of progressive transmission during training.

Closed-Loop Transmission and Semantic Quality Metric

The cloud-side decoder computes two outputs: a reconstructed latent and a predicted reconstruction error. This error is exponentiated to create a normalized quality score,

$$q(\ell) = \exp(-\hat{e}(\ell)), \quad q(\ell) \in [0, 1]$$

where $\ell$ denotes the length of the transmitted prefix. Transmission proceeds iteratively: the cloud evaluates $q(\ell)$ against a threshold $\epsilon$ and requests further chunks only if necessary, achieving closed-loop adaptive transmission (Figure 4).

Figure 4: Progressive transmission workflow: the cloud adaptively requests further latent chunks based on online semantic quality estimation.

The quality score is highly correlated with downstream task-level semantic preservation, as demonstrated empirically via high Spearman’s rank correlation with captioning output fidelity.

Experimental Setup

The system is validated on a real-world testbed: an NXP i.MX95 edge platform with hardware NPU, communicating over a 1 Mbps bandwidth-throttled link to a server with an RTX 2080 GPU. The backbone VLM is the SmolVLM-256M, a quantized 256M-parameter vision-LLM. The evaluation measures include:

• Semantic Consistency (SC): Cosine similarity in embedding space between ground-truth VLM outputs and the output generated after progressive compressed transmission.
• Latency: Total system delay under various transmission configurations and LTL prefix ratios.

SC and delay trade-offs are summarized in (Figure 5): substantial delay reductions are achieved with minimal semantic degradation, demonstrating efficient compression/utility trade-offs.

Figure 5: Semantic consistency (SC) and transmission delay as a function of latent prefix (LTL) ratio, highlighting substantial delay savings at low SC loss.

Empirical Results

At a 25% LTL ratio, end-to-end transmission delay is reduced by approximately 75% relative to baseline, while SC degradation is less than 5% across benchmark datasets (COCO Captioning, POPE). The proposed quality metric exhibits nearly perfect monotonicity with true SC, facilitating reliable control over transmission/accuracy trade-offs.

In direct comparison with Full-Edge and Full-Cloud configurations, the progressive Edge–Cloud system achieves the lowest overall latency (6.94s vs. 7.98s and 9.32s, respectively, at 1 Mbps). The MetaAE computation overhead is negligible (<0.35%) compared to the total inference cycle. Measured power traces confirm matched NPU utilization envelopes and sizable reductions in edge-side energy expenditures (Figure 6 and Figure 7).

Figure 6: On-board power profile during a Full-Edge inference cycle, indicating NPU and LLM compute bursts.

Figure 7: On-board power profile during Edge–Cloud inference, showing processing, transmission, and cloud offloading regimes.

Practical and Theoretical Implications

This progressive semantic communication paradigm achieves a strict Pareto improvement in iterative transmission cost over non-progressive baselines, scaling transmission cost linearly rather than quadratically with transmission levels. It removes the need for task- or network-specific heuristic coding and introduces a general mechanism for semantic-aware network adaptation that is compatible with future VLM architectures.

The framework enables operators to select operating points along the communication-fidelity curve dynamically, thus facilitating low-latency deployment in real-world robotic, mobile, and IoT applications under variable network constraints. The generality of the prefix-masked autoencoding protocol opens further avenues for hierarchical multimodal compression, transmission with explicit UEP, and feedback-driven interactive inference.

Conclusion

The presented framework establishes a practical and theoretically justified approach to progressive, semantic-aware, closed-loop vision data transmission in edge–cloud VLM systems. Partial and flexible reuse of transmitted latent content, together with precise quality control, significantly reduces bandwidth and energy requirements without sacrificing semantic accuracy. This methodology will likely shape future research on deployable, resource-aware multimodal inference, especially as VLM architectures continue to expand in scale and scope.