Accelerating Multi-Condition T2I Generation via Adaptive Condition Offloading and Pruning

Abstract: Text-to-image (T2I) generation using multiple conditions enables fine-grained user control on the generated image. Yet, incorporating multi-condition inputs incurs substantial computation and communication overhead, due to additional preprocessing subtasks and control optimizations. It hence leads to unacceptable generation latency. In this paper, we propose an end-edge collaborative system design to accelerate multi-condition T2I generation through adaptive condition offloading and pruning. Extensive offline profiling reveal that, different conditions exhibit significant diversity in computation and communication costs. To this end, we propose a \textit{Subtask Manager} that jointly optimizes condition inference offloading and bandwidth allocation using a heuristic algorithm, balancing local and edge execution delays to minimize overall preprocessing latency. Then, we design a lightweight feature-driven \textit{Conditioning Scale Estimator} that evaluates the contribution of each condition by analyzing its feature activation strength and overlap with other conditions. This allows adaptive conditioning scale selection and pruning of insignificant conditions, thereby accelerating the denoising process. Extensive experimental results show that our system reduces latency by nearly 25\% and improves 6\% average generation quality, outperforming other benchmarks.

• The paper presents an integrated end-edge framework that co-optimizes subtask offloading and bandwidth allocation to tackle latency challenges in multi-condition T2I generation.
• The adaptive conditioning scale estimator prunes less effective visual signals, reducing inference time by around 18% while achieving a 6% improvement in image quality.
• Experimental results reveal a 25% reduction in end-to-end latency and robust performance under varied bandwidth constraints across heterogeneous platforms.

Accelerating Multi-Condition T2I Generation via Adaptive Condition Offloading and Pruning

Introduction

The paper "Accelerating Multi-Condition T2I Generation via Adaptive Condition Offloading and Pruning" (2605.08836) addresses the escalating computational and communication overheads inherent in text-to-image (T2I) diffusion models when incorporating multiple control conditions. As the demand for fine-grained, multi-modal user guidance in image generation grows, system efficiency is constrained by substantial preprocessing workloads, constrained device resources, and increased uplink bandwidth requirements. This work systematically investigates the resource heterogeneity of typical multi-condition T2I pipelines and introduces an integrated end-edge collaborative framework incorporating adaptive subtask offloading, bandwidth allocation optimization, and feature-based condition pruning to address the core latency bottlenecks of state-of-the-art compositional T2I systems.

Figure 1: An example of a multi-condition T2I generation task decomposing user instructions into independent visual and textual preprocessing subtasks.

Resource-Diversified Subtask Profiling

The paper first characterizes the significant disparity in resource requirements across a spectrum of preprocessing subtasks used for extracting control signals (e.g., text, edge, segmentation, depth) from reference images. Empirical profiling across a range of computing platforms—including Jetson Nano, TX2, AGX Orin, and RTX 3080Ti—reveals pronounced heterogeneity in both computation and data transmission cost. For example, semantic segmentation on Jetson Nano incurs an order of magnitude greater latency compared to Canny edge extraction, while the data size of control signals varies widely. These results necessitate a joint optimization of subtask device placement and uplink resource allocation.

Figure 2: Heterogeneous computing and transmission demands across different preprocessing subtasks and hardware platforms.

Conditioning Scale Impact Analysis

A core empirical finding is the criticality of adaptively tuning the conditioning scale of each visual control signal in the generative backbone. Default uniform scale assignments frequently yield suboptimal generation quality due to over-constraining effects or redundancy among correlated controls. The ablation studies demonstrate that pruning visual conditions with minimal effective contribution can reduce inference time by around 18% without significant quality regression, whereas pruning dominant conditions results in substantial semantic degradation.

Figure 3: Ablation studies demonstrate the quality and efficiency tradeoffs incurred by varying conditioning scales for different visual control signal combinations.

System Architecture

The proposed system consists of two tightly integrated modules:

1. Subtask Manager: Given an incoming multi-condition T2I job, the manager co-optimizes subtask offloading decisions and multi-user bandwidth allocations by formulating a mixed-integer nonlinear programming (MINLP) problem. The solution minimizes the time required for all necessary controls to be available at the edge for final image generation. The manager exploits the heterogeneous workload distribution and communication bottlenecks by iteratively reassigning the most latency-critical subtasks (device or edge) and adjusting bandwidth fractions according to transmission demand severity.
2. Conditioning Scale Estimator: Upon completion of all preprocessing, the estimator computes feature-driven effectiveness and uniqueness scores for each visual condition via high-level activation and redundancy analysis. Insignificant or redundant conditions falling below a pruning threshold are filtered prior to denoising, and nonzero scales are assigned adaptively to the remaining controls to maximize semantic diversity and sample adequacy.

   Figure 4: System overview depicting collaborative local/edge execution and dynamic condition selection using adaptive offloading and pruning modules.

Experimental Results

Rigorous experiments using representative platforms, Stable Diffusion v1.5, and various ControlNet configurations validate the efficacy of the system under multi-user, multi-condition scenarios. Key quantitative findings include:

• Latency: The proposed joint optimization scheme achieves an average savings of 25% in end-to-end generation latency compared to edge-only or device-only benchmarks, and maintains strong performance even under severe uplink constraints.
• Generation Quality: Feature-guided conditioning scale assignment improves average sample quality by 6% (ImageReward) over default configurations, with only a marginal 1–2% gap from exhaustive optimal-score searching, while incurring significant inference acceleration.
• Bandwidth Sensitivity: Latency reductions persist across a wide range of uplink bandwidth budgets, highlighting robust adaptation to different network settings.
• Parameter Sensitivity: The estimator’s pruning threshold enables scalable tradeoffs between computational acceleration and preservation of compositional fidelity.

Figure 5: Aggregate latency breakdown highlighting improvements through parallel subtask execution and condition pruning.

Figure 6: Qualitative comparison of generated images under default and adaptive conditioning scale selection.

Figure 7: Subtask completion latency as a function of available uplink bandwidth, indicating strong resilience to communication bottlenecks.

Implications and Future Outlook

This work provides a systematic analysis of resource-heterogeneous generative pipelines and a practical framework for deploying controlled T2I systems in bandwidth- and compute-constrained environments such as edge networks. The robust, efficient collaboration between user devices and edge servers, together with lightweight condition selection, directly addresses real-world constraints on interactively controlled image generation.

Theoretically, the adaptive scale estimation and redundancy-suppression mechanisms are a step toward more generalizable and resource-aware diffusion control strategies applicable to various multi-modal AIGC tasks. Practically, these findings inform the deployment of interactive and compositional generative AI applications—such as content creation and mobile AR—in resource-limited or multi-tenant environments.

Anticipated future work includes extending joint optimization and offloading to multi-edge scenarios, integrating more complex dependency structures across subtasks, and generalizing condition selection to arbitrary multi-modal control signals.

Conclusion

By integrating adaptive subtask offloading, resource-aware bandwidth allocation, and feature-driven control pruning, the system described in "Accelerating Multi-Condition T2I Generation via Adaptive Condition Offloading and Pruning" (2605.08836) achieves substantial reductions in T2I generation latency and quality improvements for multi-condition diffusion models. This architecture serves as a foundation for scalable, controlled, and efficient generative AI systems deployed on edge networks and represents a relevant direction for research on collaborative and resource-efficient AIGC.