TrainDeeploy: Unified On-Device Training

• TrainDeeploy is a unified framework for on-device training that integrates graph compilation, memory scheduling, and accelerator-aware code generation for both CNNs and Transformers.
• It employs parameter-efficient fine-tuning strategies like selective layer-wise tuning and LoRA, reducing memory usage and off-chip transfers while maintaining near-full accuracy.
• The complete pipeline—from PyTorch model definition to ONNX-based static graph execution with DMA scheduling—enables efficient end-to-end training on heterogeneous ultra-low-power SoCs.

Searching arXiv for TrainDeeploy and closely related edge on-device training work. TrainDeeploy is a framework for unified efficient inference and on-device training on heterogeneous ultra-low-power System-on-Chips (SoCs), targeting the extreme edge and supporting both Convolutional Neural Networks (CNNs) and Transformer models. It provides a complete on-device training pipeline that includes graph construction, compilation, memory scheduling, accelerator-aware code generation, and execution, while also supporting parameter-efficient fine-tuning strategies such as selective layer-wise fine-tuning and Low-Rank Adaptation (LoRA). On a RISC-V-based heterogeneous SoC, it demonstrates end-to-end on-device fine-tuning of a Compact Convolutional Transformer (CCT), with throughput up to 11 trained images per second, and reports reductions in dynamic memory usage, trainable parameters and gradients, and off-chip transfer volume when LoRA is used instead of full backpropagation (Wang et al., 10 Mar 2026).

1. Definition and scope

TrainDeeploy addresses on-device tuning of deep neural networks in settings where long-term adaptation at the edge is desirable for data privacy, but where backpropagation is constrained by ultra-low-power and memory-limited hardware. The framework is positioned around the observation that the computational and memory demands of training are especially severe for attention-based models, and that these constraints are amplified on extreme-edge devices. Within that setting, TrainDeeploy unifies efficient inference and on-device training and expands prior frameworks to support both CNNs and Transformer models together with parameter-efficient tuning on extreme-edge platforms (Wang et al., 10 Mar 2026).

The framework’s scope is not limited to a single model family. Its technical summary states that the unified static graph compiler and accelerator-aware tiling apply broadly to CNNs, MLPs and Transformers. This suggests that TrainDeeploy is best understood not merely as a model-specific implementation, but as a toolchain for compiling full training workloads under severe memory and bandwidth constraints. A plausible implication is that its central contribution lies in integrating compilation, scheduling, and hardware execution rather than in proposing a new learning rule alone (Wang et al., 10 Mar 2026).

TrainDeeploy also belongs to a broader train–deploy research trajectory in which deployment is no longer treated as a terminal inference-only phase. Earlier work examined browser-based distributed training with JavaScript and WebCL (Hidaka et al., 2017), systematic discrepancies across frameworks and deployment targets such as PC, Android, iOS, and Web (Guo et al., 2019), and edge-device pipelines spanning inference, adaptation, learning, and edge-server systems (Qu, 2022). Against that background, TrainDeeploy narrows the focus to fully on-device fine-tuning on heterogeneous SoCs at the extreme edge (Wang et al., 10 Mar 2026).

2. Hardware substrate and execution model

TrainDeeploy is implemented for a heterogeneous SoC organized around a host core, a compute cluster, DMA engines, and a tiered memory hierarchy. The host core is a RISC-V RV32IMFCXpulp core that orchestrates data movement, control flow and non-GEMM operators. The compute cluster comprises 8 RISC-V cores sharing a 128 KB multi-banked L1 TCDM, together with a RedMulE FP32 GEMM accelerator implemented as a 12×4 systolic array. DMA engines move tiles between L3 and L2 and between L2 and L1 under software control (Wang et al., 10 Mar 2026).

The memory hierarchy is central to the framework’s design. L1 stores active tiles of weights, activations or gradients for each operator invocation and provides single-cycle access by cluster cores and RedMulE. L2 is a 2 MB on-chip SRAM that holds full model weights, whether frozen or PEFT-modules, as well as inputs and less-frequently used activations and gradients. L3 is a 32 MB off-chip HyperRAM used for spilled tensors when L2 capacity is exceeded, with tensors staged through L2 by DMA before tiling into L1 (Wang et al., 10 Mar 2026).

Execution is operator-centric and tile-based. The host issues operator calls; for each GEMM or convolution, the RedMulE accelerator operates on L1-resident tiles, while the cores handle tiling loops, non-GEMM and backprop operators, and optimizer updates. This partitioning of responsibilities is significant because it makes training feasible without assuming that all operations map cleanly to the accelerator. In effect, TrainDeeploy treats hardware heterogeneity as a first-class compilation target rather than a runtime inconvenience (Wang et al., 10 Mar 2026).

3. End-to-end training pipeline

TrainDeeploy’s training flow begins in PyTorch, where the model and training recipe, including loss and SGD, are defined. The forward graph is then exported in ONNX. An ONNXRuntime-based autograd stage generates a static forward+backward graph, after which optimizer update subgraphs are inserted. The output of this process is a complete training graph encoding inference, gradient flows and parameter updates (Wang et al., 10 Mar 2026).

The compiler flow extends Deeploy and has three stages: frontend, midend, and backend. The frontend parses ONNX operators and matches them to trained kernels, including RedMulE GEMM. The midend performs unified operator tiling and memory scheduling across the entire training graph. It formulates tensor lifetimes via liveness analysis and tiling constraints, including minimum tile size and L1/L2 capacity, as a 2D bin-packing problem solved by TetriSched, and assigns tensors such as activations, gradients, and PEFT modules to L1, L2, or L3 in order to minimize peak memory. The backend then generates C code that orchestrates DMA transfers, tiling, accelerator invocations, CPU kernels for non-GEMM and backprop operators, and optimizer updates (Wang et al., 10 Mar 2026).

On-device execution follows the compiled graph literally. The host loads model weights and PEFT parameters into L2. For each input sample, the input is staged from L3 to L2 to L1, the forward pass is executed operator by operator, activations are stored in L2 or L3 according to the allocation plan, and the backward pass is launched symmetrically to produce $\partial L/\partial W$ only for trainable modules and $\partial L/\partial X$. SGD updates are then applied in place, and the process repeats per sample or mini-batch (Wang et al., 10 Mar 2026).

This static-graph formulation differs in emphasis from framework studies that focus on post-training migration and conversion correctness. For example, cross-framework deployment work reported that migration and quantization can suffer from compatibility and reliability issues, including substantial accuracy decline after model conversion and platform-specific bugs (Guo et al., 2019). TrainDeeploy instead brings the full update path into a single compiled execution model on the target SoC, which suggests a strategy for reducing deployment-time semantic drift by avoiding repeated framework transitions (Wang et al., 10 Mar 2026).

4. Parameter-efficient fine-tuning mechanisms

TrainDeeploy supports two training strategies beyond full backpropagation: selective layer-wise fine-tuning and LoRA. In selective layer-wise fine-tuning, the convolutional tokenizer is frozen and the last 1 or 2 Transformer encoder blocks, comprising attention and MLP components, are selected according to an accuracy-versus-memory budget criterion. The reported effect is linear reduction in the number of trainable parameters and their gradients when fewer layers are tuned, which lowers both on-chip memory pressure and off-chip transfer volume (Wang et al., 10 Mar 2026).

LoRA is introduced through the decomposition

$$W' = W_0 + \Delta W,\quad \Delta W = BA,\quad B\in\mathbb{R}^{d\times r},\,A\in\mathbb{R}^{r\times k},\;r\ll\min(d,k).$$

Only $A$ and $B$ are trainable, while $W_0$ remains frozen. The trainable parameter count is given as

$$\#\text{param}_{\text{LoRA}} = r(d + k)\;\ll\;dk = \#\text{param}_{\text{Full}}.$$

For LoRA-2 on CCT-2, TrainDeeploy reports a 23% reduction in peak dynamic memory, a $15\times$ reduction in the number of gradients and trainable parameters, and a $1.6\times$ reduction in off-chip transfer volume between L3 and L2 compared with full backpropagation (Wang et al., 10 Mar 2026).

A concise comparison is given below.

Configuration	Trainable Params	Peak Dynamic Memory	Off-chip Traffic
LoRA-2	0.05 MB	1.4 MB	0.62×
FT-2	0.76 MB	1.8 MB	1×

The framework further states that selective layer-wise and LoRA deliver near-full accuracy, citing 96% on CIFAR-10→MNIST at less than 10% of trainable state and 23% lower dynamic memory. This places TrainDeeploy within the parameter-efficient fine-tuning lineage, but under extreme-edge constraints rather than server or desktop conditions (Wang et al., 10 Mar 2026).

In a broader deployment context, parameter-efficient adaptation also appears in other train–deploy formulations. The learngene framework for deep MIMO detection distributes a reusable neural segment constituting 10.8% of the full model and enables device-specific fine-tuning with improved convergence and scalability (Zhang et al., 21 Mar 2025). The resemblance is methodological rather than architectural: both approaches reduce the state that must be transferred or updated at deployment time. This suggests a common design principle in edge adaptation, namely to separate reusable frozen structure from compact trainable state (Zhang et al., 21 Mar 2025).

5. Empirical performance and efficiency

TrainDeeploy evaluates performance at 360 MHz on two benchmark models: CCT-2/3×2 and Deep-AE. For CCT-2/3×2, a Vision Transformer, the model has 0.28 M parameters, corresponding to approximately 1.12 MB in FP32, and requires 71–126 M FLOPs for forward plus backward execution. The framework reports throughput of 11 images per second for single-sample updates and accelerator-assisted throughput of 4.6 FLOP/cycle. For Deep-AE, an autoencoder with 0.27 M parameters and 0.8 M forward+backward FLOPs, the reported accelerator-assisted throughput is 13.4 FLOP/cycle (Wang et al., 10 Mar 2026).

For the CCT-2 comparison between LoRA-2 and FT-2, the end-to-end latency is reported as approximately 87 ms versus 200 ms on the 8-core plus RedMulE configuration. This latency difference aligns with the accompanying reductions in trainable state and off-chip traffic, indicating that TrainDeeploy’s performance gains are not solely due to arithmetic reduction but also to lower movement across the memory hierarchy (Wang et al., 10 Mar 2026).

The framework also reports that accelerator offload through RedMulE yields a 2.3–3.5× speedup, while noting that benefits diminish for tiny LoRA multiplications because of tile under-utilization. This caveat is technically important: it shows that parameter-efficient methods can lower arithmetic intensity enough to reduce accelerator efficiency at very small matrix sizes, even when overall end-to-end performance still improves (Wang et al., 10 Mar 2026).

These measurements should be interpreted against known deployment bottlenecks in other environments. Browser-based distributed training, for example, reported that bandwidth dominates when model size is large and that even 8-bit gradient representation only partially mitigates saturation beyond about 8 clients for VGG16 because of latency and bandwidth limits (Hidaka et al., 2017). TrainDeeploy’s emphasis on minimizing off-chip transfer volume inside the SoC can be read as a memory-hierarchy analogue of that communication bottleneck (Wang et al., 10 Mar 2026).

6. Position within train–deploy research

TrainDeeploy occupies a specific niche within the larger train–deploy landscape: complete on-device fine-tuning on extreme-edge hardware. Earlier browser-centric work demonstrated that distributed training could be performed in JavaScript with Sushi2 and Sukiyaki2, using WebCL and a parameter server to coordinate data-parallel updates across browsers or Node.js processes (Hidaka et al., 2017). That work established feasibility in widely diverse environments without installation, whereas TrainDeeploy targets low-power SoCs with compiled static training graphs and explicit tiling (Wang et al., 10 Mar 2026).

Systematic framework studies later showed that training and deployment are sensitive to framework-induced numerical variance, model conversion artifacts, and platform-specific reliability issues. Reported examples include validation accuracy differences across TensorFlow, PyTorch, CNTK, and MXNet under identical hyperparameters, a ResNet-20 conversion from TensorFlow to CNTK with accuracy decline from 82.66% to 74.35%, and a TF.js WebGL backend bug involving 1×1 convolution (Guo et al., 2019). In that context, TrainDeeploy’s integrated compilation path from PyTorch to ONNX to static forward+backward execution suggests a deployment model that prioritizes compilation-time control over runtime portability (Wang et al., 10 Mar 2026).

Within edge-device research more generally, four scenarios have been distinguished: inference on edge devices, adaptation on edge devices, learning on edge devices, and edge-server systems (Qu, 2022). TrainDeeploy aligns most directly with the “learning on edge devices” scenario, but it also intersects with inference and adaptation because its compiler unifies inference, gradient flows, and parameter updates in a single graph (Wang et al., 10 Mar 2026). A plausible implication is that it concretizes, for heterogeneous SoCs and small Transformer models, the type of end-to-end pipeline that edge-learning surveys and dissertations had previously framed at a higher level (Qu, 2022).

Deployment-time model selection research adds another complementary perspective. When a repository of candidate models exists, similarity-based and adaptivity-based search can identify the best model for deployment without target labels (Zhou et al., 2022). TrainDeeploy does not address this selection problem directly, but its support for selective layer-wise tuning and LoRA suggests a downstream stage in which a chosen compact model is then adapted locally on-device (Wang et al., 10 Mar 2026). This suggests a modular train–deploy stack: search at deployment time, then parameter-efficient fine-tuning on the device.

7. Limitations, extensions, and significance

The current TrainDeeploy flow uses SGD, batch size = 1, and FP32. The reported future directions are support for Adam, larger batches, mixed-precision such as FP16, and integer training. Additional extensions named in the technical summary include integration with structured sparsity, quantization-aware fine-tuning, and other PEFT methods such as adapters, as well as application to other compact Transformer architectures including MobileViT and to emerging heterogeneous AI SoCs (Wang et al., 10 Mar 2026).

Its limitations are therefore partly algorithmic and partly architectural. Full fine-tuning still maximizes adaptability but exceeds typical MCU budgets because of memory and compute demands. Even when LoRA is used, accelerator efficiency can diminish for very small updates. The framework’s reliance on static graph compilation and explicit memory scheduling also implies that its strengths are most apparent when models and training recipes can be fixed ahead of deployment (Wang et al., 10 Mar 2026).

The significance of TrainDeeploy lies in the claim that it provides the first complete on-device training pipeline for extreme-edge SoCs supporting both CNNs and Transformer models, together with multiple training strategies such as selective layer-wise fine-tuning and LoRA (Wang et al., 10 Mar 2026). In practical terms, it brings together a heterogeneous hardware runtime, a compiler with liveness-aware tensor placement, and parameter-efficient adaptation mechanisms under one end-to-end toolchain.

More broadly, TrainDeeploy exemplifies a shift in the meaning of deployment. In older train–deploy workflows, deployment commonly meant conversion, quantization, migration, or serving (Guo et al., 2019, Hrnjica, 2020). In TrainDeeploy, deployment includes continued local optimization under strict resource limits. This suggests that, for extreme-edge systems, the boundary between training and inference is no longer fixed: it becomes a compiled continuum spanning forward execution, gradient generation, and in-place parameter updates on the target device itself (Wang et al., 10 Mar 2026).