On-Device Personalization Techniques

• On-device personalization techniques are methods that tailor machine learning models to individual users using local data while ensuring strict privacy and resource efficiency.
• They employ parameter-efficient adaptation protocols, such as selective fine-tuning and lightweight submodules, to balance model performance with device constraints.
• These techniques optimize memory, compute, and energy usage through strategies like gradient-splitting, buffer management, and proactive data swapping.

On-device personalization techniques comprise algorithmic, architectural, and system-level methods for tailoring machine learning models to individual users or environments using only data and computation available on local resource-constrained devices (e.g., smartphones, wearables, edge sensors, FPGAs, MCUs). The goals are to achieve strong adaptation to user data while guaranteeing privacy (no user data leaves the device), respecting severe memory, compute, and energy constraints, and maintaining high personalization efficacy relative to traditional server-based or federated learning approaches.

1. Architectural Foundations and Core Adaptation Strategies

Modern on-device personalization typically relies on partitioned neural architectures and parameter-efficient adaptation protocols, with a focus on privacy and resource constraints. The underlying models are usually pre-trained in large-scale, speaker-, user-, or environment-independent settings. Personalization proceeds by updating only a subset of parameters—selected for maximal adaptation with minimal memory and computational overhead—or by leveraging lightweight parameter submodules such as LoRA adapters.

A representative example is end-to-end speech recognition with streaming RNN-Transducer (RNN-T) models. These models employ an 8-layer LSTM encoder, a 2-layer language-model encoder, and a joint network, with input processed as stacked 80-dimensional log-Mel features. On-device adaptation is implemented by fine-tuning a subset of layers (e.g., encoder 1–7) in floating point, while keeping the quantized inference graph on the device—no weights or user data are ever uploaded (Sim et al., 2019). This design enables high privacy and scales independently of cloud resource availability.

To perform training on-device, methods such as gradient-splitting divide the forward and backward computations into sequential sub-graphs (e.g., layers 0–3 and layers 4–7 plus LM and joint) to drastically reduce memory usage, trading modestly increased compute time for sublinear memory scaling. In the cited RNN-T example, this achieves a 45% reduction in RAM on a pixel device, with a 42% increase in per-utterance training time.

Personalization efficacy is maximized by employing sliding-window or buffer-based data management (e.g., a cache of 500 examples with a stride of 10 new examples per session), with all data and updates maintained locally.

2. Memory, Compute, and System-level Optimizations

Resource-constrained devices necessitate specialized memory- and compute-efficient training pipelines. Techniques include:

• Fine-grained Layer Scheduling: Breaking training into atomic per-layer steps (Forward, ComputeGradient, ComputeDerivative, ApplyGradient), tracking buffer temporal lifetimes and spatial aliasing to reduce memory usage to the theoretical minimum. The NNTrainer framework uses whole-program analysis to achieve 1/20th the memory required by PyTorch or TensorFlow for large models and batch sizes (Moon et al., 2022).
• Proactive Swapping: Algorithms analyze the execution order intervals of each tensor and proactively move data between main memory and flash, leveraging lookahead windowing and asynchronous I/O to minimize peak DRAM requirements, demonstrated to reduce memory from 181 MiB to 71 MiB for VGG16 (at ~20% latency overhead).
• Resource-aware Sub-model Selection: Adaptive strategies, such as freezing upper layers and only training bottom layers, utilize real-time measurement of device RAM and battery to dynamically select a sub-model that can be trained in the available footprint (Sasindran et al., 2023).
• Quantization and Selective Parameter Tuning: All methods emphasize quantized (8- or 4-bit) storage of the model, with only a subset (e.g., adapter blocks) remaining in float for updates, to reduce both memory and latency (Sim et al., 2019, Mendoza et al., 2024).

3. Learning Algorithms, Objective Functions, and Update Pipelines

On-device personalization operates in both supervised and self-supervised regimes. Traditional supervised strategies rely on user-corrected labels in a sliding-window or small-batch setup. More advanced frameworks use interaction and feedback data as self-supervised signals.

• Supervised Personalization: Given a buffer of $N$ new (input, target) pairs, the on-device objective minimizes the standard sequence loss (e.g., RNN-T or cross-entropy) solely over the trainable subset of parameters, with optional $L_2$ regularization. Standard optimizers include Adam or momentum-based SGD, with learning rates typically near $1\times10^{-4}$ or $1\times10^{-3}$ and session batch sizes of 5–10 (Tomanek et al., 2021).
• Self-supervised Schemes: Approaches such as Adaptive Self-Supervised Learning Strategies (ASLS) define a prediction loss using user feedback as pseudo-labels and anchor the personalized parameters to the base model via an $L_2$ stability term. Updates are performed on lightweight adapter modules with per-batch SGD, and continuous adaptation is orchestrated with replay buffers and periodic rehearsal to avoid catastrophic forgetting (Mendoza et al., 2024).
• Buffer Management and Data Selection: For LLMs with storage constraints, self-supervised data selection algorithms track entropy-of-embedding, domain-specific score, and in-domain dissimilarity for each incoming user interaction and maintain only a representative fixed-size buffer, as in "Enabling On-Device LLM Personalization with Self-Supervised Data Selection and Synthesis" (Qin et al., 2023).

Regularization for drift control (e.g., $L_2$ anchor to $\theta_0$), early stopping (patience on validation), and support for partial labels or implicit feedback (user engagement metrics as regression targets) are widely used to stabilize adaptation.

4. Privacy, Scalability, and Data Handling

Stringent privacy constraints are enforced by ensuring all user data and model updates remain strictly on device. Security is provided by:

• Local Training and Buffering: Only local caching is allowed, with cyclical data retention (sliding window or buffer with replacement) and immediate purging after on-device adaptation steps (Sim et al., 2019, Tomanek et al., 2021).
• No Data or Gradient Export: There is zero transmission of audio, transcripts, raw text, or parameter deltas to the cloud in the adaptation loop; the only exception is in federated or collaborative approaches where encrypted, noise-added parameter vectors may be transmitted for aggregate global model updates (Wang et al., 2019, Paulik et al., 2021).
• Replay and Differential Privacy: Techniques such as local differential privacy, on-device hashing, and obscuration of raw text are optionally used to enhance privacy during continuous learning (Mendoza et al., 2024).

Scalability is inherent: no server-side model hosting or data storage is required for basic on-device personalization, enabling deployment to arbitrary device fleets without central bottlenecks or privacy leak vectors (Sim et al., 2019). The cost is an observed relative degradation in personalization efficacy, quantified as an 18.7% loss (from 63.7% to 58.1% relative WER reduction) compared to unconstrained server-side adaptation in speech recognition (Sim et al., 2019).

5. Evaluation Metrics and Empirical Results

Robust quantitative evaluation underpins all on-device adaptation pipelines:

• Speech Recognition: Metrics include baseline WER, personalized WER, and relative WER reduction (RelWERRed), as in:

Environment	WER_baseline	WER_personalized	RelWERRed
Server-side full fine-tune	35.6%	12.9%	63.7%
On-device (500 ex, 10 stride)	35.6%	14.8%	58.1%

• LLM Personalization: User engagement, satisfaction, and latency (e.g., 0.9 s in ASLS vs 1.5 s in classical tuning). Benchmark scores (e.g., 82.7 vs 73.7 for baselines), and buffer-based ROUGE-1 gains (up to 38% over FIFO baselines with <10 MB storage) (Qin et al., 2023).
• Resource and Latency: Peak RAM, per-training-step run time, battery and energy metrics are measured across devices. NNTrainer achieves down to 220 MiB footprint for Transformer encoder-decoder, while MobileASR tracks live RAM and battery before training sessions, adapting batch size, sub-model, and halting if thresholds are violated (Moon et al., 2022, Sasindran et al., 2023).
• Ablations: Systematic analysis shows that self-supervision, buffer selection, and quantization yield steady improvements, but excessive resource constraint or update frequency can degrade personalization.

6. System Variants, Special Cases, and Deployment Guidelines

Personalization paradigms extend beyond standard supervised or reinforcement-style loops:

• Few-Shot and Rapid Adaptation: On-device few-shot learning, as in wearable human activity recognition (HAR), freezes the feature backbone and adapts only the classifier via single-step SGD per sample, achieving <1 ms latency per update and μJ-level energy per adaptation on RISC-V MCUs (Kang et al., 21 Aug 2025).
• Early Exit, Progressive Adaptation: PersEPhonEE attaches early exit heads to CNN backbones and fine-tunes only those on-device, providing up to 22.7× training speedup and 3.2× inference speedup for the same accuracy (Leontiadis et al., 2021).
• Extremely Resource-Limited Devices: MicroT incorporates a self-supervised distilled extractor and trains only a small head (linear or 2-layer) on MCUs, combining early-exit decisions to trade accuracy for up to 29% energy savings (Huang et al., 2024).
• Cloud-Device Collaboration: Protocols such as CDCDA-PLM generate user-conditioned synthetic data in the cloud with a large LLM, perform quality filtering, and send the selected samples to the device for LoRA-based PEFT adaptation. Inference is then performed entirely on-device (Zhong et al., 29 Aug 2025).
• Federated Personalization and Evaluation: Large-scale frameworks enable per-device evaluation and tuning of model parameters (ASR grammar weights, personalization vectors) in parallel across millions of clients, recording only aggregate deltas under strong differential privacy (Wang et al., 2019, Paulik et al., 2021).

Best-practice deployment guidelines emphasize (i) seed model pre-adaptation with a diverse user pool, (ii) restricting updates to a minimal, functionally critical layer slice, (iii) batch design that enables rapid on-device purge, (iv) patience-based or validation-based stopping, and (v) per-device monitoring and adaptation of batch size and learning rates (Tomanek et al., 2021, Sasindran et al., 2023).

7. Limitations, Trade-offs, and Future Directions

Despite significant advances, all surveyed on-device personalization techniques must balance privacy, memory/compute efficiency, adaptation speed, and ultimate accuracy:

• Efficacy vs. Scalability: Moving from unconstrained server-based fine-tuning to on-device protocols results in measurable degradation of personalization gains, e.g., ≈18% in speech (Sim et al., 2019).
• Resource Overhead: Split-graph and buffer reuse reduce memory but increase per-utterance or per-sample latency (42–50% in practice) (Sim et al., 2019, Moon et al., 2022).
• Adaptation Data Scarcity: Few-shot or self-supervised methods ameliorate but do not eliminate the need for labeled or meaningful user feedback; under-interacted users may experience weaker adaptation (Mendoza et al., 2024).
• Heterogeneous Feedback Noise: Self-supervised LLM methods are susceptible to adaptation misdirection from noisy or skewed feedback.
• On-device Continual Learning: Replay buffers and regularization techniques remain areas of active tuning. Privacy-preserving update mechanisms such as local differential privacy and on-device hashing are proposed for additional protection.

Emerging research priorities include meta-learning for optimal adapter initialization, smarter buffer or sample selection strategies (e.g., uncertainty sampling), full integration of differential privacy on-device, and dynamic adaptation schedules balancing frequency, resource budget, and battery life (Mendoza et al., 2024). Cloud-device hybrid methods leveraging data augmentation and explainable model selection further reduce required adaptation compute and data by exploiting similarity to prior personalized models (Wang et al., 15 Apr 2025, Zhong et al., 29 Aug 2025).

In summary, on-device personalization synthesizes advances in resource-aware learning, privacy engineering, and algorithmic adaptability to enable scalable, user-specific machine intelligence wholly under end-user data sovereignty while navigating stringent device-level constraints.