MobileForge: Dual Mobile-AI Systems

• MobileForge is a dual-use mobile-AI framework featuring two distinct systems: a firmware-based immutable foundation model with task-specific adapters and an annotation-free GUI agent adaptation mechanism.
• The firmware-oriented design employs LoRA-based adapters, modular execution paths, and secure, co-designed integration to address model fragmentation and optimize storage and memory usage on mobile devices.
• The GUI adaptation system leverages MobileGym and HiFPO to explore real apps, mine actionable tasks, and refine MLLM-based policies without relying on human-written annotations.

MobileForge is a name used for two distinct mobile-AI research constructs. In one usage, introduced as “Mobile Foundation Model as Firmware,” it denotes a system architecture in which a frozen foundation model is installed as part of the device firmware and NPU image, while individual applications contribute small offline fine-tuned adapters; the GR-T prototype instantiates this design across a broad mobile-AI benchmark (Yuan et al., 2023). In a later usage, MobileForge denotes an annotation-free adaptation system for mobile GUI agents, combining MobileGym with Hierarchical Feedback-Guided Policy Optimization (HiFPO) to adapt MLLM-based agents to target apps without human-written tasks, demonstrations, or reward labels for those apps (Liu et al., 18 Jun 2026).

1. Terminological scope

The shared label “MobileForge” covers two research agendas with different immediate objectives. The first addresses fragmentation in on-device inference by treating a large mobile foundation model as immutable firmware and shifting task specialization into compact adapters. The second addresses the adaptation problem for mobile GUI agents by constructing an annotation-free loop for exploration, curriculum mining, rollout execution, evaluation, and policy optimization.

Usage of “MobileForge”	Core construction	Primary evaluation setting
Mobile foundation model as firmware	Frozen foundation model in NPU firmware plus app-provided adapters; prototype GR-T	38 mobile AI tasks over 50 datasets
Annotation-free GUI adaptation system	MobileGym plus HiFPO for MLLM-based mobile GUI agents	AndroidWorld and MobileWorld GUI-only

This terminological overlap is substantive rather than merely stylistic. One line of work is centered on firmware, operator sets, model composition, and NPU execution; the other is centered on target-app exploration, hierarchical feedback, and GRPO-based policy improvement. A plausible implication is that “MobileForge” functions as a reusable label for mobile-AI systems that emphasize a shared substrate beneath task-specific behavior, but the two systems should not be conflated (Yuan et al., 2023, Liu et al., 18 Jun 2026).

2. Firmware-resident foundation model architecture

In the firmware-oriented formulation, MobileForge is installed as part of the device firmware and webOS image. Its approximately 10 B-parameter foundation model resides in the NPU’s weight ROM and is never modified by individual apps or by OS-level updates; only the OS loader can replace it when the OEM issues a firmware upgrade. Applications interact with the model through a unified ML service API similar to Android’s NNAPI: an app issues a run-inference call tagged with its task-ID and supplies a small-footprint adapter, after which the OS or driver loads the adapter into RAM, plugs it into the frozen foundation model on the NPU, and dispatches the mixed graph to the NPU (Yuan et al., 2023).

The architectural premise is immutability of the backbone. Backbone weights such as LLaMA-7B quantized to INT8 are baked into the NPU’s ROM image, cannot be tampered with by third-party code, and are verified and loaded unchanged when the phone boots or the OS upgrades. Security isolation is enforced by placing adapters in an isolated partition of NPU SRAM; adapters can only read from the backbone, and no writeback to the backbone ROM is permitted.

Task specialization is implemented through an adapter mechanism. Each application ships its own adapter file, with 1 M–10 M parameters stored alongside the app. The adapter is produced by offline PEFT, for example LoRA, on a server. At runtime the system stitches together the frozen foundation graph and the adapter subgraph before NPU execution. This directly targets the fragmentation identified in mobile AI, namely varied architectures, operators, and implementations that burden joint optimization of hardware, system settings, and algorithms. This suggests a co-design model in which the OS and hardware manage a common inference substrate while applications supply only task-local deltas.

3. GR-T components, adapter mathematics, and execution paths

The concrete prototype for the firmware-oriented system is GR-T. Its sub-model inventory includes an IMG_enc based on a ViT encoder with 0.63 B parameters in FP16 at 167.6 GFLOPs, a TXT_enc based on a 12-layer CLIP encoder with 0.35 B parameters in FP16 at 23.4 GFLOPs, an AUD-B_enc based on ViT (AST) with 0.09 B parameters in FP16 at 61.5 GFLOPs, an AUD-I_enc based on Whisper-tiny with 0.04 B parameters in FP16 at 26.0 GFLOPs, an IMU_enc based on a 6-layer transformer with 0.02 B parameters in FP16 at 5.14 GFLOPs, a backbone based on LLaMA-7B with 6.28 B parameters in INT8 at 312.0 GFLOPs, a TTS_dec based on Fairseq TTS with fewer than 0.01 B parameters in FP32 at 8.58 GFLOPs, an IMG_dec based on latent diffusion with 1.07 B parameters in FP16 at 267.0 GFLOPs, and a GEN_dec based on an MLP head with fewer than 0.01 B parameters in FP16 at 125.0 GFLOPs (Yuan et al., 2023).

The adapter mechanism is expressed through LoRA insertion into transformer weights. Given a weight matrix $W\in\mathbb{R}^{d\times d}$, LoRA adds a low-rank update

$$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$

During fine-tuning, the task cross-entropy

$$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$

is minimized with respect only to $\theta_{\rm adapter}=\{A,B\}$ and, for encoder-only paths, the small MLP projection; the frozen weights $W$ remain unchanged.

GR-T supports four execution paths. The full path, embed$\rightarrow$backbone$\rightarrow$gen, is used for VQA and speech understanding. The backbone$\rightarrow$gen path is used for pure-text tasks such as MT and summarization. The embed-only path is used for simple classification such as image-to-label. The gen-only path is used for unary generation such as TTS and super-resolution. The benchmark used to evaluate this design is eAIBench, comprising 38 mobile AI tasks over 50 datasets spanning five modalities: CV, NLP, audio, sensing/IMU, and multimodal. Example tasks include next-word prediction, QA, MT, sentiment, summarization, code generation, object detection, image classification, OCR, segmentation, super-resolution, ASR, SLU, emotion recognition, wake word, HAR on IMU, TTS, captioning, text-image retrieval, and VQA. Each task uses its standard metric, including accuracy, F1, BLEU, SSIM, mAP, and WER.

4. Benchmarking results, systems properties, and deployment constraints

Across eAIBench, GR-T matches or exceeds task-specific models on 85% of the 50 datasets. Fifty percent of tasks show positive gains, including gains of up to +28% on text-image retrieval and +19% on image caption, while only six tasks drop by more than 10%; the worst case is approximately –20% on very low-resource translation. In zero-shot settings, GR-T achieves roughly 70–80% of task-specific-model performance in many tasks, and fine-tuning recovers or surpasses that level. In few-shot settings with 1–10% labeled data, GR-T outperforms task-specific models by up to 67% on SLU (Yuan et al., 2023).

The system’s primary systems-level claim is scalability in storage and memory. Per-adapter size is 1–10 M parameters, less than 0.1% of 10 B. On a 12 GB RAM device, all 50 adapters plus a 4-bit quantized model, with $10\,\text{B}\times 4$ bits $\approx 5\,\text{GB}$, fit in memory. By comparison, 50 task-specific models would occupy 15.2 GB of storage and more than 40 GB peak memory across models. The storage crossover occurs at approximately 15 tasks: beyond that point, MobileForge plus adapters is smaller than the sum of task-specific models.

Operator support is another explicit systems result. GR-T uses 39 ONNX operators, versus 156 across task-specific models, and 64% of GR-T’s operators are NPU-supported, versus 33% for task-specific models. Quantization places the backbone in INT8 and adapters and generators in FP16 or FP32; reducing from Quant8 to 4-bit causes less than 1% accuracy loss.

Latency and energy results are more qualified. On Jetson Orin NX under GPU inference, MobileForge is on average 12$$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$0 slower and 19$$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$1 more energy-intensive than task-specific models; mean GR-T latency is approximately 6.8 s end-to-end for full-path tasks and 0.24 s per token in generation, with first-token latency around 6.3 s and each subsequent token around 0.24 s. The paper’s “what-if NPU” estimate assumes a 22$$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$2 speedup, as existing NPU versus CPU, and yields an estimated runtime of roughly 0.48 s per task and 1.3 J, matching or outperforming task-specific models on CPU. Full NPU deployment, however, awaits broader operator support. A common misunderstanding is to treat these NPU numbers as already realized on commodity phones; the reported result is explicitly a conditional estimate rather than an already deployed end-to-end NPU implementation.

5. Annotation-free GUI adaptation: MobileGym and HiFPO

In the GUI-agent formulation, MobileForge addresses the cost of adapting MLLM-based mobile GUI agents to real target apps. The motivating problem is that mobile apps are numerous, frequently updated, and difficult to cover with human-written tasks, demonstrations, or reward labels. MobileForge therefore defines an annotation-free adaptation system whose two tightly coupled components are MobileGym and Hierarchical Feedback-Guided Policy Optimization, or HiFPO (Liu et al., 18 Jun 2026).

MobileGym is the interaction and evaluation substrate. It explores target apps to collect reachability evidence, mines executable tasks grounded in real trajectories, executes rollouts, and provides hierarchical feedback. Exploration follows a function-aware, depth-first traversal strategy: structural anchors such as Android activities and menus are extracted from the APK; at each GUI state, an MLLM proposes concrete user goals based on the visible UI and anchors; and branches are systematically explored by restoring parent states and replaying prefixes. The resulting pool $$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$3 consists of tuples of before/after screenshot, action, element, and natural-language summary.

MobileGym-Curriculum mines tasks from explored trajectories. For each trajectory in $$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$4, a curriculum generator assesses whether the original exploration goal was reasonable and completed, then generates 3–8 task variants of the form

$$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$5

where $$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$6 is a self-contained instruction, $$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$7 is an estimated step budget, $$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$8 is core functionality, $$W' = W + \Delta W,\quad \Delta W = B\,A,\quad A\in\mathbb{R}^{r\times d},\,B\in\mathbb{R}^{d\times r},\,r\ll d.$$9 is variation type, and $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$0 is prerequisites. MobileGym-Critic then evaluates completed rollouts by producing a trajectory-level outcome $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$1, step-level process labels $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$2 with $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$3 for reasonable or unreasonable and $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$4 as natural-language rationale, and a corrective hint $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$5 when failures or unreasonable steps occur. The implementation is an agentic two-stage VLM pipeline that first describes each action and then makes final judgments and generates hints.

HiFPO converts this hierarchical feedback into policy updates. Its core quantities are the trajectory outcome $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$6, step-level process feedback

$$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$7

and corrective hints $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$8. For each task $$\mathcal{L}(\theta_{\rm adapter}) = -\sum_{(x,y)\in\mathcal{D}} \sum_{t=1}^T \log p_\theta(y_t\,|\,y_{<t},x)$$9, the algorithm performs $\theta_{\rm adapter}=\{A,B\}$0 attempts, aggregating earlier hints before later attempts:

$\theta_{\rm adapter}=\{A,B\}$1

It computes the empirical success rate

$\theta_{\rm adapter}=\{A,B\}$2

and discards tasks with $\theta_{\rm adapter}=\{A,B\}$3, keeping those with $\theta_{\rm adapter}=\{A,B\}$4. For each retained task it selects the most informative attempt using the step-quality statistic

$\theta_{\rm adapter}=\{A,B\}$5

then builds a step-level training set from those steps whose process labels indicate reasonableness.

6. Hint-contextualized GRPO, adaptation data, and empirical results

The policy update in HiFPO is hint-contextualized step-level GRPO. For each example $\theta_{\rm adapter}=\{A,B\}$6, the hint context is incorporated into the prompt $\theta_{\rm adapter}=\{A,B\}$7. The method samples $\theta_{\rm adapter}=\{A,B\}$8 candidate responses from the old policy, parses them into actions, and assigns the adaptive GUI action reward

$\theta_{\rm adapter}=\{A,B\}$9

The rewards are normalized within the group to produce $W$0, and the final clipped-objective update is

$W$1

The gradient step $W$2 performs group-relative, hint-aware policy improvement (Liu et al., 18 Jun 2026).

The annotation-free data pipeline explored 20 AndroidWorld apps via MobileGym, yielding 527 exploration trajectories, from which MobileGym-Curriculum mined 3,249 candidate tasks. Base agents were Qwen3-VL-8B and GUI-Owl-1.5-8B, and adaptation subsets of size 200, 400, and 900 tasks were studied. The main 900-task runs used eight NVIDIA A100-80GB GPUs over approximately 80 hours, producing ForgeQwen3-8B and ForgeOwl-8B.

Evaluation was conducted on AndroidWorld, using 116 held-out tasks and Pass@1/2/3, and on MobileWorld GUI-only, using 117 tasks with no adaptation rollouts from MobileWorld. On AndroidWorld, Qwen3-VL-8B base obtains 55.2% Pass@3, while ForgeQwen3 reaches 67.2% Pass@3, close to GUI-Owl-1.5-8B base at 69.0%. Adapting GUI-Owl-1.5-8B yields ForgeOwl-8B with 67.2% Pass@1, 75.0% Pass@2, and 77.6% Pass@3, and hard-task single-attempt success improves from 19.3% to 29.8%. On MobileWorld GUI-only, ForgeOwl-8B achieves 41.0% success.

Ablations identify the system’s critical components. Removing hint context drops overall rollout success from 77.0% to 52.0%. Hint-contextualized GRPO, at Pass@1 = 50.9% with 900 tasks, outperforms both no-hint SFT and hint SFT baselines. Keeping all-fail and mixed-success tasks, with $W$3, yields the best cross-domain transfer. A stronger Gemini 2.5 Pro critic helps, but even with Qwen3-VL as evaluator, MobileForge improves the base policy. Trajectory-grounded curriculum covers a broader functional spectrum than landing-screen-only task generation.

7. Limitations, misconceptions, and future directions

Both MobileForge formulations are explicitly framed with significant limitations. For the firmware-oriented system, CPU and GPU inference are too slow for real-time use on current devices, and the 9.2 B-parameter model size is described as borderline for high-end phones, with a need to shrink toward roughly 1–2 B for mass-market deployment. Specialized low-resource translation remains an underperforming task regime. Open challenges include co-training a smaller mobile-tailored foundation model for sub-10 B parameters, defining an adapter API that survives backbone upgrades such as LLaMA to LLaMA2, automating end-to-end splicing of frozen and adapter graphs into efficient NPU kernels, and addressing secure multi-tenant inference, differential privacy in adapters, and encrypted inference (Yuan et al., 2023).

For the GUI-agent system, adaptation is bounded by the diversity of explored apps and therefore reflects AndroidWorld biases. Multi-app workflows, long-horizon memorization and math tasks, and game-style interactions remain challenging. The use of automatic critics introduces questions of evaluator reliability and safety. Future directions include scaling to wider app ecosystems and multi-app orchestration, incorporating persistent user state and privacy constraints for on-device adaptation, developing more verifiable or formalized critics, and integrating synthetic environment dynamics to bootstrap adaptation in low-data regimes (Liu et al., 18 Jun 2026).

Several misconceptions are clarified by the published descriptions. First, the shared name does not denote a single unified architecture; rather, it denotes two separate systems with different substrates and objectives. Second, the 2023 system is not a report of universally deployed NPU firmware practice; its strongest latency and energy claims depend on a what-if NPU assumption and on future operator support. Third, the 2026 system is annotation-free with respect to adaptation data for target apps, not feedback-free: it relies centrally on trajectory outcomes, step-level process labels, and corrective hints generated within MobileGym. Taken together, the two uses of the term indicate a broader research pattern in mobile AI: moving from isolated, app-specific artifacts toward shared substrates, whether those substrates are immutable firmware backbones or unified adaptation loops.