VLAForge: Dual-Use VLA Framework

• VLAForge is a multi-domain framework that consolidates independently fine-tuned VLA experts into unified policies, addressing merge failures seen in naïve approaches.
• Its robotic variant utilizes masked LoRA adapters and a cross-attention-only action expert to overcome destructive parameter interference and achieve high task success rates.
• The deepfake detection adaptation leverages a frozen CLIP backbone with specialized modules to extract cross-modal cues, delivering robust cross-dataset AUROC performance.

Searching arXiv for the specified papers and related uses of the term "VLAForge". arxiv_search query="(Fu et al., 24 Nov 2025) OR VLAForge OR ForgeVLA" max_results=10

Searching arXiv for exact IDs and titles to ground the article in the current literature. VLAForge is a name used in recent arXiv literature for two distinct frameworks. In robotics, it denotes an end-to-end framework for merging independently fine-tuned Vision-Language-Action (VLA) experts into a single generalist policy, built directly on the MergeVLA architecture and evaluated on LIBERO, LIBERO-Plus, RoboTwin, and a real SO101 robotic arm (Fu et al., 24 Nov 2025). In deepfake video detection, VLAForge denotes a framework built upon a frozen CLIP backbone to exploit cross-modal semantics for discriminative forgery detection (Zhu et al., 25 Mar 2026). A closely related but differently named framework, ForgeVLA, addresses federated VLA training from distributed vision-action pairs without centralizing raw data or requiring manual annotations (Zhou et al., 8 May 2026).

1. Robotic VLAForge as a model-merging framework

In the robotic setting, VLAForge is defined as an end-to-end framework for merging independently fine-tuned VLA experts into a single generalist policy. Its immediate technical context is the observation that recent VLA models perform well when fine-tuned for a single embodiment or task family, but directly merging VLA experts trained on different tasks results in near-zero success rates. VLAForge is presented as a response to that failure mode, and the framework is described as MergeVLA plus a lightweight test-time routing mechanism (Fu et al., 24 Nov 2025).

The motivating problem is not merely multi-task training, but post hoc consolidation of separately adapted experts. The source setting assumes that each expert is adapted with LoRA adapters in the VLM backbone plus a task-specific action expert. Naïve merging is reported to produce “zero-shot collapse” of task success because independently learned task specializations do not compose under standard weight-space aggregation. This places VLAForge within the broader literature on model merging, but in a regime where existing recipes such as Task Arithmetic, TIES, and KnOTS are reported to be ineffective on vanilla VLA experts.

A plausible implication is that VLAForge is less a generic ensemble mechanism than an architectural prescription for preserving mergeability during fine-tuning. The framework therefore modifies both the parameterization of the vision-language backbone and the structure of the action expert.

2. Failure modes that motivate the design

The first empirically observed barrier is destructive LoRA parameter interference. The framework description states that LoRA-based updates satisfy $\Delta W_m = W_m - W_0$, where $W_0$ is the frozen pretrained VLM weight, and that these updates point in largely disjoint directions for different tasks. As the number of tasks grows, over $75\%$ of LoRA parameters become “selfish,” meaning relevant to exactly one task. Empirical evidence further reports that the fraction of parameters kept by exactly one task mask rises above $75\%$ even when merging only four LIBERO tasks. Straightforward averaging or sign-resolved merging therefore activates contradictory task signals and breaks the shared vision-language subspace (Fu et al., 24 Nov 2025).

The second barrier is architectural incompatibility of action experts. In dual-system VLA architectures such as VLA-Adapter, the action expert is trained from scratch and consists of alternating self- and cross-attention layers. Over depth, self-attention propagates and amplifies task-specific dependencies across blocks, causing the final layers to diverge excessively in parameter space. Empirically, while shallow blocks of different experts remain moderately similar, the relative L2 distance between their parameters in deeper layers explodes, making naïve weight-averaging completely ineffective; the reported merged success is $0\%$.

These two observations define the design space of robotic VLAForge. The first requires sparse control over which LoRA offsets are activated for a given task. The second requires action-expert blocks whose specialization remains localized rather than being recursively diffused through self-attention feedback.

3. Backbone merging with masked LoRA and modular action experts

VLAForge comprises three key components, the first of which is LoRA adapters with task masks. For each task $t$, the LoRA update is $\Delta W_t = W_t - W_0$. Rather than applying a single global merged update, the framework first computes a merged task vector

$$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$

where $\mathcal{R}(\cdot)$ is any data-free merging operator, with Task Arithmetic given as an example. It then defines a binary mask $M_t \in \{0,1\}^n$ by

$W_0$0

where $W_0$1 controls tolerance for disagreement. The final merged weight for task $W_0$2 is

$W_0$3

Across all tasks, the forward pass reuses the same merged backbone while sparsely activating only the task-relevant offsets, with the stated goal of dramatically reducing cross-task interference (Fu et al., 24 Nov 2025).

The second component is a cross-attention-only action expert. VLAForge replaces each transformer block in the action expert with a cross-attention-only module, explicitly removing self-attention. For the $W_0$4-th block, with input state $W_0$5 and VLM hidden streams $W_0$6 and $W_0$7, the block produces

$W_0$8

$W_0$9

$75\%$0

The sigmoid gate replaces the original $75\%$1 to ensure no signal inversion. Shallow blocks, up to layer $75\%$2, are simply weight-averaged across tasks, while only the deepest block or blocks $75\%$3 remain task-specific and are looked up at run time.

This decomposition is architecturally significant because it constrains where task specialization is allowed to reside. The framework description explicitly associates merged shallow blocks with transferable visuomotor priors and reserves only a small terminal head for task-specific behavior.

4. Unsupervised routing and reported performance

At inference time, VLAForge assumes that task identity is unknown. It therefore performs one routing step on the initial observation $75\%$4 to choose which mask $75\%$5 and expert head $75\%$6 to use for the entire episode. Let $75\%$7 and $75\%$8 denote the block-$75\%$9 hidden states produced by the masked backbone. Principal subspaces $75\%$0 and $75\%$1 are computed via SVD on the merged expert’s value matrices $75\%$2 and $75\%$3, keeping the top $75\%$4 right singular vectors. Each task $75\%$5 then receives scores

$75\%$6

$75\%$7

After selection, the mask and head remain fixed for all future time steps. The route-and-execute strategy is described as fully unsupervised and is reported to recover the correct skill over $75\%$8 of the time in practice (Fu et al., 24 Nov 2025).

Quantitatively, the reported success rates are benchmark-specific. On LIBERO, the table reports Single-Task Expert $75\%$9 and VLAForge $0\%$0, with “Merge w/ TIES+Masks.” On LIBERO-Plus, the values are $0\%$1 and $0\%$2, with “TA/TIES merging.” On RoboTwin in the cross-embodiment, single-task setting, the values are $0\%$3 and $0\%$4, with “$0\%$5 routed head.” On RoboTwin in the cross-embodiment, cross-task setting, the values are $0\%$6 and $0\%$7, with “$0\%$8+TIES.” On the SO101 real arm, the framework reports $0\%$9 for both single-task experts and VLAForge, with “TIES merging.”

The accompanying takeaways are also explicit. Under mixed-task evaluation on LIBERO, VLAForge achieves $t$0, just $t$1 percentage points below per-task fine-tuning at $t$2, and far above zero for naïve merging. Under seven types of visual/language shifts in LIBERO-Plus, the merged model at $t$3 outperforms specialized single-task baselines at $t$4 by $t$5 percentage points. In RoboTwin cross-embodiment tests, it recovers over $t$6 in the simpler setting and $t$7 when tasks vary. On the physical SO101 arm, evaluated with $t$8 trials per task, it matches single-skill experts at $t$9 average.

5. Generalization claims and stated limitations

The framework description attributes its generalization behavior to compact modularity: VLAForge reuses a single VLM backbone and merges most action-expert layers, relying only on a tiny per-task head and mask. Three consequences are stated. First, for cross-task robustness, the sigmoid-gated cross-attention expert preserves the VLM’s robust visual-language grounding and yields strong OOD performance under lighting, textures, viewpoint, and instruction shifts. Second, for cross-embodiment transfer, merged shallow blocks capture transferable visuomotor priors across robot morphologies, with only the deepest head layers specializing. Third, for real-world adaptability, the routing rule based on value-subspace activations generalizes from simulated hidden states to real RGB streams without retraining (Fu et al., 24 Nov 2025).

The stated limitations are equally specific. Scalability to many, defined as more than $\Delta W_t = W_t - W_0$0, skills may require hierarchical or incremental merging strategies to control mask size and routing complexity. The current routing observes only a single time step, so more dynamic or continual routing could handle task switches mid-episode. The reported system uses a $\Delta W_t = W_t - W_0$1B-parameter VLM, and it is left open how much larger backbones and higher-capacity action experts, including continuous-action flow models, can be merged under the same paradigm. The description also notes that pretraining VLA backbones on a more diverse collection of robotic data, including multi-embodiment and multi-scene data, may flatten conflicts and reduce reliance on task masks.

These limitations indicate that VLAForge is best understood as a design pattern for mergeability rather than a closed solution to lifelong generalist control. This suggests that its core contribution lies in identifying which parts of VLA fine-tuning should remain shared, which should remain sparse, and which should remain explicitly task-indexed.

6. Other uses of the name: deepfake detection VLAForge and the related ForgeVLA

A separate paper uses the same name, VLAForge, for deepfake video detection rather than robotics. In that work, VLAForge is built upon a frozen CLIP backbone, specifically ViT-L/14, and adds two independent modules: ForgePerceiver and Identity-Aware VLA Scoring. ForgePerceiver learns task-specific forgery cues without overwriting CLIP’s pre-trained knowledge. It operates on visual tokens $\Delta W_t = W_t - W_0$2 and learnable query tokens $\Delta W_t = W_t - W_0$3, computes query-conditioned forgery-aware masks, enforces orthogonality across queries via $\Delta W_t = W_t - W_0$4, and produces a global authenticity score $\Delta W_t = W_t - W_0$5 trained with cross-entropy. The identity-aware branch uses the templates “This is a real photo of <id> person.” and “This is a fake photo of <id> person.”, replaces the token embedding at the index of <id> with CLIP’s final class token $\Delta W_t = W_t - W_0$6, derives identity-aware text features, constructs a VLA attention map $\Delta W_t = W_t - W_0$7 over patch embeddings, and fuses this map with local forgery localization before producing $\Delta W_t = W_t - W_0$8. The final score is

$\Delta W_t = W_t - W_0$9

with total loss

$$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$0

Implementation details include ForgePerceiver as ViT-tiny_patch16_224, $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$1 queries, $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$2 heads, $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$3, $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$4, Adam with learning rate $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$5, weight decay $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$6, batch size $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$7, and $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$8 epochs on RTX3090; the evaluation metric is AUROC at frame and video level (Zhu et al., 25 Mar 2026).

The reported results for deepfake detection are cross-dataset. For face-swapping forgeries, VLAForge reports frame-level AUROC values of $$\tau_{\mathrm{merl}} = \mathcal{R}(\{\Delta W_t\}_t),$$9, $\mathcal{R}(\cdot)$0, $\mathcal{R}(\cdot)$1, and $\mathcal{R}(\cdot)$2 on CelebDF v1, CelebDF v2, DFDC, and DFD, respectively, with video-level values of $\mathcal{R}(\cdot)$3, $\mathcal{R}(\cdot)$4, and $\mathcal{R}(\cdot)$5 on CelebDF v2, DFDC, and DFD. For full-face generation forgeries, it reports $\mathcal{R}(\cdot)$6 on VQGAN, $\mathcal{R}(\cdot)$7 on StyleGAN-XL, $\mathcal{R}(\cdot)$8 on SiT-XL/2, $\mathcal{R}(\cdot)$9 on DiT, and $M_t \in \{0,1\}^n$0 on PixArt. The paper also reports an ablation sequence from a base model to progressively adding localization, masks, VLA scoring, and ID-priors, culminating at $M_t \in \{0,1\}^n$1 on CDF-v2 frame-level, $M_t \in \{0,1\}^n$2 on DFDC frame-level, and $M_t \in \{0,1\}^n$3 on video-level average. Strengths listed in the summary include leveraging both artifact-specific visual cues and CLIP’s intrinsic vision-language semantics, compact extra parameters of $M_t \in \{0,1\}^n$4M, and strong cross-dataset generalization; limitations include reliance on accurate extraction of a representative <id> embedding, fixed prompts, and the absence of explicit temporal modeling.

The related framework ForgeVLA, despite the similar name, addresses a different problem: federated vision-language-action learning without language annotations. It assumes one server and $M_t \in \{0,1\}^n$5 clients, each holding a private dataset $M_t \in \{0,1\}^n$6 of vision-action pairs. Each client uses an embodied instruction classifier $M_t \in \{0,1\}^n$7 to recover a pseudo-language instruction

$M_t \in \{0,1\}^n$8

To address vision-language feature collapse under highly non-i.i.d. task partitions, ForgeVLA adds a client-side contrastive planning loss against a global normalized task bank $M_t \in \{0,1\}^n$9 and replaces naïve FedAvg aggregation with a server-side adaptive aggregation quadratic program. Its reported setup uses InternVLA-M1 with $W_0$00B parameters, LoRA rank $W_0$01 on the encoder, $W_0$02 rounds, $W_0$03 local epochs, batch size $W_0$04, AdamW, $W_0$05, $W_0$06, and $W_0$07. On LIBERO-Goal, Object, Spatial, and LIBERO-10, ForgeVLA reports success rates of $W_0$08, $W_0$09, $W_0$10, and $W_0$11, respectively, with Pass@50 equal to $W_0$12 across all four suites, and it improves LIBERO-Goal by $W_0$13 percentage points over FedAvg (Zhou et al., 8 May 2026).

Taken together, these usages show that “VLAForge” is not a single established term across subfields. In robotics it denotes a mergeability-oriented architecture for consolidating VLA experts into a generalist policy; in deepfake detection it denotes a CLIP-based framework for exploiting vision-language semantics; and the similarly named ForgeVLA refers to federated VLA learning without language annotations.