Foundation-Driven Efficient Paradigm (FDEP)

• Foundation-Driven Efficient Paradigm (FDEP) is a design logic that repurposes frozen foundation models with lightweight interfaces for efficient downstream adaptation.
• It employs techniques like modular composition, proxy-space transfer, and runtime orchestration to reduce computational costs while enhancing performance.
• FDEP integrates domain priors, federated and continual strategies to ensure adaptable, resource-efficient solutions in areas such as edge AI, geoscience, and robotics.

Foundation-Driven Efficient Paradigm (FDEP) denotes, across recent literature, a class of methods in which a pretrained foundation model—or its representations, internal modular structure, latent manifold, or coordination affordances—becomes the primary reusable substrate for downstream efficiency. Rather than training task-specific systems from scratch, FDEP-style work reuses foundation-level capability through runtime orchestration, parameter-efficient adaptation, proxy-space transfer, distillation, modular composition, domain-prior fusion, or graph-level coordination. The result is not a single canonical algorithm but a recurring technical pattern spanning edge inference, low-shot geoscience prediction, retrieval, federated learning, recommender systems, robotics, and scientific discovery (Djuhera et al., 30 Nov 2025, Yang et al., 28 Apr 2026, Lu et al., 23 Feb 2025, Liu et al., 17 Oct 2025).

1. Conceptual scope and representative instances

Across the cited literature, FDEP is best understood as an umbrella design logic rather than a universally fixed formalism. In some papers it is named explicitly, as in real-time edge AI and infrared small target detection; in others it is articulated through closely related formulations such as “foundation model distillation,” “foundation model programs,” “foundation–expert,” or “knowledge-data dually driven paradigm.” What remains stable is the claim that efficiency should come from acting on foundation-model structure or priors directly, not merely from conventional task-local training or infrastructure-only scheduling (Djuhera et al., 30 Nov 2025, Letellier et al., 25 Nov 2025, Nie et al., 9 Apr 2025, Li et al., 4 Aug 2025).

This literature also broadens the meaning of efficiency. Depending on the domain, efficiency may refer to lower latency, lower FLOPs, lower GPU memory, reduced communication, lower data requirements, stronger privacy preservation, improved developer velocity, or lower coordination overhead. That breadth is central: FDEP is not synonymous with model compression alone.

Representative work	Domain	Core efficiency mechanism
"Joint Partitioning and Placement of Foundation Models for Real-Time Edge AI" (Djuhera et al., 30 Nov 2025)	Edge AI	Joint runtime optimization of model segmentation and placement
"Knowledge-Data Dually Driven Paradigm for Accurate Landslide Susceptibility Prediction under Data-Scarce Conditions Using Geomorphic Priors and Tabular Foundation Model" (Yang et al., 28 Apr 2026)	Geohazard prediction	Geomorphic prior knowledge plus tabular foundation model under scarcity
"SelaVPR++: Towards Seamless Adaptation of Foundation Models for Efficient Place Recognition" (Lu et al., 23 Feb 2025)	Visual place recognition	Frozen DINOv2 with side MultiConv adapters and binary-first retrieval
"Foundry: Distilling 3D Foundation Models for the Edge" (Letellier et al., 25 Nov 2025)	3D point clouds	SuperToken-based distillation of a self-supervised teacher
"Resource-efficient Inference with Foundation Model Programs" (Nie et al., 9 Apr 2025)	Streaming multimodal inference	Program decomposition with per-module backend allocation
"Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment" (Li et al., 4 Aug 2025)	Recommendation	Central FM plus lightweight experts via target-aware embeddings

2. Recurrent design principles and formal patterns

A recurrent principle is to freeze the high-capacity foundation substrate and move task adaptation into a much smaller interface. "SelaVPR++" freezes DINOv2 completely and moves trainable adaptation into a parallel side network with MultiConv adapters, thereby avoiding backpropagation through the backbone during training (Lu et al., 23 Feb 2025). "Leveraging Foundation Models for Efficient Federated Learning in Resource-restricted Edge Networks" applies the same logic in federated vision-language learning: CLIP remains frozen on the server, while a lightweight prompt generator is trained from class descriptions and aggregated client knowledge, eliminating local FM deployment and the need for a public dataset (Atapour et al., 2024). "See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition" makes the same pattern theoretical by casting PEFT as subspace tuning, with adaptation written as $\phi(\mathbf{W}) = g(f(\mathbf{W}))$, where reconstruction and extension act on the subspaces of frozen pretrained weights (Si et al., 2024).

A second principle is proxy-space transfer: the foundation model is first transformed into a downstream-aware teacher before knowledge is distilled into a compact model. "Towards Efficient Task-Driven Model Reprogramming with Foundation Models" constructs a proxy space by keeping the teacher extractor frozen, learning a projector and task-aligned classifier on downstream data, then progressively distilling that reprogrammed teacher into a lightweight target model with a different architecture (Xu et al., 2023). "Foundry" pushes this further by training a student to reconstruct a teacher’s token-level latent space from a compressed set of SuperTokens, preserving general-purpose transferability rather than distilling a task head (Letellier et al., 25 Nov 2025). "FedPFT: Federated Proxy Fine-Tuning of Foundation Models" applies the same idea in federated adaptation: a full-depth but compressed proxy is aligned to the hidden FM through layer-level and neuron-level distillation so that proxy gradients remain useful for the full model (Peng et al., 2024).

A third principle is modular accumulation under continual change. "A Compositional Paradigm for Foundation Models: Towards Smarter Robotic Agents" argues that FMs should evolve from monolithic systems into compositional systems of reusable specialized components, illustrated by task-wise LoRA modules and hierarchical merging in continual ViT adaptation (Quarantiello et al., 21 Oct 2025). This suggests that FDEP is not only about efficient static transfer; it is also about preserving and recombining previously paid-for computation.

3. Systems, orchestration, and coordination

Several papers show that FDEP extends beyond weight adaptation into runtime orchestration. "Joint Partitioning and Placement of Foundation Models for Real-Time Edge AI" formalizes edge inference as a joint optimization over segmentation and placement, minimizing a cost

$$\Phi(\mathbf{x}, \mathcal{C}(t)) = \alpha\,\mathcal{L}\bigl(\mathbf{x}, \mathcal{C}(t)\bigr) + \beta\,\mathcal{U}\bigl(\mathbf{x}, \mathcal{C}(t)\bigr) + \gamma\,\mathcal{P}\bigl(\mathbf{x}, \mathcal{C}(t)\bigr),$$

then extending it to runtime split revision

$$\min_{S\in \Omega,\;\mathbf{x}} \Phi\!\bigl(\mathbf{x}, S, \mathcal{C}(t)\bigr).$$

Here efficiency arises from co-optimizing where a foundation model runs and how it is internally split, under latency, utilization, bandwidth, and privacy constraints (Djuhera et al., 30 Nov 2025).

"Resource-efficient Inference with Foundation Model Programs" makes a related shift at inference time. Instead of routing an entire query to one model, it compiles tasks into programs of modular FM calls and learns a policy over backends using the reward

$$R\bigl(\vec{v}_{t},\, x_{t},\, y_{t}\bigr) = -\,\mathcal{L}\bigl(p(x_t \mid \vec{v}_t),\, y_{t}\bigr) -\lambda\,\mathcal{C}\bigl(\vec{v}_{t}\bigr).$$

This permits cheap models to solve easy subtasks while expensive backends are reserved for hard branches, yielding up to 98% resource savings in streaming binary VQA and up to 50% savings in open-form VQA (Nie et al., 9 Apr 2025).

In industrial recommendation, the same systems logic appears as architectural decoupling. "Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment" trains a central FM on lifelong, cross-surface, multi-modal user data, then transfers its knowledge to lightweight, surface-specific experts through target-aware embeddings. Experts require only 20–40% of the compute of their one-stage counterparts, while the deployed system serves tens of billions of user requests daily (Li et al., 4 Aug 2025).

"Foundation Protocol: A Coordination Layer for Agentic Society" generalizes the same pattern from model execution to multi-agent infrastructure. FP defines a graph-first coordination layer built around seven core objects—Entity, Session, Activity, Envelope, Event, Receipt/Settlement, and Provenance—and uses progressive disclosure, signed envelopes, policy checkpoints, and ledger-agnostic settlement to reduce integration and governance overhead while keeping accountability first-class (Liu et al., 22 May 2026). This suggests that, in mature FDEP systems, coordination itself becomes a reusable foundation.

4. Domain priors, modality alignment, and task-specific semantics

A notable branch of FDEP does not rely on generic foundation priors alone; it fuses them with explicit domain structure. "Knowledge-Data Dually Driven Paradigm for Accurate Landslide Susceptibility Prediction under Data-Scarce Conditions Using Geomorphic Priors and Tabular Foundation Model" combines DEM-derived geomorphic prior knowledge with a tabular foundation model, using the prior as an additional physically informed conditioning factor rather than as a final prediction. The paper’s central efficiency claim is that when the available sample size reaches approximately 30% of the total available samples, the proposed paradigm already achieves or surpasses the predictive performance of conventional baselines trained on the full available samples (Yang et al., 28 Apr 2026). This suggests that, in some domains, FDEP is most effective when broad pretrained inference is constrained by physically meaningful priors.

"Rethinking Infrared Small Target Detection: A Foundation-Driven Efficient Paradigm" makes the alignment problem explicit. It injects frozen DINOv2 semantics into encoder-decoder SIRST detectors through a Semantic Alignment Modulation Fusion module, which first aligns frozen VFM features and then modulates detector features through multiplicative gating and additive semantic enrichment. To avoid inference-time burden, Collaborative Optimization-based Implicit Self-Distillation trains a semantically enriched main branch and an original lightweight branch with shared encoder-decoder parameters, then discards the VFM path at test time. On SIRST3, MSDA-Net improves from HSE 89.72 to 91.31 while keeping inference time at 0.035 s in the full FDEP framework (Yu et al., 5 Dec 2025). The same work also introduces the Holistic SIRST Evaluation metric,

$HSE = HSE\text{-}P \times HSE\text{-}T,$

to unify pixel-level confidence quality and target-level robustness (Yu et al., 5 Dec 2025).

These works indicate that FDEP often depends on a nontrivial alignment layer between foundation priors and domain-specific signal geometry. Naive insertion of frozen features is frequently insufficient.

5. Federated, continual, and embodied variants

In federated settings, FDEP typically centralizes the heavy prior and distributes only small adaptation surfaces. "FedD2P" keeps CLIP frozen on the server, distills class-wise local knowledge into a prompt generator using linguistic class descriptions, then returns FM-derived global class knowledge to lightweight client CNNs. Across CIFAR10, SVHN, OxfordPets, EuroSAT, and DTD, the method outperforms its baselines under both homogeneous and heterogeneous client data (Atapour et al., 2024). "FedPEAT" follows a different route: clients receive compressed frozen emulators plus small trainable adapters, and only adapters are aggregated. In the reported system study, one round of computation and communication is 4.60 times shorter than full-model federated tuning, at the cost of perplexity that is 3.49 points higher (Chua et al., 2023). "Federated Foundation Models in Harsh Wireless Environments" extends the same logic conceptually, arguing that pretrained backbones plus PEFT, compressed updates, and decentralized or hierarchical coordination are necessary under intermittent connectivity, limited energy, and stale updates, and explicitly proposes evaluation criteria such as energy per improvement and communication per adaptation (Chen et al., 2 Sep 2025).

Embodied and continual variants push FDEP toward modular lifelong systems. "A Compositional Paradigm for Foundation Models" reports that hierarchical adapter merging on CUB200 with 50 tasks yields 55.17 ± 1.04 accuracy in 170.61 s, outperforming SD-LoRA and InfLoRA, and that its robotic manipulation setup reaches reward/step 0.60 and success 0.91 in 14 h, compared with 0.10 and 0.0 in 92 h for OpenVLA (Quarantiello et al., 21 Oct 2025). The robotics review "Foundation Model Driven Robotics" describes the broader systems pattern as hybrid, modular, semantics-first but embodiment-constrained: foundation models supply reusable semantic priors for perception, planning, and interaction, while classical controllers, affordance modules, memory, and safety filters preserve physical feasibility and real-time operation (Khan et al., 14 Jul 2025). In this embodied reading, FDEP is not end-to-end gigantism; it is structured reuse of broad semantic knowledge inside a physically grounded architecture.

6. Empirical status, misconceptions, and future directions

The literature indicates that FDEP should not be reduced to parameter-efficient fine-tuning. It includes subspace tuning, proxy-space reprogramming, token-space distillation, program-level backend allocation, joint placement-and-partition optimization, target-aware expert transfer, and coordination protocols (Si et al., 2024, Xu et al., 2023, Letellier et al., 25 Nov 2025, Nie et al., 9 Apr 2025). A second misconception is that FDEP always implies direct deployment of a large foundation model. Many of the strongest examples do the opposite: they use the foundation model only during training, routing, or central representation generation, then deploy a smaller branch, expert, or student (Lu et al., 23 Feb 2025, Yu et al., 5 Dec 2025, Li et al., 4 Aug 2025).

The empirical record is also uneven. Some papers provide strong experimental evidence, such as SelaVPR++, which reports average R@1 = 93.6 and total retrieval latency of 0.51 ms in its performance-focused configuration, and Foundry, whose student remains close to its teacher while reducing FLOPs from 478.128G to as low as 137.274G and running under a 6 GB budget where the baseline is OOM (Lu et al., 23 Feb 2025, Letellier et al., 25 Nov 2025). Others are partly or wholly conceptual. The edge-AI joint partitioning paper explicitly reports expected performance results rather than a built prototype, and the harsh-wireless and scientific-discovery papers are agenda-setting rather than solver-complete (Djuhera et al., 30 Nov 2025, Chen et al., 2 Sep 2025, Liu et al., 17 Oct 2025). This matters for interpretation: FDEP is already productive as a design doctrine, but its maturity varies sharply by domain.

The common limitations are similarly recurrent. Several works leave cost models abstract or not fully solver-ready; proxy and distillation methods still face domain-shift failure modes; real deployments require stronger treatment of weight transfer, cache migration, privacy guarantees, and multi-tenant fairness; and conceptual papers emphasize unresolved issues in grounding, auditability, reproducibility, authorship, and safety (Peng et al., 2024, Djuhera et al., 30 Nov 2025, Khan et al., 14 Jul 2025, Liu et al., 17 Oct 2025). A plausible synthesis is that mature FDEP will require three convergences: richer model-aware systems optimization, stronger domain-specific alignment mechanisms, and protocol-level governance for accountable multi-agent operation.

In that sense, FDEP marks less a single model family than a systems principle: foundation models become most efficient when their pretrained structure is treated as a reusable control surface—one that can be constrained by priors, exposed through programs, distilled into smaller students, partitioned across infrastructure, personalized through small deltas, or embedded inside governed agentic workflows.