Large Action Models (LAMs)

• Large Action Models (LAMs) are machine learning models that generate and execute sequences of actions in digital and physical environments.
• They integrate reasoning and decision-making with adaptable architectures, enabling applications in autonomous robotics, GUI automation, and IoT.
• Innovative training pipelines and edge deployment strategies empower LAMs to bridge the gap between human intention and practical execution.

Large Action Models (LAMs) encompass a class of machine learning models designed to generate and execute sequences of actions—often within dynamic or complex environments—in contrast to traditional models whose outputs are limited to text or class labels. LAMs have emerged in response to the limitations of models that only interpret or classify, meeting modern requirements for agentic intelligence, task automation, service composition, and real-world system integration. Research spans a breadth of domains: agentic AI and autonomous systems, scientific discovery, GUI automation, wireless communication, real-time robotics, and distributed edge intelligence, among others. Across the literature, LAMs are characterized by their ability to "bridge the gap between intention and execution," integrating reasoning, decision-making, and grounded action within adaptable architectural and deployment frameworks.

1. Conceptual Overview and Terminology

LAMs are defined by their capability to output action sequences that can be executed in external environments, whether digital (via API/tool invocation, GUI manipulation) or physical (robotic control, network operation) (Wang et al., 13 Dec 2024, Zhang et al., 5 Sep 2024). Distinct from LLMs, which map natural language inputs to text outputs, LAMs map input sequences—which may comprise text, multimodal sensory data, or environmental state—directly to atomic or composite actions.

Key subclasses and terminology include:

• Agentic LAMs: Models that autonomously generate and execute chains of actions for tasks such as service orchestration or interactive tool usage, often embodying reasoning-action interleaving (Zhang et al., 9 Mar 2025, Zhang et al., 5 Sep 2024).
• Recurrent and Parallel LAMs: Architectures based on recurrent sequence models (e.g., xLSTM) or scalable transformers, supporting efficient multi-step action generation in settings demanding fast inference or lengthy task horizons (Schmied et al., 29 Oct 2024).
• Edge LAMs: Distributed LAMs leveraging federated learning, modular deployment, and resource-aware microservice design for ubiquitous, low-latency action at the network edge—key for 6G/IoT (Wang et al., 1 May 2025, Wang et al., 6 May 2025).
• Domain-Specific LAMs: Models specialized for wireless communication (e.g., channel state feedback, semantic communications) (Guo et al., 4 Aug 2025, Zhuang et al., 13 May 2025, Ni et al., 28 Mar 2025), atomic simulation (Zhang et al., 2 Jun 2025), or audio-language understanding (Li et al., 21 Feb 2025, Song et al., 21 May 2025).

2. Principal Architectures and Methodological Frameworks

LAM research features architectural innovation tailored to scale, action grounding, cross-modality, and resource constraints.

• Transformer-based LAMs: Standard in dense architectures, sometimes extended to Mixture-of-Experts (MoE) for parameter and compute efficiency. Dense and MoE models support action composition for agentic reasoning, planning, and tool usage (Zhang et al., 5 Sep 2024).
• Recurrent LAMs (xLSTM): Enable linear-time, real-time inference crucial for robotics—with exponential gating mechanisms that support both memory retention and adaptive state tracking. The core recurrence is formulated as:

$$c_t = \alpha_t \odot c_{t-1} + (1 - \alpha_t) \odot \tilde{c}_t,\qquad \alpha_t = \sigma(W_a x_t + U_a h_{t-1} + b_a)$$

(Schmied et al., 29 Oct 2024)

• Graph Neural Network LAMs (DPA3): For atomistic/physical modeling, LAMs leverage hierarchical message passing across line graph series (encoding bond, angle, dihedral information), decoupled data embedding per dataset/task, and adherence to strict scaling laws (Zhang et al., 2 Jun 2025).
• Distributed/Edge-oriented LAMs: Use parameter-efficient fine-tuning (e.g., LoRA: $W \approx W_0 + AB$), modular microservice virtualization, looped tensor parallelism, and federated split learning to meet edge device and privacy constraints (Wang et al., 1 May 2025, Wang et al., 6 May 2025).

3. Data Generation, Training Pipelines, and Optimization

LAMs require high-quality, trajectory-rich datasets for multi-step and complex agentic tasks, often obtained through:

• LAM Simulators: Systems that combine dynamic query generation, tool collections, and feedback-driven environments for online action exploration. Trajectory feedback guides the curation of diverse, high-signal training samples (Hoang et al., 2 Jun 2025).
• Multi-Phase Training: For robust planning and tool manipulation, LAMs use multi-stage pipelines—beginning with supervised pretraining (task-plan and task-action pairs), followed by imitation of expert trajectories, self-boosted exploration, and reward-model or PPO-based reinforcement learning:

$$\mathcal{L}_{\text{PPO}}(\theta) = \frac{1}{N} \sum_{i=1}^N \sum_{t=1}^{T_i} \min\left( \frac{\pi_\theta(a_t|s_t)}{\pi_{\theta_{\text{old}}}(a_t|s_t)} \hat{A}_t,\, \text{clip}\left(\frac{\pi_\theta(a_t|s_t)}{\pi_{\theta_{\text{old}}}(a_t|s_t)}, 1-\epsilon,1+\epsilon\right) \hat{A}_t \right)$$

(Wang et al., 13 Dec 2024)

• Data Augmentation and Quality Engineering: Unification of diverse datasets, prompt format shuffling, instruction rephrasing, and synthetic trajectory synthesis (e.g., via APIGen framework) underpin high-performing open-source LAMs (Zhang et al., 5 Sep 2024).
• Low-Rank/Knowledge Distillation: Used for communication-efficient adaptation and federated training in edge and multi-modal semantic communication contexts (Wang et al., 1 May 2025, Ni et al., 28 Mar 2025).

4. Benchmarking, Evaluation Metrics, and Empirical Results

LAMs are benchmarked across task domains using domain-specific and general metrics:

• Agentic Task Leaderboards: Function-calling (e.g., Berkeley Function-Calling Leaderboard), multi-turn reasoning (ToolQuery, ToolBench), showing LAMs (like xLAM-8x22b-r) can outperform models such as GPT-4 and Claude-3 in accuracy, tool use, and program synthesis (Zhang et al., 5 Sep 2024).
• Robotics/Real-Time Control: LRAMs with recurrent backbones achieve state-of-the-art on 432 tasks, with both accuracy and inference latency outperforming transformer baselines (Schmied et al., 29 Oct 2024).
• Wireless and Semantic Comms: Multi-task LAM architectures yield marked gains in VQA, captioning (as measured by BLEU, CIDEr), and channel/beam prediction under variable SNR conditions (Ni et al., 28 Mar 2025, Zhuang et al., 13 May 2025, Guo et al., 4 Aug 2025).
• Robustness and Safety: Emerging audio-language LAMs are subject to adversarial attack and jailbreak testing (AJailBench), revealing that current models lack general robustness and that small, semantically preserved perturbations can compromise safety (Song et al., 21 May 2025).
• Interactive User-Centric Evaluation: Audio LAMs show low correlation ($\tau \leq 0.33$) between static benchmarks and real user preferences, indicating a need for interactive, human-centered evaluation beyond classic test sets (Li et al., 21 Feb 2025).

5. Applications and Real-World Deployments

LAMs underpin applications across:

• Autonomous Agent Systems: Tool orchestration (function calls, API workflows), high-level plan decomposition and execution, GUI interaction for test automation, and end-user task fulfiLLMent (Zhang et al., 27 Nov 2024, Wang et al., 13 Dec 2024).
• Edge and IoT Intelligence: Edge LAM deployment leverages federated and microservice-based inference for traffic management, industrial fault detection, and multi-modal IoT event processing (Wang et al., 6 May 2025).
• Physical Sciences: Large atomistic LAMs (e.g., DPA3) provide universal, transferable PES for molecules, materials, and catalysts, enabling zero-shot performance and scalable scientific modeling (Zhang et al., 2 Jun 2025).
• Wireless and Network Systems: LAMs achieve robust CSI feedback, channel prediction, semantic coding, and realtime beamforming, supporting next-generation (6G) communication architectures (Wang et al., 1 May 2025, Guo et al., 4 Aug 2025).
• Multi-Agent and Service Composition: LAMs, together with Large Reasoning Models (LRMs), anchor automated service composition, with coordinated loops of high-level planning and grounded, dynamically adaptive action (Georgievski et al., 24 Jul 2025).
• Safety/Robustness Frameworks: Audio-language LAMs are increasingly evaluated for their resilience to adversarial manipulation and policy violations, with leading work establishing open benchmarks for future improvement (Song et al., 21 May 2025).

6. Challenges, Limitations, and Future Directions

LAM research is actively addressing:

• Data and Generalization: Scarcity of high-quality, domain-rich, action-annotated datasets remains limiting. Multi-domain, continual learning, and data synthesis are priorities (Ni et al., 28 Mar 2025, Hoang et al., 2 Jun 2025).
• Scalability, Efficiency, and Edge Constraints: Parameter-efficient fine-tuning, LoRA, low-dimensional adaptation, and microservice virtualization are foundational, especially for edge LAM deployment and federated scenarios (Wang et al., 1 May 2025).
• Explainability and Trust: The need for more interpretable architectures and transparent decision boundary tracing is pronounced in high-stakes environments and safety-critical domains (Wang et al., 13 Dec 2024, Jiang et al., 6 May 2025).
• Human-Centric Evaluation and Safety: Developing evaluation protocols that capture user satisfaction and social acceptability is urgent, as static benchmarks fail to predict interactive success (Li et al., 21 Feb 2025). Audio-language LAMs, in particular, are being probed for adversarial resilience and secure deployment (Song et al., 21 May 2025).
• Cooperation of Reasoning and Action Systems: Deep integration of reasoning (LRMs) and action (LAMs) modules is an open field, aiming to bridge complex semantic task planning with robust, real-time execution across heterogeneous environments (Georgievski et al., 24 Jul 2025).

7. Mathematical and Formal Underpinnings

LAMs are grounded in a suite of mathematical formulations reflecting both their generic and domain-specific aims:

• Sequence-to-Action/Policy Optimization: Losses include cross-entropy for trajectory imitation, as well as RL-based objectives (e.g., PPO) for action optimization. In agentic LAMs:

$$\mathcal{L}_{\text{CE}} = \frac{1}{N} \sum_{i=1}^N \sum_{t=1}^{T_i} \mathcal{L}_{\text{CE}}(\text{LAM}(s_t), a_t)$$

• Federated/Edge Optimization: Resource-aware objective:

$$\min L(\hat{w}) + \lambda C(\hat{w}) \qquad \text{subject to}\quad R(\hat{w}) \leq R_0$$

• Scaling Laws in Atomistic LAMs: Generalization error scaling:

$$\text{MAE}(M, S, C) = \alpha_m M^{\beta_m} + \alpha_s S^{\beta_s} + \alpha_c C^{\beta_c}$$

(Zhang et al., 2 Jun 2025)

• Semantic Communication Decoding: Retrieval-augmented generation:

$$\text{Output} = \text{Gen}(z, \text{Retrieve}(\text{query}; K))$$

where $z$ is the semantic encoding and $K$ is the knowledge base (Ni et al., 28 Mar 2025).

---

LAMs represent a pivotal shift in AI from passive interpretation to dynamical, autonomous, and contextually grounded action generation. They underpin advancements in intelligence at both the individual agent and distributed system scales, and their development is driving the evolution of trustworthy, adaptive, and application-specific intelligent systems across science, communication, industry, and beyond.