--- title: Large Execution Models (LEMs) url: https://www.emergentmind.com/topics/large-execution-models-lems type: topic --- # Large Execution Models (LEMs) Searching arXiv for recent papers on Large Execution Models and closely related execution-centric model paradigms. Large Execution Models (LEMs) denote a family of model-centric and system-centric approaches in which the primary object of learning or inference is not merely a final answer but an executable process: a long-horizon action sequence, a state-transition trace, a structured orchestration, a symbolic program, or an execution policy grounded in runtime outcomes. Across the recent literature, the term is used in more than one domain-specific sense. In the broadest research sense, it refers to methods that measure, train, or operationalize reliable execution over time—especially where success depends on maintaining state, respecting constraints, and acting under feedback rather than producing a one-shot textual response [2509.09677]. In a narrower finance-specific sense, LEMs are a unified deep learning framework for benchmark-aware trade execution under bounded time horizons and execution constraints [2509.25211]. Taken together, these lines of work define an execution-centered paradigm in which models are evaluated and optimized by what they can carry through, not only by what they can state. ## 1. Execution as a primary capability A central premise of the recent LEM literature is that standard language-model evaluation obscures the capability that matters in many real systems: completing long sequences of correct actions without failure. “The Illusion of Diminishing Returns” formalizes this using the horizon-success relation $$ P(\text{success}, H) = p^H, $$ where $p$ is per-step accuracy and $H$ is task length [2509.09677]. Defining a horizon threshold $H_s$ by $p^H=s$ yields $$ H_s(p)=\left\lceil \frac{\ln(s)}{\ln(p)} \right\rceil, $$ and for $s=0.5$, $$ H_{0.5}(p)=\left\lceil \frac{\ln(0.5)}{\ln p} \right\rceil \approx -\frac{\ln 2}{\ln p}. $$ Near the high-accuracy regime, with $p=1-\varepsilon$ and $\varepsilon \ll 1$, $$ H_{0.5}\approx \frac{\ln 2}{1-p}. $$ This makes executable horizon scale roughly like the inverse residual error rate, so small absolute gains in local accuracy can imply very large gains in reliably executable task length [2509.09677]. The same paper argues that many apparent reasoning failures of LLMs on long tasks are better understood as failures of execution. To isolate execution from planning and knowledge, it supplies both the knowledge base and the plan, reducing the task to repeated retrieval and composition: $$ S_t = S_{t-1} + \sum_{i=1}^{K} \mathcal D[k_{t,i}], $$ with keys drawn from an in-context dictionary $\mathcal D : \mathcal V \to \mathbb Z$ [2509.09677]. Even when smaller models achieve 100% first-step accuracy, task accuracy collapses over turns; in the simplest multi-turn setting with $K=1$, all models except Gemma3-4B and Qwen3-4B achieve 100% accuracy on the first step, yet even Qwen3-32B drops below 50% task accuracy within 15 turns [2509.09677]. This suggests that execution reliability is a distinct capability variable rather than a by-product of one-step competence. A second execution-centric benchmark, the Self-Execution Benchmark, studies whether models can predict properties of their own future outputs—such as whether they will answer or refuse, which words they will generate in an association task, or which questions will be difficult for them [2508.12277]. The paper argues that current LLMs “lack the ability to execute themselves” and therefore must rely on internal estimations of their own behavior. Across fifteen models, performance is generally poor and often near the 50% random baseline; on the association task, the best reported model, o4-mini, reaches only 63% accuracy [2508.12277]. This suggests that self-prediction is not automatically acquired by increasing model size or reasoning sophistication. These results collectively support a general LEM view: single-step reasoning or final-answer benchmarks do not adequately characterize the ability to sustain a trajectory, preserve state, or anticipate one’s own execution behavior [2509.09677, 2508.12277]. ## 2. Mechanisms of long-horizon execution The execution literature emphasizes that failures are not merely accumulations of independent errors. In long-horizon language-model execution, per-turn accuracy itself degrades with horizon, and one identified cause is self-conditioning: models become more likely to make mistakes when the context contains their own prior mistakes [2509.09677]. In a manipulated-history experiment, turn-100 accuracy declines as the induced prior error rate increases even when context length is effectively controlled, distinguishing self-conditioning from pure long-context degradation [2509.09677]. The paper reports that scaling model size can largely solve the “just long context” part for frontier non-thinking models such as Kimi K2, DeepSeek-V3, and Qwen3-235B-Instruct-2507 under healed histories, but explicitly concludes that scaling model size does not mitigate self-conditioning [2509.09677]. A contrasting result concerns “thinking models,” i.e., models with sequential test-time compute. When thinking is enabled on Qwen3 models, the self-conditioning effect essentially disappears at turn 100, and these models can execute much longer tasks in a single turn [2509.09677]. In the single-turn execution benchmark, larger non-thinking models such as Qwen3 32B, Gemma3 27B, DeepSeek-V3 (670B), and Kimi K2 (1026B) fail to execute even a turn complexity of 2 when required to answer directly without chain-of-thought, whereas thinking counterparts are far stronger: DeepSeek-V3 without CoT fails at even two steps, while DeepSeek-R1 can execute about 200 steps in a single turn, and GPT-5 reaches over 1000 steps [2509.09677]. The paper also finds that majority voting under a matched token budget yields only marginal gains, implying that sequential test-time compute is more valuable than parallel sampling for execution tasks [2509.09677]. A plausible implication is that many LEM designs will need explicit state management and re-grounding mechanisms, not only larger base models. This interpretation is reinforced by code-execution work. “What I cannot execute, I do not understand” studies Execution Tuning (E.T.), in which code LLMs are fine-tuned on real execution traces rather than code treated as static text [2503.05703]. The paper introduces dynamic scratchpads—“self-contained intermediate computations updated by the model rather than accumulated as a history of past computations”—and shows their advantage on long executions up to 14k steps [2503.05703]. The core objectives can be summarized as learning trace distributions $$ p_\theta(\tau \mid P, x) $$ and dynamic next-state or $N$-step state prediction $$ p_\theta(s_{t+N} \mid P, x, s_t, N), \qquad N \in \{1,\dots,10\}, $$ where $P$ is program, $x$ input, and $\tau$ a serialized execution trace [2503.05703]. On CruxEval, individual full-state prediction reaches 96.3% for line-level one-step tuning and 98.8% for instruction-level one-step tuning; output prediction improves from 49.3% for direct output fine-tuning to 78.7% with scratchpad traces and 79.7% with compact scratchpads [2503.05703]. This suggests that execution supervision, especially over explicit state, can materially improve code-execution competence. A related arithmetic line of work, CAEF, trains LLMs to emulate Turing-machine-like transition systems, with the core execution equation $$ s_{i+1}, c_{i+1} = f(s_i, c_i), $$ where $s_i$ is the current machine-like state and $c_i$ the current command [2410.07896]. On LLaMA 3.1-8B, CAEF achieves nearly 100% accuracy across seven arithmetic operations, including 98.6% on 100-digit addition and 98.0% on 100-digit subtraction, substantially above direct-answer baselines in long arithmetic [2410.07896]. This supports the narrower but important claim that explicit state-transition supervision can convert a language model from direct-answer generation toward stepwise execution. ## 3. Executable representations and symbolic substrates A major branch of the literature reframes LEMs not as bigger predictors but as systems that emit executable artifacts. One early precursor is “Structured Thoughts Automaton,” which proposes a formal execution model for auto-regressive language models built around structured prompts, communication channels, and format-constrained generation [2306.10196]. Prompts act as basic blocks, prompt-local questionnaires compile to push-down automata, and branching is handled by a candidate-continuation choice algorithm that scores leaves by the geometric mean of token probabilities: $$ \left(\prod_{i=1}^{d} P(t_i \mid \text{prompt}, t_{= 1.0).to(torch.float32) ``` with iteration to fixpoint [2502.09209]. The paper further introduces soft unification against a vector database, where approximate nearest-neighbor retrieval is materialized back into executable clauses. A plausible implication is that LEMs need not internalize all reasoning in latent states if they can externalize reusable symbolic programs. A code-debugging variant of executable grounding appears in NExT, which trains PaLM 2-L to consume execution traces and generate execution-aware rationales plus repaired code [2404.14662]. The trace representation compresses program execution into inline comments attached to source lines, with per-line state changes and a `NO_CHANGE` marker [2404.14662]. NExT uses self-training over accepted rationale-fix pairs and improves pass@1 on MBPPR from 23.2 to 49.3, while retaining substantial gains even when traces are absent at test time [2404.14662]. This suggests that execution grounding can improve downstream reasoning even when execution artifacts are not available at inference. ## 4. Orchestration, research, and execution-grounded systems A second major interpretation of LEMs centers on executable coordination rather than symbolic state transition. LEMON defines an LLM-based orchestrator that emits an executable YAML specification for a multi-agent system [2605.14483]. Each specification defines roles, duties, capacities, dependencies, and execution steps. Formally, for task instance $x$, $$ y \sim \pi_\theta(\cdot \mid x), \qquad y \in \mathcal Y, $$ and the orchestration compiles to a graph $$ G = \mathcal C(y) = (V,E). $$ Each agent has a role specification $$ R_v = (a_v,b_v,d_v) $$ and a capacity level $L_v$ selecting a worker model, while execution utility is $$ \mathcal J(\theta) = \mathbb E_{x \sim \mathcal D,\, y \sim \pi_\theta(\cdot \mid x)}[\mathcal U(x,y)]. $$ LEMON combines supervised warm-starting with orchestration-level GRPO and a localized counterfactual objective that edits role, capacity, or dependency fields and applies the reward contrast only to the edited spans [2605.14483]. On six reasoning and coding benchmarks, LEMON reaches 90.72 average versus 88.53 for Tree-MAS and 86.76 for self-consistent CoT, while also improving token efficiency [2605.14483]. This is a restricted but concrete instance of a model whose direct output is an executable artifact rather than a final answer. Execution-grounded automated AI research extends this principle to algorithmic discovery. “Towards Execution-Grounded Automated AI Research” builds an automated executor that converts LLM-generated research ideas into patched codebases, launches GPU experiments, and returns empirical reward [2601.14525]. Two environments are defined: a GRPO post-training environment and a nanoGPT pre-training environment. Execution-guided evolutionary search over ideas finds a post-training method reaching 69.4% versus a 48.0% GRPO baseline, and a pre-training recipe that reaches the target validation loss in 19.7 minutes versus 35.9 minutes for the nanoGPT baseline, all within ten search epochs [2601.14525]. The paper then studies reinforcement learning from execution reward and finds a different dynamic: average reward rises, but upper-bound performance does not improve because the ideator model collapses toward simple, easy-to-execute ideas [2601.14525]. This suggests that for open-ended research-style LEMs, search over diverse executable trajectories may currently be more effective than direct policy optimization on scalar reward. Execution evaluation infrastructure also matters. ExeRScope does not propose a new model, but it provides post-hoc analytic tooling for code execution reasoning results across benchmarks such as CRUXEval, CodeMind, and REval [2501.18482]. It extracts static and dynamic program properties—constructs, complexity measures, loop length, and variable types—and correlates them with execution-reasoning accuracy. Across four benchmarks and six models, the tool shows that nested and iterative control flow, higher cyclomatic complexity, longer loops, and non-primitive output types systematically reduce execution reasoning performance [2501.18482]. A plausible implication is that LEM development requires not only new training and inference schemes but also property-aware diagnostics of where execution breaks. ## 5. Domain-specific usage: EEG models and trade execution The acronym LEM is not used exclusively for execution-centered language or code systems. In EEG research, “Large EEG Models” are also abbreviated LEMs, and the paper ECHO explicitly contrasts its decoder-centric sequence-to-sequence formulation with prior encoder-centric “Large EEG Models” [2509.22556]. In that literature, a LEM is a model trained on large-scale EEG corpora to learn representations that generalize across heterogeneous datasets and paradigms. ECHO criticizes the dominant encoder-plus-small-classifier pattern, $$ f(\mathbf X \mid t) = \mathcal C(\mathcal E(\mathbf X; \theta_d); \phi_d) \to \mathbf Y, $$ for creating a decoder bottleneck, and instead reformulates EEG modeling as sequence-to-sequence learning over serialized EEG signals, support examples, task identifiers, and labels [2509.22556]. The decoder-centric formulation enables in-context learning via support EEG examples and task/label tokens, and in multi-task evaluation ECHO outperforms state-of-the-art single-task LEM baselines across many settings [2509.22556]. Although this usage of “LEM” is domain-specific and unrelated to the generic “Large Execution Models” agenda, it is a salient source of terminological ambiguity. A second distinct usage is the finance paper “LEMs: A Primer On Large Execution Models,” where LEMs are neural execution models for order execution under bounded time horizons [2509.25211]. Here the term “execution” refers not to program or agent execution but to trade execution. The model decouples a shared market-information encoder from scenario-specific execution-allocation networks. Inputs are market and contract features $$ X \in \mathbb R^{B \times T \times D}, $$ and outputs are allocation tensors over time, scenario, side, and target type. The architecture combines feature-wise embeddings, Variable Selection Networks, Temporal Kolmogorov-Arnold Networks, causal multi-head attention, and step-wise FusedMLP allocation heads [2509.25211]. Execution feasibility is enforced by budget conservation $$ R_{s+1}=R_s-\alpha_s,\qquad \sum_{s=1}^{S}\alpha_s=1,\qquad \alpha_S=R_S, $$ and differentiable clipping $$ \mathcal S(x,u,\lambda)=x\cdot\sigma(\lambda(u-x))+u\cdot(1-\sigma(\lambda(u-x))). $$ The loss is defined over benchmark-relative deviations between achieved price and VWAP/TWAP-style benchmarks, with separate handling for volume-target and notional-target orders [2509.25211]. Empirically, the paper reports strong benchmark-relative improvements on Binance crypto data and DOW Jones equities, especially when the model can stop anywhere in a bounded interval $[T_{\min},T_{\max}]$ [2509.25211]. This is a legitimate “Large Execution Models” line, but it belongs to quantitative finance rather than the broader execution-centered foundation-model agenda. The coexistence of these meanings is an important source of confusion. In one usage, “execution” means carrying out a long-horizon cognitive or agentic process; in another, it means market order execution; and in EEG work the acronym LEM refers to “Large EEG Models” rather than execution at all [2509.22556, 2509.25211]. ## 6. Limitations, controversies, and open directions Several recurring limitations appear across the literature. First, execution competence remains brittle even when local task competence is high. Long-horizon LLM benchmarks show that models with perfect or near-perfect first-step accuracy can still fail within a handful of turns, and self-conditioning is not removed by scale alone [2509.09677]. Second, many execution-centered systems rely on carefully engineered interfaces: state serializations in CAEF [2410.07896], dynamic scratchpads and iterator annotations in E.T. [2503.05703], code-slice construction in AutoExe [2505.13452], or structured YAML schemas in LEMON [2605.14483]. This suggests that a substantial fraction of present-day LEM capability is scaffolded rather than native. Third, execution-grounded learning from scalar reward is not straightforward. In automated AI research, RL from execution reward improves average reward but not best-found performance because it collapses onto simple, executor-friendly ideas [2601.14525]. A plausible implication is that discovery-oriented LEMs may require diversity-preserving search or archive-based optimization rather than conventional expected-reward maximization. Fourth, symbolic and retrieval-augmented execution systems remain only partially sound. Logic-program extraction produces executable theories, but the underlying rules may be noisy or inconsistent [2502.09209]. Soft unification over vector databases is explicitly approximate and is useful as abduction rather than as a logically sound replacement for exact unification [2502.09209]. Similarly, AutoExe concedes that theorem-prover-backed systems still dominate in exact arithmetic or strongly formal domains even if LLM-based code reasoning is better at loops, heap reasoning, and natural-language specifications [2505.13452]. Finally, evaluation remains fragmented. Code-execution papers measure output prediction, state prediction, or repair [2503.05703, 2404.14662, 2501.18482]; long-horizon language papers measure turns, complexity, and self-prediction [2509.09677, 2508.12277]; orchestration papers measure benchmark scores and token efficiency [2605.14483]; execution-grounded research measures discovered method quality [2601.14525]. This suggests that “Large Execution Models” is currently better understood as a cross-cutting research direction than as a single standardized model class. A plausible synthesis is that the field is converging on three shared principles. First, execution should be represented explicitly, whether as action trajectories, symbolic programs, orchestration graphs, or state-transition traces. Second, execution should be evaluated by carried-through outcomes under constraints, not only by one-shot accuracy or textual plausibility. Third, models that externalize or inspect execution—through traces, symbolic artifacts, structured controllers, or runtime feedback—appear systematically more reliable than models asked to infer long-horizon behavior implicitly. That does not yet define a single canonical LEM architecture, but it does define an emerging paradigm centered on executable reasoning, runtime grounding, and constraint-respecting behavior [2509.09677, 2503.05703, 2605.14483, 2601.14525]. Source: https://www.emergentmind.com/topics/large-execution-models-lems