---
title: 'Program-of-Layers (PoLar): Dynamic LLM Execution'
url: https://www.emergentmind.com/topics/program-of-layers-polar
type: topic
---

# Program-of-Layers (PoLar): Dynamic LLM Execution

Searching arXiv for the cited PoLar paper and closely related context papers.
Program-of-Layers (PoLar) is an inference framework for large language models in which the layers of a pretrained model are treated as a library of atomic functions that can be composed into an input-specific execution program, rather than being executed only in the default fixed order and fixed depth. In this formulation, pretrained layers can be skipped or looped, so that a frozen model admits multiple latent computations beyond the standard forward pass; empirically, many of these alternative programs are shorter than the original execution, and some correct errors made by standard inference [2606.06574].

## 1. Definition and formal structure

PoLar formalizes a pretrained LLM with \(D\) transformer layers as a collection of functions
\[
f_i : \mathbb{R}^{T \times d} \rightarrow \mathbb{R}^{T \times d}, \quad i \in \{0,\ldots,D-1\}.
\]
The conventional execution path is the fixed-depth program
\[
\pi_0 = (0,1,\ldots,D-1),
\]
whereas PoLar permits an input-dependent program
\[
\pi = (i_1, i_2, \ldots, i_K), \quad i_k \in \{0,\ldots,D-1\},
\]
with induced computation
\[
F_{\pi} = f_{i_K} \circ \cdots \circ f_{i_1}.
\]

Within this formulation, **skipping** omits layers from the default sequence, and **repeating** or **looping** reuses layers. The central conjecture is that, for a given input, multiple distinct executions of programs-of-layers can yield correct predictions. This shifts the object of analysis from the pretrained parameters alone to the pair consisting of a frozen pretrained model and an execution program over its layers.

The learned PoLar method does not predict arbitrary layer-index sequences directly. Instead, it represents execution programs by partitioning depth into contiguous segments
\[
[0=s_1, s_2),\ [s_2, s_3),\ \ldots,\ [s_M, s_{M+1}=D),
\]
with a maximum segment length
\[
s_{j+1}-s_j \le K_{\max},
\]
and assigns each segment one of three operations:
\[
\{\textsf{skip},\textsf{keep},\textsf{repeat}\}.
\]
The semantics are
\[
\textsf{skip} : \emptyset,\qquad
\textsf{keep} : [s_j,\ldots,s_{j+1}-1],\qquad
\textsf{repeat} : [s_j,\ldots,s_{j+1}-1,\ s_j,\ldots,s_{j+1}-1].
\]

This design constrains the program space to short contiguous modules with bounded local recurrence. The stated constraint is \(K_{\max}=4\), and the repeat operator corresponds to one additional pass through the selected segment [2606.06574].

## 2. Training-free discovery of alternative executions

Before introducing a learned predictor, PoLar establishes the existence of alternative valid programs through a training-free analysis based on Monte Carlo Tree Search. Search begins from the default program \(\pi_0=(0,1,\ldots,D-1)\) and explores skip and repeat operations over contiguous subsequences. In the MCTS experiments, the skip length \(k\) and repeat count \(r\) are bounded by \(4\).

A program is valid for an input if it yields the correct prediction, with reward
\[
r(\pi, x) = \mathbf{1}\{F_{\pi}(x) = y\}.
\]
The appendix gives the tree policy as
\[
\mathrm{UCB}(\pi) = \frac{R(\pi)}{v(\pi)} + c \sqrt{\frac{\ln V}{v(\pi)} - \lambda \frac{|\pi|}{D},
\]
balancing cumulative reward, exploration, and a preference for shorter programs.

The principal empirical findings are quantitative. On DART-Math, **Loop** consistently outperforms **Skip**, and **Skip+Loop** is best in every reported setting. For example, on DART-Math level DM-1, LLaMA-3.2-3B-Instruct improves from **37.9** for Base to **84.7** for Skip+Loop; Qwen2.5-3B-Instruct improves from **25.4** to **87.4**; and Qwen3-8B improves from **40.7** to **91.3**. On DM-5, the corresponding Base \(\rightarrow\) Skip+Loop changes are **27.1 \rightarrow 59.1**, **1.2 \rightarrow 44.5**, and **8.8 \rightarrow 45.7**.

The search analysis also characterizes the structure of valid programs. Among inputs already correct under the original model, **75.5%** admit shorter valid programs. Among inputs initially wrong but corrected by an alternative program, **36.2%** admit shorter correcting programs. Segment structure is highly local: **54.5%** of segments consist of a single layer, **over two-thirds** contain at most two consecutive layers, and segments with predominantly non-consecutive layers account for **less than 3.2%** of cases. Most segments are repeated **at most once**. These observations motivate the learned segment-and-operation representation used by PoLar [2606.06574].

## 3. Program prediction network and execution procedure

Because MCTS is too expensive for routine inference, PoLar introduces a lightweight prediction network that outputs a program before the frozen LLM is executed. The base LLM remains fully frozen; only the predictor is trained.

The predictor first encodes the input \(x\) with the frozen embedding model **Qwen3-Embedding-0.6B**, producing
\[
\mathbf{H}=E(x)\in\mathbb{R}^{T\times d_q},
\]
followed by projection
\[
\tilde{\mathbf{H}}=\mathbf{H}\mathbf{W}_h \in \mathbb{R}^{T\times d}.
\]
Each depth position has a learnable layer embedding
\[
\mathbf{e}_i\in\mathbb{R}^{d}, \qquad \mathbf{E}\in\mathbb{R}^{D\times d},
\]
which serves as a query in multi-head cross-attention:
\[
\mathbf{X}=MHA(\mathbf{Q},\mathbf{K},\mathbf{V}), \quad \mathbf{Q}=\mathbf{E},\ \mathbf{K}=\tilde{\mathbf{H}},\ \mathbf{V}=\tilde{\mathbf{H}},
\]
yielding \(\mathbf{X}\in\mathbb{R}^{D\times d}\). A lightweight transformer over the depth dimension then produces
\[
\mathbf{X}'=Enc_{\text{layer}}(\mathbf{X})\in\mathbb{R}^{D\times d}.
\]

Two heads predict segmentation and operations:
\[
\boldsymbol{\ell}^{\text{seg}}=\mathbf{X}'\mathbf{W}_{\text{seg}}+\mathbf{b}_{\text{seg}} \in\mathbb{R}^{D},
\]
\[
\boldsymbol{\ell}^{\text{op}}=\mathbf{X}'\mathbf{W}_{\text{op}}+\mathbf{b}_{\text{op}} \in\mathbb{R}^{D\times 3}.
\]
These define
\[
p^{\text{seg}}_i=\sigma(\ell^{\text{seg}}_i), \qquad
\mathbf{p}^{\text{op}}_i=Softmax(\boldsymbol{\ell}^{\text{op}}_i).
\]

Training supervision comes from valid execution programs collected offline by MCTS and parsed into a binary segmentation mask \(\mathbf{z}^{\text{seg}*}(x)\) and operation labels \(\mathbf{z}^{\text{op}*}(x)\). The losses are a binary cross-entropy segmentation term
\[
\mathcal{L}_{\text{seg}} = -\sum_{i=0}^{D-1}\Big[ \mathbf{z}^{\text{seg}*}_i\log p^{\text{seg}}_i +(1-\mathbf{z}^{\text{seg}*}_i)\log(1-p^{\text{seg}}_i) \Big],
\]
and a masked cross-entropy operation term
\[
\mathcal{L}_{\text{op}} = -\sum_{i=0}^{D-1} m_i\cdot \log \mathbf{p}^{\text{op}}_i\big[\mathbf{z}^{\text{op}*}_i\big],
\qquad m_i=\mathbf{z}^{\text{seg}*}_i,
\]
with total objective
\[
\mathcal{L}=\mathcal{L}_{\text{seg}}+\mathcal{L}_{\text{op}}.
\]

At inference time, PoLar deterministically thresholds segmentation logits to produce segment boundaries, enforces the maximum segment length constraint, computes operation probabilities at each segment start, and then applies a small beam search over segment-level operation assignments. The frozen pretrained LLM is executed once per candidate program. This distinguishes PoLar from sequential layer-wise routers: it predicts the execution program upfront, over contiguous layer segments, rather than making local routing decisions during the forward pass [2606.06574].

## 4. Empirical performance, efficiency, and generalization

PoLar is evaluated on fully frozen instruction-tuned LLMs: **LLaMA-3.2-3B-Instruct**, **Qwen1.5-MoE-A2.7B-Chat**, **Qwen2.5-3B-Instruct**, and **Qwen3-8B**. The principal in-distribution benchmark is **DART-Math**, with five difficulty levels **DM-1** through **DM-5**; each level contains **2,000 examples**, split into **1,250 train**, **250 validation**, and **500 test**. Evaluation uses direct prompting only, with the model required to output only the final answer as \(\boxed{ANSWER}\), and a maximum output length of **50 tokens**.

The primary metric is **pass@\(\mathbf{k}\)** accuracy. On LLaMA-3.2-3B-Instruct, pass@1 improves from **42.4 / 28.6 / 27.2 / 27.6 / 28.6** for Base across DM-1 to DM-5 to **46.2 / 30.2 / 28.2 / 28.8 / 30.2** for PoLar. At pass@5, Base (sampling) gives **47.6 / 43.2 / 32.8 / 32.8 / 35.6**, whereas PoLar reaches **68.4 / 48.0 / 46.0 / 40.4 / 45.8**, for gains of **+20.8**, **+4.8**, **+13.2**, **+7.6**, and **+10.2**.

Comparable pass@5 gains appear on the other backbones. For **Qwen1.5-MoE-A2.7B-Chat**, Base (sampling) gives **40.0 / 25.6 / 18.6 / 15.0 / 11.8** and PoLar gives **62.0 / 44.0 / 33.0 / 25.4 / 23.2**. For **Qwen2.5-3B-Instruct**, Base gives **42.2 / 30.2 / 20.4 / 15.8 / 13.0** and PoLar gives **59.8 / 40.6 / 28.2 / 18.0 / 22.8**. For **Qwen3-8B**, Base gives **48.4 / 27.8 / 13.6 / 13.4 / 10.6** and PoLar gives **55.6 / 44.8 / 25.4 / 18.8 / 17.8**.

The efficiency results are also explicit. For **Qwen1.5-MoE-A2.7B-Chat with 24 layers**, the Base model uses **24.00** average layers and **373.45 ms** latency. PoLar on **DM-1** uses **23.30** average layers, **311.41 ms** latency, **0.83x** relative runtime, and an accuracy gain of **+5.8**. On **DM-5**, it uses **23.76** average layers, **353.31 ms** latency, **0.95x** relative runtime, and an accuracy gain of **+1.2**. The overhead breakdown is: **13.23 ms** for one LLM layer, **0.99 ms** for the predictor head, **0.11 ms** for beam search, **1.95 ms** for the encoder, and **3.05 ms** total additional overhead, equal to **0.8%** of a full forward pass and about **0.23 equivalent layers**.

Out-of-distribution evaluation trains PoLar on the union of DART-Math training data across all difficulties and tests zero-shot on **ASDiv**, **MAWPS**, and **MMLU-Pro** subject subsets. For Qwen1.5-MoE-A2.7B-Chat, PoLar improves over Base on all reported OOD tasks, including **ASDiv: 59.1 \(\rightarrow\) 63.8**, **MAWPS: 41.7 \(\rightarrow\) 46.7**, **MMLU-Pro Math: 13.9 \(\rightarrow\) 18.5**, **Phys: 15.6 \(\rightarrow\) 20.3**, **Chem: 13.8 \(\rightarrow\) 18.3**, **Law: 16.6 \(\rightarrow\) 20.4**, and further improvements across Eng, Other, Econ, Health, Psych, Bus, Bio, Phil, and Hist. Appendix results report similarly broad gains on the other backbones [2606.06574].

## 5. Position within dynamic-depth research and terminological scope

PoLar is situated within adaptive-compute work, but its program space is broader than early-exit and skip-only approaches. Early-exit methods such as FastBERT, DeeBERT, PABEE, and Faster Depth-Adaptive Transformers reduce depth while preserving the original order. Skip-only methods such as ShortGPT, MindSkip, FlexiDepth, and LayerDrop-style approaches omit layers but do not model recurrence. Recurrent-depth systems such as Universal Transformers, looped transformers, and Inner Thinking Transformer rely on architectural redesign or special recurrent training. PoLar differs by applying dynamic execution to **frozen pretrained models**, with both skipping and repetition inside one unified program space.

The closest comparison identified in the source material is **DR.LLM**, which also learns from MCTS supervision and supports both skipping and repetition. The stated distinction is that DR.LLM performs sequential layer-wise routing based on intermediate hidden states and is limited to single-layer recurrence, whereas PoLar predicts the entire execution program upfront and operates on contiguous layer segments. This enables patterns such as
\[
4 \rightarrow 5 \rightarrow 4 \rightarrow 5,
\]
which lie outside a single-layer routing space.

The term “PoLar” also requires disambiguation. It is distinct from **“PoLAR: Factorizing Extent and Mode in Latent Actions for Robot Policy Learning”** [2606.21139], where “PoLAR” expands to **Polar Latent Actions with Radial structure** and refers to latent-action geometry in robot learning rather than dynamic LLM execution. It is also distinct from **“Polar probe linearly decodes semantic structures from LLMs”** [2605.14125], which studies a geometric “polar” code in layer-local subspaces and reports that semantic structures are most linearly decodable in middle layers. A plausible implication is that contemporary usage has attached “polar” or “PoLar” to several different research programs—dynamic execution, latent-action geometry, and geometric semantic probing—that are methodologically adjacent in their focus on internal structure, but not terminologically interchangeable [2606.06574].

## 6. Limitations, caveats, and open directions

Several limitations are explicit. The full program space is exponentially large, which is why MCTS is used only as an offline diagnostic and data-generation mechanism rather than as a practical inference-time procedure. The learned representation is restricted to **contiguous segments**, **\(K_{\max}=4\)**, and only the operation set \(\{\textsf{skip},\textsf{keep},\textsf{repeat}\}\), with recurrence limited to a single additional pass. The paper notes that deeper repeated execution could be represented by extending the vocabulary to \(\{\textsf{repeat-}2,\ldots,\textsf{repeat-}k\}\), but does not do so.

The experimental scope is also narrow relative to the generality of the claim. The reported evaluations focus mainly on **mathematical reasoning**, **arithmetic word problems**, and **MMLU-Pro** subject subsets. The work does not provide a detailed per-layer heatmap or semantic analysis of which absolute depth positions are usually skipped or repeated. It also does not present a formal study of activation-distribution shift or instability induced by looping pretrained layers, beyond the empirical observation that reuse can help and that most useful loops are shallow.

The impact discussion identifies additional open questions around **robustness**, **calibration**, **interpretability**, and **safety**, especially because execution becomes input-dependent. This suggests that PoLar is not only a dynamic-depth method but also a change in the unit of explanation for LLM inference: instead of treating the standard forward pass as the model, it treats the latent computation path as a variable object that must itself be predicted, audited, and interpreted [2606.06574].

Source: https://www.emergentmind.com/topics/program-of-layers-polar