Function Encoder (FE) Policies

• Function Encoder (FE) Policies are representations that decompose policies into a fixed set of basis functions with dynamically computed coefficients from task data.
• They leverage an encoder–decoder architecture and Hilbert-space methods to enable robust interpolation, extrapolation, and efficient transfer in diverse control scenarios.
• FE policies facilitate modular design in federated reinforcement learning and visuomotor control, supporting rapid adaptation and closed‐form aggregation without full retraining.

Searching arXiv for papers on function encoders and FE policies to ground the article. Function Encoder (FE) policies are policy representations in which a policy is either represented directly as a weighted combination of basis functions or conditioned on a compact coefficient vector computed from data by a function encoder. Across reinforcement learning, differentiable predictive control, federated reinforcement learning, and visuomotor imitation, the common structure is an encoder–decoder decomposition: a finite set of basis functions or a fixed encoder defines a function space, task-specific coefficients are inferred from data, and a policy head maps state together with those coefficients to values, logits, or actions. In the Hilbert-space formalization, FE learns basis functions $g_1,\ldots,g_k$, computes coefficients $c$, and defines $$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$; in application-specific instantiations, the same idea appears as linear readouts over fixed nonlinear encoders, FE-conditioned policies $u_k=\pi_\psi(x_k,z,r_k)$, and standardized latent interfaces between perception and control (Ingebrand et al., 30 Jan 2025, Iqbal et al., 7 Nov 2025, Hou et al., 27 May 2026, Jian et al., 2024, Ingebrand et al., 2024).

1. Core definition and architectural forms

In the most general formulation, FE learns a finite set of differentiable basis functions $g_1,\ldots,g_k: X \to Y$ parameterized by $\theta$, computes task-specific coefficients $c \in \mathbb{R}^k$ from data, and reconstructs a function by

$$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$

Within this framework, a policy is either a function in a Hilbert space represented directly as

$\pi(x) = \sum_j c_j g_j(x),$

or a downstream policy network conditioned on the coefficient vector $c$ inferred from task-defining data (Ingebrand et al., 30 Jan 2025).

Applied FE policy designs in the cited literature fall into three recurring forms. In federated reinforcement learning, each client uses a fixed encoder $c$0 and a linear readout. For policy-based FE, logits are

$c$1

In differentiable predictive control, the encoder computes an FE embedding $c$2 from short data and the control policy is explicitly conditioned on it,

$c$3

In zero-shot reinforcement learning, the FE representation of a reward or transition function is a context vector $c$4, and standard RL function approximators are conditioned by concatenation: $c$5 The common FE property across these variants is that task information enters through coefficients or a fixed encoder interface rather than through end-to-end policy retraining (Hou et al., 27 May 2026, Iqbal et al., 7 Nov 2025, Ingebrand et al., 2024).

A practically important distinction is that the encoder map need not itself be a neural network. In zero-shot RL via function encoders, the paper states that “the encoder $c$6 is not a neural network: it is the linear map formed by inner products with learned bases.” In federated RL, the key FE property is instead that $c$7 is fixed, possibly randomized/nonlinear, and the trainable parameters enter linearly, so aggregation and compilation can be done in closed form (Ingebrand et al., 2024, Hou et al., 27 May 2026).

2. Function-space geometry, least-squares encoding, and transfer

The Hilbert-space treatment gives FE policies a geometric interpretation. Let $c$8 be a Hilbert space of functions $c$9, let $$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$0 be source predictors, and let FE learn a subspace $$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$1. The framework distinguishes three kinds of transfer: $$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$2 Type 1 is interpolation within the convex hull; Type 2 is extrapolation to the linear span; Type 3 is extrapolation outside the span via orthogonal projection onto the learned subspace. For Type 3, FE computes

$$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$3

where $$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$4 and $$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$5. This least-squares geometry is the formal mechanism by which FE supports interpolation, extrapolation within span, and best-in-subspace approximation outside span (Ingebrand et al., 30 Jan 2025).

Training is correspondingly based on least-squares reconstruction. FE uses Monte Carlo approximation of the Hilbert-space inner product,

$$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$6

solves the normal equations

$$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$7

and optimizes basis parameters $$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$8 through the average source reconstruction error together with the regularizer

$$F_\theta(D)(x)=D_\theta(x,E(D))=\sum_{j=1}^k c_j\,g_j(x)$$9

The paper proves a universal approximation theorem: for any $u_k=\pi_\psi(x_k,z,r_k)$0 and any $u_k=\pi_\psi(x_k,z,r_k)$1, there exist neural network basis functions $u_k=\pi_\psi(x_k,z,r_k)$2, a finite $u_k=\pi_\psi(x_k,z,r_k)$3, and coefficients $u_k=\pi_\psi(x_k,z,r_k)$4 such that

$u_k=\pi_\psi(x_k,z,r_k)$5

A plausible implication is that FE policies inherit approximation properties from the learned function basis rather than from a single monolithic policy network (Ingebrand et al., 30 Jan 2025).

The zero-shot RL formulation instantiates the same principle with scalar perturbing functions $u_k=\pi_\psi(x_k,z,r_k)$6, such as reward functions $u_k=\pi_\psi(x_k,z,r_k)$7 or component-wise transition functions. FE represents

$u_k=\pi_\psi(x_k,z,r_k)$8

and computes coefficients from few-shot task data $u_k=\pi_\psi(x_k,z,r_k)$9 by

$g_1,\ldots,g_k: X \to Y$0

Because the encoder mapping is linear, “if $g_1,\ldots,g_k: X \to Y$1, then $g_1,\ldots,g_k: X \to Y$2.” The runtime mechanism for zero-shot transfer is therefore to collect input-output samples of the current reward or transition function, compute $g_1,\ldots,g_k: X \to Y$3, and deploy the FE-augmented policy without additional learning (Ingebrand et al., 2024).

3. Linear-head FE policies in reinforcement learning and federation

FedQHD provides the most explicit policy-theoretic account of FE in federated reinforcement learning. Each client $g_1,\ldots,g_k: X \to Y$4 uses a fixed encoder

$g_1,\ldots,g_k: X \to Y$5

and a linear readout. For value-based FE,

$g_1,\ldots,g_k: X \to Y$6

For policy-based FE,

$g_1,\ldots,g_k: X \to Y$7

When all clients share the same encoder $g_1,\ldots,g_k: X \to Y$8, function-space consensus equals weighted parameter averaging: $g_1,\ldots,g_k: X \to Y$9 and $\theta$0 uses softmax over $\theta$1. When encoders are heterogeneous, the server builds a federated teacher on a shared anchor-state set and each client compiles that teacher into its local representation by a single ridge solve,

$\theta$2

The same closed-form compilation used for Q-values therefore extends to FE policies through logits (Hou et al., 27 May 2026).

A central analytic object is the federation gap, defined relative to a client-specific oracle projection. FedQHD decomposes this gap into encoder heterogeneity, anchor conditioning, and ridge shrinkage: $\theta$3 The paper identifies $\theta$4 as the well-conditioned regime in which the anchor-induced amplification does not dominate, and states that if $\theta$5 and $\theta$6, then $\theta$7 for a modest constant $\theta$8. This makes the anchor-to-dimension ratio a concrete FE design rule in heterogeneous federation (Hou et al., 27 May 2026).

Empirically, on four continuous-state, discrete-action tasks—CartPole, Acrobot, LunarLander, and MountainCar—FedQHD “matches or outperforms FedAvg-DQN and heterogeneous distillation baselines on most settings,” is “best non-oracle in 5/8 task variants,” and is “markedly faster than DQN-based baselines and distillation (minutes vs tens of minutes per run for $\theta$9 clients).” In heterogeneous settings, “compiled error scales $c \in \mathbb{R}^k$0 (log–log slope $c \in \mathbb{R}^k$1),” and ablations “confirm the phase transition around $c \in \mathbb{R}^k$2” (Hou et al., 27 May 2026).

4. FE-conditioned predictive control and zero-shot adaptation

In differentiable predictive control, FE policies are explicit neural control laws conditioned on an FE embedding of the underlying dynamics. The FE-NODE model represents a parametric family of dynamics by a coefficient vector $c \in \mathbb{R}^k$3 that linearly combines learned neural ODE basis functions: $c \in \mathbb{R}^k$4 The control policy is

$c \in \mathbb{R}^k$5

or equivalently $c \in \mathbb{R}^k$6. The FE embedding is inferred from short online measurement data $c \in \mathbb{R}^k$7 by a Tikhonov-regularized least-squares solve,

$c \in \mathbb{R}^k$8

where $c \in \mathbb{R}^k$9 and $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$0 are evaluated via Monte Carlo from measured data. The paper states that coefficients are recomputed in closed form at each step and that “no gradient through $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$1” is used during basis learning (Iqbal et al., 7 Nov 2025).

The differentiable predictive control objective is end-to-end over FE-NODE rollouts: $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$2 with dynamics

$$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$3

At deployment, zero-shot adaptation consists of collecting a small probe dataset $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$4, computing $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$5, and applying the trained policy immediately “without any online optimization or retraining.” The reported practical configuration uses “online coefficient estimation: $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$6 samples,” a “4-layer MLP, hidden size 256” for the policy, RK4 integration, Adam, and “single NVIDIA RTX 5090 GPU; PyTorch + NeuroMANCER” (Iqbal et al., 7 Nov 2025).

The empirical comparison against white-box MPC is reported as follows:

System	FE-DPC	WB-MPC
Van der Pol	0.002683 (0.53s)	0.002653 (1.21s)
Two Tank	0.008452 (1.13s)	0.004164 (6.75s)
GO	0.180299 (5.89s)	0.032320 (136.07s)
Quadrotor	0.022003 (1.93s)	0.024208 (155.85s)

The paper reports “speedups: ~2.3× to ~71.5× over MPC at inference, with competitive accuracy,” and further states that FE-DPC “maintains stability/tracking under abrupt dynamics changes mid-simulation in all domains.” A plausible implication is that FE policies are especially effective when task variability is concentrated in a low-dimensional coefficient space that can be identified quickly from probe data (Iqbal et al., 7 Nov 2025).

5. Perception-conditioned FE policies in visuomotor control

Perception Stitching operationalizes FE policies as modular visuomotor policies with a standardized encoder–policy interface. The policy decomposes into perception and control: $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$7 for MLP-based policies, and

$$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$8

for RNN-based policies. The visual encoder is “ResNet-18 + spatial softmax + linear projection to a 256-D latent per camera,” but this latent is not consumed directly. Instead, after normalization, the method computes a 256-D relative representation by cosine similarities to 256 anchors: $$E(D) := c(D) \in \mathbb{R}^k,\qquad D_\theta(x,c) := \sum_{j=1}^k c_j\, g_j(x \mid \theta_j),\qquad F_\theta(D)(x) := D_\theta(x, E(D)).$$9 With two cameras and an 8-D proprioceptive input embedded through a 64-D linear layer, the action decoder receives a 576-D input. This relative-representation interface is the mechanism that permits direct stitching of encoders trained under different visual conditions (Jian et al., 2024).

The training objective is behavior cloning with an additional disentanglement regularizer: $\pi(x) = \sum_j c_j g_j(x),$0 The disentanglement loss uses off-diagonal covariance penalties on the pre-relative-representation latent $\pi(x) = \sum_j c_j g_j(x),$1: $\pi(x) = \sum_j c_j g_j(x),$2 Anchor selection is by k-means over an expert dataset in one environment, followed by exact trajectory replay in the second environment so that anchor indices correspond across domains. Zero-shot stitching then plugs, for example, $\pi(x) = \sum_j c_j g_j(x),$3 and $\pi(x) = \sum_j c_j g_j(x),$4 into the action decoder $\pi(x) = \sum_j c_j g_j(x),$5 and deploys in the mixed environment without any fine-tuning (Jian et al., 2024).

The reported results emphasize zero-shot transfer under substantial visual shift. On basic simulation tasks, “PeS averages $\pi(x) = \sum_j c_j g_j(x),$6 (Push) and $\pi(x) = \sum_j c_j g_j(x),$7 (Lift).” On hard tasks, “PeS achieves averages $\pi(x) = \sum_j c_j g_j(x),$8 (Can), $\pi(x) = \sum_j c_j g_j(x),$9 (Stack), $c$0 (Door), while all baselines are $c$1 on average across these tasks.” In real-world zero-shot transfer, PeS reports “Reach (100%), Push (85%), Lift (80%), Stack (45%) success,” while the Devin et al. 2017 baseline is “0% across all tasks.” The paper further states that PeS “dramatically reduces cosine distances (mean 0.050 vs 0.828 for baseline) and lowers L2 distances (mean 3.78 vs 5.76)” in latent analysis. This suggests that, in perception-driven FE policies, the standardized interface may be at least as important as the encoder backbone itself (Jian et al., 2024).

6. Assumptions, limitations, and current points of debate

The FE literature is explicit that these policy constructions depend on strong structural assumptions. In FedQHD, “encoder stationarity” is required: the encoder $c$2 must remain fixed during aggregation, because “learning $c$3 breaks closed-form consensus.” Performance also depends on “anchor coverage,” since anchors must cover client state supports, and on “linear head sufficiency,” because severe heterogeneity or too-small $c$4 yields a large heterogeneity floor $c$5. The paper also states that “highly nonlinear policies outside the encoder span are not recoverable via linear heads; compilation discards the unrepresentable component (projection residual), per the invisibility theorem” (Hou et al., 27 May 2026).

In FE-conditioned predictive control, the assumptions are different but comparably restrictive. The method assumes “smooth, differentiable dynamics and differentiable solver/integrator,” “availability of short probe data $c$6 at test time for coefficient estimation,” and offline data coverage that adequately spans target variability. The paper notes that “stiff dynamics and long horizons may challenge standard MLP NODEs and RK4,” that “performance degrades if test-time dynamics lie far outside the training subspace,” and that there is “no explicit disturbances/noise model” beyond regularization and data averaging (Iqbal et al., 7 Nov 2025).

The Hilbert-space FE formalism makes the dependence on geometry explicit. FE guarantees are with respect to a chosen inner product, and Monte Carlo inner products induce weighting by the sampling distribution $c$7. The paper therefore warns that when $c$8 varies across tasks, “inner-product geometry differs task-to-task.” Likewise, zero-shot RL via FE assumes “task relatedness,” “sufficient basis coverage,” and access to input-output samples of the perturbing function; encoder errors from “small $c$9, non-uniform sampling, or out-of-distribution functions degrade transfer” (Ingebrand et al., 30 Jan 2025, Ingebrand et al., 2024).

A related diagnostic issue appears in encoder-conditioned vision-language-action policies. Under a frozen-backbone grafting protocol, “the small-backbone winner does not reliably select the large-backbone top tier: SigLIP is best on SmolVLA across both suites, while on $c$00 DINOv2-small leads the spatial suite and the object suite is a seed-sensitive near-tie band.” The same study reports that “three of the four backbone-suite comparisons (and 11 of 12 seed-level cells) support backbone-dependent rankings,” and that the grafting wrapper is “non-neutral with opposite sign across backbones (+45-56% MSE on the SmolVLA native tower, -50-52% on $c$01).” Although this result is not itself a function-encoder theorem, it reinforces a broader point in the FE-policy literature: encoder–policy interfaces are often protocol-dependent, and conclusions about transfer are conditional on the wrapper, projection, or compilation mechanism used (Zeng et al., 12 Jun 2026).

Taken together, these limitations indicate that FE policies are not a single algorithmic recipe but a family of function-space policy constructions. Their recurring advantages—closed-form aggregation, zero-shot task conditioning, and modular encoder reuse—depend on whether the relevant task variation can be represented by a stable coefficient vector or a fixed encoder span. Their recurring failure modes arise when that function space is poorly chosen, insufficiently covered by data, or mismatched to the policy architecture that consumes it.