---
title: Element-wise Linear Modulation (EiLM)
url: https://www.emergentmind.com/topics/element-wise-linear-modulation-eilm
type: topic
---

# Element-wise Linear Modulation (EiLM)

Searching arXiv for recent EiLM/FiLM-related papers to ground the article.
Element-wise Linear Modulation (EiLM) denotes a conditioning mechanism in which a context-dependent vector, or a context-dependent tensor matching the hidden-state shape, applies an element-wise affine transformation to intermediate representations. In its most compact form, the operation is written as \(y = \gamma \odot x + \beta\), where \(\gamma\) and \(\beta\) are conditioning-dependent scale and shift terms and \(\odot\) is element-wise multiplication. Across recent work, EiLM appears both explicitly and implicitly: SongEcho formulates it as an extension of FiLM for temporally aligned melody control, the Parametrized Random Projection layer uses learnable element-wise modulation around a fixed random projection, and phase-aware quadruped skateboarding as well as graph message passing adopt FiLM formulations whose mathematics is explicitly element-wise affine [2602.19976; 2512.13480; 2602.09370; 1906.12192].

## 1. Core mathematical form

The defining property of EiLM is that conditioning is vector-valued rather than a single scalar gate. The modulation therefore acts coordinate by coordinate, allowing different hidden dimensions to be emphasized, suppressed, or shifted. SongEcho gives the explicit EiLM formulation as
$$
h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,
$$
with
$$
(\gamma_i, \beta_i)=f_i(c),
$$
where \(h_i \in \mathbb{R}^{B \times T \times D_i}\), \(c \in \mathbb{R}^{B \times T \times M}\), and \(\gamma_i, \beta_i \in \mathbb{R}^{B \times T \times D_i}\). In this formulation, the modulation parameters “precisely match the shape of the hidden states,” so the affine transform is not merely per-channel but per element in the sequence [2602.19976].

The same paper also introduces a zero-initialized variant,
$$
\text{EiLM-zero}(h_i \mid c_i) = (\gamma_i + 1)\odot h_i + \beta_i, \qquad (\gamma_i,\beta_i)=f_i(c_i),
$$
so that at initialization the module behaves like identity. This is used to avoid destabilizing a pretrained backbone.

A related formulation appears in phase-aware quadruped skateboarding, where each layer first computes
$$
z = \mathbf{W}_\ell \mathbf{h}_{\ell-1} + \mathbf{b}_\ell,
$$
and then applies a phase-conditioned affine transform,
$$
\mathbf{h}_\ell = \sigma\!\left(\gamma_\ell(\Phi_t) \odot z + \beta_\ell(\Phi_t)\right).
$$
Here the common form \(y = \gamma \odot x + \beta\) is retained, but the conditioning signal is a cyclic phase embedding rather than a temporally aligned melody representation. The shared structure across these formulations is that adaptation occurs through learned scale and bias vectors, not through a full conditioned weight matrix [2602.09370].

## 2. Relation to FiLM and the meaning of “element-wise”

EiLM is closely related to Feature-wise Linear Modulation (FiLM), and in several papers the distinction is primarily one of emphasis and tensor shape. GNN-FiLM defines message transformations through target-node-conditioned vectors \(\boldsymbol\gamma\) and \(\boldsymbol\beta\), applying them to incoming messages by Hadamard product and bias addition:
$$
\boldsymbol\beta^{(t)}_{\ell, v}, \boldsymbol\gamma^{(t)}_{\ell, v} = g(\state{v}{t} \; ; \; \parameters_{g, \ell})
$$
$$
\state{v}{t+1} = \sigma\left( \sum_{\typededge{u}{\ell}{v} \in \edges} \boldsymbol\gamma^{(t)}_{\ell, v} \odot \weights_{\!\ell} \state{u}{t} + \boldsymbol\beta^{(t)}_{\ell, v} \right).
$$
The paper is explicit that these are “element-wise affine transformations,” “feature-wise linear modulation,” and a way to “dynamically up-weight and down-weight features.” It also contrasts this with scalar attention, emphasizing that the target node controls a vector-valued affine modulation rather than a single coefficient shared across all dimensions [1906.12192].

Phase-Aware Policy Learning makes the same equivalence explicit from the opposite direction. Its FiLM layers use
$$
\gamma_\ell(\Phi_t) \odot z + \beta_\ell(\Phi_t),
$$
and the paper states that this is “directly analogous to what is often called element-wise linear modulation (EiLM): a per-dimension affine transform of activations, rather than a global scalar conditioning.” It further notes that the modulation is “vector-valued, not a single scalar gate” and is applied “element-wise to capture heterogeneous phase effects” [2602.09370].

A common misconception is therefore that “feature-wise” implies a coarse channel gate while “element-wise” implies a fundamentally different class of operators. In these papers, feature-wise and element-wise coincide whenever the feature dimension is modulated coordinate by coordinate. SongEcho extends this logic by requiring the modulation parameters to match not only feature dimension but the full time dimension of the hidden states; this suggests that EiLM is best understood as the most explicit tensor-shaped form of FiLM rather than as an unrelated mechanism [2602.19976].

## 3. Realizations in graph and vision architectures

In graph neural networks, EiLM appears as message modulation conditioned on the receiving node. GNN-FiLM retains the standard message-passing decomposition into message computation, aggregation, and update, but changes the message transformation so that the target node determines how incoming source-node information is interpreted. The paper also gives an empirically better variant in which nonlinearity is applied before aggregation:
$$
\state{v}{t+1} = l\left( \sum_{\typededge{u}{\ell}{v} \in \edges} \sigma\left(\boldsymbol\gamma^{(t)}_{\ell, v} \odot \weights_{\!\ell} \state{u}{t} + \boldsymbol\beta^{(t)}_{\ell, v} \right) \; ; \; \parameters_{l} \right).
$$
The stated motivation is that graph tasks often require the relevance of incoming information to depend on the receiving node, and that with a linear \(g\) the message function becomes bilinear in source and target representations [1906.12192].

In vision, Focal Modulation Networks do not explicitly use the name EiLM in the provided description, but the final injection step is an element-wise modulation:
$$
y_i = q(x_i) \odot m(i,\cdot),
$$
more explicitly,
$$
y_i = q(x_i) \odot h\!\left(\sum_{\ell=1}^{L+1} g_i^\ell \cdot z_i^\ell\right).
$$
The modulator is built by hierarchical contextualization with depth-wise convolutions, gated aggregation across focal levels, and a projection \(h\) that produces the final context-dependent modulator. The paper characterizes this as “element-wise modulation or affine transformation to inject the aggregated context into the query,” and contrasts it with generic concatenation or addition. This suggests that focal modulation can be read as a context-conditioned, multi-scale EiLM mechanism in which query features are rescaled channel by channel by aggregated visual context [2203.11926].

The graph and vision cases illustrate two recurring structural roles for EiLM-like operations. In graphs, modulation determines how one node receives messages from its neighbors. In vision, modulation injects aggregated context back into the token at the same location. In both settings, the conditioning signal is not simply appended to the input; it is converted into an affine transform that changes the behavior of existing representation channels.

## 4. Phase-aware and time-aligned conditioning

A central use case for EiLM is multimodal behavior that varies systematically across phases or time steps. In quadruped skateboarding, the task is described as a cycle of behaviors including “pushing,” “carving/steering,” “mounting/transition,” and “foot planting,” with distinct contact patterns, dynamics, reward objectives, critic value distributions, and optimal action structures. To encode this cycle, Phase-Aware Policy Learning defines a phase clock
$$
\phi_t \in [0,2\pi), \qquad \phi_t = 2\pi t/T_\phi \bmod 2\pi,
$$
maps it to motion modes, and embeds it as
$$
\Phi_t = (\cos \phi_t,\ \sin \phi_t)^\top \in \mathbb{R}^2.
$$
The sinusoidal embedding is used to condition FiLM layers in both actor and critic MLP backbones, with Table 1 giving the architectures as critic \(f_\theta^{critic}\): input size 1702, MLP \(\mathcal{F}(1024,512,256,1)\), and actor \(f_\theta^{actor}\): input size 91, MLP \(\mathcal{F}(512,256,128,12)\). The paper’s interpretation is that one unified policy can “share robot-specific knowledge” while adapting to phase-specific variations [2602.09370].

SongEcho addresses a different alignment problem: cover song generation requires melody to be injected with “precise temporal alignment” while preserving lyrics and timbre and generating coherent accompaniment. The model therefore extends FiLM to EiLM so that modulation parameters match the full \(B \times T \times D_i\) shape of the hidden states. Its pipeline extracts vocal pitch \(p\) using RMVPE at 100 Hz, encodes pitch with a melody encoder \(\mathcal{E}\) into melody features \(m^0\), interpolates them to the backbone sequence length, refines them through Instance-Adaptive Condition Refinement (IACR),
$$
c_i = \text{IACR}(m, h_i),
$$
and then applies EiLM,
$$
h_i^m = \text{EiLM}(h_i \mid c_i).
$$
The IA-EiLM module is inserted before the FFN layer in each Transformer block, with the explanation that self-attention performs global token interaction and may disturb melody preservation, whereas FFN is more local [2602.19976].

IACR further refines the conditioning features by interacting with the current hidden states:
$$
h'_i = L_{h_i}(h_i), \quad m'_i = L_{m_i}(m)
$$
$$
c_i = \tanh(h'_i)\odot \tanh(m'_i).
$$
The stated motivation is that a static conditioning vector may be incompatible with a given hidden-state instance. In this setting, EiLM is not merely a generic conditioning block but a mechanism for framewise alignment and hidden-state-compatible control.

## 5. Efficiency-oriented parameterization

EiLM can also be used as the main trainable component of an efficient linear layer. The Parametrized Random Projection (PRP) layer is defined by
$$
\mathbf{y} = \big( P^\top (\mathbf{x} \odot \boldsymbol{\alpha}) \big) \odot \mathbf{w} + \mathbf{b},
$$
where \(P\) is a fixed random matrix, \(\boldsymbol{\alpha}\) is a learnable input modulator, \(\mathbf{w}\) is a learnable output modulator, and \(\mathbf{b}\) is a learnable bias. The paper rewrites the corresponding effective linear operator as
$$
W_{\text{PRP} = \operatorname{diag}(\mathbf{w})\, P^\top\, \operatorname{diag}(\boldsymbol{\alpha}).
$$
Its central claim is that the design “decouples feature mixing from adaptation,” keeping the expensive mixing matrix fixed while making the layer adaptive through only lightweight element-wise parameters [2512.13480].

The parameterization changes the trainable parameter count from
$$
d_{\text{in} \cdot d_{\text{out} + d_{\text{out}
$$
for a standard fully connected layer to
$$
d_{\text{in} + 2d_{\text{out}
$$
for PRP. The fixed projection \(P\) can be Gaussian, sparse ternary, or orthogonal. The paper connects this design to the Johnson–Lindenstrauss lemma as a motivation for using a fixed random matrix as a stable feature-mixing mechanism.

An important limitation is stated explicitly: if \(P\) is stored densely, the product
$$
P^\top (\mathbf{x}\odot \boldsymbol{\alpha})
$$
still costs roughly the same dense matmul FLOPs as a standard layer, \(\mathcal{O}(d_{\text{in}d_{\text{out})\). The savings are therefore mainly in the number of learned parameters, gradient computation for those parameters, and optimizer-state storage. A common misconception is that element-wise modulation necessarily yields lower raw multiply-accumulate cost; PRP shows that this is not automatic and depends on the structure or sparsity of the fixed mixing operator [2512.13480].

## 6. Empirical behavior, comparisons, and limitations

Empirical work on EiLM and EiLM-like mechanisms repeatedly isolates the value of multiplicative, coordinate-wise conditioning relative to simpler or more fragmented alternatives. In phase-aware quadruped skateboarding, replacing FiLM with standard MLPs leads to “narrower command-tracking coverage,” “higher errors,” and “worse robustness.” Removing exteroceptive sensing causes “frequent constraint violations,” “failure to recover from drift or perturbations,” and “collapse in many cases.” A Mixture-of-Experts MLP actor fails to produce effective skateboarding because “the pushing expert collapsed,” “phase-specific experts were too isolated,” “information sharing was poor,” the “parameter count was too large,” and “fewer parallel environments could be trained due to memory limits.” This suggests that element-wise modulation can be preferable to hard expert separation when the task requires both specialization and shared structure [2602.09370].

SongEcho provides a direct EiLM ablation against element-wise addition. Replacing EiLM with element-wise addition and removing IACR yields melody metrics RPA \(0.6336 \rightarrow 0.6799\), RCA \(0.6476 \rightarrow 0.7000\), and OA \(0.6683 \rightarrow 0.6793\) when EiLM is restored. Adding IACR further improves RPA \(0.6799 \rightarrow 0.7080\), RCA \(0.7000 \rightarrow 0.7339\), OA \(0.6793 \rightarrow 0.6952\), CLAP \(0.2999 \rightarrow 0.3243\), and reduces FD \(75.28 \rightarrow 42.06\). The method uses only **49.1M trainable parameters**, which is **3.07%** of ACE-Step + SA ControlNet, **14.8%** of ACE-Step + SA ControlNet + LoRA, and **26.0%** of ACE-Step + MuseControlLite [2602.19976].

Focal Modulation Networks report that replacing multiplication with addition causes a performance drop of **0.7%**, while removing global aggregation causes a drop of **0.3%**, top-only aggregation causes a drop of **0.4%**, and removing gating causes a drop of **0.4%**. These results support the claim that hierarchical context, gating, and element-wise modulation jointly matter, rather than context aggregation alone [2203.11926].

GNN-FiLM reports strong but task-dependent performance. On PPI it achieves **0.992 micro-F1**, with the table highlighting **0.992 ± 0.000**, and it converges in about **150 epochs** versus **400–700 epochs** for other models. On QM9 it is reported as the best or near-best model on most properties, and on VarMisuse it is competitive but not always best. The paper also notes that the variant with nonlinearity before aggregation worked better across all experiments, and that extensive hyperparameter search revealed that some previously published baselines were much weaker than they appeared in earlier work [1906.12192].

PRP shows that EiLM-style parameter reduction can remain competitive or even surpass dense baselines in some settings, but also exposes its trade-offs. On MNIST, Standard FC has **535,818 params, 97.79% accuracy**, PRP has **3,108 params, 91.66% accuracy**, and Low-rank FC has **6,990 params, 55.78% accuracy**. On CIFAR-10, the classifier head changes from **1,182,986 params** to **4,372 params**, total model size changes from **1,470,890** to **292,276**, and accuracy changes from **86.21%** to **87.40%**. On TinyImageNet, Standard total params are **15,219,720**, PRP total params are **5,590,224**, and accuracy changes from **45.84%** to **47.44%**. At the same time, the paper emphasizes “No FLOP reduction in the dense case,” a “Lower absolute accuracy ceiling,” and “Subspace restriction” as limitations [2512.13480].

Taken together, these results characterize EiLM less as a single architecture than as a recurring design pattern: a context signal is converted into element-wise affine coefficients that modulate hidden representations without discarding a shared backbone. The exact benefit varies by domain—message conditioning in graphs, context injection in vision, phase specialization in control, temporal alignment in music, or parameter-efficient linear operators—but the recurring empirical theme is that multiplicative, coordinate-wise conditioning is more expressive than addition and more integrated than hard expert separation.

Source: https://www.emergentmind.com/topics/element-wise-linear-modulation-eilm