Element-wise Linear Modulation (EiLM)

• EiLM is a conditioning mechanism that applies per-element affine transformations using context-dependent scale and shift parameters.
• It refines feature adaptation by enabling coordinate-wise modulation, offering enhanced granularity over scalar gating techniques like FiLM.
• EiLM is applied in graph networks, vision tasks, and temporal alignment, achieving efficient parameterization and improved performance.

Searching arXiv for recent EiLM/FiLM-related papers to ground the article. Element-wise Linear Modulation (EiLM) denotes a conditioning mechanism in which a context-dependent vector, or a context-dependent tensor matching the hidden-state shape, applies an element-wise affine transformation to intermediate representations. In its most compact form, the operation is written as $y = \gamma \odot x + \beta$, where $\gamma$ and $\beta$ are conditioning-dependent scale and shift terms and $\odot$ is element-wise multiplication. Across recent work, EiLM appears both explicitly and implicitly: SongEcho formulates it as an extension of FiLM for temporally aligned melody control, the Parametrized Random Projection layer uses learnable element-wise modulation around a fixed random projection, and phase-aware quadruped skateboarding as well as graph message passing adopt FiLM formulations whose mathematics is explicitly element-wise affine (Li et al., 23 Feb 2026, Szorc, 15 Dec 2025, Yoon et al., 10 Feb 2026, Brockschmidt, 2019).

1. Core mathematical form

The defining property of EiLM is that conditioning is vector-valued rather than a single scalar gate. The modulation therefore acts coordinate by coordinate, allowing different hidden dimensions to be emphasized, suppressed, or shifted. SongEcho gives the explicit EiLM formulation as

$$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$

with

$(\gamma_i, \beta_i)=f_i(c),$

where $h_i \in \mathbb{R}^{B \times T \times D_i}$, $c \in \mathbb{R}^{B \times T \times M}$, and $$\gamma_i, \beta_i \in \mathbb{R}^{B \times T \times D_i}$$. In this formulation, the modulation parameters “precisely match the shape of the hidden states,” so the affine transform is not merely per-channel but per element in the sequence (Li et al., 23 Feb 2026).

The same paper also introduces a zero-initialized variant,

$$\text{EiLM-zero}(h_i \mid c_i) = (\gamma_i + 1)\odot h_i + \beta_i, \qquad (\gamma_i,\beta_i)=f_i(c_i),$$

so that at initialization the module behaves like identity. This is used to avoid destabilizing a pretrained backbone.

A related formulation appears in phase-aware quadruped skateboarding, where each layer first computes

$\gamma$0

and then applies a phase-conditioned affine transform,

$\gamma$1

Here the common form $\gamma$2 is retained, but the conditioning signal is a cyclic phase embedding rather than a temporally aligned melody representation. The shared structure across these formulations is that adaptation occurs through learned scale and bias vectors, not through a full conditioned weight matrix (Yoon et al., 10 Feb 2026).

2. Relation to FiLM and the meaning of “element-wise”

EiLM is closely related to Feature-wise Linear Modulation (FiLM), and in several papers the distinction is primarily one of emphasis and tensor shape. GNN-FiLM defines message transformations through target-node-conditioned vectors $\gamma$3 and $\gamma$4, applying them to incoming messages by Hadamard product and bias addition:

$\gamma$5

$\gamma$6

The paper is explicit that these are “element-wise affine transformations,” “feature-wise linear modulation,” and a way to “dynamically up-weight and down-weight features.” It also contrasts this with scalar attention, emphasizing that the target node controls a vector-valued affine modulation rather than a single coefficient shared across all dimensions (Brockschmidt, 2019).

Phase-Aware Policy Learning makes the same equivalence explicit from the opposite direction. Its FiLM layers use

$\gamma$7

and the paper states that this is “directly analogous to what is often called element-wise linear modulation (EiLM): a per-dimension affine transform of activations, rather than a global scalar conditioning.” It further notes that the modulation is “vector-valued, not a single scalar gate” and is applied “element-wise to capture heterogeneous phase effects” (Yoon et al., 10 Feb 2026).

A common misconception is therefore that “feature-wise” implies a coarse channel gate while “element-wise” implies a fundamentally different class of operators. In these papers, feature-wise and element-wise coincide whenever the feature dimension is modulated coordinate by coordinate. SongEcho extends this logic by requiring the modulation parameters to match not only feature dimension but the full time dimension of the hidden states; this suggests that EiLM is best understood as the most explicit tensor-shaped form of FiLM rather than as an unrelated mechanism (Li et al., 23 Feb 2026).

3. Realizations in graph and vision architectures

In graph neural networks, EiLM appears as message modulation conditioned on the receiving node. GNN-FiLM retains the standard message-passing decomposition into message computation, aggregation, and update, but changes the message transformation so that the target node determines how incoming source-node information is interpreted. The paper also gives an empirically better variant in which nonlinearity is applied before aggregation:

$\gamma$8

The stated motivation is that graph tasks often require the relevance of incoming information to depend on the receiving node, and that with a linear $\gamma$9 the message function becomes bilinear in source and target representations (Brockschmidt, 2019).

In vision, Focal Modulation Networks do not explicitly use the name EiLM in the provided description, but the final injection step is an element-wise modulation:

$\beta$0

more explicitly,

$\beta$1

The modulator is built by hierarchical contextualization with depth-wise convolutions, gated aggregation across focal levels, and a projection $\beta$2 that produces the final context-dependent modulator. The paper characterizes this as “element-wise modulation or affine transformation to inject the aggregated context into the query,” and contrasts it with generic concatenation or addition. This suggests that focal modulation can be read as a context-conditioned, multi-scale EiLM mechanism in which query features are rescaled channel by channel by aggregated visual context (Yang et al., 2022).

The graph and vision cases illustrate two recurring structural roles for EiLM-like operations. In graphs, modulation determines how one node receives messages from its neighbors. In vision, modulation injects aggregated context back into the token at the same location. In both settings, the conditioning signal is not simply appended to the input; it is converted into an affine transform that changes the behavior of existing representation channels.

4. Phase-aware and time-aligned conditioning

A central use case for EiLM is multimodal behavior that varies systematically across phases or time steps. In quadruped skateboarding, the task is described as a cycle of behaviors including “pushing,” “carving/steering,” “mounting/transition,” and “foot planting,” with distinct contact patterns, dynamics, reward objectives, critic value distributions, and optimal action structures. To encode this cycle, Phase-Aware Policy Learning defines a phase clock

$\beta$3

maps it to motion modes, and embeds it as

$\beta$4

The sinusoidal embedding is used to condition FiLM layers in both actor and critic MLP backbones, with Table 1 giving the architectures as critic $\beta$5: input size 1702, MLP $\beta$6, and actor $\beta$7: input size 91, MLP $\beta$8. The paper’s interpretation is that one unified policy can “share robot-specific knowledge” while adapting to phase-specific variations (Yoon et al., 10 Feb 2026).

SongEcho addresses a different alignment problem: cover song generation requires melody to be injected with “precise temporal alignment” while preserving lyrics and timbre and generating coherent accompaniment. The model therefore extends FiLM to EiLM so that modulation parameters match the full $\beta$9 shape of the hidden states. Its pipeline extracts vocal pitch $\odot$0 using RMVPE at 100 Hz, encodes pitch with a melody encoder $\odot$1 into melody features $\odot$2, interpolates them to the backbone sequence length, refines them through Instance-Adaptive Condition Refinement (IACR),

$\odot$3

and then applies EiLM,

$\odot$4

The IA-EiLM module is inserted before the FFN layer in each Transformer block, with the explanation that self-attention performs global token interaction and may disturb melody preservation, whereas FFN is more local (Li et al., 23 Feb 2026).

IACR further refines the conditioning features by interacting with the current hidden states:

$\odot$5

$\odot$6

The stated motivation is that a static conditioning vector may be incompatible with a given hidden-state instance. In this setting, EiLM is not merely a generic conditioning block but a mechanism for framewise alignment and hidden-state-compatible control.

5. Efficiency-oriented parameterization

EiLM can also be used as the main trainable component of an efficient linear layer. The Parametrized Random Projection (PRP) layer is defined by

$\odot$7

where $\odot$8 is a fixed random matrix, $\odot$9 is a learnable input modulator, $$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$0 is a learnable output modulator, and $$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$1 is a learnable bias. The paper rewrites the corresponding effective linear operator as

$$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$2

Its central claim is that the design “decouples feature mixing from adaptation,” keeping the expensive mixing matrix fixed while making the layer adaptive through only lightweight element-wise parameters (Szorc, 15 Dec 2025).

The parameterization changes the trainable parameter count from

$$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$3

for a standard fully connected layer to

$$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$4

for PRP. The fixed projection $$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$5 can be Gaussian, sparse ternary, or orthogonal. The paper connects this design to the Johnson–Lindenstrauss lemma as a motivation for using a fixed random matrix as a stable feature-mixing mechanism.

An important limitation is stated explicitly: if $$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$6 is stored densely, the product

$$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$7

still costs roughly the same dense matmul FLOPs as a standard layer, $$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$8. The savings are therefore mainly in the number of learned parameters, gradient computation for those parameters, and optimizer-state storage. A common misconception is that element-wise modulation necessarily yields lower raw multiply-accumulate cost; PRP shows that this is not automatic and depends on the structure or sparsity of the fixed mixing operator (Szorc, 15 Dec 2025).

6. Empirical behavior, comparisons, and limitations

Empirical work on EiLM and EiLM-like mechanisms repeatedly isolates the value of multiplicative, coordinate-wise conditioning relative to simpler or more fragmented alternatives. In phase-aware quadruped skateboarding, replacing FiLM with standard MLPs leads to “narrower command-tracking coverage,” “higher errors,” and “worse robustness.” Removing exteroceptive sensing causes “frequent constraint violations,” “failure to recover from drift or perturbations,” and “collapse in many cases.” A Mixture-of-Experts MLP actor fails to produce effective skateboarding because “the pushing expert collapsed,” “phase-specific experts were too isolated,” “information sharing was poor,” the “parameter count was too large,” and “fewer parallel environments could be trained due to memory limits.” This suggests that element-wise modulation can be preferable to hard expert separation when the task requires both specialization and shared structure (Yoon et al., 10 Feb 2026).

SongEcho provides a direct EiLM ablation against element-wise addition. Replacing EiLM with element-wise addition and removing IACR yields melody metrics RPA $$h^m_i = \text{EiLM}(h_i \mid c) = \gamma_i \odot h_i + \beta_i,$$9, RCA $(\gamma_i, \beta_i)=f_i(c),$0, and OA $(\gamma_i, \beta_i)=f_i(c),$1 when EiLM is restored. Adding IACR further improves RPA $(\gamma_i, \beta_i)=f_i(c),$2, RCA $(\gamma_i, \beta_i)=f_i(c),$3, OA $(\gamma_i, \beta_i)=f_i(c),$4, CLAP $(\gamma_i, \beta_i)=f_i(c),$5, and reduces FD $(\gamma_i, \beta_i)=f_i(c),$6. The method uses only 49.1M trainable parameters, which is 3.07% of ACE-Step + SA ControlNet, 14.8% of ACE-Step + SA ControlNet + LoRA, and 26.0% of ACE-Step + MuseControlLite (Li et al., 23 Feb 2026).

Focal Modulation Networks report that replacing multiplication with addition causes a performance drop of 0.7%, while removing global aggregation causes a drop of 0.3%, top-only aggregation causes a drop of 0.4%, and removing gating causes a drop of 0.4%. These results support the claim that hierarchical context, gating, and element-wise modulation jointly matter, rather than context aggregation alone (Yang et al., 2022).

GNN-FiLM reports strong but task-dependent performance. On PPI it achieves 0.992 micro-F1, with the table highlighting 0.992 ± 0.000, and it converges in about 150 epochs versus 400–700 epochs for other models. On QM9 it is reported as the best or near-best model on most properties, and on VarMisuse it is competitive but not always best. The paper also notes that the variant with nonlinearity before aggregation worked better across all experiments, and that extensive hyperparameter search revealed that some previously published baselines were much weaker than they appeared in earlier work (Brockschmidt, 2019).

PRP shows that EiLM-style parameter reduction can remain competitive or even surpass dense baselines in some settings, but also exposes its trade-offs. On MNIST, Standard FC has 535,818 params, 97.79% accuracy, PRP has 3,108 params, 91.66% accuracy, and Low-rank FC has 6,990 params, 55.78% accuracy. On CIFAR-10, the classifier head changes from 1,182,986 params to 4,372 params, total model size changes from 1,470,890 to 292,276, and accuracy changes from 86.21% to 87.40%. On TinyImageNet, Standard total params are 15,219,720, PRP total params are 5,590,224, and accuracy changes from 45.84% to 47.44%. At the same time, the paper emphasizes “No FLOP reduction in the dense case,” a “Lower absolute accuracy ceiling,” and “Subspace restriction” as limitations (Szorc, 15 Dec 2025).

Taken together, these results characterize EiLM less as a single architecture than as a recurring design pattern: a context signal is converted into element-wise affine coefficients that modulate hidden representations without discarding a shared backbone. The exact benefit varies by domain—message conditioning in graphs, context injection in vision, phase specialization in control, temporal alignment in music, or parameter-efficient linear operators—but the recurring empirical theme is that multiplicative, coordinate-wise conditioning is more expressive than addition and more integrated than hard expert separation.