Multiplicative Fusion: Mechanisms & Applications

• Multiplicative fusion is a method that fuses multiple data sources via elementwise or tensor multiplication, enforcing agreement and enhancing selectivity.
• It is implemented in diverse neural architectures—such as EEG decoding, emotion recognition, and image restoration—to improve robustness and performance.
• The technique underpins algebraic and combinatorial models, enabling structured higher-order interactions and efficient gating, while addressing modality-specific artifacts.

Multiplicative fusion refers to a class of operations that combine multiple sources of information or representations—modalities, features, controllers, or algebraic elements—by means of elementwise, outer-product, or probabilistic multiplication, often to enforce interaction or agreement between sources and enhance selectivity or invariance beyond what is possible with additive or concatenative schemes. Multiplicative fusion appears in neural architectures, probabilistic control, combinatorics, algebraic structures, and stable homotopy theory, among other domains. Its principal utility is in selectively gating or modulating jointly salient interactions while suppressing spurious or modality-specific artifacts.

1. Mathematical Foundations and Core Mechanisms

At its core, multiplicative fusion typically implements either Hadamard (elementwise) product or higher-order tensor (outer) products between two or more vectors, matrices, or functions, optionally followed by affine, normalization, and nonlinearity layers. For two aligned feature vectors $\mathbf{x}_1, \mathbf{x}_2 \in \mathbb{R}^d$, the most canonical case is the Hadamard product: $$\mathbf{z} = \mathbf{x}_1 \odot \mathbf{x}_2, \quad z_i = (x_1)_i \cdot (x_2)_i \quad (i=1,\ldots,d).$$ This strict gating ensures that large activations only propagate when both sources agree componentwise.

More generally, in multimodal or tensor-based settings, fusion considers all monomials up to a specified interaction order $D \leq M$. For feature embeddings $f^m \in \mathbb{R}^{n_m}$ for $m=1,\ldots,M$, the $I$-th interaction of modalities is given as the tensor outer product: $$f^I = \bigotimes_{m \in I} f^m, \quad I \subseteq \{1,\ldots,M\}, |I| \le D,$$ which is then projected and aggregated as in interpretable tensor fusion mechanisms (Varshneya et al., 2024). These algebraic operations underlie both feature-level fusions in neural architectures and the structural multiplicativity found in commutative algebras (Guilhot et al., 2022).

2. Architectural Realizations in Deep Learning

Multiplicative fusion is implemented in diverse neural architectures across modalities and domains:

• Dual-Stream Neural Decoding: In "ASPEN: Spectral-Temporal Fusion for Cross-Subject Brain Decoding," temporally and spectrally encoded EEG features, each linearly projected to $\mathbb{R}^d$, are fused by elementwise multiplication after affine transforms:

$$\mathbf{z} = (\mathbf{W}_s \mathbf{x}_s)\odot (\mathbf{W}_t \mathbf{x}_t),$$

enforcing cross-domain agreement and resulting in superior cross-subject generalization (Lee et al., 18 Feb 2026).

• Multimodal Emotion Recognition: "M3ER" uses samplewise multiplicative fusion of predicted class probability vectors across facial, textual, and speech cues, upweighting reliable modalities during training via a multiplicative loss and proxy feature generation for noisy modalities (Mittal et al., 2019).
• Transformer and Convolutional Networks: ELMformer for raw image restoration applies multiplicative gating both in the input fusion (color × spatial branches) and at the self-attention level, by elementwise multiplying channelwise subwindow attention weights into global attention outputs (Ma et al., 2022). MSConv for face recognition fuses parallel convolutions of different receptive fields via multiplication (salient feature path), achieving gradient scaling and selectivity unattainable by additions or concatenations (Zhou et al., 8 Mar 2025).
• Feature-Value Conditioning: In MedFuse, numerical values modulate feature identity embeddings multiplicatively via projector-parameterized gates, enabling value-dependent higher-order feature interactions critical for irregular clinical time series (Hsieh et al., 12 Nov 2025).

3. Theoretical Rationale, Properties, and Comparisons

Multiplicative fusion can be rationalized through several theoretical and empirical lenses:

• Feature-wise Gating: The elementwise product acts as an AND gate at the feature level, propagating only those components where all sources register strong, aligned responses. This enhances robustness to modality-specific noise, suppresses spurious activations, and enforces model invariance (Lee et al., 18 Feb 2026, Zhou et al., 8 Mar 2025).
• Higher-Order Interaction Disentanglement: Tensor-based multiplicative fusion explicitly captures monomial interactions unreachable by additive linear rules, and, under centering/normalization, prevents higher-order fusions from spuriously reencoding lower-order effects (Varshneya et al., 2024).
• Uncertainty and Probabilistic Fusion: In reinforcement learning, multiplicative fusion of Gaussian policies (distribution over actions) retains high-probability regions supported by both prior and learned policies, reducing risk and providing principled fallback behavior under uncertainty (Rana et al., 2020).
• Comparison with Additive/Concatenative Schemes: Additive fusion is susceptible to domination by single-modal artifacts. Concatenation increases output dimensionality and necessitates additional mixing layers. Multiplicative fusion is parameter efficient, supports modularity, and achieves stricter agreement gating without these drawbacks (Lee et al., 18 Feb 2026, Zhou et al., 8 Mar 2025, Ma et al., 2022).

4. Applications and Empirical Outcomes

Multiplicative fusion underpins advances in a broad array of application domains:

Application Area	Fusion Site/Type	Core Empirical Benefit
EEG decoding (ASPEN)	Temporal × spectral streams (Hadamard)	State-of-the-art cross-subject
Emotion recognition (M3ER)	Probability fusion (power-product)	+5% accuracy, robustness
Raw image restoration (ELMformer)	Color × spatial features, local attention	Higher PSNR, ⅓ compute cost
Clinical time series (MedFuse)	Feature × value embedding (Hadamard, block)	Expressive, portable features
Face recognition (MSConv)	Parallel receptive fields (multiplicative)	Boosted TAR @ low FAR
Safe RL/control (MCF)	Policy × prior distribution (product)	RL sample efficiency, safety

Multiplicative fusions have been shown to outperform additive and attention-based baselines in unseen-subject inference (Lee et al., 18 Feb 2026), noisy/missing-modality conditions (Mittal et al., 2019), and computationally constrained vision tasks (Ma et al., 2022, Zhou et al., 8 Mar 2025). Empirically, Hadamard modulation yields superior dependency modeling in irregular, multivariate settings (Hsieh et al., 12 Nov 2025), and tensor fusion enables interpretable, order-disentangled multimodal models (Varshneya et al., 2024).

5. Algebraic, Combinatorial, and Structural Contexts

• Fusion Algebras and Graphs: The concept of a positively multiplicative graph generalizes the notion of adjacency via multiplication in an algebra with nonnegative structure constants. The c_{ij}^k coefficients in $b_i \cdot b_j = \sum_k c_{ij}^k b_k$ correspond precisely to (quantum) fusion rules and, by extension, to path-counting interpretations, alcove walks, and harmonic function classification on graded graphs (Guilhot et al., 2022).
• Fusion Coefficients: In the representation theory of affine Lie algebras, fusion coefficients enumerate combinatorial fusion rules via alternating sums over cylindric tableaux, expressible as sums of positive and negative contributions according to ribbon tabloid structure (with positive rules for special cases) (Morse et al., 2012).
• Multiplicative Structure in Homotopy Theory: The stable splitting of $\Omega SL_n(\mathbb{C})$ (affine Grassmannian) is coherently multiplicative at the $A_\infty$-level but fails to be strictly $E_2$-split due to higher coherence obstructions. Passing to MU-theory (complex bordism) eliminates these, yielding a true $E_2$ fusion algebra structure (Hahn et al., 2017).

6. Flexible and Differentiable Interpolation: Addiplication

A differentiable continuum between addition and multiplication is available via the "addiplication" transfer function, defined for real $\alpha$ as

$$x \oplus_\alpha y = \exp^{(\alpha)}\left( \exp^{(-\alpha)}(x) + \exp^{(-\alpha)}(y) \right),$$

where $\exp^{(\alpha)}$ denotes the functional $\alpha$-th iterate of the exponential (Urban et al., 2015). This construction allows each neuron or layer to smoothly and automatically interpolate between pure additive and multiplicative behavior via trainable parameters, broadening the expressive space for learned fusion operations. While the mathematical apparatus is fully developed, large-scale empirical studies remain a target for future work.

7. Limitations, Interpretability, and Computational Considerations

While multiplicative fusion strategies offer key advantages in selectivity, expressiveness, and invariance, certain trade-offs are noted:

• Computational Cost: Elementwise and low-order products are efficient, but full tensor-product interactions suffer from exponential growth in representation size, necessitating learned low-rank projections or blockwise gating (Varshneya et al., 2024), or Hadamard-restricted schemes (Hsieh et al., 12 Nov 2025).
• Numerical Stability: Multiplicative gating may excessively suppress gradients when any input is small, motivating careful initialization and, in some settings, normalization or attention before or after fusion (Lee et al., 18 Feb 2026, Zhou et al., 8 Mar 2025).
• Interpretability: Interpretable tensor fusion frameworks formalize modality and interaction relevance scores, allowing decomposition and visualization of learned higher-order interactions (Varshneya et al., 2024).
• Task-specificity: Some domains (e.g., RL with explicit probabilistic priors) align with product fusion for theoretical reasons tied to support and uncertainty, whereas others may occasionally benefit from additive or attention-based alternatives (Rana et al., 2020).
• Algebraic Obstructions: In stable homotopy, multiplicative splittings may lack strict higher coherence unless specialized to complex bordism or similar settings (Hahn et al., 2017).

Taken together, multiplicative fusion encompasses a broad family of structurally principled, context-adaptive mechanisms distinguished by their gating, selectivity, and capacity to encode or reveal high-order interactions across diverse theoretical and applied domains.