---
title: Element-Wise Multiplication-Summation (EWMS)
url: https://www.emergentmind.com/topics/element-wise-multiplication-summation-ewms
type: topic
---

# Element-Wise Multiplication-Summation (EWMS)

Element-Wise Multiplication-Summation (EWMS) denotes a computation pattern in which feature tensors, vectors, or structured coefficient arrays interact through element-wise multiplication together with summation, reduction, or residual-like addition. In the literature considered here, the term has both a narrow and a broad sense: in "A Lightweight Group Multiscale Bidirectional Interactive Network for Real-Time Steel Surface Defect Detection" it is a named, parameter-free operator embedded in the Group Multiscale Bidirectional Interactive module of GMBINet, while several related papers do not formalize EWMS as a standalone acronym but use closely related mechanisms built from summation, Hadamard multiplication, residual summation, broadcasting, product-to-sum conversion, or diagonal extraction [2508.16397] [2406.07488] [2406.04170] [2409.17502] [1412.1145] [2506.23906] [2501.05223] [2606.06456].

## 1. Terminology and conceptual scope

The terminology is not uniform across papers. GMBINet explicitly introduces a parameter-free Element-Wise Multiplication-Summation operation as the interaction rule inside the Bidirectional Progressive Feature Interactor, with the stated purpose of providing cross-scale communication between multiscale feature groups without the overhead of extra convolutions, attention blocks, or channel-alignment layers [2508.16397]. ReduceFormer, by contrast, states that the paper does not formally name a module “EWMS” as a standalone acronym in the main text, but repeatedly uses global summation and element-wise multiplication to replace standard self-attention [2406.07488]. The Deeper-PINN work is described as squarely about an EWMS-like idea, but again not as a named standalone primitive; its architecture is based on repeated element-wise multiplicative feature fusion followed by residual addition [2406.04170].

The same pattern extends beyond neural feature interaction. The broadcast product paper defines a new tensor operator \(\boxdot\) that performs shape-aligned element-wise multiplication after duplication/broadcasting, and then develops marginalization and least-squares identities around it [2409.17502]. The matrix-multiplication survey does not present EWMS as a named primitive, but gives an explicit route from packed multiplication to inner product and summation via binary segmentation [1412.1145]. The MMV-RAM paper treats element-wise multiplication, segmented scan, and segmented sum as separate but composable primitives accelerated by matrix multiplication hardware [2506.23906]. EVA-S2PLoR uses secure 2-party vector element-wise multiplication as a core PPML building block [2501.05223], and the quantum element-wise transforms paper formalizes entrywise multiplication and entrywise polynomial application on block-encoded matrices [2606.06456]. This suggests that EWMS is better understood as a family of multiplication-plus-summation constructions than as a universally standardized operator.

## 2. Formal EWMS operator in GMBINet

In GMBINet, the input tensor \(F_{in} \in \mathbb{R}^{c \times w \times h}\) is first split evenly into \(n\) channel groups,
\[
x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.
\]
Each group is processed with depthwise convolution using different dilation rates \(1,2,\ldots,n\) to obtain multiscale representations [2508.16397].

EWMS appears inside the Bidirectional Progressive Feature Interactor, where it implements the interaction function \(f_{inter}\). During forward guidance, the current group \(x_i\) is modulated by the previous scale’s output \(y_{i-1}\) through a sigmoid gate,
\[
x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.
\]
Here \(\sigma(\cdot)\) is the sigmoid function, \(\times\) denotes element-wise multiplication, and the final \(+x_i\) is the element-wise summation term. The same type of interaction is also used in the backward enhancement path, where higher-level features refine lower-level ones. EWMS is therefore embedded directly into the progressive bidirectional interaction process rather than used as a standalone fusion stage after all scales are computed [2508.16397].

Architecturally, EWMS sits inside the BPFI, which is one of the central components of the Group Multiscale Bidirectional Interactive module. The full pipeline is
\[
\text{group multiscale feature extraction} \rightarrow \text{bidirectional progressive interaction via BPFI and EWMS} \rightarrow \text{multiscale feature fusion by concatenation and a pointwise convolution},
\]
with
\[
F_{out} = F_{in} + f_{pw}\big(cat(y_1^{en}, y_2^{en}, \ldots, y_n^{en})\big),
\]
where \(f_{pw}\) is a \(1\times 1\) pointwise convolution and \(cat\) is concatenation. In this formulation, EWMS acts before the final cross-scale concatenation/fusion stage and serves as the local interaction rule that enhances each scale pair progressively [2508.16397].

The operator is explicitly described as parameter-free. It introduces no learnable weights and no convolutional layers; its computational burden consists only of sigmoid, element-wise multiplication, and element-wise addition. Because the operation is performed on tensors of identical shape, \(\sigma(y_{i-1})\) and \(x_i\) are required to be compatible channel-wise and spatially, so the gating mask can be applied element by element. The paper states that the guidance feature and the target feature are aligned in the BPFI pathway so that multiplication and addition occur without shape transformation, which makes the operator especially suitable for real-time deployment [2508.16397].

## 3. Interaction logic, design trade-offs, and reported behavior

The design rationale in GMBINet is stated in comparative terms. Pure element-wise addition is cheap, but it can cause semantic dilution and weak noise suppression because it simply mixes signals without emphasizing important regions. Pure element-wise multiplication can suppress irrelevant activations and highlight target areas, but it may over-filter and discard fine detail. Concatenation preserves more information, but usually requires extra convolutions to align channels and perform fusion, which increases parameters and FLOPs. EWMS merges these behaviors in a compact form: the sigmoid-gated multiplication acts as soft attention on \(x_i\), and the subsequent summation restores the original feature content to reduce information loss [2508.16397].

The paper also contrasts EWMS with attention mechanisms. Compared with attention mechanisms, EWMS does not learn extra projection matrices, query-key-value mappings, or channel/spatial attention parameters; the modulation comes directly from the feature tensor itself. The authors further state that EWMS helps the network “emphasize salient regions via element-wise multiplication and integrate complementary context through summation,” a property presented as useful for defect detection where fine boundaries and low-contrast regions matter [2508.16397].

The ablation study compares EWMS against “w/ Sum”, “w/ Multiply”, and “w/ Concat”, with the same overall Group Multiscale Bidirectional Interactive structure but different interaction operators. The reported results are that “w/ Sum” performs better than “w/ Multiply” in some metrics, but both are weaker than EWMS; “w/ Concat” improves performance more than simple sum or multiply, but increases parameters from \(0.19\) M to \(0.53\) M and FLOPs from \(0.39\) G to \(0.89\) G, while reducing speed to \(932\) FPS. EWMS achieves MAE \(0.0126\), WF \(0.9273\), OR \(0.8653\), SM \(0.9334\), PFOM \(0.9077\), and IoU \(0.8728\), with only \(0.19\) M parameters, \(0.39\) G FLOPs, and \(1048\) FPS [2508.16397].

At the model level, GMBINet is reported to deliver real-time speeds of \(1048\) FPS on GPU and \(16.53\) FPS on CPU at \(512\) resolution, using only \(0.19\) M parameters, with experiments conducted on SD-Saliency-900 and NRSD-MN. Additional evaluations on NEU-CLS are described as confirming strong generalization ability beyond surface defect detection [2508.16397].

## 4. Related neural formulations: attention surrogates, multiplicative feature fusion, and broadcasted Hadamard structure

ReduceFormer replaces standard self-attention with a simpler attention-like computation built from global summation, element-wise multiplication, and ReLU. Its core pattern is a tensor-reduction attention surrogate: reduce feature maps globally with summation, multiply reduced summaries element-wise with feature tensors, reduce again, and then use the resulting global descriptor to modulate the query features. Inside the ReduceFormer block, \(ReLU(K)\) is first reduced over spatial dimensions into \(\mathbf{SUM}^K\), this is broadcast and multiplied with \(V\), another global summation yields \(\mathbf{SUM}^V\), and \(ReLU(K)\) is then element-wise multiplied with \(\mathbf{SUM}^V\) before another reduction produces \(\mathbf{SUM}^{KV}\). The paper emphasizes that this design avoids explicit \(QK^\top\), Softmax, and large attention maps, and reports up to \(37\%\) lower latency and up to \(44\%\) higher throughput. On NVIDIA DRIVE Orin, ReduceFormer-B1 at \(r224\) achieves \(0.68\) ms latency versus \(0.90\) ms for EfficientViT-B1, and on L40 it reaches \(4149\) images/s versus \(3067\) images/s [2406.07488].

Element-wise multiplication is also central in Deeper-PINNs. The initial feature map is formed by
\[
H^{(1)} = \big(W_1 x + b_1\big)\odot \big(W_2 x + b_2\big),
\]
and each shortcut block contains two further multiplicative layers,
\[
h^{(l)} = \sigma\!\left(W_1^{(l)} H^{(l)} + b_1^{(l)}\right)\odot \sigma\!\left(W_2^{(l)} H^{(l)} + b_2^{(l)}\right),
\]
\[
\tilde{h}^{(l)} = \sigma\!\left(W_1^{(l)} h^{(l)} + b_1^{(l)}\right)\odot \sigma\!\left(W_2^{(l)} h^{(l)} + b_2^{(l)}\right),
\]
followed by residual accumulation,
\[
H^{(l+1)} = \tilde{h}^{(l)} + H^{(l)}.
\]
The paper argues that multiplication maps features into high-dimensional, non-linear spaces and helps avoid initialization pathology in PINNs. Reported best relative \(L^2\) errors are \(1.68\times 10^{-5}\) for the Allen–Cahn equation, \(2.38\times 10^{-6}\) for the 2D Helmholtz equation, and average relative \(L^2\) error \(3.37\times 10^{-3}\) for the 1D advection equation [2406.04170].

The broadcast product paper formalizes a shape-aligned generalization of Hadamard multiplication,
\[
{Z} = {X} \boxdot {Y} := \mathrm{bc}({X}, \mathrm{size}({Y})) \odot \mathrm{bc}({Y}, \mathrm{size}({X})) = {X}^\square \odot {Y}^\square.
\]
If the two tensors have the same shape, then \({X} \boxdot {Y} = {X}\odot {Y}\); otherwise, singleton modes are duplicated to align shapes. The paper uses this operator to express masking and FiLM, and introduces marginalization identities such as
\[
\Vert {X} \boxdot {Y} \Vert_F^2 = \Vert {X}^\square \odot {Y}^\square \Vert_F^2 = \Vert {X}_\square \odot {Y}_\square \Vert_F^2.
\]
In this line of work, EWMS-like behavior is not restricted to same-shape tensors but includes explicit duplication and contraction structure [2409.17502].

## 5. Algebraic, finite-element, segmented, and quantum formulations

An algebraic interpretation appears in the matrix-multiplication survey through inner product and summation by binary segmentation. For vectors \({\bf u}=(u_i)_{i=0}^{n-1}\) and \({\bf v}=(v_i)_{i=0}^{n-1}\), the paper defines
\[
u(x)=\sum_{i=0}^{n-1} u_i x^i,\qquad v(x)=\sum_{i=0}^{n-1} v_i x^{n-1-i},
\]
so that in the product \(q(x)=u(x)v(x)\), the middle coefficient satisfies
\[
q_{n-1}=\sum_{i=0}^{n-1} u_i v_i = {\bf u}^T {\bf v}.
\]
Evaluating at \(x=2^k\) and applying binary segmentation yields the inner product from one large-integer multiplication; summation is recovered as a special case by setting one vector to all ones. The paper does not present EWMS as a standalone formal primitive, but it provides a direct route from coordinatewise products to a recovered sum [1412.1145].

In hierarchical FEM, the product-to-sum method replaces products of basis and weighting polynomials with short sums of other polynomials. The general identity is
\[
P_{m_1}^{(Y_iY_j)}(u)\,P_{m_2}^{(Y_iY_j)}(u) = \sum_{k_1=0}^{K_1} a_{k_1}\,P_{k_1}^{(Y_iY_j)}(u),
\]
which converts a multidimensional matrix element into
\[
M^{Y_iY_j}_{m_1m_2n_1n_2p_1p_2} = \sum_{k_1,k_2,k_3} a_{k_1}a_{k_2}a_{k_3} \int P_{k_1}^{(Y_iY_j)}(u)\, Q_{k_2}^{(Y_iY_j)}(v)\, R_{k_3}^{(Y_iY_j)}(w)\, \alpha^{s}_{Y_iY_j}(u,v,w)\,du\,dv\,dw.
\]
For the 2D example with Chebyshev polynomials, identities such as
\[
U_{m_1}(u)\,U_{m_2}(u) = T_{|m_1-m_2|}(u)-T_{m_1+m_2+2}(u)
\]
lead to a four-term decomposition of \(M^{uu}_{m_1m_2n_1n_2}\). The reported integral-count reduction factor is roughly \(D/16\), and the asymptotic operation-count reduction is about \(D/15\) [1703.09619].

At the systems level, MMV-RAM studies segmented scan and segmented sum with a matrix-multiplication unit. The central decomposition is
\[
\text{Segmented Sum} \cong \text{DIFF} \circ \text{COMPRESS} \circ \text{SCAN},
\]
with
\[
\text{DIFF}(x)_i = x_i - x_{i-1},\quad x_{-1}=0.
\]
Segmented scan is defined by
\[
y(i)= \begin{cases}
x(i), & i=0 \text{ or } f(i)=1,\\
y(i-1)+x(i), & \text{otherwise},
\end{cases}
\]
and the main theorem gives an \(O(\log_s n)\)-step MMV-RAM algorithm, while any polynomial-work VCU-only algorithm requires \(\Omega\!\left(\frac{\log n}{\log\log n}\right)\) steps. The same paper gives element-wise integer vector multiplication in \(O(\log_s B)\) steps with work \(O\!\left(M(nB/s)+nsB^2\right)\) [2506.23906].

The quantum element-wise transforms paper formalizes the entrywise product
\[
C=A\circ B,\qquad C_{ij}=A_{ij}B_{ij},
\]
and entrywise polynomial application
\[
f^\circ(M)=\big[f(M_{ij})\big]_{ij},
\]
with \(f(x)=\sum_{k=1}^d c_k x^k\) and \(c_0=0\). Its central identity is that \(A\circ B\) can be extracted from \(A\otimes B\) by permutation and postselection,
\[
(I_n\otimes \langle 0^n|)P(A\otimes B)P^\dagger (I_n\otimes |0^n\rangle)=A\circ B.
\]
The main theorem states that, given a controlled block encoding of \(A\), one can prepare a block encoding of \(f^\circ(A/\alpha)\) requiring \(\mathcal{O}(d)\) controlled-\(U\) copies, with ancilla cost \(b=\mathcal{O}(a+n\log d)\). The paper highlights a reduction in auxiliary space from prior \(\tilde{\mathcal{O}}(ad+nd)\) to \(\mathcal{O}(a+n\log d)\) [2606.06456].

## 6. Secure computation, optimization variants, and interpretive limits

In privacy-preserving machine learning, EVA-S2PLoR builds directly on secure 2-party vector element-wise multiplication, S2PVEM, defined by
\[
\boldsymbol{v_a} + \boldsymbol{v_b} = \boldsymbol{a} \odot \boldsymbol{b}.
\]
The protocol disguises the input vectors as matrices \(A\) and \(B\), invokes secure 2-party matrix multiplication, and extracts the diagonal:
\[
\boldsymbol{v_a} + \boldsymbol{v_b} = diag2v(V_a + V_b) = diag2v(A \times B) = (\boldsymbol{a_1}\cdot \boldsymbol{b_1}, \dots, \boldsymbol{a_n}\cdot \boldsymbol{b_n})^T = \boldsymbol{a}\odot \boldsymbol{b}.
\]
The derived protocols include secure addition-to-multiplication conversion, reciprocal, and sigmoid. Reported communication rounds are \(6\) for S2PVEM, \(7\) for S2PVATM, \(13\) for S2PVR, \(19\) for S2PVS, \(43t\) for S2PLoRT, and \(31\) for S2PLoRP. The abstract states that sigmoid performance improves by about \(10\) orders of magnitude compared to most frameworks, and verification failure probabilities are bounded by powers of \(1/4^l\) [2501.05223].

A different approximation appears in online learning. The modified online Newton step paper replaces a full \(d\times d\) Hessian with first- and second-order quantities of size \(m\times d\), using element-wise multiplication:
\[
d_t = A^{-1} \bigodot b,\qquad w_{t+1} = w_t - \eta_t d_t,
\]
with \(\eta_t=\frac{1}{2\sqrt{t}}\). The paper claims memory \(\mathcal{O}(md)\), per-iteration time \(\mathcal{O}(md)\), and regret
\[
R_G(T) \le ( \|F\|^2 + \|k\|^2 ) \sqrt{T},
\qquad
\limsup_{T\to\infty} \frac{R_G(T)}{T} \le 0.
\]
Its stated purpose is to retain matrices of the same size as the model while reducing computation on large multiclass datasets [1904.05633].

A common misconception is to treat EWMS as a single fixed formula. The cited literature does not support that reading. In GMBINet, EWMS is a parameter-free local interaction rule based on sigmoid-gated multiplication plus element-wise summation [2508.16397]. In ReduceFormer, the closest equivalent is repeated global summations and element-wise multiplications used to circumvent the inner product calculation of \(K\) and \(V\) [2406.07488]. In hierarchical FEM, the closest analogue is product-to-sum conversion that reduces multidimensional integration to a smaller library of reusable integrals [1703.09619]. In the quantum setting, the central objects are entrywise products and entrywise polynomial transforms assembled by LCU [2606.06456]. This suggests that EWMS is a useful cross-domain descriptor for low-cost multiplicative interaction plus accumulation, but its exact mathematical object remains domain-specific and must be read from the local definitions of each paper.

Source: https://www.emergentmind.com/topics/element-wise-multiplication-summation-ewms