Element-Wise Multiplication-Summation (EWMS)

• Element-Wise Multiplication-Summation (EWMS) is a computation pattern that performs element-wise multiplication followed by summation to facilitate cross-scale feature interactions in deep networks.
• It is implemented as a parameter-free operator in architectures like GMBINet, using sigmoid gating to modulate features without extra convolutions or attention blocks.
• EWMS achieves high efficiency and real-time performance (e.g., 1048 FPS in GMBINet) while preserving critical feature details by blending multiplicative and additive operations.

Element-Wise Multiplication-Summation (EWMS) denotes a computation pattern in which feature tensors, vectors, or structured coefficient arrays interact through element-wise multiplication together with summation, reduction, or residual-like addition. In the literature considered here, the term has both a narrow and a broad sense: in "A Lightweight Group Multiscale Bidirectional Interactive Network for Real-Time Steel Surface Defect Detection" it is a named, parameter-free operator embedded in the Group Multiscale Bidirectional Interactive module of GMBINet, while several related papers do not formalize EWMS as a standalone acronym but use closely related mechanisms built from summation, Hadamard multiplication, residual summation, broadcasting, product-to-sum conversion, or diagonal extraction (Zhang et al., 22 Aug 2025, Yang et al., 2024, Jiang et al., 2024, Matsui et al., 2024, Pan, 2014, Sobczyk et al., 30 Jun 2025, Tao et al., 9 Jan 2025, Rossi et al., 4 Jun 2026).

1. Terminology and conceptual scope

The terminology is not uniform across papers. GMBINet explicitly introduces a parameter-free Element-Wise Multiplication-Summation operation as the interaction rule inside the Bidirectional Progressive Feature Interactor, with the stated purpose of providing cross-scale communication between multiscale feature groups without the overhead of extra convolutions, attention blocks, or channel-alignment layers (Zhang et al., 22 Aug 2025). ReduceFormer, by contrast, states that the paper does not formally name a module “EWMS” as a standalone acronym in the main text, but repeatedly uses global summation and element-wise multiplication to replace standard self-attention (Yang et al., 2024). The Deeper-PINN work is described as squarely about an EWMS-like idea, but again not as a named standalone primitive; its architecture is based on repeated element-wise multiplicative feature fusion followed by residual addition (Jiang et al., 2024).

The same pattern extends beyond neural feature interaction. The broadcast product paper defines a new tensor operator $\boxdot$ that performs shape-aligned element-wise multiplication after duplication/broadcasting, and then develops marginalization and least-squares identities around it (Matsui et al., 2024). The matrix-multiplication survey does not present EWMS as a named primitive, but gives an explicit route from packed multiplication to inner product and summation via binary segmentation (Pan, 2014). The MMV-RAM paper treats element-wise multiplication, segmented scan, and segmented sum as separate but composable primitives accelerated by matrix multiplication hardware (Sobczyk et al., 30 Jun 2025). EVA-S2PLoR uses secure 2-party vector element-wise multiplication as a core PPML building block (Tao et al., 9 Jan 2025), and the quantum element-wise transforms paper formalizes entrywise multiplication and entrywise polynomial application on block-encoded matrices (Rossi et al., 4 Jun 2026). This suggests that EWMS is better understood as a family of multiplication-plus-summation constructions than as a universally standardized operator.

2. Formal EWMS operator in GMBINet

In GMBINet, the input tensor $F_{in} \in \mathbb{R}^{c \times w \times h}$ is first split evenly into $n$ channel groups,

$$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$

Each group is processed with depthwise convolution using different dilation rates $1,2,\ldots,n$ to obtain multiscale representations (Zhang et al., 22 Aug 2025).

EWMS appears inside the Bidirectional Progressive Feature Interactor, where it implements the interaction function $f_{inter}$. During forward guidance, the current group $x_i$ is modulated by the previous scale’s output $y_{i-1}$ through a sigmoid gate,

$x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$

Here $\sigma(\cdot)$ is the sigmoid function, $F_{in} \in \mathbb{R}^{c \times w \times h}$0 denotes element-wise multiplication, and the final $F_{in} \in \mathbb{R}^{c \times w \times h}$1 is the element-wise summation term. The same type of interaction is also used in the backward enhancement path, where higher-level features refine lower-level ones. EWMS is therefore embedded directly into the progressive bidirectional interaction process rather than used as a standalone fusion stage after all scales are computed (Zhang et al., 22 Aug 2025).

Architecturally, EWMS sits inside the BPFI, which is one of the central components of the Group Multiscale Bidirectional Interactive module. The full pipeline is

$F_{in} \in \mathbb{R}^{c \times w \times h}$2

with

$F_{in} \in \mathbb{R}^{c \times w \times h}$3

where $F_{in} \in \mathbb{R}^{c \times w \times h}$4 is a $F_{in} \in \mathbb{R}^{c \times w \times h}$5 pointwise convolution and $F_{in} \in \mathbb{R}^{c \times w \times h}$6 is concatenation. In this formulation, EWMS acts before the final cross-scale concatenation/fusion stage and serves as the local interaction rule that enhances each scale pair progressively (Zhang et al., 22 Aug 2025).

The operator is explicitly described as parameter-free. It introduces no learnable weights and no convolutional layers; its computational burden consists only of sigmoid, element-wise multiplication, and element-wise addition. Because the operation is performed on tensors of identical shape, $F_{in} \in \mathbb{R}^{c \times w \times h}$7 and $F_{in} \in \mathbb{R}^{c \times w \times h}$8 are required to be compatible channel-wise and spatially, so the gating mask can be applied element by element. The paper states that the guidance feature and the target feature are aligned in the BPFI pathway so that multiplication and addition occur without shape transformation, which makes the operator especially suitable for real-time deployment (Zhang et al., 22 Aug 2025).

3. Interaction logic, design trade-offs, and reported behavior

The design rationale in GMBINet is stated in comparative terms. Pure element-wise addition is cheap, but it can cause semantic dilution and weak noise suppression because it simply mixes signals without emphasizing important regions. Pure element-wise multiplication can suppress irrelevant activations and highlight target areas, but it may over-filter and discard fine detail. Concatenation preserves more information, but usually requires extra convolutions to align channels and perform fusion, which increases parameters and FLOPs. EWMS merges these behaviors in a compact form: the sigmoid-gated multiplication acts as soft attention on $F_{in} \in \mathbb{R}^{c \times w \times h}$9, and the subsequent summation restores the original feature content to reduce information loss (Zhang et al., 22 Aug 2025).

The paper also contrasts EWMS with attention mechanisms. Compared with attention mechanisms, EWMS does not learn extra projection matrices, query-key-value mappings, or channel/spatial attention parameters; the modulation comes directly from the feature tensor itself. The authors further state that EWMS helps the network “emphasize salient regions via element-wise multiplication and integrate complementary context through summation,” a property presented as useful for defect detection where fine boundaries and low-contrast regions matter (Zhang et al., 22 Aug 2025).

The ablation study compares EWMS against “w/ Sum”, “w/ Multiply”, and “w/ Concat”, with the same overall Group Multiscale Bidirectional Interactive structure but different interaction operators. The reported results are that “w/ Sum” performs better than “w/ Multiply” in some metrics, but both are weaker than EWMS; “w/ Concat” improves performance more than simple sum or multiply, but increases parameters from $n$0 M to $n$1 M and FLOPs from $n$2 G to $n$3 G, while reducing speed to $n$4 FPS. EWMS achieves MAE $n$5, WF $n$6, OR $n$7, SM $n$8, PFOM $n$9, and IoU $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$0, with only $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$1 M parameters, $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$2 G FLOPs, and $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$3 FPS (Zhang et al., 22 Aug 2025).

At the model level, GMBINet is reported to deliver real-time speeds of $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$4 FPS on GPU and $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$5 FPS on CPU at $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$6 resolution, using only $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$7 M parameters, with experiments conducted on SD-Saliency-900 and NRSD-MN. Additional evaluations on NEU-CLS are described as confirming strong generalization ability beyond surface defect detection (Zhang et al., 22 Aug 2025).

4. Related neural formulations: attention surrogates, multiplicative feature fusion, and broadcasted Hadamard structure

ReduceFormer replaces standard self-attention with a simpler attention-like computation built from global summation, element-wise multiplication, and ReLU. Its core pattern is a tensor-reduction attention surrogate: reduce feature maps globally with summation, multiply reduced summaries element-wise with feature tensors, reduce again, and then use the resulting global descriptor to modulate the query features. Inside the ReduceFormer block, $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$8 is first reduced over spatial dimensions into $$x_i \in \mathbb{R}^{\frac{c}{n} \times w \times h}, \quad i \in \{1,2,\ldots,n\}.$$9, this is broadcast and multiplied with $1,2,\ldots,n$0, another global summation yields $1,2,\ldots,n$1, and $1,2,\ldots,n$2 is then element-wise multiplied with $1,2,\ldots,n$3 before another reduction produces $1,2,\ldots,n$4. The paper emphasizes that this design avoids explicit $1,2,\ldots,n$5, Softmax, and large attention maps, and reports up to $1,2,\ldots,n$6 lower latency and up to $1,2,\ldots,n$7 higher throughput. On NVIDIA DRIVE Orin, ReduceFormer-B1 at $1,2,\ldots,n$8 achieves $1,2,\ldots,n$9 ms latency versus $f_{inter}$0 ms for EfficientViT-B1, and on L40 it reaches $f_{inter}$1 images/s versus $f_{inter}$2 images/s (Yang et al., 2024).

Element-wise multiplication is also central in Deeper-PINNs. The initial feature map is formed by

$f_{inter}$3

and each shortcut block contains two further multiplicative layers,

$f_{inter}$4

$f_{inter}$5

followed by residual accumulation,

$f_{inter}$6

The paper argues that multiplication maps features into high-dimensional, non-linear spaces and helps avoid initialization pathology in PINNs. Reported best relative $f_{inter}$7 errors are $f_{inter}$8 for the Allen–Cahn equation, $f_{inter}$9 for the 2D Helmholtz equation, and average relative $x_i$0 error $x_i$1 for the 1D advection equation (Jiang et al., 2024).

The broadcast product paper formalizes a shape-aligned generalization of Hadamard multiplication,

$x_i$2

If the two tensors have the same shape, then $x_i$3; otherwise, singleton modes are duplicated to align shapes. The paper uses this operator to express masking and FiLM, and introduces marginalization identities such as

$x_i$4

In this line of work, EWMS-like behavior is not restricted to same-shape tensors but includes explicit duplication and contraction structure (Matsui et al., 2024).

5. Algebraic, finite-element, segmented, and quantum formulations

An algebraic interpretation appears in the matrix-multiplication survey through inner product and summation by binary segmentation. For vectors $x_i$5 and $x_i$6, the paper defines

$x_i$7

so that in the product $x_i$8, the middle coefficient satisfies

$x_i$9

Evaluating at $y_{i-1}$0 and applying binary segmentation yields the inner product from one large-integer multiplication; summation is recovered as a special case by setting one vector to all ones. The paper does not present EWMS as a standalone formal primitive, but it provides a direct route from coordinatewise products to a recovered sum (Pan, 2014).

In hierarchical FEM, the product-to-sum method replaces products of basis and weighting polynomials with short sums of other polynomials. The general identity is

$y_{i-1}$1

which converts a multidimensional matrix element into

$y_{i-1}$2

For the 2D example with Chebyshev polynomials, identities such as

$y_{i-1}$3

lead to a four-term decomposition of $y_{i-1}$4. The reported integral-count reduction factor is roughly $y_{i-1}$5, and the asymptotic operation-count reduction is about $y_{i-1}$6 (Khodapanah, 2017).

At the systems level, MMV-RAM studies segmented scan and segmented sum with a matrix-multiplication unit. The central decomposition is

$y_{i-1}$7

with

$y_{i-1}$8

Segmented scan is defined by

$y_{i-1}$9

and the main theorem gives an $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$0-step MMV-RAM algorithm, while any polynomial-work VCU-only algorithm requires $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$1 steps. The same paper gives element-wise integer vector multiplication in $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$2 steps with work $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$3 (Sobczyk et al., 30 Jun 2025).

The quantum element-wise transforms paper formalizes the entrywise product

$x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$4

and entrywise polynomial application

$x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$5

with $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$6 and $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$7. Its central identity is that $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$8 can be extracted from $x_i^{en} = \sigma(y_{i-1}) \times x_i + x_i.$9 by permutation and postselection,

$\sigma(\cdot)$0

The main theorem states that, given a controlled block encoding of $\sigma(\cdot)$1, one can prepare a block encoding of $\sigma(\cdot)$2 requiring $\sigma(\cdot)$3 controlled-$\sigma(\cdot)$4 copies, with ancilla cost $\sigma(\cdot)$5. The paper highlights a reduction in auxiliary space from prior $\sigma(\cdot)$6 to $\sigma(\cdot)$7 (Rossi et al., 4 Jun 2026).

6. Secure computation, optimization variants, and interpretive limits

In privacy-preserving machine learning, EVA-S2PLoR builds directly on secure 2-party vector element-wise multiplication, S2PVEM, defined by

$\sigma(\cdot)$8

The protocol disguises the input vectors as matrices $\sigma(\cdot)$9 and $F_{in} \in \mathbb{R}^{c \times w \times h}$00, invokes secure 2-party matrix multiplication, and extracts the diagonal: $F_{in} \in \mathbb{R}^{c \times w \times h}$01 The derived protocols include secure addition-to-multiplication conversion, reciprocal, and sigmoid. Reported communication rounds are $F_{in} \in \mathbb{R}^{c \times w \times h}$02 for S2PVEM, $F_{in} \in \mathbb{R}^{c \times w \times h}$03 for S2PVATM, $F_{in} \in \mathbb{R}^{c \times w \times h}$04 for S2PVR, $F_{in} \in \mathbb{R}^{c \times w \times h}$05 for S2PVS, $F_{in} \in \mathbb{R}^{c \times w \times h}$06 for S2PLoRT, and $F_{in} \in \mathbb{R}^{c \times w \times h}$07 for S2PLoRP. The abstract states that sigmoid performance improves by about $F_{in} \in \mathbb{R}^{c \times w \times h}$08 orders of magnitude compared to most frameworks, and verification failure probabilities are bounded by powers of $F_{in} \in \mathbb{R}^{c \times w \times h}$09 (Tao et al., 9 Jan 2025).

A different approximation appears in online learning. The modified online Newton step paper replaces a full $F_{in} \in \mathbb{R}^{c \times w \times h}$10 Hessian with first- and second-order quantities of size $F_{in} \in \mathbb{R}^{c \times w \times h}$11, using element-wise multiplication: $F_{in} \in \mathbb{R}^{c \times w \times h}$12 with $F_{in} \in \mathbb{R}^{c \times w \times h}$13. The paper claims memory $F_{in} \in \mathbb{R}^{c \times w \times h}$14, per-iteration time $F_{in} \in \mathbb{R}^{c \times w \times h}$15, and regret

$F_{in} \in \mathbb{R}^{c \times w \times h}$16

Its stated purpose is to retain matrices of the same size as the model while reducing computation on large multiclass datasets (Charanjeet et al., 2019).

A common misconception is to treat EWMS as a single fixed formula. The cited literature does not support that reading. In GMBINet, EWMS is a parameter-free local interaction rule based on sigmoid-gated multiplication plus element-wise summation (Zhang et al., 22 Aug 2025). In ReduceFormer, the closest equivalent is repeated global summations and element-wise multiplications used to circumvent the inner product calculation of $F_{in} \in \mathbb{R}^{c \times w \times h}$17 and $F_{in} \in \mathbb{R}^{c \times w \times h}$18 (Yang et al., 2024). In hierarchical FEM, the closest analogue is product-to-sum conversion that reduces multidimensional integration to a smaller library of reusable integrals (Khodapanah, 2017). In the quantum setting, the central objects are entrywise products and entrywise polynomial transforms assembled by LCU (Rossi et al., 4 Jun 2026). This suggests that EWMS is a useful cross-domain descriptor for low-cost multiplicative interaction plus accumulation, but its exact mathematical object remains domain-specific and must be read from the local definitions of each paper.