Sub-information Excitation Convolution

• Sub-information Excitation Convolution is a refinement of squeeze-and-excitation that recalibrates CNN channels using compressed intermediate descriptors instead of a single pooled vector.
• It integrates competitive channel gating by jointly processing residual and identity branch information through convolution on re-imaged descriptor maps.
• Empirical results across residual architectures indicate modest accuracy gains with controlled parameter overhead by leveraging structured intermediate cues.

Sub-information Excitation Convolution can be understood as an interpretive umbrella for squeeze-and-excitation mechanisms that generate channel recalibration from compressed intermediate descriptors rather than from a single residual-branch vector alone, and that, in the strongest case, apply convolution to a rearranged representation of those descriptors before producing channel gates. The term itself is not defined in the cited papers. Its closest explicit precursor is the competitive inner-imaging formulation of channel excitation in residual networks, where squeezed residual and identity descriptors are reorganized into a small two-dimensional structure and processed by convolution (Hu et al., 2018). A looser but relevant adjacent line is the staged modification of squeeze and excitation in residual CNNs, where channel information is compressed and restored gradually through intermediate bottleneck states rather than by a single abrupt reduction and expansion (NV, 2023).

1. Conceptual basis in squeeze-and-excitation networks

The underlying reference point is the original squeeze-and-excitation pattern used inside CNNs and especially residual blocks. Given a convolutional feature tensor

$$\mathbf{U} = [\mathbf{u}_1,\dots,\mathbf{u}_C] \in \mathbb{R}^{H \times W \times C},$$

the squeeze step aggregates each channel spatially by global average pooling,

$$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$

producing a descriptor $\mathbf{z} \in \mathbb{R}^{C}$. The excitation step maps that descriptor through a bottleneck MLP,

$$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$

with

$$\mathbf{W}_1 \in \mathbb{R}^{\frac{C}{r} \times C}, \qquad \mathbf{W}_2 \in \mathbb{R}^{C \times \frac{C}{r}},$$

and the resulting channel weights recalibrate the feature maps by

$\tilde{\mathbf{x}}_c = s_c \cdot \mathbf{u}_c.$

Within a residual block, the SE unit is inserted after the convolutional transformations and before residual addition, so that a common form is

$$\mathbf{y} = \mathbf{x} + F_{\text{SE}}(F_{\text{res}}(\mathbf{x})) .$$

These definitions are the formal substrate from which later variants depart (Hu et al., 2018).

Within this baseline, channel attention is computed from a globally pooled summary and modeled by dense projections. What later work modifies is either the source of the summary, the internal route through the bottleneck, or the operator used to model inter-channel dependence. In that sense, sub-information excitation is best treated as a refinement of the excitation stage rather than as a different family of convolutional backbones.

2. Competitive excitation from intermediate branch information

The most explicit foundation for Sub-information Excitation Convolution is the competitive squeeze-and-excitation mechanism for residual networks. In that formulation, the residual branch output is

$$\mathrm{U}_{r}=F_{res}(\mathrm{x}, \mathrm{w}_{r})= [\mathrm{u}_{r}^{1}, \mathrm{u}_{r}^{2}, \dots, \mathrm{u}_{r}^{C}],$$

and the identity branch is

$$\mathrm{X}_{id} = [\mathrm{x}_{id}^{1}, \mathrm{x}_{id}^{2}, \dots, \mathrm{x}_{id}^{C}].$$

Both are globally squeezed: $$\hat{u}_{r}^{c} = \frac{1}{W \times H} \sum_{i=1}^{W}\sum_{j=1}^{H}u_{r}^{c}(i, j), \qquad \hat{x}_{id}^{c} = \frac{1}{W \times H} \sum_{i=1}^{W}\sum_{j=1}^{H}{x}_{id}^{c}(i, j).$$ The excitation vector is then generated jointly from the residual and identity descriptors: $$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$0 where

$$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$1

The final block takes the form

$$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$2

This mechanism is called “competitive” because the identity pathway conditions how much residual correction should be injected, even though only the residual branch is rescaled (Hu et al., 2018).

This is a strong match to the “sub-information” idea because the gate is no longer derived from one pooled vector alone. Instead, it depends on intermediate branch summaries: the residual mapping and the identity mapping. The paper does not define sub-information as a formal term, but it explicitly shifts channel excitation toward compressed branch-level information. That is a more specific and structurally richer dependency than standard residual-only SE.

A further implication is that excitation becomes branch-relative rather than branch-isolated. The identity stream acts as a conditioning signal that indicates what is already being forwarded. This suggests a formulation in which channel importance is assessed against a baseline carried by the skip path, not merely from the transformed features themselves.

3. Convolution over re-imaged descriptors: inner-imaging

The decisive “convolution” component appears in the inner-imaging mechanism. After global pooling, the residual and identity descriptors are stacked row-wise into a pair-view map: $$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$3 This produces a small image-like representation of size $$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$4. The paper then applies convolutions to this descriptor image rather than relying only on a vector MLP. For the pair-view version with $$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$5 kernels,

$$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$6

followed by BN and then

$$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$7

A $$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$8 pair-view alternative is also defined, and a folded inner-imaging mode reshapes the $$z_c = \frac{1}{H W}\sum_{i=1}^{H}\sum_{j=1}^{W} u_c(i,j),$$9 map into a more square matrix $\mathbf{z} \in \mathbb{R}^{C}$0, enabling $\mathbf{z} \in \mathbb{R}^{C}$1 convolution: $\mathbf{z} \in \mathbb{R}^{C}$2 after which the result is flattened and sent to the FC excitation layers (Hu et al., 2018).

This formulation is the clearest technical realization of excitation by convolution over compressed intermediate information. The convolved object is not the original spatial feature map; it is a rearranged representation of globally squeezed channel descriptors. The resulting channel gates therefore arise from convolution over a secondary internal representation. This suggests that “Sub-information Excitation Convolution” is most faithfully instantiated when the excitation operator acts on re-indexed summaries of intermediate information rather than on the raw backbone activations.

The method remains hybrid rather than purely convolutional end to end. Even in inner-imaging mode, the convolved descriptor is still fed through FC layers before the final sigmoid gate is produced. The paper therefore extends SE toward convolutional channel-dependency modeling without abandoning the SE bottleneck structure altogether.

4. Progressive bottleneck processing in SE variations

A second, more indirect line of relevance comes from staged modifications of squeeze-and-excitation blocks. This work proposes four variants inserted inside residual/basic blocks, mainly in ResNet-18: slow squeeze and excite, slow squeeze and slow excite, squeeze and slow excite, and squeeze–learn–excite / Bump module (NV, 2023).

In standard SE, the bottleneck transition is a one-step reduction and one-step restoration,

$\mathbf{z} \in \mathbb{R}^{C}$3

The proposed variants replace that abrupt transition by staged intermediate states. In slow squeeze and excite, the squeeze is performed in two reduction steps, conceptually

$\mathbf{z} \in \mathbb{R}^{C}$4

An abstract structural summary is

$\mathbf{z} \in \mathbb{R}^{C}$5

In slow squeeze and slow excite, both sides of the bottleneck are staged,

$\mathbf{z} \in \mathbb{R}^{C}$6

with the corresponding abstraction

$\mathbf{z} \in \mathbb{R}^{C}$7

In squeeze and slow excite, the squeeze is standard but the excitation is staged. In Bump, an extra learning layer is inserted at the compressed state: $\mathbf{z} \in \mathbb{R}^{C}$8 abstractly

$\mathbf{z} \in \mathbb{R}^{C}$9

The paper’s stated motivation is to avoid a “sudden change in the network weights” and to create a “smooth transition of layer weights.” It argues that gradual compression and gradual expansion may preserve weaker or “unimportant” information that might otherwise be discarded too aggressively in a single bottleneck step. The paper does not define exact intermediate dimensions, the reduction ratio $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$0, or exact activation choices for all variants, so these formulas should be treated as interpretive structural abstractions rather than paper-quoted equations (NV, 2023).

These variants are adjacent rather than direct instances of sub-information excitation convolution. They do not partition channels into groups, use spatial subregions, or apply convolution to descriptor images. Their relevance lies instead in the idea that compressed channel information can be processed through intermediate subspaces before final gate generation. This suggests a broader interpretation of sub-information excitation in which “sub-information” refers not only to branch summaries but also to staged latent bottleneck states.

5. Empirical evidence across residual architectures

The competitive inner-imaging work evaluates on CIFAR-10, CIFAR-100, SVHN, and ImageNet. On pre-act ResNet-164, SE-ResNet-164 reports 4.85 error on CIFAR-10 and 22.61 on CIFAR-100, while CMPE-SE-164 reports 4.72 / 22.38 for the double-FC version, 4.59 / 22.41 for $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$1, 4.57 / 22.35 for $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$2, and 4.60 / 22.38 for folded $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$3. On WRN-28-10, SE-WRN-28-10 reports 3.88 / 19.05, whereas CMPE-SE-WRN-28-10 reports 3.66 / 18.69 for double FC, 3.73 / 18.71 for $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$4, 3.58 / 18.55 for $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$5, and 3.59 / 18.47 for folded $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$6. On ImageNet with ResNet-50, SE-ResNet-50 gives 23.29 / 6.62 top-1/top-5, while CMPE-SE-ResNet-50 gives 23.06 / 6.46 for double FC, 22.97 / 6.41 for $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$7, and 22.79 / 6.35 for folded $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$8. The paper also states that the parameter overhead over SE residual networks is around $$\mathbf{s} = \sigma\!\left(\mathbf{W}_2 \,\delta(\mathbf{W}_1 \mathbf{z})\right),$$9, depending on variant (Hu et al., 2018).

The staged SE-variation work evaluates mainly on ResNet-18 over CIFAR-10, CIFAR-100, and SVHN, with an additional 3-layer CNN experiment on CIFAR-10. On ResNet-18 for CIFAR-10, the default model gives 73.130 / 97.620 top-1/top-5, the SE module gives 75.580 / 97.210, slow squeeze and excite gives 74.370 / 96.910, squeeze and slow excite gives 75.630 / 97.160, slow squeeze and slow excite gives 76.020 / 97.700, and Bump gives 76.020 / 97.810. On CIFAR-100, default gives 72.870 / 92.380, SE gives 74.460 / 93.190, slow squeeze and excite gives 73.340 / 92.960, squeeze and slow excite gives 73.500 / 93.910, slow squeeze and slow excite gives 73.250 / 92.860, and Bump gives 73.960 / 93.200. On SVHN, the table reports default 91.211 / 98.851, SE 91.795 / 98.855, slow squeeze and excite 92.048 / 98.959, squeeze and slow excite 92.244 / 98.871, slow squeeze and slow excite 92.187 / 98.782, and Bump 91.576 / 98.882, with the paper also remarking that the accuracy differences are extremely small and that practical significance is unclear (NV, 2023).

Taken together, these results support two distinct empirical patterns. First, branch-aware excitation with inner-imaging convolution shows consistent improvements over standard SE across multiple residual architectures, including ImageNet. Second, gradual bottleneck staging can help in some settings, especially CIFAR-10 on ResNet-18, but its gains are less uniform: standard SE remains best on CIFAR-100 top-1 in that study.

6. Scope, limitations, and precise interpretation

Several boundaries are explicit. The competitive inner-imaging work does not implement a formal normalized competition such as a softmax across branches; its “competition” is a learned joint conditioning function. Its convolution is applied to an artificial descriptor image whose spatial neighborhood structure is induced by stacking and folding rather than inherited from the original feature map. Even in inner-imaging mode, the final excitation remains partly MLP-based because the convolved descriptor still passes through FC layers before gate generation (Hu et al., 2018).

The staged SE-variation work has more substantial ambiguities. It does not specify the exact reduction ratio $$\mathbf{W}_1 \in \mathbb{R}^{\frac{C}{r} \times C}, \qquad \mathbf{W}_2 \in \mathbb{R}^{C \times \frac{C}{r}},$$0, the hidden dimensions of the intermediate FC layers, or formal equations for expressions such as “half of the reduction” and “reduction//2.” The Bump module is inconsistently described as using a dense/FC layer and a $$\mathbf{W}_1 \in \mathbb{R}^{\frac{C}{r} \times C}, \qquad \mathbf{W}_2 \in \mathbb{R}^{C \times \frac{C}{r}},$$1 convolution layer, even though the squeezed representation is also described as $$\mathbf{W}_1 \in \mathbb{R}^{\frac{C}{r} \times C}, \qquad \mathbf{W}_2 \in \mathbb{R}^{C \times \frac{C}{r}},$$2, which makes the FC interpretation the coherent one. The paper reports no parameter count, FLOP increase, latency, or memory overhead, and it does not include multiple runs, error bars, or parameter-matched ablations (NV, 2023).

These limits matter for terminology. A strict reading would reserve Sub-information Excitation Convolution for mechanisms that satisfy three properties simultaneously: the excitation is derived from compressed intermediate descriptors, those descriptors contain more structure than a single residual-only vector, and convolution is used to model dependencies within the compressed representation. Under that stricter criterion, the inner-imaging competitive SE mechanism is a direct precursor, whereas the staged squeeze/excite variants are adjacent inspiration rather than a formal instance.

A plausible implication is that the phrase names a design direction rather than a settled architecture. In the available literature, its most concrete form is branch-competitive channel attention with convolution over re-imaged squeezed descriptors, while its broader conceptual perimeter includes gradual processing of compressed channel information inside the SE bottleneck.