Growth Bound Matrices (GBM)

• Growth Bound Matrices are matrices that bound each partial derivative in a neural network, providing a detailed measure of input-output sensitivity and robustness.
• Explicit constructions for LSTM, S4, and CNN architectures demonstrate how GBM serves as a regularization technique to minimize sensitivity during training.
• Minimizing GBM leads to smoother decision boundaries and improved adversarial robustness while maintaining or enhancing clean accuracy.

Growth Bound Matrices (GBM) are a coordinate-wise sensitivity object for a neural mapping $\mathcal{F}:\mathbf{X}\to\mathbf{Y}$, defined by a matrix $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$ whose entries upper-bound the magnitude of each partial derivative $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$. In "Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approach" (Bouri et al., 14 Jul 2025), GBM is introduced as a regularization technique for NLP models under word substitution attacks, with explicit constructions for Long Short-Term Memory (LSTM), State Space models (S4), and Convolutional Neural Networks (CNN). The central motivation is to replace a single coarse scalar sensitivity measure with a matrix of elementwise derivative bounds that tracks how each input coordinate can affect each output coordinate, and to minimize that matrix-derived sensitivity during training.

1. Formal definition and robustness interpretation

For a neural mapping

$$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$

a matrix $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$ is called a Growth Bound Matrix if, for every output coordinate $i$ and input coordinate $j$,

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$

Each entry $(\mathcal{M})_{ij}$ is therefore an upper bound on the magnitude of the partial derivative of output component $i$ with respect to input component $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$0. The intended use is adversarial robustness in text classification, where synonym substitutions perturb word embeddings while ideally preserving semantics (Bouri et al., 14 Jul 2025).

The perturbation analysis is expressed through a mean-value-theorem style bound. For $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$1,

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$2

The paper’s main theoretical claim, stated as Proposition 1, is the row-wise deviation bound

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$3

This supports the interpretation that reducing GBM reduces a worst-case output deviation bound. The authors interpret this as a certification-style robustness guarantee: if perturbations are small enough and the output gap is large enough, the predicted class should not change.

GBM is also related to Lipschitz continuity. If

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$4

then $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$5 is a Lipschitz constant of $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$6. The paper emphasizes that GBM is richer than a scalar Lipschitz constant because it preserves directional sensitivity information.

2. Architecture-specific constructions

The paper derives explicit GBMs for three architectures: LSTM, S4/state space models, and CNNs. The constructions differ substantially because the underlying Jacobian structure differs substantially (Bouri et al., 14 Jul 2025).

Architecture	GBM construction	Notable feature
LSTM	Decomposed into blocks for $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$7, $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$8, and $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$9	Requires interval bounds on gate preactivations and derivatives
S4	Derived from the effective linear input-to-output Jacobian blocks	Direct because the map is linear
CNN	Derived from convolution weights and max-pooling structure	Activation derivative bounded by associated convolution weight

For LSTM, the cell is written as

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$0

with gates

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$1

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$2

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$3

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$4

The cell is treated as an input-output map

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$5

mapping $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$6. The GBM is decomposed into three blocks corresponding to sensitivity with respect to the current word embedding, the previous hidden state, and the previous cell state. For the previous cell state, the derivative simplifies to

$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$7

Because exact ranges of $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$8, $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert$$9, and related quantities are needed to bound the derivatives, the paper introduces auxiliary interval-bounding propositions for $$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$0 and $$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$1, plus interval bounds on gate preactivations and on $$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$2. This is what makes LSTM GBM the most involved case.

For S4, the continuous-time system is

$$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$3

After bilinear discretization, the recurrence becomes

$$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$4

where

$$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$5

$$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$6

The model is viewed as

$$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$7

Because this map is linear, the GBM is direct and clean: for S4 the GBM is essentially the absolute value of the effective linear input-to-output Jacobian blocks.

For CNNs, the paper uses TextCNN-style 1D convolutions with max pooling. For kernel size $$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$8,

$$\mathcal{F}:\mathbf{X}\to\mathbf{Y},\qquad x\mapsto \mathcal{F}(x),$$9

followed by

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$0

The derivation uses the fact that the activation $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$1 is Lipschitz with constant at most $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$2 for the choices considered, ReLU or tanh, so the derivative magnitude is bounded by the associated convolution weight. Max pooling does not amplify sensitivity beyond the max of the contributing positions.

3. Regularization objective and optimization

The paper does not merely compute GBM after training; it minimizes GBM during training as a robustness regularizer (Bouri et al., 14 Jul 2025). The objective is

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$3

with

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$4

The hyperparameter $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$5 controls the robustness/accuracy tradeoff: $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$6 gives standard training, while larger $\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$7 imposes stronger pressure to reduce GBM and improve robustness, potentially at the cost of clean accuracy.

The training pipeline is described as embedding the input text, computing cross-entropy loss, computing GBM loss, combining them, and updating the model parameters using Adam. Pretrained embeddings are frozen, and only the classifier/model parameters are updated. For LSTM, the GBM of the final hidden state is computed using dedicated interval-bound subroutines; for S4 and CNN, GBM is more direct because the Jacobian structure is simpler.

In the discussion, the paper further claims that minimizing GBM encourages smoother decision boundaries, which improves both adversarial robustness and generalization on clean data. A plausible implication is that GBM is intended to serve simultaneously as a local sensitivity control mechanism and as a structural regularizer.

4. Evaluation protocol in NLP robustness

The empirical study evaluates GBM on two standard text classification datasets: IMDB, with 25,000 training and 25,000 test samples for binary sentiment classification, and Yahoo! Answers, with 1.4 million training and 50,000 test samples for 10 classes (Bouri et al., 14 Jul 2025). The models studied are CNN, BiLSTM, S4, and BERT-based setups for additional comparison.

The main adversarial attacks are GA, PWWS, PSO, and TextFooler. The paper notes the synonym sources used by these attacks: PWWS uses WordNet synonyms; TextFooler and GA use counter-fitted embeddings; PSO uses sememe-based substitutions from HowNet. Attacks are generated using OpenAttack and TextAttack. Because attacks are expensive, the experiments use 1000 randomly sampled test instances per dataset; for the BERT/TextFooler comparison, 500 IMDB samples are used.

The perturbation setting is defined by

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$8

nearest neighbors in GloVe space within Euclidean distance

$\mathcal{M}\in\mathbb{R}^{n_y\times n_x}$9

This defines the synonym set $i$0. The reported metrics are Clean Accuracy (CA), the accuracy on unperturbed test data, and Accuracy Under Attack (AUA), the accuracy under adversarial attack.

Representative optimization settings are also given. For BiLSTM, the paper uses batch size $i$1, hidden size $i$2, learning rate $i$3, and weight decay $i$4. For S4, it uses batch size $i$5, hidden size $i$6, a smaller learning rate for $i$7, no weight decay on those parameters, and $i$8 versus $i$9 for others. For CNN, it uses 128 kernels, kernel sizes $j$0, learning rate $j$1, and weight decay $j$2. All experiments were run on a single NVIDIA A100 80GB GPU.

5. Quantitative findings

Across CNN and BiLSTM on IMDB and Yahoo! Answers, the paper reports that GBM consistently outperforms the baselines IBP, ATFL, SEM, and ASCC in attack robustness, while often preserving or improving clean accuracy (Bouri et al., 14 Jul 2025).

Setting	GBM result	Comparison
IMDB, BiLSTM, PSO	$j$3 AUA	$j$4 over the best baseline
IMDB, CNN, PSO	$j$5 AUA	$j$6 over the best baseline
Yahoo! Answers, BiLSTM	up to $j$7 AUA	up to $j$8
Yahoo! Answers, CNN	$j$9 AUA	up to $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$0
IMDB, BERT/TextFooler	$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$1 under attack, $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$2 clean	$$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$3 over Text-CRS

On IMDB, GBM also gives strong clean accuracy: CNN reaches $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$4, higher than all listed baselines, and BiLSTM reaches $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$5, near the best clean performance. For S4, which the paper presents as the first systematic robustness analysis of S4 in NLP, standard S4 has the highest clean accuracy on IMDB at $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$6, and GBM improves attack robustness substantially, reaching $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$7 under PWWS, $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$8 under PSO, and $$\left\lVert \frac{\partial \mathcal{F}^i}{\partial x^j}(x)\right\rVert \le (\mathcal{M})_{ij},\qquad \forall x\in \mathbf{X}.$$9 under TextFooler.

The paper also emphasizes computational efficiency relative to IBP. For BiLSTM on IMDB, IBP requires $(\mathcal{M})_{ij}$0 min/epoch, whereas GBM requires $(\mathcal{M})_{ij}$1 min/epoch, which is reported as over $(\mathcal{M})_{ij}$2 faster. For CNN on IMDB, IBP requires $(\mathcal{M})_{ij}$3 min/epoch and GBM requires $(\mathcal{M})_{ij}$4 min/epoch.

Sensitivity visualizations are reported as well. BiLSTM total sensitivity drops from $(\mathcal{M})_{ij}$5 to $(\mathcal{M})_{ij}$6, CNN total sensitivity drops from $(\mathcal{M})_{ij}$7 to $(\mathcal{M})_{ij}$8, and S4 total sensitivity drops from $(\mathcal{M})_{ij}$9 to $i$0. These numbers support the paper’s claim that GBM regularization meaningfully suppresses input-output sensitivity.

6. Terminological scope and acronym disambiguation

The acronym “GBM” is heavily overloaded in arXiv usage, and Growth Bound Matrices are distinct from several established meanings.

In high-energy astrophysics and cosmology, GBM commonly means the Fermi Gamma-ray Burst Monitor. "Size of Shell Universe in Light of Fermi GBM Transient Associated with GW150914" studies a possible gamma-ray transient seen by Fermi GBM in temporal and positional coincidence with GW150914, and uses the reported $i$1 s delay to constrain a shell-universe or spherical brane-world model (Gogberashvili et al., 2016). "Background fitting of Fermi GBM observations" develops Direction Dependent Background Fitting (DDBF) for Fermi GBM CTIME data, using direction-dependent variables derived from spacecraft geometry rather than a simple polynomial in time (Szécsi et al., 2013). "A Proposal to Localize Fermi GBM GRBs Through Coordinated Scanning of the GBM Error Circle via Optical Telescopes" uses GBM to denote the detector whose error circle is tiled by a distributed telescope network for optical afterglow searches (Ukwatta et al., 2011).

In stochastic-process theory, GBM commonly means Geometric Brownian Motion. "Phenomenology of stochastic exponential growth" treats GBM as the canonical model of stochastic exponential growth and argues that it fails to reproduce approximately stationary mean-rescaled distributions at long times (Pirjol et al., 2017). "Ergodicity breaking in geometric Brownian motion" analyzes non-ergodicity in GBM and shows that diversification via partial ensemble averaging delays but does not eliminate the difference between ensemble-average and time-average growth rates (Peters et al., 2012).

In numerical linear algebra, "On a perturbation analysis of Higham squared maximum Gaussian elimination growth matrices" studies matrices that attain the maximum growth factor under partial pivoting, with the classical bound

$i$2

and analyzes how perturbations reduce that worst-case growth (Edelman et al., 2024). The connection to Growth Bound Matrices is described there as conceptual and structural rather than terminological.

Accordingly, Growth Bound Matrices in the sense of (Bouri et al., 14 Jul 2025) are a matrix-based robustness formalism for neural networks, not a gamma-ray detector, not geometric Brownian motion, and not the growth factor of Gaussian elimination.