Channel-wise Sensitivity Analysis

• Channel-wise Sensitivity Analysis is a framework for quantifying how modifications to individual channels affect overall model performance, crucial for tasks like quantization and pruning.
• It employs both second-order Hessian trace methods and first-order gradient-based approaches to precisely measure each channel's influence on the loss, aiding resource allocation and interpretability.
• Empirical results demonstrate that leveraging channel-wise sensitivity leads to superior accuracy-compression trade-offs in deep neural models and enhanced anomaly detection in time series.

Channel-wise sensitivity analysis encompasses a set of methodologies for quantifying the relative impact of interventions, perturbations, or modifications—either to network parameters or input features—at the granularity of individual channels (output feature maps or input variables) within deep neural models. Central to its motivation is the observation that channels within a layer, or variables within a multivariate time series, can exhibit widely varying influence on global objectives such as predictive accuracy, robustness, or anomaly detection metrics. Rigorous channel-wise sensitivity analysis facilitates efficient resource allocation in quantization, interpretability for time series, and principled pruning, serving as a foundation for state-of-the-art compression and interpretability strategies in modern machine learning (Qian et al., 2020, Wang et al., 2024).

1. Conceptual Foundations and Motivation

In the context of neural networks, channel-wise sensitivity refers to the degree by which perturbing or quantizing the weights or activations associated with a specific channel alters the network's loss. Not all channels contribute equally: some are highly sensitive, such that coarse quantization or pruning induces pronounced loss changes, while others are redundant or insensitive. Correctly identifying sensitive channels is crucial, for example, in mixed-precision quantization, where the precision budget should preferentially be allocated to the most sensitive channels (Qian et al., 2020).

In multivariate time series analysis, each channel often represents a distinct sensor or input signal. Channel-wise sensitivity, in this domain, measures how changes in a particular channel's data influence the loss or output of the model. This is essential for interpretability, anomaly detection, and channel pruning in forecasting (Wang et al., 2024).

2. Second-order and First-order Sensitivity Quantification

A core technical challenge in channel-wise sensitivity analysis is effective quantification of each channel's impact. Two dominant strands exist:

Second-order (Hessian-based) approaches:

These methods assess curvature information of the loss landscape. For a parameter vector $w$ (often layer weights), the Hessian $H_w$ provides second derivatives that capture how sharply the loss increases near the solution. Channel-wise Hessian trace—specifically, the trace of the Hessian restricted to the coordinates associated with channel $j$—approximates the expected sensitivity of loss to perturbations of that channel. Large channel-wise Hessian traces indicate higher channel sensitivity. Hutchinson’s stochastic trace estimator enables tractable empirical estimation via randomized projections:

$$\mathrm{Tr}\bigl(H_w^{(j)}\bigr)\approx \frac{1}{m}\sum_{k=1}^m (z_k^{(j)})^T H_w z_k^{(j)}$$

where $z_k^{(j)}$ is a random vector masked to channel $j$ (Qian et al., 2020).

First-order (gradient-based, influence function) approaches:

These methods, rooted in robust statistics, rely on the gradient of the loss with respect to parameters or inputs. For multivariate time series, channel-wise influence functions decompose the effect of infinitesimal upweighting of a specific channel in the training data on the loss/parameters. The TracIn approximation enables scalable computation by replacing the costly Hessian inverse with a learning-rate-scaled identity, measuring pairwise inner-products of per-channel gradients:

$$\mathrm{CInf}_{i,j} = \eta \cdot (\nabla_\theta \ell(x'_i;\hat{\theta}))^T (\nabla_\theta \ell(x_j;\hat{\theta}))$$

where $i, j$ index channels in test and train samples, respectively (Wang et al., 2024).

3. Channel-wise Sensitivity Analysis in Quantization and Pruning

Mixed-precision quantization benefits from channel-wise sensitivity measurement by enabling differentiated allocation of bit-width across channels. The CW-HAWQ framework uses channel-wise Hessian traces to order channels by sensitivity and applies a deep reinforcement learning (DRL) agent to optimally partition the set of channels among available bit-widths, subject to a global compression constraint. Rather than search over the exponentially large space of per-channel assignments, CW-HAWQ employs a ratio-based strategy, where the DRL agent emits a vector of ratios specifying the fraction of remaining channels to quantize at each bit level. The prioritized list of channels according to Hessian trace underpins all assignments (Qian et al., 2020):

Channel Sensitivity Ranking Step	Description
Compute channel-wise traces	Using Eq. (2)/(4), for each channel in the layer
Sort channels	Descending by trace (most to least sensitive)
Assign bits by ratios	DRL agent allocates top channels to high bits, subject to ratio

Channel-wise influence functions in time series enable principled channel selection (pruning). Channels with low self-influence can be eliminated with minimal performance loss. Computational complexity is significantly reduced compared to second-order methods, and influence-based pruning is empirically superior to random or naive approaches (Wang et al., 2024).

4. Algorithmic Implementations

CW-HAWQ (Neural Network Quantization):

• Input: Pre-trained full-precision model, target compression ratio.
• For each layer, compute channel-wise Hessian traces; sort channels.
• Use a DDPG-based DRL agent whose action space represents bit-allocation ratios across bit-widths.
• Apply quantization and one-epoch fine-tuning in two stages (activations then weights).
• Output: Quantized model with per-channel bit assignments dictated by sensitivity (Qian et al., 2020).

Channel-wise Influence Function (MTS):

• Input: Model, per-channel loss, learning rate, test and training samples.
• For each channel, compute gradient of per-channel loss.
• Channel-wise influence matrix: pairwise gradient inner products, scaled by learning rate.
• For anomaly detection: evaluate self-influence for each channel in the test sample; spikes indicate anomalies.
• For pruning: aggregate self-influence of each channel on validation data and select least influential channels for removal (Wang et al., 2024).

5. Empirical Results and Comparative Analysis

CW-HAWQ achieves superior trade-offs in accuracy versus compression. For example, on ResNet-50/ImageNet at average weight-bit 2.61 and activation-bit 4.0, Top-1 accuracy drop is 0.20%, outperforming both HAWQ-V2 and AutoQ at matched model sizes. For aggressive compression, CW-HAWQ maintains higher accuracy than previous methods, and ablations confirm the added granularity of channel-wise (vs. layer-wise) Hessian trace further improves performance. Per-channel traces typically span multiple orders of magnitude, justifying fine-grained allocation (Qian et al., 2020).

In MTS anomaly detection, channel-wise self-influence scores achieve higher F1 scores compared to classical reconstruction error or vanilla TracIn. For instance, on the SMD dataset, channel-wise influence yields an F1 of 58.8% (GCN-LSTM), exceeding the PCA-baseline. In channel pruning for forecasting, influence-guided selection enables iTransformer and PatchTST models to retain baseline performance with as little as 45–50% of the input channels compared to >80% required under random drop (Wang et al., 2024).

6. Practical Methodologies and Extensions

Practical guidelines include:

• Begin with a pre-trained model for both quantization and time series sensitivity analysis.
• For second-order methods, estimate channel-wise traces using Hutchinson’s method with 10–50 projections per channel; for influence functions, compute channel-wise gradients on subsets of parameters (e.g., final layer) for tractability.
• Channel sorting based on sensitivity forms the backbone for downstream bit allocation or pruning.
• In RL-based approaches, enforce compression or resource constraints by clamping agent actions dynamically.
• Applications generalize to a broad class of architectures—CNNs, RNNs, transformers, TCNs, and even Gaussian processes—provided per-channel losses and gradients are available (Qian et al., 2020, Wang et al., 2024).

Channel-wise sensitivity analysis can be adapted for more complex influence approximations by incorporating low-rank Hessian inversion techniques (e.g., Lanczos, K-FAC) or leveraging gradients for channel-embedding layers. In transfer learning, channel-wise influence offers a principled route for identifying source channels with maximal benefit for target domains (Wang et al., 2024).

7. Impact and Theoretical Implications

Channel-wise sensitivity analysis provides a rigorous, scalable framework for both interpretability and efficient resource utilization in complex models. By leveraging precise measurements of channel contributions via Hessian-based or gradient-based proxies, modern frameworks such as CW-HAWQ and channel-wise influence functions enable dramatic improvements in quantization accuracy, channel pruning, and model interpretability. The substantial empirical gains highlight the inadequacy of naive, layer-wise, or feature-agnostic sensitivity strategies. Future extensions may incorporate more sophisticated curvature approximations and adapt channel-wise approaches to additional domains such as transfer learning and automated architecture search (Qian et al., 2020, Wang et al., 2024).