1-D CNN Feature Extractor

• 1-D CNN feature extractor is a specialized neural network model that uses layered 1-D convolutions to extract localized features from sequential data like time series and audio signals.
• It employs multiple convolutional, activation, and pooling layers to build hierarchical representations, efficiently downsampling data for subsequent analysis.
• The design supports applications in fault diagnosis and domain adaptation by leveraging adversarial training and optimized parameter allocation for robust performance.

A 1-dimensional convolutional neural network (1-D CNN) feature extractor is a hierarchical neural architecture designed to learn and extract meaningful representations from univariate or multivariate sequential data, such as time series, audio, vibration signals, or other 1-D signals. It employs stacked convolutional layers and channel-wise operations to progressively distill raw input sequences into lower-dimensional, semantically salient features suitable for downstream supervised or unsupervised tasks. The 1-D CNN paradigm enables the discovery of localized and hierarchical patterns across the input domain, holding significant utility in domains ranging from condition monitoring to signal processing and function approximation.

1. Formal Structure and Architectural Variants

A generic 1-D CNN feature extractor processes an input vector $x \in \mathbb{R}^d$ through a sequence of convolutional, activation, and pooling modules. Each convolutional layer applies a set of $n_j$ 1-D kernels $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$ (with kernel size $s_j$ and stride $t_j$) across the channel outputs of the preceding layer, interleaved with nonlinearities (typically ReLU) and optional channel-wise biases $B^{(j)}_\ell$. This mechanism is formalized as:

$$h_j(x)_\ell = \sigma\left( \sum_{i=1}^{n_{j-1}} W^{(j)}_{\ell,i} *_{t_j} h_{j-1}(x)_i + B^{(j)}_\ell \right)$$

where $*_{t_j}$ denotes 1-D convolution with stride $t_j$, and $\sigma$ is typically ReLU activation. Spatial dimensionality $n_j$0 and channel number $n_j$1 evolve with depth, with typical practice employing downsampling (stride equal to kernel size), causing $n_j$2 to halve at each layer, and channel counts $n_j$3 that grow then contract for parameter efficiency (Li et al., 2022).

The A2CNN architecture exemplifies a concrete 1-D CNN feature extractor for bearing fault diagnosis, operating on FFT amplitude vectors $n_j$4 through a hierarchy of five convolutional+ReLU+max-pooling blocks and two fully connected layers:

• Conv1: 8 filters, kernel=32, stride=2, output $n_j$5
• Pool1: pool=2, stride=2, output $n_j$6
• Conv2: 16 filters, kernel=16, stride=2, output $n_j$7
• Pool2: pool=2, stride=2, output $n_j$8
• Conv3: 32 filters, kernel=8, stride=2, output $n_j$9
• Pool3: pool=2, stride=2, output $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$0
• Conv4: 32 filters, kernel=8, stride=2, output $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$1
• Pool4: pool=2, stride=2, output $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$2
• Conv5: 64 filters, kernel=3, stride=2, output $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$3
• Pool5: pool=2, stride=2, output $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$4
• FC1: 500 units, ReLU
• FC2: $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$5 units (softmax label outputs) (Zhang et al., 2018)

2. Theoretical Foundations and Feature Extraction Guarantees

Rigorous analysis demonstrates that deep 1-D multi-channel CNNs, with appropriate layer depths, kernel sizes (=stride), and channel allocation, have the expressivity to serve as exact linear feature extractors. For an input $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$6 and a dictionary of $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$7 target feature vectors $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$8, a $W^{(j)}_{\ell,i} \in \mathbb{R}^{s_j}$9-layer network can realize $s_j$0 via multi-resolution convolutions. Intermediate layers yield all $s_j$1-length patch features of the target vectors, and the parameter count is bounded by $s_j$2, considerably more efficient than fully-connected layers (Li et al., 2022). This reveals a theoretical equivalence between deep 1-D CNNs and certain classical transforms (e.g., wavelets, SVD factorizations) in their capacity for structured linear feature extraction.

3. Feature Map Dimensions and Downsampling

At each layer, the output feature map length $s_j$3 is determined by the input length $s_j$4, kernel size $s_j$5, stride $s_j$6, and padding $s_j$7:

$s_j$8

For max-pooling with pool size $s_j$9:

$t_j$0

Channels per layer match the number of filters. Stride equaling kernel size yields rapid downsampling, consistent with practical approaches where spatial dimension is halved per layer, concentrating high-level receptive fields at depth (Zhang et al., 2018, Li et al., 2022).

4. Partial Layer Untying and Domain Adaptation

In domain adaptation scenarios, e.g., A2CNN, feature extractors are instantiated per domain (source $t_j$1, target $t_j$2). The lower layers (up to $t_j$3) are shared across domains, enforcing basic filter consistency, while the upper $t_j$4 layers are "untied" (domain-specific weights), allowing target-specific high-level feature adaptation. This yields a compromise between domain invariance (via tied layers) and adaptation flexibility (via untied layers), supporting robust transfer learning under covariate shift (Zhang et al., 2018).

5. Adversarial Training and Objective Functions

The 1-D CNN feature extractor can be integrated into an adversarial training scheme that enforces extracted features to be both discriminative for source labels and invariant across domains. Key loss terms are:

• Source label cross-entropy:

$t_j$5

• Domain discriminator loss (binary cross-entropy):

$t_j$6

• Adversarial loss for target extractor:

$t_j$7

The overall objective combines $t_j$8 and $t_j$9 for balance. Training follows pre-training on source, then alternating domain discriminator and adversarial feature extractor updates. Gradient reversal layers can implement adversarial updates efficiently (Zhang et al., 2018).

6. Practical Implementation Notes and Computational Considerations

1-D CNN feature extractors admit efficient implementation:

• Layer depth $B^{(j)}_\ell$0 suffices to extract $B^{(j)}_\ell$1-dimensional features with $B^{(j)}_\ell$2 parameters (Li et al., 2022).
• Pooling and strides are matched for maximal downsampling and parameter efficiency.
• Channel counts double then halve, peaking at $B^{(j)}_\ell$3, minimizing memory and computation cost.
• Post-convolutional features can be fed directly into any fully-connected network, ResNet module, or classifier head for supervised tasks.
• ReLU activations can be effectively linearized via large positive biases where needed (Zhang et al., 2018, Li et al., 2022).

7. Extensions and Analytical Perspectives

Recent analysis establishes deeper connections between 1-D and 2-D CNN architectures, using vectorization to show that multi-layer 2-D convolutions can be reformulated as 1-D convolutions with appropriately structured filters. This provides insights on how CNNs extract singular values and enables novel theoretical guarantees on their function approximation rates, which surpass those of shallow or fully-connected architectures under certain regularity conditions. Open directions include generalizing these results to stride-1 architectures, grouped/dilated convolutions, and quantifying the effects of finite precision and stochastic optimization (Li et al., 2022).