1D Attention U-Net

• One-Dimensional Attention U-Net is a U-Net backbone adapted for 1D signals with temporal attention gates that modulate skip connections for selective feature fusion.
• It replaces 2D operations with 1D convolutions, pooling, and upsampling, making it versatile for applications such as adversarial speech enhancement and medical imaging segmentation.
• Empirical results show modest improvements in speech and imaging tasks, highlighting design trade-offs and the potential for further enhancements using adversarial and multi-head attention strategies.

One-Dimensional Attention U-Net denotes a family of encoder–decoder U-Net architectures that operate on one-dimensional sequences and augment skip-connected feature fusion with attention mechanisms. In the speech-defense formulation of U-Net$_{At}$, the model is explicitly described as a “one-dimensional U-Net with down-sampling blocks and a sequential attention gate embedded up-sampling blocks,” operating directly on single-channel waveforms and using temporal attention to enhance adversarial speech before automatic speech recognition (ASR) (Yang et al., 2020). More generally, the architectural logic descends from Attention U-Net, in which trainable attention gates modulate encoder features before concatenation with decoder features; this mechanism is dimension-agnostic and transfers from 2D or 3D imaging to 1D sequence domains by replacing convolution, pooling, and upsampling operators with their 1D counterparts (Oktay et al., 2018).

1. Definition, scope, and representative formulations

A One-Dimensional Attention U-Net is not a single canonical model. Within the literature represented here, it refers to a 1D U-Net backbone whose skip connections, decoder blocks, or decoder-side feature aggregation are conditioned by an attention mechanism defined along the sequence axis. In the most literal formulation, all convolutions, poolings, and upsamplings are 1D, and the attention mechanism is applied over temporal or sequential positions rather than over 2D pixels or 3D voxels (Yang et al., 2020).

Paper	Domain	1D-relevant formulation
(Oktay et al., 2018)	Medical image segmentation	Additive attention gates on skip connections; dimension-agnostic transfer to 1D
(Yang et al., 2020)	Adversarially robust speech enhancement	Time-domain 1D U-Net with temporal self-attention in upsampling blocks
(Mulat et al., 2024)	Salient object detection	Attention-gated skips plus multi-scale side outputs and fusion, directly mappable to 1D
(Silva et al., 28 Nov 2025)	Seismic horizon interpretation	Context Fusion Attention gates combining semantic, spatial, and edge-aware heads
(Liu et al., 2022)	3D intracerebral haemorrhage segmentation	Factorized orthogonal attention constructed from 1D axis summaries

The term therefore encompasses several related but non-identical design lines. One line uses additive attention gates, as in Attention U-Net, to suppress irrelevant encoder activations before skip concatenation. A second line uses temporal scaled dot-product attention over sequence features, as in U-Net$_{At}$. A third line extends skip gating with auxiliary structure such as multi-scale side outputs, explicit gradient features, or factorized axis-wise attention (Oktay et al., 2018).

2. Canonical encoder–decoder structure in one dimension

The core backbone remains the U-Net encoder–decoder. For 1D signals, the input is a sequence tensor, and the network repeatedly compresses the sequence through downsampling, then reconstructs it through upsampling while reinjecting higher-resolution encoder features through skip connections. In the speech formulation, the waveform input is

$\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$

and the model adopts a Wave-U-Net style architecture that operates directly in the time domain via 1D convolutions (Yang et al., 2020).

In U-Net$_{At}$, the encoder is a sequence of 1D convolutional downsampling blocks that produce multi-scale temporal feature maps. The decoder consists of 1D upsampling followed by convolutional refinement, with skip connections from matching encoder scales. The paper states that it uses block number $L = 17, C = 1$ for the experiments, following Macartney and Weyde’s validated Wave-U-Net setting. The final layer is a 1D convolution with filter size $1$ and $2 \times 1$ filters, corresponding conceptually to two reconstructed sources: clean speech $S_1$ and an adversarial or noise source $S_{\text{adv}}$ (Yang et al., 2020).

Generic 1D adaptations of Attention U-Net retain the same structural pattern. In the original Attention U-Net, the encoder performs repeated convolutions with ReLU and batch normalization followed by downsampling by a factor of $2$ at each scale, and the decoder upsamples, concatenates gated skip features, and applies further convolutions. The formulation is independent of spatial dimensionality: for 1D sequence data, Conv3d or Conv2d becomes Conv1d, max-pooling becomes MaxPool1d, and trilinear or bilinear interpolation becomes 1D interpolation (Oktay et al., 2018). SalFAU-Net preserves the same logic while attaching a side-output prediction head to each decoder block, a pattern that the authors explicitly describe as directly translatable to 1D signals (Mulat et al., 2024).

3. Attention formulations along the sequence axis

The principal distinction between a plain 1D U-Net and a One-Dimensional Attention U-Net lies in how skip features are filtered before decoder fusion. In Attention U-Net, the gate receives an encoder feature vector $_{At}$0 at position $_{At}$1 and a gating feature $_{At}$2 from a coarser decoder scale, then computes a scalar coefficient $_{At}$3 that rescales the encoder features:

$_{At}$4

$_{At}$5

Here $_{At}$6 is ReLU, $_{At}$7 is sigmoid, and the linear maps are implemented as kernel-size-$_{At}$8 convolutions. In 1D, the index $_{At}$9 ranges over sequence positions rather than pixels or voxels (Oktay et al., 2018).

U-Net$\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$0 uses a different attention formalism. The attention gate is attached to the upsampling blocks, and the paper states that it adopts the scaled dot-product softmax function for self-attention transformation. Its high-level formulation is

$\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$1

where $\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$2 denotes the corresponding cropped encoder feature map on the skip path. The attention is explicitly temporal: the network forms a query from the decoder state at time $\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$3, compares it against keys over temporal positions, and aggregates skip features into a context vector that emphasizes relevant time segments (Yang et al., 2020).

Subsequent variants enrich the gate without changing the encoder–decoder principle. SalFAU-Net keeps additive attention gates on all skip connections and supplements them with multi-scale side outputs whose fused prediction is supervised by binary cross-entropy (Mulat et al., 2024). CFA U-Net replaces a single encoder projection inside the gate with three heads—semantic, spatial, and Sobel-derived edge-aware—so that the gate fuses context, local neighborhood structure, and geometric gradients before producing its attention coefficient (Silva et al., 28 Nov 2025). Viola-Unet goes further by constructing 3D attention from orthogonal 1D axis summaries, showing that low-dimensional factorizations of attention can provide long-range context without full volumetric self-attention; in a 1D reduction, this collapses to a sequence-axis attention map optionally combined with channel modulation (Liu et al., 2022).

4. Objectives, supervision, and optimization regimes

A One-Dimensional Attention U-Net is architecturally defined, not loss-defined. The objective depends on the task. In adversarial speech enhancement, U-Net$\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$4 is trained as a source-separation or waveform-regression model and then strengthened through adversarial training:

$\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$5

with $\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$6. The paper distinguishes two perturbation regimes: externally generated ASR attacks used as data, and internal FGSM perturbations of the enhancement input used for regularization (Yang et al., 2020).

In segmentation-oriented formulations, the losses reflect dense labeling. Attention U-Net uses a Sørensen–Dice loss and deep supervision at several decoder scales (Oktay et al., 2018). SalFAU-Net uses deep supervision with four side losses and one fusion loss, each instantiated as pixel-wise binary cross-entropy; its total loss is the weighted sum of the side and fusion terms (Mulat et al., 2024). CFA U-Net uses a composite BCE plus Dice loss with $\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$7, which the authors present as suitable for sparse-label regimes and class imbalance (Silva et al., 28 Nov 2025). Viola-Unet, in contrast, is trained with a Dice loss plus Focal loss and an SGD optimizer with Nesterov momentum $\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$8, cosine annealing, and learning-rate warm-up (Liu et al., 2022).

These differences matter because attention does not eliminate task-specific optimization trade-offs. A gate may improve localization, continuity, or robustness, but the observed operating point remains strongly coupled to the loss, supervision density, and post-processing pipeline.

5. Empirical behavior across domains

The most explicit literal 1D example is adversarially robust speech enhancement. U-Net$\mathbf{m} \in [-1,1]^{L \times C},\quad C = 1,$9 is inserted in front of Mozilla DeepSpeech, enhancing adversarial or noisy waveforms before ASR. On standard noisy speech, the paper reports that U-Net-based methods outperform a plain DNN baseline, and that attention gives slight additional gains, including PESQ $_{At}$0 and SNR $_{At}$1. On over-the-air adversarial examples without adversarial training, enhancement increases SNR but degrades PESQ, STI, and STOI relative to raw adversarial input, indicating that naive enhancement can hurt perceptual quality. With adversarial training of the enhancement model, U-Net$_{At}$2 reaches PESQ $_{At}$3, STI $_{At}$4, STOI $_{At}$5, and SNR $_{At}$6 from adversarial inputs that begin at PESQ $_{At}$7, STI $_{At}$8, STOI $_{At}$9, and SNR $L = 17, C = 1$0 dB; the abstract separately reports improvements from $L = 17, C = 1$1 to $L = 17, C = 1$2 in PESQ, from $L = 17, C = 1$3 to $L = 17, C = 1$4 in STI, and from $L = 17, C = 1$5 to $L = 17, C = 1$6 in STOI (Yang et al., 2020).

The same work reports ASR robustness gains. Under gradient-based attack, U-Net$L = 17, C = 1$7 reduces WER to $L = 17, C = 1$8 for DeepSpeech and to $L = 17, C = 1$9 for DeepSpeech+AdvT, compared with $1$0 and $1$1 without speech enhancement; under evolutionary attack, it reduces WER to $1$2 and $1$3, compared with $1$4 and $1$5 without speech enhancement. The abstract highlights absolute WER decreases of $1$6 under gradient-based perturbation and $1$7 under evolutionary-optimized perturbation. In a stronger adaptive attack, the required SNR of adversarial noise to reach $1$8 targeted attack success shifts from approximately $1$9 dB or $2 \times 1$0 dB without enhancement to $2 \times 1$1 dB or $2 \times 1$2 dB with U-Net$2 \times 1$3, implying that substantially more audible perturbations are required for a successful attack (Yang et al., 2020).

Outside speech, attention-gated U-Nets show a recurring pattern: modest architectural changes at skip fusion often yield meaningful gains on localization-sensitive tasks. Attention U-Net improves pancreas Dice on CT-150 from $2 \times 1$4 to $2 \times 1$5, improves recall from $2 \times 1$6 to $2 \times 1$7, and reduces pancreas surface-to-surface distance from $2 \times 1$8 mm to $2 \times 1$9 mm (Oktay et al., 2018). SalFAU-Net is reported as competitive on six salient-object-detection datasets in terms of MAE, F-measure, s-measure, and e-measure (Mulat et al., 2024). An attention-based U-Net for lane detection reports $S_1$0 accuracy, $S_1$1 precision, $S_1$2 recall, and $S_1$3 IoU, with an absolute IoU gain of $S_1$4 percentage points over the re-implemented Brad et al. baseline (Tangestanizadeh et al., 2024). CFA U-Net reaches a validation IoU of $S_1$5 and MAE of $S_1$6 ms on the Mexilhão dataset and $S_1$7 surface coverage on the F3 Block dataset under sparse conditions (Silva et al., 28 Nov 2025). Viola-Unet improves mean cross-validation DSC from $S_1$8 to $S_1$9 over nnU-Net-base and attains the best NSD and RVD among challenge submissions with $S_{\text{adv}}$0 NSD and $S_{\text{adv}}$1 RVD (Liu et al., 2022).

6. Conceptual boundaries, misconceptions, and open directions

Several misconceptions recur around the term. First, “one-dimensional” does not mean channel-only attention. In the speech formulation, it means that the network operates on 1D sequences along time and that all convolutions, poolings, and upsamplings are 1D; the attention itself is also temporal (Yang et al., 2020). Second, an Attention U-Net is not necessarily Transformer-based. The best-known formulation uses additive gating on skip connections rather than a full self-attention stack, and its essential operation is relevance-weighted feature selection before concatenation (Oktay et al., 2018). Third, attention does not guarantee monotonic quality improvement: in adversarial speech enhancement, attention without adversarial training improves SNR while degrading PESQ, STI, and STOI relative to raw adversarial input (Yang et al., 2020).

The literature also indicates substantive design variability. Some models gate every skip connection, whereas original Attention U-Net notes that very low-level skip connections are typically not gated because they are not yet highly semantic (Oktay et al., 2018). Some models prioritize precision at the risk of missing uncertain regions: the seismic results explicitly state that Attention U-Net is high-precision but low-recall under sparse data, while CFA U-Net mitigates that trade-off through semantic, spatial, and edge-aware context fusion (Silva et al., 28 Nov 2025). This suggests that the gate design, not merely the presence of attention, governs the precision–recall balance.

Open directions are stated explicitly in several sources. For adversarially robust speech enhancement, the cited future work includes multi-head self-attention, multi-resolution or multi-band attention, and joint end-to-end training with ASR (Yang et al., 2020). For lane detection, future work is directed toward low light, shadows, rain, snow, and blurred lane markings (Tangestanizadeh et al., 2024). A plausible implication is that One-Dimensional Attention U-Net will remain less a fixed architecture than a design pattern: a 1D U-Net backbone whose skip pathways or decoder features are modulated by task-specific attention operators chosen to emphasize relevant subsequences, suppress irrelevant context, and stabilize multi-scale reconstruction.