Bidirectional Feature Extraction

• Bidirectional feature extraction is a computational technique that processes input data in both forward and backward directions to capture richer contextual features.
• It employs architectures like BiLSTMs, convolutional blocks, and reversible fusion modules to enhance performance in language, vision, and acoustic tasks.
• Empirical studies show that integrating bidirectional flows boosts accuracy and efficiency, reducing training costs while improving multi-scale feature representation.

Bidirectional feature extraction refers to a broad class of computational architectures and algorithms that process input data sequentially (or recursively) in both forward and backward directions—often using parallel or intertwined modules—with the explicit aim of capturing contextual dependencies from both past and future (or hierarchical parent and child) structures. This bidirectionality, realized in forms ranging from recurrent networks (e.g., BiLSTM, bidirectional reservoir computing) to bidirectional convolutional and reversible fusion blocks, enables richer feature representations and superior performance on tasks where local and global context jointly determine the semantics of observations. Contemporary research demonstrates its impact across diverse domains including language processing, computer vision, acoustic modeling, structured prediction, and hyperspectral imaging.

1. Core Principles and Theoretical Foundations

Bidirectional feature extraction fundamentally augments classical unidirectional paradigms by enabling models to access contextual signals in both directions along either time, sequence, spatial, or tree axes. In neural sequence models, this is realized by independently (or jointly) processing input via a forward path (e.g., left-to-right for text, low-to-high for spectral bands) and a backward path (reverse order). The per-token or per-frame feature representations from both directions are then fused, typically via concatenation, summation, or attention-based weighting.

A canonical instantiation employs bidirectional LSTMs (BiLSTMs), in which for each time step $t$:

• The forward recurrent state $h_t^f$ is computed left-to-right
• The backward state $h_t^b$ is computed right-to-left
• The final encoding is $h_t = [h_t^f; h_t^b]$ (concatenation)

Similar principles are applied in convolutional, graph-based, and reservoir-computing architectures, where bidirectional passes traverse either sequence or hierarchical structures to propagate information from both ancestors and descendants (e.g., bottom-up and top-down in trees) (Chalapathy et al., 2016, Kiperwasser et al., 2016, Luo et al., 2018, Liu et al., 2017, Yang et al., 2024, Yang et al., 2024).

2. Bidirectional Feature Extraction Architectures Across Modalities

2.1 Sequence and Structured Prediction

In language sequence labeling and parsing, bidirectional feature extraction—often via stacked BiLSTM or BiLSTM-CRF—permits robust token-wise encoding that leverages both left and right context. In dependency parsing, BiLSTM representations provide highly compact and effective feature sets for both greedy transition-based and globally optimized graph-based decoders (Kiperwasser et al., 2016). In aspect term extraction, models like BiDTree propagate information both from dependents to head (bottom-up) and from head to dependents (top-down) on dependency trees, yielding richer syntactic features (Luo et al., 2018).

2.2 Multiscale and Multidirectional Aggregation

Feature pyramid architectures in vision routinely exploit bidirectional fusion to aggregate semantic information across scales. For example, RevBiFPN introduces reversible bidirectional fusion modules (RevSilos) that perform bottom-up coarse aggregation, then top-down refinement, merging features across scales in both directions while maintaining invertibility and minimizing memory (Chiley et al., 2022). In speaker verification, BMFA iteratively refines multiscale features via both top-down and bottom-up branches, with attention-based fusion (AFM) adaptively weighting the contribution from each direction (Qi et al., 2021).

2.3 Spectral-Spatial Feature Fusion in Hyperspectral Imaging

Bidirectional spectral processing is critical for high-dimensional data such as hyperspectral images (HSIs). Methods such as Bi-CLSTM (Liu et al., 2017), HSIMamba (Yang et al., 2024), and the SS-non-Linear Model (Yang et al., 2024) employ parallel 1-D convolutions or recurrent state updates along increasing and decreasing spectral band orders. The resultant forward and backward spectral features are fused—typically summed or concatenated—before passing to a spatial refinement block, enabling comprehensive spectral-spatial context integration. Experimental results demonstrate that the omission of either direction (forward or backward) leads to marked performance degradation.

2.4 Bidirectional Information Flow in Relation Extraction

In structured information extraction, bidirectionality extends to combinatorial frameworks. The BiRTE model (Ren et al., 2021) launches parallel subject-to-object and object-to-subject entity pairing modules, each guided by a shared encoder, recovering relational triples missed by strictly unidirectional models. The interplay of feature flows between the two extraction directions, combined with shared gradient-aware learning rates, achieves state-of-the-art extraction F1 across complex relational datasets.

2.5 Reservoir Computing and Lightweight Real-time Systems

Parallel bidirectional reservoir architectures, as in PBRC for sign language recognition (Singh et al., 22 Dec 2025), provide an efficient, trainable yet non-gradient-based approach to bidirectional feature extraction for temporal signals. Two echo state network-based bidirectional modules process time series both in forward and reversed temporal order, their concatenated states forming a compact input for a linear classifier. The approach achieves real-time inference and orders-of-magnitude reduction in training cost compared to deep learning baselines.

2.6 Generative and Inverse Problems

Neural vocoders such as BiVocoder (Du et al., 2024) integrate bidirectional feature extraction and waveform synthesis, processing amplitude and phase spectra via parallel ConvNeXt V2 branches. The learned feature embedding supports both direct acoustic modeling (analysis) and high-fidelity waveform reconstruction (synthesis) with explicit bidirectionality in information flow.

3. Mathematical Formulations and Fusion Mechanisms

Bidirectional architectures typically instantiate two mirrored computational graphs per input sequence, spatial axis, or graph/tree, with outputs $h_t^{\rightarrow}$ and $h_t^{\leftarrow}$. Fusion strategies include:

• Concatenation: $h_t = [h_t^{\rightarrow}; h_t^{\leftarrow}]$ (standard in BiLSTM, Bi-CLSTM, BiDTree)
• Summation / Averaging: $h_{fused} = h_t^{\rightarrow} + h_t^{\leftarrow}$ or mean across positions
• Attention/Gating: Learnable or data-dependent mixture weights, as in AFM for adaptive fusion (Qi et al., 2021), or light dynamic gates in HSIMamba/SS-non-Linear (Yang et al., 2024, Yang et al., 2024)

In multiscale architectures (e.g., RevBiFPN), bidirectional flow is realized as stacked pairs of coarse-to-fine and fine-to-coarse modules, with additive fusion at each scale and invertible mappings for memory efficiency (Chiley et al., 2022).

For tree-structured data, bidirectional propagation involves distinct parameter sets for bottom-up and top-down recursions. In the BiDTree framework for aspect term extraction, this produces per-node embeddings concatenating both dependency directions (Luo et al., 2018).

4. Empirical Benefits and Quantitative Impact

Bidirectional feature extraction consistently yields empirical gains:

• In sequence labeling, BiLSTM-CRF architectures outperform both purely CRF and unidirectional LSTM baselines by 1–2 F1 points, with F1 reaching 83.88 on i2b2/VA concept extraction (Chalapathy et al., 2016).
• In dependency parsing, BiLSTM-based features attain or surpass the state of the art without elaborate feature engineering (Kiperwasser et al., 2016).
• In HSI classification, fully bidirectional models (HSIMamba, SS-non-Linear, Bi-CLSTM) yield OA gains of 2–6% over CNN or transformer architectures, and significantly improved Kappa statistics (Yang et al., 2024, Yang et al., 2024, Liu et al., 2017).
• In relation extraction, BiRTE restores triples lost by unidirectional pipelines, achieves up to 93.6% F1, and reduces extraction errors from entity failures (Ren et al., 2021).
• For memory-limited or real-time domains, bidirectional reservoir computing (PBRC) cuts training time by 2–3 orders of magnitude versus Bi-GRU while maintaining competitive accuracy (Singh et al., 22 Dec 2025).
• In speech, BiVocoder attains analysis–synthesis UTMOS = 4.06, outperforming HiFi-GAN and APNet (Du et al., 2024).

Ablation studies repeatedly confirm that disabling one directional module degrades accuracy, with full bidirectional fusion essential for optimal representation (Yang et al., 2024, Yang et al., 2024, Liu et al., 2017, Luo et al., 2018).

5. Computational Trade-offs, Efficiency, and Limitations

Bidirectional designs typically require duplicated computation per input position but do not inherently double model complexity because of parameter sharing, fusion, and optimization. Recent developments in reversible and lightweight model classes (e.g., RevBiFPN, PBRC, HSIMamba) address memory and inference constraints, enabling bidirectional feature extraction under strict hardware budgets (Chiley et al., 2022, Singh et al., 22 Dec 2025).

Complexity analyses reveal:

• Linear scaling in sequence or band length for bidirectional CNNs and state-space models (Yang et al., 2024, Yang et al., 2024)
• Avoidance of quadratic overhead typical of self-attention models
• O(n) forward and backward passes in BiLSTM-based systems
• O(1) memory per module in fully reversible fusions (Chiley et al., 2022)

Potential constraints include fixed receptive fields in narrow-kernel convolutions, the memory/compute for parallel branches, or limited spatial context in lightweight spatial blocks.

6. Applications, Variants, and Extensions

Bidirectional feature extraction underpins state-of-the-art results in:

• Sign language recognition (Singh et al., 22 Dec 2025)
• Dependency parsing, sequence labeling, and relational triple extraction (Kiperwasser et al., 2016, Chalapathy et al., 2016, Ren et al., 2021)
• Speaker verification via bidirectional multiscale aggregation (Qi et al., 2021)
• Hyperspectral image analysis (Liu et al., 2017, Yang et al., 2024, Yang et al., 2024)
• Neural vocoding (analysis–synthesis pipelines) (Du et al., 2024)
• Aspect-based sentiment analysis using tree-structured bidirectional propagation (Luo et al., 2018)

Variants adapt bidirectionality to graph, tree, multiscale, or hybrid domains. Potential extensions include multiscale/dilated kernels for broader receptive fields, deeper spatial blocks for richer context, or cross-domain transfer to biomedical and physical sensor streams (Yang et al., 2024, Yang et al., 2024).

---

Bidirectional feature extraction constitutes a foundational mechanism for effective, context-rich representation learning in sequential, spatial, and structured data, with modern instantiations emphasizing parallelization, memory efficiency, and empirical robustness across modalities. Its continued evolution drives progress in both accuracy and deployability in contemporary machine learning systems.