BEVANet: Bilateral Visual Attention Network
- The paper introduces BEVANet, a real-time segmentation network that combines a bilateral architecture with sparse decomposed large-kernel attention to balance global context with fine detail recovery.
- It employs multi-scale context aggregation using a Deep Large Kernel Pyramid Pooling Module and dynamic kernel selection, achieving high accuracy (up to 81.0% mIoU) on benchmarks like Cityscapes.
- The boundary-guided adaptive fusion module enhances contour delineation by optimally fusing semantic and detail features, demonstrating improved segmentation of fine structures under real-time constraints.
Searching arXiv for BEVANet and closely related bilateral attention segmentation papers. arxiv_search(query="BEVANet Bilateral Efficient Visual Attention Network Real-Time Semantic Segmentation", max_results=5, sort_by="relevance") Bilateral Efficient Visual Attention Network (BEVANet) is a real-time semantic segmentation architecture that addresses the joint requirement for large receptive fields, refined contour recovery, and strict efficiency constraints by combining a bilateral two-branch topology with large-kernel attention, multi-scale context aggregation, and boundary-aware fusion. In its reported Cityscapes configuration, BEVANet operates above the paper’s real-time threshold of $30$ FPS, reaching $32.8$ FPS with mIoU after ImageNet pretraining and $32.9$ FPS with mIoU without pretraining, while a smaller BEVANet-S variant attains $79.4$ FPS and mIoU on CamVid (Huang et al., 10 Aug 2025).
1. Design objective and conceptual framing
BEVANet is formulated for real-time semantic segmentation, where every pixel must be assigned a class label under deployment constraints typical of autonomous driving. The central design problem is presented as a three-way trade-off: semantic understanding requires large receptive fields and long-range dependency modeling, boundary quality requires high-resolution detail preservation, and real-time inference requires moderate FLOPs, parameter count, and latency (Huang et al., 10 Aug 2025).
The architecture is positioned between two familiar limitations. Traditional CNN-based real-time segmenters rely on small kernels such as and , so receptive-field expansion typically depends on deeper stacks, dilation, or pooling, which increases computation and can degrade spatial detail. Vision transformers, by contrast, model long-range dependencies effectively but incur high computational cost because global self-attention is quadratic in spatial resolution. BEVANet responds by using large-kernel attention as an efficient approximation to global contextual modeling, then embedding that mechanism inside a bilateral segmentation network rather than a pure transformer pipeline (Huang et al., 10 Aug 2025).
A crucial interpretive point is that the model does not treat “context” and “detail” as decoder-stage corrections to one another. Instead, it treats them as coequal streams that must interact repeatedly during feature extraction. This suggests that BEVANet’s contribution is not merely a new attention block, but a coordinated design in which receptive-field expansion, branch communication, and fusion are optimized together.
2. Bilateral architecture and information flow
BEVANet is organized as a Bilateral Architecture (BA) with two branches. The high-level semantic branch is heavily downsampled and processes low-resolution feature maps to capture global context. The low-level detail branch maintains higher spatial resolution, fixed at $1/8$ of the input, and is responsible for edges, contours, and other high-frequency structures. For a $32.8$0 Cityscapes image, that detail representation is $32.8$1 (Huang et al., 10 Aug 2025).
The pipeline begins with shared early processing, after which the network splits into the two branches. The semantic branch continues downsampling through stages such as $32.8$2 resolution and processes features with Efficient Visual Attention (EVA) blocks. At the top of the branch, features pass through the Deep Large Kernel Pyramid Pooling Module (DLKPPM), which enriches multi-scale contextual information. The detail branch largely remains at $32.8$3 resolution and receives semantic guidance from the semantic branch through repeated branch communication. Final fused features are upsampled to input resolution and passed through a $32.8$4 classifier to produce the segmentation map (Huang et al., 10 Aug 2025).
This bilateral structure is explicitly described as inspired by PIDNet, but the substitution of conventional blocks with EVA and the replacement of simpler fusion with Boundary Guided Adaptive Fusion (BGAF) materially changes the balance between context capture and boundary delineation. An important empirical caution follows from the ablation study: bilateral topology alone is not sufficient. A BA variant using only convolutional blocks and BAG reaches $32.8$5 FPS but only $32.8$6 mIoU, below the PIDNet baseline at $32.8$7 FPS and $32.8$8 mIoU. The accuracy gains therefore depend on the specific attention, pooling, and fusion mechanisms rather than on branch duplication by itself (Huang et al., 10 Aug 2025).
3. Efficient Visual Attention: EVA, SDLSKA, and CKS
The principal computational primitive in BEVANet is the EVA block. Each EVA block contains Large Kernel Attention (LKA), implemented through Sparse Decomposed Large Separable Kernel Attentions (SDLSKA) and Comprehensive Kernel Selection (CKS), followed by a Convolutional Feed-Forward Network (CFFN). The role of EVA is to enlarge the effective receptive field while remaining compatible with real-time inference (Huang et al., 10 Aug 2025).
SDLSKA replaces a dense large kernel with a sparse decomposition. The module extracts large features using a $32.8$9 convolution, followed by 0 and 1 strip convolutions with dilation rate 2, achieving a receptive field of 3. The decomposition combines strip convolutions with depthwise, pointwise, and dilated convolutions to reduce parameters while preserving large-kernel behavior. Relative to a dense 4 kernel, this substantially lowers cost because the expensive full spatial kernel is replaced by a small 2D kernel and two sparse 1D kernels (Huang et al., 10 Aug 2025).
CKS operates on top of SDLSKA as a dynamic kernel-selection mechanism. It integrates features from multiple kernel sizes and shapes and performs joint channel-wise and spatial-wise adjustment. The paper contrasts this with SKNet- and LSKNet-style strategies that treat channel and spatial dimensions separately. In the reported ablation, SDLSKA with simple addition reaches 5 mIoU; replacing addition with LSKNet-style selection yields 6; and using CKS further improves performance to 7 at 8 FPS, compared with 9 FPS for the LSKNet-style selection. The resulting gain is modest in absolute terms but is achieved with negligible speed loss, which is precisely the regime that matters for real-time segmentation (Huang et al., 10 Aug 2025).
The technical significance of EVA lies in the fact that it is neither a transformer-style global attention operator nor a conventional local convolution block. It is best understood as a large-kernel, sparse, decomposed attention surrogate whose receptive-field behavior is tuned by kernel decomposition and dynamic selection. This makes it central to BEVANet’s performance-efficiency trade-off.
4. Multi-scale context aggregation and boundary-guided fusion
BEVANet complements EVA with two higher-level modules: DLKPPM for semantic context enrichment and BGAF for semantic-detail fusion under boundary guidance. These modules occupy different positions in the architecture and address distinct failure modes (Huang et al., 10 Aug 2025).
DLKPPM is the top-level context module in the semantic branch. It is inspired by DAPPM and retains a hierarchical-residual multi-scale structure, but it integrates large kernel attention and dilated convolutions at each scale. The purpose is to expand receptive fields to approximately $32.9$0, reduce the information loss typically associated with pyramid pooling, and preserve more spatial structure while aggregating multi-scale context. In the ablation study, DAPPM reaches $32.9$1 mIoU at $32.9$2 FPS, PAPPM reaches $32.9$3 at $32.9$4 FPS, and DLKPPM reaches $32.9$5 at $32.9$6 FPS. The module therefore improves mIoU by $32.9$7 over DAPPM at a cost of $32.9$8 FPS (Huang et al., 10 Aug 2025).
BGAF governs interaction between the upsampled semantic features $32.9$9, the detail features 0, and a boundary feature 1. Semantic and detail inputs are first refined by BatchNorm, ReLU, and convolution. Boundary guidance is then converted into a balancing weight through another BatchNorm–ReLU–convolution stack followed by sigmoid. The module forms an adaptive mixture
2
adds a shortcut residual, and applies a final convolution to produce the fused feature. The intended effect is to emphasize detail near boundaries while relying more heavily on semantic context where contour information is less critical (Huang et al., 10 Aug 2025).
The ablation results show that the fusion choice is consequential. BAG yields 3 mIoU at 4 FPS, Light_BAG yields 5 at 6 FPS, and BGAF yields 7 at 8 FPS. BGAF is therefore slower, but it provides the strongest boundary-aware accuracy improvement, including better segmentation of traffic signs, pedestrians, traffic cones, and vegetation in the qualitative examples reported by the paper (Huang et al., 10 Aug 2025).
5. Training protocol and empirical performance
BEVANet is evaluated on Cityscapes and CamVid. Cityscapes contains 9 training, $79.4$0 validation, and $79.4$1 test images at $79.4$2 resolution with $79.4$3 semantic classes. CamVid contains $79.4$4 images with $79.4$5 classes at $79.4$6 resolution. ImageNet pretraining is performed with random cropping to $79.4$7, random horizontal flip, $79.4$8 epochs, batch size $79.4$9, SGD, learning rate 0, weight decay 1, and momentum 2. Segmentation training uses 3 epochs, batch size 4, and base learning rate 5 on Cityscapes, and 6 epochs, batch size 7, and base learning rate 8 on CamVid, together with a poly learning-rate schedule and Online Hard Example Mining (OHEM). Inference is reported on an NVIDIA RTX 3090 using PyTorch 9, CUDA 0, Ubuntu 1, and batch size 2 for FPS measurement (Huang et al., 10 Aug 2025).
| Setting | Speed / complexity | Result |
|---|---|---|
| Cityscapes, ImageNet pretraining | 32.8 FPS, 238.2 GFLOPs, 58.6M params | 81.0% mIoU |
| Cityscapes, no pretraining | 32.9 FPS, 238.2 GFLOPs | 79.27% mIoU |
| CamVid, BEVANet-S | 79.4 FPS, 20.1 GFLOPs | 83.1% mIoU |
On Cityscapes with ImageNet pretraining, BEVANet is compared against DDRNet-23 and PIDNet-M. DDRNet-23 reports 3 FPS, 4 GFLOPs, 5M parameters, and 6 mIoU; PIDNet-M reports 7 FPS, 8 GFLOPs, 9M parameters, and 0 mIoU; BEVANet reports 1 FPS, 2 GFLOPs, 3M parameters, and 4 mIoU. On Cityscapes without pretraining, PIDNet-S reaches 5 mIoU at 6 FPS, PIDNet-M reaches 7 at 8 FPS, PIDNet-L reaches 9 at $1/8$0 FPS, and BEVANet reaches $1/8$1 at $1/8$2 FPS. The reported interpretation is that BEVANet is less dependent on pretraining than competing baselines while still maintaining real-time speed (Huang et al., 10 Aug 2025).
The component-wise ablations attribute the overall gain from the $1/8$3 PIDNet baseline to three specific additions: SDLSKA with CKS contributes about $1/8$4 mIoU, DLKPPM contributes $1/8$5 relative to DAPPM, and BGAF contributes $1/8$6 relative to BAG. This decomposition is important because it shows that the final performance is distributed across receptive-field design, multi-scale context encoding, and boundary-aware fusion rather than concentrated in a single module.
6. Relation to broader bilateral-attention research and stated limitations
BEVANet belongs to a broader family of bilateral or dual-path visual models, but the term “bilateral” is used differently across that literature. In “Deep learning in a bilateral brain with hemispheric specialization,” a bilateral architecture means two specialized hemispheres, one biased toward local features and the other toward global features, whose outputs are combined by a weighted attention-like head (Rajagopalan et al., 2022). In BOAT, “bilateral local attention” means the sequential combination of image-space local attention and feature-space local attention within each transformer block (Yu et al., 2022). In BAFNet, the bilateral principle appears as a low-resolution dependency path and a high-resolution remote-local path for lightweight semantic segmentation in remote sensing, with feature aggregation through channel attention (Wang et al., 2024). This suggests that BEVANet should be understood as a segmentation-specific member of a broader bilateral design tradition: one branch captures large-scale semantic dependency, the other preserves fine spatial structure, and performance depends on how the two are made to interact.
At the same time, the reported results delimit the scope of the contribution. The full BEVANet model uses $1/8$7M parameters and $1/8$8 GFLOPs, which are moderate in the context of Cityscapes real-time segmentation but still substantial for highly constrained edge deployment. The model is validated on urban driving datasets, specifically Cityscapes and CamVid, so cross-domain behavior is not established. The paper also notes potential difficulties for even higher-resolution inputs, such as $1/8$9K imagery, and implies that extremely thin structures and severe occlusions remain challenging even with BGAF (Huang et al., 10 Aug 2025).
The paper’s own extension path is correspondingly pragmatic. It points to lighter large-kernel attention variants, pruning, and quantization for deployment efficiency, and it suggests that the bilateral backbone could be adapted to instance segmentation, object detection, panoptic segmentation, and cross-domain segmentation. These directions are consistent with the wider bilateral-attention literature, but BEVANet’s current empirical identity remains specific: a real-time semantic segmentation network that combines bilateral branch communication, sparse decomposed large-kernel attention, dynamic kernel selection, large-kernel pyramid pooling, and boundary-guided adaptive fusion (Huang et al., 10 Aug 2025).