---
title: 'BiFPN: Bidirectional Feature Pyramid Network'
url: https://www.emergentmind.com/topics/bi-directional-feature-pyramid-network-bifpn-514d3886-2972-4ea4-b2af-fdaedf6e14e4
type: topic
---

# BiFPN: Bidirectional Feature Pyramid Network

A Bi-Directional Feature Pyramid Network (BiFPN) is a neural architecture for multiscale feature fusion, designed to enhance feature representations for tasks such as object detection and structured prediction. BiFPNs combine bottom-up and top-down pathways, enabling efficient bidirectional information flow across spatial resolutions and semantic hierarchies. Core principles include fast, normalized, learnable weighted fusion and connectivity optimizations that enable improved accuracy and computational efficiency compared to standard Feature Pyramid Networks (FPNs) and similar designs.

## 1. Historical Development and Motivation

The motivation for BiFPN stems from the limitations of conventional one-way FPNs, which perform single-direction top-down fusion to transfer semantic information from deeper (coarser) layers to shallower (finer) ones. This design omits a bottom-up path, failing to propagate fine-grained spatial details back to coarser scales. Early bidirectional approaches, such as the Bidirectional Pyramid in BPN ["Single-Shot Bidirectional Pyramid Networks for High-Quality Object Detection" 1803.08208], introduced explicit bottom-up and top-down passes to enrich all pyramid levels with both semantic and spatial cues. Subsequent work, including EfficientDet ["EfficientDet: Scalable and Efficient Object Detection" 1911.09070], generalized bidirectional fusion, introduced weighted fast-normalized fusion, and iterated the process across multiple stacked blocks. Variants such as the Residual Bi-Fusion FPN ["Residual Bi-Fusion Feature Pyramid Network for Accurate Single-shot Object Detection" 1911.12051], MF-PAM’s BiFPN ["MF-PAM: Accurate Pitch Estimation through Periodicity Analysis and Multi-level Feature Fusion" 2306.09640], and the fully reversible RevBiFPN ["RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network" 2206.14098] further diversified the architectural landscape.

## 2. Core Architectural Principles

BiFPNs operate over a set of multiscale feature maps produced by a CNN backbone, typically denoted as $\{P_i\}$. They organize feature flow into interleaved top-down and bottom-up fusion passes, connecting all relevant pyramid levels bidirectionally. The central components are:

- **Top-down pathway:** Propagates high-level semantics from deeper/coarser feature maps towards higher-resolution layers via upsampling and fusion.
- **Bottom-up pathway:** Routes fine spatial details from shallow/finer maps to deeper ones via downsampling and fusion.
- **Bidirectional cross-scale connectivity:** Each level is fused using information from adjacent (and sometimes skip-connected or lateral) nodes in both directions.
- **Learnable fast-normalized weighted fusion:** Feature fusion is parametrized by per-input positive scalars and normalized with a small $\epsilon$ to ensure numerical stability:

  $$
  \tilde{P}_i = \frac{\sum_{j \in \mathcal{I}(i)} w_j X_j}{\sum_{j \in \mathcal{I}(i)} w_j + \epsilon}
  $$

  where $w_j$ is a learnable scalar (enforced non-negative via ReLU), and $\mathcal{I}(i)$ indexes the inputs to fusion node $i$ [1911.09070, 2306.09640].

- **Depthwise-separable convolutions and block repeats:** Fusion outputs are processed by depthwise-separable convolutions (often 3×3, sometimes kernel=5 in variants) and activation functions (e.g., Swish/SiLU). The BiFPN module may be stacked multiple times to deepen fusion.

## 3. Mathematical Formulation and Fusion Strategies

Fusion operations are explicitly defined:

- **Weighted sum normalization:** Both EfficientDet [1911.09070] and MF-PAM [2306.09640] utilize normalized, non-negative learnable scalar weights to fuse each input node at a fusion point. Weights are initialized to 1.0 and learned via backpropagation, with normalization to avoid degenerate solutions and facilitate stable training.
- **Fusion node computation (EfficientDet):**

  At input feature set $\{X_j\}$ and learned weights $\{w_j\}$:
  $$
  P_i = \frac{\sum_j w_j X_j}{\sum_j w_j + \epsilon}
  $$
- **Fusion node computation (MF-PAM):**
  
  Top-down pass (for $i = 1\ldots4$):
  $$
  P_{i}^{\mathrm{mid}} = \mathrm{DSC}\left(\frac{\alpha_{i} P_{i}^{\mathrm{in}} + \beta_{i} P_{i+1}^{\mathrm{in}}}{\alpha_{i} + \beta_{i} + \epsilon}\right)
  $$
  Bottom-up pass (for $i = 2\ldots5$):

  $$
  P_{i}^{\mathrm{out}} = \mathrm{DSC}\left(\frac{\gamma_{i} P_{i}^{\mathrm{in}} + \delta_{i} P_{i}^{\mathrm{mid}} + \theta_{i} P_{i-1}^{\mathrm{out}}}{\gamma_{i} + \delta_{i} + \theta_{i} + \epsilon}\right)
  $$

  where $\mathrm{DSC}(\cdot)$ is a depthwise-separable convolution with activation [2306.09640].

- **Residual variant:** The Residual Bi-Fusion FPN performs fusion via concatenation followed by 1×1 conv "gating" and residual addition without explicit sigmoid gating or channel-wise masking [1911.12051].

## 4. Implementation Details and Variants

Key implementation parameters and notable BiFPN variants include:

- **EfficientDet BiFPN:** Five pyramid levels ($P_3$–$P_7$), fusion at each level, pruned single-input nodes, repeated BiFPN blocks ($L$ stack depth), all fusion post-processed by 3×3 ConvDW + BatchNorm + Swish. Upsampling mainly by nearest-neighbor, downsampling typically by max pooling or strided depthwise conv [1911.09070].
- **Bidirectional Pyramid (BPN):** Four pyramid stages ($C_3$–$C_6$), with three quality stages: (1) direct use, (2) bottom-up FP (deconv upsampling, lateral sum, 3×3 conv), (3) top-down "reverse FPN" (stride=2 conv downsampling, fusion with previous, lateral input, 3×3 conv), all at 256 channels [1803.08208].
- **MF-PAM BiFPN:** Modified for 1-D feature fusion for pitch estimation. Five levels, pre-size block ensures same temporal stride, single BiFPN block (no repeat), channels = 48, uniform 1-D depthwise-separable conv (kernel=5), no extra skip connections, strictly follows fast-normalized weighted fusion [2306.09640].
- **Residual Bi-Fusion (ReBiF Net):** Employs a three-way fusion (upsampled, native, reorganized) via standard 1×1 conv acting as implicit gating followed by residual aggregation. A final learned 1×1 conv module (“BiFusion Module”) fuses reorg'ed multi-scale information to improve large-object localization and alleviate shift-variance [1911.12051].
- **RevBiFPN:** Stacks invertible "RevSilo" building blocks for bidirectional multi-scale fusion, enabling exact recomputation of feature activations during backpropagation and reducing memory requirements from $O(D)$ to $O(1)$ in network depth, with minimal empirical accuracy loss [2206.14098].

| Variant         | Input Pyramid | Fusion Strategy        | Key Params               | Unique Aspects            |
|-----------------|--------------|-----------------------|--------------------------|---------------------------|
| EfficientDet    | 5 levels     | Weighted, normalized  | Channels 64–160, repeats | Pruned nodes, skip edges  |
| BPN             | 4 levels     | Sum + convolution     | 3×3 conv, 256 channels   | 3 quality stages, deconv  |
| MF-PAM          | 5 levels     | Fast-normalized, 1D   | Channels 48, kernel=5    | Audio, pre-sizing block   |
| ReBiF Net       | 3–6 levels   | Residual, concat+conv | 1×1 and 3×3 conv         | Reorg, residual fusion    |
| RevBiFPN        | N-levels     | Reversible coupling   | MBConv/F blocks          | $O(1)$ activation memory  |

## 5. Empirical Impact and Ablation Results

Empirical studies across multiple domains confirm the performance and efficiency benefits of BiFPN-based fusion:

- **Object detection (EfficientDet):** Replacing FPN with BiFPN on EfficientNet-B3 improves mAP on COCO from 40.3 to 44.4 (single scale/val), with parameter count dropping from 21 M to 12 M and FLOPs from 75B to 24B. Weighted fusion confers an additional +0.5 AP [1911.09070].
- **High-quality detection (BPN):** The Bidirectional Pyramid in BPN yields substantial gains at higher IoU thresholds (e.g., mAP@0.7), both on VOC and COCO, due to improved localization from bidirectional feature flow [1803.08208].
- **Pitch estimation (MF-PAM):** In extreme noise, inclusion of a light BiFPN improves raw pitch accuracy by ≈4 % versus a single-level LSTM baseline. Full multi-level fusion gains an additional ≈1 %, both with minimal parameter overhead (≤0.4 M) [2306.09640].
- **Residual Bi-Fusion Net:** Introduces consistent gains for small and large objects in detection, with up to ∼7 points of AP increase for small objects and robust improvements even as pyramid depth increases [1911.12051].
- **RevBiFPN:** Achieves nearly 20× reduction in train-time memory (e.g., RevBiFPN-S6: 0.25 GB/sample vs. EfficientNet-B7: 5.05 GB) with similar or improved accuracy (84.2% vs. 84.3% ImageNet top-1), with compute overhead being amortized by larger feasible batch sizes [2206.14098].

## 6. Theoretical Considerations and Computational Efficiency

- **Parameter and compute scaling:** BiFPN architectures, by employing depthwise-separable convolutions and pruning redundant fusion nodes, reduce parameter count and computational FLOPs by up to ∼2× relative to fully connected or standard FPNs, with ablated EfficientDet-B3 results showing a 0.88× parameter ratio and 0.68× FLOPs versus repeated FPN [1911.09070].
- **Convex normalization of fusion weights:** Enforcing $w_j \geq 0$ with normalization ensures that fusion remains a convex linear combination, stabilizing learning dynamics and making fusion coefficients interpretable [1911.09070, 2306.09640].
- **Reversible architectures:** RevBiFPN architectures achieve constant activation memory with only a modest computational penalty (≈2× MACs in worst case for full recomputation), unlocking the ability to train deeper or wider networks under fixed hardware constraints [2206.14098].

## 7. Application Domains and Extensions

BiFPN has been widely applied across domains:

- **Object Detection and Segmentation:** Original context (BPN, EfficientDet, ReBiF Net, RevBiFPN) for improved multiscale detection, especially high-precision and small object localization.
- **Audio/Speech Structured Prediction:** MF-PAM adapts BiFPN for 1-D temporal fusion in robust pitch estimation, demonstrating the architecture’s applicability beyond traditional 2-D raster domains [2306.09640].
- **Reversible and Memory-Efficient Backbones:** RevBiFPN and related reversible pyramid networks extend bidirectional pyramid ideas to ultra-large scale training [2206.14098].

A plausible implication is that any structured prediction or dense prediction task requiring rich cross-scale feature synthesis can benefit from BiFPN principles, whether using full spatial pyramids or specialized streaming architectures.

---

BiFPN constitutes a class of architectures exploiting bidirectional, learnable, normalized multiscale feature fusion, yielding improved trade-offs in accuracy, efficiency, and scalability compared to classic FPN paradigms. Its variants have demonstrated consistent empirical improvements in detection, segmentation, regression, and structured prediction under diverse constraints and modalities.

Source: https://www.emergentmind.com/topics/bi-directional-feature-pyramid-network-bifpn-514d3886-2972-4ea4-b2af-fdaedf6e14e4