---
title: EfficientNet Architecture Family
url: https://www.emergentmind.com/topics/efficientnet-architecture-family
type: topic
---

# EfficientNet Architecture Family

EfficientNet is a family of convolutional neural network (CNN) architectures optimized for classification accuracy, parameter efficiency, and training/inference speed, based on systematic network scaling and neural architecture search (NAS). Originating from work by Tan and Le at Google Research, EfficientNet has two primary generations: EfficientNetV1, which introduced compound scaling and a new baseline discovered via NAS, and EfficientNetV2, which advanced the methodology with training-aware architecture search, new fused block designs, and progressive training methods. Both generations have become standard references for high-performance, resource-efficient ConvNet design in computer vision applications [1905.11946] [2104.00298].

## 1. Neural Architecture Search and EfficientNet-B0 Baseline

The foundation of the EfficientNet family is the EfficientNet-B0 architecture, obtained through neural architecture search on a variant of the MnasNet search space. The search space includes:

- Operator choices: mobile inverted bottleneck convolutions (MBConv), with varying kernel sizes (3×3, 5×5), expansion ratios (1 or 6), optional squeeze-and-excitation (SE) modules, and residual skips.
- Stage-wise variation: each of the eight main stages (following an initial 3×3 Conv stem) can select its operator configuration and number of block repeats.

The NAS process optimizes a multi-objective reward: $\text{ACC}(m) \cdot [\text{FLOPs}(m)/T]^w$ where $T=400$M FLOP target and $w=-0.07$, enforcing a trade-off between accuracy and computational cost. The resulting architecture (EfficientNet-B0) forms the template for subsequent scaling [1905.11946].

The structure of EfficientNet-B0 is shown below:

| Stage | Operator          | Resolution | Channels | Repeats |
|-------|-------------------|------------|----------|---------|
| 1     | Conv3×3           | 224×224    | 32       | 1       |
| 2     | MBConv1 k3        | 112×112    | 16       | 1       |
| 3     | MBConv6 k3        | 112×112    | 24       | 2       |
| 4     | MBConv6 k5        | 56×56      | 40       | 2       |
| 5     | MBConv6 k3        | 28×28      | 80       | 3       |
| 6     | MBConv6 k5        | 14×14      | 112      | 3       |
| 7     | MBConv6 k5        | 14×14      | 192      | 4       |
| 8     | MBConv6 k3        | 7×7        | 320      | 1       |
| 9     | Conv1×1 → Pool→FC | 7×7        | 1280     | 1       |

Key operator details:
- Each MBConv: $1 \times 1$ expansion (BatchNorm + SiLU), $k \times k$ depthwise (BatchNorm + SiLU), $1 \times 1$ projection (BatchNorm), SE module between depthwise and projection, and skip if input/output shapes match.
- All activations use SiLU/Swish: $f(x) = x \cdot \sigma(x)$.

## 2. Compound Model Scaling

EfficientNet introduces "compound scaling," a principled method to simultaneously scale the three architectural dimensions: network depth $d$, width $w$, and input resolution $r$, controlled by a global user-specified coefficient $\phi$:

$$
d = \alpha^\phi, \quad w = \beta^\phi, \quad r = \gamma^\phi
$$

The coefficients $\alpha, \beta, \gamma \geq 1$ are chosen such that $\alpha \cdot \beta^2 \cdot \gamma^2 \approx 2$ (doubling FLOPs for each $\phi$), reflecting the cost structure of Conv layers. For EfficientNet, grid search yields $\alpha=1.2$, $\beta=1.1$, $\gamma=1.15$. Integral rounding is applied to block counts and resolution.

This compound method outperforms traditional strategies that scale only a single axis, avoiding diminishing returns caused by excessive depth, width, or resolution alone. Empirically, uniform compound scaling delivers substantial accuracy improvements for fixed computation budgets [1905.11946].

## 3. EfficientNet-V1 and V2 Families: Model Variants and Scaling Progression

The original EfficientNet-V1 family comprises B0 to B7, with each successive model scaled according to the compound coefficient $\phi$:

| Model | $\phi$ | Input    | Params | FLOPs | Top-1 Acc |
|-------|--------|----------|--------|-------|-----------|
| B0    | 0      | 224×224  | 5.3M   | 0.39B | 77.1%     |
| B1    | 1      | 240×240  | 7.8M   | 0.70B | 79.1%     |
| B2    | 2      | 260×260  | 9.2M   | 1.0B  | 80.1%     |
| B3    | 3      | 300×300  | 12M    | 1.8B  | 81.6%     |
| B4    | 4      | 380×380  | 19M    | 4.2B  | 82.9%     |
| B5    | 5      | 456×456  | 30M    | 9.9B  | 83.6%     |
| B6    | 6      | 528×528  | 43M    | 19B   | 84.0%     |
| B7    | 7      | 600×600  | 66M    | 37B   | 84.3%     |

Regularization (dropout, stochastic depth, AutoAugment) increases with model scale. At ImageNet scale, B7 achieves 84.3% top-1 accuracy, matching far larger models (e.g., 556M-param GPipe) with an approximately $8\times$ reduction in size and $6\times$ increase in speed [1905.11946].

EfficientNetV2 refines model scaling by capping max resolution at $480\times480$, heuristically increasing block counts in later stages. It further reduces over-parameterization at high input resolutions and improves parameter efficiency by adding layers where useful [2104.00298].

## 4. Architectural Innovations: Fused-MBConv and Operator Search

EfficientNetV2 introduces the Fused-MBConv module. In standard MBConv, a $1\times1$ expansion precedes a $k\times k$ depthwise convolution. Fused-MBConv merges these into a single $k\times k$ regular convolution, then optionally SE/projection. Parameters for Fused-MBConv are $M_{in} \cdot M_{out} \cdot k^2$; for MBConv, the total is $(M_{in} \cdot r) \cdot M_{in} \cdot 1^2$ [expand], $+ (M_{in} \cdot r) \cdot k^2$ [depthwise], $+ (M_{in} \cdot r) \cdot M_{out} \cdot 1^2$ [project].

Fused-MBConv reduces memory-access overhead in early stages (where input dimensions are large, and $M_{in}$, $M_{out}$ are small). EfficientNetV2's architecture search operates in a space containing both MBConv and Fused-MBConv blocks, expansion ratios of $\{1,4,6\}$, kernel sizes $\{3,5\}$, and stagewise combinations. This approach systematically selects block types and placements for training and inference efficiency on modern accelerators [2104.00298].

## 5. Progressive Learning and Adaptive Regularization

EfficientNetV2 introduces progressive training in which input image size increases over $M$ training stages: $S_i = S_0 + (S_e - S_0)i/(M-1)$. Training at smaller sizes early accelerates convergence and reduces computation, while later large images provide high final accuracy. To counteract accuracy drop typical of progressive resizing, EfficientNetV2 linearly interpolates regularization strength (dropout, RandAugment magnitude, mixup $\alpha$) in lockstep with image size:

$\phi^k_i = \phi^k_0 + (\phi^k_e - \phi^k_0)\frac{i}{M-1}$

This "adaptive regularization" significantly improves accuracy during progressive learning, mitigating capacity overfitting in later stages and stabilizing training dynamics [2104.00298].

## 6. Performance, Transfer, and Empirical Findings

EfficientNet architectures show strong performance on both standard and transfer learning tasks:

- On ImageNet, B7 (V1) reaches 84.3% top-1 accuracy at 66M parameters and 37B FLOPs. EfficientNetV2-L achieves 85.7% at 120M parameters and 53B FLOPs.
- EfficientNetV2-M matches B7’s accuracy (84.7%) with much faster training (13h vs 139h on TPUv3), and 2–3$\times$ inference speedup at similar accuracy.
- Transfer learning results are consistently state-of-the-art: on CIFAR-100, EfficientNetV1-B7 and V2-L achieve 91.7% and 92.3%, respectively; Flowers-102 achieves 98.8% with V1-B7, matched by V2-L [1905.11946] [2104.00298].
- With ImageNet21k pretraining, EfficientNetV2-L achieves 86.8% top-1, outperforming ViT-L/16 (85.3%) while requiring fewer parameters (120M vs 304M) and less compute (53B vs 192B FLOPs).

## 7. Impact and Directions

EfficientNet's compound scaling and NAS framework have set new standards for scaling ConvNet architectures and resource-efficient model design for both academia and production. The Fused-MBConv and adaptive progressive training of EfficientNetV2 demonstrate the impact of coupling architecture search with training dynamics and hardware-aware optimization.

A plausible implication is that the EfficientNet family’s principles—systematic multidimensional scaling, block-level heterogeneity, and training-aware search—represent an enduring blueprint for the design of high-accuracy, efficient backbones in computer vision, including beyond classification to detection and segmentation [1905.11946] [2104.00298].

Source: https://www.emergentmind.com/topics/efficientnet-architecture-family