---
title: VGGNet Architecture and ResSquVGG16 Variant
url: https://www.emergentmind.com/topics/vggnet-architecture
type: topic
---

# VGGNet Architecture and ResSquVGG16 Variant

The VGGNet architecture is a deep convolutional neural network architecture originally developed for visual recognition tasks. VGG-16, a widely adopted instantiation, comprises 13 convolutional layers and 3 fully connected layers, employing a uniform structure of small 3×3 convolution filters and 2×2 max-pooling throughout. Recent advances have sought to address the architecture’s parameter redundancy and training efficiency, leading to variants such as Residual-Squeeze-VGG16 (ResSquVGG16), which integrates parameter-efficient Fire modules adapted from SqueezeNet and residual skip connections to mitigate degradation effects. The ResSquVGG16 model achieves VGG-comparable accuracy on large-scale scene recognition with substantial reductions in model size and training time [1705.03004].

## 1. Architectural Specification of VGG-16

VGG-16, introduced by Simonyan and Zisserman, is defined by a deep design with a fixed scheme of stacking multiple convolutional layers using $3 \times 3$ filters—each stride 1, pad 1—and periodic max-pooling layers of $2 \times 2$, stride 2. The sequence for an input $224 \times 224$ RGB image is as follows:

- **Block 1:**  
  - Conv1\_1: 64 filters, $3 \times 3$  
  - Conv1\_2: 64 filters, $3 \times 3$  
  - MaxPool1: $2 \times 2$, stride 2
- **Block 2:**  
  - Conv2\_1: 128 filters, $3 \times 3$  
  - Conv2\_2: 128 filters, $3 \times 3$  
  - MaxPool2
- **Block 3:**  
  - Conv3\_1, Conv3\_2, Conv3\_3: 256 filters, $3 \times 3$  
  - MaxPool3
- **Block 4:**  
  - Conv4\_1, Conv4\_2, Conv4\_3: 512 filters, $3 \times 3$  
  - MaxPool4
- **Block 5:**  
  - Conv5\_1, Conv5\_2, Conv5\_3: 512 filters, $3 \times 3$  
  - MaxPool5
- **Fully Connected:**  
  - FC6: 4096 units  
  - FC7: 4096 units  
  - FC8: 1000 units (originally for ImageNet)  

The model exhibits a uniform architectural principle, leading to a total parameter count approaching $138$ million ([1705.03004], Table 1).

## 2. Integration of SqueezeNet Fire Modules

Parameter compression in ResSquVGG16 is achieved by replacing VGG-16’s convolutional (and fully-connected) blocks with SqueezeNet Fire modules. The original first convolution is retained but modified to $3 \times 3$, stride 2, 64 filters. All subsequent conv layers (Conv1\_2 through Conv5\_3) and FC layers (FC6–FC8) are replaced by 12 Fire modules and 3 $1 \times 1$ conv layers:

- **Fire Module (per Iandola et al. [10]):**  
  - Squeeze: $1 \times 1$ conv, $s_{1\times1}$ filters  
  - Expand: Parallel $1 \times 1$ conv ($e_{1\times1}$), $3 \times 3$ conv ($e_{3\times3}$, pad 1).

The parameter economy is:

\[
P_{\rm fire} = C\,s_{1\times 1} + s_{1\times 1}\,e_{1\times 1} + 9\,s_{1\times 1}\,e_{3\times 3}
\]

Compared to a single $3 \times 3$ conv layer with $9\,C\,M$ parameters, the Fire module provides a substantial reduction, with the ratio $r$:

\[
r = \frac{C\,s + s\,e_1 + 9\,s\,e_3} {9\,C\,M}
\]

Pooling is inserted after each group of 1–3 Fire modules to mirror VGG’s max-pooling schedule. The three $1 \times 1$ conv layers at the tail end substitute for the fully connected layers, tailored here for 365 scene classes [1705.03004].

## 3. Residual (Shortcut) Connections

To suppress degradation with increased depth, ResSquVGG16 supplements its compressed architecture with shortcut (residual) connections. These are inserted after sequences of two or more consecutive Fire modules without intervening pooling. The residual mapping is:

\[
y_\ell = F\bigl(x_\ell, \{W_\ell\}\bigr) + x_\ell, \qquad F(x_\ell, \{W_\ell\}) = \text{(Fire module stack)}
\]

If channel dimensions are mismatched, a projection via $W_s$ is performed:

\[
y_\ell = F(x_\ell) + W_s x_\ell
\]
with $\mathrm{shape}(W_s x_\ell) = \mathrm{shape}(F(x_\ell))$.

Concretely, four skip connections are instantiated:

| Skip | Source         | Destination | Channels | Projection |
|------|---------------|-------------|----------|------------|
| 1    | Pool1 Output  | Fire3 Out   | 64       | No         |
| 2    | Pool2 Output  | Fire6 Out   | 128      | No         |
| 3    | Pool3 Output  | Fire9 Out   | 256      | No         |
| 4    | Pool4 Output  | Fire12 Out  | 512      | No         |

All are element-wise additions [1705.03004].

## 4. Complete Layer-Wise Layout: ResSquVGG16

The composition of ResSquVGG16 is as follows:

1. Conv1: $3 \times 3$, stride 2, $64$ filters → ReLU  
2. Fire1: $s=16$, $e_1=64$, $e_3=64$ → Scale → ReLU  
3. Pool1: $3 \times 3$, stride 2  
4. Fire2: $s=16$, $e_1=64$, $e_3=64$ → ReLU  
   Fire3: $s=16$, $e_1=64$, $e_3=64$ → ReLU  
5. Pool2: $3 \times 3$, stride 2  
6. Fire4: $s=32$, $e_1=128$, $e_3=128$ → ReLU  
   Fire5: $s=32$, $e_1=128$, $e_3=128$ → ReLU  
   Fire6: $s=32$, $e_1=128$, $e_3=128$ → ReLU  
7. Pool3: $3 \times 3$, stride 2  
8. Fire7: $s=48$, $e_1=192$, $e_3=192$ → ReLU  
   Fire8: $s=48$, $e_1=192$, $e_3=192$ → ReLU  
   Fire9: $s=48$, $e_1=192$, $e_3=192$ → ReLU  
9. Pool4: $3 \times 3$, stride 2  
10. Fire10: $s=64$, $e_1=256$, $e_3=256$ → ReLU  
    Fire11: $s=64$, $e_1=256$, $e_3=256$ → ReLU  
    Fire12: $s=64$, $e_1=256$, $e_3=256$ → ReLU  
11. Pool5: $3 \times 3$, stride 2  
12. Conv6: $1 \times 1$, $4096$ units → ReLU  
    Conv7: $1 \times 1$, $4096$ units → ReLU  
    Conv8: $1 \times 1$, $365$ units → Softmax

Scale layers (Caffe-type BatchNorm replacements) and ReLU are applied after each Fire or conv module [1705.03004].

## 5. Performance and Empirical Comparison

ResSquVGG16 was trained on MIT Places365-Standard (1.8M images, 365 classes) from scratch, using 4 GTX Titan X GPUs (Caffe + DIGITS), over 50 epochs. Metrics:

| Metric                | VGG16 Fine-tuned | ResSquVGG16        |
|-----------------------|------------------|--------------------|
| Training Time         | 3d 16h           | 2d 19h             |
| Model Size            | 10.6 GB          | 1.23 GB            |
| Top-1 Accuracy        | 54.00%           | 51.68%             |
| Top-5 Accuracy        | 84.30%           | 82.04%             |

ResSquVGG16 matches VGG16 within $\sim$2.3 pp in both Top-1 and Top-5 validation accuracy, while reducing training time by approximately $23.9\%$ and storage by $88.4\%$ ([1705.03004], Table 2).

## 6. Implications and Contributions

The integration of Fire modules with residual connections in ResSquVGG16 demonstrates that compression strategies can replace large-weight networks such as VGG-16 while incurring minimal performance loss. This suggests further exploration in deep network design may emphasize parameter efficiency and residual learning, both in supervised training from scratch and transfer scenarios. The architectural modifications retain the internal macro-structure of VGG-16, preserving its depth, while exploiting sub-layer parameter sharing and shortcut propagation for practical gains in computational and memory efficiency [1705.03004].

Source: https://www.emergentmind.com/topics/vggnet-architecture