---
title: 'FPG-NAS: Efficient 6DoF Pose Estimation'
url: https://www.emergentmind.com/topics/fpg-nas
type: topic
---

# FPG-NAS: Efficient 6DoF Pose Estimation

Searching arXiv for FPG-NAS and closely related FPGA-aware NAS benchmarks and methods.
FPG-NAS, short for **FLOPs-Aware Gated Differentiable Neural Architecture Search**, is a differentiable neural architecture search framework for efficient **6DoF object pose estimation** from a single RGB image. It is formulated for **keypoint-based pose estimation**, where the network predicts 2D keypoint heatmaps and the resulting keypoints are used by a downstream geometric solver such as PnP. The framework combines a **task-specific search space** for pose estimation, a **differentiable gating mechanism** that permits **discrete multi-candidate operator selection**, and a **FLOPs-regularized objective** that enforces explicit compute budgets. The reported search space contains approximately \(10^{92}\) architectures, and the method is evaluated on **LINEMOD** and **SPEED+** under strict FLOPs constraints [2508.03618].

## 1. Problem setting and conceptual scope

FPG-NAS targets the estimation of an object’s **3D rotation** and **3D translation** relative to the camera from a single RGB image. The paper places this problem in robotics, AR, autonomous systems, and spacecraft applications, and emphasizes that practical deployment is constrained not only by pose accuracy but also by **computational cost**. In the setting considered, the core model is a **heatmap-based keypoint detector** rather than a direct regression architecture. This matters because the searched network is optimized for **heatmap regression**, not image classification, and because the search space includes a **multi-scale fusion layer** motivated specifically by keypoint localization [2508.03618].

The paper identifies two main deficiencies in prior work. First, efficient 6DoF estimators had been obtained mainly through **manually designed lightweight networks**, compression, quantization, or distillation, rather than through a NAS procedure specialized to the pose-estimation task. Second, conventional differentiable NAS, especially **DARTS-style softmax relaxation** followed by **winner-takes-all argmax discretization**, was argued to be restrictive for 6DoF estimation. The stated concern is that pose estimation may benefit from **complementary operations across layers and scales**, whereas one-op-per-layer discretization can discard operator combinations that preserve spatial precision and geometric cues [2508.03618].

Within that framing, FPG-NAS is positioned as, to the authors’ knowledge, the **first differentiable NAS framework specifically designed for 6DoF object pose estimation**. Its defining characteristics are a pose-oriented search space, multi-candidate layer selection, and an explicit FLOPs-aware optimization criterion.

## 2. Search space and architecture parameterization

The searched network follows a fixed macro-architecture with **layer-wise search**. The overall model contains a **stem**, **16 searchable backbone layers**, **1 searchable fusion layer**, and a fixed **heatmap prediction head**. The stage structure is described with progressively changing channels and resolutions: \(16\) channels at \(W/2 \times H/2\), \(24\) at \(W/4 \times H/4\), \(40\) at \(W/8 \times H/8\), \(80\) at \(W/16 \times H/16\), \(112\) at \(W/16 \times H/16\), \(192\) at \(W/32 \times H/32\), and \(320\) at \(W/32 \times H/32\). The fusion layer consumes features from stages \(S\{1,2,4,6\}\) and outputs 64 channels, while the fixed head uses a depthwise convolution to predict \(K\) keypoint heatmaps at \(W/4 \times H/4\) resolution [2508.03618].

The searchable backbone block is inspired by **EfficientNet** and **GhostNet**. Each candidate bottleneck block contains a **Ghost module** with ghost ratio \(2\), a **depthwise convolution**, **batch normalization**, **SiLU activation**, a **Squeeze-and-Excitation** module, a second **Ghost module**, and an optional residual connection when the input and output dimensions match. Three hyperparameters define the block family: depthwise kernel size \(K \in \{3,5,7\}\), expansion rate \(e \in \{1,3,6\}\), and Ghost module kernel size \(K_{\text{ghost}} \in \{3,5\}\). This yields
\[
3 \times 3 \times 2 + 1 = 19
\]
candidate blocks per searchable backbone layer, where the additional candidate is a skip connection [2508.03618].

The fusion layer is itself searchable and has **4 candidates**: **SimpleFusion**, **DilatedFusion**, **SEFusion**, and **AttentionFusion**. This is a central task-specific design choice. In standard pose pipelines, cross-scale fusion is often hard-coded through deconvolution, concatenation, or FPN-like modules; FPG-NAS instead includes that decision inside the search space. Because the framework allows **multi-candidate selection** in each of the 16 backbone layers, each layer can activate any non-empty subset of the 19 candidates, producing
\[
2^{19}-1
\]
possible selections per backbone layer. Including the 4 fusion choices, the total search space is
\[
4 \times (2^{19} - 1)^{16} \approx 10^{92}.
\]
This very large combinatorial space is one of the paper’s main motivations for using a differentiable supernet formulation rather than black-box enumeration [2508.03618].

## 3. Gated differentiable search mechanism

FPG-NAS follows the standard **supernet-then-derive-final-architecture** pattern of differentiable NAS, but it replaces DARTS-style normalized competition with **independent differentiable gates**. Each candidate operation in each searchable backbone layer has an architecture parameter \(\alpha\), and its gate is defined as
\[
g(\alpha) = \frac{\alpha^2}{\alpha^2 + \epsilon}.
\]
The paper describes this as a **differentiable polarized gate**: if \(\alpha\) is driven toward \(0\), the gate goes to \(0\); if \(\alpha\) remains nonzero, the gate tends toward \(1\). The gate therefore approximates a binary on/off decision while remaining differentiable [2508.03618].

For searchable layer \(l\), the output is
\[
x_{l+1} = \sum^{N}_{i=1} g(\alpha_i) \cdot b_{l,i}(x_l),
\]
where \(b_{l,i}(x_l)\) is the output of candidate block \(i\) at layer \(l\), and \(N=19\). This means that a layer output is the **sum of all active candidate blocks**, each modulated by its gate. The distinction from DARTS is explicit. In DARTS, operators are coupled through a softmax and final discretization often keeps only the top-1 operator; in FPG-NAS, operators are **not forced to compete in a simplex**, and **multiple operators can remain active** after search [2508.03618].

The fusion layer is handled differently. Because it has only four candidates and because the FLOPs budget already induces sparsity, the paper uses **standard Gumbel-Softmax** for fusion selection rather than polarized gates. The initialization and schedule are partially specified: all architecture parameters \(\alpha\) are initialized to \(1\), \(\epsilon = 0.1\) initially, and \(\epsilon\) is **gradually decayed during training** to improve gradient stability. The exact decay formula is not given [2508.03618].

The search is framed in standard bilevel DNAS form, with architecture parameters optimized on validation loss and network weights optimized on training loss. In implementation, all models use **AdamW**; architecture weights use a **cosine-decay learning rate** from \(10^{-2}\) to \(10^{-4}\), while network weights use a fixed learning rate of \(10^{-3}\). Additional training details include **AMP with dynamic loss scaling** and **gradient clipping** with max norm \(1.0\) [2508.03618].

## 4. FLOPs-aware objective and training pipeline

The defining efficiency mechanism in FPG-NAS is its **budget-exceed penalty**. During search, the training loss combines the task loss with a FLOPs regularization term that is activated only when the architecture exceeds a target compute budget. Architectures within budget are not penalized; architectures above budget incur a linear excess-FLOPs penalty through a ReLU term. The total FLOPs are computed from the current relaxed architecture state by weighting backbone candidate costs with their gate activations, adding the fusion-layer contribution, and including the fixed stem and head cost. The regularization strength \(\lambda\) is increased linearly from **0.1 to 1** during training, so early search emphasizes task learning and later search applies stronger pressure toward the budget [2508.03618].

The task loss is dataset-specific. On **LINEMOD**, the task loss is **KL divergence**; on **SPEED+**, it is **mean squared error (MSE)**. Search proceeds under explicit FLOPs budgets of **10 GFLOPs** and **5 GFLOPs**, producing the reported models **Ours\(_A\)** and **Ours\(_B\)**, respectively. Search duration is **20 epochs** on LINEMOD and **30 epochs** on SPEED+, after which the discovered architectures are retrained on the full datasets for the same number of epochs. For preprocessing, LINEMOD images are cropped around the target object, and SPEED+ search is performed on a proxy subset consisting of one-third of the training data before final training on the full dataset [2508.03618].

The reported search cost on NVIDIA A100 is about **0.21 GPU-days per class** for LINEMOD and about **1.8 GPU-days** for SPEED+. The final discrete architecture is derived from the learned gate activations and fusion choices, but the paper does **not** specify a precise thresholding rule for backbone-gate discretization. It does, however, state that the discovered architectures **consistently select multiple blocks per layer**, which is presented as a direct consequence of the independent-gating design [2508.03618].

## 5. Empirical performance and comparative evaluation

On **LINEMOD**, the 10 GFLOPs regime, **Ours\(_A\)**, reports **10.03 G** FLOPs and **90.69** mean ADD(-S). The comparison group reported in the paper includes **YOLO6D** at **26.1 G, 55.95**, **PVNet** at **72.7 G, 86.27**, **HRPose** at **15.5 G, 89.21**, and **ADLP** at **17.3 G, 90.40**. The paper highlights object-level gains on challenging categories such as **Ape**, **Cam**, and **Glue**. In the 5 GFLOPs regime, **Ours\(_B\)** reports **4.37 G** FLOPs and **85.19** mean ADD(-S), compared with **ADLP** at **4.8 G, 84.85** and **Lite-HRPE** at **8.9 G, 87.57** [2508.03618].

On **SPEED+**, the reported 10 GFLOPs model is compared against **YOLOv8s-pose** and **SPNv2 (\(\phi=0\))**. For **Synthetic**, FPG-NAS reports \(E_T = 0.035\), \(E_R = 1.355^\circ\), and \(E_{\text{pose}} = 0.029\), improving over SPNv2’s \(0.050\), \(1.441^\circ\), and \(0.033\). For **Lightbox**, it reports \(E_T = 0.321\), \(E_R = 15.303^\circ\), and \(E_{\text{pose}} = 0.320\), again improving over SPNv2’s \(0.447\), \(16.804^\circ\), and \(0.368\). For **Sunlamp**, however, SPNv2 remains better: FPG-NAS reports \(E_T = 0.416\), \(E_R = 22.823^\circ\), and \(E_{\text{pose}} = 0.467\), versus SPNv2’s \(0.372\), \(19.366^\circ\), and \(0.401\) [2508.03618].

The paper also compares FPG-NAS against alternative differentiable NAS methods at roughly 10 GFLOPs. On LINEMOD, **DARTS** reports **10.20 G, 85.94**, **DNAL** reports **10.35 G, 87.54**, and **Ours\(_A\)** reports **10.03 G, 90.69**. On SPEED+, the reported pose error values are **0.046 / 0.356 / 0.529** for DARTS on Synthetic/Lightbox/Sunlamp, **0.036 / 0.339 / 0.509** for DNAL, and **0.029 / 0.320 / 0.467** for FPG-NAS. These comparisons are used to support the claim that **independent gating** provides better architecture exploration than both **DARTS softmax mixing** and **DNAL’s scaled sigmoid weighting** [2508.03618].

## 6. Terminological ambiguities, relation to hardware-aware NAS, and limitations

FPG-NAS should be distinguished from several adjacent acronyms and research directions. It is a **FLOPs-aware** method for 6DoF pose estimation, not an **FPGA-aware NAS** framework. FPGA-oriented NAS addresses a different optimization target: deployment-conditioned latency, throughput, or FPGA resource utilization. Examples include **FNAS**, which inserts a latency estimator and pruning stage before training so that sampled architectures satisfy a required FPGA latency bound [1901.11211]; **HW-NAS-Bench**, which packages architecture-level accuracy together with device-specific hardware costs, including estimated FPGA latency and energy on a Xilinx ZC706 through a Vivado HLS flow [2103.10584]; **Accel-NASBench**, which supports zero-cost accelerator-aware search with real end-to-end FPGA throughput and latency measurements on **ZCU102** and **VCK190** under **Xilinx Vitis AI DPU** deployment [2404.08005]; and **SNAC-Pack**, which performs multi-objective global search with Optuna and NSGA-II using a synthesis-trained surrogate for FPGA utilization and latency, followed by QAT, pruning, and hls4ml synthesis [2605.16138].

A separate source of confusion is the acronym **FPG** itself. In detection literature, **Feature Pyramid Grids** is abbreviated **FPG**, but that work is **not a NAS method**; it is a manually designed deep feature pyramid proposed partly as a regular, search-free alternative to NAS-designed pyramid architectures such as NAS-FPN [2004.03580]. Accordingly, “FPG-NAS” in the pose-estimation paper refers specifically to **FLOPs-Aware Gated Differentiable NAS**, not to Feature Pyramid Grids and not to FPGA-aware NAS.

The main limitations reported for FPG-NAS are method-specific. The authors explicitly note that multi-candidate selection may activate **blocks with overlapping characteristics**, introducing redundancy and motivating future **block merging strategies**. The demonstrated setting is limited to **heatmap-based keypoint pipelines** rather than direct pose regression or dense correspondence methods. The exact final gate discretization rule is not specified, and performance is **not best on Sunlamp** within SPEED+ [2508.03618]. A plausible implication is that FPG-NAS addresses **compute-sensitive architecture search** at the level of FLOPs budgets, whereas deployment scenarios dominated by latency, memory traffic, or FPGA resource composition may require the kind of deployment-conditioned objectives developed in hardware-aware NAS.

Source: https://www.emergentmind.com/topics/fpg-nas