FPG-NAS: Efficient 6DoF Pose Estimation

• FPG-NAS is a FLOPs-aware differentiable NAS framework designed specifically for 6DoF object pose estimation using keypoint-based methods.
• It integrates a task-specific search space, a gated multi-candidate operator selection mechanism, and a FLOPs-regularized objective to manage strict compute budgets.
• Empirical results on datasets like LINEMOD and SPEED+ demonstrate competitive performance and efficiency compared to traditional NAS methods.

Searching arXiv for FPG-NAS and closely related FPGA-aware NAS benchmarks and methods. FPG-NAS, short for FLOPs-Aware Gated Differentiable Neural Architecture Search, is a differentiable neural architecture search framework for efficient 6DoF object pose estimation from a single RGB image. It is formulated for keypoint-based pose estimation, where the network predicts 2D keypoint heatmaps and the resulting keypoints are used by a downstream geometric solver such as PnP. The framework combines a task-specific search space for pose estimation, a differentiable gating mechanism that permits discrete multi-candidate operator selection, and a FLOPs-regularized objective that enforces explicit compute budgets. The reported search space contains approximately $10^{92}$ architectures, and the method is evaluated on LINEMOD and SPEED+ under strict FLOPs constraints (Ousalah et al., 5 Aug 2025).

1. Problem setting and conceptual scope

FPG-NAS targets the estimation of an object’s 3D rotation and 3D translation relative to the camera from a single RGB image. The paper places this problem in robotics, AR, autonomous systems, and spacecraft applications, and emphasizes that practical deployment is constrained not only by pose accuracy but also by computational cost. In the setting considered, the core model is a heatmap-based keypoint detector rather than a direct regression architecture. This matters because the searched network is optimized for heatmap regression, not image classification, and because the search space includes a multi-scale fusion layer motivated specifically by keypoint localization (Ousalah et al., 5 Aug 2025).

The paper identifies two main deficiencies in prior work. First, efficient 6DoF estimators had been obtained mainly through manually designed lightweight networks, compression, quantization, or distillation, rather than through a NAS procedure specialized to the pose-estimation task. Second, conventional differentiable NAS, especially DARTS-style softmax relaxation followed by winner-takes-all argmax discretization, was argued to be restrictive for 6DoF estimation. The stated concern is that pose estimation may benefit from complementary operations across layers and scales, whereas one-op-per-layer discretization can discard operator combinations that preserve spatial precision and geometric cues (Ousalah et al., 5 Aug 2025).

Within that framing, FPG-NAS is positioned as, to the authors’ knowledge, the first differentiable NAS framework specifically designed for 6DoF object pose estimation. Its defining characteristics are a pose-oriented search space, multi-candidate layer selection, and an explicit FLOPs-aware optimization criterion.

2. Search space and architecture parameterization

The searched network follows a fixed macro-architecture with layer-wise search. The overall model contains a stem, 16 searchable backbone layers, 1 searchable fusion layer, and a fixed heatmap prediction head. The stage structure is described with progressively changing channels and resolutions: $16$ channels at $W/2 \times H/2$, $24$ at $W/4 \times H/4$, $40$ at $W/8 \times H/8$, $80$ at $W/16 \times H/16$, $112$ at $16$0, $16$1 at $16$2, and $16$3 at $16$4. The fusion layer consumes features from stages $16$5 and outputs 64 channels, while the fixed head uses a depthwise convolution to predict $16$6 keypoint heatmaps at $16$7 resolution (Ousalah et al., 5 Aug 2025).

The searchable backbone block is inspired by EfficientNet and GhostNet. Each candidate bottleneck block contains a Ghost module with ghost ratio $16$8, a depthwise convolution, batch normalization, SiLU activation, a Squeeze-and-Excitation module, a second Ghost module, and an optional residual connection when the input and output dimensions match. Three hyperparameters define the block family: depthwise kernel size $16$9, expansion rate $W/2 \times H/2$0, and Ghost module kernel size $W/2 \times H/2$1. This yields

$W/2 \times H/2$2

candidate blocks per searchable backbone layer, where the additional candidate is a skip connection (Ousalah et al., 5 Aug 2025).

The fusion layer is itself searchable and has 4 candidates: SimpleFusion, DilatedFusion, SEFusion, and AttentionFusion. This is a central task-specific design choice. In standard pose pipelines, cross-scale fusion is often hard-coded through deconvolution, concatenation, or FPN-like modules; FPG-NAS instead includes that decision inside the search space. Because the framework allows multi-candidate selection in each of the 16 backbone layers, each layer can activate any non-empty subset of the 19 candidates, producing

$W/2 \times H/2$3

possible selections per backbone layer. Including the 4 fusion choices, the total search space is

$W/2 \times H/2$4

This very large combinatorial space is one of the paper’s main motivations for using a differentiable supernet formulation rather than black-box enumeration (Ousalah et al., 5 Aug 2025).

3. Gated differentiable search mechanism

FPG-NAS follows the standard supernet-then-derive-final-architecture pattern of differentiable NAS, but it replaces DARTS-style normalized competition with independent differentiable gates. Each candidate operation in each searchable backbone layer has an architecture parameter $W/2 \times H/2$5, and its gate is defined as

$W/2 \times H/2$6

The paper describes this as a differentiable polarized gate: if $W/2 \times H/2$7 is driven toward $W/2 \times H/2$8, the gate goes to $W/2 \times H/2$9; if $24$0 remains nonzero, the gate tends toward $24$1. The gate therefore approximates a binary on/off decision while remaining differentiable (Ousalah et al., 5 Aug 2025).

For searchable layer $24$2, the output is

$24$3

where $24$4 is the output of candidate block $24$5 at layer $24$6, and $24$7. This means that a layer output is the sum of all active candidate blocks, each modulated by its gate. The distinction from DARTS is explicit. In DARTS, operators are coupled through a softmax and final discretization often keeps only the top-1 operator; in FPG-NAS, operators are not forced to compete in a simplex, and multiple operators can remain active after search (Ousalah et al., 5 Aug 2025).

The fusion layer is handled differently. Because it has only four candidates and because the FLOPs budget already induces sparsity, the paper uses standard Gumbel-Softmax for fusion selection rather than polarized gates. The initialization and schedule are partially specified: all architecture parameters $24$8 are initialized to $24$9, $W/4 \times H/4$0 initially, and $W/4 \times H/4$1 is gradually decayed during training to improve gradient stability. The exact decay formula is not given (Ousalah et al., 5 Aug 2025).

The search is framed in standard bilevel DNAS form, with architecture parameters optimized on validation loss and network weights optimized on training loss. In implementation, all models use AdamW; architecture weights use a cosine-decay learning rate from $W/4 \times H/4$2 to $W/4 \times H/4$3, while network weights use a fixed learning rate of $W/4 \times H/4$4. Additional training details include AMP with dynamic loss scaling and gradient clipping with max norm $W/4 \times H/4$5 (Ousalah et al., 5 Aug 2025).

4. FLOPs-aware objective and training pipeline

The defining efficiency mechanism in FPG-NAS is its budget-exceed penalty. During search, the training loss combines the task loss with a FLOPs regularization term that is activated only when the architecture exceeds a target compute budget. Architectures within budget are not penalized; architectures above budget incur a linear excess-FLOPs penalty through a ReLU term. The total FLOPs are computed from the current relaxed architecture state by weighting backbone candidate costs with their gate activations, adding the fusion-layer contribution, and including the fixed stem and head cost. The regularization strength $W/4 \times H/4$6 is increased linearly from 0.1 to 1 during training, so early search emphasizes task learning and later search applies stronger pressure toward the budget (Ousalah et al., 5 Aug 2025).

The task loss is dataset-specific. On LINEMOD, the task loss is KL divergence; on SPEED+, it is mean squared error (MSE). Search proceeds under explicit FLOPs budgets of 10 GFLOPs and 5 GFLOPs, producing the reported models Ours$W/4 \times H/4$7 and Ours$W/4 \times H/4$8, respectively. Search duration is 20 epochs on LINEMOD and 30 epochs on SPEED+, after which the discovered architectures are retrained on the full datasets for the same number of epochs. For preprocessing, LINEMOD images are cropped around the target object, and SPEED+ search is performed on a proxy subset consisting of one-third of the training data before final training on the full dataset (Ousalah et al., 5 Aug 2025).

The reported search cost on NVIDIA A100 is about 0.21 GPU-days per class for LINEMOD and about 1.8 GPU-days for SPEED+. The final discrete architecture is derived from the learned gate activations and fusion choices, but the paper does not specify a precise thresholding rule for backbone-gate discretization. It does, however, state that the discovered architectures consistently select multiple blocks per layer, which is presented as a direct consequence of the independent-gating design (Ousalah et al., 5 Aug 2025).

5. Empirical performance and comparative evaluation

On LINEMOD, the 10 GFLOPs regime, Ours$W/4 \times H/4$9, reports 10.03 G FLOPs and 90.69 mean ADD(-S). The comparison group reported in the paper includes YOLO6D at 26.1 G, 55.95, PVNet at 72.7 G, 86.27, HRPose at 15.5 G, 89.21, and ADLP at 17.3 G, 90.40. The paper highlights object-level gains on challenging categories such as Ape, Cam, and Glue. In the 5 GFLOPs regime, Ours$40$0 reports 4.37 G FLOPs and 85.19 mean ADD(-S), compared with ADLP at 4.8 G, 84.85 and Lite-HRPE at 8.9 G, 87.57 (Ousalah et al., 5 Aug 2025).

On SPEED+, the reported 10 GFLOPs model is compared against YOLOv8s-pose and SPNv2 ($40$1). For Synthetic, FPG-NAS reports $40$2, $40$3, and $40$4, improving over SPNv2’s $40$5, $40$6, and $40$7. For Lightbox, it reports $40$8, $40$9, and $W/8 \times H/8$0, again improving over SPNv2’s $W/8 \times H/8$1, $W/8 \times H/8$2, and $W/8 \times H/8$3. For Sunlamp, however, SPNv2 remains better: FPG-NAS reports $W/8 \times H/8$4, $W/8 \times H/8$5, and $W/8 \times H/8$6, versus SPNv2’s $W/8 \times H/8$7, $W/8 \times H/8$8, and $W/8 \times H/8$9 (Ousalah et al., 5 Aug 2025).

The paper also compares FPG-NAS against alternative differentiable NAS methods at roughly 10 GFLOPs. On LINEMOD, DARTS reports 10.20 G, 85.94, DNAL reports 10.35 G, 87.54, and Ours$80$0 reports 10.03 G, 90.69. On SPEED+, the reported pose error values are 0.046 / 0.356 / 0.529 for DARTS on Synthetic/Lightbox/Sunlamp, 0.036 / 0.339 / 0.509 for DNAL, and 0.029 / 0.320 / 0.467 for FPG-NAS. These comparisons are used to support the claim that independent gating provides better architecture exploration than both DARTS softmax mixing and DNAL’s scaled sigmoid weighting (Ousalah et al., 5 Aug 2025).

6. Terminological ambiguities, relation to hardware-aware NAS, and limitations

FPG-NAS should be distinguished from several adjacent acronyms and research directions. It is a FLOPs-aware method for 6DoF pose estimation, not an FPGA-aware NAS framework. FPGA-oriented NAS addresses a different optimization target: deployment-conditioned latency, throughput, or FPGA resource utilization. Examples include FNAS, which inserts a latency estimator and pruning stage before training so that sampled architectures satisfy a required FPGA latency bound (Jiang et al., 2019); HW-NAS-Bench, which packages architecture-level accuracy together with device-specific hardware costs, including estimated FPGA latency and energy on a Xilinx ZC706 through a Vivado HLS flow (Li et al., 2021); Accel-NASBench, which supports zero-cost accelerator-aware search with real end-to-end FPGA throughput and latency measurements on ZCU102 and VCK190 under Xilinx Vitis AI DPU deployment (Ahmad et al., 2024); and SNAC-Pack, which performs multi-objective global search with Optuna and NSGA-II using an overview-trained surrogate for FPGA utilization and latency, followed by QAT, pruning, and hls4ml synthesis (Weitz et al., 15 May 2026).

A separate source of confusion is the acronym FPG itself. In detection literature, Feature Pyramid Grids is abbreviated FPG, but that work is not a NAS method; it is a manually designed deep feature pyramid proposed partly as a regular, search-free alternative to NAS-designed pyramid architectures such as NAS-FPN (Chen et al., 2020). Accordingly, “FPG-NAS” in the pose-estimation paper refers specifically to FLOPs-Aware Gated Differentiable NAS, not to Feature Pyramid Grids and not to FPGA-aware NAS.

The main limitations reported for FPG-NAS are method-specific. The authors explicitly note that multi-candidate selection may activate blocks with overlapping characteristics, introducing redundancy and motivating future block merging strategies. The demonstrated setting is limited to heatmap-based keypoint pipelines rather than direct pose regression or dense correspondence methods. The exact final gate discretization rule is not specified, and performance is not best on Sunlamp within SPEED+ (Ousalah et al., 5 Aug 2025). A plausible implication is that FPG-NAS addresses compute-sensitive architecture search at the level of FLOPs budgets, whereas deployment scenarios dominated by latency, memory traffic, or FPGA resource composition may require the kind of deployment-conditioned objectives developed in hardware-aware NAS.