UAGDet: Uncertainty-Aware Graph Detection

• The paper introduces a framework that estimates per-instance uncertainty and refines only ambiguous object detections using a graph-based approach.
• UAGDet builds on Faster R-CNN with RoITransformer and employs Monte Carlo dropout to differentiate between certain and uncertain proposals.
• Empirical evaluations on DOTA datasets reveal significant mAP gains, showcasing the efficiency and effectiveness of selective graph refinement.

UAGDet, or Uncertainty-Aware Graph network for object DETection, is a framework designed to improve object detection in aerial images by incorporating structured graph-based reasoning focused selectively on high-uncertainty object proposals. By modeling contextual relations between detections and refining only uncertain predictions via message passing on a graph neural network (GNN), UAGDet achieves state-of-the-art results on challenging benchmarks with minimal additional computational overhead (Kim et al., 2022).

1. Motivation and Problem Setting

Aerial imagery presents unique object detection challenges: images typically contain numerous objects spanning a wide range of scales, and many targets are small, rotated, or heavily occluded. Standard convolutional detectors, when faced with such complexity, often struggle with ambiguous or noisy predictions—improvements in difficult cases frequently require fine-grained local context or global scene reasoning. However, naïvely refining the representation of all detected objects can degrade results for already certain detections and is computationally wasteful. UAGDet addresses these issues through two main mechanisms: (i) explicit estimation of per-instance uncertainty and (ii) graph-based feature propagation focused exclusively on uncertain objects.

2. Initial Detection Pipeline and Proposal Representation

UAGDet builds upon a two-stage object detector, specifically Faster R-CNN extended with RoITransformer for accurate handling of rotated bounding boxes. The base detector produces for each input image a set of region proposals via Region Proposal Network (RPN), aligned convolutional features with RoIAlign, and initial classification and regression outputs: $$\{\,prop_i\}_{i=1}^N,\; conv\_feature \;=\; RPN(image), \quad \hat{f}_i^{conv} = RoIAlign(prop_i,conv\_feature),$$

$$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$

where $\widehat{p}_i \in \mathbb{R}^C$ is the class probability vector, and $\widehat{bbox}_i \in \mathbb{R}^5$ encodes the rotated bounding box $(x, y, w, h, \theta)$. The detector is trained with a sum of cross entropy (classification) and SmoothL1 (regression) losses, and non-maximum suppression (soft NMS, low threshold) is applied to yield $N_{nms}$ surviving proposals.

To enable graph construction, pairwise distances between proposals are computed:

• Spatial distance: $d_{ij}^{spa} = \| c_i - c_j \|_2^2$, where $c_i$ is center of $i$.
• Semantic distance: $$d_{ij}^{sem} = \| \hat{f}_i^{conv} - \hat{f}_j^{conv} \|_2^2$$.

3. Uncertainty Estimation and Node Partitioning

For each proposal, UAGDet estimates uncertainty using lightweight Monte Carlo dropout applied solely to the final fully connected classification and regression layers. Specifically, for $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$0 stochastic forward passes, the framework records $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$1; the proposal's uncertainty is then defined as the standard deviation: $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$2 Proposals are sorted by $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$3. The lowest half (most certain) form the source set $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$4; the highest half (most uncertain) form the target set $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$5.

4. Graph Construction and GNN Refinement

A directed bipartite graph $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$6 is constructed, with edges from sources to targets whenever $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$7 or $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$8 is below a specified threshold. This structure encodes the hypothesis that certain contextually similar objects can improve predictions for ambiguous detections.

Each node $$\widehat{p}_i = cls\_head(\hat{f}_i^{conv}), \quad \widehat{bbox}_i = reg\_head(\hat{f}_i^{conv}),$$9 is represented by concatenated features: $\widehat{p}_i \in \mathbb{R}^C$0 with total dimension $\widehat{p}_i \in \mathbb{R}^C$1. Edge weights are defined by normalized inverse spatial distance: $\widehat{p}_i \in \mathbb{R}^C$2 providing scale invariance.

The node features are updated by two layers of graph convolution: $\widehat{p}_i \in \mathbb{R}^C$3

$\widehat{p}_i \in \mathbb{R}^C$4

yielding GNN-refined target features $\widehat{p}_i \in \mathbb{R}^C$5.

5. Selective Representation Update and Uncertainty-Weighted Loss

After GNN propagation, only the uncertain (target) proposals are updated: $\widehat{p}_i \in \mathbb{R}^C$6 For each target, the final class prediction $\widehat{p}_i \in \mathbb{R}^C$7 is obtained. The classification loss is uncertainty-weighted, sharpening the focus on hard examples: $\widehat{p}_i \in \mathbb{R}^C$8 The refinement loss takes the form: $\widehat{p}_i \in \mathbb{R}^C$9 and is combined with the Faster R-CNN base loss: $\widehat{bbox}_i \in \mathbb{R}^5$0

6. Training and Inference Workflow

The UAGDet training and inference processes are summarized algorithmically:

• Training: For each image minibatch, proposals and uncertainties are extracted using MC-Dropout. Proposals are split into sources and targets, a directed source-to-target graph is constructed, and two-layer GCN is applied to the node features. Only targets undergo feature replacement and refinement via the GNN. The total loss is backpropagated.
• Inference: The pipeline proceeds through basic detection, uncertainty estimation, node partition, graph construction, and GCN-based refinement for targets only. The final output aggregates predictions from both source (unchanged) and target (refined) proposals, with a final NMS post-processing step.

7. Empirical Evaluation and Comparative Results

UAGDet was benchmarked on the DOTA dataset of aerial imagery, containing either 2,806 images with 188k instances (v1.0) or 403k instances (v1.5), each annotated with rotated bounding boxes for $\widehat{bbox}_i \in \mathbb{R}^5$115–16 classes. The mean Average Precision (mAP) is reported below for baseline detectors and UAGDet:

Model	DOTA-v1.0 mAP	DOTA-v1.5 mAP
Mask R-CNN	70.7	62.7
GFNet	71.6	—
RoITransformer	73.4	65.5
UAGDet	75.1	68.0

UAGDet achieves a +1.7 mAP gain over RoITransformer on DOTA-v1.0 and +2.5 on DOTA-v1.5, supporting the effectiveness of targeted uncertainty-aware refinement in the presence of complex and ambiguous object arrangements. The selective mechanism preserves or improves certainty for easy cases while allocating resources to the hardest predictions, with only a lightweight computational addition (Kim et al., 2022).

8. Conceptual Significance and Implications

UAGDet demonstrates the utility of uncertainty modeling and graph-based reasoning for object detection in imagery characterized by high object density, variable scales, and complex backgrounds. By limiting representational updates to uncertain nodes and leveraging semantically/spatially local context through a directed graph structure, the framework mitigates the risk of negative interference on confident predictions and computational inefficiency present in blanket GNN integration. A plausible implication is that, within other dense detection contexts, uncertainty-aware selective refinement may yield similar performance and efficiency gains, especially where ambiguous cases are disproportionately challenging.