RFAssigner: Receptive Field Label Assignment

• RFAssigner is a label assignment strategy for dense object detection that improves positive sampling for small objects using a point-based prior augmented with Gaussian receptive field similarity.
• It models both ground-truth boxes and feature locations as Gaussian distributions and employs the Gaussian Combined Distance to effectively measure receptive-field compatibility.
• The method enhances multi-scale learning by selecting additional high-quality positives via statistical and ambiguous-matching rules, leading to improved AP on benchmarks like MS-COCO.

Searching arXiv for the primary paper and a few directly related dense-detection assignment baselines. RFAssigner is a label assignment strategy for dense object detection that is designed to improve multi-scale learning, particularly by addressing the tendency of existing assigners to provide too few positive samples for small objects. It first establishes an initial positive set with a point-based prior, then uses a Gaussian Receptive Field (GRF) distance to identify additional positive samples among previously unassigned locations, with the stated goal of producing a more balanced learning process across object scales. In the paper “RFAssigner: A Generic Label Assignment Strategy for Dense Object Detection” (Guan et al., 3 Jan 2026), it is presented as a generic training-time assignment module rather than a detector architecture, an auxiliary branch, or an inference-time post-processing component.

1. Problem setting and motivation

Dense detectors train on a very large set of candidate samples, such as anchors or feature-map locations, and the label assignment rule determines which of these candidates are treated as positives or negatives. The paper places RFAssigner in the context of the long-standing distinction between anchor-based and anchor-free detection, and argues that many practical differences between detectors are ultimately differences in how positive and negative samples are defined. It emphasizes that scale imbalance is a persistent consequence of standard assignment rules: small objects often attract too few positive samples, whereas larger objects naturally attract more (Guan et al., 3 Jan 2026).

The paper attributes this imbalance to the geometry of existing assigners. In anchor-based settings, small objects may have poor overlap with predefined anchors. In point-based, anchor-free settings, such as FCOS-style assignment, a point typically needs to lie inside a ground-truth box, and small boxes may cover very few feature locations. The paper further argues that recent weighting-based or dynamic assignment methods, including methods such as AutoAssign and DW, do not fully solve this issue because they still begin from candidate sets that may already be too sparse for small objects. This suggests that reweighting alone is insufficient when the underlying positive support is too limited.

RFAssigner is therefore motivated by a specific diagnosis: the training signal for small objects is often weak not only because weights are imperfect, but because the number of appropriate positives is itself insufficient. Its response is to retain a conventional point-based prior and then supplement it with receptive-field-aware positives drawn from the unassigned pool.

2. Gaussian receptive field formulation

The technical core of RFAssigner is the representation of both ground-truth objects and feature-location receptive fields as Gaussian distributions. A ground-truth box is modeled as

$$\boldsymbol{\mu}_{gt} = \begin{bmatrix} x_{gt} \ y_{gt} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{gt} = \begin{bmatrix} \frac{w_{gt}^{2}}{4} & 0 \ 0 & \frac{h_{gt}^{2}}{4} \end{bmatrix},$$

where $(x_{gt}, y_{gt})$ is the box center and $(w_{gt}, h_{gt})$ are its width and height. Each feature point’s receptive field is modeled analogously as

$$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$

The paper notes that, unlike RFLA, it uses the full theoretical receptive field radius in this construction (Guan et al., 3 Jan 2026).

Similarity between the two Gaussians is measured through Gaussian Combined Distance (GCD), which the paper chooses over WD, KLD, and NWD because it is described as both scale-invariant and symmetric. The assignment score used by RFAssigner is the receptive-field similarity

$$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$

High RFD indicates that an unassigned feature location has a receptive field that is well matched to the object in both spatial position and scale. This is the mechanism by which RFAssigner can recover plausible positive samples even when the original point prior is too restrictive.

To make the matching multi-scale, the paper adopts four GRF scales for each feature pyramid level: $1.0\times$, $0.75\times$, $0.50\times$, and $0.25\times$ of the layer’s theoretical receptive field. This suggests that receptive-field compatibility is treated as a family of scale-dependent tests rather than a single fixed match criterion.

3. Assignment mechanism

RFAssigner operates in two stages. First, it uses a standard point-based prior to generate an initial binary positive mask $M_p$. Second, it examines the unassigned candidate locations and evaluates their RFD scores relative to each ground-truth object. The paper describes supplementary positive selection in terms of both score statistics and an ambiguous-matching rule (Guan et al., 3 Jan 2026).

In the statistical description, the method takes the top 9 candidates by RFD score, computes their mean $(x_{gt}, y_{gt})$0 and standard deviation $(x_{gt}, y_{gt})$1, and then selects candidates whose RFD exceeds $(x_{gt}, y_{gt})$2. This is presented as a way to avoid adding low-quality samples. The more explicit ambiguous-matching rule then focuses on candidates whose RFD lies in the interval $(x_{gt}, y_{gt})$3, ranks them by RFD, and forms a supplementary mask $(x_{gt}, y_{gt})$4. The final positive mask is

$(x_{gt}, y_{gt})$5

This preserves all positives from the original point prior and adds only previously unassigned locations.

A central design choice is that ambiguous matching only changes the positive branch. The negative branch continues to follow the center-prior mechanism of DW. This is an important point of interpretation: RFAssigner does not replace the full optimization structure of DW-style assignment, but selectively enlarges the positive set where the paper judges point-based supervision to be insufficient.

The detector loss is written as

$(x_{gt}, y_{gt})$6

with

$(x_{gt}, y_{gt})$7

and

$(x_{gt}, y_{gt})$8

Here the positive set defined by $(x_{gt}, y_{gt})$9 determines which samples enter the positive branch. A plausible implication is that the main novelty lies in the composition of the supervised positive set rather than in a new classification or regression objective.

4. Position relative to prior assignment methods

RFAssigner is framed as neither a purely point-based assigner nor a purely cost-based assigner. Relative to FCOS-style point priors, its novelty is that it preserves the geometric prior but supplements it with receptive-field-aware positives. Relative to methods such as AutoAssign and DW, the paper’s argument is that these methods refine weighting but still inherit a candidate scarcity problem for small objects. Relative to RFLA, the paper’s claim is that RFLA behaves more like anchor-based hard assignment and is less compatible with modern soft assignment frameworks (Guan et al., 3 Jan 2026).

The paper’s empirical comparison supports this framing. On AI-TOD-v2, RFAssigner improves over DW from $(w_{gt}, h_{gt})$0 AP to $(w_{gt}, h_{gt})$1 AP, over AutoAssign from $(w_{gt}, h_{gt})$2 AP to $(w_{gt}, h_{gt})$3 AP, and over DW + RFLA from $(w_{gt}, h_{gt})$4 AP to $(w_{gt}, h_{gt})$5 AP. On MS-COCO-2017, the comparison with DW + RFLA is particularly sharp: DW + RFLA reaches $(w_{gt}, h_{gt})$6 AP, whereas RFAssigner reaches $(w_{gt}, h_{gt})$7 AP. The paper uses this to argue that receptive-field matching can help small objects, but that hard RF-based assignment may degrade broader-scale detection unless integrated more carefully.

Conceptually, RFAssigner differs from methods such as ATSS, PAA, OTA, SimOTA, or TaskAligned-style assignment because it is not formulated as a global cost-optimization or transport problem. Instead, it is best understood as a prior-enhanced assigner: it starts from a point-based prior and uses GRF similarity to repair the scale imbalance that the paper identifies as the key weakness of existing schemes.

A likely misconception is to treat RFAssigner as a detector architecture or as a small-object-specific auxiliary head. The paper’s presentation suggests the opposite: it is a training-time assignment strategy intended to alter supervision while leaving architecture and inference intact.

5. Integration into dense detectors

The implementation reported in the paper uses MMDetection with a ResNet-50 backbone and FPN, pretrained on ImageNet, and trained for 12 epochs under a $(w_{gt}, h_{gt})$8 schedule. Optimization uses SGD with momentum $(w_{gt}, h_{gt})$9, weight decay $$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$0, batch size $$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$1, initial learning rate $$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$2, and learning-rate decay at epochs $$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$3 and $$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$4. Inference uses a score threshold of $$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$5 and NMS IoU threshold $$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$6 (Guan et al., 3 Jan 2026).

The paper emphasizes that RFAssigner is integrated at the label assignment stage, specifically in the loss computation phase. It requires no architectural change, no auxiliary head, and no new inference-time component. It therefore adds no inference overhead. The only reported systems-level cost is a minor increase in training memory consumption proportional to the number of anchor points.

This makes the method “generic” in a narrow but important sense: it can be attached to FCN-based dense detectors without changing the detector head or post-processing. At the same time, the paper states a limitation: current receptive-field calculation methods are tailored to fully convolutional network architectures, so the method is presently limited to FCN-based detectors. This suggests that the genericity claim is architectural rather than universal.

6. Empirical performance, ablations, and limitations

The paper evaluates RFAssigner on three datasets with different scale distributions: AI-TOD-v2, VisDrone-2019, and MS-COCO-2017. AI-TOD-v2 is described as highly tiny-object dominated, with 700,621 instances in 28,036 images across 8 categories and a mean object size of 12.8 pixels. VisDrone-2019 is characterized by dense drone imagery with many small and medium objects. MS-COCO-2017 provides a broad general-purpose object scale distribution (Guan et al., 3 Jan 2026).

The headline results can be summarized briefly:

Dataset	Reported best RFAssigner result
AI-TOD-v2	$$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$7 AP with RFAssigner*
VisDrone-2019	$$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$8 AP
MS-COCO-2017	$$\boldsymbol{\mu}_{tr} = \begin{bmatrix} x_{tr} \ y_{tr} \end{bmatrix}, \qquad \boldsymbol{\Sigma}_{tr} = \begin{bmatrix} \frac{w_{tr}^{2}}{4} & 0 \ 0 & \frac{h_{tr}^{2}}{4} \end{bmatrix}.$$9 AP

On AI-TOD-v2, the strongest result is obtained by RFAssigner*, which uses P2–P6 FPN levels and reaches $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$0 AP, compared with $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$1 AP for DW* + RFLA. On VisDrone-2019, RFAssigner reaches $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$2 AP and the best reported $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$3 of $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$4. On MS-COCO-2017, it reaches $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$5 AP, slightly exceeding DW at $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$6 AP while maintaining or slightly improving $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$7, $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$8, and $$\mathbf{RFD} = \exp\left( -\sqrt{\mathbf{D}_{gc}^{2}\left(\mathcal{N}_{gt}, \mathcal{N}_{tr}\right)} \right).$$9.

The ablation study on receptive-field distance is central to the method’s justification. Using AI-TOD-v2, the paper reports $1.0\times$0 AP for WD, $1.0\times$1 for KLD, $1.0\times$2 for NWD, and $1.0\times$3 for GCD, supporting the choice of GCD as the default metric. Threshold ablations further report that the ambiguous-matching defaults of lower threshold $1.0\times$4 and upper threshold $1.0\times$5 give the best AI-TOD-v2 result. When the upper threshold is relaxed to $1.0\times$6, AP drops by about $1.0\times$7, which the paper interprets as evidence that ambiguous matching is functionally important rather than incidental.

The paper’s main limitation is architectural scope: RF computation is currently tied to FCN-based detectors. It also introduces training-time hyperparameters, including the top-9 selection and ambiguous-matching thresholds. A plausible qualification is that, although the paper claims strong performance “without requiring auxiliary modules or heuristics,” the method still depends on a small but meaningful set of selection rules and thresholds. Even so, within the paper’s framing, these are treated as assignment hyperparameters rather than architectural heuristics.

RFAssigner can therefore be understood as a two-stage label assignment strategy: first, retain a standard point-based positive prior; second, use Gaussian receptive-field similarity to add high-quality ambiguous positives from the unassigned pool. The paper’s central claim is that this directly addresses positive-sample scarcity for small objects while preserving performance on medium and large objects, and its experiments are presented as evidence that the approach generalizes across datasets with very different object-scale distributions (Guan et al., 3 Jan 2026).