---
title: 'Pseudo-Boxes: Surrogate Representations'
url: https://www.emergentmind.com/topics/pseudo-boxes
type: topic
---

# Pseudo-Boxes: Surrogate Representations

Searching arXiv for recent and relevant papers on “pseudo boxes” and closely related usages across detection, tracking, 3D perception, geometry, and non-signalling boxes.
Pseudo-boxes are box-like surrogate representations used when exact box annotations, exact geometry, or unique box identity are unavailable, unreliable, or intentionally relaxed. In contemporary arXiv literature, the expression appears most prominently in computer vision, where pseudo-boxes are teacher-generated 2D or 3D bounding boxes on unlabeled data and function as supervisory signals in semi-supervised, weakly supervised, unsupervised, and domain-adaptive detection. Related work in tracking replaces a direct rectangular representation by a learned point set that induces a pseudo bounding box. In combinatorial geometry, an analogous phenomenon appears when one planar shape folds into multiple non-isomorphic boxes. Outside geometric perception, “box” also denotes non-signalling conditional distributions and cryptographic S-boxes, producing a separate but terminologically adjacent lineage [2303.02998][2408.00619][2309.13257][2506.01079][1111.1781][1411.2503].

## 1. Scope and principal meanings

Across the cited literature, pseudo-boxes are not a single standardized object but a family of box-like substitutes. In 2D and 3D perception, they are predicted bounding boxes used as pseudo supervision; in tracking, they are boxes recovered from learned sample points; in combinatorial folding, they are planar structures compatible with multiple box realizations; and in non-signalling theory, a “box” is a probability distribution rather than a geometric region.

| Research area | Box-like object | Representative role |
|---|---|---|
| SSOD / weak supervision | Predicted 2D bounding box | Supervision on unlabeled or weakly labeled images |
| Unsupervised / UDA 3D detection | Pseudo 3D bounding box | Self-training target under noisy LiDAR geometry |
| Single-object tracking | Point-induced pseudo bounding box | Internal object representation |
| Combinatorial geometry | Common unfolding of multiple boxes | Non-unique box realization |
| Non-signalling theory | Conditional probability “box” | Resource-theoretic object |
| Cryptography | S-box from pseudo-random or pseudo-extension design | Nonlinear substitution component |

The dominant computer-vision meaning is especially precise. PCL distinguishes **pseudo-labels**, meaning the full teacher-predicted supervisory signal on unlabeled data, from **pseudo boxes**, meaning specifically the localization component of those pseudo-labels [2303.02998]. This distinction matters because classification noise and localization noise behave differently. The perception literature therefore treats pseudo-boxes not merely as weak labels, but as geometric hypotheses whose errors can corrupt assignment, regression targets, and later rounds of self-training.

## 2. Pseudo-boxes in 2D detection

In semi-supervised object detection, the canonical setting uses a labeled set \(D_s=\{x_i^s,y_i^s\}_{i=1}^{N_s}\), an unlabeled set \(D_u=\{x_i^u\}_{i=1}^{N_u}\), and a mean-teacher pipeline in which the teacher produces pseudo-labels for weakly augmented unlabeled images and the student is trained on both labeled and pseudo-labeled data. PCL writes the overall loss as
\[
\mathcal{L}= \frac{1}{N_s} \sum_{i=1}^{N_s} \mathcal{L}_s(x^s, y^s)+  \frac{1}{N_u} \sum_{i=1}^{N_u} \alpha \mathcal{L}_u(x^u, y^u),
\]
and argues that localization noise arises both in pseudo-box generation and in later learning, where noisy boxes alter proposal assignment and regression targets [2303.02998].

The core empirical objection to naive pseudo-box use is that class confidence is an imperfect proxy for localization quality. PseCo reports that pseudo-box precision is about **81% at IoU = 0.3**, but only **31% at IoU = 0.9**, and explicitly states that classification confidence and localization precision are not strongly correlated [2203.16317]. This makes confidence thresholding alone insufficient: a pseudo box can have the right category and a high score while still being spatially coarse.

Several methods therefore modify either the pseudo boxes themselves or the way they are consumed. PCL introduces **pseudo-label correction** through **multi-round refining** and **multi-vote weighting**. Refinement iteratively feeds initial pseudo boxes \(B^0\) back into the teacher RCNN head,
\[
B^r, S^r = \mathcal{M}(F, B^{r-1}),
\]
and then locally perturbs each refined box with jittered boxes that are rescored and fused by
\[
b_c = \sum_{i=1}^{n_j} \hat{B}_j^i \times \frac{S_j^i}{\sum_{i=1}^{n_j} S_j^i}.
\]
PCL further proposes an unsupervised regression weight
\[
w_i^{reg} = sigmoid\!\left(\frac{1}{V_{IoU}^{\lambda}}\right),
\]
which is negatively correlated with teacher-student IoU. On COCO-standard, it reports **24.36**, **33.28**, and **36.47 mAP** under **1%**, **5%**, and **10%** labeled splits, improving on SoftTeacher by **3.90**, **2.54**, and **2.43 mAP** [2303.02998].

PseCo addresses noisy pseudo boxes differently. Its **Prediction-guided Label Assignment (PLA)** scores proposals by
\[
q = s^{\alpha} \times u^{1-\alpha},
\]
combining classification score and localization quality, and its **Positive-proposal Consistency Voting (PCV)** estimates pseudo-box reliability by
\[
\sigma^{j} = \frac{\sum_{i=1}^{N}u_{i}^{j}}{N},
\]
then uses \(\sigma^j\) as an instance-wise weight in unlabeled regression. This explicitly treats coarse pseudo boxes as potentially useful rather than automatically discardable, and on COCO it reports gains over Soft Teacher of **2.0**, **1.8**, and **2.0** points under **1%**, **5%**, and **10%** labeling ratios [2203.16317].

“Mind the Gap” pushes the same idea into a dedicated refinement module. It trains separate category and box polishing networks on labeled images using synthesized noisy pseudo-label/ground-truth pairs. For pseudo bounding boxes, the network extracts RoI features from the original pseudo box, **four shifted boxes**, and **two enlarged boxes**, concatenates the resulting **7 RoI** features, and predicts the deviation between the pseudo box and the true box. The paper further disentangles polished pseudo categories and polished pseudo boxes for separate classification and regression, allowing more unlabeled objects to enter training [2207.08185].

A different branch of work avoids sparse pseudo-boxes altogether. Dense Teacher argues that pseudo-box pipelines are a lossy intermediate representation because they require score thresholding, NMS, box filtering, label assignment, and hyper-parameter tuning. It replaces sparse pseudo-boxes with **Dense Pseudo-Labels (DPLs)**, namely the teacher’s post-sigmoid dense outputs,
\[
\vec{y}_i \triangleq \vec{p}_i^t,
\]
followed by top-\(k\%\) region selection based on
\[
S_i = \max_{c \in [1, C]} p^t_{i,c}.
\]
This reframes pseudo-boxes as one possible interface, rather than the inevitable one, for unlabeled supervision [2207.02541].

Weak supervision yields yet another interpretation. “Semi-Weakly Supervised Object Detection by Sampling Pseudo Ground-Truth Boxes” generates pseudo GT boxes online from selective-search proposals rather than from a teacher network. Proposal probabilities are maintained by
\[
\theta_k = \frac{ \exp\{ \frac{s_k}{T} \} }{\sum_j \exp \{ \frac{s_j}{T} \} },
\]
and **\(K=5\)** proposals per class per image are sampled as pseudo GT, with IoU-based score propagation
\[
s_{l,c} = (1 - \gamma) s_{l,c} + \gamma \cdot s_{d,c}.
\]
This replaces offline pseudo-label generation by an online sampled latent-variable mechanism [2204.00147].

## 3. Pseudo 3D boxes in unsupervised and domain-adaptive detection

In unsupervised 3D detection, pseudo-boxes are typically initial 3D bounding boxes obtained from clustering or object discovery, then reused in self-training. UL3D emphasizes that such pseudo 3D boxes are not uniformly wrong: errors can be coordinate-specific in **location / position** \((x,y,z)\), **shape / size**, and **orientation** \(yaw\). It therefore estimates uncertainty at the box-coordinate level by comparing a primary and an auxiliary detector,
\[
U = \left| B_{1} - B_{2} \right|,
\]
with \(U \in \mathbb{R}^{N\times 7}\), and rectifies the dense-head loss by
\[
\mathcal{L}_{i_u}= \frac{\mathcal{L}_{i_o}}{\exp\left(U\right)} + \lambda \cdot U.
\]
This yields coordinate-wise partial trust rather than whole-box acceptance or rejection. The paper reports gains of **+6.9% AP\(_{BEV}\)** and **+2.5% AP\(_{3D}\)** on nuScenes, and **+4.1% AP\(_{BEV}\)** and **+2.0% AP\(_{3D}\)** on Lyft [2408.00619].

A complementary strategy improves pseudo-boxes before training the detector. “Enhancing Pseudo-Boxes via Data-Level LiDAR-Camera Fusion for Unsupervised 3D Object Detection” argues that label-level fusion is limited because it does not repair sparse LiDAR geometry. DFU3D instead uses **SEEM** for instance segmentation and **DepthAnything** for dense depth estimation, performs bi-directional LiDAR-camera fusion, and then filters the resulting pseudo points locally and globally. Its local filtering keeps pseudo points satisfying
\[
F = \left \{ v_j \mid E(r_i,v_j) \leq \lambda \cdot E(r_i, o) \right \} \cup R,
\]
with \(\lambda=0.01\), while global filtering retains points
\[
U = \left \{ f_m \mid E(f_m, f_n) \leq \Phi_m \right \}.
\]
Dynamic self-evolution is triggered when
\[
|t_{e} - t_{e-1}| \leq \psi p e^{-p},
\]
with \(\psi=0.1\). On nuScenes validation, the method reports **28.4 mAP** in class-aware detection [2508.20530].

Cross-dataset 3D UDA introduces a further complication: pseudo boxes can be unreliable because LiDAR beam-count mismatch changes instance point populations. PERE formalizes pseudo-box confidence intervals with \([T_{neg},T_{pos}] = [0.25,0.6]\), discards low-confidence boxes, keeps high-confidence boxes, and treats middle-confidence boxes as unreliable. Its **Complementary Augmentation** chooses between removing all points inside an unreliable box or replacing that region by a transformed high-confidence box of the same class, using
\[
\mathcal{P}_{P}=(u_b-T_{neg})/(T_{pos}-T_{neg}), \qquad \mathcal{P}_R=1-\mathcal{P}_P.
\]
The same framework also adds interpolation and extrapolation proposals and RoI feature alignment losses to reduce beam-count-induced pseudo-label degradation [2404.19384].

Taken together, these 3D methods show three distinct pseudo-box doctrines: **coordinate-wise uncertainty weighting**, **data-level pseudo-box enhancement before detector training**, and **pseudo-label refinery that edits or replaces ambiguous pseudo-box regions**. The shared assumption is that pseudo 3D boxes are necessary but structurally noisy, and that robust learning requires modeling where that noise resides rather than only how confident the detector appears to be.

## 4. Point-induced pseudo-boxes in single-object tracking

RTrack introduces a different notion of pseudo-box. Instead of treating the object as a directly regressed rectangle, it represents the target by a learned point set
\[
\mathcal{S} = \{(x_k, y_k)\}_{k=1}^{n},
\]
with default \(n=9\), and derives a pseudo bounding box from those points. The refinement stage predicts offsets
\[
\mathcal{S}_r = \{(x_k + \Delta x_k, y_k + \Delta y_k)\}_{k=1}^{n},
\]
and the final pseudo-box is recovered by a conversion \(\mathcal{T}\), with the main implementation using a **min-max operation over both axes of the TrackPoints** [2309.13257].

This design is motivated by the claim that a genuine axis-aligned box is a poor internal representation for tracking deformable or rotating targets because it encloses cluttered background and does not explicitly capture arbitrary appearance. RTrack therefore preserves benchmark compatibility—its output is still a box—but moves the representation burden to a deformable point set that can highlight informative local areas before collapsing back to a rectangle.

The optimization strategy is equally important. RTrack uses a **one-to-many leading assignment strategy**: the init stage is center-based and one-to-one, while the refine stage selects top-\(K\) candidate positives by IoU and sets a dynamic threshold as the **sum of the mean and variance of the candidate samples**. This relaxes the extreme sparsity of one-positive-sample training in SOT. The final losses include weighted focal classification, stage-weighted GIoU detection losses with \(\lambda_{init}=1\) and \(\lambda_{refine}=2\), and a mutual-affinity term
\[
\mathcal{L}' = \mathcal{L} + \lambda_{corr}\mathcal{L}_{corr},
\qquad
\mathcal{L}_{corr} = 1 - \rho(\widehat{\mathrm{IoU}}, \widehat{\mathrm{s}}).
\]

Empirically, the paper reports **74.6% mAO** for **RTrack-256** and **76.4% mAO** for **RTrack-384** on GOT-10k, with **2.1M** and **3.6M** training pairs respectively, and states that training cost is reduced to **10%** of previous SOTA trackers’ training costs [2309.13257]. In this literature, a pseudo-box is therefore not a noisy label but an induced box recovered from a more flexible latent representation.

## 5. Non-unique box realizations in combinatorial geometry

A combinatorial analogue of pseudo-box behavior appears in the study of common box unfoldings. “Unfolding Boxes with Local Constraints” examines polyominoes that can be folded into multiple non-isomorphic rectangular boxes. The canonical example is area **22**, where one polyomino folds into both \((1,1,5)\) and \((1,2,3)\). The paper defines \(\mu(k)\) as the smallest area allowing a common unfolding of \(k\) boxes and proves
\[
\mu(3) > 58,
\]
thereby ruling out **46**, **54**, and **58** as the smallest areas allowing a common unfolding of three boxes and refuting a conjecture of Xu et al. (2017) [2506.01079].

The technical contribution is a SAT encoding that replaces global connectivity and acyclicity constraints by local constraints on cut-edge structures, square orientations, and directed edge patterns. The paper begins from four necessary properties of valid cut-edge graphs—connectivity, touching all 8 corners, acyclicity, and a \(2\times2\) local necessity condition—and then enforces local orientation and sink-directed constraints with a finite set of forbidden induced subgraphs. This makes large-scale search feasible: the abstract reports that previous approaches could find common unfoldings of two boxes only up to area **88**, whereas the new method scales **beyond 150**, and that enumeration scales from area **30** to **60** [2506.01079].

Here the pseudo-box phenomenon is not noisy supervision but **non-unique inverse realization**: a single planar object is compatible with several latent boxes. The common unfolding behaves like a box net but does not determine a unique box identity.

A useful geometric contrast is provided by the literature on ordinary axis-parallel boxes. “Empty axis-parallel boxes” studies true Cartesian products of intervals in \([0,1]^d\), proving
\[
m_d(n)\geq \frac{1}{n}\cdot \frac{2d}{e}\bigl(1-4dn^{-1/d}\bigr)
\]
and constructing point sets whose largest empty axis-parallel box has volume at most
\[
8000\,\frac{d^2\log d}{n}.
\]
This paper concerns exact boxes rather than surrogate ones, but it provides the canonical baseline geometry against which more generalized box-like objects can be contrasted [2009.05820].

## 6. Non-geometric box formalisms and cryptographic name overlaps

In non-signalling resource theory, a box is a family of conditional probability distributions rather than a geometric region. The basic object is
\[
P(ab|xy),
\]
or more generally \(P(\mathbf{a},\mathbf{b}|\mathbf{x},\mathbf{y})\), subject to non-signalling constraints. The 2011 no-broadcasting result studies locality-preserving operations \(\Lambda\) and defines anti-Robustness by
\[
\bar R(A) = \max_X \left\{ q \,\middle|\, qA + (1-q)X \in LR_{ns} \right\}.
\]
Its main theorem states that any nonlocal \(2\times2\) box cannot be broadcast by linear operations that map local boxes to local boxes and fully non-signalling boxes to fully non-signalling boxes [1111.1781]. In this setting, “box” is already a formal resource-theoretic noun; the term has no relation to bounding boxes, but it preserves the broader idea of a boxed abstract object manipulated under structural constraints.

Cryptography introduces a different terminological branch through S-boxes. “On the Design of Chaos-Based S-boxes” argues that many chaos-based constructions overemphasize **average coordinate nonlinearity value (ACNV)** and underreport true vectorial nonlinearity relevant to linear cryptanalysis. Starting from pseudo-random bijective S-boxes, its heuristics repeatedly reach **ACNV = 114.0**, and a combined method reaches **ACNV = 114.5**, outperforming previously published chaotic-function-based S-boxes on that comparison metric [2006.10289]. This is a “pseudo-random box” usage rather than a pseudo bounding box usage.

“Generating S-Boxes from Semi-fields Pseudo-extensions” is closer in vocabulary to the present topic because it constructs S-Boxes from a **pseudo-extension** of a semifield. It defines pseudo-irreducibility for
\[
P=X^2+\alpha X+\beta
\]
by the condition
\[
\forall \gamma\in \mathbb{S}_{2^4},\quad [(\alpha-\gamma)\gamma-\beta]\neq 0,
\]
and then a pseudo-inverse map
\[
T' : (\mathbb{S}_{2^4})^2 \to (\mathbb{S}_{2^4})^2.
\]
The paper reports **12,781 non equivalent S-Boxes** and **2,684 APN functions** [1411.2503]. These cryptographic objects are substitution boxes, not geometric boxes, but they show how “pseudo-” and “box” combine in yet another formal lineage.

Across these non-geometric literatures, the shared element is lexical rather than representational. A non-signalling box is a probability table; an S-box is a vectorial Boolean function; a pseudo-extension-based S-Box is a cryptographic design constructed from semifield algebra. The connection to perception pseudo-boxes lies mainly in the reuse of the word “box,” not in a common mathematical ontology.

Pseudo-boxes therefore form a cross-disciplinary but non-unified concept. In 2D and 3D perception they are chiefly surrogate bounding boxes used under annotation scarcity and geometric noise; in tracking they are boxes induced from flexible point sets; in combinatorial geometry they arise as non-unique box realizations; and in non-signalling theory and cryptography they reflect entirely different formal uses of “box.” The strongest common thread is methodological: a pseudo-box is rarely the final object of interest, but rather an intermediate, substitute, or multiply realizable structure that makes learning, inference, or formal analysis possible when a unique, exact, or fully observed box is unavailable.

Source: https://www.emergentmind.com/topics/pseudo-boxes