Object-Space Error (OSE): Definitions and Applications

• Object-Space Error (OSE) is a family of object-related error measures—not one universal metric—covering geometric mismatch, unknown-object misclassification, pose uncertainty, shape deviation, and reconstruction residuals.
• Applications vary by field: IoU-based localization evaluates detection geometry, A-OSE counts unknown objects labeled as known classes, robotics integrates pose-error distributions with task tolerances, and manufacturing models CAD-surface deviations.
• OSE values require careful interpretation because minimizing an objective does not guarantee downstream success; researchers should pair each measure with complementary outcomes such as AP, unknown recall, task success, or Euclidean reconstruction accuracy.

Object-Space Error (OSE) is an ambiguous term used across several research areas. In object detection, it may refer informally to spatial mismatch between predicted and ground-truth bounding boxes, although this usage is often represented by localization error rather than a named OSE metric. In open-world object detection, OSE commonly denotes the semantic error of assigning an unknown object to a known class, usually reported as Absolute Object-Space Error (A-OSE or AOSE). In robotics, OSE describes a probability distribution over object-frame pose errors evaluated against task-specific tolerances. In initialization-free bundle adjustment, OSE is a depth-scaled algebraic surrogate for image reprojection error. Related uses include spatially structured image error, manufactured-part shape deviation, and errors in object-oriented spatial statistics. These meanings are conceptually related by their association with an object, object representation, or object-defined coordinate space, but they are not interchangeable metrics.

1. Terminology and conceptual scope

OSE does not have a single cross-disciplinary definition. Its interpretation depends on the object representation and the downstream decision being evaluated:

• Geometric support: spatial mismatch between a predicted region and an object’s ground-truth support, such as a bounding box or segmentation region.
• Semantic open-set confusion: an unknown object incorrectly assigned to a known class.
• Pose uncertainty: error in an estimated object frame relative to the true frame.
• Object-space reconstruction residual: an algebraic residual associated with projective cameras and 3D landmarks.
• Manufactured shape deviation: a spatially correlated displacement field over a nominal CAD surface or mesh.
• Spatially structured image discrepancy: a proximity-sensitive residual that emphasizes concentrated image differences.
• Entity-assignment error: discrepancy between estimated and true time-use fractions assigned to GIS objects.

The term therefore does not identify a unique mathematical object. A meaningful OSE definition must specify the object domain, error representation, matching rule, normalization, and evaluation target. For example, a bounding-box IoU threshold, an unknown-object count, a six-dimensional pose distribution, and a projective algebraic residual measure fundamentally different quantities.

A particularly important distinction is between object-space error as a phenomenon and OSE as a reported metric. Several studies use OSE-like concepts without naming or measuring a standalone OSE quantity. The object-detection study in (Borji, 2020), the manufactured-part study in (Babu et al., 2020), the spatial-error study in (Gudi et al., 2022), and the hard-sample pose-estimation study in (Li et al., 2024) are examples. By contrast, open-world detection explicitly reports A-OSE or AOSE, robotics evaluates object-pose error distributions against acceptable task-error spaces, and initialization-free bundle adjustment defines OSE residuals algebraically.

2. OSE in object detection

Bounding-box localization

In conventional object detection, an OSE-like error naturally denotes incorrect spatial support: a predicted box may be shifted, enlarged, reduced, or otherwise have an incorrect position or extent relative to the ground-truth box. The principal operational quantity is intersection-over-union:

$$\operatorname{IoU}(B_p,B_g)= \frac{|B_p\cap B_g|}{|B_p\cup B_g|},$$

where $B_p$ is a predicted box and $B_g$ is a ground-truth box.

The study “Empirical Upper Bound, Error Diagnosis and Invariance Analysis of Modern Object Detectors” (Borji, 2020) does not define or name OSE. Its closest category is localization error, consisting of predictions satisfying

$0.1<\operatorname{IoU}_{\max}<0.5.$

The diagnostic procedure replaces the coordinates of such a prediction with the coordinates of the ground-truth target having maximum IoU, while preserving its confidence and label:

$$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$

The resulting AP increase estimates the performance impact of spatial misalignment under this intervention. It is not a mean displacement, mean boundary error, mean $1-\operatorname{IoU}$, or standalone OSE score. The paper does not decompose localization into center displacement, width error, height error, boundary error, or object-extent error.

Detection AP combines spatial alignment with classification, object discovery, confidence ranking, and duplicate suppression. At an IoU threshold, a detection is a true positive if its target has not already been assigned and its IoU is sufficient; otherwise it is a localization error or duplicate. Consequently, the difference between $AP_{0.5}$, $AP_{0.75}$, and $AP_{0.5:0.95}$ reflects increasingly strict spatial requirements but is not itself an OSE measurement.

The paper reports that classification confusion and misses generally contribute more error than localization and duplicates, although localization is substantial and often exceeds duplicate error. Localization correction raises FCOS from $42.8$ to $B_p$0 AP on COCO and from $B_p$1 to $B_p$2 AP on VOC under the intervention protocol. These gains quantify the effect of localization-like errors, not the fraction of predictions affected by OSE.

Object extent and scale

The nearby-box experiments in (Borji, 2020) sample rectangles around each target subject to an IoU constraint. They probe translation and box extent by asking whether a nearby box can be easier to classify than the exact target box. In most cases, surrounding context reduces classification accuracy; improvements occur only in a few cases, notably for medium and small objects in FASHION and small objects in COCO. The sampling strategy covers translation but not scale variation.

Small objects are particularly difficult. On COCO, the empirical upper-bound AP is $B_p$3 for small objects, $B_p$4 for medium objects, and $B_p$5 for large objects, while the corresponding best-detector values are approximately $B_p$6, $B_p$7, and $B_p$8. These gaps indicate severe small-object difficulty, but they do not isolate geometric localization from classification, objectness, and missed-detection errors.

3. OSE in open-world object detection

In open-world object detection (OWOD), OSE has a more specific semantic meaning. The detector must identify known classes while detecting objects belonging to classes not yet introduced. A central failure occurs when an unknown object is confidently localized but assigned a known-class label. This prevents the object from being passed to an oracle for future annotation and may contaminate incremental learning.

At task $B_p$9, let $B_g$0 denote known classes and $B_g$1 unknown classes. An unknown ground-truth object contributes to A-OSE when it is matched by a sufficiently confident prediction whose label belongs to $B_g$2, typically under an IoU threshold of $B_g$3. Operationally,

$B_g$4

A-OSE is an absolute count, not a percentage or normalized rate. It should be distinguished from U-Recall, which measures unknown objects detected as unknown; Wilderness Impact, which measures precision degradation caused by unknown objects; mAP, which evaluates known-class detection; and ordinary false positives, which need not overlap an actual unknown object.

The OW-RCNN study (Pershouse et al., 2023) evaluates four sequential tasks of 20 classes using combined Pascal VOC and MS-COCO data. On MS-COCO, OW-RCNN reports A-OSE values of $B_g$5, $B_g$6, and $B_g$7 for Tasks 1–3. The method does not dominate every baseline in every task: thresholded Faster R-CNN has lower A-OSE in Task 1. The reported reduction of up to $B_g$8 refers to relative improvement against selected comparison methods, not to a percentage-valued A-OSE.

The detector reduces A-OSE through class-agnostic detection, probability-based unknown detection, prior-task class handling, and representation learning. In particular, Gaussian-mixture modeling over known-class logits converts suspicious known predictions into unknown predictions. A-OSE must nevertheless be interpreted jointly with U-Recall and known-class mAP, since a detector that suppresses most detections can obtain low A-OSE through failure to detect objects.

The study “Wasserstein Distance-based Expansion of Low-Density Latent Regions for Unknown Class Detection” (Mallick et al., 2024) uses the equivalent terminology Absolute Open-Set Error, written $B_g$9. It defines AOSE as the count of unknown-object detections erroneously assigned to known classes. Its proposed OD-CWA method combines the Contrastive Feature Learner, Unknown Probability Learner, Class Wasserstein Anchor, and spectral normalization. On VOC–COCO scenarios with 20, 40, and 60 unknown classes, OD-CWA reduces AOSE relative to the Open-Det baseline from $0.1<\operatorname{IoU}_{\max}<0.5.$0 to $0.1<\operatorname{IoU}_{\max}<0.5.$1, from $0.1<\operatorname{IoU}_{\max}<0.5.$2 to $0.1<\operatorname{IoU}_{\max}<0.5.$3, and from $0.1<\operatorname{IoU}_{\max}<0.5.$4 to $0.1<\operatorname{IoU}_{\max}<0.5.$5, respectively.

AOSE is an output-level detection count. It is not a Wasserstein distance, latent-space overlap, density statistic, AUROC, FPR, or general classification error. Wasserstein distance and density modeling are mechanisms intended to reduce AOSE, not definitions of the metric.

4. OSE as geometric shape and spatial error

Manufactured-part geometry

In manufacturing and design, object-space error concerns deviation of a manufactured part from nominal CAD geometry. “Object shape error modelling and simulation during early design stage by morphing Gaussian Random Fields” (Babu et al., 2020) models this deviation using Morphing Gaussian Random Fields (MGRF).

Let nominal mesh coordinates be $0.1<\operatorname{IoU}_{\max}<0.5.$6, with nominal surface normals $0.1<\operatorname{IoU}_{\max}<0.5.$7. MGRF models a scalar normal-deviation field $0.1<\operatorname{IoU}_{\max}<0.5.$8, yielding the non-ideal geometry

$0.1<\operatorname{IoU}_{\max}<0.5.$9

The field is spatially correlated, rather than composed of independent point perturbations. A Gaussian Random Field is characterized by a mean and covariance function. The covariance scale $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$0 controls variation magnitude, while correlation lengths $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$1 control spatial smoothness. Short correlation lengths generate rapidly varying roughness; long lengths generate smooth, large-scale form variation. Squared-exponential, periodic, and summed kernels represent roughness, waviness, bending, and technological patterns.

MGRF separates systematic prescribed deformation from random correlated form error:

$$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$2

The prescribed component $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$3 represents designer-controlled local or global defects, while $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$4 represents stochastic form variation. Conditional Gaussian-process regression propagates deviations prescribed at selected key points across the mesh. Conditional simulation then generates multiple non-ideal parts with the intended spatial behavior.

The methodology supports global bending, twisting, springback-like deformation, local dents, flange variation, multi-scale form error, and process-specific periodic patterns. It can generate statistically form-tolerance-conforming parts and use current measurements, historical data, FEM data, or manually specified covariance parameters. Its main limitations are the surface-normal displacement assumption, Gaussian finite-dimensional distributions, computational cost of covariance factorization, dependence on key-point selection, and limited modeling of volumetric or non-Gaussian defects.

Spatially structured image error

“Proximally Sensitive Error for Anomaly Detection and Feature Learning” (Gudi et al., 2022) introduces Proximally Sensitive Error (PSE), not OSE. PSE addresses the inability of pixel-wise MSE to represent spatial arrangement. Given residual image $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$5, it convolves the residual with a Gaussian kernel and then averages squared responses:

$$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$6

Nearby residuals reinforce one another, whereas isolated residuals receive less regional reinforcement. PSE can therefore emphasize concentrated anomaly regions such as sunglasses, cracks, or damaged parts relative to spatially dispersed noise.

PSE is only a weak object-space proxy. It uses no segmentation masks, object correspondences, bounding boxes, connected components, object labels, or semantic embeddings. A compact texture artifact can receive a high score, while a semantically meaningful but spatially dispersed change may not. The Gaussian scale $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$7 controls regional emphasis and must be estimated separately for different object categories.

On MVTecAD, PSE achieves $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$8 average precision versus $$B_p\leftarrow B_g^*, \qquad B_g^*=\arg\max_{B_g}\operatorname{IoU}(B_p,B_g).$$9 for MSE; on AR Face, it achieves $1-\operatorname{IoU}$0 versus $1-\operatorname{IoU}$1. These results support spatial concentration as an anomaly signal but do not establish an object-aware OSE metric.

GIS entity-assignment error

In object-oriented spatial statistics, the corresponding error is the absolute error in an entity-specific time-use fraction:

$1-\operatorname{IoU}$2

The entities are GIS polygons and road segments. GPS observations are assigned to the nearest entity, and irregularly sampled observations receive time weights. The estimator is

$1-\operatorname{IoU}$3

The error bound in (Wu et al., 8 May 2026) decomposes into stochastic measurement error, systematic nearest-entity misclassification bias, temporal gaps and boundary crossings, and day-to-day behavioral variability. This is an object-space interpretation in which “object” means a GIS entity rather than a physical object or image region.

5. OSE in object pose estimation and robotics

Pose-error representation

In robotic manipulation, OSE is the error in an estimated object frame relative to the true object frame. “Robotic Task Success Evaluation Under Multi-modal Non-Parametric Object Pose Uncertainty” (Naik et al., 2024) represents OSE as a six-dimensional distribution over translation and rotation:

$1-\operatorname{IoU}$4

If $1-\operatorname{IoU}$5 is the estimated camera-to-object transformation and $1-\operatorname{IoU}$6 is a possible true pose, the object-frame error is

$1-\operatorname{IoU}$7

The pose error is propagated to the tool pose. The relevant quantity is not merely translation or angular distance but the probability that the resulting tool pose remains compatible with the task:

$1-\operatorname{IoU}$8

The estimated error distribution may be multi-modal because of occlusion, symmetry, ambiguous visual features, and viewpoint limitations. The acceptable task-error space may also be multi-modal because robot reachability, collision constraints, contact geometry, and grasp stability can create disconnected success regions.

Acceptable task-error space

The framework discretizes a bounded six-degree-of-freedom error space $1-\operatorname{IoU}$9. For each error hypothesis, an offline simulation evaluates whether the task succeeds. The acceptable region is

$AP_{0.5}$0

At runtime, the estimated distribution is mapped to the discrete error bins and integrated over the precomputed success map:

$AP_{0.5}$1

Thus, OSE acceptability is task-conditioned rather than a universal tolerance. The same pose error may be harmless for base-pose inverse-kinematics availability but catastrophic for grasp stability. The acceptable region depends on object geometry, tool pose, robot kinematics, environment, contact dynamics, and task definition.

In experiments on five YCB objects, the method achieved $AP_{0.5}$2 grasp success with $AP_{0.5}$3 failures over 100 trials, compared with $AP_{0.5}$4 success and $AP_{0.5}$5 failures for the best baseline. For IK availability, it achieved $AP_{0.5}$6 success with no failures over 50 trials. These results concern task success under pose uncertainty, not a universal scalar OSE magnitude.

Pose-space hard-sample synthesis

“Targeted Hard Sample Synthesis Based on Estimated Pose and Occlusion Error for Improved Object Pose Estimation” (Li et al., 2024) models estimator difficulty over a camera-to-object viewsphere and an occlusion space. It primarily uses keypoint error during training, while ADD or other pose-error metrics may be substituted. Evaluation uses ADD-S, the symmetry-aware average distance between model points under predicted and ground-truth poses, normalized by object diameter. A detection is correct when

$AP_{0.5}$7

The method estimates error as a function of viewpoint and visible surface region, then synthesizes targeted pose and occlusion examples. On ROBI, correct detection rates improve from $AP_{0.5}$8 to $AP_{0.5}$9 for DIN and from $AP_{0.75}$0 to $AP_{0.75}$1 for D-Sub. On the T-LESS bin-picking scene, Pose+Occ. reaches $AP_{0.75}$2 ADD-S detection rate and $AP_{0.75}$3 $AP_{0.75}$4, compared with $AP_{0.75}$5 and $AP_{0.75}$6 for the baseline.

The study does not define a separate OSE metric. It uses object-space pose metrics and error distributions to target difficult views and occlusion configurations.

6. OSE in projective reconstruction and bundle adjustment

Algebraic definition

In initialization-free bundle adjustment, OSE has a precise algebraic meaning. “Initialization-Free Bundle Adjustment Revisited: A Controlled Experimental Study” (Weber et al., 18 Aug 2026) considers projective cameras $AP_{0.75}$7, homogeneous landmarks $AP_{0.75}$8, and image observations $AP_{0.75}$9. The perspective projection is

$AP_{0.5:0.95}$0

The OSE residual removes the depth denominator:

$AP_{0.5:0.95}$1

If $AP_{0.5:0.95}$2, then

$AP_{0.5:0.95}$3

OSE is therefore a depth-scaled algebraic surrogate for reprojection error, not reprojection error itself. It is bilinear: linear in landmarks when cameras are fixed and linear in camera parameters when landmarks are fixed.

The principal formulations include pOSE, rOSE, RpOSE, expOSE, and pOSE+rot. They augment the basic residual with affine, depth, radial, exponential, or relative-rotation constraints. The objectives are weighted least-squares problems, with $AP_{0.5:0.95}$4 in the implementation.

Variable projection

The bilinear structure enables Variable Projection. Writing the residual as

$AP_{0.5:0.95}$5

where $AP_{0.5:0.95}$6 denotes nonlinear camera variables and $AP_{0.5:0.95}$7 landmark variables, the landmarks are eliminated by

$AP_{0.5:0.95}$8

The resulting reduced camera-only objective is

$AP_{0.5:0.95}$9

This avoids scene-specific geometric initialization such as relative-pose estimation and triangulation. It does not eliminate all initialization dependence: the camera initialization distribution acts as an implicit prior.

Optimization–reconstruction gap

After OSE minimization, the result is only a projective reconstruction. A metric upgrade seeks a transformation $42.8$0 satisfying

$42.8$1

The upgrade may be unstable because of camera degeneracy, sparse observations, weak landmark connectivity, projective ambiguity, cheirality violations, and ill-conditioned self-calibration.

The central finding of (Weber et al., 18 Aug 2026) is an optimization–reconstruction gap: projective solutions with similarly low OSE values can yield substantially different Euclidean reconstructions after metric upgrade. Low OSE is therefore not a certificate of valid 3D reconstruction. Final reprojection-refinement residual is more predictive of metric-upgrade success than the optimized OSE objective.

Across the evaluated formulations, pOSE+rot achieves the strongest average performance, with rotation AUC@20 of $42.8$2, translation AUC@20 of $42.8$3, and landmark RMSE of $42.8$4 after linear metric upgrade. Unit-circle initialization outperforms normal and uniform initialization on average, with rotation AUC@20 of $42.8$5, translation AUC@20 of $42.8$6, and landmark RMSE of $42.8$7. Cauchy robustification can rescue catastrophic failures but has limited average effect.

The appropriate interpretation is that OSE is an optimization device for constructing projective solutions. Reliable evaluation must continue through reprojection refinement, metric upgrade, cheirality checks, and Euclidean camera and landmark comparison.

7. Distinctions, limitations, and disambiguation

OSE should not be conflated with the following quantities:

• IoU localization error: measures overlap between predicted and ground-truth detection regions; it does not encode open-set semantic confusion or task-conditioned pose tolerance.
• A-OSE or AOSE: counts unknown objects assigned known labels; it is not a geometric distance.
• ADD-S: measures symmetry-aware model-point displacement under pose estimates; it is not an unknown-object rejection metric.
• PSE: aggregates image residuals spatially through Gaussian convolution; it is not explicitly object-aware.
• Manufactured shape deviation: represents a spatial field over nominal geometry; it is not a detector error count.
• Reprojection error: measures image-plane residuals; projective OSE is depth-scaled and algebraic.
• Task success probability: integrates an estimated pose-error distribution over an acceptable task-error space; it is not a raw pose-error magnitude.
• OSE-style subspace embedding: in randomized numerical linear algebra, OSE means oblivious subspace embedding, unrelated to object-space error. It requires simultaneous two-sided norm preservation on a fixed subspace, whereas OSI provides isotropy in expectation and one-sided subspace injectivity (Townsend et al., 11 Apr 2026).

Across these domains, several general principles recur. First, object-relative errors are usually more informative than coordinate-independent parameter errors when object geometry or semantics matter. Second, spatial or object structure must be specified explicitly if the metric is expected to distinguish meaningful object-level differences from arbitrary residuals. Third, absolute counts and raw objectives require normalization or complementary metrics for comparison across datasets and task sizes. Fourth, minimizing an error objective does not necessarily imply downstream validity: low localization loss may coexist with classification failure, low AOSE may arise from missed detections, and low projective OSE may coexist with an unstable metric upgrade. Finally, OSE is most informative when paired with the operational outcome it is intended to predict—detection AP, unknown recall, task success, manufacturing quality, activity-space recovery, or Euclidean reconstruction accuracy.