Functional Object Canonicalization

• Functional object canonicalization is a method that transforms observed objects into normalized forms, reducing pose, scale, and viewpoint variabilities.
• It employs techniques such as normalized coordinate spaces, affine-invariant transformations, and learned canonical poses to improve tasks like reconstruction, segmentation, and manipulation.
• This approach bridges invariant representation and functional alignment in robotics, enabling efficient perception and control using weak, self-, or test-time supervision.

Functional object canonicalization denotes a family of methods that map an observed object, object-conditioned action, or object-related representation into a canonical form in which nuisance variability is reduced and task-relevant structure is preserved. In current vision and robotics literature, the canonical object may be a dense coordinate field such as the Normalized Object Coordinate Space (NOCS), an affine-invariant pointwise representation, a learned canonical pose, a shared functional frame defined by affordances, or an object-centric observation-action frame used by a control policy. Across these formulations, the common objective is to disentangle intrinsic object structure or use from pose, scale, viewpoint, and related confounders so that downstream tasks such as reconstruction, pose estimation, segmentation, retrieval, deformation, anomaly detection, and manipulation can operate on a more stable representation (Sajnani et al., 2020, Di et al., 2023, Xu et al., 23 Sep 2025, Ma et al., 2024).

1. Conceptual scope and problem setting

Functional object canonicalization sits at the intersection of canonical representation learning, equivariant or invariant modeling, and object-centric control. In its geometric form, canonicalization expresses shape in a coordinate system normalized for scale, rotation, and translation, as in category-level canonical reconstruction with NOCS. In its functional form, canonicalization aligns objects by affordance-bearing structure rather than by geometry alone, so that semantically similar action regions such as handles or pouring regions can be represented in shared frames across instances and categories (Sajnani et al., 2020, Xu et al., 23 Sep 2025).

A central theoretical perspective is that canonicalization is a route to invariance or equivariance without necessarily constraining the full downstream architecture. One formulation uses a canonicalization function $h$ and a prediction function $f$ through

$\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$

with equivariance guaranteed if $h$ is itself equivariant. A related perspective formalizes canonicalization as a set-valued map to orbit representatives and shows an equivalence between frames and canonical forms: for every frame there exists an orbit canonicalization, and vice versa, with canonicalization removing redundancies induced by automorphisms (Kaba et al., 2022, Ma et al., 2024).

This scope also explains why the term “functional” has two distinct but compatible uses in the recent literature. In some works it refers to functionality in the sense of object use, affordance, and manipulation; in others it refers to the role canonicalization plays in implementing invariant or equivariant function classes. A plausible implication is that the contemporary field treats canonicalization not as a single algorithmic primitive but as a design pattern for reducing nuisance variability while preserving the degrees of freedom relevant to the task.

2. Canonical representations and mathematical constructions

The dominant object-centric representation in dense pose and reconstruction work is NOCS. In DRACO, NOCS is a unit cube $([0,1]^3)$ in which corresponding parts of all object instances of a category are mapped to semantically consistent 3D locations, enabling dense canonicalization from one or more RGB images under weak supervision. In Object Pose Transformer, canonical coordinates are referenced in $[-0.5, 0.5]^3$, and the model jointly predicts depth, point maps, camera parameters, and normalized object coordinates to support both category-level absolute $SA(3)$ pose and unseen-object relative $SE(3)$ pose (Sajnani et al., 2020, Li et al., 24 Mar 2026).

For point-set and partial-shape settings, canonicalization is often expressed as explicit decoupling of extrinsic factors. ShapeMatcher uses a progressive vector neuron MLP backbone to produce translation-decoupled, rotation-decoupled, and scale-decoupled point-wise features. Its canonicalized point cloud is written as

$S_{tgt}^{c} = s_{tgt}R_{tgt}S_{tgt} + T_{tgt},$

where $R_{tgt}, T_{tgt}, s_{tgt}$ are the predicted pose and scale terms. The stated aim is disentangling inherent structure of the object from pose and size so that segmentation, retrieval, and deformation operate in a shared canonical space (Di et al., 2023).

The broader invariant-learning literature supplies a more abstract formulation. A canonicalization $f$0 is $f$1-invariant if $f$2, and canonical averaging is performed over $f$3 rather than over a potentially much larger frame. The size relations

$f$4

make explicit that canonicalization can be strictly more efficient when automorphism groups are nontrivial. The same paper states that a canonicalization is universal if and only if it is essentially an orbit canonicalization (Ma et al., 2024).

These constructions show that canonical forms need not be limited to rigid pose normalization. They may be dense coordinate fields, affine-invariant latent features, set-valued orbit representatives, or learned group elements. This suggests that “canonical object” is best understood as a task-conditioned representative of an equivalence class induced by nuisance transformations.

3. Supervision regimes and learning pipelines

The learning regime for canonicalization varies from weak supervision to self-supervision to label-free inference-time conditioning. DRACO is a canonical example of weak supervision: it requires camera poses, semantic keypoints, and object segmentation masks during training, but no ground-truth depths or NOCS maps; at inference, only a single RGB image is needed. The architecture uses a ResNet-50 encoder with depth, mask, and NOCS decoder heads. Its dense NOCS supervision is synthesized indirectly: C3DPO learns a canonicalizing transform from backprojected depths and 2D annotations, those transforms are used to lift pixels to 3D and canonicalize them, and the NOCS head is trained with geometric, photometric, perceptual, and smoothness losses (Sajnani et al., 2020).

ShapeMatcher exemplifies self-supervised joint optimization. It trains canonicalization, segmentation, retrieval, and deformation together, with a full branch for complete shapes and a partial branch for occluded inputs. No ground-truth pose, scale, or segmentation labels are used. Canonicalization is regularized by a reconstruction loss and an orthogonality penalty, while partial-full consistency losses enforce that canonicalized partial and full observations remain aligned (Di et al., 2023).

Object Pose Transformer extends canonicalization to unseen objects by conditioning NOCS prediction on an object-centric latent embedding $f$5 rather than on semantic labels at inference time. During training, embeddings from views of the same physical object are brought together through a supervised InfoNCE objective, and the NOCS head is conditioned on $f$6 using FiLM. The resulting model is category-agnostic at test time and unifies absolute and relative pose estimation within a single feed-forward architecture (Li et al., 24 Mar 2026).

The comparative lesson is that dense ground-truth canonical labels are no longer assumed to be mandatory. Weak cross-view cues, photometric consistency, partial-full consistency, and object-centric contrastive learning can all act as substitutes for direct canonical supervision. A plausible implication is that the field increasingly treats canonicalization as a latent structure induction problem rather than as a straightforward regression task.

4. Data alignment, robustness, and test-time canonicalization

A persistent difficulty is that many canonicalizers implicitly rely on aligned training data. “Robust Canonicalization through Bootstrapped Data Re-Alignment” addresses this directly by positing that real-world datasets violate the canonicalization prior. Its proposed bootstrapping algorithm selects a fraction $f$7 of high-loss samples, applies the estimated inverse transformation to realign them, and iteratively contracts pose variance. The update is formalized as

$f$8

with variance evolution

$f$9

Under $\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$0 for $\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$1, the paper gives exponential convergence $\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$2, where $\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$3 (Schmidt et al., 9 Oct 2025).

A complementary line moves canonicalization entirely to test time. “Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring” formulates canonicalization as

$\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$4

where any OOD score can serve as the energy. Across around twenty OOD scores and nine search algorithms, the paper reports that distance-based scores paired with random search and local refinement perform best overall. Because transforming an already aligned input can hurt accuracy, it introduces gated mechanisms that canonicalize only when the OOD score indicates this is needed, and accept the transformed sample only if the score improves (Lindner et al., 23 Jun 2026).

These results clarify two technical issues often obscured in broader discussions. First, canonicalization can fail because training data are not actually canonicalized. Second, canonicalization can fail because the canonicalizer is applied indiscriminately, including to already in-distribution inputs. Both issues are treated as alignment problems, but one is dataset-level and iterative, whereas the other is per-sample and test-time.

5. Functional alignment in robotics and embodied perception

In robotic manipulation, functional object canonicalization is explicitly tied to affordance and action semantics. FUNCanon converts long-horizon manipulation tasks into Actor, Verb, Object chunks and performs “functional object canonicalization for functional alignment and automatic manipulation trajectory transfer, mapping objects into shared functional frames using affordance cues from large vision LLMs.” Candidate affordance regions are selected by a multimodal binary classifier

$\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$5

and the functional region set is

$\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$6

From these regions, the framework computes a functional direction vector

$\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$7

aligns source and target objects by

$\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$8

and transfers trajectories by

$\phi(x) = h'(x)\, f\left(h(x)^{-1} x\right),$9

The intent is to allow action reuse across objects that are functionally similar even when they are not geometrically aligned (Xu et al., 23 Sep 2025).

AnyMug operationalizes a closely related idea for closed-loop mug-handle grasping. Its observation-action canonicalization transforms both the depth observation and the predicted end-effector action into a shared mug-centric frame. The mug center becomes the origin, the handle is rotated to a fixed canonical direction, and the frame is recomputed at every control step from the current pose estimate. Observation canonicalization uses image-plane handle direction

$h$0

alignment rotation

$h$1

and an affine warp $h$2. Action de-canonicalization rotates canonical-frame commands back into the world frame: $h$3 The paper’s emphasis is that the policy sees a consistent mug-centered view and emits actions in a canonical direction regardless of mug placement (Qiu et al., 19 Jun 2026).

PiCo broadens this embodied view by combining Active Physical Canonicalization with Neural Latent Canonicalization for robotic visual anomaly detection. The robot actively reorients the object to reduce geometric uncertainty at its source, seeking

$h$4

where uncertainty is based on reconstruction loss. Residual nuisance is then handled through photometric normalization,

$h$5

feature-level denoising, and contextual reasoning. This shifts canonicalization from passive representation learning to an embodied perception-control loop (Yan et al., 24 Mar 2026).

Taken together, these works make clear that functional canonicalization in robotics is not merely pre-alignment of sensor data. It can also be a control-theoretic interface that converts action primitives, visual inputs, and even robot pose selection into object-centric computations.

6. Empirical landscape, recurring misconceptions, and technical limits

The empirical literature reports consistent gains from canonicalization, but the gains depend strongly on the task, the nuisance group, and whether the canonical form is geometric or functional. Representative reported results include the following (Sajnani et al., 2020, Xu et al., 23 Sep 2025, Qiu et al., 19 Jun 2026, Yan et al., 24 Mar 2026, Di et al., 2023).

System	Canonicalization target	Reported outcome
DRACO	Dense NOCS and object-centric depth	$h$6 mAP at $h$7, versus $h$8 for XNOCS
ShapeMatcher	Affine-invariant canonical shape space	Chamfer improves from $h$9 with no decoupling to $([0,1]^3)$0 with translation, rotation, and scale decoupling
FUNCanon	Shared functional frames for action transfer	RLBench success rates $([0,1]^3)$1, $([0,1]^3)$2, and $([0,1]^3)$3 for Pose, Inst., and Cat.
AnyMug	Shared mug-centric observation-action frame	$([0,1]^3)$4 on unseen upright mugs and $([0,1]^3)$5 on unseen inverted mugs in simulation; $([0,1]^3)$6 zero-shot real success
PiCo	Active physical and latent canonical manifold	$([0,1]^3)$7 O-AUROC in static settings and $([0,1]^3)$8 accuracy in active closed-loop scenarios

Several misconceptions recur in this area. One is that canonicalization is interchangeable with architectural equivariance. The learned-canonicalization literature explicitly presents canonicalization as an alternative that can be plugged into non-equivariant backbones, while the canonicalization perspective on invariant and equivariant learning shows that frame design can often be reduced to canonical form design rather than to symmetry constraints at every layer (Kaba et al., 2022, Ma et al., 2024).

A second misconception is that a single canonical representative always exists and can always be chosen without loss. The canonicalization perspective states that not all objects can have a unique size-$([0,1]^3)$9 canonical form under group constraints, and the zero-shot test-time literature notes that highly symmetric inputs may have non-unique canonical forms. This suggests that ambiguity is intrinsic in some group actions rather than a defect of a specific implementation (Ma et al., 2024, Lindner et al., 23 Jun 2026).

A third misconception is that canonicalization is uniformly beneficial. Several papers qualify this. Robust bootstrapped re-alignment argues that canonicalizers trained under misaligned data can become brittle when the aligned-data prior is false. Test-time canonicalization can reduce accuracy on already aligned inputs unless a gate is used. In robotics, active canonicalization arises precisely because passive perception under adverse viewpoints or illumination may not admit a sufficiently informative canonical representation from a single observation (Schmidt et al., 9 Oct 2025, Lindner et al., 23 Jun 2026, Yan et al., 24 Mar 2026).

The current trajectory of the field therefore points in two directions at once. One direction is toward more explicit object-centric coordinate systems and learned group actions for perception. The other is toward function-aware frames, embodied reorientation, and action-conditioned alignment for control. A plausible implication is that future systems will combine geometric canonical coordinates, affordance-based functional frames, and selective or active canonicalization policies within a single object-centric pipeline.