FunCanon: Canonicalization in Vision & Robotics

• FunCanon is a research label that unifies distinct canonicalization approaches across computer vision, robotics, and humor modeling to simplify downstream tasks.
• In computer vision, it integrates GAN inversion, latent mapping, and swapped StyleGAN modules to perform unified frontalization and stylization from arbitrary poses.
• In robotics, it employs functional canonicalization to align objects into shared frames, enabling transferable manipulation skills with improved efficiency and success rates.

FunCanon is a research label used in arXiv literature for distinct machine-learning systems organized around the idea of canonicalization. In computer vision, it denotes a framework for generating and editing stylized character portraits in a canonical frontal view from a single image without 3D supervision (Kwak et al., 2022). In robotics, it denotes a framework for learning pose-aware manipulation primitives by canonicalizing objects into shared functional frames and training an action-centric diffusion policy on aligned trajectories (Xu et al., 23 Sep 2025). In Chinese humor modeling, the term appears only as a prospective label for a broader canon of humor resources and models rather than as the name of the reported model itself (Yu et al., 26 Mar 2025).

1. Namesake usages and canonicalization regimes

The label has multiple uses across otherwise unrelated research areas. The common element is not a shared implementation, but the use of a canonical representation to simplify downstream generation or control.

Usage	Domain	Canonical target
FunCanon	Portrait synthesis and editing	Frontal face view in StyleGAN latent space
FUNCanon	Robotic manipulation	Shared functional object frame
“FunCanon” in CFunModel	Humor modeling	A hypothetical canon of humor resources and models

In the 2022 vision system, canonicalization means mapping an arbitrary-pose portrait into a frontal latent code while preserving identity and enabling stylization (Kwak et al., 2022). In the 2025 robotics system, canonicalization means aligning objects by affordance-relevant regions such as handles, spouts, and cavities so that trajectories can be transferred across instances and categories (Xu et al., 23 Sep 2025). In the humor-model paper, by contrast, “FunCanon” is not introduced as a trained architecture; it is used as a conceptual label for a multi-task humor ecosystem built around CFunSet and CFunModel (Yu et al., 26 Mar 2025).

2. Canonical-view portrait synthesis: objective and problem setting

The vision version of FunCanon addresses a specific synthesis problem: given a single user-provided portrait, possibly in profile or three-quarter view and possibly already stylized, generate a stylized character in a canonical frontal view while preserving identity and visual quality (Kwak et al., 2022). The target domains explicitly include cartoons, paintings, avatars, and realistic portraits. The paper motivates canonical view as a prerequisite for downstream tasks including 3D modeling and reconstruction, animation and facial motion, metaverse avatars and social-media profile icons, and editing pipelines that assume frontal input.

The difficulty is framed as a compound domain-and-pose problem. Traditional face frontalization methods are trained on real faces and degrade when inputs belong to non-real domains such as cartoons or paintings. Latent-space pose editing methods in StyleGAN, including InterFaceGAN, SeFa, and GANSpace, can move faces toward frontal pose, but supervised variants require semantic labels and unsupervised variants are described as too coarse, with identity drift and inaccurate frontalization. Cascaded pipelines also fail systematically: frontalize-then-stylize amplifies intermediate artifacts and often loses identity, while stylize-then-frontalize applies a real-face frontalizer to stylized images and produces distorted outputs (Kwak et al., 2022).

FunCanon therefore formulates frontalization and stylization as a single latent-space transformation. The inference path is

$I_{\text{out}} = G_s(T(E(I))),$

where $E$ is a GAN inversion encoder, $T$ is a latent mapper from arbitrary pose to canonical frontal pose, and $G_s$ is a swapped StyleGAN that injects target-domain style while preserving low-level pose geometry (Kwak et al., 2022). The stated objective is a unified framework that learns an accurate frontalization mapping in StyleGAN latent space, applies stylization through a swapped StyleGAN, and trains from unlabeled 2D image sets without 3D ground truth.

3. Vision-system architecture and training procedure

The architecture combines a NeRF-based GAN, StyleGAN2, pSp inversion, a latent mapper, and a swapped generator (Kwak et al., 2022). The NeRF-based GAN is pi-GAN, used only during training to synthesize paired views of the same identity: a random view and a canonical frontal view. These pairs provide pseudo ground truth for frontalization without requiring real 3D labels. StyleGAN2 pretrained on FFHQ serves as the main face generator, and pSp maps input images into the StyleGAN $\mathcal{W}^+$ latent space.

The latent space is written as

$$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$

with $L=14$ at $256^2$ resolution and $\mathbf{w}^{(i)} \in \mathbb{R}^{512}$ (Kwak et al., 2022). Early layers encode pose and global structure, while later layers encode local details, texture, and style. This decomposition is operationalized twice. First, the latent mapper $T$ is designed to modify only the first four latent vectors, because pose is concentrated in early layers and because these layers remain unswapped during style interpolation. Second, the swapped generator $E$0 keeps the early low-resolution layers from the original FFHQ StyleGAN and replaces later layers with layers from a transferred StyleGAN trained on a stylized target domain. The specific split reported is $E$1–$E$2 from the original generator and $E$3–$E$4 from the stylized generator (Kwak et al., 2022).

Training proceeds by generating NeRF pose pairs $E$5, inverting them with pSp to obtain $E$6 and $E$7, applying the mapper $E$8, decoding $E$9 and $T$0 with the original StyleGAN, and optimizing latent, reconstruction, and LPIPS losses:

$T$1

$T$2

$T$3

$T$4

with $T$5 and $T$6 (Kwak et al., 2022). The system does not use 3DMMs, pose estimators, landmark-based labels, or manual yaw/pitch annotations. Style transfer to cartoon, MetFaces, and BitmojiFaces domains is obtained by fine-tuning StyleGAN2 with adaptive discriminator augmentation.

4. Portrait results, editability, and limitations

The reported experiments compare FunCanon against two cascaded stylization-frontalization pipelines and against standard frontalization baselines such as CONFIG, Rotate-and-Render, pSp-FF, InterFaceGAN, and SeFa (Kwak et al., 2022). Quantitatively, the main comparison is among frontalize-then-stylize, stylize-then-frontalize, and the unified FunCanon pipeline, using FID for stylized image quality, ArcFace-based identity similarity for identity preservation, and runtime for generating three stylized outputs from one input.

Approach	FID	ID / Runtime
Front. → Style.	95.25	77.10 / 1.84 s
Style. → Front.	86.64	– / 4.63 s
Ours	65.21	82.26 / 0.26 s

These results show the best reported FID, the highest reported identity similarity, and the shortest reported runtime for the unified method (Kwak et al., 2022). The paper attributes the runtime advantage to the absence of heavy 3D fitting and rendering at inference. A common misconception is to treat the method as a 3D-aware inference system because it uses pi-GAN; in fact pi-GAN supplies training pairs only, while final inference is purely 2D through StyleGAN inversion, latent mapping, and swapped generation.

Qualitatively, the paper reports that FunCanon preserves facial shape, features, expression, and global appearance more reliably than CONFIG, Rotate-and-Render, and pSp-FF, especially under large pose changes (Kwak et al., 2022). Relative to InterFaceGAN and SeFa, it avoids explicit pose or landmark supervision while remaining more consistent across pSp and e4e encoders. The method also preserves editability: because the output remains in $T$7, semantic directions from InterFaceGAN, GANSpace, or SeFa can be applied after frontalization. The paper explicitly demonstrates smile, age, and gender edits using

$T$8

The reported limitations are tied to the training scaffold and the data regime. Training requires pi-GAN, which is described as computationally heavy and complex. NeRF artifacts and limited resolution can influence the latent mapping, although this is mitigated by StyleGAN reconstruction. Extreme poses and occlusions are not formally evaluated and are suggested as possible sources of incomplete frontalization, minor artifacts, or identity drift. Stylization also remains domain-specific, requiring separate ADA transfer and generator swapping for each target domain (Kwak et al., 2022).

5. FUNCanon in robotics: action primitives and functional object frames

The robotics framework titled FUNCanon addresses a different canonicalization problem: how to learn general-purpose manipulation skills that are reusable across tasks and object categories, pose-aware, and transferable without new human demonstrations (Xu et al., 23 Sep 2025). Its starting point is the observation that end-to-end policies trained on long-horizon demonstrations tend to be task-specific, instance-specific, hard to scale, and fragile under sim-to-real transfer.

FUNCanon represents manipulation as a sequence of reusable action primitives, each defined by an Actor–Verb–Object triplet,

$T$9

and a long-horizon task as

$G_s$0

Task descriptions are decomposed into these AVO steps using GPT-4o and vision models through

$G_s$1

The reported motivation is that “grasp,” “pour,” and “insert” should be learned as transferable interactions rather than as isolated task-specific trajectories (Xu et al., 23 Sep 2025).

Functional canonicalization is the key mechanism that supports this transfer. Given an object mesh $G_s$2, the system renders RGB-D views, extracts DINOv2 features, lifts them into 3D, and clusters them with KMeans into candidate regions

$G_s$3

A multimodal LLM then classifies whether each region is relevant to an action $G_s$4, a role $G_s$5 (active or passive), and an object category $G_s$6,

$G_s$7

This yields a functional region set

$G_s$8

From these regions, FUNCanon computes a functional direction vector

$G_s$9

and aligns source and target objects by solving

$\mathcal{W}^+$0

Because meshes are normalized and Z-axis aligned, the paper reduces this to rotation around the Z axis. Once objects share a functional frame, sub-trajectories can be transferred by

$\mathcal{W}^+$1

with the paper noting that rotation is also applied in practice (Xu et al., 23 Sep 2025). The resulting frame is intended to make a kettle, pitcher, or watering can comparable at the level of affordances rather than appearance.

6. FuncDiffuser, experiments, and sim-to-real behavior

The policy learner in FUNCanon is FuncDiffuser, an object-centric and action-centric diffusion policy conditioned on canonicalized state (Xu et al., 23 Sep 2025). Its state is

$\mathcal{W}^+$2

where $\mathcal{W}^+$3 is the relative pose feature between actor and object, $\mathcal{W}^+$4 and $\mathcal{W}^+$5 are PointNet++ functional features of actor and object point clouds, and $\mathcal{W}^+$6 is a CLIP verb embedding. The policy is written as $\mathcal{W}^+$7 and trained with standard diffusion denoising MSE. The reported architecture uses a 3-layer MLP pose encoder with a 64-dimensional embedding, PointNet++ for point-cloud encoding, a UNet-style diffusion core, DDIM with 100 training timesteps and 10 inference timesteps, and AdamW with learning rate $\mathcal{W}^+$8, 5000 epochs, and batch size 64.

Experiments span simulation and real-robot deployment. Simulation uses RLBench tasks—Put A in B, Pour A in B, and Water B with A—under pose-level, instance-level, and category-level variation (Xu et al., 23 Sep 2025). Real-world deployment uses a Franka Emika 7-DOF arm, Intel RealSense cameras, FoundationPose for 6D tracking, and TripoSR when CAD models are unavailable. Evaluation uses Success Rate, sub-task Success Rate, 25 episodes with 3 seeds in simulation, and 50 trials in the real world.

Method	Simulation average SR (pose / instance / category)	Real-world SR (Pick & Place / Pick, Pour L1 / Pick, Pour L2)
FunCanon	72.4 / 75.8 / 68.0	88 / 90 / 88
SPOT	61.3 / 62.9 / 60.9	52 / 76 / 60
3DA	34.0 / 33.3 / 32.2	48 / 54 / 62

The paper reports gains of $\mathcal{W}^+$9, $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$0, and $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$1 over 3DA for pose-, instance-, and category-level simulation averages, and roughly $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$2 to $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$3 over SPOT across those settings (Xu et al., 23 Sep 2025). Real-world gains are reported as $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$4 to $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$5 over SPOT depending on the task and at least $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$6 over 3DA, with the largest improvements on pouring tasks where spout and handle alignment are especially important.

Ablation studies reinforce the importance of the canonicalization scheme. On category-level Pick, Pour L2, the full system attains Sub1-SR $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$7, Sub2-SR $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$8, and Overall SR $$\mathbf{w}^+ = [\mathbf{w}^{(1)}, \mathbf{w}^{(2)}, \dots, \mathbf{w}^{(L)}],$$9; removing affordances (“Geom-only”) reduces overall SR to $L=14$0, removing functional frames (“No-Func-Frame”) reduces it to $L=14$1, and replacing chunked primitives with a monolithic trajectory policy reduces it to $L=14$2 (Xu et al., 23 Sep 2025). A common misconception is to interpret FUNCanon as merely another pose-conditioned diffusion policy. The paper instead positions the method around three coupled biases: AVO chunking, VLM-derived affordance recognition, and functional-frame alignment prior to diffusion training.

The reported limitations are also concrete. Real-world failures often stem from 6D pose estimation errors on symmetric objects, low-texture surfaces, occlusions, and RealSense noise. RLBench does not support re-grasping or actions outside predefined behaviors, limiting task diversity. Functional classification depends on VLM quality, and rigid-object canonicalization does not directly solve small handles, deformable objects, or articulated objects (Xu et al., 23 Sep 2025).

7. Comparative perspective and conceptual significance

The two named FunCanon systems are technically unrelated, but they instantiate two sharply defined uses of canonicalization. The portrait model canonicalizes viewpoint within a StyleGAN latent manifold, holding identity fixed while replacing late-layer appearance statistics with a target artistic domain (Kwak et al., 2022). The robotics model canonicalizes functional geometry within object-centric 3D frames, holding affordance relations fixed while transferring actor–object trajectories across instances and categories (Xu et al., 23 Sep 2025).

This suggests a shared methodological role for canonicalization as an inductive bias rather than as a single algorithmic recipe. In the vision system, the canonical variable is frontal pose, and the benefit is simultaneous frontalization, stylization, and latent editability. In the robotics system, the canonical variable is affordance-aligned object orientation, and the benefit is compositional skill reuse, automatic trajectory transfer, and stronger sim-to-real generalization. The name overlap therefore reflects a common abstraction—mapping diverse inputs into a stable frame before learning or generation—while the actual representations, objectives, and benchmarks are distinct.

A further source of confusion is the appearance of the term in humor-model literature. There, “FunCanon” is not the title of the trained model; the reported contributions are CFunSet and CFunModel, and “FunCanon” is used only as a hypothetical ecosystem label for humor resources, tasks, and evaluation practices (Yu et al., 26 Mar 2025). Within arXiv-indexed usage, the term is therefore best understood as a cross-domain naming coincidence centered on canonicalization, not as a single unified research program.