Pose-in-Context (PiC): Unified 3D Motion Learning

• Pose-in-Context (PiC) is a pose-centric formulation that models pose relative to contextual evidence, unifying cross-domain 3D motion tasks.
• It employs a dual prompt design with Task-Guided and Task-Unified Prompts alongside a two-stream transformer to learn shared motion representations.
• PiC replaces domain-specific output heads with prompt-conditioned inference, enabling cross-task transfer and reducing ambiguity through contextual cues.

Pose-in-Context (PiC) denotes a pose-centric formulation in which pose is modeled relative to contextual evidence rather than as an isolated set of joint coordinates. In its most explicit usage, PiC is introduced as an in-context learning framework for cross-domain 3D human motion modeling: a single model receives a prompt pair and a query, infers the hidden task specification from the prompt, and predicts the output for the query without domain-specific heads or multi-stage training (Liu et al., 14 Aug 2025). In related literatures, closely allied pose-in-context formulations treat context as joint dependencies, detector feature maps, scene affordances, occlusion structure, or multimodal sensory evidence, and use those signals to reduce ambiguity, improve controllability, or enforce plausibility (Ma et al., 2021, Zhao et al., 2023, Brooks et al., 2021).

1. PiC as an in-context learning formulation for 3D human motion

PiC is defined around the in-context mapping

$$\mathcal{M}\Big([<\mathrm{input}>_P,<\mathrm{target}>_P]^D,\;<\mathrm{input}>_Q\Big)\longrightarrow <\mathrm{output}>_Q,$$

where $D$ is a domain combining task, modality, and dataset (Liu et al., 14 Aug 2025). The central premise is that the model need not be given explicit task labels or domain-specific switches; instead, it infers the task from an example in the prompt and applies that latent specification to the query. Within the original formulation, PiC is pose-centric rather than mesh-centric, and it targets pose-based tasks across multiple datasets in a unified representation.

The task scope reported for PiC includes 3D Pose Estimation from 2D to 3D, Future Pose Estimation, Pose (3D) Joint Completion, Motion Prediction, and Motion In-Between, with data drawn from three datasets, including Human3.6M, AMASS, and FreeMan (Liu et al., 14 Aug 2025). All tasks are cast into a unified masked sequence-to-sequence regression format over joint-position sequences, so that cross-task transfer does not require changing the output space or attaching specialized heads. This unification is the defining technical move of PiC: the model is asked to learn cross-domain motion reasoning through prompt-query structure rather than through architectural compartmentalization.

A notable implication of this design is that PiC treats domain as a compositional object. Because $D$ combines task, modality, and dataset, prompting is not merely few-shot supervision; it is also the mechanism by which the model identifies which domain-specific semantics are active for the current inference. In the original PiC, that compositionality remains restricted to pose, but it establishes the template later extended by Human-in-Context (HiC) (Liu et al., 14 Aug 2025).

2. Prompt construction, network design, and training mechanics

PiC uses a dual prompt design consisting of a Task-Guided Prompt (TGP) and a Task-Unified Prompt (TUP) (Liu et al., 14 Aug 2025). The TGP is a randomly selected pair $$[\mathbf{X}_i^{\mathrm{in}}, \mathbf{X}_i^{\mathrm{gt}}]^D$$ from the same domain and serves as a hard prompt drawn from real data. The TUP is a learnable soft prompt that encodes shared task or domain priors and is implemented either as an averaged pose projected into feature space or as a directly learnable feature. During training and inference, PiC constructs context from the combination of a sampled TGP and the shared TUP.

The prompt and query are encoded as

$$\mathbf{X}_P=\mathcal{E}\Big([\mathbf{X}_i^{\mathrm{in}},\mathbf{X}_i^{\mathrm{gt}}]^D\Big),\qquad \mathbf{X}_Q=[\mathcal{E}(\mathbf{Q}_j^D),\mathbf{U}],$$

where $\mathcal{E}(\cdot)$ is the encoder and $\mathbf{U}$ is the TUP (Liu et al., 14 Aug 2025). The backbone is a two-stream transformer-like architecture with one stream for the prompt and one for the query. Each stream alternates spatial and temporal attention, and a representative layer is written as

$$\mathbf{X}_P^{l+1} = \alpha\,\mathcal{T}_1\big(\mathcal{S}_1(\mathbf{X}_P^l)\big) + \beta\,\mathcal{S}_2\big(\mathcal{T}_2(\mathbf{X}_P^l)\big),$$

with $\mathcal{S}$ denoting spatial attention, $\mathcal{T}$ temporal attention, and $D$0 learnable weights. After $D$1 layers, the prompt and query streams are aggregated as

$D$2

followed by additional processing and decoding (Liu et al., 14 Aug 2025).

Training is unified across all supported tasks and datasets in a single process. Prompts are sampled from the same domain as the query, all TGPs share the same TUP, and supervision uses standard losses appropriate to the prediction target, including MPJPE (Liu et al., 14 Aug 2025). The absence of domain-specific heads is not a minor implementation detail; it is the architectural expression of PiC’s claim that prompt-conditioned inference can replace explicit task routing for pose-based motion domains.

3. Limitations of the original PiC and its evolution

The original PiC is described as a proof-of-concept framework, and its limitations are stated with unusual clarity (Liu et al., 14 Aug 2025). First, modality diversity is narrow: PiC covers pose-based representations only and cannot generalize to mesh-based tasks or bridge pose-to-mesh gaps. Second, the prompting strategy is naive: TGPs are selected randomly and independently of the query, so contextual or stylistic similarity between prompt and query is not guaranteed. Third, contextual dependency handling is limited: the model uses a global attention-based architecture and does not explicitly model fine-grained dependencies across local/global or spatial/temporal scopes.

These limitations motivate HiC, which extends PiC to broader cross-domain 3D human motion modeling (Liu et al., 14 Aug 2025). HiC combines pose and mesh representations, expands task coverage, increases dataset scale, introduces a max-min similarity prompt sampling strategy, and uses a dual-branch context injection architecture for improved contextual dependency handling. The quantitative contrast reported between the two frameworks is concise:

Framework	Data Scale / Modalities	Tasks / Datasets
PiC	0.18M / pose	5 / 3
HiC	3.83M / pose+mesh	10 / 4

The PiC-to-HiC transition clarifies what PiC accomplished and what it left unresolved. PiC established that in-context learning can serve as a unifying interface for pose-centric motion tasks; HiC argues that such a unification requires better prompting, broader representations, and richer context injection to scale beyond pose-only domains (Liu et al., 14 Aug 2025).

4. Context modeling as the technical lineage of pose-in-context estimation

Long before PiC was named as an in-context learning framework, 3D human pose estimation had already treated context as a primary mechanism for reducing ambiguity. A unified formulation is given by

$D$3

where $D$4 are unary joint terms and $D$5 are pairwise context terms over body edges $D$6 (Ma et al., 2021). In that view, both pictorial structure models (PSM) and graph neural networks (GNN) are special cases. PSM makes limb-length constraints explicit through

$D$7

whereas GNNs learn the pairwise functions with message passing and end-to-end training (Ma et al., 2021).

The comparative result is that end-to-end training in GNNs and limb-length constraints in PSMs are complementary rather than mutually exclusive. ContextPose combines them through an attention mechanism and a differentiable soft limb-length loss,

$D$8

thereby reducing absurd 3D pose estimates with incorrect limb lengths and improving generalization (Ma et al., 2021). This is an important correction to a common oversimplification: context in pose estimation is not exhausted by graph connectivity; anatomical constraints and training regime both matter.

Other works operationalize context differently. "Global Context for Convolutional Pose Machines" introduces a U-shaped context module to expand the empirical receptive field of CPMs and reports 87.9% PCKh on Look Into Person; a smaller version is 2.9% less accurate while running more than 160 frames per second (Osokin, 2019). "A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimation" argues that temporal context is overused when spatial context in detector feature maps is discarded: its Context-Aware PoseFormer uses a single frame, achieves MPJPE $D$9 mm and PA-MPJPE $D$0 on Human3.6M, surpasses MixSTE at 40.9 mm with 243 frames and MHFormer at 43.0 mm with 351 frames, and reduces computation from 14.2 GFLOPs to 0.6 GFLOPs relative to MHFormer (Zhao et al., 2023). Together, these works show that pose-in-context estimation may refer to explicit body-graph reasoning, expanded receptive fields, or reuse of detector-side visual context, but in each case the shared goal is ambiguity reduction.

5. Pose relative to scenes, interactions, and human placement

A second major meaning of pose-in-context appears when pose is conditioned on, or used to infer, scene structure. In human-object interaction detection, "Exploring Predicate Visual Context in Detecting Human-Object Interactions" proposes Predicate Visual Context (PViC), a two-stage transformer-based detector that enriches human-object pair representations with contextual visual information via explicit queries, cross-attention, and box pair positional embeddings (Zhang et al., 2023). On HICO-DET, PViC reaches 34.69 mAP, 32.14 mAP on rare classes, and 35.45 mAP on non-rare classes; with Swin-L it reaches 44.32 mAP. On V-COCO it reaches 62.8 AP in S1 and 67.8 AP in S2 (Zhang et al., 2023). The underlying claim is that coarse object identity and box geometry are insufficient for subtle or ambiguous interactions, and that pose and local scene cues must be reintroduced into the HOI representation.

In generative modeling, "Hallucinating Pose-Compatible Scenes" inverts the usual conditioning direction by asking what human pose reveals about a scene (Brooks et al., 2021). It trains a conditional GAN on a "Humans in Context" meta-dataset of 19,503,700 frames and 229,595 video clips, uses dual pose conditioning through keypoint heatmaps and a pose latent, removes style mixing, doubles channel width relative to StyleGAN2, and outperforms pose-conditioned StyleGAN2 and Pix2Pix/Pix2PixHD baselines on both percent of correct keypoints and FID (Brooks et al., 2021). Here pose is not merely an object attribute; it is treated as a carrier of action semantics, affordances, and object-interaction cues from which scene compatibility can be hallucinated.

Related image-synthesis work uses pose and context jointly for explicit compositional control. "Person-in-Context Synthesis with Compositional Structural Space" defines a layout $D$1, with object classes $D$2, boxes $D$3, and pose keypoints $D$4, and encodes context and person structure in separate branches before composing them into a shared structural space (Yin et al., 2020). On COCO-Stuff 128$D$5128 person split it reports FID 77.8 and classification accuracy 50.17%, versus 78.2 and 41.10% for LostGAN; on Visual Genome 128$D$6128 it reports FID 58.74 and accuracy 57.49% (Yin et al., 2020). "Person-In-Situ" adds explicit 3D pose control and occlusion-aware depth reasoning for human insertion into images, using latent diffusion models in either a two-stage explicit-depth pipeline or a direct implicit model; the direct model reports SSIM 0.723, MSE 0.0177, CLIP similarity 0.893, Depth SSIM 0.896, and Depth MSE 0.0141, improving over Kulal et al. on all listed metrics (Masuda et al., 7 May 2025). These works collectively broaden pose-in-context from estimation to scene-consistent synthesis and insertion.

6. Multimodal, embodied, and 3D-aware extensions

Pose-in-context reasoning has also expanded beyond image-only human motion. In robotics, PoseIt introduces a visual-tactile dataset for grasp stability under different holding poses, with 1840 grasp data points from 26 household objects and 16 holding poses per object (Kanitkar et al., 2022). It frames the task as predicting whether an object will remain stable when shaken in a given held pose and reports 85.2% accuracy for Vision+Tactile with LSTM+DRS under random uniform pose split, 77.3% on unseen objects, and gains of 13.2% over vision alone and 3.4% over tactile alone (Kanitkar et al., 2022). Here pose-in-context is operationalized as stability under functional repositioning rather than as static grasp success.

A different robotics usage appears in "Pose Imitation Constraints for Collaborative Robots," which introduces PIC and PICs as octant-based hard and soft constraint systems for human pose imitation on Baxter and YuMi (Gonzalez et al., 2020). Compared with FABRIK, the reported mean pose accuracy is 25% for FABRIK, 53% for PIC, and 58% for PICs; for incision occlusion, the reported means are 10% for FABRIK, 4% for PIC, and 9% for PICs (Gonzalez et al., 2020). The point is not merely anthropomorphic motion, but human-like motion subject to workspace context and visibility constraints.

Vision-language and customization systems use yet another variant. "Make VLM Recognize Visual Hallucination on Cartoon Character Image with Pose Information" introduces pose-aware in-context visual learning and reports hallucination-detection improvements from 50% to 78% for GPT-4V and from 57% to 80% for Gemini Pro Vision when pose information is added (Kim et al., 2024). "Pose-ICL: 3D-Aware In-Context Learning for Pose-Controllable Subject Customization" moves back toward the in-context formulation, but now for generative subject customization rather than motion modeling (Han et al., 9 Jun 2026). Its Surface-Anchored Position Embedding,

$D$7

anchors attention to surface coordinates of a volumetric bounding box and modulates attention as

$D$8

yielding pose MAE 10.75 and DINO-I 0.758 on 3D assets, and pose MAE 13.20 and DINO-I 0.758 on real-world subjects, while remaining tuning-free (Han et al., 9 Jun 2026). This suggests that the PiC idea has broadened from pose-conditioned prediction to a general strategy for injecting structured contextual geometry into in-context inference.

Across these variants, a stable pattern emerges. Whether the target is 3D motion prediction, HOI classification, scene generation, robotic manipulation, hallucination detection, or subject customization, pose is most effective when contextualized by structure outside the joint list itself: graph constraints, detector features, scene depth, interaction regions, or cross-modal measurements. PiC, in that sense, names both a specific framework for cross-domain 3D human motion modeling and a wider research principle: pose becomes informative when treated as a relation to context rather than as an isolated coordinate array.