Articulated Search

Updated 9 July 2026

Articulated search is a family of problems where the search target is a structured object constrained by kinematics, sequential decisions, or interaction logic across domains like information retrieval, vision, and robotics.
Interactive systems use facets and context-aware guidance to enable iterative query reformulation and improve search relevance, as evidenced by significant increases in query submissions and result explorations in medical and archival tasks.
In computer vision and robotics, articulated search drives real-time 3D pose estimation and joint parameter optimization, achieving sub-millimeter accuracy and considerable speedups through methods like deep learning and factor graph inference.

Articulated search denotes a family of search problems in which the search space is constrained by articulated structure, sequential decisions, or both. In information retrieval and interface design, it refers to exploratory and iterative search in which users clarify information needs, reformulate queries, and refine results through facets and context-aware guidance. In computer vision, graphics, and robotics, it refers to searching over articulated poses, skeletal configurations, kinematic trees, joint parameters, actionable grasps, or physically plausible manipulation trajectories under structural constraints imposed by joints, links, or full-body interaction models. The literature also uses the term for web-based source search, where large models reason over directional choices in a simplified visual environment (Vakharia et al., 2013, Zhang et al., 2016, Zhu et al., 14 Feb 2025, Liu et al., 2024).

1. Scope and conceptual structure

The term spans several technically distinct but structurally related settings. In each case, the search target is not an unconstrained label or ranking score; it is a structured object whose valid states are restricted by articulation, kinematics, or staged interaction logic. This commonality explains why the term appears in search interfaces, 3D pose estimation, articulated object modeling, and robot manipulation.

Domain	Search target	Representative formulations
Information retrieval	refinement of search results and clarification of information needs	faceted search, snippets, tags, context-aware tips
Vision and graphics	articulated pose, canonical correspondence, joint state, kinematic structure	sequential search on manifolds, inverse rendering, root-finding, topology inference
Robotics and manipulation	actionable grasp, articulation model, motion trajectory	factor graphs, dense articulation prediction, online estimation

A survey of 3D modeling of human-made articulated objects frames the broader technical context as the joint problem of geometry modeling and articulation modeling. It emphasizes the “dual complexity of geometry and articulation,” the diversity of joint types and topologies, the prevalence of noisy and incomplete observations, the difficulty of generalization to unseen objects, and the importance of physical plausibility and constraints (Liu et al., 2024). This suggests that articulated search is best understood as a constrained inference problem whose search variables may include part segmentation, motion parameters, interaction priors, or user reformulation strategies, depending on the application.

2. Interactive information retrieval and web-based decision support

In search interfaces for spontaneous speech archives, articulated search is explicitly tied to exploratory and iterative search and to facet-based refinement. A design for a historical archive with “nearly 52,000 audiovisual testimonies” combines a search bar with auto-suggest, faceted search by Time Period, Location, and Subject, popular and suggested searches, a featured interview box, tags, highlighted search items, snippets, direct links to audio/video, and transcript-view markup such as <spk#>, <s>, and << text >> (Vakharia et al., 2013). Evaluation with 8 diverse university students reported 100% completion for the human-transcript task and 42.8% for the ASR-transcript task, while qualitative findings indicated that highlighted keywords and tags were most helpful and that users often judged relevance without opening full transcripts (Vakharia et al., 2013). The emphasis is not on a novel ranking algorithm; it is on interface elements that let users articulate and revise a search trajectory over noisy, spontaneous speech data.

A later context-aware interactive search companion places this logic directly inside the search engine result page. The system presents stage-specific guidance in a right-hand sidebar and triggers interventions for clarifying information need, improving query reformulation, result exploration, and bias mitigation. The controlled implementation uses rule-based triggers such as initial page load, query submission, 20 seconds of no click after a query, and return from a visited result. In a between-groups, pre-registered study with 170 participants performing medical search tasks, companion users submitted 75% more queries (1.96 vs. 1.12) and viewed roughly twice as many results (2.64 vs. 1.30), with improvement on the hardest topics (Bink et al., 16 Jan 2026). Here, articulated search is operationalized as guided reformulation and reflective search behavior rather than as latent-state inference.

A distinct but related web setting appears in AutoS $^2$ earch, which addresses source search in industrial safety management systems. The system operates on a web-based display of a $20 \times 20$ grid, invokes a multi-modal LLM to convert visual observations into structured language descriptions for four directional regions, and then uses chain-of-thought prompting to choose among A, B, C, and D according to prioritized rules: shortest distance to the dense source-probability region first, highest density of nearby unexplored space second (Zhu et al., 14 Feb 2025). On 20 challenging scenarios, the best AutoS $^2$ earch variants achieved 98% and 97% success, compared with 100% for human-AI collaboration, while reducing dependence on crowdsourced labor (Zhu et al., 14 Feb 2025). This use extends articulated search from document retrieval to staged, language-mediated decision support.

3. Search over articulated pose and configuration spaces

A foundational formulation appears in “Learning to Search on Manifolds for 3D Pose Estimation of Articulated Objects,” which casts 3D pose estimation from a single depth image as structured prediction over skeletal models on manifolds. Each joint transformation is an element of $\mathrm{SE}(3)$ , and the pose is represented by tangent vectors in $\mathrm{se}(3)$ composed by forward kinematics:

$\begin{pmatrix} \mathbf{x}_t' \ 1 \end{pmatrix} = e^{\hat{\xi}_1} e^{\hat{\xi}_2} \cdots e^{\hat{\xi}_t} \begin{pmatrix} \mathbf{x}_t \ 1 \end{pmatrix}.$

The paper introduces two learning-to-search variants: deep-L2S-para, which predicts all tangent vectors at once, and deep-L2S-seq, which predicts them sequentially along the kinematic chain or tree (Zhang et al., 2016). On the reported benchmarks, deep-L2S-seq reached 6.35 mm on mouse, 0.66 mm on fish, and 14.15 mm on human hand, while remaining real-time at 176.2 FPS, 31.7 FPS, and 35.5 FPS respectively (Zhang et al., 2016). The central idea is that search proceeds through valid skeletal actions rather than independent Euclidean regression of joints.

Neural field methods extend this view by treating articulation as a latent variable recovered by rendering-based search. CLA-NeRF is a “Category-Level Articulated Neural Radiance Field” trained on RGB images with camera poses and part segments, without CAD models or depth. At inference it estimates articulated pose by inverse rendering, optimizing the articulation variable $\mathbf{a}$ against the photometric error of the rendered image (Tseng et al., 2022). Its field jointly supports view synthesis, part segmentation, and articulated pose estimation across five categories on synthetic and real-world data (Tseng et al., 2022). The search problem is therefore analysis-by-synthesis over a continuous articulation space.

ARAH addresses a different search subproblem: finding both the ray-surface intersection in posed space and the corresponding point in canonical space for an articulated human SDF. It formulates a joint root-finding problem over canonical point $\hat{\mathbf{x}}$ and depth $d$ :

$\begin{aligned} f_{\sigma_f}(\hat{\mathbf{x}}) &= 0, \ LBS_{\sigma_\omega}(\hat{\mathbf{x}}) - (\mathbf{c} + d\mathbf{v}) &= \mathbf{0}, \end{aligned}$

and solves it with Newton’s method with Broyden’s update (Wang et al., 2022). The reported complexity drops from $20 \times 20$ 0 for naive alternation to $20 \times 20$ 1 for the joint formulation, with about 5× speedup, while improving generalization to out-of-distribution poses (Wang et al., 2022). In this setting, articulated search is correspondence search under deformation.

4. Structural search: parts, joints, and kinematic topology

Beyond pose estimation, many systems search for the articulated structure itself. Articulate AnyMesh addresses open-vocabulary conversion of a rigid 3D mesh into an articulated object. Its pipeline first performs movable part segmentation using a vision-LLM, DINO-X, and SAM, then estimates articulation via geometry-aware visual prompting over connecting areas between neighboring parts, and finally refines geometry and texture with RichDreamer using Score Distillation Sampling under randomized joint states (Qiu et al., 4 Feb 2025). The articulation estimator handles revolute and prismatic joints, infers axes and limits from rendered visual prompts, and reports strong out-of-domain articulation results; for unseen categories, the summary reports joint angle error of 4.81° for the proposed method, compared with 42.23° for NAP and 58.64° for CAGE (Qiu et al., 4 Feb 2025). This suggests a search regime in which semantic grounding and geometric reasoning jointly determine functional joints.

SPLATART pursues articulated Gaussian splatting with estimated object structure from posed images, with a subset containing image-space part segmentations. Its five stages are per-scene semantic Gaussian splat training, cross-scene part pose estimation, joint parameter estimation, tree-structure generation, and novel configuration rendering (Lewis et al., 13 Jun 2025). The pipeline explicitly “disentangles the part separation task from the articulation estimation task,” performs joint estimation post-facto, and is applied not only to tools and cabinets but also to articulated serial chain manipulators (Lewis et al., 13 Jun 2025). On the synthetic Paris dataset it reports 1.33° mean axis angle error, and on a Franka Panda example it reports joint estimation errors as low as 0.03–1.17° (Lewis et al., 13 Jun 2025). The design target is deeper kinematic trees than those typically studied in articulated object research.

Kinematify generalizes structural search to high-DoF objects synthesized from arbitrary RGB images or textual descriptions. It reconstructs part-aware 3D meshes, builds a contact graph, orients that graph into a kinematic tree with Monte Carlo Tree Search, classifies joint types with a vision-LLM, and optimizes joint parameters with the Distance-Weighted Contact-Aware Virtual Linkage objective (Wang et al., 3 Nov 2025). The MCTS reward combines structural regularity, static support, contact consistency, symmetry consistency, and hierarchy, and the final system reports axis angle error 16.06, axis position error 0.36, and tree edit distance 1.32, outperforming the listed baselines (Wang et al., 3 Nov 2025). Here articulated search is a combinatorial search over topologies followed by continuous optimization over joint parameters.

5. Manipulation-oriented search and online estimation

When the goal is physical interaction, articulated search typically couples perception with action selection. AO-Grasp generates 6 DoF grasps for articulated objects directly from segmented partial point clouds. Its Actionable Grasp Point Predictor uses PointNet++ with an MLP head to score surface points for stable and actionable grasps, and Contact-GraspNet is used to assign grasp orientations for the top- $20 \times 20$ 2 points (Morlans et al., 2023). The accompanying AO-Grasp Dataset contains 78,273 positively labeled actionable grasps on synthetic articulated objects, and the reported results are 45.0% grasp success in simulation versus 35.0% for the highest baseline, and 67.5% success on 120 real-world scenes versus 33.3% for the baseline (Morlans et al., 2023). The search target is not merely a stable contact but a contact that can actuate the object.

FlowBot++ predicts dense per-point articulation flow and dense articulation projection from a partial point cloud of a specified part. For revolute joints, the axis direction is recovered from the cross product of projection and flow, and for prismatic joints the average flow direction defines the axis (Zhang et al., 2023). It then generates multi-step trajectories for rotation or translation, reducing sensitivity to occlusion and noise. The reported execution time drops from about 17s to about 1.2s, and real-world experiments on a Sawyer robot demonstrate zero-shot deployment on real point clouds without finetuning (Zhang et al., 2023). This is a search procedure over feasible motion directions and axis parameters grounded in dense local geometry.

“Online Estimation and Manipulation of Articulated Objects” integrates search and control through a factor graph that fuses visual affordance priors, kinematics, force sensing, and an analytical articulation model based on Screw Theory. The articulation transform is parameterized as

$20 \times 20$ 3

and the factor graph performs MAP inference over poses, screw parameters, and configuration variables using affordance, force, kinematic, and articulation factors (Buchanan et al., 4 Jan 2026). A force factor enforces that valid motion is orthogonal to the measured reaction force, written as $20 \times 20$ 4, which allows the system to update its hypothesis when a pull or push fails (Buchanan et al., 4 Jan 2026). In real hardware experiments, the robot achieved a 75% success rate for autonomous opening of unknown articulated objects (Buchanan et al., 4 Jan 2026). The search process is explicitly online: initial visual prediction before contact, followed by rapid hypothesis revision during manipulation.

ArtiPoint extends online estimation to ego-centric, in-the-wild RGB-D videos with dynamic camera motion and partial observability. It extracts interaction intervals from hand segmentation, obtains object masks near the hand, tracks keypoints with deep any-point tracking, lifts them to 3D, smooths trajectories, and fits articulation parameters in a factor graph using a shared twist $20 \times 20$ 5 across the motion segment (Werby et al., 1 Sep 2025). On Arti4D, the full system reports 14.54° primary angular error, 17.14° revolute angular error, 0.07 m revolute positional error, and 0.98 revolute type accuracy (Werby et al., 1 Sep 2025). This is articulated search under clutter, egocentric motion, and intermittent visibility rather than under laboratory isolation.

6. Benchmarks, interaction corpora, and emerging evaluation regimes

Benchmarks have expanded the operational meaning of articulated search by making pose variation, structure variation, and interaction context explicit. ArtiAD is introduced as “the first large-scale benchmark for articulated 3D anomaly detection,” with 15,229 point clouds across 39 object categories, dense joint-angle variations, six structural anomaly types, part-level motion labels, and seen/unseen articulation splits (Gan et al., 29 Apr 2026). Its SPA-SDF baseline replaces the rigid prior with a pose-conditioned implicit field, estimates the articulation state by minimizing reconstruction energy, and scores anomalies as point-wise deviations from the learned manifold. The reported object-level AUROC is 0.884 on seen configurations and 0.874 on unseen configurations (Gan et al., 29 Apr 2026). This benchmark reframes articulated search as recovery of the articulation state needed to separate pose-induced geometry from true structural defects.

CHAIRS brings full-body human-object interaction into the search space. It contains 17.3 hours of multi-view RGB-D sequences, 1390 recordings from 46 participants, and 81 sittable objects, 28 of which have at least one articulated joint (Jiang et al., 2022). The dataset provides 3D meshes of both humans and articulated objects, motion-captured object-part poses, and a post-processing pipeline for temporal alignment, spatial alignment, and penetration removal (Jiang et al., 2022). The accompanying model reconstructs articulated object pose and shape from an image and an estimated human pose, then refines the result with a learned interaction prior implemented as a conditional VAE (Jiang et al., 2022). A plausible implication is that articulated search in HOI is increasingly mediated by physically plausible human context rather than by object-only observation.

Arti4D supplies an ego-centric, scene-level counterpart for articulation estimation in the wild. It contains 45 RGB-D sequences across four real-world scenes and 414 human-object interaction episodes, with articulation labels and ground-truth camera poses (Werby et al., 1 Sep 2025). Its design foregrounds dynamic camera motion, clutter, missing depth, and repeated interactions, thereby shifting evaluation away from fixed-camera, isolated-object assumptions (Werby et al., 1 Sep 2025).

Across these resources, a recurring conclusion is that rigid priors, single-state modeling, and category-specific assumptions are limiting. The survey on human-made articulated objects identifies open challenges in robustness to occlusion and clutter, category-agnostic inference, physical plausibility, and reconstruction of interior or hidden geometry (Liu et al., 2024). Recent systems respond by moving toward pose-conditioned implicit fields, post-facto joint estimation, Monte Carlo tree search over kinematic graphs, factor-graph fusion of vision and proprioception, and interaction-aware priors. This suggests that articulated search is evolving from a narrow problem of estimating a joint angle or part mask into a broader program of structured inference over geometry, motion, interaction, and decision-making.