Waiting for Hands (WfH)

• Waiting for Hands is an XR performance-installation that critiques hand-centered interaction norms and exposes inherent ableist design assumptions.
• It repurposes alternative controllers by integrating mouth-based interactions, transforming disability lived experience into a critical design strategy.
• The work employs layered staging and interruption, blending live performance with digital elements to foster a reflective, non-empathic viewing experience.

Waiting for Hands (WfH) is an interactive eXtended Reality (XR) performance-installation and research-creation artwork created by Yesica Duarte and Puneet Jain, in collaboration with Hema Kumari, an Indian disabled artist, singer, and activist living with Rheumatoid Arthritis (RA). Presented in the xCoAx 2025 paper “Challenging Disability and Interaction Norms in XR: Cooling Down the Empathy Machine in Waiting for Hands,” the project repurposes interaction norms in XR through Alternative Controllers and stages an absurd XR performance that critiques the framing of Virtual Reality as the “ultimate empathy machine.” Rather than using XR to immerse a viewer smoothly into another person’s experience, WfH uses friction, interruption, absurdity, and critical distance to challenge sentimentalized disability narratives and the able-bodied assumptions embedded in hand-centered XR interaction (Duarte et al., 21 Jul 2025).

1. Origins, collaborators, and basic conception

WfH was developed during the 2024 artistic residency BODIES-MACHINES-PUBLICS at Khoj International Artists’ Association in Delhi, India. It is explicitly positioned not as an accessibility exercise or technical prototype, but as a critical artwork that interrogates both the ableist body assumptions embedded in XR and the cultural framing of XR as an “empathy machine.” Hema Kumari’s arthritis makes conventional hand-based XR interaction painful or impossible, and WfH treats that exclusion as a design and political problem rather than as a mere need for accommodation (Duarte et al., 21 Jul 2025).

The project’s central claim is that mainstream XR interaction is built around a normative, non-disabled body, especially around hand-based gestures and controllers. In this account, XR contains a “corporeal standard” that excludes people whose movement does not fit those assumptions. WfH therefore rejects the idea that the primary goal is to make a dominant interaction paradigm slightly more inclusive. Instead, it proposes that XR storytelling and interaction norms themselves should be rethought through disabled lived experience.

A defining feature of WfH is its refusal of transparent identification. The audience is not invited to “become Hema,” nor to feel that disability has been understood through a temporary simulation of impairment. The work redirects attention toward the norms and infrastructures of XR itself: who it is designed for, what body it assumes, and how its interfaces encode able-bodiedness. This suggests that WfH belongs less to assistive-interface optimization than to disability-centered critique through performance.

2. Theoretical and critical framework

WfH combines disability studies, HCI, design research, media theory, and radical theatre. From HCI and accessibility research, it draws on critiques of XR’s “corporeal standard,” “ability assumptions,” and ability-based design. From disability studies, it uses Arseli Dokumaci’s notion of “activist affordances” and Margaret Price’s concept of crip spacetime, defined as “a material-discursive reality experienced by disabled people ... whose meaning is made through the relations among words, bodies, objects, technologies, and environments.” In WfH, these ideas are applied to XR as a socially and materially organized space rather than a neutral immersive medium (Duarte et al., 21 Jul 2025).

Hema Kumari’s everyday use of her mouth and teeth is foundational to the work’s interaction design. The paper describes actions such as pulling blankets with her teeth, unscrewing bottle caps between her teeth, and squeezing lemons with her mouth when her hands are in pain. These practices are treated not as compensatory curiosities but as sources of interaction logic. WfH therefore shifts the design center of XR away from the hand as the unquestioned natural default.

The project is also shaped by Disability Justice, especially through Shayda Kafai’s emphasis on collective liberation and on giving voice to those who are “unseen, marginalized, and forgotten.” The stated stance is one of refusal: refusal of voyeuristic consumption, refusal of pity, refusal of objectifying disabled life as emotional content for others. In that sense, WfH does not attempt to rehabilitate XR as benevolent technology; it asks what ethical stance XR performance can take while resisting spectacle.

Two theatrical lineages are central. From Brecht’s Epic Theatre, WfH takes the Verfremdungseffekt or alienation effect, using interruption rather than absorption. From Beckett’s Theatre of the Absurd, especially Waiting for Godot, it takes repetition, uncertainty, disjointed temporality, anti-resolution, and existential stasis. The title Waiting for Hands echoes Beckett, and the work’s dramaturgy uses uncertainty and anti-resolution to destabilize any straightforward empathic or moral reading. The paper also invokes McLuhan’s distinction between hot and cool media, presenting XR as typically a “hot medium” and WfH as an attempt to “cool down” XR by introducing ambiguity, divided attention, and visible mediation.

3. Performance structure and staging

WfH is described as an 8-minute work-in-progress performance for eight people total: six seated audience members and two participants on stage wearing XR headsets. A projected video documentary plays on the stage wall behind the live action. The two XR participants sit between the projection and the audience, facing one another across a table, so their bodies partially block the projected documentary. This staging prevents uninterrupted documentary spectatorship and produces a layered scene in which film, live performance, and hidden XR action interfere with one another (Duarte et al., 21 Jul 2025).

The performance unfolds in two acts, separated by a two-minute interval. In Act One, the projected video introduces Hema Kumari and her life as a disabled artist in Delhi. At the same time, the XR participants encounter a virtual scene “centered around lemons.” Initially they use their hands to pick floating virtual lemons; as the act proceeds, the interaction shifts toward mouth-based controls, and they navigate a surreal highway surrounded by lemons.

In Act Two, the curtains reopen to reveal that the table is covered with real lemons. Hema returns on screen, now speaking more directly about disability. The headset passthrough mode is activated, described as the RGB camera feed that lets XR users see the material world, so the performers can now see one another and the physical lemons in a mixed-reality configuration. They pick up the actual lemons and follow guided instructions that lead them to handle these objects in “unexpected ways.” Eventually the interaction extends outward to the seated audience through the lemons, and the performance ends with Hema appearing on screen singing a Bollywood folk song.

The seated audience thus never encounters a single coherent layer. They watch Hema’s documentary, but they also watch live XR users whose strange actions partially obscure that documentary. The work calls these onstage XR users “unreliable actors.” Their gestures ambiguously echo, distort, or interrupt the filmic layer rather than clarifying it. The result is a sustained instability of attention: the audience must negotiate between screen, stage, apparatus, and gesture, rather than consuming a unified story.

4. Alternative Controllers and embodied interaction

WfH’s Alternative Controllers are described as physical input mechanisms distinct from traditional handheld or console-style controllers, following the “Alt Ctrl” tradition. In this project, the relevant controller is specifically a camera-based mouth and lip tracker, integrated into XR and “co-developed with and for individuals with disabilities.” The two XR participants each wear a headset “adapted with a mouth and face tracker,” and the paper relates this to earlier work in which mouth gestures including chin, jaw, and tongue movements were detected to enable movement and interaction in VR/MR (Duarte et al., 21 Jul 2025).

The work’s embodied actions foreground the mouth as a primary interface. The paper describes “strange mouth and hand movements,” mouth-based controls, and mouth/lip tracking. The visible strangeness of these gestures is deliberate. It denaturalizes hand-centered XR interaction by showing that what appears “natural” in XR is often simply conventional. WfH asks why mouth-led gestures should appear aberrant, except that design conventions have already coded some bodies and motions as normal and others as deviant.

Technically, the paper remains deliberately sparse. It does not provide precise hardware names for the headsets, the mouth tracker, the cameras, or any microcontrollers or sensors. It does not give fabrication details, electronics schematics, tracking pipelines, calibration procedures, software frameworks, machine learning models, or interaction-state diagrams. No formal mappings, equations, or signal-processing formulas are presented. The main explicit clue beyond mouth/face tracking is the use of the headset passthrough function based on RGB cameras in Act Two. Accordingly, WfH should not be classified as a low-level interface paper in the usual systems sense.

The conceptual mapping, however, is clear. Standard XR controllers assume grasping ability, finger dexterity, comfortable arm extension, and hand-centered spatial manipulation. WfH rejects those assumptions by centering body parts and gestures that become meaningful through disabled lived practice. The shift from hand-picking virtual lemons to mouth-based navigation on a surreal lemon highway is thus both an interactional remapping and a critique of how “default” embodiment is encoded in XR.

5. Aesthetics, ethics, and anti-empathy politics

A common expectation in disability-related XR is that the medium should simulate another person’s condition so that non-disabled users can “step into someone else’s shoes.” WfH opposes that logic directly. The paper argues that such simulations often flatten disability into a temporary, consumable sensation and reduce it to pity, inspiration, or sentimental education for a presumed non-disabled observer. WfH refuses to offer Hema’s disability as a neatly consumable emotional experience and instead redirects attention toward the assumptions of able-bodiedness built into XR and interaction design (Duarte et al., 21 Jul 2025).

Its aesthetic method is estrangement rather than immersion. Instead of transparency, it offers interruption; instead of empathic fusion, uncertainty; instead of passive spectatorship, “active negotiation of meaning rather than passive consumption of experience.” The lemons are central to this effect. They recur across virtual and physical registers—as floating objects, a surreal environment, a table covering, and props offered to audience members—without being reduced to a single symbolic code. Their recurrence creates structure without explanatory closure, producing the Beckettian tone the paper seeks.

The performance’s visual obstruction is equally important. The two XR participants physically block parts of the projected documentary, so the audience cannot settle into a stable documentary relation to Hema’s story. This produces a layered and unstable spectatorship in which mediation is always visible. In Brechtian terms, XR becomes a distancing device rather than an absorptive one.

Ethically, the work is explicit that it wants to avoid objectifying Hema or portraying her disability as something to be pitied. The gaze is redirected away from disability as content and toward able-bodied norms as infrastructure. The disabled subject is therefore not staged as a problem to be explained, but as a standpoint from which the medium’s assumptions can be exposed. The paper frames this as asking how XR can “attune participants to a non-normative body schema while resisting the voyeuristic gaze and the sentimentalization of disability.” In this formulation, accessibility becomes “not an act of objectification but a reflective space.”

6. Documentation, limitations, and relation to adjacent hand-centered research

WfH is presented as a work-in-progress, and its published account is stronger as research-creation documentation than as technical or empirical evaluation. The paper does not provide a systematic account of audience reception. There is no formal user study, no interview data, no coded themes from spectators, and no deployment metrics. Its methodology is best understood as research-through-design and research-creation, grounded in collaborative making, disability-informed design, and artistic experimentation rather than controlled evaluation (Duarte et al., 21 Jul 2025).

This also defines its relation to adjacent hand-centered and controller-centered research. Unlike systems that aim at continuous 3D hand pose tracking on commodity wearables using built-in speaker and microphone hardware (Kim et al., 25 Feb 2026), or wearable exoskeleton systems for collecting dexterous hand demonstrations with explicit sensing, synchronization, and replay pipelines (Zhu et al., 20 Feb 2026), WfH does not prioritize precise hand-state estimation, kinematic fidelity, or benchmarked capture quality. Its contribution is instead to repurpose interface design as dramaturgy and critique.

That distinction matters because WfH is sometimes liable to be misread as either a technical accessibility retrofit or a disability-simulation system. The paper rejects both classifications. It is not simply trying to make an existing hand-centered XR paradigm more usable for more bodies, and it is not staging disability as immersive content for a non-disabled audience. Rather, it uses accessibility as a generative design force and an aesthetic strategy. A plausible implication is that WfH belongs to a line of work in which interaction design is not only a means of control, but also a means of exposing how control schemes encode social norms.

Within that frame, the project’s significance lies in how it repositions XR. It uses mouth-based interaction, theatrical alienation, visual obstruction, and absurd repetition to cool down a medium often described as immersive and sensory-dominant. What replaces seamless access is friction; what replaces identification is attunement; what replaces spectacle is critique.