GravField: MR Nudging for Collective Movement
- GravField is a mixed reality system that employs live-configurable digital physics to subtly guide group movement without explicit instructions.
- It utilizes MR objects like Rope, Spring, and Magnetic Field, each with parameterized dynamics and audiovisual mappings, to induce emergent intercorporeal coordination.
- The system integrates real-time orchestration by an Object Jockey and qualitative movement studies, emphasizing embodied exploration and user agency.
Searching arXiv for the exact GravField paper and closely related MR movement-guidance work to ground the article in the relevant literature. GravField is a co-located mixed reality performance system designed to nudge collective movement without relying on explicit instructions, choreography, or physical actuation. Participants wearing optical see-through MR headsets encounter shared virtual objects—such as ropes, springs, and magnetic particle fields—whose parameterised “digital physics” are continuously reconfigured by an “Object Jockey” (OJ). The system was developed to investigate how live-configurable MR objects can shape open-ended, intercorporeal movement while preserving agency, and it combines audiovisual feedback, real-time orchestration, and qualitative movement study within a performance-oriented HCI framework (Hu et al., 17 Sep 2025).
1. Conceptual orientation
GravField is positioned between prescriptive guidance and puppeteering. On one side are XR coaching systems that specify the “correct” movement; on the other are actuation-based approaches, such as EMS, that physically drive the body. GravField instead uses nudges: it alters the perceptual and relational conditions of movement so that some actions become more rewarding or more sensible than others, while participants remain free to move as they wish. This nudging operates through affordances of the virtual objects, reward structures in the audiovisual mappings, and live reconfiguration of those mappings during performance (Hu et al., 17 Sep 2025).
A central term in the system is intercorporeal coordination. In GravField, this refers to relations between bodies in which each person’s movement and perception are continually shaped by others. The design therefore emphasizes distance, tension, shared momentum, clustering, mirroring, relay sequences, and orbital movement, not as pre-scripted behaviors but as patterns that emerge from shared interaction with MR objects.
Another key term is cognitive-somatic cues. Because the objects are intangible, the system makes their “digital physics” perceptible through tightly coupled visual and sonic mappings. Visual features such as drooping versus taut lines, particle density, flow, or wave amplitude are paired with audio features such as pitch, loudness, filter brightness, and glitchiness. These cues provide what the paper describes as as-if haptic information: participants interpret sound and image through felt bodily experience and move as if they were sensing forces.
The design is explicitly open-ended. Participants are told there is no right way to move; mappings remain legible but not fully transparent; and the OJ periodically alters parameters to “defamiliarize” the interaction and reopen exploration. This suggests that GravField should be understood less as an instruction system than as an embodied, relational environment for exploratory movement.
2. System composition and technical implementation
GravField involves three participant roles. The first is the group of 2–4 people wearing HoloKit X + iPhone optical see-through headsets and moving together in the same room. The second is the Object Jockey, a facilitator-performer described as akin to a live-coder or musician, who introduces and reconfigures virtual objects in real time. The third is the spectator, who views the shared scene through iPads or TV-based spectator views.
Technically, the system is built around a Unity-based “Live Orchestration System” running on a MacBook Pro. The server receives real-time 6DoF headset data via Unity Netcode, simulates MR object state, and drives both visual updates and audio mappings. Spatial alignment uses ARKit and InstantCopresence, with a QR code on the floor for initial alignment and shared spatial anchors thereafter. Audio synthesis is handled through Ableton Live, with OSC data exchange and the CO/DA live-coding environment for more advanced mappings. The OJ controls parameters through TouchOSC on a tablet, with dedicated control panels for Rope, Spring, and Magnetic modalities (Hu et al., 17 Sep 2025).
The system prioritizes perceptual coherence over exact physical accuracy. Its physics is described as pseudo-physical and stylized rather than as a full rigid-body simulation. Rope and spring behavior are driven by simple parametric curves and kinematic quantities rather than by full mass-spring solvers. Only head-mounted tracking is used, so body pose is inferred only partially—for example, height is approximated by head -position. Audio and visuals are intentionally expressive rather than photorealistic, functioning as indexes of force rather than as literal simulations.
This technical architecture is integral to the system’s conceptual aims. Low-latency networking and interpretable control surfaces enable the OJ to adjust the experiential physics in performance time, while the simplified physical model preserves robustness and legibility.
3. Digital physics and MR objects
GravField’s interactive vocabulary is built around three MR objects (MROs): Rope, Spring, and Magnetic Field. Each is a simple metaphor with highly parameterised dynamics and cross-modal mappings.
| MRO | Key parameters | Salient mappings |
|---|---|---|
| Rope | , , , | controls amplitude; controls pitch |
| Spring | , , | pitch 0; low-pass cutoff 1 |
| Magnetic Field | 2, 3 | particle density/motion and glitch intensity vary with proximity and pole arrangement |
The Rope MRO connects participants with a virtual rope. Its digital physics parameters are weight 4 and elasticity 5, while derived kinematic variables include rope-center velocity 6 and centripetal acceleration 7. Higher 8 produces louder sound, and higher 9 produces higher pitch. The paper frames this as mimicking physical intuition: harder swinging yields a louder, higher “whoosh.” Live adjustment of 0 and 1 makes the rope feel heavier or lighter in both behavior and sound.
The Spring MRO is a taut virtual spring connecting participants. It exposes separation distance 2, relative height 3, and an abstract “tension” 4, described as conceptually akin to 5, though not formalized. Audio mappings are explicit: pitch is proportional to distance 6, and the low-pass filter cutoff is proportional to height 7, so higher stance produces brighter sound while crouching muffles it. The OJ can also remap spring visual amplitude to microphone volume, allowing clapping or shouting to increase visible oscillation.
The Magnetic Field MRO is a particle-based vector field representing attraction and repulsion. Its parameters include the sum of pairwise distances 8 and each participant’s magnetic “monopole” 9. These drive both visual density and motion of the field lines and audio properties such as glitch intensity and collision-like textures. The paper notes that with three or more participants the result resembles a visible “three-body problem,” producing chaotic and emergent patterns.
A common misconception would be to treat these objects as literal physical simulations. The paper instead presents them as perceptible, responsive metaphors whose digital physics need to remain legible enough for embodied interpretation while still being open to live deformation.
4. Live orchestration and the Object Jockey
The OJ is the central adaptive agent in the system. This role is described as part conductor, part live coder, and part improvising musician. The OJ observes engagement, introduces or attaches objects to particular participants, and continuously tunes mappings in order to reward interesting movement, shift group attention, or re-ignite exploration when boredom or fatigue begins to appear (Hu et al., 17 Sep 2025).
This live orchestration can occur at multiple levels. The OJ can adjust parameters such as rope mass or width, spring wavelength or shake strength, and magnetic polarity or per-field strengths. The OJ can also alter mapping curves, thresholds, and which variables control which audiovisual features. In some cases, visuals can be muted temporarily so that participants rely on audio only, thereby increasing interpretive ambiguity.
The paper describes several empirically observed strategies. One is to allow participants to explore until a pattern stabilizes and begins to saturate. Another is to monitor signs of fatigue or disengagement, such as slower movement or headset fiddling. The OJ then “bends” the physics—for example, increasing rope heaviness and tempo to encourage broader steps, increasing spring distance sensitivity to redistribute agency, or assigning height-mapping dominance to a single participant so that a new role structure emerges.
This live reconfiguration is presented as the mechanism by which GravField maintains metastability. Patterns are not simply produced and repeated; they are allowed to form, then destabilized and redirected before they become fixed routines. A plausible implication is that the OJ is not merely a controller of parameters but an active participant in the system’s intercorporeal dynamics, mediating between computational state and collective movement sense-making.
5. Empirical study and movement dynamics
The reported study involved 25 participants and 2 OJs in a large 8×16 m multimedia room. Participants were drawn from backgrounds including dance, live coding, new media art, music, and HCI, selected for comfort with improvisation and ambiguity. Sessions consisted of 15–20 minutes of GravField experience with 2–3 participants, followed by 15–30 minute post-session group interviews in the same space (Hu et al., 17 Sep 2025).
The data corpus included multi-angle video, system and ambient audio, semi-structured interviews, field notes, OJ autoethnographic reflections, and soma trajectory drawings in which participants traced dimensions such as Interest, Familiarity, Inclusion of Other in Self, and Sense of Agency. Analysis used bricolage analysis, combining HCI, movement, and music lenses, with open coding of transcripts. The study did not report detailed quantitative kinematics metrics such as RMS speeds or synchrony indices; its emphasis was qualitative and trajectory-based.
The movement findings are organized around a recurrent six-stage pattern termed the Live Nudging Spiral. In Learning, participants tentatively probe the object and discover mappings. In Internalizing, the mappings become intuitive and are integrated into body schema. In Coordination, group timing and role structure emerge without instructions. In Exploration, participants perturb stable patterns through experimentation, theatricality, or voice input. In Boredom/Fatigue, novelty decays and movement amplitude drops. In Reconfiguration, the OJ alters the physics or swaps objects, returning the group to a new learning phase.
Different MROs supported distinct movement dynamics. Rope often produced mirrored swinging, synchronized pulses, 360° spins, and triadic relay patterns. Spring supported push–pull behavior, alternating squats, and role redistribution when mappings changed. Magnetic fields encouraged more exploratory circling, orbital trajectories, and anthropomorphic interpretations such as “electron” behavior. The paper also reports an audio-only Spring vignette—the “heartbreak” configuration—in which one participant lay on the floor while others moved away, leading to a separation metaphor that disappeared when explicit visuals were restored.
These findings support the claim that GravField shapes intercorporeal choreography without scripts. The movement patterns are not authored in advance; they emerge from the coupling among digital physics, audiovisual feedback, participant improvisation, and OJ intervention.
6. Design principles, limitations, and significance
The paper formulates several design principles from GravField. It recommends beginning with grounded physical metaphors and then bending them; making effort, momentum, and tension audible and visible through synesthetic mappings; maintaining metastable open-endedness through multiple shallow attractors; balancing legibility and ambiguity; treating proxemics as a core design material; supporting shared authorship among participants, system, and OJ; and embedding ethics-by-design so that movement nudging remains voluntary and safe (Hu et al., 17 Sep 2025).
Several limitations are also explicit. Evaluation is mostly qualitative, and future work is suggested to complement it with movement metrics without undermining open-endedness. Sensing is limited to head tracking, constraining limb-level nuance and accessibility possibilities. The repertoire includes only three object metaphors. Spectator views are informative but not embodied. The paper also flags broader ethical questions concerning programmable reality, norms, consent, and the use of MR to influence physical behavior beyond staged settings.
Within HCI and mixed reality research, GravField contributes a concrete model of MR-mediated intercorporeal perception–action loops in which bodily action, live-configured digital physics, social coordination, and multisensory feedback are tightly entangled. It also contributes to soma design by prioritizing felt, qualitative movement experience over performance metrics, and to mixed reality performance by showing how live coding logics can be extended to group movement via parameterised virtual objects.
A frequent misunderstanding would be to read GravField either as an instruction system or as a hidden actuation system. The paper argues for neither interpretation. Its central claim is narrower and more technical: live-configurable MR objects, coupled to perceptible audiovisual mappings and moderated by an OJ, can guide collective movement through embodied, exploratory nudging while preserving participant agency.
