---
title: 'Commonaiverse: Embodied Affective AI'
url: https://www.emergentmind.com/topics/commonaiverse
type: topic
---

# Commonaiverse: Embodied Affective AI

Searching arXiv for the Commonaiverse paper and closely related affective-computing work.
Commonaiverse is an interactive multimedia installation that explores human emotion through full-body movement tracking, real-time AI interpretation, and adaptive audiovisual feedback. It is organized as a three-phase experience—Teaching Phase, Exploration Phase, and Cosmos Phase—in which participants enter a private immersive space, teach the system how they want emotions to be expressed through their bodies, experiment with the AI’s interpretations, and then receive a visualized “cosmos” or constellation-like artifact summarizing their session [2509.22168]. Within affective computing, the project is defined by its departure from face-centric emotion recognition and from fixed, top-down emotion taxonomies. Instead, it presents emotion as embodied, social, contextual, and co-created between participants and machine, and it frames this shift as both an artistic and a research intervention into the politics, biases, and surveillance implications of emotion AI [2509.22168].

## 1. Conceptual orientation

Commonaiverse was designed in response to several limitations that its authors identify in mainstream affective computing: overreliance on facial-expression analysis, use of predefined, fixed emotion categories, dependence on static datasets, weak handling of cultural variation, insufficient attention to full-body movement, and ethically problematic uses of emotion AI in surveillance, productivity monitoring, and commodification of affect [2509.22168]. The installation therefore asks how AI might engage with emotion in a way that is embodied, adaptive, collaborative, and culturally situated rather than extractive or reductionist.

The system is explicitly situated at the intersection of affective computing, multimedia interaction, embodied AI, and participatory design. It remains an affective computing system because it interprets and responds to emotional expression, but it departs from conventional affective computing by emphasizing body movement rather than mainly facial or vocal features, by avoiding sole dependence on universal labels, and by using participant-generated interaction data rather than pre-made training datasets [2509.22168]. It is also a multimedia system because it integrates vision-based pose tracking, AI interpretation, real-time projection and laser visuals, sound synthesis and modulation, and a web-based post-session visualization [2509.22168].

A central claim of the project is that emotion is not well captured by systems derived from universal facial categories. The authors align themselves with constructionist critiques associated with Russell and Barrett and describe emotion as constructed, contextual, embodied, and relational [2509.22168]. Operationally, however, the implementation retains some explicit labels such as happiness, relaxation, anger, sadness, and joy. This indicates a hybrid structure: the system critiques fixed emotion taxonomies, yet still uses recognizable emotional labels while allowing participants to teach how those labels are embodied in a particular session [2509.22168]. This suggests that the project is less a rejection of labeling per se than a reallocation of interpretive authority from static datasets to situated interaction.

## 2. Spatial setting and participant experience

The installation is designed for at least two participants and lasts approximately 15–20 minutes, with each phase typically taking 5–7 minutes [2509.22168]. The authors emphasize the minimum of two participants because they treat emotional communication as inherently social and interdependent rather than purely individual. The experience proceeds through a fixed sequence: participants enter an enclosed, private room; the system detects their presence; they are guided into an initial learning interaction; they demonstrate emotions with their bodies; the system learns from those demonstrations; they move freely while the AI interprets them in real time; the room responds audiovisually; and the session concludes with a “cosmos” summarizing the interaction, retrievable afterward through a QR code [2509.22168].

The room is described as enclosed, private, and immersive. These properties were chosen to reduce external judgment, encourage authentic expression, support uninhibited bodily performance, and reinforce the sense that emotion is shared within a bounded social environment [2509.22168]. Earlier design iterations envisioned LED strips or low-resolution grids on the walls, but this concept was abandoned because the LED wall was found to be too visually dominant, drawing attention away from participants’ embodied interaction with one another [2509.22168]. The final room configuration instead used reflective fabric walls, three lasers mounted on the ceiling, projected abstract light patterns creating ripple-like effects, a main display interface visible during interaction, an external panel for the final Cosmos visualization, and six speakers for spatial sound [2509.22168].

The resulting environment is not merely a display surface but a responsive interaction chamber. The paper does not specify exact room dimensions, exact camera placement, or the exact type or model of camera hardware used for MoveNet input [2509.22168]. This omission is significant for reproducibility, but it also indicates that the installation is presented primarily as a design and conceptual system rather than a narrowly instrumented benchmark.

## 3. Three-phase interaction model

### Teaching Phase

The Teaching Phase is conceptually central to Commonaiverse’s participatory framing. Participants are asked to demonstrate specific emotional states using full-body movement. The paper gives examples such as sadness expressed by slowly crouching and hunching the shoulders, and joy expressed by leaping with outstretched arms [2509.22168]. It also mentions sadness, joy, and anger, while elsewhere the implementation refers to happiness, relaxation, anger, and sadness; the paper does not fully clarify whether the interaction always uses exactly the same fixed set [2509.22168].

Technically, MoveNet tracks participants’ full-body poses, the movement data are analyzed in real time, and the AI builds a “data map” connecting bodily gestures to emotion labels [2509.22168]. The significance of this phase lies in the inversion of the usual affective-computing workflow: rather than subjecting participants to a pre-trained universal model, the installation presents them as teachers of the system. The emotional lexicon is therefore described as co-created rather than imposed from a prior dataset [2509.22168].

### Exploration Phase

In the Exploration Phase, participants engage in unscripted movement, free-form bodily expression, and ongoing collaborative interaction with each other [2509.22168]. The AI attempts to infer emotional meaning from the movements it learned during teaching, and the system responds with visual and sonic feedback. This creates a closed loop: participants move, the system interprets, the room responds audiovisually, and participants can then react to that response [2509.22168].

This phase functions both as interaction and as informal evaluation. Participants can implicitly assess whether the AI’s interpretations align with their intentions, while the system continuously reevaluates emotional states over time [2509.22168]. The project’s authors frame this loop as evidence that affective AI can be collaborative and exploratory rather than a hidden classification engine.

### Cosmos Phase

The final Cosmos Phase produces a digital “cosmos,” conceptualized as a constellation of shared experiences [2509.22168]. The visualization includes emotional exchanges during the session, movement patterns, session metrics, and an abstract 3D generative form; later in the paper these forms are described as unique crystal-like installations generated in WebGL [2509.22168]. The output appears on an external panel and is also made available through a unique QR code.

This phase has several roles. It provides artistic closure, a reflective summary of the interaction, a representation of collective dynamics, and a concrete mechanism for participant access to their own session data [2509.22168]. The authors explicitly present this as a contrast to extractive AI systems in which user data are silently harvested and retained elsewhere. A plausible implication is that the Cosmos artifact functions as both a visualization and a symbolic reallocation of data agency.

## 4. Technical architecture

The reported system pipeline begins when participants enter the installation and their body movements are captured. MoveNet estimates full-body keypoints; Python scripts process and filter pose data; pose features are sent via Open Sound Control; a multi-recommender AI system interprets emotional states; TouchDesigner receives movement and emotion streams; TouchDesigner controls laser visuals and the display interface; Max/MSP receives data via OSC and modulates sound; session data are sent to a WebGL application; a crystal or cosmos visualization is generated; and a QR code gives access to the resulting artifact [2509.22168].

The paper names the principal software frameworks directly: MoveNet, TensorFlow Lite, custom Python scripts, OSC, TouchDesigner, Max/MSP, and a WebGL application [2509.22168]. The hardware specifications are less complete. The paper specifies three ceiling-mounted lasers, six speakers, reflective fabric walls, and display interfaces, but it does not specify CPU or GPU details, camera models, exact compute devices, projector models, or laser models [2509.22168].

The system is designed for real-time processing and low latency, and MoveNet was chosen partly because it supports such responsiveness [2509.22168]. Yet the paper does not provide measured latency numbers, frame rates, throughput metrics, or synchronization timings. This limitation means that the technical discussion remains architectural and component-based rather than performance-characterized in a systems sense.

## 5. Motion tracking and inference

MoveNet was selected because, according to the paper, it offers high accuracy, robustness in controlled and real-world conditions, lightweight architecture, suitability for edge devices, and real-time performance with low latency [2509.22168]. It estimates 17 key body points corresponding to major joints and body landmarks, though the paper does not list the 17 points individually [2509.22168].

Data from MoveNet were processed with custom Python scripts using TensorFlow Lite. These scripts handled model configuration, interpretation of keypoint data, and transmission of pose estimates to multimedia components via OSC [2509.22168]. To improve real-time stability, the authors applied smoothing algorithms to interpolate noisy frames, dynamic calibration to adapt to participant height and posture variation, and confidence thresholding on each keypoint to remove inaccurate detections [2509.22168]. The paper does not specify the exact smoothing method, threshold values, calibration formula, or temporal filter parameters.

A notable feature of Commonaiverse is its multi-recommender AI system. Rather than relying on a single classifier, the installation uses multiple specialized recommenders that analyze different aspects of movement and interaction, including amplitude, speed, frequency, and interpersonal proximity [2509.22168]. These recommenders are then aggregated into a final consensus, though the paper does not provide a formal fusion equation, weighting rule, voting mechanism, or confidence-combination formula [2509.22168].

Three recommender modules are described as implemented. REC1, the Behavioral Data Recommender, extracts movement features such as speed, amplitude, and participant proximity. REC2, the Contextual Recommender, adds situational and group information, including number of participants and spatial relationships. REC3, the Longitudinal Emotion Recommender, tracks emotional trends over time and allows the system to adapt to sustained states, shifting states, and inconsistencies over a session [2509.22168]. Additional modules—REC4 for facial expression, REC5 for voice tone, and REC6 for physiological signals—are discussed only as possible future expansions [2509.22168].

The installation’s AI operation is presented in three processing stages: Preparation Phase, Detection Phase, and Evaluation and Adaptation Phase. In preparation, the system initializes and detects predefined emotions using metrics such as movement amplitude, speed, and proximity. During detection, it continuously interprets incoming movement data and dynamically updates predictions. During evaluation and adaptation, it monitors emotional states over time and reevaluates predictions if behavior shifts [2509.22168]. The paper does not specify whether these recommenders are trained machine-learning models in the conventional sense; there are no explicit references to SVMs, neural classifiers, loss functions, or conventional training procedures [2509.22168]. The safest characterization is therefore a modular, feature-driven, real-time interpretive architecture.

## 6. Audiovisual outputs and embodied feedback

Commonaiverse uses audiovisual feedback not as a decorative layer but as the medium through which AI interpretation becomes perceptible. Using TouchDesigner, the installation maps incoming OSC data to laser projections, geometric patterns, color schemes, and dynamic visual complexity and fluidity [2509.22168]. Increased movement intensity or speed yields greater complexity or fluidity of visuals, while changes in emotional state alter color and overall aesthetic tone [2509.22168]. These visuals are projected onto reflective fabric walls, producing ripple-like effects.

A main display visible to participants shows movement metrics such as quantity, speed, and range of motion, alongside emotion-specific values including happiness, relaxation, and sadness [2509.22168]. This display makes the system’s inferences partly legible in real time, which distinguishes the installation from opaque affect-recognition systems whose internal judgments remain hidden from the people being measured.

Sound is generated using Max/MSP and delivered through six speakers to create a spatialized auditory field [2509.22168]. The paper describes several explicit movement-to-sound mappings: faster movement produces faster tempo; emotion interpretations guide harmonic shifts between major and minor tonalities; more active movement generates more complex rhythms or syncopation; and larger or more intense movement increases loudness or sonic intensity [2509.22168]. The soundscape is thus affectively parameterized without being reduced to a single classification readout.

Taken together, these mappings form a closed interaction loop in which movement is captured, emotional interpretation is inferred, audiovisual parameters are adjusted, and participants respond to the resulting environment [2509.22168]. This is a defining feature of the installation. The AI does not merely label bodies; it transforms the space around them, making interpretation materially interactive.

## 7. Ethics, inclusivity, and relation to affective computing

One of the most developed aspects of the project is its ethical critique of existing emotion-recognition technologies. The paper criticizes systems tied to workplace productivity monitoring, consumer surveillance, policing, commodification of affect, and broader surveillance-capitalist extraction [2509.22168]. Against this background, Commonaiverse is designed to emphasize collaboration instead of extraction, participant teaching rather than opaque pre-trained labeling, reflective and artistic interaction, and the return of session data to participants through the Cosmos phase [2509.22168].

The paper also foregrounds bias and representation. Conventional emotion-recognition systems are described as encoding racial bias, cultural bias, gender bias, and Western normative assumptions [2509.22168]. Commonaiverse attempts to respond by relying on live participant input, decentralizing universal assumptions, and enabling a situated emotional lexicon. The claim is not that these problems are solved, but that the installation shifts the locus of emotional interpretation toward local performance and shared negotiation.

Privacy and ownership are discussed in similarly conceptual terms. By letting participants see the final representation, access it via QR code, and revisit their data artifact, the system frames session data as something users can reclaim rather than something silently harvested [2509.22168]. However, the paper does not specify data retention policy, anonymization procedures, storage architecture, or consent protocol details. This is an important limitation: the project is stronger as a critique and design alternative than as a fully specified privacy-preserving system.

The installation’s most important theoretical move is its reframing of emotion as contextual, embodied, social, culturally variable, dynamic, and sometimes overlapping rather than discretely separable [2509.22168]. The teaser figure description mentions an emotional distribution visualized in two axes based on Russell and Scherer, suggesting a dimensional framing related to arousal/valence-like space, but the paper does not formally explain the model or provide exact dimensions or equations [2509.22168]. This partial formalization is characteristic of the work as a whole: conceptually ambitious, technically concrete at the system-assembly level, but not mathematically explicit.

## 8. Deployment, claims, and limitations

Commonaiverse was exhibited at the CCCB, Centre de Cultura Contemporània de Barcelona, during the exhibition *AI: Artificial Intelligence*, from October 18, 2023 to March 17, 2024 [2509.22168]. The paper does not present a conventional user study with participant counts, demographics, interviews, surveys, quantitative metrics, or statistical analysis [2509.22168]. Instead, it offers design rationale, reflections from iterative prototyping, qualitative observations from deployment, and a critical discussion of aims and limitations.

The main claims are correspondingly conceptual and design-oriented. The authors argue that Commonaiverse demonstrates the value of moving beyond facial-expression-only systems, using full-body interaction for emotional expression, making affective AI more collaborative and participant-driven, foregrounding social and cultural complexity, and supporting user agency while reducing bias by avoiding static datasets [2509.22168]. These claims are not validated through comparative experiments, and the paper explicitly notes several limitations: real-time processing can miss subtle or overlapping emotional states; performance can degrade in noisy or crowded conditions; reliance on visual and auditory feedback may limit emotional depth for some users; movement-based interpretation can introduce bias toward certain physical capabilities; and cultural adaptability requires further exploration [2509.22168].

These limitations are substantial. They indicate that the installation should not be read as a solved alternative to mainstream affective computing, but as a design argument and prototype of another direction. This suggests that its significance lies less in algorithmic novelty narrowly construed than in the reorganization of affective AI around embodiment, co-creation, and interpretive visibility.

## 9. Place within broader digital and commons-oriented discourse

Although Commonaiverse is principally an affective AI installation, its participatory and anti-extractive framing resonates with broader research on collaborative governance and digital commons. Work on commons-governed artificial intelligence argues that AI institutions differ from both market and state frames when resources are “pooled, accountable, collectively stewarded” under self-given rules rather than unilateral private control or top-down command [2606.15466]. Commonaiverse does not present itself as a full commons-governed AI institution in that formal sense, but its emphasis on participant teaching, shared meaning-making, and partial recovery of data agency suggests an adjacent design logic.

A related philosophical literature on digital commons argues that systems such as Wikipedia and arXiv show how digital tools can support collective knowledge without being reduced to extractive platform logics, and that the key issue is not openness alone but collaborative governance, verifiability, and contributive participation [2509.01014]. Commonaiverse operates in a different domain—emotion, movement, and multimedia installation rather than shared textual knowledge—but it belongs to the same family of interventions that oppose opaque mediation with visible, participatory infrastructures.

This broader context matters because affective computing has often been criticized precisely for turning lived expression into a hidden, monetizable signal. Commonaiverse instead stages emotional interpretation as a shared event in a bounded environment, renders parts of the inference process legible through displays and audiovisual responses, and returns a session artifact to participants [2509.22168]. A plausible implication is that the installation should be understood not only as an HCI prototype but also as a contribution to debates over how AI systems distribute interpretive power.

## 10. Significance

Commonaiverse contributes a collaborative, embodied affective AI installation centered on full-body movement rather than face-only emotion analysis; a three-phase participatory interaction design consisting of Teaching, Exploration, and Cosmos phases; a real-time technical pipeline integrating MoveNet, a multi-recommender AI architecture, TouchDesigner, Max/MSP, and WebGL; a participant-driven approach to emotion interpretation intended to shift from top-down universal categorization toward co-created, culturally situated meaning; and a critical ethical reframing of emotion AI that emphasizes agency, transparency, and resistance to extractive surveillance logics [2509.22168].

Its broader significance lies in offering a countermodel to emotion AI systems built on static datasets, universalized labels, and hidden inference. The project does not provide a formal algorithmic theory, a conventional empirical evaluation, or a complete governance and privacy framework. Instead, it provides a concrete socio-technical argument: emotional interpretation can be negotiated rather than imposed; embodied interaction can exceed the expressive limits of face-centric pipelines; and multimedia installations can function as laboratories for more ethical, participatory, and culturally situated forms of AI [2509.22168].

Source: https://www.emergentmind.com/topics/commonaiverse