MindCube: Interactive Device & Spatial Benchmark

• MindCube is a dual-purpose system integrating a sensor-rich interactive device for emotion and musical exploration with a vision-language benchmark evaluating spatial reasoning.
• The device uses precise sensor fusion and two mapping techniques—AI-driven generative and non-AI expressive—to deliver real-time emotion-regulation feedback.
• The benchmark challenges models with cognitive mapping, perspective taking, and mental simulation tasks, yielding quantifiable improvements in spatial intelligence.

MindCube is both a technical artifact—a sensor-rich, fidget-cube-inspired interactive device for embodied emotion and musical exploration—and the name of a vision-language reasoning benchmark specifically constructed to probe spatial mental modeling under conditions of viewpoint variability and partial observability. The dual usage reflects its cross-disciplinary role at the intersection of affective computing, embodied musical interfaces, and spatial cognition in artificial intelligence models.

1. MindCube as Sensor-Driven Interactive Device

The MindCube device consists of a 3.3 cm × 3.3 cm × 3.3 cm handheld cubic module integrating a diverse suite of sensors and user inputs. Its hardware formulation comprises an Inertial Measurement Unit (ICM-20498) delivering 3-axis accelerometer, gyroscope, and magnetometer data; four mechanical push-buttons with hardware debounce; a rolling disk using a quadrature mouse-wheel encoder for discrete rotation ($\Delta\theta$); a two-axis analog joystick (continuous $X,Y$ deflections); a PWM-driven linear vibration motor; visual notification via RGB LED; and a 100 mAh Li-Po battery supporting approximately 3 h Bluetooth Low Energy (BLE) operation. The electronics stack utilizes Nordic nRF52832 (ARM Cortex-M4, BLE), structured over three PCBs for control, button input, and connectivity (Liu et al., 22 Jun 2025).

Firmware is developed in C/C++ using the Arduino framework, with all sensors polled at 20 Hz. Sensor data is packetized using Consistent Overhead Byte Stuffing (COBS) and transmitted via BLE to a Python frontend (Bleak library), which parses the BLE stream, decodes the COBS-framed packet, and dispatches structured sensor data to either an AI-driven sonification pipeline or a modular synthesizer setup, specifically VCV Rack, via a TCP interface. This architecture supports both standalone expressive mappings and deep generative audio processes.

2. Signal Processing and Sonification Mappings

MindCube operates as a musical interface and emotion regulation controller via two core sonification mappings (Liu et al., 22 Jun 2025):

• AI-Driven Generative Mapping: User activity, computed as a real-time RMS-style metric based on weighted rolling standard deviations of all 16 sensor signals, serves as a proxy for affective state. Low RMS activity (user is calm) conditions the pipeline to generate more lively music via a latent-diffusion model, while high activity (user agitated) invokes calming music. The pipeline computes the metric, embeds it as a 1024-dimensional vector for cross-attention in a latent diffusion model, produces a latent sequence, and decodes it into audio buffers using a RAVE decoder.
• Non-AI Expressive Mapping: Sensor fusion (accelerometer + gyroscope) infers roll ($\theta$) and pitch ($\phi$) angles through a complementary filter, mapped to VCV Rack control voltages: $\theta$ modulates filter cutoff (100 Hz–5 kHz), $\phi$ controls LFO rate (0.1–10 Hz), encoder steps select sequencer indices, joystick $X,Y$ control stereo pan and modulation index, and button presses gate ADSR envelopes.

Both approaches are implemented with normalized mappings and low-latency streaming (measured round-trip latency: $\sim$1.05 s; 512-step diffusion $\sim$0.90 s, RAVE decode $\sim$0.15 s), sufficient for responsive emotion-regulation feedback.

3. Vision–Language Benchmark for Spatial Mental Modeling

MindCube also denotes a vision–language benchmark comprising 21,154 multiple-choice questions paired with either short videos or sets of images, crafted to systematically evaluate three key capabilities in models: (1) cognitive mapping (allocentric spatial positions of occluded or out-of-view objects), (2) perspective taking (both egocentric and allocentric orientation inference), and (3) mental simulation for "what-if" dynamics after hypothetical scene transformations (Yin et al., 26 Jun 2025, Lian et al., 29 Sep 2025).

Each benchmark instance presents a canonical camera trajectory—Rotation (in-place camera rotation), Around (circular path around object cluster), or Among (camera moving between multiple objects with occlusion events)—along with a question requiring precise spatial reasoning about object relationships, permanence, and re-identification across disjoint observations. All questions are multiple-choice (four to six options), evaluated by exact-match accuracy.

Table 1: MindCube Question Taxonomy

Category	Task Description	Input Modality
Cognitive Mapping	Infer hidden object identity/position from partial views	Images + textual QA
Perspective Taking	Reason from alternate viewpoint (object/human/egocentric)	Images + textual QA
Mental Simulation	Predict post-movement spatial relationships (“what if…?”)	Images + movement spec

4. Experimental Benchmarks, Model Performance, and Analysis

Baseline zero-shot accuracy on MindCube demonstrates significant limitations in existing vision–LLMs (VLMs), with raw-QA models operating near-random (20.4%–38.9% overall), highlighting a persistent challenge in robust spatial mental modeling under partial observability (Lian et al., 29 Sep 2025). Fine-tuning on the Euclid30K geometry surrogate dataset, constructed from a broad set of formal geometry VQA tasks, yields across-the-board improvement: For instance, Qwen2.5VL-3B improves from 20.4% to 38.9% (Δ+18.5 points), while RoboBrain2.0-32B jumps from 29.2% to 38.8%. Most pronounced are gains in Among and Rotation categories, with geometry-tuned models displaying enhanced object consistency, relational inference through occlusion, and more reliable counting when objects are hidden.

Key limitations persist for tasks demanding 3D mental rotation (Rotation) and temporal ordering, attributed to the predominance of 2D geometric content in Euclid30K and absence of temporal cues (Lian et al., 29 Sep 2025).

5. Scaffolding Methods for Enhancing Spatial Reasoning

Multiple approaches are investigated to scaffold spatial reasoning in VLMs (Yin et al., 26 Jun 2025):

• Unseen Intermediate View Generation: Inserting synthetic interpolated views between observed frames produces negligible performance gain (+0.1% accuracy).
• Free-form Chain-of-Thought Reasoning: Prompted stepwise reasoning modestly improves accuracy (to 40.5% from 37.8%).
• Cognitive Maps: Explicit 2D or augmented spatial graphs (object and camera positions/orientations) must be generated by the model (static provision is ineffective); dynamic, model-generated maps, when combined with downstream reasoning, underpin the most significant improvements.
• Map-Then-Reason Paradigm: The highest supervised fine-tuning result (60.8%) follows a pipeline where the model first synthesizes a structured cognitive map from the visual input, then chains reasoning on this intermediate representation to select an answer. Reinforcement learning (RL) using Group Relative Policy Optimization, initialized from “map-then-reason” fine-tuning, pushes performance to 70.7% (Δ+32.9% over baseline).

Fine-tuning only the LLM component (with a frozen visual encoder) yields over 98% of the possible gain, suggesting that the primary challenge is reasoning over spatial structure, not visual perception.

6. Implications for Cognitive Modeling and Spatial Intelligence

MindCube exposes the deficiencies in spatial generalization in state-of-the-art VLMs. Scaffolding internal structured spatial representations—maps that encode allocentric and egocentric relationships and can be manipulated in reasoning chains—produces substantial quantitative and qualitative improvements, closing much of the gap between model and human inference (Yin et al., 26 Jun 2025). Results with geometry-driven surrogate curricula (Euclid30K) confirm that inductive priors about parallelism, angle/distance invariance, and object permanence are transferrable from synthetic domains to dynamic, partially observed scenes such as those in MindCube (Lian et al., 29 Sep 2025).

A plausible implication is that sequential, layered curriculum schedules—progressing from broad formal geometric reasoning to domain-specific spatial navigation—can further enhance spatial intelligence in multimodal agents.

7. Future Directions and Applications

Research directions proposed include:

• Expanding curriculum datasets with temporal and 3D-geometric content to improve mental rotation and ordering skills.
• Introducing hybrid mappings for emotion-centered sonification—blending generative AI output and modular synthesis for richer musical expressivity (Liu et al., 22 Jun 2025).
• Conducting controlled user studies to calibrate mappings between sensor-derived activity and validated affective states, and assess real-world efficacy in emotion regulation.
• Embedding the MindCube interface in VR/AR for context-sensitive soundscapes controlled by embodied gestures.
• Generalizing the “map-then-reason” paradigm to hierarchical, multi-room, or multi-agent spatial tasks, and integrating metric-aware 3D cognitive maps for next-generation scene reasoning (Yin et al., 26 Jun 2025, Lian et al., 29 Sep 2025).

MindCube defines a rigorous testbed for both sensor-driven interactive emotional interfaces and foundational research in spatial mental modeling, with measurable impact demonstrated via quantitative metrics and ablative analysis across multiple model architectures.