Reshaping Inclusive Interpersonal Dynamics through Smart Glasses in Mixed-Vision Social Activities

Abstract: Meaningful social interaction is vital to well-being, yet Blind and Low Vision (BLV) individuals face persistent barriers when collaborating with sighted peers due to inaccessible visual cues. While most wearable assistive technologies emphasize individual tasks, smart glasses introduce opportunities for real-time, contextual support in social settings. To explore how smart glasses affect interpersonal dynamics and support inclusion in mixed-vision groups, we developed a smart glasses-based system, CollabLens, as a technology probe and employed it in four workshop sessions. We found that smart glasses can meaningfully support inclusive collaboration through expanding BLV participants' assistive networks with more flexible, independent access to visual information. While sighted participants viewed smart glasses as a promising medium that fosters interpersonal connection, they revealed uncertainty in adapting their helping behaviors. We concluded by discussing and synthesizing challenges and opportunities for designing smart glasses that provide seamless interaction experiences and enhance reciprocal mixed-vision social inclusion.

• The paper introduces CollabLens, a smart glasses system that integrates large multimodal models to provide BLV users with real-time contextual information for independent social engagement.
• The paper employs mixed-method workshops, including quantitative surveys with high inclusion scores and qualitative interviews, to assess the impact on group dynamics and autonomy.
• The paper highlights challenges such as hardware ergonomics and LMM reliability, underscoring the need for adaptive multimodal interactions in assistive technology.

Reshaping Inclusive Interpersonal Dynamics through Smart Glasses in Mixed-Vision Social Activities

Introduction

The persistent inaccessibility of visual cues in mixed-vision collaboration challenges the participation and inclusion of Blind and Low Vision (BLV) individuals in social activities. "Reshaping Inclusive Interpersonal Dynamics through Smart Glasses in Mixed-Vision Social Activities" (2604.07232) examines the role of smart glasses, integrating Large Multimodal Models (LMMs), in mediating these dynamics. The authors introduce CollabLens, a technology probe designed to facilitate real-time environmental and information access for BLV participants and evaluate its impact through mixed-methods workshop studies.

System Architecture and Interaction Model

CollabLens capitalizes on advancements in LMMs for real-time, context-sensitive environmental description and query-based interaction, implemented on RayNeo X2 smart glasses (Figure 1). The system operates in two primary modes: Context Mode for generic environmental scanning and Query Mode for targeted user queries. Input is triggered via a tactile button, with audio and video streams processed on a local server and responses delivered in audio format through earphones—emphasizing private, non-disruptive feedback.

Figure 1: CollabLens pipeline, featuring video/audio streaming, two user activation modes, LMM-based visual understanding, and feedback through audio output.

This architecture specifically targets hands-free usability, low latency, and multimodal information processing—key requirements for BLV users' seamless integration into collaborative social settings.

Workshop Design and Methodology

The evaluation is grounded in four structured workshop sessions, each with mixed BLV and sighted participants engaging in multi-phase collaborative activities mirroring typical mixed-vision contexts (Figure 2). The tabletop game-based design scaffolds the investigation of group dynamics, agency, and inclusive participation, enabling both quantitative and qualitative data capture.

Figure 2: Workshop procedure includes multi-stage collaborative games representing typical mixed-vision collaborative tasks.

Materials provided include smart glasses, earphones, tactile interaction components, and specialized game assets (Figure 3), ensuring ecological validity and repeatability of observed behaviors.

Figure 3: Session materials comprising the smart glasses prototype, earphones, auxiliary input devices, and collaborative game artifacts.

Findings: Dynamics, Affordances, and Challenges

Quantitative Insights

Comprehensive survey data (PGIS and IRIS subscales) show high mean scores (>4.2/5 on inclusion, >7/9 on responsiveness and supportiveness), with negligible differences between BLV and sighted participants. Both groups perceived smart glasses as supporting group inclusion and positive interpersonal interactions.

Usage Patterns and BLV Agency

Observation and interviews revealed that:

• Primary Utilization: BLV cohorts used smart glasses predominantly for independent access to textual, object, and environmental information, minimizing reliance on sighted peers.
• Social Cue Interpretation: Interest in using smart glasses for non-verbal cue interpretation (expression, gesture) was low—BLV participants typically leveraged alternative, non-visual strategies refined through experience.
• Autonomy and Efficiency: There was a marked preference for scenarios where glass-driven access avoided social burdening or delays imposed by human mediation.

Affordances and Limitations

Participants highlighted:

• Wearability and Social Integration: The form factor contributed to unobtrusiveness and group homogeneity, but hardware limitations (weight, comfort, battery life, camera alignment) impaired longer-term usability.
• Interaction Constraints: Voice input, while efficient, induced social pressure in group contexts. Discreet activation, gesture- or haptics-based triggering, and private audio delivery modes were proposed to improve acceptability.
• Reliability and Latency: LMM inaccuracies (hallucinations, verbosity) and systemic latency disrupted conversational flow and impaired trust—prompting fallback to established support channels.

Mediated Group Dynamics and Reciprocal Inclusion

Smart glasses established a re-negotiated assistance dynamic, shifting BLV participants' roles from help-seeker to co-independent collaborator. Sighted participants, while initially uncertain about when to intervene, noted both diminished help burden and greater mutual engagement. Still, BLV participants stressed that technology alone does not instill inclusion, which relies substantially on interpersonal adaptation and group culture.

Theoretical and Practical Implications

Enriching Assistive Networks

The persistent preference for technology-enabled autonomy, even in collaborative contexts, underscores the continued salience of independence as a foundational value for BLV users—a finding consistent with prior literature on the social costs of help-seeking and the invisible labor of accessibility management [Shinohara & Wobbrock 2011, Branham & Kane 2015]. However, the interdependence framework remains critical: genuine inclusion arises not simply from technological independence but from the flexible orchestration of support relationships and accessible communication modalities [Bennett, Brady, & Branham 2018].

Designing for Seamlessness and Social Alignment

Real-world adoption will require prioritizing:

• Hardware Ergonomics: Optimizing weight, fit, and battery duration, with explicit attention to use cases involving extended wear and varied environments [Danielsson et al. 2020].
• Interaction Modality Diversity: Moving beyond voice, toward gesture, haptic, or intent-aware triggers to minimize social disruption [Kim et al. 2011, Lee et al. 2018].
• LMM Trust Calibration: Addressing hallucinations and non-determinism to reduce information asymmetries that disproportionately affect marginalized users in group contexts [Tang et al. 2025, Adnin & Das 2024].

Reciprocal and Contextual Inclusion

Introducing smart glasses alters not only the participation of BLV individuals but also the expectations and behaviors of sighted peers. For inclusion to be substantive, systems must enable flexible, context-sensitive switching between autonomous and co-dependent modes, support mutual situational awareness, and incorporate signaling to clarify when assistive mediation is active (e.g., via visual indicators or explicit handovers).

Broader Considerations and Limitations

The study’s structured, game-based approach provides strong ecological validity, but longitudinal, in-situ deployments are required to assess adaptation over time and across broader populations (e.g., low-vision, technology-averse individuals, and heterogeneous cultural backgrounds). Privacy and bystander concerns, while not foregrounded here, warrant proactive design—especially as smart glasses move from group-confined to public deployment [Iqbal & Campbell 2023, Zhang et al. 2025].

Future Directions

Extensions of this research should investigate:

• Advanced multimodal interaction paradigms (e.g., gaze, gestures, intent estimation) for minimal social footprint.
• Long-term integration strategies and behavioral adaptation in naturalistic social ecologies.
• Trust-building interventions for LMM-powered assistive devices, including social transparency and explainability [Ehsan et al. 2021, Xu et al. 2023].
• Cross-cultural studies addressing diverse norms on disability, privacy, and technology adoption.

Conclusion

This work empirically substantiates the role of LMM-enabled smart glasses in facilitating flexible, reciprocal, and context-aware participation of BLV individuals in mixed-vision social activities. While current affordances support substantial gains in independent information access and reduce the assistance burden on sighted peers, critical challenges persist: ergonomic limitations, trust in LMM outputs, and the intricacies of social integration. The findings emphasize that inclusion in mixed-vision contexts is not reducible to technological mediation alone, but emerges from the dynamic negotiation of interpersonal relationships, roles, and shared agency. Responsive, adaptive, and unobtrusive design will be central to realizing the full potential of smart glasses as mediators of genuinely inclusive social interaction.

---

References

• (2604.07232): "Reshaping Inclusive Interpersonal Dynamics through Smart Glasses in Mixed-Vision Social Activities"
• Branham & Kane. "The Invisible Work of Accessibility: How Blind Employees Manage Accessibility in Mixed-Ability Workplaces" [10.1145/270(0648.28098)64]
• Bennett, Brady, & Branham. "Interdependence as a Frame for Assistive Technology Research and Design" [ASSETS ‘18]
• Danielsson et al. "Augmented reality smart glasses in industrial assembly: Current status and future challenges" [Journal of Industrial Information Integration, 2020]
• Ehsan et al. "Expanding Explainability: Towards Social Transparency in AI systems" [CHI '21]
• Iqbal & Campbell. "Adopting smart glasses responsibly: potential benefits, ethical, and privacy concerns" [AI and Ethics, 2023]
• Shinohara & Wobbrock. "In the shadow of misperception: Assistive technology use and social interactions" [CHI 2011]
• Xu et al. "XAIR: A Framework of Explainable AI in Augmented Reality" [CHI 2023]
• Tang et al. "Everyday Uncertainty: How Blind People Use GenAI Tools for Information Access" [CHI 2025]
• Adnin & Das. "How Blind People Use and Understand Generative AI Tools" [ASSETS 2024]
• Lee et al. "Interaction Methods for Smart Glasses: A Survey" [IEEE Access, 2018]
• Kim et al. "Designing of multimodal feedback for enhanced multitasking performance" [CHI 2011]