Cam-2-Cam: Exploring the Design Space of Dual-Camera Interactions for Smartphone-based Augmented Reality (2504.20035v3)

Abstract: Off-the-shelf smartphone-based AR systems typically use a single front-facing or rear-facing camera, which restricts user interactions to a narrow field of view and small screen size, thus reducing their practicality. We present Cam-2-Cam, an interaction concept implemented in three smartphone-based AR applications with interactions that span both cameras. Results from our qualitative analysis conducted on 30 participants presented two major design lessons that explore the interaction space of smartphone AR while maintaining critical AR interface attributes like embodiment and immersion: (1) Balancing Contextual Relevance and Feedback Quality serves to outline a delicate balance between implementing familiar interactions people do in the real world and the quality of multimodal AR responses and (2) Preventing Disorientation using Simultaneous Capture and Alternating Cameras which details how to prevent disorientation during AR interactions using the two distinct camera techniques we implemented in the paper. Additionally, we consider observed user assumptions or natural tendencies to inform future implementations of dual-camera setups for smartphone-based AR. We envision our design lessons as an initial pioneering step toward expanding the interaction space of smartphone-based AR, potentially driving broader adoption and overcoming limitations of single-camera AR.

Cam-2-Cam: Exploring Dual-Camera Interactions for Smartphone-Based Augmented Reality

In the paper titled "Cam-2-Cam: Exploring the Design Space of Dual-Camera Interactions for Smartphone-based Augmented Reality," the authors explore augmented reality (AR) applications on smartphones that employ dual-camera configurations. Traditional smartphone AR systems often rely on either the front-facing or rear-facing camera, limiting the interaction potential. This research introduces Cam-2-Cam, a novel interaction concept utilizing both cameras concurrently to broaden the interaction field and enhance user engagement with AR content.

The authors implemented Cam-2-Cam in three distinct smartphone-based AR applications: Face TriggAR, Mouth Craft, and Mirror ThrowAR. Each application leverages different input modalities—face gestures like winking and mouth opening or hand gestures for object manipulation—and examines the interplay between simultaneous and alternating camera captures. These implementations are designed to test the viability and effectiveness of dual-camera setups in creating cohesive and immersive AR experiences on smartphones.

Key Findings

The paper involved a qualitative analysis based on user feedback from thirty participants interacting with these applications. The results highlight two primary design lessons:

1. Balancing Contextual Relevance and Feedback Quality: The authors found that creating AR interfaces that resonate with real-world contexts while providing high-quality feedback can significantly enhance user immersion and engagement. Applications like Face TriggAR and Mirror ThrowAR demonstrated strengths in familiar interactions—users associated gestures with real-world activities such as aiming and throwing—while Mouth Craft, despite its lack of intuitive context, was praised for its immediate multimodal feedback.
2. Preventing Disorientation with Simultaneous and Alternating Capture: Seamless camera transitions that maintain spatial orientation and context between user actions and AR feedback are crucial. The paper suggests that implementing picture-in-picture (PiP) methods or self-viewing interfaces could mitigate disorientation, leading to a sense of connectedness between the spaces captured by each camera.

Implications and Future Directions

The paper opens up a new avenue in smartphone AR research, suggesting the potential for richer interaction spaces without altering device hardware. Expanding the interaction capabilities of smartphones through dual-camera setups could lead to broader adoption and novel applications in areas like gaming, virtual exhibitions, and training environments. The paper emphasizes the importance of understanding natural user strategies, such as mental mapping and self-orientation, to further refine dual-camera interfaces.

Future research could explore the nuances of transitioning between camera views and quantifying responsiveness to enhance user experience and engagement. Moreover, comparisons between dual-camera and single-camera setups could delineate the trade-offs in immersion and functionality, providing further insight into the design of intuitive and immersive AR interfaces.

In conclusion, the Cam-2-Cam concept represents a significant step toward overcoming the limitations of smartphone AR by leveraging dual-camera interactions, enhancing user engagement through immersive and intuitive interfaces. This research provides a foundational framework for developing and implementing dual-camera AR applications, paving the way for more expressive and interactive mobile AR experiences.