Passive Screen-to-Camera Communication (2403.16185v1)
Abstract: A recent technology known as transparent screens is transforming windows into displays. These smart windows are present in buses, airports and offices. They can remain transparent, as a normal window, or display relevant information that overlays their panoramic views. In this paper, we propose transforming these windows not only into screens but also into wireless transmitters. To achieve this goal, we build upon the research area of screen-to-camera communication. In this area, videos are modified in a way that smartphone cameras can decode data out of them, while this data remains invisible to the viewers. A person sees a normal video, but the camera sees the video plus additional information. In this communication method, one of the biggest disadvantages is the traditional screens' power consumption, more than 80% of which is used to generate light. To solve this, we employ novel transparent screens relying on ambient light to display pictures, hence eliminating the power source. However, this comes at the cost of a lower image quality, since they use variable and out-of-control environment light, instead of generating a constant and strong light by LED panels. Our work, dubbed PassiveCam, overcomes the challenge of creating the first screen-to-camera communication link using passive displays. This paper presents two main contributions. First, we analyze and modify existing screens and encoding methods to embed information reliably in ambient light. Second, we develop an Android App that optimizes the decoding process, obtaining a real-time performance. Our evaluation, which considers a musical application, shows a Packet Success Rate (PSR) of close to 90%. In addition, our real-time application achieves response times of 530 ms and 1071 ms when the camera is static and when it is hand-held, respectively.
- K. Qian et al., “Aircode: Hidden screen-camera communication on an invisible and inaudible dual channel,” in USENIX NSDI, 2021.
- K. Zhang et al., “Chromacode: A fully imperceptible screen-camera communication system.” ACM Mobicom, 2018.
- A. Wang et al., “Inframe++: Achieve simultaneous screen-human viewing and hidden screen-camera communication,” in ACM MobiSys, 2015.
- V. Nguyen et al., “High-rate flicker-free screen-camera communication with spatially adaptive embedding,” in IEEE INFOCOM, 2016.
- “Videowindow,” https://www.videowindow.eu/, last accessed: March 11, 2024.
- “Demo, black and white,” https://youtu.be/S_hxk3TQCmI, 2023.
- “Demo, colored,” https://youtu.be/7FoaacEiG60, 2023.
- R. Doherty et al., “Keeping users in the flow: mapping system responsiveness with user experience,” Elsevier Procedia Manufacturing, 2015.
- V. Tran et al., “Deeplight: Robust & unobtrusive real-time screen-camera communication for real-world displays,” in IEEE/ACM IPSN, 2021.
- M. Izz et al., “Uber-in-light: Unobtrusive visible light communication leveraging complementary color channel,” in IEEE INFOCOM, 2016.
- S. Abe et al., “Imperceptible color vibration for screen-camera communication via 2d binary pattern,” in ITE Transactions on Media Technology and Applications, 2020.
- R. Bloom et al., “Luxlink: Creating a wireless link from ambient light,” in ACM SenSys, 2019.
- X. Xu et al., “Enabling practical visible light backscatter communication for battery-free iot applications,” in ACM Mobicom, 2017.
- S. Ammar et al., “Sun-fi: Architecting glass for sunlight data transmission,” IEEE Communications Magazine, 2023.
- Z. Yang et al., “Wearables can afford: Light-weight indoor positioning with visible light,” in ACM MobiSys, 2015.
- J. Li, et al, “Retro-vlc: Enabling battery-free duplex visible light communication for mobile and iot applications,” in HotMobile, 2015.
- S. K. Ghiasi et al., “A principled design for passive light communication,” in ACM Mobicom, 2021.
- M. Chávez Tapia et al., “Sunbox: Screen-to-camera communication with ambient light,” in ACM IMWUT, 2022.