Papers
Topics
Authors
Recent
Search
2000 character limit reached

LensLeech: On-Lens Interaction for Arbitrary Camera Devices

Published 30 Jun 2023 in cs.HC | (2307.00152v1)

Abstract: Cameras provide a vast amount of information at high rates and are part of many specialized or general-purpose devices. This versatility makes them suitable for many interaction scenarios, yet they are constrained by geometry and require objects to keep a minimum distance for focusing. We present the LensLeech, a soft silicone cylinder that can be placed directly on or above lenses. The clear body itself acts as a lens to focus a marker pattern from its surface into the camera it sits on. This allows us to detect rotation, translation, and deformation-based gestures such as pressing or squeezing the soft silicone. We discuss design requirements, describe fabrication processes, and report on the limitations of such on-lens widgets. To demonstrate the versatility of LensLeeches, we built prototypes to show application examples for wearable cameras, smartphones, and interchangeable-lens cameras, extending existing devices by providing both optical input and output for new functionality.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Alexander C. Abad and Anuradha Ranasinghe. 2020. Visuotactile Sensors With Emphasis on GelSight Sensor: A Review. IEEE Sensors Journal 20, 14 (July 2020), 7628–7638. https://doi.org/10.1109/JSEN.2020.2979662
  2. Patrick Baudisch and Gerry Chu. 2009. Back-of-Device Interaction Allows Creating Very Small Touch Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1923–1932. https://doi.org/10.1145/1518701.1518995
  3. Gary Bradski. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000).
  4. Issues and Techniques in Touch-Sensitive Tablet Input. In Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’85). Association for Computing Machinery, New York, NY, USA, 215–224. https://doi.org/10.1145/325334.325239
  5. Google Cardboard. 2014. Website. Retrieved Apr 5, 2023 from https://arvr.google.com/cardboard/.
  6. Improved GelSight Tactile Sensor for Measuring Geometry and Slip. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 137–144. https://doi.org/10.1109/IROS.2017.8202149
  7. GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 1927–1934. https://doi.org/10.1109/IROS.2018.8593661
  8. deForm: An Interactive Malleable Surface for Capturing 2.5D Arbitrary Objects, Tools and Touch. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). Association for Computing Machinery, New York, NY, USA, 527–536. https://doi.org/10.1145/2047196.2047265
  9. Optical Pressure Sensing for Tangible User Interfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS ’11). Association for Computing Machinery, New York, NY, USA, 45–48. https://doi.org/10.1145/2076354.2076362
  10. An Object Profile Detection by a High Resolution Tactile Sensor Using an Optical Conductive Plate. In Proceedings.14 Annual Conference of Industrial Electronics Society, Vol. 4. 982–987. https://doi.org/10.1109/IECON.1988.666278
  11. ISO 8995-1:2002 2002. Lighting of work places – Indoor: Part 1. Standard. International Organization for Standardization, Geneva, CH.
  12. Micah K. Johnson and Edward H. Adelson. 2009. Retrographic Sensing for the Measurement of Surface Texture and Shape. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 1070–1077. https://doi.org/10.1109/CVPR.2009.5206534
  13. Wolfgang Kabsch. 1976. A Solution for the Best Rotation to Relate Two Sets of Vectors. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography 32, 5 (Sept. 1976), 922–923. https://doi.org/10.1107/S0567739476001873
  14. Evaluation of a Vision-Based Tactile Sensor. In IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA ’04. 2004, Vol. 2. 1542–1547 Vol.2. https://doi.org/10.1109/ROBOT.2004.1308043
  15. Takuya Kitade and Wataru Yamada. 2019. Prismodule: Modular UI for Smartphones Using Internal Reflection. In The Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST ’19). Association for Computing Machinery, New York, NY, USA, 119–121. https://doi.org/10.1145/3332167.3356892
  16. Nintendo Labo. 2018. Website. Retrieved Apr 5, 2023 from https://web.archive.org/web/20180117220613/https://labo.nintendo.com/.
  17. DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor With Application to In-Hand Manipulation. IEEE Robotics and Automation Letters 5, 3 (July 2020), 3838–3845. https://doi.org/10.1109/LRA.2020.2977257
  18. Attaching Objects to Smartphones Back Side for a Modular Interface. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST ’17). Association for Computing Machinery, New York, NY, USA, 51–52. https://doi.org/10.1145/3131785.3131810
  19. Bokode: Imperceptible Visual Tags for Camera Based Interaction from a Distance. In ACM SIGGRAPH 2009 Papers (SIGGRAPH ’09). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/1576246.1531404
  20. Vision Based Tactile Sensor Using Transparent Elastic Fingertip for Dexterous Handling. In Mobile Robots: Perception & Navigation, Sascha Kolski (Ed.). Pro Literatur Verlag, Germany / ARS, Austria. https://doi.org/10.5772/4771
  21. Nobuyuki Otsu. 1979. A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on Systems, Man, and Cybernetics 9, 1 (Jan. 1979), 62–66. https://doi.org/10.1109/TSMC.1979.4310076
  22. Mats Petter Petterson and Tomas Edso. 2003. Patent US-6548768-B1: Determination of a Position Code.
  23. DeforMe: Projection-Based Visualization of Deformable Surfaces Using Invisible Textures. In SIGGRAPH Asia 2013 Emerging Technologies (SA ’13). Association for Computing Machinery, New York, NY, USA, 1–3. https://doi.org/10.1145/2542284.2542292
  24. Ariadna Quattoni and Antonio Torralba. 2009. Recognizing Indoor Scenes. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 413–420. https://doi.org/10.1109/CVPR.2009.5206537
  25. PhotoelasticTouch: Transparent Rubbery Tangible Interface Using an LCD and Photoelasticity. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST ’09). Association for Computing Machinery, New York, NY, USA, 43–50. https://doi.org/10.1145/1622176.1622185
  26. Sauron: Embedded Single-Camera Sensing of Printed Physical User Interfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ’13). Association for Computing Machinery, New York, NY, USA, 447–456. https://doi.org/10.1145/2501988.2501992
  27. Flexibles: Deformation-Aware 3D-Printed Tangibles for Capacitive Touchscreens. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1001–1014. https://doi.org/10.1145/3025453.3025663
  28. Dothraki: Tracking Tangibles Atop Tabletops Through De-Bruijn Tori. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’21). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3430524.3440656
  29. Carmelo Sferrazza and Raffaello D’Andrea. 2019. Design, Motivation and Evaluation of a Full-Resolution Optical Tactile Sensor. Sensors 19, 4 (Jan. 2019), 928. https://doi.org/10.3390/s19040928
  30. Kazuhiro Shimonomura. 2019. Tactile Image Sensors Employing Camera: A Review. Sensors 19, 18 (Jan. 2019), 3933. https://doi.org/10.3390/s19183933
  31. Fat Finger Worries: How Older and Younger Users Physically Interact with PDAs. In Human-Computer Interaction - INTERACT 2005 (Lecture Notes in Computer Science), Maria Francesca Costabile and Fabio Paternò (Eds.). Springer, Berlin, Heidelberg, 267–280. https://doi.org/10.1007/11555261_24
  32. GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger. arXiv:2103.12269 [cs] (March 2021). arXiv:2103.12269 [cs]
  33. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17 (2020), 261–272. https://doi.org/10.1038/s41592-019-0686-2
  34. ClipWidgets: 3D-printed Modular Tangible UI Extensions for Smartphones. In Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’22). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3490149.3501314
  35. GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger. In

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up