ThinTact:Thin Vision-Based Tactile Sensor by Lensless Imaging

Abstract: Vision-based tactile sensors have drawn increasing interest in the robotics community. However, traditional lens-based designs impose minimum thickness constraints on these sensors, limiting their applicability in space-restricted settings. In this paper, we propose ThinTact, a novel lensless vision-based tactile sensor with a sensing field of over 200 mm2 and a thickness of less than 10 mm.ThinTact utilizes the mask-based lensless imaging technique to map the contact information to CMOS signals. To ensure real-time tactile sensing, we propose a real-time lensless reconstruction algorithm that leverages a frequency-spatial-domain joint filter based on discrete cosine transform (DCT). This algorithm achieves computation significantly faster than existing optimization-based methods. Additionally, to improve the sensing quality, we develop a mask optimization method based on the generic algorithm and the corresponding system matrix calibration algorithm.We evaluate the performance of our proposed lensless reconstruction and tactile sensing through qualitative and quantitative experiments. Furthermore, we demonstrate ThinTact's practical applicability in diverse applications, including texture recognition and contact-rich object manipulation. The paper will appear in the IEEE Transactions on Robotics: https://ieeexplore.ieee.org/document/10842357. Video: https://youtu.be/YrOO9BDMAHo

• The paper introduces ThinTact, a novel vision-based tactile sensor utilizing lensless imaging and a fast reconstruction algorithm to achieve sub-10mm thickness and a large sensing area.
• Experimental validation demonstrates ThinTact's performance with 0.18 mm lateral resolution, improved depth accuracy via Real2Sim transfer, and capability in complex manipulation tasks.
• ThinTact's design enables tactile sensing integration in space-constrained environments, offering significant potential for enhancing robotic manipulation capabilities.

ThinTact: A Thin, Lensless Vision-Based Tactile Sensor

The paper introduces a novel tactile sensor design termed "ThinTact," addressing a significant constraint in vision-based tactile sensors: thickness. Traditional tactile sensors, utilizing lens-based designs, impose a minimum thickness due to the lens requirements, limiting their application in scenarios with spatial constraints. ThinTact emerges as a solution, employing lensless imaging to achieve a thickness of less than 10 mm while maintaining a large sensing field exceeding 200 mm², thus enhancing its feasibility in tight operational environments.

Technical Contributions and Methodology

ThinTact implements a mask-based lensless imaging system, transferring contact information from the sensor's surface into CMOS signals without the use of lenses. This design choice not only reduces thickness but also eradicates the dependencies between working distance and field of view (FOV) found in conventional lens-based systems. A real-time reconstruction algorithm, grounded in frequency-spatial-domain joint filtering via discrete cosine transform (DCT), is introduced to quickly process these signals into usable tactile data. This algorithm is notably faster than prevalent optimization-based methods.

Additionally, the paper explores mask optimization via a genetic algorithm alongside a system matrix calibration algorithm to enhance sensing accuracy and robustness. The optimized mask shows superior performance in scene reconstruction, offering improvements in image uniformity and noise management compared to Maximum-Length-Sequence-based masks.

Experimental Validation

A series of qualitative and quantitative evaluations validate ThinTact's functionality. Resolution tests with USAF-1951 resolution charts indicate a lateral resolution threshold of 0.18 mm. Moreover, depth accuracy is assessed through a series of indentation experiments, which benefit significantly from a Real2Sim transfer process, improving depth measurement accuracy markedly compared to direct real-sensor outputs.

In practical terms, ThinTact's utility is demonstrated through texture recognition tasks and complex object manipulations, such as handling delicate items like fragile chips and pencil leads, which highlight the sensor’s high sensitivity and resolution capabilities. Additionally, ThinTact proves effective in scenarios demanding finesse, such as test tube insertions and the manipulation of flat plates and drawers, areas where traditional thick sensors would struggle.

Implications and Future Directions

From a theoretical perspective, ThinTact’s innovation lies in applying lensless imaging to tactile sensors, challenging traditional sensor design norms and opening avenues for embedding tactile sensors in previously inaccessible applications. Practically, its integration into robots could enhance manipulation capabilities in cluttered and space-constrained environments.

Future developments could explore reductions in CMOS thickness for even slimmer designs, enhancing integration possibilities further. Additionally, real-time processing efficiencies could continue to improve, potentially involving the exploration of event-based sensors to overcome current data constraint challenges. The combination of multiple sensing modalities could also be investigated to enhance dynamic response capabilities.

Overall, ThinTact marks a substantial step towards more adaptable and versatile tactile sensing technologies, offering significant potential for advancing robotic interaction capabilities in complex environments.