Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
121 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments (2310.09441v1)

Published 13 Oct 2023 in cs.CV, physics.bio-ph, and q-bio.QM

Abstract: Tracking microrobots is challenging, considering their minute size and high speed. As the field progresses towards developing microrobots for biomedical applications and conducting mechanistic studies in physiologically relevant media (e.g., collagen), this challenge is exacerbated by the dense surrounding environments with feature size and shape comparable to microrobots. Herein, we report Motion Enhanced Multi-level Tracker (MEMTrack), a robust pipeline for detecting and tracking microrobots using synthetic motion features, deep learning-based object detection, and a modified Simple Online and Real-time Tracking (SORT) algorithm with interpolation for tracking. Our object detection approach combines different models based on the object's motion pattern. We trained and validated our model using bacterial micro-motors in collagen (tissue phantom) and tested it in collagen and aqueous media. We demonstrate that MEMTrack accurately tracks even the most challenging bacteria missed by skilled human annotators, achieving precision and recall of 77% and 48% in collagen and 94% and 35% in liquid media, respectively. Moreover, we show that MEMTrack can quantify average bacteria speed with no statistically significant difference from the laboriously-produced manual tracking data. MEMTrack represents a significant contribution to microrobot localization and tracking, and opens the potential for vision-based deep learning approaches to microrobot control in dense and low-contrast settings. All source code for training and testing MEMTrack and reproducing the results of the paper have been made publicly available https://github.com/sawhney-medha/MEMTrack.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Real-time optoacoustic tracking of single moving micro-objects in deep phantom and ex vivo tissues. Nano letters, 19(9):6612–6620, 2019.
  2. Medical imaging of microrobots: Toward in vivo applications. ACS nano, 14(9):10865–10893, 2020.
  3. Three dimensional microrobot tracking using learning-based system. International Journal of Control, Automation and Systems, 18:21–28, 2020.
  4. Image analysis driven single-cell analytics for systems microbiology. BMC systems biology, 11:1–21, 2017.
  5. Howard C Berg. Random walks in biology. Princeton University Press, 1993.
  6. Simple online and realtime tracking. In 2016 IEEE International Conference on Image Processing (ICIP), pp.  3464–3468. IEEE, 2016.
  7. Associative image analysis: a method for automated quantification of 3d multi-parameter images of brain tissue. Journal of neuroscience methods, 170(1):165–178, 2008.
  8. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020.
  9. Visual object tracking using adaptive correlation filters. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.  2544–2550, 2010. doi: 10.1109/CVPR.2010.5539960.
  10. Usmicromagset: Using deep learning analysis to benchmark the performance of microrobots in ultrasound images. IEEE Robotics and Automation Letters, 8:1–8, 2023.
  11. Optimizing the restored chemotactic behavior of anticancer agent salmonella enterica serovar typhimurium vnp20009. Journal of biotechnology, 251:76–83, 2017.
  12. Trackpad: Software for semi-automated single-cell tracking and lineage annotation. SoftwareX, 11:100440, 2020. ISSN 2352-7110. doi: https://doi.org/10.1016/j.softx.2020.100440. URL https://www.sciencedirect.com/science/article/pii/S2352711019302390.
  13. Icy: an open bioimage informatics platform for extended reproducible research. Nature methods, 9(7):690–696, 2012.
  14. Bringing trackmate into the era of machine-learning and deep-learning. BioRxiv, pp.  2021–09, 2021.
  15. Multiple dense particle tracking in fluorescence microscopy images based on multidimensional assignment. Journal of structural biology, 173(2):219–228, 2011.
  16. András Frank. On kuhn’s hungarian method—a tribute from hungary. Naval Research Logistics (NRL), 52(1):2–5, 2005.
  17. Laptrack: linear assignment particle tracking with tunable metrics. Bioinformatics, 39(1):btac799, 2023.
  18. 3d tracking of single nanoparticles and quantum dots in living cells by out-of-focus imaging with diffraction pattern recognition. Scientific Reports, 5(1):1–10, 2015.
  19. Ross Girshick. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, pp.  1440–1448, 2015.
  20. Region-based convolutional networks for accurate object detection and segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1):142–158, 2016. doi: 10.1109/TPAMI.2015.2437384.
  21. Cell tracking with deep learning and the viterbi algorithm. In 2018 International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS), pp.  1–6. IEEE, 2018.
  22. Determining optical flow. Artificial Intelligence, 17:185–203, 08 1981. doi: 10.1016/0004-3702(81)90024-2.
  23. Control and autonomy of microrobots: Recent progress and perspective. Advanced Intelligent Systems, 4(5):2100279, 2022.
  24. RE Kalman. A new approach to liner filtering and prediction problems, transaction of asme. Journal of Basic Engineering, 83(1):95–108, 1961.
  25. Bioimagexd: an open, general-purpose and high-throughput image-processing platform. Nature methods, 9(7):683–689, 2012.
  26. Tlm-tracker: software for cell segmentation, tracking and lineage analysis in time-lapse microscopy movies. Bioinformatics, 28(17):2276–2277, 2012.
  27. H. W. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83–97, 1955. doi: https://doi.org/10.1002/nav.3800020109. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/nav.3800020109.
  28. Data-driven statistical modeling of the emergent behavior of biohybrid microrobots. APL Bioengineering, 4(1):016104–016104, 2020.
  29. Micro-rocket robot with all-optic actuating and tracking in blood. Light: Science & Applications, 9(1):84, 2020.
  30. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp.  2980–2988, 2017.
  31. An iterative image registration technique with an application to stereo vision. In IJCAI’81: 7th international joint conference on Artificial intelligence, volume 2, pp.  674–679, 1981.
  32. Cellprofiler 3.0: Next-generation image processing for biology. PLoS biology, 16(7):e2005970, 2018.
  33. Dissection of molecular assembly dynamics by tracking orientation and position of single molecules in live cells. Proceedings of the National Academy of Sciences, 113(42):E6352–E6361, 2016.
  34. Chapter nine - methods for cell and particle tracking. In P. Michael conn (ed.), Imaging and Spectroscopic Analysis of Living Cells, volume 504 of Methods in Enzymology, pp.  183–200. Academic Press, 2012. doi: https://doi.org/10.1016/B978-0-12-391857-4.00009-4. URL https://www.sciencedirect.com/science/article/pii/B9780123918574000094.
  35. Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831, 2016.
  36. Convolutional neural networks automate detection for tracking of submicron-scale particles in 2d and 3d. Proceedings of the National Academy of Sciences, 115(36):9026–9031, 2018. doi: 10.1073/pnas.1804420115. URL https://www.pnas.org/doi/abs/10.1073/pnas.1804420115.
  37. Real-time imaging and tracking of microrobots in tissues using ultrasound phase analysis. Applied Physics Letters, 118(1):014102, 2021.
  38. Tumblescore: Run and tumble analysis for low frame-rate motility videos. BioTechniques, 62:31–36, 01 2017. doi: 10.2144/000114493.
  39. Yolo9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  7263–7271, 2017.
  40. You only look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.  779–788, 2016. doi: 10.1109/CVPR.2016.91.
  41. Single-particle virus tracking. Cold Spring Harbor Protocols, 2011(9):1978–1987, 2011.
  42. I.F. Sbalzarini and P. Koumoutsakos. Feature point tracking and trajectory analysis for video imaging in cell biology. Journal of Structural Biology, 151(2):182–195, 2005. ISSN 1047-8477. doi: https://doi.org/10.1016/j.jsb.2005.06.002. URL https://www.sciencedirect.com/science/article/pii/S1047847705001267.
  43. Fiji: an open-source platform for biological-image analysis. Nature methods, 9(7):676–682, 2012.
  44. NIH image to ImageJ: 25 years of image analysis. Nat. Methods, 9(7):671–675, July 2012.
  45. Ysmr: a video tracking and analysis program for bacterial motility. BMC bioinformatics, 21(1):1–8, 2020.
  46. Medical micro/nanorobots in precision medicine. Advanced Science, 7(21):2002203, 2020.
  47. Deepbacs for multi-task bacterial image analysis using open-source deep learning approaches. Communications Biology, 5(1):688, 2022.
  48. Cellprofiler 4: improvements in speed, utility and usability. BMC bioinformatics, 22:1–11, 2021.
  49. Supersegger: robust image segmentation, analysis and lineage tracking of bacterial cells. Molecular microbiology, 102(4):690–700, 2016.
  50. Introduction to data mining (2nd edition). Pearson Education Limited, 2018.
  51. Trackmate: An open and extensible platform for single-particle tracking. Methods, 115:80–90, 2017. ISSN 1046-2023. doi: https://doi.org/10.1016/j.ymeth.2016.09.016. URL https://www.sciencedirect.com/science/article/pii/S1046202316303346. Image Processing for Biologists.
  52. Deep learning-based 3d magnetic microrobot tracking using 2d mr images. IEEE Robotics and Automation Letters, 7(3):6982–6989, 2022.
  53. Biomanufacturing and self-propulsion dynamics of nanoscale bacteria-enabled autonomous delivery systems. Applied Physics Letters, 105(17):173702, 2014.
  54. Diatrack particle tracking software: Review of applications and performance evaluation. Traffic, 18(12):840–852, 2017.
  55. End-to-end representation learning for correlation filter based tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  2805–2813, 2017.
  56. Dissecting the cell entry pathway of dengue virus by single-particle tracking in living cells. PLoS pathogens, 4(12):e1000244, 2008.
  57. Trends in micro-/nanorobotics: materials development, actuation, localization, and system integration for biomedical applications. Advanced Materials, 33(4):2002047, 2021.
  58. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arxiv 2022. arXiv preprint arXiv:2207.02696, 2022.
  59. Qianqian Wang and Li Zhang. Ultrasound imaging and tracking of micro/nanorobots: From individual to collectives. IEEE Open Journal of Nanotechnology, 1:6–17, 2020.
  60. Image segmentation and dynamic lineage analysis in single-cell fluorescence microscopy. Cytometry Part A: The Journal of the International Society for Advancement of Cytometry, 77(1):101–110, 2010.
  61. Biohybrid robots: recent progress, challenges, and perspectives. Bioinspiration and Biomimetics, 18(1):015001, 2023.
  62. Simple online and realtime tracking with a deep association metric. In 2017 IEEE International Conference on Image Processing (ICIP), pp.  3645–3649. IEEE, 2017.
  63. Automatic tracking of escherichia coli bacteria. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2008: 11th International Conference, New York, NY, USA, September 6-10, 2008, Proceedings, Part I 11, pp.  824–832. Springer, 2008.
  64. Robust and repeatable biofabrication of bacteria-mediated drug delivery systems: Effect of conjugation chemistry, assembly process parameters, and nanoparticle size. Advanced Intelligent Systems, 4(3):2100135, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - sawhney-medha/MEMTrack: Bacteria Detection and Tracking in complex environments such as Collagen (1 star)