Learning Deformable 3D Graph Similarity to Track Plant Cells in Unregistered Time Lapse Images
Abstract: Tracking of plant cells in images obtained by microscope is a challenging problem due to biological phenomena such as large number of cells, non-uniform growth of different layers of the tightly packed plant cells and cell division. Moreover, images in deeper layers of the tissue being noisy and unavoidable systemic errors inherent in the imaging process further complicates the problem. In this paper, we propose a novel learning-based method that exploits the tightly packed three-dimensional cell structure of plant cells to create a three-dimensional graph in order to perform accurate cell tracking. We further propose novel algorithms for cell division detection and effective three-dimensional registration, which improve upon the state-of-the-art algorithms. We demonstrate the efficacy of our algorithm in terms of tracking accuracy and inference-time on a benchmark dataset.
- Z. Orynbayeva, R. Sensenig, and B. Polyak, “Metabolic and structural integrity of magnetic nanoparticle-loaded primary endothelial cells for targeted cell therapy,” Nanomedicine, vol. 10, no. 10, pp. 1555–1568, 2015.
- S. C. Fry, “Primary cell wall metabolism: tracking the careers of wall polymers in living plant cells,” New phytologist, vol. 161, no. 3, pp. 641–675, 2004.
- H. Shen, G. Nelson, D. E. Nelson, S. Kennedy, D. G. Spiller, T. Griffiths, N. Paton, S. G. Oliver, M. R. White, and D. B. Kell, “Automated tracking of gene expression in individual cells and cell compartments,” Journal of The Royal Society Interface, vol. 3, no. 11, pp. 787–794, 2006.
- K.-K. Liu, C.-C. Wang, C.-L. Cheng, and J.-I. Chao, “Endocytic carboxylated nanodiamond for the labeling and tracking of cell division and differentiation in cancer and stem cells,” Biomaterials, vol. 30, no. 26, pp. 4249–4259, 2009.
- Y. Refahi, A. Zardilis, G. Michelin, R. Wightman, B. Leggio, J. Legrand, E. Faure, L. Vachez, A. Armezzani, A.-E. Risson et al., “A multiscale analysis of early flower development in arabidopsis provides an integrated view of molecular regulation and growth control,” Developmental Cell, vol. 56, no. 4, pp. 540–556, 2021.
- K. Koch, B. Bhushan, and W. Barthlott, “Diversity of structure, morphology and wetting of plant surfaces,” Soft Matter, vol. 4, no. 10, pp. 1943–1963, 2008.
- H. Zhang, H. Rong, and D. Pilbeam, “Signalling mechanisms underlying the morphological responses of the root system to nitrogen in arabidopsis thaliana,” Journal of experimental botany, vol. 58, no. 9, pp. 2329–2338, 2007.
- E. Shani, O. Yanai, and N. Ori, “The role of hormones in shoot apical meristem function,” Current opinion in plant biology, vol. 9, no. 5, pp. 484–489, 2006.
- S. W. Paddock, “Confocal laser scanning microscopy,” Biotechniques, vol. 27, no. 5, pp. 992–1004, 1999.
- L. Willis, Y. Refahi, R. Wightman, B. Landrein, J. Teles, K. C. Huang, E. M. Meyerowitz, and H. Jönsson, “Cell size and growth regulation in the arabidopsis thaliana apical stem cell niche,” Proceedings of the National Academy of Sciences, vol. 113, no. 51, pp. E8238–E8246, 2016.
- A. Gupta, A. Aich, K. Rodriguez, G. V. Reddy, and A. K. Roy-Chowdhury, “Deep quantized representation for enhanced reconstruction,” in 2020 IEEE 17th International Symposium on Biomedical Imaging Workshops (ISBI Workshops), 2020, pp. 1–4.
- B. MiCHAEL pOLLAN, “The intelligent plant,” New Yorker, p. 93, 2013.
- W. Jiang, L. Wu, S. Liu, and M. Liu, “Cnn-based two-stage cell segmentation improves plant cell tracking,” Pattern Recognition Letters, vol. 128, pp. 311–317, 2019.
- M. Liu, Y. Liu, W. Qian, and Y. Wang, “Deepseed local graph matching for densely packed cells tracking,” IEEE/ACM transactions on computational biology and bioinformatics, vol. 18, no. 3, pp. 1060–1069, 2019.
- M. Liu, A. K. Roy-Chowdhury, and G. V. Reddy, “Robust estimation of stem cell lineages using local graph matching,” in 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 2009, pp. 194–201.
- M. Liu, P. Xiang, and G. Liu, “Robust plant cell tracking using local spatio-temporal context,” Neurocomputing, vol. 208, pp. 309–314, 2016.
- V. E. Mendia and D. Sarkar, “Optimal broadcasting on the star graph,” IEEE Transactions on Parallel & Distributed Systems, vol. 3, no. 04, pp. 389–396, 1992.
- A. Chakraborty and A. K. Roy-Chowdhury, “Context aware spatio-temporal cell tracking in densely packed multilayer tissues,” Medical image analysis, vol. 19, no. 1, pp. 149–163, 2015.
- C. Sutton, A. McCallum et al., “An introduction to conditional random fields,” Foundations and Trends® in Machine Learning, vol. 4, no. 4, pp. 267–373, 2012.
- A. T. Ihler, J. W. Fisher III, A. S. Willsky, and D. M. Chickering, “Loopy belief propagation: convergence and effects of message errors.” Journal of Machine Learning Research, vol. 6, no. 5, 2005.
- M. Liu, Y. He, W. Qian, Y. Wei, and X. Liu, “Cell population tracking in a honeycomb structure using an imm filter based 3d local graph matching model,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 15, no. 5, pp. 1706–1717, 2017.
- M. Liu, Y. Liu, and W. Qian, “A multi-seed 3d local graph matching model for tracking of densely packed cells,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 1075–1079.
- Y. Xie, M. Liu, S. Zhou, and Y. Wang, “A deep local patch matching network for cell tracking in microscopy image sequences without registration,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 19, no. 6, pp. 3202–3212, 2022.
- H. Zhu, D. Liu, S. Zhang, Y. Zhu, L. Teng, and S. Teng, “Solving the many to many assignment problem by improving the kuhn–munkres algorithm with backtracking,” Theoretical Computer Science, vol. 618, pp. 30–41, 2016.
- S. Beucher and F. Meyer, “The morphological approach to segmentation: the watershed transformation,” Mathematical morphology in image processing, vol. 34, no. 1993, p. 49, 1993.
- D. Eschweiler, M. Rethwisch, S. Koppers, and J. Stegmaier, “Spherical harmonics for shape-constrained 3d cell segmentation,” in 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE, 2021, pp. 792–796.
- D. Eschweiler, T. V. Spina, R. C. Choudhury, E. Meyerowitz, A. Cunha, and J. Stegmaier, “Cnn-based preprocessing to optimize watershed-based cell segmentation in 3d confocal microscopy images,” in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019, pp. 223–227.
- D. Eschweiler, R. S. Smith, and J. Stegmaier, “Robust 3d cell segmentation: Extending the view of cellpose,” in 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 2022, pp. 191–195.
- K. Mkrtchyan, A. Chakraborty, and A. K. Roy-Chowdhury, “Automated registration of live imaging stacks of arabidopsis,” in 2013 IEEE 10th International Symposium on Biomedical Imaging. IEEE, 2013, pp. 672–675.
- R. B. Rusu, N. Blodow, and M. Beetz, “Fast point feature histograms (fpfh) for 3d registration,” in 2009 IEEE international conference on robotics and automation. IEEE, 2009, pp. 3212–3217.
- Z. Zhang and W. S. Lee, “Deep graphical feature learning for the feature matching problem,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 5087–5096.
- D. Mazia, “Mitosis and the physiology of cell division,” in The cell. Elsevier, 1961, pp. 77–412.
- S. Basu, D. Bhattacharyya, and R. Banerjee, “Mapping the distribution of packing topologies within protein interiors shows predominant preference for specific packing motifs,” BMC bioinformatics, vol. 12, no. 1, pp. 1–26, 2011.
- A.-A. Liu, J. Tang, W. Nie, and Y. Su, “Multi-grained random fields for mitosis identification in time-lapse phase contrast microscopy image sequences,” IEEE transactions on medical imaging, vol. 36, no. 8, pp. 1699–1710, 2017.
- Q. Chen, L. Sun, E. Cheung, and A. L. Yuille, “Every view counts: Cross-view consistency in 3d object detection with hybrid-cylindrical-spherical voxelization,” Advances in Neural Information Processing Systems, vol. 33, pp. 21 224–21 235, 2020.
- D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497.
- C. Kim, F. Li, and J. M. Rehg, “Multi-object tracking with neural gating using bilinear lstm,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 200–215.
- P. Voigtlaender, M. Krause, A. Osep, J. Luiten, B. B. G. Sekar, A. Geiger, and B. Leibe, “Mots: Multi-object tracking and segmentation,” in Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 2019, pp. 7942–7951.
- J. Peng, C. Wang, F. Wan, Y. Wu, Y. Wang, Y. Tai, C. Wang, J. Li, F. Huang, and Y. Fu, “Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer, 2020, pp. 145–161.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.