Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Atom-Level Optical Chemical Structure Recognition with Limited Supervision (2404.01743v1)

Published 2 Apr 2024 in cs.CV

Abstract: Identifying the chemical structure from a graphical representation, or image, of a molecule is a challenging pattern recognition task that would greatly benefit drug development. Yet, existing methods for chemical structure recognition do not typically generalize well, and show diminished effectiveness when confronted with domains where data is sparse, or costly to generate, such as hand-drawn molecule images. To address this limitation, we propose a new chemical structure recognition tool that delivers state-of-the-art performance and can adapt to new domains with a limited number of data samples and supervision. Unlike previous approaches, our method provides atom-level localization, and can therefore segment the image into the different atoms and bonds. Our model is the first model to perform OCSR with atom-level entity detection with only SMILES supervision. Through rigorous and extensive benchmarking, we demonstrate the preeminence of our chemical structure recognition approach in terms of data efficiency, accuracy, and atom-level entity prediction.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Rdkit: Open-source cheminformatics. accessed on 01.02.2022.
  2. Rethinking class activation mapping for weakly supervised object localization. In European Conference on Computer Vision, pages 618–634. Springer, 2020.
  3. Bounding boxes, segmentations and object coordinates: How important is recognition for 3d scene flow estimation in autonomous driving scenarios? In Proceedings of the IEEE International Conference on Computer Vision, pages 2574–2583, 2017.
  4. Decimer - hand-drawn molecule images dataset, 2022.
  5. Img2mol–accurate smiles recognition from molecular graphical depictions. Chemical science, 12(42):14174–14181, 2021.
  6. Weakly supervised localization and learning with generic knowledge. International journal of computer vision, 100(3):275–293, 2012.
  7. Optical structure recognition software to recover chemical information: Osra, an open source solution, 2009.
  8. The chembl database in 2017. Nucleic acids research, 45(D1):D945–D954, 2017.
  9. Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440–1448, 2015.
  10. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014.
  11. Road-r: The autonomous driving dataset with logical requirements. arXiv preprint arXiv:2210.01597, 2022.
  12. Content-based image retrieval: A review of recent trends. Cogent Engineering, 8(1):1927469, 2021.
  13. Cede: A collection of expert-curated datasets with atom-level entity annotations for optical chemical structure recognition.
  14. Cede: A collection of expert-curated datasets with atom-level entity annotations for optical chemical structure recognition. Advances in Neural Information Processing Systems, 35:27114–27126, 2022.
  15. Cross-domain weakly-supervised object detection through progressive domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5001–5009, 2018.
  16. Weakly supervised object localization with progressive domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3512–3520, 2016.
  17. Stereoisomerism and local chirality. Journal of the American Chemical Society, 106(11):3319–3328, 1984.
  18. H. L. Morgan. The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service. Journal of Chemical Documentation, 5(2):107–113, 1965.
  19. Chemgrapher: optical graph recognition of chemical compounds by deep learning. Journal of chemical information and modeling, 60(10):4506–4517, 2020.
  20. Self-labeling of fully mediating representations by graph alignment. In Benelux Conference on Artificial Intelligence, pages 46–65. Springer, 2021.
  21. Weakly supervised knowledge transfer with probabilistic logical reasoning for object detection. In The Eleventh International Conference on Learning Representations (ICLR), 2023.
  22. Indigo: universal cheminformatics api. Journal of cheminformatics, 3(Suppl 1):P4, 2011.
  23. Molscribe: Robust molecular structure recognition with image-to-graph generation. Journal of Chemical Information and Modeling, 63(7):1925–1934, 2023.
  24. Decimer: towards deep learning for chemical image recognition. Journal of Cheminformatics, 12(1):1–9, 2020.
  25. Kohulan Rajan et al. Decimer.ai: an open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications. Nat Commun, 14(5045), 2023.
  26. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.
  27. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28, 2015.
  28. Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 50(5):742–754, 2010. PMID: 20426451.
  29. On learning to localize objects with minimal supervision. In International Conference on Machine Learning, pages 1611–1619. PMLR, 2014.
  30. Taffee T Tanimoto. Elementary mathematical theory of classification and prediction. 1958.
  31. Art2real: Unfolding the reality of artworks via semantically-aware image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5849–5859, 2019.
  32. Revisiting knowledge transfer for training object class detectors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1101–1110, 2018.
  33. David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31–36, 1988.
  34. Chempix: automated recognition of hand-drawn hydrocarbon structures using deep learning. Chem. Sci., 12:10622–10633, 2021.
  35. Boosting weakly supervised object detection with progressive knowledge transfer. In European conference on computer vision, pages 615–631. Springer, 2020.
  36. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016.
  37. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020.
  38. Object detection in 20 years: A survey. Proceedings of the IEEE, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Martijn Oldenhof (6 papers)
  2. Edward De Brouwer (21 papers)
  3. Adam Arany (17 papers)
  4. Yves Moreau (26 papers)
Citations (1)
View on Semantic Scholar