Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Evaluating the method reproducibility of deep learning models in the biodiversity domain (2407.07550v1)

Published 10 Jul 2024 in cs.IR

Abstract: AI is revolutionizing biodiversity research by enabling advanced data analysis, species identification, and habitats monitoring, thereby enhancing conservation efforts. Ensuring reproducibility in AI-driven biodiversity research is crucial for fostering transparency, verifying results, and promoting the credibility of ecological findings.This study investigates the reproducibility of deep learning (DL) methods within the biodiversity domain. We design a methodology for evaluating the reproducibility of biodiversity-related publications that employ DL techniques across three stages. We define ten variables essential for method reproducibility, divided into four categories: resource requirements, methodological information, uncontrolled randomness, and statistical considerations. These categories subsequently serve as the basis for defining different levels of reproducibility. We manually extract the availability of these variables from a curated dataset comprising 61 publications identified using the keywords provided by biodiversity experts. Our study shows that the dataset is shared in 47% of the publications; however, a significant number of the publications lack comprehensive information on deep learning methods, including details regarding randomness.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (85)
  1. Biodivnere: Gold standard corpora for named entity recognition and relation extraction in the biodiversity domain. Biodiversity Data Journal, 10.
  2. How reproducible are the results gained with the help of deep learning methods in biodiversity research? Biodiversity Information Science and Standards, 7.
  3. An automated deep learning based satellite imagery analysis for ecology management. Ecol. Inform., 66(101452):101452.
  4. Integrating multi-sensors data for species distribution mapping using deep learning and envelope models. Remote Sens. (Basel), 13(16):3284.
  5. An alternative approach for mapping burn scars using landsat imagery, google earth engine, and deep learning in the brazilian savanna. Remote Sens. Appl. Soc. Environ., 22(100472):100472.
  6. AI naturalists might hold the key to unlocking biodiversity data in social media imagery. Patterns, 1(7):100116.
  7. Vegetation detection using deep learning and conventional methods. Remote Sens. (Basel), 12(15):2502.
  8. Country-wide retrieval of forest structure from optical and SAR satellite imagery with deep ensembles.
  9. An automated light trap to monitor moths (lepidoptera) using computer vision-based tracking and deep learning. Sensors (Basel), 21(2):343.
  10. Taxonomic classification of ants (formicidae) from images using deep learning. bioRxiv, page 407452.
  11. A deep learning approach to species distribution modelling. In Multimedia Tools and Applications for Environmental & Biodiversity Informatics, pages 169–199. Springer International Publishing, Cham.
  12. Predicting animal behaviour using deep learning: GPS data alone accurately predict diving in seabirds. Methods Ecol. Evol., 9(3):681–692.
  13. Understanding deep learning in land use classification based on sentinel-2 time series. Sci. Rep., 10(1):17188.
  14. Modelling animal biodiversity using acoustic monitoring and deep learning. In 2021 International Joint Conference on Neural Networks (IJCNN). IEEE.
  15. Automatic standardized processing and identification of tropical bat calls using deep learning approaches. Biol. Conserv., 241(108269):108269.
  16. Mapping potential plant species richness over large areas with deep learning, MODIS, and species distribution models. Remote Sens. (Basel), 13(13):2490.
  17. Applications for deep learning in ecology. Methods in Ecology and Evolution, 10(10):1632–1644.
  18. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46.
  19. Extreme deep learning in biosecurity: the case of machine hearing for marine species identification. J. Inf. Telecommun., 2(4):492–510.
  20. Pollen analysis using multispectral imaging flow cytometry and deep learning. New Phytol., 229(1):593–606.
  21. Camera assisted roadside monitoring for invasive alien plant species using deep learning. Sensors (Basel), 21(18):6126.
  22. Deep learning pipeline. Apress: Berkeley, CA, USA.
  23. CityNet—Deep learning tools for urban ecoacoustic assessment. Methods Ecol. Evol., 10(2):186–197.
  24. A checklist for maximizing reproducibility of ecological niche models. Nature Ecology & Evolution, 3(10):1382–1395.
  25. Image-based taxonomic classification of bulk biodiversity samples using deep learning and domain adaptation.
  26. Trade-off between deep learning for species identification and inference about predator-prey co-occurrence: Reproducible R workflow integrating models in computer vision and ecological statistics.
  27. What does research reproducibility mean? Science Translational Medicine, 8(341):341ps12–341ps12.
  28. GPAI (2022). Biodiversity and artificial intelligence, opportunities and recommendations report.
  29. Tree cover estimation in global drylands from space using deep learning. Remote Sens. (Basel), 12(3):343.
  30. Deep-Learning convolutional neural networks for scattered shrub detection with google earth imagery.
  31. Whale counting in satellite and aerial images with deep learning. Sci. Rep., 9(1):14259.
  32. Do machine learning platforms provide out-of-the-box reproducibility? Future Generation Computer Systems, 126:34–47.
  33. Reproducibility standards for machine learning in the life sciences. Nature Methods, 18(10):1132–1135.
  34. Heredia, I. (2017). Large-scale plant classification with deep neural networks. In Proceedings of the Computing Frontiers Conference, New York, NY, USA. ACM.
  35. BatNet++: A robust deep learning-based predicting models for calls recognition. In 2020 5th International Conference on Smart Grid and Electrical Automation (ICSGEA). IEEE.
  36. Recognition of endemic bird species using deep learning models. IEEE Access, 9:102975–102984.
  37. Automated extraction of phenotypic leaf traits of individual intact herbarium leaves from herbarium specimen images using deep learning based semantic segmentation. Sensors (Basel), 21(13):4549.
  38. Reconstruction of damaged herbarium leaves using deep learning techniques for improving classification accuracy. Ecol. Inform., 61(101243):101243.
  39. Bag of features (BoF) based deep learning framework for bleached corals detection. Big Data Cogn. Comput., 5(4):53.
  40. Fish classification using DNA barcode sequences through deep learning method. Symmetry (Basel), 13(9):1599.
  41. Deep learning improves acoustic biodiversity monitoring and new candidate forest frog species identification (genus platymantis) in the Philippines. Biodivers. Conserv., 30(3):643–657.
  42. Automatic windthrow detection using very-high-resolution satellite imagery and deep learning. Remote Sens. (Basel), 12(7):1145.
  43. Deep learning for large scale biodiversity monitoring. In Bloomberg Data for Good Exchange Conference.
  44. From human experts to machines: An llm supported approach to ontology and knowledge graph construction. arXiv preprint arXiv:2403.08345.
  45. Leonelli, S. (2018). Rethinking reproducibility as a criterion for research quality. In Including a symposium on Mary Morgan: curiosity, imagination, and surprise, volume 36, pages 129–146. Emerald Publishing Limited.
  46. A novel deep learning based approach for seed image classification and retrieval. Comput. Electron. Agric., 187(106269):106269.
  47. Columnar cactus recognition in aerial images using a deep learning approach. Ecol. Inform., 52:131–138.
  48. Bat detective—deep learning tools for bat acoustic signal detection. PLoS Comput. Biol., 14(3):e1005995.
  49. Automatic annotation of coral reefs using deep learning. In OCEANS 2016 MTS/IEEE Monterey. IEEE.
  50. A continental-scale assessment of density, size, distribution and historical trends of farm dams using deep learning convolutional neural networks. Remote Sens. (Basel), 13(2):319.
  51. Semantic segmentation of tree-canopy in urban environment with pixel-wise deep learning. Remote Sens. (Basel), 13(16):3054.
  52. Deep learning for species identification of modern and fossil rodent molars.
  53. Using deep learning for image-based plant disease detection. Front. Plant Sci., 7:1419.
  54. Hierarchical mapping of brazilian savanna (cerrado) physiognomies based on deep learning. J. Appl. Remote Sens., 15(04).
  55. Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. Proceedings of the National Academy of Sciences, 115(25):E5716–E5725.
  56. Evaluation of deep learning techniques for deforestation detection in the brazilian amazon and cerrado biomes from remote sensing imagery. Remote Sens. (Basel), 12(6):910.
  57. Improving reproducibility in machine learning research (a report from the neurips 2019 reproducibility program). The Journal of Machine Learning Research, 22(1):7459–7478.
  58. Potamitis, I. (2016). Deep learning for detection of bird vocalisations. arXiv preprint arXiv:1609.08408.
  59. Raff, E. (2019). A step toward quantifying independently reproducible machine learning research. Advances in Neural Information Processing Systems, 32.
  60. Identifying land patterns from satellite imagery in amazon rainforest using deep learning.
  61. Harnessing deep learning in ecology: An example predicting bark beetle outbreaks. Front. Plant Sci., 10:1327.
  62. Assessment of deep learning techniques for land use land cover classification in southern new caledonia. Remote Sens. (Basel), 13(12):2257.
  63. ” which camera trap type and how many do i need?” a review of camera features and study designs for a range of wildlife research applications. Hystrix.
  64. Fusing shallow and deep learning for bioacoustic bird species classification. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE.
  65. Understanding experiments and research practices for reproducibility: an exploratory study. PeerJ, 9:e11140.
  66. Machine learning pipelines: Provenance, reproducibility and FAIR data principles. In Glavic, B., Braganholo, V., and Koop, D., editors, Provenance and Annotation of Data and Processes - 8th and 9th International Provenance and Annotation Workshop, IPAW 2020 + IPAW 2021, Virtual Event, July 19-22, 2021, Proceedings, volume 12839 of Lecture Notes in Computer Science, pages 226–230. Springer.
  67. Deep learning and citizen science enable automated plant trait predictions from photographs. Sci. Rep., 11(1):16395.
  68. Identification of animals and recognition of their actions in wildlife videos using deep learning techniques. Ecol. Inform., 61(101215):101215.
  69. Would ecology fail the repeatability test? BioScience, 66(2):98–99.
  70. Applications of deep convolutional neural networks to digitized natural history collections. Biodivers. Data J., 5:e21139.
  71. Sashimi: A toolkit for facilitating high‐throughput organismal image segmentation using deep learning. Methods Ecol. Evol., 12(12):2341–2354.
  72. Stark, P. B. (2018). Before reproducibility must come preproducibility. Nature, 557(7706):613–614.
  73. Machine learning to classify animal species in camera trap images: Applications in ecology. Methods in Ecology and Evolution, 10(4):585–590.
  74. A practical taxonomy of reproducibility for machine learning research.
  75. Development of spectral-phenological features for deep learning to understand spartina alterniflora invasion. Remote Sens. Environ., 242(111745):111745.
  76. Coral reef fish detection and recognition in underwater videos by supervised machine learning: Comparison between deep learning and HOG+SVM methods. In Advanced Concepts for Intelligent Vision Systems, Lecture notes in computer science, pages 160–171. Springer International Publishing, Cham.
  77. A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecol. Inform., 48:238–244.
  78. A new method to control error rates in automated species identification with deep learning algorithms. Sci. Rep., 10(1):10972.
  79. Demystifying the landscape of ecological data repositories in the United States. BioScience, 67(12):1044–1051.
  80. Weinstein, B. G. (2018). Scene‐specific convolutional neural networks for video‐based biodiversity detection. Methods Ecol. Evol., 9(6):1435–1441.
  81. Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks. Remote Sens. (Basel), 11(11):1309.
  82. Investigation of different CNN-based models for improved bird sound classification. IEEE Access, 7:175353–175361.
  83. Detection and annotation of plant organs from digitised herbarium scans using deep learning. Biodivers. Data J., 8:e57090.
  84. Automated conservation assessment of the orchid family with deep learning. Conserv. Biol., 35(3):897–908.
  85. Towards an IoT-based deep learning architecture for camera trap image classification. In 2020 IEEE Global Conference on Artificial Intelligence and Internet of Things (GCAIoT). IEEE.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Waqas Ahmed (46 papers)
  2. Vamsi Krishna Kommineni (3 papers)
  3. Birgitta König-Ries (13 papers)
  4. Jitendra Gaikwad (1 paper)
  5. Luiz Gadelha (6 papers)
  6. Sheeba Samuel (13 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now