Developing an AI-based Integrated System for Bee Health Evaluation

Abstract: Honey bees pollinate about one-third of the world's food supply, but bee colonies have alarmingly declined by nearly 40% over the past decade due to several factors, including pesticides and pests. Traditional methods for monitoring beehives, such as human inspection, are subjective, disruptive, and time-consuming. To overcome these limitations, artificial intelligence has been used to assess beehive health. However, previous studies have lacked an end-to-end solution and primarily relied on data from a single source, either bee images or sounds. This study introduces a comprehensive system consisting of bee object detection and health evaluation. Additionally, it utilized a combination of visual and audio signals to analyze bee behaviors. An Attention-based Multimodal Neural Network (AMNN) was developed to adaptively focus on key features from each type of signal for accurate bee health assessment. The AMNN achieved an overall accuracy of 92.61%, surpassing eight existing single-signal Convolutional Neural Networks and Recurrent Neural Networks. It outperformed the best image-based model by 32.51% and the top sound-based model by 13.98% while maintaining efficient processing times. Furthermore, it improved prediction robustness, attaining an F1-score higher than 90% across all four evaluated health conditions. The study also shows that audio signals are more reliable than images for assessing bee health. By seamlessly integrating AMNN with image and sound data in a comprehensive bee health monitoring system, this approach provides a more efficient and non-invasive solution for the early detection of bee diseases and the preservation of bee colonies.

• The paper presents an integrated system using an Attention-based Multimodal Neural Network (AMNN) that fuses visual data from YOLOv5 and audio features for robust bee health evaluation.
• It employs detailed data acquisition from 25 beehives, utilizing image and audio preprocessing alongside feature extraction methods like MFCC and Chromagram.
• The study demonstrates the method’s practical value by achieving 92.61% accuracy, thereby enhancing early detection of pesticide effects and supporting sustainable beekeeping practices.

Developing an AI-based Integrated System for Bee Health Evaluation

Introduction

The paper "Developing an AI-based Integrated System for Bee Health Evaluation" (2401.09988) introduces a comprehensive AI-driven approach targeting the enhancement of honey bee colony health monitoring. Honey bees are critical to global agriculture, yet their populations have experienced a drastic 40% decline over the last decade due to factors like pesticides and pests. Traditional beekeeping methods are often labor-intensive, subjective, and may disrupt the bees. The research leverages an Attention-based Multimodal Neural Network (AMNN) combining computer vision and signal processing to assess bee health using both visual and audio data, achieving remarkable accuracy.

Methods

Data Acquisition and Preprocessing

The data includes images and audio from 25 beehives at various locations in California, capturing bee activities over 150 gigabytes of video data. A Raspberry Pi camera and microphone are utilized for image and sound collection, ensuring high-resolution recordings necessary for detailed analysis.

Bee Object Detection

The bee images undergo preprocessing and are annotated using YOLO format for image detection models. Audio clips are similarly annotated, focusing on distinguishing bee sounds from background noise. Data augmentation techniques like blurring and transformation play a crucial role in improving model robustness.

Figure 1 illustrates the distinct audio feature patterns when bees are absent versus present, providing significant insights into behavior identification.

Figure 1: The audio features show different patterns when bees are absent (left) vs. present (right). The top-to-bottom representation includes the original audio wave, Mel Spectrogram, MFCC, STFT, and Chromagram.

Feature Extraction

The study emphasizes audio feature extraction using Mel Spectrogram, MFCC, STFT, and Chromagram. These features effectively capture intricate details of bee sounds, offering valuable cues for behavioral analysis. The transformation process standardizes techniques for accurate sound feature recognition.

Model Development

The development incorporates multiple deep learning models for health assessment:

1. YOLOv5 Model: A robust object detection tool applied to bee images, optimized for high precision in identifying and cropping bee-centric data (Figure 2).

   Figure 2: Summary of YOLO v5 model performance

2. Audio Object Detection Models: Four distinct 1D CNN models employ different audio features. The Chromagram model achieves the highest accuracy, emphasizing pitch dynamics as crucial for distinguishing bee sounds.
3. Image Health Assessment Models: Implemented models like Inception v3, MobileNet v2, CNN, and VGG16, each displaying efficacy in health classification through bee images. VGG16 offers superior performance due to its feature map depth (Figure 3).

   Figure 4: Compressed self-designed CNN structures for (A) audio object detection, (B) visual health assessment, and (C) audio health assessment.

   

   Figure 3: Visualization of VGG16's feature maps, highlighting key regions of an input image across the convolutional layers.

4. Attention-based Multimodal Neural Network (AMNN): This model merges visual and audio data via attention mechanisms, improving robustness and accuracy in health assessment. Detailed pseudocode and architecture are provided, showcasing dynamic feature weighting and integration for optimal evaluation.

Results

Performance Analysis

The AMNN model significantly improves bee health assessment accuracy, surpassing isolated visual and audio models with an accuracy of 92.61%. This integration method achieves high F1-scores across health conditions, enhancing predictive consistency during environmental stressors like pesticide exposure.

Training and Inference Times

The study reports marginal increases in processing time for AMNN, yet benefits in predictive accuracy justify this minor cost. Training and inference times across models are documented, highlighting computational efficiency even in multimodal approaches.

Discussion

The paper illuminates the essential role of integrating visual and audio data, where subtle cues from audio significantly enhance behavioral assessments, particularly in detecting pesticide exposure. Bee sounds offer more consistent indicators due to their reliance on sound-based communication methods. Future studies might broaden across species and environments to bolster bee health monitoring frameworks.

Conclusion

The research provides a novel system integrating AI with practical beekeeping strategies, contributing substantially to early detection of bee health issues. This comprehensive methodology offers beekeepers precise real-time monitoring tools for identifying emergent threats, thereby fostering sustainable conservation practices. The paper underlines the importance of advancing multimodal models in ecological applications.