Snakes often pose a danger on farms, especially those with venomous species that can cause serious medical emergencies. This XIAO ESP32S3 Sense-driven Snake Recognition System aims to detect snakes in farm fields and alert people via LoRaWAN.
Software: Arduino, Edge Impulse
Industry: Agriculture, Urban Wildlife Monitoring, Zoos and Aquariums, Conservation Biology, Tourism and Nature Conservation, Research and Education
Snakes often enter farms, threatening human safety. Chloe Zhang‘s research highlights the neglected issue of snake bites in many countries, especially in Africa, Asia, and Latin America. The numbers are alarming: Asia sees around 2 million snakebite cases each year, and in Africa, an estimated 435,000 to 580,000 people require treatment annually. These bites from venomous snakes can lead to severe medical emergencies, including paralysis, respiratory problems, and bleeding disorders.
There are three significant challenges within Chloe’s project. Firstly, Wi-Fi and cellular networks are ill-suited for the vast and remote expanse of agricultural fields. Secondly, to balance outdoor deployment feasibility and cost-effectiveness, a compact equipment solution is crucial. Lastly, the project must achieve long-distance transmission capabilities while minimizing power consumption, demanding an optimal synergy of range and energy efficiency.
- Seeed Studio XIAO ESP32S3 Sense: is a cost-effective device that comes with an OV2640 camera sensor, a digital microphone, and SD card support. Its AI capabilities empower it to excel in image recognition, ensuring precise image detection and prediction.
- Grove – Wio-E5 (STM32WLE5JC): LoRaWAN’s extensive coverage and low power usage make this LoRa module a great choice for sending XIAO image recognition results to the IoT platform. Its plug-and-play simplicity, along with built-in AT commands and Arduino compatibility, enhance its appeal.
- Grove Base for XIAO: is used to connect the XIAO ESP32S3 Sense to the Grove-Wio E5.
- SenseCAP M2 Multi-Platform LoRaWAN Indoor Gateway(SX1302) – EU868: Chloe seeks a budget-friendly yet high-quality gateway, and the M2 is the perfect choice. It meets her requirements without breaking the bank and offers strong performance.
Training object detection model with Edge Impulse
1. Gathering dataset
Training a target recognition model begins with acquiring a high-quality dataset of target images. You can source relevant datasets from platforms like Kaggle, Roboflow, and OpenDataLab. Download your chosen dataset in COCO format for a compressed dataset package.
Upon decompression, you’ll find “train” and “test” folders containing data for model training and validation. Each folder includes a JSON file with image labels, eliminating the need for manual labeling. If manual labeling is necessary, consider using the labelme annotation tool.
2. Build a new project on Edge Impulse
- Create a new project.
- Click on “Data acquisition” and then the “upload” button. Select a folder, choose files, and specify “train” or “test” depending on the folder’s content. Then, upload the data.
- After uploading the data, you can view the entire dataset in the Edge Impulse platform. Check the labels for the entire dataset.
- Click “Create Impulse” and change the resize mode to “Fit longest axis.” Save the impulse.
- Click “Image” and generate features.
- Begin model training and deploy it in the Arduino IDE.
- Once model training is complete, click on “Deployment,” choose the Arduino library, disable the EON Compiler, select “Quantized (int8),” and then build.
- Edge Impulse’s developer model has limitations on dataset size and training duration. Prior complexity assessment is essential to ensure your model adheres to these constraints.
- Parameter changes are required based on data set size and model structure, and the default parameters are used here for training
3. Compiling code using Arduino IDE
- Installation of Object Detection Libraries
Upon completing the steps in the document, access a compressed package containing the trained model and necessary library files required for Arduino operation.
Installing libraries in Arduino IDE, refer to the Installing Libraries | Arduino Documentation
- Implementing the Trained Model on XIAO
With the deep learning dependency libraries configured in the previous section, the trained model can now be used for inference. Initiate the camera to capture real-time image data. Successful initialization will prompt the serial port to display “camera initialized.” For a visual representation, refer to the figure below.
Subsequently, obtain image data through camera invocation and execute forward propagation prediction with the loaded model to determine the image’s category. The prediction corresponds to different categories, represented using sequential numbering for classification. Specifically, for snake detection, “1” denotes snake detection, and “0” signifies no snake detection. Once this classification data is available, transmit it through LoRa communication.
3. Transmit data to TTN and achieve visualization on Datacake
- Data Transmission to The Things Network
Consult the guide “Sending Wio-E5 Data to Datacake via TTN – Hackster.io” for instructions on creating an application, adding a gateway, and binding devices. It’s essential to have your own LoRaWAN gateway to utilize The Things Network (TTN). As shown in the image below, XIAO successfully uploads recognition results to TTN via the Grove-Wio-E5.
- Data Visualization Using Datacake
For data visualization, refer to the same resource, “Sending Wio-E5 Data to Datacake via TTN – Hackster.io” Before visualizing the data, the data must be decoded to obtain the necessary information. This can be done by navigating to the device -> Configuration -> Payload Decoder page, as illustrated in the figure below:
You can then replace the code in the Payload Decoder with the code Chloe provided. Once all the tasks are accomplished, you can visualize the data from XIAO on the dashboard.
Notice that the value remains at 0 when no snakes are present. However, if a snake is detected, the number will change. A “1” indicates the presence of snakes during that specific time period.
Once the program is uploaded to XIAO, the camera captures snake images for object detection. Using x, y, width, and height data, we can determine the target’s position within the image. If needed, additional equipment, like a steering gear head, can be added to track and recognize the target by keeping it in the image center. This project benefits from LoRaWAN technology, providing ample coverage for farm areas, extending up to several kilometers.
In conclusion, Chloe Zhang has been working on a snake detection project and plans to expand it to count and identify snake species using advanced AI and computer vision techniques. This development has significant implications for wildlife conservation, ecological research, and snakebite prevention. She is committed to advancing her project and encourages others to explore AI and LoRaWAN technology for their unique endeavors.
Learn More Project Details on Hackster: LoRaWAN Based TinyML Snake Recognition System
Please feel free to reach out to [email protected] for any inquiries or if you’d like to engage in further project discussions. Your questions and interest are welcomed.