AI-Powered Camera

I have been always very passionate about wildlife conservation, especially finding tech-based solutions to prevent the poaching of wildlife and mitigate the growing issue of Human-Wildlife conflict, rising rapidly in South Asia due to the growth of both human and wildlife populations near the wildlife habitats. So to continue my passion, in 2020 I joined RESOLVE, an NGO based in Washington DC as a Conservation Technology Fellow and started the development of TrailGuard AI, an AI-powered camera that can be placed on forest trails and send images in real-time (under 30 seconds) of humans and wildlife species crossing the camera.

Drawbacks of Camera Technology currently available in the Conservation market :

The battery-powered camera technology that is currently used for monitoring wildlife and poacher’s movement is very ineffective and has several drawbacks :

• Bulky size: These cameras are very bulky, and almost 30-40% of these cameras get stolen.
• Short battery life: The batteries of these cameras last only a few weeks, a big drawback given it is not very easy to frequently revisit these locations in the forest due to difficult terrain.
• No Real-Time Image Transmission: These cameras store the images on their SD card, there is no other way to remotely access data. Accessing data requires revisits.
• Lacking AI on the Edge to Allow Filtering: These cameras do not have any onboard AI chip, to do inference on the images taken. (Un)Reliability in field conditions: The mechanical enclosures of these cameras often fail in rainy and moist conditions.
• None of them are made in India: These cameras have to be imported to India from outside, making them quite expensive due to customs duties and taxes.

1.) Camera Development (Technology R&D and Production) :

To overcome the challenegs associated with traditional camera traps, the development of the camera system was started by keeping these key features in mind :

The Camera is divided into two parts, the Camera unit system and the Communication unit system. Both these parts are joined together with an M12 cable and communicate with each other via UART and GPIOs.

a.) Development of Camera unit System :

• The major components of the camera unit are Atmega328PB MCU, Intel’s Myriad X chip and an ultra low-light Sony IMX462 image sensor.
• The camera is always in low-power sleep mode. When any movement happens in front of the camera, the MCU gets an interrupt from the PIR sensor, and powers on the Myriad X which then captures the image. This duration from PIR trigger to image capture is around 200 ms, ensuring no objects are missed by the camera.
• After capturing the image, Myriad X runs the AI model (developed by CVEDIA), if the probability of the detected object is more than a threshold, the image is sent to the communication unit system via UART. After this camera goes back to sleep mode, waiting for the next PIR trigger.

b.) Development of Communication unit System to send data in real-time :

• The major components of the communication unit are, ESP32s3, SIM7600 Cellular Modem and Seeed Studio LoRa E5 module.
• The communication unit is always in power-down mode to preserve the battery. When any object of interest is detected on the camera unit side, the camera unit wakes up the communication unit and sends the image data to the ESP32 of the communication unit via UART.

Type 1.) Cellular Communication unit :

• After receiving the data, the ESP32 first waits for the SIM7600 modem to connect to a cellular network, once connected, the ESP32 sends the image jpeg data using MQTT AT commands to the SIM7600 modem which then sends it to the server.
• After the data is sent to the server, the communication unit goes back to sleep.

Type 2.) LoRa Communication unit :

• After receiving the data, the ESP32 converts this jpeg data to ssdv bytes, and transfers this data to LoRa E5 modem via UART.
• The LoRa E5 modem breaks this data into packets of 255 bytes and sends it to the Gateway LoRa unit placed in another location.
• The Gateway LoRa unit is in duty cycle receiver mode to conserve power. But after receiving the first packet from the Transmitter LoRa unit, the Gateway LoRa unit exits the duty cycle mode, and goes to constant receive mode till it receives the entire data.
• After receiving the entire data from the Transmitter LoRa unit, the Gateway LoRa unit sends this data to the ESP32 via UART, which then sends it to the server via the SIM7600 modem on it.

(Note: The LoRa Communication unit (TX and Gateway) is useful for areas that have no cellular connectivity. So the Camera unit with LoRa TX Communication unit can be placed inside the forest area with no network and can send the data to the LoRa Gateway Communication unit, which is already placed in an area with good cellular coverage and can send data to the server end. The problem with this system was that LoRa is more suitable for line-of-sight transmissions, so after several experiments, we found this solution to be not so suitable for dense vegetation forest areas. We decided to stick with the Cellular Communication unit and strategically place the cameras on the periphery of the parks)

c.) Development of Cloud based Server :

• Linode was used as a cloud-based server.
• MQTT Broker service was running on the server end, waiting to receive data from the camera end.
• The data received was converted into a jpeg image, ran through a Yolov5 model and based on the location and camera id, was sent to the respective end user.
• The Yolov5 object detection model was used on the server end to remove any false positives. Yolov5 has 80 classes, so to detect tigers, the model was retrained using the tiger images dataset and performed fairly well.
• The data was displayed in a centralized dashboard, where users can log in and look for data based on: location, date-time, and species detected.

2.) Deployments :

• We deployed the system in Kanha-Pench Tiger Reserve and Dudhwa Tiger Reserve with the respective forest department officials. Both these regions are hotspots for poaching and human-wildlife conflict issues.

• The deployment was done on the periphery of the parks, so any potential human intruder trying to enter the park can be caught in real-time or any wildlife coming out of the park can be detected before it enters the nearby villages.

Images sent in real-time from India :

Tigers and Elephants detected in real-time while trying to enter villages :

A gang of Poachers detected in Dudhwa Tiger Reserve, who were later arrested :

Published on the Front page of the Hindu Newspaper

Camera hidden on the Tree that detected the poachers in Dudhwa Tiger Reserve :

Organizing workshops to introduce the technology to the Forest Rangers and Village Communities :

Deploying the Cameras with the Forest Rangers :

Demonstrating the Camera system to the Government Officials of India :

3.) Media Coverage of the Work and Technology :

Featured in the Award-winning CNN Documentary : Mission Tiger
(Documentary Link : CNN Mission Tiger)
(Article Link : CNN Mission Tiger)

The Hindu Article : Arrest of Poachers in Dudhwa
(Article Link : The Hindu)

The Times of India Article : Real-Time detection of Tigers in the Kanha-Pench Tiger Reserve

Work Funded by : This entire work was funded by Mr. Vinod Khosla (Khosla Ventures). I had sent an email to Mr. Khosla for support and he gave me a $50k grant to start this work in India. After seeing the results from India, Mr. Khosla subsequently invested $600K to continue the work in India. (Tweet Link)

Note : Currently this system is under production at VVDN in Delhi!