Vision-Based Object Detection and Tracking Using Drone

Project Overview

This project presents a vision-based object detection and tracking system that utilizes a Tello drone to track an iRobot as the target object. The system incorporates a custom-trained YOLOv5 deep learning model for accurate object detection in real-time, as well as a combination of centroid tracking and PID control algorithms to manage the drone's movement. The drone autonomously maintains a safe distance from the iRobot while following it through various environments.

Video Demonstration

Motivation and Project Applications

The motivation behind this project is the increasing need for efficient, versatile, and accurate object detection and tracking systems in various domains. The proposed vision-based object detection and tracking system has numerous potential applications, including surveillance, search and rescue operations, wildlife observation and research, inspection and monitoring, autonomous navigation, sports and entertainment, and retail and inventory management.

Objectives and Deliverables

This project is guided by two primary objectives:

1. Implement an accurate object detection system for iRobot
2. Combine the object detection system with a tracking system for accurately tracking the iRobot

The key deliverables resulting from this project include:

• Custom-trained YOLOv5 model for iRobot detection
• Integrated object detection and tracking system
• System performance evaluation using a custom dataset of iRobot images

Installation and Usage

To use this project, follow these steps:

1. Clone this repository to your local machine
2. Install the required dependencies using pip install -r requirements.txt
3. Launch the system using python main.py

You can also modify the system to track other objects of interest by adjusting the object detection model as needed.

Contributors

• Shravan Nimma

• Mounika Chintam

License

Code

You can find the source code for this project on GitHub:

Data

iRobot Dataset

The iRobot dataset consists of images and annotations of the iRobot Roomba robot vacuum cleaner. The dataset includes a total of 1600 images, with 1200 images for training and 400 images for testing.

The annotations were created using the YOLOv5 annotation tool, and include bounding boxes and class labels for the iRobot Roomba object.

Data Augmentation

To increase the size and diversity of the training dataset, data augmentation techniques were applied to the iRobot dataset. The following data augmentation techniques were used:

• Random horizontal flip
• Random vertical flip
• Random rotation
• Random zoom
• Random brightness
• Random contrast

Results

Object Detection Metrics

Here are the precision, recall, and F1 scores for the custom-trained YOLOv5 model:

• Case 1 (before augmentation, only iRobot images):
• Precision: 0.85
• Recall: 0.81
• mAP_0.5:0.95: 0.85
• Case 2 (training 1, iRobot images and augmentation data):
• Precision: 0.90
• Recall: 0.92
• mAP_0.5:0.95: 0.95
• Case 3 (all data, including similar confusable objects):
• Precision: 0.93
• Recall: 0.96
• mAP_0.5:0.95: 0.96

Object Detection Performance Images

Here are sample images showing the object detection performance for each case:

Acknowledgments

We would like to thank our project advisor and colleagues for their support and feedback throughout the development process. We also acknowledge the contributions of the YOLOv5 and OpenCV communities, whose open-source software made this project possible.