Single-handed Gesture UAV Control and Video Feed AR Visualization for First Responders

Abstract

Unmanned Aerial Vehicles (UAVs) are becoming increasingly widespread in recent years, with numerous applications spanning multiple sectors. UAVs can be of particular benefit to first responders, assisting in both hazard detection and search-and-rescue operations, increasing their situational awareness without endangering human personnel. However, conventional UAV control requires both hands on a remote controller and many hours of training to control efficiently. Furthermore, viewing the UAV video-feed on conventional devices (e.g. smartphones) require first responders to glance downwards to look at the screen, increasing the risk of accident. To this end, this work presents a unified system, incorporating single-hand gesture control for UAVs and an augmented reality (AR) visualization of their video feed, while also allowing for backup remote UAV control from any device and multiple-recipient video streaming. A modular architecture allows the upgrade or replacement of individual modules without affecting the whole. The presented system has been tested in the lab, and in field trials by first responders.