Augmented Reality Control System

What Problem Needed To Be Solved?

When it comes to Surf Life Saving a set of binoculars is a crucial piece of equipment used to identify distressed swimmers. Once identified current methods of rescue not only place the rescuer in varying levels of danger but consume a considerable amount of precious time available to the distressed. The use of drones has started being trialled in rescue situations but still rely on trained pilots to pilot the drone to the distressed location for delivery of a flotation device.


What Was The Solution?

In Short: Augmented reality control system that determines GPS coordinates of the distressed for drone navigation using visuals alone.

Detailed: SAPHI Engineers designed a new control system for Surf Life Saver Drones by switching out the surf lifesavers binoculars for a virtual reality headset that controls a camera. This creates an Augmented Reality Control System which works by applying a digital overlay to the world we see with our eyes. In this application, a crosshair is used to line up the distressed for the GPS coordinate calculation. From the height, angle and orientation of the camera, it is possible to determine the GPS coordinate of the distressed based on visuals alone.

The VR headset has a first-person view from the camera’s location on top of the lifeguard tower. When the surf lifesaver moves their head the roll, pitch and yaw sensor data are used to change the angle of the camera. Once the GPS coordinate is calculated, the drone is deployed to fly to the location of the distressed and give aid in the form of a flotation device and verbal commands.

Benefits of such a control system being used in surf lifesaving situations include;

  • Reduced response time between identifying the distressed and getting to their location.
  • Minimum training required to use a VR
  • Audio link between the shoreline and the distressed.


What Skills Did SAPHI Use To Implement The Solution?

SAPHI engineers implemented a complete end-to-end solution by using a broad range of skills within the areas:

  • Control System Design
  • Embedded Systems
  • Power Systems
  • Robot Kinematics
  • User Interface Design
  • Wireless Communications


Completion Date: 2017
Location: Newcastle, NSW, Australia
School: University of Newcastle