Teaching Drones How to Navigate Real-World Weather

A deep-learning method that can help drones cope with new and unknown wind conditions.

Time-lapse photo shows a drone equipped with Neural-Fly maintaining a figure-eight course amid stiff winds at Caltech’s Real Weather Wind Tunnel. (Image: Caltech)

Right now, drones are either flown under controlled conditions, with no wind, or are operated by humans using remote controls. However, for drones to autonomously perform necessary but quotidian tasks, such as delivering packages or airlifting injured drivers from a traffic accident, drones must be able to adapt to wind conditions in real time.

To face this challenge, a team of engineers from Caltech has developed Neural-Fly, a deep-learning method that can help drones cope with new and unknown wind conditions in real time just by updating a few key parameters. Neural-Fly is described in a study published in Science Robotics.

Neural-Fly was tested at Caltech’s Center for Autonomous Systems and Technologies (CAST) using its Real Weather Wind Tunnel, a custom 10-foot-by-10-foot array of more than 1,200 tiny computer-controlled fans that allows engineers to simulate everything from a light gust to a gale.

Engineers test Neural-Fly in the open air on Caltech’s campus. (Image: Caltech)

“The issue is that the direct and specific effect of various wind conditions on aircraft dynamics, performance, and stability cannot be accurately characterized as a simple mathematical model,” said Soon-Jo Chung, Bren Professor of Aerospace and Control and Dynamical Systems and Jet Propulsion Laboratory Research Scientist. “Rather than try to qualify and quantify each and every effect of turbulent and unpredictable wind conditions we often experience in air travel, we instead employ a combined approach of deep learning and adaptive control that allows the aircraft to learn from previous experiences and adapt to new conditions on the fly with stability and robustness guarantees.”

Neural-Fly gets around these challenges by using a so-called separation strategy, through which only a few parameters of the neural network must be updated in real time.

After obtaining as little as 12 minutes of flying data, autonomous quadrotor drones equipped with Neural-Fly learn how to respond to strong winds so well that their performance significantly improved (as measured by their ability to precisely follow a flight path). The error rate following that flight path is up to 2.5 times to 4 times smaller compared to drones equipped with the current state of the art adaptive control algorithms that identify and respond to aerodynamic effects but without deep neural networks.

Though landing might seem more complex than flying, Neural-Fly, unlike the earlier systems, can learn in real time. It can respond to changes in wind on the fly, and it does not require tweaking after the fact.

Neural-Fly performed as well in flight tests conducted outside the CAST facility as it did in the wind tunnel. Further, the team has shown that flight data gathered by an individual drone can be transferred to another drone, building a pool of knowledge for autonomous vehicles.

For more information, contact Robert Perkins at This email address is being protected from spambots. You need JavaScript enabled to view it.; 626-395-1862.