Deep Reinforcement Learning Achieves Multifunctional Morphing Airfoil Control

Smooth camber morphing aircraft offer increased control authority and improved aerodynamic efficiency. Smart material actuators have become a popular driving force for shape changes, capable of adhering to weight and size constraints and allowing for simplicity in mechanical design.

An image of the morphing airfoil system within the 1' × 1' (30 cm × 30 cm) wind tunnel. Also visible in the upper left corner is the Keyence 2D profilometer used to measure the true deflection of the trailing edge. (Image: University of Michigan)

Uncrewed aerial vehicles (UAVs) are growing in popularity for both civilian and military applications, which makes improving their efficiency and adaptability for various aerial environments an attractive objective. Many studies pursue this goal using morphing techniques that incorporate shape changes not typically seen in traditional aircraft. Due to weight and volume constraints consistent with smaller flight vehicles, smart materials, such as macro fiber composites (MFCs), have been used to achieve the desired shape changes. Macro fiber composites are low-profile piezoelectric actuators which have gained substantial attention within the morphing aircraft community. Piezoelectric actuators operate by generating strain when voltage, and hence an electric field, is applied to the electrodes. Piezoelectric actuators are also well known for their capabilities to produce high force-output and a highspeed actuation response. Unlike traditional piezoelectric actuators, which are composed of solid piezoelectric material, MFCs are manufactured using a series of thin piezoceramic rods in a composite laminate layup allowing them to exhibit excellent flexibility while still maintaining the performance benefits attributed to traditional piezoelectric actuators. Furthermore, MFCs exhibit large out-of-plane curvatures when bonded to a thin inextensible substrate, like steel shim, which shifts the structure’s neutral axis. This behavior is attractive for camber morphing airfoil applications and has spurred a large subset of research in the field of morphing aircraft.

Though the field of morphing aircraft is brimming with novel morphing mechanisms, camber morphing wings and airfoils have proven to be especially beneficial due to their ability to increase control authority and improve efficiency. MFC actuators have been widely used in camber morphing wings, in part because they are capable of seamlessly generating cambered actuation allowing them to serve both as the airfoil skin and actuator. Furthermore, the lightweight nature of MFCs and their rapid actuation response are advantageous in UAV applications. Reductions in aircraft weight lead to greater fuel efficiency and rapid actuation allows for greater maneuverability. MFC-driven camber morphing has been applied to several UAV control problems including localized optimization for adverse aerodynamic disturbance and stall recovery, as well as improved efficiency and control effectiveness in roll and pitch for a rudderless aircraft. Finally, pitch and yaw control effectiveness and yaw stability were also demonstrated in an avian-inspired rudderless UAV with a camber morphing MFC tail actuator.

Although MFCs have demonstrated potential for camber morphing applications, they have drawbacks. While piezoelectric actuators generate high force output, the thin and flexible nature of MFCs make them vulnerable to displacement under large out-of-plane forces. As the wings of aircraft are the primary lifting surface, large aerodynamic loads are prone to inducing aeroelastic deformation of MFC-actuated airfoils, reducing the total camber. However, this can be remedied using control algorithms which utilize feedback to tune the actuator’s input voltage such that the desired camber or trailing edge displacement is achieved. Though this has proven successful, the inherent hysteresis of MFCs is a challenging hurdle for traditional control algorithms, many of which are linear by nature. In contrast, deep reinforcement learning (DRL) is well-equipped to manage nonlinear control relationships and may be a promising alternative.

Reinforcement learning (RL) is a means of autonomously achieving control in a manner analogous to that seen in biological systems. Like biological learning, trial and error is used in conjunction with a reward system to meet a specified goal. Each reinforcement learning problem consists of two fundamental parts, the agent, or object of concern whose actions are determined by a learned policy, and the environment in which the agent observes its state and performs actions. A state’s value is measured as the long term expected reward to be received after residing in that specific state. If the state space is large and best represented as continuous, function approximation is used to reduce memory requirements. Artificial neural networks (ANNs) are an effective method for function approximation because of their ability to accurately represent nonlinear functions when trained on large quantities of data. Recent work has combined multi-layered ANNs with RL to create the subfield DRL, which has found remarkable success in many simulated and game-based environments. Success in perfectly controlled environments, such as games and simulations, continues to advance the field; however, there is a growing need to apply the knowledge gained through simulation to robotics and other physical hardware environments.

This work was performed by Kevin P. T. Haughn, Lawren L. Gamble, and Daniel J. Inman for the University of Michigan. For more information, download the Technical Support Package (free white paper) at under the Aerospace Defense & Technology category. AFOS-0087

This Brief includes a Technical Support Package (TSP).
Document cover
Deep Reinforcement Learning Achieves Multifunctional Morphing Airfoil Control

(reference AFOS-0087) is currently available for download from the TSP library.

Don't have an account? Sign up here.