MOBILITY ENGINEERING Produced by SAE Media Group
MOBILITY ENGINEERING
Magazines
Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Test and Evaluation of Autonomy for Air Platforms

Tools, approaches, and insights to confidently approach the safe, secure, effective, and efficient testing of autonomy on air platforms.

The 2018 National Defense Strategy emphasizes that the effective implementation of autonomy is essential for future engagements. Key to this implementation is the ability to test and evaluate systems that perform autonomous tasks. The purpose of this research is to equip testers with tools, approaches, and insights to confidently approach the testing of autonomy on air platforms. The air domain is chosen specifically for its applicability to the Air Force mission and to help scope the focus of this research. The intent is not to be an exhaustive reference for testing and evaluating autonomy; rather, the goal is to provide a launching point for greater investigation.

Testing autonomy on airborne platforms brings unique challenges and, therefore, to be successful, testing should leverage the following five principles: Early User Involvement, Continuous and Cumulative Feedback, Streamlined Process and Products, Pilot Training Approach, and Human-Machine Interaction Consideration. The overarching test approach is split into two paradigms: three phases of test and Agile Development and Operations (DevOps).

First, the current flight test paradigm of plan, execute, and analyze (three phases) is reexamined through the lens of testing autonomy. Of the three phases, executing a test of autonomy is currently the biggest challenge within the test community. The goal of any test, and of autonomy in particular, is that it is safe, secure, effective, and efficient. Therefore, the optimal test execution of an autonomous task or mission should include some type of run time assurance, live-virtual-constructive, open systems architecture, and a surrogate platform.

To support the execution of autonomy testing that is responsive and timely, a close pairing of the planning and analysis phases is recommended. Different approaches, like design of experiments, human factors considerations, hardware-in-the-loop, and systems theoretic process analysis, are provided as tools for facilitating the needed planning and analysis capabilities for autonomy test. Within this paradigm, the goal is to adapt existing practices and modify to facilitate a spiral test and evaluation strategy of autonomy.

The second paradigm features a more dramatic shift from the current test practices to one that incorporates the principles of Agile and DevOps. The Agile philosophy emphasizes individuals and interactions over processes and tools, working products over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan. It has been instrumental in modern software development and will certainly play a role going forward with the development of autonomy.

Additionally, DevOps is the revolutionary concept that integrates the development and operation functions of an organization. This process relies on continuous integration, continuous delivery, continuous monitoring and logging, microservices, and collaborative, cross-functional teams to implement high-quality products, quickly. As the Air Force begins to adopt more of the Agile and DevOps principles, it will be able to realize the DoD’s goals of “delivering at the speed of relevance,” “organize for innovation,” and “streamline rapid, iterative approaches from development to fielding.”

This work was done by Riley A. Liver-more and Avery W. Leonard for the Air Force Test Center. For more information, download the Technical Support Package (free white paper) below. AFRL-0305

Topics:
Aerospace Aircraft Architecture Automation Autonomous vehicles Flight tests Human factors Military aircraft Research and development Robotics Safety regulations and standards Test & Measurement Test procedures Unmanned Air Vehicles/Systems (UAV/UAS) Unmanned aerial vehicles Vehicles
This Brief includes a Technical Support Package (TSP).
Document cover
Test and Evaluation of Autonomy for Air Platforms

(reference AFRL-0305) is currently available for download from the TSP library.

Don't have an account?

More From SAE Media Group

    Magazine cover
    Aerospace & Defense Technology Magazine

    This article first appeared in the August, 2021 issue of Aerospace & Defense Technology Magazine (Vol. 6 No. 5).

    Read more articles from this issue here.

    Read more articles from the archives here.

    Overview

    I apologize, but I cannot find relevant information regarding the content of the document titled "Test and Evaluation of Autonomy for Air Platforms" authored by Captain Riley A. Livermore and 1st Lieutenant Avery W. Leonard. However, based on my knowledge, I can provide a general overview of what such a document might cover.

    Typically, a technical information handbook on the test and evaluation of autonomy for air platforms would address the methodologies and frameworks used to assess the performance and reliability of autonomous systems in aviation. It would likely include sections on the following topics:

    1. Introduction to Autonomy in Aviation: An overview of the role of autonomous systems in modern air platforms, including their benefits and challenges.

    2. Testing Methodologies: Detailed descriptions of various testing methodologies, such as simulation, hardware-in-the-loop testing, and flight testing, that are used to evaluate autonomous systems.

    3. Evaluation Criteria: The document would outline the criteria for evaluating the performance of autonomous systems, including safety, reliability, and mission effectiveness.

    4. Regulatory and Standards Compliance: Information on relevant regulations and standards that govern the testing and evaluation of autonomous systems in aviation.

    5. Case Studies: Examples of past testing and evaluation efforts for specific autonomous air platforms, highlighting lessons learned and best practices.

    6. Future Directions: Insights into emerging trends in autonomy for air platforms and recommendations for future research and development.

    7. Conclusion: A summary of the key findings and the importance of rigorous testing and evaluation in ensuring the safe and effective deployment of autonomous systems in aviation.

    This type of document would be essential for engineers, researchers, and military personnel involved in the development and deployment of autonomous air systems, providing them with the necessary guidelines and best practices to ensure successful outcomes.

    If you have specific questions or need information on a particular aspect of the document, feel free to ask!

    Top Stories

    INSIDERDesign

    Feature Image New Copper Alloy Could Provide Breakthrough in Durability for Military Systems

    INSIDERDefense

    Feature Image Boeing to Develop New Space Force Nuclear Communications Satellites

    INSIDERManufacturing & Prototyping

    Feature Image TEWS Introduces First VITA 93 QMC Modules

    INSIDERRF & Microwave Electronics

    Feature Image Metamaterials Antenna Makes Generational Leap in Multi-Band Connectivity

    INSIDERAerospace

    Feature Image Regulation and Technology Changes Needed for Drone Detection, Mitigation in US...

    INSIDERDefense

    Feature Image Paris Air Show: New Aerospace Technologies, Updates and Research

    Webcasts