Update on Machine-Learned Correctness Properties

A novel method which has the potential for improving the U.S. Navy’s ability to perform continuous assurance on autonomous and other cyberphysical systems.

August 1, 2023

Naval Postgraduate School, Monterey, CA

A novel new method has the potential to provide formal guarantees about the dependability of autonomous systems from the realm of doctoral-level experts into the domain of system developers and engineers. (Image: Kristian/Adobe Stock)

Autonomous systems are poised to provide transformative benefits to society. Autonomous vehicles (AVs) have the potential to reduce the frequency and severity of collisions, enhance mobility for blind, disabled, and underage drivers, lower energy consumption and environmentally harmful emissions, and reduce population density in metropolitan regions. In civilian aviation, increasingly autonomous systems could mitigate two of the most costly features of human pilots: the cost associated with training and paying highly skilled operators, and the reduced efficiency incurred by flight time limitations and crew rest requirements.

Additionally, autonomous air traffic management systems could reduce the cognitive burden on air traffic controllers by automating the monitoring and analysis of high volumes of data, alerting a human operator only when complex decisions must be made to mitigate risk. Within the power distribution industry, innovations in “micro-grid” technology can allow better utilization of alternate energy sources while decreasing vulnerability to failure compared to current centralized power distribution, but such decentralization necessitates highly adaptive autonomous systems to carefully synchronize energy production and consumption. Medical devices are currently designed to function for a large group of patients with similar conditions, but adaptive patient-specific algorithms could respond more effectively to individual patient needs, increasing lifespan and quality of life.

In military aviation, Automatic Ground Collision Avoidance System (AGCAS) has saved the lives of eleven pilots by fusing terrain mapping and sensor data to autonomously maneuver to avoid ground contact if necessary. Elsewhere in the military, autonomous weapons systems are increasingly critical to enhancing warfighting capability. In today’s warfighting environment, autonomous vehicles, unlike unmanned vehicles which require uninterrupted interactive remote control, are becoming increasingly necessary for operation in denied, degraded, or otherwise unsafe environments.

Also, the vast number of sensors streaming data from these systems is overwhelming the ability of human analysts to conduct processing, exploitation, and dissemination (PED) for intelligence purposes. Algorithms supporting autonomous operation, combined with advances in edge computing, can be used to lower the cognitive burden on human analysts by performing autonomous data triage for alerting three humans to data patterns of interest. Despite the potential advantages to be derived from employing autonomous systems, evidence-based assurance of dependability is a key to obtaining users’ trust in these systems.

Such assurance of dependability is often lacking, leading to a lack of trust. When we look at the optimistic statements made within the last five years forecasting the rapid proliferation of AVs, it is clear that such optimism did not survive contact with reality. For these reasons, AVs have become emblematic of the issues associated with trust in autonomy, and therefore, we use them as the system of interest for our research and experimentation. For AVs, the primary barrier to garnering trust in their dependability is answering the question “Are they safe enough?”

This work was performed by James Michael, Doron Drusinsky, and Matthew Litton for the Naval Postgraduate School. For more information, download the Technical Support Package (free white paper) below. NPS-08235