Autonomous Surveillance Technologies Relating to Dismounted Soldiers

The development of the autonomous applications for dismounted Soldier systems is paramount to defeating our adversaries, such as China and Russia, in future combat. A comprehensive literature review is necessary to assist in defining the best path forward.

May 1, 2023

Army Research Laboratory, Aberdeen Proving Ground, MD

An overview of the type of situational awareness that exists for UAV surveillance. (Image: Army Research Laboratory)

The development of the artificial intelligence/machine learning (AI/ML) applications for dismounted Soldier systems is paramount to defeating our adversaries, such as China and Russia, in future combat. A comprehensive AI/ML literature review is a first step toward defining what exists and what can be applied and researched for our nation’s defense in future warfare. There is a clear need to use the latest AI/ML technologies in threat identification and elimination without U.S. lives lost. A comprehensive literature review is necessary to assist in defining the best path forward. In theory, networked unmanned aerial vehicles (UAVs) using onboard cameras may assist in successful navigation and threat identifications for ground troops. Furthermore, UAVs used as a surveillance and situational awareness (SA) tool may also be feasible to house weaponry to eliminate these threats.

As next-generation AI/ML-enabled optical systems, visual enhancement systems, and accessories are developed for use by the Army, a comprehensive examination of the human systems integration implications for networked use of these systems for teams of Soldiers/Operators is needed to examine human performance relative to teams equipped with legacy weapons and enablers.

The U.S. Army Combat Capabilities Development Command (DEVCOM) Analysis Center, known as DAC, Human Systems Integration Division proposed a comprehensive literature review of autonomous applications, AI, ML, and networked UAVs relative to use of dismounted Soldier systems (i.e., weapon systems and enablers), to include enabling swarmed unmanned aerial systems (UASs), as a conceptual documentation of the potential for autonomous systems during operational tasks. The primary search engines used were the ARL Online Library Catalog (WebCat) and Google Scholar.

To develop a networked UAV system to accurately identify and eliminate targets to dismounted Soldiers, the design of the HMI should be considered. HMI development considers human information processing and cognition and is necessary to optimize cognitive fightability. This is further dependent on how human senses are used to perceive the environment and how they may affect HMI modalities.

Models of information processing (human behavior classification, stimulus-central processing response, and effects of attention-sharing and multi-tasking), modalities of information (visual, auditory, and haptic), modalities of system control (the means by which the Soldier’s input or control commands into the system), and the analysis of Soldier needs (identify information needs and implement the best-suited modality) should be reviewed when implementing or designing HMI.

UAVs will play a vital role in the U.S. Army’s future force, providing visual feedback of the battlefield through the utilization of sensors, cameras, and munitions. An important factor for future building is the analysis of the balance of munitions, sensors, and fuel. UAV operators provide the eyes for the fight. The CL I and CL II UAVs provide battlefield signature patterns that keep the dismounted Soldiers alive.

To have successful wins, Soldiers must get inside the OODA by having a better SA than the threat by making quicker decisions and changing the situation in a way that is unobservable and incomprehensible by the opponent at a given time. AI can strengthen SA by identification of enemy presence, movement, identification, and mitigation.

UAVs can help improve SA with future advancements of data processing, fusion, and analytics. SA is paramount for dismounted Soldiers to engage efficiently. SA encompasses assessment (machine), awareness (user), and understanding (user– machine teaming). To address the multifaceted problem statement concerning UAVs, visual/optic systems, AI/ML, network teaming, and the use of legacy equipment, this literature review covers many interlocking factors. Among the numerous considerations are massive communications, high-level data traffic, low on-board energy storage, network configuration, different modalities to augment human cognition, speech, and audio, and UAV navigation, deployment optimization, and coordination.

This work was performed by Patricia M. Burcham for the Army Research Laboratory. For more information, download the Technical Support Package (free white paper) here under the Vehicles and Robotics category. ARL-0113