Characterizing Motion Prediction in Small Autonomous Swarms

Despite the expanded use of robotic swarms, little is known about how swarms are perceived by human operators. To characterize human-swarm interactions, we evaluate how operators perceive swarm characteristics, including movement patterns, control schemes, and occlusion.

June 1, 2023

US Army DEVCOM Soldier Center, Medford, MA

Robotic swarms, such as those that use drones pictured here, are becoming more common, and researchers are trying to understand how to better track their dynamic cognitive movements. (Image: Mike Mareen/Adobe Stock)

Prior investigations of swarm robot control focus on optimizing communication and coordination between agents, with at most one human control scheme, or with discrete (rather than continuous) human control schemes. In these studies, focus tends to be on human-robot interactions, including human-machine gesture interfaces, human-machine interaction during conversation, or evaluation of higher-level mental states like comfort, happiness and cognitive load. While there is early work in human control of Unmanned Arial Vehicles (UAVs) and interface design, there are few systematic studies of how human operators perceive fundamental properties of small swarms of ground-based semi-autonomous robots. Therefore, the goal of this study is to better understand how humans perceive swarms of semi-autonomous agents across a range of conditions.

Given fundamental properties of tracking swarms derive from basic cognitive processes, prior research offers clues to understand the limits and abilities of human swarm operators. First, object tracking is a ubiquitous real-world visual phenomenon that has been thoroughly investigated. It is established that humans can accurately track the path of single-object motion in straight lines, arcs, and sinusoids. Even through occlusion, humans can accurately predict the future location of a single moving object.

Further, work on motion extrapolation and tracking through occlusion for single objects suggests that humans can accurately align their eyes with the position of an occluded target. It has been suggested that target velocity information encoded prior to occlusion is used to update the tracking mechanism at the correct rate. While humans cannot usually produce smooth eye movements in the absence of a moving target, they can produce anticipatory smooth pursuit eye movements scaled according to the expected velocity of an upcoming target or a target reappearing after occlusion.

Second, multiple object tracking (MOT) defines humans’ ability to simultaneously track small numbers of moving objects. The boundaries of multiple tracking ability suggest capacity limits of approximately 3–5 items.

However, this capacity limit is impacted by multiple factors, including object speed, stimulus complexity, individual variability and motion of the observer. Other variables such as expertise can also impact one’s ability to track multiple objects. For instance, team sports, video game playing, and military activities are thought to impact one’s MOT capacity. While the MOT literature informs our general understanding of tracking multiple objects, work on the perception of ensembles is useful for applications relevant to robot swarms. For instance, objects in groups greater than 3–4 are perceived more like a group or ensemble. When this grouping occurs, humans can quickly estimate average properties such as hue, facial expression, motion, size, and orientation. Furthermore, perception of dynamic ensembles is impacted by group properties, such as group size and inter-object spacing. Specifically, multiple object tracking tends to decline as inter-object spacing dynamically shift from a groups motion trajectory. Overall, humans accurately estimate the position centroid of groups of objects. Nevertheless, the study of ensemble perception to date primarily relies on rapid exposure to static stimuli.

The current investigation lies at the intersection of object tracking and ensemble perception. Specifically, the goal is to characterize the cognitive processes involved in tracking dynamic ensembles (i.e., swarms). While little work has been done on this particular topic thus far, the literature indicates there are indeed perceptual challenges, such as high cognitive demands and non-intuitive behaviors that play a role in the control of semi-autonomous swarms.

This work was performed by Seth Elkin-Frankston for the U.S. Army Combat Capabilities Development Command Soldier Center, Natick, MA. For more information, download the Technical Support Package (free white paper) under the Communications category.