MOBILITY ENGINEERING Produced by SAE Media Group
MOBILITY ENGINEERING
Magazines
Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Quantifying Eye Movement Trajectory Similarity for Use in Human Performance Experiments in Intelligence, Surveillance, and Reconnaissance (ISR) Research

Using ScanMatch, a Matlab software package, to determine the similarity of two strings of eye tracking data provides useful background and recommendations for potential applications in Intelligence, Surveillance, and Reconnaissance (ISR) research.

The performance of persons who watch surveillance videos, either in real-time or recordings, can vary with their level of expertise. It is reasonable to suppose that some of the performance differences might be due, at least in part, to the way experts scan a visual scene versus the way novices might scan the same scene. For example, experts might be more systematic or efficient in the way they scan a scene compared to novices. Even within the same person, video surveillance performance can vary with factors such as fatigue. Again, differences in the way their eyes scan a scene might account for some of the differences. Full Motion Video (FMV) “Eyes-on” intelligence analysts, in particular, actively scan video scenes for items of interest for long periods of time.

To better understand the characteristics of scanning behavior of Intelligence, Surveillance, and Reconnaissance (ISR) analysts, it is important to track eye movement characteristics. It is relatively simple to model eye characteristics over time such as pupil dilation or eyelid opening over time. It is also common to characterize different types of eye movement over time.

However, when it comes to making comparisons between two sequences of eye data, it is much more challenging and by default, most commercial eye tracking systems do not come equipped with analytic measures of comparing trajectory morphologies. However, there are a variety of applications where this comparison can be useful to ISR research:

  • Comparing eye scanning movements before and after critical events, such as detecting a target;

  • Comparing scanpaths between two different analysts;

  • Comparing scanpaths of analysts over time (e.g. early in the day vs. later in an analyst’s shift);

  • Comparing scanpaths across different surveillance tasks;

  • Comparing scanpaths in simple versus complex surveillance scenarios.

This research explores some common metrics for quantifying the similarity between two eye movement scanpaths, with an emphasis on the Matlab toolbox ScanMatch. ScanMatch is a Matlab package that computes a similarity score between two scanpaths. Both experiments involving pseudodata with known conclusions and experimental data from Piaseki (2016) were analyzed using ScanMatch.

In research on human analyst performance in ISR tasks, it is useful to collect eye tracking metrics of analysts as they perform search operations. In particular, information about fixations and saccades (i.e., the jumps from one fixation to another), has been useful for yielding information regarding:

  • Workload;

  • Fatigue;

  • Attention; and even

  • Inattention blindness.

Fixation and saccade locations can serve as markers for attention because where an analyst is looking on the screen is often highly correlated with what he or she is attending to. However, highly correlated does not mean perfectly correlated with conscious attention, as a well-known inattention blindness study by Drew, Vo, and Wolfe (2013) demonstrates. In their study, radiologists examining an X-ray often fixated on a gorilla figure embedded in the X-ray and made repeated backtracks or re-fixations to it but did not consciously notice the anomaly. Inattention blindness and associated re-fixation and backtrack patterns are important in understanding ISR analyst performance.

Among the rich variables that are collected in eye tracking data, one that centers on the temporal aspects of fixation patterns but whose analysis is frequently neglected due to its complexity, is the scanpath. A scanpath is defined by the temporal sequence of point-by-point (x,y) screen coordinates of where a person is looking on the screen. The accompanying figure shows three example notional scanpaths, although the temporal direction is not shown. At a minimum, scanpaths encompass at least one full fixation-saccade-fixation sequence. Clearly, scanpaths capture the fixation, re-fixation, and backtrack patterns that reveal an analyst's attention, conscious or otherwise.

This work was done by Dr. Mary E. Frame for the Air Force Research Laboratory. For more information, download the Technical Support Package (free white paper) below. AFRL-0303

Topics:
Aerospace Cameras Data Acquisition Data acquisition and handling Detectors Displays/Monitors/HMIs Human factors Imaging Imaging and visualization Medical, health and wellness Medical, health, and wellness Mental processes Optics Psychiatry and psychology Research and development Sensors Surveillance Surveillance and Security Surveillance and Security Threat Location, Targeting, & Surveillance Video
This Brief includes a Technical Support Package (TSP).
Document cover
Quantifying Eye Movement Trajectory Similarity for Use in Human Performance Experiments in Intelligence, Surveillance, and Reconnaissance (ISR) Research

(reference AFRL-0303) is currently available for download from the TSP library.

Don't have an account?

More From SAE Media Group

    Magazine cover
    Aerospace & Defense Technology Magazine

    This article first appeared in the June, 2021 issue of Aerospace & Defense Technology Magazine (Vol. 6 No. 4).

    Read more articles from this issue here.

    Read more articles from the archives here.

    Overview

    The document titled "Quantifying Eye Movement Trajectory Similarity for Use in Human Performance Experiments in Intelligence, Surveillance, and Reconnaissance (ISR) Research" is an interim report published by the Air Force Research Laboratory in June 2018. The report is authored by Dr. Mary E. Frame and focuses on the analysis of eye movement trajectories as a means to enhance understanding of human performance in ISR tasks.

    The primary objective of the research is to develop methodologies for quantifying the similarity of eye movement patterns among individuals engaged in ISR activities. Eye movements are critical indicators of cognitive processes and can provide insights into how individuals gather and process visual information in complex environments. By analyzing these movements, researchers aim to identify performance metrics that can be used to improve training and operational effectiveness in ISR missions.

    The report outlines the significance of eye tracking technology in capturing detailed data on eye movements, including fixation points, saccades, and overall gaze patterns. It discusses the potential applications of this technology in evaluating human performance, particularly in high-stakes scenarios where situational awareness and decision-making are crucial.

    The document also emphasizes the importance of establishing a framework for comparing eye movement trajectories. This involves developing algorithms and statistical methods to quantify trajectory similarity, which can help in identifying patterns associated with expert versus novice performance. By understanding these differences, the research aims to inform training programs that can enhance the skills of ISR operators.

    Additionally, the report highlights the collaborative nature of the research, involving contributions from various experts in the fields of human factors, cognitive psychology, and data analysis. It underscores the commitment to scientific and technical information exchange, ensuring that findings are accessible to a broader audience, including foreign nationals.

    In conclusion, this interim report serves as a foundational step in advancing the understanding of eye movement dynamics in ISR contexts. By quantifying trajectory similarity, the research aims to provide valuable insights that can lead to improved training methodologies and operational strategies, ultimately enhancing the effectiveness of ISR missions. The findings are expected to contribute significantly to the field of human performance research and its applications in military and intelligence operations.

    Top Stories

    INSIDERWeapons Systems

    Feature Image AUSA 2025: The Army's New Anti-Vehicle Terrain Shaping Munition is Ready for...

    INSIDERUnmanned Systems

    Feature Image Meet Arc: Inversion's New Autonomous Space Vehicle for Logistics and Hypersonic...

    INSIDERAerospace

    Feature Image Mercury Signs Embedded Production Agreement for AeroVironment’s Satellite...

    INSIDERManned Systems

    Feature Image AUSA 2025: Secretary Driscoll Wants Army to Save Time and Money by 3D-Printing...

    INSIDERSoftware

    Feature Image Helsing Unveils New Autonomous Fighter Jet 'CA-1 Europa'

    PodcastsAerospace

    Feature Image Autonomous Targeting Systems for a New Autonomous Ground Vehicle

    Webcasts