360-Degree Visual Detection and Target Tracking on an Autonomous Surface Vehicle

June 1, 2011

Office of Naval Research, Arlington, Virginia

Operation of autonomous surface vehicles (ASVs) poses a number of challenges, including vehicle survivability for long-duration missions in hazardous and possibly hostile environments, loss of communication and/or localization due to environmental or tactical situations, reacting intelligently and quickly to highly dynamic conditions, re-planning to recover from faults while continuing with operations, and extracting the maximum amount of information from onboard and offboard sensors for situational awareness. Coupled with these issues is the need to conduct missions in areas with other possible adversarial vessels, including the protection of high-value fixed assets such as oil platforms, anchored ships, and port facilities.

An autonomy system for an ASV detects and tracks vessels of a defined class while patrolling near fixed assets. The ASV’s sensor suite includes a wide-baseline stereo system for close-up perception and navigation (less than 200 m), and a 360-degree camera head for longer-range contact detection, identification, and tracking. Situation awareness for the addressed patrol missions is primarily determined through processing images from the 360-degree camera head in the perception system called Surface Autonomous Visual Analysis and Tracking (SAVAnT). The SAVAnT system is integrated into the CARACaS (Control Architecture for Robotic Agent Command and Sensing) autonomy architecture, enabling the ASV to reason about the appropriate response to the vessels it has identified, and then to execute a particular motion plan.

CARACaS is composed of a dynamic planning engine, a behavior engine, and a perception engine. The SAVAnT system is part of the perception engine, which also includes a stereo-vision system for navigation. The dynamic planning engine leverages the CASPER (Continuous Activity Scheduling Planning Execution and Replanning) continuous planner. Given an input set of mission goals and the autonomous vehicle’s current state, CASPER generates a plan of activities that satisfies as many goals as possible while still obeying relevant resource constraints and operation rules. CARACaS uses finite state machines for composition of the behavior network for any given mission scenarios. These finite state machines give it the capability of producing formally correct behavior kernels that guarantee predictable performance.

For the behavior coordination mechanism, CARACaS uses a method based on multiobjective decision theory (MODT) that combines recommendations from multiple behaviors to form a set of control actions that represents their consensus. CARACaS uses the MODT framework, coupled with the interval criterion weights method, to systematically narrow the set of possible solutions (the size of the space grows exponentially with the number of actions), producing an output within a time span that is orders of magnitude faster than a brute-force search of the action space.

SAVAnT receives sensory input from an inertial navigation system (INS) and six cameras, which are mounted in weather-resistant casings, each pointed 60 degrees apart to provide 360-degree capability, with 5-degree overlap between each adjacent camera pair. The core components of the system software are as follows. The image server captures raw camera images and INS pose data and “stabilizes” the images (for horizontal, image-centered horizons). The contact server detects objects of interest (contacts) in the stabilized images and calculates absolute bearing for each contact. The OTCD server (object-level tracking and change detection) interprets series of contact bearings as originating from true targets or false positives, localizes target position (latitude/longitude) by implicit triangulation, maintains a database of hypothesized true targets, and sends downstream alerts when a new target appears or a known target disappears.

This work was done by Michael T. Wolf, Christopher Assad, Yoshiaki Kuwata, Andrew Howard, Hrand Aghazarian, David Zhu, Thomas Lu, Ashitey Trebi-Ollennu, and Terry Huntsberger of NASA’s Jet Propulsion Laboratory, California Institute of Technology, for the Office of Naval Research. ONR-0024

---

---

This Brief includes a Technical Support Package (TSP).

360-Degree Visual Detection and Target Tracking on an Autonomous Surface Vehicle

(reference ONR-0024) is currently available for download from the TSP library.

LOGIN TO DOWNLOAD

Don't have an account? Sign up here.

---