High-Performance Fault-Tolerant xEmbedded Computing (HPFEC) Benchmark Suite

This work arises out of the Next Generation Space Processor (NGSP) project, a collaborative effort among several NASA centers and the Air Force Research Laboratory. The goal of NGSP is to develop the next generation of radiation-hardened, fault-tolerant, space-worthy computing systems consisting of hardware and system software. To drive and evaluate the design, each of the NGSP collaborators contributed benchmarks. This article describes the benchmarks developed at the Jet Propulsion Laboratory (JPL). Collectively, these benchmarks are called the HighPerformance Fault-Tolerant Embedded Computing (HPFEC) benchmark suite.

The HPFEC suite consists of four synthetic applications divided into 29 components. The synthetic applications are (1) real-time guidance, navigation, and control; (2) automatic scheduling and planning; (3) identification of closed shapes in an image; and (4) classification of features in an image according to their size, shape, surface reflectance, and texture. In developing these benchmarks, JPL adopted the following approach:

  1. Study current and anticipated future applications that JPL develops or supports.
  2. Extract key algorithm components from the applications. Emphasize capabilities that are beyond the capabilities of current flight systems, but that become feasible in an NGSP system.
  3. Combine the components into synthetic applications, specifying both computation and dataflow.
  4. Provide specifications of the components and of the synthetic applications. Leave the implementation details unspecified, so that a system designer can implement the benchmark specification in the best way for the target platform.
  5. Provide reference implementations that illustrate the input, computation, and output.
  6. Provide instructions on what to implement, run, measure, and report. The HPFEC benchmark suite has the following key features:
    • It provides synthetic applications that test the interaction of several compute kernels in a realistic setting.
    • It provides a library of components that may be combined into new applications.
    • It covers sequential, parallel, distributed, and real-time performance.
    • It covers power management and fault tolerance, including injection of faults to simulate system failure.
    • It specifies conditions for evaluating the behavior of an application when run to stress-induced failure.

The benchmarks are useful to anyone evaluating an embedded system.

This work was done by John Y. Lai, Robert L. Bocchino, Timothy K. Canham, Chris Flatley, Kim P. Gostelow, Raphael R. Some, David A. Rennels, Hans P. Zima, and William D. Whitaker of NASA's Jet Propulsion Laboratory. This software is available for license through the Jet Propulsion Laboratory, and you may request a license at: https://download.jpl.nasa.gov/ops/request/request_introduction.cfm . NPO-49492