Designing and Assessing Vehicle Safety Functions with a Use Case Approach

The required capabilities of a vehicle’s systems and components will vary depending on the scenarios and use cases in which they are expected to safely operate. This commonsense understanding is consistent with the framework for evaluating, improving, and documenting expected vehicle performance based on the Safety of the Intended Functionality (SOTIF) standard.

SOTIF, described in ISO/PAS 21448:2019, is a complement to the Functional Safety requirements outlined in ISO 26262. Whereas Functional Safety analysis identifies and mitigates potentially hazardous failures of vehicle components or systems, the SOTIF approach provides a methodology for identifying and maximizing the range of scenarios in which a vehicle can be expected to function safely under normal operation or with reasonably foreseeable misuse. A corollary result of SOTIF methodology is the identification of performance limitations and triggering events that may produce potentially hazardous outcomes, along with consideration of whether the resulting risk is acceptable. Because the SOTIF framework is versatile, it can be applied to define the capabilities and restrictions of any safety function, either in autonomous vehicles  (AVs) or specific advanced driver assistance systems  (ADAS) features.

Taken together, the conditions and scenarios in which a safety function can be expected to operate compose that function’s Operational Design Domain (ODD). Constraints on ODDs might include “roadway types, speed range, environmental conditions (e.g., weather, daytime, nighttime), [and] prevailing traffic laws”.1 Well-defined ODDs represent a useful basis for conveying vehicle feature capabilities to consumers through online databases, new vehicle stickers, marketing descriptions, and/or user’s manuals. As the variety of vehicle performance capabilities continues to expand, the need for clear consumer education about what safety functions can and cannot do in specific situational contexts becomes increasingly important. Even more, for ADAS and AV applications, the parameters of safe operation will depend largely on the vehicle’s sensor and processing system’s ability to accurately gather and interpret data about the surrounding environment. Thus, as this article will demonstrate, clearly defining a safety feature’s intended ODD also highlights required levels of sensor performance.

After briefly introducing the methodology for achieving SOTIF through the design, verification, and validation phases of development, this article examines two common performance limitations for Pedestrian Automatic Emergency Braking (PAEB): lighting conditions and speed. This focus on PAEB is based on its being a foundational element for both ADAS and AV applications, with improved PAEB functionality offering the potential to improve roadway safety dramatically. This examination highlights how improved, or additional, sensor technology can expand the currently limited ODDs of camera and radar-based PAEB features. Finally, this technical article considers how SOTIF analyses can be utilized to support an ADAS feature rating system, along with a proposed sticker label to educate consumers about the ODDs of safety functions.

SOTIF in practice

ISO/PAS 21448 defines “Safety of the Intended Functionality” as “the absence of unreasonable risk due to hazards resulting from functional insufficiencies of the intended functionality or by reasonably foreseeable misuse.” The document offers guidelines for achieving SOTIF through careful methods during the design, verification, and validation phases. The standard “is intended to be applied to intended functionality where proper situational awareness is critical to safety, and where that situational awareness is derived from complex sensors and processing algorithms; especially emergency intervention systems (e.g., emergency braking systems) and Advanced Driver Assistance Systems (ADAS).”2

Analyzing the safety of a vehicle system’s intended functionality involves considering the logical scenarios that the system will likely encounter. As shown below, these scenarios can be organized according to whether they have been identified for analysis (known vs. unknown) and whether they can lead to hazardous outcomes (safe vs. a unsafe).3

Scenario classification for SOTIF

The classification process allows for the identification of scenarios that point to performance limitations of the function (i.e., dangerous inaction) along with scenarios that might trigger hazardous events (i.e., dangerous reaction). It allows for these resulting hazards to be mitigated either through improved performance or through documentation of limitations to the function’s ODD; e.g., in user’s manuals or other consumer-oriented materials. Any scenario that has been evaluated through vehicle performance testing can be classified under either the “known safe” or “known unsafe” categories. Continual improvement enables automakers to shift scenarios from the category of “known unsafe” to “known safe,” which also supports the process of developing systems robust enough to safely address a range of unknown scenarios (Area 3) and minimize possible unknown hazards (Area 4).

Identifying the intended ODD of a safety function at the design stage can help automakers and component suppliers establish and build toward specific engineering requirements. During the validation phase, however, it is unlikely that safe performance within any ODD can be completely captured and assessed through a single testing protocol. Rather, multiple tests will be required, with each test evaluating only a subset of the conditions or scenarios within that function’s intended ODD. If the function fails to operate safely in certain conditions or scenarios it was intended to handle, those conditions either need to be removed from the function’s ODD or the function needs to be improved and retested until it operates as intended.

Diagram of SOTIF validation process, highlighting steps in the design, implementation, and testing phases to ensure that the safety function performance aligns with design intent.

Within this framework, the ODDs of both ADAS features and AV functionalities are defined by the known scenarios in which they can safely operate. It is the responsibility of automakers to limit AV or ADAS feature usage to the scenarios they know the vehicle can safely manage by discouraging or prohibiting usage in scenarios that are known to be potentially hazardous. As a matter of public safety, it is critical that enginers, automakers, municipalities, and consumers know how well a vehicle’s safety features can be expected to perform in the range of scenarios they will logically encounter.

Achieving SOTIF in nighttime PAEB performance

Most new vehicles sold in 2018 (56%) were equipped with PAEB as either a standard or optional feature.4 Various testing organizations, including NHTSA and AAA, have demonstrated that current PAEB systems frequently fail in dark conditions, however.5 It is significant because improved nighttime PAEB performance can reduce the number of pedestrian lives lost annually. In 2018, 76% of the 6,283 total pedestrian crash fatalities occurred in dark conditions.6 In fact, that year more pedestrians were fatally struck in “dark-not lighted” conditions than in daylight conditions.7 No statistic available indicates how many of these pedestrian fatalities involved a vehicle equipped with PAEB. However, as the studies mentioned indicate, failures of current PAEB systems in nighttime conditions illustrate this feature does not safely function within a commonplace roadway scenario. As a result, increased testing of pedestrian scenarios needs to be conducted under nighttime conditions; if these tests continue to produce hazardous outcomes, then the high failure rate of these ADAS features needs to be addressed.

Images from NHTSA’s report, “Objective Test Procedures for Pedestrian Automatic Emergency Braking Systems,” presented at the SAE Government/Industry Meeting  in 2017.8

One possible response to this assertion could state that, within the context of ADAS, PAEB failures do not fall under SOTIF consideration because human drivers are ultimately responsible for the operation of their vehicle, no matter what features it has. However, the SOTIF standard states that “the hazards caused by a potentially hazardous system behavior, due to a triggering event, are considered both for use cases when the vehicle is correctly used and for use cases when it is incorrectly used in a reasonably foreseeable way.”9 The example of foreseeable misuse given in the document is “lack of driver attention while using a level 2 driving automation.”10 If this reasoning applies not only to hazardous actions taken by the vehicle, but also hazardous inaction due to performance limitations, driver inattention does not exempt ADAS features from achieving SOTIF compliance.

Even more, within a SOTIF framework, the key question is whether automakers intend for the feature to be available at night. If an automaker’s PAEB function is not actually intended to operate safely in dark conditions, one option is to limit the feature’s intended ODD. This limitation would need to be made transparently clear to the public, not only as a disclaimer in the user’s manual but listed in the feature description on marketing materials, and perhaps through other consumer-oriented methods to be explored below. Numerous examples exist of ADAS feature names that include terminology related to the feature’s intended ODD. For example, “Traffic Jam Assist” indicates that the feature will function only at low speeds, and “Highway Pilot” indicates that the feature is not intended for urban or country roads. Similarly, current camera and radar-based PAEB systems could be labelled as “Daytime PAEB,” because they cannot be expected to function safely in nighttime conditions.

If automakers whose PAEB fails in dark conditions intend for this feature to be available and provide safety at night, they need to improve the feature’s performance, most likely by enhancing their vehicles’ perception capabilities. Tests show that lidar (light detection and ranging)-based PAEB solutions perform significantly better in nighttime conditions than current camera and radar-based approaches.11 This example illustrates the importance of sensor redundancy, or the utilization of multiple sensor modalities, within the SOTIF context. Often, sensor redundancy is cited as an important consideration for achieving functional safety compliance, in that if one sensor malfunctions another sensor is there to avoid or mitigate hazardous outcomes. Within a SOTIF framework, though, sensor redundancy provides the benefit of expanding the ODDs of vehicle features. While one or even two sensors in combination might have weaknesses that lead to unintended performance limitations, implementing additional sensor modalities can cover these weaknesses and thereby extend functionality to a broader range of scenarios. Adding lidar to current camera and radar-based PAEB solutions, thus extending functionality from daylight to nighttime scenarios, is just one example illustrating this concept.

Achieving SOTIF for PAEB at various speeds

Along with lighting conditions, vehicle speed is another parameter for defining safety function ODDs. The speed at which features such as PAEB can be expected to operate safely is largely an outcome of the maximum object detection range enabled by onboard sensors and perception software; longer perception range translates to function availability at higher speeds. Conversely, exceeding the speed defined in a safety function’s ODD can expose a hazardous performance limitation.

The table below provides a sample framework for defining the speed constraint of PAEB functions’ intended ODDs. The minimum required distance for a vehicle system to detect and identify a collision- relevant obstacle can be deduced from the maximum speed at which a PAEB feature is intended to function. The minimum required detection ranges listed below factor in the distance travelled by the vehicle during the time it takes to determine that an imminent collision requires braking. This calculation is based on the expectation that onboard perception software will detect an imminent collision after processing five frames of data at 50 milliseconds per frame, or 0.25 second total. The table also includes the distance traveled during a brake actuation response time of 0.1 second and includes a two-meter buffer between the vehicle and object when the vehicle comes to a stop.12 Finally, the minimum detection ranges factor in the braking distance required to stop when travelling at the intended maximum speed. The braking distances listed below reflect performance of an automobile in good condition travelling along a level and dry highway without loose material.13 Calculating the perception range required to come to a full stop is a more conservative approach than considering the range required to enable a safe lane change. An even more cautious approach could be formulated by considering the stopping distances on wet or down-sloping roads, or by including a slower data frame rate or processing speed.

A table like the one above is useful during the design and verification phases of PAEB development to ensure that the integrated sensor, software, and actuator components support the intended ODD of the function. Importantly, this analysis moves beyond the level of sensor specifications, such as range, resolution, and field of view to evaluate the range at which vehicle systems can produce and process sensor data to determine that a detected object requires vehicle response. It highlights the importance of perception software in maximizing safety function ODDs. Vehicles must not only be equipped with sensors capable of providing data of the requisite quality and quantity but must also process this data with software that enables their systems to translate sensor output into appropriate action.

Multiple combinations of sensor, software, processing, and actuation componentry are possible that can enable PAEB functions to safely operate at high speeds. For example, it is possible that certain software approaches are better than others at performing perception tasks with relatively sparse input data from the vehicle’s sensors. As a result, assessment and validation of vehicle function ODDs shall occur based on their performance at the system level, rather than being based on specific performance metrics of any given component or sensor. As we will see, validation of vehicle safety functions at the system level also produces results that can effectively convey performance limitations to consumers.

Implementing ODDs in consumer education

A use case approach to defining and validating the ODDs of safety functions enables vehicle performance to be assessed and described in standardized, transparent ways. In turn, this approach fulfills the need for a straight-forward method of supporting comparisons of market offerings conducted by safety-conscious consumers. Identifying and testing against the key performance parameters of various safety functions enables vehicle ODDs to be systematically defined and assessed. The results of these assessments can be clearly communicated to consumers via a rating system, as proposed in a previous paper.14 Even more, quantified assessment results can be combined with ODD descriptions on new vehicle stickers, such as the one below.

*AAA, Consumer Reports, J.D. Power and National Safety Council Recommended Nomenclature for Advanced Driver Assistance Technologies

Like current crashworthiness ratings, the data for a sticker to inform consumers about the ODDs of safety functions would need to be gathered through tests conducted prior to product release. While this would require heavy testing initially, as new features are often improved iterations of previous offerings, tests in subsequent years would be able to focus on updating ODDs based on improvements in functionality. By designing, evaluating, and marketing vehicle safety functions based on the use cases in which these functions demonstrate reliably safe performance, automakers, regulators, and testing organizations can help ensure that the public’s expectations are properly aligned with vehicles’ capabilities. This SOTIF-based approach provides not only a guideline for intentional development of safety functions, but also serves as a useful framework for encouraging well-informed, responsible, and safe implementation on roadways.

References

  1. Schwalb, Edward, “Analysis of Safety of the Intended Function,” National Highway Traffic Safety Administration, 2019.
  2. “Road Vehicles — Safety of the Intended Functionality,” ISO-PAS 21448, First Edition, 2019, pg. 1.
  3. Ibid, pg. 7.
  4. https://www.aaa.com/AAA/common/aar/files/Research-Report-Pedestrian-Detection.pdf,pg 8.
  5. https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/sae2017albrecht.pdf
  6. https://www.aaa.com/AAA/common/aar/files/Research-Report-Pedestrian-Detection.pdf
  7. https://www.ghsa.org/sites/default/files/2020-02/GHSA-Pedestrian-Spotlight-FINAL-rev2.pdf
  8. https://cdan.nhtsa.gov/query; National Highway Traffic Safety Administration (NHTSA) Motor Vehicle Crash Data Querying and Reporting; Pedestrians Involved in Motor Vehicle Crashes; Filter Selected: Light Condition: Dark - Not Lighted; Person Type: Pedestrian; Years: 2018.
  9. https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/sae2017albrecht.pdf
  10. “Road Vehicles — Safety of the Intended Functionality,” ISO-PAS 21448, First Edition, 2019, pg. iv.
  11. Ibid.
  12. “Pedestrian Automatic Emergency Braking (PAEB) Nighttime Testing by Velodyne Lidar,” March 18, 2021. https://www.youtube.com/watch?v=HQJRREwck6s.
  13. Prohaska, R. and Devlin, P, “Combined Brake and Steering Actuator for Automatic Vehicle Research,” California PATH Working Paper, 1998, pg 1.
  14. Code of Virginia, 46.2-880. “Tables of Speed and Stopping Distances”.
  15. Heeren, D. and Gradu, M., “An ADAS Feature Rating System: Proposing a New Industry Standard,” SAE Technical Paper 2019-24-0251, 2019.

About the authors

David Heeren, Director, Technology Research

David is Director, Technology Research at Velodyne Lidar. In this role, he examines product, industry, policy, and technology trends to help steer Velodyne’s product development and lead the company’s white paper projects. Prior to joining Velodyne, David earned a PhD from UC Santa Barbara.

Mircea Gradu, Senior Vice President Automotive Programs

Mircea started his career at Daimler-Benz AG in Stuttgart, Germany and prior to Velodyne he led engineering and quality at Hyundai Motor America. His past experience includes corporate officer in charge of Transmission Powertrain and Driveline at Fiat Chrysler and automotive executive at the Timken Company. Mircea has been awarded 56 patents on mechatronic automotive systems, published over 40 technical papers and is a board member and the 2018 President of SAE International. He was named one of the 50 most influential automotive executives by Motor Trend, and received the SAE Ed Cole Award for Innovation.

Mircea holds a master’s degree in mechanical engineering from the Polytechnic Institute of Bucharest and a doctorate in mechanical engineering from the University of Stuttgart, Germany.

For more information

Disclaimer: Text and images reproduced with permission from Velodyne Lidar. The views and opinions expressed are those of the authors and do not reflect the view, policy, or position of SAE.