Silicon Valley Summit Refines Autonomous Safety Thinking

World Safety Summit on Autonomous Technology looks to limit operational design domain (ODD), some noting SAE Level-5 autonomy no longer a relevant goal.

A Zoox developmental AV in Las Vegas; a company executive said Zoox intends to create SAE Level 5 vehicles but relegate them to Level 4 mobility service. (Zoox)

Until recently, leading autonomous vehicle (AV) technologists posited that ultra-safe, go-anywhere robotaxis would soon be on the road. But questions about those timelines –and the safety of testing AVs on public roads – emerged in 2018 after a series of high-profile AV accidents. Consumers have since been caught in the middle between promises of eliminating highway fatalities that conflict with real-world incidences of deadly crashes.

At the World Safety Summit, Aurora’s Chris Urmson said that building a culture of safety is the paramount factor in AV development. (Velodyne Lidar)

The World Safety Summit on Autonomous Technology was organized by Velodyne Lidar to help clear up the confusion. The second annual event, which took place in Santa Clara, Calif. in October, gathered some of the AV industry’s sharpest thinkers. While attendees did not walk away from the day’s speeches and panel sessions with a well-defined roadmap, the summit yielded several common-sense guidelines for addressing AV safety.

All AVs have domain limitations

If there’s only one insight derived from the 2019 World Safety Summit, it’s that no self-driving vehicle can safely perform all driving functions “under all conditions.” This is effectively the SAE International definition of Level 5 automated operation in its universally known J3016 Standard. Times have changed since the standard was issued in 2016 and presenters at the summit now acknowledge that Level 5 is aspirational, and not a relevant goal.

Marta Hall, president and chief business Development officer for Velodyne Lidar, believes that AVs should operate at lower speeds to increase safety. (Velodyne Lidar)

“I don't even think about Level 5,” said Larry Burns, the former vice-president of Research and Development for General Motors. “It's not even on my radar screen.” Burns, who is also the author of the recently released book Autonomy, gave one of the event’s three keynote speeches. SAE’s system of levels was designed only as an initial conceptual framework. “As an engineer, I felt they were quite liberating and ingenious,” Burns said. But the conversation has since shifted to defining a vehicle’s operational design domain (ODD), the many scenarios and conditions that limit their operations.

Companies such as Zoox, which are creating AVs without steering wheels or foot pedals, also acknowledged that Level 5 operation is beyond its scope. “Zoox is a Level 5 vehicle that is optimized and designed to be able to perform in almost any environment, but it will be a Level 4 [conditional automated driving] service,” said Bert Kaufman, head of corporate and regulatory affairs at Zoox. “It will be constrained to ODDs.”

The geographical region is a common ODD. For example, most AV companies will entirely avoid certain zones, particularly rural areas. Jean and Eddie Rowe traveled to the summit from their hometown of Schaefferstown, Pennsylvania to participate in the panel, “The Voice of the AV Rider” and recently traveled to Detroit to take their first ride in a self-driving vehicle. They were impressed. The Rowes, as aging baby boomers, believe they could greatly benefit from the mobility provided by AVs. However, they question if a self-driving service will reach their region during their lifetime. “Where we live, there are roads that don’t have any lines,” said Jean.

Velodyne Lidar and other vendors demonstrated their technology in the Levi’s Stadium parking lot. (Velodyne Lidar)

Regardless of geographical constraints, setting speed limits also will be helpful, according to Marta Hall, president and chief business development officer at Velodyne Lidar. “I hear our engineers talk about the complexity of the problem with full autonomy,” she said. “I’ve also heard them say that traveling at 35 miles per hour (56 km/h) is one-thousand times less complicated than 65 miles per hour (105 km/h).”

Hall said that AV fleet customers using Velodyne Lidar want their sensors to have a range of 200 to 300 meters (656 to 984 ft). But that long-range perception should be combined with limiting vehicle speed to about 35 to 40 miles per hour. “That's the key,” she said. “Slow it down so the car can react in time and get everything correct.”

More simulation, fewer miles on public roads

AV companies proudly report the number of miles – sometimes in the millions – that test vehicles travel on public roads. More miles, it has been assumed, means giving AV algorithms more opportunity to learn about roadway behavior. At the same, logging a lot of miles allows engineers to better understand challenging conditions that force autonomous systems to disengage.

However, panelists at the summit questioned the value of prematurely placing AVs on public roads, only to study how they fail. “Disengagement reports really don't mean anything,” said Danny Shapiro, senior director of automotive at Nvidia. “When you're testing in the real world, the safety driver takes over and never gives the system a chance to determine what it would actually do to prevent that accident.” Shapiro instead advocated that companies use extensive simulation to study dangerous scenarios with numerous random factors such as time of day and weather.

Colm Boran, senior manager of AV system engineering and safety at Ford, agreed. “There's a lot of work to be done in a simulation environment” to ensure that the AV’s software is “vetted and tested in lots of scenarios first.” Boran said that simulation should be followed by similarly rigorous testing on closed test tracks. “After we've become satisfied with that, then we put them on public roads,” he said.

Empower safety drivers to ground vehicles

Chris Urmson, co-founder and chief executive of Aurora, shared some of his company’s safety strategies, starting with the critical role played by vehicle safety operators behind the wheel. In the event’s opening keynote speech, Urmson said that Aurora’s operators are employees rather than contactors. “We want them to have an ownership stake in the outcome of what we build,” he said. Furthermore, Aurora’s safety operators are aggressively screened and then put through a six-week training process.

“Safety is not one of the things that you bolt on at the end of a process,” he said. “It's something that you have to be thinking about throughout the process.” For Aurora, that means giving all employees the power to ground vehicles. “When they see something that's not quite right, let's bring everything back, shut it down, and understand how we can do better.”

Meanwhile, SAE recently updated is J3018 Standard, which provides AV testing guidelines. The new rules stipulate the required skills needed for testing prototypes at various stages of development, how long a test driver can work without a break and how to ensure that the safety driver maintains attention.