Virtual Validation on the Rise, but Physical Testing Remains Crucial

April 17, 2018

Terry Costlow

Members of the “Validation Is the Path to Getting on the Road for Automated Vehicles” panel at SAE International’s WCX 2018 conference (image: Terry Costlow).

The more vehicles do to take control and perform safety-related maneuvers without driver input, the smaller will be their margin for error. Ensuring that all the sensors, controllers and actuators work exactly as expected will require immense amounts of validation and verification.

Ford's James Buczkowski speaks in a session at the WCX 2018 conference in Detroit (image: Terry Costlow).

Designs are usually made with models, facilitating simulation during design. That sets the stage for validation, which helps ensure that systems meet customer requirements—and verification, which looks deeply into actions to confirm that systems function as expected.

Panelists from a range of transportation companies examined these issues in a panel, “Validation Is the Path to Getting on the Road for Automated Vehicles” at SAE International’s recent WCX 2018 conference. The consensus: although virtual testing plays a dominant role in system development, it still hasn’t replaced physical tests with prototypes.

“No company has nailed it, where they can say the validations are perfect every time,” said Ford’s James Buczkowski. “Physical tests remain part of the process.”

Five years ago, many observers predicted that validation and verification would all but replace physical testing. However, the heightened effort to automate driving and push towards full autonomy may make it more difficult to do away with physical testing, even though it typically is far more expensive than virtual tests.

“Everything is moving towards doing all the testing in a virtual world,” said Paul Decker, National Automotive Center, TARDEC. “The challenges of autonomy, with things like sensor fusion and radar return, have pushed back the effort to do everything virtually.”

Regardless of whether developers are performing virtual or physical tests, measurement techniques may have to evolve. Most companies still use driving miles as their metric, even for tests conducted on virtual vehicles.

But aggregrate mileage may not be the best yardstick for automated and autonomous driving.

“We need to do some metrics, like saying the number of scenarios tested instead of saying it was tested for 100 million miles,” Decker said.

Meanwhile, physical tests may also change; simply driving the prototype vehicle down a freeway isn’t a good test of an automated safety system.

“Not all miles are created equal,” said Ed Straub of the American Center for Mobility. “You need to clearly define the operating domain and find areas that are critical.”