'Road Race' for AV Testing May Be Slowing

So real it's not real: a highly accurate digital model of a nocturnal wet-road surface using rFpro’s simulation technology. (rFpro)

Chris Hoyle, Technical Director of software specialist rFpro, believes the race by auto and technology companies to be ahead of competitor programs involving autonomous vehicle (AV) testing on public roads is losing momentum.

Digital "twin" of real road at Applus+IDIADA proving ground in Spain created via Lidar scans. (rFpro)

“This trend follows reports of autonomous vehicles being involved in fatal accidents. Now there is more competition to deliver the highest level of validation before a vehicle turns a wheel on the highway. This re-alignment of priorities is good news for all parties because it will accelerate improvements in AV safety,” Hoyle asserted.

To support this, the world’s proving grounds and test tracks are likely to take on an ever more significant role in the pursuit of establishing AV safety, but with the need to narrow the testing gap between virtual and physical environments. To help achieve that, rFpro has developed what it terms “digital twins” (virtual and physical) of proving grounds in Spain with Applus+ IDIADA and the UK with Millbrook. Hoyle believes this capability could be used by vehicle manufacturers, system-suppliers and government agencies for the testing of AVs to be safe for use on public roads.

In a bold statement he told Automotive Engineering: “Such twinning will reduce, or eventually obviate, the need for on-road autonomous-vehicle testing.” The software for AVs (and ADAS) needs to be trained and tested on a digital copy of the proving ground or other test area before the vehicle is ready for real-world testing.

Autonomous vehicle testing in simulation. (rFpro)

Phase-based laser scanning survey data is used by rFpro to create models with an accuracy to about 1 mm in Z, X and Y axes.

A digital model of Millbrook (or any other proving ground) becomes a simulated part of an OEM’s continuous integration (CI) software development toolchain. A virtual rFpro test track can be populated by an OEM’s development vehicles together with swarm traffic (generic traffic following the rules of the road) and programmed traffic (tasked to carry out a specific maneuver).

At Applus+IDIADA’s facility, lighting is modeled for day of the year, time of day and weather conditions at its specific latitude and longitude. Ad hoc behavior by pedestrians suddenly stepping into the road is a part of its capability. Another significant aspect of rFpro’s software is its TerrainServer model that facilitates a high-definition simulation of a road surface. As well as modelling proving grounds, rFpro has an extensive library of public roads, including those in many capital cities across the world, plus motorways, autobahns and U.S. interstates. The company also helped Jaguar Land Rover (JLR) to model its customer evaluation route in England, enabling the manufacturer to validate results on roads in the immediate area of its Gaydon R&D center in the English Midlands.

Library of AV scenarios Now Hoyle envisages the creation of a national–and ideally, global—virtual library of possible scenarios to which AVs would be exposed, with every new failure mode discovered added to the database. For an AV to be “certified” as ready for road testing, it would have to successfully negotiate hundreds of thousands of such scenarios to prove that it was an order of magnitude safer than a human driver.

“The final step would be to precisely replicate a sample of the tests at a proving ground to physically correlate the results,” Hoyle said.

Subjecting AVs to scenarios using models of public roads is critical to effectively training artificial intelligence (AI), stressed Hoyle: “The range of scenarios that could be faced by an AV interacting with other road users is almost infinite and you cannot ‘calibrate’ the response of AI systems because they have to learn by experience.”

rFpro technical director Chris Hoyle envisages an expansive digital library of possible driving scenarios to which AVs would be exposed. (rFpro)

It would mean that new situations would be continually encountered by development vehicles which, if undertaken on the road, would be too risky and too slow to accumulate the necessary experience. But if the systems could be validated first in a virtual environment, it would save time and cost as well as improving safety.

But the creation of a suitable virtual environment introduces its own challenges, from identifying and selecting the scenarios to be tested to ensuring the accuracy of the simulation and reaching agreement on the appointment of an independent agency, such as Euro NCAP, to administer the tests.

Demonstrating that simulation is a valid replacement for large elements of on-road AV testing will significantly reduce development time and cost and improve the safety of other road users, stated Hoyle. He expects successful project completion will lead to a legislative framework for AV certification that includes a virtual sign-off before real-world trials could begin.

“The key to success is the level of accuracy we achieve when replicating the real world in simulation, which enables the various sensors used on AVs to react naturally, making the test results completely representative,” Hoyle explained. “Our library of real roads created through highly-precise scanning technology forms the basis of the simulation. As it is a digital platform, users have control of all the variables, such as traffic, pedestrians, weather and location, enabling them to test every eventuality.”

Helping AI act like a human

Hoyle is confident that virtual testing will help manufacturers’ use of a more-sophisticated approach in algorithm development, taking account of AV occupant needs as well as safety requirements.

“Current efforts are largely directed towards the avoidance of other road users, but for AVs to achieve widespread consumer acceptance, they must provide a comfortable and reassuring experience for the vehicle occupants,” stressed Hoyle. “An experienced human driver doesn’t drive in a straight line, but plots a path around potholes and broken surfaces and reacts progressively to changes in distant traffic lights, or anticipates the actions of merging traffic.”

If the reactionary nature of human behavior is not factored in, the supervised learning process used by the AI in a AV would produce fundamental errors in the architecture of the algorithms, potentially requiring reworking completely from scratch, Hoyle said.

“Simulation enables manufacturers to introduce humans into the development cycle at a much earlier stage, preventing wasted time training AI. Best of all it’s perfectly safe; nobody can actually die in simulation!”