Speeding AV Testing with a Digital Twin

rFpro creates a digital twin of University of Michigan’s Mcity testing facility to accelerate AV development.

Traffic lights, pedestrians, vehicles and a buildings’ facade; the downtown scene in Mcity’s digital twin. (UofM)

A European company has developed time-saving simulation software for users of the University of Michigan’s Mcity CAV (connected and autonomous vehicle) testing facility in Ann Arbor. It has created what it terms a “highly accurate digital twin” of Mcity with precise details of materials used and its roadside infrastructure. Applying survey-grade lidar equipment, U.K.-based rFpro has captured a digital copy of the 32-acre test area to a high level of accuracy.

Not a specter but an Mcity pedestrian-crossing test. (University of Michigan)

“This enables vehicle manufacturers and technology suppliers to greatly accelerate development using simulation,” said Matt Daley, rFpro’s managing director. He explained that established methods to create training and test data for CAV systems involve the manual annotation of real world data, frame by frame, to pick out features such as pedestrians, other vehicles, road markings, road signs, and traffic signals so that the software correctly identifies everything.

“It can take several man-years to annotate an hour of source video,” Daley said. “Doing this in a virtual world enables manufacturers to create synthetic data up to 10,000 times faster - with none of the human error of manual annotation - allowing the amount of testing to be massively increased.”

V2X and 5G

The purpose-built Mcity outdoor lab incorporates 16 acres of roads and traffic infrastructure, a railroad crossing, short tunnel, a steel-deck bridge, roundabouts and a mix of road surfaces. Mcity’s capabilities include V2X and 5G connectivity. Tests at the facility can also be used to validate the simulated tests and ensure high accuracy.

The Mcity digital twin adds to rFpro’s library of similar work, including Applus+ IDIADA in Spain and Millbrook in the U.K., and motorsport venues such as the Nürburgring and Pikes Peak. Using a digital twin of a proving ground helps OEMs speed development by testing in a fully representative virtual environment before validation on the actual track. Daley explained that most artificial intelligence (AI) systems, particularly those that deploy supervised learning, learn by experience. Their training data sets must continually grow to include any new situation that might be faced in the real world. But physical testing typically is too slow.

“Testing in a virtual environment is now the only cost effective and also time effective way to introduce self-learning systems to the limitless number of scenarios that can occur in the real world,” he asserted. “It permits a CAV to interact with any combination of other roadusers, vehicular or pedestrian, in any weather or lighting conditions, safely and repeatably.” Ultimately, the CAV is still validated in the physical environment, so a digital twin complements the proving ground rather than competing with it.

Phase-based scanner

Highly accurate rFpro digital model of an example wet road surface, including puddles and lighting. (rFpro)
Matt Daley, rFpro’s managing director: Testing in a virtual environment is “the only cost-effective and also time-effective way to introduce self-learning systems to the limitless number of scenarios that can occur in the real world.” (rFpro)

Daley cites absolute accuracy as the overarching priority for the success of a digital-twin creation: “We capture road surface detail to within 1mm from phase-based laser scanning survey data, providing accurate surface information into our customers’ vehicle models. This enables the simulation of ride quality, road noise and other factors which affect the comfort of CAV occupants.” It also ensures that sensor mountings are exposed to representative vibration levels to ensure they function correctly over different surfaces, he said.

Road layouts, lanes and junctions are incorporated within the rFpro digital-twin simulation using the industry’s three most common formats for network descriptions: OpenDRIVE, IPG ROADS and SUMO formats. This ensures compatibility of the rFpro software with all major vehicle modeling tools at Mcity and other sites. The Mcity control network collects data using wireless, fiber optics, Ethernet and a real-time kinematic positioning. Patent-pending augmented reality (AR) testing technology allows physical test vehicles to interact with virtual connected vehicles in real time inside the facility .

Huei Peng, the Roger L. McCarthy Prof. of Mechanical Engineering at the University of Michigan, said: “The development of CAVs is safer, faster, and cheaper by strategically combining controlled tests and simulations, rather than relying too heavily on public road testing.” Added Daley: “Proving grounds have always been very popular with our OEM customers and rFpro has produced private models of those areas for many years. What is exciting now is the range of publicly accessible models we have available including locations like Mcity that have been purpose-built for autonomous vehicle testing.”