Horiba to Expand Virtual AV Vetting with Physical-Body Testing
Moving past simulation and stimulation, Horiba’s upcoming AV testing tool will use manipulated physical models to vet traffic scenario performance.
Measurement and analysis specialist Horiba has a host of virtual tools available for suppliers and OEMs to aid the development of autonomous vehicle (AV) sensor suites. The Kyoto, Japan-based outfit offers tools in the digital realm to model everything from a proposed lidar sensor to managing the complex task of sensor fusion for entire AV arrays. Horiba’s latest “virtual” tool, however, will use physical models to vet the performance of AV sensing.
In October 2021 in Novi, Michigan, Horiba demonstrated a scaled-down version of its upcoming lab-based Scenario CAV (Connected and Autonomous Vehicle) system, which can manipulate physical models around a vehicle under test (VUT) to simulate real-world scenarios. The new system will provide a stage of testing beyond fully digital simulation models, adding another layer of confidence to AV sensor suites in advance of real-world testing. The goal is to verify autonomous systems modeled in the digital domain actually “see” objects and obstacles as intended.
Beyond simulation and stimulation
According to Leo Breton, Horiba’s technology development director, testing protocols for AV sensors fit into three rough categories: sensor simulation, sensor stimulation and physical body testing. The first category, which often comes earliest in the development process, substitutes virtual signals for sensor output, bypassing the vehicle sensor with computer-generated signals designed to emulate that sensor.
“If you're sitting in a laboratory and you have test equipment all over, the sensor – a radar system, for example – is going to be depicting all of these objects that are in the laboratory environment,” Breton said. “We take the signals from the radar system and replace them with signals that have a meaning that we're interested in. The presence of a vehicle, five meters in front of the vehicle going at certain speed, for example, and see how the vehicle reacts. That's sensor simulation.”
Sensor stimulation, according to Breton, is where you are actively engaging the vehicle’s onboard sensors. “If you've got a radar system, for example, you could be generating a signal, a radar return,” he explained, that the vehicle receives that is not based on the signal that the vehicle emitted from its radar system. “It's a radar return that shows the presence of a vehicle or object at a certain location, moving at a certain speed. You're stimulating the sensors, but there's no physical connections between your test systems and the sensors on the vehicle.”
The third category, physical body testing, is in effect an extension of sensor stimulation, where physical models are located at appropriate times and locations to determine how the vehicle, guided by its AV sensor suite, reacts. “One example of that,” Breton offered, “is you have the vehicle out on a test track, and you have a fake pedestrian cross in front of the vehicle. That's a physical body and you can watch the reaction of the vehicle.”
Bringing the AV test track inside
Physical body testing is a crucial step in developing effective AV sensor suites, but if only available outdoors, it becomes weather dependent. Horiba’s new Scenario CAV system seeks to leverage much of the benefit of test-track-based physical body testing with the controlled environment and 24/7 testing ability of a lab. “The ultimate test is the real world,” Breton noted. “Once you put that sensor suite into a vehicle, it's possible that you get some kind of interference from the vehicle itself. So any of these things will be followed up by some overall type of physical body testing in the end.”
“I call this physical-body-type testing the ‘space telescope test,’ where you're testing the whole thing together without interfering with its operation,” Breton said. “When the Hubble Space Telescope was launched, it had a problem where the images were blurry. NASA had contractors who built the various modules, and each module was tested on the ground and each performed according to a spec, but the entire space telescope was not tested as a single system on the ground.”
The Scenario CAV setup, Breton claimed, came mostly from the desire within the industry to be able to test various standard scenarios, such as highway merges or emergency braking scenarios. The 1:24-scale benchtop prototype previewed in October (top) has given way to production of a full-scale setup, which is already underway.
“The demonstration prototype has overhead arms so that they're not interfering, not being picked up by the radar of the vehicle,” Breton said of the Scenario CAV system. “And it's moving these lightweight automotive lookalike objects that have similar radar cross sections to the back of a real vehicle, that can be moved around in desired relationships to the vehicle under test sitting on the dynamometer. But they're lighter and smaller than a real vehicle so they can be moved quickly with a lesser infrastructure.”
Confirming function and efficiency
Such a test system opens a wealth of development opportunities, Breton claimed, citing adaptive cruise control as just one system ripe for optimization. “There are multitude of questions related to testing the adaptive cruise. Does it work properly? Does it work as designed? What's the fuel economy impact of having the adaptive cruise on versus a human driver?”
Efficiency, Breton noted, will quickly become a hallmark for evaluating ADAS systems. “The fuel economy impacts are very important, but they're not getting much attention yet because [AV] functionalities right now are primarily talked about from the convenience and the safety points of view,” he said. “But they also have the ability to improve energy efficiency.”
Dyno-based physical body testing will permit development engineers to tune ADAS systems for both effectiveness and efficiency, a crucial aspect as more products become electrified. “Vehicles will share information with the infrastructure to know when traffic lights are red up ahead, and with other vehicles to do cooperative merging,” Breton noted. “All of those factors come into play in trying to maximize the efficiency.”
A second major function of a controlled-environment physical body test would be for regulators, letting them craft and enact standardized testing for AV functionality. “If you are a regulator, you're not going to be knowledgeable with all the technical details of each of the sensors,” Breton commented from an experienced perspective, having worked at the EPA for 17 years.
“You want to test the whole system, and you want to do away with all the assumptions that have been made,” he continued. “Every time you do a test of a specific sensor, you have to make some assumptions. But the final, ‘space telescope-type test’ is done with the elimination of all those assumptions. That's where this physical-body-type testing comes into play.”
Ahead of the AV curve
One of the biggest challenges for Horiba, Breton acknowledged, was keeping its development tools on pace with an industry advancing so rapidly. “Everyone wants these capabilities, but the sensors, the methods, the fusion algorithms, everything is evolving very quickly. All of these technologies on the vehicles are new, and so the testing technologies are just as new or even newer,” he said. “Typically, customers want more and more virtual work because it can be done in any weather and can be done indoors.”
“Any given sensor may not even be available in two or three years, so we can't be developing just based on the specific sensor,” Breton acknowledged. “We have to develop methods and technologies that go beyond a specific sensor. In the future, maybe everyone will settle out on one or two systems once things are fully evolved 10 to 15 years from now. But right now, things are evolving very quickly. We have to keep that in mind and not make our solutions dependent upon specific models of sensors.”