Making Data Logging, Replay, and Prototyping More Efficient

High levels of continuity and compatibility are vital to avoid interruptions in the development process – and reduce cost.

July 1, 2020

Frank Mertens and Marius Müller

Because different sensor types have different strengths and weaknesses in recognizing information, the data is superimposed during sensor fusion to create the most realistic environmental simulation possible. (Image: dSPACE)

The development of advanced driver assistance (ADAS) and highly automated driving systems poses great challenges for OEM and supplier engineers. It is important, therefore, that powerful, flexibly configurable development systems optimally support the various use cases in data-driven development to make it as efficient as possible.

In the case of functionality for both ADAS autonomous driving, the chain of effects can be divided into several steps, starting with environment detection with the aid of extensive sensor technology – including camera, radar, lidar and ultrasonics – and ending with the final implementation of the driving strategy and actuator control (Fig. 2).

Fig.2: Chain of effects for driver assistance and autonomous driving. (Image: dSPACE)

In the next step, perception, the recorded sensor data must be interpreted in order to recognize objects and their relative movement. Because different sensor types have different strengths and weaknesses in recognizing information, the data is superimposed during sensor fusion to create the most realistic environmental simulation possible.

In combination with other environment information based on map data, vehicle position, and vehicle-to-everything (V2X) information, for example, this environment simulation forms the basis for predicting the vehicle movement. This allows for the calculation of ideal trajectories for route planning and the specification of target values for vehicle control.

Development approaches

Various development approaches have been established to develop new functions as efficiently as possible across all steps in the chain of effects. First, new algorithms are often developed as prototypes and tested in the laboratory. In addition to classical control algorithms, for example, for actuator control, modern-day development increasingly uses methods of artificial intelligence (AI) such as neural networks for perception and sensor fusion. For their training, enormous volumes of real environment data are required. These can be recorded via data logging during numerous test drives in the field.

After validation in the laboratory, the overall functionality must be validated and optimized under real conditions and in the context of the entire vehicle by means of rapid prototyping. Subsequently, a production implementation is derived from the optimized overall function, which has to be tested in the laboratory by software-in-the-loop (SIL), hardware-in-the-loop (HIL) simulation, or increasingly data replay/reprocessing.

To save time and costs, the development phases – from data logging to prototyping and testing – must be as efficient as possible. It is important to ensure that the development system used can be optimally designed for the respective phase. There should also be as much continuity as possible between the phases, and systems re-use in all circumstances. The characteristics and specific variants of these systems in connection with the individual development phases are explained below in more detail using the example of the dSPACE Autera product family.

High-end data logging

Test vehicles are equipped with various high-resolution sensors to record the environment data. The sensors often provide the data to be captured via various raw data interfaces, such as GMSL, FPD Link or CSI, but also via Ethernet and CAN FD. The dSPACE automotive-grade AUTERA AutoBox offers a modular and configurable interface for data logging that allows for optimum adaptation to the sensors.

It is important that despite the modularity, all data is time-synchronously provided with an accurate time stamp, if required, even across system boundaries, and can be reliably backed up with high bandwidth to a data storage device that has several terabytes (TB) memory space. Therefore, data can be recorded with a bandwidth of up to 50 Gbit/s with the AutoBox. The storage size of 32 TB (in the future 64 TB) allows for uninterrupted measurement runs that last several hours even at maximum bandwidth, or days at a lower band-width. In addition, the memory units can be conveniently exchanged during operation, which extends the recording duration for test drives as required (Fig.3).

Fig. 3: Comprehensive prototyping solution for the entire chain of effects. (Image: dSPACE)

In this context, the possibility of integrating GPUs or FPGAs for intelligent data analysis must also be emphasized. It helps reduce data volumes to relevant data in a targeted and automated manner while the data is being captured.

To further process the recorded data in the laboratory, it generally has to be uploaded to data centers. The Autera Upload Station enables this via a 100-Gigabit Ethernet interface, for example.

Comprehensive prototyping

The above-mentioned requirements for data logging systems essentially also apply to prototyping systems in real test drives. Moreover, it is useful to record sensor data or internal calculation variables simultaneously during prototyping.

Prototyping, however, focuses on direct data processing for the validation and optimization of the algorithms. It is important that prototyping systems, in contrast to production hardware, have sufficient performance reserves so that restrictions are not necessary in early phases of algorithm development. In cases where AI technologies are used, it is also useful to be able to extend the system with hardware accelerators such as GPUs or FPGAs. Therefore, the AutoBox is based on server technology and optionally allows for the integration of powerful hardware accelerators from different manufacturers.

In cases where a final actuator ECU is not yet available (or where an existing one has to be developed further), the use of separately and scalably coupled rapid control prototyping (RCP) systems such as the dSPACE MicroAutoBox has proven to be a good solution. They can be used to conveniently implement and execute control algorithms with strict real-time requirements as well as directly control the actuators via discrete I/O interfaces with the aid of power output stages.

Besides the hardware, the system software plays a significant role in efficient prototyping. Generally, these prototyping systems use Linux-based operating systems, which offer a high degree of flexibility and versatility. For the specific implementation and execution of the algorithms, they are often extended by specialized software, for example, Robot Operating System (ROS) or RTMaps from Intempora. To increase usability and productivity, tools such as RTMaps have an intuitive graphical user interface for the configuration of sensor technology as well as algorithm implementation based on C++, Python, or CUDA.

The algorithms developed this way now calculate the control variables for controlling the vehicle (brake, steering, chassis, drive, etc.) and must be able to exchange data with the actuator control units in the vehicle network. This requires the support of the various automotive buses and networks, including frequently used description formats such as AUTOSAR XML or FIBEX. The Autera AutoBox can therefore be tool-supported and configured for communication via CAN FD, Ethernet (1000BASE-T, 1000BASE-T1, 10GBASE-T), and, in the future, FlexRay on the basis of these description formats.

Continuous data replay

Frank Mertens (left) is director of strategic product management, Real-Time Test & Development Solutions. Marius Müller (right) is product manager for prototyping systems. Both are with dSPACE GmbH.

After its validation and optimization in real test drives, the algorithm must be implemented in the series ECU and then tested in the laboratory using data replay/ reprocessing, among other methods. Especially for the validation of perception and fusion algorithms, the data recorded and annotated during data logging (ground truth) must be played back chronologically and highly synchronously to the way they were originally recorded.

To keep the effort in the development process low, high continuity and compatibility from data logging to the data replay system is advisable. Ideally, a similar system structure is used, for example, to reuse the storage units from data logging and to play back the data from these units during replay. If the data is stored in data centers and a direct streaming connection is desired, the data replay system can also be directly connected with high bandwidth.

If transmission fluctuations occur while streaming data from the data centers, sufficiently large caches in the data replay system can compensate for these fluctuations by buffering the data. Also, data replay systems can be used for closed-loop applications by executing synthetic sensor models or coupling them with HIL systems.

A scalable system architecture with high computing power, high bandwidths, and a variety of expansion options, such as powerful data storage, hardware accelerators as well as raw data, bus and network interfaces in combination with comprehensive software forms the basis for an efficient development system. A high level of continuity and compatibility from data logging to prototyping and data replay is also important to avoid interruptions in the development process.