MOBILITY ENGINEERING Produced by SAE Media Group
MOBILITY ENGINEERING
Magazines
Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Connectivity Solutions for AVs

The promises of fully connected autonomous vehicles are great, but so are the challenges.

The industry is working on all levels to optimize the new 5G and C-V2X connectivity platforms. Continental’s new flexible platform (shown) enables OEMs to equip vehicles for different technology preferences.

The synergies between electrification, connectivity and automation in future vehicles are creating new business models and causing developments in each to be accelerated. EVs, for example, need connectivity to assist with finding charging stations and autonomous vehicles (AVs) require significant levels of electric power to support their compute demands. But perhaps the greatest synergy lies between connectivity and autonomy.

Most AV developers, particularly for mainstream applications, want to minimize reliance on V2X connectivity because it might introduce interoperability requirements and a need for standards that may slow down deployment and eliminate competitive advantage. In general, increased connectivity creates cybersecurity risks and latency, coverage and band-width issues. Despite these drawbacks, there is broad agreement that AVs need connectivity for up-to-date high-resolution maps as road conditions and landmarks can change due to construction, for example.

For Mobility-as-a-Service (MaaS) applications — the AV as “robotaxi” — there is also a general understanding that passengers may need to have a direct connection with a call-center operator for situations where the passenger needs assistance—perhaps for medical reasons or for personal safety. This may require that the passenger presses a button to be connected to an operator or it could depend on remote sensing of the passenger (e.g. biometric screening in the seat and interior cameras) with automatic notification to the operator even in situations where the passenger is unable to initiate contact.

In late 2018 Continental, Ericsson, Nissan, NTT DOCOMO, OKI and Qualcomm Technologies joined forces to successfully perform cellular V2X trials. Conducted in Japan, the testing proved the viability of direct and network-based communications between vehicles, to the infrastructure and to vulnerable road users tested under varying conditions. The companies observed a mean latency of 20 ms for direct communication and a nearly error-free communication even at longer distances such as 1.2 km with unobstructed line-of-sight condition as measured by the C-V2X test system.

Health and wellness monitoring is expected to grow in importance as at-risk persons may be alone in AVs. Integrating information from the person’s smartphone with that provided by in-vehicle sensing can help improve passenger health and safety—especially if coupled with baseline data from a cloud service that stores personal information in a secure and private manner.

OTA’s promise — and challenges

Dr. Chris Borroni-Bird

Besides maps and customer service, connectivity is also likely to be required for Over-the-Air (OTA) software updates; ironically, the need for connectivity increases cybersecurity risks which drives a need for OTA updates. Software can continuously be developed throughout the vehicle’s lifecycle to increase the functionality of the vehicle’s embedded hardware (e.g. new ADAS or self-driving features). Or the vehicle might be tuned differently because actual customer usage may turn out to be different or less demanding than was assumed during development — such as extending the EV battery’s state of charge or tuning the suspension.

OTA updates bring with them two types of challenges, however. The first involves the practical matter of how to reliably execute the update. A safety-related OTA update needs to occur as soon as possible, but the vehicle may be in an area where there is no cellular or Wi-Fi connectivity at that time. Even if there is connectivity, the update may take a long time and cause inconvenience for the vehicle’s owner (this may be less of an issue with an AV fleet owner), especially if the update is interrupted and has to be restarted from scratch. The update may even require that the EV is not charging at the same time and delaying charging may mean reduced vehicle range and the associated inconvenience.

These issues, though troubling, are less worrying than the second type of challenge: ensuring the security of the vehicle and its associated cloud system service while the update occurs. Cybersecurity risks can harm the vehicle systems and render vehicle operation unsafe with loss of control and vulnerability to theft. Although non-connected vehicles are susceptible to hacking, the probability and severity of hacking will be significantly increased with connected vehicles in the future, especially connected automated vehicles which will have many more attack surfaces. Fortunately, mobility-tech companies are developing holistic approaches to end-to-end security between vehicle and cloud.

OTA software updates for critical vehicle functionality may need to be delivered from an OEM-controlled gateway to a dedicated modem on the vehicle, separate from the customer - facing one needed for delivering Internet content to the vehicle’s infotainment system (this need for cellular modems makes adding C-V2X functionality easier). OTA solutions are necessary for securing the vehicle throughout its lifecycle because even secure embedded vehicle electrical and electronic systems can become vulnerable over time to increasingly sophisticated cyberattacks if there is no updating of the security protections.

Robotaxi deployment hurdles

Several companies are developing solutions that will enable robotaxi operation in an automated MaaS model. This is SAE Level 4 where the AV may have no ability to be manually driven but can operate in a geo-fenced zone, such as part of a metropolitan area, that has favorable regulatory and climate conditions for AV deployment.

The currently favored approach to developing a robotaxi service is to do testing in the geo-fenced location environment with a safety driver who takes over control of the vehicle when the AV system is challenged to make the correct decision. This testing is intended to allow the AV software to be improved by learning how it fails in the real-world without jeopardizing safety for other road users. The work is complemented with extensive simulation of conditions that may occur very rarely during the miles accumulated in physical testing.

What makes the development of an AV solution even more challenging is that the vehicle not only needs to drive safely without causing any collisions, but it also needs to behave in a manner similar to that of human-driven vehicles (i.e. roadworthiness). It needs to be able to infer the expected trajectories of other road users and to be able to proceed in ambiguous situations and make necessary adjustments based on the actions of other road users.

It may be possible to achieve a viable performance using the AV’s embedded sensors and software, but this has not been proven in practice so far. It will be argued here that connectivity with roadside infrastructure and with a Tele-Operator may allow a viable AV to be developed sooner without jeopardizing road safety and traffic flow.

Various reasons have been given for why V2V has not been considered integral to AV deployment. There has been a lack of global (and even regional) agreement on standards for DSRC and the emerging challenge it faces from C-V2X has exacerbated this. The benefits of V2V require large-scale deployment which will take several years to achieve. Meantime, there is no perceived business model incentivizing automakers to introduce it on their vehicles. Infrastructure-based solutions for improving commercial vehicle operation have been limited mainly to pilot programs and don’t have the same scale as for equipped passenger vehicles.

Since some cities are keen to promote the deployment of AVs because they promise to improve mobility and safety, it is reasonable to imagine that funding for I2V installations (e.g. vision-based sensors plus wireless communications) at those locations which are challenging for AVs could help to accelerate safe and viable AV deployment in perhaps a more cost-effective and timely manner than without such support. A simple example might be that of an AV having trouble quickly negotiating an intersection in a safe manner as it climbs a hill. With the addition of vision-based sensors, such as camera, lidar or infrared camera, information about presence of absence of cross-traffic and of potential violations can be wirelessly communicated to the AV and allow it to make a more robust and faster decision on how best to proceed.

The additional cost to the infrastructure and vehicles in the fleet needs to be weighed against the extra time and effort required to improve the software to a point that the same level of safety and traffic flow is achieved. In this scenario, the AV fleet does not need to communicate with other vehicles and the number of roadside installations is finite and relatively small. If a city funds the roadside installation it is possible that this could be used for the benefit of all road users and not just the AV fleet developer, whereas if the AV fleet developer were to fund the installation this may or may not be the case. As the number of vehicles in an AV fleet increases there could also be value in each AV being able to sense their surroundings and share this information with other vehicles in its fleet.

Tele-Operation

Recent events in 2019 have illustrated the potential benefits that may be realized if Tele-Operation is used to complement AV deployment. One incident involved a driver sleeping while in Level 2 autonomy mode. There are also complaints from the general public that robotaxis can stop unexpectedly and hesitate to advance, necessitating a switchover to manual mode. In both cases, a Tele-Operator could conceivably take over control of the vehicle and drive it to safety or enable it to proceed forward without holding up traffic.

It can be argued, in fact, that Tele-Operation may be necessary in the near-term for a fleet operator because it not only improves passenger satisfaction if the vehicle is not overly cautious, but it can also reduce the manpower to rebalance stranded vehicles. Corner-cases that can confuse AVs and cause hesitation may hurt customer acceptance but tele-operation might alleviate the problem and allow a viable robotaxi service to be implemented sooner. Since passengers in the robotaxi may need to contact a remote service center operator for other reasons (e.g. for safety, security or comfort) this connection may already exist between the vehicle and a remote operator.

Tele-Operation solutions require that the:

  • vehicle senses its environment
  • vehicle communicates to a remote Tele-Operator its 360° awareness of the situation
  • Tele-Operator sends commands to vehicle to control braking, steering, acceleration

Stereoscopic images or 3D sensor date (e.g. lidar, radar and camera) representing what the vehicle sees is probably necessary because it is more difficult to determine depth visualization with monoscopic images. Color images are easier for humans to process than black and white images (shadows, in particular, make it harder to see with monochrome images). Virtual Reality Head-Mounted Displays (HMDs) tend to provide a more compelling immersion experience for the Tele-Operator than traditional large simulator screens. HMD security authentication will be necessary in order to permit secure access to the vehicle in order to take over control.

Low wireless communications latency is critical between the vehicle and Tele-Operator and should be at least 10 times faster than the vehicle’s mechanical latency. Therefore, the time for both uploading high-quality sensed information, remote processing and downloading commands will need to be <<100 ms. This is particularly challenging for highway speed operation and may justify initial deployment with robotaxis operating in a city center where vehicle speeds are typically <50 kph (30 mph). Bandwidth requirements can be reduced with smart data compression protocols and by sending only the “delta” image to the Cloud (and perhaps with only the edges of the object rather than the entire object).

A high bandwidth connection will be needed in case of decreased performance (multiple users, obstructions, etc.). It may also be appropriate to select routes for where the signal strength is known to be high if it does not affect vehicle safety or trip duration negatively.

The vehicle needs to have, at all times, a planned emergency stop function in case connectivity fails. The Tele-Operator, ideally, has approximately 10 seconds to adjust to seeing a new environment when they are required to engage. Actuator control is relatively straight-forward for by-wire vehicles so this should not be an issue.

Although Wi-Fi can be faster than 3G or even LTE, the latter should be adequate and may enable large area coverage more affordably. Future 5G systems with pico-cells can further improve Quality of Service (coverage, bandwidth, latency). Rather than sharing with other road users, a dedicated cellular connection might improve service reliability. Moreover, each fleet operator may decide to have its own Tele-Operation center since knowing what causes the Tele-Operation system to engage will be information for the company and tied in with its proprietary software development.

The cost of the hardware to enable tele-operation can be modest, especially for low speed robotaxi operation, and it should be possible for one Tele-Operator to manage many vehicles since robotaxis typically do not fail at the same time and/or can be scheduled to move at different times. Robotaxi development can learn from each “remote control” engagement and this should lead to improved algorithms for all the vehicles in the fleet.

Tele-Operation could become a more active area for development because corner-cases can delay robotaxi deployment and business model monetization. Some of these applications could include:

  • failure for the driver to respond to take over, as requested, in L2+ operation
  • taking over if the AV systems degrades below a certain level (either due to system failure or environmental conditions) and manual control is impossible
  • augmenting robotaxis in the shared vehicle fleet.

Other examples might include moving autonomous, electric vehicles at night to charging stations and returning them to the customer’s parking spot in the morning prior to being used by the customer; this may be particularly attractive in regions, such as China, where homes may not have a charge port and where a remote charging infrastructure can be less expensive to install and have greater utilization to reduce costs. Another example might be driving the passenger(s) in autonomous mode to a local hospital or caregiver or even, in some cases, locking the vehicle and driving the vehicle to a local police station. While these corner-cases can be achieved without Tele-Operation, the use of Tele-Operation can make the performance more reliable and improve the perceived safety of the robotaxi fleets.

Changing the competitive dynamic

It is possible that, over time, the AV software becomes so good that Tele-Operation is no longer required. However, in the near-term there are substantial challenges to solving the many corner-cases and there can be a benefit in leveraging Tele-Operation to accelerate AV commercialization. An AV company wishing to lead in market deployment must balance safety, roadworthiness and business viability. How safe is safe enough?

One approach to reducing some of this uncertainty might be to leverage connectivity for V2I and Tele-Operation to ensure safety and roadworthiness are acceptable from the start and to have the AV software learn from these connectivity interventions. This might allow an AV company having slightly inferior AV software to still offer the best overall AV performance. Over time, improved software from machine learning could allow reliance on Tele-Operation connectivity to be reduced while I2V connectivity should evolve to V2X as more vehicles are able to communicate with the infrastructure and with each other. This is likely to happen as vehicles increasingly come with an embedded cellular connection (to support enterprise diagnostics and OTA updates as well as to provide infotainment) and C-V2X functionality can be added relatively easily and affordably. This should reinforce safety and might even allow AV costs to be reduced over time, as sensing hardware might be simplified if V2X connectivity is ubiquitous in the geo-fenced location or even more broadly.

The development of safe and well-behaved AVs is essential to their social acceptance and implementation, but it is insufficient from the perspective of optimizing the road transport system. City planners are also aiming to improve traffic and connectivity will be needed to ensure that the actions of independent AVs are co-ordinated to smooth traffic flow and increase throughput for the same average traffic speed, or level of congestion. In addition, future urban transport systems will need to emphasize multi-modal integration and encourage diversity of transport. This includes “first-mile/last-mile” micro-mobility modes, so that the optimal solution for individuals and the city as a whole are more closely aligned. It can only effectively be achieved when there is wireless connectivity linking the different modes.

Dr. Chris Borroni-Bird is co-author of Reinventing the Automobile: Personal Urban Mobility for the 21st Century, with Dr. Larry Burns and the late Prof. Bill Mitchell (MIT Press, 2010). He has led advanced automotive-related activites at Chrysler, GM, Qualcomm MIT Media Lab and Waymo. He is the founder of Afreecar LLC, where he consults on future mobility and is developing a novel solar-powered solution for the developing world. He holds 50 patents, many related to the ‘skateboard’ platform concept.

Topics:
Automotive Connectivity Software

More From SAE Media Group

    Magazine cover
    Autonomous Vehicle Engineering Magazine

    This article first appeared in the July, 2019 issue of Autonomous Vehicle Engineering Magazine (Vol. 6 No. 7).

    Read more articles from this issue here.

    Read more articles from the archives here.

    Top Stories

    INSIDERDefense

    Feature Image F-35 Proves Nuke Drop Performance in Stockpile Flight Testing

    INSIDERMaterials

    Feature Image Using Ultrabright X-Rays to Test Materials for Ultrafast Aircraft

    INSIDERManufacturing & Prototyping

    Feature Image Stevens Researchers Test Morkovin's Hypothesis for Major Hypersonic Flight...

    INSIDERManufacturing & Prototyping

    Feature Image New 3D-Printable Nanocomposite Prevents Overheating in Military Electronics

    INSIDERRF & Microwave Electronics

    Feature Image L3Harris Starts Low Rate Production Of New F-16 Viper Shield

    INSIDERRF & Microwave Electronics

    Feature Image F-22 Pilot Controls Drone With Tablet

    Webcasts