Navigating the Language of Vehicle Autonomy

While public misconceptions of ‘self-driving’ bloom, its engineers are conveying rapidly evolving technology towards clear autonomous demarcations.

Safe deployment of automated vehicle technology will depend on clear language demarcating various levels of autonomy. (Magna)

Call them what you may – autonomous vehicles, automated vehicles, self-driving vehicles – they are coming. But you can’t buy one yet, and like many technological revolutions, capabilities will arrive in stages. Such conveyances will methodically ascend a ladder of automated functionality over the next decade, eventually leading to something autonomous that can drive itself. Expect much confusion along the way. What’s in a name? A lot, potentially, as lives will be lost if end users, consumers, don’t understand the gradations of autonomy.

A graphic plotting escalating advanced driver assist technologies provides some sense of the complexity of overlapping technologies involved in vehicle automation. (Hitachi Astemo)

This was brought into sharp focus by a recent New York Times article  characterizing an SAE Level 2 system as “self-driving,” and the deserving backlash  that followed from enthusiast forums. Organizations such SAE International have been working for years with a host of industry partners and other standards organizations to alleviate such scenarios. This is a not an easy task, with technology advancing so quickly there are often not common terms to describe it.

The Partners for Automated Vehicle Education (PAVE) is a coalition of nearly 90 members (including SAE International) tasked with educating the public on autonomous vehicle technology. (PAVE)

One of the first productive steps along those lines was the 2014 publishing of SAE’s J3016  Recommended Practice, a “Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles.” This evolving document has helped define the vocabulary around which the industry is rapidly expanding, its most recent  and already fourth revision in April 2021 helping harmonize terms globally with ISO’s 22736.

Such cooperation among the world’s standards organizations speaks volumes to the importance of having a common tongue, as was the document’s availability. “I would give quite a bit of credit to SAE as an organization for making that document freely available as an SAE standards document. That is extremely rare,” explained George Nicols, an advanced technology standards engineer at Toyota Motor North America, who has been involved with SAE J3016 since 2017. “I think that speaks to just how valuable and significant the document is for distribution through the public.”

Media inflection point

George Nicols, an advanced technology standards engineer at Toyota Motor North America, who has been involved with SAE J3016 since 2017, points to the challenges communicating the shared role between current automated vehicle features and the driver. (George Nicols)
Lee Aho, regional business unit head for ADAS products for Hitachi Astemo, explained that “automated” connotates different meanings between engineers and the driving public. (Hitachi Astemo)
Sharath Reddy, Magna’s senior VP of Electronics, said “driver in the loop” provides a very clear distinction for its development teams. (Magna)
Nikhil Gupta, Magna’s senior engineering director for ADAS features, explained that even “simple” features such as automated lane change can involve significant compute assets depending on the operational design domain (ODD). (Magna)

The kerfuffle around the New York Times article was likely the first of many misguided descriptions around the language of autonomy, but could be seen more hopefully as an inflection point for the industry on which to build. Describing any SAE Level 2 system as “self-driving” could inspire dire consequence, but few automotive technologies are initially understood by the driving public (see “ABS”).

Ed Niedermeyer, the former communications director at PAVE (Partners for Automated Vehicle Education), was active in real-time via social media channels during the dust-up with the New York Times. The organization he represented is a coalition of nearly 90 members (including SAE International) tasked with educating the public on autonomous vehicle technology. “The AV industry talks about the need for public trust,” Niedermeyer explained. “In order to earn the public's trust, we all have to be speaking the same language, and actually understand what we're talking about. Education is a fundamental aspect of that, and how PAVE thinks about its role in this ecosystem.”

Niedermeyer noted that because all stakeholders are watching the technology evolve in real time, challenges will be ongoing. “The New York Times situation is a blatant example of the challenges-remaining part of it,” he said. “What that situation shows is the challenge is not just getting people aligned around language or even basic distinctions. Even more simple than the SAE levels is helping people understand – including the very intelligent and worldly folks who work at the New York Times – that these differences matter.”

“What was really tough about the New York Times situation is that not only did they make a factual error that potentially has safety implications, which would be disappointing enough on its own,” Niedermeyer explained. “But when this was pointed out to them, the editors said, ‘Americans don't understand the difference between driver assistance and autonomous driving, and we don't expect them to.’”

“That sort of attitude really galvanized us at PAVE, because it shows we have not done enough to explain why this distinction matters,” Niedermeyer said. “I think it's really easy for people outside of industry to ask, ‘Why are you nerds quibbling over some words that kind of all sound the same to us. What's the big deal?’ I think that we still have some progress to make on what's the big-deal part, as well as clarifying the distinction itself.”

Defining the autonomous convention

An ongoing challenge for engineers is a taxonomy that keeps up with the torrid pace of technology advancement. More so than many projects for a standards organization, it’s required near constant iteration for J3016. “The initial publication of J3016 occurred in 2014, and last year we published the fourth version of the document,” Nicols stated. “Updating every several years might not sound really fast, but in the standards world, I would say that that is fairly rapid updating that speaks to how it is a living document.”

Although the public quickly attached itself to the term “autonomous,” that has certainly not been the case in engineering circles. “We have the word ‘autonomous’ being a deprecated term,” Nicols detailed. “Right from day one in J3016, using that term is not appropriate from our engineering perspective. I think it's been more reflective of a common use among the public that aren't involved directly with automation itself... used more casually or colloquially.”

Nicols noted that within engineering circles, distinguishing among terms is something well understood. “Those of us working specifically around automation recognize that using the term autonomous can be misleading and has its own connotation,” he said. “If you describe them as autonomous, they could be understood as having their own decision-making process, almost like a self-governance. That really is not appropriate because these are automated systems currently being developed. Even those not introduced to market yet are programmed. They are systems that follow an algorithm.”

“This is not a science fiction type of device. It is not that something has been created to be used on a vehicle that basically has its own direction on what it wants to do,” Nicols emphasized. “You could argue that sometime in the distant future these vehicles would operate literally on their own accord, but that's certainly not the case currently. It's because of that connotation we think that this is not an appropriate term – the term autonomous – to use for the technologies being currently developed.”

Automated vs autonomous

According to Nicols, the distinction between autonomous and automated is easier to discern at the engineering systems level. “The phrase ‘automated vehicle’ is another deprecated term within 3016, in that the automation is part of a vehicle,” he said. “This is probably a subtlety that isn't as widely recognized, but what we try to emphasize in 3016 is that automation applies to a feature or an aspect of a vehicle, not the vehicle itself.”

Providing an example using windshield wipers, he noted, “In some sense this is an automated device, in that you turn a switch and the windshield wipers operate without any further need for a human to take some kind of action. It's just that single action being programmed,” he said. “That's a simplistic and exaggerated example, but you can take that and extend it to the systems that we are discussing in 3016 that are automated for the purposes of operating a vehicle in motion – operating the steering system, operating the acceleration and braking, or even more complex configurations as we get into the higher levels of automation.”

Where language meets development

Within engineering groups creating the technology, there appears little ambiguity as to where driver involvement begins and ends. “I don't think we'd throw around the term automated. Our discussion is whether the driver is engaged or not,” explained Lee Aho, regional business unit head for ADAS products for Hitachi Astemo. “We say: ‘Feet off; hands off; eyes off; and then finally, mind off.’”

“If we say automated, that means different things between you and I, and the average guy walking around on the street,” Aho noted. “If we say this stuff is automated, somebody on the street might say, ‘Hey, great. I can get in the car and not have to do anything.’ But somebody still has to push the button. Let's take a silly example: paper shredder. This is an automated piece of equipment. But what happens if I get my tie stuck? There's got to be some human oversight or intervention, somebody has to shut the thing down before I strangle myself.”

Referencing the SAE Levels of Automation , Aho added, “Level 5 is you get in the car and you tell it where you want to go and it takes you there, you don't need to be involved. But with Level 4, there is still the possibility that the vehicle says, ‘Hey, I'm looking at the conditions around me and it's not safe. So I'm transferring control back to you.’ You can't go in the back and take a snooze when you've got Level 4 features engaged. Until you get to Level 5, you're always going to be engaged at some point with these features.”

“I think we would say we're working on advanced driving assistance systems [ADAS],” Aho stated. “But the things that are employed in vehicles today and in the near future are still Level 1, 2 and Level 3 features. What we do is work with the OEMs to see what kinds of things they want, what kind of driver experience they are trying to achieve. We offer the different pieces, and we integrate them with the things that they already have in the vehicle in order to get toward this Level 5.”

Driver in the loop

Speaking with two principals at Tier 1 Magna revealed that the borders defining the levels of vehicle automation are just as absolute: “When we think of something autonomous, it's basically the vehicle operating under all conditions without human supervision,” stated Sharath Reddy, Magna’s senior VP of Electronics. “Whereas automated means the driver is in the loop. That's the fundamental difference from Magna’s perspective.”

“I think when the industry started using these terms of ‘self driving,’ ‘autonomous,’ ‘automated,’ I think the maturity of the industry was not there,” he said. “SAE with its 3016 standard have done a pretty good job of laying out what Level 1 is, what Level 2 is, etc. I think that has helped us define more clearly to avoid any confusion.”

“As we get into more feature content, especially in this Level-2, Level-3 area, that's where the industry is going very, very fast right now,” Reddy noted. “We're getting into more features which are more automated, but when you look at autonomous features, we're looking into limited cases. These are shuttles for example on dedicated lanes. It's a completely different use case from our perspective.”

“For autonomous, it's not having the driver in the loop, or the potential for the driver to takeover,” emphasized Nikhil Gupta, Magna’s senior engineering director for ADAS features. “Whereas for automated and assisted, basically the driver is in absolute control. The systems may be helping you a bit, be it lane centering, be it adaptive cruise control, but the driver is eventually in control. The driver is in the loop and can take over the system anytime the system so determines.”

“If you look at some of the systems in the market today, they're able to drive on highways pretty much with the driver hands off, feet off,” Gupta said. “And to some, it may appear the systems are driving themselves and hence the line between automated or assisted and autonomous gets blurred. But autonomous really means the driver does not have to be in the loop. Whereas for automated, no matter if the car is driving itself for a while, eventually the responsibility as well as the onus of control lies with the driver.”

Communicating the message

Much of how forthcoming automated technology will begin to be applied will depend on where it is expected to operate, known as an operational design domain (ODD) such as a “highways only” feature. But much complexity remains even within specific ODD’s. “Take a simple example of automated lane change,” Gupta offered. “There's a big difference, especially if you're driving in Germany on the autobahn. The speeds may require you to add additional sensors to support a feature like that, and may also require additional processing and compute capability to make sure they can fuse all the sensors and make the right decisions.”

“But drivers don't understand that,” Gupta emphasized. “They don't know what's under the hood, how many controllers they're running, what happens if the system will fail and will it stop in the middle of the lane or go to a safe spot on the road shoulder. Those things are nuances, but those nuances are what differentiates assisted or automated system from autonomous systems. And while today from a user experience, the systems are driving themselves, they're nowhere close to being autonomous.”

“We all have a responsibility in the industry right now,” Reddy noted. “Yourself in the press, we as Tier 1’s, the OEMs, how we communicate this to the consumer buying the car. Because if you use some of the easy terms like ‘self-driving car,’ people can misinterpret what that feature means and cause themselves trouble, or liabilities for themselves or for the OEM,” Reddy explained. “I think it's very important for us to stick to the SAE standards. I think the SAE standard has been very, very good in terms of defining the different levels.”

“I would speculate that part of the difficulty that the broader public might have in understanding the [SAE] levels is they might tend to think in terms of either a system does everything automatically by itself, or I as a human need to operate the vehicle,” Toyota’s Nicols noted. “Even with the current systems that are available on the market, there is a shared role between what the automated system does and what a driver still needs to do. I think that's probably an area where we can work on explaining that situation more clearly to everyone. You cannot buy a self-driving vehicle today. There is no such thing.”