Far and Away: Remote Drivers Monitor Autonomous Vehicles

Remote operators are helping autonomous shuttles and other AVs navigate through complex situations.

Texas A&M is moving drivers from the vehicle to a remote operations center. (Texas A&M)

Eliminating the safety “watchdogs” who typically ride in autonomous vehicles (AVs) is a big step for technologists and legislators alike. A shuttle at Texas A&M university is among the first on public streets to replace these drivers with a remote operator who monitors vehicle behavior from an operations center. Teleoperations increasingly is considered as a way to augment automated-vehicle systems when they encounter situations that are difficult to analyze.

Phantom’s operations centers give remote drivers a broad view of the vehicle’s operating area. (Phantom Driver)

Startups like Phantom Auto, Starsky Robotics, Veniam and Designated Driver are establishing operations centers where remote drivers constantly watch for challenges that aren’t easily handled by AV algorithms. Larger players such as Valeo, Uber and General Motors also are developing teleoperation strategies. Texas A&M’s shuttle started a test to see how Designated Driver’s systems work on public streets. The route of an autonomous shuttle in Bryan, Texas, was doubled early this fall to cover a larger portion of the busy downtown shopping area.

Remote drivers monitor vehicles, authorizing restarts when shuttles stop for signals or pedestrians in their path. “Our goal is to eliminate the human safety driver in the shuttle,” said Srikanth Saripalli, an associate professor who heads the Texas A&M University shuttle program. “However, to get there, we believe that remote human supervision is necessary, specifically for operating in complex downtown environments.”

Market analysts predict that teleoperations will become an important part of autonomy. Michael Ramsey, automotive analyst at Gartner, noted that autonomous trucks in mining sites and other early adopters are overseen by remote drivers. That may well be a common model. “As soon as autonomous vendors take people out, I promise you there will be remote operators watching their operations,” Ramsey said. “Anyone who has a large number of autonomous vehicles will need a control center. It will be a requirement from a business-operations standpoint.”

However, teleoperation isn’t always viewed as a necessity. TuSimple, which recently added UPS as a minority partner, is developing large trucks that will drive themselves from loading dock to loading dock. Remote overseers aren’t in the company’s road map. “It’s not something we see as viable now,” said Koabi Brooks, general manager at TuSimple. “There are major latency issues; 250-millisecond latency is needed for controls. What happens if the connections degrade?”

Latency upgrades expected

Designated Driver believes that view will be somewhat limited, since most early applications won’t have the same latency requirements as a fully-loaded semi-truck at 70 mph (113 kph); shuttles and taxis in cities often don’t surpass 40 mph (64 kph). Existing cellular links meet almost all of today’s demands and pending 5G cellular is promised to eventually slash latency times and create more-realistic viewpoints for remote operators.

“5G is not a necessity, we have done everything over 4G LTE, including controlling a car on the coast of England from our headquarters in Portland,” said Walter Sullivan, CTO at Designated Driver. “5G does add performance that will be beneficial. We built a car with Samsung to help demonstrate their 5G technology and we added virtual reality for the remote operator. That really gave the remote operator a different perspective.”

Those operation centers are a central element for success. Staffing centers with drivers who can quickly analyze situations and take action will be a significant barrier to entry, according to Sullivan. That may prevent OEMs and other large automotive suppliers from entering the market. Various models will emerge. “We have two types of teleoperation,” Sullivan said. “One is a remote driver, where the human operator is driving the vehicle. The one that will be more frequently engaged is remote assistance. Autonomous systems are fully functional, fully controlling the vehicle. A human provides information to improve operations.”

Operations centers may also address robotic package deliveries, using operators to help autonomous robots navigate tricky last-mile deliveries. That strategy is gaining importance at Phantom Auto, which sees growth for delivery robots coming before autonomous vehicle take off.

Valeo is exploring still another tack. It thinks teleoperations can replace driving instructors. Teleoperators can step in to handle tricky operations. “Remote parties can assist drivers and relieve them of certain driving tasks or even switch to manual-driving mode, for example serving as a great learning platform for student drivers,” said Guillaume Devauchelle, vice president of innovation at Valeo. “This serves as the best of both worlds, letting students learn to drive without being in harm’s way and ensuring they are fully immersed in an authentic driving experience.”

Designated Driver is focusing on shuttles and other on-road vehicles. Sullivan also noted that the technology is easy to add to existing vehicles. Adding modems and access to the CAN bus are fairly minor technical challenges. “We use four cellular links that use different providers,” Sullivan said. “Vehicles also need a drive-by-wire system that lets us remotely access primary vehicle systems: throttle, brakes and steering. We may need to honk the horn, operate the turn signals and lock or unlock doors, so it’s helpful to have more access to the network.”