Detecting Pedestrians

Safety of vulnerable road users is driving new technologies as pedestrian deaths rise worldwide.

May 1, 2018

Terry Costlow

Small pedestrians such as children are more difficult for human drivers and advanced vehicle sensor systems to see and identify. (Image: BMW)

Automakers and legislators are focusing on pedestrians, aiming to reduce injuries that are rising as more people worldwide migrate to cities. Camera systems have surpassed passive technologies to become the key technology for protecting people on foot who venture into harm’s way with cars and trucks on the roadway.

The death rates are already soaring for pedestrians, cyclists and motorcyclists — an aggregate group called Vulnerable Road Users (VRU). The World Health Organization said that nearly half the 1.25 million people killed in traffic accidents in 2016 (the most recent data available) are VRUs. In the U.S., VRU deaths reached the highest levels in more than a decade. NHTSA said that in 2016, pedestrian deaths increased by 9% to 5,987. By comparison, motorcyclist fatalities increased by 5.1% to 5,286 and bicyclist deaths rose 1.3%, reaching 840.

Cameras with greater acuity and processing power are vital for fast and accurate pedestrian identification. (Image: ZF)

The auto industry is racing to help reduce this trend, leveraging the rapid advances in advanced safety systems. Legislators are also ramping up their efforts.

“We’re focused intensely on VRUs—pedestrians, cyclists and motorcyclists,” said Kay Stepper, Vice President of Driver Assistance and Automated Driving for Chassis Systems Control at Bosch North America. “European regulations call out VRUs, and in the U.S. they’re looking at back-over avoidance legislation.”

Government interests are going beyond forcing automakers to react. Many cities are beginning to monitor pedestrian movement as part of their “smart cities” programs. Understanding patterns that cause accidents can help urban planners make changes to enhance safety.

“Downtown Las Vegas is an innovation district; we have lidar systems at intersections to detect whether anyone is in the intersection,” said Joanna Wadsworth, Program Manager for the City of Las Vegas. “We’ll work with Cisco on dashboarding to make that data available. That data’s also helpful to us as planners; it’s important to know how many people are crossing where.”

The need to look very closely

Automotive engineers and technologists are moving away from passive safety systems that lift the vehicle’s hood on impact and deploy exterior airbags that cushion pedestrians. Those systems posed many design challenges but offered minimal benefits.

“Market interest has shifted to avoidance and mitigation instead,” said Aaron Jefferson, Director, Product Planning, ZF Global Electronics. “It was difficult to package [exterior] airbags and there was the question of what happened to the pedestrian after the collision.”

ZF is developing a system called X2Safe that alerts both drivers and pedestrians, through their mobile phones, when accidents seem likely. (Image: ZF)

The change comes as cameras and radar are becoming commonplace, helping stop vehicles before accidents occur. Tweaking systems to spot VRUs in addition to cars leverages existing technology, reducing costs and space requirements.

Camera systems are evolving rapidly as developers increasingly use them to spot people who may be in danger of being hit. It’s harder to see and identify comparatively small VRUs than cars, so higher resolution is important. Humans move more freely than cars; newer systems are also looking to the side to see people who may drift in front of the vehicle. Those two approaches can be at odds with each other.

“It’s always an engineering compromise, looking at range or field of view,” Stepper said. “When it comes to VRUs, there’s great advantage to increasing resolution. There’s talk of a minimum camera resolution of one megapixel, and there’s already a lot of effort to go to two megapixels, four megapixels and beyond.”

Developers are also finding ways to spot people when cars are turning. Right-hand turns are often dangerous, particularly in busy cities. People may walk in front of the car while the driver is watching traffic coming from the left.

“Today, typical forward-looking cameras have a field of view of around 50 degrees, focusing on vehicles and pedestrians who step out in front of the car,” noted Andy Whydell, Vice President, Systems Planning and Strategy at ZF. “Next-generation systems increase that to around 100 degrees, which lets them see pedestrians when cars are turning around corners. As vehicles pull around for a right-hand turn, cameras detect pedestrians and slow the vehicle down.”

VRUs are far more unpredictable than vehicles, in terms of their next move, because they have more degrees of freedom. Crowds also pose significant challenges — one person can move in a different direction, breaking away from the others to enter harm’s way.

“In urban environments, it’s not just one pedestrian or cyclists; they’re often in groups,” Stepper said. “The trick is to be able to resolve groups of humans down to an individual human being. Another problem is that it’s not as obvious to predict the movement of pedestrians, they can move in 360 degrees. The trick here is to use predictions based on recent data sets. We are using artificial intelligence to support us on this research.”

Looking forward

Camera suppliers are developing products that can spot pedestrians in low light, such as this one from Denso. (Image: Denso)

As safety and autonomous systems evolve, more technologies will be used to protect VRUs. Artificial intelligence (AI) can help systems discern a human from a pole or perform other complex recognition tasks. Making these decisions in all light conditions can be a problem for current-generation camera systems.

“The technology of image recognition is being improved with AI,” said Takayuki Nagai, Director of Advanced Driver Assistant Systems for Denso International America. “Additionally, each sensor’s performance is improving every day. For example, we now produce a vision sensor that can recognize pedestrians at night.”

Recognizing VRUs quickly and accurately is no simple task. It is difficult to avoid false positives, for example. Avoiding unexpected stops will be important in crowded urban areas since quick stops may result in rear-end collisions. That’s prompting researchers to look at ways to use multiple sensors to ensure that potential safety threats are indeed real. Radar may augment cameras.

“We have proven that it’s possible implementations can use solely radar to detect pedestrians and still meet NCAP requirements,” Stepper said.

Developers are also exploring ways to alert pedestrians when they’re entering a dangerous area. Cell phones may become part of an alert scheme designed to protect people.

“We have an R&D program for a cloud-based warning system,” ZF’s Jefferson said. “It allows bidirectional messages to be sent to pedestrians who are about to step into the path of a vehicle. The system will send a message to both the pedestrian and the vehicle.”