Cameras Look to Go the Distance

Automakers seek vision systems with greater distances, improved reliability, and more functionality, thanks to ruggedized complementary metal-oxide semiconductor technologies.

June 1, 2015

Terry Costlow

Increasing distance is a focal point for GM’s camera design teams.

Forward-looking cameras are quickly becoming standard features on cars, improving safety and convenience. Engineering teams are burning the midnight oil to improve distance capabilities and safety while adding more functions and improving performance by leveraging advances in cameras, processors, and software.

The CMOS (complementary metal-oxide semiconductor) imagers used in consumer products are helping make it more cost effective for automakers to add ruggedized cameras. Relatively few cars had cameras at the start of the decade, but most will have them by its end.

TRW is using higher resolution cameras to increase distance detection.

“The camera market will go from around 30 million units in 2014 to nearly 100 million by 2019,” said Ian Riches, Global Automotive Practice Director at Strategy Analytics. “By 2021 there will be one or more cameras on every car.”

Faraway look

When cameras are compared to other forward-looking sensors, distance is a weakness. Cameras have displaced radar in some applications, but radar still offers far better distance capabilities.

“Camera technologies are not as good as radar,” said John Capp, Director, Global Vehicle Safety at General Motors. “Cameras can see about 50-60 yards, radar’s range is about two to three times that. That’s why higher-end systems pushing the envelope for autonomous driving combine radar and cameras.”

Redundancy and reliability are important elements in Continental’s design plans.

Automotive-grade CMOS imagers are being improved to close that gap. The drive to higher resolution in consumer products is helping automakers collect information before objects get too close.

“Higher resolution imagers are going from 752 x 480 pixel formats to HD imaging at 1280 x 960,” said Andy Whydell, Product Planning Director for Global Electronics at TRW Automotive. “That improves range by about 50%.”

Once cameras are added, developers want to make them do more. The combination of higher resolutions and faster processors lets these cameras perform more functions.

“Traffic-light recognition is now available in the U.S., and there’s a move to get to 1000 road signs that can be detected,” said Martin Duncan, Business Unit Director at STMicroelectronics. “Cameras are also looking at the road profile so the active suspension can adjust for potholes. Cameras can also see if something’s on the road, detecting an object that measures 10 cm at a distance of 40 meters.”

Changes will be made

Packaging inside camera modules helps TRW remove heat, which is a major issue for automakers.

As imagers and lenses improve, other system components such as micro-controllers are also advancing. There’s also a lot of work on algorithms and software, where refinements help systems understand what they’re seeing.

“The use of cameras is currently limited by the ability of the algorithm to detect objects in the image,” said Ron Schubert, Director of Denso International America’s Body Components Group. “This is impacted by both the detail available in the image (resolution) and the ability of the algorithm to decide if an object falls into a specific category (identification).”

Going forward, it’s likely that the processing architectures for cameras will move from dedicated processors in the camera module to a centralized sensor fusion box. These centralized boxes will combine data from radars and other sensors.

This ECU will include processors that decide whether or not to activate brakes or steering, among other tasks. As these boxes get more complex, some are likely to use real-time operating systems.

Vehicles are using multiple cameras for safety, so Continental is putting more emphasis on safety integrity levels.

“Cameras have been kind of dumb systems that could be handled with a home-grown operating system, but when you get into ADAS, you need hard real-time responses to process graphics and understand what’s going on,” said Andrew Poliak, Global Director, Business Development for QNX Software Systems. “Some of those smart ADAS cameras will start appearing around model year 2018. When you get to a central sensor fusion box, you definitely need real-time capabilities.”

As cameras perform more tasks, TRW is moving to advanced networks and control architectures.

While various forms of software will play a major role in engineering efforts, old-time mechanical issues must still be dealt with. Removing heat is an important factor for rooftop-area cameras. The packages that house electronics must be very compact while providing high levels of protection and heat removal. Chipmakers play a major role in cooling.

“We use expensive packaging, with copper fillers and metal lids,” STMicroelectronics’ Duncan said. “We need to get as much heat out as possible.”

When semiconductors remove heat, they transfer it to the system package. The Tier 1s that integrate these components are also improving their cooling techniques.

“We’re always working to advance packaging,” TRW’s Whydell said. “For camera modules, we use more expensive materials like magnesium, which is lightweight and has good thermal characteristics.”

Safe and reliable

When ECUs decide to steer or hit the brakes, safety and reliability requirements climb beyond automotive’s already-stringent demands. Single points of failure aren’t acceptable. Engineers are revising architectures to provide high reliability while living within cost guidelines.

“As ADAS functions move from informing and warning the driver to autonomous functions such as automated braking and steering, the need for higher safety levels and redundancy increases,” said Dean McConnell, ADAS Business Unit Director at Continental Automotive. “This is driving certain function integration into the sensors, but also system integration of multiple sensor inputs, and fusion, which sometimes happens in the sensors, sometimes in a separate ECU.”

Every element in the system must meet performance demands. Engineers striving to meet safety requirements are also being pushed to alter networks to provide more bandwidth. Security is a factor that’s seeing increased interest.

“Cameras are becoming more highly-developed safety products that are generally rated to higher automotive safety integrity levels,” Whydell said. “The architecture of electronics, software design and testing are changing. Many companies are also moving to higher speed buses, going from CAN to Ethernet. Cyber security is also on the drawing boards.”

The magic of semiconductor pricing is helping engineers meet the combined challenges of enhancing performance and improving reliability. CMOS imagers are becoming far more affordable, making it practical to use a few cameras to gain different benefits.

“With camera prices coming down, some companies are using three cameras mounted in the one box,” Duncan said. “They can be a wide angle, a conventional camera, and a medium camera, which help you see different objects. That also gives you some redundancy.”