Owl AI’s 3D Thermal Sensors See Further into the Dark

November 3, 2023

Sebastian Blanco

The race to develop low-cost and effective sensors for automated driving requires all-day effort. Getting sensors to accurately identify the world around them is easiest on a clear, sunny day, but that solves just part of the problem. One possible solution for driving in darker hours is thermal cameras, including those from the appropriately named Owl Autonomous Imaging. Owl is developing thermal-ranging 3D sensors that, the company claims, can detect objects in any light or visibility conditions.

Speaking at AutoSens Europe 2023, Owl co-founder and CEO Chuck Gershman said his company wanted to create 3D thermal cameras that can identify objects no matter the time of day or the weather because today’s sensing systems do not recognize pedestrians as well as they should.

“Generally speaking, if the sun’s out, no clouds, no fog or rain, [you’re] good to go,” Gershman said. “Turn out the lights, though, and that can be deadly.”

Road fatality statistics prove that automated-driving sensors need to accurately identify people at night. In 2021, 50% of all pedestrian deaths in the U.S. happened between 6 pm and midnight, with another 24% happening between midnight and 6 am.

Founded in 2019, Owl is based in Rochester, New York, the former home of Kodak, and draws engineering talent from people who used to work there. Owl’s thermal cameras are intended for SAE Level 2 through Level 4 automated driving technologies, as well as construction, agriculture and smart-infrastructure applications. The company is moving away from today’s analog-based thermal sensors to a nearly completely digital thermosensor called the 3D Thermal Ranger.

“This allows us to put more pixels in the same amount of silicon area and reduce the power by over an order of magnitude per pixel,” Gershman said. “We build the highest-resolution cameras today in thermal that you can use in automotive, 1280 by 800 [pixels]. We built all the software to leverage that information [because] taking RGB software out of the box and applying that to thermal will give you a response, but it won’t give you the best response.”

Gershman said Owl’s thermal sensors need to be combined with other types of sensors to make a complete stack. There are four vital components for an AV system, he said, and the first three — detection, classification and range — are required before you can involve the fourth, software and hardware components that can take action. There’s no single sensor solution that manages all four. Visual (RGB) cameras can detect and classify quite well, he said, and provide decent range information when used in a stereo pair, but require light to work. Radar and lidar, which don’t need a well-lit environment to work, are terrible at classification, even as they’re great for depth. A 3D thermal sensor can provide ranging ability at a lower cost than lidar, Gershman asserted, while boosting a visual camera’s capabilities.

“Thermal supplements the visual cameras and supplements them from the point of view of classification,” he said. “When RGB cameras fail to classify, thermal cameras continue to identify the object, and that’s the fundamental value proposition.”

More data, more time to react

Before Owl developed the 3D Thermal Ranger, combining a thermal camera with an RGB system might have required three computer boards. Owl’s digitized sensor and integrated logic board means the unit is smaller and there’s no cooling system necessary, even though the pixel count went from 300,000 to more than 1 million. The unit cost, Gershman said, is just “a couple hundred dollars.”

Owl’s system converts the thermal data into a Cartesian coordinate system, Gershman said, so all the data is in the same format as it would be coming from a radar or lidar system. Standard headlights today illuminate somewhere between 40-50 m (131-164 ft), while Owl’s thermal cameras can identify and classify an object at around 180 m (590 ft) and detect objects at up to 400 m (1,312 ft).

“That’s significantly more time to react,” he said. “That’s your fundamental value. You can take this off the highway and put it into the urban environment. And you can use thermal cameras to see bicycles behind you in the dark, you can use them to see pedestrians walking their dog across the street on the side of your car in the dark, if you were to surround the vehicle with a thermal solution.”

Owl offers a lower-cost, 1280x400-resolution sensor for potential side and rear applications, but exactly where thermal sensors might be placed on a future production vehicle – Gershman would not disclose any names of OEMs or Tier 1 suppliers Owl might be working with – is still up for debate. If the goal is to fuse the visual and thermal data streams, then the cameras should be as physically close to each other as possible, perhaps mounted on the same rigid structure.

Thermal sensors still will need to ride alongside standard cameras for a while, Gershman said, because road infrastructure relies on pure visual cues such as lane markings, traffic lights and signs. Think of the sensors as slices of Swiss cheese, he said, since any one type has holes but, together, they make a solid block.

“We have over 100 years of roads out there that were designed for people with eyes and all the infrastructure was designed around people,” he said. “We’re not throwing any of that away. Your visual cameras have a home for a long, long time.”

---