Volvo CE Pursues AI for Operator-Assist Functions on Path to Full Automation

Telematics platforms connected to the cloud, such as Volvo CareTrack, could be used as the medium for more AI-based solutions. (all images: Volvo CE)

While true machine learning employed in fully automated vehicles and equipment—where algorithms trained by data become brain-like, gaining skills such as pattern recognition and problem-solving—is still a work in progress, technologies employing the early stages of artificial intelligence (AI) are already hard at work.

In the construction-equipment sector, early adopters of AI are deploying the technology to increase the efficiency, safety, and quality of their machines on worksites. Volvo CE falls squarely in this category, according to Dr. Fares Beainy, Strategy & Business Development Manager for Advanced Engineering & Core Technologies.

Dr. Fares Beainy

At the last Bauma exhibition in Munich in 2016, Volvo CE released the in-house developed Volvo Co-Pilot system, which uses a tablet computer to deliver a range of intelligent machine services, such as Load Assist, Dig Assist, Compact Assist and Pave Assist applications. Co-Pilot is designed for use on various machines like excavators and pavers.

Compact Assist, currently in use by many of its customers, employs AI at its core, in specific artificial neural networks, which is one type of machine learning method, according to Beainy.

Two module options are available. Intelligent Compaction provides real-time insight using pass mapping, temperature mapping and Compaction Measurement Value mapping, helping to prevent over-compaction. Intelligent Compaction with Density Direct (for compactors only) allows operators to account for the effects of the entire paving train by calibrating to cores or an asphalt density gauge. The patented process can provide an estimate of the density value, with an average tolerance of +/-1.5%.

A number of engineers at the Volvo Group, and specifically at Volvo CE, are actively working with AI, according to Beainy. He could not share exactly how many engineers for strategic purposes but did allow that the number has increased steadily over the past several years.

An early example of an operator-assist technology that “employs AI at its core” is Volvo CE’s Compact Assist, launched in 2015.

“There are always collaborations between the different business areas inside the Volvo Group, especially on the R&D side,” he said, commenting on whether AI strategies and know-how from the passenger vehicle and heavy-duty truck areas inform the off-highway business, and vice versa. “As you would expect, the challenges are different for the products of each business unit—the pains and gains for our customers are different—but we still try to share fundamental knowledge as much as possible.”

Adding AI-capability

Three major components are needed to create an AI system, according to Beainy: data/information acquisition module, a computing module, and a user or machine interface module. The data acquisition module in most cases consists of one or many types of sensors such as a camera, radar or accelerometer. Lidar—a sensing method utilizing laser light pulses to measure range—is being explored in off-highway as well.

The interface to the operator—the proper frequency and amount of feedback given—is as important as the precision of a safety system, according to Beainy. Dig Assist function is shown.

The computing module entails some sort of computer or a tablet, as in the case of Volvo Co-Pilot, or GPUs, and the interface is either a signal that would control the machine, or a human interface such as a display, speaker or haptic feedback.

“Another way to make our machines AI-capable is through connectivity, or what is mostly known as connected machines,” Beainy said. “Most of our machines are connected to the cloud through the Volvo CareTrack. Such a platform could be used as the medium for more AI-based solutions.”

Volvo CE has been developing AI algorithms that detect and decipher specific objects using several computer vision methods. If industrialized, such an advanced system would send a warning message to the operator to reduce the risk of accidents.

“This is still under development, so I can’t share too many details for now,” Beainy said, adding, “What I can say is that we are working closely with some of our innovative customers to best understand the needs and the best way to deliver the feedback to the operator. The interface to the operator is as important as the precision of a safety system. The proper frequency and amount of feedback given to the operator is crucial for any safety system.”

Current state-of-the-art AI algorithms, especially for computer vision applications, require quite a bit of processing power. “This is a challenge for our machines given the stringent mechanical and environmental requirements our ECUs [electronic control units] have,” Beainy said. “This makes such computers power-supply hungry, as well as more expensive.

“Another big challenge is public data availability that are specific for construction or off-road environments. Such data is needed in the initial training of AI algorithms.”

Combating operator shortage

By performing mundane, repetitive and sometimes dangerous labor-intensive tasks, operator-assist functions and AI technologies help alleviate some of the burden from machine operators, particularly those who are inexperienced. For instance, an assist function can observe the area in front of a wheel loader bucket and guide it to dig and load with minimal input from the operator. Such a function could be optimized to get a truck loaded faster, safer, and using less fuel with fewer emissions.

“In many areas of the world, there is a shortage of skilled operators,” Beainy said. “AI could be the solution to enable many countries, cities, companies to overcome the limiting factors of such shortages. Such a benefit is sometimes overlooked.”

Beainy admitted it’s hard to predict a roadmap for how AI will advance, in Volvo’s construction equipment and in the industry in general. “It is a combination of market demand, introduction of legislations that require specific features that need AI technologies to work properly, and technological advancement that could alleviate some of the challenges that we face today,” he said.

Setting the uncertain timetable aside, Beainy is certain AI will continue to advance and play a critical role in construction equipment and jobsites: “I envision AI to continue to be used to introduce more operator-assist functions, then cloud analytics, and in the future, it will be the core of our autonomous machines.”