Autonomous Jobsites Edge Closer to Reality

Heavy-equipment manufacturers and tech suppliers are solving the challenges to creating construction sites with fewer humans on the ground.

SafeAI and Obayashi demo a retrofitted autonomous Caterpillar 725 articulated haul truck on a construction site in Japan. (SafeAI - Niall David Photography)

Will the construction industry ever get to a point where the sight of humans on a jobsite is a rarity? Judging by the increasing number of projects demonstrating autonomous construction equipment at jobsites in the U.S., Europe and Asia, the answer likely is “yes” – eventually.

The Exosystem from Built Robotics includes an all-weather enclosure, proximity radar, 360° cameras, GPS and a powerful liquid-cooled computer that enable machines to operate autonomously. (Built Robotics)

Consider the partnership between California-based SafeAI and Japanese construction company Obayashi that demoed this past summer a retrofitted autonomous Caterpillar 725 articulated haul truck operating on a construction site in Japan for the first time. Prior to the public demo, which the companies consider a critical first step toward securing government approval for autonomous construction in Japan, SafeAI and Obayashi spent six weeks extensively testing in the country to confirm all the features previously developed in a U.S.-based proof of concept.

SafeAI’s retrofit autonomous system is powered by “industry-specific” AI technology. It utilizes off-the-shelf hardware (lidar, radar, camera, drive-by-wire system, control hardware), proprietary autonomous-vehicle and site-operations management software, and perception-based localization with GNSS to detect surroundings and enable real-time decision-making. Working from a cloud-based project model, the process doesn’t rely completely on GPS and network availability and offers mixed-fleet capabilities, said Bibhrajit Halder, founder and CEO of SafeAI.

ASI’s Mobius platform enables multi-vehicle command, control and monitoring to help jobsites leverage the benefits of autonomous vehicle navigation and traffic management. (Autonomous Solutions, Inc.)

“From here, we are focused on securing autonomous vehicle buy-in from the government/industry organizations and executing a ramp-up plan for the construction industry that starts in Japan and expands globally from there,” Halder said. The company will continue to work with the government and industry consortium groups to review findings and evaluate necessary regulatory changes, he added.

The promise of significant productivity gains, along with safer work environments, is driving the development of self-operating assets. SafeAI claims that companies can add thousands of hours of productivity annually, completing major projects 20% faster and at 25% lower cost, by operating autonomous construction equipment.

“We firmly believe the future of the construction industry is dependent on off-road autonomy,” said Shinya Sugiura of Obayashi’s Business Innovation division. “We’re committed to bringing this technology to Japan, where it will have a massive impact on safety, sustainability and productivity.”

Working in concert

Teleo’s Supervised Autonomy retrofit system is now in beta-testing on North American jobsites. (Teleo)
Trimble’s Earthworks Horizontal Steering Control automatically steers a soil compactor using a 3D model or compaction pass line, improving compaction productivity and quality by precisely controlling overlap between passes. (Trimble)

All industries, including construction, have been the beneficiaries of U.S. defense-related research, according to Halder. This included the DARPA (Defense Advanced Research Projects Agency) Grand Challenge in the early 2000s, designed to accelerate autonomous vehicle technologies. “That was a trigger point,” Halder said. “It was a massive success that really sparked autonomy in this country.”

The military is interested in autonomous systems “primarily as force multipliers and for occupant safety,” Brendan Chan, senior chief engineer of autonomy and active safety at Oshkosh Corp., said at a past SAE COMVEC conference. “There is quite a lot of synergy in the autonomy space, although the application may be different. It all boils down to the use case and the operational design domain.”

While defense may have sparked interest in autonomy, the commercial sector now is “much more mature” in its development of autonomous and unmanned vehicles, according to Dana J.H. Pittard, Allison Transmission’s VP of Defense Programs. Experimentation with leader-follower trucks has led to soldiers being able to issue digital commands for autonomous vehicles to form into convoys, drive in formation and retrieve loads.

“That’s a start,” Pittard said at 2022’s SAE COMVEC. “Where we want to eventually get to, though, is when you see tanks or infantry fighting vehicles moving on the battlefield, you should have no idea whether or not they’re manned or unmanned,” he said.

Though further along, commercial industries still have much work to do before automated-driving features can operate vehicles under all conditions. “No one has a true [SAE] Level 5 system yet,” said William Nassauer, manager of product strategy for Komatsu America’s autonomous systems, mining technology solutions. As with the progression in passenger vehicles, construction equipment will transition from assist features to task automation to task autonomy.

Equipment automation should be considered in the context of total jobsite autonomy, with several autonomous machines working in concert, said Fred Rio, product manager for Construction Digital and Technology at Caterpillar. “On a jobsite, all machines have a shared mission,” Rio said, “and no one machine can accomplish it without the other machines. The true quantum step in value will be when you can get them to all work together.”

Retrofitting existing machines

ULC Technologies’ Robotic Roadworks and Excavation System uses a robotic arm on a tracked undercarriage to automate tasks in complex underground utility repairs. (ULC Technologies)

Several companies in addition to SafeAI – including Autonomous Solutions, Inc. (ASI), Built Robotics, and Teleo – are building retrofit kits that aim to take the operator out of the cab. ASI defines three different types of controls: remote control, where the operator is line-of-sight with the machine; teleoperation, or non-line-of-sight operation that’s still one operator on one machine; and autonomy, in which an operator can remotely oversee the operation of an entire fleet of machines.

“We look at it as finding the best solution for the situation, but our experience and focus is really on autonomy,” said Matt Nielsen, business development manager at ASI. The company has expanded its Mobius autonomy platform to include a Mobius for Drills application in collaboration with Epiroc Drilling Solutions, which complements similar programs for autonomous haulage and blasting. The Mobius system enables multi-vehicle command, control and monitoring to help jobsites leverage the benefits of autonomous vehicle navigation and traffic management, Nielsen said.

Teleo’s Supervised Autonomy retrofit is specifically designed to include operators, according to co-founder and CEO Vinay Shet. “We’re combining the best of both worlds – the experience and expertise that their operators have with the advancements in technology,” he said. “This is letting their operators do a lot more than previously.”

Teleo claims that with one operator remotely controlling two or more machines, fleets can see 30% or more in operational savings and a payback in less than a year. The company, which has partnered with Deere dealer RDO Equipment, among others, now is beta-testing its system on North American jobsites.

The human element

“The change management is significant in adopting autonomous machines,” Caterpillar’s Rio said. Because of their autonomous experience in mining, Caterpillar, Komatsu and ASI have developed a structured approach to onboarding the technology to their customers.

“Our customers are going to be changing mentalities,” Komatsu’s Nassauer said. “They’ve got to maintain their site in a different way, use workers in different ways and transition operators into supervisory roles. There’s a lot of learning involved.”

Understanding a jobsite – including what each machine is doing each day – and how the inputs and outputs work is an important step in becoming autonomous, said Michael Gidaspow, Komatsu America’s VP of products. “They’ll have to give the machines specific instructions on exactly where and when to go,” he said.

To be attractive, autonomy also must ultimately be easier to use, said Finlay Wood, Trimble’s general manager of off-road autonomy. “We don’t want them to go and hire a whole group of IT specialists; there’s no point in it being more complex.” Earlier in 2022, Trimble began offering Horizontal Steering Control functionality on the Trimble Earthworks Grade Control Platform for soil compactors. It’s reportedly the industry’s first automatic steering control solution for soil compactors, and the next step towards Trimble’s autonomous vision, Wood said.

As part of the move towards autonomous, Built Robotics envisions a new job: Robotic Equipment Operator (REO). “Fifty percent of this effort is developing the robot and 50% is how you deploy and get people to manage it effectively,” said Erol Ahmed, director of communications at Built Robotics. “REOs are the people on the front lines. They go through a 30-hour training to run and manage these machines.” The company has partnered with the International Union of Operating Engineers to offer this certification to its members.

In addition to autonomous-machine research, some companies are investigating job-specific robotic units. For example, ULC Technologies’ Robotic Roadworks and Excavation System (RRES) uses a robotic arm on a tracked undercarriage to perform tasks aimed at executing precision bores in complex underground utility repairs.

“It automates this operation, from above-ground scanning and identifying where the underground assets are, to reinstating the road when the job is done,” said Ali Asmari, director of infrastructure automation and AI at ULC Technologies. After scanning, the onboard software creates a 3D model of what’s underground to guide the rest of the operation. The sensor box then is swapped for a variety of road-cutting, air, vacuum, repair and backfill tools.

Although the RRES was created for one utility customer, its applications are broad, Asmari said. ULC is actively pursuing new opportunities with other companies, including how each of the tools can be used separately.

Reaching a tipping point

Will construction ever see a “no-entry” site where no humans are on the job or even necessary? Perhaps, said SafeAI’s Halder, but it’s still years away.

There will be a tipping point, however – for example, if using autonomous machines gives a 20% improvement in productivity. “The moment one contractor completes a $100-million project for $80 million because of autonomy, it’s game over,” Halder said. “Everybody has to do it because you can’t compete anymore.”

“The industry is absolutely massive, the pain points are huge, and it’s early days for autonomy,” Teleo’s Shet said. “To be honest, there’s not enough companies doing what we’re doing.”

“There’s a huge appetite and interest in autonomy,” Built Robotics’ Ahmed agreed. “Maybe construction needs to develop its own set of autonomy goalposts, ones that are specific to its needs and show that each level is valuable.”

The Association of Equipment Manufacturers  (AEM) submitted content serving as the basis for this article. AEM produces the ConExpo-Con/Agg  trade show, which takes place March 14-18, 2023, in Las Vegas. The trade group consists of more than 1,000 off-road equipment manufacturers and suppliers representing more than 200 product lines.