How Thunderbolt 4 Helps Bring Fault-Tolerant, Distributed Systems to Market

September 1, 2023

In an embedded world gone SOSA sensational, one might believe that centralized ATR-style OpenVPX systems are the best way to architect your next rugged system. While these chassis are routinely and successfully deployed on airborne, shipboard, and vetronics platforms, they are big, heavy, costly, and a real challenge to cool and connect. An alternate but equivalent rugged, deployable approach uses one or more small form factor chassis modules, distributed into any available space in the vehicle, interconnected via Apple® and Intel’s® 40Gbps Thunderbolt™ 4, a commercial open standard that uses USB Type-C connectors with a single, thin bi-directional copper or fiber cable.

With 4, 8, even 16 3U or 6U LRU (line replacement unit) boards inside an ATR chassis, 600 watts is on the low end of systems that can push well over 2,000 watts in a 200 square inch footprint or less. Assuming one can find the space for such a chassis in the vehicle or platform, there’s also the issue of cooling it. Vetronics chassis must be bolted to a cold plate—no easy challenge in space-constrained armored vehicle interiors.

Other ATR chassis are air-cooled with fans, blowing heater-hot air from exhaust vents—sometimes too close to operators. All high performance electronic systems will get hot, and sub-dividing heat into smaller units is one way to deal with it. Breaking the system down into smaller, cooler small form factor (SFF) loads not only improves crew comfort with less heat per SFF but makes it easier to provide cooling to each lower-wattage load.

Broken Apart, Connected by Wire

Thunderbolt 4 uses the Type-C connector but makes available all of these interfaces on one copper or fiber cable. (Image: Intel Corp.)

If an eight-slot ATR chassis of OpenVPX modules consists of a single-board computer (SBC), a multiport Ethernet switch for in-chassis and out-of-chassis networking, a graphics processing unit (GPU) or co-processor, mass storage, low-speed I/O such as serial and 1553, plus multiple high-speed sensor I/O cards—this is notionally four to eight or more functional blocks. If this centralized system is to be distributed into multiple SFF modules for mounting simplicity and heat load optimization, they need a high-speed connection between them. This isn’t just a case of a remotely located storage drive that is randomly accessed; a distributed computing architecture physically divides processing nodes and must therefore have a very fast—but long-distance—interconnect scheme.

Thunderbolt 4 is the backbone of GMS’ X9 Spider system architecture, enabling distributed, ruggedized computing modules to support the demanding needs of next-generation warfare.

Thunderbolt 4 is the ideal “cable plant” on top of which to build distributed SFF computer architecture. An open-standard interface that’s a superset of the open standard USB4, myriad Thunderbolt 4 devices are available and interoperable—from cameras to disk drives to PCIe co-processing chassis and embedded computers. Thunderbolt 4 is a thin, bi-directional cable that operates at 40 Gbps and uses the USB-IF’s Type-C interface—the same one available on smart phones, laptops, and tablets. Most civilian devices can plug into Thunderbolt 4 and at least get power and USB 2.0/3.x data transfer. But there’s so much more available.

What is Thunderbolt 4?

Thunderbolt 4 is more than USB operating at 40 gigabits per second (Gbps). While USB 3.2 Gen 2 is 10 Gbps and USB4 is 20 Gbps, Thunderbolt 4 also provides DisplayPort alt mode, PCI Express 3.0, networking up to 10 Gbps, and Power Delivery up to 100 watts. On a single cable not unlike what’s available to charge a cell phone, two 4K displays can be driven simultaneously (or a single 8K display) with PCIe traffic, networking, and USB data at over 10 Gbps. The 100 watts of power is bi-directional, interconnecting distributed modules such that whichever has more power available can send that power to up/downstream SFF modules. Thunderbolt 4 devices can also be daisy-chained up to six hops. In this manner, fault-tolerant distributed systems can be envisioned.

In a rugged, small form factor architecture, Thunderbolt 4’s Gen 3 PCI Express extension with power is a compelling feature, allowing a compact 60 watt CPU-based mission computer to be installed in one location, and interconnected with a distant GPU or artificial intelligence (AI) co-processor located close to an EO/IR sensor or SDR front end at the far end of the ship, airplane fuselage or ground vehicle. A single thin copper or fiber cable connects them.

Distributed Computing Architecture Benefits

Unlike the single, dense, centralized ATR chassis with its high heat load and single-site cooling requirements, Thunderbolt 4-connected small form factor modules can be tucked into any available location on the platform. Thunderbolt 4 can provide 40 Gbps connectivity, PCIe bus extension, and even power up to 100 watts. Distributing the modules makes system configuration simpler while easing the cooling requirement as each module now is a fraction of the power of a centralized ATR. And there’s no penalty for distance between modules due to Thunderbolt’s bi-directional speed. Finally, lighter-weight Thunderbolt 4 cables can be easier to route and save weight in a platform compared to traditional MIL-SPEC cables, while still providing ample speed, EMI immunity, and harsh environment ruggedness.

This article was written by Chris Ciufo, Chief Technology Officer, General Micro Systems. For more information, visit here .