Beyond VMEbus A New Concept
Deployability and life-cycle management of large and disparate collections of weapons, combat and C4ISR computing systems is one of the most critical challenges facing the Department of Defense today. Numerous defense programs, each defining separate, and oftentimes unique system architectures, configurations, and compositions, result in a wide range of processing and control systems that create complexity and drive excessive lifecycle costs.
Although serious efforts have been made to standardize on a set of common hardware solutions such as NAVSEA Acoustic Rapid COTS Insertion (A-RCI), Common Processing (CPS), and Common Display Systems (CDS), that limited commonality has not extended across any one ship, let alone an entire fleet. This lack of a truly common modular infrastructure has led to:
Difficult, asynchronous technology insertion cycles & delayed modernization due to expensive, time-consuming integration and shipboard industrial work;
DMS/MS-driven logistics that lead to the purchase of inefficient, obsolete, end-of-life products;
Non-uniform system administration & management creating unnecessary complexity;
Failure to achieve broad-based economies of scale due to small quantity purchases and a variety of “spares.”
These challenges not only make it difficult to upgrade to the latest technology but may also limit the government's ability to repurpose used - but still viable - equipment that may have residual value to other programs with less funding. If the DoD is to get onto, and stay on, the commercial technology cam, current efforts to standardize common hardware platforms must be accelerated. What is needed is a standardized, Modular Open Systems Approach (MOSA) compliant architecture that is at once scalable, extensible, serviceable and available. Once deployed, it should be populated with a relatively small set of mechanically robust, Commercial Off-The-Shelf (COTS), loosely-coupled, hardware modules compatible with mission critical land, sea, and airborne applications—truly COTS products with the end in mind.
The Impact of VMEbus
To understand the importance of standards-based technology in the DoD market, one of the best known examples is VMEbus. In the 1980's and 90's VMEbus overcame a slew of competitors to become arguably the leading embedded bus-board architecture for a wide range of commercial, industrial, military and aerospace applications. Over the years, it became the “party candle” of bus structures—whenever a competitor arose adopting some form of the Eurocard packaging, whether it be Multibus II, CompactPCI, Futurebus, ATCA, etc., the VME community would morph their product just enough to not only keep it alive but also maintain its dominance as the embedded computing architecture of choice. The compact (6U x 160mm deep) form factor, flexible packaging, pin-and-socket connector, and robust commercial/industrial construction enabled it to soldier on as an embedded computing platform for much longer than anyone could have originally predicted—often using the backplane as little more than a physical structure with power and ground distribution.
The success of VME in the DoD market was largely attributed to its:
industry standard architecture;
broad range of suppliers;
adequate performance for most applications.
However, despite its achievements, VME's days were numbered due to the rise of fast pseudo-serial interconnects such as RapidIO, PCI Express, Ethernet, Infiniband and Myrinet. Additional drawback features also contributed to its decline, such as:
Backplane limitations: signaling features were often underutilized, with the backplane itself being used for power.
Upgrade Complexity: the VME standard allowed custom backplane pin assignments and custom backplanes; board models were rarely interchangeable without backplane or other design changes.
Inefficient Sparing: Lack of fully compatible modules and lack of purchasing coordination between programs led to the need to stock a variety of spares.
Several generations of bus protocol enhancements and higher performance VMEbus interface devices helped hide the inherent liability of VME's relatively low-speed, shared parallel bus structure. In fact, much effort went towards the development of highly integrated VME Single Board Computers (SBCs) and auxiliary local IO bus expansion capabilities, all to remove data traffic from its most fundamental resource — the bus itself.
When blade systems began to take market share and Sun Microsystems introduced the 1U “pizza box,” the same capabilities were now available in more cost-effective 1U servers — with commercial motherboard production volumes that were orders of magnitude greater than any VMEbus SBC. The highly integrated systems with packaged ATX-style motherboards, industry standard IO, and Ethernet interconnects were not only cost-effective, but easier to integrate and offered greater flexibility.
Possibly the most obvious example of a successful transition from SBCs to 1U rack-mount servers in a mission critical system is the NAVSEA, Acoustic Rapid COTS Insertion (ARCI) program. By transitioning to 1U rack servers, the ARCI IPT has been able to simultaneously reduce costs in hardware, the technology insertion process, and related shipboard industrial work, while speeding the deployment of higher-performance, contemporary technology unavailable on any suitable bus structure.
Looking across the universe of military embedded systems, one can see the proliferation of 1U “pizza boxes” across virtually every major weapons, combat, and C4ISR system, including:
The U.S. Navy
Aegis Weapons & Combat Systems
AN/SQQ-89 Anti-Submarine Warfare System
Submarine Warfare Federated Tactical System (SWFTS) & Tech Insertion Hardware (TIH)
Acoustic - Rapid COTS Insertion (A-RCI)
Consolidated Afloat and Network Enterprise Services (CANES)
The U.S. Army
Distributed Common Ground Station (Army- DCGS)
Intelligence Fusion System (IFS)
Warfighter Information Network -Tactical (WIN-T)
Battle Command Common Services (BCCS)
Yesterday's Wisdom is Today's Liability
Clearly, there has been an obvious “sea change” in the way many embedded systems have been architected over the past five to ten years. The only thing certain about computing systems is that change happens, and yesterday's conventional wisdom is today's liability.
Contemporary computing technologies, virtually across the board (Mil-Aero, Commercial Data Centers, Cloud Computing, etc.) are being driven by a seismic shift in how applications are being deployed—driving the way hardware is, or will be deployed in the very near future. In summary, technology is increasingly leaning to:
Everything-Over-Ethernet Ethernet is ubiquitous and, in many instances, has become the common network, or fabric for virtually all IO in the data center.
Virtualize Everything, Minimize Appliances
Given that racks and racks of servers were typically underutilized, or utilized in a random fashion, server virtualization has enabled more uniform loading and resiliency of the data center.
Virtualizing storage has resulted in the deployment of fast and highly available NAS and SAN virtual appliances based on standard rack servers and local (direct) attached storage.
Virtualizing the network allows for more efficient use of higher bandwidth “pipes” to the point that it's often less expensive to deploy fewer of the latest, highest performance network or fabric devices.
Highly Composable Special Purpose Modules
In alignment with MOSA guidelines - modules that may be combined with other modules to provide new types of functionality.
A set of modular building blocks representing a wide range of functional elements (processing, co-processing, storage, graphics, fabric extenders, etc.) that are disaggregated & highly modular with minimal dependencies (i.e., loosely-coupled).
In deference to disciplined standards and availability of broad market commodities, such as those found in web-scale data centers, Themis introduced the ResHD product line, which incorporates five simple, yet powerful concepts:
Micro containers - A small set of standardized, and composable compute, storage and IO modules. A library of common modules, designed to abstract fast-changing technologies from the platform infrastructure, that may be combined to resolve a much larger array of specific system functions.
Platforms - A platform is the immutable HW infrastructure that supports multiple generations of micro containers by functioning as a “hotel” structure with integrated interconnect fabric. A means to build N systems from M modules., where N >> M.
Composability - The ability to combine common, interdependent modules in diverse ways to achieve different functionalities.
Convergence - The aggregation of two or more functions on a single device by way of virtualization. For example, zero rack volume virtual storage arrays by converging storage and virtual machines on the same servers.
Unified Hyperscale Fabric - Leaf & Spine interconnect fabric and Equal Cost Multi-Path (ECMP) or Open Shortest Path First (OSPF) protocols to provide predictable performance in terms of bandwidth, blocking ratios, latencies, jitter, and resiliencies.
Currently, the 20” deep ResHD offers six different types of modules, some of which have integrated key partner technologies: processor, storage, IO, management, two types of switches (one integrates a 100 Gigabit/sec Mellanox switch), and a fabric extender (with Bay Microsystems FX50).
Bringing Datacenter Strategies to Mission Critical Applications
Given that it may be a while before DoD weapons and combat systems embrace virtualization, the seeds can be planted today for near certain future occurrences.
The U.S. Navy, for example, can begin to replace legacy VMEbus systems, while simultaneously deploying an infrastructure able of accommodating future virtualized systems in incremental and manageable steps.
To this end, small form factor platforms like the Themis HDversa (6U x 10” (254mm) deep x 19” rack-mount), roughly the volume of a 6U VMEbus card cage, are designed to accommodate small form factor modules. As part of the Themis ResHD family, HD-versa provides the ability to configure a number of small, lightweight, special purpose micro containers — each sharing common electrical, physical & environmental characteristics, including:
Input power and cooling methodology;
General purpose IO integration;
Environmental resilience, e.g., kinet ics, temperature, humidity;
Life-cycle support, and Technology Insertion (TI).
The modular HDversa embraces Ethernet as the fabric of choice; providing an easy and proven path to virtualization. The standardized compute, storage and IO modules have a firmly defined backplane interconnect, unlike VME's user-customizable IO. Upon release, it will provide twelve module bays that accommodate a range of configurable modules, including:
Single-wide, compute modules featuring a single Intel Xeon D - 1500 embedded series processor, (4) DIMM sockets, (2) 10GbE, (2) 1GbE, (1) IPMI2;
Double-wide, storage server modules featuring all of the compute module functions with added (4) removable 2.5” SAS or SATA storage devices;
Double-wide, NVMe storage server modules, identical to (2.) while replacing the SAS or SATA devices with U.2 NVMe storage devices;
Double-wide, PCIe expansions also featuring all compute module functions, but adding PCIe expansion with either (2) PCIe x8/Gen3 or (1) PCIe x16/Gen3 slot;
Single-wide, 16-port 10GbE managed switches;
Single-wide, 16-port 1GbE managed switches and/or resource manager modules.
As with the full-length (20” depth) ResHD, the shallow-depth (10” depth) modules are optimized for all of the same common characteristics, but feature less SWaP and a single processor (8/16c Xeon D-15xx series), better aligning it with target applications best served by this level of performance.
Themis has positioned ResHD and HDversa to bring the latest thinking in datacenter and cloud scalability, extensibility, availability and serviceability within a SWaP-C (cost & cooling) optimized platform to Mil & Aerospace mission critical systems. These systems support existing bare metal applications while simultaneously providing a platform for taking intermediate steps toward server, storage and network virtualization strategies. More importantly, they are designed to ameliorate the previously identified problems associated with a plethora of unconstrained, and ad-hoc system configurations; deployed on a single platform, without regard for any adjacent systems. Whereas the current and future versions of RESHD address the high end, HDversa is targeted for applications where less Space, Weight, and Power (SWaP) are of greater utility than maximizing the power and density of cores, storage or network bandwidth. In effect, it's just good enough…
Voltair once said, “perfect is the enemy of the good”. What he meant was, the good should never be held hostage to the pursuit of perfection.
This article was written by Rick Studley, Chief Technologist of Mercury Systems' Trusted Mission Solutions group (Fairfax, VA). For more information, visit here .