Designing Multi-Channel Microwave Radio Systems Using Optical Interconnects

Designing multi-channel microwave radio systems using optical interconnects

Imagine if it was possible to build multi-channel microwave radio systems leveraging optical rather than copper interconnects. Several benefits of this paradigm shift seem likely to occur including:

  • Streaming sample control and configuration data, as well as reference clock and synchronization signals via fiber, simplifies radio front-end design and signal distribution whilst simultaneously reducing cable mass.

  • Eliminating copper signal wiring in future radio designs offers increased architectural freedom and flexibility whilst reducing crosstalk and inter-channel interference for better performance.

  • The optical concept substantiates a valuable separation of front-end analog design from back-end digital signal processing heralding the arrival of fully digital antennas with enhanced operation arising from advanced electronic beam steering.

On initial consideration, this optical transport idea may not seem especially innovative, given the profusion of optical links widely used across global data centers. There is a strong case for using fiber as highlighted in the accompanying sidebar. However, in the context of radio design, there are novel engineering challenges to be resolved.

It is important to highlight that this approach aims to transport all signaling between the analog signal environment and digital domains and thus must manage low-speed control signaling, handle reference clock distribution and, critically, ensure deterministic multi-channel synchronization. This is key since signal spatial (phase) information must be preserved if highly desirable electronic beam steering capabilities are to be facilitated.

Project Objective

Until now, two copper-based serial interfaces have been used for linking broadband data converters to digital signal processors. They are the industry standard JESD204 link and the alternative, license-free ESIstream system from Teledyne e2v.

Despite optical data links powering the modern global broadband backbone, to date there have been few efforts to replace traditional copper in radio frontend design. The optical digital harness (ODH) project set out here to explore the capabilities and limitations of a prototype optical data link bridge, the intention being the reliable transport of four types of system signals. They are, in addition to the obvious sample data:

  • A reference clock;

  • All single-ended signals with a frequency lower than ~100 MHz, including SPI and general purpose I/Os (GPIO);

  • A proprietary synchronization signal designed to facilitate system-wide deterministic sampling;

  • Data converters samples data.

In Teledyne e2v’s data converter sphere, determinism (i.e., the mitigation of undesirable pseudo-random metastability) is assured by a novel method called SYNC chaining – the distribution of a simple, single-edge sync signal together with a sync flag, allowing a massively parallel system of converters to maintain synchronization throughout. However, what is a relatively straightforward approach to the problem and easily implemented in copper, becomes more challenging in the shift to a fiber transport layer.

An experimental fiber-based prototype has been designed to evaluate its suitability as an alternative to copper as well as to identify any performance effects introduced by temperature and other environmental factors.1 In the initial experimental phase, the physical fiber link is a modest ten meters – enabling significant separation between the microwave RF analog and data processing domains.

ODH Implementation

The main challenges defined at the start of this project were twofold:

  1. Ensuring the reliable transport of relatively slow control signals of which the most important is;
  2. The delivery of error-free, deterministic synchronization to multiple converters to ensure system-level determinism and thus synchronous sampling. This remains on the work in progress list at this point.

Earlier efforts working with fiber links have been described in a combined control and data context. One of note, from Lincoln University in the UK, describes an FPGA-based project2 to aggregate SPI and GPIO data, to be sent over a single optical link. The article describes data coding approaches for extended transmission distances and illuminated the challenges arising. The earlier conclusions were:

  • Care to establish reliable link synchronization, necessitating a master reference clock.

  • Encoding is a key success factor. In the example described, 8b/10b coding proved orders of magnitude better than other simpler coding schemes (such as exclusive OR - XOR, or Grey codes).

  • Finally, if latency is important, as it is for this experimental work, the authors detailed how re-synchronizing the link could compensate for known system latencies.

Figure 1 - Optical Digital Harness Conceptual Design.

This previous work forms a basis from which to extend the current prototype work. A new solution proposed (Figure 1) takes economic, low-power programmable logic devices (here FPGAs) to be configured as simple coding engines with the objective to format, aggregate and serialize slow control vectors generated directly from system GPIO and SPI messages. Encoded message vectors are then transported across three parallel optical lanes with a reference clock and a discrete coded SYNC signal.

Experimental Implementation

The starting point for the prototype is an existing high-end IC evaluation system. A system comprising an advanced microwave front end card for the EV12AQ600 was selected. This quad channel, 12-bit 6.4 Gsps broadband RF ADC with the ability to sample up to 6.4 GHz offers a suitable FMC based target ADC for the prototype work. The EV12AQ600 has been designed to mount directly onto a powerful signal processing (DSP) card using a standard FPGA Carrier card format (FMC) connector. The FMC provides a useful physical break in the existing interconnect and, thus, an ideal insertion point for a modified fiber-based digital transport layer as shown.

Figure 2 - Manchester Encoding of the SYNC Edge.

Taking inventory of the various digital control and clock signals needed to bridge the FMC break point, three discrete fiber channels are required for control signaling headed to the remote receiver ADC. Moreover, two further channels handle data and SSO communication back to the DSP card.

Focusing on a practical solution, the search for cost-optimized, low-power programmable logic devices (PLD) was on. Following a market scan, Artix 7 FPGAs emerged as a suitable choice for the code engines. Referring to the system partitioning shown in Figure 2, it should be apparent that for future implementations the encode engine resources could easily be merged with the DSP FPGA for economy. Less than 5% of the Artix 7 resources are used today. Prototype expediency and programming ease meant that the coding engines would remain as discrete blocks in the early design.

Note also how the need for remote access to a reference clock forced the decode end of the prototype to require a local clock generator. The physical layer is implemented using Radiall D-lightsys® VCSEL optical drivers. These drivers claim to support data rates from 100 MHz to 10 GHz with excellent clock jitter performance over this range. Radiall’s rugged D light portfolio offers three data rate grades (10 Mbps, 5 & 12 Gbps) and multiple package options. These components are configured either as four-channel duplex transceivers or 12-channel simplex for receiver and transmitter applications. The drivers are protocol agnostic and offer both standard LVDS and CML electrical interfaces.

Two Key System Challenges

Two core issues are addressed in this prototype. They are:

  1. Encoding the non-toggling SYNC signal to establish a robust optical SYNC-chain solution for system-wide application, ensuring determinism;
  2. Ensuring full rate link locking.

Providing System-Wide Deterministic Synchronization

SYNC chaining has been documented in several previous articles, and as stated previously, is a critical success factor for this project if a truly useful optical alternative to copper is to emerge. The challenge in transitioning to robust SYNC distribution over fiber is that the SYNC signal is flagged by a relatively slow, 10ns logic level change. Given the ‘un-clocked’ nature of SYNC, how can its precise level transition be transported over fiber?

Manchester encoding is a simple phase-shift keying technique that provides a suitable option. Binary data gates the phase of a clock (Figure 2), effectively ensuring a mid-bit level transition, helpfully maintaining DC balance (in copper interfaces) and provides a regular source of data edges to ensure the optical link remains locked yet can distinguish SYNC assertion.

Manchester code is easily implemented by exclusive OR-ing (XOR) the SYNC signal with the reference clock which is already needed throughout the system. Even though a steady stream of zeros precedes SYNC assertion seen here, the receiver registers, within half a reference clock period, SYNC’s low to high transition. Incidentally, latency might impact system-level synchronization and is part of the challenge to ensure determinism with copper-based links. With fiber, latency is orders of magnitude lower than copper. Accordingly, adjusting for differing SYNC travel times across multi- array systems becomes a vanishingly small problem in most practical cases giving the optical approach an edge.

  • Enhanced performance – No crosstalk, Zero EMI

  • Long distance communication >> 20 m

  • Reduced weight, increased data density

Full Rate Link Locking

Success here relies on two factors. The first is inherent to the digital design of the coding engines. Moreover, the coding engine relies on the embedded PLL’s ability to discriminate clock edges and synchronize to the reference clock. External to the code engine, is the optical link characteristics. Over a wide range of data rates, the performance of the link varies. For this reason, the optical transceivers provide internal current modulation control. This aspect of the prototype performance is not yet characterized and will be explored in greater detail as the project looks to performance over temperature.

Results So Far

Despite extensive digital simulation, the proof of operation is only possible once the hardware is in place and ready for power-up. Perhaps unsurprisingly, despite a rigorous round of checking, a couple of minor hardware errors slipped into the initial design which required some minor reworking.

Initial power-up also disappointed. Despite the evidence from comprehensive simulation effort, it proved challenging to track down the source of a problem preventing the receive end to lock to the reference clock. Suspicion started to form around the quality of the received serialized reference clock. This source became clear when lowering the reference clock rate from the nominal maximum 12.8 Gbps lane rate of the EV12AQ600 to a substantially reduced level at 9.0 Gbps (4.5 GHz reference clock) finally produced the sought- after link locking.

Conclusions

Despite not having attained full rate data signaling, the experimental design is proven to be robust over a ten-meter fiber although at a slower data rate than targeted. Furthermore, once link synchronization is established, the prototype runs non-stop without loss of link locking or change in bit-error rate (BER). Better still, the Radiall optical transceivers provide digital control of the optical modulation current. This permits further tweaking of optical transmission characteristics. This should ensure, that with some further optimization, that the data transmission eye can be enlarged to allow full rate transport at 12.8 Gbps soon.

Several important issues require further verification before the development team can claim success. The most important of these is to verify that rather than simply demonstrating SYNC decoding at a single receive location, that a wider system-level deterministic multi-channel operation is possible. Moreover, that determinism can be maintained over varying environmental conditions - particularly temperature and voltage shifts.

Time will tell, but progress to date is encouraging and we stand on the brink of a major paradigm shift towards advanced digital, smart microwave antenna design.

References

  1. Can deterministic digital phased array control be delivered over fiber?’, MWJ Nov. 2021
  2. Cross-channel synchronization is further complicated by metastability – an inherent, pseudo-random behavior of digital systems. It can be mitigated through careful design.

This article was written by Stéphane Breysse, Application Engineer, Teledyne e2v (Chelmsford, UK). For more information, visit here .