Eyes in the Sky: HD Ethernet Cameras Take Flight Test Imagery to New Heights
Rugged High-Speed Cameras Capture Critical Flight Test Video Data
There is increasing demand for high-quality High Definition (HD) video for airborne applications such as Flight Test Instrumentation (FTI). Ideally, such new camera solutions can reduce the weight and difficultly of installing wiring, and enable data to be coherently combined with image data. Ethernet cameras can address these needs with built-in compression and multiple output streams. Additionally, as Ethernet-based networks have become an attractive choice for FTI applications, we see increased requirements for integrating Ethernet-based cameras with FTI data acquisition equipment, network recorders, and telemetry systems as this removes duplication of wiring and devices.
Using an Ethernet camera that supports onboard compression enables video compression to be removed from the FTI Data Acquisition System (DAS), or it can eliminate a dedicated unit. The camera can be connected via an Ethernet switch directly into the system, like any other data acquisition unit. Even better, because there is no need for dedicated hardware compression, SWaP is minimized and installation wiring greatly simplified.
As legacy airborne cameras are rapidly becoming obsolete, designers, systems integrators and end users have sought up-to-date digital video alternatives that offer higher quality images. Older cameras typically use coaxial wiring, which while fairly immune to noise and generally well understood, is heavy and can create installation headaches. In addition, the move to HD also adds complexity and limits the number of video frames that can be transmitted and stored without additional conversion hardware, because of the sheer size of the data they generate.
One approach for meeting the need for higher performance FTI cameras without adding complexity is to use IP cameras that utilize Ethernet wiring, switches and recorders. Ethernet IP cameras offer several key benefits when compared to traditional Composite Video Baseband Signal (CVBS), such as simplified installation, reduced system weight, and high-quality images. Even better, the required infrastructure is often already installed on the aircraft for other data acquisition purposes.
Using Cameras in Test Applications
Video is often used during a test campaign as a virtual “witness” to events (excluding high-speed video which is used for applications such as time magnification and trajectory analysis). It is generally not used as a primary data source for measuring phenomena about the aircraft, but it is a very useful tool nonetheless. When the image data is properly correlated with data from other sensors, busses, etc., one can correlate the physical event with imagery. This is especially useful for environments, like the cockpit, for example, where you can see how the pilots and operators are interacting with instruments and controls for user interface analysis and training (e.g. an organization may be looking at reaction times). Another example is to check that the instruments are displaying the same information as the bus to confirm the data the pilot is getting is accurate.
Many customers now want better quality video than older SD cameras offer. They also want cameras that can cope with being pointed into bright objects (such as the sun). Older cameras tend to use Charge-Coupled Device (CCD) sensors that suffer from pixel bleed and become washed out when encountering bright objects. However, transmitting raw HD video adds complexity and limits the number of video frames that can be transmitted and stored. Synchronizing this video data with other flight test parameters from a Data Acquisition System (DAS) can also pose a challenge. Dedicated compression cards can solve some of these issues, but they have negative implications for size, weight and power (SWaP) – all critical factors for FTI.
The extreme environmental conditions typical of FTI applications require highly reliable cameras ruggedized far beyond the levels supported by most industrial or commercial cameras. Today, a typical FTI system designer uses separate camera and video compression systems, or stand-alone video cameras, with simple recording capability. The video is not usually synchronized to other cameras or DAS data. For flight test applications, camera data for telemetry needs to be coherently synchronized and available for storage. Ideally, the data in the telemetry stream should be highly compressed to minimize downlink bandwidth. Recorded data, on the other hand, should be minimally compressed to provide maximum quality for onboard and post flight analysis.
Camera Functions and Features
Various image-processing functions are essential for delivering the appropriate image quality during test flights. Rolling shutter designs, common in consumer cameras, capture an image frame by rapidly scanning vertically or horizontally, but the time difference between different parts of the frame can result in the distortion of moving elements (such as spinning rotor blades) as shown in Figure 1. For flight tests, where the subject being imaged is rotating or moving with high velocity, a ‘global shutter’ is required to eliminate this smearing effect (Figure 1). Global shutters, which use simultaneous acquisition to capture the entire frame in a single instant of time, eliminate motion-induced distortion. Rolling shutters are generally cheaper, however, and can perform better at low light levels, so they can still be useful in some applications.
The more modern CMOS sensors used in many HD cameras today are immune to pixel bleed that CCD sensors suffer from. They can also commonly utilize Wide Dynamic Range (WDR) techniques to enhance the illumination of a scene. WDR techniques identify particularly bright and dark portions of images and control the saturation of pixels in those areas. WDR is important for achieving good image quality and is becoming common in commercial electronics such as cameras and TVs (WDR is often referred to as HDR in these applications).
Environmental Factors and Conditions in FTI
During flight tests, the aircraft must execute maneuvers not often encountered during typical operation. The aircraft and its systems must be pushed to their limits to prove the validity of the design assumptions and to record the safe operational limits. Cameras for FTI must be designed to meet stringent and harsh environmental requirements in order to withstand the extreme vibration, shock, humidity and temperature. For example, an FTI camera may need to operate on a runway at 50°C, and shortly afterwards at -30°C. Such thermal differences can change electronic component impedances as a result of temperature or moisture condensation.
FTI cameras should have rugged optical windows made of sapphire glass, for example, to protect against scratches and breakage. And they should feature ruggedized connectors to maximize the camera's availability. To ensure that environmental requirements are met, the camera should be qualified to DO-160, MIL-STD-461, MIL-STD-464, MIL-STD-810 and MIL-STD-704 at certified laboratories. Manufacturer testing, quality management (ISO 9100, EN/AS 9100), and other manufacturing standards (IPC-A-610E, IPC-A-600, IPC J-STD-011, IPC/WHMAA-A-620) need to be addressed during the design stage.
Cameras in FTI Applications
FTI camera data is typically sent to ground via a telemetry device, and to a recorder. Full HD video at 60 fps can take up to 3 Gbps of bandwidth per channel. If several HD cameras are required the bandwidth required for uncompressed video can overload the data acquisition system. Also, the transmission bit rate will affect the video quality. Lowering the bit rate will reduce the video quality unless the frame rate is decreased.
To maintain video quality, changing the frame rate has a linear effect on the suggested bit rate. Video compression is one method for overcoming some of these problems. Compressing HD video using an industry standard algorithm can reduce the bandwidth to a more reasonable amount without significantly affecting the image quality.
Today's popular compression schemes for flight test include MPEG2 (DVD videos), H.264 (Blu-ray video discs and online streaming), JPEG 2000 (archiving, medical imagery and digital cameras). H.265 is a newer standard that is similar to H.264 but offers about double the data compression ratio at the same level of video quality (or higher quality at the same bit rate). It is not yet common in airborne imaging due to its higher processing requirements.
A camera that supports onboard compression and Ethernet packet-based transmission can easily output multiple compression streams over one link. It can do this by creating two data streams, each of which stores video data in Ethernet packets. One set of packets can contain high bit rate data, the other low bit rate. This can be particularly useful for FTI. For example, two compression rates can be defined for the same channel over the same Ethernet connection, enabling the user to set one data rate for the recorder and a second data rate for Pulse Code Modulation (PCM) transmission. Having this multiple video output from the camera has the significant advantage of removing the need for separate video compression devices.
Video Transfer and Playback
Moving from the more traditional analog camera data to a packetized format requires a paradigm shift. One is now moving data, not video, so you'll need something that can read the data to display it. One tactic is to wrap industry standard image data in such a way that it can be unwrapped into an easily readable format for industry standard displays. Recorders can store Ethernet data easily – any pain in terms of displaying this data will be in the software, and this is relatively trivial as long as you're using the right formats. Latency is an important issue for replaying data. Currently, a software solution that can parse video from Ethernet data (such as the widely used Video LAN Client (VLC)) may take a second or two to display what the camera has captured.
While this is not a big problem for ground personnel who are looking at the feed, it can be an issue for those in the air. A pilot may be examining or trying to control an object in real-time and the feedback delay between their actions and the information on the screen can be jarring. To achieve lower latencies, one needs to process data as quickly as possible in the encoder and decoder. This may have an impact on image quality depending on the required bit rate and movement in the scene.
For example, if a high-quality reproduction of a complex moving scene is required, the processing overhead is going to be larger and more time consuming than a lower bitrate stream of a static image. One way to reduce latency significantly in most applications is to implement a hardware decoder, as opposed to software. This would likely reduce the time impact of decompression from approximately 1 second to 60 ms.
A common requirement for airborne displays is to show multiple camera images on one screen. For traditional cameras, such displays require a multiplexer to do the physical switching of the signals. This means extra hardware and a lot of wiring which increases system weight.
An advantage of IP cameras is that they produce multiple streams and their data can be “logically” routed. Physical routing requires dedicated hardware to take the signals and direct and/or combine them, whereas logical routing uses the existing Ethernet switch network to route packets to a recorder, display, processing system, and transmitter as required.
This article was written by Russell C. Moore, Director of Advanced Imaging & Video Systems, Teletronics – A Curtiss-Wright Company (Newtown, PA). For more information, visit here .