Software Suite for a Large Defense System

A suite of software tools was developed to enable networks composed of many hundreds, thousands, or even millions of commodity computers to protect themselves against a variety of security threats. These tools include Anagram, a content-based anomaly detection (AD) tool; ASSURE, which provides automatic software self-healing; and Aeolos, a distributed intrusion detection and event correlation infrastructure.

The work demonstrated that it is possible to construct a software self-healing mechanism that identifies new instances of known classes of failures (e.g., buffer overflows, input-drive application crashes), creates candidate fixes, tests them in an isolated environment, and (if successful) applies them to the production system.

Anagram models a mixture of highorder n-grams (n > 1) designed to detect anomalous and “suspicious” network packet payloads. By using higher-order n-grams, Anagram can detect significant anomalous byte sequences, and can generate robust signatures of validated malicious packet content. The Anagram content models are implemented using Bloom filters, reducing space requirements and enabling privacy-preserving cross-site correlation.

ASSURE (Automatic Software Self-healing Using REscue points) introduces rescue points to retrofit legacy applications with exception-handling capabilities that mimic system behavior under anticipated error conditions. This behavior is induced to recover from unanticipated software faults while maintaining system integrity and availability. Rescue points are locations in existing application code for handling programmer-anticipated failures, which are automatically repurposed and tested for safely enabling general fault recovery. When a fault occurs at an arbitrary location in the program, ASSURE restores execution to the closest rescue point and induces the program to recover execution by virtualizing and using its existing error-handling facilities.

Aeolos is a new framework for Collaborative Distributed Intrusion Detection, or CIDS, that relies on the combination of a decentralized, robust, and scalable P2P architecture, paired with compression algorithms, anonymity, and privacy mechanisms to detect attacks accurately and rapidly while encouraging participation. It uses a hierarchical distributed hash table (HDHT), which scales well for large-scale alert dissemination; multiple federations of HDHTs to ensure resiliency, provide verifiability, and detect uncooperative peers; Bloom filters for effective data privacy and an efficient source of keys for the HDHT; and anonymous but differentiable cryptographic signatures to preserve anonymity.

This work was done by Steven M. Bellovin, Salvatore J. Stolfo, and Angelos D. Keromytis of Columbia University for the Air Force Research Laboratory. AFRL-0120



This Brief includes a Technical Support Package (TSP).
Document cover
Software Suite for a Large Defense System

(reference AFRL-0120) is currently available for download from the TSP library.

Don't have an account?



Magazine cover
Defense Tech Briefs Magazine

This article first appeared in the August, 2009 issue of Defense Tech Briefs Magazine (Vol. 3 No. 4).

Read more articles from this issue here.

Read more articles from the archives here.


Overview

The document titled "Large Scale System Defense," published in October 2008 by Columbia University, presents a comprehensive investigation into techniques for enhancing the security of networks composed of numerous commodity computers. The research, sponsored by the Air Force Research Laboratory, focuses on developing self-protecting systems capable of defending against various security threats.

Key findings from the report include the development of several innovative prototypes aimed at improving network security. One significant advancement is an automatic patch generation system that can detect previously unknown attacks and create effective fixes. This system has demonstrated a success rate of over 95% in maintaining the integrity and availability of target applications while imposing minimal performance overhead. Additionally, the report introduces a technique for in situ testing of security patches, allowing for the evaluation of fixes without disrupting the stability or functionality of production systems.

Another notable contribution is Anagram, a new content-based anomaly detection system that identifies anomalous traffic with high accuracy and low false positive rates. This system is part of a broader effort to enhance distributed anomaly detection capabilities, which also includes Aeolos, an infrastructure for distributed intrusion detection and event correlation.

The report also discusses the importance of better patch management and artificial diversity in enhancing system security. The findings emphasize the need for adaptive and resilient security measures that can evolve in response to emerging threats. The research highlights the feasibility of constructing software self-healing mechanisms that can identify and address known classes of failures, such as buffer overflows and application crashes.

Overall, the document outlines a multifaceted approach to large-scale system defense, integrating various techniques and technologies to create a robust security framework. The findings underscore the potential for automated systems to significantly improve the resilience of networks against cyber threats, paving the way for future advancements in cybersecurity.

The report is structured with a table of contents that includes sections on executive summaries, detailed technical findings, and an appendix listing deliverables such as software, papers, and conference publications. It serves as a valuable resource for researchers and practitioners in the field of cybersecurity, providing insights into innovative solutions for protecting large-scale systems.