MOBILITY ENGINEERING Produced by SAE Media Group
MOBILITY ENGINEERING
Magazines
Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Magazine cover

Current Issue

Archives

Manifold-Based Image Understanding

This method supports the entire image processing chain, from data acquisition through processing, and subsequent archiving and data mining.

The rapid growth of sensing and imaging technology, combined with the need for near-real-time action based on the sensed data, has rendered automatic processing, understanding, and decision-making vital to our national security. A unified theory and practical toolset was developed for the analysis and processing of signal and image manifolds for signal and image understanding purposes. The unifying theme is the multiscale geometric structure of signal and image families and manifolds. Specifically, theory and tools were developed for (1) model-based signal and image recognition and registration, (2) sensing and compressing data on manifolds, and (3) data-driven manifold modeling and learning.

  1. Model-based signal recognition and registration: It was determined how to best understand and infer signal and image information based on prior models for potential targets. The result was the smashed filter, a new tool for compressive classification and recognition.
  2. Sensing and compressing data on manifolds: Efficient sampling and measurement schemes were developed for manifold-modeled data. The result was in proving the sufficiency of random projections to compressively capture signals on a manifold, and applying this result to the theory of compressive sensing.
  3. Data-driven manifold modeling and learning: To bridge the gap to practice, a new theory and algorithms were developed for learning manifold models for signal and image families. Applications of these ideas include adaptation to novel targets, signal and image database mining, and compression.

The Single-Pixel Compressive Imaging Camera. Incident lightfield (corresponding to the desired image x) is reflected off a digital micromirror device (DMD) array whose mirror orientations are modulated in the pseudorandom pattern Øm supplied by a random number generator. Each different mirror pattern produces a voltage at the single photodiode that corresponds to one measurement Ym.
Compressive sensing (CS) enables the reconstruction of a sparse or compressible image or signal from a small set of linear, non-adaptive (even random) projections. However, in many applications, including object and target recognition, one is interested in making a decision about an image rather than computing a reconstruction. A framework was proposed for compressive classification that operates directly on the compressive measurements without first reconstructing the image. The resulting dimensionally reduced matched filter is called the smashed filter. The first part of the theory maps traditional maximum likelihood hypothesis testing into the compressive domain; it was found that the number of measurements required for a given classification performance level does not depend on the sparsity or compressibility of the images but only on the noise level.

The second part of the theory applies the generalized maximum likelihood method to deal with unknown transformations such as the translation, scale, or viewing angle of a target object. The fact that the set of transformed images forms a low-dimensional, nonlinear manifold in the high-dimensional image space was exploited. The number of measurements required for a given classification performance level grows linearly in the dimensionality of the manifold, but only logarithmically in the number of pixels/samples and image classes. Using both simulations and measurements from a new, single-pixel compressive camera, the effectiveness of the smashed filter for target classification, using very few measurements, was demonstrated.

In addition to the computational and storage savings afforded by compressive classification, the proposed method shares many advantages previously shown for CS reconstruction. In particular, random projections enable universal estimation and classification in the sense that random projections preserve the structure of any low-dimensional signal class with high probability. In this context, it means that one need not know what the classes are or what the classification algorithm will be prior to acquiring the measurements. Additionally, compressive measurements are progressive in the sense that larger numbers of projections translate into higher classification rates due to increased noise tolerance, and democratic in that each measurement can be given equal priority because classification rates depend only on how many measurements are received, not on the particular subset received.

It was demonstrated that, thanks to the pronounced structure present in many signal classes, small numbers of non-adaptive compressive measurements can suffice to capture the relevant information required for accurate classification. Simple parametric models impose a low-dimensional manifold structure on the signal classes within the high-dimensional image space, and the geometric structure of these manifolds is preserved under their projection to a random, lower-dimensional subspace.

The GMLC-based smashed filter is readily implementable with CS hardware, such as the single-pixel compressive imaging camera, and shares many of the attractive features of CS in general, including simplicity, universality, robustness, democracy, and scalability, which should enable it to impact a variety of different applications. More sophisticated algorithms could exploit the manifold structure to more efficiently obtain the ML estimates required by the smashed filter. For example, rather than an exhaustive nearest-neighbor search, which could be computationally prohibitive for a large training set, a greedy approach might offer similar performance at significant computational savings; other approaches that exploit the smoothness of the manifolds could also be beneficial.

This work was done by Richard G. Baraniuk of Rice University for the Office of Naval Research. For more information, download the Technical Support Package (free white paper) at www.defensetechbriefs.com/tsp  under the Physical Sciences category. ONR-0020

Topics:
Data acquisition and handling Engine components Engine mechanical components Engines Imaging and visualization Manifolds Optics Physical Sciences Powertrains
This Brief includes a Technical Support Package (TSP).
Document cover
Manifold-Based Image Understanding

(reference ONR-0020) is currently available for download from the TSP library.

Don't have an account?

More From SAE Media Group

    Magazine cover
    Defense Tech Briefs Magazine

    This article first appeared in the October, 2010 issue of Defense Tech Briefs Magazine (Vol. 4 No. 5).

    Read more articles from this issue here.

    Read more articles from the archives here.

    Overview

    The document titled "Manifold-Based Image Understanding" is the final technical report for ONR Grant N00014-06-1-0829, authored by Richard G. Baraniuk. It addresses the challenges posed by the rapid growth of sensing and imaging technologies, which have made automatic processing and understanding of data crucial for national security.

    The report outlines key factors complicating data acquisition and processing, including the increasing volume and diversity of sensor data, as well as the changing operating conditions under which data is collected. These challenges necessitate a unified mathematical and algorithmic foundation for effective data fusion and processing.

    The project's primary objective was to develop a comprehensive theory and practical tools for analyzing and processing signal and image manifolds. The focus was on the multiscale geometric structure of signal and image families, leading to three main areas of development:

    1. Model-based Signal Recognition and Registration: The project explored methods for understanding and inferring information from signals and images based on prior models of potential targets. A significant outcome was the creation of the "smashed filter," a novel tool designed for compressive classification and recognition.

    2. Sensing and Compressing Data on Manifolds: The report discusses the theoretical justification for using random projections to capture information from signals with manifold structures. This has implications for the field of compressive sensing, which aims to efficiently acquire and reconstruct signals.

    3. Data-driven Manifold Modeling and Learning: The project also focused on developing new theories and algorithms for learning manifold models that can effectively represent families of signals and images.

    The report emphasizes the importance of a coordinated approach to signal and image understanding, advocating for the integration of various methodologies to address the complexities of data processing. It concludes with a summary of the project's accomplishments and potential future work, highlighting the need for continued research in this area.

    Overall, the document presents a significant contribution to the fields of image processing and machine learning, offering insights into how geometric structures can enhance the understanding and classification of complex data. The findings have broad applications, particularly in areas requiring real-time data analysis and decision-making.

    Top Stories

    INSIDERDefense

    Feature Image F-35 Proves Nuke Drop Performance in Stockpile Flight Testing

    INSIDERMaterials

    Feature Image Using Ultrabright X-Rays to Test Materials for Ultrafast Aircraft

    INSIDERManufacturing & Prototyping

    Feature Image Stevens Researchers Test Morkovin's Hypothesis for Major Hypersonic Flight...

    INSIDERManufacturing & Prototyping

    Feature Image New 3D-Printable Nanocomposite Prevents Overheating in Military Electronics

    INSIDERRF & Microwave Electronics

    Feature Image L3Harris Starts Low Rate Production Of New F-16 Viper Shield

    INSIDERRF & Microwave Electronics

    Feature Image F-22 Pilot Controls Drone With Tablet

    Webcasts