SITIS Topic Details |
||||||
| Proposals Accepted: | |
| Program: | STTR |
| Topic Number: | AF10-BT15 (AirForce) |
| Title: | Saliency Annotation of Image and Video Data | Research & Technical Areas: | Information Systems |
| Objective: | Enable efficient image analyst performance as part of an intelligence, surveillance, and reconnaissance system via automated region-of-interest (“saliency”) annotation of image and video data.
| Description: | Human image analysts expected to remain integral elements in ISR systems for the foreseeable future. Among many reasons for continued reliance on human analysts are: (i) fully autonomous ISR systems have not reached the necessary levels of operational performance; (ii) urban clutter remains very challenging for conventional statistical signal processing methods; and (iii) collateral damage can be so catastrophic that final confirmation is required before a target is engaged.
While human analyst resources are limited and costly, several technical trends in ISR continue to mount increasing demands upon them: (i) persistent operation is increasingly desired of ISR platforms; (ii) heterogeneous sensor suites are becoming increasingly necessary to address more challenging clutter environments; and (iii) ISR sensors are generating increasingly large volumes of imagery. These trends, together with the perishable nature of information contained in the raw data and the need for fusion of information extracted by different analysts from different segments of raw data, create a critical and growing need for automated capabilities to enable analysts to perform more efficiently.
Achieving an effective capability for saliency annotation of video and image data is expected to entail fusion of intrinsic information within the collected data with side information from other sources, such as sensor data from other modalities, intelligence or environmental information, and historical or geographical context. Information from some important sources may not be quantitative in nature (i.e., “soft information”). Additionally, automated assessment of saliency necessarily involves some level of analytical and algorithmic modeling of relevant human cognitive processes to ensure that annotations are consistently of high relevance to the human analyst (e.g., so that images are not cluttered with annotations of the obvious, but that important features most likely to be missed in rapid human analysis are marked). Recent research in human-in-the-loop systems, bio-inspired multi-layer architectures, dimensionality reduction, data representation, learning theory, and sensor management are all potentially valuable resources for attaining the desired capabilities.
Regardless of the specific approach, proposed solutions should set forth quantifiable metrics of performance, including algorithm complexity and scalability as well as fidelity and latency, and should consider achievable performance and trade-offs in terms of these metrics.
| PHASE I: Develop an analytical formulation and perform a preliminary proof of concept with video and/or still image data. Devise a plan for full theoretical development, implementation, and detailed experimental validation of the methodology using representative ISR streaming video.
| PHASE II: Fully develop rigorous mathematical foundations for saliency annotation of image/video data. Instantiate a prototype capability, either as a stand-alone system or as a module of an existing system, suitable for demonstration and evaluation of the saliency augmentation technique developed. Evaluate the system's performance using multimode ISR data. Devise a specific plan for transition to practice.
| PHASE III | DUAL USE COMMERCIALIZATION:
Military Application: DoD transition opportunities include ISR systems involving human analysis of image and video data collected by UAVs, satellites, manned aircraft, special forces, and other intelligence assets.
Commercial Application: Successful realization of mathematically sound multisensor saliency methodology will enable a broad spectrum of technologies for homeland security and industrial surveillance applications.
| References: | 1. T. Serre, A. Oliva and T. Poggio, “A Feedforward Architecture Accounts for Rapid Categorization”, Proceedings of the National Academy of Sciences (PNAS), vol. 104, no. 15, 6424-6429, 2007. 2. J. B. Tenenbaum, V. de Silva, and J. C. Langford, “A Global Geometric Framework for Nonlinear Dimensionality Reduction,” Science, pp. 2319-2323, 2000. 3. S. T. Roweis, et al., Nonlinear Dimensionality Reduction by Locally Linear Embedding, Science, pp. 2323-2326, 2000. 4. D. Cohn, Z. Ghahramani, and M. Jordan, “Active learning with statistical models,” Journal of Arti?cial Intelligence Research, pp. 129–145, 1996. 5. A.O. Hero, D. Castanon, D. Cochran, and K. Kastella, Foundations and applications of sensor management, Springer, 2008. |
| Keywords: | Data Clustering, Feature Extraction, Statistical Learning Theory, Dimensionality Reduction |
Questions and Answers: |
Q: 1. Is there any requirment for real time processing? |
A: . . . response pending . . . |
As of midnight September 1, questions for solicitations SBIR 10.3 and STTR 10.B will no longer be accepted.
To read the solicitation for full proposal preparation and submission details click here. |