|Acquisition Program: || Objective: ||Develop a method and prototype for representing and incorporating information quality into the human-machine information fusion environment to reduce multiple sources of data bias arising from objective (physical, sensor) and subjective (human) data inputs.
|| Description: ||Information fusion in the complex battlespace confronting today’s commanders is a challenging problem in several respects. First is the large number of machine sensors providing constant streaming data that is of a variety of types, contains a range of uncertain, redundant, or erroneous data, and requires correlations that can further confound organic errors (Gorski, Wilson, Elhajj & Tan, 2005). Second is the harder problem of consolidating and correlating objective (machine sensor) inputs with subjective (human) reports. This higher-level fusion problem requires a human analyst to integrate information and draw inferences from a large number of heterogeneous sensor reports while maintaining an underlying sense of the competence and veracity of sources (Wright & Laskey, 2007). However, determining information quality in the information fusion space is a complicated endeavor and requires a multidisciplinary approach. As Bosse and Rogova (2011) note, the nature of these challenges is likely to include the variable quality and unreliability of sources, low fidelity of reports, insufficient resolution of data, and the contradictory/redundant nature of reports. As tools and methods are developed to address these problems, researchers may encounter additional problems. Some of these identified by Bosse and Rogova (2011) include the need for a quality ontology, methods to assess incoming data for quality, the need to combine characteristics into a single quality measure, how to understand context in the fusion process, and how to account for subjective biases in the information quality computation. Uncertainty in sensor data is one focus area for quality measures. Uncertainty arises in several levels of the information fusion process, including the data level (involving uncertainties with the source and content of information), the fusion level (involving the correlation of individual bits of information and the strength of evidence for a threat) and the model level (involving the reliability of the model used to structure and report information) (Kruger, M. 2008). Though improving correlations and quality of machine-based sensor fusion is important, the human sensor domain is increasingly important to military decision making. Though the challenges associated with information quality in this domain are not new to the social/behavioral/ethnography communities, they are a new challenge to the military information fusion world. For this group, the challenge is multi-dimensional. Not only are methods required to improve the basic level of quality in analysis of reporting; tools are needed to associate these subjective reports with objective machine-based sensor reports. In the human domain, research challenges include retrieving data from human sensors in a form suitable for computational analysis, data and knowledge elicitation based on decision requirements, representing first and second order uncertainty, and dealing with reporting and observational biases (Hall, Graham, & Rimland, 2011).
|| ||PHASE I: Complete a literature review, feasibility study and research plan that establishes the proof of principle of the approach for improving information quality in the information fusion problem. Place this problem in the military decision making domain through selection of a suitable dataset that includes both machine and human data inputs.
Identify the critical technology issues that must be overcome to achieve success. Clearly demonstrate how subjective and objective reports can be correlated and provide methods to establish the means by which bias have been addressed and the manner in which they have been incorporated into the solution.
|| ||PHASE II: Produce a prototype method/tool/algorithm(s) capable of producing meaningful information quality advances to a problem set and that addresses sources of bias that are likely to exist in the data. Demonstrate the capability with a relevant dataset in a relevant environment that includes military subject matter experts. Develop a set of metrics that can be used to measure the improvements of this capability over standard fusion tools.
|| ||PHASE III: Produce a system capable of deployment in an operational setting of interest. Test the system in an operational setting in a stand-alone mode or as a component of larger system. The work should focus on capability required to achieve transition to program of record of one or more of the military Services. The system should provide metrics for performance assessment.
|| References: ||
Gorski, J., Wilson, L., Elhajj, I., & Tan, J. Data Fusion and Error Reduction Algorithms for Sensor Networks. 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005. (IROS 2005).
Hall, D.L.; Graham, J.; More, L.D.; Rimland, J.C. Test and evaluation of soft/hard information fusion systems: A test environment, methodology and initial data sets, Information Fusion (FUSION), 2010 13th Conference on Information Fusion, July 2010.
Kruger, M. “Two Maybes’, One ‘Probably’ and One ‘Confirmed’ Equals What? Evaluating Uncertainty in Information Fusion for Threat Recognition/” Proceedings MCC2008, Cracow, Poland, Sept. 2008.
Rein, K.; Schade, U. How certain is certain? Evaluation of uncertainty in the fusion of information derived from diverse sources, Information Fusion (FUSION), 2009 12th Conference on Information Fusion, July 2009.
Rogova, G.L.; Bosse, E. Information quality in information fusion, Information Fusion (FUSION), 2010 13th Conference on Information Fusion, July 2010.
Wright, E.J and Laskey, K.B. Credibility Models for Multi-Source Fusion, Proceedings of the Ninth International Conference on Information Fusion, July 2006.|
|Keywords: ||information quality, data bias, information fusion, subjective sensor reports, objective sensor reports|
Questions and Answers:
Q: What is the Program of Record (PoR) transition target for this technology?
A: Could be DCGS-A, CPOF, and/or any tools to support the Company Intel Support Team (COIST).
Q: Has Phase II funding already been identified or allocated?
Q: Is there a constraint for the software environment that the system must ultimately execute within?
A: The Army is not building any more new, stand-alone systems...all new capabilities must work within existing systems. So, anything new must be capable of being inserted into existing systems.
Q: Can a Use Case be provided that would demonstrate the type of analysis that motivated this topic?
A: Here are some vignettes that support all 4 topics.
HERE IS ONE: In a COIST, several analysts will be working at an intel station and can be expected (now or in future) to be linked to neighboring COISTs and to higher intel echelons (Bn, Bde, Div). The Company patrols have been asked to be on the lookout for three HVIs and have been given descriptions and pictures of these individuals. They have also been given geo-locations of possible hotspots that could be meeting places, bomb assembly locations, gun caches, or safe houses. These locations cross the AO of three Companies, and it is expected that the HVIs move between these AOs, but at an unknown rate and at unknown track(s). If each COIST operates only within their AO, they will be unsuccessful in identifying suspicious behavior and activity. The COIST analysts in each of the AOs need a CVA method/technique/capability that allows them to share information across a larger geographic space and identify behavior/activity that may not look suspicious at a village level, but would look suspicious at the
HERE IS A SECOND: Soldiers are operating in an unfamiliar AO where they do not understand the norms of society and the underlying basis for behaviors and actions. They do not have any knowledge of religious, social, political, or ethnic celebrations or rituals. They do not have any basis upon which to judge anomalous or potentially risky behavior. For example, they do not understand marriage ceremony activity that could involve large numbers of people gathering in a public space and behaving in loud,
semi-organized behavior (dancing, mingling, congregating in small sub-groups, firing weapons in the air, etc.). The Soldiers also do not understand the pattern of life behavior for the groups in the society, such as market days, religious days, how children attend school, and how to identify the influential members of society. The Soldiers need an 'overlay' of the cultural markers for this society, helping them to distinguish large ethnic groups, regional locations, religious sensitivities, and high-level patterns of life.
HERE IS A THIRD: Think of Hurricane Katrina, before, during, and after (say 5 days total). Imagine a control center with many displays, computers, and small teams representing engineers (levee watch), health care (hospital evacuation), transportation (evacuation planning, bus schedules, pick-up locations), weather (storm track, timing of storm, wind strength, etc.), public affairs (public information, warning, evacuation routes), local/state/regional/national leaders (coordinate evacuation, shelter,
disaster response), and on and on. As the storm track continues to New Orleans, each team must monitor the larger scenario as they continue to plan within their own task. How many busses are needed for the 9th ward? What is the latest possible time for the busses to arrive/depart and by what route, to what destination? How is this plan altered if the storm speeds up/slows down? How is it altered if a levee breaks? You could also think of the Arab Spring demonstrations in Egypt. Government leaders around the globe were trying to understand this behavior and how it spread to other nations
in the region, and what the implications of the demonstrations would be for many issues.
HERE IS A FOURTH: Optimize actions: given a developing section of the world, global leaders must determine how best to deploy infrastructure resources, such as road-building, electric grids, water structures, etc. Forecast future states: given a developing section of the world, state and defense department leaders must try to plan for future conflicts and/or disaster relief efforts.
Q: In this application, are the collaborator’s considered to be peers each capability of performing the same type of analysis with equivalent proficiency? Or are the collaborator’s considered to be focused on a particular analysis aspect to a problem? That is, are the collaborators peers or does each collaborator function with a certain role?
A: Well, it could be both. Collaborators could be COIST analysts operating in adjoining sectors (most likely) or they could be collaborators in a vertical sense (coordinating with battalion analysts).
Q: Will the following information be made available during the Phase I effort?
a. Description or workflow currently used to address analysis problems motivating this topic.
b. Current visualization tools/techniques used to present, analyze, and modify a knowledge used by representative operators.
A: a. The COIST is very new and is being implemented in a variety of ways...the functions are not standardized yet...but you can look at the COIST Handbook (google it) and get some ideas of what is intended.
b. Current vis tools would include any that are available on analyst notebook, axis pro, etc.
Q: Is this technology viewed as being follow-on to an existing established system or a totally new capability designed from scratch?
A: Follow-on to an existing established system.
Q: Which organization within the US Army will be the technical monitor of this effort, should it be awarded?
A: Army Research Lab.
Q: The topic descriptions, keywords, and answers to posted questions for OSD11-DR2 thru OSD11-DR4 appear to be very similar. They all refer to aspects soft information fusion, visualization, collaboration, and decision aides. Can you please clarify the emphasis of this topic as opposed to the other two topics?
A: It is correct that DR-2, 3, and 4 have a similar flavor. This is how I have been differentiating them: DR2 focuses more on the process of decision making, DR3 addresses lower-level visualization of information, and DR4 focuses on data and user interactions in a collaborative sense.
Hope that helps.