| Acquisition Program: | PEO Simulation, Training, and Instrumentation |
Objective: | To use multiple video sources, virtual (visual) databases, geospatial data sets, and frontline sensors to complement each other in an integrated fashion. This will allow the current and future force to see the battlefield in ways that were previously impossible, taking full advantage of available sensing platforms. The Dynamic Integrated Video/Virtual View (DIV3) will dynamically use the best available source of visual, command and control, and geospatial information to provide an enhanced battlefield view. For example, the DIV3 will enhance live video feeds in a manner comparable to a heads-up display (HUD) to show overlay symbology (e.g., a planned route or departure point), known or suspected unit locations, recent changes to the environment, and geospatial data such a river depth. This research will address the operational gaps such as Ability to Conduct Joint Urban Operations; Timeliness of Analysis and Information Dissemination; and Train the Force How and As it Fights.
| Description: | The Dynamic Integrated Video/Virtual View (DIV3) will provide enhanced situational awareness of the battlespace in terms of predictive analysis and decision-making techniques that will help the commander/operator estimate enemy courses of action and adaptively plan friendly actions. This effort will focus on a key enabling technology for DIV3, namely the ability to incorporate innovative technologies (tools & algorithms) for merging/fusing information from 7 major data sources: (1) live/stored video, (2) imagery (overhead, still pictures), (3) geographic information system (GIS) data, such as cultural features and attribution, (4) C4I data (overlay symbols, unit locations, etc.), (5) operator annotations & input, (6) mission planning systems, to include Computer Generated Forces applications used in training, and (7) changes between previously sensed data and newly acquired data (e.g., the sudden appearance of a crater on a road, or the destruction of a bridge). All of this information will be geo-referenced, integrated, and displayed as a mixture of video and virtual (computer-generated) images. The data fusion capabilities will be well-suited to use within an intuitive user interface, taking advantage of touch screens, motion sensors, voice commands, etc., to allow the observer to select the viewpoint and the nature of the enhanced information displayed.
The M&S Training and Operational communities have identified the lack of critical technologies, techniques, and algorithm and for merging/fusing information data sources to aid decision-making capabilities. To achieve these goals, DIV3 will require technology to merge these disparate data sources in real-time (or near real-time). This technology will also support the ability to detect and discretely identify changes to geospatial information based upon frontline sensors, whether overflight (UAV) or ground (UGV). Ground sensors could include human observation (data entry), photos, video, GPS data, or the proximity sensors of a UGV. Regardless of the source, the observed environment must be compared to pre-existing geospatial data, with detected changes being discretely identified, and then integrated back into the electronic terrain representations in a way that is clearly distinguishable to the human observer and underlying services (e.g. automated route planning).
A typical scenario for the use of the merging/fusing technology is to assist a future force soldier or an Future Combat System (FCS) operator to be able to select portions of the merged video/virtual display to gain additional information such as notes taken during mission rehearsal, detailed attribution such as the number of rooms in a building (if available), or links to alternative data sources (floor plans, map views, etc.). By using technology/algorithm to merge the information sources, the operator will be able to select variables such as projected weather, time of day, sensor in use, to wargame possible scenarios. This will allow the future force soldier to see the battlefield from an enemy’s position (or likely position).
This technology could be used outside of military applications, such as automatically noticing faults in routing networks for applications like MapQuest, extending assisted-navigation applications to provide richly detailed, 3D directions with visually hi-lited routes, providing virtual reality tours for the tourism industry, and supporting police or security systems.
| | PHASE I: Conduct industry and technology surveys to determine the best starting points for merging/fusion technologies, including approaches for visualizing the merged data. Consider and document approaches for comparing sensed data to pre-existing data sets to identify and categorize changes, georeference such data against video sources, etc. Provide a report illustrating how continued research would be conducted, including a description of how the Phase I DIV3 data merge capability maps into FCS technologies.
| | PHASE II: Provide initial concepts on software algorithms to address merging and visualization of various sources. Provide a basic demonstration of selected DIV3 data merge and display capabilities as a proof of concept and describe an approach for a more complete implementation. Develop a prototype DIV3 capability targeted to key areas not addressed by existing or near-term technology. Provide a detailed demonstration of the resultant capability as it applies to the Future Combat System (FCS). Conduct experiments to evaluate usability from the standpoint of future force soldiers.
| | PHASE III: Broaden the implementation of DIV3 to a complete data merge and visualization capability ready for transition to targeted military applications. Include evaluation of applicability to commercial uses such as navigation systems, airport security, urban planning and emergency preparedness. Possible uses including automatically noticing faults in routing networks for applications like MapQuest, extending assisted-navigation applications to provide richly detailed, 3D directions with visually hi-lited routes, providing virtual reality tours for the tourism industry, and supporting police or security systems.
| References: | 1) Efficient Environment Database Generation - Potential TDB Process Enhancements, by Litton TASC for STRICOM, December 2000.
2) Environmental Data Modeling for Simulation System Requirements Specification, by Dale D. Miller, Annette Janett, Mary Kruck, Richard Schaffer, Paul A. Birkel, Bernard Gajkowski, and Pamela Woodard, IITSEC 2000 Proceedings.
3) Integrated Command And Control Toolset For Global Aerospace Operations - Rodney Davis, Eric Sorton, Ross Manges, John Ward, David Hobbs, Command and Control Technologies, 1425 Chaffee Drive, Titusville Florida 32780 USA 321-264-1193; davisrd@cctcorp.com
4) Bauer, J., Karner, K., Schindler, K, Klaus, A. & Zach, C. (2003)
5) Segmentation of Building Models from Dense 3D Point Clouds. Workshop of the Austrian Association for Pattern Recognition, Laxenburg, Austria, 2003.
6) Brenner, C. (2000). Towards Fully Automatic Generation of City Models. ISPRS, vol. XXXIII, Amsterdam, 2000.
7) Anon, Blue Force Tracker and Army Aviation Operations in Afghanistan, circa 2003.(www.quada.org/chapter/Drum/blue_force_tracker_and_army_avia.htm)
8) Baddeley, A, ‘Bowmen Aims at Interoperability Target’, Military Information Technology, 11 March 2004.
9) Dunn, R, Blue Force Tracking: The Afghanistan and Iraq Experience and its implications for the US Army. San Diego, Northrop Grumman, October 2003.
10) Erwin, S, Army to Upgrade Land Warrior System with Blue Force Tracker, National Defense Magazine, February 2004 (www.nationaldefensemagazine.org/article.cfm?Id=1339)
11) French, M, ‘Marines Build Bridge to New Radio System’, FCW.COM 23 February 2004, (www.fcw.com/fcw/articles/2004/0223/tec-marines-02-23-04.asp).
12) Gourley, S, ‘OIF Lessons Highlight Comms Compatibility: Marines coordinate joint service and multinational C4I capabilities.’ Military Information Technology Online Edition: 2003 (www.mit-kmi.com/articles.cfm? DocID=467).
13) Elaksher, A. F. & Bethel, J. S. (2002). Reconstructing 3D Buildings from LIDAR Data. Photogrammetric Computer Vision Symposium, September 9 - 13, 2002, Graz, Austria.
14) Früh, C. & Zakhor, A (2003). Constructing 3D City Models by Merging Ground-Based and Airborne Views. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2003, pp. 562 – 569.
15) Früh, C. & Zakhor, A. (2004). An Automated Method for Large-Scale, Ground-Based City Model Acquisition. International Journal of Computer Vision, Vol. 60, No. 1, October 2004, pp. 5 - 24.
16) Haala, N. & Brenner, C. (1999). Extraction of Buildings and Trees in Urban Environments. ISPRS Journal of Photogrammetry & Remote Sensing, 54 (1999), pp.
17) Boiney, L. Team Decision Making in Time-Sensitive Environments. In Proceedings of the 2005 Command and Control Research and Technology Symposium. Department of Defense Command and Control Research Program: McLean, VA, June 2005.
18) Brickman, B. J., Hettinger, L. J., Stautberg, D. K., Haas, M. W., Vidulich, M. A., and Shaw, R. L. “The global implicit measurement of situation awareness: implications for design and adaptive interface technologies.” In M. W. Scerbo & M. Mouloua (Eds.), Automation Technology and Human Performance: Current Research and Trends. Mahwah, NJ: Lawrence Erlbaum Associates, 1999.
19) Crabtree, A. Designing Collaborative Systems: A Practical Guide to Ethnography. Series: Computer Supported Cooperative Work. Springer, 2003.
| | Keywords: | Simulation, video, data fusion, situational awareness, battle space and command, situational awareness, battle command, synthetic natural environment (SNE) |