SITIS Archives - Topic Details
Program:  SBIR
Topic Num:  N102-147 (Navy)
Title:  Develop Valid Performance Measures for Multi-tasking Environments
Research & Technical Areas:  Human Systems

Acquisition Program:  Combat Systems Training ACAT II, Warfare Systems Training PEO IWS
 RESTRICTION ON PERFORMANCE BY FOREIGN CITIZENS (i.e., those holding non-U.S. Passports): This topic is “ITAR Restricted”. The information and materials provided pursuant to or resulting from this topic are restricted under the International Traffic in Arms Regulations (ITAR), 22 CFR Parts 120 - 130, which control the export of defense-related material and services, including the export of sensitive technical data. Foreign Citizens may perform work under an award resulting from this topic only if they hold the “Permanent Resident Card”, or are designated as “Protected Individuals” as defined by 8 U.S.C. 1324b(a)(3). If a proposal for this topic contains participation by a foreign citizen who is not in one of the above two categories, the proposal will be rejected.
  Objective:  Develop a measurement model of performance in multi-tasking environments. It will consider environmental variation and develop measurement tools that can be flexibly applied to different types of multi-tasking environments. It will develop a general system by which performance in real-world multi-tasking environments can be defined and assessed. Less funding for ship operating time requires more robust multi-system integrated simulation capability than is provided in the current Battle Force Tactical Trainer (BFTT). Currently, ships use a single training event focused on navigation and tactics while other Sailor skill sets atrophy. The enhanced capability of the evolved BFTT capability provided by the Total Ship Training System TSTS) will provide more realistic training as well as mission rehearsal capability via the capacity to integrate all ship systems for a more robust, multi-tasking training context. This will require a different model of performance definition, capture and assessment to support individual, team, strike force, strike group, joint and coalition training. Work definition, measurement and assessment methodologies that focus on the multi-tasking environment will better support manpower and training system requirements analysis. The TYCOM, Numbered Fleets, and System Technical Warrants must define the operational/technical requirements and standards to achieve capabilities through training. Execution of these functions will enable the Surface Navy to provide optimally trained Sailors in support of optimal and minimum manned ships.
  Description:  The requirement to reduce crew size on new platforms i.e., LCS has imposed added demands on already over-burdened personnel to multi-task in many military assignments. Yet, research has shown that multi-tasking can have serious negative consequences such as increased error, burnout, stress, and attrition as well as decreased morale. The ostensible productivity benefits of multi-tasking may be illusory because multi-tasking can have the effect of decreasing the quality of overall job performance. Moreover, adding a multi-tasking job requirement changes the quality of the work itself. Hence, the selection factors that were originally used to assign personnel to jobs, the training that was originally designed to develop the skills necessary to the job demands, and the measures of job performance that were good indicators of overall performance for the original job, may no longer be valid when attempting to predict performance when the additional requirement of multi-tasking is imposed. The typical performance assessments developed for most work environments are not applicable to multi-tasking environments in which accuracy or thoroughness may not be valued as highly as efficiency, the ability to make hard tradeoffs, and rapidly prioritize tasks. Hence new performance measures are needed that are tailored for multi-tasking environments that will be better predictors of operational performance. In a constrained fiscal environment it is critical to achieve responsible allocation of resources to well-defined training requirements. Current acquisition efforts support using shipboard systems that will double as "trainers" to provide warfare conditions resulting in more platform/command intra- and interaction and coordination to meet mission requirements and goals. It is imperative that manning models/training and performance assessments capture and then support any additional tasking/training associated with enhanced "own ship" and integrated platform, Joint, and Coalition at-sea and shore-based distributed training exercises and real world missions. Introduction of the LCS and its associated mission modules introduces increased demands for multi-tasking training. The TSTS acquisition training model and approach targets enhancing real time organic individual and coordinated shipboard combat system training capability which includes providing operational, proficiency and maintenance training for various team training efforts (e.g., Watch Team, Combat Systems, Damage Control, Engineering Control and Ships Navigation Teams) concurrently with process and outcome measures associated with platform mission tasking/accomplishment in support of dynamic Shipboard/Battle Group Doctrine.

  PHASE I: The research team will conduct R&D to develop innovative measurement methods for assessing performance in multi-tasking environments that are typical of today's shipboard operations. Increasingly, the cognitive demands of virtually every DoD tactical system employed, especially weapons systems, sensor systems, and command and control systems, far outweighs the sensory, physical and psychomotor demands placed on performers. Candidate metrics will be designed and a proof-of-concept established to select a set of the most promising measures of multi-tasking performance applicable to a wide range of environments. A model of cognitive skill variation in multi-tasking environments will be developed. This model will serve as a guide for design and selection of performance measures.
  PHASE II: Complete the needed R&D to implement measurement and assessment methodologies to support manpower and training system requirements analysis. This will enable TYCOM, Numbered Fleets, and System Technical Warrants to better define the operational/technical requirements and standards to achieve capabilities through training. Execution of these functions will enable the Surface Navy to provide optimally trained Sailors in support of optimal and minimum manned ships. Develop a system to identify and evaluate multi-tasking environments. The system will determine the appropriate performance metrics applicable to different kinds of environments. The system will be validated in at least two multi-tasking environments whose characteristics differ substantially according to the model. Performance measures will be generated and tested in empirical studies in coordination with NAVSEA's TSTS ONR Adaptive Training initiative.

  PHASE III: This company is expected to work with the Navy to adapt the tool for use with the Combat Systems Training ACAT II, Warfare Systems Training System. This tool would be modified by the company to support the evolutionary development of the Combat Systems Training Program from BFTT to TSTS by adapting the model to meet the Navy's need to implement critical skill sets aboard ship with ship needs and schedule to meet Required Operational Capabilities and Projected Operational Environment (ROC & POE) needs. The tool would be adapted to provide a validated multi-tasking assessment modeling tool that will be used to address Human Factors/training requirements during the ship design process for a new platform. The tool will incorporate unique platform design requirements to assess the manning and training impact on the crew and operators by the addition of new capabilities installed through the use of Mission Modules. PRIVATE SECTOR COMMERCIAL POTENTIAL/

  DUAL-USE APPLICATIONS: This product will also support any industry where the demands to multi-task are high. Applicable domains that may be interested in reducing risk in predicting and optimally supporting manpower and training requirements in multitasking environments include air traffic control, emergency services, Homeland security, and medicine. As the private sector downsizes, employers will find greater emphasis on multi-tasking. How can systems be designed to accommodate multi-tasking with out compromising quality, performance or safety? This has direct applicability in public safety and transportation industries, i.e., Railway management, emergency response planning, Airline and Cruise line management.

  References:   (1) COMNAVSURFORINST 1500.57 (Draft) "Surface Warfare Training Strategy". (2) Ackerman, P. (1992). Predicting individual differences in complex skill acquisition: Dynamics of ability determinants. Journal of Applied Psychology, 77, 598-614. (3) Meyer, D.E. & Kieras, E.E. (1997). A computational theory of executive cognitive processes and multiple-task performance: Part I. Basic mechanisms. Psychological Review, 104, 3-65. (4) Gilbert, T.F. (1996). Human competence: Engineering worthy performance, Silver Spring, MD; International Society for Performance Improvement. (5) Joslyn, S. & Hunt, E. (1998). Evaluating individual differences in response to time-pressure situations. Journal of Experimental Psychology: Applied, 4(1), 16-43. (6) Spiker, A. & Dick, R. Cognitive design patterns for extreme tactical environments. Industry/Interservice Training, Simulation and Education Conference, Orlando, FL., Nov 30, 2007.

Keywords:  Multi-tasking, cognitive skill, performance assessment, job selection, job placement

Questions and Answers:

No questions posed on this topic at this time

Record: of