SITIS Archives - Topic Details
Program:  SBIR
Topic Num:  N07-047 (Navy)
Title:  Three-Dimensional Control Panel Simulation
Research & Technical Areas:  Air Platform, Information Systems, Human Systems

Acquisition Program:  PMA-205 Aviation Training Systems
  Objective:  Develop an innovative, reconfigurable, conformable screen capable of simulating, visually and tactically, the control panels, buttons, keyboards and switches of a variety of aircraft.
  Description:  Nearly ten years ago, Hoffman (1998) provided converging evidence for the value of adding physical qualities to virtual objects used in training. That same year Peterson, Wells, Furness, and Hunt provided evidence that an interface that uses the body enhances certain components of navigation in virtual environments. Historically these requirements have been met by building custom mock-ups for each cockpit or operator interface training device. Today researchers are now exploring how to train tomorrow’s surgeons with the use of simulated images with physical qualities by leveraging recent developments in the modeling and simulation industry (Hatwell, et al, 2003; Kahol, et. al, 2005). Such innovations could also enable a single training device to serve for multiple training requirements (at sea) where available space is at a premium. These developments include but are not limited to: organic light emitting display (OLED) panels, flexible organic light emitting displays (FOLEDs), and fiber optics. Active materials used for shape deformation such as the shape memory alloys used in the Smart Wing program, piezoelectric materials such as isotropic wafers, and anisotropic forms (i.e., active fiber composites, macro fiber composites, and single crystal fiber composites, or even the new flexible transistors that provide paper-like electronic ink displays are also options for consideration. A deformable material with photorealistic images of a control panel could present an unlimited number of interactive trainer control panels, in one training device. These advancements could address the requirements for forward deployed mission rehearsal trainers, such as those cited in the Navy Aviation Simulation Master Plan. Such multi-platform trainers would enable tomorrow’s at sea trainee to observe and then interact with physical representations of objects and surfaces in a digitized/morphed “mock-up” of any cockpit or control panel during mission rehearsals. Development of a prototype methodology that enables visually simulated panels, buttons, key boards, and switches to cross from a flat panel touch-screen visual presentation to an actual or representative physical state would result in a reconfigurable conformable screen that could simulate the control panels, buttons, keyboards and switches of a variety of aircraft visually and tactically. Prior research indicates that unlike the visual and auditory sensory systems, only the haptic system is capable of implementing direct action (NSF Workshop). Haptic interfaces such as joysticks and force-reflecting mechanisms can create a feeling of immersion, provided they convey a realistic sense of touch and exploration to the user. Considering the highly tactual nature of naval tasks (i.e., manually locating and operating navigational controls), the creation of effective haptic environments is viewed as a critical step in realizing the potential of VE training technologies. As Srinivasan et. al. suggested, "It is quite likely that much greater immersion in a virtual environment can be achieved by the synchronous operation of even a simple haptic interface with a visual display, than by large improvements in the fidelity of the visual display alone."

  PHASE I: Determine the feasibility of generating two different representative physical surfaces from digitized images of an aircraft's control panel's and recommend an approach.
  PHASE II: Develop, evaluate, and refine the prototype to demonstrate at least two different training applications identified by the fleet. It is possible the sponsor would seek development of a complete re-conformable cockpit for demonstrations of transforming the panels from one aircraft type to another, and then back again. Additional tailoring for demonstrations could be pursued through options as funds became available.

  PHASE III: Transition the prototype-training panel to Navy, Marine Corps, and Coast Guard training facilities throughout the world.

  PRIVATE SECTOR COMMERCIAL POTENTIAL: Developments and innovations from this effort could be applied to industrial training devices, the prototyping of controls, architectural modeling, the computer gaming industry, and to other tactile simulations.

  References:  1. Mark A. DeSorbo, “Start-Up Aims To Revolutionize Print With Electronic Ink,”. Computer Graphics World, http://cgw.pennnet.com/Articles 2. “Flexible Microelectronics & Displays Workshop,“ US Display Consortium (www.usdc.org ) 3. Beverly Rosenbaum, “The Eyes Have It.” HAL PC, Trumors, January 2002. http://www.hal-pc.org/journal/2003/03_jan/column/trumors/trumors.html 4. Cesnik, Carlos “Wing Shape Deformation for High Performance Vehicles,” ICASE Morphing Seminar Series, NASA LaRC, 26 June 2002. 5. K. Kahol, P. Tripathi, and S. Panchanathan, “Tactile Cueing in Haptic Visualization,” to appear in Proc. ACM Workshop on Haptic Visualization at AMC Computer Human Interface Conference (CHI 2005). 6. Hatwell, Y., Streri, A., and Gentaz, E., Touching forKnowing. Cognitive psychology of haptic manual perception. Philadelphia, PA, USA: John Benjamins Publishing Company, 2003. 7. Hoffman, H.G. (1998). Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments. Proceedings of the IEEE Virtual Reality Annual International Symposium '98, Atlanta GA, p. 59-63. IEEE Computer Society, Los Alamitos, California. 8. Barry Peterson, Maxwell Wells, Thomas A. Furness III, Earl Hunt (1998). The Effects of the Interface on Navigation in Virtual Environments. Presented at the Human Factors and Ergonomics Society 1998 Annual Meeting. 9. Srinivasan, M.A., Salisbury, J.K., Brock, D., & Beauregard, G.L . (1995). Haptic interfaces for naval training with virtual environments (Final report on contract no. N61339-93-C-0083).

Keywords:  Tactile Stimulation; Reconfigurable Training Surfaces; Mock-ups; Flexible Microelectronics; Organic Light Emitting Display Panels; Flexible Organic Light Emitting Displays

Questions and Answers:
Q: A 2-D panel only requires knowing the overall active area dimensions and the "pixel" size for sensitivity. If a 3-D effect is needed (and it IS feasible although "joysticks" require more work to reconfigure than buttons), then more information is needed for the Z-axis besides "pixel" size -- depth of movement, need for angular movements (sliders, joysticks)?
A: The 3D (z-axis) data could be obtained with a stereo digital image of the panel. As a button is pressed, a second image would reveal the range of movement. Sliders, knobs, joysticks are all used in cockpits. Whether or not these could also be generated depends on the inventors approach to the problem. All innovative and plausible concepts that are proposed for different types of controls will be considered. It may be that different types of solutions would be optimal for different types of controls. It is likely that a proposal which best addresses one type of control, could result in a successful Phase I award - if it appears to be the most plausible solution for that particular type of control.
Q: A 2-D panel only requires knowing the overall active area dimensions and the "pixel" size for sensitivity. If a 3-D effect is needed (and it IS feasible although "joysticks" require more work to reconfigure than buttons), then more information is needed for the Z-axis besides "pixel" size -- depth of movement, need for angular movements (sliders, joysticks)?
A: The 3D (z-axis) data could be obtained with a stereo digital image of the panel. As a button is pressed, a second image would reveal the range of movement. Sliders, knobs, joysticks are all used in cockpits. Whether or not these could also be generated depends on the inventors approach to the problem. All innovative and plausible concepts that are proposed for different types of controls will be considered. It may be that different types of solutions would be optimal for different types of controls. It is likely that a proposal which best addresses one type of control, could result in a successful Phase I award - if it appears to be the most plausible solution for that particular type of control.

Record: of