| |
|
Welcome
to "Team Aegis Simulation
Platform
(TASP)"
Project
What is TASP ?
The
recent trend of manpower reductions necessitates the implementation
of highly automated systems. Therefore, a decision maker must
not only function under conditions of target ambiguity, time
pressure, and information overload, he/she must also be able
to effectively operate somewhat autonomous pieces of machinery.
To support service members who must work in these environments,
researchers should first seek to understand fundamental decision
making processes in the context of situations that are dynamic,
complex, and highly automated. The purpose of this project is
to extend an existing suite of tools known as the Wright State
Aegis Simulation Platform (WASP) to better enable researchers
to assess and model human performance in a team environment.
WASP is a Naval Air Defense Warfare simulator developed by Wright
State University for the Naval Air Warfare Center Training Systems
Division (NAWCTSD). we
use a simplified naval command and control team consisting of
a Tactical Action Officer (TAO), an Identification Supervisor
(IDS), and the Electronic Warfare Supervisor (EWS). We assume
that each member of the team must accomplish specified tasks
based on the tactical situation, rules of engagement, and commanding
officer (CO) directives. The TAO supports the CO to command,
maneuver the ship, and define operational parameters. The TAO
also can give the order to deploy weapons and provide verbal
feedback to team member requests. The IDS is responsible for
the supervision and control of the identification function.
The IDS controls the Identification Friend-Foe (IFF) functions,
manages air tracks in the system, and issues verbal level warnings
to perceived hostile aircraft. The EWS is responsible for supervising
the operation of electronic support measures. Furthermore, the
EWS is responsible for the proper characterization of tracks
and association of sensors to tracks.
The proposed research will produce a team human-in-the-loop
simulation testbed, "Team Aegis Simulation Platform"
(TASP) to support modeling efforts to improve training effectiveness.
The proposed effort will extend WASP by adding more tasks and
responsibilities in a three-person team configuration, incorporate
hooks for software models of each role to be able to interact
with the simulation in real time, incorporate hooks for diagnostic
algorithms and feedback algorithms, enable automatic collection
of performance data to include automatic speech recognition,
develop diagnostic algorithms consistent with the Team Dimensional
Training methodology, design and develop software agents to
identify and measure team time windows.
Draft
Description of Work:
Develop
TASP :
The investigators will work with the subject matter expert (supported
by NAWCTSD) to develop and implement a final list of user requirements
into a team task. The task will include additional functions
to require simultaneous task execution to examine sequential
as well as parallel operator processing. The simulation will
facilitate interaction among three players and will contain
a team infrastructure that allows dynamic message passing and
synchronization of events among all team members. The tool will
extend ScriptMaker (Rothrock, 1995; Hodge et al., 1992) to enable
experimenters and controllers to incorporate additional elements
to enhance simulation complexity. Performance will be captured
and measured via standard time windows (Rothrock 2001). The
standard automated data logging found in WASP (Rothrock, 1995)
will be retained to include individual time windows, platform
state information at prescribed time intervals, state information
of the task environment at prescribed time intervals, and all
user interactions in the form of keystrokes and mouse actions.
The investigators will develop software to enable a constructive
simulation capability using an alternative cognitive modeling
paradigm.
Design
and develop team time windows and speech recognition capability:
Time
Windows:
The use of time windows for operator and team
performance measurement obviates some basic obstacles regarding
data collection and assessment. Vreuls and Obermayer (1985)
argue that the tendency to comprehensively record everything
that seems reasonable, converting the raw data into many measures
(e.g., average error and reaction time), and then attending
to those measures that show differences as a result of a battery
of statistical tests is misguided. We concur, and propose that,
if we use an informed method of data collection using time windows,
we can attain results that follow from planned comparisons and
are task independent.
We propose to establish team time windows to prescribe interactions
to occur between team members and in response to appropriate
events. To establish these windows, we will determine a task
structure based on the following factors:
User requirements on how the human-in-the-loop simulation is
to function;
Rules of engagement or standard operating procedures that dictate
action requirements between specific time intervals; and
Modes of user interaction that will be used
in the simulation.
A time window has been defined as a construct that specifies
a functional relationship between a required situation and a
time interval that indicates availability for action. A time
window does not specify what action must be taken, but only
that there exists an action which will result in the required
situation.
At the onset of operator interaction, all time windows are designated
as inactive and represented by the set U0. Until a time window
is designated as open, it remains inactive. Time windows are
designated as open if the availability for action exists for
a required situation at the current point in time space. The
set of open time windows at time t is designated as Ot. When
a required situation no longer exists (e.g., because the operator
is not longer able to act on the object for which the required
situation is desired), the corresponding time window is designated
as closed. The set of closed time windows at time t is denoted
as Ct. The membership of U, O, and C is defined to be persistent
over time, and will remain the same (i.e., Ut+1 = Ut, Ot+1 =
Ot, and Ct+1 = Ct) unless designated otherwise.
To complete the constraint specified by situativity (??) theory
(Greeno, 1998) in a temporal context, one must define operator
action and the relationship between action and time window.
An operator action is defined here as a two-tuple that includes
a detectable act performed by the operator at a specific point
in time. The relationship between action and time window can
be described by two Boolean indicator functions, 
, such that, for l=1, the function evaluates whether an action
meets the required situation specified by a time window, and
for l=2, the function evaluates the relevance of an action toward
a time window.
Thus,
Six predicates,  ,
for k=1 to 6, will now be constructed to characterize fundamental
relationships between time windows and operators actions over
a time interval T. In particular, the truth value,  ,
of each predicate is evaluated for a time interval that starts
when operator interaction in the task begins (T+) and ends when
operator interaction ceases
(T-). Given that bj occurs at time s, equations to
evaluate the first five predicates are listed as follows:
An
on-time action that results in a required situation,  ,
is formally defined as,
; (1)
An
early action that results in a required situation,  ,
is defined as,
; (2)
A
late action that results in a required situation,  ,
is defined as,
; (3)
An
action that is relevant toward a required situation, but does
not result in it,
,
is defined as,
; (4)
An
action with no corresponding time window,  ,
is defined as,
; (5)
Because the sixth predicate is based on a time window instead
of action, the equation to evaluate it is defined separately
as follows:
A
time window that has been missed,  ,
is defined as,
; (6)
Based on these six predicates, we can characterize fundamental
relationships between time windows and operators actions over
a time interval T.
In a team environment, we submit that there are two types of
time windows - individual and team. Unlike the time window for
individuals, a team time window specifies the availability for
action by the team.
In our proposed team time window framework, we will maintain
two distinct time window management systems (see Figure 1).
Local time windows represent required situations for which only
one member is able to meet. Global time windows, in contrast,
represent situations for which more than one member on the team
is able to meet. The manipulation of global time windows is
consistent with individual time windows and can be summarized
by predicate equations.
To
illustrate let us consider some examples based on the simplified
command and control team described earlier. While the roles
of the EWS and IDS are mostly distinct, both are partly responsible
for the identification of tracks. As a new track appears, then,
a global time window, specifying that a properly identified
aircraft is required, will be created for which either the EWS
or the IDS can execute actions to satisfy. If either the the
EWS or IDS alerts the command and control staff, the action
is considered on time and the window is considered met. If the
EWS and IDS never alert the command and control staff, the window
would close when the track is no longer available, and the global
time window would be considered missed.
The
rules that dictate the opening, satisfaction, and closing of
global time windows should be constructed based on a thorough
analysis of the domain. This analysis should be conducted not
only from the perspective of the physical environment, command
directives, and crew capabilities, but also from quantitative
factors that influence Team Dimensional Training (TDT).
Speech
Recognition Capabilities:
Based
on detailed assessments by members of the modeling team at NAWCTSD,
it was determined that the role of speech in team interactions
cannot be substituted. In fact, removing speech from the simulation
would fundamentally change the nature of the task in a negative
way. We do propose to build a speech recognition application
to complement the TASP simulation with sufficient processing
capabilities to recognize the need for time window manipulation,
and effect changes on the time window maintenance system to
enable real-time performance assessment.
We have assessed existing speech recognition software and will
begin development using a package called Nuance V-Builder (for
details, see Nuance). V-Builder
provides useful speech recognition features to include dynamic
grammars and "hot-word" recognition (i.e., recognition of key
words).
Our proposed design for the speech recognition application is
shown in Figure 2. Our local and global time window maintenance
systems are updated via a real-time event tracker on each workstation.
The speech recognition application would therefore interact
with the event tracker to assess time window manipulations extracted
through speech commands as well as to alert the speech recognition
application to listen to expected commands.
The
proposed speech recognition application not only aids the manipulation
of time windows, but also enables the objective measurement
of multiple Team Dimensional Training (TDT) components. An application
of the proposed TASP simulation toward assessment using TDT
is shown as Table 1.
Develop
behavioral-based diagnostic measures:
To
assess teamwork involves both an understanding of the team interaction
processes and their affect on the resultant performance. Time
windows will serve as an initial measure to assess performance;
however, team interaction measures that mirror TDT will also
be developed as well (e.g., Communication: interactions; Information
Exchange; information shared, Supporting behavior; task workload
shifting, Initiative/Leadership; establishing task priorities).
While some elements could be gathered from WASP, team communication
transcripts will also be needed. Utilizing the voice recognition
technology discussed above, the dialogue can be coded and entered
into the stream of team events for post-hoc assessment. The
behavioral-based measures extracted through SMEs will be used
in the development of such measures. The potential to replace
or augment current transcript analysis with computational measures
that combine components of TDT and time windows will provide
the Navy a broader capability by which to implement team performance
assessment without the logistical issues.
Initial validation of behavioral-based diagnostic algorithms
using the Team Dimensional Training methodology will be done.
The
strength of any team measurement methodology lies in its expandability
to other types of tasks. Harvey (2001) proposed that tasks vary
along the broad categories of task structurability, task uncertainty,
and task scope that create their complexity. We propose that,
by combining the framework to specify task complexity in the
team complexity space with the use of time windows, profiles
of the way teams interact given different task environments
can be established. In order to establish these profiles, we
must first determine the feasibility of regions of the team
task complexity space to enable implementation of performance
measurement systems. By sampling the space with different tasks,
one could potentially build different team performance profiles
that characterize individual and team responses as well as team
interaction.
To facilitate this exploratory study, it is envisioned that
scenarios would be embedded within initial validation that represent
a scenario along a vector of the complexity space (refer to
Figure 3).
Figure 3: Single Dimension Scenarios
WSU will be responsible for experimental development of this
study and will work with the NAWCTSD to determine how to facilitate
the exploration within initial validation. WSU will conduct
the data analysis of the experimental data related to task complexity.
In addition, WSU will also complete a task breakdown of the
scenarios developed in initial validation to determine how they
would fit within the complexity space as illustrated in Figure
4. This would allow WSU researchers to explore what this space
looks like or even if the dimensions are representative of actual
scenario complexity.
Figure 4: SME Tasks within Complexity Space
We
feel that this initial study will allow WSU to determine if
it is reasonable to represent a task's complexity along these
dimensions. If the results show promise, future NAWCTSD work
could look at characterizing the space by investigating other
scenarios. Ultimately, if this complexity space could be realized,
algorithms as opposed to traditional SME evaluation could develop
scenarios in the future.The research group at Wright State University
will coordinate closely with experts at NAWCTSD to ensure that
team measures are designed and implemented in manner that is
consistent with TDT methodology.
References:
Greeno, J.G. (1998). The Situativity of Knowing, Learning, and
Research. American Psychologist-53(1), 5-26.
Harvey, C. M. (2001). Gauging team tasks: How can one improve
the process?, In Proceedings of the 2001 Summer Computer Simulation
Conference, July 15-19, 2001.
Hodge, K.A., Rothrock, L., Kirlik, A.C., Walker, N., Fisk, A.D.,
Phipps, D.A., and Gay, P.E. (1995). Training for Tactical Decision
Making Under Stress: Towards Automatization of Component Skills
(Human Attention Performance Laboratory Technical Report-9501):
Georgia Institute of Technology, School of Psychology.
Rothrock, L. (2001). Using Time Windows to
Evaluate Operator Performance. International Journal of Cognitive
Ergonomics-5(2), 95-119.
Rothrock, L. (1995). Performance Measures and Outcome Analyses
of Dynamic Decision Making in Real-time Supervisory Control.
Ph.D. dissertation, School of Industrial and Systems Engineering,
Georgia Institute of Technology, Atlanta.
Vreuls, D. and Obermayer, R.W. (1985). Human-System Performance
Measurement in Training Simulators. Human Factors-27(3), 241-250.
|
