A Psychological Fidelity Approach to Simulation-Based Training: Theory, Research, and Principles Steve W. J. Kozlowski and Richard P. DeShon Michigan State University Draft 1: 20 July 2000 Final Submission: 20 September 2002 Kozlowski, S. W. J. & DeShon, R. P. (2004). A psychological fidelity approach to simulation-based training: Theory, research, and principles. In E. Salas, L. R. Elliott, S. G. Schflett, & M. D. Coovert (Eds.), Scaled Worlds: Development, validation, and applications (pp. 75-99). Burlington, VT: Ashgate Publishing.
39
Embed
A psychological fidelity approach to simulation-based training - iacmr
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A Psychological Fidelity Approach to Simulation-Based Training: Theory, Research, and Principles Steve W. J. Kozlowski and Richard P. DeShon Michigan State University
Draft 1: 20 July 2000 Final Submission: 20 September 2002
Kozlowski, S. W. J. & DeShon, R. P. (2004). A psychological fidelity approach to simulation-based training: Theory, research, and principles. In E. Salas, L. R. Elliott, S. G. Schflett, & M. D. Coovert (Eds.), Scaled Worlds: Development, validation, and applications (pp. 75-99). Burlington, VT: Ashgate Publishing.
Psychological Fidelity 2 Abstract Transfer in terms of skill maintenance and generalization is one of the key challenges in effective training design. Training design for critical military tasks often entails the use of simulations that are high on physical fidelity to minimize skill decrements that impede training transfer. Although this is an effective approach to resolving the transfer problem, it can be costly and inefficient. Moreover, a new wave of advanced training technologies based on a distributed network architecture is emerging. These emerging training technologies will rely on low fidelity synthetic tasks to address the transfer problem. This chapter presents a theoretically based strategy for training research and design that focuses on psychological fidelity--an explicit effort to model underlying psychological constructs and processes responsible for effective performance. Theoretically driven psychological fidelity is essential for enhancing the training to transfer linkage. We argue that a focus on psychological fidelity can leverage the training potential of cost-efficient low fidelity simulations used in distributed training systems, and can enhance the effectiveness of high fidelity simulations as well. The chapter presents the principles of this approach, illustrates their research application, and describes the advantages of explicitly considering psychological fidelity in training research and design.
Psychological Fidelity 3
A Psychological Fidelity Approach to Simulation-Based Training: Theory, Research, and Principles
Steve W. J. Kozlowski and Richard P. DeShon Michigan State University
Many critical activities, such as air traffic control, industrial process control, and military
command and control, are accomplished by individuals and teams interacting through complex,
technology mediated systems. These task environments, which can be characterized as dynamic
decision making (DDM) situations, place high demands on the skills and capabilities of operators.
DDM tasks are dynamic, ambiguous, and emergent, necessitating rapid assessment of the situation as it
unfolds, diagnosis and prioritization of possible actions, and implementation of appropriate task
strategies. DDM tasks place heavy demands on decision makers, necessitating high levels of expertise
to enable the strategic action and adaptive performance required for team effectiveness.
How can training be used to develop the knowledge and skills essential to strategic and adaptive
performance? From a training design perspective, there are two critical and related issues that need to
be addressed: skill acquisition and transfer (Goldstein, 1993). Skill acquisition concerns learning the
knowledge and skills necessary for effective performance. Transfer concerns the transportability of
trained knowledge and skills from the training context to the performance environment, and focuses on
issues of retention, maintenance, and generalization (Baldwin & Ford, 1988). Although both issues are
important, transfer is the more challenging issue (Barnett & Ceci, 2002). Moreover, transfer is directly
relevant to adaptabilityBan essential aspect of team effectiveness in DDM environments.
There are two general approaches for resolving the transfer problem through training design.
One approach addresses physical fidelity, whereas the other approach addresses psychological fidelity.
The physical fidelity approach focuses on the use of high fidelity simulation during the skill acquisition
phase in an effort to minimize or eliminate skill degradation during transfer. High fidelity simulation is
an effort to design a training context that physically reproduces the actual performance environment to
Psychological Fidelity 4 the greatest possible extent. The equipment to be used; its controls, reactions, and behavior; its look,
feel, and motion are made to be as realistic as possible. Training or practice scenarios are often based
on actual events to further enhance the realism of the training experience. The essence of this training
strategy is that the emphasis on realism will minimize differences between the training and performance
contexts, thus enhancing the potential for knowledge and skill transfer. It is a theoretically based (i.e.,
identical elements theory: Thorndike & Woodworth, 1901), tested, and effective approach (Druckman
& Bjork, 1991) to the transfer issue. It is the dominant approach to training design for critical military
tasks such as aviation crews and command and control teams. However, it is also a training strategy
that can be costly, time consuming, and inefficient.
The other approach -- one that we incorporate in our research -- focuses on psychological
fidelity in training design. Psychological fidelity concerns the extent to which the training environment
prompts the essential underlying psychological processes relevant to key performance characteristics in
the real-world setting. In other words, it is an effort to evoke the central psychological constructs and
mechanisms responsible for on-the-job performance. Whereas the physical fidelity approach attempts to
accomplish this implicitly by replicating the performance environment, the psychological fidelity
approach represents an effort to model this explicitly by using basic theory to guide research and
training design. By doing so, it has the potential to enable the use of cost-effective low fidelity
simulations during training that can nonetheless maximize transfer in terms of retention and, more
importantly, generalization.
It is important to recognize that these two approaches are not competing alternatives; rather,
they are complementary. It is simply that the physical fidelity approach dominates training design for
transfer, and the psychological fidelity approach is an emerging perspective. One can consider a
training system to be a series of episodes or experiences that systematically build key skills from basic
to strategic to more complex adaptive skills (Kozlowski, 1998). Low fidelity simulation has an
Psychological Fidelity 5 important role to play in such a system. Moreover, a new wave of distributed training technologies is
emerging that will make extensive use of low fidelity synthetic tasks. These technologies will be based
on networked architectures linking distributed PC platforms that can run complex interactive
simulations. Individuals, teams, and teams of teams will linked together to engage in common training
experiences. The cost advantage and training potential of these emerging training systems is
inescapable. However, the technologies are merely media for delivering information and experience;
they are not instruction per se. To be effective training systems, they will have to meet the challenges
of skill acquisition and transfer. Thus, a critical question concerns how these challenges can be met
through the use of low fidelity simulations. We assert that psychological fidelity is an essential feature
of training design regardless of the level of physical fidelity of the simulation. Indeed, when coupled
with the physical fidelity approach, the psychological fidelity approach can improve the cost-benefit and
overall effectiveness of the training system.
Our purpose in this chapter is to explicate the principles of psychological fidelity and to
illustrate how we utilize these principles to guide our basic research on training design for skill
acquisition and generalization. The hallmark of our approach to enhancing psychological fidelity is the
centrality of theory; it is essential at each phase of the research process. We begin with a brief
discussion of adaptability and provide an overview of the theoretical heuristics that guide our research
program on the development of adaptive performance skills. We next define and describe the essential
elements of psychological fidelity, and the research strategies that translate the elements into
experimental designs. We provide an illustration of this approach through work in our research program
on training and developing adaptive performance. And, finally, we close with a brief discussion
regarding the advantages of our approach as a research strategy, and as a method for developing tools
for simulation-based training.
Psychological Fidelity 6
Problem Background and Theoretical Overview
What is Adaptability, Why is it Important, and How is it Developed?
Dynamic problem situations create challenges for decision makers and place a premium on the
capability to adapt individual and team performance to the shifting demands of the emerging problem
situation (Orasanu & Connolly, 1993). The problem is ill structured, with incompatible or shifting
goals. Diagnostic information is difficult to obtain, and is often ambiguous or conflicting when it is
available. The situation is dynamic and emergent, responsive to decision maker actions, but also subject
to unpredictable shifts. Individual decision makers are embedded in teams, and must coordinate their
individual efforts with multiple players. Often there are significant time pressures and high stress.
Thus, DDM situations call for more than the static and routine application of well-learned
knowledge. Such situations necessitate what Holyoak (1991) describes as Aadaptive expertise,@ and what
we refer to as adaptability or adaptive performance. Adaptive performance builds on a foundation of
basic domain knowledge and the routine expertise that guides performance in typical situations.
However, adaptive performance goes beyond procedural knowledge of an automatic sort. It requires
active cognitive monitoring to develop a deep comprehension of the conceptual structure of the problem
domain. Adaptive experts understand when and why particular procedures are appropriate, and also
when they are not. Comprehension entails mindful processing, allowing adaptive experts to recognize
shifts in the situation that necessitate adaptability (Smith, Ford, & Kozlowski, 1997).
A key factor for the development of adaptive performance skills is active learning during skill
acquisition. Active learning enhances the development of metacognitive and self-regulatory skills.
Metacognition refers to executive-level processes entailing knowledge, awareness, and control of
cognitive activity involved in goal attainment (Flavell, 1979). Self-regulation occurs at a more micro-
level, and entails the planning, monitoring, and adjustment of cognitive and task strategies necessary to
accomplish sub-goals. In addition to cognitive and task-relevant strategies, self-regulatory skills entail
Psychological Fidelity 7 the capability to manage affect. Complex tasks require focused attention and cognitive effort. Tasks that
are difficult mean many errors and frustrations early in the learning process. The negative affect that
accompanies failure to meet expectations draws attention away from the task and must be managed.
Effective management of the learning process enhances self-efficacy, a sense of self-perceived task
competency that allows the individual to tackle difficult tasks and persist in the face of novel challenges.
These capabilities are also important for maintaining motivation under challenging and shifting
The essential element of this example is the way in which different training manipulations are
combined together to create a more complex strategy. Here, the constructs to be combined have to have
Psychological Fidelity 27 some common underpinnings to guide their combination. In the psychological fidelity approach, that
common underpinning is basic theory; it guides what should be combined and how it should be
combined. The result is data to support an effective training strategy; a strategy that is theoretically
based and applications relevant; a strategy that by design has the potential to generalize to alternative
simulation platforms -- at the same or higher fidelity -- to the extent that they entail the same cognitive
and performance requirements and the same underlying psychological processes.
Discussion and Conclusion
We noted at the onset of this chapter that issues of transfer effectiveness pose a critical
challenge for training researchers and designers. Moreover, training design for complex DDM tasks
must not only be sensitive to transfer as skill reproduction, but more importantly it must be concerned
with training design that enhances transfer as knowledge and skill generalization; that is, as
adaptability. This focus on enhancing the adaptive capabilities of individuals and teams through training
design is at the center of our research. The principles of psychological fidelity form the core of our
strategy for accomplishing this goal.
The dominant approach to meeting the challenges posed by the problem of training transfer has
been to focus on high physical fidelity simulations. High fidelity simulations are designed to emulate the
complexity of real-world tasks, without entailing the costly consequences of failure that are often
associated with such systems. It is an elegant strategy because it seeks to minimize the transfer problem
by closing the gap between training and the real-world task. Training and task performance merge. It is
an effective strategy that has served the training community wellBand it will continue to do so.
Nevertheless, in spite of its many strengths, high physical fidelity simulation possesses several
limitations as a training design strategy. It requires significant investment in dedicated facilities and
hardware. It must be staffed by expert trainers and coaches. There must be support facilities for
trainees. There must be coverage of trainee, travel, maintenance, and lost-work costs. Moreover, the
Psychological Fidelity 28 considerable investment required to establish and maintain such facilities almost always ensures that
training demands will exceed the capacity of the training system. Such facilities, then, often become
major bottlenecks for training. Thus, cost-effectiveness and efficiency become troublesome factors for
training systems constructed around high fidelity simulations in a time of increasingly tight training
budgets and correspondingly greater demands for high-level skills.
These challenges and constraints are driving the push for new, more cost-effective, and more
efficient training technologies. Increased computer power, declining costs, and enhanced connectivity
are making possible a host of new and advanced training technologies that allow training to be
distributed. The potential of these technologies is creating a move to push training out of centralized
facilities to far-flung distributed systems (Kozlowski & Bell, 2002). By distributing training across
computer networks, high facilities and maintenance costs are eliminated. Distributing training, by
necessity, also entails a push away from high-cost high physical fidelity simulation systems and a shift
toward lower cost low fidelity systems that run on common PC platforms.
Distributed training is a general label used to describe training systems in which trainees are
geographically separated from an instructor and/or other trainees, and can assume two primary forms.
One form of distributed training uses advanced video-conferencing and communication technologies to
enable an instructor to hold class for trainees in geographically remote locations. A second form of
distributed training--and the form relevant to our training research--focuses on interactive multi-media
applications in which information (e.g., web-based training) and simulation-based practice can be
provided over the Internet or internal intranets. This form of distributed training has the potential to
enable a new architecture for training design for complex military DDM tasks (e.g., command and
control). Multiple, geographically dispersed trainees (as individuals, teams, and teams of teams) will be
able to engage in sophisticated simulation-based practice to hone their basic skills and to develop
higher-level strategic and adaptive skills.
Psychological Fidelity 29
The cost logic of this shift to a distributed training design strategy is compelling, but will it be
effective? Will it meet the challenges posed by the transfer problem? In our view, the critical challenge
to realizing the promise of distributed training is the need for effective instructional principles and
strategies--grounded in psychological theory and research--to guide the design and application of system
features and capabilities. The technology is merely a medium for delivering information or experience;
it is not instruction per se. The psychological fidelity approach to training research and design is
predicated on meeting this challenge; on realizing the potential of this new generation of training tools;
on developing theoretically-based and applications-relevant instructional principles that enhance skill
acquisition and adaptability for individuals and teams.
Psychological Fidelity 30 References
Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Students' learning
strategies and motivational processes. Journal of Educational Psychology, 80, 260-267.
Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, 63-105.
Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes, 50, 248-287.
Bandura, A., & Wood, R. (1989). Effect of perceived controllability and performance standards on self-regulation of complex decision-making. Journal of Personality and Social Psychology, 56, 805-814.
Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn? A
taxonomy for far transfer. Psychological Bulletin, 128, 612-637. Bell, B. S., & Kozlowski, S. W. J. (2002). Adaptive guidance: Enhancing self-regulation,
knowledge and performance in technology-based training. Personnel Psychology, 55, 267-306. Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). Goal orientation in organizational
research: A conceptual and empirical foundation. Organizational Behavior and Human Decision Processes, 67, 26-48.
Cannon-Bowers, J. A., & Salas, E. (1997). A framework for developing team performance measures in training. In M. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement (pp. 45-62). Mahwah, NJ: LEA.
Carver, C. S. & Sheier, M. F. (1990). Principles of self-regulation: Action and emotion. In E. T.
Higgins and R. M. Sorrentino (Eds.), Handbook of motivation and cognition (Vol. 2, pp.3-52). New York: Guilford.
Coovert, M. D., & Thompson, L. F. (2001). Computer supported cooperative work: Issues and
implications for workers, organizations, and human resource management. Thousand Oaks, CA: Sage. DeShon, R. P., Kozlowski, S. W. J., Schmidt, A. M., Wiechmann, D., & Milner, K. R. (2001,
April). Developing team adaptability: Shifting regulatory focus across levels. In S. W. J. Kozlowski & R. P. DeShon (Chairs), Enhancing team performance: Emerging theory, instructional strategies, and evidence. Symposium presented at the 16th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
DeShon, R. P., Kozlowski, S. W. J., Wiechmann, D., Milner, K. R., Davis, C. A., &
Schmidt, A. M. (April, 2000). Training and developing adaptive performance in teams and individuals. In S. W. J. Kozlowski (Chair), Developing complex adaptive skills: Individual- and team-level training strategies. Symposium to be presented at the 15th Annual Conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
Psychological Fidelity 31
DeShon, R. P., Milner, K. R., Kozlowski, S. W. J., Toney, R. J., Schmidt, A., Wiechmann, D., & Davis, C. (1999, April). The effects of team goal orientation on individual and team performance. In D. Steele-Johnson (Chair), New Directions in goal orientation research: Extending the construct, the nomological net, and analytic methods. Symposium conducted at the 14th Annual Conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
Druckman, D., & Bjork, R.A. (1991). In the mind=s eye: Enhancing human performance. Washington, DC: National Academy Press.
Dwyer, D. J., Oser, R. L., Fowlkes, J. E., & Lane, N. E. (1997). Team performance measurement in distributed environments: The TARGETS methodology. In M. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement (pp. 137-154). Mahwah, NJ: LEA.
Flavell, J.H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906-911.
Frese, M., & Altmann, A. (1989). The treatment of errors in learning and training. In L. Bainbridge & S. A. R. Quintanilla (Eds.), Developing skills with information technology (pp. 658-682). New York: Wiley.
Gagne, R. M. (1970). Conditions of learning. New York: Holt, Rinehart, & Winston.
Goldstein, I. L. (1993). Training in organizations: Needs assessment, development, and evaluation. Pacific Grove, CA: Brooks/Cole.
Holyoak, K.J. (1991). Symbolic connectionism: Toward third-generation theories of expertise. In K.A. Ericsson & J. Smith, Toward a general theory of expertise (pp. 301-336). Cambridge, UK: Cambridge University Press.
Johnston, J. H., Smith-Jentch, K. A., & Cannon-Bowers, J. A. (1997). Performance measurement tools for enhancing team decision-making training. In M. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement (pp. 311-327). Mahwah, NJ: LEA.
Karoly, P. (1993). Mechanisms of self-regulation: A systems view. Annual Review of Psychology, 44, 23-52.
Kluger, A.N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284.
Kozlowski, S. W. J. (1998). Training and developing adaptive teams: Theory, principles, and research. In J. A. Cannon-Bowers & E. Salas (Eds.), Decision making under stress: Implications for training and simulation (pp. 115-153). Washington, DC: APA Books.
Psychological Fidelity 32
Kozlowski, S. W. J., & DeShon, R. P. (1999, June). TEAMSim: Examining the development of basic, strategic, and adaptive performance. Presented at the International Synthetic Task Development Conference/Scaled Worlds: Current Issues in Simulation-Based Human Performance Research, University of Georgia, Athens, GA.
Kozlowski, S. W. J., Gully, S. M., McHugh, P. P., Salas, E., & Cannon-Bowers, J. A. (1996). A dynamic theory of leadership and team effectiveness: Developmental and task contingent leader roles. In G. R. Ferris (Ed.), Research in personnel and human resource management (Vol. 14, pp. 253-305). Greenwich, CT: JAI Press.
Kozlowski, S. W. J., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of work performance: Implications for staffing, personnel actions, and development (pp. 240-292). San Francisco: Jossey-Bass.
Kozlowski, S. W. J., Gully, S. M., Salas, E., & Cannon-Bowers, J. A. (1996). Team leadership and development: Theory, principles, and guidelines for training leaders and teams. In M. Beyerlein, D. Johnson, & S. Beyerlein (Eds.), Advances in interdisciplinary studies of work teams: Team leadership (Vol. 3, pp. 251-289). Greenwich, CT: JAI Press.
Kozlowski, S. W. J., Gully, S. M., Brown, K. G., Salas, E., Smith, E. A., & Nason, E. R. (2001). Effects of training goals and goal orientation traits on multi-dimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes, 85, 1-31.
Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in
organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco, CA: Jossey-Bass.
Kozlowski, S. W. J., & Salas, E. (1997). An organizational systems approach for the implementation and transfer of training. In J. K. Ford, S. W. J. Kozlowski, K. Kraiger, E. Salas, & M. Teachout (Eds.), Improving training effectiveness in work organizations (pp. 247-287). Mahwah, NJ: Lawrence Erlbaum Associates.
Kozlowski, S. W .J., Toney, R. J., Mullins, M. E., Weissbein, D. A., Brown, K. G., & Bell, B. S. (2001). Developing adaptability: A theory for the design of integrated-embedded training systems. In E. Salas (Ed.), Advances in human performance and cognitive engineering research (pp. 59-123). Amsterdam: JAI/Elsevier Science.
Kraiger, K., Ford, J.K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311-328.
MacMillan, J., Serfaty, D., Young, P., Klinger, D., Thordson, M., Cohen, M., & Freeman, J. (1998). A system to enhance team decision-making performance. Woburn, MA: Aptima, Inc.
Psychological Fidelity 33
Mullins, M. E., Kozlowski, S. W. J., Toney, R. J., Brown, K. G., Weissbein, D. A., & Bell, B. S. (1999, April). Adaptive performance: Mastery versus performance goals and feedback consistency. Paper presented at the 14th Annual Conference of the Society for Industrial and Organizational Psychology, Atlanta, GA.
Nicholls, J.G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, 328-346.
Orasanu, J., & Connolly, T. (1993). The reinvention of decision making. In G. Klein, J. Orasanu, R. Calderwood, & C. Zsambok (Eds.), Decision making in action: Models and methods (pp. 3-20). Norwood, NJ: Ablex Publishing.
Ree, M. J., & Earles, J. A. (1991). Predicting training success: Not much more than g. Personnel Psychology, 44, 321-332.
Smith, E. M., Ford, J. K., & Kozlowski, S. W. J. (1997). Building adaptive expertise: Implications for training design. In M. A. Quinones & A. Dudda (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp.89-118). Washington, DC: APA Books.
Steinberg, E. R. (1989). Cognition and learner control: A literature review, 1977-1988. Journal of Computer-Based Instruction, 16, 117-121.
Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes, 37, 60-82.
Psychological Fidelity 34 Table 1 Basic Elements of the Psychological Fidelity Approach Basic Elements
Grounds the task domain in its underlying # cognitive requirements # performance specifications
Simulation Selection or Construction
Links: # basic theory # CTA # scaled world
Scenario Design and Measurement
# Translate critical CTA requirements to specific scenario features # Operationalize measures to capture critical psychological constructs, processes, and outcomes
Psychological Fidelity 35 Table 2 Research Design Strategies Basic Elements
Function
Theory of Instructional Design
Guides the design of training manipulations to leverage key constructs/mechanisms specified by basic psychological theory. Two research strategies: # Research on Apure@ manipulations (isolated
constructs) designed to assess the impact on relevant constructs and processes. Appropriate for basic research to evaluate basic theory or to assess new manipulation tools.
# Research on complex training strategies (a
combination of several Apure@ constructs) designed to optimize the instructional impacts on underlying processes. Appropriate for research to assess training and/or transfer effectiveness.
Theory of Training Effectiveness and Evaluation
Guides research design features: # Assessment of multidimensional training
outcomes to capture cognitive, affective, and behavioral domains.
# Specifies developmental assessments at
multiple time points during skill acquisition. # Specifies mulitlevel assessment -- individual
and team -- in team training contexts. # Specifies assessment of skill generalization
and adaptability.
Psychological Fidelity 36 Table 3 Illustration of the Psychological Fidelity Approach
Basic Elements
Function
Basic Psychological Theory: # Self-Regulation and Action Initiation
Source of: # psychological constructs
individual differences in cognitive ability and learning and performance goal orientation, variation in goal types, goal levels, goal commitment, performance monitoring, feedback interpretation/attribution, self-efficacy, self-satisfaction, knowledge, performance, adaptive performance
effort devoted to study and practice, focus of study and practice (content, sequence), cooperation and coordination, basic, strategic, and adpative performance
Simulation Selection or Construction # Simulation Features: - scripted, event-based scenarios - dynamic, emergent situations - variable information processing demands - variable cue ambiguity and conflict - variable target priorities - shifting task strategies and trade-offs - variable difficulty, dynamics, and complexity - data to assess situational awareness, priorities, and strategies - data to assess mutual performance monitoring and coordination - data to assess basic, strategic, and adaptive performance
Links: # basic theory executive level knowledge, awareness, and control of cognitive activity in goal attainment; self-regulation processes in dynamic planning, monitoring, and adjustment of cognitive and task strategies; managing of affect/emotional control; persistence in the face of uncertainty, change, and difficulty; monitoring of teammates, work-load sharing; adaptation of individual and team performance # CTA requirements dynamic, emergent, ambiguous, information processing demands, shifting goal priorities/task strategies, requirements for team coordination and adaptive performance # scaled world requirements dynamic, emergent, ambiguous, information processing demands, shifting goal priorities/task strategies, requirements for team coordination and adaptive performance
Scenario Design and Measurement
Activities: # Translate critical CTA requirements to specific scenario features: dynamic, unfolding events; ambiguous cues for decision making; conflicting goals necessitate prioritization and trade-off strategies for individuals; shifting and unpredicatable overloads necessitate team cooperation and coordination of effort; dramatic increases in task difficulty and complexity require individual and team adaptation # Operationalize measures to capture critical psychological constructs, processes, and outcomes: cognitive ability, learning goal orientation, performance goal orientation, goal type, goal levels, goal commitment, feedback monitoring, task strategies, performance attribution, performance-goal discrepancies, self-efficacy, knowledge, performance, adaptive performance
Psychological Fidelity 38 Table 4 Illustration of Research Design Strategies
Basic Elements
Function
Theory of Instructional Design: # Integrated-Embedded Training Design - Self-regulation Theory # Team Compilation Theory - Self-regulation Theory
Guides the design of training manipulations to leverage key constructs/mechanisms specified by basic psychological theory. Two research strategies: # Research on Apure@ manipulations (isolated
constructs) designed to assess the impact on relevant self-regulation constructs and processes. Appropriate for basic research to evaluate basic theory or to assess new manipulation tools.
- mastery vs. performance goal frames; - feedback (individual, team, both) - team-level development vs. shifted individual to team-level development # Research on complex training strategies (a
combination of several Apure@ constructs) designed to optimize the instructional impacts on underlying processes. Appropriate for research to assess training and/or transfer effectiveness.
- sequenced mastery goals/feedback vs. sequenced performance goals/feedback
Theory of Training Effectiveness and Evaluation # Kraiger et al. (1993) # Kozlowski (1998) # Kozlowski & Salas (1997) # Kozlowski et al. (2001)
Guides research design features: # Assessment of multidimensional training outcomes to capture cognitive, affective, and behavioral domains. # Specifies developmental assessments at multiple time points during skill acquisition. # Specifies mulitlevel assessment -- individual and team -- in team training contexts. # Specifies assessment of skill generalization and adaptability.
Psychological Fidelity 39
Figure 1. Schematic Research Model
Research Factors:• pure constructs• training strategies