7/29/2019 Udlaip Pellegrino
1/14
How People Learn:ow People Learn:Incorporating Theory and Research intoncorporating Theory and Research intoInstructional Practicenstructional PracticeJames W. PellegrinoJames W. Pellegrino
Distinguished Professor in Psychology and Education
Center for the Study of Learning, Instruction and Teacher Development
Department of Psychology
University of Illinois at ChicagoUniversity of Illinois at Chicago
Overview of TodayOverview of Today!!s Presentations Presentation
!How People Learn- What we know, its
implications, & illustrative applications toinstructional design in bioengineering
!Knowing What Students Know- Understandinthe nature of assessment, especially its integralrole in the teaching and learning process
! Linking the Scholarship of Discoverywith theScholarship of Teachingin domains of
engineering education
Overview of TodayOverview of Today!!s Presentations Presentation
!How People Learn- What we know, its
implications, & illustrative applications toinstructional design in bioengineering
!Knowing What Students Know- Understanding
the nature of assessment, especially its integralrole in the teaching and learning process
! Linking the Scholarship of Discoverywith theScholarship of Teachingin domains of
engineering education
Advances in Sciences ofAdvances in Sciences of
Thinking & LearningThinking & Learning
!The most important cognitive principles
about thinking and learning are derivedfrom study of the nature of competenceand the development of expertise inspecific curriculum domains."Characteristics of expertise*
"Knowledge organization*
"Metacognition
"Multiple paths to competence
"Preconceptions and mental models*
"Situated knowledge and expertise*
Advances in Sciences ofAdvances in Sciences of
Thinking & LearningThinking & Learning
!The most important cognitive principles
about thinking and learning are derivedfrom study of the nature of competenceand the development of expertise inspecific curriculum domains.
""Characteristics of expertiseCharacteristics of expertise*
"Knowledge organization*
"Metacognition
"Multiple paths to competence
"Preconceptions and mental models*
"Situated knowledge and expertise*
7/29/2019 Udlaip Pellegrino
2/14
Characteristics of ExpertiseCharacteristics of Expertise
!Experts have well-organized knowledge:" their knowledge is organized to support understanding
and it is conditionalized for use.
" they have fluent access to their knowledge andrecognize patterns and chunks.
" they have domain-specific problem solving strategies
"
expertise is acquired over time and depends onmultiple, contextualized experiences.
!Questions --"What are examples and features of expertise and its
consequences in the engineering domain?
"What assumptions can be made about the necessaryconditions & time course of acquiring expertise?
What Do You NoticeWhat Do You Notice
& Understand?& Understand?
What Did You NoticeWhat Did You Notice
& Understand?& Understand?
What Would You ReportWhat Would You Report
About What You Saw?About What You Saw?
A NoviceA Novice
An Expert in Bio-ImagingAn Expert in Bio-Imaging
Innovation
Efficiency
AdaptiveExpert
Routine
Expert
OverwhelmedNovice
Novice
Optimal Adaptability Corridor
OAC
Adaptive Experts are both efficientAdaptive Experts are both efficient
and innovativeand innovative
7/29/2019 Udlaip Pellegrino
3/14
Advances in Sciences ofAdvances in Sciences of
Thinking & LearningThinking & Learning
!The most important cognitive principlesabout thinking and learning are derivedfrom study of the nature of competenceand the development of expertise inspecific curriculum domains."Characteristics of expertise*
""Knowledge organization*Knowledge organization*
"Metacognition
"Multiple paths to competence
"Preconceptions and mental models*
"Situated knowledge and expertise*
Knowledge OrganizationKnowledge Organization
!Effective knowledge organization in areassuch as bioengineering means thatpersons:" have a deep foundation of factual and procedural knowledge,
" understand facts, ideas and procedures in the context of aconceptual framework,
" organize knowledge into schemas that facilitate retrieval andapplication
!Questions --" What defines the key conceptual, procedural
knowledge & schemas for areas of engineering?
" Are existing ABET documents and sources anadequate starting place?
" What must be done to capture core competencies?
Novice Map: Biomedical engineeringNovice Map: Biomedical engineering
!
Expert Map -- Biomedical engineeringExpert Map -- Biomedical engineering
Learning With UnderstandingLearning With Understanding
!Problem of transfer -- Judd!s 1908 dartthrowing experiment with children
!Learning facts versus understanding theircontext and significance"Geography -- memorizing names and places vs
understanding how natural features define borders &importance of water -- transfer across continents
" History -- facts and dates of civil war versusunderstanding conditions of life and true causes
" Biology -- properties of arteries and veins versus howthose properties relate to the functions they serve
" Engineering and Technology -- specific design artifactsvs understanding systems, constraints, & processes
Advances in Sciences ofAdvances in Sciences of
Thinking & LearningThinking & Learning
!The most important cognitive principles
about thinking and learning are derivedfrom study of the nature of competenceand the development of expertise inspecific curriculum domains."Characteristics of expertise*
"Knowledge organization*
""MetacognitionMetacognition
"Multiple paths to competence
"Preconceptions and mental models*
"Situated knowledge and expertise*
7/29/2019 Udlaip Pellegrino
4/14
MetacognitionMetacognition
!Competent performers consciously keeptrack of their own thinking and adjust theirunderstanding while they learn or solve aproblema process called metacognition." self-aware learners can explain which strategies they
used and why
" less competent students monitor their thinkingsporadically and ineffectively.
!Questions --" How does metacognition develop for specific
engineering content areas?
"What does this monitoring look like?
"What is specific to areas of engineering?
Advances in Sciences ofAdvances in Sciences of
Thinking & LearningThinking & Learning
!The most important cognitive principlesabout thinking and learning are derivedfrom study of the nature of competenceand the development of expertise inspecific curriculum domains."Characteristics of expertise*
"Knowledge organization*
"Metacognition
""Multiple paths to competenceMultiple paths to competence
"Preconceptions and mental models*
"Situated knowledge and expertise*
Multiple Paths to CompetenceMultiple Paths to Competence
!Not all persons learn in the same way orfollow the same paths to competence." problem solving strategies become more effective over
time and with practice
" the growth process is not a simple, uniformprogression, nor is there movement directly fromerroneous to optimal solution strategies.
!Questions --"What does this look like for specific areas of
engineering?
"What specific patterns exist in the growth ofunderstanding and competence?
Advances in Sciences ofAdvances in Sciences ofThinking & LearningThinking & Learning
!The most important cognitive principlesabout thinking and learning are derivedfrom study of the nature of competenceand the development of expertise inspecific curriculum domains."Characteristics of expertise*
"Knowledge organization*
"Metacognition
"Multiple paths to competence
""Preconceptions and mental models*Preconceptions and mental models*
"Situated knowledge and expertise*
Preconceptions & Mental ModelsPreconceptions & Mental Models
!Students come to the classroom withknowledge representations containing pre-conceptions about how the world works." If their initial understanding in a domain is not engaged
they may fail to grasp new concepts & information thatare the focus for learning.
!Questions --"What are the preconceptions and mental models that
apply to the domain of engineering?
# e.g. -- Constraints? Systems? Design? Invention?
"Which are serious concerns for future learning?
" How can these be identified and externalized?
LionniLionni!!ssFish is FishFish is Fish
7/29/2019 Udlaip Pellegrino
5/14
The FishThe Fish!!s Image of Birdss Image of Birds The FishThe Fish!!s Image of Cowss Image of Cows
The FishThe Fish!!s Image of Peoples Image of People Some Analogs to Fish is FishSome Analogs to Fish is Fish
!Young children who believe the earth is flat.
!Physics students who assume force of thehand when a ball is thrown into the air
!Electrical engineering students who use waterpipe analogies for electricity
!Student beliefs that history is about the goodguys vs the bad guys
!Students at multiple ages beliefs about seasons-- distance from sun not tilt
!Professors who believe that learning is aprocess of absorbing knowledge transmitted bythe instructor
Advances in Sciences ofAdvances in Sciences of
Thinking & LearningThinking & Learning
!The most important cognitive principles
about thinking and learning are derivedfrom study of the nature of competenceand the development of expertise inspecific curriculum domains."Characteristics of expertise*
"Knowledge organization*
"Metacognition
"Multiple paths to competence
"Preconceptions and mental models*
""Situated knowledge and expertise*Situated knowledge and expertise*
Situated Knowledge & Expertise (1)Situated Knowledge & Expertise (1)
!Knowledge frequently develops in a highlycontextualized and inflexible form, and
often does not transfer very effectively." Transfer depends on the development of an explicit
understanding of when and how to apply what hasbeen learned.
!Questions --" What constitutes evidence of transfer in areas of
engineering?
" How context bound is knowledge of engineering andhow much does current practice constrain transfer?
" To what extent is training for transfer part of theteaching & learning process?
7/29/2019 Udlaip Pellegrino
6/14
Situated Knowledge & Expertise (2)Situated Knowledge & Expertise (2)
!There are important relationships amonglearners and the contexts in which they
learn which define major parts of knowingand expertise." Expert performers, through interactions with peers,
build communities of practice and understanding whichare distributed and build on the learning of others.
!Questions --"What are the communal and participatory practices that
constitute part of the domains of engineering?
" How is community established and supported in areasof engineering?
Applying the Principles of HPL:Applying the Principles of HPL:
A Design Framework forA Design Framework for
Effective Learning EnvironmentsEffective Learning Environments
!Goal -- to significantly impact the nature andquality of bioengineering and biomedicalengineering education
!Approach -- combine expertise in multiplecontent domains with expertise from the learningsciences and use the power of informationtechnologies to redesign learning environments
Examples fromExamples fromVaNTHVaNTHofof
Applying HPL conceptsApplying HPL concepts
!Domain analysis" Develop taxonomies of core knowledge and competencies -- e.g.,
bio-optics, systems physiology, bio-imaging, bio-mechanics, bio-transport, bio-tech
!HPL-based modular course design" Apply HPL principles to the design of instructional materials and
modules for core concepts
" Conduct design experiments
!Conduct research on instructional processes &student learning outcomes" Evidence of changes in instructional practices
" Student evaluations of the features of the learning environments
" Levels of student engagement
!But what about the assessment of studentlearning?
Partial List of Concepts and Skills foPartial List of Concepts and Skills fo
Introductory BiomechanicsIntroductory Biomechanics! FUNDAMENTALS
" units
" vector analysis, vector components, vector products
" forces
" moments, couples
" equivalent force systems, resultant force systems
" composite body center of gravity, center of mass
" centroid, distributed body centroid, distributed load
! MUSCULOSKELETAL SYSTEM AND STATICS" levers, musculoskeletal machines
" bone, joints, tendon, ligament, muscle
" equilibrium, free body diagrams
" types of support
" 2 force and 3 force system statics, method of sections
" equilibrium analysis - upper extremities
" equilibrium analysis - lower extremities
! DYNAMICS" linear kinematics, linear kinetics
" angular kinematics, angular kinetics
" impulse-momentum methods
" area and mass moments of inertia, moment of impulse
7/29/2019 Udlaip Pellegrino
7/14
Bioreactor Design ChallengeBioreactor Design ChallengeDifferences in Teaching Methods in HPL and non-HPL
Biomechanics Lessons
010
20
30
40
50
ContentK
nowl
edge
ProceduralKn
owled
ge
Questio
n-An
swer
Professor-g
uidedP-S
Inde
pend
entP
-S
Grou
pP-S
Percentof
Possible
Instances non-HPL
HPL
Figure 2: Comparison of Student Course Evaluation Questions
1
2
3
4
5
6
Class
Participation
Lecture Group Work Student
Presentations
Real Life
Problems
Discussed Ideas
Frequency
HPL Non HPL
** = Significant at p
7/29/2019 Udlaip Pellegrino
8/14
Overview of TodayOverview of Today!!s Presentations Presentation
! How People Learn- What we know, its implications, &illustrative applications to instructional design inbioengineering
! Knowing What Students Know- Understanding the
nature of assessment, especially its integral role in theteaching and learning process
! Linking the Scholarship of Discoverywith theScholarship of Teachingin domains of engineeringeducation
AAssessment as a Process ofssessment as a Process ofReasoning from EvidenceReasoning from Evidence
! cognition" model of how students
represent knowledge & developcompetence in the domain
! observations" tasks or situations that allow
one to observe students!performance
! interpretation"
method for making sense of thedata
observation interpretation
cognition
Must becoordinated!
Why Cognitive Models of KnowledgeWhy Cognitive Models of Knowledge
in Content Domains are Criticalin Content Domains are Critical
! Tell us what are the important aspects ofknowledge that we should be assessing." Give deeper meaning and specificity to standards
!Give us strong clues as to how such knowledgecan be assessed" Suggest what can and should be assessed at points proximal or
distal to instruction
!Can lead to assessments that yield moreinstructionally useful information -- within andacross levels and contexts
!Can guide the development of systems ofassessments" Comprehensive, Coherent & Continuous
Rationalist &Rationalist &SocioculturalSociocultural
PerspectivesPerspectives
!Rationalist: Focused on the nature ofcompetence and the development of knowledge
in specific curriculum domains or topic areaslike bioengineering, electrical engineering etc.." Individual cognition -- the mind of the individual
!Sociocultural: Focused on the nature ofpractice and forms of participation in
communities of practice like bioengineering." Distributed cognition -- the collective mind
7/29/2019 Udlaip Pellegrino
9/14
Contrasting & ComplementaryContrasting & Complementary
EmphasesEmphases
Forms of
Mediated Activity
Mental
Representations
Understanding
Knowledge
Trajectories of
Participation
Trajectories of
Learning
UnderstandingDevelopment
Communal
Practices
Task
Analysis
UnderstandingPerformance
Sociocultural
Perspective
Rationalist
Perspective
Foci for
Explanation
Generalizations AboutGeneralizations About
PerformancePerformance
Expert" performance develops in communities that value certain forms o
knowledge and activity, like modeling in science or design inengineering.
" knowledge is tuned to specific patterns of activity, l ike solvingcertain kinds of problems or designing classes of objects and
tools." performance increases in scope and precision with multiple,
contextualized experiences.
" no magic levers: practice, disciplined inquiry.
! Implication -- Assessments must be designed tocapture the complexity of expertise andcompetent performance, ranging from mentalprocesses to participation in forms of practice
Generalizations aboutGeneralizations aboutDevelopmentDevelopment
!Not all persons learn in the same way or follow
the same paths to competence." Conceptual change is often not a simple, uniform progression, nor
is there movement directly from erroneous to optimal solutionstrategies or incorrect to correct representations.
" Intermediate forms of knowledge may not resemble expert forms,so simple building block relations may not hold.
" Participation often starts at the edges and becomesprogressively more aligned with core disciplinary practices.
! Implication - Assessments should identify specific
strategies, knowledge representations, and formsof activity with respect to the role they play in
developmental trajectories.
Generalizations about KnowledgeGeneralizations about Knowledge
!Disciplinary Knowledge" Is organized in ensembles that facilitate its use.
" Is amplified by processes of self regulation, or metacognition,where learners spontaneously evaluate their knowledge and itslimits.
" Is developed in communities that foster identity and interest.
! Implications for Assessment -- multiple
questions" Knowledge Issues - Specific Facts, Procedures, Schemas
" Reflection Issues - Articulation, Evaluation
" Practice Issues - Why prove? Model?
APPLICATIONS of KWSK CONCEPTS:
the ASSESSMENT SPACE in VaNTH
CHARACTERISTICS of the LEARNING ENVIRONMENT
META-COGNITIVE
SKILLS
DECLARATIVE
KNOWLEDGE
PROCEDURAL
KNOWLEDGE
PROBLEM
SOLVINGTRANSFER
ADAPTIVE
EXPERTISE
Key Knowledge Outcomes
PRECONCEPTIONS &
MISCONCEPTIONS
APTITUDE
MOTIVATION
ATTITUDES
FUNDS OF
KNOWLEDGE
Individual Differences
Examples of Assessment inExamples of Assessment inVaNTHVaNTH
!Learning environment characteristics"
HPL-ness of a module, course or program" Impact of course redesign on student engagement
!Formative assessment during learning" Use of personal response systems & diagnostic
assessment tasks during learning
" Importance of formative assessment
!Acquisition of student knowledge" Application of concept maps
" Expert-novice differences
7/29/2019 Udlaip Pellegrino
10/14
Assessment During InstructionAssessment During Instruction
! Large lecture courses hinder instructors! abilityto judge how well students grasp coursematerial." Difficult to establish a sense of classroom community where
students will ask questions when confused.
!One tool that removes these roadblocks tolearning is the Personal Response System(PRS)" Students use hand-held remotes to respond to questions
during class time; individual answers are fed into a computerprogram that calculates group statistics and immediately reportsthem to the professor in the form of a histogram.
" Responses can also be displayed on a screen that is visible tothe entire class.
A B C D
0
10
20
30
40
50
60
70
Q. What percentage of falls occur in
people older than 70 years?
A. 10%
B. 30%
C. 60%
D. 90%
*
Importance of Formative AssessmentImportance of Formative Assessment! As instruction is occurring, teachers need
information to evaluate whether their teachingstrategies are working.
! They also need information about the currentunderstanding of individual students and groupsof students so they can identify the mostappropriate next steps for instruction.
! Students need feedback to monitor their ownlearning success and to know how to improve.
! Black & Wiliam (1998) reviewed impact offormative assessment practices on learning
outcomes -- effect sizes ranging from .4 - .7! Similar results for higher ed courses -- Nyquist &
Cordray (2004)
Concept Mapping as aConcept Mapping as a
Form of StudentForm of Student
Assessment and InstructionAssessment and InstructionJoan M.T. Walker & David S. Cordray
Department of Psychology and Human Development
Paul H. King
Biomedical EngineeringVanderbilt University
Three Major QuestionsThree Major Questions
!Can concept maps differentiatenovices from experts?
!Can concept maps captureindividual student growth over time?
!Can a concept map be used as aninstructional framework thatenhances student learning?
Study 1: Novice-Expert ContrastStudy 1: Novice-Expert Contrast
Participants" Undergraduates (n = 8)
" Graduate students (n = 9)
" Faculty (n = 3)
Sophisticated tools" Pencils, paper and post-it notes
Focus question:What are the 10-20 most important concepts in BMEand how are they related?
7/29/2019 Udlaip Pellegrino
11/14
Novice MapNovice Map
!
Expert MapExpert Map
Density ScoresDensity Scores
Mean NodeGroup Nodes Lines Line/Node Range
Undergrad 20.0 24.5 1.22 14-24
Graduate 16.7 19.8 1.19 11-21
Faculty 16.0 25.7 1.61 11-24
Qualitative DifferencesQualitative Differences
Novices emphasize domain contenbiotechnology, physiology
Experts emphasize"Higher order principles
# synthesis of engineering and medicine
"Application of principles
# interdisciplinary communication
Study 2: Individual GrowthStudy 2: Individual Growth
Participants"4 seniors in year-long design course#worked in pairs at 2 time points in Fall 2001
#3rd map constructed in Spring 2002
Focus question:What is your current conceptualunderstanding of what is involved in theBME design process?
Pair One:Pair One: Time 1Time 1
7/29/2019 Udlaip Pellegrino
12/14
Pair One: Time 3Pair One: Time 3 Pair Two: Time 1Pair Two: Time 1
Pair 2: Time 3Pair 2: Time 3Validity RatingsValidity Ratings
0
20
40
60
invalid 3 0 0 0 2 1
partially valid 6 2 0 1 3 0moderately valid 5 11 12 12 5 9
valid 0 8 16 20 22 52
Pair 1 Pair 2 Pair 1 Pair 2 Pair 1 Pair 2
Time 1 Time 2 Time 3
Qualitative DifferencesQualitative Differences
!Greater differentiation"more precise vocabulary
!Fewer dead ends" increased integration
!More coherent structure" awareness of the big picture
!Reviewed all Time 1 maps" Complained about quality of own Time 1 map
" Articulated differences between pairs
!Provided tangible evidence of theirintellectual growth, something traditionalassessment had not offered
Student ReflectionsStudent Reflections
7/29/2019 Udlaip Pellegrino
13/14
Study 3: Development of ExpertiseStudy 3: Development of Expertise
! "Mined! map structures to establish a benchmark
of expertise in bioengineering design.
!Extracted the contents of bioengineering design
experts! maps, reduced them to a set of 27
unique concepts, and then sorted them intoconcept categories
! 6 key areas pertinent to pertinent to
bioengineering design:" the design process (e.g., prototyping, protocols),
" the technical and interpersonal skills necessary for design,
" overriding social concerns (e.g., ethics),
" genesis of the design (e.g., scientific need, client need),
" market opportunities and constraints
" Walker, King, Cordray & Fries, 2004
Figure 4: Expert and Novice Definitions of the Biodesign
Process
0
20
40
60
80
100
Desig
nproc
essEth
ics
Comm
unica
tion
Backg
round
Genes
isofd
esign
Marke
ting
Perce
ntofcontentcoverage Time 1
Time 3
Expert
*
*
** * *
Assessment Issues & OpportunitiesAssessment Issues & Opportunities
! Importance of an integrated assessment &evaluation effort in the engineering educationprocess" Critical to advancing student learning and the instructional process
" Responds to the need for internally & externally acceptable indices ofstudent achievement
" Demonstrates efficacy of the instructional process & program
! A persistent issue needing attention" HELP! -- Can anyone out there from the A&E group tell me what to do?
I need a test!!!
" Assessment design must be a collaborative enterprise
!Opportunity to advance knowledge
" If programs are planful & systematic they can address all their A&Eneeds ---> respond thoughtfully to ABET requirements
" Possible to show how the ideas in HPL & KWSKcan be made to workin actual instructional practice
Overview of TodayOverview of Today!!s Presentations Presentation
! How People Learn- What we know, its implications, &illustrative applications to instructional design inbioengineering
! Knowing What Students Know- Understanding thenature of assessment, especially its integral role in theteaching and learning process
! Linking the Scholarship of Discoverywith theScholarship of Teachingin domains of engineeringeducation
Linking Research & PracticeLinking Research & Practice
7/29/2019 Udlaip Pellegrino
14/14
Areas of Engineering EducationAreas of Engineering Education
Needing Further AttentionNeeding Further Attention
!Building the cumulative knowledge base forteaching, learning & assessment indomains of engineering" Defining core knowledge constructs
" Research on fundamental teaching & learning issues
" Research on instructional and assessment practices
!Applying HPL & KWSK principles to thesystematic analysis of:" Existing educational programs, materials & tools
" Assessments used for high stakes decisions
" Faculty & TA training models, methods & content
Working in PasteurWorking in Pasteur!!s Quadrants Quadrant
Practical Valueractical Valuelow high
low
high
TheoreticalValue
Pasteurasteur
Edisondison
Bohrohr