Brian Mavis, PhD Portfolio Use to Develop Teaching Skills and Meet Program Goals.

Brian Mavis, PhD

• Portfolio Use to Develop Teaching Skills and Meet Program Goals

• What is a portfolio?

• How can it be used to document accomplishments related to teaching?

• What resources are available to guide faculty and RPT committees?

Objectives

• Living document

• Extension of your CV

• Personal annual report (annual review)

• Necessary part of many promotion and tenure packages

What is a portfolio?

• Originally conceptualized like those used with artists or architects

• Demonstrates quality of your work

• Records breadth of your work

• Illustrates professional development

What is a portfolio?

“…a method of encouraging adult and reflective learning … based on developing a collection of evidence that learning has taken place”

What is a portfolio?

Snadden and Thomas 1998, p. 192

Personal reflection• Central to successful portfolio• Explains –What is included?–Why it is included?– How it is organized?– How it relates to program or

institutional goals?

What is a portfolio?

• Tell your story

–Where have you been?

–What have you done?

–What have you learned?

–Where are you going?

• How to build a portfolio?

– Find a place to store your work

– Keep everything

– Ask for it in writing

– Be organized

– Paper vs electronic

What is a portfolio?

What is a portfolio?

Documenting Competence

Miller GE. The Assessment of Clinical Skills/Competence/Performance; Acad Med 1990 65(9):63-67. Adapted by Drs R. Mehay & R. Burns, UK (Jan 2009).

Why a portfolio?

Why a portfolio?

How can portfolios be used to document accomplishments related to teaching?

• AAMC Taskforce on Educator Evaluation: 2010 – 2012

• The Charge: To provide resources that will aid decision-makers in developing clear, consistent and efficient evaluation processes for faculty with a career focus in education

Documenting Accomplishments

Task Force MembersMaryellen Gusic

Indiana UniversityChair of the Task Force

Jonathan AmielColumbia University

Brian MavisMichigan State University

Suzanne RoseUniversity of Connecticut

Constance BaldwinUniversity of Rochester

Kathe NelsonUniversity of Alabama

Deborah SimpsonMedical College of Wisconsin

Latha ChandranSUNY Stony Brook

Lois NoraThe Commonwealth

Medical College

Henry StrobelUniversity of Texas

Medical School at Houston

Ruth-Marie E. FincherGHSU/Medical College of

Georgia

Jamie PadmoreMedStar Health

Craig TimmUniversity of New Mexico

Nancy LowittUniversity of Maryland

Pat O’SullivanUCSF

Tom ViggianoMayo Medical School

• Teaching• Learner Assessment• Curriculum Development• Mentoring and Advising• Educational Leadership and

Administration

What do educators do?

Simpson et al, 2007

Four typical indicators of competence:• Quantity• Quality• Scholarly approach• Scholarship

Evaluating the work of educators?

• Quantity– Duration, number, scope of teaching

activities

• Quality– Teaching effective and well-received

• Scholarly approach– Incorporates best practices

• Scholarship– Workshops, peer-reviewed presentations– Adoption by others

Contributions in Teaching

• Scholarship involves:

– Discovery of new knowledge

– Application of knowledge

– Integration of knowledge

– Dissemination of knowledge

Scholarship Reconsidered

Boyer, 1990

1. Clear goals2. Adequate preparation3. Appropriate methods4. Significant results5. Effective presentation6. Reflective critique

Glassick’s Criteria

Glassick, 2000

Let’s Focus on Teaching

• Learning objectives for teaching session/curriculum are:– Clearly stated– At level appropriate for learners– Specified to measure learner’s

performance

1. Clear Goals

• Learning objectives are:– Based on documented needs– SMART (specific, measurable,

achievable, realistic and timely)– Address multiple domains (e.g.,

knowledge, skills and/or attitudes)

1. Clear Goals

• Congruence/integration with other curricular components

• Use of best practices

• Necessary resource planning

2. Adequate Preparation

• Best Practices– Content is up-to-date and evidence-

based– Content is logically integrated with

other curricular components– Content to be covered appropriate for

time available– Content depth and breadth matched

to learners’ needs

2. Adequate Preparation

• Resource Planning– Specific needed resources are

specified– Needed resources are available– Adequate preparation for use of

technology

2. Adequate Preparation

• Teaching methods aligned with learning objectives

• Methods are feasible, practical and ethical

• Innovative teaching methods used to achieve learning objectives

3. Appropriate Methods

• Chooses teaching strategies that incorporate a variety of approaches

• Variety of approaches is evidence-based

• Uses interactive approaches and promotes self-directed learning

• Includes strategies for monitoring learner progress

• Provides evidence of innovation

3. Appropriate Methods

• Satisfaction/reaction of learners• Learning: Measures knowledge,

skills, attitudes and/or behaviors• Application: desired performance

demonstrated in other settings• Impact: educational programs and

processes here or elsewhere

4. Significant Results

Kirkpatrick & Kirkpatrick, 2006

• Satisfaction/Reaction– Teaching ratings by learners or

peers/experts– Compare learner ratings across

teachers

• Learning–Measurable changes in knowledge,

skills, etc.– Comparison to benchmarks or prior

data

4. Significant Results

• Application– Demonstration of knowledge, skills, etc. in

subsequent settings or curricular components

• Impact– Evaluation by knowledgeable peers,

educational leaders, etc.– Internal or external awards or recognition

4. Significant Results

• Recognized as valuable (internally or externally) through:– Peer review– Dissemination– Use by others

5. Effective Presentation

• Invitations to conduct faculty development, workshops, presentations

• Peer review of other teachers

• Dissemination and adoption of teaching materials or methods

5. Effective Presentation

• Ongoing improvement– Personal reflection– Learner performance data– Evaluation results– Peer review

6. Reflective Critique

• Critical analysis of teaching activities using information from others and self-reflection

• Evidence of continuous quality improvement of teaching activities

6. Reflective Critique

1. Clear goals2. Adequate preparation3. Appropriate methods4. Significant results5. Effective presentation6. Reflective critique

Glassick’s Criteria

Glassick, 2000

Toolkit

Criteria Teaching

Assessm’t

Curric Develop

mt

Mentoring/

Advising

Leadership/Admin

Clear goals √ √ √ √ √AdequatePreparation

√ √ √ √ √

AppropriateMethods

√ √ √ √ √

Significant Results

√ √ √ √ √

Effective Presentation

√ √ √ √ √

Reflective Critique

√ √ √ √ √

AAMC Toolbox for Evaluating Educators

• Available through MedEdPortal:www.mededportal.org/publication/9313

Where to Find It

• Boyer EL. Scholarship reconsidered: priorities of the professoriate. San Francisco, CA. Jossey-Bass Publishers; 1990.

• Glassick CE. Boyer’s expanded definition of scholarship, the standards for assessing scholarship and the elusiveness of the scholarship of teaching. Acad Med. 2000; 75:877-880.

• Kirkpatrick DL and Kirkpatrick JD. Evaluating Training Programs: The Four Levels (3rd Ed). San Francisco, CA: Berrett-Koehler Publishers, 2006.

• Miller GE. The Assessment of Clinical Skills/Competence/Performance; Acad Med 1990 65(9):63-67.

• Simpson D, Fincher RM, Hafler JP, Irby DM, Richards BF, Rosenfeld GC, Viggiano TR. Advancing educators and education by defining the components and evidence associated with educational scholarship. Med Educ. 2007;41:1002-1009.

• Snadden D. & Thomas ML. The use of portfolio learning in medical education. Med Teach. 1998; 20: 192-199.

References

• Baldwin C, Chandran L, Gusic M. Guidelines for evaluating the educational performance of medical school faculty: priming a national conversation. Teach Learn Med. 2011; 23(3):285-97.

• Hutchings, P. and Shulman, L.S. (1999). The scholarship of teaching: new elaborations and developments. Change, 31(5), 10-5.

• Van Tartwijk, J. & Driessen, EW. Portfolios for assessment and learning: AMEE Guide No. 45. Med Teach. 2009; 31: 790-801.

Additional Resources