1 The Accreditation Team Experience: Ten Steps for Team Success Middle States Workshop for New Team Chairs and Evaluators October 4, 2006 Patricia A. McGuire.

1

The Accreditation Team Experience: Ten Steps for Team Success

Middle States Workshop for New Team Chairs and EvaluatorsOctober 4, 2006

Patricia A. McGuirePresident, Trinity (Washington) University

2

WHAT’S IT LIKE TO SERVE ON A MIDDLE STATES ACCREDITATION TEAM?

…IN FIVE WORDS OR LESS!

• INTENSE

• TEAMWORK

• STIMULATING

• INTELLECTUAL

• COLLEGIAL

3

TEN STEPS TO TEAM SUCCESS

1. Know “Accreditation”

2. Know the Process

3. Be Insightful About

Institutional Needs and

Expectations

4. Know Your

Characteristics!

5. Know Your Role and

Play It Well

6. Analyze Self-Study and

Prepare Field Work

Based on Analysis

7. Interview, Observe,

Review

8. Write a Great Report

9. Know Your MSA

Resources

10. Good Home Training

4

STEP ONE:

KNOW WHAT“ACCREDITATION”

MEANS

5

“Accreditation”

• Characteristics of Excellence

“Accreditation is the means of self-regulation and peer review adopted by the educational community.

The accrediting process is intended to strengthen and sustain the quality and integrity of higher education, making it worthy of public confidence and minimizing the scope of external control.

The extent to which each educational institution accepts and fulfills the responsibilities inherent in the process is a measure of its concern for freedom and quality in higher education and its commitment to striving for and achieving excellence in its endeavors.” (p. iv)

6

STEP TWO:

KNOW HOW ACCREDITATION

IS DONE

7

AC YR ONE___________________________________________________________AC YR TWO________________________ _____________________

AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUNE

AUGUST YEAR ONE

OUTLINESTEERING CMTEDATA COLLECTIONDOCUMENTATION

SEPT YR ONE

STAFF VISIT

OCTOBER YR ONE

DESIGN FOR SS SUBMITTED

COMMITTEESUNDERWAY

SPRING YR ONE

SPRING 05COMMITTEESDEVELOP REPORTS

END YR ONE

BY END OF YRALL COMMITTEEREPORTS DUE

SUMMERYEAR TWO

SELF STUDYDRAFTED

FALL YEAR TWO

SELF STUDYCIRCULATED FOR COMMENTAND REVISED

WINTER YR TWO

SELF-STUDYSUBMITTEDTO MSA

SPRINGYEAR TWO

TEAMVISIT

TEAM REPORT TOMSA

FALL YR TWO

PRELIM VISITTEAM CHAIR

ACCREDITATION AND PLANNING CONCEPTUAL TIMETABLETWO-YEAR PROCESS

PRIOR YEAR:PLANNING

ASSESSMENTDATA COLLECTION

ANALYSIS OFCHARACTERISTICS

COMMISSIONACTION

ONACCREDITATION

8

STEP THREE:

BE INSIGHTFULABOUT INSTITUTIONAL

NEEDS ANDEXPECTATIONS

9

What Do Institutions Expect of Teams and Chairs?

• Respect for Time and Effort in Self-Study; Respect for the Institution’s expertise about itself

• Collegiality with appropriate professional distance

• Fairness and Integrity

• “Compassionate Rigor”

• Expertise in Applying Standards and Interpreting Institutional Conditions

• Thorough Preparation and Earnest Interest in the Story of the Institution

• Consistency with Role Expectations and Conduct of the Visit

10

STEP FOUR:

KNOW YOUR CHARACTERISTICS!

11

TEAM ANALYSIS OF THECHARACTERISTICS OF EXCELLENCE

• ALL TEAM MEMBERS SHOULD READ THE CHARACTERISTICS THOROUGHLY AND FLAG ISSUES PARTICULARLY RELEVANT TO THE AREA EACH MEMBER WILL REVIEW AT THE INSTITUTION UNDER STUDY

• ANALYZE SELF-STUDY IN LIGHT OF THE CHARACTERISTICS

• PREPARE INTERVIEW QUESTIONS DRAWN FROM THIS ANALYSIS

12

STEP FIVE:

KNOW YOUR ROLEAND PLAY IT WELL!

13

Role of the Team Chair

14

Role of the Team Chair

ORGANIZER

Set Dates for Visits Conduct Preliminary Visit Manage Communications Final Report

Review Visit Arrangements Review Documents Make Team Assignments Chair’s Brief

Analyze SS Oral Exit Interview Eval. Cmte. Meeting

FACILITATOR/COACH

Develop Strategy for the Team Visit

Review with Team the Role of Team Members and Ground Rules for Visits

Review with Team Characteristics of Evidence and related accreditation materials

LEADER

Team and Institutionlook to YOU as the embodiment of

the Middle States system and arbiter of the ethics of peer review

15

When Team Chairs Go Bad…

Runaway Team – Chair Loses Control of Team Direction

Bully on the Team – Chair Fails to Confront/Resolve

Institutional Hide ‘n’ Seek – Chair Ignores Reality

Scrooge – Chair Keeps Team Working Past Reasonable Hour

Leviathan – Chair As Control Freak

Terminator – Chair Bullies/Threatens Institution

16

Role of An Evaluator• You are Invited because of your expertise, but remember the “peer” in “peer

review”

• Expected to be thoroughly prepared in advance of the visit:– KNOW Characteristics thoroughly and other MSACHE documents– ANALYZE Self-Study (see Handbook pp. 24-26)– BE READY to engage the interview process appropriately

• Cooperate with Team Chair in Site Visit Assignments, and maintain collegiality with team members

• Prepare each interview with care, and record notes carefully

• INTERVIEW, do not monologue! (If you hear yourself talking for more than 3 minutes, stop! Ask questions, don’t make speeches.)

• Refrain from making statements comparing this institution to yours

• Avoid Prescriptive Pronouncements

• Make Every Effort to Understand the Institutional Mission, Goals and Strategies

• Make sure that evaluative comments are rooted in Characteristics

17

When Good Teams Go Bad…• The Inspector General Persona and ‘Gotcha!’ Games

• “Well, at MY Institution here’s how it’s done….”

• “I am the Great Oz and I know what you need to be doing…”

• “Let’s slam them on this …”

• “Could You Believe That President??”

• “I’ve arranged a small meeting with a few disaffected faculty to hear what’s REALLY going on at this place.”

• “Forget the Characteristics, this place is a mess and we need to fix it or shut it down.”

• “They’re lying.”

18

STEP SIX:

ANALYZE THE SELF-STUDYAND PREPARE FIELD WORKBASED ON THIS ANALYSIS

19

PRIOR TO THE VISIT: TEAM ASSIGNMENTS

• Chair: Establish a pre-visit work plan for the team to guide members in preparation

– Identify key areas for each team member to review, but stress that all team members must read the entire self-study and participate in the full review

– Ask team members to prepare written outline of the issues and questions they are likely to review on the campus visit

– Review the team outlines and questions to be sure that all team members understand their roles and are heading in the right direction prior to the visit

• Team: – Study Characteristics and Designs to create mental framework for analysis– Read and Analyze Self-Study – Prepare written outline of issues to probe during campus visit and likely

questions to ask– Identify relationship to Characteristics for issues– Communicate with team chair about any questions/concerns regarding

role, issues, preparation

20

Sample Matrix for Team Assignments

Area of Inquiry

Questions

Self-Study Sections

Characteristics

Team Member #1 Outcomes Assessment

Chapters 2 and 3 Standards 8 and 14

Team Member #2 Governance and Administration

Chapters 1 and 5 Standards 4 and 5

Team Member #3 Planning and Resources

Chapter 1,4,5,6,7 Standards 1,2,3,7

Team Member #4 Student Services Chapter 6,7 Standards 5,7,9,11

Team Member #5 Admissions and Enrollment Mgmt

Chapters 1,2,3,6 Standards 6,8,9,11

Team Member #6 General Education and Undergrad Prgms

Chapters 2,3,4 Standards 11,12,13,14

Team Member #7 Faculty and Graduate Programs

Chapters 2,3,4,7 Standards 10,11,12,13,14

21

Understanding the Self-Study

• Designs: “The self-study process enables the campus community to examine the institution’s strengths and challenges relative to accreditation standards. The process also enables each institution to develop solutions to problems, and to identify opportunities for growth and development. The aim is to understand, evaluate and improve, not simply to describe or defend.”

(emphasis added) (p. 3)

22

Understanding the Self-Study

• Designs:

Self-Study =

• Continuous Process in Planning Cycle

• Accountability to Constituencies

• Measurements of Goal Achievements

• Meet Accreditation Standards

• Fulfill Institutional Mission

23

Problem Set: Self-Study Meltdowns and Team Response Strategies

• Self-Study lacks the basics

• SS is disorganized and badly written

• Gaps in Data

• Too superficial/too self-congratulatory without analysis/too descriptive without analysis

• Overly self-critical where data and information suggests otherwise

• Political tensions on campus come through in report and create analytical skews in the narrative and data

• Self-study is not truthful in part, fact only discovered through the team visit

• Team learns during visit that the self-study was largely written by administration with little campus engagement

• Chair discuss with MSA staff

• Chair Discuss with MSA staff and Institution prior to the visit, correct as possible in advance

• Be prepared to identify what data sets are necessary for the team visit to be successful and request the data

• Team must probe beneath the writing to get the insightful analysis that the document may lack. Need to distinguish between writing style preferences of team members and real problems with the self-study

• Chair and team discuss gaps between reality and statements in the report and reasons

• Team cannot get drawn into campus political issues; need to pierce through the tension to get at facts and analysis of performance and outcomes

• Chair must discuss issues with the institutional president as soon as the facts are in hand; consult with MSA staff if necessary

• Problems of collegiality in the self-study process can be addressed in the team report.

PROBLEM ACTION STEP

24

Problem Set: Self-Study Meltdowns and Team Response Strategies

• Institution plans a selected topics self-study but needs a comprehensive review

• As Designs makes clear, this kind of problem should be addressed in the early stages of self-study preparation, see pp. 10-11.

– Institution should submit proposal for selected topics 24 months in advance

– Commission staff evaluate proposal and compliance with all accreditation standards

– (Option 1) Chair’s Preliminary Visit should review the plan and the documentation supporting compliance with standards NOT addressed in the selected topics self-study

• Chair and “Generalist Evaluator” write a report, see p. 11

– (Option 2) Chair and Generalist Evaluator review the comprehensive compliance documentation during the regular visit, with team focused on special topics, and Chair/Evaluator write report on compliance with other standards

• In all events, the Chair should consult as early as possible with the MSA staff if there are questions, differences of opinion, or new data on this topic

PROBLEM ACTION STEP

25

Problem Set: Self-Study Meltdowns and Team Response Strategies

• Institution chose a selected topics model of self-study and the report seems fine, but during the team visit it becomes clear that there are major problems at the institution that a focused self-study ignored, whereas a comprehensive self-study would have addressed the problems

• Team member(s) ignore selected topics and insist on pursuing other avenues of inquiry

• Chair should seek guidance from MSA staff. Might be appropriate to address the problem in the team report. Campus visit should ensue on the basis of the report given, and that report should be addressed, but the gaps should also be illuminated and addressed for further action.

• Chair is in charge of the team and responsible to ensure that the team stays focused

PROBLEM ACTION STEP

26

STEP SEVEN:

INTERVIEW, OBSERVE, REVIEW: MAKE GOOD

USE OF TIME ON CAMPUS

27

MAXIMIZING TIME ON THE CAMPUS VISITINTERVIEWS, OBSERVATIONS, REVIEWS

• Chair: Prepare interview list and schedule well in advance in consultation with the institution and the team; changes can and will occur during the visit, but have a game plan in advance and change strategically

• Team: All members should have a prepared set of issues and interview questions for the appointments

• Observe: An important part of the campus visit is observation of the environment, facilities, availability of services, general conditions;

• Review: A large part of time on campus is also spent reviewing source documents that support assertions in the self-study; the team should come prepared with a list of topics that bear additional research time when on campus

28

How to Conduct an InterviewDuring a Team Visit

• Preparation: Interviews test and illuminate the self-study. Good interviews start with thorough knowledge of the self-study and institutional materials. Chair makes interview assignments according to the expertise of the team, but team members also identify potential interviews, and the final schedule reflects dialogue between team chair and members about assignments.

• Develop Interview Objectives and Questions: For each interview, the team member should know the objectives of the interview, develop clear questions, and be sure those questions relate to the self-study and accreditation criteria.

• Seek Dialogue, NOT Cross-Examination: Questions should be designed to evoke analysis and dialogue. This is not a cross-examination or interrogation, and questions that make the interviewee feel put on the spot are generally not helpful.

• Set the Context and Explain the Objectives: At the outset of the interview, be sure to get the names and positions of everyone in the room. Greet the individual or group in a friendly way, and remind all of the purpose of accreditation. Frame the objectives of the interview and general topics under review in this session. Set the time limit as well.

• Avoid Monologue and Prescription: Team members must not use interview times to lecture the interviewee, monologue about your opinions, or offer prescriptions about what the institution should do. You are there to learn, test assertions constructively and push analysis by institutional participants.

• Take Careful Notes: Interviews form an important part of the background for the team report, and often a statement in an interview can shape the team’s understanding of some dimension of the self-study. Notes taken during the interview facilitate accuracy and completeness.

29

STEP EIGHT:

WRITING A GREAT TEAM REPORT

30

TEMPLATE FOR TEAM REPORTS(See pp. 32-38 “Chairing and Hosting”)

I. Context and Nature of the Visita) Institutional Overviewb) Scope of Institutionc) Self-Study Process

II. Affirmation of Continued Compliance With Eligibility RequirementsIII. Compliance with Federal/State/Other Regulatory/Other Accreditation

IV. Compliance with Accreditation StandardsFor each standard (or by major heading in the self-study, with standards identified) the report should follow this outline:

a) Summary of Evidence and Findingsb) Significant accomplishments/progress/exemplary practicesc) Suggestions for Improvementd) Recommendationse) Requirements

V. Summary of Team Recommendations and Requirements

Standard Recommendation Requirement

31

Suggestions, Recommendations, Requirements: What’s the Difference?

• SUGGESTIONS

– may reflect ideas for improvement in performance within accreditation standards based on the professional experience of team members or the guidance of CHE publications

• RECOMMENDATIONS

– identify areas where the institution needs improvement in order to continue to meet accreditation standards

– must state specifically how the institution might be at risk of non-compliance;

– team report must be specific about the standard and how the institution needs to improve;

– note that a recommendation warrants Commission follow-up with a progress letter or monitoring report

• REQUIREMENTS

– address specific points where the institution fails to meet the accreditation standards

– and set forth steps the institution should take to comply;

– specific citation to the standard, required element and failure to meet the requirement;

– note that a requirement necessitates Commission action of postponement, warning, probation or show cause

32

STEP NINE:

KNOW YOUR MSA RESOURCES!

33

Resources for Team Support

• All of the Middle States Documents

• Middle States Staff

• Experienced Evaluators

• Team Chair

34

STEP TEN:

GOOD HOME TRAINING

35

Good Home Training

• Things that shouldn’t need to be said, but sometimes have to be:– Leave your biases at home.– If you have a conflict of interest, reveal it before the visit.– Respect the privacy and reputation of the institution. Do not ‘share’

what you have learned about the institution when you get home.– Keep your sense of humor apparent, but mild. This is not the time to try

out for Second City.– Do not be dour, or angry, or difficult, this is a TEAM and we all need to

get along.– Bar bills should not be necessary at all, but pay for your drinks

personally if you choose to have a drink.– Same for movies and personal phone bills.– There are no job opportunities for you at this school.– If you hate the experience, do not volunteer for this gig again.– If you love the experience definitely sign-up again.– Be open to learning new things you didn’t anticipate.– Get to bed at a reasonable time each night.