Marvin W. Peterson & Thomas E. Perorazio The National Center for Postsecondary Improvement, Project 5.2 The University of Michigan Ross A. Griffith, Wake.

Marvin W. Peterson & Thomas E. PerorazioMarvin W. Peterson & Thomas E. PerorazioThe National Center for Postsecondary Improvement, Project 5.2The National Center for Postsecondary Improvement, Project 5.2

The University of MichiganThe University of Michigan

Ross A. Griffith, Ross A. Griffith,

Wake Forest UniversityWake Forest University

American Association for Higher Education American Association for Higher Education Assessment ConferenceAssessment Conference

Denver, CO - June 25, 2001Denver, CO - June 25, 2001

Institutional Strategies for IntegratingInstitutional Strategies for IntegratingStudent Assessment with Student Assessment with

Academic Planning and ManagementAcademic Planning and Management

Session Purposes

Discuss Initial Findings from the Case Study Phase of NCPI Research Project on Institutional Support for Student Assessment

Provide Detailed Examples of Two Institutions That Have Integrated Assessment Into their Academic Management Processes

Outline the Lessons Learned Applicable to Assessment Practitioners

The National Center for Postsecondary Improvement

Main Project Web Site

http://www.stanford.edu/group/ncpi/

University of Michigan - Project Five

http://www.umich.edu/~ncpi/

NCPI Project Area 5:

Student Learning and Assessment

5.3 Academic Programs and Students:

Educational Reform and Innovation in Teaching, Learning, and Assessment Practices

Metadata Project: Review of National Student, Faculty, and Institutional Data Resources

5.1 State Policies and Regional Accreditation Practices of Assessment for Student Learning

5.2 Institutional Support for Enhancing Student Assessment and Performance

5.2 Institutional Support for Enhancing Student Assessment and Performance

Project 5.2’s Purpose

To Examine How

Postsecondary Institutions

Support and Promote the

Uses and Impacts of Student

Assessment Information for

Institutional Improvement

To Examine How

Postsecondary Institutions

Support and Promote the

Uses and Impacts of Student

Assessment Information for

Institutional Improvement

Overview of Project 5.2

Phase OnePhase One

•Review & Synthesis of the Literature •Developed a Conceptual Framework of Institutional Support

Phase TwoPhase Two

National Survey of Institutions•Comprehensive Inventory Based on Conceptual Framework•1394 Usable Responses -- 55.1%

Phase ThreePhase Three

Case Studies of Institutions with Comprehensive Student Assessment Efforts

Phase ThreePhase Three

Case Studies of Institutions with Comprehensive Student Assessment Efforts

Institutional Context

Assessment Culture and Climate

Institutional Approach to Student Assessment

Institution-wide Support for Student Assessment

Assessment Management Practices & Policies

External Influences on Student Assessment

Uses & Impacts of Student Assessment

Phase I: Conceptual Framework of Organizational and Administrative Support for

Student Assessment

Phase II: National InventoryResearch Questions

What Approaches to Student Assessment Have Institutions Adopted?

What Organizational and Administrative Support Patterns and Assessment Management Practices for Student Assessment Have Institutions Developed?

How Have Institutions Used Student Assessment Information and What Impacts Has It Had?

How Do Patterns of Student Assessment Approach, Organizational and Administrative Support Patterns and Practices, and Uses and Impacts of Student Assessment Vary by Institutional Characteristics?

How Do External Groups Influence Institutional Adoption of Various Measures of and Approaches to Student Assessment?

How Are External Influences on Institutional Approaches to and Organizational and Administrative Support Patterns and Practices for Student Assessment Related to the Use and Impact of Student Assessment Information?

Phase III: Case Study Research

Seven Institutions With Extensive Involvement in Student Assessment

•Review of Response Patterns to National Inventory of Institutional Support for Student Assessment (ISSA) •Campus Site Visits by Research Teams•Collection & Synthesis of Documents, & Records •Interviews & Focus Groups With Faculty, Administrators, & Students

Research Purposes

1) To Examine the Internal Dynamics of Institutional Support for Student Assessment

2) To Identify Institutional Strategies for Supporting, Promoting & Integrating Student Assessment

3) Assess the Culture & Climate for Using Student Assessment Data for Academic Improvement

Integrating Student Assessment with Academic Management for Institutional

Improvement

Institution’s Academic Management

Approach

Use in Academic Decision Making

AssessmentPurposeStructureApproach

AssessmentLeadershipSupportCoordination

Links to Management Process/

Improvement

Discussion of Institutions

Student Assessment Strategy Purposes of Assessment Assessment Structure Approach to Assessment Leadership, Support, & Coordination Links to Academic Management Processes Utilization in Academic Decision Making Summary - Culture for Student Assessment

Brief Institutional Storyline Institution’s Academic Management Approach

•Primary Mission & Governance Pattern•Key Academic Management Processes

The Story of

Institution Developed a Quality Initiative Based on a Baldridge

Model

Assessment Adopted as a Means of

Academic ImprovementAssessment Activity Gradually

Integrated into Culture of Quality and Improvement

for Many Years

Primary Mission & Governance Pattern

Vision

•Emphasis on Culture of Quality; Sets Goals

Statement of Mission•Focuses on Continuous Improvement•Committed to Using Assessment to Enhance

Performance

Cultural Core Values•Define the University; Drive the Mission & Vision

Decision Drivers•Priorities Regarding Essential Functions

Strategic Planning Council

Seven StepPlanning Process

Program Management

Student AffairsKQIs

Office of Assessment, Information, & Analysis

Approach to Academic Improvement

Key Academic Management Processes

Assessment Purpose -- NMSU

Initiating Conditions

Internal- President’s Focus on Culture of Quality (early

1980s)

External- State Mandated Assessment by Governor (late

1980s)

President’s Early Efforts ResistedCulture of Quality

Focused on Academic Improvement in Undergraduate Education•Adapted From Baldridge National Quality Award Criteria

Assessment Structure -- NMSU

Formal, University-wide Strategy

Uses Baldridge Quality Award Criteria As an Assessment Framework

Comprehensive Approach -- Cognitive and Affective Learning

College and Post-college Experiences

Collected by by Office of Assessment, Information, and Analysis (OAIA)

Seven Step Planning Process

Assessment Approach -- NMSU

Quantitative & Qualitative Data Gathered From Recruitment Until Post-graduation

Battery of Cognitive Measures During College Years

Regular Collection Schedule

Data on Needs or Expectations of Students Using Local and National Surveys & Listening Posts Assessments are required--Explained in UG Catalog

Student progress tracked and enforced through restrictions on registration and final transcripts

Leadership, Support, & Coordination

Leadership

Long-term, Continuous Presidential Promotion

Strong Support by Provost Through OAIA•Other Groups:

•President’s Cabinet & Deans, Baldridge Category

Council•Faculty Senate Assessment Committee•Student Success Task Force

•Faculty Provide Significant, but Hesitant, Support Coordination

•OAIA-- Compiles and Distributes Assessment Data

•Formal Grid Details Methods, Uses, & Frequencies for Assessment Activity

Links to Academic Management Process/

Improvement

Annual Departmental Reports & Course Reviews

Primary Link Between Action Planning & Assessment

Deans Uses Assessment Data to Identify Areas for

Improvement

Focus Is on Student Performance and Experience DataPresident's Cabinet Reviews Reports & Provides Feedback to Unit

Leaders

Strategic Planning Council

Representatives From Administrative, Academic, and Student Affairs

Assessment Info Used:

•To Evaluate Organizational Performance on Strategic Initiatives (SI) •The Development of New Key Quality Indicators (KQI) for Each SI

Links to Academic Management Process/ Improvement

OAIA Produces Three Types of Academic Information:

1) Outcomes and Performance;

2) Student and Stakeholder Satisfaction Ratings;

and

3) Operational Effectiveness

Data Is Distributed Through Dashboards and Profiles

Tracking Systems to Indicate Deviations From Intended Targets

Can Be Accessed Through Campus Network

Measurement of of Key Quality Indicators

Seven Step Planning Process

•To Meet Stakeholder Requirements

Utilization in Academic Decision Making-- NMSU

Planning

Phase I Review of Mission and Analysis of Progress on Current Goals

Phase II Identification of Strategic Initiatives

Phase III Action Planning and Development

Actions Plans Are Developed to Improve Performance on Key Quality Indicators

•Purpose Is to Increase Organizational Performance

•Assessment Information Part of KQIs in Dashboards & Profiles Critical to Developing Agenda for Planning Efforts

Utilization in Academic Decision Making

Program Management

•Annual Reports & Reviews•Benchmarking•Stretch Goals

Quality

Student Affairs-Academic Affairs Alliance

Development of Student Life & Experiences Surveys/Studies

Links to Educational uses of Assessment Information

NMSU Applies Annually for Baldridge Quality Award

Summary-- Culture for Assessment

Baldridge Quality Criteria Drive Assessment FrameworkIntegration of Assessment into Quality, Planning,

and Program Management Processes

Data-Driven System & Reports

Strong, Central Administrative Support for

Quality Approach & Assessment

Key Coordination Role Played by OAIA --

Commitment to Assessment•As a Means of Visibility for Quality Emphasis•Increasingly for Institutional Improvement

The Story of

Assessment Data Essential to Cycles of Institutional Planning,

Program Review, & Evaluation

Assessment Information Used by Decision Makers at All Levels to

Evaluate Progress Towards Goals

Administration Expects that Assessment Activity Will Occur for

the Purpose of Academic Improvement

Primary Mission & Governance Pattern

No Formal Statements Regarding Assessment

WFU Administration believes Assessment is a Necessary Management Tool to Maintain & Improve its Quality and Reputation

•Obtaining Data Crucial to Decision Making

Institutional Strategic Planning

Evaluation Committee

Institution-wideAssessment

Departmental Review

Student Life

Office of InstitutionalResearch

Approach to Academic Improvement

Key Academic Management Processes

Assessment Purpose -- WFU

Initiating Conditions•Internal- Foundations for Institutional Cycles of Planning Initiated by President & Vice-president

•External- Accreditation Focus on Institutional Effectiveness

Program Planning & Evaluation Committees Use Assessment Data

•To Address Major Campus Needs •Ongoing Strategic Planning

Program Planning & Evaluation Committees Use Assessment Data

•To Address Major Campus Needs •Ongoing Strategic Planning

Evaluate Strategic Plan The Plan for the Class of 2000

•Assessment Now a Management Tool in a Continual Process of Planning, Implementation, and Evaluation

Assessment Structure -- WFU

1) University-wide- OIR- Oversees Assessment of the Process

•Administers National & Local Surveys to Students & Faculty•Analyzes, Stores, & Disseminates Data•Works With Evaluation Committee & Other Depts. •Only Office to Have Dedicated Funds for Assessment

1) University-wide- OIR- Oversees Assessment of the Process

•Administers National & Local Surveys to Students & Faculty•Analyzes, Stores, & Disseminates Data•Works With Evaluation Committee & Other Depts. •Only Office to Have Dedicated Funds for Assessment

Three Level, Decentralized Process

3) Student Life- Focuses on its own programs

•Focus is on student experiences & satisfaction•Important for Intellectual Climate Planning

2) Departmentally Based- Program Review Process •Primary Planning Unit on Campus •13-month Process, 7 Year Cycle for All Programs •Review of Curriculum & Department/ Program Performance•$2,500 Budgeted for Each Review

Assessment Approach-- WFU

Departments Conduct Own Assessment of Programs & Satisfaction

Significant Autonomy & Discretion Granted to Evaluate Their Students

Primarily Quantitative Focus on Satisfaction & Experiences

Little Emphasis on Cognitive or Affective Measures Some Determined by Dept. In Departmental Review

Institution-wide Surveys of Students, Faculty, & Alumni

National Surveys Administered by Office of Institutional

ResearchAlso Some Locally Developed Instruments & Course

Evaluations

Leadership & Support

Institutional Support from Executive

•President’s and Provost’s Seminars, Conferences, & Retreats

•Executive Council & Reynolda Cabinet Receive Reports & Use the Information in Strategic Planning

Coordination

OIR Shares Its Information With Departments Upon Request

Student Life Provides Its Data to Departments

Information Systems Office Sends Course Evaluation Results to Department Heads Online

OIR Role in Accreditation Allows It to Maintain Focus on Institutional Effectiveness Criteria

OIR Has the Primary Role In:

•Administering Institution-wide Student Assessments•Coordinating Program Planning•Conducting Data Analysis

Information Flow/ Link to Improvement

Data Driven Approach to Management•Measuring Progress on Strategic Plan Goals Results in Steps Toward Improvement

OIR & Departments Report to Evaluation Committee (EC) for the Strategic Plan

Reviews Departmental Reports on How SA Info Was Used to Monitor Progress on Goals

EC Monitors 27 Areas of QualityEC Members Serve As Liaisons to Other Committees and Departments

Annual Reports by Departments to Deans•Must State Progress on Goals and Set New Ones

Utilization in Academic Decision Making

Institutional Strategic Planning•Prior Data Streams Produced the Development, Implementation

and Evaluation of Plan for the Class of 2000

•Recent Student Life Data Critical to Next Wave of Planning on Intellectual Climate

Program Review & Departmental Evaluations

•One Full Cycle Completed

•Departmental and Curricular Changes, including Program

Restructuring or Elimination

•Departments use OIR Data & their own to Improve Teaching & Learning

Utilization in Academic Decision Making

Evaluation Committee•Assesses Laptop Initiative and Effect of Technology on Academics

•Purpose Remains Measurement of Progress Towards Academic & Educational Goals

Student Life•Units Engage in Planning & Evaluation

•Many Changes to Services Made Based on Satisfaction Survey Results

Summary-- Culture for Assessment

Academic Management is Well-designed

Integrated Institutional Planning, Program Review, & Academic Evaluation Approach that is Improvement Driven

Student Assessment Integrated into All Academic Management Processes

Strong, Central Support for Data Driven Management

Intensive Involvement of Departments in Program Review

Key Support & Coordination Roles Played by OIR

Commitment to Assessment is Embedded in Culture, but Not as an End in ItselfWide Support for Student Assessment

Conclusions

Lessons & Insights

Emphasis is on Integrating Student Assessment into Key Elements & Academic Decision Making Arenas:

Assessment Used at All Levels of Institution

Tied to Academic Improvement

Embedded in Academic Management Policies & Practices

Assessment Data Key to Management & Evaluation Processes

Need a Clear Understanding of: The Institution’s Academic Management Approach

& its Key Elements

The Role of Assessment in Their Various Elements

Lessons & Insights An Effective Student Assessment Strategy Requires:

Clearly Understood Purpose for Assessment

Committed, Upper-Level Leadership

Coordination Across Support Offices &

Departments

Flow of Information to Decision Makers at All

Levels

Link of Data to Problem Identification and Goal

SettingBuilding a Culture that Values Assessment

Requires: Long-term Effort Linking Assessment to Academic Management Processes Concerned with Academic Improvement

Wake Forest Case Study Visit September 2000

Wake Forest Honored to Be Selected for the Case Study

Faculty, Students and Administrators Were Most Cooperative in Participation of Interviews and Focus Groups

Interviews Included President, Vice Presidents, Deans,

Associate Provost, Advising Director, Director of Teaching

& Learning Center, Director of Institutional Research,

Chief Information Officer, Faculty Senate President, Chair

of Curriculum Committee, Chairs of Academic

Departments and Members of Evaluation Committee Focus Groups Included Four Faculty Members and Four

Departmental Chairs Active in Assessment and 10 “Good” Students

Lessons Learned Considerable Assessment With Decided

Outcomes Took Place at Institutional Level, e.g. Program Review and Surveys of Students and Faculty

Assessment at Departmental Level Is Decentralized and Varies Considerably

Qualitative Assessment Could Be Emphasized More at the Institutional Level

Combination of SACS 1995-97 Self Study and 1992-95 Strategic Plan Strengthened Assessment at Wake Forest