Culture of Assessment Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office of Education Innovation University of Kentucky College of Pharmacy Closing the Loop
64
Embed
Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Culture of AssessmentCulture of Assessment
CREATING A CULTURE OF ASSESSMENT
Dean Kenneth B. Roberts, Ph.D.Heidi M. Anderson, Ph.D.
Eleanora Bird, M.S.Jeff Cain, M.S.
Office of Education InnovationUniversity of Kentucky College of Pharmacy
Closing the Loop
Culture of AssessmentCulture of Assessment
2
Outline Literature “Creating a Culture of
Assessment” UKCOP Case Study
Created an office to handle the College assessment efforts
Faculty development and involvement Student involvement Documenting current assessment efforts
Future Directions at UKCOP
Culture of AssessmentCulture of Assessment
Creating a Culture
of
Assessment 3
Culture of AssessmentCulture of Assessment
4
Culture of Assessment1. Leadership must be ‘on board’
Mission, resources, use results for improvement
2. Involve all stakeholders Faculty Students Alumni
Culture of AssessmentCulture of Assessment
5
Culture of Assessment3. Make assessment relevant
Course reviews Surveys, rubrics
4. Focus on teaching and learning rather than assessment Focus conversations not on
assessment, but on helping students learn
Culture of AssessmentCulture of Assessment
6
Culture of Assessment5. Provide opportunities to learn about
assessment6. Minimize the burden of assessment on
faculty and students Provide centralized leadership,
coordination and support for assessment Provide resources Keep paperwork to a minimal
Culture of AssessmentCulture of Assessment
7
Culture of Assessment7. Start small, Set up clear expectations and be
flexible8. Celebrate and reward assessment efforts
Give faculty and staff written recognition Encourage leaders to commend publicly Provide special funding Honor faculty and staff who obtain external
grants that will strengthen assessment
Culture of AssessmentCulture of Assessment
8
Culture of Assessment9. Use assessment results appropriately
Keep faculty, students and staff informed
Don’t penalize faculty whose assessments results are less than positive
10. Don’t expect everyone to get on board
Culture of AssessmentCulture of Assessment
(UKCOP)
9
Culture of AssessmentCulture of Assessment
10
Vision Office of Education Innovation (OEI)
was developed to assist faculty Teaching Learning ASSESSMENT Technology Scholarship of Teaching
Today
Culture of AssessmentCulture of Assessment
11
Mission The mission of the “Office of Education
Innovation” (OEI) is to provide assistance to faculty in developing and evaluating the professional pharmacy and graduate programs within the University of Kentucky, College of Pharmacy (UKCOP).
Culture of AssessmentCulture of Assessment
12
Community Dean’s Leadership Team Office of Academic Affairs Office of Education Innovation (4 FTEs)
Assistant Dean Assessment Coordinator Information Technology Officer Administrative Assistant
Faculty Students Other Participants
TEAM
Culture of AssessmentCulture of Assessment
13www.mc.uky.edu/Pharmacy/oei
Culture of AssessmentCulture of Assessment
14
Who the office works with? The “Office of Education Innovation” (OEI)
works closely with: UKCOP Course Coordinators/Directors UKCOP Individual Faculty UKCOP Departments UKCOP Students External Constituents to College
Culture of AssessmentCulture of Assessment
15
OEI Activities and Services Curriculum
Assessment Instructional Technology Faculty Development (Enrichment) Scholarship of Teaching
Today
Culture of AssessmentCulture of Assessment
UKCOP Faculty Development: How, What and
Why of Assessment?
16
Culture of AssessmentCulture of Assessment
17
Why do assessment? Accountability
ACPE Accreditation University Review Curriculum committee requirements
Enhance student’s educational development Decision-making
Culture of AssessmentCulture of Assessment
18
Accreditation July 2000 accreditation standards require
assessment activities to be aligned with mission and goals: achievement indicators, formative and summative formats, outcomes assessment, and systematic assessment plan.
Culture of AssessmentCulture of Assessment
19
Assessment ProcessDriven by Mission
Used forImprovement Ongoing
Use Multiple Measures
Culture of AssessmentCulture of Assessment
20
What is the Assessment Process? “Systematic collection, review and use of
information about educational programs undertaken for the purpose of improving student learning and development”
Foundation Knowledge Critical Thinking Information Management Communications Professionalism Social Interaction, Citizenship, Leadership Life-long Learning
Culture of AssessmentCulture of Assessment
31
UKCOP Professional Outcomes Patient Care
Medication Distribution Human Resources Management and
Operations Research Teaching
Culture of AssessmentCulture of Assessment
32
Faculty Involvement Focus on Teaching/Learning
Generate the appropriate assessment questions
Determine appropriate ways to measure outcomes
Decide on when to assess the students Prepare and utilize the reports
Culture of AssessmentCulture of Assessment
33
Faculty Involvement
Developing Questions to Ask: What do we want students to know, and do? How will we know they are learning what we
hope they will? What is the effect of our curricula, courses,
teaching, styles and educational tools? Did our changes bring about the effect we
desired?
Culture of AssessmentCulture of Assessment
Student Involvemen
t
34
Culture of AssessmentCulture of Assessment
35
Student Involvement Incoming orientation:
Explain the College’s assessment, rationale relating to accreditation standards, role of OEI
Review IRB protocol and informed consent Assessments throughout their 4 years Anonymous (code for tracking) Reports in aggregate fashion
Culture of AssessmentCulture of Assessment
36
Student Participation Curriculum Committee
Student Liaison Committee (SLC) meets 3x per semester
Assessment Committee Course
Performance Assessments Other
Accreditation Visit, Program Review
Culture of AssessmentCulture of Assessment
UKCOP Case Study:Documenting Current Effort
37
Culture of AssessmentCulture of Assessment
38
Documenting Current Efforts
Purpose Determine what assessments were already
taking place within the College
Develop an assessment inventory list and appropriate questions about student learning
Culture of AssessmentCulture of Assessment
39
Self-assessment of outcome abilities Attitudes towards peer/self assessment Performance records from courses CAPP results (integrated abilities) Student portfolios Yearly interviews Post-graduation surveys
determine types of assessment data that might be useful to gather for the program’s assessment functions Student records Program evaluations Student performance and competencies
Needs Analysis
Culture of AssessmentCulture of Assessment
42
Data Audit Class of 2002-2005: Student and educational
outcomes records Data obtained from various offices and files within
the college: Demographic Entry-exam scores Learning styles GPAs
Needs Analysis
Culture of AssessmentCulture of Assessment
43
Data Audit Data portfolios for several student cohorts
were examined Faculty Other sources:
Institutional Research Office Boards of Pharmacy State/local pharmacy associations
Needs Analysis
Culture of AssessmentCulture of Assessment
44
Data Audit Report
Culture of AssessmentCulture of Assessment
45
Success External IR officer requested to use the
college audit form as a model for the campus
Data are being collected from all audiences in a wide variety of venues
Some data are being disseminated in meaningful way
Data AuditNeeds Analysis
Culture of AssessmentCulture of Assessment
46
Lessons Learned from Data Audit
Amount & breadth of information was considerable
Gaps/Duplication
Some areas require more or different data
Culture of AssessmentCulture of Assessment
47
Lessons Learned from Data Audit
Length and frequency of surveys Faculty engagement Incorporate findings into College
assessment plan Reassess
Culture of AssessmentCulture of Assessment
48
Phase 2Recommendations about research
questions to assessment committee
I-E-O data management model used to classify student and program data (Astin, 1993)
I=inputs E=environments O=outcomes
Culture of AssessmentCulture of Assessment
49
Phase 2A. Content Analysis
B. Educational Impact
Which educational experiences/environments maximize student learning during the first year of pharmacy school?
Culture of AssessmentCulture of Assessment
50
Phase 2Ho: Structured Clinical Instructive Models
(SCIMs), prior to students participating in EPPE, statistically improve student critical thinking skills.
Ho: Videotaped counseling sessions statistically improve student communication skills during the first year of pharmacy school.
Culture of AssessmentCulture of Assessment
51
PHASE 3
Align and map curricular outcomes
Disseminate reports and use for improvement
Culture of AssessmentCulture of Assessment
Future Assessment Directions
52
Culture of AssessmentCulture of Assessment
53
Next Steps OEI reviewing literature Formulating appropriate research questions Prioritization and approval of research
questions by assessment committee Identifying appropriate measures Conducting the analysis Reporting and using the results for
improvement
Culture of AssessmentCulture of Assessment
54
Assessing Faculty Teaching
Classroom teaching Portfolio
Student advising Scholarship of
Teaching Publications Presentations
Supervision of clinical experiences
Supervision of graduate students
Consultation about teaching
Next Step:
Culture of AssessmentCulture of Assessment
Other Activities of the Office of
Education Innovation (OEI)
55
Culture of AssessmentCulture of Assessment
56
Instructional Technology Introduce and foster the use of
innovative technologies that enhance teaching, learning and assessment
Consultation and chair side assistance Remove obstacles