10/22/2019 1 Maybe We Should Ask Them: When It’s Time for Meta‐Assessment IUPUI Assessment Institute October 14, 2019 Cynthia Groover, EdD Assessment and Accreditation Specialist [email protected]How We Do What We Do • Business and Finance • Information Technology • Student Affairs and Enrollment Management • Academic Affairs • Office of the President – Advancement – External/Government Affairs
13
Embed
How We Do What We Do - Assessment Institute · OIE Website External Resources Face to Face Feedback Electronic Feedback Review Team Off Cycle OIE Off Cycle OIE Rubric Divisional Example
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
10/22/2019
1
Maybe We Should Ask Them: When It’s Time for
Meta‐AssessmentIUPUI Assessment Institute
October 14, 2019
Cynthia Groover, EdDAssessment and Accreditation Specialist
• Business and Finance• Information Technology• Student Affairs and Enrollment Management• Academic Affairs• Office of the President
–Advancement –External/Government Affairs
10/22/2019
2
Who Are We Targeting?
Resources and Processes in Question
• OIE Website
• OIE Rubric
• Electronic Feedback
• Face to Face Feedback
• Consultation with IE Review Team Off Cycle
• Consultation with OIE Staff Member Off Cycle
• Divisional Example
10/22/2019
3
Programmatic Assessment Focus
What are the strengths and weaknesses of the processes and resources in place in support of administrative and student affairs assessment?
10/22/2019
4
Programmatic Assessment
Shutt, Garrett, Lynch, and Dean (2012)
Any program “should continue to undergo evaluation where it can be modified to ensure that every element contributes to the program’s outcomes” (p. 78).
Fink (2013)
Institutions need to look at the impact of multiple resources and processes in place to determine if they have the “right set of activities” in place to positively impact assessment practices across campus (p. 47).
Getting Uncomfortable
Meyer and Murrell (2014)
“ask the tough questions and to get the news that something is not working (or working as assumed) and should therefore be revised or eliminated” (p. 4).
10/22/2019
5
Guiding Questions
1. What are the perceived strengths and weaknesses of the processes and resources in place to develop knowledge of and confidence in assessment and how does perceived utility differ among divisions of the institution?
2. How do assessment coordinators perceive their own knowledge of and confidence in assessment?
3. What is the relationship between knowledge of and confidence in assessment and the utility of resources in place?
4. What is the relationship between knowledge of and confidence in assessment and number of assessment cycles in which participants have engaged?
Data Collection
Why E‐mail ‐ Social Desirability Bias
Saturation sampling
Population
N = 85
n = 61
71.7%
10/22/2019
6
Research Question One
1. What are the perceived strengths and weaknesses of the resources in place to develop knowledge of and confidence in assessment and how does perceived utility differ among divisions of the institution?
Note. n = 61. *Denotes significant at the p < 0.05 level
Knowledge Confidence Composite and Resource Utility Take 2
OIE Website
External Resources
Face to Face
FeedbackElectronic Feedback
Review Team Off
Cycle
OIE Off
Cycle OIE RubricDivisional Example
KCC Pearson Correlation
0.33 0.29 0.35** 0.34** 0.54** 0.55** 0.42** 0.14
N 35 33 60 60 43 44 44 45
10/22/2019
12
Research Question Four
4. What is the relationship between knowledge of and confidence in assessment and the number of assessment cycles in which participants have engaged?
Knowledge Confidence Composite and Experience
Assessment Cycles
KCC 0.11Note. n = 60. *Denotes significant at the p < .05 level.
10/22/2019
13
Using What We Know Now
• We have some disconnects to address
– Perception versus all the data
• We have some relationships to build
– Functioning within a very different environment
– Posting resources may help some
– Being the resource will help more
• We need to keep asking those questions
– And be ready to respond
References
Fink, L. D. (2013). Innovative ways of assessing faculty development. New Directions for Teaching & Learning, 2013(133), 47‐59. doi: 10.1002/tl.20045
Meyer, K. A., & Murrell, V. S. (2014). A national survey of faculty development evaluation outcome measures and procedures. Online Learning, 18(3).
Rodgers, M., Grays, M., Fulcher, K., & Jurich, D. (2013). Improving academic program assessment: A mixed methods study. Innovative Higher Education, 38(5), 383‐395. doi: 10.1007/s10755‐012‐9245‐9
Shutt, M. D., Garrett, J. M., Lynch, J. W., & Dean, L. A. (2012). An assessment model as best practice in student affairs. Journal of Student Affairs Research and Practice, 49(1), 65‐82.