Developing an Assessment Plan Owens Community College Assessment Day Workshops November 13-14, 2009 Anne Fulkerson, Ph.D. Institutional Research
Dec 23, 2015
Developing an Assessment Plan
Owens Community College Assessment Day WorkshopsNovember 13-14, 2009
Anne Fulkerson, Ph.D.
Institutional Research
Agenda
What is assessment? Why assess? Types of assessment Basic steps in the assessment planning process
Identifying outcomes Defining measures Evaluating measures Other important considerations
OCC assessment plan template & rubric Sample assessment plan How IR can help
What is Assessment?
“Assessment is the ongoing process of:
Establishing clear, measurable expected outcomes of student learning.
Ensuring that students have sufficient opportunities to achieve those outcomes.
Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations.
Using the resulting information to understand and improve student learning.”
Suskie (2004), pg. 3
Why Assess?
Demonstrate effectiveness Improve student learning
Measure Learning
Adapted from Suskie (2004) and Maki (2004)
Provide Learning Opportunities
Establish Learning Outcomes
Implement Change
Types of Assessment
Formative—assessment that takes place while learning is in progress (or while a new program is being developed) in order to provide feedback for improvement
Summative—assessment that takes place at the end of a course or program to document that learning has occurred or to make judgments about the efficacy of a program
Establish Mission Statement
Identify Student Learning Outcomes
Student Learning Outcomes: Specifiable activities, products, skills, abilities, behaviors, attitudes, or pieces of knowledge that students attain as a result of their involvement in a particular set of educational experiences.
Represent the most important competencies that all students should possess
Do not reflect ALL learning that might occur Often aligned with accrediting body standards No magic number, but keep them manageable
Expressing Student Learning Outcomes
Too vague:Students will demonstrate information literacy skills
Too specific:Students will be able to use institutional online services toretrieve information
Better:Students will locate information and evaluate it criticallyfor its validity and appropriateness
Expressing Student Learning Outcomes
Fuzzy Terms: Action Words:
Know
Understand
Become aware of
Appreciate
Think critically
Demonstrate knowledge
Learn
Write proficiently
Remember
Identify
Perform
Create
Define
Summarize
Explain
Discuss
Describe
Solve
Find/Locate
Analyze
Evaluate
Apply
Debate/Argue
Synthesize
Integrate
Research
Choose
Construct
Design
Develop
Organize
Use
Bloom’s Taxonomy (Revised)
Based on an adaptation of Anderson & Krathwohl (2001)http://uwf.edu/cutla/assessstudent.cfm
Th
inki
ng
Ski
lls
Not Measurable:
Recognize a need for lifelong learning and plan for personal andprofessional growth
Demonstrate an historical knowledge of the symphonic, string orchestra, and chamberensemble repertoire
Sample Student Learning Outcomes
Measurable:
Describe and adopt a plan for ongoing professionaldevelopment and lifelong learning
Describe the historical development of the symphonic, string orchestra, and chamberensemble repertoire
http://uwf.edu/cutla/assessstudent.cfm
Identify Opportunities for Students to Achieve Each Learning Outcome
Course / Activity / Experience
Learning Outcome
1.
2.
3.
4.
5.
Define Measures for Each Outcome
Types of Measures:
Quantitative—numeric (e.g., test scores, placement rates, GPA, structured surveys)
Qualitative—described by words rather than numbers (e.g., interviews, focus groups, observations, rubrics)
Direct—require students to display knowledge or skill (e.g., tests, performances, assignments)
Indirect—second-hand evidence (e.g., surveys, course evaluations, journal reflections)
Word of Caution
Don’t measure everything, just because you can Pick the best measures Keep it manageable
Determine How Measures will be Evaluated Criterion-based benchmarks—compares
student performance with a pre-established standard.
Norm-referenced benchmarks—compares student performance with a standardized norm or a group of peers.
Value-added approach—compares student performance at two points in time to see if they have improved.
Longitudinal/historical approach—compares current students with prior students.
Other Things to Consider
What resources are needed?
Who’s responsible for collecting and analyzing the data?
Build support through participation in the planning process
Assessment plans are perpetual drafts
OCC Assessment Plan Templatehttps://intranet.owens.edu/committees/outcomes/index.html
Program Mission: Program Vision:
Program Level Student Learning Outcomes Program Student Learning Outcomes # 1: Where taught? Where
Where measured?
Direct measure(s)
Indirect measure(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 2. Where taught?
Where measured?
Direct measure(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 3 Where taught? Where
measured? Direct measure(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 4 Where taught? Where
measured? Direct measurer(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmarks(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 5 Where taught? Where
measured? Direct measurer(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
OCC Assessment Plan & Report Rubrichttps://intranet.owens.edu/committees/outcomes/index.html
Elements (1) Developing Established Exemplary Program goals and intended student learning outcomes (Appropriateness)
Lack of or incomplete inclusion of some established criteria.
Describe how the goals & SLO align with College &/or School mission & vision. Describe how the goals & SLO link to accreditation standards. if app Describe how the goals & SLO link to industry and/or academic standards.
Provides evidence supporting the statements in “Established” category.
Evidence is appropriate for the measurement of student learning (Evidence)
Lack of or incomplete inclusion of some established criteria.
Evidence that the measurement instrument is aligned with goals and/or SLO. Benchmarks in place and evaluated regularly Artifacts when utilized are evaluated with rubrics, when appropriate. At least one direct & one indirect measure as appropriate.
Normed evidence Provide information on validity & reliability of instrumentation. Rubrics are constructed and/or agreed upon by most stakeholders. Multiple direct & indirect measures.
Individuals responsible for delivering the program engage in systematic analysis of evidence. (Analysis of Evidence & Shared Responsibility)
Lack of or incomplete inclusion of some established criteria.
Describe how the data was analyzed. Provide evidence that appropriate stakeholders were involved in the analysis of data. Describe how the data was used to make decisions about program improvement. Describe action steps needed to improve data collection and/or program. Provide a summary of the analysis with references to data.
Provide evidence that the previous plan of action was implemented. Provide evidence that the previous action plan was assessed. Provide results of the implementation of the action plan.
Results are shared with essential stakeholders
(Shared Responsibility)
Lack of or incomplete inclusion of some established criteria.
Evidence that the report was shared with all faculty and advisory committee members where applicable.
Evidence that the report was shared with additional stakeholder groups.
Sample Assessment Plans
Owens— https://www.owens.edu/portrait/index.html
College of Charleston, SC— http://spinner.cofc.edu/~oap/docs.html
Supplemental Instruction
How IR Can Help
Data support and analysis Assessment consulting
Survey development Rubric development Identifying appropriate metrics Identifying existing data sources
References
Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing: Herndon, VA.
Palomba, C. & Banta, T.W. (1999). Assessment Essentials. Planning, Implementing, and Improving Assessment in Higher Education. John Wiley & Sons: San Francisco, CA.
Suskie, L. (2004). Assessing Student Learning: A Common Sense Guide. Jossey-Bass: San Francisco, CA.
Owens Community College, Student Learning Assessment Committee (2009). Building your SLAC assessment plan: Instructions for the SLAC assessment plan template version #1: intranet.owens.edu/committees/outcomes/index.html.
University of West Florida, Center for University Teaching, Learning, & Assessment. http://uwf.edu/cutla/Assessres.cfm.