Tia A. Minnis Assistant Director for Assessment Office of Planning Assessment and Research
2013 TAM | OPAR
A faculty Driven Process
Student Learning Outcomes (SLOs) Are and Are NOT!
Learning Outcomes at Different Levels. Defining Assessment of SLOs.
Why Do We Assess Student Learning? Faculty Role in Developing SLOs. Steps in Developing & Assessing SLOs.
Group Activity. Questions, Comments, Conversation.
2013 TAM | OPAR
Statements that define the goals of the learning experiences.
What a student should be able to do, know, or value/think as a result of engaging in that learning experience. Expressed as knowledge, skills, and attitudes or values.
Define impact – how the student has changed.
2013 TAM | OPAR
The same as satisfaction with education, professors, facilities, activities, or experiences
The same as outputs, such as grades, GPA, retention rate, graduation rate, etc.
2013 TAM | OPAR
Why are grades not enough? Grades can be subjective
Grades may reflect student behaviors (class participation, attendance, cooperation, etc.)
Grades may be inconsistent between instructors
Grades are not enough to say the we know that the students learned
Student Achievement is not the same as student learning
2013 TAM | OPAR
2013 TAM | OPAR
•Students who complete this course can calculate and interpret a variety of descriptive and inferential statistics
Course Level
(CLOs)
• Students who complete the Psychology program can use statistical tools to analyze and interpret data from psychological studies
Program Level
(PLOs) • Students who graduate
in business can apply quantitative methods to business-related problems
School Level (SLOs)
•Graduates from our campus can apply quantitative reasoning to real-world problems.
Institutional Level (ILOs)
The systematic collection, review and use of information about educational programs undertaken for the purpose of improving student learning and development.
-Ted Marchese
Assessment is a systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions that affect student learning.
-Barbra E. Walvoord, 2010 2013 TAM | OPAR
Assessment is the ongoing process of: Establishing clear, measurable expected outcomes of
student learning Ensuring that students have sufficient opportunities to
achieve those outcomes Systematically gathering, analyzing, and interpreting
evidence to determine how well student learning matches our expectations
Using the resulting information to understand and improve student learning
-Linda Suskie, 2009
2013 TAM | OPAR
Establishing measurable SLO
Provide Sufficient Learning
opportunities
Systematically Gather, Analyze, & Interpret Evidence
of Student Learning
Using The Resulting to
Improve Student Learning
2013 TAM | OPAR | CAU
On-going Process
Regional Accreditation (SACS COC) Core Requirements: 2.5 and 2.12 Comprehensive Standards: 3.3.1.1 and 3.5.1 Federal Requirements: 4.1
School and Program Accreditation SAS: NASPA, ABET, ACEJMC, APA SOB: AACSB, SOE: NCATE, CACREP, SSW: CSWE
2013 TAM | OPAR
Improve the quality of student learning experience Curriculum assessment and change Evaluation of instruction Effective use of resources Improved Communication Advising Tools Improving promotional materials Program Validation and Credibility Provide accountability to stakeholders Targets for assessment and accreditation Institutional Effectiveness
2013 TAM | OPAR
We assess SLOs for two primary reasons: Improvement - Assessment can improve the quality on
teaching learning, program and services, and planning and decision making.
Accountability – Assessment can validate current programs, services, and teaching and learning efforts and thereby secure continued support for them by demonstrating their effectiveness to concerned audiences.
Source: adapted from Linda Suskie (Assessing Student Learning, 2009)
2013 TAM | OPAR
Collaborative Faculty-Driven Process Faculty Role: Identify student learning outcomes Identify the Means of Assessment (Tool, Method,
& Criteria) Analyze results, determine appropriate
improvements, and write annual assessment reports.
2013 TAM | OPAR
Faculty Role Hindrance to Participation
Insufficient Time Compromised Autonomy Meaningless – Data gather that no one will use Complicated Process
Utilization-Focused Evaluation (Paton, 2008) Evaluation judged by utility and actually use (Useful and Used) Focus on intended use by intended users Collaborative effort by the users for their intended uses Evidence need to identify program strengths and areas of
concern
2013 TAM | OPAR
Faculty Benefits: Shared understanding and commitment
Inform Pedagogical Practices Changes to the curriculum, requirements, programmatic
structure, etc.
Secure Needed Resources Changes to funding, planning, and policies that support
student learning
Faculty Development
2013 TAM | OPAR
WHEN DEVELOPING YOUR STUDENT LEARNING OUTCOMES REMEMBER:
Student learning outcomes (SLOs) are statements that specify what students will know, be able to do or demonstrate and think or value, after successfully completing the program/curriculum. Focus on Mastery of Program Content.
Ask yourself: what are the most important things a student should know, be able to do or demonstrate and think or value after completing my program?
Relate the SLO to the overall goal and objectives of the academic program.
SLO are expressed as knowledge, skills, and attitudes or values.
2013 TAM | OPAR
WHEN DEVELOPING YOUR STUDENT LEARNING OUTCOMES REMEMBER:
SLOs specify an action by the student that must be observable (able to be demonstrated) and measurable!
SLOs serve as benchmark assessments for completion of your
program! SLOs are related to the mission of the program or department,
school, and institution. Identify 5-8 student learning outcomes.
2013 TAM | OPAR
THE SLO’S SHOULD INCLUDE THE FOLLOWING:
An Action Verb that results in overt behaviors that can be observed and measured.
Sample action verbs are:
Analyze, Apply, Argue, Arrange, Assemble, Assess, Calculate, Categorize, Choose, Classify, Compare, Compile, Compute, Create, Criticize, Critique, Defend, Define, Demonstrate, Describe, Design, Develop, Differentiate, Discuss, Distinguish, Estimate, Examine, Explain, Formulate, Identify, Illustrate, Indicate, Interpret, Label, List, Locate, Manage, Memorize, Order, Operate, Organize, Plan, Practice, Predict, Prepare, Propose, Question, Rate, Recognize, Repeat, Report, Reproduce, Review, Revise, Schedule, Select, Solve, State, Translate, Use, Utilize, Write
Avoid verbs that are unclear and call for covert, internal behavior which cannot
be observed or measured. Appreciate, Know, Learn, Understand, Become aware of, Become familiar with….
2013 TAM | OPAR
The verb used may vary depending on the domain of learning and the appropriate level of thinking
2013 TAM | OPAR
THE SLO’S SHOULD INCLUDE THE FOLLOWING:
Language that indicate the BIG PICTURE rather than nuts and bolts. Does the language describe what students can do, know, think or
value as a result of their matriculation (observable/demonstrable)?
Does it ask students to apply what they’ve learned by producing something?
Does it address student competency rather than content coverage?
2013 TAM | OPAR
Each SLO should be S.M.A.R.T • Specific • Measurable (Result-Oriented) • Attainable (Realistic) • Relate to the mission (Relevant) • Time-bound
2013 TAM | OPAR
Each SLO should be S.M.A.R.T
The SLO should be Specific to the program The SLO’s should be specific to the program. Include in clear and
definite terms the expected abilities, knowledge, values and attitudes a student who graduates from the program is expected to have.
The SLO should be Measurable and result oriented
The intended outcome should be one for which it is feasible to collect accurate and reliable data.
The intended outcome should produce results that can identify problems and assist with program improvement.
2013 TAM | OPAR
Each SLO should be S.M.A.R.T
The SLO should be Attainable and realistic Consideration was given to the available resources (e.g., staff,
technology, assessment support, institutional level surveys, etc.) in determining whether the collection of data for each student learning outcome is a realistic expectation.
There are a realistic proportion of students able to achieve the stated specific level (e.g., 80% of the students will pass the written portion of the comprehensive exam with a minimum of 75 points on the assessment rubric).
2013 TAM | OPAR
Each SLO should be S.M.A.R.T
The SLO should be Related and relevant to the department’s/schools’ mission It should focus on intended outcomes that are critical to the
program and its mission statement.
The SLO should be Time-bound When defining the outcomes, it is important to indicate when the
outcome will be assessed (e.g. each semester, spring, annually, etc.), and for what period of time.
2013 TAM | OPAR
WHEN DEVELOPING YOUR STUDENT LEARNING OUTCOMES USE:
Appropriate Sources for SLOs Mission statements Written descriptions of the program Employers Your “ideal” graduate Teaching goals inventory Accreditation agencies Professional/certification exams
Established Baseline Data Set goal that are attainable/realistic
2013 TAM | OPAR
When writing student learning outcomes try using templates similar to these: Students will be able to ___________ (ACTION VERB- know, do,
value) _______________ (CONTENT AREA).
As a result of students participating in ________ (PROGRAM), they
will be able to _____________ (ACTION VERB- know, do, value) _________________ (CONTENT AREA).
______________ (PROGRAM) seniors will be able to
_______________ (ACTION VERB- know, do, value) _____________ (CONTENT AREA).
2013 TAM | OPAR
WHEN ASSESSING YOUR STUDENT LEARNING OUTCOMES:
Use Multiple Measures or Means of Assessment: Direct Assessment Methods as the primary MOA Indirect Assessment Methods as the secondary MOA
At least 1 direct measure must be used Qualitative/Quantitative Formative/Summative
2013 TAM | OPAR
Direct Measures (all students)
– Capstone projects/senior projects
– Samples of student work (Portfolios)
– Examinations – Oral Presentation – Observations of student
behavior (Internships) – Theses/Dissertations – Pre- and post-tests
Indirect Measures (sample)
– Alumni, employer, and student surveys
– Questionnaires – Focus groups – Interviews
2013 TAM | OPAR
Appropriate measures of Learning Outcomes include:
WHEN ASSESSING YOUR STUDENT LEARNING OUTCOMES :
Develop Scoring Guide/Rubric Identify what competencies are assessed Identify how scores are assigned to each competency Ensure that appropriate assessment measures are used to assess
the degree to which learning is attained
Identify the Criteria for Success Overall Passing Score (NOT Grade) % of students expected to achieve passing score
2013 TAM | OPAR
WHEN ASSESSING YOUR STUDENT LEARNING OUTCOMES :
Identify the course/s where learning and/or assessment will occur Identify how the course relates to the SLO
Introduced; Reinforced; Assessed Curriculum Mapping
Assessment reports should: Indicate how the results will be used for improvement Include follow-up action plans when the criteria are not met Be generated annually and evaluated by appropriate faculty/staff members Used in periodic program reviews – Longitudinal Assessment Stored centrally
2013 TAM | OPAR | CAU
Developing Student Learning Outcomes is: Collaborative Faculty-Driven Process
Student Learning Outcomes should: Be expressed as knowledge, skills, and attitudes or values that students acquire as a
result of engaging in that learning experience. Focus on program content mastery (not individual course learning outcomes) Define impact and specify an action by the student that is observable (able to be
demonstrated) and measurable Be assessed using appropriate direct/primary MOA and when applicable,
indirect/secondary MOA
Assessment of SLOs should: Be on-going, not episodic Address the degree to which learning has occurred and be distinct from grades Should indicate how the results will be used for continuous improvements
2013 TAM | OPAR | CAU
Instructions: o Divide into groups with members of your program/department o Use the SLO exercise sheet provided to develop 3 outcomes (one for
each domain). o Use your check sheet to review your SLO o Select a spokesperson from the group to share your SLO
2013 TAM | OPAR
2013 TAM | OPAR
Tia A. Minnis Assistant Director for Assessment
Office of Planning Assessment and Research [email protected]
404-880-6294 (Tel) 404-880-8737 (Fax)