Northern Illinois University March 2009 Susan Hatfield Winona State University [email protected] Leave No Outcome Behind: Teaching, Learning & Assessment.

Northern Illinois University

March 2009

Susan HatfieldWinona State University

[email protected]

Leave No Outcome Behind:Teaching, Learning &

Assessment

Plan

Assessment

Teaching

Learning

Assessment

A few things you need to know

1. Assessment is about student learning

Assessment Learning

National Norms

Trend Lines

Instrument Driven Outcome Driven

Relational Data

Targets & Goals

AnalysisCollection

How do we compare? What does it mean?

2. It isn’t going away

3. You’re probably already doing it

Evaluation Assessment

QuizzesCount toward final grade

Used to see if students understand

TestsScored and returned

Scored, tabulated. returned & discussed; adjustments to syllabus

RubricsReturned to studentswith grade

Returned after being aggregated & analyzed; adjustments to syllabus

4. The rules have changed

INDIRECTMEASURES

1990 20102000

Indirect Measures of Learning

• Alumni, employer, and student surveys (including satisfaction surveys)

• Exit interviews of graduates and focus groups graduate follow up studies

• Retention and transfer studies• Length of time to degree• ACT scores • Graduation and transfer rates• Job placement rates

DIRECTMEASURES

INDIRECTMEASURES

1990 20102000

Direct Measures of Learning

• Capstone experience• Standardized tests• Performance on national licensure certification or

professional exams • Locally developed tests • Essay questions blind scored by faculty• Juried review of senior projects• Externally reviewed exhibitions performances• Evaluation of internships based upon program

learning outcomes

PROCESS MEASURES

OUTCOMEMEASURES

1990 20102000

PROGRAMASSESSMENT

CLASSROOMASSESSMENT

1990 20102000

INSTITUTIONAL RESPONSIBILITY

PROGRAMRESPONSIBILITY

1990 20102000

INSTITUTIONALEFFECTIVENESS

STUDENTLEARNING

1990 20102000

5. It’s a bumpy ride

Evolutionary Trajectories

Beginning

Mature

MakingProgress

0402 03 05 06 07 08 09

6. It’s all about student learning outcomes

Program LevelStudent Learning Outcomes

• Students should be able to<<action verb>> <<something>>

Program LevelStudent Learning Outcomes

• Learner Centered• Specific• Action oriented• Cognitively appropriate for the program level

KNOWLEDGECOMPREHENSION

APPLICATIONANALYSISSYNTHESISEVALUATION

CiteCountDefineDraw

IdentifyList

NamePointQuoteRead

ReciteRecordRepeatSelectState

TabulateTell

TraceUnderline

AssociateClassifyCompareComputeContrast

DifferentiateDiscuss

DistinguishEstimateExplainExpress

ExtrapolateInterpolate

LocatePredictReportRestateReview

TellTranslate

ApplyCalculateClassify

DemonstrateDetermineDramatize

EmployExamineIllustrateInterpretLocate

OperateOrder

PracticeReport

RestructureScheduleSketchSolve

TranslateUse

Write

AnalyzeAppraiseCalculate

CategorizeClassifyCompareDebate

DiagramDifferentiateDistinguish

ExamineExperiment

InspectInventoryQuestionSeparate

SummarizeTest

ArrangeAssemble

CollectComposeConstruct

CreateDesign

FormulateIntegrateManageOrganize

PlanPrepare

PrescribeProduceProposeSpecify

SynthesizeWrite

AppraiseAssessChoose

CompareCriticize

DetermineEstimateEvaluate

GradeJudge

MeasureRankRate

RecommendReviseScoreSelect

StandardizeTest

Validate

Lower division courseoutcomes

KNOWLEDGECOMPREHENSION

APPLICATIONANALYSISSYNTHESISEVALUATION

CiteCountDefineDraw

IdentifyList

NamePointQuoteRead

ReciteRecordRepeatSelectState

TabulateTell

TraceUnderline

AssociateClassifyCompareComputeContrast

DifferentiateDiscuss

DistinguishEstimateExplainExpress

ExtrapolateInterpolate

LocatePredictReportRestateReview

TellTranslate

ApplyCalculateClassify

DemonstrateDetermineDramatize

EmployExamineIllustrateInterpretLocate

OperateOrder

PracticeReport

RestructureScheduleSketchSolve

TranslateUse

Write

AnalyzeAppraiseCalculate

CategorizeClassifyCompareDebate

DiagramDifferentiateDistinguish

ExamineExperiment

InspectInventoryQuestionSeparate

SummarizeTest

ArrangeAssemble

CollectComposeConstruct

CreateDesign

FormulateIntegrateManageOrganize

PlanPrepare

PrescribeProduceProposeSpecify

SynthesizeWrite

AppraiseAssessChoose

CompareCriticize

DetermineEstimateEvaluate

GradeJudge

MeasureRankRate

RecommendReviseScoreSelect

StandardizeTest

Validate

Upper divisionCourse / Program

outcomes

7. Assessment requires exertion and intention

Exertion without Intention

Intention without Exertion

Intention and Exertion

Teaching and Learning

Campus Climate

Meteorological metaphor Seasonal Changing Uneven Perpetuated by individuals Something an university HAS

Campus Culture

Anthropological Metaphor Deep rooted Defended Perpetuated by structures,

policies, procedures, behaviors What a university IS: Who you

ARE

Culture

Cultures cannot be consciously created -- they can be promoted, but an college’s culture arises from the interaction of multiple variables.

Evidence of Culture

Vocabulary Metaphors Legends, stories, mythologies,

folklore Symbols Rites & Rituals

Evidence of Culture

Written Materials Formal & Informal Policies and

Procedures Organizational Structure Social Knowledge Reward Structure

Student Learning: Climate or

Culture?

Focus on Student Learning:Climate or Culture?

Temporary Ongoing

External Internal

Accreditation Improvement

Surface Embedded

Personality Driven Structurally Driven

Isolated Pervasive

Focus on Student Learning:Climate or Culture?

Temporary Ongoing

External Internal

Accreditation Improvement

Surface Embedded

Personality Driven Structurally Driven

Isolated PervasiveLEADERSHIP

KNOWLEDGE

SUPPORT & RESOURCES

COMMITMENT

Reflection

Does NIU have a climate or a culture of student learning?Must exceed the Difficulty of the Task

Moving toward a Culture of Student

Learning

A Culture of Learning requires a fundamental shift away from a focus on Teaching

TEACHER CENTERED

LEARNER CENTERED

Knowledge is communicatedfrom professor to student

Students passively receive information

Emphasis on acquisition ofknowledge in a specific context

Professor’s role is to be primary information giver and evaluator

Students construct knowledgethrough gathering and

synthesizing information

Students are actively involved

Emphasis on applying knowledgeto new situations and issues

Professors role is to coach and facilitate

Teaching and assessing are separate

Teaching and assessing areintertwined

TEACHER CENTERED

Assessment is usedto monitor learning

Emphasis is on theright answers

Desired learning isassessed indirectly

Focus is on a single discipline

Assessment is used topromote and diagnose learning

Emphasis is on generatingbetter questions and learning

from errors

Desired learning is assessed directly

Approach is compatible with interdisciplinary learning

Culture is competitiveand individualistic

Culture is collaborative,cooperative, supportive

LEARNER CENTERED

Reflection

What elements of the teacher-centered focus are embedded in the culture of NIU?

What elements of the learner-centered focus are embedded in the culture of NIU?

Shifting from a Teaching Focused to a Learning Focused

Institution

Reflection on Teaching

How did you get to be a good teacher?

What Good Teachers Do

Understand learning theoryPrepare carefullyIdentify learning outcomesConsider instructional strategiesRespect their studentsAssess their students and themselves

Bane, K. What Good Teachers Do

Facilitating Learning: The Seven Principles for Good Practice

1. Student-Faculty Contact 2. Cooperative Learning 3. Active Learning 4. Prompt Feedback 5. Time on Task 6. High Expectations 7. Respect for Diverse Talents and Ways

of Learning

Assessment should be viewed as“ a strategy of inquiry into student

learning and actions taken to improve student learning.”

According to the HLC:

From the HLC powerpoint presentation Making a Difference in Student Learning: Assessment as a Strategy of Inquiry

1. How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students?

2. What evidence do you have that students achieve your stated learning outcomes?

3. In what ways do you analyze and use evidence of student learning?

Guiding Questions - Student Learning

4. How do you ensure shared responsibility for student learning and assessment of student learning?

5. How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning

6. In what ways to do you inform the public and other stakeholders about what and how well your students are learning?

Guiding Questions - Student Learning

Relatively Easy Changes Faculty Can

Make to Enhance Learning

Course Design Instead of…. Try…….

Letting the textbook determine the content of the course

Focus the course on the specific learning outcomes for the course

Instruction Instead of…. Try…….

Focusing on “getting through the material”

Using Classroom Assessment Techniques to make sure students understand before moving on to the next concept

Feedback Instead of…. Try…….

Trusting students understand when they don’t seem to have any questions.

Asking students to demonstrate what they have learned through a one minute paper or summary and the end of the class period.

Quizzes Instead of…. Try…….

Giving points for quizzes that count toward the final grade.

Use quizzes frequently to see how students if students are progressing in their learning.

Exams Instead of…. Try…….

Returning exams without discussion

Conducting an analysis of which items students missed, and figure out how to help them understand.

Rubrics Instead of…. Try…….

Returning the rubric to the students with your evaluation of their performance

Aggregating your student’s performance on each dimension of the rubric to see in what areas they are having problems.

Course Structure Instead of…. Try…….

Moving forward even if students are lagging behind.

Readjusting the syllabus or offering additional resources and support for those students who haven’t yet grasped the material.

Program Level Assessment Instead of…. Try…….

Having a huge list of every single concept to be learned in the program

Identifying the top five or six things you want students to know or be able to do and map those outcomes to the courses in the curriculum.

Assessment Instead of…. Try…….

Conducting an assessment program without telling the students about it…

Making the learning outcomes for the course and program explicit to the students so they know what is expected of them.

Assessment Instead of…. Try…….

Collecting tons of data

Focus on a few key learning outcomes and collect, discuss and analyze that data.

Assessment Instead of…. Try…….

Attempting to revise an entire course.

Take one class or lesson (this semester!) and try to approach it differently.

Reflection

What can you do to help transform NIU toward a culture of student learning?

Northern Illinois University

March 2009

Susan HatfieldWinona State University

[email protected]

Leave No Outcome Behind:Teaching, Learning &

Assessment