Practical Perspectives on Measuring Student Success - L. Mercado IGA 2013 FINAL

Practical Perspectives on Measuring Student Success

Leonardo A. Mercado, M.Ed., M.B.A.

“The Big Picture”

Why is assessment SO important?

Proficiency Targets

Standards

• CEFR: Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has.• WIDAGrades 1 – 2, Speaking, Level 1 EnteringName everyday objects depicted visually in real-life contexts (e.g. "paper" in a classroom scene)

Standards

Learning Goals, Objectives, Outcomes, & Standards

Formative vs. Summative Assessment

• Formative: A variety of teacher practices that have the purpose of gathering information in order to enhance learning outcomes.

• Summative: A formal process by which student performance is

measured in order to determine a level of attainment for grading and promotional purposes.

Exercise 1: Formative Assessment

Please offer possible reasons as well as a measure to take for the following situations:

1. The teacher is asking questions in the past tense and most students are responding in the present.

2. At least 50% of the class is not on task. 3. Students are not volunteering for a given speaking activity.

PRINCIPLES ON ASSESSMENT

“Art & Science” Principle

Teaching is the art and science of getting people to learn…

“Art & Science” Principle

Teaching is the art and science of getting people to learn…

“Rehearsal” Principal: Grading

“Rehearsal Principle”

Never grade your students on their ability to carry out a task unless they have had an opportunity to

practice something similar first.

Example: “Rehearsal Principle”

• Course book speaking activity:

Name a nice electronics store. Where is it? What is it like?

• Speaking activity for evaluation:

Name a good restaurant. Where is it? What is it like?

Must share:

- Time allotment- Vocabulary- Grammar- Theme or topic- Nature of task or

activity

“Sampling Principle”: Grading

• When you evaluate students, what do you look for?• How much is enough to measure performance?• What should I focus on?• How long should an in-class activity for grading last?

Sampling Principle: Grading

Before grading your students, make certain you know what you are going to measure, what language is required to

complete the activity task, and what the criteria for success are.

“Feedback” Principle

• What kind of feedback do you give students in the classroom?• Should you give them feedback during a speaking or writing

test?• How much and how specific should feedback on performance

be?

“Feedback” Principle

Feedback should be sufficient so that students understand what did they right and what they could have done better, and how. It should always be delivered on a timely basis.

“Construct” Principle

Have a clear idea as to what an area of evaluation means and what it is measuring.

“Construct” Principle

What does it mean when you are evaluating…

• Reading?• Effort?• Speaking?• Pronunciation?

“Authenticity” Principle

Students should be tested on things that are similar to what they have done in the classroom.

PRACTICAL INSTRUMENTS & IDEAS

Instruments

Should…• lead to “reliable” and “valid” assessments.• be user-friendly and practical for teachers.• be easy to understand for students.

Two Types of Instruments

Analytic • Diagnostic • Focused criteria• Traceable feedback• More objective

Holistic• User-friendly, “easier” to

understand• Comprehensive• Higher level proficiency

Assessment Instrument 1: Speaking

AREASCORE0 1

Accuracy

The student was mostly unable to use the target form(s) successfully or only did so after receiving assistance.

The student was able to use the target form(s) successfully most of the time without assistance.

Communication

The student's language was incomprehensible at least 50% of the time, not in accordance with a successful completion of the task, or ambiguous because of misused vocabulary.

The student's language was comprehensible most of the time, in accordance with the successful completion of the task, and characterized by a correct use of vocabulary.

B02 / Page 39 / Unit 6 / Activity 8

Objective: To ask/answer questions about you/your family’s activities using simple present tense.

EXAMPLES SCORE0 - 2T: Does your mother work?

A: Yes, I do. 0

B: Yes, she does. She work in a hospital 1

C: Yes, she does. She works in a hospital.

She’s a nurse.

2

Assessment Instrument 1: Speaking

• Listen to the student respond to the question. What would you grade him using Assessment Instrument 1?

Assessment Instrument 2: SpeakingCriteria to evaluate performance in Oral Examinations

Rubric Reference Performance Descriptors Rating

3 Very Good / Good

Completes activity or task thoroughly through the use of ample general language resources as well as extensive knowledge of topic and subject matter

Performs communicative function completely according to activity or task Pronunciation is highly intelligible. Fluent most or all of the time with the possibility of having occasional hesitations and

pauses.

+

2 Fair / Average

Completes activity or task by exhibiting variety of general language resources as well as knowledge of topic and subject matter, although details or elaboration may be lacking at times

Performs communicative function completely most of the time according to activity or task

Pronunciation is usually intelligible, although some utterances are difficult to understand.

Pauses and hesitations are noticeable, leading to slower speech.

+

1 Needs reinforcement

Is not able to complete one or more of the activities or tasks, exhibiting a lack of general language resources as well as serious knowledge gaps in relation to the topic or subject matter. Details and attempts at elaboration are scant.

Performs communicative functions according to activity or task, but significant difficulty or unsuccessful attempts are evident at least 50% of the time.

Pronunciation errors clearly affect intelligibility, with many utterances being difficult to understand.

Pauses, hesitations and false starts are frequent and speech is slow overall.

-

0 Unsuccessful

Cannot perform most or any of the activities or tasks. Cannot perform most or any of the communicative functions as required by the

activities or tasks. Pronunciation is rarely intelligible. Speech is halted, inarticulate and fragmented making communication impossible.

-

Assessment Exercise 2

• Listen to the student present her personal portfolio. What would you grade her using Assessment Instrument 2?

Assessment Instrument 3: Writing

AREASCORE0 1

Accuracy

The student was mostly unable to use the target form(s) successfully or only did so after receiving assistance.

The student was able to use the target form(s) successfully most of the time without assistance.

Communication

The student's language was incomprehensible at least 50% of the time, not in accordance with a successful completion of the task, or ambiguous because of misused vocabulary.

The student's language was comprehensible most of the time, in accordance with the successful completion of the task, and characterized by a correct use of vocabulary.

1. In the summer I visiting my family and I went to the beach, then I shopping. After I was eating in a restaurant, the food delicious. My family was happy I can went.

2. I was having a great time at the beach when suddenly I received a call from my brother. He told me that my dog was sick. I returned very fast because I was very worried. When I arrived the dog was sleeping the doctor went to see him and gave him medicine.

3. Something happened when I was visiting my family I went to the beach and I was having a good time, when I received a call that my dog is sick. I was going very fast. I worry very much.

BASIC 12 (Unit 12) pages 79-80 WRITING EXAMPLEObjective: To use the past continuous and simple past in short paragraph

about something that happened to you. Use adverbs.

Assessment Instrument 4: Writing

Assessment Design

• LGOOS: Link• Medium: Activity /

Task / Prompt• Sample Type & Size• Authenticity• Timing• Interpretation:

feedback / grades

Assessment Design

Please choose one of the following alternatives and design an assessment accordingly:• To order food in a restaurant using will have / would

like• To guess the meaning of words from context.

WRAPPING IT UP

General Recommendations

• Plan ahead• Time yourself• Inform, no unpleasant

surprises• Be a good listener• Give helpful feedback; don’t

de-motivate• Link performance to LGOO’s• Self-assess your

assessments• Clean the slate

Conclusion

• Assessment & Evaluation are not easy

• We owe it to our students to be as objective as possible. A lot is at stake.

• Improve your techniques and instruments over time. • More knowledge about the area, the easier the task.

THANK YOU!!!!

[email protected]