Using Methodology Feedback to Improve Courses: A … · To consider how methodology feedback might be used ... free of any bias in approach, method, or technique. 17. ... Second Language

Using Methodology Feedback to Improve Courses: A Case StudyPaul C. Corrigan

Introduce ILOs

Assessment (a.k.a. feedback) Group Activity 1: Warm Up Purposes and Types of Assessment Group Activity 2: Effectiveness of Assessment

Methodology Feedback Group Activity 3: Warm-up What is Methodology Feedback? Group Activity 4: Critical Analysis and Application

Case Study: Methodology Feedback

Summary and Conclusion

Outline of Today’s Seminar

22

Intended Learning Outcomes ILO 1

To review and gain further knowledge about assessment/ feedback

ILO 2 To critically examine the concept of ‘methodology

feedback’ ILO 3

To consider how methodology feedback might be usedfor your own teaching

ILO 4 To examine a case study of methodology feedback

Introduction

33

ASSESSMENT

4

In groups of about 3-4, discuss these questions and be ready to share your ideas with the rest of the seminar participants

In your courses:1. What types of assessment/feedback do you use?2. What are your purposes of assessment?

Activity 1 – Warm Up

55

Do you recognize these from your discussion of assessment?Some definitions of assessment terminologyDiagnostic Indicates how current performance differs from expected

performance. Can be used to identify problems that a pupilmay be experiencing.

Formative An assessment that helps pupils learn; results in actions thatare successful in closing the gap between current andexpected performance.

Summative An assessment that is used to certify or record end of courseperformance or predict potential future attainment; the finalproduct of a unit or course; an examination grade.

Evaluative Assessment information that is used to judge the performanceof schools or teachers; league tables.

6

Purposes and Types of Assessment It is not the assessment itself that is diagnostic,

formative, summative or evaluative but the waythat the information collected is used. For example, descriptive or prescriptive – but could

be others

As William and Black (1996, in Weeden, Winterand Broadfoot, 1996) suggest, ‘These terms aretherefore not descriptions of kinds of assessmentbut rather of the use to which information arisingfrom the assessments is put.” 7

Two broad categories of purpose areDescriptive and non-judgmental ORPrescriptive and judgmental

Some Purposes of Assessment & Type

88

Teachers monitor students’ learning For example, a quiz for this purpose

Could be judgmental/prescriptive Could be descriptive and non-judgmental to give

teacher an idea of what needs to be re-taught


99

Students monitor their own learning For example, answering questions at the end of a

chapter for this purpose; Probably descriptive and non-judgmental


101010

Teachers’ effectiveness is monitored For example, a teaching feedback form for this

purpose Could be descriptive to show areas where improvement

could be made Could be judgmental – e.g., contract renewal

Social purposes Institutions and students are assessed and described

for the other benefits for the society For example, university league tables, recruitment of

graduates


1111

Activity 2 In groups of about 3-4, discuss this question

and be ready to share your ideas with the rest of the seminar participants:

How can you tell if that type of assessment is providing the information you need for a purpose?

12

Do you recognize any of your ideas from the discussion in these concepts?Definition of validity, reliability and manageability

Validity of assessment method

To what extent does an assessment measure whatit sets out to measure?

Reliability of assessment

How consistent is the measurement of resultsbetween different teachers, between differenttest situations, etc.?

Manageability of assessment

Can the assessment be conducted without toomuch disruption to normal teaching?

1313

Course/Programme Assessment Purpose

Descriptive: to understand what is taking place, perhaps to make changes without judging a person

Prescriptive: to judge quality and prescribe changes Etc.

Type survey forms oral feedback from tutors hearsay focus groups with students focus groups with other stakeholders Etc. 14

Course/Programme Assessment How can you tell if the types of assessment just

shown are valid, manageable and reliable? Are some more so than others?

15

METHODOLOGY FEEDBACK

16

Activity 3 In groups of about 3-4, discuss these

statements: which of them do most persons inyour group believe to be true? Materials are only as good as the teachers who use

them The attempt to produce ‘teacher proof’ materials is

both futile and undesirable Materials should be ‘neutral’; that is, they should be

free of any bias in approach, method, or technique

17

Methodology Feedback Methodology Feedback assumes that the same

materials could be taught in different ways But effectively or not is another question

In collecting feedback, both materials andteaching methods thus should be taken into account

Two major factors to take into account forcourse/programme quality are materials andmethods

18

Methods Techniques

Materials

Methodology

Methodology: Methods and Materials

1919

Prescribed WHAT Within an OBTL framework the content within teaching

and learning activities (TLA) which are constructivelyaligned with course intended outcomes

Actual WHAT The actual content that students were exposed to

during the course

Effective HOW Effective methods used to teach / students to learn the

content Is linked to students learning strategies

Methodology: The WHAT and HOW

20

X

X

X X

Methodology Feedback Table

21

Actual “What”

Effective “How”

Prescribed “What”

Methodology




X

X

X X

Methodology Feedback Table

22

Actual “What”

Effective “How”


Methodology

What is supposed to be taught isbeing taught and in an effectiveway

What is supposed to be taught isbeing taught but not in aneffective way

What is supposed to be taught isnot being taught but what is beingtaught is taught effectively

What is supposed to be taught isnot being taught but what isbeing taught is not being taughteffectively




1. The prescribed content is being covered2. The prescribed content is being covered, but the

methods are not consistent with students’ learningstrategies

3. The prescribed content is not being taught, butthe methods are consistent with students’ learningstrategies

4. The prescribed content is not being taught, andthe methods are not consistent with students’learning strategies

What can a Methodology Feedback Table can tell us?

23

Remediation1. Change the teaching methods to coordinate with students’

learning strategies (unless you want the students to learn in anew way)

2. Re-examine content for relevancy If relevant, then explain to tutor(s) changes that should be

made in the actual content presented If not relevant, revise the content according to appropriate

academic procedures3. The third & fourth situations Is the content relevant or not relevant? (See situation above) Are the methods preventing the content from being

covered?

What can a Methodology Feedback Table can tell us?

24

Sometimes, the HOW is part of the WHAT that is to say the methods is part of the content

Why is this so? In teacher training or professional development

courses, methods are modeled for the participants That is, methods such as role play, teacher fronted,

student-centered methods are modeled so that thestudents can learn them experientially and laterexplicitly reflect on them

Methodology

25

Literature Support for Methodology Feedback Sax (1997) implies that feedback on both

materials and methods should be obtained forprogramme evaluation

Nunan (1988 ) says a key question in assessmentof a programme is “Are the materials, methods,and activities consistent wit the pre-specifiedobjectives?” which he places this under the heading“Methodology”

From these writers it is inferred, therefore, thatmethodology is comprised of two majorcomponents - materials and methods

26

Other Support for Methodology Feedback Our teaching methods are not ‘neutral’ They reflect conscious and/or unconscious

assumptions about teaching and education Classical humanism Approach

Content is most important

Reconstructionism Approach The teacher is just a curriculum implementator

Progressivism ApproachMethod is seen as very important

27

Edward Anthony (1963) described the relationship among approach, method and technique

28

Approach (Education Philosophy) / Method / Technique

28

Method• Techniques• Techniques• Techniques




29

APPROACH

29


For example, in a given situation: The Progressive Approach is in use Discussion is a method used in that approach Requesting students to first discuss with one partner in

English before talking to the class is a discussion technique

30

Activity 4 Discuss what are the pros and cons of Methodology

Feedback? Discuss the main implicit or explicit approach in one of

your courses. Are the teachers seen mostly as implementors?Or is method given a lot of importance?Or is content the most important thing?

Within that main approach, what methods andtechniques are used?

How would you gather valid, reliable, andmanageable feedback on both the content andmethods?

31

CASE STUDY: METHODOLOGY FEEDBACK

32

Case Study Outline Background Pilot Programme Preliminary Analysis of Feedback for each Course

33

Background:Hong Kong’s Quality Enhancement Grant Scheme Promoting Teaching Effectiveness in English as the Medium of

Instruction in Hong Kong’s Self-financing, Post-secondary Sector

1

• Research the general and special needs of teachersusing English as the Medium of Instruction in the self-financing, post-secondary education sector in Hong Kongand interpret the results within a critical literature review

2

• Develop printed and web resources for a pilotprogramme of four mini-courses in teaching in themedium of English for teachers in the self-financing post-secondary education sector.

34

34

Background: Hong Kong’s Quality Enhancement Grant Scheme

3• Offer the pilot programme to teachers in the sector

and assess the results to make improvement

4• Offer the revised programme on a wider scale

and assess the results

5• Offer the revised programme on a wider scale

and assess the results

35

35

Pilot Programme:Structure, Registrants and Dates Four Courses

Second Language Learning and the Classroom Communicative Teaching Methods and Techniques Language Awareness Developing Curriculum and Materials in English

20 registered attendees Post-secondary teachers in different subject areas

August 2010

36

Pilot Programme: CILOs Second Language Learning and the Classroom

1. Demonstrate knowledge of recognized approaches in SLA

2. Analyze features of those approaches which are implicit or explicit in their classroom methods and techniques

3. Synthesize a coherent approach to second language learning for their classroom

4. Develop a strategy for applying a coherent approach.

37

Pilot Programme: CILOs Language Awareness

1. Demonstrate awareness of British English, American English, and World Englishes, including ‘Hong Kong English’

2. Apply vocabulary learning skills3. Demonstrate awareness of common writing errors and

their repair at the sentence and discourse levels4. Demonstrate awareness of common pronunciation and

intonation errors and their repair5. Describe and apply sociolinguistic appropriateness in

different communicative contexts.38

Pilot Programme: CILOs Communicative Teaching Methods and Technique

1. Describe a range of communicative teaching methods and techniques

2. Identify which methods and techniques are suitable for their subject.

39

Pilot Programme: CILOs Developing Curriculum and Materials in English

1. Describe how to apply existing communicative methods and techniques to curriculum and materials development in their subjects

2. Describe how to synthesize new communicative methods and techniques to curriculum and materials development in their subjects

3. Develop lesson plans which employ, and justify the use of, such methods and techniques

40

Pilot ProgrammeEmbedded Methods: Second Language Learning and the Classroom

Teacher fronted PowerPoint presentation Preview questions, presentation of materials

Small group discussions Consolidation, elaboration, critical thinking, application

Debate

Small group discussions Consolidation, elaboration, critical thinking, application

Small group discussions and presentations with visual aids Consolidation, reformulation, cooperative learning

4141

Pilot ProgrammeEmbedded Methods: Second Language Learning and the Classroom

Jigsaw reading

Quizzes by students Review materials, draft quiz, administer / take it, review scores,

seek clarification

Student presentations of course units

Written reflection

4242

Pilot ProgrammeEmbedded Methods: Communicative Teaching Methods and Techniques

4343


Survey questionnaire Read survey, complete survey, discuss answers in small group to

raise self-awareness

Quizzes by students Review materials, draft quiz, administer / take it, review scores,

seek clarification

Pilot ProgrammeEmbedded Methods: Communicative Teaching Methods and Techniques

4444

Groups discussion Discuss, evaluate and apply

Debate Reading Extrapolating and applying

Short essay Group presentation on course units Short reflective essay

Pilot ProgrammeEmbedded Methods: Language Awareness

4545


Video Preview, watching the video, review and application

Debate Group writing Look-and-Describe Experiential method: Reading Determining the meaning of vocabulary

Pilot ProgrammeEmbedded Methods: Language Awareness

4646

Video Teacher summary, student prediction, sequencing of actions,

cloze activity, role play

Imagine-and-speak Read-and-lookup Student presentation of course units Short reflective essay

Pilot ProgrammeEmbedded Methods: Developing Curriculum and Materials in English

4747


Survey questionnaire Read survey, complete survey, discuss answers in small group

to raise self-awareness

Group Discussion Discuss, evaluate and apply

Practicum Students develop and deliver lessons in cooperating what

they learnt in the programme

Student developed quiz Short reflective essay

Methodology Feedback Form / Sample

48

Case Study: Preliminary Analysis

of Feedback for each Course

49

Second Language Learning and the Classroom Quantitative Feedback

Materials: Satisfactory to Good Teaching: Satisfactory to Good

Qualitative Feedback

Too much material Too theoretical More practical ideas for EMOI Discussions and Peer sharing were helpful

Remediation

Remedy1. Revise course materials to proceed from concreteclassroom cases to theory

Remedy2. Make certain students understand the Intended LearningOutcomes of this course, and not confuse them with other courses50

Quantitative Feedback Materials: Good Teaching: Good to Very Good

Qualitative Feedback Materials were fine, but too many High praise for the teacher

Remediation Remedy1.

Make materials more concise

51

Communicative Teaching Methods and Techniques

Quantitative Feedback

Materials: Satisfactory to Good Teaching: Good to Very Good


Good interactive methods but not enough content Liked specific strategies but would like more strategies Material appropriate for some, too easy for others

Remediation

Remedy1. Introduce more material with additional strategies Remedy 2. Possibly stream participants by language proficiency or

scaffold stronger with weaker for peer-teaching/learning

52

Language Awareness

Quantitative Feedback

Materials: Satisfactory to Good Teaching: Good to Very Good


Materials not used much Materials not enough High Praise for teacher’s method

Remediation

Remedy1. Provide and review with tutor the material earlier and how to use it.

53

Developing Curriculum and Materials in English

Thoroughly explain the rationale of the entireprogramme curriculum to teachers of the coursesduring the interviews

Ensure, when hired, that teachers understand howthe specific courses fit together in the entireprogramme

Thoroughly explain to teachers how to use thematerials for each course, including the importanceof modeling methods as part of the content

54

Case Study: Preliminary Analysis of Feedback for the Programme

Summary & Conclusion Methodology Feedback includes feedback on both

course teaching methods and materials It provides richer information for making decisions

about how to revise courses than traditional feedback

It might be beneficial to try it, at least informally, for a course and see how useful it is

Thanks to: The participants in the pilot programme EDGE for inviting me You for attending Ms. Judy Lip for assisting with the research

Anthony, E. M. (1963). Approach, method and technique.English Language Teaching, 17, 63-67.

Broadfoot P. M. (1996). Education, Assessment andSociety. USA: Open University Press.

Broadfoot, P. M., Weeden, P. & Winter, J. (2002).Assessment: What’s in it for schools?. London:RoutlegdeFalmer.

Nunan, D. (1988). The Learner-Centred Curriculum.Cambridge: Cambridge University Press

Sax, G. (1997). Principles of Educational andPsychological Measurement and Evaluation. USA:Wadsworth Publishing Company.

References

5757

Tables from Slide 6 & 13 cited from Broadfoot, P. M.,Weeden, P. & Winter, J. (2002). Assessment: What’s in it forschools?. London: RoutlegdeFalmer.

Slide 23 adapted from Nunan D. (1988). The Learner-Centred Curriculum. Cambridge: Cambridge UniversityPress.

Statement no. 1&2 from Slide 17 cited from Nunan D.(1988). The Learner-Centred Curriculum. Cambridge:Cambridge University Press.

Citations

58

Using Methodology Feedback to Improve Courses: A … · To consider how methodology feedback might be used ... free of any bias in approach, method, or technique. 17. ... Second Language

Documents