Combatting survey fatigue: uncovering the student learning experience using the

ID 164. Combatting survey fatigue: uncovering the student learning experience using the

Nominal Group Technique

Funded by ELESIG

Tunde Varga-Atkins, eLearning Unit Jaye McIsaac, Educational Development

University of Liverpool11-13 Sep 2012, ALT-C

Outcomes

•To consider the use of Nominal Group Technique as a student evaluation strategy •To question the use (and concept) of technology in relation to an evaluation process

HE context: surveys

One solution: face-to-face Nominal Group Technique sessions

What is theNominal Group Technique?

• Structured group activity • One given topic• Facilitates group

decision-making• Immediate results• Reducing researcher/participant

bias- Delbecq & Van de Ven (1971)

Nominal Group Technique Stages

1) Individual responses 2) Clarification and consolidation

3) Ranking responses

‘Nominal’ Group = only in name group, most activities are individual

Example question: ‘What to keep doing (feedback)?’

What to keep doing? (feedback)Stage 1: individual responses

Annotated ,personalised feedback.

Model answers

Giving model answers for essay titles

Audio feedback for everything!

Audio feedback giving extensive comments

Drafts that must be submitted and marked prior final work

1 2 3

45 6

What to keep doing? (feedback)Stage2 : clarification and

consolidation

Model answers

Giving model answers for essay titles

Audio feedback for everything!

Audio feedback giving extensive comments

Drafts that must be submitted and marked prior final work

35

24

Annotated ,personalised feedback.

6

1

Audio feedback

Model answers

Personalised feedback

What to keep doing? (feedback)Stage3 : ranking responses

Item no. Item description

5 points 1 Annotated, personalised feedback4 points 2 Giving model answers3 points 4 Audio feedback

2 points 7 etc.1 point 10 etc.

The top five qualities of good feedback

Timing, quick

Detailed, on how to improve Applicabl

e to future work Personal

Consistent bw markers

1

2

3

4

5

is a useful student evaluation technique

Conclusion: The Nominal Group Technique

… (minor) problems of efficiency

Credit by Nathan T. Baker

Can technology improve (time & resource)

efficiencyof the NGT process?

Presumption!

No single technology could

replicateall the 3 stages of

NGT.Technologies we tried: ID275, Thu 9am

and…

Our assumption:

Technology can improve (time)

efficiency of the NGT process.

revisited…

learning (evaluation)

technology

+

= efficiency ?

Problem of lack of theorising technology (Oliver 2012)

Paper

• ubiquitous• manipulable• overlay• multi-items• legible• visible (Luff et al 2007)

• text entry• legibility• visibility• scoring• manipulable• multiitems

Digital

Conclusion: can technology enhance efficiency of the process?

• ubiquitous• manipulabl

e• overlay• multi-items• text entry• legibility• visibility

Image by Paul Duvall

Finally…thanks to

Contact: –Tünde Varga-Atkins, [email protected] –Jaye McIsaac, [email protected]

References • Delbecq, A., Van de Ven, Andrew, & Gustafson, D. (1975).

Group techniques for program planning : a guide to nominal group and Delphi processes. Glenview  Ill.: Scott  Foresman.

• Luff et al (2007) Augmented Paper: Developing relationships between digital content and paper

• Oliver, M. (2012) Learning with technology as a coordinated sociomaterial practice: digital literacies as a site of praxiological study

• Varga-Atkins, McIsaac et al (2011) Using the nominal group technique with clickers to research student experiences of e-learning: a project report [http://slidesha.re/xQlBCg ]

• Varga-Atkins, McIsaac et al (2011) The Nominal Group Technique – a practical guide for facilitators [http://slidesha.re/AmYOgv]

Guide to NGT & Project report

See References

On slideshare.net , search for Nominal Group Technique