Does it Click? Assessing the Role of Student Response Systems - FR Arico

Does it Click?

Assessing the Role of Student Response

Systems in the Formation of Academic

Self-Efficacy Beliefs within a

Diverse Student Population

Fabio R. Aricò

Kathleen Lane

SRHE – Dec 2014

ACKNOWLEDGEMENTS

Chris Thomson – UEA alumnus and Research Assistant

UEA-HEFCE Widening Participation Teaching Fellowship

HEA – Teaching Development Grant Scheme

2

OUTLINE

1. Overview of the “When Student Confidence Clicks Project”

2. Focus Group interviews: recruitment protocols findings

3. Final remarks: the role of Student Response SystemsFurther research

3

ETHICAL REMARK

You will be presented with data collected during focus group sessions.

Students involved have given informed consent for us to analyse their responses and present the results of this analysis.

I can assist with ethical queries as well, please ask me.

4

1. When Student Confidence Clicks: the role of Academic Self-Efficacy in Learning

5

PROJECT DEVELOPMENTS

Project Resources: https://sites.google.com/site/fabioarico/hea_tdg

Project Workshop: Wednesday 3 Sep 2014 - University of East Anglia

6

BLENDED LEARNING ENVIRONMENT

• Intense use of Student Response Systems (clickers).

• Focus on increasing students’ Academic Self-Efficacy beliefs Self-Assessment is a fundamental component.

• Intense use of SRS + VLE dialogue with the students – closing feedback loop blended learning, but also blended surveying!

• Learning analytics to discern and to disentangle relationship between different aspects of the learning experience.

7

TEACHING PROTOCOL – the module

Introductory Macroeconomics Level 1 – compulsory year-long module - 170 students

Lectures traditional frontal-teaching (10 per sem.)

Seminars small group, pre-assigned problem sets (4 per sem.)

Workshops large group, problem-solving sessions (4 per sem.)

Support Sessions non-compulsory drop-in sessions (4 per sem.)

8

DATASETS

Student Q1 Q2 Q3 …

1 0 1 1

2 1 0 0

3 1 1 …

…

performance per questionconfidence by question

pe

rform

ance

pe

r stud

en

tco

nfid

en

ce by stu

de

nt

9

longitudinal study- across all lectures- across all seminars- across all workshops

Intermediate and final attainment outcomes- course test- final exam

2. Focus Group interviews:recruitment protocolsand findings

10

11

FOCUS GROUPS: set-up

2 focus group sessions

Nov 2013 Role of SRS in learningConfidence and self-efficacy

10 participants5 home students5 overseas students

Mar 2014 Experience with the moduleOpinions on learning environment

2 separate groups (on same day)6 high-performing students4 low-performing students

12

FOCUS GROUPS: recruitment protocol

• Invitation email sent to all students (first-come-first-served)

different email sent to high/low performing students2 different time-slots without revealing criteria.

• Students invited to collect a ticket from School’s General Office

Nov focus group: disclose domicile statusMar focus group: disclose time-slot on invitation email.

• Tickets collected at focus group session for participation.

• Facilitators not involved in “Introductory Economics” teaching.

Preserve anonymity and disclose as little information as possible. Control for number of participants (as many as tickets available). Allow for balanced representation within a diverse student population.

13

FOCUS GROUPS: general considerations

Interview schedules at a glance

• Students’ perceptions of the impact of use of technology in the module.

• Reflecting on different teaching innovations within the module.

• Reflecting on student confidence (self-assessment, pressure, comparison).

Findings: preamble

• Students displayed different learning styles and diverse preferences.

Diversity also revealed in student attitudes towards clickers.

• Attention often diverted to whole teaching styles, teaching resources, approachability of teaching team.

14

FOCUS GROUPS: 1st Session (Nov 2013)

Interview schedule

• investigate the role of clickers and learning technology

Findings

• Initially: not much endorsement of clickers

pace ‘too fast/too slow’ – ‘just one more thing to do’

• Deeper probing: recognised role of clickers on learning and motivation

‘really useful’ – ‘can tell the truth’ – ‘interactive’ – ‘know where you are’

• Students appreciate anonymity and feel more confident inanswering questions and participating in class.

15

FOCUS GROUPS: 1st Session (Nov 2013)

I found it’s really useful because we don’t have anything in my country … …the computer asks you if you are confident or not, and you can say [‘yes’], if ‘no’ you can tell the truth.

you are confident to answer the question even if you are not confident about the answer

I think it actually motivates me to focus on what I’ve actually missed, like go to the lecturer or study more

I know that I’m very weak in this area and I need to work on it more

There is not always time to finish all the questions that you’ve prepared … and those are the questions we’re going to get in the exams

16

FOCUS GROUPS: 2nd Session (Mar 2014)

Interview schedule

• Explore student confidence, study habits, motivation factors.

Findings

• Students acknowledge that the material had become ‘harder’ in second term.

• A difference emerged: high-performing students developed a learning style and a learning strategy – higher awareness and control over learning.Low-performance students less worried as First Year ‘doesn’t matter’.

• Some low-performing students recognised the role of clickers in highlighting difficulties and motivating them to seek for help. Stronger change in study habits, like ‘working harder’ or ‘not missing lectures’.

17

FOCUS GROUPS: 2nd Session (Mar 2014)

The adversity of not doing well on clicker questions:It stressed me out… I worked harder! It gave me the motivation to look over it [the material] because I wanted to correct it.

when I read the hand-outs myself…after the lecture, I understand so much more than when I’m actually in the classroom.

It sort of engages you a lot more in lectures… and especially in the workshops because you go through loads of different questions and sort of click away and that way you can know where you are

[I] avoided it [the Library] but it had become her second home

I know that I need to go in and speak to someone or go to a support session can know where you are. … [W]hen I wasn’t doing so well, I felt a little bit low but then I realised that the clickers are a way of letting me know that, so I can go and improve

Work out what works best for you quite quickly –whether making notes or going through the slides when you go home is best

I am confident if I stick to my revision schedule

only needing 40% to pass

doesn’t actually matter

3. Focus Group interviews:closing remarks andfurther research

18

19

FOCUS GROUPS: closing remarks

• The diversity in the opinions gathered within the focus group sessions are underpinned by diverse background, reactions to the university environment and by diverse learning styles.

• Evidence from the qualitative data supports the claim that clickers contribute both directly and indirectly to student confidence about their learning as well as their level of engagement with the module;

directly when students recognise the role of clickers asa powerful self-assessment and self-regulation device;

indirectly when students feel, however, empowered to take more active participation in their learning.

20

FOCUS GROUPS: related literature

• There is no literature assessing the role of SRS on the formationof Academic Self-Efficacy beliefs.

• Nielsen et al., (2013), “Teaching with Student Response Systems (SRS): Teacher-centric Aspects that can Negatively Affect Students’ Experience of Using SRS”, Research in Learning Technology:

• Similarities:

Student call for consistency across teaching sessions and lecturers.

Two-way commitment of students and lecturers.

It is not about fancy technology and ‘fun’ for its own sake.It is still about good teaching practice: thinking, preparing, explaining.

FURTHER RESEARCH

• “Assessing Student Self-Assessment” – SRHE Dec 2014

inconsistency between different assessment methods.

• “Peer-Instruction Unveiled” – NR SRHE Dec 2014

relationship between 1st and 2nd response – ‘learning gains’

relationship between learning gains and student confidence.

• Learning analytics at student-level

investigate the role of demographics.

21

PROJECT DEVELOPMENTS

Project Resources: https://sites.google.com/site/fabioarico/hea_tdg

Project Workshop: Wednesday 3 Sep 2014 - University of East Anglia

22