Teaching research methods in the Social Sciences ... · 1 Teaching research methods in the Social Sciences Facilitator: Natasha Taylor Leeds, 21 May 2015

1

Teaching research methods in the Social Sciences

Facilitator: Natasha Taylor

Leeds, 21 May 2015

Joan Smith and Phil Wood, University of Leicester

Abstract

The presentation outlines an action research project in which we trialled strategies to

develop EdD students’ critical writing and peer reviewing skills, by engaging them in a

process of peer assessment. The project began with an intensive, residential critical

writing weekend and culminated in the establishment of a student-led, peer reviewed

journal. We report on students’ perceptions of their involvement in the project as a

transformational experience. Students were interviewed before and after the

intervention, and reported shifts in their understanding of criticality, peer review and

critical writing. Perhaps more importantly, there were significant shifts in their

perceptions of themselves as capable of and confident in their criticality. Learning was

enhanced through critical friendship, and students derived a sense of belonging to a

research community in which they were playing a meaningful role. Critical friendships

were established that would thrive beyond the life of the project. We consider the

implications of the project for teaching and supervisory pedagogies at postgraduate level.

2

Developing doctoral students’ critical writing and

reviewing skills through peer assessment

Developing doctoral students’ critical writing skills through peer

assessment Joan Smith, Phil Wood, Gareth Lewis & Hilary Burgess

(University of Leicester School of Education) Project funded by HEA Social Sciences Strategic

Priorities Teaching Research Methods in the Social Sciences

Issues

• Challenges of developing critical writing (students and supervisors)

• Affective barriers to accepting feedback

• Isolation of school-based researchers

• Need to develop pedagogies to develop criticality & resilience, & induct students into peer review

The project: ‘writing as social practice’ (Kamler & Thomson, 2006: 5)

Students were offered opportunities to work with others to:

• Develop an awareness of what critical writing is

• Practise and develop their own critical writing

• Provide constructive critical feedback via a peer reviewing system to other students;

• Become integrated into a research community in a meaningful way.

The project

• Intensive residential critical writing weekend

• Forming of editorial board

• The Bridge https://journaleducationalresearchinformedpractice.wordpress.com

• Summer School workshop led by students

Data collection

• Pre- and post-weekend interviews

• Notes taken during roundtable

• Students’ written feedback on drafts

• Observation notes from editorial board meeting

• The Bridge

Initial findings • Understanding of critical writing considerably enhanced through

engagement with the process

• Shift in understanding of criticality as negative and destructive to supportive and developmental (caring?)

• Parallels drawn between giving feedback & teaching (caring?);

• Shifting identity from initial alienation and feelings of inadequacy to growing sense of positive researcher identity: key difference & keyword marking the ‘before’ and ‘after’ is ‘confidence’

• Apparent that this is less to do with technical skills; much more bound up with affect, emotion, anxiety and belonging

• Evidence that learning is enhanced via social aspects of the peer assessment and review experience, and engagement in a community

• Critical friendships established/strengthened through peer assessment and review

• Support group moved from camaraderie to critical friendship

Emotion, affect, anxiety, insecurity ‘I think I’m going to find [peer reviewing] quite challenging. .. .Partly, because they’re writing in an area that’s not my area. So I don’t really know much about it. ..So I end up saying really pathetic things like ‘That's really interesting’ and then I think ‘what use is that to anybody?’ …[It’s difficult] partly because I don’t know [the other students] well enough yet. ..So …we’re having a congenial conversation instead of a deep conversation… we’re all being terribly polite and nice to each other…We don’t know enough about what each other is doing in order to be critical…somebody gave me one of their scripts to read and my first reaction was ‘this is too descriptive’. There were no references in it. It was very narrative. But it was just her first, you know, initial thoughts. She’d got it down on paper and was going to go away and work on it. So I think it’s just…it’s easier not to say anything, because you’re just being destructive, not helpful… I didn’t feel sure of my grounding…’ (Eloise, Interview 1)

Emotion, affect, anxiety, insecurity

‘I’ve read some of the other students’ stuff … but the problem is … you don’t want to upset anyone. You look at their work…but you just say, ‘Oh, it’s good’, and…we’re not really doing anyone any favours there’ (Peter, interview one).

Shifting identity & growing confidence

‘I felt much more confident by the end of the weekend. I understand what critical writing looked and felt like. I think beforehand, we’ve had a couple of meetings at the library, my little, sort of, support group, and I’ve not really felt able to give anybody any meaningful feedback about their writing, and we didn’t know where to start. And it’s just having that structure, in a pro-forma that was provided to sort of give the feedback in the beginning and emailed out before the course started that just helped to…I understood what I was being asked to comment on and how.’ (Eloise, interview 2)

Shifting identity & growing confidence ‘When I sit down at the table with our peers, really, I’m probably the least confident of anyone there, you know, because I’m not always that confident in my ability… In the past I’ve worked on group stuff before and I haven’t really contributed that well, because I haven’t felt…everyone’s seemed to understand what’s going on except me… it’s always in the…back of my head really… when I read my work and read…others’ in the group…peers’, sometimes I feel mine is just too simplistic…I tend to use straightforward words, let’s say non-academic words... that’s something that I know I need to get away from’ (Peter, interview one). ‘You know, when I look back on (the critical writing weekend) now, I thoroughly enjoyed it and I got a lot of things out of it, and the thing is, like I mentioned, … before … to me it’s a confidence thing, and towards the end of it I certainly felt a bit more confident in me, about my own ability’ (Peter, interview two).

Shifting identity & growing confidence

‘Before it was a case of reading it and you might say, ‘Oh yeah that’s good… maybe a couple of grammatical mistakes’ – you wouldn’t go too deep. Whereas I think now we don’t mind stepping that little bit further, you know what I mean? And I think … two or three meetings’ time, I think we’ll be quite OK with it. It is just being prepared to…let’s say there’s something bad in what you’ve done, and they’re really going to point these things out then, you know, you want to have that little bit of a bond…you want a reasonably good bond with them’ (Peter, interview two).

Sense of belonging to a community of researchers

‘I think it was really great [that there were students from all stages of the doctorate involved]. Really enjoyed that, really enjoyed meeting them all, they’re all just such lovely people and it makes you feel, I think it helps that we’re all distance learners, even people like me who live nearby. Everyone’s all scattered to the four winds that it, it just gives you a feeling of belonging to somebody, that you don’t get as the kind of student we are’ (Eloise, interview 2)

Final thoughts

• Residential weekend transformational and empowering: students taking work forward

• Importance of critical friendships

• Too important to leave to chance!

References

Kamler, B. & Thomson, P. (2006) Helping doctoral students write: pedagogies for supervision. Abingdon: Routledge

Liam Foster & Tom Clark, University of Sheffield

Abstract

Whilst recent funding initiatives are a welcome development, there is continuing concern

about the level of quantitative skills possessed by UK social science students specifically

and their levels of employability more generally. Indeed, the difficulties of developing such

experience is likely to remain particularly acute for those students, institutions and

departments who find themselves outside the reach of those initiatives. This paper

explores the inquiry-based strategies we have been developing in the Department of

Sociological Studies at the University of Sheffield. Designed to enhance students’

experience of doing sociological research, we will discuss how, with the assistance of the

HEA, we developed of a series of student-led quantitative workbooks, worksheets, and

associated teaching strategies and subsequently embedded them within a ‘research

methods spine’. The paper considers some of the difficulties of employing an inquiry-

based framework and outlines some considerations for those wishing to adopt a ‘student-

focused’ and ‘research-led’ approach to the learning and teaching of research methods.

17

Embedding inquiry-based approaches to quantitative

research within a 'research methods spine'

Embedding inquiry-based approaches to quantitative research within a ‘methods spine’

Tom Clark and Liam Foster [email protected] and [email protected]

28/05/2015 © The University of Sheffield

Contents

• The wider context of teaching quantitative

research methods

• Inquiry-based learning

• Developing ‘a methods spine’

• Doing quantitative sociological research

• An integrated approach

Wider context – top down

• Lots of ‘top-down’ movement toward quantitative social research

• Continuing drives toward promoting statistical skills among social

scientists (Nuffield/ESRC/HEFCE funding)

• Employment

• Post-Graduate Research (and beyond)

• Big data?

• High quality data – and lots of it

• Research mindedness in a research saturated society

28/05/2015 © The University of Sheffield

Wider context - bottom up

• From the perspective of UG students, there are any

numbers of constraints – and opportunities

• ‘my level of ability is…in a formative stage…I’m not a natural

mathematician’

• ‘literature surrounding doing quantitative research is often quite

inaccessible and dry’

• But…

• ‘I would like to learn more about quantitative research as quite often

interesting and/or relevant facts within sociology that support points both

in my own essays and within general reading are of a quantitative

statistical nature’. Flo

28/05/2015 © The University of Sheffield

Esther says…

• ‘I think I’m weaker on analysis because we haven’t ever

had to actually do it, and the whole maths thing puts me

off a little bit. I think it would be good to learn more about

quantitative research ... I think that most students who

do sociology that I know struggle with thinking

mathematically, which puts them off wanting to do

anything with statistics because it seems a bit

complicated’

28/05/2015 © The University of Sheffield

Elina says...

• ‘I feel that my biggest weakness at the moment in

quantitative social research is statistical techniques and I

find that it makes quantitative research seem more

complex. In addition, as it is an important part of the

research process I feel that in order to consider

conducting a quantitative social research it would be

essential for students to learn the basics in the simplest

form’

28/05/2015 © The University of Sheffield

A methods spine

• 5 (core) modules across three levels

• Introduction to social research (L1: 10 credits)

• Doing social research (L1: 10 credits)

• Doing quantitative sociological research (L2: 20

credits)

• Doing qualitative sociological research (L2: 20

credits)

• Dissertation (L3: 40 credits)

28/05/2015 © The University of Sheffield

Doing quantitative sociological research

• Aims to provide a theoretical and practical foundation for conducting

independent quantitative sociological research

• Design and sampling

• Data collection (primary and secondary)

• Descriptive and inferential analysis (bivariate)

• Assessment

• Poster presentation (survey-based ‘research commission’ –

group-led)

• Research report (using ESDS data-sets - individual)

28/05/2015 © The University of Sheffield

28/05/2015 © The University of Sheffield

How do we teach it?

• Blended learning

• 16 lectures, 2 lab-based workshops, 7 group-focussed seminars, 5 inquiry-

based workbooks

• HEA funding enabled us to continue to develop workbooks used within the

previous courses in our home department (see Foster and Clark, 2013).

• ‘Research Rationales, Research Questions and Research Hypothesis’

• ‘Designing Variables and Understanding Levels of Measurement’,

• ‘Describing Data’,

• ‘Using Chi-Square, Phi & Cramer’s V’

• A guide to analysing data using PASW (IBM SPSS)

28/05/2015 © The University of Sheffield

Quantitative Workbooks

28/05/2015 © The University of Sheffield

Why do we teach in this way?

• Inquiry-based learning, student-centred, inclusive (peer support)

• Quantitative analysis is not just a series of techniques: it is part of a process of

research – so we start right at the beginning and integrate themes throughout

• Our students think in narratives, not in numbers

• They can, however, understand:

Basic numerical operations

Basic issues in research design

• Alignment between learning and teaching and assessment is vital, but pace of

learning is often variable – secure resources are necessary for students to connect

their L&T with assessment tasks in their own time

• Understanding the ideas and techniques at work behind non-parametric tests (now)

provide a secure and accessible platform for more complex analysis (later)

• Grounded in examples from the UK Data Service, and student assessments

28/05/2015 © The University of Sheffield

What do our students think?

• ‘The information is rich and detailed without being over complicated.

Examples that are relative to students current life experience are

used which creates interest and also gives inspiration for topics of

research relevant to students lives. In terms of experience of

quantitative methods within sociological research these workbooks

are effective as they provide simple exercises in order to practise

methods that are previously only abstract to the student. After

completing them I feel more confident in my own statistical

knowledge. Overall I feel these workbooks would be a very useful

resource to me as a student who both struggles with maths as a

subject and wants to learn as much as possible about this

methodology’. Flo.

28/05/2015 © The University of Sheffield

What do our students think? (2)

• ‘The books definitely enhanced my knowledge of

quantitative social research and demonstrated new

aspects of it and how it can be used in practice ... I was

very pleasantly surprised with how student friendly these

workbooks were and they definitely changed my attitude

towards quantitative research and statistics. Moreover I

feel that I gained a lot from participating in this project

and hopefully will have a chance to use in the future’.

Elina

28/05/2015 © The University of Sheffield

What do our students think? (3)

• ‘I found it interesting because it was put

into a sociological context, so there was

some sort of relevance to it, and the sort of

topics I am interested in were integrated

into the statistics’. Adam

28/05/2015 © The University of Sheffield

What do our students think? (4)

• ‘The work books definitely enhanced knowledge

and experience. They didn’t really give me a

huge interest in quantitative methods (I think I’ll

always be put off a little by the maths side of it),

but everything was explained clearly and easily

and I feel as though I know a lot more about it’.

Esther.

28/05/2015 © The University of Sheffield

Difficulties and challenges

• Integrated curriculum design

• Level of ambition has to match capacity - and be understood in relation

to the programme it sits within

• Embedding method

• More substantive modules need to take some strain

• Contradictory messages from within

• Department (and Faculty) are supportive, but developing open-access

resource is not a ‘research priority’

• Contradictions from outside

• ‘[at A Level] we spent most of the time ripping into quantitative stuff

rather than looking at why it might be useful’ (Adam)

28/05/2015 © The University of Sheffield

But…

• Our IBL research methods modules that include the workbooks are

more popular amongst students than they were when they were

primarily didactic

• Provided a solid foundation for a more advanced L3 (F6) module

‘How to Lie with Statistics’

• Student feedback has not been problematic – which is actually very

positive - students are far from resistant

• Any negative comments are primarily directed to group work and

the use of IT

• More students do go onto conduct quantitative dissertations and some

are going on to PGR study (and employment)

28/05/2015 © The University of Sheffield

Karen Burrows, Amy Pearson and Nick Wragg, University

Centre, Grimsby College

Abstract

Teaching research methods is problematic; research suggests that the terminology is

challenging and is often referred to as inaccessible and ‘complicating common sense’.

This session aims to address these problems by showcasing an innovative and creative

resource in teaching research methods; and the emphasis is on the use of games in the

classroom. Developed over a 2 year period, the resource consists of a board game in

the traditional sense called Methodology: building examples and meaning within higher

education. Players are grouped into teams of 4 and work their way around the board

collecting cards related to research methods. The game is designed to encourage the

students to analyse and evaluate plus consider the appropriateness of research design

and to match their chosen methods to a particular question and aims and objectives.

The game is highly adaptable and can be tailored to the needs of both the tutor and the

student group.

36

Methodology: building examples and meaning

within higher education

HEA Conference, Leeds, May 21st 2015

‘METHODOLOGY Building examples and meaning within Higher

Education’

Karen Burrows, Amy Pearson and Nick Wragg

Background/History

• Originally aimed at Chinese learners

• Encouraging engagement with research methods

• Developed in 2013 from a 3 stage process:

– Problem solving worksheet

– Concept mapping group activity

– Initial board game

Testing and Feedback

• Feedback from HEA Summer Conference 2013

• Internal testing (non-traditional, Chinese learners)

• Quickly became apparent the game engaged a full range of students and staff

• HEA Workshop October 2013

• External testing (selected universities, traditional pathway students)

• Ongoing Peer review leading to current version

Philosophy and Approach

Strategy/Time

Horizons

Data Collection/Data Type

Ethics Research Design Terminology

Research (1)__________ refer to the appropriateness of your (2)__________ in relation to the rights of those who become the subject of your work or affected by the work. They also relate to yourself and ensuring no harm comes to you and other researchers. Ethical concerns can occur at all stages of your (3)__________ project: when seeking access: during data (4)__________: as you analyse (5)__________ and when you report your findings.

behaviour collection data

ethics research

Questionnaire

Sample

Probability

Interview

Quantitative

Qualitative

d _ d _ c _ _ v _

_ n _ u _ t _ v _

_ n t _ _ p r _ _ _ v _ _ _

_ o _ i _ i v _ _ _

_ r _ g _ _ t _ _ m

_ _ a _ i _ m

RESEARCH METHODOLOGY

Board Game

Original

• Chinese Focused

• Emphasis on three stage build

• Methodology Foundations

• Framework – Philosophy

– Approach

– Strategy

– Data Collection Methods

– Time Horizons

Final Version

• Aim to engage all students (differentiation via card booster packs)

• Focus on standalone game – Additional squares, resources

• Methodology Foundations and wider considerations

• Framework

-Philosophy -Strategy -Data Collection Methods

-Sampling -Ethics -Questions and Considerations

Game Playing in HE

With the use of technology an everyday occurrence, students using social media and playing games in these forums, the question posed is, why not use games as a teaching and learning method? This would also provide a useful method for the international learners as arguably game playing (as in the social forums) is universal, perhaps on the same basis as numbers. So game playing would go far in overcoming the language and cultural barriers that international students and UK teachers may find.

Our Game Rationale

The objective is to build a logical research methodology and be able to define and justify methodological choices. The first to build a logical methodology and justify it to the rest of the group successfully is the winner. The methodology is built via card collection with cards you deem to be appropriate kept and recorded on the methodology sheet; warning other players may steal a card you need.

D

ata

co

llectio

n

me

tho

ds

Inte

rview

acce

ss grante

d

by rich

data

sou

rce-Take

a d

ata colle

ction

card

Mo

ve

fo

rwa

rds

3

sp

ace

s...

Qu

estio

nn

aire

resp

on

se rate

s slo

w-m

iss A G

o

Choose your

time horizon

Your tutor has concerns that your research methods do not match your

research question – miss a go!!!

SWOP A CARD

Lap top stolen-

Discard one card of your

choice

Philosophy

Data collection

ETHICS

Move forwards

3 spaces... Philosophy

You critique your

colleagues research

question- STEAL A

CARD from your

opponent

SWITCH

ETH

IC

S

Yo

ur

tuto

r h

as

co

nc

ern

s o

ve

r

pla

gia

ris

m -

go

ba

ck

to

sta

rt

an

d ju

sti

fy y

ou

r

se

lec

tio

n!!

!

Yo

u c

riti

qu

e

yo

ur

co

lle

ag

ue

s

res

ea

rch

qu

es

tio

n-

ST

EA

L A

CA

RD

fro

m y

ou

r

op

po

ne

nt

Da

ta

co

lle

cti

on

m

eth

od

s

Mo

ve

fo

rwa

rds

3

pa

ce

s..

.

‘METHODOLOGY Building examples and meaning within

Higher Education’

• An engaging method for teaching methodology, and judging learning through formative assessment

• Ideal for seminar activities and independent student learning

• Currently in development for manufacture (Available for pre-order – see us for details)

• Booster Packs in the development stage

Impact of Research 1

The HE landscape is changing:-

• QAA Auditing Requirements

• Student as Producer

• Student Voice and Expectations

• Continuing issues over funding

• Continuing dynamism in areas of distance/blended learning

Impact of Research 2

• Paper One - It’s All in the Game – Embedding Research through Game Playing

• Book One – Publishing Contact, The Challenges of teaching and learning – Chinese Students

• PhD Research Project – Karen Burrows

• Continuing research based on progression of boardgame

Thank You

• We welcome you to examine the game in the following session.

Contact us at:

• [email protected]