Session 3 Research Methods - Data Analysis - Case Study Example

New Media Research Methods

Part 1 – How research methods relate to the research questionPart 2- Qualitative and Quantitative Part 3 – Data collection and analysis – Case Study ExamplePart 4 – Presentation and analysis

Recap from last week

• Qualitative and Quantitative Research Methods

• Interviews and surveys• Types of questions• Designing questionnaires and interviews • Good practice

Project team: Nick Weldin, Karen Bunning and

Gosia Kwiatkowska

Case Study – Symbol Surfing Project

Funded by Esmee Fairbairn Trust

Symbol Surfing Project - Background

• Ubiquity of computer technology and the WWW – access and participation problems

• Accessibility issues – keyboard/mouse

• Match between the interface and the individual capacity of the user

Symbol Surfing Project - Background

• The role of the user/the role of the supporter

• Interactions: student/supporter; student/computer + supporter/interface/computer

• Teacher initiating role to align the student – operational engagement

Symbol Surfing Project - Background

• Communication – symbols, signs, pictures e.g. Makaton, Widgits etc

• Use user experience (using symbols) but for different purpose – control the computer

Symbol Surfing Project - Background

• Benefits – low cost, using existing skills, control, etc

• Exploring the use of freely available symbol recognition software for PMLD community

• Accessing and controlling computers by PMLD

Research questions

• What are the ways in which individuals are able to use and engage with a simple symbol recognition system to access media on a computer?

• What role is played by the supporter during user activity with this system?

Symbol Surfing Project - Method

• Design – an action research, ethics, consent• Settings – specialised collage (2), secondary

part of special school (2), residential setting (1). Supporters all knew the participants and understood their communication e.g. gestures, facial expressions etc

• Sample – five people with PMLD, age range 15 – 28, difficult to engage, English – first language,

Symbol Surfing Project – Data Collection

• Four visits, monthly intervals, • Symbol surfing software was installed

• Two video cameras were used – one to capture the screen and the other to capture the engagement

Symbol Surfing Project – Data Collection

• Environment – familiar• Researchers role –

marginal participants – guiding the supporter if required

• Field notes were recorded • Length of the sessions –

varied

Symbol Surfing Project – Data Collection

• The formats : model, withdraw support, observe• Number of cards used – varied• Video footage – combined and displayed side by

side• Sampling of video – 60 sec at 2 min intervals• Data transcribed through repeated viewings

1 3 5 7 9 11 13 1517

Symbol Surfing Project – Data Collection

• Transcriptions template – actions participant/ supporter/ researcher

• Actions: – vocal, – non vocal communication behaviours, – computer related actions, – contextual information

Symbol Surfing Project –Data Analysis

• Coding framework – structural linguistics – to capture user attempts at manipulating the symbol tools, but also recording user responses to the activity and the facilitation role performed by the supporter

• Initiating moves, response moves, pre-initiating moves and self expression

Symbol Surfing Project –Data Analysis

• Refining the coding framework – viewing video and observing behaviours

• Definition for each category with examples

Coding Framework Used

• Transcriptions reviewed with the video footage

• Code annotations applied to transcripts

Symbol Surfing Project –Data Analysis

Symbol Surfing Project –Data Analysis

• Frequency of occurrence for each code per trial was summarised in a prepared excel spreadsheet

• An overall mean scores were calculated – to explore user engagement across trials

Symbol Surfing Project - Reliability

• Two researchers reviewing each transcripts and assigning the code via a process of consensus, which informed the definition of each code.

• Also independent coding on 10% of data – Cohen’s Kappa Coefficient

Symbol Surfing Project - Findings

• Individual use of and engagement with symbol recognition system

• Levels of ‘self expression’ • ‘Basic’ and ‘Functional actions’ with symbols

varied – factors: alertness and physical demands of manipulating the symbol, attention

• Role of supporter during user activity- move types: ‘instruction’, ‘query’, ‘technical’, feedback’ – facilitating engagement

The following slides are for you to look at independently as they cover material covered by Tony and myself

in previous lectures.

For next week please be ready to discuss your research question and

your proposal.

Emotional Tools - Recap

• Emotional Text Analysis – locating emotion in what users have to say

• Facial Recognition – observing emotional responses to experiences

• EEG – observing brainwaves (relaxed and attentive states)

• GSR – skin conductance

• Blending of the above with eye tracking

Emotional Analysis of The School Plan

Nice

Pleasant

Fun

Active

Nasty

Unpleasant

Sad

Passive

High Imagery

Low Imagery

Sample 1Total words 27201Sentence Length 18.72Sentences 1453Periods 614Question Marks 466% Nice 2.06% Pleasant 4.38% Fun 7.80% Active 4.43% Nasty 1.74% Unpleasant 1.52% Sad 4.42% Passive 13.87% High Imagery 2.57% Low Imagery 39.27

Emotional text recognition using Whissell’s Dictionary of Affective Language

Blair’s Speech on War in Iraq

0

5

10

15

20

25

Series1

Nice 2.04

Pleasant 3.53

Fun 3.79

Active 4.02

Nasty 4.3

Unpleasant 4.53

Sad 4.53

Passive 19.62

Analyzing User Feedback• What is your goal?

– “I’m not sure. I am a little confused at this point as to what I am supposed to be doing, It’s not obvious.”

• What did you expect when you did that?– “I was hoping that if I clicked this icon I would be able to move to the

help page, but I am surprised that didn’t work.” • Can you tell me what you were thinking?

– “I’m not sure. I think I’m supposed to be moving through this section of the maze, but there is no help and I’m getting lost.”

• How did you feel about that process?– “In a word confused. It looked good to begin with, but I am lost and

feel like giving up.”

Analyzing User Feedback

0

2

4

6

8

10

12

14

Series1

Nice 4.3

Pleasant 2.69

Fun 2.15

Active 3.76

Nasty 1.08

Unpleasant 5.91

Sad 1.61

Passive 11.83

http://compling.org/emotion_in_text_cgi/DAL_app/index.php

match tools to your questions• For questions on factual clarification• Use polls and surveys

• For questions on opinion• Use surveys, interviews, focus groups

• For questions on experience• Interviews, focus groups,

observations, user testing, and ethnographies

• For questions on concepts• surveys, interviews,

ethnographies, and user testing

• For questions on emotions• Surveys, interviews, focus groups,

and observations

CorrelationsInterview, Questionnaire & Focus Group

References• Bunning, K., Kwiatkowska, G., Weldin, N. (2012) Journal

of Assistive Technology, V24 n4 p259-270• May T (1997) Social research: issues, methods, progress

Buckingham: OUP• Rugg, D. (1941) Experiments in wording question. II

Public Opinion Quarterly, 5:91-92. • Silverman D (1997) Qualitative research: theory, method,

practice London: Sage • Silverman D (2004) Doing Qualitative Research: A

Practical Handbook London: Sage