Online Surveys as Data Collection Instruments in Education Research: A Feasible Option? Online Surveys as Data Collection Instruments in Education Research:

Online Surveys as

Data Collection Instruments

in Education Research:

A Feasible Option?

EMASA 2012 - Dr Lorinda Minnaar

Introduction

Study Title School Governance and Leadership for Sustainable Quality Education for All

Study ThemeSchool governance and leadership

Key Purpose of the StudyTo explore and establish the ability of parent members of school governing bodies to govern and manage schools’ finances

Research DesignApproachLarge-scale, exploratory, quantitative study

SampleAll (1433) schools in the Western Cape (WCED)

ParticipantsSchool principals

Data Collection Instrument Online survey

Motivation for Choice of an Online Survey

• Large sample – 1433 schools• Financial and human resource

constraints• An online survey is a

convenient and cost-effective data collection method

• Access to“Checkbox”

Review of Literature on Online

Surveys

Problem Solution

Despite an extensive search, existing literature documenting the feasibility of this data collection method in an education research context, seems deficient

Adapt existing literature to the purpose and design of the study, located within a South African educational research context

Types of Online SurveysE-mail an introductory letter or invitation to respondents with a Uniform Resource Locator (URL) hyperlink to a web-based survey Send respondents a survey embedded in an e-mail message with or without an attachmentPlace a general request for respondents in an electronic communication environment such as a web page to complete a survey

Advantages of Online Surveys

Global reachLarge samples easily accessedTime-efficient and convenientLow preparation and administration costsEnvironmentally friendlyQuestion diversityQuestion sequencing capabilitiesQuestion branching capabilitiesEase of data entry and analysis

Disadvantages of Online Surveys

Perceptions as SpamLack of participant online

experience and expertiseComputer configurationImpersonalSecurity and confidentiality

Planning the Online Survey

Planning the online survey entailed:• Drafting the hard copy • Placing instructions, questions and answer

options in simple tables• Continually revising instructions and questions

to ensure they did not contain design errors such as confusing ambiguities, double, split and negative questions and value-laden concepts

• Consulting an expert in the field of school governance to seek confirmation that questions were valid, meaningful, grammatically correct and appropriate to the research theme

Designing the Online Survey

Designing the online survey entailed the following methodological considerations:• Selecting a visual layout comprising design

elements such as background colour, font face, size and colour

• Uploading Stellenbosch University’s oak leaf logo, which not only lent colour and formality to the survey but immediately informed participants of the survey’s origin

• Avoiding and eliminating complex, vague and ambiguous instructions that may perplex and frustrate participants to such an extent that they exit the survey before completing it

• Checking the survey’s length to ensure it was not too long and that participants would not spend too much time completing it

Long surveys with matrix type questions that have numerous

items and alternatives, increase respondent fatigue and boredom and frequently result in

respondents adopting a “satisficing” behaviour where they select the same scale-point to rate all items without paying them much attention (Mora 2010)

Designing the Online Survey (Cont).

Question FormatsRadio Buttons A list from which a participant selects one option

Question Formats (Cont).

Checkboxes A list from which the participant may select multiple options

Question Formats (Cont).

Drop-Down Lists A space-saving format

Question Formats (Cont).

Open-Ended Single-Line Text Allows freeform text with formatting rules

Question Formats (Cont).

Open-Ended Multi-Line Text Allows freeform text within a box with a specified number of rows and columns

Question Formats (Cont).

Matrixes and Rating Scales Explore the frequency of participants’ behaviour or their attitudes toward certain phenomena

Activating the Online Survey

Activating the online survey included the following aspects:• Sourcing the e-mail addresses of schools in the

sample• Checking e-mail addresses for errors• Deliberating on which day, week and month to

activate the survey• Determining the period permitted for participants to

respond and the final date for submission• The frequency of reminders to be sent to participants• Compiling an appropriate invitation to principals to

participate in the survey

Activating the Online Survey (Cont).

The invitation to participate in the online survey

Response Rates and Results

Activation and

Reminders

Number of Surveys sent

Number of Surveys

Completed and Returned

Response Rate

1st 1 433 64 4%

2nd 1 433 44 3%

3rd 1 433 37 2%

4th 1 433 220 15%

24%Overall

Response Rate

Low response rates appear typical of online surveys

Response Rates and Results

(Cont).

B a r/Co lu m n P lo t o f Co u n t o f 1

S p re a d sh e e t1 4 3 v*6 c

Co u n t o f 1

1 8 %

1 5 %1 5 %

1 %0 %

0 %

3 Re le va n t2 In te re sti n g

1 Q u i ckO th e r

4 L o n g5 Co m p l i ca te d

Q u a l i ty o f su rve y

0

5

1 0

1 5

2 0

2 5

3 0

3 5

4 0

4 5

1 8 %

1 5 %1 5 %

1 %0 %

0 %

Participants’ Responses to the Quality of the Survey

Conclusion“How feasible are Internet-based online surveys used as data collection instruments for educational research?”

• Denscombe (2009, 281) asserts “There are still several unanswered questions concerning the use of online surveys as data collection instruments for research purposes”.

• Despite implementation of salient methodological considerations, the relatively low response rate alerts educational researchers to use online surveys as data collection instruments circumspectly.