Top Banner
Parlindungan Parlindungan Pardede Pardede Christian University of Indonesia Christian University of Indonesia
31

Survey Research for Elt

Oct 10, 2014

Download

Documents

this slides are used in my research classes at UKI
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Survey Research for Elt

ParlindunganParlindungan PardedePardedeChristian University of IndonesiaChristian University of Indonesia

Page 2: Survey Research for Elt

Definition

• Survey research designs are procedures inquantitative research in which investigatorsadminister a survey to a sample or to theentire population of people to describe theattitudes, opinions, behaviors, orcharacteristics of the population.

• It is a useful design to use when researchersseek to collect data quickly and economically,study attitudes and opinions, and surveygeographically dispersed individuals.

Page 3: Survey Research for Elt

Goals of Surveys• To describe trends, such as high school students’

preferences in having native speaker teachers or non-native ones.

• To determine individual opinions about policy issues,such as whether English should be taught in primaryschools.

• To identify important beliefs and attitudes ofindividuals, such as college students’ perceptions onthe use of blogs to develop writing skills.

• To get information necessary to evaluate programs inschools, such as the success of using multimedialaboratory in English teaching.

Page 4: Survey Research for Elt

Key Characteristics

1. Sampling from a population2. Collecting data through questionnaires or

interviews3. Designing instruments for data collection4. Obtaining a high response rate

Page 5: Survey Research for Elt

Population, Target Population, and Sample

Senior High SchoolStudents in Jakarta

Students of SMAN16 Jakarta

11th Graders ofSMAN 16 Jakarta

Page 6: Survey Research for Elt

Types of Sampling MethodsSampling Techniques

Non-RandomSampling Techniques

RandomSampling Techniques

ConvenienceSampling

JudgmentalSampling

QuotaSampling

SnowballSampling

SystematicSampling

StratifiedSampling

ClusterSampling

Other SamplingTechniques

Simple RandomSampling

Page 7: Survey Research for Elt

Convenience Sampling

Convenience sampling attempts to obtain a sample ofconvenient elements. Often, respondents are selectedbecause they happen to be in the right place at the right time.

– use of students, and members of social organizations– mall intercept interviews without qualifying the

respondents– department stores using charge account lists– “people on the street” interviews

Page 8: Survey Research for Elt

Judgmental Sampling

Judgmental sampling is a form of convenience sampling inwhich the population elements are selected based on thejudgment of the researcher.

– test markets– purchase engineers selected in industrial marketing

research– bellwether precincts selected in voting behavior research– expert witnesses used in court

Page 9: Survey Research for Elt

Quota SamplingQuota sampling may be viewed as two-stage restricted judgmental sampling.– The first stage consists of developing control categories, or quotas, of

population elements.– In the second stage, sample elements are selected based on convenience

or judgment.

Population Samplecomposition composition

ControlCharacteristic Percentage Percentage NumberSexMale 48 48 480Female 52 52 520

____ ____ ____100 100 1000

Page 10: Survey Research for Elt

Snowball Sampling

In snowball sampling, an initial group of respondents isselected, usually at random.

– After being interviewed, these respondents are asked toidentify others who belong to the target population ofinterest.

– Subsequent respondents are selected based on thereferrals.

Page 11: Survey Research for Elt

Simple Random Sampling

• Each element in the population has a known and equalprobability of selection.

• Each possible sample of a given size (n) has a known andequal probability of being the sample actually selected.

• This implies that every element is selected independently ofevery other element.

Page 12: Survey Research for Elt

Systematic Sampling• The sample is chosen by selecting a random starting point and then

picking every ith element in succession from the sampling frame.• The sampling interval, i, is determined by dividing the population size N by

the sample size n and rounding to the nearest integer.• When the ordering of the elements is related to the characteristic of

interest, systematic sampling increases the representativeness of thesample.

• If the ordering of the elements produces a cyclical pattern, systematicsampling may decrease the representativeness of the sample.For example, there are 100,000 elements in the population and a sampleof 1,000 is desired. In this case the sampling interval, i, is 100. A randomnumber between 1 and 100 is selected. If, for example, this number is 23,the sample consists of elements 23, 123, 223, 323, 423, 523, and so on.

Page 13: Survey Research for Elt

Stratified Sampling

• A two-step process in which the population is partitioned intosubpopulations, or strata.

• The strata should be mutually exclusive and collectivelyexhaustive in that every population element should beassigned to one and only one stratum and no populationelements should be omitted.

• Next, elements are selected from each stratum by a randomprocedure, usually SRS.

• A major objective of stratified sampling is to increaseprecision without increasing cost.

Page 14: Survey Research for Elt

Stratified Sampling• The elements within a stratum should be as homogeneous as possible, but

the elements in different strata should be as heterogeneous as possible.• The stratification variables should also be closely related to the

characteristic of interest.• Finally, the variables should decrease the cost of the stratification process

by being easy to measure and apply.• In proportionate stratified sampling, the size of the sample drawn from

each stratum is proportionate to the relative size of that stratum in thetotal population.

• In disproportionate stratified sampling, the size of the sample from eachstratum is proportionate to the relative size of that stratum and to thestandard deviation of the distribution of the characteristic of interestamong all the elements in that stratum.

Page 15: Survey Research for Elt

Cluster Sampling• The target population is first divided into mutually exclusive and

collectively exhaustive subpopulations, or clusters.• Then a random sample of clusters is selected, based on a probability

sampling technique such as SRS.• For each selected cluster, either all the elements are included in the

sample (one-stage) or a sample of elements is drawn probabilistically(two-stage).

• Elements within a cluster should be as heterogeneous as possible, butclusters themselves should be as homogeneous as possible. Ideally, eachcluster should be a small-scale representation of the population.

• In probability proportionate to size sampling, the clusters are sampledwith probability proportional to size. In the second stage, the probabilityof selecting a sampling unit in a selected cluster varies inversely with thesize of the cluster.

Page 16: Survey Research for Elt

Types of Cluster Sampling

Cluster Sampling

One-StageSampling

MultistageSampling

Two-StageSampling

Simple ClusterSampling

ProbabilityProportionate

to Size Sampling

Page 17: Survey Research for Elt

Three Types of Information Surveys Can Provide

• Factual information, like the characteristics of individualteachers and learners (e.g., students’ age, gender,ethnicity, language background, proficiency level, etc.).

• Behavioral information, i.e. the one that describe whatstudents or teachers have done or regularly do in terms oftheir language teaching and learning, such how oftenstudents look up unfamiliar words in a dictionary or makean outline before they write an essay.

• Attitudinal information, which depict the opinions, beliefs,or interests of teachers or learners. These questions areoften used in needs analysis when researchers want togather information on such topics as what learning goalsstudents have or what skill areas they are most interestedin.

Page 18: Survey Research for Elt

Types of Surveys

Survey

Cross-Sectional

data are collected atone point in time

Longitudinal data are collectedover time

Page 19: Survey Research for Elt

CROSS-SECTIONAL vs. LONGITUDINAL SURVEY

Page 20: Survey Research for Elt

Types of Survey: CROSS-SECTIONAL

A cross-sectional study is one that produces a ‘snapshot’ ofa population at a particular point in time. The researchercollects data at one point in time in order to:(1) to examine current attitudes, beliefs, opinions, or

practices;(2) to compare two or more educational groups (students

with students, students with teachers, students withparents) in terms of attitudes, beliefs, opinions, orpractices;

(3) to measure community needs of educational services asthey relate to programs, courses, or school facilitiesprojects;

(4) to evaluate a program, such as a survey that providesuseful information to decision makers.

Page 21: Survey Research for Elt

Types of Survey: LONGITUDINAL

• The researcher collects data to study individuals over time. Thisdesign is differentiated into:(1) Trend study, involves identifying a population and examining

changes within that population over time, e.g., Gallup Poll, whichis used during elections to monitor trends in the population ofvoters from the primary to the final election;

(2) cohort study, in which a researcher identifies a subpopulationbased on some specific characteristic and then studies thatsubpopulation over time. For instance, a group of 18-year-oldstudents is studied in the year 2001. Five years later (in 2006), agroup of 23-year-olds is studied. (They may or may not be thesame individuals studied in 2001.) Five years after that (in 2011),a group of 28-year-olds is studied.

Page 22: Survey Research for Elt

Surveys

Questionnaire

Telephone

Mail

Web/Email

Face-to-FaceInterviews

Page 23: Survey Research for Elt

Survey Procedure

Page 24: Survey Research for Elt

Questionnaire DesignTips for designing questionnaire:1) Consider whether a survey instrument is available to

measure your variables2) Consider modifying an existing instrument3) design your own instrument by following 3 steps:

Write different types of questions, including personal,attitudinal, and behavioral questions; sensitivequestions; and closed-and open-ended questions.Use strategies for good question construction, i.e. usingclear language, making sure the answer options do notoverlap, and posing questions that are applicable to allparticipants.Perform a pilot test of the questions, and make revisionbased on obtained feedback.

Page 25: Survey Research for Elt

Guidelines for Designing Questionnaire1. Keep the questionnaire sufficiently short (30 minutes maximum).2. Avoid jargon. Seek simplicity but avoid being condescending.3. Keep questions short, as long and complex questions are difficult to understand.4. Split double-barrelled questions, e.g. How long have you studied English and

been in receipt of Government support? into two questions.5. Avoid leading questions which encourage a particular answer, e.g. Do you agree

that your supervisor is supportive?6. Avoid negatively framed questions or statements which are difficult to

understand, particularly when asked to agree or disagree.7. Make sure respondents have the knowledge, otherwise you may get false

answers.8. Check terms are suitable for the context in which they are used, as meanings

may vary for different age groups, religions, cultures etc.9. Ensure the frame of reference for each question is clear e.g. when asking for the

frequency of an event, specify the time period.10. Avoid creating opinions. Respondents do not necessarily hold opinions on

topics. Allow a no opinion alternative.11. Use personal wording if you want the respondents to express their feelings.

Page 26: Survey Research for Elt

A good questionnaire …• is complete, i.e. gets all the data you need;• is short, i.e. doesn't abuse the respondents’ time or concentration;• asks only relevant questions;• gives clear instructions;• has precise, unambiguous and understandable questions;• has objective questions, i.e. doesn't suggest answers;• starts with general questions;• has appropriate questions;• puts sensitive questions at the end; is complete, i.e. gets all the data you need;• is short, i.e. doesn't abuse the respondents7 time or concentration;• asks only relevant questions;• 0 gives clear instructions;• has precise, unambiguous and understandable questions;• has objective questions, i.e. doesn't suggest answers;• starts with general questions;• has appropriate questions;• puts sensitive questions at the end;

Page 27: Survey Research for Elt

Data Analysis Techniques• DESCRIPTIVE. Deals with the question of ‘what’

things are like, not ‘why’ they are that way, andincludes means, standard deviations, frequencycounts, graphs, and charts.

• ANALYTICAL. Seeks to explain relationships,causes or consequences, and include bivariateand multivariate analyses such as correlations,cross-tabulations and regressions

• CONTEXTUAL. Narrows down the context byreinterpreting the data for subgroups. E.g. EFLstudents vs. ESL students, academic vs non-academic employees.

Page 28: Survey Research for Elt

Instrument Validity

• Construct validity: Does the questionnaire really measure theconstruct being examined?

• Criterion-related validity. Does the instrument accuratelypredicts (predictive validity) or diagnoses (concurrent validity)some particular variable (criterion).

• Content validity. Does the contents of the questionnaire reallymeasure the variable being measured ? To achieve this,compare your questionnaire to existing related instruments. Ifnone exists, gather expert opinion on each question on theinstrument to determine whether or not it actually tests whatit is supposed to.

Page 29: Survey Research for Elt

Measures for Assuring a Survey’s Reliability

• Giving the same survey on two occasions to thesame individuals and checking the consistency ofthe same response to the same item.

• Having the same individuals taking two forms of asurvey.

• check the internal consistency of responses in asurvey, i.e. seeing how consistently the samerespondents answer similar questions formulatedin different forms

Page 30: Survey Research for Elt

References

McKay, S. L. (2006). Researching second language classrooms.New Jersey: Lawrence Erlbaum Associates, Inc., Publishers

Burns. A. (2010). Doing action research in english languageteaching: A guide for practitioners. New York: Routledge:

Creswell, J. W. 2008. Educational research: Planning,conducting, and evaluating quantitative and qualitativeresearch. New Jersey: Pearson

Denscombe, M. (2010). The good research guide for small-scalesocial research projects. New York: McGraw-Hill

Goddard, W & Melville, S. (2006). Research methodology: Anintroduction. Lansdowne: Juta & Co, Ltd.

Ross, Kenneth N. (ed.). (2005). Educational research: Somebasic concepts and terminology. Paris: International Institutefor Educational Planning/ UNESCO.

Page 31: Survey Research for Elt

Examples• A Survey of English Language Teaching in Lithuania: 2003-2004 by

Ministry of Education and Science of the Republic of Lithuania.(Available online athttp://old.smm.lt/en/stofedu/docs/edu_reform/english%20language%20teaching.pdf)

• A Survey on the Iranian ELT Community's Attitudes to CriticalPedagogy by Hossein Davari, Abutaleb Iranmehr, Seyyed MahdiErfani. (Available online athttp://www.ccsenet.org/journal/index.php/elt/article/view/14567)

• Survey of The English Language Needs of the Senior Students ofPhysical and Biomedical Sciences by: Inga Rozgien and AlbinaTre iokait . (Available online athttp://www.uki.vu.lt/file/Verbum/2011/2/rozgiene_treciokaite.pdf)