SURVEYS Sherry Woosley & Cindy Miller. Overall Surveys.

SURVEYS

Sherry Woosley & Cindy Miller

Overall

Surveys

Survey Planning

What are you

trying to accomplis

h?

Surveys should be used to:

Explore attitudes, opinions, experiences, expectations, and needs

Gather information from and about large populations

Make comparisons among subgroups of the population

Compare results from year to year Gather statistically representative data

Surveys should not be used:

For audiences that are uncomfortable with numbers/statistics

For small numbers of participants

Without a clear understanding of the issues

When investigating issues of a sensitive or intrusive nature

Advantages

Can gather information from large numbers

Results may be generalizable to a larger population

Allow for statistical analysis that examines relationships among variables

Disadvantages

Important issues can be overlooked on surveys when the questions and responses are predetermined.

The quality of survey data is strongly dependent on the survey design.

Response rates and response bias are difficult to control.

Frequently Asked Questions – Other Big Picture Issues

Surveys

FAQ – What about an on-line survey?

To decide think about: Who is in your population? What is in your survey?

Do students care about this topic? How long is the survey?

What opportunities do you have to administer the survey?

What resources do you have?

On-line surveys do NOT necessarily have lower response rates.

FAQ – How many respondents do I need?

The important issues are: What percentage of people didn’t respond? How well do your respondents match your

non-respondents? Do you have enough respondents to do the

analysis you want?

There is no magic number! In most cases, a sample size calculator

does NOT adequately answer this question.

FAQ – What types of analysis are appropriate?

1. Descriptives: Looking at how students responded

2. Differences: Comparing groups of students

3. Relationships: Looking at which variables may be related

4. Change: Looking longitudinallyThe type of analysis depends on (1) the questions being asked and (2) the statistical

skills of the researcher.

FAQ – What is included in a survey report?

Need or problem and research questions Survey characteristics Survey methods (administration, sample,

analysis) Results Conclusions Implications

Audience needs will determine the level of detail included in a report.

Questions / Items

Surveys

Good Survey Questions

Wording should be simple, clear, direct, non-ambiguous, concrete, and uniformly understood.

There is no one “correct” response scale for all survey items.

Question Types

Open ended Single response Essay/paragraph

Closed Categorical (categories) Scales (Likert) Rankings Choose multiples or choose only one

Good Surveys…

Use neutral language Generate a variety of responses Use simple sentences (avoid compound

sentences and multiple phrases) Consist of only one question (beware of

double-barreled questions)

Bad Surveys…

Ask things you already know Ask things that your respondents cannot

answer Be too intrusive or personal Include

universals (always, all, none) limiters (only, just) double negatives abbreviations unconventional phrases

Bad Examples – Can you identify what went wrong?

How many times did you call your parents last year?

How much time does your teacher spend preparing for class?

How often do you eat donuts and/or drink coffee?

How often have you been harassed on this campus?

How often have you participated in ACCESS?

How often are you merely late for a class?

How often do you miss class and feel bad about it

afterwards?

How often do you never miss a class?

Strategies to Ensure Good Questions

Have the questions reviewed by experts Have the questions reviewed by

potential respondents Have the questions reviewed by

colleagues Adopt or adapt questions that have been

used successfully on other surveys Pilot test surveys

Discussion

Your turn… What did you bring? What questions do you still

have? How can you share your

expertise with others?

Questions?

Thanks for attending today’s session. More questions? Feel free to contact us.

Sherry [email protected]

Cindy [email protected]