Erich NW Network 2009 Session -- Measuring and Monitoring With Surveys

8/14/2019 Erich NW Network 2009 Session -- Measuring and Monitoring With Surveys

1/52

1

Measuring and Monitoring

with SurveysErich Stiefvater, NWREL

NW Network Conference 2009

March 2-4, 2009


2/52

2

Guiding Questions

Why should I use surveys?

What survey tool is right for me?

How do I design an effective survey?


3/52

3

Session Roadmap

Take workshop survey

General discussion

Sign up for free accountSurvey-building practice

Report out

Wrap-up


4/52

4

Why should I use surveys?


5/52

5

A Variety of Uses

Surveys can help you:

Investigate community needs

Design new programs and servicesEvaluate impacts and outcomes

Assess constituent satisfaction

Plan eventsTest communication media and

messages


6/52

6

Survey Pros

Surveys:

Are ideal for gathering data and feedback

that can be quantified

Collect information in a standard,

systematized way

Yield datasets that can be sorted, filtered,

cross-tabbed, and graphed to providedeeper analysis


7/52

7

Survey Cons

Surveys:

Take time to plan for, design, and collect

and analyze responses

Can yield unreliable or unusable

information if poorly planned or designed

Are often ignored by targeted users


8/52

8

Questions to Ask Yourself

What question do I need answered,

or what decision do I need to make?

What data or feedback will help meanswer that question or make that

decision?

Is a survey the best way to get thatdata or feedback?


9/52

9

What survey tool is right for me?


10/52

10

Paper or Online?

Paper-survey tools

Desktop-publishing software (e.g., Word)

Optical-scan readers (e.g., Scantron)Online-survey tools

Survey software (e.g., Zoomerang)

Web-design tools (e.g., Dreamweaver)Widgets and templates in social-

networking software (e.g., Google Docs)


11/52

11

A Myriad of Options


12/52

12

How to Pick

Price

Maximum numbers of:

SurveysQuestions

Responses

FeaturesStorage

Support


13/52

13

Basic Tool Features

Battery of common question types

Save/copy/edit surveys

Launch via email or URLView survey results online

Knowledge base/FAQs/tutorials


14/52

14

Advanced Tool Features

Survey templates you can adapt

Skip/branch logic

Customizable appearance and messages

Upload and manage email lists

Filter and cross-tabulate results

Summary reports in several formats

Export survey dataPhone and email tech support


15/52

15

Questions to Ask Yourself

How often will I create surveys?

How many people will I typically be

surveying?How complex will my surveys be?

How long will I need to save my data?

What combination of pricing andfeatures would best meet my needs?


16/52

16

How do I designan effective survey?


17/52

17

Survey-Design Process


18/52

18


19/52

19

Determine Your Goal

What do I want to know?

A clear goal:

Tells you what information to collectHelps you identify participants

Helps you write questions

Begins with analysis in mind


20/52

20

Sample Goal Statements-1

Investigate community needs

What gaps exist in the continuum ofhomeless services in our town?

Design new programs and servicesWhat new recreation classes would localseniors be most interested in?

Evaluate impacts and outcomesHow many of our graduates find a job orgo on to college within a year?


21/52

21

Sample Goal Statements-2

Assess constituent satisfaction

What do our AmeriCorps members thinkof our new staff mentoring program?

Plan eventsWhat topics do our partners want includedon the agenda for the gang summit?

Test communication media andmessages

Does our website meet visitors needs?


22/52

22

Determine Your Audience

Who can give me the data or feedback

I need?

Audience analysis:Identifies potential participants

Helps you figure out how to reach them

Determines the number of responses youneed


23/52

23

Determine Your Methodology

Key questions to ask

Number of responses needed

Print or online surveyHow will be marketed

Offer incentive or not

Format raw data will need to be in

Format of finished analysis


24/52

24


25/52

25

General Design Tips

Introduce the survey

Keep it short

Move from broad to narrow questionsPut the most important questions first

Limit use of open-ended questions

Keep answer schemes consistent

End with profile questions


26/52

26

Introduction Example


27/52

27

Question Types

Multiple Choice-Single Answer


28/52

28

Question Types (contd)

Multiple Choice-Multiple Answer


29/52

29


Rating Scale-Single Answer

Note consistent scaling


30/52

30


Rating Scale-Table/Matrix


31/52

31

Choose Words Carefully

Make sure your survey questions are:

Brief, clear, and direct

Free of jargon or confusing languageAsking only one thing each

Free of bias


32/52

32

Question Example 1

What do you think?


33/52

33

Question Example 1 (contd)

biased jargon/unclearbiased

assumption/leadingdouble-barreled

question


34/52

34


Try this


35/52

35

Question Example 2

What do you think?


36/52

36


slightly

unclear


37/52

37


Try this


38/52

38

Skip/Branch Example


39/52

39

Skip/Branch Plan


40/52

40


41/52

41

Deployment Options

Send link via email

Embed URL in webpage or domain


42/52

42

Pre-Test Your Survey

Have colleagues test/review:

Link (make sure it works)

Email invitation subject line and textQuestions/instructions

Appearance and navigation


43/52

43


44/52

44

Monitor Your Survey

Review real-time collection stats

Send reminders if needed


45/52

45


46/52

46

View Quick Reports . . .


47/52

47

. . . Or Download Full Ones


48/52

48

Some Can be Customized


49/52

49


50/52


51/52

51

Thank you, and please contact me with

any questions

Erich Stiefvater

(503) 275-0761

[email protected]

Happy Surveying!


52/52

Resources and References

Zoomerang. (n.d.). 10 Tips to Improve Your Surveys. (Online).

Retrieved February 22, 2009 from http://www.zoomerang.com.

Griffiths, K. & Toledano, Y. (August 26, 2008). Creating effective online

surveys. (Online). Retrieved February 23, 2008 from

https://cc.readytalk.com/cc/schedule/display.do?rfe=wviry6spmzb3&udc

=pi7os9o8bpc7.

Nielsen, J. (February 2, 2004). Alertbox: Keep online surveys short.

(Online). Retrieved February 24, 2008 fromhttp://www.useit.com/alertbox/20040202.html.

Kasunic, M. (September 2005). Designing an effective survey. (Online).

Retrieved February 24, 2008 from

http://www.sei.cmu.edu/pub/documents/05.reports/pdf/05hb004.pdf.

Dillman, D.A. (2007). Mail and internet surveys: The tailored designmethod. Hoboken, NJ: John Wiley & Sons, Inc.

Erich NW Network 2009 Session -- Measuring and Monitoring With Surveys

Documents