NECC 2007 - MP356 Creating Web-Based Online Surveys.

NECC 2007 - MP356

Creating Web-Based Online Surveys

2

Jeff Sun [email protected]

www.sun-associates.com/necc2007

You? Who’s working on projects now? Existing survey work?

3

Objectives

To understand the role of data collection in an overall evaluation process

To understand the role of surveys as one tool in an evaluation data collection process

To become initially familiar with tools for creating online surveys

Others?

4

Why Evaluate?

To fulfill program requirementsProject sustainabilityGeneration of new and improved project

ideas

5

By Definition, Evaluation…

Is both formative and summative Helps clarify project goals, processes, products Should be tied to indicators of success written for your

project’s goals Is not a “test” or simply a checklist of completed

activities Qualitatively, are you achieving your goals? What adjustments can be made to your project to

realize greater success?

6

Data Collection vs. Evaluation

Evaluation is more than data collectionEvaluation is about…

Creating questions Creating indicators Collecting data Analyzing and using data

Data collection occurs within the context of a broader evaluation effort

7

An Iterative Process

Evaluation breaks your vision down into increasingly observable and measurable pieces.

8

Tools and Techniques

Surveys, Focus Groups and Interviews Measuring satisfaction and determining attitudes

ObservationsArtifact AnalysisTriangulation with data from administrators and

staff Do other groups confirm that teachers are being

served?

9

Surveys

Online vs. Paper Is there sufficient connectivity? Often works best if people complete the

instruments all at the same time Same goes for paper surveys

Online surveys provide immediate dataSpreadsheets which can be exported to a

variety of different programs for analysis

10

Surveys

Online VIVED Profiler LoTi Zoomerang SurveyMonkey.com

11

Make Your Own!

www.sun-associates.com/surveyws/surveys.html www.sun-associates.com/necc2007/monday/y2survey.html

Based on a CGI script on your web serverOutputs to a text file, readable by ExcelWorks with yes/no, choose from a list, and free text

input (no branching)

12

The Survey Script

www.bignosebird.com/carchive/bnbform.shtml

You need access to your WWW server’s directory structure If you do not have this, you should use some of the

commercial survey services discussed earlier

You need FTP access to your WWW serverThe bnbform.cgi script goes in your /cgi directoryThe readme.txt file contains all the installation info

necessary

13

Once Installed

Surveys are HTML forms that make use of the POST command <FORM METHOD="POST"

ACTION="/cgi-bin/bnbform.cgi" ENCTYPE="x-www-form-urlencoded">

Let’s examine an example… www.sun-associates.com/necc2007/monday/y2survey.html

14

For Today’s Exercise

Create a folder called <yourinit>testsite on your desktop

In that folder, create a folder called wwwIn the /www folder, create a folder called

<yourinit>test

15

www.sun-associates.com/necc2007/monday/y2survey.html

Save this as “sample.html” as Page SourcePlace that file in your <yourinit>test folder

16

Start Dreamweaver /Site/New SiteLocal Root Folder = <yourinit>testsiteHttp address = www.sun-associates.comRemote Info

Access = FTP Host = ftp.business.earthlink.net Host Dir = leave blank Login = wfp44289%sun-associates.com Pswd = neccmp356

17

Creating Forms

Dreamweaver (or another HTML authoring tool) is recommended

It’s best to start with a completed survey form and then modify That way, you get all of the correct <form>

tags and hidden files You can use the previous example as your

template (just Save As… to save the html) Let’s do that now

18

Things to Do

(in “Design” view) Delete text between first line on the page and the text just before the SUBMIT button

Create new form elements in the resulting space Pull Down List Text Area Radio Button Pair (yes/no)

Modify the hidden elements (in “Code” view) at the end of the source code

<P><!-- SCRIPT CONFIGURATION SECTION --><INPUT TYPE="hidden"NAME="data_order" VALUE="district,other_dist,…8i"><INPUT TYPE="hidden" NAME="outputfile" VALUE="neccwsy2.log"> <INPUT TYPE="hidden" NAME="form_id" VALUE="SAMPLE Y2 Expectations Survey"> <INPUT TYPE="hidden" NAME="ok_url" VALUE="http://www.sun-associates.com/thanks.html"><INPUT TYPE="hidden" NAME="not_ok_url" VALUE="http://www.sun-associates.com/sorry.html"><!-- END OF SCRIPT CONFIGURATION SECTION -->

19

Survey Considerations

There actually is a science involved in survey creation

Issues of validity both in questions and instruments

Sample size Attaining a sufficiently large sample to test

validity Piloting

20

Survey Tips

Keep them short (under 10 minutes)Avoid huge long checklistsAllow for text commentsSupport anonymity

But allow for categorical identifications -- school, job function, grade, etc.

21

Coordinate and support survey administration Avoid the “mailbox stuffer” Work with building leaders Provide clear deadlines

22

Three Big Points

Surveys alone mean nothing TRIANGULATE!

100% response rate is virtually impossible On the other hand, nearly 100% is very

possible if you follow our tips!

Share the data No one wants to fill in forms for no purpose

23

Data Dissemination

Compile the reportDetermine how to share the report

School committee presentation Press releases Community meetings

24

10 Tips for Data Collection

Ground your data collection within a larger evaluation framework Know why you’re asking what you’re asking

Not all data is quantitativeCast a wide net

It’s all about stakeholders

Dig deep Try to collect the data that can’t easily be observed or

counted

25

Use confirming sources Triangulate! Surveys alone do nothing.

Have multiple writers Stakeholders and different perspectives

Think before you collect Choose questions carefully and with regard to

what you really expect to find

26

Set (reasonable) expectations for participation Time and effort

Forget about mailbox surveys Usually waste more time than their value

Report back Don’t be a data collection black hole!

27

More Information

[email protected] 978-251-1600 ext. 204

www.sun-associates.com/evaluationwww.sun-associates.com/necc2007www.edtechevaluation.com