Software/Tools For Running Web Experiments By Robert Slater Texas A&M -Corpus Christi.

Software/Tools For Running Software/Tools For Running Web ExperimentsWeb Experiments

ByBy

Robert SlaterRobert Slater

Texas A&M -Corpus ChristiTexas A&M -Corpus Christi

ConsiderationsConsiderations

SkillsSkills TechnologyTechnology Data PrivacyData Privacy FeaturesFeatures

SkillsSkills

What is your proficiency?What is your proficiency?– HTML, ASP, VB, JAVA, NONEHTML, ASP, VB, JAVA, NONE

Hire external partyHire external party– Do they understand what you are doing?Do they understand what you are doing?– How important is their friendship?How important is their friendship?

Learn new skills?Learn new skills?– learning curve.learning curve.

TechnologyTechnology

What type of computer systems will access What type of computer systems will access the software?the software?– Windows XP, ME, 2000, 98, 95, 3.1, DOS, MACWindows XP, ME, 2000, 98, 95, 3.1, DOS, MAC

Will the code you have work on the different Will the code you have work on the different operating systems or web browsers?operating systems or web browsers?

Do you have a web-server you can use to Do you have a web-server you can use to host your study?host your study?

Would a third party notify you in advance of Would a third party notify you in advance of server maintenance?server maintenance?

Data PrivacyData Privacy

Where will the data be stored and who will Where will the data be stored and who will have access to it? have access to it?

If you use a third party hosting site can you If you use a third party hosting site can you keep your data confidential?keep your data confidential?

Can you control backup copies of your Can you control backup copies of your data?data?

FeaturesFeatures

What kind of features do you need in your study? What kind of features do you need in your study? I needed:I needed:

– Randomization into treatment groupsRandomization into treatment groups– Piping with mathematical manipulationPiping with mathematical manipulation– Multiple Question formatsMultiple Question formats– Data ExportData Export– CustomizationCustomization– User sign in options (specific/anonymous)User sign in options (specific/anonymous)– Java script capability within survey pagesJava script capability within survey pages

My SolutionMy Solution

Adapted an open source survey software to meet Adapted an open source survey software to meet experimental needs (www.ClassApps.com)experimental needs (www.ClassApps.com)

ASP based open source softwareASP based open source software Multiple database support – Multiple database support –

– Supports data export to excelSupports data export to excel

Professional looking finished productProfessional looking finished product No programming experience neededNo programming experience needed Quick learning curveQuick learning curve

SelectSurveyASPSelectSurveyASP

SelectSurveyASPSelectSurveyASP

Finished ProductFinished Product

Survey OptionsSurvey Options

Export OptionsExport Options

Limitations and SolutionsLimitations and Solutions

Software not designed for multiple Software not designed for multiple treatments:treatments:– Create multiple questionnaires based on Create multiple questionnaires based on

treatment groups treatment groups Created one master survey and copied it to create Created one master survey and copied it to create

separate treatmentsseparate treatments

Software would not randomize subjects:Software would not randomize subjects:– Use separate web page and basic java script to Use separate web page and basic java script to

randomize subjects into treatment groupsrandomize subjects into treatment groups Software allowed hyperlinks to surveysSoftware allowed hyperlinks to surveys

Multiple TreatmentsMultiple Treatments

Limitations and Solutions (cont)Limitations and Solutions (cont)

Software allowed for piping but not Software allowed for piping but not mathematical manipulation as needed in my mathematical manipulation as needed in my study. study.

I needed to be able to:I needed to be able to:– Take users estimate of Earnings Per ShareTake users estimate of Earnings Per Share– Give them management’s estimate that was Give them management’s estimate that was

calculated as a percentage of user’s response.calculated as a percentage of user’s response.

PipingPiping

Take data input by participants and use in Take data input by participants and use in the study.the study.– Personalize questionsPersonalize questions– To develop rankings using data input by users.To develop rankings using data input by users.

Taking User InputTaking User Input

Using the InputUsing the Input

Capture Input for RankingCapture Input for Ranking

Ranking The Captured InputRanking The Captured Input

Piping SolutionPiping Solution

Program already allowed for piped Program already allowed for piped responses.responses.

Open source software allowed me to Open source software allowed me to manipulate the ASP source code to perform manipulate the ASP source code to perform mathematical function on piped response. mathematical function on piped response.

Users Input EPSUsers Input EPS

Design ScreenDesign Screen

Pipe InstructionsPipe Instructions

Actual Question TextActual Question Text

User View Piped ResponseUser View Piped Response

ConclusionsConclusions

Quick learning curve. Little expertise necessary.Quick learning curve. Little expertise necessary. Customizable software allowed for unique Customizable software allowed for unique

functionality.functionality. Locally housed database on web server allowed Locally housed database on web server allowed

for data privacy and further customized data for data privacy and further customized data gathering. (Timed responses)gathering. (Timed responses)

Customized programming problems led to Customized programming problems led to incompatibility with some operating systems. incompatibility with some operating systems. – Participants were dropped from the study before Participants were dropped from the study before

completion. completion.

Other Commercial SoftwareOther Commercial SoftwareZoomerang™ SuperSurvey Greenfield Online

Infopoll Perseus PollCat

Inquisite Cool Surveys Survey System

Apian Software Hosted Survey SurveyView

StatPac SurveyGold SurveySite

InstantSurvey EZSurvey Mercator

SurveyCrafter PollPro LiveSurveys

InSite ObjectPlanet SurveySaid

QuickSenseActive Websurvey SumQuest

CustomerSat StatSurvey SurveyMonkey

*(list of survey companies found on www.surveymonkey.com/Pricing.asp)

Recent Academic Studies on Web Recent Academic Studies on Web ExperimentsExperiments

Bryant, B., Hunton, J., and Stone, D., 2004, Bryant, B., Hunton, J., and Stone, D., 2004, “Web Experiments: Prospects and “Web Experiments: Prospects and Possibilities for Behavioral Accounting Possibilities for Behavioral Accounting Research,” Behavioral Research In Research,” Behavioral Research In Accounting, 16: 1-24Accounting, 16: 1-24

Birnbaum, M., 2004, “Human Research and Birnbaum, M., 2004, “Human Research and Data Collection via The Internet,” Annual Data Collection via The Internet,” Annual Review of Psychology, 55: 803-32Review of Psychology, 55: 803-32