Copyright Shanna Smith & Tom Bohman (2003). This work is the intellectual property of the authors. Permission is granted for this material to be shared.

Copyright Shanna Smith & Tom Bohman (2003).

This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes,

provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To

disseminate otherwise or to republish requires written permission from the authors.

Learning about your IT community via online surveys

Shanna E. Smith, Ph.D.

Thomas M. Bohman, Ph.D.ITS Research Consulting

The University of Texas at Austin

Four key topics

• Defining and maximizing survey benefits

• Online surveys: Advantages and disadvantages

• A multiple-method approach

• Realizing survey benefits

Potential Survey Benefits• Documents services

– Usage of current services– Perceived importance of services

• Evaluates services– General satisfaction– Problem areas

• Informs managerial decisions– Reactions to policy– Needs assessment and marketing

• Communicates care and concern

Designing to Maximize Benefits

• Buy-in is required– Sponsor– Other stakeholders– Developer

• Questions must be relevant and useful

• Questions should adhere to social science research guidelines and practices

• Sampling and data collection must be unbiased to allow generalization of results

Possible Survey Methods

Telephone Mail Online

Online Surveys

• Advantages:– Inexpensive, efficient data collection– Questionnaire flexibility

• Disadvantage:– Possibility of sample bias

Respondent currentlyuses• austin.utexas.edu or forum.utexas.edu• uts.cc.utexas.edu• a non-UT fee based account• and ‘my other e-mail account’

The user answerssatisfaction questions onlyregarding the services that arecurrently used.

Online Surveys

• Advantages:– Inexpensive, efficient data collection– Questionnaire flexibility

• Disadvantage:– Possibility of sample bias

Generalizing to your Population

Population: all the people to whom you want

to apply your survey results. For example:

• All students who use computer labs at your University

• All student residents at your University

• All faculty teaching graduate courses at your University

A Population

A representative sample

A biased sample

Who might be more likely to respond to your online survey?

• People who more widely use computers and the Internet

• People who already use your services

A multiple-method strategy

• Choose a random sample• Send an initial contact through postal mail • E-mail all respondents with an invitation to

complete the survey online• For those without valid e-mail addresses,

send a paper version of the survey• Send reminders!• For those who do not respond within two

weeks, attempt a telephone interview

ITS Survey 2002 Responses

Telephone24%

Paper3%

Web73%

Realizing Survey Benefits

• Report results in timely, appropriate fashion– General results– Individualized reports for each stakeholder

• Acknowledge public feedback– Publicly available results– Press releases

• Keep interest piqued

ITS Survey 2002 Results Areas• Usage and Satisfaction: Familiarity, usage, satisfaction with core

services, e.g., Internet access, e-mail, Help Desk• Computer Labs: Usage of student computer labs and satisfaction with

lab software, hardware, proctor helpfulness, etc. • Technical Support: Types of computer problems experienced by

faculty and staff, and where they go for help• Computer Use: Purchase information, operating system, age, and other

details regarding home and office computers• Training: Necessity for various types of technical training and

preferences regarding training methods • UT Direct: Usage of UT Direct portal and its various components• Needs Assessment: Importance of e-mail, the campus network, and

other services to respondents' work at UT • Computer Security: Familiarity with ITS acceptable use policies,

usage of anti-virus software, password sharing behavior• Specific Issues: Questions of interest to particular Directors, e.g.,

familiarity with Microsoft Campus Agreement.

Example: the Help Desk and Instant Messaging

• Faculty and staff were more likely to use the Help Desk, but all groups who used it were uniformly satisfied with the service.

• Over half of the UT population uses Instant Messaging, but most of them are students.

Instant Messaging

0%10%20%30%40%50%60%70%80%

Fac

ulty

Staf

f

Gra

ds

Und

ergr

ads

% useIM

• Undergraduates are most likely to use IM

• Undergraduates were most satisfied with IM (94% said they were ‘satisfied’ or ‘very satisfied’).

For more results. . . See our ITS Survey 2002 website at http://www.utexas.edu/its/surveys/2002/