The Survey as an Assessment Method: Why, when and how surveys provide evidence to inform decision-making Carrie Towns, Office of Institutional Research.

The Survey as an Assessment Method:

Why, when and how surveys provide evidence to inform decision-making

Carrie Towns, Office of Institutional Research and Planning and the Information Services

Assessment CouncilApril 13, 2006

assessment

an ongoing process in which services, resources and performance are measured against the expectations of users, and improvements are made to satisfy user needs effectively and efficiently.

What do we need to know?

Who can tell us?

How can we get the information?

What will it enable us to do?

How much will it cost?

Definition of Survey

An assessment tool/system for collecting information that is used to describe, compare, and explain knowledge, attitudes, and behavior of a defined group or groups of respondents.

Conduct a Survey When:

Need input from a large, well-defined group.

Have a set of focused questions developed to meet a specified objective.

Results will inform specific decisions.Time and other resources permit.

Characteristics of a Good Survey

Specific objectivesStraightforward questionsSound research designAppropriate resources

Types of Surveys

Information gathering attitudes/opinion/behavior

Market research toolsPublic relations toolsEducational tools

Survey Process

Set survey context – specify objectivesEstablish target audience – sampling

frameDetermine modePrepare cost estimateEstablish tentative calendarDesign survey

Survey Process cont.

Develop questionsPlan for analysisField testAdminister surveySummarize and interpret dataReport results

Survey Context – Specify Objectives

Who is asking for the information?What do they want to learn?How will the information be used?Starting point – existing instrument?

Establish Target Audience/Sampling Frame

Who can provide the information?Type of analysis needed – how detailedNumber of subgroups of interestPlans for follow-ups

Modes of Surveys

WebPaper and pencilTelephoneFocus Groups

Prepare Cost Estimate

OIRP professional time $25/hourCommercial tools and associated costsOther costs

Establish Tentative Calendar

DevelopmentField testingAdministrationProgramming and analysisReporting

Design Survey

Length and layoutOrganizationQuestion formats

Characteristics of Good Questions

Make sense to the respondent Are concrete Use conventional language Avoid emotionally-charged language Avoid negative phrasing Ask for only one piece of information Have a specific purpose

Plan for Analysis

Presentation of data drives item development

Different audiences require different level of analysis

Field Test/Revise and Fine Tune Instrument

Test drive instrument

Administer SurveyData collection phase

Summarize Data

Examine the dataRun preliminary analysisDig in – interpret and draw conclusions

Report ResultsWritten/oral

Information Services Assessment Council members

Susanne Clement, Libraries

Jill Glaser, IT

Ryan Papesh, NTS

Thelma Simons, IT

John Stratton, Libraries

Bill Myers, IS

Call on ISAC members to:

Consult, advise and assist in the development of assessment initiatives.

Identify other campus resources for assessment-related services.

Provide oversight and assure coordination with other IS assessment activities.