Top Banner

of 37

Survey Research in Design

Jan 27, 2015




Presentation to the Master's of Design students at Ontario College of Art and Design, Oct. 14, 2010.

  • 1. Using Surveys in Design
    Sam Ladner, PhD
    October 14, 2010

2. Key Themes
When to use surveys in design
Qualvs quant
From interviews to instruments
Survey basics
Should we do a survey?
Asking good questions
The limits of surveys

  • Creativity

4. Complexity 5. Compromise 6. ChoiceThe Four Cs ofResearch
BorjaDe Mozota, Brigitte. 2003. Design Management: Using Design to Build Value. New York: All Worth Press.
7. Make your choice:
8. What research design would you choose?
9. Quantitative Research in Design
10. What do quant researchers worry about?
I really spend a lot of time wondering how to measure things.
I want to make sure others can repeat my findings.
I want to know what causes something else.
I wonder how small patterns generalize to big patterns.
11. What do qual researchers worry about?
I really want my research approach to be flexible and able to change.
I want to describe the context in a lot of detail.
I want to see the world through the eyes of my respondents.
I want to show how social change occurs. Im interested in how things come to be.
12. Qualitative vs. Quantitative
13. What is your orientation?
14. A famous sampling mistake
15. Even with proper samplingbeware!
predicting behavior on the basis of knowledge of attitude is a very hazardous venture.
16. What exactly IS a sample?
17. What exactly IS a sample?
A subset of the population, selected by either probability or non-probability methods. If you have a probability sample you simply know the likelihood of any member of the population being included (not necessarily that it is random).
18. Assumptions of quantitative sampling
We want to generalize to the population.
Random events are predictable.
We can compare random events to our results.
Probability sampling is the best approach.
19. Assumptions of qualitative sampling
Social actors are not predictable like objects.
Randomized events are irrelevant to social life.
Probability sampling is expensive and inefficient.
Non-probability sampling is the best approach.
20. What kind of sample would you use for a design project?
21. From interviews to instruments
concepts arecategories for the organization of ideas and observations (Bulmer, 1984: 43)
22. A variable varies
23. What is varying?
24. Concepts and variables
25. What is a measure?
Something that can be counted.
Number of books
26. What is an indicator?
Something that indicates the presence of something else.
Number of hospital visits
27. What is survey research?
Asking structured questions
Print or Online surveys
Telephone surveys
Structured face-to-face interviews
28. Surveys versus other qualitative methods
29. Is a survey the right research method?
I know a lot about the topic
I need to summarize the findings in numerical format
I dont need a lot of detail and nuance
I need to generalize the results to the population
I need to predict the likelihood of a certain thing happening
I want to measure incremental change
I dont really know much about the topic
I dont need to summarize the findings in numerical format
I need to communicate a lot of detail and nuance
Im interested in case study information
Im not planning on predicting the likelihood of anything
I dont need to measure any change
30. Is it time to do a survey?
Clear idea of all possible questions and answers?
Formulated all questions and possible answers?
Do a survey
Pilot test the questions
Discovered questions and answers through
Qual research?
Do more Qual research
31. Asking good questions
Mutually exclusive and exhaustive
No overlap between answers
All possible answers
No double-barreled questions
Wrong: What do you think about our response times and customer service?
Right: What is your opinion of our response time?
Focus on opinions or beliefs of participants
Wrong: How usable is our Web site?
Right: What are users opinions of the sites look and feel?
Close-ended is best
Cuts down on analysis time
Demographic questions at the end
32. The lay of the land: descriptive statistics
Summarizes the responses
What is the most typical response?
How much variation is there between responses?
Standard deviation
Inter-quartile range
33. Predicting behavior: inferential statistics
Infers or predicts future behavior
Which respondents are more likely to
Which factors most influence
34. The tools you need
Online Survey Tool
Statistical Analysis Software
35. The skills you need
Knowledge of the topic
Ability to write good questions
Familiarity with the survey tool
Some basic knowledge of statistics
A framework for writing the report
Important to write the outline of the final report before you start!
36. The payoff youll get
More men are dissatisfied with the experience.
Younger users are much more unhappy with customer service that older users.
Higher-income individuals are the most unlikely to recommend this product to friends and family.
Men are more likely to prefer a two-step process than women.
37. Whats wrong with quant research?
Yeah and they treat people like theyre test tubes or something.
Its the way they do thingsit makes it hard for people to see their research as relevant to them.
Well their research is just so static. Real life actually changes.
They always seem to measure artificial things and say theyre really precise.
38. What wont you learn from your survey?
39. What you wont learn
Why arent people using our product?
How can we innovate on our current design?
Why dont people finish the checkout process?
What are the main characteristics of the people who dont use our product?
40. Copernicus ConsultingDesign research and strategy