Survey Research: Design, Samples and Response Mayyada Wazaify, PhD Reference: 1. Smith F. Survey Research: Design, Samples and Response In: Research Methods.

Survey Research: Design, Samples and Response Mayyada Wazaify, PhD Reference: 1. Smith F. Survey Research: Design, Samples and Response In: Research Methods in Pharmacy Practice. 1 st Ed, 2002:pp:1-31 2. The School of Pharmacy. Queens University of Belfast. An Introduction to Research Methods in Pharmacy Practice. 2000 3. Abudahab S. Clinical Research Course/ Faculty of Rehabilitation Sciences. University of Jordan. 2009

Survey verb (of a person or their eyes) look carefully and thoroughly at (someone or something), especially so as to appraise them. noun a general view, examination, or description of someone or something.

Surveys are appropriate when the question requires obtaining information FROM subjects, rather than measuring performance

Introduction What are surveys? A survey is composed of a series of questions that are posed to a group of subjects, and may be conducted as an oral interview or a written questionnaire In clinical research, surveys are used to: Describe attitudes and values Describe levels of knowledge or experience Describe current practices Describe characteristics of specific groups Assess outcomes related to function and quality of life (Standardized questionnaires e.g. SF-36)

Response Rate Ideally 66% In interviews >> questionnaires Drop and pick Q >> than postal Q Face-to-face interviews >> telephone interviews Can be increased if: Questionnaire is well-designed Enclose a friendly cover-letter/cover memo/introduction (background of the survey, subjects help is vital to the success of the project) or send an advance letter Enclose a stamped envelope Explain how respondents have been selected

Interviews and Questionnaires Interviews Face-to face or over the phone Face-to-face tends to be more effective Major disadvantage: cost and time Structured vs. Semi structured vs. In-depthSemi structured Structured: consists of standardized set of questions that will be asked The three main pitfalls that can occur during qualitative research work are: - equipment failure, - environmental hazards and - transcription errors (Easton et al., 2000).

Exercise-1: give examples on structured, semi-structured and in- depth interviews relating to each of the following topic: - Barriers and facilitators of attending CE courses - Identifying and treating drug misusers and abusers in community pharmacy

Face-to-face interviews

Advantages Can provide explanations if required Better response rate Can explore issue in-depth Gain trust and confidence Opportunity to probe Disadvantages Questionnaire may influence the interview Time consuming expensive

Interviews and Questionnaires Cont. Questionnaires Structured surveys that are self-administered using pen and paper Advantages: More efficient than interviews Data can be gathered from a large sample over wide geographical distribution in relatively short time Standardized everyone is exposed to the same questions in the same way Provide anonymity Disadvantages: Misunderstanding or misinterpreting questions or response choices Unknown accuracy or motivation of respondent

Self-report Respondents report of their attitudes, knowledge, practices, etc. Recall bias is of concern when the respondents are asked to remember past events You need to understand your target population and the respondents abilities to answer your questions Do you know what is proxy report ?

Design of Surveys The research question: what do you want to assess? Guiding questions Delineate what the researcher is trying to find out Reflect descriptive interests or expected relationships among variables How much do mothers of children with asthma know about the asthma? How did mothers of children with asthma cope with their children special needs?

Design of Surveys Cont. The research question Cont. Hypotheses Mothers of children with asthma will demonstrate more knowledge about asthma more than mothers who do not have any child with asthma Questionnaire outline Listing of all items needed to answer guiding questions and/or hypothesis Often more than one item is needed to address each question and/or hypothesis Demographic information (income, ethnicity, marital status)

Design of Surveys Cont. Review of existing instruments Apply or adapt already existing instruments Designing the instrument Write all the questions that reflect the purpose of the study Group and organize questions to reflect each category or topic Watch the flow of your questions Questions should proceed from general to specific Less sensitive questions should precede more sensitive ones Make it as friendly as possible, the less complicated the more responses you will get

Design of Surveys Cont. Preliminary drafts Panel of colleagues for review (Face Validity) Constructive criticism Ongoing process may take several times Pilot testing and revisions Small representative sample (5-10 individuals) Interview respondents to determine where questions were unclear or misleading Monitor missing questions and inconsistencies Monitor time Length of questionnaire In case of interviews --- the interviewers should be trained on how they present the survey, how questions are asked, and how probing follow-up questions are used (if they are to be allowed)

Design of Surveys Cont. Administration of the Survey As much as possible, a probability sample should be selected For interview surveys limited within a local geographic area Introduce the Survey include a Cover letter The purpose and importance of the survey Indicate why the respondents has been chosen for the survey Confidentiality How will the data be used How long it will take Thank the respondents for their participation Sign the letter NO longer than one page

Constructing Survey Questions Open-Ended: Useful for probing respondents feeling and opinions, without biases and limits from the researcher Advantages: Allows a wide variety of responses Good for exploring a topic Does not superimpose answers and expectations Disadvantage: Can be difficult to summaries, code and analyse Unpopular if included in a self-administered Q Time-consuming to answer

Constructing Survey Questions Closed-ended Identification questions Selection questions Yes/No questions Advantages: Easier to analyze Good if questionnaire is long Better is motivation of respondent is low Does not descriminate against less articulate subjects Disadvantages: Narrows the answer Restricts spontaneity of respondents Good design of question is important to avoid ambiguity e.g. Two-way question Likert scale Semantic differential scale Check-lists Ranking

Examples: Close-ended questions Format For questions that in on a continuum, provide range of responses (3-5) How important do you think it is to have an orientation program for first-year students? a- Very important b- Important c- somewhat important d- Not important e. Unsure

Constructing Survey Questions Cont. Close-ended questions Cont. Format Cont. When a series of questions use the same format, a grid or checklist can provide a more efficient presentation. For each of the following questions, please indicate your level of satisfaction in the first-year students orientation program: Not satisfied Somewhat satisfied SatisfiedHighly satisfied Unsure Topics discussed Time of the orientation Method of presentation

Constructing Survey Questions Cont. Close-ended questions Cont. Format Cont. Rank-order questions. The following are the topics discussed in the orientation. Please order them in terms of importance to you, from 1 (most important) to 5 (least important) ______ Using the library ______ Dinning options on campus ______ Sports centers and sports activities on campus ______ Campus security ______ Time management

Constructing Survey Questions Cont. Close-ended questions Cont. Format Cont. Branching 1- Did you attend the opening ceremony of the orientation? a. No Skip to Question 4 b. Yes 2- Do you believe the ceremony was well-organized? a. No b. Yes Skip to Question 4 3- what are the reasons that made you feel that the ceremony was not organized? Please list as much as possible

Constructing Survey Questions Leading questions They are worded so that they are not neutral; they suggest what answer should be given and indicate the questioners own point of view. In questionnaires, leading questions must be avoided as they will bias the responder to give the answer that the questioner wants. e.g. As a caring parent, would you not agree that you should provide your family with a private health care? YES.. NO

Wording of Questions Simple, Simple, Simple Sentences should be concise and grammatically correct Common language for the lowest educational level that might be encountered The shorter the better BUT keep in mind clarity Avoid idioms or sublet cultural expressions / Avoid Jargon The Response Rate (e.g. not too many open-ended) Provide definitions Avoid double-barrelled or double negative questions

Scales Purpose: to distinguish among people who demonstrate different intensities of the characteristics that is being measured Types Categorical: based on nominal measurements Used for variables such as gender, ethnicity, religion, etc. Continuous: represent characteristics that exist on a continuum Ordinal: variables are collapsed into ranks (mild, moderate, severe) Summative: presents a total score with all items contributing equal weight to total Cumulative: demonstrate an accumulated characteristic, with items representing an increasing amount of the attribute being measured

Scales Cont. Likert Scale Summative scale Most often used to assess attitudes or values Includes 5 categories: Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree Modified to have 7 categories: Somewhat Agree, Somewhat Disagree. Or 4 categories: eliminating Neutral Each choice along the scale is assigned a point value Actual values are not important as long as there is consistency Positive items are coded 5 through 1 while negative items are coded 1 through 5 Overall score is computed by adding points for each item Items are measuring the same thing Each item reflects EQUAL element of the characteristic being studied

Scales Cont. Semantic Differential Measure the individual feelings about a particular object or concept based on a continuum that extends between two extreme opposites Pairs of words that reflect opposite feelings 7-point scale is used; middle representing neutral position Different from Likert scale: 2 extremes are labeled Continuum is not based on agree/disagree Assessed by assigning a value from 1-7 to each of the spaces within each adjective pair (1 negative and 7 positive) Alternate the position of negative and positive values Slow Fast

Scales Cont. Visual Analogue Scale (VAS) Simplest method to assess the intensity of a subjective experience A 100mm line is usually drawn, with word anchors on either end that represent extreme of a characteristic Intermediate levels along the line are NOT defined Respondents are asked to mark along the line corresponding to their perceived level for that characteristic VAS measures the distance of the mark from the left hand anchor in millimeters No Pain Pain as bad as it can be

Scales Cont. Guttman Scales Cumulative scale Presents an increasing intensities of the characteristic being measured One dimension within a set of responses If we have 6 items that represent the same dimension, the respondent who agrees with item 2 will also have agreed with item 1. Only ONE way to achieve the same score in a Guttman score Each item is scored as 1 = Agree or 0 = disagree 1- I believe that this country should allow more immigrants in. 2- I would be comfortable with new immigrants moving into my community. 3- It would be fine with me if new immigrants moved onto my block. 4- I would be comfortable if a new immigrant moved next door to me. 5- I would be comfortable if my child went to school with a new immigrant. 6- I would permit a child of mine to marry an immigrant.

Analysis of Survey Data Collate responses and enter then into a computer Each item on the survey is a data point, and must be given a variable name (usually item number) Researcher should go through each survey, making sure that the respondents has correctly filled out the survey Code closed ended questions Descriptive statistics are used to summarize responses Quantitative data: averages and SD Categorical: Percentages Should always be reflected to the total sample size Cross-tabulation Chi-square analysis

Exercise-2: How to code open-ended questions in surveys?

e.g. How would you like the service provided by your community pharmacy improved? 1. Give your customers somewhere to sit 2. Provide a quiet area to discuss health issues privately 3. Lower prices 4. More opportunity to talk with pharmacist 5. Faster dispensing of medicines 6. Less out of stock medicines which we have to call back for 7. More counter staff 8. Wider aisles to allow prams 9. Friendly, more helpful staff 10. Wider product range Cluster the responses into the following 4 categories: Improved dispensing service 4,5,6 Improved shop design 1,2,8. Increased product range/Lower price 3,10. Better trained staff 7,9. Consider the following answers which were obtained from the above survey open-ended question.

Informed Consent Obtain a consent before you collect any dataconsent

Exercise-3: In surveys related to events, collecting data can be conducted by 2 methods: self-completion by participants (e.g. diaries) or by the researcher (e.g. direct observation of the practice setting or retrospective analysis of records). List down the main advantages and disadvantages of each method

Survey Research: Design, Samples and Response Mayyada Wazaify, PhD Reference: 1. Smith F. Survey Research: Design, Samples and Response In: Research Methods.

Documents

performance slide

interviews questionnaires

community pharmacy slide

survey research

motivation of respondent

questionnaires interviews

interviews data

depth interviews