Different Approaches for Data Collection/Data Analysis Experiment Survey Quasi-Experiment Secondary Data Analysis Content Analysis Historical Comparative.

Different Approaches for Data Collection/Data Analysis

• Experiment• Survey • Quasi-Experiment • Secondary Data Analysis• Content Analysis• Historical Comparative

Methods (Archival Analysis )

• Participant Observation • Ethnography• Individual Qualitative

Interview• Textual Analysis • Focus Group Discussion • Del-Phi Method

Secondary Data Analysis

Secondary Data Analysis• A type of research in which data collected by others are

reanalyzed. Primary Data Analysis• Original analysis of the data collected in a study. Meta-Analysis• “Analysis of analysis” • Quantitative procedure for summarizing or integrating the

findings obtained from a literature. • Uses the results of individual research projects on the same

topic as data points for a statistical study of the topic.

Secondary Data Analysis Advantage• Saves cost for data collection. • Saves time required for data collection. • May obtain data otherwise impossible.

Disadvantage• Data collected may not be suitable for the researcher’s purposes.

(Validity)• All necessary data may not be available in existing data. • Requires time to search for the data set. • Original data set may not be accurate. • Existing data sets may contain more variables than what the

researchers need. Need time to sort out the data.

Content Analysis

General usage of the term • Refers to any of several research techniques used to

analyze the content of written, spoken, or pictorial communication such as books, newspapers, television programs, or interview transcripts.

Specific usage of the term• Refers to a quantitative technique that attempts to

quantify the meaning of the content of the written, spoken or pictorial communication.

Content Analysis1. Determine “Unit of Analysis”2. Determine “Indicators”3. “Code” the data

ExampleResearch Question: What values and beliefs were expressed

by advocacy coalition about Social Security? Unit of Analysis: Congressional testimony on Social

Security reform from 1983 to 2004. Indicators: Values---Statement on what they like. Beliefs---Statement on what things should be.

Content Analysis

1. Issue of “Sampling” e.g. Randomly sample 20 congressional testimony each

month.

2. Issue of “Coding” • Coding Sheet • Intercoder Reliability

Exercise: Content Analysis of Course Syllabus

• Content: PA410U Syllabus

• Coding: Civic Capacity (CCI ) Initiative Matrix

Historical Comparative Methods (Archival Analysis)

• Focus on identifying historical trends or patterns. • Focus on comparing the historical trends or

patterns.

Quantitative approach: • Secondary Data analysis. • Content analysis of archival data.

Qualitative approach: • Oral history. • Case study.

Qualitative Research Methods

• Examples: • Participant Observation • Ethnography• Individual Qualitative

Interview• Textual Analysis • Focus Group Discussion • Del-Phi Method

Key features of Qualitative Research

1. Collection primarily of qualitative rather than quantitative data Qualitative methods emphasize observations about natural behavior and

artifacts that capture social life as it is experienced by the participants rather than the numerical representations of the categories predetermined by the researcher.

2. Exploratory research question. Qualitative researchers typically begin their projects seeking to discover

what people think and how they act, and why, in some social setting.

3. Inductive reasoning (Reasoning that moves from more specific kinds of statement to more general ones) Only after immersing themselves to many observations, do qualitative

researchers try to develop general principles to account their observations.

Key features of Qualitative Research

4. A focus on human subjectivity. Qualitative methods emphasize the meanings that participants attach to events and that people give to their lives.

5. Reflexive research design. In the qualitative methods, the research design may need to be reconsidered or modified in response to new developments, or to changes in some other component as research progresses.

6. Sensitivity to the subjective role of the researcher. Qualitative researchers should be sensitive to the role they play in the process of data collection. “Researcher as an instrument”

Ways to collect qualitative data

1. Participant Observation 2. Ethnography3. Individual Interview

a) Semi-structured interviewb) Unstructured interview

4. Textual Analysis 5. Focus Group Discussion 6. Del-Phi Method

Participant Observation

•  Sometimes referred to as “fieldwork” in anthropology.

• A method in which natural social processes are studied as they happen.

• Researcher takes a role of “participant”

Ways to collect qualitative data

1. Participant Observation 2. Ethnography3. Individual Interviews

Individual Interviews

Semi-structured interview• Interviews with an interview guide containing

primarily open-ended questions that can be modified for each interview.

 Unstructured interview • The interviewer starts with only a general sense of

the topics to be discussed and creates questions as the interaction proceeds.

Interview Guide

• A list of topics to cover and the order in which to cover them that can be used to guide less structured interviews.

• The questions must be judged its appropriateness for each interview, and may be re-ordered, and re-worded if necessary.

Types of Questions

Open-ended questions vs. Close-ended question Open-ended • “How would you describe the performance of the new Japanese Prime

Minister”?

Closed-ended • “Do you think the new Japanese Prime Minister is doing a good job?”

Probes Questions that allow the interviewee to further expand on

their earlier comments.  • “Can you elaborate on what you mean by . . .” • “Why is that so?”

Ways to collect qualitative data

1. Participant Observation 2. Ethnography3. Individual Interview4. Textual Analysis 5. Focus Group Discussion 6. Del-Phi Method

Focus Groups

• A type of group interview.• Focuses on group interaction on a topic selected

by the researcher. • Ideally 4- 12 participants. • The interaction is directed by a moderator who

asks questions and keeps the discussion on the topic.

 Reference: Dr. David Morgan (Institute on Aging, PSU) • “Focus groups as qualitative research”(1995). Sage. • Annual Review of Sociology 22, p. 129-153 (1996)

Ways to collect qualitative data

1. Participant Observation 2. Ethnography3. Individual Interview4. Textual Analysis 5. Focus Group Discussion 6. Del-Phi Method

Different Purposes for Research • Pure Research : Discovery of knowledge for the sake of

knowledge. • Applied Research: Conducted with some practical

purposes in mind. • Exploratory Research: Conducted in order to

explore the phenomena. • Explanatory Research : Conducted in order to

explain the phenomena.

• Evaluation Research: Conducted in order to develop and “evaluate” programs.

Evaluation research

• Research specifically designed to assess the impact of a specific program, policy or legal change.

• Often the focus of an evaluation is whether the program, policy, or law has succeeded in effecting intentional or planned change.

Developing and Evaluating a Program

Research Research Program Program

Needs assessmentDesign a program

Implement a programProcess Evaluation

Outcome Evaluation

Redesign a program Or Terminate a program

Types of Evaluation

1. Needs Assessment • An Analysis of whether a problem exists, its severity, and an

estimate of essential services.

2. Process Evaluation/Formative Analysis • Research that monitors a program or policy to evaluate the

process of implementation of the program. • Provides the program staff with evaluation of the current

program and suggestions for improving the program design and service delivery.  

Types of Evaluation

3. Outcome evaluation (Impact analysis, summative analysis)

• Research that is designed to “sum-up” the effects of program, policy, or law in accomplishing the goal or intent of the program, policy or law.

4. Cost-Benefit Analysis • Research that compares a program’s costs to its benefits.

• A study designed to weigh all expenses of a program (its costs) against the monetary estimates of the program’s benefits (even putting dollar values on intangible benefits.)

Selecting a study design

Quasi-Experiment • An experiment design that is missing one or more

aspects of the classic controlled experiment.

Panel Study (Longitudinal Study)

A study design in which data are collected about one sample at least two times and all variables are measured, not controlled by the researcher.  

Selecting a study design

Cross-sectional Study • A study design in which data are collected once

for all the variables of interest using one sample.• Case Study A research strategy that focuses on one case (an

individual, a group, an organization, and so on) with in its social context at one point in time, even if that one time spans months or years.

Summary Summary

Group Exercise Group Exercise