Top Banner
Chapter 4 Chapter 4 Conceptualization and Measurement
53

Chapter 4 Chapter 4 Conceptualization and Measurement.

Dec 28, 2015

Download

Documents

Magnus Sparks
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Chapter 4 Chapter 4 Conceptualization and Measurement.

Chapter 4Chapter 4Conceptualization and

Measurement

Page 2: Chapter 4 Chapter 4 Conceptualization and Measurement.

Introduction

IF IT EXISTS ---- IT CAN BE MEASURED Measurement is used to gain mathematical

insight into our data. When we measure we can compare our data to

a standard used for evaluation Through measurement we can inspect, analyze

and interpret our information Measurement is essential to Quantitative

research

Page 3: Chapter 4 Chapter 4 Conceptualization and Measurement.

Conceptualization. The process of specifying what we mean by a term.

A concept is a mental image that summarizes a set of similar observations,feelings or ideas.

Some concepts are more abstract than others

Page 4: Chapter 4 Chapter 4 Conceptualization and Measurement.

Introduction, cont.

Concepts like “social health,” “prejudice,” and even “binge drinking” require an explicit definition before they are used in research because we cannot be certain that all readers will share a particular definition or that the current meaning of the concept is the same as it was when previous research was published.

It is especially important to define clearly concepts that are abstract or unfamiliar.

When we refer to concepts like “social control,” “anomie,” or “social health,” we cannot count on others knowing exactly what we mean.

Page 5: Chapter 4 Chapter 4 Conceptualization and Measurement.

Conceptualization in Practice

If we are to do an adequate job of conceptualizing, we must do more than just think up some definition, any definition, for our concepts (Goertz 2006).

We have to turn to social theory and prior research to review appropriate definitions.

We should understand how the definition we choose fits within the theoretical framework guiding the research and what assumptions underlie this framework.

Page 6: Chapter 4 Chapter 4 Conceptualization and Measurement.

Conceptualization in Practice, cont.

Example: Do you have a clear image in mind when you hear the term youth gangs?

“Neither gang researchers nor law enforcement agencies can agree on a common definition . . . and a concerted national effort . . .failed to reach a consensus” (Howell 2003:75).

If you use the term you must have a clear definition of how you are using it.

Page 7: Chapter 4 Chapter 4 Conceptualization and Measurement.

Conceptualization in Practice, cont.

Decisions about how to define a concept reflect the theoretical framework that guides the researchers.

For example, the concept “poverty” has always been somewhat controversial, because different conceptualizations of poverty lead to different estimates of its prevalence and different social policies for responding to it.

Conceptualization is specifying the dimensions of and defining the meaning of the concept

Page 8: Chapter 4 Chapter 4 Conceptualization and Measurement.

Variables

A variable is any observation that can take different values

Gender, Age, Religion, Ethnicity are variables

Variables have attributes or values

Page 9: Chapter 4 Chapter 4 Conceptualization and Measurement.

Attributes (Values) of Gender are: 1. Male 2. Female The main concept in a question on a

questionnaire is the variable being measured.

Did you vote in the last election. The main concept is Vote.

Page 10: Chapter 4 Chapter 4 Conceptualization and Measurement.

An indicator is the response to a single question

Variable Vote (Did you vote?) Indicator = your answer - Yes The values or attributes of the variable

vote are: 1. Yes, voted or 2. No, didn’t vote.

Page 11: Chapter 4 Chapter 4 Conceptualization and Measurement.

From Concepts to Observations, cont. Operationalization is the process of connecting

concepts to observations. It is the process of choosing the variable to represent the concept.

You can think of it as the empirical counterpart of the process of conceptualization.

When we conceptualize, we specify what we mean by a term.

Example: “Race” is an important concept in social research, but we know it is a social construct and not based on any biological reality. What people mean by “race” has varied over time and from place to place. The term Race is often confused with Ethnicity.

Page 12: Chapter 4 Chapter 4 Conceptualization and Measurement.

From Concrete to Abstract Concepts Concepts vary in their level of abstraction, and

this, in turn, affects how readily we can specify the indicators pertaining to the concept.

Would you now consider “race” to be an abstract or a concrete concept?

A very abstract concept like social status may have a clear role in social theory but a variety of meanings in different social settings.

We often use abstract concepts in our theories: crime, abuse, deterrence, substance abuse

Page 13: Chapter 4 Chapter 4 Conceptualization and Measurement.

From Concrete to Abstract Concepts, cont. Usually, the term variable is used to refer to

some specific aspect of a concept that varies, and for which we then have to select even more concrete indicators.

The important thing to keep in mind is that we need to define clearly the concepts we use and then develop specific procedures for measuring variation in the variables related to these concepts.

Page 14: Chapter 4 Chapter 4 Conceptualization and Measurement.

Measurement Operations

The deductive researcher proceeds from defining concepts in the abstract (conceptualizing) to identifying variables to measure, and finally to developing specific measurement procedures.

We use variables which are derived from concepts in our hypotheses and research questions?

Page 15: Chapter 4 Chapter 4 Conceptualization and Measurement.

PROCESS

Theory – Punishment deters crime We can define the concept crime with

the variable ‘physical spousal abuse’ We can define the concept punishment

with the variable ‘arrest’

Page 16: Chapter 4 Chapter 4 Conceptualization and Measurement.

Will arresting physical abusers of spouses deter repetition of the offense? (Research Question)

If spousal abusers are arrested on 1st offense then recidivism (repeating offense) will be reduced (Hypothesis)

Page 17: Chapter 4 Chapter 4 Conceptualization and Measurement.

Some concepts are more concrete than abstract i.e. Age – time elapsed since birth

But we may have a different meaning in mind – How old in hours, month, years

How we will use the variable is important in defining it

Page 18: Chapter 4 Chapter 4 Conceptualization and Measurement.

Sometimes we may have to use qualitative means to define a concept.

Instead of using a scientific meaning of ‘diversity’ we may want to use the general public’s meaning of diversity for our research or we may want to know the difference in the meaning of rape for women versus men

Page 19: Chapter 4 Chapter 4 Conceptualization and Measurement.

Available versus Empirical Data

Where does our data come from?PRIMARY OR EMPIRICAL DATA – Collected

by the researcher (not from library research)AVAILABLE OR SECONDARY DATA Government reports are rich and readily

accessible sources of social science data. Organizations ranging from nonprofit service

groups to private businesses also compile a wealth of figures that may be available to some social scientists for some purposes.

Page 20: Chapter 4 Chapter 4 Conceptualization and Measurement.

In addition, the data collected in many social science surveys are archived and made available for researchers who were not involved in the original survey project.

Page 21: Chapter 4 Chapter 4 Conceptualization and Measurement.

Using Available Data, cont.

Before we assume that available data will be useful, we must consider how appropriate they are for our concepts of interest.

We may conclude that some other measure would provide a better fit with a concept or that a particular concept simply cannot adequately be operationalized with the available data.

We also cannot assume that available data are accurate, even when they appear to measure the concept in which we are interested in a way that is consistent across communities.

Page 22: Chapter 4 Chapter 4 Conceptualization and Measurement.

Using Available Data, cont.

Government statistics that are generated through a central agency like the U.S. Bureau of the Census are often of high quality, but caution is warranted when using official data collected by local levels of government.

Page 23: Chapter 4 Chapter 4 Conceptualization and Measurement.

Constructing Questions

Asking people questions is the most common and probably the most versatile operation for measuring social variables. i.e. survey research (Quantitative)

Most concepts about individuals can be defined in such a way that measurement with one or more questions becomes an option.

Of course, in spite of the fact that questions are, in principle, a straightforward and efficient means to measure individual characteristics, facts about events, level of knowledge, and opinions of any sort, can easily result in misleading or inappropriate answers.

Page 24: Chapter 4 Chapter 4 Conceptualization and Measurement.

Constructing Questions, cont.

Memories and perceptions of the events about which we might like to ask can be limited, and some respondents may intentionally give misleading answers.

For these reasons, all questions proposed for a study must be screened carefully for their adherence to basic guidelines and then tested and revised until the researcher feels some confidence that they will be clear to the intended respondents and likely to measure the intended concept (Fowler 1995).

Alternative measurement approaches will be needed when such confidence cannot be achieved.

Page 25: Chapter 4 Chapter 4 Conceptualization and Measurement.

Types of Questions Used in Social Research Measuring variables with single questions is

very popular. Public opinion polls based on answers to single

questions are reported frequently in newspaper articles and TV newscasts.

Single questions can be designed with or without explicit response choices. o closed-ended (fixed-choice) questiono open-ended questions

Page 26: Chapter 4 Chapter 4 Conceptualization and Measurement.

Closed Ended –survey provides preformatted response categories for the subject to circle or check

Closed ended: What is your Ethnicity? 1. Anglo 2.Hispanic 3. African

American 4. Asian, 5 Native American

Page 27: Chapter 4 Chapter 4 Conceptualization and Measurement.

Open-ended: respondent replies in his or her own words either by writing or talking

Open ended: How would you describe your ethnic background?

Page 28: Chapter 4 Chapter 4 Conceptualization and Measurement.

Making Observations-qualitative

Observations can be used to measure characteristics of individuals, events, and places.

The observations may be the primary form of measurement in a study, or they may supplement measures obtained through questioning.

Direct observations can be used as indicators of some concepts.

Observations may also supplement data collected in an interview study.

Page 29: Chapter 4 Chapter 4 Conceptualization and Measurement.

Collecting Unobtrusive Measures

Qualitative Unobtrusive measures allow us to collect data about

individuals or groups without their direct knowledge or participation. (Can be qualitative or quantitative)

Eugene Webb and his colleagues (2000) identified four types of unobtrusive measures: physical trace evidence, archives (available data), simple observation, and contrived observation (using hidden recording hardware or manipulation to elicit a response).

Unobtrusive measures can also be created from such diverse forms of media as newspaper archives or magazine articles, TV or radio talk shows, legal opinions, historical documents, personal letters, or e-mail messages.

Page 30: Chapter 4 Chapter 4 Conceptualization and Measurement.

Combining Measurement Operations Using available data, asking questions, making

observations, and using unobtrusive indicators are interrelated measurement tools, each of which may include or be supplemented by the others.

Researchers may use insights gleaned from questioning participants to make sense of the social interaction they have observed.

Unobtrusive indicators can be used to evaluate the honesty of survey responses.

Page 31: Chapter 4 Chapter 4 Conceptualization and Measurement.

Combining Measurement Operations, cont. The choice of a particular measurement method is

often determined by available resources and opportunities, but measurement is improved if this choice also takes into account the particular concept or concepts to be measured.

Questioning can be a particularly poor approach for measuring behaviors that are very socially desirable, such as voting or attending church, or that are socially stigmatized or illegal, such as abusing alcohol or drugs.

People have the tendency to answer questions in socially approved ways.

Page 32: Chapter 4 Chapter 4 Conceptualization and Measurement.

Exhibit 4.9

Page 33: Chapter 4 Chapter 4 Conceptualization and Measurement.

Triangulation

Triangulation—the use of two or more different measures of the same variable—can strengthen measurement considerably.

When we achieve similar results with different measures of the same variable, particularly when they are based on such different methods as survey questions and field-based observations, we can be more confident in the validity of each measure.

Page 34: Chapter 4 Chapter 4 Conceptualization and Measurement.

Levels of Measurement

When we know a variable’s level of measurement, we can better understand how cases vary on that variable and so understand more fully what we have measured.

Level of measurement: The mathematical precision with which the values of a variable can be expressed. (has implications for the type of statistics to be used)

The nominal level of measurement, which is qualitative, has no mathematical interpretation; the quantitative levels of measurement— ordinal, interval, and ratio—are progressively more precise mathematically.

Page 35: Chapter 4 Chapter 4 Conceptualization and Measurement.

Nominal Level of Measurement

The nominal level of measurement identifies variables whose values have no mathematical interpretation; they vary in kind or quality but not in amount (they may also be called categorical or qualitative variables).

Gender is a nominal variable. Nationality, occupation, religious affiliation, and

region of the country are also measured at the nominal level.

Page 36: Chapter 4 Chapter 4 Conceptualization and Measurement.

Nominal Level of Measurement, cont. Although the attributes of categorical variables do

not have a mathematical meaning, they must be assigned to cases with great care. The attributes we use to measure, or categorize, cases must be mutually exclusive and exhaustive:

A variable’s attributes or values are mutually exhaustive if every case has at least one attribute.

i.e. Religion: 1.Catholic, 2. Jewish, 3.Protestant, 4. Other

Page 37: Chapter 4 Chapter 4 Conceptualization and Measurement.

The different values of a variable measured at the ordinal level must also be mutually exclusive and exhaustive.

variable’s attributes are exclusive when every case can be classified into only one of the categories.

Grouped Age: 1.0-20, 2. 21-30, 3. 31-40 etc.

Page 38: Chapter 4 Chapter 4 Conceptualization and Measurement.

Ordinal Level of Measurement

The first of the three quantitative levels is the ordinal level of measurement.

At this level, the numbers assigned to cases specify only the order of the cases, permitting “greater than” and “less than” distinctions. i.e. Heat= 1. low, 2. medium, 3. high

Or, if the response choices to a question range from “very wrong” to “not wrong at all”, for example. (Some researchers use this type of ordinal level data as Interval, i.e. Likert Scales)

Page 39: Chapter 4 Chapter 4 Conceptualization and Measurement.

Ordinal Level of Measurement, cont.

Of course, an ordinal rating scheme assumes that respondents have similar interpretations of the terms used to designate the ordered responses.

This is not always the case, because rankings tend to reflect the range of alternatives with which respondents are familiar. Ugly, Ordinary, Pretty (inbalance in the terms used)

Page 40: Chapter 4 Chapter 4 Conceptualization and Measurement.

Interval Level of Measurement

The numbers indicating the values of a variable at the interval level of measurement represent fixed measurement units but have no absolute, or fixed, zero point.

The numbers can therefore be added and subtracted, but ratios are not meaningful.

Again, the values must be mutually exclusive and exhaustive.

This level of measurement is represented by the difference between two Fahrenheit temperatures. We assume equal intervals between attributes.

Page 41: Chapter 4 Chapter 4 Conceptualization and Measurement.

60 degrees may be 30 degrees hotter than 30 degrees, but 60 degrees cannot be said to be twice as hot as 30 degrees because there is no absolute zero to set up a ratio relationship

Page 42: Chapter 4 Chapter 4 Conceptualization and Measurement.

There are very few true interval-level measures in social science.

Interval assumes equal distance between attributes

Can’t say between low, medium, high there is equal distance but some use Likert construction as such

Page 43: Chapter 4 Chapter 4 Conceptualization and Measurement.

Likert Scale 1. Very satisfied 2. Satisfied 3.

dissatisfied 4. very dissatisfied Notice there is no middle ground i.e. 1. Very satisfied 2. Satisfied

3.satisfied/dissatisfied 4. dissatisfied 5. Very dissatisfied

Page 44: Chapter 4 Chapter 4 Conceptualization and Measurement.

Ratio Level of Measurement

The numbers indicating the values of a variable at the ratio level of measurement represent fixed measuring units and an absolute zero point (zero means absolutely no amount of whatever the variable indicates). i.e. Age is continuous (numbers indicating values are points on a continuum

For most statistical analyses in social science research, the interval and ratio levels of measurement can be treated as equivalent.

On a ratio scale, 10 is 2 points higher than 8 and is also 2 times greater than 5—the numbers can be compared in a ratio.

Page 45: Chapter 4 Chapter 4 Conceptualization and Measurement.

Ratio Level of Measurement, cont.

Of course, the numbers also are mutually exclusive and exhaustive, so that every case can be assigned one and only one value.

Age is ratio:Ages range in order from 0 to 100 Test scores range in order from 0 to 100 Also people’s ages can be represented by

values ranging from 0 years (or some fraction of a year) to 120 or more. A person who is 30 years old is 15 years older than someone who is 15 years old (30 - 15 = 15) and is twice as old as that person (30/15 = 2).

Page 46: Chapter 4 Chapter 4 Conceptualization and Measurement.

Comparison of Levels of Measurement Researchers choose levels of measurement in

the process of operationalizing variables; the level of measurement is not inherent in the variable itself. Many variables can be measured at different levels, with different procedures.

It is usually is a good idea to try to measure variables at the highest level of measurement possible. The more information available, the more ways we have to compare cases.

Page 47: Chapter 4 Chapter 4 Conceptualization and Measurement.

Measurement Validity

Measurement validity refers to the extent to which measures indicate what they are intended to measure.

When a measure “misses the mark”—when it is not valid—our measurement procedure has been affected by measurement error.

Measurement error can arise in three general ways.

Page 48: Chapter 4 Chapter 4 Conceptualization and Measurement.

Measurement Validity, cont.

1. Idiosyncratic individual errors are errors that affect a relatively small number of individuals in unique ways that are unlikely to be repeated in just the same way.

Example: Individuals make idiosyncratic errors when they don’t understand a question, when some unique feelings are triggered by the wording of question, or when they are feeling out of sorts due to some recent events.

Page 49: Chapter 4 Chapter 4 Conceptualization and Measurement.

Measurement Validity, cont.

2. Generic individual errors occur when the responses of groups of individuals are affected by factors that are not “what the instrument is intended to measure.”

For example, individuals who like to please others by giving socially desirable responses may have a tendency to say that they “agree” with statements, simply because they try to avoid saying they “disagree” with anyone.

Page 50: Chapter 4 Chapter 4 Conceptualization and Measurement.

Measurement Validity, cont.

3. Method factors can also create errors in the responses of most or all respondents. Questions that are unclear may be misinterpreted by most respondents, while unbalanced response choices may lead most respondents to give positive rather than negative responses.

For example, if respondents are asked a question with unbalanced response choices, they are more likely to respond that something is wrong than if they are asked the question with the balanced response choices.

Page 51: Chapter 4 Chapter 4 Conceptualization and Measurement.

Reliability

Reliability means that a measurement procedure yields consistent scores when the phenomenon being measured is not changing (or that the measured scores change in direct correspondence to actual changes in the phenomenon).

If a measure is reliable, it is affected less by random error, or chance variation, than if it is unreliable.

Reliability is a prerequisite for measurement validity: We cannot really measure a phenomenon if the measure we are using gives inconsistent results.

Page 52: Chapter 4 Chapter 4 Conceptualization and Measurement.

Ways to Improve Reliability and Validity

Whatever the concept measured or the validation method used, no measure is without some error, nor can we expect it to be valid for all times and places.

Remember that a reliable measure is not necessarily a valid measure.

Unfortunately, many measures are judged to be worthwhile on the basis of only a reliability test.

Page 53: Chapter 4 Chapter 4 Conceptualization and Measurement.

Conclusions, cont.

Statistical tests can help to determine whether a given measure is valid after data have been collected, but if it appears after the fact that a measure is invalid, little can be done to correct the situation, so do pretests.

If you cannot tell how key concepts were operationalized when you read a research report, don’t trust the findings.

And if a researcher does not indicate the results of tests used to establish the reliability and validity of key measures, remain skeptical.