研究设计之 测量 School of Public Administration & Policy Dr. Kaifeng Yang.

研究设计之测量

School of Public Administration & PolicyDr. Kaifeng Yang

IssuesIssues

概念操作化– 概念化– 指标化

测量的层次复合指数测量测量的好坏：效度与信度提高效度与信度的途径

概念操作化概念操作化沟通绩效 ?

目标清晰度 ?

例子例子 沟通绩效 ?

目标清晰度 ?

从名义从名义 // 概念定义到操作定义概念定义到操作定义

Nominal (Conceptual) Definition

A definition that is simply assigned to a term without any claim that the definition represents a “real” entity. It is A) defined by others; b) what it is vs. what it is not; c) shared; d) realistic and sensible.

Operational (Working) Definition

Specifies precisely how a concept will be measured

菜谱

概念化概念化

The process … we specify what we mean when we use particular terms. It involves describing the essential features of the terms. Sometimes, it involves indicators we’ll be using to measure our concept and the different aspects of the concept (dimensions).

Indicator(指标 ): a sign of the presence or absence of the

concept Dimension (纬度 /变量 ): a specifiable aspect of a

concept

爱？

指标化指标化 : Political Alienation: Political AlienationConceptualization

Defining the concept

Nominal Definition(Variables)

Operational Definition

(Indicators)

Instrumentation

Political distrust, powerlessness, and meaninglessness

Political Distrust: the general feeling that government isno longer of the people, by the people, and for the people

Indicators are responses to the following questions:1. Do you think that quite a few of the people running the government are a little crooked, not very many are, or do you think that hardly any are crook at all?2. How much of the time do you think you can trust

the government to do what is right?3. Would you say that the government is pretty much

run by a few big interests looking out for themselvesor that it is run for the benefit of all people?

General Social Survey, 2000, National Opinion Research

Theory

abstract

DataConcrete

指标化：指标化： Other ExamplesOther Examples Literacy:

– The completion of six years of formal education– Those nations in which at least 50 percent of the population has

had six years of formal education, as indicated in a publication of the United Nations

Democracy:– A system of government in which public officials are selected in

competitive elections– Those countries in which the second-place finishers in elections

for the chief executive office has received at least 25 percent of the vote at least once in the past eight years

To operationalize your concepts is arbitrary, but you have to make some justification if not based on common literature or policy.

指标化：例子指标化：例子

Social Status

Education

Income

Occupation

Numbers of school years completed

Cross family income last year

Blue collar, white collar, or professional

PoliticalParticipation

Voting

Campaign contribution

Attend political rally

“Did you vote in the last…”

“Did you contribute to …”

“Have you attended a …”

Concept Variable Operational Definition

In the sample article? Communication PerformanceCommunication Performance

Conceptualization

Nominal Definition(Variables)

Operational Definition

(Indicators)

Instrumentation

External Communication Performance

p.49

Reverse: Reverse: 从问题从问题 // 数据到概念？数据到概念？

“How many times a week do you drink an alcoholic beverage?”1. None 2. 1~2 3. 3~5 4. 5~7 5. 7~10 6. >10

Percentage of a nation’s population that is literate “I often feel lonely”

1. agree 2. disagree

“number of hours of sleep each night” “number of trips taken abroad each year by presidents”

指标与概念的不完全契合指标与概念的不完全契合

概念 指标 2

指标 3

指标 5指标 4

指标 1

Level of MeasurementLevel of Measurement

1. “How many times a week do you drink an alcoholic beverage?”

1. 1. None 2. 1~2 3. 3~5 4. 5~7 5. 7~10 6. >10

2. “How many times a week do you drink an alcoholic beverage?” _____

3. “Do you think you are addicted to alcohol? 1. Yes2. No

4. “I am alcoholic.” 1. Strongly agree 2. Agree 3. Neutral 4. Disagree 5. Strongly Disagree

Precision/Level of MeasurementPrecision/Level of Measurement

Nominal Measures（定类）exhaustive and mutual exclusive categories; none is more or less than another

Ordinal Measures（定序）can logically rank-order: with more or less of particular attribute

Interval Measures（定距）How much smaller or larger; the distance between each unit on the scale must be the same;

Ratios Measures（定比）Interval measures; the zero point has a true meaning (absence of the characteristic being measured)

Level of MeasurementLevel of Measurement

Level of MeasurementLevel of Measurement

选择测量层次选择测量层次

如果可能，选择定比 /定距受其他因素影响

– 变量本身的特点– 敏感性（如收入）– 信度与效度

PracticesPractices

You have data on the annual salary of 200 employees of an organization. How the data can be presented as ratio, interval, ordinal, and nominal data?

复合指数测量：什么是？复合指数测量：什么是？

复合指数测量：为什么？复合指数测量：为什么？

概念 指标 2

指标 3

指标 5指标 4

指标 1

例子：怎么测量国家的“自由”程度？例子：怎么测量国家的“自由”程度？

(Country) 1 2 3 4 5

Privately owned newspapers 1 0 0 0 1

Legal right to form political parties 1 1 0 0 0

Contested elections for public offices

1 1 0 0 0

Voting rights for most of the adults 1 1 0 1 0

Limitations on government’s ability to incarcerate citizens

1 0 0 0 1

Index score 5 3 0 1 2

1= Yes; 0=No.

李克特量表李克特量表 Likert Format (for other formats, refer to the textbook)

SD

1

D

2

U

3

A

4

SA

5

Privately owned newspapers

Legal right to form political parties

Contested elections for public offices

Index score for Observation x 15 or 15/3=5

测量的好坏？测量的好坏？How do you measure individuals’ alcohol addiction?

“How many times a week do you drink an alcoholic beverage?”

1. 1. None 2. 1~2 3. 3~5 4. 5~7 5. 7~10 6. >10

“How many times a week do you drink an alcoholic beverage?” _____

“Do you think you are addicted to alcohol? 1. Yes2. No

“I am alcoholic.” 1. Strongly agree 2. Agree 3. Neutral 4. Disagree 5. Strongly Disagree

How many bottles of alcohol do you normally drink for a week? ____

How many bottles of alcohol do you normally drink for a day?

测量的准确度测量的准确度 To what extent are you measuring what you intended

to measure? Ideal: Measurement error is 0

Measurement = Intended characteristic + Systematic error + Random error

Systematic error is durable Random error is temporary

Pre-test = Math Ability + ____+_____

准确度准确度 : : 信度信度

Reliability: produce same results over repeated applications. Stability and Consistency

– Test-Retest method 再测信度* Two times * maturation * sensitivity

– Alternative format复本信度* two times *maturation

– Split-Halves method (multiple measures, one concept)折半信度– Sub-sample method 亚样本信度

Split-HalvesSplit-Halves

Measuring Performance measurement (PM) effectiveness

1. Our PM information can be trusted2. Our PM helps managerial decisions3. Our PM improves communication between city hall and us4. Our PM help elected officials formulate budget5. Our PM indictors reflect our management effectiveness6. Our PM indictors are reliable7. Our investment in PM is worthwhile 8. Our PM improves productivity9. Our PM motivates employees10. Our PM helps adjusting strategic planning

准确度：效度准确度：效度 Validity: the extent to which an empirical measure

adequately reflects the real meaning of the concept under consideration.– Face validity表面效度 : Does it appear…? Judgment!– Content validity内容效度 : Got everything in?– Consensual validity共识效度 : Everybody agrees!– Construct validity—Criterion-related validity

* Predictive validity * Concurrent validity– Interitem association– Discriminant validity

测量误差与信度、效度测量误差与信度、效度Measurement = Intended characteristic + Systematic error + Random error

• Systematic error is durable, affecting validity

• Random error is temporary, affecting more on reliability

Which of the following questions concerns reliability?

1. What does the Scholastic Aptitude Test (SAT) really measure?

2. Do attitude scales really tap personal feelings or just what people think investigators want to hear?

3. Is poverty best represented by measures of low absolute income (a minimum subsistence) or low relative income (e.g., 50% of national median income)?

4. How closely does self-reported church attendance correspond to actual church attendance?

5. Are scores on the even-numbered and odd-numbered items on this test highly correlated?

Research indicates that in recent years respondents’ answers to survey questions measuring racial prejudice underestimate actual racial prejudice. This underestimate is an example of __________ measurement error, which affects measurement __________.

random; reliability random; reliability and validity systematic; validity systematic; reliability and validity

Systematic error would be least likely to occur with which of the following measures?

1. a miscalibrated yardstick that is an inch short2. an IQ test developed for English-speaking U.S.

citizens that is administered to French Canadians3. a question measuring child abuse that asks, “how

often do you hit your child when she doesn’t obey you?”

4. scores on the multiple-choice portion of an exam 5. reports of gambling winnings on IRS tax return

Validity and ReliabilityValidity and Reliability• Reliability is a necessary, but insufficient, condition, for validity

• validity is more important, but reliability is more easily tested

• Think about possible errors

Low V

Low R

Low V

High R

Low V

Low R

High V

High R

提高效度与信度：减少误差提高效度与信度：减少误差

Participant (object who)Situational factor (where, how, when)Measurer (subject who)Instrument (what)

Facial attractiveness?

提高效度与信度：减少误差提高效度与信度：减少误差

Participant (object who)客体 /对象Situational factor (where, how, when)情境与程序

Measurer (subject who)主体Instrument (what)工具

Facial attractiveness?

提高信度的途径提高信度的途径

访谈、试测增加同类问题 /指标去掉“区分性”较低的指标

提高效度的途径提高效度的途径访谈、试测去掉“区分性”较低的指标多种测量方法

SPSSSPSS

学生设计的例子学生设计的例子