Chapter 5 Producing Data. 5.1 Objectives Define population and sample Explain how sampling differs from a census Explain what is meant by a voluntary.

Chapter 5

Producing Data

5.1 Objectives

Define population and sample Explain how sampling differs from a census Explain what is meant by a voluntary response sample Give an example of a voluntary response sample Explain what is meant by convenience sampling Define what it means for a sampling method to be biased Define, carefully, a simple random sample List the four steps involved in choosing a SRS Explain what is meant by systematic random sampling Use a table of random digits to select a simple random sample

5.1 Objectives

Define a probability sample Given a population, determine the strata of interest, and select a

stratified random sample Define a cluster sample Define undercoverage and nonresponse as sources of bias in

sample surveys Give an example of response bias in a survey question Write a survey question in which the wording of the question is

likely to influence the response Identify the major advantage of large random samples

Observational Study versus Experiment Observational Study

Observe individuals and measure variables of interest but do not attempt to influence the response.

Experiment Deliberately impose some treatment on

individuals in order to observe their responses.

A response variable measures an outcome of a study. An explanatory variable helps explain or influences changes in a

response variable.

Population versus Sample

Population The entire group of individuals that we want

information about. Sample

A part of the population that we actually examine in order to gather information

Census…Sampling

Census This attempts to contact every individual in the

entire population Sampling

This involves studying part in order to gain information about the whole

Why sample?

What are some reasons why we use samples instead of a census?

Example

Page 333 #5.1 Students as customers. A committee relations in a college town plans to

survey local businesses about the importance of students as customers.From telephone book listings, the committee chooses 150 businesses at random. Of these, 73 return the questionnaire mailed by the committee.

What is the population for this sample survey? What is the sample? What is the proportion of people that did not

respond?

Types of Samples

Voluntary Response Sample Consists of people who choose themselves by

responding to a general appeal. Biased because people with strong opinions

(especially negative opinions) are most likely to respond.

Types of Samples

Convenience Sampling Choosing individuals that are easiest to reach.

Bad Sampling Techniques

Bias

A sampling method is biased if it systematically favors certain outcomes.

A statistic is said to be unbiased if the mean of the sampling distribution equals the mean of the population.

Example

Page 334 #5.8: Explain it to the congresswoman You are on the staff of a member of Congress who is

considering a bill that would provide government-sponsored insurance for nursing home care.

You report that 1128 letters have been received on the issue, of which 871 oppose the legislation. “I’m surprised that most of my constituents oppose the bill. I thought that it would be quite popular,” says the congresswoman.

Are you convinced that a majority of the voters oppose the bill?

How would you explain the statistical issue to the congresswoman?

Sampling Methods

Simple Random Sample (SRS) A SRS of size n consists of n individuals from the

population chosen in such a way that every set of size n has an equally likely chance to be the sample actually selected. Think of putting everyone’s name in a hat and drawing.

Random Digits A table of random digits is a long string of the

digits 0 – 9 with these two properties Each entry in the table is equally likely to be any of the

10 digits The entries are independent of each other

Using the table to select a SRS Step 1: Label

Assign a numerical label to every individual in the population

Step 2: Table Use table B to select numbers at random

Step 3: Stopping Rule Indicate when you should stop sampling

Step 4: Identify Sample Use the labels to identify the subjects selected in

the sample

Example

Page 341 #5.10 Spring Break Destinations. A campus newspaper plans a major article on

spring break destinations. The authors intend to call a few randomly chosen resorts at each destination to ask about their attitudes toward groups of students as guests.

The table contains the resorts listed in one city. The first step is to label this population as shown.

Enter Table B at line 131, and choose three resorts.

Example continued

01 Aloha Kai 08 Captiva 15 Palm Tree 22 Sea Shell

02 Anchor Down 09 Casa del Mar

16 Radisson 23 Silver Beach

03 Banana Bay 10 Coconuts 17 Ramada 24 Sunset Beach

04 Banyan Tree 11 Diplomat 18 Sandpiper 25 Tradewinds

05 Beach Castle 12 Holiday Inn 19 Sea Castle 26 Tropical Breeze

06 Best Western 13 Lime Tree 20 Sea Club 27 Tropical Shores

07 Cabana 14 Outrigger 21 Sea Grape 28 Veranda

Gettysburg Address

Sampling Methods

Probability Sample Sample chosen by chance. We must know what samples are possible and

what chance, or probability, each possible sample has.

Sampling Methods

Stratified Random Sample First divide the population into groups of

individuals (called strata). Individuals in each strata are similar in some way that is

important to the response Then choose a separate SRS in each stratum and

combine these SRSs to form the full sample.

The River Problem

Sampling Methods

Cluster Sampling Divide the population into groups, or clusters. Some of these clusters are randomly selected. All individuals in the chosen clusters are selected

to be in the sample.

Cautions about Sample Surveys Undercoverage Bias

Occurs when some groups in the population are left out of the process of choosing the sample.

Cautions about Sample Surveys Nonresponse Bias

Occurs when an individual chosen for the sample can’t be contacted or does not cooperate.

Cautions about Sample Surveys Response Bias

Influences caused by the behavior of the respondent or of the interviewer

GIGO

Garbage In Garbage Out Need to learn about producing data that can be

used for making valid statements. Need to recognize techniques that produce

biased samples.

Question Wording

The wording of the question is the most important influence on the answers given in a sample survey. Confusing questions Leading Questions Choice of words

Ask

Insist on knowing the following before believing a survey. The exact question asked The rate or nonresponse The date of the survey The method of the survey

Samples Vary, Parameters are Fixed

5.2

Designing Experiments

5.2 Objectives

Define experimental units, subjects and treatments. Define factor and level. Given the number of factors and the number of levels for each

factor, determine the number of treatments. Explain the major advantage of an experiment over an

observational study. Give an example of the placebo effect. Explain the purpose of a control group. Explain the difference between control and a control group. Discuss the purpose of replication, and give an example of

replication in the design of an experiment. Discuss the purpose of randomization in the design of an

experiment. Given a list of subjects, use a table of random numbers to assign

individuals to treatment and control groups.

5.2 Objectives

List the three main principles of experimental design. Explain what is means to say that an observed effect is

statistically significant. Define a completely randomized design. For an experiment, generate an outline of a completely

randomized design. Define a block. Give an example of a block design in an experiment. Explain how a block design may be better than a completely

randomized design. Give an example of a matched pairs design, and explain why

matched pairs are an example of block designs. Give an example in which lack of realism negatively affects our

ability to generalize the results of a study.

Vocabulary

Experimental Units The individuals on which the experiment is done.

Subjects When the units are humans they are called subjects.

Treatment A specific experimental condition applied to the units is

called a treatment.

Response variables measures an outcome of a study These are also called dependent variables.

Explanatory variables help explain or influences changes in a response variable. These are also called factors, and independent

variables.

Principles of Experimental Design Control

The purpose of control is to try to eliminate the confounding effects of lurking variables.

Replication

Randomization

Control

Placebo Effect

Control Group

Example

Page 357 # 5.35: Improving Response Rate How can we reduce the rate of refusals in telephone surveys?

Most people who answer at all listen to the interviewer’s introductory remarks and then decide whether to continue.

One study made telephone calls to randomly selected households to ask opinions about the next election. In some calls the interviewer gave her name, in others she identified the university she was representing, and in still others she identified both herself and the university.

For each type of call the interviewer either did or did not offer to send a copy of the final survey results to the person interviewed. Do these differences in the introduction affect whether the interview is completed?

Identify the experimental units or subjects, the factors, the treatments and the response variables.

Example Page 357 # 5.37: Sham Operation

In the mid 1900s, a common treatment for angina (a disease marked by brief attacks of chest pain caused by insufficient oxygen to the heart) was called internal mammary ligation. In this procedure doctors made small incisions in the chest and tied knots in two arteries to try and increase blood flow to the heart. It was a popular procedure-90% of patients reported that it helped reduce pain. In 1960, Seattle cardiologist Dr. Leon Cobb carried out an experiment where he compared ligation with a procedure in which he made incisions but did not tie off the arteries. This sham operation proved just as successful, and the ligation procedure was abandoned as a treatment for angina. What is the response variable in Dr. Cobb’s experiment? Dr. Cobb showed that the sham operation was just as successful as

ligation. What term do we use to describe the phenomenon that many subjects report good results from a pretend treatment?

The ligation procedure is an example of the lack of an important property of a well designed experiment. What is that property?

Principles of Experimental Design Control

Replicate Purpose is not to eliminate chance variation but to

reduce its role and increase sensitivity of the experiment to differences between treatments.

Randomize

Principles of Experimental Design Control

Replicate

Randomize Use chance to assign experimental units to treatments in

an attempt to balance groups according to different variables

Statistically Significant An observed effect so large that it would rarely

occur by chance is called statistically significant.

Different Experiments

Comparative Experiments:

1) Treatment → Observation

2) Observation → Treatment → Observation

Completely Randomized Experiments:

Randomized Experiments

Group 1 Treatment 1 Measure

Random Compare

Allocation Results

Group 2 Treatment 2 Measure

Example

Page 364 # 5.39: Treating Prostrate Disease A large study used records from Canada’s

national health care system to compare effectiveness of two ways to treat prostrate disease. The two treatments are traditional surgery and a new method that does not require surgery. The records described many patients whose doctor had chosen each method. The study showed that the patients treated by the new method were significantly more likely to die within 8 years.

Page 364 # 5.39: Treating Prostrate Disease Continued Further study of the data showed that this conclusion was

wrong. The extra deaths among patients who got the new method could be explained by lurking variables. What lurking variables might be confounded with a doctor’s choice of surgical or nonsurgical treatment?

You have 300 prostate patients who are willing to serve as subjects in an experiment to compare the two methods. Use a diagram to outline the design of a randomized comparative experiment. (when using the diagram to outline the design of an experiment, be sure to indicate the size of the treatment groups and the response variable.)

Example

Page 364 # 5.40: Headache Relief Doctor’s identify “chronic tension-type headaches” as

headaches that occur almost daily for at least 6 months. Can antidepressant medications or stress management training reduce the number and severity of these headaches? Are both together more effective than either alone?

Investigators compared four treatments: antidepressant alone, placebo alone, antidepressant plus stress management, and placebo plus stress management.

Outline the design of the experiment. The headache sufferers named in the following table have agreed to participate in the study.

Use table B line 130 to randomly assign the subjects to the treatments.

Acosta Farouk Liang Solomon

Asihiro Fleming Maldondado Trujillo

Bennett George Marsden Tullock

Bijalis Han Montoya Valasco

Chen Howard O’Brian Vaughn

Clemente Hruska Ogle Wei

Duncan Imrani Padilla Wilder

Durr James Plochman Willis

Edwards Kaplan Rosen Zhang

Calculator Practice

Selecting random samples by calculator

1. STAT/4:clrList/2nd/L1/ENTER.This clears any data in list L1.

2. MATH/PRB/5:randInt(a, b, c)→L1This will choose c numbers between a and b.

They will be stored in list L1.

Calculator Practice

To “seed” the calculator so that all of us get the same random numbers

Calculator Practice

Repeat #5.40 using the graphing calculator to randomly assign the subjects to the treatments.

Remember…

Questions to ask before you believe a poll. Who carried out the survey? What was the population? How was the sample selected? How large was the sample? What was the response rate? How were the subjects contacted? When was the survey conducted? What were the exact questions asked?

Why use experiments?

The goal of much research is to establish a causal link between a particular treatment and a response.

Experiments are much better at this than are observational studies.

In observational studies you cannot guarantee that you have controlled for the effects of lurking variables, but you can in a well-designed experiment.

What is control?

Control is the overall effort to minimize variability in the way the experimental units are obtained and treated. The simplest form of control is comparing several

treatments in the same environment.

A control group is the group that gets a sugar pill, no treatment, a fake surgery…

Why replicate?

The purpose of replication is not to eliminate chance variation (an impossible task) but to reduce its role and increase the sensitivity of the experiment to differences between treatments. Use enough subjects to reduce chance variation.

Randomization

The rule used to assign experimental units to treatments.

Some believe this is the most important principle of experimental design…since it allows us to assert that the treatment groups are essentially similar. That is, that there is no systematic difference between them before treatments are administered.

Some attempt to balance groups according to different variables. The problem is that some variables are difficult to measure, others are hidden or not thought about/known until after the experiment. Lurking variables are everywhere

Completely Randomized Design All experimental units have been randomly

assigned to treatment groups.

There are ways that we can improve upon studies with blocking and matching. “Improvement” refers to the ability to have a greater degree of control over lurking variables.

Blindness

Blinding The subject does not know which treatment they

are receiving

Double-blinding Neither the subject nor the those who measure

the response variable know which treatment a subject received This is a way to control the placebo effect. Avoids unconscious bias

Blocking

A block is a group of experimental units or subjects that are known to be similar in some way that is thought to systematically affect the response to the treatments.

Block Design…The random assignment of units is carried out separately within each block.

Blocking

You block to control for the variables you know about that might influence the response; you randomize to control for the variables you do not know about.

Matched Pairs Designs

A type of block design in which just two treatments are compared.

Subjects are matched in pairs such that variables are as similar as possible.

In some matched pair experiments subjects receive both treatments and the subject serves as his/her own control.

Example

Page 375 #5.54: Fabric Science A maker of a fabric for clothing is setting up a new line to

“finish” the raw fabric. The line will use either metal rollers or natural-bristle rollers to raise the surface of the fabric; a dyeing cycle time of either 30 or 40 minutes; and a temperature of either 150° or 175° Celsius. An experiment will compare all combinations of these choices. Three specimens of fabric will be subjected to each treatment and scored for quality. What are the factors and the treatments? How many

individuals (fabric specimens) does the experiment require? Outline a completely randomized design for this experiment.

(you need not actually do the randomization.)

Example

Page 376 #5.58: Will taking antioxidants help prevent colon cancer? People who eat a lot of fruits and vegetables have lower rates of

colon cancer than those who eat little of these foods. Fruits and vegetables are rich in “antioxidants” such as vitamins A, C and E. Will taking antioxidants help prevent colon cancer?

A clinical trial studied 864 people who were at risk of colon cancer. The subjects were divided into four groups: daily beta-carotene (which is made into vitamin A in the body), daily vitamins C and E, all three vitamins every day, and daily placebo. After four years, the researchers were surprised to find no significant difference in colon cancer among the groups. What are the explanatory and response variables in this experiment? Outline the design of the experiment.

Example

Page 376 #5.58: Will taking antioxidants help prevent colon cancer? continued

Assign labels to the 864 subjects and use Table B, starting at line 118 to assign the first 5 subjects for the beta-carotene group.

The study was double-blind. What does this mean? What does “no-significant difference” mean in describing the

outcome of the study? Suggest some lurking variables that could explain why people

who eat lots of fruits and vegetables have lower rates of colon cancer. The experiment suggests that these variables, rather than the antioxidants, may be responsible for the observed benefits of fruits and vegetables.

Example

Page 377 #5.60: Acculturation rating There are several psychological tests that measure the

extent to which Mexican Americans are oriented towards Mexican/Spanish or Anglo/English culture. Two such tests are the Bicultural Inventory (BI) and the Acculturation Rating Scale for Mexican Americans (ARSMA). To study the correlation between the scores on these two tests, researchers will give both tests to a group of 22 Mexican Americans. Briefly describe a matched pairs design for this study. In

particular, how will you use randomization in your design? You have an alphabetized list of the subjects (numbered 1 to

22). Carry out the randomization required by your design and report your results.

What to REMEMBER?

Randy … “Randomize” & Blake … “Block” (if needed) Can’t… “Control” Reply … “Replicate”

and...

Double Blind is BEST!