Exploring Data

Exploring Data

Individuals & Variables• Individuals are the objects described by a

set of data. Individuals may be people animals, or things

• A variable is any characteristic of an individual. A variable can take different values for different individuals.

• Does the distribution have one or more peaks (modes) or is it unimodal?

• Is the distribution approximately symmetric or is it skewed in one direction? Is it skewed to the right (right tail longer) or left?

• Shape: The distribution is roughly symmetric with a single peak in the center.

• Center: You can see from the histogram that the midpoint is not far from 110. The actual data shows that the midpoint is 114.

• Spread: The spread is from 80 to about 150. There are no outliers or other strong deviations from the symmetric, unimodal pattern.

Example DescriptionExample Description

Calculator ExampleCalculator Example

Text

(To save data for later use on home screen type L1 -> Prez)

President Age President Age President Age

Washington 57 Lincoln 52 Hoover 54

J Adams 61 A Johnson 56 F D Roosevelt 51

Jefferson 57 Grant 46 Truman 60

Madison 57 Hayes 54 Eisenhower 61

Monroe 58 Garfield 49 Kennedy 43

JQ Adams 57 Arthur 51 LB Johnson 55

Jackson 61 Cleveland 55 Nixon 56

Van Buren 54 B Harrison 55 Ford 61

WH Harrison 68 Cleveland 55 Carter 52

Tyler 51 McKinley 54 Reagan 69

Polk 49 T Roosevelt 52 GHW Bush 64

Taylor 64 Taft 51 Clinton 46

Fillmore 50 Wilson 56 G W Bush 54

Pierce 48 Harding 55

Buchanan 65 Coolidge 51

Calc continuedCalc continued

• Frequency shortcut: If you have a dataset comprised of 75 3’s and 35 4’s for example, you can enter the values in list 1 and the frequencies in list 2 then pull 1 variable stats:

• Stats-edit- L1: 3, 4 L1: 75, 35 stat-calc-1var stats L1,L2 enter

Relative frequency/Cumulativ

e Frequency

Relative frequency/Cumulativ

e Frequency

• A histogram does a good job of displaying the distribution of values of a quantitative variable, but tells us little about the relative standing of an individual observation.

• So, we construct an ogive (“Oh-Jive”) aka a relative cumulative frequency graph.

Step 1- Construct table

Step 1- Construct table

• Decide on intervals and make a frequency table with 4 columns: Freq, Relative frequency, cumulative frequency, and rel. cum. Freq.

• To get the values in the rel. freq. column, divide the count in each class interval by the total number of observations. Multiply by 100 to convert to %.

• In Cum freq column, add the counts that fall in or below the current class interval

• for rel. cum. freq. column, divide the entries in the cum freq column by total number of individuals.

President Age President Age President AgeWashington 57 Lincoln 52 Hoover 54J Adams 61 A Johnson 56 F D Roosevelt 51Jefferson 57 Grant 46 Truman 60Madison 57 Hayes 54 Eisenhower 61Monroe 58 Garfield 49 Kennedy 43JQ Adams 57 Arthur 51 LB Johnson 55Jackson 61 Cleveland 55 Nixon 56Van Buren 54 B Harrison 55 Ford 61WH Harrison 68 Cleveland 55 Carter 52Tyler 51 McKinley 54 Reagan 69Polk 49 T Roosevelt 52 GHW Bush 64Taylor 64 Taft 51 Clinton 46Fillmore 50 Wilson 56 G W Bush 54Pierce 48 Harding 55Buchanan 65 Coolidge 51

Class Frequency Relative Frequency Cumulative frequency

Relative cumulative frequency

40-44 2 2/43=0.047 2 2/43=0.04745-49 6 6/43=0.140 8 8/83=0.18650-54 13 13/43=0.302 21 21/43=0.48855-59 12 12/43=0.279 33 33/43=0.76760-64 7 7/43=0.163 40 40/43=0.93065-69 3 3/43=0.07 43 43/43=1.000TTotal 43

Step 2 & 3Step 2 & 3• Label and scale your axes and title your graph. Vertical axis always Relative Cum. Freq. Scale the horizontal axis according to your choice of class intervals and the vertical axis from 0% to 100%.

• Plot a point corresponding to the rel. Cum. freq. in each class interval at the LEFT ENDPOINT of the NEXT class interval. (example, the 40 to 44 interval, plot a point at a height of 4.7% above the age value of 45.

• Begin with 0% you should end with 100%. Connect dots

To Locate an individual within distribution: What about Clinton? He was 46. To find his relative standing, draw a vertical line up from his age (46) on the horizontal axis until it meets the ogive. Then draw a horizontal line from this point of intersection to the vertical axis. Based on our graph his age places him at the 10% mark which tells us that about 10% of all US presidents were the same age as or younger than Bill Clinton when they were inaugurated.

To locate a value corresponding to a percentile, do the opposite. Ex: 50th percentile, 55 years old.

• Whenever data are collected over time, plot observations in time order. Displays of distributions such as stemplots and histograms which ignore time order can be misleading when there is systematic change over time.

Shows change in gas price over time. Shows TRENDS

Exploring DataExploring Data1.2 Describing Distributions with Numbers

YMS3e

AP Stats at CSHNYCMs. Namad

1.2 Describing Distributions with NumbersYMS3e

AP Stats at CSHNYCMs. Namad

Sample DataSample DataConsider the following test scores for a small class:75 76 82 93 45 68 74 82 91 98

Plot the data and describe the SOCS:

What number best describes the “center”?What number best describes the “spread’?

scores40 50 60 70 80 90 100

Collection 1 Dot Plot

scores40 50 60 70 80 90 100

Collection 1 Dot Plot Shape?Outliers?Center?Spread?

Measures of CenterMeasures of CenterNumerical descriptions of distributions begin with a measure of its “center”.

If you could summarize the data with one number, what would it be?

x x1 x2 ... xn

n

x xi

n

x Mean: The “average” value of a dataset.

Median: Q2 or M The “middle” value of a dataset.

Arrange observations in order min to max

Locate the middle observation, average if needed.

Mean vs. MedianMean vs. MedianThe mean and the median are the most common measures of center.

If a distribution is perfectly symmetric, the mean and the median are the same.

The mean is not resistant to outliers.

You must decide which number is the most appropriate description of the center...

Measures of SpreadMeasures of SpreadVariability is the key to Statistics. Without variability, there would be no need for the subject.

When describing data, never rely on center alone.

Measures of Spread:

Range - {rarely used...why?}

Quartiles - InterQuartile Range {IQR=Q3-Q1}

Variance and Standard Deviation {var and sx}

Like Measures of Center, you must choose the most appropriate measure of spread.

QuartilesQuartilesQuartiles Q1 and Q3 represent the 25th and 75th percentiles.

To find them, order data from min to max.Determine the median - average if necessary.The first quartile is the middle of the ‘bottom half’.The third quartile is the middle of the ‘top half’.

19 22 23 23 23 26 26 27 28 29 30 31 32

45 68 74 75 76 82 82 91 93 98

med Q3=29.5Q1=2

3

med=79Q1 Q3

5-Number Summary, Boxplots

5-Number Summary, Boxplots

The 5 Number Summary provides a reasonably complete description of the center and spread of distribution

We can visualize the 5 Number Summary with a boxplot.

MIN Q1 MED Q3 MAX

min=45 Q1=74 med=79 Q3=91 max=98

45 50 55 60 65 70 75 80 85 90 95 100

Quiz ScoresOutlier?

Determining OutliersDetermining Outliers

InterQuartile Range “IQR”: Distance between Q1 and Q3. Resistant measure of spread...only measures middle 50% of data.

IQR = Q3 - Q1 {width of the “box” in a boxplot}

1.5 IQR Rule: If an observation falls more than 1.5 IQRs above Q3 or below Q1, it is an outlier.

“1.5 • IQR Rule”“1.5 • IQR Rule”

Why 1.5? According to John Tukey, 1 IQR seemed like too little and 2 IQRs seemed like too much...

1.5 • IQR Rule1.5 • IQR Rule

To determine outliers:

Find 5 Number Summary

Determine IQR

Multiply 1.5xIQR

Set up “fences” Q1-(1.5IQR) and Q3+(1.5IQR)

Observations “outside” the fences are outliers.

Outlier ExampleOutlier Example

0 10 20 30 40 50 60 70 80 90 100Spending ($)

IQR=45.72-19.06IQR=26.66IQR=45.72-19.06IQR=26.66 1.5IQR=1.5(26.66)

1.5IQR=39.991.5IQR=1.5(26.66)1.5IQR=39.99

All data on p. 48.

outliers}

fence: 45.72+39.99= 85.71

fence: 19.06-39.99= -20.93{

Standard DeviationStandard DeviationAnother common measure of spread is the Standard Deviation: a measure of the “average” deviation of all observations from the mean.

To calculate Standard Deviation:Calculate the mean.Determine each observation’s deviation (x - xbar).“Average” the squared-deviations by dividing the total squared deviation by (n-1).This quantity is the Variance.Square root the result to determine the Standard Deviation.

Standard DeviationStandard DeviationVariance:

Standard Deviation:

Example 1.16 (p.85): Metabolic Rates

var (x1 x )2 (x2 x )2 ... (xn x )2

n 1

sx (xi x )2n 1

1792 1666 1362 1614 1460 1867 1439

Standard DeviationStandard Deviation1792 1666 1362 1614 1460 1867 1439

x (x - x) (x - x)2

1792 192 36864

1666 66 4356

1362 -238 56644

1614 14 196

1460 -140 19600

1867 267 71289

1439 -161 25921

Totals: 0 214870

Metabolic Rates: mean=1600

Total Squared

Deviation214870

Variance

var=214870/6

var=35811.66

Standard Deviation

s=√35811.66

s=189.24 calWhat does this

value, s, mean?

Linear TransformationsLinear TransformationsVariables can be measured in different units (feet vs meters, pounds vs kilograms, etc)

When converting units, the measures of center and spread will change.

Linear Transformations (xnew=a+bx) do not change the shape of a distribution.

Multiplying each observation by b multiplies both the measure of center and spread by b.

Adding a to each observation adds a to the measure of center, but does not affect spread.

Data Analysis ToolboxData Analysis ToolboxTo answer a statistical question of interest:Data: Organize and Examine

Who are the individuals described?What are the variables?Why were the data gathered?When,Where,How,By Whom were data gathered?

Graph: Construct an appropriate graphical display

Describe SOCS

Numerical Summary: Calculate appropriate center and spread (mean and s or 5 number summary)

Interpretation: Answer question in context!

Chapter 1 SummaryChapter 1 SummaryData Analysis is the art of describing data in context using graphs and numerical summaries. The purpose is to describe the most important features of a dataset.