Transcript
PSYCHOLOGY 3800 LAB 002
• two-way ANOVA feedback link
• chi-square (!2) test: overview and applications
• example analysis in SPSS- omnibus test
• post hoc options in a chi-square analysis
• this week’s assignment
Feedback on the assignment can be found on the lab blog in the form of a list of commonly made errors:
http://uwo3800g.tumblr.com/post/79396735608/assignment-5-commonly-made-errors
• in this unit, we are no longer obtaining scores from participants, calculating means based on the scores, and comparing those means using statistical tests
" instead, we are working with frequency data and nominal variables
frequency data - the number of times that some event occurs - the number of objects or individuals who fit a give criterion or category - represented as a count or tally
nominal variables - classify data into categories - values in each category represent tallies rather than scores
(one variable; not carried out in your assignment)
Application #1: Goodness of Fit
• testing whether observed frequencies are the same as theoretical (expected) frequencies for a single nominal variable
Example: A fair coin should come up heads half the time during a coin toss
" toss a coin repeatedly and record the number of heads " compare the observed number of heads to the expected number of heads to determine whether these values differ significantly " for this test, we want a small chi-square value (and a non-significant result), telling us that our observed values are not differing significantly from what is expected theoretically
…and don’t forget: correlation (or relation) ! causation
(two variables)
Application #2: Test of Association
• testing whether two nominal variables are related
• this test is similar to a correlation, only we cannot determine the degree of the relation (only whether or not there is some association)
(two variables)
Application #2: Test of Association
Example: Is pet ownership related to an individual’s experience with depression?
Procedure: • obtain sample of individuals
• ask them to indicate their pet status: o currently own a pet o previously owned a pet o never owned a pet
• ask them to indicate their experience with depression o have episodes of major depression o have episodes of mild depression o have no experience with depression
• record number of individuals who fit in each category
Hypotheses:
HO: there is no significant relation between pet ownership and experience with depression (variables are independent)
HA: there is a significant relation between pet ownership and experience with depression (variables are dependent)
(two variables)
Application #2: Test of Association
Design:
3 (depression) x 3 (pet ownership) chi-square test for association
Depression Pet ownership Current Previous Never
Severe 20 5 16 Mild 11 14 34 None 4 5 15
Example Data: Observed Frequency of Pet Ownership and Experience with Depression
35 24 65
41
59
24
Total participants: 124 " 35 + 24 + 65 = 124 or " 41 + 59 + 24 = 124
(two variables)
Application #2: Test of Association
Example Data: Calculating Expected Frequency of Pet Ownership and Experience with Depression
(two variables)
Application #2: Test of Association
Depression Pet ownership Current Previous Never
Severe
Mild
None !
(35)(41)124
=11.57
!
(35)(59)124
=16.65
!
(35)(24)124
= 6.77!
(24)(41)124
= 7.94
!
(24)(59)124
=11.42
!
(24)(24)124
= 4.64!
(65)(41)124
= 21.49
!
(65)(59)124
= 30.93
!
(65)(24)124
=12.58
35 24 65
41
59
24
!
E =column sum " row sum
total
(two variables)
Application #2: Test of Association
Depression Pet ownership
Current Previous Never
Severe 20 5 16
Mild 11 14 34
None 4 5 15
Depression Pet ownership
Current Previous Never
Severe 11.57 7.94 21.49
Mild 16.65 11.42 30.93
None 6.77 4.64 12.58
The chi-square test compares the observed values to the expected values:
observed
expected
The chi-square test compares the observed values to the expected values:
• the manner in which expected values are derived suggests consistent effect across all cells (think main effect… no relation between variables can be concluded)
• if test is significant: some cells’ observed values differs from expected values, no longer suggesting a consistent effect (think interaction)
• significant effect points to significant relation between variables
• omnibus test cannot give us more details than this… must do post hoc procedures
(two variables)
Application #2: Test of Association
• independent replication o each individual contributes to count in one cell only o allows us (and SPSS) to derive an accurate total value to calculate the expected frequencies
• normal sampling distribution of (E-O) o distribution not tested directly o assumed to hold true if many low expected values are not calculated for the data o can assess low expected count via Cochran’s rule
• cannot conduct chi-square test reliably if: • expected value < 1 for 1 or more cells
• expected < 5 for 20% or more cells
• check footnote of “Chi-Square Tests” table in output
• if either of the above options is true: a) collapse cells (in collapsing, will have to adjust p-value)
or b) get more subjects (or, for your assignment: increase the counts
within your cells)
pet ownership
1 = current 2 = previous 3 = never
depression status
1 = severe 2 = mild 3 = none
there are 20 individuals who currently own a pet (Pet: 1) and
have severe depression (Depression: 1)
there are 34 individuals who never owned a pet (Pet: 3) and have mild
depression (Depression: 2)
To make sure that SPSS reads the structure of your data correctly, do the following:
Data ! Weight Cases…
Analyze " Descriptive Statistics " Crosstabs
specify that you would like Chi-square output
Statistics Menu
Request: -observed counts -expected counts -unstandardized residuals
Cells Menu
Click “OK” to run analysis (output will open up in separate output window)
Observed Values
• these are the values you typed into SPSS (or I did, in this case)
Expected Values
• these are the values that SPSS calculates based on the row and column totals, and the total sample size
Residual Values
• these values represent the difference between the observed and expected values in each cell
Example: 20 – 11.6 = 8.4
"2(4) = 13.062, p < .05
• no issues with expected count identified (need at least 20% of cells to have expected frequency less than 5 to violate Cochran’s rule)
" a significant relation exists between pet ownership and depression " post hoc analyses needed to dissect effect
Significance of Relation Between Variables
Option #1: Extract Cells • extract cells from full data table that have large residuals • large residuals suggest some interaction • choosing to ignore levels that do not suggest an association
Overall goal: create a 2 x 2 table that is more easily interpretable (i.e., compare two levels of one variable to 2 levels of the other)
Option #2: Collapse Cells • collapse cells that exhibit similar types of effects • will allow for a clearer, more magnified understanding of effects
! re-run chi-square test on reduced data
• the highlighted 4 cells have the largest residuals • can extract these 4 cells to create a 2 x 2 table
Data " Select Cases…
select “If condition is satisfied” and click on “If…”
in this window, we identify which cells we wish to extract for additional analysis
((Depression=1) & (Pet=1)) | ((Depression=1) & (Pet=3)) | ((Depression=2) & (Pet=1)) | ((Depression=2) & (Pet=3))
Note: clicking “OK” will bring up your output window,
but this is just to show you the syntax that you ran… there are no new results to
examine just yet.
Inspect your data. Make sure that only the cells you want active are marked with a “1” under the “filter_$” column (and are not crossed out)
We have selected: current pet owners (1) with severe depression (1) current pet owners (1) with mild depression (2) non-pet owners (3) with severe depression (1) non-pet owners (3) with mild depression (2)
Analyze " Descriptive Statistics " Crosstabs
Re-run the chi-square test " SPSS will have remembered your previous selections so simply click “OK” in the menu that appears
this is your extracted data (it will help you with
your conclusion)
this is your "2 test (it will help you to
determine the significance
of your effect)
• when you start collapsing/extracting cells, you are (in essence) making it easier for yourself to obtain significant results
• need to adjust our p-values to be more conservative (more strict) to ensure that our results are sound
• to ensure accurate results: use the Bonferroni correction to adjust p-values when data is extracted, and a conservative p-value when the data is collapsed or collapsed/extracted (combination of both)
Using a Bonferroni Correction: Steps
(1) calculate the number of possible 2 x 2 comparisons (k) that could be made using the following formula:
(2) divide .05 (the largest acceptable p-value) by k
(3) compare the significance value of your "2 test (performed on your new table) to your Bonferroni adjusted p-value
" if significance value in table is smaller than the adjusted p-value, conclude that test was significant at p < .05
!
k =r!
2!(r " 2)!#
c!2!(c " 2)!
r = number of rows in original table c = number of columns in original table
used when analyzing purely extracted cells
Using a Bonferroni Correction: Example
(1) number of possible 2 x 2 comparisons (k):
(2) divide .05 by k:
(3) D.R. reject HO (and conclude significant effect) if p < .006
!
k =3!
2!(3 " 2)!#
3!2!(3 " 2)!
!
=(3)(2)(1)(2)(1)(1!)
"(3)(2)(1)(2)(1)(1!)
!
=6
(2)(1)"
6(2)(1)
!
k =62"62
!
= 3 " 3
!
= 9
!
.059
= .006
p-value is indeed < .006
!2(1) = 8.194, p < .05
• no issues with expected count identified (Cochran’s rule not violated)
" conclusion
• for individuals who currently own pets: greater frequency of severe depression reported than mild depression
• effect is reversed for people who have never owned pets: greater frequency of mild depression rather than severe depression
• cause/effect cannot be concluded so consider possible explanations
• the highlighted cells for mild depression and no depression show pattern in same direction (increasing frequency across pet ownership categories) • can be collapse and compared against severe depression
• the highlighted cells for previous pet owners and non-pet owners show pattern in same direction • can be collapse and compared against current pet owners
• create two new variables representing the collapsed categories: " dep_severe_other " pet_current_other
• specify 0 decimal places for both
• specify both as a nominal measure
• coding those with severe depression as 1 • coding those with mild or no depression as 2
• coding current pet owners as 1 • coding previous and non-pet owners as 2
Data " Select Cases…
select “All cases” to turn off previous filters
Analyze " Descriptive Statistics " Crosstabs
Re-run the chi-square test with the new re-coded variables
this is your collapsed data (it will help you with your
conclusion)
this is your "2 test (it will help you to
determine the significance
of your effect)
(1) compare the significance value of your "2 test (performed on your new table) to a conservative p-value
" ideal choice: p < .001
Pick a Conservative p-Value: Steps
used when analyzing purely collapsed cells or a combination of collapsed and
extracted cells
(2) if significance value in table is smaller than the adjusted p-value, conclude that test was significant at conservative value
p-value is indeed < .001
!2(1) = 12.774, p < .001
• no issues with expected count identified (Cochran’s rule not violated)
" conclusion
• for individuals with severe depression: relatively similar frequencies in terms of current ownership vs. no current ownership
• for individuals with limited-to-no experience with depression: substantially fewer current pet owners versus no current pet ownership
• cause/effect cannot be concluded so consider possible explanations
• APA style results section
• 2 pages maximum (not including additional pages for tables)
• no data file available " you will be making up your own data o significant obtained chi-square (interaction) o Cochran’s rule is not violated o data makes sense (theoretically, in real world) o total of at least 600 participants (can be more)
• please include all output and hand calculations
Introduction • description of study • description of variables • design used in analysis • hypotheses (Ho, HA)
Overall Chi-Square Test • assessment of assumptions • obtained statistic and significance • conclusion
Post Hoc Test • description of approach and rationale • assessment of assumptions • obtained statistic and significance (with adjusted p-values as needed) • conclusion
Tables • two tables: original data (main analysis), post hoc data • APA formatting for both tables
Final conclusion • discussion of all findings in applied, non-statistical terms
top related