Top Banner
DRAWING INFERENCES FROM GROUP DATA: Construc)ng 2 nd level models, working with ROIs, and avoiding sta)s)cal circularity
64

DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Mar 11, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

DRAWINGINFERENCESFROMGROUPDATA:Construc)ng2ndlevelmodels,workingwithROIs,

andavoidingsta)s)calcircularity

Page 2: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Soyouhavegroupdata…Whatnow?

¨  Whole-brainvoxelwiseanalysis¤ ConstrucJngappropriate2ndlevelmodels

n Characterizingwithin-subject(repeatedmeasures)effects

n Characterizingbetween-subject(group)effects

n Usingcovariatesatthe2ndlevel

¨  Region-of-interest(ROI)analysis¤ ChoosingyourROIs¤ Avoiding“voodoo”

n a.k.a.inferenJalcircularity;non-independenceerrors

Page 3: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  YouhavedatafromagroupofsubjectsandyouwanttolookforacJvitychangesacrosscondiJons

¨  Thesimplestapproachistocreatecontrastimagesforeachsubjectandthenconductaone-samplet-testacrosssubjects

¨  ContrastimagesaresimplyweightedlinearcombinaJonsofcondiJon-specificbetaimages

¤  e.g.,CondA-CondB:[1-1]

¤  ReferredtoasCOPEimagesinFSLand.conimagesinSPM

Page 4: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  ContrastswithmorethantwocondiJons:

¤  CondAandCondBvs.CondC:[11-2]

n NOTE:IfCondA&Bhavevastlydifferent#’softrials,itmaybebecertorunanewGLMthatcombinesthesecondiJons,oralternaJvelyyoucouldweighteachcondiJonbythe#oftrials

¤  (CondA–CondB)vs.(CondC–CondD):[1-1-11]

n  TheinteracJonterm!

¨  Nullhypothesis:

¤  acrosssubjectmeanofcontrastvariable=0

Page 5: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  Probingforparametriceffectswithlinearcontrastcoefficients

¤  3condiJons:[-101]

¤  4condiJons:[-3-113]

¤  5condiJons:[-2-1012]

¤  6condiJons:[-5-3-1135]

¨  Canalsoprobefor2ndorder(quadraJc)and3rdorder(cubic)effects

¨  Anysetofcontrastcoefficientscanwork,solongastheysumtozero

¤  WriteyourcontrastcoefficientsasposiJvenumbersandthensubtractthemeanfromeachnumber

¤  Butnotethatcustomcontrastsmightnotbeorthogonaltoeachother

Page 6: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Example:MappinglinearandquadraJccontrasts

Page 7: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Example:Mappingcustomcontrasts

Todd&Marois(2004)Nature

Page 8: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AlternaJveapproach:Separatepairwisetests

Chenetal.(2013)AJNRAmJNeuroradiol

Page 9: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AlternaJveapproach:Separatepairwisetests

Chenetal.(2013)AJNRAmJNeuroradiol

Page 10: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Workingwithrepeatedmeasuresdata

¨  Probingforparametriceffectswithlinearcontrastcoefficients

¤  3condiJons:[-101]

¤  4condiJons:[-3-113]

¤  5condiJons:[-2-1012]

¤  6condiJons:[-5-3-1135]

¨  Canalsoprobefor2ndorder(quadraJc)and3rdorder(cubic)effects

¨  Anysetofcontrastcoefficientscanwork,solongastheysumtozero

¤  WriteyourcontrastcoefficientsasposiJvenumbersandthensubtractthemeanfromeachnumber

¤  Butnotethatcustomcontrastsmightnotbeorthogonaltoeachother

Page 11: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Theone-samplet-test(inFSL)

TheimagesinpucedintoEV1canbe:Ø  thebetaesJmatesfromasinglecondiJonØ  thedifferencebetweentwocondiJonsØ  anylinearcontrastbetweencondiJons

Take-homepoint:Aone-samplet-testcanaccomplishalot!

Page 12: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AnalternaJve(slightlymoreinvolved)waytotestforadifferencebetween2condiJons

¨  FSLrepresentaJonof2ndlevelmodelforpairedt-test

8subjects2condiJons(A&B)

Page 13: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Whythetwoapproachesareequivalent

PLoSBiology(2015)

Page 14: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

WhatifonesubjectismissingdatafromagivencondiJon?

¨  RelaJvelycommonsituaJontobefacedwith¤ E.g.,AnalysisofCorrectvs.Incorrecttrials

n  subjectmayhavenoincorrecttrials

¨  PotenJalsoluJons:¤ Analysiscanberunomipngthatsubject’sdata

¤ Or,analysiscanberunasanunpairedt-test(essenJallyconsideringthetwocondiJonsastwoseparategroupsofsubjects)n MoreconservaJve,butmaysomeJmesbeuseful,especiallyifmulJplesubjectsaremissingdata

Page 15: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Between-subjectmodels

¨  Wehavetwogroupsofsubjects(e.g.,9paJentsand7controls)withpotenJallydifferentcross-subjectvariance.¤ Two-SampleT-Test(equalvariancenotassumed)

¨  SpecifytwogroupmembershipssothatFSL’sFEATesJmateseachgroup'scross-subjectvarianceseparately.¤ SPM’sdefaultisalsotoassumeunequalvariance

¨  WewanttotestwhetherornotacJvitybetweenthetwogroupsisequivalentforagivencondiJon

Page 16: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Two-sampleunpairedt-test

Page 17: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Howabouttwogroupswithtwocondi)ons?

¨  E.g.,Comparingthemagnitudeofpre-vs.post-trainingacJvaJonchangesinpaJentsandcontrols¤ Within-subjectfactor:pre-vs.post-training¤  Across-subjectfactor:paJentsvs.controls

¨  Oneeasy-to-implementapproach:¤  FirstcomputeA-Bdifferenceimageforeachsubject¤  Thenperformsimpletwo-sample(unpaired)t-testtofindgroupdifferencesn  i.e.,groupxcondiJoninteracJon

¨  ThisbasicapproachcangeneralizetoanylinearcombinaAonofrepeatedmeasuresfactors

Page 18: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Howabouttwogroupswithtwocondi)ons?

¨  AnalternaJveapproach(slightlymoreinvolved):

Page 19: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

SpecificaJonofmorecomplexgroup-levelmodelscangettrickyinFSL

¨  SeeFSLFEATuserguideforexamplesofmulJ-factorANOVAmodels

¨  hCp://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

Page 20: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Introducingcovariatesintoyourmodels

¨  Singlegroupwithcovariate¤ Youhaveasinglegroupofsubjectsandyoualsohavemeasuredage.YouwouldliketoseeifthereisanageeffectonbrainacJvity.

n Whatwouldthemodellooklike?

n Whatcontrastswouldyouspecifyfortheageeffect?

Page 21: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Use age Use demeaned age

•  Both models will give exactly the same result for C2, but C1 will be different.

Singlegroupwithcovariate

SlidefromJeaneCeMumford

Page 22: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Simulateddata

Takehome:MeancenteringisonlynecessaryifyouwantyourPEofcolumnof1stobetheoverallmean

SlidefromJeaneCeMumford

Page 23: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

TwogroupswithaconJnuouscovariate

¨  Wehavetwogroupsandaconfoundingcovariate(e.g.,depressionscore).

¨  OurprimaryinterestisinthedifferenceofmeanbrainacJvaJonbetweenthetwogroups.

¨  Wewanttomakesurethisdifferencewasn’tduetobetweengroupdifferencesindepression.

¨  Whatshouldthemodellooklike?

SlidefromJeaneCeMumford

Page 24: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

¨  Okaytodemeantheconfoundingmeasureacrossallsubjects

¤  Butonlynecessaryifyouintendtolookateachgroupvs.baseline(e.g.,[100])

¨  ButdoNOTdemeantheconfoundingmeasurewithingroup

¤  Thiscouldremoveanyconfoundingeffectthemeasuremighthave

TwogroupswithaconJnuouscovariate

SlidefromJeaneCeMumford

Page 25: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Whyyoushouldn’tdemeanwithingroup

¨  Whatifthisiswhatyourdatalooklike?¤ ApparentdifferenceinmeansisclearlyduetorangeofXsampled,notthegroupmembership

SlidefromJeaneCeMumfordFormorediscussionofissuessurroundingmean-centering:hCp://mumford.fmripower.org/mean_centering/

Page 26: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

FormoreinformaJononmean-centering

hCp://mumfordbrainstats.tumblr.com

Page 27: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

TesJngtheinteracJon

¨  WhatifyouwanttotestwhethertherelaJonshipbetweenbrainacJvityanddepressionscorediffersbetweenyourgroups?¤ mean-centeringisnotnecessaryhere

010-1SlidefromJeaneCeMumford

Page 28: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 29: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 30: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 31: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 32: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 33: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

2ndlevelanalysesinSPM

Page 34: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

AlternaJvestovoxelwiseanalysis

¨  ConvenJonalfMRIstaJsJcscomputeonestaJsJcalcomparisonpervoxel.¤  Advantage:candiscovereffectsanywhereinbrain.¤  Disadvantage:lowstaJsJcalpowerduetomulJplecomparisons.

¨  SmallVolumeCorrecAon:OnlyruntestsonasmallproporJonofvoxels(byreducingthesearchspace,allowsformorelenientvoxel-levelstats)

¨  Region-of-interest:PooldataacrossaregionforsinglestaJsJcaltest.

SVC ROI SPM

Example: how many comparisons on this slice?

• Voxelwise: 1600

• SVC: 57

• ROI: 1

SlidefromChrisRorden

Page 35: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

WhyuseROIs?

¨  ConvenientwaytoalleviatethemulJplecomparisonsproblemsthatariseinwhole-brainanalyses¤  ShouldsAlladjuststatsfor#ofROIstested!

¨  Allowsformorehypothesis-drivenanalyses¤  ExploringtheenJredatasetcanbeunwieldyandsomeJmesleadstounfocusedandhighly-speculaJvepapers

¤  ROIresultsareeasiertopresentanddiscuss

¤ HemodynamicJmecourseplotscanbeinformaJve

¤  ROIsdon’trequiresubjectstoacJvatetheexactsamevoxeln  ButanROI-onlyanalysisisvulnerabletoTypeIIerrors!

Page 36: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Atlas-based(e.g.,AALatlas)

Page 37: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n AutomatedsegmentaJon-based(e.g.,Freesurfer)

Page 38: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n AutomatedsegmentaJon-based(e.g.,Freesurfer)

Page 39: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Hand-tracedusinganatomicallandmarks

Page 40: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Coordinate-based(e.g.,basedonapreviousstudy)

Page 41: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Coordinate-based(e.g.,basedonapreviousstudy)

Page 42: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Coordinate-based(e.g.,basedonameta-analysis)

Page 43: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

!

Page 44: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

Poweretal.(2013)CurrOpinNeurobio

Page 45: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

Gordonetal.(2014)CerebralCortex

Page 46: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ Anatomically-defined

n Network-based

Gordonetal.(2014)CerebralCortex

Page 47: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Region-of-interestanalysis

¨  ChoosingtherightROI(s)¤ FuncAonally-defined

n BasedonaparJcularacJvaJoneffectinyourdatan ROI-definingcontrastneedstobeorthogonaltostaJsJcaltestsconductedonextracteddata

n  Independent“localizer”taskscanbeusefulhere

n AVOIDDOUBLE-DIPPINGATALLCOSTS!

Page 48: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

Ø  IdenAfyaclusterthatshowsabovesignificantacJvityforbothCatsandDogs(viaconjunc)onanalysis)

Ø  TestwhetherthisregionshowssignificantacJvityforDogs>Snakes

Ø  TestwhetherthisregionshowssignificantacJvityforHorses>Snakes

Ø  IdenAfyaclusterthatshowsCats>Dogseffect

Ø  TestwhetherthisregionshowssignificantacJvityforCats>Snakes

Ø  TestwhetherthisregionshowssignificantacJvityforSnakes>Dogs

Ø  TestwhetherthisregionshowssignificantacJvityforDogs>Snakes

Page 49: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

Ø  IdenAfyaclusterthatshowssignificanteffectofsJmulustypeinone-wayANOVA(F-test)thatincludesalltaskcondiJons(Cats,Dogs,Snakes,Horses)

Ø  TestwhetherthisregionshowssignificantacJvityforCats>Dogs

Ø  TestwhetherthisregionshowssignificantacJvityfor(Cats&Dogs&Horses)>Snakes

Page 50: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

StrongMemory

ModerateMemory

WeakMemory

***

StrongMemory

ModerateMemory

WeakMemory

*

*

IdenJfyROIsbasedon2contrasts:StrongMemory>WeakMemoryStrongMemory>ModerateMemory

Okaytolookatplotstomakesurethatnothingweirdisgoingon,buteffectsizeswillbeinflatedandsignificancebracketsarehighlymisleading

Page 51: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Isthisokay?

StrongMemory

ModerateMemory

WeakMemory

***

StrongMemory

ModerateMemory

WeakMemory

*

*

IdenJfyROIsbasedonanatomicalboundaries

NotappropriatetodrawanyconclusionsaboutthesetworegionsshowingdissociableprofilesofacJvityunlessyoudirectlytestforaregionxcondiJoninteracJon

Page 52: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

The“implied”interacJon

Nieuwenhuisetal.(2011)NatureNeuroscience

Page 53: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

The“implied”interacJon

Nieuwenhuisetal.(2011)NatureNeuroscience

Page 54: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

The“implied”interacJon

Nieuwenhuisetal.(2011)NatureNeuroscience

¨  “…theerrorofcomparingsignificancelevelsisespeciallycommonintheneuroimagingliterature,inwhichresultsaretypicallypresentedincolor-codedstaJsJcalmapsindicaJngthesignificancelevelofaparJcularcontrastforeach(visible)voxel.

¨  Avisualcomparisonbetweenmapsfortwogroupsmighttempttheresearchertostate,forexample,that“thehippocampuswassignificantlyacJvatedinyoungeradults,butnotinolderadults”.

¨  However,theimpliedclaimisthatthehippocampusisacJvatedmorestronglyinyoungeradultsthaninolderadults,andsuchaclaimrequiresadirectstaJsJcalcomparisonoftheeffects.

¨  Similarly,claimsaboutdifferencesinacJvaJonacrossbrainregionsmustbesupportedbyasignificantinteracJonbetweenbrainregionandthefactorunderlyingthecontrastofinterest.”

Page 55: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

ROIdefiniJonisaffectedbynoise

SlidefromNikoKriegeskorte

true region

overfitted ROI

RO

I-ave

rage

ac

tivat

ion

overestimated effect

independent ROI

Page 56: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

Baker, Hutchison, & Kanwisher

(2007)

High selectivity from pure

noise. SlidefromEdVul

ROIdefiniJonisaffectedbynoise

Page 57: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

EdVul

Page 58: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

Page 59: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodooPuzzlinglyhighcorrelaJons

Page 60: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

r = 0.82

Neuralcorrelatesofhumanvirtuejudgment

r = -0.83

Page 61: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

Thisfigureistherealdata

Page 62: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

VoodoocorrelaJons

“Tosumup,then,weareledtoconcludethatadisturbinglylarge,andquite

prominent,segmentoffMRIresearchonemoJon,personality,andsocialcogniJon

isusingseriouslydefecJveresearchmethodsandproducingaprofusionofnumbersthatshouldnotbebelieved.Althoughwehavefocusedhereon

studiesrelaJngtoemoJon,personality,andsocialcogniJon,wesuspectthatthequesJonableanalysismethodsdiscussedherearealsowidespreadinotherfields

thatusefMRItostudyindividualdifferences,suchascogniJve

neuroscience,clinicalneuroscience,andneurogeneJcs.”

Vuletal.(2009)

Page 63: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

ToavoidselecJonbias,wecan...

...performanonselecAveanalysisOR...makesurethatselecJonandresultsstaJsJcsareindependentunderthenullhypothesis,becausetheyareeither:¨  inherentlyindependent¨  orcomputedonindependentdata

¤ e.g.,separatestudy;funcAonallocalizer;cross-validaAon

e.g.independentcontrasts

e.g.whole-brainmapping(noROIanalysis)

SlidefromNikoKriegeskorte

Page 64: DRAWING INFERENCES FROM GROUP DATA...Working with repeated measures data ¨ You have data from a group of subjects and you want to look for acJvity changes across condiJons ¨ The

FormoreinformaJon…

¨  PuzzlinglyHighCorrelaAonsinfMRIStudiesofEmoAon,Personality,andSocialCogniAon.Vul,E.,HarrisC.,Winkielman,P.,&Pashler,H.(2009)PerspecJvesonPsychologicalScience,4,274-290.[FormerlyJtled:VoodooCorrelaJonsinSocialNeuroscience].

¨  Circularanalysisinsystemsneuroscience–thedangersofdoubledipping.KriegeskorteN,SimmonsWK,BellgowanPSF,BakerCI.(2009)NatureNeuroscience12(5):535-40.

¨  Everythingyouneverwantedtoknowaboutcircularanalysis,butwereafraidtoask.KriegeskorteN,LindquistMA,NicholsTE,PoldrackRA,VulE.(2010)JCerebBloodFlowMetab.30(9):1551-7.

¨  Voodooandcircularityerrors.Vul,E.andPashler,H.(2012).NeuroImage,62,945-948.


Related Documents
Who Are The “Expressive Attentives?” Identifying Social ......poll data with respect to this group, whom we label “expressive attentives,” in order to draw inferences about

Who Are The “Expressive Attentives?” Identifying Social....

Category: Documents
4th IMPRS Astronomy Summer School Drawing Astrophysical Inferences from Data Sets

4th IMPRS Astronomy Summer School Drawing Astrophysical...

Category: Documents
Rational Inferences and Bayesian Inferences

Rational Inferences and Bayesian Inferences

Category: Documents
Chapter 12 Inferences on Categorical Data and Qualitative Data

Chapter 12 Inferences on Categorical Data and Qualitative...

Category: Documents
Drawing Statistical Inferences from Historical Census Data - ipums usa

Drawing Statistical Inferences from Historical Census Data -...

Category: Documents
Variance Estimation: Drawing Statistical Inferences from IPUMS-International Census Data

Variance Estimation: Drawing Statistical Inferences from...

Category: Documents
1 Single Sample Inferences Greg C Elvers. 2 Single Samples A single sample implies that you have collected data from one group of people or objects You.

1 Single Sample Inferences Greg C Elvers. 2 Single Samples A...

Category: Documents
Ir inferences

Ir inferences

Category: Documents
Making INFERENCES

Making INFERENCES

Category: Documents
Inferences from sample data Confidence Intervals Hypothesis Testing Regression Model.

Inferences from sample data Confidence Intervals Hypothesis....

Category: Documents
African Lions: Modeling Population Growth Date: · PDF fileWhat inferences can we make about ... • Apply models to real data sets • Make inferences about new ... Lion population

African Lions: Modeling Population Growth Date: · PDF...

Category: Documents
CCGPS ADVANCED ALGEBRA UNIT 1- Inferences & Conclusions from Data.

CCGPS ADVANCED ALGEBRA UNIT 1- Inferences & Conclusions from...

Category: Documents