Advanced Statistics

AdvancedStatistics

Page1

AdvancedStatisticsPaoloColettiA.Y.2010/11Free UniversityofBolzanoBozen

TableofContents1. Statisticalinference................................................................................................................21.1 Populationandsampling..............................................................................................................................................22. Dataorganization...................................................................................................................42.1 Variablesmeasure.......................................................................................................................................................42.2 SPSS..............................................................................................................................................................................42.3 Datadescription...........................................................................................................................................................53. Statisticaltests........................................................................................................................73.1 Example........................................................................................................................................................................73.2 Nullandalternativehypothesis..................................................................................................................................113.3 TypeIandtypeIIerror...............................................................................................................................................113.4 Significance.................................................................................................................................................................123.5 Acceptandreject........................................................................................................................................................123.6 Tailsandcriticalregions.............................................................................................................................................133.7 Parametricandnonparametrictest..........................................................................................................................153.8 Prerequisites...............................................................................................................................................................154. Tests.....................................................................................................................................164.1 Studentsttestforonevariable.................................................................................................................................164.2 Studentsttestfortwopopulations...........................................................................................................................164.3 Studentsttestforpaireddata..................................................................................................................................184.4 Ftest...........................................................................................................................................................................194.5 Onewayanalysisofvariance(ANOVA)......................................................................................................................204.6 JarqueBeratest..........................................................................................................................................................224.7 KolmogorovSmirnovtest...........................................................................................................................................224.8 Signtest......................................................................................................................................................................234.9 MannWhitney(Wilcoxonranksum)test..................................................................................................................264.10 Wilcoxonsignedranktest..........................................................................................................................................284.11 KruskalWallistest......................................................................................................................................................304.12 Pearsonscorrelationcoefficient................................................................................................................................324.13 Spearman'srankcorrelationcoefficient.....................................................................................................................344.14 Multinomialexperiment.............................................................................................................................................365. Whichtesttouse?................................................................................................................416. Regressionmodel.................................................................................................................436.1 Theleastsquaresapproach........................................................................................................................................436.2 Statisticalinference....................................................................................................................................................466.3 Multivariateandnonlinearregressionmodel...........................................................................................................476.4 Multivariatestatisticalinference................................................................................................................................486.5 Qualitativeindependentvariables.............................................................................................................................496.6 Qualitativedependentvariable..................................................................................................................................506.7 Problemsofregressionmodels..................................................................................................................................51

AdvancedStatistics

Page2

1. StatisticalinferenceStatistic is the scienceofdata. This involves collecting, classifying, summarizing,organizing,

analyzing, and interpreting numerical information. A population is a set of units (usually people,objects, transactions, etc.) thatwe are interested in studying. A sample is a subset of units of apopulation,whose elements are called cases or,when dealingwith people, subjects. A statisticalinference is an estimate, prediction, or some other generalization about a population based oninformationcontainedinasample.

For example,wemay introduce a variable whichmodels the temperature atmidday inJanuary.Clearlythisisarandomvariable,sincethetemperaturefluctuatesrandomlydaybydayand,moreover, temperatures of the future days cannot be even determined now.However, from thisrandom variables we have data, measurements done in the past. In statistics people deal withobservationsor,inotherwords,realizations, , , ..., ofarandomvariable .Thatis,eachof isarandomvariablethathasthesameprobabilitydistributionas itsoriginatingrandomvariable. It characterizes the th performance of the stochastic experiment determined by the randomvariable . Given this information,wewant to characterize the distribution of or some of itscharacteristics, like theexpectedvalue. In thesimplestcases,wecanevenestablishvia theoreticalconsiderationstheshapeofthedistributionandthentrytoestimatefromthedataitsparameters.

In other words, statistical inference concerns the problem of inferring properties of anunknowndistributionfromdatageneratedbythatdistribution.Themostcommontypeofinferenceinvolvesapproximatingtheunknowndistributionbychoosingadistributionfromarestrictedfamilyof distributions.Generally the restricted family of distributions is specified parametrically. For thetemperatureexamplewecanassumethat benormallydistributedwithaknownvariance andanexpectedvaluetobedetermined.Amongallnormaldistributionwiththisvariancewewanttofindtheonewhichisthemostlikelycandidateforhavingproducedthefinitesequence , ,..., oftemperatureobservedinthepastdays.

Making inferenceaboutparametersofadistribution,peopledealwithstatisticorestimates.Any function , , , of the observations is called a statistic. For example, the samplemean is a common statistic, typically used to estimate the expected value. Thesample variance is anotheruseful estimate.Being a functionof randomvariables, a statistic is a random variable itself. Consequently we may, and will, talk about itsdistribution.

1.1 PopulationandsamplingA statistical researchcananalyzedata from theentirepopulationoronlyona sample.The

population isthesetofallobjects forwhichwewantto infer informationorrelations. Inthiscase,dataset iscompleteandstatistical researchsimplydescribes thesituationwithoutgoingon toanyother objective andwithout using any statistical test.When data are instead available only on asample,a subsetof thepopulation, statistical researchanalyseswhether informationand relationsfoundonthesamplecanbeextendedontheentirepopulationfromwhichthesamplecomesfromortheyarevalidonlyforthatparticularsamplechoice.

Therefore,samplechoiceisaveryimportantanddelicateissueinstatisticalresearches.Many

AdvancedStatistics

Page3

statisticalmethods let us extend results (estimates or tests results) found on the sample to thepopulation, provided that sample is a random sample, a sample whose elements are randomlyextracted from the populationwithout any influence from the researcher, from previously takensampleselementsorfromotherfactors.Buildingsuchasample,however, isadifficulttasksinceaperfectly random selection is almost a utopia. For example, any random sampling on peoplewillnecessarilyincludepeoplewhoareunwillingtogiveinformation,whohavedisappeared,andwholie;thesepeoplecannotbeexcludednorreplacedwithothers,becauseotherwisethesamplewouldnotberandomanymore.Apreviouslyrandomsamplewithexcludedelementscanscrewtheestimates:inour example, problematic people are typically old andwith low education, thus unbalancing oursampleinfavorofyoungandeducatedsubjects.

A common strategy tobuilda samplewhichbehaves likea random sample is the stratifiedsampling.Withthismethod,thesample ischosenrespectingtheproportionsofthevariableswhichare believed to be important for the analysis andwhich are believed to be able to influence theanalysis results. For example, ifwe analyze peoplewe should take care to build a samplewhichreflectsthesexproportions,age,educationand incomedistribution,theresidence(towns,suburbs,countryside)proportions,etc. In thisway,thesamplewillreflectexactlythepopulationat least forwhattheconsideredvariablesareconcerned.Wheneverapersonisnotavailableforanswering,wereplacehimwith another onewith the same variables values.Obviously these variablesmust bechosenwithcareandwithalookatpreviousstudiesonthesametopic,balancingtheirnumbersincetoo fewvariableswillcreateabadlystratifiedsample,whileatoomanywillmakesamplecreationandpeoplessubstitutionverydifficult.

Anotheraspectisthesamplesize.Obviously,thelargerthesamplethebetter.However,thisrelationisnotdirect,i.e.doublingthesamplesizedoesnotyielddoublybetterresults.Therelationinmany statistical tests goes approximately like , which means that we need to quadruple thesamplesizetogetdoublybetterresults.Inanycase,itismuchmoreimportanttohavearandomorwellstratifiedsampleratherthananumeroussample.Qualityismuchbetterthanquantity.

Acommonmistakerelatedtosamplesize issupposing that itshouldbeproportional tothepopulation.This is,at least forall the testanalyzed in thisbook, false: for largepopulations, testsresultsdependonlyontheabsolutesizeandnotontheproportion.Thus,apopulationof1000withasampleof20doesnotyieldbetterresultscomparedtoapopulationof5000withasampleof20.

AdvancedStatistics

Page4

2. Dataorganization2.1 Variablesmeasure

Inastatisticalresearchwefacebasicallythreetypesofvariables: scalevariablesarefullynumericalvariableswithanintrinsicmathematicalmeaning.Forexample,

a temperature or a length are scale variables since they are numeric and anymathematicaloperationonthesevariablesmakessense.Alsoacountisanumericalvariable,eventhoughithasrestrictions(cannotbenegativeandisinteger),becauseitmakessensetoperformmathematicaloperationson it.However,numericalcodessuchasphonenumbersor identificationcodesarenot scale variables even though they seem numeric, since nomathematical operationmakessenseonthemandthenumberisusedonlyasacode;

nominalvariablesrepresentcategoriessuchassex,nationality,degreecourse,plantstype.Thesevariablesdividethepopulationintogroups.Variablessuchasidentificationnumberarenominalsincetheydividethesampleintocategories,eventhougheachcaseisasinglecategory;

ordinalvariablesareamidwaybetweennominalandscalevariables.Theyrepresentcategorieswhichdonothaveamathematicalmeaning(eventhoughmanytimescategoriesareidentifiedbynumbers,suchasinaquestionnairesanswers)butthesecategorieshaveanordinalmeaning,i.e.canbeput inorder.Typicalexamplesarequestionnairesanswerssuchthatverybad,bad,good,verygood,orsometimeissuessuchasfirstyear,secondyear,thirdyear.

Ordinalandnominalvariables,oftenreferredtoascategorical,areusedinSPSSintwoways:asvariablesbythemselves,suchasinmultinomialexperiments(seesection4.14)and,moreoften,asawaytosplitthesampleintogroupstoperformtestsontwoormorepopulations,suchasStudentTtestfortwopopulations(seesection4.2),ANOVA(seesection4.5),MannWhitney(seesection4.9)andKruskalWallis(seesection4.11).2.1.1 Grouping

It isalsoacommonproceduretodegradescalevariablethemtoordinalvariables,arbitrarilyfixingintervalsorbinsandgroupingthecasesintotheirappropriatebin.Forexample,anagevariableexpressedinyearscanbedegradedtoanordinalvariabledividingthesubjectsintoyoung,upto25,adult,from26to50,old,from51to70,veryold,71andover.Thenewvariablesthatweobtainare suited for different statistical testswhich open upmore possibilities. However, any groupingprocedure reduces the information that we have introducing arbitrary decisions in the data andpossiblebiases.Forexample,ifoursamplehasaverylargecountforpeopleofage26,thepreviousarbitrarychoiceof25asalimitforyounggrouphasputmanypeople,whoaremoresimilarto25yearsoldpeopleratherthanto50yearsoldpeople,intotheadultgroup.SPSS:TransformRecodeintoDifferentVariables2.2 SPSS

SPSSmeansStatisticalPackage forSocialSciencesand it isaprogram toorganize statisticaldata and perform statistical research. SPSS organizes data in a sheet calledDataViewwhich is adatabase table,moreor less likeExcels tables.Eachcase isrepresentedbyanhorizontal linesandidentified very often by the first variable which is an ID number. Variables instead use vertical

AdvancedStatistics

Page5

columns.UnlikeExcelandlikedatabasetables,SPSSdatatableisextremelywellstructuredandeachvariablehasalotoffeatures.ThesefeaturesarefoundinVariableViewsheet: Name:feelfreetouseanymeaningfulname,butwithoutspecialcharactersandwithoutspaces.

Whendatahavemanyvariablesitisagoodideatoindicatenamesasv_followedbyanumber(itwillbepossibletoindicateahumanreadablenamelater).

Type:numeric is themost common type. String shouldbeusedonly for completely free text,whilecategoricalvariablesshouldbenumericwithanumbercorrespondingtoeachcategory(itwillbepossible to indicateahuman readablename later);acommonmistake isusingastringvariableforacategoricalvariable,whichhastheimpactthatSPSSwillrefusetoperformcertainoperationswiththatvariable.

Widthanddecimals Label:this isthevariables labelwhichwillappear inchartsandtables insteadofthevariables

name. Values: this feature represents the association between values and categories. It is used for

categoricalvariables,which,assaidbefore,shouldusenumbers foreachcategory. In this fieldvalues labels can be assigned and in charts and tables these labels will appear instead ofnumbers.Obviouslyscalevariablesshouldnotreceivevalueslabels.

Missing:wheneveravariablesvalueisunknownforacertaincaseaspecialnumericcodeshouldbeused,traditionallyanegativenumber(ifthevariablehasonlypositivenumbers)orthelargestpossiblenumbersuchas9999.Ifthisnumberisinsertedhereamongthemissingvalues,SPSSwillsimplyignorethatcasewheneverthatvariableisinvolvedinanyoperation.Itisalsopossible,inData View, to clear the cell completely and SPSS will indicate it with a dot which is asystemmissingnumber(sameeffectasmissingvalue).

Measure:variablesmeasuremustbecarefullyindicated,sinceitwillhaveimplicationsonwhichoperationsmaybedoneonthevariable.

SPSShasfourbasicmenus: Transform: thismenu letsusbuildnewvariablesormodifyexistingones,usuallyworkingona

casebycasebase,thusperformingonlyhorizontaloperations.Veryusefularecommands:o compute,whichbuildanewvariable,typicallyscale,usingmathematicaloperations;o recode,whichbuildanewvariable,typicallycategorical,usingrecoding;

Data: this menu lets you rearrange your data in a more global way. Very useful are thecommands:o split,splitsthefileusinganominalorordinalvariableinsuchawaytobeabletoanalyzeit

automaticallyingroups;o select,letsusfilteroutsometemporarilyundesiredcases;o weight,letsusweightthecasesusingavariablewhenevereachcaserepresentsseveralcases

withthesamedata(allthestatisticswilluseanewsamplesizebasedontheweights); Analyze:thismenuisthecoreofSPSSwithallthestatisticaltestsandmodels; Graphs:thisisthemenutocreatecharts.2.3 Datadescription

SPSSoffersavarietyofnumericalandgraphicaltoolstoquicklydescribedata.Thechoiceofthetooldependsonvariablesmeasure:

AdvancedStatistics

Page6

SPSS:AnalyzeDescriptiveStatisticsFrequencies Frequenciesisindicatedasadescriptionforasinglecategoricalvariable,whileforascalevariablefrequencytablebecomestoo longandfullofsinglecases.However, it isalwaysagood ideatostartanystatisticalresearchwithfrequenciesforeveryvariable,includingscaleones,tospotoutdataentrymistakeswhichareverycommoninstatisticaldata.

SPSS:GraphsChartBuilderPie/Polar Piechartisindicatedasagraphforasinglenominalandordinalvariable.

SPSS:GraphsChartBuilderBar Pie charts are indicated as a graph for a single categorical variable. Using colors andthreedimensionalitytheyworkalsofortwooreventhreenominalandordinalvariables.

SPSS:AnalyzeDescriptiveStatisticsDescriptivesDescriptivestatistics(mean,median,standarddeviation,minimum,maximum,range,skewness,kurtosis)isindicatedasadescriptionforasinglescalevariableandusuallyitdoesnotmakesenseforcategoricalvariables.

SPSS:GraphsChartBuilderHistogram Histogramisindicatedasagraphforasinglescalevariable.Variablevaluesaregroupedintobinsforthevariablerepresentation.Thechoiceofbinninginfluencesthehistogram.

SPSS:GraphsChartBuilderBoxplot Boxplotisindicatedasagraphforasinglescalevariable.Thecentrallinerepresentsthemedianandtheboxrepresentsthecentral50%ofthevariablesdistributiononthesample.Boxplotsmaybeusedalsotocomparethevaluesofascalevariablebygroupsofacategoricalvariable.

SPSS:AnalyzeDescriptiveStatisticsCrosstabsContingencytable(seesection4.14.2)isindicatedasadescriptionfortwocategoricalvariables.

SPSS:AnalyzeCompareMeansMeansMeanscomparisonisawaytocomparethemeansofascalevariableforgroupsofacategoricalvariable,usuallyfollowedbyStudentsTtestorANOVA(seesections4.2and4.5).

SPSS:AnalyzeCorrelateBivariate Bivariatecorrelation(seesections4.12and4.13)isadescriptionforthelinearrelationbetweentwoscalevariables.

SPSS:GraphsChartBuilderScatter/Dot Scatterplotisindicatedasagraphfortwoscalevariables.

AdvancedStatistics

Page7

3. StatisticaltestsStatisticaltestsareinferencetoolswhichareabletotellustheprobabilitywithwhichresults

obtainedonthesamplecanbeextendedtothepopulation.Everystatisticaltesthasthesefeatures:

the null hypothesis H0 and its contradictory hypothesis H1. It is very important that thesehypothesesarebuiltwithoutlookingatthesample;

a sample of observations , , . . . , and a population, to which we want to extendinformationandrelationsfoundonthesample;

prerequisites, special assumptions which are necessary to perform the test. Among theseassumptionsthereisalways,eventhoughwewillnotrepeatiteverytime,thatdatamustcomefromarandomsample;

the statistic , , . . . , , a function calculated on the data,whose value determines theresultofthetest;

astatisticsdistribution fromwhichwecanobtainthetestssignificance.Whenusingstatisticalcomputerprograms,significanceisautomaticallyprovidedbytheprogramnexttothestatisticsvalue;

significance,alsocalledpvalue, fromwhichwecandeductwhetheracceptingor rejectingnullhypothesis.

3.1 ExampleInordertoshowalltheelementsofastatisticaltest,werunthroughaverysimpleexample

andwewill,later,analyzethetheoreticalaspectsofallthetestssteps.WewanttostudytheageofInternetusers.Ageisarandomvariableforwhichwedonothave

any idea of the distribution nor its parameters. However,wemake the hypothesis that age is acontinuousrandomvariablewithanexpectedvalue.Wewanttocheckwhethertheexpectedvalueis35yearsornot.Weformulatethetestshypotheses:

H0: Eage 35 H1: Eage 35

Ofthisrandomvariabletheonlythingweknowaretheobservationsonarandomsampleof100users,whichare:25;26;27;28;29;30;31;30;33;34;35;36;37;38;30;30;41;42;43;44;45;46;47;48;49;50;51;52;20;54;55;56;57;20;20;20;30;31;32;33;34;35;36;37;38;39;40;41;42;43;44;45;46;47;48;49;50;20;21;22;23;24;25;26;27;28;29;30;31;32;33;34;35;36;37;38;39;40;35;36;37;35;36;37;35;36;37;35;36;37;35;36;37;35;36;37;35;36;37;35.

Nowwecalculatetheageaverageonthesample, age 36.2,which isanestimationforthe expected value. We compare this result with the 35 of the H0 hypothesis and we find adifferenceof 1.2.Atthispoint,weaskourselveswhetherthisdifferenceislargeenough,implyingthattheexpectedvalueisnot 35 andthusH0mustberejected,orissmallandcanbecausedbyanunluckychoiceofthesampleandthereforeH0mustbeaccepted.

Thisconclusion inastatisticalresearchcannotbedrawn fromasubjectivedecisionwhetherthedifferenceislargeorsmall.Itistakenusingformalargumentsandthereforewemustrelyonthis

AdvancedStatistics

Page8

statisticfunction:age hypothesizedexpectedvalue

samplevariance

It is noteworthy to look at this statistic numerator.When the averageof age is very close to thehypothesized expected value, the statisticwill be close to 0. On the other hand,when the twoquantitiesareverydifferent,comparedtothesamplestandarddeviation,thestatistic isvery large.Statisticsvalueisalsoinfluencedbythesamplesnumberofelements :thelargeristhesample,thelargerthestatistic.

Summingup,considering that inourcase thesamplestandarddeviation is 8.57,statistic is1.40.Thesituationistherefore

AtthispointweaskourselveswhereisexactlythepointwhichseparatedtheH0truezonefromtheH0 falsezone.To find itout,wecalculatetheprobabilitytoobtainanevenworseresultthantheonewehavegotnow.ThemeaningofworseinthissituationisworseforH0,thereforeanyresultlargerthan 1.40 orsmallerthan 1.40.Weusecentrallimittheoremwhichguaranteesusthat,if is large enough and if the hypothesized expected value is the real expected value of thedistribution(i.e.H0 istrue),ourstatistichasastandardnormaldistribution. Infact,theonlyreasonwhywehavebuiltthisstatisticinsteadofusingdirectlythedifferenceatthenumeratorisbecauseweknowthestatisticsdistribution.Thereforeweknowthattheprobabilityofgettingavaluelargerthan1.40 orsmallerthan 1.40 is1 16%.Thisvalueiscalledsignificanceorpvalue.

If significance is large itmeans that, supposing H0 to be true and taking another random

sample, theprobabilityofobtaining aworse result is large and therefore the result thatwehave

1 This value can be calculated through normal distribution tables or using English Microsoft Excel functionNORMDIST(1.4;0;1;TRUE)whichgivestheareaunderthenormaldistributionontheleftof1.4,equalto8%.Areaontherightof+1.4isobviouslythesame.

3 2 1 0 1 2 3+1.41.4

H0probablyfalse 0

H0probablytrue +1.40

H0probablyfalse

AdvancedStatistics

Page9

obtainedcanbeconsidered tobe reallyclose to 0,somethingwhichpushesus toaccept the ideathatH0 is true.When, instead,significance issmall, itmeans that ifwesuppose thatH0 is truewehaveasmallprobabilityofgettingsuchabadresult,somethingwhichpushesustobelievethatH0befalse.Intheexamplessituationwehaveasignificanceof 16%,whichusuallyisconsideredlarge(thechosencutpointistypically 5%)andthereforeweacceptH0.

Aslightlydifferentmethod,whichyieldstothesameresult,isfixingthecutpointapriori,letssay 5%,andfindingthecorrespondingcriticalvalueafterwhichthestatisticisintherejectionregion.Inourcase,consideringtwoareasof 2.5% onthe leftandontherightside,thecriticalvalueforastandardnormaldistributionis2 1.96.

Atthispointthesituationis

The firstmethod givesus an immediate and straightforward answer and in fact is theone

typicallyusedbycomputerprograms.Thesecondmethodinsteadismoresuitedforonetailedtestsandiseasiertoapplyifacomputerisnotavailable.

Anexampleofaonetailedtestisthesituationwhenwewanttocheckwhethertheexpectedvalueoftheageissmallerorlargerthan35.Wewritethehypothesesinthisway:


Inthiscase,thedifferenceof 1.2 betweensampleaverageand 35,sinceitispositive,leadsustostronglybelievethatH0betrue.Infact,nowthesituationofthestatisticisdifferentfrombefore,i.e.

2 This value canbe calculated throughnormaldistribution tablesorusingEnglishMicrosoftExcelNORMINV(2.5%;0;1)which gives the critical value1.96 forwhich the areaunder thenormaldistributionon the leftof it is2.5%.Due tosymmetricityofthedistribution,criticalvalueontherightisobviously+1.96.

3 2 1 0 1 2 3+1.961.96

H0 probably false H0 probably false

0

H0 probably true

+1.40 +1.961.96

AdvancedStatistics

Page10

InfactherewedonothaveanydoubtsincethestatisticvaluefallsrightinthemiddleoftheH0truearea.

Writinghoweverthehypothesesinthisway: H0: Eage 35 H1: Eage 35

Inthiscase,thesituationofthestatisticis

andherewehavethesameproblemofdeterminingwhether 1.40 iscloseto 0 orfarawayfromit.Asusual,todetermineitwehavetwomethods.Thefirstonecalculatestheprobabilityofgettingaworseresult,whereworsemeansworseforH0. Inthissituation,however,aworseresult is largerthan 1.40,while results smaller than 1.40 are strongly in favor of H0. The statistic is alwaysdistributedlikeastandardnormal,underthehypothesisthatH0betrue,

and thearea, thus the significance, is 8%.Using the secondmethod the criticalvalue isnot 1.96anymore,but3 1.64.Thecriticalregion is largerthanbefore,sincenowthe 5% isallconcentratedontheleftpart.

3 ThisvaluecanbecalculatedthroughnormaldistributiontablesorusingEnglishMicrosoftExcelNORMINV(5%;0;1)whichgivesthecriticalvalue1.64forwhichtheareaunderthenormaldistributionontheleftofitis5%.Duetosymmetricityofthedistribution,criticalvalueontherightisobviously+1.64.

3 2 1 0 1 2 3+1.4

H0 probably false H0 probably true

0

H0 probably true

+1.40 +1.64

H0probablytrue0

H0probablytrue+1.40

H0probablyfalse

H0probablyfalse0

H0probablytrue+1.40

H0probablytrue

AdvancedStatistics

Page11

3.2 NullandalternativehypothesisThehearthofastatisticaltestisnullhypothesisH0,whichrepresentstheinformationthatwe

are officially trying to extend from the sample to the population4. It is important that the nullhypothesis gives us additional information, since we need to suppose it to be true and use itsinformation toknowthestatisticsdistribution. If, inthepreviousexample,thenullhypothesishadnotgivenustheadditionalinformationthattherealexpectedvaluebe 35,wecouldnotusethefactthat that statistic function be normally distributed. Therefore, the null hypothesis must alwayscontainanequality,whilestrictinequalitiesarereservedforH1.Whenthetestisonetailed,wewritethe null hypothesis in the form of a nonstrict inequality such as Eage 35 for practicalpurposes, but theoreticallywe shouldwrite the equality Eage 35 and simply not take intoaccountthe Eage 35 possibility.

Forexample,usablehypothesesareE 35ordistributionof isexponentialoreven and areindependent.Ontheotherhand,hypothesessuchasE 35ordistributionof isnotexponentialarenotacceptable.Also and aredependent isnotacceptable,since itdoesnotprovideuswithanyinformationonhowtheyaredependent.

TogetherwithnullhypothesiswealwayswritealternativehypothesisH1,which isthe logicalcontradictionofnullhypothesis.3.3 TypeIandtypeIIerror

Once the statistic iscalculatedwemust takeadecision:acceptH0or rejectH0.WhenH0 isrejected,wefaceoneofthetwofollowingsituations: nullhypothesisisreallyfalseandwerejectedit:verygood; nullhypothesisisreallytrueandwerejectedit:wecommittedatypeIerror.IfweacceptH0,wefaceoneofthetwofollowingsituations: nullhypothesisisreallyfalseandweacceptedit:wecommittedatypeIIerror; nullhypothesisisreallytrueandwerejectedit:verygood.

Thereare twodifferent typesoferrors thatwemaycommitwhen takingadecisionafterastatistical test and itwould bewonderful ifwe could reduce at the same time the probability ofcommittingboth errors.Unfortunately, the onlymethod to reduce the probability to commit botherrorsistakingalargesample,hopefullytakingtheentirepopulation.Thisthingisclearlynotfeasibleinmanysituationswheregatheringdataisveryexpensive.

There is amethod to reduceprobabilityof committing a type Ierror: rejectingonly in thesituationswhereH0isevidentlyfalse.InthiswayatypeIerrorwillbeveryraresincewearerejectinginveryfewsituations.Unfortunately,ifwerejectwithparsimony,wewillacceptveryoftenandthismeanscommittingalotoftypeIIerrors.Samethingif,viceversa,werejecttoomuch:wewillcommitveryfewtypeIIerrorsbutmanytypeIerrors.

Thus,wemustdecidewhicherroristhemoresevereoneandtrytoconcentrateonreducingthe probability of committing it. Every statistical research concentrates on type I errors, trying to

4 As we will see later, it is instead H1 the information that we will be able to extend to the population, while,unfortunately,itisneverpossibletoextendH0.

AdvancedStatistics

Page12

reduce theprobabilityofcommitting themundera significance levelusually 5% or 1%.Usinganexampledrawnfromajuridicalsituation:

H0:suspectdeserves0yearsofprison(suspectisinnocent) H1:suspectdeserves>0yearsofprison(suspectisguilty)

Inthiscase,atype Ierrormeanscondemningan innocent,whileatype IIerrormeansan innocentverdictforaguilty. It iscommonbeliefthat inthiscaseatype Ierrorshouldbeavoidedatallcost,whileatypeIIerrorbeacceptable.

Thereasonwhystatisticaltestsconcentratetheirattentiononavoidingtype Ierrorsderivedfromthehistoricaldevelopmentofsciencewhichtakesascorrectthecurrenttheories(H0)andtriestominimizetheerrortodestroy,bymistake,awellestablishedtheoryinfavorofnewtheories(H1).Itisthereforeaconservativeapproach.Forexample:

H0:hearthpumpsblood H1:hearthdoesnotpumpblood

Atype Ierror inthiscasewouldbeadisastersince itwouldmeanrejectingthecorrecthypothesisthatbloodispumpedbyhearth,givingusnoothercluesinceH1carriesonlyanegativeinformation.3.4 Significance

Significance or pvalue is the probability of committing a type I error. This probability iscalculatedassumingthatH0betrueandcomparingthevalueofthestatisticthatwecalculateonoursamplesdatawith the statisticsdistribution.A small significancemeans that ifwe rejectwehaveonlya smallprobabilityof committingamistake,and thereforewewill reject.A large significancemeansthat ifwerejectwearefacinga largeprobabilityofcommittingamistake,andthereforewewillacceptH0.

Another equivalent definition for the significance is the probability of obtaining, takinganother randomsample,anequalorworsestatisticsvalueunderthehypothesis thatH0betrue.AsmallsignificancemeansthatthestatisticsvalueisreallybadandthereforewewillrejectH0.AlargesignificancemeansthatthestatisticsvalueismuchbetterthanwhatweexpectedandthereforewewillacceptH0.

Sincewetrytominimizetype Ierrors,wewillfixaverysmallsignificance levelunderwhichnullhypothesis is rejected,usually 5% or 1%. In thisway,probabilityofa type Ierror is lowandwhenwerejectwearealmostsurethatH0isreallyfalse.

Confidenceisequalto 100% minusthesignificance.3.5 Acceptandreject

Attheendofthestatisticaltestwemustdecidewhetheracceptingorrejecting: ifsignificanceisabovethesignificancelevel(usually 5% or 1%),weacceptH0; ifsignificanceisbelowthesignificancelevel,werejectH0.ItisveryimportanttounderlinethefactthatwhenwerejectwearealmostsurethatH0isfalse,sincewearekeepingtypeIerrorsunderasmallsignificancelevel.However,whenweacceptwemaynotsaythatH0betrue,sincewedonothaveanyestimationontype IIerrors.Therefore,rejecting isasurething,whileacceptingisanoanswerandfromitwearenotallowedtodrawanyconclusion.

AdvancedStatistics

Page13

Thisapproachiscalledfalsification,sinceweareonlyabletofalsifyH0andnevertoproveit.IfweneedtoprovethatH0betrue,wemustrewritethehypothesesandputtheinformationwewanttoextendtothepopulationintheH1hypothesisinstead,performthetestagainandhopetoreject.

Another important effect thatwemust underline is the sample size.When sample size isextremely small,data are almost random andprobabilityof committing type I error is very large.Thereforesignificanceisverylargeand,usingthetraditionalsmallsignificancelevels,wewillaccept.Thereforea statistical testwith fewdataautomaticallyaccepts everything, since itdoesnothaveenoughdatatoprovethatH0befalse.Again,acceptingmustneverimplythatH0betrue.3.5.1 Paradox

Usingthefalsificationapproachwecan,throughasmartchoiceofnullhypotheses,accepttwocontradictorynullhypotheses.Usingassampletheoneofthepreviousexampleandformulatingthehypotheses


weaccept Eage 35 withasignificancelevelof 5%.Usinginsteadthesehypotheses H0: Eage 36 H1: Eage 36

we accept Eage 36 with a significance level of 5%.We have thus accepted two hypotheseswhichsaydifferentandcontradictorythings.This isonlyanapparentparadox,sinceacceptingdoesnotmean that theyare truebutonly that theymightbe true.Therefore, for thepopulation fromwhichoursampleisextracted,theexpectedvaluemightbe 35 or 36 (ormanyotherclosevalues,suchas 35.3, 36.5, 37,etc.).This isduetoarelativelysmallsizeofthesample; ifwe increasethesamplesize,theintervalofvaluesforwhichweacceptwoulddecrease.3.6 Tailsandcriticalregions

Statistical testswhere thenullhypothesis containsanequalityandalternativehypothesisanot equality are twotailed tests. Statistical testswhere the null hypothesis contains a nonstrictinequalityandalternativehypothesisastrictinequalityareonetailedtests,suchas


Thenameof these testscomes from thenumberofcritical regions.Acritical region isanarea forwhichnullhypothesisisrejectedwhenthestatisticsvaluefallsinthatarea,accordingtothesecondmethodthatwehaveseen intheexample3.1.Thenumberofcriticalregions,whichusuallyarefarawayfromthecenterofthedistributionandthereforearecalledtails,determinesthenameofthetesttwotailedoronetailed.

twotailedtest

critical region critical region

0 +CC

AdvancedStatistics

Page14

onetailedtestwithcriticalregionontheright onetailedtestwithcriticalregionontheleftThepointwherethecriticalregionstartsiscalledcriticalvalueandisusuallycalculatedfrom

tablesofthestatisticsdistribution.Inthetwotailedtestthetworegionsarealwayssymmetric,whileforonetailedtestwefacetheproblemofdeterminingonwhichsideistherejectionregion.

Inordertofindwherethecriticalregion is inonetailedtests,wetrytoseewhathappens ifwehaveanextremely largepositivevalueforthestatistic. Ifsuchanextremely largepositivevalue(which,beingverylarge,isforsureintherighttail)isnotinfavorofnullhypothesis,itmeansthattheright tail isnot in favorofnullhypothesisandtherefore it istherejectionregion.Otherwise, if thisextremely large value of the statistic is in favor of the null hypothesis, the right region is not arejectionregionandthecriticalregionisontheleft.Forexample,weconsiderexample3.1


andweusethesamestatistic agesamplevariance .Whenthisstatisticsvalueispositiveandextremelylarge,itmeansthattheaverageofageismuchmorethanthehypothesizedexpectedvalueand this is a clear indication that the real expected value is much larger than 35. This is incontradictionwithnullhypothesiswhichsays thatexpectedvaluemustbesmallerorequal to 35.Thereforeapositivevalueof thestatistic,on theright tail, iscontradictingnullhypothesisand thismeansthatrighttailisacriticalregion.

Consideringinsteadhypotheses

H0: Eage 35 H1: Eage 35,

whenthestatisticsvalue ispositiveandextremely large, itmeansthattheaverageofage ismuchmorethanthehypothesizedexpectedvalueandthisisaclearindicationthattherealexpectedvalueismuchlargerthan 35.Thisisexactlywhatthenullhypothesissays.Thereforeapositivevalueofthestatistic,ontherighttail,isinfavorofthenullhypothesisandthismeansthatrighttailisnotcriticalregion.Thereforethecriticalregionisontheleft.

Someimportantfeaturestonoteoncriticalvalues:

decreasing significance level implies that criticalvaluegoesaway from 0.This isevident ifweconsiderthefactthatdecreasingthesignificance levelweareevenmoreafraidoftype Ierrors

critical region

01.41

critical region

0 +1.41

critical region

0C critical region

0 +C

AdvancedStatistics

Page15

andthereforewerejectwithmuchmorecare,thusreducingtherejectionzone; thecriticalvalueofaonetailed test isalwayscloser to 0 than thecriticalvalueof twotailed

tests.This isbecausethecriticaltailofaonetailedtestmustcontaintheprobabilitythat foratwotailedtestinsplitintworegionsandthereforethezonemustbelarger;

for each twotailed test there are two corresponding onetailed tests. One of them has thestatisticsvaluecompletelyon theothersideof therejection region, therefore for thisonewealways accept. This is the reason why using the significance method to determine whetheracceptingorrejectingcanbemisleadingforonetailedtests,since it isnotevidentwhetherthetesthasanobviousacceptverdictornot.

3.7 ParametricandnonparametrictestThereareparametricandnonparametricstatisticaltests.Aparametrictest impliesthatthe

distributioninquestionisknownuptoaparameterorseveralparameters.Forexample,itisbelievedthatmanynaturalphenomenaarenormallydistributed.Estimating and ofthephenomenonisaparametricstatisticalproblem,becausetheshapeofthedistribution,anormalone,isknownuptothese two parameters. On the other hand, nonparametric test do not rely on any underlyingassumptionsabouttheprobabilitydistributionofthesampledpopulation.Forexample,wemaydealwithcontinuousdistributionwithoutspecifyingitsshape.

Nonparametrictestsarealsoappropriatewhenthedataarenonnumericalinnaturebutcanberanked,thusbecomingranktests.Forexample,tastetestingfoodswecansaywe likeproductAbetter thanproductB,andBbetter thanC,butwecannotobtainexactquantitativevalues for therespectivemeasurements.Otherexamplesaretestswherethestatisticisnotcalculatedonsamplesvaluesbutontherelativepositionsofthevaluesintheirset. 3.8 Prerequisites

Each test, especially parametric ones,may have prerequisiteswhich are necessary for thestatistictobedistributedinaknownway(andthusforustocalculateitssignificance).

A typical prerequisite formany parametric tests is that the sample comes from a certaindistribution.Toverifyit: if data are not individualmeasures but are averages ofmany data, the central limit theorem

guaranteesusthattheyareapproximatelynormallydistributed; ifdataaremeasuresofanaturalphenomena,theyareoftenaffectedbyrandomerrorswhichare

normallydistributed; wecanhypothesizethatdatacomesfromacertaindistributionifwehavetheoreticalreasonsto

doit; wecanplotthehistogramofthedatatohaveahintontheoriginalpopulationsdistribution, if

thesamplesizeislargeenough; we can perform specific statistical tests to check the populations distribution, such as

KolmogorovSmirnovorJarqueBeratestsfornormality.Everytesthasasaprerequisitethatthesamplebearandomsample,eventhoughwewillnot

indicateit.

AdvancedStatistics

Page16

4. Tests4.1 StudentsttestforonevariablePrerequisites:variablenormallydistributed(ifsamplevarianceisused).H0:expectedvalue= Statistic:

nm

/ variancesampleor population average sample

Statisticsdistribution:Studentstwith 1 degreesoffreedom;when 30standardnormal. SPSS:AnalyzeCompareMeansOneSampleTTest

WilliamStudentGosset

(18861937)Studentsttestistheonewehavealreadyseenintheexampleinitslargesampleversion.Itis

atestwhichinvolvesasinglerandomvariableandcheckswhetheritsexpectedvalueis ornot. Forexample,taking 32 andasampleof 10 elements:25;26;27;28;29;30;30;31;33;

34 H0: E 32 H1: E 32

Sampleaverage is 29.3 andsamplestandarddeviation is 2.91.Statistic istherefore 2.94 and itssignificanceis5 1.7%.H0isrejectedsince 1.7% isbelowsignificancelevel;thismeansthatextractinganothersampleof 10 elementsfromadistributionwithanexpectedvalueequalto 32,wehaveaverysmallprobabilityofgettingsuchbadresults.Wecanthussaythatexpectedvalueisnot 32.

Aswecaneasilysee,Studentsttestforonevariableisexactlythetestversionoftheaverageconfidenceinterval.4.2 Studentsttestfortwopopulations

Prerequisites: twopopulationsAandBand thevariablemustbedistributednormallyon the twopopulationsH0:expectedvalueonpopulationA=expectedvalueonpopulationBStatistic:

BA

BA

nnn n n 11

2 varianceB populationor sample1A variance populationor sample1

average B sample averageA sample

Statisticsdistribution:Studentstwith 2 degreesoffreedom;when 31 standardnormal.SPSS:AnalyzeCompareMeansMeans SPSS:AnalyzeCompareMeansIndependentSamplesTTest

5 Significance canbe calculated in twoways. (1)UsingStudents tdistribution table. (2)UsingEnglishMicrosoftExcelfunctionTDIST(2.94;9;2)whichgivesusthesumofthetwotailsareas,thoseontheleftof2.94andontherightof+2.94.

AdvancedStatistics

Page17

This test is used whenever we have two populations and one variable calculated on thispopulation and we want to check whether the expected value of the variable changes on thepopulations.

Forexample,wewanttotest H0: Eheightformale Eheightforfemale H1: Eheightformale Eheightforfemale

Wetakeasampleof 10 males(180;175;160;180;175;165;185;180;185;190)e 8 female(170;175; 160; 160; 175; 165; 165; 180). We suppose that males and females heights are normallydistributedwith thesamevariance.Malessampleaverage is 177.5 while for female it is 168.75.Statisticsvalue is 2.18.Since it isonetailed testwedraw thegraph tohaveaclear ideawheredoesthestatisticfall.

If the statisticwereextremely large, thiswouldbe strongly incontradictionwithH0and thereforerejectionregioninontheright.

Critical value for onetailed test is6 1.76 and therefore we reject. Using instead the significancemethod,afterhavingcheckedthatstatisticdoesnotfallontheH0truearea,weget7 asignificanceof 2.2% andthereforewereject,meaningthatmalepopulationhasanexpectedheightsignificantlylargerthanfemalepopulation.

6 Criticalvaluecanbecalculatedintwoways.(1)UsingEnglishMicrosoftExcelfunctionTINV(5%;16),whichgivesusthecriticalvalueforthetwotailedtest,thereforeprobabilitysplitinto2.5%and2.5%.Foronetailedtestprobabilitymustbedoubled,TINV(10%;16),sinceinthiswayitwouldbesplitinto5%and5%.(2)UsingStudentstdistributiontable.7 Significancecanbecalculated in fourways.(1)Usingoneofthestatisticalttests (Zweistichprobenttest) intheDataAnalysis tookpak inMicrosoft Excel, choosing among known variances (in this case populations variances have to beindicated explicitly), equal andunknown, different and unknown (in these latter two casespopulations variances areestimatedfromsampledataautomaticallybyExcel),whichgivesusstatisticsvalueanditssignificance.(2)UsingEnglishMicrosoftExcelfunctionTTESTwhichgivesusthesignificancedirectlyfromthedata,choosingtype=2ifwesupposeequalvariancesortype=3 ifwesupposedifferentvariances.(3)UsingEnglishMicrosoftExcel functionTDIST(2.18;16;1)whichgivesustheareaofoneofthetwotails.(4)UsingStudentstdistributiontable.


0

H0 probably true

+1.76 +2.18

0

H0probablytrue +2.18

AdvancedStatistics

Page18

4.3 StudentsttestforpaireddataPrerequisites: two variables and on the same population and must be normallydistributedH0: E E,whichmeans E E Thetestcanalsobeperformedwithnullhypothesis:H0: E E Statistic:weuse asvariableandweperformStudentsttestforonevariableStatisticsdistribution:sameasStudentsttestforonevariableSPSS:AnalyzeCompareMeansPairedSamplesTTest

Thistest isusedwheneverwehaveasinglepopulationandtwovariablescalculatedon thispopulationandwewanttocheckwhethertheexpectedvalueofthesetwovariablesisdifferent.

Forexample,wewanttotestwhetherpopulationsincomeinacountryhaschanged.Wetakeasampleof 10 peoplesincomeandthenwetakethesame 10 subjectsincomethenextyear

Income2010(thousands)

Income2011(thousands)

Difference20102011

20 21 123 23 034 36 253 50 +343 40 +345 44 +136 12 +2476 80 444 45 112 15 3

Twothingsareveryimportanthere.Thesubjectsmustbeexactlythesame,noreplacementisclearlypossible.Whencalculating thedifference thesign is important,so it isagood idea toclearlywritewhatissubtractedfromwhat,especiallyforonetailedtests.

Hypothesesare: H0: Eincomefor2010 Eincomefor2011 0 H1: Eincomefor2010 Eincomefor2011 0

Sampleaverageforthedifferenceis 2.0 andsamplestandarddeviationis 8.07.Statisticis 0.78with8 asignificanceof 45.3%.H0isthusaccepted.Thisdoesnotmeanthatincomehasremainedthesame,butsimplythatourdataarenotabletoprovethatithaschanged.

8 Significancecanbecalculated infourways.(1)WiththeStudentsttestforonevariableformulausingm=0.(2)UsingEnglishMicrosoftExcelfunctionTTESTwhichgivesusthesignificancedirectlyfromthedata,choosingtype=1.(3)Usingthe statistical t test (Zweistichproben t testbeiabhngigStichproben) in theDataAnalysis tookpak inMicrosoftExcel,whichgivesusstatisticsvalueanditssignificance.(4)UsingStudentstdistributiontable.

AdvancedStatistics

Page19

4.4 FtestPrerequisites:twopopulationsAandBandthevariablemustbedistributednormallyonthetwopopulationsH0:VaronpopulationA=VaronpopulationBStatistic: sampleAvariance/sampleBvarianceStatisticdistribution:FishersFdistributionwith 1 and 1degreesoffreedom

GeorgeWaddel

Snedecor(18811974)

RonaldFisher

(18901962)ThenameofthistestwascoinedbySnedecorinhonorofFisher.Itchecksthevariancesoftwo

populations. It is interestingtonotethat,unlikealltheothertests,statisticsbestvalue forH0 is 1andnot 0.SinceFdistribution isonlypositiveandnot symmetric, specialcaremustbe taken intoaccount on the statistics positionwhen calculating the significance since it can bemisleading. Inparticular,theopposingstatisticsvalueisnottheoppositebutthereciprocal.

Forexample,supposingthatheightformaleandfemaleisnormallydistributed,wetest H0: Varheightformale Varheightforfemale H1: Varheightformale Varheightforfemale.

Weusetheprevioussampleandwegetasamplevarianceof 84.7 formaleand 55.4 for female.Statistic is thus 1.53. Degrees of freedom are 9 and 7. The two critical values are9 4.82 and

. 0.21 andthereforeweacceptH0.Usingthesignificancemethod,afterhavingcheckedthatthestatistic isontherightof 1,wegetanareaof 29% fortherightpartandthereforesignificance is58%.

9 Calculation of critical values or significance can be done in different ways. (1) Using the statistical F test(ZweiStichproben FTest) in the Data Analysis tookpak in Microsoft Excel, which gives us statistics value and itssignificance.(2)UsingEnglishMicrosoftExcelfunctionFTESTwhichgivesusthesignificancedirectlyfromthedata.Thismethod canbemisleadingwhen statistic ison the leftof1. (3)UsingEnglishMicrosoftExcel functionFDIST(1.53;9;7)whichgivesustheareaoftherighttail. (4)UsingEnglishMicrosoftExcel functionFINV(2.5%;9;7)and1/FINV(2.5%;9;7) toget the twocriticalvalues.Payattention to the inverteddegreesof freedom for the secondcalculation. (5)UsingFdistributiontable,whichhoweverusuallyprovidesonlythecriticalvalues.

1

critical region critical region

4.821.530 0.21

AdvancedStatistics

Page20

4.5 Onewayanalysisofvariance(ANOVA)Prerequisites: populations, variable is normally distributed on every populationwith the samevarianceH0:expectedvalueofthevariableisthesameonallpopulationsStatistic: VarianceBetweenVarianceWithin

Statisticdistribution:FishersFdistributionwithdegreesoffreedomequalto 1 and SPSS:AnalyzeCompareMeansMeans SPSS:AnalyzeCompareMeansOneWayANOVA

This test is the equivalent of Students t test for two unpaired populations when thepopulationsaremorethantwo.Wenotethatifonlyonepopulationhasanexpectedvaluedifferentfromtheother,thetestrejects.Therefore,arejectionguaranteesusthatpopulationsdonothavethesameexpectedvaluebutdoesnottelluswhichpopulationsaredifferentandhow.OptimalstatisticvalueforH0is 0 and,sinceFdistributionhasonlypositivevalues,thistesthasonlytherighttail.

Forexample,wehaveheightsforyoung(180;170;150;160;170),adults(170;160;165)andold(155;160;160;165;175;165)andwewanttocheck

H0: Eheightforyoung Eheightforadults Eheightforold H1:atleastoneofthe Eheight isdifferentfromtheothers

We supposeheights arenormallydistributedwith the same variance. Fromdatawegeta sampleaverageof 166 foryoung, 165 foradultsand 163.3 forold.Nowweaskourselveswhetherthesedifferencesarelargeenoughtosaythattherearedifferencesamongpopulationsexpectedvaluesornot.

Theoriginsoftheanalysisofvariancelieinthesplittingofsamplesvarianceinthisway10:

10 Variance

2

AdvancedStatistics

Page21

Variance 1

1

We now define the samples variance between groups as a measure of the averages variationsbetweenvaluesofdifferentgroups

variancebetween 1 11

and the samples variancewithin group as ameasureof the variations among valuesof the samegroup

variancewithin 1 1

Theideabehindthetestistocomparethesetwomeasures:ifthevariancebetweenismuchlargerthanthevariancewithin,itmeansthatatleastonepopulationissignificantlydifferentfromtheothers,while if the variance between is not large compared to the variancewithin itmeans thatvariationsduetoachangeinthepopulationhavethesamesizeasvariationsduetoothereffectsandcanthusbeconsiderednegligible.Simplifyingthe 1/ thestatisticis

variancebetweenvariancewithin 1

1

1

whichisdistributedasaFishersFdistributionwith 1 and degreesoffreedom.Rejectionregionisclearlyontheright,sincethatareaistheonewhereVarianceBetweenismuchlargerthanVarianceWithin.

Goingbacktoourexample,statisticsvalueis . ./ 0.136 withdegreesoffreedom2 and 11 andasignificance11 of 87.4% andthereforeweaccept.

0

11 Significancecanbecalculatedindifferentways.(1)UsingtheonewayANOVA(ANOVA:EinfaktorielleVarianzanalyse)in theDataAnalysis tookpak inMicrosoft Excel,which gives us statistics value and its significance. (2)Using EnglishMicrosoftExcel functionFDIST(0.136;2;12)whichgivesustheareaoftherighttail. (3)UsingFdistributiontable,whichusuallyprovidestherightsidecriticalvalues.

AdvancedStatistics

Page22

4.6 JarqueBeratestPrerequisites:none.H0:variablefollowsanormaldistributionStatistic:

4Kurtosis sampleskewness sample

6

22n

Statistic distribution: JarqueBera distribution. When 2000, chisquaredistributionwith2degreesoffreedom.

CarlosJarque AnilBeraThis testcheckswhetheravariable isdistributed,on thepopulation,according toanormal

distribution.ItusesthefactthatanormaldistributionhasalwaysaskewnessandaKurtosisof 0.Itsstatisticisclearlyequalto 0 ifthesamplesdatahaveaskewnessandKurtosisof 0 andincreasesifthesemeasuresaredifferentfrom 0.Thestatisticismultipliedby ,meaningthatifwehavemanydatatheymusthavedisplayverysmallskewnessandKurtosistogetalowstatisticsvalue.

SamplesskewnessandsamplesKurtosisarecalculatedas

1

1

1

1

3.

4.7 KolmogorovSmirnovtestPrerequisites:none.H0:variablefollowsaknowndistributionStatistic: sup numberofsampledata , where isthecumulativedistributionoftheknownr.v.Statisticdistribution:KolmogorovdistributionSPSS:AnalyzeNonparametricTestsOneSample

AndreyKolmogorov

(19031987)

VladimirIvanovich

Smirnov(18871974)

Thisisaranktestwhichcheckswhetheravariableisdistributed,onthepopulation,accordingtoaknowndistributionspecifiedbytheresearcher.Thetestforeach calculatesthethedifferencebetweenthepercentageofsamplesdatasmallerthanthis andtheprobabilityofgettingavaluesmallerthan fromtheknowndistribution.Clearly,ifsamplesdataaredistributedaccordingtotheknown distribution, these differences are very small for every since the percentage of smallervalues reflects exactly the probability of finding smaller values. The statistic is defined as themaximum,forallthe ,ofthesedifferences.

Forexample,wewant to checkwhetherdata3;4;5;8;9;10;11;11;13;14 come fromaN9; 25 distribution. For 2 , numberofsampledata N9;252 |0 0.05| 0.05 ; for 3 , numberofsampledata N9;253 |0 0.08| 0.08 ; for 4 ,

numberofsampledata

N9;254 |0.1 0.12| 0.02; for 5 , numberofsampledata N9;255 |0.2 0.16|

AdvancedStatistics

Page23

0.04 andsoon.Obviously,thiscalculation isnotdoneonlyfor integervaluesbutforallvaluesanddoingitmanuallyis,inmanycases,averyhardtask.Inthiscase,themaximumis 0.21 obtainedforavalueof immediatelyafter 11.Itssignificanceismuchlargerthan 5% andthereforeweaccept.4.8 SigntestPrerequisites:continuousdistribution.H0:medianis Statistic:outcomesontheleftorontherightof Statisticdistribution: B; 50%;for 10 ... ~N0; 1SPSS:AnalyzeNonparametricTestsOneSample

Signtestisaranktestwhichteststhecentraltendencyofaprobabilitydistribution.Itisusedtodecideonwhetherthepopulationmedianequalsornotthehypothesizedvalue.

Consider theexamplewhen 8 independentobservationsofa random variable havingacontinuousdistributionare0.78,0.51,3.79,0.23,0.77,0.98,0.96,0.89.Wehavetodecidewhetherthedistributionmedian isequalto 1.00.Weformulatethetwohypotheses:

H0: 1.00 H1: 1.00

Ifthenullhypothesisistrue,weexpectapproximatelyhalfofthemeasurementstofalloneachsideofthehypothesizedmedian.Ifthealternativeistrue,therewillbesignificantlymorethanhalfononeofthesides.Thus,ourteststatisticwillbeeither or .Thesetwoquantitiesdenotethenumberof observations falling below and above 1.00 . Since was assumed to have a continuousdistribution, P 1.00 0. Inotherwords,everyobservation fallseitherbelowofabove 1.00,neverhittingthisvalueitself.Consequently, 8.Inpracticeitcanbethatanobservationisexactly 1.00. In this situation, since this observation is strongly in favorofH0 hypothesis,wewillconsiderittobelongto when islargerandto when islarger.

Notethatthischoiceofteststatisticdoesnotrequirehavingexactvaluesoftheobservations.Infact,itisenoughtoknowwhethereachobservationislargerorsmallerthan 1.00.Tothecontrary,the corresponding small sample parametric test (which is the Students t test for one variable)requiresexactvaluesinordertocalculatethesamplesaverageandvariance.

Nowwe take andconsider thesignificanceof this test.This is theprobability (assumingthatH0 is true)ofobservingavalueof the teststatistic that isat leastascontradictory to thenullhypothesis,andthussupportivetothealternativehypothesis,astheactualonecomputedfromthesampledata. Inour case 7.There are twomore contradictoryoutcomesof theexperiment:when 8, the casewhen all observations have fallen on the same side of the hypothesizedmedian, and when 0. And there is a result which is as contradictory as the one we have, 1.Thussignificanceequals P 7 P 8 P 1 P 0.

Notethatthedistributionof hasabinomialdistribution B8; 0.5.Indeed,ifwesupposethatH0 is correct, having an outcome on the leftof 1.00 is an eventwith probability 50%.Andhaving outcomes on the left of 1.00 on a total of 8 independent observations is a binomialwith 50% and 8.Therefore,rememberingthat

AdvancedStatistics

Page24

PB; !! !1

wecancalculate12 P 7 P 8 0.035.Remembering that thebinomialdistribution intheparticularcaseof 50% issymmetricandtherefore P 1 P 0 P 7 P 8, we get that significance is 7%. Setting a significance level of 5%, we accept nullhypothesismeaningthatourdataarenotabletosupportthehypothesisthatmedianisnot 1.00.

Thecorrespondingonetailedtestisusedtodecideonwhetherthedistributionmedianequalstothehypothesizedvalueorfallsbelow/exceedsit.Referringtothesetofdataconsideredabove,thecorrespondingtwomutuallyexclusivehypothesesread,forexample:

H0: 1.00 H1: 1.00

Asteststatisticwechoose .Inordertofindoutwhereistherejectionregion,wenotethatwhenourstatisticishugetheobservationsfallingbelow 1 willbemorenumerousthantheonesexceeding1 and this is in favorwith the alternative hypothesis. Thus the zone on the right is the rejectionregion,while the zone on the left,where is small, is not a rejection region. Because 7,there is only one more contradictory to H0 outcome is 8. Thus the significance equalsP 7 P 8 . The random variable has always a binomial distribution whoseprobability of a success is 1/2 and we conclude that the significance is PB8; 50% 7 PB8; 50% 8 3.5%.

Thus,whenH0 istrue,theprobabilityto faceanoutcomeascontradictoryastheactuallyobservedoneoranoutcomemorecontradictorytoH0,equals 3.5%.Consequently,thesampledatasuggestthatifwerejectH0wemaybewronginonly 3.5% ofthecases.

Note that,as comparedwith the twotailed test,now theprobabilityof type Ierror is twotimessmalleralthoughthesample informationremainsthesame.This isnotsurprisingbecausetheonetailed test starts fromamorepreciseguess, it startswith the implicithypothesis that canneverbelargerthan 0.

Ifwemaketheotheronetailedtestinstead: H0: 1.00 H1: 1.00,

ifwetake asstatistic, inordertofindoutwhere istherejectionregion,wenotethatwhenourstatisticishugetheobservationsfallingbelow 1 willbemorenumerousthantheonesexceeding 1andthisisinfavorwiththenullhypothesis.Thereforelargervaluesofthestatisticareallinfavorof

12 Thesequantitiescanbemucheasilycalculatedintwodifferentways:(1)usingbinomialdistributioncumulativetables,which givedirectly PB; and in our case P(B(8;50%)=7) + P(B(8;50%)=8) = 100% P(B(8;50%)6); (2) usingEnglishMicrosoftExcelfunction100%BINOMDIST(6;8;50%;TRUE)whichgivesus100%P(B(8;50%)6).


4

H0 probably true

7

AdvancedStatistics

Page25

H0.Thereforetherejectionregionisnowforsmallvaluesofthestatistic

Withoutevencalculatingthesignificance,itisevidentthatwemustacceptH0.Inanycase,theworsecasesare 6, 5, 4, 3, 2, 1 and 0.Therefore, P 7 PB8; 50% 7 0.996.

Recall that the normal distribution provides a good approximation for the binomialdistributionwhenthesamplesizeislarge(usually 10).Thus,usingthecentrallimittheorem,wemayuse N0.5; 0.25 toapproximatethedistributionofourstatistic.Usingstandardization

0.5 0.25 ~N0; 1,

where isourstatistic or .Duetotechnicalreasons13 acorrectionof 0.5 isappliedtotheformula

0.5 0.5 0.25 ~N0; 1,

Forexample,wehaveasampleof 30 elementswith 18 elementsonthe leftof 2.00 and12 elementsontherightof 2.00 andwewanttotest

H0: median 2.00 H1: median 2.00.

13 Atechnicalproblemwhichariseswheneverwetrytoapproximateadiscretedistribution(B; 50% inourcase)withacontinuousone (N0.5; 0.25 inourcase).Discreteprobabilitydistributiondoesnothaveanyprobability fornonintegervalues,whilecontinuousonedoes.

Thereforewehavetodecidewhattodowiththevaluesbetween 12 and 13,wherethebinomialdistributiondoesnotexists, however the normal distribution has a consistent probability. We take a compromise, taking for the normalapproximationsall the valuesup to 12.5. Thereforeweadd a 0.5 to theprevious formula. It is alwaysanadditionwheneverweareonthelefttail,whileitisclearlyasubtractionwheneverweareontherighttailandhavethusa sign:... ~N0; 1

H0 probably true H0 probably false

4

H0 probably true

7

AdvancedStatistics

Page26

We takeas statistic . Since it isaonetailed testwehave to seewhere the rejection region is.Supposingavery largevalue for thestatistic, i.e. 30, thismeans thatprobably themedian ismuchlargerthanthehypothesizedvalueandthisisinfavorofH0.Therefore,rejectionregionisnotforlargestatisticsvalueanditisontheotherside,theleftone.ValuesmoreorequalcontradictorytoH0arethus 12.Usingtheexactcalculationyieldsto PB30; 50% 12 18.07%,whileusingapproximatedcalculation14 wehave

P N0; 1 12 0.5 0.5 300.25 30 PN0; 1 0.9129 18.06%.Inbothcasesweaccept,meaningthatoursampledataarenotabletoprovethatH0bewrong.4.9 MannWhitney(Wilcoxonranksum)testPrerequisites:thetwoprobabilitydistributionsarecontinuous H0:positionofdistributionforpopulationA=positionofdistributionforpopulationBStatistic:sumofranksofthesmallergroupStatisticdistribution:Wilcoxonranksumtableor N0; 1 whensampleislargeandtablesarenotavailableAlternativestatistic: sumofranksofthesmallergroupminus 1/2,where isthesizeofthesmallergroupAlternativestatisticdistribution:MannWhitneytableor N0; 1 whensampleislargeandtablesarenotavailableSPSS:AnalyzeNonparametricTestsIndependentSamples

Supposetwo independentrandomsamplesaretobeusedtocompare twopopulationsandwe are unwilling to make assumptions about the form of the underlying population probabilitydistributions (andthereforewecannotperformStudentst test fortwopopulations)orwemaybeunable toobtainexact valuesof the samplemeasurements. If thedata canbe ranked inorderofmagnitude, theMannWhitney test (also calledWilcoxon rank sum test) can be used to test thehypothesisthattheprobabilitiesdistributionsassociatedwiththetwopopulationsareidentical.

Forexample,supposesixeconomistswhoworkforthegovernmentandsevenwhoworkforuniversities are randomly selected, and each one is asked to predict next year's inflation. Theobjective of the study is to compare the government economists' predictions to those of theuniversityeconomists.Assumethegovernmenteconomistshavegiven:3.1,4.8,2.3,5.6,0.0,2.9.Theuniversityeconomistshavesuggested insteadthefollowingvalues:4.4,5.8,3.9,8.7,6.3,10.5,10.8.That is, there is a random variable equal to the next year's inflation given by a governmentaleconomist. Asking governmental economists about their prediction, we observe independentoutcomes, ,of .Aswell, there isanotherrandomvariable equal tothenextyear's inflationgivenbyauniversityeconomist.Approachingauniversityeconomistconcerninghis forecastof the

14 Theprobabilityofanormaldistributioncanbecalculatedintwoways:(1) looking intoastandardnormaldistributiontable;(2)usingEnglishMicrosoftExcelfunctionNORMDIST(2.5/SQRT(0.25*30);0;1;TRUE).

AdvancedStatistics

Page27

inflationrate,weobserveanindependentoutcomes, ,ofthisrandomvariable.Wehavetodecidewhether and have the same distributions or not, basing our decision only on the sampleobservations,whichistheonlyinformationwehave.

H0: the probability distribution corresponding to the government economistspredictionsofinflationrateisinthesamepositionastheuniversityseconomistsone

H1: the probability distribution corresponding to the government economistspredictionsofinflationrateisinadifferentpositionastheuniversityseconomistsone

Tosolvethisproblem,wefirstrankallavailablesampleobservations,fromthesmallest(arankof 1) to the largest (a rank of 13): 1 0.0, 2 2.3, 3 2.9, 4 3.1, 5 3.9, 6 4.4, 74.8, 8 5.6, 9 5.8, 10 6.3, 11 8.7, 12 10.5, 13 10.8. The test statistic for theMannWhitneytestisbasedonthetotalsoftheranksforeachofthetwosamplesthatis,onranksums. If the two rank sums arenearlyequal, the implication is that there isnoevidence that theprobabilitydistributionsfromwhichthesamplesweredrawnaredifferent.Ontheotherhand,whenthetworanksumsdiffersubstantially,itsuggeststhatthetwosamplesmayhavecomefromdifferentdistributions.Wedenote the rank sum forgovernmentaleconomistsby and that foruniversityeconomists by . Then 4 7 2 8 1 3 25 and 6 9 5 11 10 12 13 66.Thesumof and willalwaysequal 1/2,thatisthesumofallintegersfrom 1through . In the particular case in hands, 6 , 7 , 13 , and 1313 1/2 91.Since isfixed,asmallvaluefor impliesa largevaluefor (andviceversa)andalargedifferencebetween and .Therefore,thesmallerthevalueofoneoftheranksums,the greater the evidence to indicate that the sampleswere selected from different distributions.However,whencomparingthesetwovalues,wemustalsotakeintoaccountthefactthata maybesmallduetothefactthatthecorresponding issmall;inourcase, g maybesmallerbecausethegovernmentalsamplehaslesssubjects.Thetestsstatisticisanyofthetworanksums.Criticalvaluesfor thisstatisticaregiven inappropriateWilcoxon ranksum tables.We take g and lookingat thetablefor 6 and 7 weget,forasignificancelevelof5%,criticalvaluesof 28 and 56.

Since our statistic is in the critical region,we reject,meaning that our data confirm that the twodistributionsaredifferent.

NotethattheassumptionsnecessaryforthevalidityoftheMannWhitneytestdonotspecifytheshapeofprobabilitydistribution.However,thedistributionsareassumedtobecontinuoussothatthe probability of tied measurements is zero, and, consequently, to each measurement can beassignedauniquerank.Inpractice,however,roundingofcontinuousmeasurementsmaysometimesproduceties.As longasthenumberofties issmallrelativetothesamplesizes,theMannWhitneytestprocedure isapplicable.On theotherhand, the test isnot recommended tocomparediscretedistributionsforwhichmanytiesareexpected.Tiesmaybetreatedinthefollowingway:assigntiedmeasurementstheaverageoftherankstheywouldreceiveiftheywereunequal.Forexample,ifthethirdrankedandfourthrankedmeasurementsaretied,weassigntoeachonearankof 3.5.Ifthethirdranked,fourthrankedandfifthrankedmeasurementsaretied,weassigntoeachonearank


5625 28

AdvancedStatistics

Page28

of 4. Returning toourexample,wemay formulate thequestionmoreexactly: is it true that the

university economists' predictions tend to be higher than the predictions of the governmentaleconomists? In other words, is the density shifted to the right with respect to density ? Conceptuallythisshiftequalsthesystematiccomponentinthedifferencebetweenthepredictionsofagenericuniversityeconomistandagenericgovernmenteconomist.Thatis:

H0: the probability distribution corresponding to the government economistspredictionsofinflationrateisinthesamepositionorshiftedtotherightwithrespecttotheuniversityseconomistsone

H1: the probability distribution corresponding to the government economistspredictions of inflation rate is shifted to the left with respect to the universityseconomistsone

Wehaveto findouttherejectionregion.Wetake g asstatisticandsupposethat itsvalue isverylarge.Thismeans thatgovernmentaleconomistsmakepredictionswith larger ranksand thuswithhigher values than universitys economists. This is strongly in favor ofH0 and therefore rejectionregion isontheotherside,the leftone.Criticalvaluesaredifferentandtheyare,forasignificancelevel of 5%, 30 and 54. Statistic falls in the rejection region and thus our data confirms thatgovernmentalpredictionsareshiftedtotheleft.

Whensamplesize, or , is largerthan 10,tablesdonotprovideuswithcriticalvalues

anymore.Inthesecasesstatisticdistributioncanbeapproximatedwithanormaldistribution 1 2 1 12

N0; 1.

4.10 WilcoxonsignedranktestPrerequisites:thedifferenceisarandomvariablehavingacontinuousprobabilitydistribution.

H0:positionofdistributionforvariableA=positionofdistributionforvariableBStatistic:sumofranksofdifferencesStatisticdistribution:Wilcoxonsignedranktableor N0; 1whensampleislargeandtablesarenotavailableSPSS:AnalyzeNonparametricTestsRelatedSamples FrankWilcoxon(18921965)

Rank tests can also be employed to compare two probability distributionswhen a paireddifferencedesignisused.Forexample,consumerpreferencesfortwocompetingproductsareoftencomparedbyanalyzingtheresponsesinarandomsampleofconsumerswhoareaskedtorateboth

H0 probably true H0 probably false H0 probably true

5425 30

AdvancedStatistics

Page29

products. Thus, the ratingshavebeenpairedoneach consumer.Consider forexample a situationwhen 10 students have been asked to compare the teaching ability of two professors, say and . Each of the students grades the teaching ability on a scale from 1 to 10,with highergradesimplyingbetterteaching.Theresultsoftheexperimentareasfollows:

student signof rankof 1 6 4 2 2 + 52 8 5 3 3 + 7.53 4 5 1 1 24 9 8 1 1 + 25 4 1 3 3 + 7.56 7 9 2 2 57 6 2 4 4 + 98 5 3 2 2 + 59 6 7 1 1 210 8 2 6 6 + 10

Here and arethegradesassignedbyeachStudentstoprofessor and .Sincethisisapaireddifferenceexperiment,weanalyzethedifferencesbetweenthemeasurements.Examiningthedifferencesallowsremovingapossiblecommoncausalitybehindtheseratings.Infact,thefourthandthesixthstudentsseemtohavegivenhigherthanotherstudentsratingstobothprofessors.

This rank test requires thatwecalculate the ranksof theabsolutevaluesof thedifferencesbetween the measurements. Since there are ties, the tied absolute differences are assigned theaverage of the ranks theywould receive if theywere unequal but successivemeasurements. Forexample,theabsolutevalue 3 appearstwotimes.Ifthesewereunequalmeasurements,theirrankswouldhavebeen 8 and 7.Thustherankfor 3 equals 7.5.Inthesameway,therankfor 2equals 5,therankfor 1 is

2.Aftertheabsolutedifferencesareranked,thesumof

theranksofthepositivedifferencesoftheoriginalmeasurements, ,andthesumoftheranksofthenegativemeasurements, ,arecomputed.Inourcase: 5 7.5 2 7.5 9 5 10 46 and 2 5 2 9.Nowwearereadytotestthenonparametrichypotheses:

H0:theprobabilitydistributionsoftheratingsforprofessor isinthesamepositionastheoneforprofessor , 1 2

H1: the probability distributions of the ratings for professor is in a differentpositionastheoneforprofessor , 1 2

As the test statisticwe use any . Themore the difference between and , the greater theevidencetoindicatethatthetwoprobabilitydistributionsdifferinlocation.Notethatalsoforthistestthe sum of is fixed and equal to 1/2. Left critical value is tabulated,while rightcriticalvaluecanbefoundforsymmetricity.Inourcase,wetakeforexample whichis8.Theleftcriticalvalue,forasignificancelevelof 5%,is 8.Theothercriticalvalueis 1/2 8 55 8 47.


27.5

H0 probably true

47 98

46

AdvancedStatistics

Page30

Asitcanbeseenintheschema,thistestisperfectlysymmetricandwhenone fallsintothecentralregion,theotherautomaticallydoesthesame.Viceversa,whenone falls intoarejectionregion,theotherfalls intotheotherrejectionregion. Inourexampleweacceptandthereforeourdataarenotabletoprovethatthetwodistributionsaredifferent.

Obviously,also forthistestwehaveonetailedversions.This isperformed intheusualway,takingcaretochooseonestatisticanddecidewhichtherejectionregionforthatstatisticis.

Sincewehaveassumedthatthedistributionofadifference iscontinuous,theremaynotbedifferenceswhichareexactly 0.However,inpractice,theymayoccurduetorounding:insuchcases,wemust decidewhether assigning their rank to or to . For the twotailed test there is nosolution. Since a difference of 0 is in favor of H0 hypothesis, assigning it to either statistic canunbalancethesituationandpushinfavorofH1.Moreover,adifferenceof0isstronglyinfavorofH0,butitwouldhavethesmallerrank.So,thetwotailedtestcannotbeperformedatallifwehaveany0 difference.However,theonetailedtestcanbeperformed.Forexample:

H0:theprobabilitydistributionsoftheratingsforprofessor isinthesamepositionorshiftedtotheleftwithrespecttotheoneforprofessor , 1 2, 1 2 0

H1: the probability distributions of the ratings for professor is shifted to the rightwithrespecttotheoneforprofessor , 1 2, 1 2 0

A difference of 0 is in favor of H0 hypothesis which includes also all the negative differences.Therefore,any 0 differencesrankisassigned,withthesehypotheses,to .

When 25 statistics tables are not available anymore. Statistics distribution can beapproximatedwith:

1 4 12 1 24 N0; 1,

whereitisbettertotakeasstatisticthesmallerbetween and ,sinceusuallystandardnormaldistributiontablesprovidetheareaontheleft.4.11 KruskalWallistestPrerequisites:thereare 5 ormoremeasurementsineachsample;the probabilitydistributionsfromwhichthesamplesaredrawnarecontinuous H0:positionofdistributionofpopulationsisthesameStatistic:

3 1

Statisticdistribution:chisquaredistributionwith 1 degreesoffreedomSPSS:AnalyzeNonparametricTestsIndependentSamples

WilliamHenry

Kruskal(19192005)

WilsonAllen

Wallis(19121998)

The KruskalWallis test is the MannWhitney test when more than two populations areinvolved.ItscorrespondingparametrictestistheAnalysisofVariance.

Forexample,ahealthadministratorwants to compare theunoccupiedbed space for three

AdvancedStatistics

Page31

hospitals. She randomly selects 10 different days from the records of each hospital and lists thenumber of unoccupied beds for each day. Just as with two independent samples, we base ourcomparisonontheranksumsforthesethreesetsofdata.Tiesaretreatedas intheMannWhitneytestbyassigningtheaveragevalueoftherankstoeachofthetiedobservations:

Hospital1 Hospital2 Hospital3Beds Rank Beds Rank Beds Rank6 5 34 25 13 9.538 27 28 19 35 263 2 42 30 19 1517 13 13 9.5 4 311 8 40 29 29 2030 21 31 22 0 115 11 9 7 7 616 12 32 23 33 2425 17 39 28 18 145 4 27 18 24 16 120 210.5 134.5

Wetest H0: the probability distributions of the number of unoccupied beds have the same

positionforallthreehospitals H1:at leastoneofthehospitalshasprobabilitypositiondifferentwithrespecttothe

others.

Theteststatistic,called ,is ,where denotesthenumberofdistributionsinvolved, is the number of measurements available for the th distribution, is thecorrespondingranksum, / isthemeanrank forpopulation and . . . /

(remembering that the sum of ranks is fixed, as for MannWhitney and

Wilcoxontests) isthemeanrankforthewholepopulation.As itcanbeseenfromtheformula,thisstatisticmeasurestheextenttowhichthe ranksdifferwithrespecttotheaveragerank.Notethat statisticisalwaysnonnegative.Ittakesonthevaluezeroifandonlyifallsampleshavethesamemeanrank,thatis forall .Thisstatisticbecomesincreasinglylargeasthedistancebetweenasamplemeanrank andthemeanrankforthewholepopulationgrows.

However,theformulathatisusedforpracticalcalculationsisaneasierone15:

12 1

3 1

In our case 3, 10 and 30. is

.

. 3 31

15

2

2

1

1

1241211231

AdvancedStatistics

Page32

6.097. The statistics distribution is, under the hypothesis that the null hypothesis is true,

approximately a chi square distribution with 1 degrees of freedom. This approximation isadequateaslongaseachofthe samplesizesisatleast 5.Chisquaredistributionhasonlyonetailontherightandthustherejectionregionforthetestislocatedintherighttail.Inourcase 3,sowe are dealingwith a chi square distributionwith 2 degrees of freedom. Using the significancemethod,we find16 a significance of 4.74% whichmeans thatwe reject.Using the critical regionmethodwith 5% significancelevel,wegetacriticalvalueof 5.99.

4.12 PearsonscorrelationcoefficientPrerequisites:coupleddata H0: Corr,

Statistic: Statisticdistribution:Studentstwith 2 degreesoffreedomSPSS:AnalyzeCorrelateBivariate

KarlPearson(18571936)

Consider two random variables, and ,ofwhichwehaveonly couplesofoutcomes,; . It is important that theoutcomes thatwehaveare incouples, sinceweare interesting inestimatingthecorrelationbetweenthetwovariables.WeuseasestimatorthePearsonscorrelationcoefficientwhichisdefined,throughtheintroductionofthe (sumofsquares)quantity,as

.

As it canbe seen from the formulas,quantities have two equivalentdefinitions,ofwhich thelatter is easier to use in practical calculations while the former is more useful for theoreticalconsiderations.Inparticular,wecanimmediatelyobservefromtheseconddefinitionthat and arestrictlypositiveandthereforethesquarerootandthedenominatorarewelldefined.Intheparticularcasewhenall the orall the have the samevalue, thecorresponding quantitybecomes 0 andthePearsoncorrelationcoefficient isnomoredefined.This isaveryrarecaseandcorresponds to thesituationwhen thereareonlyconstantoutcomes for randomvariable or ;clearly, fromconstantoutcomeswecannotestimateanythingconcerning thebehaviorof randomvariables.

is the estimation of the variance of random variable , while is the

16 Significancecanbecalculatedintwodifferentways.(1)UsingEnglishMicrosoftExcelfunctionCHIDIST(6.097;2)whichgivesustheareaofthelefttail.(2)Usingchisquaredistributiontable,whichusuallyprovidestheleftsidecriticalvalues.


6.0975.990

AdvancedStatistics

Page33

estimationofthevarianceofrandomvariable and istheestimationfor Cov, .Sincethe correlation is exactly Corr, Cov,VarVar , Pearsons correlation coefficient is theestimationforthecorrelation.

The sign of is determined only by the sign of . It can moreover be easilydemonstrated17 that and therefore the value of must lie between 1and 1,independentlyfromhowlargeorsmallarethenumbers and .Inotherwords, isascalelessvariable.Avalueof nearorequaltozeroisinterpretedaslittleornocorrelationbetween and . In contrast, the closer comes to 1 or 1, the stronger is the correlation of thesevariables. Positive values of imply a positive correlation between and . That is, if oneincreases,theotheroneincreasesaswell.Negativevaluesof implyanegativecorrelation.Infact, and move intheoppositedirections:when increases, decreasesandviceversa.Insum,thiscoefficientofcorrelationrevealswhetherthereisacommontendencyinmovesof and .

We have a test to checkwhether CorrX, Y is different from 0,meaning that there is alinearrelationbetweenrandomvariables and .Thistestusesthefactthatstatistic

21

isdistributedlikeaStudentstdistributionwith 2 degreesoffreedom.Weremindthefactthatindependenceimplieszerocorrelationbutnotviceversa:therefore,whenthecorrelationisdifferentfrom 0,wearesurethatthetworandomvariablesaredependent.

Forexample,supposewehavethese 11 couplesofdata 2 3 4 3 5 6 7 3 1 3 4 5 5 7 5 7 7 14 5 3 1 12

we get 4 9 16 9 25 36 49 9 1 9 16 11 3.7 30.18 , 10 15 28 15 35 42 98 15 3 3 48 11 3.7 6.5 47.36 and 138.73.Therefore ... 0.732 with 11 couplesofdataandthevalueofourstatisticis .. 3.223 and a significance, for the twotailed test, of 1.04% . Therefore, taking asignificance level of 5%, 11 couples of data with Pearsons correlation coefficient of 0.732 areenough toprove that the correlation isdifferent from 0 and therefore the two variables arenotindependent.

17 This factobtainsby applying the CauchySchwarz inequality, | | || || ||||, to the vectors and with and .

AdvancedStatistics

Page34

4.13 Spearman'srankcorrelationcoefficientPrerequisites:coupledrankeddataorcoupleddatafromcontinuousdistributions H0:ranksareuncorrelated Statistic:SpearmansrankcorrelationcoefficientStatisticdistribution:SpearmantableSPSS:AnalyzeCorrelateBivariate CharlesSpearman

(18631945)The Spearman's rank correlation coefficient is the non parametric versionof the Pearsons

correlationcoefficient.Takingthesamedataofthepreviousexample,

2 3 4 3 5 6 7 3 1 3 4 5 5 7 5 7 7 14 5 3 1 12

this time instead of taking the values, we assign ranks. It is important that ranks be assignedindependentlyfor and ,yetmaintainingthecoupledpositionofthedata:

2 4.5 7.5 4.5 9 10 11 4.5 1 4.5 7.5 4.5 4.5 8 4.5 8 8 11 4.5 2 1 10

TheSpearmansrankcorrelationcoefficient, , iscalculatedexactlyasPearsonscorrelationcoefficient:

Where,exactlyasforPearsonscorrelationcoefficient,

,

1

Thevalueof always fallsbetween 1 and 1,with 1 indicatingperfectpositivecorrelationand 1 for perfect negative correlation. The closer falls to 1 or 1 , the greater thecorrelationbetweentheranks.Conversely,thenearer isto 0,thelessthecorrelation.

ForSpearmans rank correlationwehave,however,additional information since the valuesused in the calculation must be integer numbers between 1 and . Therefore, throughmathematicalcalculations,wecanderive18 analternativeformulavalidonlywhentherearenottied

18 Starting from the consideration that 1 2 3 1 we can obtain asimplification

and

. Moreover, since

1 2 3 1

AdvancedStatistics

Page35

ranks: 1 6n 1

.

where ,thedifferencebetweentherankofthe thmeasurementinthefirstsetandtherankofthe thmeasurementinthesecondset.Wecanseethatifallranksareidentical,thatis, forevery ,then 1.Wemusttakecaretorememberthatthisformulaisvalidonlywhentherearenotiedranks.

Returningtoourexample,weseethat

2.5 0 0.5 0 1 2 0 0 1 3.5 2.5 31,

consequently, 1 6 31 11 11 1 0.859. The fact that is close to 1 indicatesthattherankingsgivenbythetwomagazinestendtoagree,buttheagreementisnotperfect.

If the sets of ranks are formed by values taken by independent realizations of randomvariables and , the Spearman's rank correlation coefficient may be used for testingwhetherthevalueof Corr, isdifferentfrom 0.Thestatisticisthecoefficientitself.Inthepreviousexample,with 11 andasignificancelevelof 5% wehaveacriticalvalueof 0.623,

andthereforewereject,meaningthattheranksarecorrelatedandthere isarelationbetweentheorderofthetwovariables.

Spearmans rank correlation coefficient can be used, as every other rank test, in all thesituationswhereeffectivemeasuresarenotavailableandonlyranksareprovided.Supposetennewcarmodelsareevaluatedbytwoconsumermagazinesandeachmagazineranksthebrakingsystemofthe cars from 1 (best) to 10 (worst).Wewant to determinewhether themagazines' ranks arerelated. If they are, we may conclude that these rankings contain useful information about thebreakingsystem.Otherwise, if therankingsgivenby the twomagazinesarenotrelated,weshouldnotregardtheserankingascontainingusefulinformationsincetheyarecontradictoryandwedonotknowwhichonetouse.Lettheranksgivenbythetwomagazinesbeasfollows:

Carmodel 1 2 3 4 5 6 7 8 9 10

we see that

. Finally, taking into account that

, we obtain

.Consequently,

1 .


0H0 probably true

0.623

0.623

0.859

AdvancedStatistics

Page36

Rankgivenbymagazine1 4 1 9 5 2 10 7 3 6 8Rankgivenbymagazine2 5 2 10 6 1 9 7 3 4 8

Inthiscasedataarealreadyrankedandthecoefficientcanbecalculateddirectly.4.14 Multinomialexperiment

Manybusinessanalysesconsistofenumeratingthenumberofoccurrencesofsomeevent.Forexample,wemaycountthenumberofconsumerswhochooseeachofthethreebrandsofcoffee,orthenumberofsalesmadebyeachoffiveautomobilesalespeopleduringamonth.Whenthere isasingle scale to classifydata,as inallexamplesabove,wehaveaonedimensional classification. Insome caseswemay collect the countdata characterizing several factors.Forexample,wemaybeinterested in investigatingwhether the colorof automobilepurchased is related to the sexof thebuyer. In this casewe are dealingwith a two dimensional classification. The corresponding dataconstituteacontingencytable.Countdataaretraditionallyanalyzedusingtables. 4.14.1 OnedimensionalclassificationPrerequisites: 5 forall H0: forall or,equivalently, forall Statistic:tableschisquare

,

Statisticdistribution:chisquarewith 1 degreesoffreedomSPSS:AnalyzeNonparametrictestsOneSample

Thepropertiesoftheonedimensionalmultinomialexperimentareasfollows: theexperimentconsistsof identicaltrials; thetrialsareindependent; thereare possibleoutcomestoeachtrial; theprobabilitiesofthe outcomes,denotedby , , ..., ,remainthesame fromtrialto

trial,where 1 (therefore there is no other possible outcome outside theonesweareconsidering);

therandomvariablesofinterestarethecounts , ,..., ineachofthe cells. Forexample,supposea largesupermarketchainconductsaconsumerpreferencesurveyby

recording thebrandofbreadpurchasedbycustomers in itsstores.Assume thechaincarries threebrands of bread, A, B and C. The brand preferences of a random sample of 150 consumers areobserved,and the resultingcountdataareas follows:A: 61,B: 53,C: 36.Do thesedata indicatethatapreferenceexistsforanyofthesebrands?

Our consumer preference survey satisfies the properties of amultinomial experiment. Theexperiment consists in randomly sampling 150 buyers from a large population of consumerscontaining anunknownproportion whopreferbrandA, aproportion whopreferbrandB,andaproportion whopreferthestorebrand,C.Approachingabuyerconcerninghispreference,weperformasingletrialthatcanresultinoneofthreeoutcomes:theconsumerprefersbrandA,BorC.Probabilitiesof theseoutcomesare , ,and , respectively.Thebuyer'spreferenceofanysingleconsumerinthesampledoesnotaffectthepreferenceofanother.Consequently,thetrialsare

AdvancedStatistics

Page37

independent. The recorded data are the numbers of buyers in each of the consumer preferencecategories. Thus, the consumer preference survey satisfies the five properties of a multinomialexperiment.

Note that we may talk about the proportions , , as probabilities because in apopulationconsisting totallyof agents, preferbrandA, opt forbrandB,and forbrand C. Consequently, the probability to choose randomly a customerwho buys A, B, Cwill becorrespondingly / , / and / .Thatiswhywemaytalkabout asaproportionaswellasaboutaprobability.The threeprobabilities , , areunknownandwewanttousethesurveydatatomakeinferencesabouttheirsize.

Thegeneralformforatestofahypothesisconcerningmultinomialprobabilitiesisasfollows: H0: , , ..., ,where , , ..., representthehypothesizedvaluesof

themultinomial probabilities ( 1/3 in the above examplewith three types ofbread)

H1:at leastoneofthemultinomialprobabilitiesdoesnotequal itshypothesizedvalue, inotherwords,thereisan suchthatthecorrespondingactualprobability doesnotcoincidewithitshypothesizedvalue , .

Webuildatableofobservedcountsandatableofpredictedcounts A B C A B C61 53 36 50 50 50observedcounts predictedcountsunderH0hypothesis

Theteststatisticisthetableschisquare,ameasurecalculatedas

,where arecalledobservedcounts, . . . isthetotalsamplesize.Thisstatisticisdistributed as a chi square distributionwith 1 degrees of freedom.Observing the chi squarestatistic,itisevidentthatwhentheobservednumbersareverydifferentfromthepredictedcounts, , thevalueof thestatistic isvery large,whilewhen theobservednumberscoincideswith thepredictedones thestatistic iszero.Therefore, rejection region isonlyon the right.This testworksonlyifthepredictedcountsareall 5,whileitisnotimportantthattheobservedonesbeatleast5.

Inourparticularexample, // /

/ /

/ 6.52. Sincehere 3 , we are dealing with a chi square distribution with 2 degrees of freedom. Statisticssignificance is19 3.84% and therefore we reject, meaning that consumers preferences are not

19 Significancecanbecalculatedindifferentways:(1)usingEnglishMicrosoftExcelfunctionCHIDIST(6.52;2)whichgivesustheprobabilityoftherighttailofchisquaredistribution;(2)lookingintochisquaretableswhichusuallyprovidecriticalvaluesfordifferentsignificance levels;(3)usingEnglishMicrosoftExcelfunctionCHIINV(5%;2)whichgivesusthecriticalvaluecorrespondingto5%significancelevel;(4)testcanbeperformedalsousingEnglishMicrosoftExcelfunctionCHITESTwhich,giventheobservedtableandthepredictedtable,givesusthevalueofchisquarestatisticandthenusingCHIDISTsignificancecanbefound.

AdvancedStatistics

Page38

uniformandthatthere isat leastonetypeofbreadthathasaprobabilitydifferentfrom 1/3.Ifwewanttousethecriticalregionsmethod,critical

Advanced Statistics

Documents