-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
1/22
Regression analysis generates an equation to describe the
statistical relationship betweenone or more predictor variables and
the response variable. After you use Minitab StatisticalSoftware
http://www.minitab.com/enus/products/minitab/ to fit a regression
model, andverify the fit by checking the residual plots
http://blog.minitab.com/blog/adventuresinstatistics/whyyouneedtocheckyourresidualplotsforregressionanalysis,
youll want tointerpret the results. In this post, Ill show you how
to interpret the pvalues and coefficientsthat appear in the output
for linear regression analysis.
How Do I Interpret the PValues in Linear RegressionAnalysis?The
pvalue for each term tests the null hypothesis that the coefficient
is equal to zero noeffect. A low pvalue < 0.05 indicates that
you can reject the null hypothesis. In otherwords, a predictor that
has a low pvalue is likely to be a meaningful addition to your
modelbecause changes in the predictor's value are related to
changes in the response variable.
Conversely, a larger insignificant pvalue suggests that changes
in the predictor are notassociated with changes in the
response.
In the output below, we can see that the predictor variables of
South and North aresignificant because both of their pvalues are
0.000. However, the pvalue for East 0.092 isgreater than the common
alpha level of 0.05, which indicates that it is not
statisticallysignificant.
Typically, you use the coefficient pvalues to determine which
terms to keep in theregression model. In the model above, we should
consider removing East.
59
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
How to Interpret Regression Analysis Results: Pvaluesand
CoefficientsJim Frost
http://blog.minitab.com/blog/adventuresinstatistics 1 July,
2013.
98 778 153
Master StatisticsAnytime,AnywhereQuality Trainer teachesyou how
to analyzeyour data anytime youare online.
Take the Tour! http://www.minitab.com/products/quality
trainer/?WT.ac=BlogQT
The Minitab Blog http://blog.minitab.com
Data Analysis http://blog.minitab.com/blog/dataanalysis2
Quality Improvement
http://blog.minitab.com/blog/qualityimprovement2
Project Tools http://blog.minitab.com/blog/projecttools2
Minitab.com http://www.minitab.com
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
2/22
How Do I Interpret the Regression Coefficients for
LinearRelationships?Regression coefficients represent the mean
change in the response variable for one unit ofchange in the
predictor variable while holding other predictors in the model
constant. Thisstatistical control
http://blog.minitab.com/blog/adventuresinstatistics/atributetoregressionanalysis
that regression provides is important because it isolates the role
of onevariable from all of the others in the model.
The key to understanding the coefficients is to think of them as
slopes, and theyre oftencalled slope coefficients. Ill illustrate
this in the fitted line plot below, where Ill use apersons height
to model their weight. First, Minitabs session window output:
The fitted line plot shows the same regression results
graphically.
The equation shows that the coefficient for height in meters is
106.5 kilograms. Thecoefficient indicates that for every additional
meter in height you can expect weight toincrease by an average of
106.5 kilograms.
The blue fitted line graphically shows the same information. If
you move left or right alongthe xaxis by an amount that represents
a one meter change in height, the fitted line rises orfalls by
106.5 kilograms. However, these heights are from middleschool aged
girls andrange from 1.3 m to 1.7 m. The relationship is only valid
within this data range, so we wouldnot actually shift up or down
the line by a full meter in this case.
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
3/22
If the fitted line was flat a slope coefficient of zero, the
expected value for weight wouldnot change no matter how far up and
down the line you go. So, a low pvalue suggests thatthe slope is
not zero, which in turn suggests that changes in the predictor
variable areassociated with changes in the response variable.
I used a fitted line plot because it really brings the math to
life. However, fitted line plots canonly display the results from
simple regression, which is one predictor variable and theresponse.
The concepts hold true for multiple linear regression, but I would
need an extraspatial dimension for each additional predictor to
plot the results. That's hard to show withtoday's technology!
How Do I Interpret the Regression Coefficients forCurvilinear
Relationships and Interaction Terms?In the above example, height is
a linear effect; the slope is constant, which indicates that
theeffect is also constant along the entire fitted line. However,
if your model requirespolynomial or interaction terms, the
interpretation is a bit less intuitive.
As a refresher, polynomial terms model curvature in the
datahttp://blog.minitab.com/blog/adventuresinstatistics/curvefittingwithlinearandnonlinearregression,
while interaction terms indicate that the effect of one
predictordepends on the value of another predictor.
The next example uses a data set that requires a quadratic
squared term to model thecurvature. In the output below, we see
that the pvalues for both the linear and quadraticterms are
significant.
The residual plots not shown indicate a good fit, so we can
proceed with the interpretation.But, how do we interpret these
coefficients? It really helps to graph it in a fitted line
plot.
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
4/22
You can see how the relationship between the machine setting and
energy consumptionvaries depending on where you start on the fitted
line. For example, if you start at a machinesetting of 12 and
increase the setting by 1, youd expect energy consumption to
decrease.However, if you start at 25, an increase of 1 should
increase energy consumption. And ifyoure around 20, energy
consumption shouldnt change much at all.
A significant polynomial term can make the interpretation less
intuitive because the effect ofchanging the predictor varies
depending on the value of that predictor. Similarly, asignificant
interaction term indicates that the effect of the predictor varies
depending on thevalue of a different predictor.
Take extra care when you interpret a regression model that
contains these types of terms.You cant just look at the main effect
linear term and understand what is happening!Unfortunately, if you
are performing multiple regression analysis, you won't be able to
use afitted line plot to graphically interpret the results. This is
where subject area knowledge isextra valuable!
Particularly attentive readers may have noticed that I didnt
tell you how to interpret theconstant
http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowtointerprettheconstantyintercept.
Ill cover that in my next post!
Be sure to:
Check your residual plots so you can trust the
resultshttp://blog.minitab.com/blog/adventuresinstatistics/whyyouneedtocheckyourresidualplotsforregressionanalysisAssess
the goodnessoffit and Rsquared
http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowdoiinterpretrsquaredandassessthegoodnessoffit
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
5/22
If you're learning about regression, read my regression
tutorialhttp://blog.minitab.com/blog/adventuresinstatistics/regressionanalysistutorialandexamples!
CommentsName: Lovemore Friday, January 24, 2014
That's sounds great but for me I am finding difficult how do I
instigate a six sigma project in a medical laboratory using soof
the Minitab tools
Name: Henry Mwangi Thursday, February 20, 2014
Thank you for an elaborate explanation on the interpreting reg
coefficients and mostly the pvalue.
Name: Deeps Dee Thursday, March 27, 2014
It has been useful for my thesis whereby I've been struggling to
interpret my results :sThank you for the explanation.
Name: taiwo lucas Wednesday, April 2, 2014
Thank you very much the explanation really help me in my
thesis.God bless you.
Name: O.Jobi Saturday, May 10, 2014
This is very helpful information for my dissertation page
4&5.
Name: yashika Tuesday, May 13, 2014
really i was confused and you clear this concept of regression
coefficient. very good explanation.can you do this with ttest
explanation also?
You Might Also Like:Regression Analysis: How to Interpret the
Constant Y Intercept
http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowtointerprettheconstantyinterceptRegression
Analysis: How Do I Interpret Rsquared and Assess the
GoodnessofFit?http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysishowdoiinterpretrsquaredandassessthegoodnessoffitWhy
Are There No P Values for the Variables in Nonlinear Regression?
http://blog.minitab.com/blog/adventuresinstatistics/whyaretherenopvaluesforthevariablesinnonlinearregressionRegression
Analysis Tutorial and Examples
http://blog.minitab.com/blog/adventuresinstatistics/regressionanalysistutorialandexamples
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
6/22
Name: omid Saturday, June 7, 2014
hi dear,I am doing a censored least absolute deviation model
using STATA, when I got output there was a column indicated
with"Bias" , does it mean Pvalue ?
Name: Jim Frost Monday, June 9, 2014
Hi Omid,
Thanks for your question. I can't really offer guidance about
using Stata. However, bias and Pvalue are not synonymous, sothat's
probably not what the output means.
I suspect it has to do with the censoring in your data.
Regression with censored data can cause biased estimates becauseyou
may be less likely to observe the response value for certain
classes of observations. In other words, the model that fitsthe
observed responses may not provide an unbiased fit for the censored
observations.
Minitab can perform regression with censored data and can assume
different distributions. In Minitab: Stat >Reliability/Survival
> Regression with Life Data.
You can try a free 30 day trial of Minitab 17
here:http://it.minitab.com/enus/products/minitab/freetrial.aspx
http://it.minitab.com/enus/products/minitab/freetrial.aspx
Thanks for writing!Jim
Name: Mrv Yrd Wednesday, August 27, 2014
Hi Jim,First of all Thank you for the useful information! I am
little confused about p value and significance for regression. If
our pvalue is 0.02 for SLR can we say that regression analysis is
statistically significant at 95% confidence level ? Or should we
sayit is significant at 98%? My second question is that if we are
not given the p value for the variable and the constant for SLR,
but the regression pvalue is smaller than 0.05 , can we conclude
the factor significantly affects the response ? Thank you in
advance.
Name: Jim Frost Thursday, August 28, 2014
Hi,
Typically you choose the significance level before the study,
and that's the level you cite after the analysis. For example,
youcan state that the SLR is statistically significant at the the
0.05 level. Or for multiple regression, identify the variables
thatare significant at that level e.g. 0.05. You typically don't
change the significance level to match your pvalues.
However, I'd also report the exact pvalues as well. The exact
pvalue is important in terms of understanding the liklihoodthat
your test drew the correct conclusions. I cover that in this
post:http://blog.minitab.com/blog/adventuresinstatistics/fiveguidelinesforusingpvalueshttp://blog.minitab.com/blog/adventuresinstatistics/fiveguidelinesforusingpvalues
For your second question. Yes, in a simple linear regression
model Y = a + bX, the regression pvalue in the ANOVA is fora test
of the hypothesis that the linear coefficient is zero.
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
7/22
Thanks for reading!Jim
49Comments TheMinitabBlog Login1
Share SortbyOldest
Jointhediscussion
Reply
Joel 6monthsagoHello,
Ifittedthemodely=a+bX1+cX2+dX1.X2+e(X1)^2+f(X2)^2onadatasetbutIhavesomeproblemsininterpretingthepvaluesofthecoefficients.IfIusenormalizedvaluesforX1andX2(smallestvalue:1,largestvalue:+1)andIperformaregressionIgetdifferentpvaluesforthecoefficientsa,bandc(notd,eandf)comparedtottherealvalues.Infactformydatasetp0.05fortherealvalues.SoIguessnormalizationistobedonealwaystoanalyzedata?
Thanksinadvance.
Jol1
Reply
inez
6monthsagoInmylinearregressionresults,whatdothetvaluesmean?caniputthemintableofresults?
Reply
JimFrostAtMinitab 6monthsagoMod
>inezHiInez!Thanksforwritingwiththeexcellentquestion!
Thetvalueisastatisticthatmeasurestheratiobetweenthecoefficientanditsstandarderror.
Minitabusesthetvaluetocalculatethepvalue,whichyouusetomakeadecisionaboutthestatisticalsignificanceofthetermsandmodel.
Asufficientlylargeratioindicatesthatthecoefficientestimateisbothlargeandpreciseenoughtobesignificantlydifferentfromzero.Conversely,asmallratioindicatesthatthecoefficientestimateistoosmallortooimprecisetobecertainthatthetermhasaneffectontheresponse.
Youcanusethetvaluetodeterminewhethertorejectthenullhypothesis.However,thepvalueisusedmoreoftenbecauseitiseasiertointerpret.
Unlessyouhaveaspecialneedtoincludeit,Iwouldnotincludeitinyourresults.
Jim
Cain
5monthsagoHowcanItellthelevelofsignificancefromanoutput?IhaveanexamusingminitabandI'mnotsure
Recommend
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
8/22
Reply
Reply
JimFrostAtMinitab 5monthsagoMod
>CainHi,thatsoundslikeatrickquestiontome.Thesignificancelevel(alpha)issomethingthatyoushouldchoosebeforeyouperformyourstudy.Afteryouperformtheanalysis,youcomparethepvaluesintheoutputtoyoursignificancelevel.
Jim
Reply
WDC123
5monthsagoHiJim,IfIreducethemodelbytakingouttermswithpvalueslessthank0.05andthennoticethatRsquaredhasalsoreducedhowdoIexplain.ShouldIconsiderleavinginsometerms?
Reply
JimFrostAtMinitab 4monthsagoMod
>WDC123Hi,typicallyyouconsiderremovingpredictorsfromthemodelifthepvalueisgreaterthanyoursignificancelevel.I'llassumethatiswhatyoumeanttotype!:)
It'sfairlytypicalfortheRsquaredtodeclineasyouremovepredictors,evenwhenthosepredictorsarenotsignificant.
Hereareacoupleofsuggestions:*UseadjustedRsquaredtocomparemodelswithdifferentnumbersofterms.*Don'tchoosethemodelbasedsolelyonthehighestRsquaredbecausethatcanleadyouastray.*Useyourexpertise,theory,andcommonsenseratherthanrelyingsolelyonsimplisticmodelselectionrules.
Foryourcase,don'tfeellikeyoushouldincludethoseinsignificantpredictorsjusttogetthehigherRsquared.However,youcanconsiderincludingthemiftheorysuggeststhattheybelonginthemodel.Ingeneral,youshouldalreadyhaveanideaofwhattheimportantvariablesarealongwiththeirrelationships,coefficientsigns,andeffectmagnitudesbasedonpreviousresearch.
There'snotalwaysaclearansweronwhichpredictorsyoushouldincludeinyourmodel.Useboththestatisticaloutputandtheoretical/subjectareaconsiderationstohelpyoudecide.
Thanksforwritingwiththegreatquestion!Selectingthecorrectmodelhasalwaysbeenaveryinterestingsubjectforme!
Jim
Ronja
4monthsagoHello,myquestionisquitesimilartotheothers:inordertodevelopaforecastIwanttousemultipleregression.Itriedvariousindependentvariablesthatwouldallmakesense(meaningtheyallmayhaveanimpactontheforecast)togainthebestsuitedequationfortheforecast,butIfinditdifficulttochoosetherightsetofvariables.Withtheonesetofindependentvariables,mypvaluesarehigherthan0,05(theyare0,12)howevertheRsquaredishighestwith0,9904.Takingouttermswithpvalueshigherthan0.05won'tworksincethentherewon'tbeanyleft.Withtheotherset,myRsquaredisjust0.8473howeverthepvaluesarelessthan0,05.Howdoyouselecttherightset?DoyouweighthepvaluehigherortheRsquaredoristhereanothertermIshouldconsiderformy
Share
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
9/22
Reply
DoyouweighthepvaluehigherortheRsquaredoristhereanothertermIshouldconsiderformyselection?
Thankyouverymuchinadvance!!!
Ronja
Reply
JimFrostAtMinitab 4monthsago
seemore
Mod >Ronja
HiRonja,
Selectingthecorrectmodelcanbeaverydifficultprocessinsomecases.Readmyresponsetothecommentdirectlyaboveyours(toWDC123)becauseitappliestoyourcaseaswell.
Specifically,don'tfeellikeyoumustgetthehigherRsquaredbecauseit'spossibletohaveanRsquaredthatistoohighandcauseproblems.YourRsquaredof0.99maybetoohighandcouldindicatethatyou'reoverfittingthemodel.Also,youshoulduseadjustedRsquaredtocomparemodelswithdifferentnumbersofpredictorsratherthanRsquared.
IsuggestthatyoureadmyblogpostaboutadjustedRsquared,whichcoversalloftheabovepoints.
AsforpvaluesversusadjustedRsquaredvalues,researchhasshownthatusingpvaluesinastepwisemannergenerallyworksbetterthanusingadjustedRsquaredtopickthecorrectmodel.However,usinganysimplemodelselectionprocedurelikethatgenerallydoesnotpickthecorrectmodel.I'vewrittenanotherpostaboutthisissuewhereIcomparestepwisetobestsubsetsregression.
Theimplicationsofthesefindingsareprofoundevenifyou'renotusingeitheroftheseautomatedmethods.Thefindingsshowthatchoosingthecorrectmodelisasmuchascienceasitisanart.The
Reply
Fiachra
4monthsagoHi,AfterrunningmyregressionIendedupwithpvalueslike6.9345E05.WhatdoesthisEmeanandhowdoIworkoutthePvaluethanks.
Reply
JimFrostAtMinitab 4monthsagoMod
>FiachraThatiscalledscientificnotationandisusedtowriteverylargeandverysmallnumbers.ItworksbyshiftingthedecimalpointleftorrightbythenumberofplacesindicatedaftertheE,whichstandsforexponent.
The05indicatesthatyouneedtotakethe6.9345andshiftthedecimalpointtotheleftby5places.So,yourpvalueis0.000069345.That'saverylowvaluesoitisverysignificant!
Jim
Fiachra
4monthsago>JimFrostAtMinitabThanksamillion!myheadwaswreckedthinkingitwassomethingmuchmorecomplex.IdohaveoneotherquestionhoweverinarecentmcqIwasgivenaregressionoutputbasedonsalary=b1+b2(Rank).(Rankbeingthequalityoftheindividualsuniversity,thebestwasawardedarankof1andtheworstarankof142).Thecoefficientstheregressionproducedfortheinterceptandrankwere56063and206.731respectively.Bothhadveryverylowp
Share
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
10/22
Reply
fortheinterceptandrankwere56063and206.731respectively.Bothhadveryverylowpvaluessotheyweresignificant.Thequestionwaswhatisthetrueeffectofaoneplaceincreaseinuniversityrankingsonsalaries.TheanswerIgavewas206.731butthecorrectansweristhatitcannotbedeterminedfromthesefigures.(Figuresbeingapictureofaregressionoutputinexcel).Whyisthisthecorrectanswer?Ithoughtthiswouldhavebeenexactlywhatthecoefficientintheregressionindicates.Thanks.
1
Reply
JimFrostAtMinitab 4monthsagoMod
>FiachraIfthequestionaskedyouspecifically,whatwasthe"true"effect,youhavetorememberthatregression,andotherstatisticaltechniques,canonlyprovideanestimateofthetrueeffect.It'sgenerallyimpossibletoeverknowthetrueeffectitselfbecauseyou'reworkingwithasampleofthepopulationratherthantheentirepopulation.
Instead,inferentialstatisticscanonlyprovideanestimateofthetrueeffectandgiveyouaconfidenceintervalforarangeofvaluesthatislikelytocontainthetrueeffect.Inregressionanalysis,thecoefficientsaretheparameterestimates.
Reply
sewnsew
4monthsagoihavearegressionmodelhowdoIcalculatethechangeinpwhenItakeoutvariablesoraddvariablebackintoamodeltoseewhichhasthemostpredictivevalue?InthedataIhave,Ihaveachangeinp,butinSPSS,Idon'tseeanythingthatshowsorrelatestothechangeinp,sowhenIrerunthedata,Idon'tknowwhattolookfororwhattointerpretasachangeinp.Thanks.
Reply
JimFrostAtMinitab 4monthsagoMod
>sewnsewHi,Ican'tspeaktowhatyouseeinothersoftwarepackages.Also,I'mnotsurewhichpyouarereferringto.
Youmaywanttolookattheadjustedsumsofsquaresintheoutput.Thisindicatestheuniqueportionofthetotalsumsofsquaresthateachtermexplainsregardlessoftheordertheywereenteredinthemodel.Ifyouwanttofindouthowmuchvariationeachpredictorvariableaccountsforinamodel,thisiswhatyouneed.
Jim
Reply
Scott
4monthsagoIsthereanywaytoset/holdaparticularregressionequationcoefficientataparticularvalue,andthenperformtheregressionanalysis?
Inmyexample,Iamanalyzingpsioutvaluebasedonanumberofinputs,IwanttoholdPsiIncoefficientat1,andlettheothervariablesbeapartoftheregression.Hopethismakessense,:/
Thanks!
JimFrostAtMinitab 4monthsagoMod >ScottHiScott,
Share
Share
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
11/22
Reply
HiScott,
That'saninterestingquestion.
Typically,you'refittingamodellikethis:Y=B0+BX1+BX2+BX3...whereyouestimatetheBsfromthedata.
Youwanttofitthis:Y=B0+1X1+BX2+BX3...wherethefirstcoefficientis1.
WhatyoucantrydoingismovingthetermwiththefixedcoefficientovertotheYsideoftheequation:Y1X1=B0+BX2+BX3
You'dhavetocreateanewcolumnofresponsedatawhereyoutaketheoriginalmeasureandsubtractoutthe1X1.Inyourcase,you'dtaketheoutputPSIandsubtracttheinputPSIforeachobservationandusethenewlycalculatedvaluesastheresponse.Then,includetherestofthepredictorsinthemodel.
You'dessentiallybelookingathowthepredictorsarerelatedtothechangeinPSIratherthantheabsolutePSI,whichsoundspromisingifIunderstandyourscenariocorrectly.
Theestimatesfortheotherpredictorswouldbethevaluesifforcedthefirstpredictortoequal1.You'dhavetobecarefulhowyouinterpretthemodelfitvalues.Forexample,Rsquaredindicateshowmuchvariationyouaccountforwiththenewresponsevariable.
Jim
Reply
sewnsew
4monthsagoInmyhomogeneoussubsetstheNisdifferentthantheNthatIgotwhenIranfrequencies.Why?Isthisnormal?
Reply
SharonEdgeWilkie
4monthsagoThisisaPostHocquestion.WhyaretheNinmyhomogeneoussubsetsnotthesameastheNinmyfrequencycharts?
Reply
PatrickKajubili
4monthsagoHi,Iamstilljuniorinthefield.iwanttoknowifihaveF714andSig761inmyANOVAtablewhatdoesthismean?Havingsiglikedoesshowmodelfit?
JimFrostAtMinitab 4monthsagoMod
>PatrickKajubiliHiPatrick,theFstatisticisatestoftheoverallsignificanceoftheregressionmodel.WhileRsquaredandadjustedRsquaredtellyoutheoveralldegreeofthefitforaregressionmodel,theydon'tprovideaformalhypothesistestfortheoverallfit.
That'swheretheFtestanditsassociatedpvaluecomesin.
ThenullhypothesisfortheFtestisthatallofthecoefficientsintheregressionmodelequalzero.Ifallthecoefficientsequalzero,thisisequivalenttosayingthatthefittedvaluessimplyequalthemeanof
Share
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
12/22
Reply
theresponsevariable.Inotherwords,yourmodelpredictstheresponsenobetterthanusingtheresponsemean.
Thealternativehypothesisisthattheydon'tallequalzero.Or,thatyourmodeldoesprovidebetterpredictionsthanjustusingthemean.
Alowpvaluemeansthatyoucanrejectthenullandconcludethatyourmodelisbetterthanjustusingthemeanandthatatleastonecoefficientdoesn'tequalzero.
You'llstillneedtochecktheresidualplotsbecausethistestwon'ttellyouwhetherthemodelprovidesanadequate,unbiasedfit.InthebulletsneartheendofthispostIprovidealinktoablogpostIwroteaboutcheckingtheresidualplots.
Thanksforwriting!Jim
Reply
Marija 3monthsagoHello,Ineedyurhelpaboutmyexamquestion:
(i)EstimatethefollowingregressionsPRICE=b1+allindependentvariables+utLnPRICE=b1+allindependentvariables+utAccordingtotherelevantcriteria,judgewhichoneisbetter.Continueworkingwiththebetterfromthetwo.Fullyinterpret(statisticalandeconomicsignificance)theresultsofhedonichousepriceestimation.Myquestion:Whicharethecriteriatodecidewhichisabetterregression?
Ihavecalcualtedthembothandherearetheresults(valuesonlyfromthevariableswithsig.MarijaHiMarija,
InadditiontothefactthatIreallyshouldnotansweryourexamquestionforyou,Ireallycan'tanswerthequestionwiththeinformationthatyouprovided.Thereisinsufficientinformationtobeabletochoose.But,Icangiveyousomegeneralguidelinesonhowtochoose.
Youshouldchecktheresidualplotsforbothmodels.Iftheplotslookgoodforonemodelbutnottheother,thatwillhelpyouchoose.
Youshouldalsolookatthecoefficientsforthepredictorsanddeterminewhethertheymatchtheory.Forexample,ifonemodelsuggeststhatagoodcharacteristiclowerstheprice(negativecoefficient),youshouldseriouslyquestionthatmodel.
Thosearethetypesofthingsyouneedtoassesstodeterminewhichmodelisbetter.Irecentlywroteablogpostabouthowtochoosethebestregressionmodel.Ithinkthatwillhavealotofhelpful
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
13/22
Reply
wroteablogpostabouthowtochoosethebestregressionmodel.Ithinkthatwillhavealotofhelpfulinformationforyou!
Goodluckwithyourtest!Jim
Reply
wuyr
3monthsagoHelloJim,Thanksalotforyourposting.Itisveryhelpful.IhaveanofftopicMinitabquestion,andhopingthatyoucouldhelpmeout.DoesminitabhasafunctionlikeVlookupinexcel?Thanksalot.Yan
Reply
JimFrostAtMinitab 3monthsagoMod
>wuyrHi,thankyouforthenicecomment!
Unfortunately,Minitabdoesn'thaveanexactlyequivalentfunction.However,inMinitab,youcanuseControlFtousetheFindinDataWindowfunction.Thiswillsearchwithinacolumnforaspecificvalue,eitherexactmatchornot.Whenitfindsamatchinacell,youcanlookattheassociatedinformationinthethatrowasawaytomimicthefunctionalityofVLOOKUP.
Jim
Reply
JackWotton 3monthsagoHijim,
I'mabletoexplainmyresultsthroughthepvalue,s=,rsq,andthegraphs.butiamunsureonothervaluesthathaveshownupe.g.,DF,SS,MF,F,(howtointerprettheresidualerrortomyresults?whatdoesDF20,SS235.57MS11.78allmean)ithinkthismostlyrelatestotheanalysisofvarience.hopeyourabletohelpasihaveadissertationhandinnextmonth)cheers
Jack
JimFrostAtMinitab 3monthsagoMod >JackWottonHiJack,
Alotofthesestatisticsarethe"behindthescenes"typeofnumbersthatMinitabneedstocalculateinordertocomputethemorecommonstatisticsthatpeopleneed,likethepvalues,Rsquared,adjustedRsquared,andS.Unlessyouhaveaspecialneed,youoftendon'tneedthestatisticsthatyoulist.
I'llrunthroughthemingeneralforyou.Ifyouneedmoredetailedinformationabouthowthey'recalculated,youcanalwayslookattheMethodsandFormulaHelpinMinitab:Help>MethodsandFormulas.TheMinitabGlossary(Help>Glossary)alsohasdefinitionsoftheseterms.
DF:Thedegreesoffreedom(DF)describetheamountofinformationyourdataprovidethatyoucan"spend"toestimatethevaluesofunknownpopulationparameters,andcalculatethevariabilityoftheseestimates.Degreesoffreedomareaffectedbythesamplesizeandthenumberofparametersinyourmodel.Increasingyoursamplesizeprovidesmoreinformationaboutthepopulation,and
Share
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
14/22
Reply
seemore
inyourmodel.Increasingyoursamplesizeprovidesmoreinformationaboutthepopulation,andconsequentlyincreasesthedegreesoffreedompresentinyourdata.Addingparameterstoyourmodel(byincreasingthenumberoftermsinaregressionequation,forexample)"spends"informationfromyourdata,andlowersthedegreesoffreedomavailabletoestimatethevariabilityoftheparameterestimates.
Reply
JackWotton
3monthsago>JimFrostAtMinitabThankyousomuchforyourhelp:)
Reply
Fardeen
3monthsagoHiMrJim.Imhavinggreatproblemsindoingmydissertation.Idontknowhowtomakeuseofregression.Iwouldbegratefulifyoucouldhelpme.Isthereasitewhereitshowsclearlytouseregression?Thanks
Reply
JimFrostAtMinitab 3monthsagoMod >FardeenHiFardeen,
Irecommendthatyoureadmyregressiontutorialwithexamples.Ithinkthiswillansweralotofyourquestions.
Bestofluckwithyourdissertation!Jim
Reply
becbec
2monthsago>JimFrostAtMinitabHiJim,thankyousomuchfortheinformativediscussionshere.Iammakingmythesishowever,Iamfindingdifficultiesininterpretingmydata.Whatdoesthisresultmeanifmyconstanttvalueis7.114,pvalue=.000,LIFCAStvalue=10.228pvalue.000,LERIANStvalueis2.971pvalue.003,andEFCOStvalue,2.186andpvalue.029.iwouldappreciateyourhelp.thanks.
JimFrostAtMinitab 2monthsagoMod >becbecHi,
Withtheinformationyouprovide,Ican'tbesurethatyourmodelmakessensetheoreticallyorwhetherthemodelprovidesanadequate,unbiasedfittothedata.Onethingyoushoulddoisdefintelycheckyourresidualplots.
Assumingthemodelisgood,here'swhatyou'vegot.
Youhaveaconstanttermthatissignificantlydifferentfromzero.However,theconstanttermusuallyhasnomeaningfulinterpretation.There'salinktoablogpostIwroteaboutwhythisistrueneartheendofthisblogpost(beforethecommentssection).
Share
Share
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
15/22
Reply
Youhave3significantpredictors.Thissuggeststhatchangesineachpredictorarerelatedtochangesintheresponse.Forexample,aoneunitincreaseinLIFCASisrelatedtoanincreaseinthemeanresponsevalueequaltotheLIFCAScoefficient.SameforLERIANS.ForEFCOS,everyoneunitincreaseisrelatedtoadecreaseinthemeanresponse(youdidn'tincludethecoefficientsbutfromthetvalueIknowthattheEFCOScoefficientisnegative).
Typically,youdon'tneedtoworryaboutthetvaluesandinsteadfocusonthepvaluesandcoefficients.
Youmightwanttoreadmyblogpostaboutchoosingthebestregressionmodeltohelpyoubesurethatyoudohavethebestmodel!
Bestofluckwithyourthesis!Jim
Reply
dunmao 2monthsago
seemore
>JimFrostAtMinitab
HiJim,
Couldyoupleasegivemeadirectionforthefollowingquestion?
Myquestions:IamdoingridershipmodelingusingmultiplelinearregressionmethodinExcelsoftware.Mydependentvariableisboardings,threeindependentvariablesarepopulation,feederbusservices,andemploymentdata.Eventhoughtheconstantismeaninglessdiscussedfromyourdiscussgroup.Inmycase,thepvalueforYinterceptis0.6(greatthan5%),howevertheYinterceptcanminimizetheresidual(observeddatapredictedvalue).Seetheregressionresult:
RSquare=0.943573,
PvalueforYintercept=0.6,Pvaluesforthethreeindependentvariablesarelessthan5%
AccuracyValidationwithoutYintercept(ObservedPredicted):
Reply
JimFrostAtMinitab 2monthsagoMod
>dunmaoHi,Irepliedtoyourquestionintheotherpostwhereyousharedyourcomment.Youcanfindithere.
Theshortansweris,yes,youshouldalmostalwaysincludetheconstantregardlessofthepvalue!
Jim
dunmao 2monthsago>JimFrostAtMinitabHiJim,
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
16/22
Reply
HiJim,
Thankyousomuchforyourquickresponse!
Iwanttoincludetheconstanteventhoughthepvalueoftheconstantisgreatthan5%.Theconstantcanbeexplainedasanadjustedfactorinmypredictionmodeltominimizetheerror.
Youranswerconfirmsmytestresults.
Thanksagain,Hope
Reply
JimFrostAtMinitab 2monthsagoMod
>dunmaoHi,you'reverywelcome!
Justtoclarifyonepoint.Yougenerallyshouldincludetheconstantregardlessofthepvalue.Youdon'tneedajustificationtoincludetheconstant.Instead,youneedaverystrongjustificationtoevenconsidernotincludingtheconstant.
Infact,I'veneverpersonallyworkedwitharegressionmodelwhereIfeltjustifiedtonotincludetheconstant.Aregressionmodelwithouttheconstantisveryrarebecausethepotentialforintroducingbiasisveryhigh.
Jim
Reply
dunmao 2monthsago
seemore
>JimFrostAtMinitab
HiJim,
Icomeback.IhaveanotherpredictionmodelwithYinterceptpositive.Seethefollowings:
AccuracyValidationwithoutYintercept(ObservedPredicted):
Predictedmodel:DV_37pm=0.441*IV2+0.179*IV3Error=Observed(3559)predicted(3961)=402(overestimated1678)
AccuracyValidationwithYintercept
Predictedmodel:DV_37pm=0.441*IV2+0.179*IV3+0.714Error=Observed(3559)predicted(3971)=412(overestimated412)
JimFrostAtMinitab 2monthsagoMod >dunmaoHi,
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
17/22
Reply
Hi,
YoushouldalmostalwaysincludetheYinterceptinthemodel.Irecommendthatyoudoleaveitinthemodel.
ThisistrueregardlessofthePvalue.Ihopeyou'vereadmypostabouttheregressionconstant?Ishowthereasonswhyyoushouldalwaysincludeitinthemodel.
Ifyou'reboundanddeterminedtoconsiderremovingit,thereareimportantconsiderationsyoumustevaluatefirst.
1)Checkthestandarderroroftheregression.TheErrorisinyouroutputisnotthestandarderrorbecauseSisalwayspositive.Yourerrorreductionisnotsubstantialanywayonlyfrom412to402.Theminisculereductioninerrorsuggestsyoumightaswellleavetheconstantinthemodel.
2)Checkyourresidualplots.Inparticular,besurethattherearenononrandompatternsforeithermodel.Thisisespeciallyimportantinthemodelwithouttheconstantbecauseoftenremovingtheconstantintroducesabiasthatyou'llseeintheresidualplots.Ifyouremovetheconstantandyouseeapatterninresiduals,puttheconstantbackinyourmodel.
But,really,youshouldincludetheconstantevenwiththehighpvalue.It'snothurtinganythinganditislikelyhelpingreducebiasinyourmodel.
JIm
dunmao 2monthsago>JimFrostAtMinitabThankyouJim!
Iwanttolearnmore,soIcomparethetwocasesWiththeconstantinmypredictionmodelANDwithouttheconstantinmypredictionmodel.WITHtheconstantinmypredictionmodel:Standarderror:82Residualplot:73.08%ofprobabilityoutputofthesampledatafitsanormaldistribution.WITHOUTtheconstantinmypredictionmodel:Standarderror:78Residualplot:73.08%ofprobabilityoutputofthesampledatafitsanormaldistribution.Therearenononrandompatternsforeithermodel.
Frommyunderstanding,theconstantissmall,sothereisnopatternintheresidualsdistributions.
Lastquestion:iftheconstantisbig,itcausestheerrorreductionsubstantial,doIstillneedtokeeptheconstant?(sorry,Idon'thavetheregressionresults,butIwanttoknowifthecaseexists.)
Thankyou,Hope
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
18/22
Reply Hope
Reply
JimFrostAtMinitab amonthagoMod >dunmaoHi,
Givenwhatyousay,theredoesn'tseemtobeanynumericreasontonotremovetheconstant.
However,beforeyoudothat,askyourselfifit'stheoreticallyjustifiedthatifyousetallofthepredictorstozero,you'dexpecttheresponsetoequalzeroaswell.Preferably,youwouldalsohavemeasuredvaluesnear/atthisallzeroregiontoconfirmthattheregressionlinetrulygoesthroughtheorigin.
It'sonlywhentheconstantissmallthatyouhaveachance(smallchance)toremoveitfromthemodel.Ifitislarge,removingitfromthemodelwillalmostcertainlybiasyourmodel!Iwouldneverremovealargeconstant.
Jim
Reply
dunmao amonthago>JimFrostAtMinitabHiJim,
Thankyousomuchforyourexplanation!Icompletelyunderstandtheconstant(regardlessofpvalue)now.
NowIhaveanewregressionresult:
R=99.35%,
AdjustR=99.06%
DV=20+0.129*IV1+0.178*IV2+0.078*IV3
Errors=observed(4088)predicted(4052)=36
Averageerrors=5.75%
Questions:WhyistheRsobigat99.35%?Maybesomeonewouldaskmeaboutthequestion.However,thisistrueregressionresult.Howwouldyouexplaintheresult?
Thankyouagain,
Hope
JimFrostAtMinitab amonthagoMod >dunmaoHi,
Withoutknowingthespecificsofthemodelandthestudyarea,it'simpossibletosayforsure.IfIremembercorrectly,youaremodelingridershipovertime.Iftherearetrendsinthedatathataffectbothsidesoftheequation,thisisaproblemandcanoftenproduceinflatedRsquaredvalueslikethis.Youshouldplotthevariablestoseeif
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
19/22
Reply
produceinflatedRsquaredvalueslikethis.Youshouldplotthevariablestoseeiftheyarestationary(constantmeanandvarianceovertime)ornonstationary(upwardordownwardtrendornonconstancevariance).
Ifyouhavenonstationarydata,youmustmakeitstationarybydifferencingthedatasothateachdatapointisthechangeinvaluebetweenconsecutivepoints.Usingregressionanalysiswithtimeseriesdatainvolvesadditionalconsiderationslikethis.Unfortunately,Idon'thaveahandyreferencetoreferyoutoobutyoushouldperformsomeadditionalresearchtoensurethatyouendupwithavalidmodel.
Jim
Reply
dunmao amonthago
seemore
>JimFrostAtMinitab
HiJim,
Inoticedanewquestion:
AsItoldyouIhavedonethetestingasthefollows:=====================================================Whenletintercept=0,theregressionresult:
Rsquared=0.96
AdjustedRsquared=0.88
StandardError=78
Observations=14
ANOVA:dfRegression:2Residual:12
Reply
Em
2monthsagoHi,thankyouforyourextremelyhelpfulblogs!Iwaswondering,ifyoucanhelpmeoutwithmymultipleregressionanalysis.ForthePearsoncorrelation,Ifoundthatonlyoneofmypredictorsissignificant(p=0.037).However,Idon'tquiteunderstandwhyinthettestsection,noneofmyindependentvariablesmakeasignificantcontributiontothemodel.Howisitpossible?Icouldn'tfigureoutthelinkbetweenthetwo.Canyouexplainthis?Thanksinadvance!
JimFrostAtMinitab 2monthsagoMod >EmHi,
ThePearsoncorrelationpvaluesandregressionpvaluestestdifferentthingssotheanswersmaynotagree.Thecorrelationpvalueonlytestsonepairofvariablesatatimewithoutconsideringtheothervariables.Theregressionpvaluesfactorinalltheotherpredictorvariablesthatareincludedinthemodel.
Fromwhatyouwrite,itsoundsasthoughthecorrelationpairthatisissignificantisoneofthe
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
20/22
Reply
Fromwhatyouwrite,itsoundsasthoughthecorrelationpairthatisissignificantisoneofthepredictorsandtheresponsevariable.Tryaregressionmodelwithjustthatonepredictor.Itshouldbesignificantinaregressionmodelbyitself.Then,addintheotherpredictors.Ifthesignificancegoesaway,itindicatesthattheotherpredictor(s)areaccountingforsomeofthesamevarianceintheresponse.Bysplittingupthevariancethatisaccountedforbetweenthevariables,itmaybethatnonearesignificantwhenthereismorethanoneinthemodel.
Also,checkyourVIFsinthefullmodel.It'spossiblethatmulicollinearity(correlationbetweenthepredictors)issappingthesignificanceofthepredictors.Theproblemsassociatedwithmulticollinearitydonotoccuronlywhenthereisastrongcorrelationbetweenindividualpairsofpredictors.Theseproblemscanoccurwhenthereisamoderatecorrelationbetweenanumberofpredictors.ThismoderatecorrelationmaynotbesignificantwhenyoulookatthePearsoncorrelationbetweenpairsbutcanbedetectedwithVIFs.ReadmoreaboutthisinmypostaboutmulticollinearityandVIFs!
Ihopethishelpsandthanksforwriting!Jim
Reply
Sayeed
2monthsagoHeyJim,howdoyouinterpretanadjustedRSquareresult.Foreg,Ihadtofindthecorelationbetweenexchangerateandstockprice,ItgavemeananswersayingtheadjustedRSquaretobe0.3925.Isthereacorelationandifthereisthanhowdoyouwritethat?Thanksinadvance
Reply
JimFrostAtMinitab 2monthsagoMod >SayeedHiSayeed,
That'sagreatquestion!
I'vewrittenabouthowweoftenuseadjustedRsquaredtohelpincludethecorrectnumberofpredictorsinthemodel.
However,thereisaspecificinterpretationforadjustedRsquare.AdjustedRsquaredprovidesanunbiasedestimatedofthestrengthoftherelationshipbetweenthepredictorsandresponse.
RegularRsquaredisthestrengthofrelationshipinyoursamplebutitisabiasedestimateofthepopulationbecauseittendstobetoohigh.AdjustedRsquaredis"shrunken"soitisnotbiased.
Foryourresults,themodelaccountsforanestimated39.25%ofthevariabilityintheresponseinthepopulation.WhatevervaluetheregularRsquaredis,itonlyappliestoyoursample.
IwroteanentirepostaboutthisthatIrecommendyouread:Rsquaredshrinkage.
Thanksforwriting!Jim
Javaid 2monthsago>JimFrostAtMinitabIhaveaquestion:
Share
Share
Share
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
21/22
RegressionEquation
MR=0.00349+0.003154A+0.16467B+0.000595C
andthatgivesmeModelSummary
SRsqRsq(adj)Rsq(pred)
0.001568899.56%99.30%98.23%
AmIcorrectinassumingthatthevalueofRsqis0.9956?
Visit Us at Minitab.comBlog Map
http://blog.minitab.com/sitemap.html |
Legalhttp://www.minitab.com/legal/ | Privacy
Policyhttp://www.minitab.com/legal/#privacypolicy |
Trademarkshttp://www.minitab.com/legal/trademarks/
Copyright 2015 Minitab Inc. All rights Reserved.
Who We AreMinitab is the leading provider of software and
services for qualityimprovement and statistics education. More than
90% of Fortune 100companies use Minitab Statistical Software, our
flagship product, andmore students worldwide have used Minitab to
learn statistics than anyother package.
Minitab Inc. is a privately owned company headquartered in State
College,Pennsylvania, with subsidiaries in the United Kingdom,
France, andAustralia. Our global network of representatives serves
more than 40countries around the world.
AuthorsCarly
Barryhttp://blog.minitab.com/blog/realworldqualityimprovementPatrick
Runkelhttp://blog.minitab.com/blog/statisticsandqualitydataanalysisJoel
Smithhttp://blog.minitab.com/blog/funwithstatisticsKevin
Rudyhttp://blog.minitab.com/blog/thestatisticsgameJim
Frosthttp://blog.minitab.com/blog/adventuresinstatisticsGreg
Foxhttp://blog.minitab.com/blog/dataanalysisandqualityimprovementandstuffEric
Heckmanhttp://blog.minitab.com/blog/startingoutwithstatisticalsoftwareDawn
Kellerhttp://blog.minitab.com/blog/adventuresinsoftwaredevelopmentEston
Martzhttp://blog.minitab.com/blog/understanding
-
5/18/2015
HowtoInterpretRegressionAnalysisResults:PvaluesandCoefficients|Minitab
http://blog.minitab.com/blog/adventuresinstatistics/howtointerpretregressionanalysisresultspvaluesandcoefficients
22/22
statisticsKaren
Meldrumhttp://blog.minitab.com/blog/statisticstipsfromatechnicaltrainerBruno
Scibiliahttp://blog.minitab.com/blog/applyingstatisticsinqualityprojectsEduardo
Santiagohttp://blog.minitab.com/blog/understandingstatisticsanditsapplicationCody
Steelehttp://blog.minitab.com/blog/statisticsandqualityimprovement