THEIL INEQUAKITY COEFFICIENT.pdf

RESEARCH NOTES

AND COMMUNICATIONS

CONTENTS

Theil's Forecast Accuracy Coeflicient: A ClarificationSpatial IVIcasurcment of Retail Store Demand

Friedhelni Hliemel 444David B. MacKay 447

Theil's Forecast Accuracy Coefficient: A

Clarification

FRIEDHELM BLIEMEL*

INTRODUCTION

Theil's coefficient of inequality is one of the statisticalforecasting evaluators frequently cited in the literature[1-7]. However, there seems to be some confusionabout this coefficient which may stem from the fact thatTheil himself proposed two different formulae at differ-ent times under the same name. Both are called "coeffi-cient of inequality" and labeled with the symbol "U."In addition, there are two different interpretations possi-ble for the formula first proposed.

THEIL'S OWN PROPOSALS

In Economic Policy and Forecast, Theil proposes thefollowing formula as a measure of forecast accuracy:

* Friedhelm Bliemel is Assistant Professor of Business Ad-ministration, Queen's University, Kingston, Canada.

1/2

(1) U =

"where Ai are the actual observations and Pi are thecorresponding predictions" [6, p. 32]. This coefficientis denoted as UI in this article. In Applied EconomicForecasting, Theil suggests as a measure of forecastquality;

(2) U =

1 = 1

= Ull

"where {Au Pi) stands for a pair of predicted and ob-served changes'' [7, p. 28]. T'his coefficient is denotedhere as UII.

Journal of Marketing Researeh,Vol, X (November 1973), 444-6

THEIL'S FORECAST ACCURACY COEFFICIENT; A CLARIFICATION 445

The formal difference between UI and UII is the ab-sence or presence of the P-tcrm in the denominator. ThisP-term causes UI to be bounded between 0 and 1. Thereis no finite upper boundary for UII, where the P-term ismissing.

Theil states for UI:

"We have U = 0 in the case of equality: P, — A,for all /. This is clearly the case of perfect forecast. Wehave U = 1 (the "maximum inequality") if there iseither a negative proportionality or if one of the varia-bles is identically zero" [6, p. 32].

In comparison, the corresponding statement for UIIwould be;

"We have U — 0 in the case of equality as above:P, = A, for all /. This is again the case of perfect fore-cast. We have U — 1 when the prediction method isnaive no-change extrapolation or when it leads to thesame standard deviation of forecast error as thatmethod."

THE UI FORMULA

Interpretation Problems and Their Consequences

Theil's formulation that ''At are the actual observa-tions and Pi are the corresponding predictions" does notmake it clear whether absolute values (e.g., next yearwe will sell 20,000 trucks) or changes in absolute values(e.g., next year we will sell 1,000 more trucks than thisyear) are forecasted. In some secondary literature aboutTheil's coefficient one finds that same ambiguous formu-lation [2, 5], while other writers specify which interpre-tation they prefer to use [3, 4]. This ambiguous andmultiple sort of interpretation of a "standard" formula-tion should be avoided, especially in view of the factthat Theil himself uses UI only on change data.

Interpretation A and Its Implications

If one interprets Pi and A, as the absolute^ values forforecast and outcomes (interpretation A) and computesUI under this assumption, then the following implica-tions must be kept in mind:

I. UI will be a proxi-variable for the standard error ofthe forecast, where this error is expressed as a frac-tion of the sum of the standard values for the fore-cast series and the series of actual outcomes. Forexample, a value of .06 for Ul would mean thatthe standard error of the forecast is .06 times aslarge as the standard value of all forecasts plus thestandard value of all outcomes added together. Thisis indeed a complicated relationship. Its informationvalue seems doubtful when comparing forecastingmethods with each other, since the error generatedby a particular forecasting method is scaled down

' The term "absolute" is used in the meaning defined in theprevious paragraph.

by the variation in the predictions generated by thesame method.

2. UI is, of course, bounded by 0 and 1. Here, thelower boundary is the ideal ca^e of perfect forecast,while the upper boundary stands for a number ofsituations which are trivial or impossible for al!practical purposes of sales forecasting, namely: (a)^ , = 0 for all Ai. This would niean we try re-peatedly to forecast sales where there are none nowand were none in the past; (b) Pi = O for all P*. Thiswould mean we repeatedly forecast zero sales wheresales have been made both now and in the past; and(c) Ai and P, are negative proportional. This wouldmean that it is possible to have negative sales orforecasts of negative sales.

Thus the information obtained from UI under interpre-tation A is not very enlightening. A Ul-value computedfor a forecasting series would tell the reader little ornothing which could give him a feeling for the relativereliability of the applied forecasting method.

Interpretation B and Its Implications

If one means by Ai and Pi the observed changes andthe predicted changes and computes UI under this as-sumption, then the following implication arises: Thevalues for UI become inconclusive and thus their infor-mation content is limited. We can easily show this in ageneral way of reasoning as below.^ UI is also boundedby 0 and 1. The lower boundary is, as before, the caseof ideal forecast. The upper boundary is reached in thecase that all Pi are zero.^ Under interpretation B, thatcase would be equivalent to the most naive forecastingmodel, the no-change model, which says: There will beno change; the forecast for the next period is equal tothe actual sales of this period. While it is nice to havethis case of the simple no-change model as a referencepoint, it is also desirable that the value of UI indicateswhether a forecasting method gives results better orworse than the no-change model. But precisely that in-formation cannot be found from UL The coefficient as-sumes the upper boundary of "maximal inequality" whenusing the no-change model. Any other forecastingmethod, regardless whether it is better or worse thanthe no-change model, will yield lower UI values, loweror equal to unity. Thus, UI should not be used at all forranking alternative forecasting methods.^* ^

^When proposing UII, Theil himself expressed the draw-hacks of UI in a footnote by stating that the denominator of UIdepends on the forecasts and that it is therefore not true thatUI is uniquely determined by the mean square prediction error.

'Other cases, which need not be treated here, show the in-conclusiveness of UI.

' Hirsch and Lowell demonstrate the limitedness of UI by thefollowing example (paraphrased): "Suppose that the forecasterknows that /( — 0 and also knows the standard deviation 5:4, andnothing more. He might simply forecast no change, in whichcase be would have a mean square error of SA' and a UI — 1.Alternatively, he might draw his forecasts randomly from a dis-

446 JOURNAL OF MARKETING RESEARCH, NOVEMBER 1973

UII, THE COEFFICIENT WITHOUT PROBLEMS

The second version, namely Ull, which apparentlyhas not yet been adopted to a great extent by scholarlywriters, is unambiguous and easy to understand and tointerpret. UII reaches its lower boundary of UII = 0at perfect forecasts. It assumes the value of 1 when aforecasting method delivers forecasts with the samestandard error as the naive no-change extrapolation. Itincreases monotonically as the standard error forecast-ing improves over the no-change extrapolation. If UII islarger than 1, the forecasting method applied is to berejected because it cannot beat the most simple no-change extrapolation. Because of the superiority of theseproperties, it is advisable to use only UII (and not UI)

tribution with mean zero and a variance of S/. Although thesecond strategy is manifestly worse than the first (it yields amean square error of 2 SA'), it receives a better rating in termsof UI, namely UI = \/V2 rather than unity."

" One might argue against this recommendation by proposingthat good forecasting models will be better than Ihe naive no-change forecasting model, and thus UI will be appropriate forcomparing "better" models in the region where UI is small. Thisproposition implies that indeed one is sure from other indicatorsthat the forecasting models are better than the no-change model.In view of the state of art in forecasting, "better" forecasts donot seem warranted. Evidence to this account is given byZarnovitz [8]. He compared forecasts made by professionalteams to the no-change alternative by computing R,, the ratioof mean square error of the experts' forecasts to the mean squareerror from the no-change model. For sectoral economic fore-casts made during 1958-63, Zarnovitz found: "While no morethan one-eighth of them exceeds the ratio of 1,0, only aboutone-fourth are less than 0.6" [8, p. 40],

when assessing the reliability of sales forecasting meth-ods.

SUMMARY

It has been shown that one version of Theil's coeffi-cient of inequality has little or no value as an index toassess forecast accuracy, although the use of this versionis still suggested by many scholarly writers. In contrast,another and even simpler version of Theil's coefficientof inequality gives more meaningful information aboutthe accuracy of forecasting methods. It is suggested thatonly this latter version be applied.

REFERENCES

1. Bliemel, Friedhelm. "Forecasting Short-Terni Market Po-tential for a Capital Good on the Basis of EstablishmentData: A Simulation Study of Methods and Data Quality,"unpublished doctoral dissertation, Purdue University, 1972.

2. Bond, Richard O, and D. B, Montgomery. "FORAC MOD I:A Computer Program for Forecast Evaluation Statistics,"Marketing Science Institute Paper, September 1970,

3. Chisholm, Roger K. and G. R. Whitaker, Jr, ForccastitigMethods. Homewood, 111.: Irwin, 1971.

4. Hirsch, Albert A, and M. C. Lovell. Sales Anticipation andInventory Behavior. New York: John Wiley & Sons, 1969.

5. Montgomery, David B. "FORAC MOD I: Program forForecast Evaluation Statistics," Journal of Marketing Re-search, 9 (May 1972), 200.

6. Theil, Henry. Applied Econotnic Forecasts. Amsterdam-North Holland, 1966.

7- • Economic Forecasts and Policy. Amsterdam: NorthHolland, 1965.

8. Zarnovitz, Victor. "An Appraisal of Short-Term EconomicForecasts." Occasional Paper 104, National Bureau of Eco-nomic Research, 1967,