Taking Uncertainty Into Taking Uncertainty Into Account: Account: Bias Issues Arising from Uncertainty Bias Issues Arising from Uncertainty in Risk Models in Risk Models John A. Major, ASA John A. Major, ASA Guy Carpenter & Company, Inc. Guy Carpenter & Company, Inc.
21
Embed
Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Taking Uncertainty Into Taking Uncertainty Into Account:Account:
Bias Issues Arising from Uncertainty in Risk Bias Issues Arising from Uncertainty in Risk ModelsModels
John A. Major, ASAJohn A. Major, ASAGuy Carpenter & Company, Inc.Guy Carpenter & Company, Inc.
N=20 observationsN=20 observations T = sample mean; T = sample mean; =1 true mean=1 true mean MLE EP curve:MLE EP curve:
q-exceedance point (PML, VaR)q-exceedance point (PML, VaR)
Sampling Distribution of TSampling Distribution of T
Estimated PDFsEstimated PDFs
Client QuestionsClient Questions
What is the 1 in 100-yr PML (1% VaR)?What is the 1 in 100-yr PML (1% VaR)? What is probability of exceeding 4.605?What is probability of exceeding 4.605? Can you give me an EP curve to answer Can you give me an EP curve to answer
these and similar questions?these and similar questions? Does sampling error affect the answer?Does sampling error affect the answer? Can I get unbiased answers?Can I get unbiased answers?
3 Kinds of Bias3 Kinds of Bias
““dollar” or X-bias:dollar” or X-bias: the average of PML dollar estimatesthe average of PML dollar estimates
““probabilistic” or P-bias:probabilistic” or P-bias: the average the average truetrue exceedance probability of exceedance probability of estimatedestimated PML points PML points
““exceedance” or Q-bias:exceedance” or Q-bias: the average estimated exceedance the average estimated exceedance
probabilityprobability
qq vsXE Xˆ
qvsXQE qˆ
qvsQE qXˆ
Exponential MLE is X-Exponential MLE is X-unbiasedunbiased
TE
qq qqTEXE X)ln()ln(ˆ
Exponential MLE is X-Exponential MLE is X-unbiasedunbiased
for small qfor small q Expected actual risk is greater than Expected actual risk is greater than
Predictive vs. Model DensityPredictive vs. Model Density
Which to use?Which to use? MLE curve is X-unbiasedMLE curve is X-unbiased
no uncertainty adjustment, but...no uncertainty adjustment, but... on average, gets right $ answeron average, gets right $ answer
Predictive curve is P-unbiasedPredictive curve is P-unbiased ““takes uncertainty into account” and...takes uncertainty into account” and... on average, reflects true exceedance pron average, reflects true exceedance pr
But they disagree...But they disagree... and it gets worse...and it gets worse...
for small qfor small q Expected estimated risk is greater than Expected estimated risk is greater than
the true risk (at the specified threshold)the true risk (at the specified threshold) Uncertainty now causes risk to be Uncertainty now causes risk to be overoverstated!stated!
Exponential MLE is Q-biasedExponential MLE is Q-biased
qQE q Xˆ
Exponential MLE is Q-biasedExponential MLE is Q-biased
Correcting for Q-biasCorrecting for Q-bias
Minimum Variance Unbiased EstimatorMinimum Variance Unbiased Estimator standard procedure in classical statisticsstandard procedure in classical statistics
Rao-Blackwell TheoremRao-Blackwell Theorem Expectation of unbiased estimator, Expectation of unbiased estimator,
conditional on sufficient statisticconditional on sufficient statistic Exponential model:Exponential model: MVUE result:MVUE result:
TxxQ exp)(
1
1)(
n
nT
xxQ
MVUE vs. Model DensityMVUE vs. Model Density
ParadoxParadox
Say we get an estimated T=1 (correct)Say we get an estimated T=1 (correct) MLE says XMLE says X.01.01=4.605, Pr{X>4.605}=1%=4.605, Pr{X>4.605}=1%
Predictive: XPredictive: X.01.01=5.179 is p-unbiased=5.179 is p-unbiased risk is greater than MLE answer because risk is greater than MLE answer because
impact of uncertaintyimpact of uncertainty MVUE: Pr{X>4.605}=.69% is q-unbiasedMVUE: Pr{X>4.605}=.69% is q-unbiased
risk is less because MLE tends to overstate risk is less because MLE tends to overstate exceedance probabilityexceedance probability
How the Paradox ArisesHow the Paradox Arises
ConclusionsConclusions
Uncertainty induces bias in estimatorsUncertainty induces bias in estimators Biases operate in different directionsBiases operate in different directions
depends on the question being askeddepends on the question being asked There is no monolithic “fix” for taking There is no monolithic “fix” for taking
uncertainty into accountuncertainty into account Predictive distribution fixes p-bias, Predictive distribution fixes p-bias, while making q-bias worsewhile making q-bias worse
RecommendationsRecommendations
First: Show modal estimates (MLE etc.)First: Show modal estimates (MLE etc.) Second: Show effect of uncertaintySecond: Show effect of uncertainty
Keep uncertainty distinct from randomnessKeep uncertainty distinct from randomness Sensitivity testing w.r.t. parameters Sensitivity testing w.r.t. parameters Confidence intervals on estimatorsConfidence intervals on estimators
Third: Adjust for bias only as necessaryThird: Adjust for bias only as necessary Carefully attend to the question askedCarefully attend to the question asked Advise that bias adjustment is equivocalAdvise that bias adjustment is equivocal