DATA QUALITY and ANALYSIS Strategy for Monitoring Post-fire Rehabilitation Treatments Troy Wirth and David Pyke Troy Wirth and David Pyke USGS – Biological Resources Division USGS – Biological Resources Division Forest and Rangeland Ecosystem Science Forest and Rangeland Ecosystem Science Center Center Corvallis, Oregon Corvallis, Oregon U.S. Department of Interior U.S. Geological Survey Supported by USGS - BLM Interagency Agreement #HAI040045
39
Embed
DATA QUALITY and ANALYSIS Strategy for Monitoring Post-fire Rehabilitation Treatments Troy Wirth and David Pyke USGS – Biological Resources Division Forest.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DATA QUALITY and ANALYSIS
Strategy for Monitoring Post-fire Rehabilitation Treatments
Troy Wirth and David PykeTroy Wirth and David PykeUSGS – Biological Resources DivisionUSGS – Biological Resources DivisionForest and Rangeland Ecosystem Science CenterForest and Rangeland Ecosystem Science CenterCorvallis, OregonCorvallis, Oregon
U.S. Department of InteriorU.S. Geological Survey
Supported by USGS - BLM Interagency Agreement #HAI040045
Data QualityData Quality
Assess the ability of the data to determine Assess the ability of the data to determine treatment successtreatment success
Ability to achieve high data quality will Ability to achieve high data quality will depend on variabilitydepend on variability
Confidence IntervalsConfidence Intervals Construct a simple Construct a simple
confidence interval around confidence interval around data to determine data to determine precision of estimateprecision of estimate
The narrower the The narrower the confidence interval, the confidence interval, the more precise the estimatemore precise the estimate
Must specify the alpha Must specify the alpha level level
n
stX critical
Alpha level, or type I error is the probability of Alpha level, or type I error is the probability of declaring there is no difference when there is. declaring there is no difference when there is. Specifies the width of the confidence interval (1 – Specifies the width of the confidence interval (1 – alpha)alpha)
Equations which estimate the number of Equations which estimate the number of samples required to meet your sampling samples required to meet your sampling objective objective For single populations (quantitative objective)For single populations (quantitative objective)
Confidence level (Type I error rate)Confidence level (Type I error rate) Confidence interval widthConfidence interval width
For detecting difference between two For detecting difference between two populationspopulations
Confidence levels (Type I and II error rate)Confidence levels (Type I and II error rate) Minimum detectable changeMinimum detectable change
Iterative processIterative process
Single Population Sample SizeSingle Population Sample Size
Calculate sample size Calculate sample size estimate for each estimate for each parameter of interestparameter of interest
Result will depend on Result will depend on variability of datavariability of data
For example, using For example, using equation for single equation for single population:population:
2
22
d
sZn
= sample size required = alpha level for specified
level of confidence
s = standard deviation
n
d = desired precision level (absolute term)
Single Population Sample Size Single Population Sample Size Estimation ExampleEstimation Example
Alpha = 0.1Alpha = 0.1 X = 18.5 % cover of X = 18.5 % cover of
Need 3 more samples (precision achieved 4.7)Need 3 more samples (precision achieved 4.7)
2
22
d
sZn
Improving Data QualityImproving Data Quality
In order to increase data quality (achieve In order to increase data quality (achieve sample size) you need to:sample size) you need to: Reduce standard deviation (variability)Reduce standard deviation (variability) Increase the number of samplesIncrease the number of samples
In order to reduce sample size estimates In order to reduce sample size estimates without more samples you can:without more samples you can: Increase alpha (less confidence)Increase alpha (less confidence) Increase precision or MDC (detect a larger Increase precision or MDC (detect a larger
difference)difference)
Graphical Analysis Using Graphical Analysis Using Confidence IntervalsConfidence Intervals
Compare treatment results to quantitative Compare treatment results to quantitative objectives or control areas objectives or control areas
Several types of analysis to fit your situationSeveral types of analysis to fit your situation Types of graphical analysis:Types of graphical analysis:
Comparison of treatment to quantitative standardComparison of treatment to quantitative standard Seeded plants vs. quantitative standardSeeded plants vs. quantitative standard All plants at treatment plots change from time 1 to time 2All plants at treatment plots change from time 1 to time 2
Comparison of two populations (seeded/unseeded)Comparison of two populations (seeded/unseeded) Treatment vs. controlTreatment vs. control Treatment vs. control (change from time 1 to time 2)Treatment vs. control (change from time 1 to time 2)
Flowchart for Graphical Analysis Flowchart for Graphical Analysis DensityDensity
Graphical AnalysisGraphical Analysis (comparison to a standard)(comparison to a standard)
Specify quantitative objectiveSpecify quantitative objective Determine desired alpha level and precisionDetermine desired alpha level and precision Collect data at treatment plotsCollect data at treatment plots Graph mean with confidence interval of Graph mean with confidence interval of
desired width (typically 80 or 90%)desired width (typically 80 or 90%) Graphically compare to quantitative standard Graphically compare to quantitative standard
to determine which situation exists to determine which situation exists Need mean, standard deviation, and nNeed mean, standard deviation, and n Use ES&R Equation spreadsheet to helpUse ES&R Equation spreadsheet to help
Graphical AnalysisGraphical Analysis (comparison to a quantitative objective)(comparison to a quantitative objective)
D: The sample mean is above the quantitative objective, but the lower limit of the confidence interval is below the objective.
Comparison to a Quantitative ObjectiveComparison to a Quantitative Objective
ObjectiveProbablyNot Met
Veg
eta
tio
n P
aram
eter
0
1
2
3
4
5
Management Objective
Objective Not Met
Objective MayBe Met
Check CI
ObjectiveSurpassed
A B C D
C: The sample mean is above the objective but the lower confidence limit is below the objective.
Comparison to a Quantitative ObjectiveComparison to a Quantitative Objective
ObjectiveProbablyNot Met
Veg
etat
ion
Par
amet
er
0
1
2
3
4
5
Management Objective
Objective Not Met
Objective MayBe Met
Check CI
ObjectiveSurpassed
A B C D
B: The sample mean is below the quantitative objective, but the upper limit of the confidence interval is above the objective.
Comparison to a Quantitative ObjectiveComparison to a Quantitative Objective
ObjectiveProbablyNot Met
Veg
etat
ion
Par
amet
er
0
1
2
3
4
5
Management Objective
Objective Not Met
Objective MayBe Met
Check CI
ObjectiveSurpassed
A B C D
A: The sample mean and confidence interval (CI) fall below the objective. Conclude that the objective has not been met.
Comparison to a Quantitative ObjectiveComparison to a Quantitative Objective
ObjectiveProbablyNot Met
Veg
eta
tio
n P
aram
eter
0
1
2
3
4
5
Management Objective
Objective Not Met
Objective MayBe Met
Check CI
ObjectiveSurpassed
A B C D
Graphical AnalysisGraphical Analysis (Treatment v. Control)(Treatment v. Control)
Confidence interval of the difference between the Confidence interval of the difference between the treatment and controltreatment and control
Uses the difference between the means of two Uses the difference between the means of two treatments and constructs a single CI using the variance treatments and constructs a single CI using the variance from both estimates (SE)from both estimates (SE)
A mean of 0 represents no difference between the two A mean of 0 represents no difference between the two treatmentstreatments
Express the quantitative objective as an absolute value Express the quantitative objective as an absolute value or as a multiple of the control.or as a multiple of the control.
Use the mean and CI to make a determination of Use the mean and CI to make a determination of treatment effecttreatment effect
Graphical AnalysisGraphical Analysis (Treatment v. Control)(Treatment v. Control)
Using the ESR monitoring spreadsheetUsing the ESR monitoring spreadsheet Specify the desired alpha level Specify the desired alpha level Enter the mean, standard deviation, and N from the Enter the mean, standard deviation, and N from the
data collected at the treatment and control plotsdata collected at the treatment and control plots Specify the level of quantitative objective (multiple of Specify the level of quantitative objective (multiple of
control or absolute difference)control or absolute difference) Make interpretation based on graphical analysis Make interpretation based on graphical analysis
of the CI of the difference between the two of the CI of the difference between the two treatmentstreatments
Graphical AnalysisGraphical Analysis (Treatment v. Control)(Treatment v. Control)
Quantitative objective: twice that of control plot Quantitative objective: twice that of control plot (2) – note that because it is a CI of difference (2) – note that because it is a CI of difference the original amount is subtracted from the original amount is subtracted from quantitative objectivequantitative objective
ControlControl X = 2.0X = 2.0 S = 0.5S = 0.5 N = 7N = 7
TreatmentTreatment X = 4.9X = 4.9 S = 0.9S = 0.9 N = 7N = 7
Treatment vs. ControlTreatment vs. ControlV
egea
tio
n P
aram
eter
-2
0
2
4
6
8
10
A B C D E F
Ecological
Significance
A: The mean and confidence interval for the difference between the two means is completely above the level of ecological significance (5 plants/m2).
Veg
eati
on
Par
amet
er
-2
0
2
4
6
8
10
A B C D E F
Ecological
Significance
B: The difference of the mean between is above the level of ecological significance, but the lower confidence limit for the difference is below the level of ecological significance.
Treatment vs. ControlTreatment vs. Control
Veg
eati
on
Par
amet
er
-2
0
2
4
6
8
10
A B C D E F
Ecological
Significance
C: The difference between the two means is below the level of ecological significance, but the upper confidence limit for the difference is above the level of ecological significance.
Treatment vs. ControlTreatment vs. Control
Veg
eati
on
Par
amet
er
-2
0
2
4
6
8
10
A B C D E F
Ecological
Significance
D: The mean and confidence interval of the difference is below the level of ecological significance. Conclude that there is no ecologically significant difference between the control and treatment plots.
Treatment vs. Control Treatment vs. Control
Veg
eati
on
Par
amet
er
-2
0
2
4
6
8
10
A B C D E F
Ecological
Significance
E: The mean of the difference is above zero, but the lower confidence limit is below 0 (no difference) and the upper confidence limit is above 5 plants/m2.
Treatment vs. ControlTreatment vs. Control
Veg
eati
on
Par
amet
er
-2
0
2
4
6
8
10
A B C D E F
Ecological
Significance
F: The mean of the difference is above 0, but the lower confidence limit is below 0 and the upper confidence limit is below the level of ecological significance.
Treatment vs. ControlTreatment vs. Control
Flowchart for Graphical Analysis Flowchart for Graphical Analysis DensityDensity
Graphical AnalysisGraphical Analysis (Treatment vs. Control T2-T1)(Treatment vs. Control T2-T1)
Confidence interval of the difference in change between Confidence interval of the difference in change between two time periods between treatment and controltwo time periods between treatment and control
Uses the difference between the change in the means of Uses the difference between the change in the means of treatment and control and constructs a single CI using treatment and control and constructs a single CI using the variance from both estimates (SE)the variance from both estimates (SE)
A mean of 0 represents no difference in change A mean of 0 represents no difference in change between the two treatmentsbetween the two treatments
Express the quantitative objective as an absolute value Express the quantitative objective as an absolute value or as a multiple of the control.or as a multiple of the control.
Use the mean and CI to make a determination of Use the mean and CI to make a determination of treatment effecttreatment effect
Pla
nts
/m2
1
2
3
4
5
6
7
8
Control Treatment
Year 1 Year 3
Pla
nts
/m2
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Control Treatment
Year 1 Year 3
Appears that there is a difference in year three when there actually was not.
Accounting for initial difference in the degree of change
Veg
eati
on
Par
amet
er
-2
0
2
4
6
8
10
A B C D E F
Ecological
Significance
B: The difference of the mean between is above the level of ecological significance, but the lower confidence limit for the difference is below the level of ecological significance.
Change in Treatment vs. Control Change in Treatment vs. Control
Graphical AnalysisGraphical Analysis (Treatment at two time periods, T2-T1)(Treatment at two time periods, T2-T1)
Confidence interval of the change between the two time Confidence interval of the change between the two time periodsperiods
Treats the two time periods as paired, reducing Treats the two time periods as paired, reducing variabilityvariability
A mean of 0 represents no change between the two A mean of 0 represents no change between the two time periodstime periods
Express the quantitative objective as the desired change Express the quantitative objective as the desired change between the two different time periodsbetween the two different time periods
Use the mean and CI compared to the quantitative Use the mean and CI compared to the quantitative objective to make a determination of successobjective to make a determination of success
Paste graphs directly into reports and describe quantitative results e.g.Perennial Grass DensityThe density of perennial grasses is significantly greater in the treatment plots as compared to the control plots. We are 90% confident that the difference is between 1.06 to 4.54 plants/m2 greater than the control plots with a mean of 2.8 plants/m2)
Confidence Interval of the Difference Between Treatment and Control Plots
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Para
mete
r
ReportingReporting
ReportingReporting
Link reports back to quantitative objectivesLink reports back to quantitative objectives Re-assess whether objectives were Re-assess whether objectives were
reasonable and possible reasons for reasonable and possible reasons for success and failure.success and failure.
Make recommendations for future Make recommendations for future improvements to implementation and improvements to implementation and monitoringmonitoring