Top Banner
Mesocyclone Detection Algorithm Neural Network (MDA NN) Briefing for the TAC Arthur Witt, NSSL
26

Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Apr 19, 2018

Download

Documents

dinhtruc
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Mesocyclone Detection Algorithm Neural Network (MDA NN)

Briefing for the TAC

Arthur Witt, NSSL

Page 2: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

ObjectiveObjective

Measure the skill of the MDA NN vsMDA and TDAUse County Warning Scoring methodology instead of time-window scoring

Page 3: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Original MDA NNOriginal MDA NN

Found errors in the data used to train the original NNDeveloped a new NN instead of correcting the original NN

Page 4: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

New MDA NNNew MDA NN

Uses fewer input parameters than the original NNDeveloped on a larger data set than the original NNDesigned to minimize over-fitting of the training data

Page 5: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

New MDA NNNew MDA NN

Like the original NN, has both radar-only and radar + near-storm environment (NSE) componentsLike the original NN, predicts the probability of tornadoUnlike the original NN, wasn't developed to predict the probability of severe wind

Page 6: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

County Warning ScoringCounty Warning Scoring

An automated algorithm scoring system using simulated county warningsMethodology is similar to that used by the NWS to determine severe weather warning performance

Page 7: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Scoring algorithm outputScoring algorithm output

Output scored indirectly via simulated warningsSimulated warnings are issued based on categorical output or bythresholding a parameterPerformance measures are calculated from simulated warnings and ground-truth verification

Page 8: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Procedure for issuing a warningProcedure for issuing a warning

Select a warning parameter

Select a warning threshold

Set duration time for warnings

Page 9: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Procedure for issuing a warningProcedure for issuing a warning

For each volume scan:• Check if parameter ≥ warning threshold• If yes, and storm not in county already

being warned• Calculate areal coverage of warning• Use “default warning polygon” criteria

from AWIPS WarnGen program

Page 10: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Polygon criteriaPolygon criteria

2 miles upstream and 6 miles either side of locationWidens out by a factor of 0.012 for each mile along the pathLength determined from motion vector and duration of warning

Page 11: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Example of warning polygonExample of warning polygon

Page 12: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Performance measuresPerformance measures

warned tornado reportsPOD = -------------------------------

total tornado reports

unverified county warningsFAR = -----------------------------------

total county warnings

warned tornado reportsCSI = ------------------------------------------------------------------

total tornado reports + unverified county warnings

Page 13: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Test dataTest data

36 severe weather events (storm cases)284 tornado reports• 5 cases with 0 reports (null cases)• 14 cases with 1 - 5 reports• 8 cases with 6 - 10 reports• 9 cases with >10 reports

Page 14: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

National coverage from 32 sitesNational coverage from 32 sites

Page 15: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Primary evaluationPrimary evaluation

Simulated tornado warningsAnalysis domain: 230 km from each radar siteAlgorithm predictors:• MDA – strength rank ≥ 5 with time continuity• MDA+NN• MDA+NN+NSE

Page 16: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Secondary evaluationSecondary evaluation

Simulated tornado warningsAnalysis domain: 100 km from each radar siteAlgorithm predictors:• MDA – strength rank ≥ 5 with time continuity• MDA+NN• MDA+NN+NSE• TDA – default parameter settings

Page 17: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Results Results -- Primary evaluationPrimary evaluation

MDA – all cases combined:• POD = 69%• FAR = 91%• CSI = 10%

Page 18: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Results Results -- Primary evaluationPrimary evaluation

Page 19: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Results Results -- Primary evaluationPrimary evaluation

Page 20: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Results Results -- Secondary evaluationSecondary evaluation

MDA:• POD = 79%• FAR = 89%• CSI = 13%

TDA:• POD = 66%• FAR = 90%• CSI = 12%

Page 21: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Results Results -- Secondary evaluationSecondary evaluation

Page 22: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

Results Results -- Secondary evaluationSecondary evaluation

Page 23: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

ConclusionsConclusions

Primary results – only small improvement in skillSecondary results – somewhat greater improvement in skill, but only at higher warning thresholdsNo improvement in skill when NSE data is included

Page 24: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

NonNon--performance NN Issuesperformance NN Issues

Consolidates many parameters into a single probability-based forecast of tornadoUseful “screening tool” in active severe weather situationsProbabilities provide a measure of confidence and are easier to use and understand than many algorithm parameters

Page 25: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

NonNon--performance NN Issuesperformance NN Issues

Acts like a “black box” – many forecasters don't like thisWith increasing emphasis on analysing base data, NN output may be ignoredIf character of input data changes, it may be necessary to retrain the NN

Page 26: Mesocyclone Detection Algorithm Neural Network … Detection Algorithm Neural Network (MDA NN) ... has both radar- ... zAn automated algorithm scoring system using simulated county

RecommendationRecommendation

There is insufficient evidence to support this particular NN being added to the operational WSR-88D system at this time