Matthew Greenstein Matthew Greenstein | | METEO 485 METEO 485 | | Apr. Apr. 26, 2004 26, 2004 Using Neural Networks and Using Neural Networks and Lagged Climate Indices to Lagged Climate Indices to Predict Monthly Temperature Predict Monthly Temperature and Precipitation Anomalies and Precipitation Anomalies
40
Embed
Matthew Greenstein | METEO 485 | Apr. 26, 2004 Using Neural Networks and Lagged Climate Indices to Predict Monthly Temperature and Precipitation Anomalies.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
2. Root relative squared error• Relative to error if prediction
= average of actual values
• Outliers are penalized strongly
ii
iii
aa
ap
2
2
)(
)(
Results IResults I
NE TemperatureNE Temperature
• Linear regression: r = 0.1067, RRSE = 102.25%
• Neural nets:Neural Net Layers
8,4,2 9,6
Momentum
0.3 0.3
Learning Rate
0.3 0.3
Epochs 500 500
R -0.0205 -0.1190
RRSE 100.25% 100.30%
Results IResults I
SE TemperatureSE Temperature
• Linear regression: r = 0.0352, RRSE = 104.78%
• Neural nets:Neural Net Layers
4,2 15,7
Momentum
0.3 0.3
Learning Rate
0.2 0.2
Epochs 500 500
R 0.0372 -0.0389
RRSE 100.13% 100.74%
Results IResults I
SW TemperatureSW Temperature
• Linear regression: r = 0.036, RRSE = 103.40%
• Neural nets:Neural Net Layers
9,3 9,3
Momentum
0.2 0.4
Learning Rate
0.2 0.2
Epochs 500 500
R 0.063 0.025
RRSE 99.99% 99.97%
Results IResults I
NW TemperatureNW Temperature
• Linear regression: r = 0.011, RRSE = 103.88%
• Neural nets:Neural Net Layers
Auto 9,5
Momentum
0.05 0.8
Learning Rate
0.2 0.2
Epochs 500 500
R 0.176 0.224
RRSE 98.66% 99.08%
Results IResults I
NE PrecipitationNE Precipitation
• Linear regression: r = 0.073, RRSE = 101.044%
• Neural nets:Neural Net Layers
5,3 5,3
Momentum
0.5 0.2
Learning Rate
0.2 0.1
Epochs 500 2000
R 0.061 0.115
RRSE 99.732% 99.999%
Results IResults I
SE PrecipitationSE Precipitation
• Linear regression: r = 0.063, RRSE = 104.14%
• Neural nets:Neural Net Layers
8,3 Auto
Momentum
0.7 0.5
Learning Rate
0.2 0.2
Epochs 500 500
R 0.0651 0.1179
RRSE 99.85% 101.05%
Results IResults I
SW PrecipitationSW Precipitation
• Linear regression: r = 0.187, RRSE = 98.83%
• Neural nets:Neural Net Layers
9,5 9,5
Momentum
0.5 0.3
Learning Rate
0.2 0.2
Epochs 500 500
R 0.280 0.289
RRSE 92.25% 96.12%
Results IResults I
NW PrecipitationNW Precipitation
• Linear regression: r = 0.091, RRSE = 101.49%
• Neural nets:Neural Net Layers
9,5 Auto
Momentum
0.3 0.3
Learning Rate
0.5 0.6
Epochs 500 500
R 0.052 0.098
RRSE 99.91% 102.71%
Results IResults I
• Putrid results !!
• Not worth trying NC/SC… away from oceans
• RRSE ~ 100%, r ~ 0.10
• No big improvement over linear regression
• SW Precipitation predictedthe best (although still bad)…El Nino-related?
Method IIMethod II
• Predict positive or negative anomaly instead of actual value!
• Anomalies changed to binary (1, 0) predictands
• Vary indices used
• Does that cause significant changes?
• This became the most interesting part of the study
• Limited time available: NE T, NE P, SW P
Method IIMethod II
Skill scoresSkill scores
• Many scores provided
• 3 used
1. Percent Correctly Classified
2. TP (True Positive) Rate
3. TN (True Negative) Rate
Results IIResults II
NE Temperature NE Temperature
Neural Net Setup
Auto No Month
Only Nino:Nino 3.4
No Nino’s
No Nino’s, SOI, MEI
More Epochs
Momentum
0.5 0.5 0.5 0.5 0.5 0.5
Learning Rate
0.5 0.5 0.5 0.5 0.5 0.5
Epochs 500 500 500 500 500 1000
Classified Correctly
56.04%
54.59% 51.69% 54.59% 53.62% 56.52%
TP Rate .617 .628 .606 .617 .553 .628
TN Rate .513 .478 .442 .487 .522 .513
Auto = WEKA automatically chooses node setup
Results IIResults II
NE PrecipitationNE Precipitation
Neural Net Setup
Auto No Month
Only Nino:Nino 3.4
No Nino’s
No Nino’s, SOI, MEI
Auto
Momentum
0.3 0.3 0.3 0.3 0.3 0.3
Learning Rate
0.2 0.2 0.2 0.2 0.2 0.2
Epochs 500 500 500 500 500 1000
Classified Correctly
54.11%
48.79% 53.14% 49.28% 51.21% 53.62%
TP Rate .664 .573 .691 .645 .755 .645
TN Rate .402 .392 .351 .320 .237 .412
** Changing the epochs results in overfitting!
Results IIResults II
SW PrecipitationSW Precipitation
Neural Net Setup
Auto No Month
Only Nino:Nino 3.4
No Nino’s
No Nino’s, SOI, MEI
Momentum
0.1 0.1 0.1 0.1 0.1
Learning Rate
0.2 0.2 0.2 0.2 0.2
Epochs 500 500 500 500 500
Classified Correctly
60.39% 60.39% 62.32% 61.83% 59.42%
TP Rate .611 .611 .646 .628 .841
TN Rate .596 .596 .596 .606 .298
** Changing the epochs did not change the ‘Only Nino: Nino 3.4” value
DiscussionDiscussion
• NE Temperature 94 +, 113 –
• Predict negative correct 54.59%
• Best neural net: 56.52% correctly classified
• NE Precipitation 110 +, 97 –
• Predict positive correct 53.14%
• Best neural net: 54.11 % correctly classified
• SW Precipitation 113 +, 94 –
• Predict positive correct 54.59%
• Best neural net: 62.32% correctly classified
DiscussionDiscussion
• These types of neural nets do not provide significant skill over ‘guessing’
• Similar to Method I, not significant difference in skill of logistic regression versus neural nets
• There is some sensitivity to which variables are included in the neural net… even though the decay factor would attempt to eliminate useless interactions
• Different sensitivities in each region
• Using the ‘auto’ setting for layers produced better results
DiscussionDiscussion
• The study was originally supposed to predict the anomaly, but predicting the sign of the anomaly seems to show more promise
• Time constraints prevented a more in depth look at Method II possible Meteo 485 project in future semesters
• Missing June – Sept data could have caused problems with this study
Future WorkFuture Work
1. Obtain missing PNA & EPO data
2. Build neural nets for other regionsof the country for Method II
3. Use different lag times and combinations of lag times
4. Use different climate indices
5. Omit different indices from current set
6. Try other tools that WEKA offers
Special thanks to…Special thanks to…
• Jeremy Ross
For gathering anomaly data
• Climate Diagnostics Center (CDC)
For climate indices
• Dr. George Young
Neural net info from Meteo 474 notes
Useful InfoUseful Info
• WEKA website with software downloads: http://www.cs.waikato.ac.nz/ml/weka/