YOU ARE DOWNLOADING DOCUMENT

Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Category:

Documents

Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Please tick the box to continue:

Transcript
Page 1: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Makingpredictions

Page 2: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Sofar...Build,check&selectmodelsfordetectability

Build,check&selectmodelsforabundance

Makesomeecologicalinferenceaboutsmooths

Whataboutpredictions?

Page 3: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Let'stalkaboutmaps

Page 4: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Whatdoesamapmean?Grids!

Cellsareabundanceestimate

“snapshot”

Sumcellstogetabundance

Sumasubset?

Page 5: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Goingbacktotheformula(Count)Model:

Predictions(index ):

Needto“fill-in”valuesfor , and .

= exp[ + s( ) + s( )] +nj Aj p̂j β0 yj Depthj ϵj

r

= exp[ + s( ) + s( )]nr Ar β0 yr Depthr

Ar yr Depthr

Page 6: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

PredictingWiththesevaluescanusepredictinRpredict(model, newdata=data)

Page 7: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Predictiondata x y Depth SST NPP DistToCAS EKE off.set126 547984.6 788254 153.5983 8.8812 1462.521 11788.974 0.0074 1e+08127 557984.6 788254 552.3107 9.2078 1465.410 5697.248 0.0144 1e+08258 527984.6 778254 96.8199 9.6341 1429.432 13722.626 0.0024 1e+08259 537984.6 778254 138.2376 9.6650 1424.862 9720.671 0.0027 1e+08260 547984.6 778254 505.1439 9.7905 1379.351 8018.690 0.0101 1e+08261 557984.6 778254 1317.5952 9.9523 1348.544 3775.462 0.0193 1e+08 LinkID Nhat_tw126 1 0.01417657127 2 0.05123483258 3 0.01118858259 4 0.01277096260 5 0.04180434261 6 0.45935801

Page 8: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

AquickwordaboutrastersWehavetalkedaboutrastersabit

InR,thedata.frameiskingFortunatelyas.data.frameexistsMakeour“stack”andthenconverttodata.frame

Page 9: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Predictors

Page 10: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

MakingapredictionAddanothercolumntothepredictiondata

Plottingtheneasier(inR)

predgrid$Nhat_tw <- predict(dsm_all_tw_rm, predgrid)

Page 11: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Mapsofpredictionsp <- ggplot(predgrid) + geom_tile(aes(x=x, y=y, fill=Nhat_tw)) + scale_fill_viridis() + coord_equal()print(p)

Page 12: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

TotalabundanceEachcellhasanabundance,sumtogettotal

sum(predict(dsm_all_tw_rm, predgrid))

[1] 2491.864

Page 13: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

SubsettingRsubsettingletsyoucalculate“interesting”estimates:

# how many sperm whales at depths less than 2500m?sum(predgrid$Nhat_tw[predgrid$Depth < 2500])

[1] 1006.272

# how many sperm whales North of 0?sum(predgrid$Nhat_tw[predgrid$x>0])

[1] 1383.742

Page 14: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Extrapolation

Page 15: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Whatdowemeanbyextrapolation?Predictingatvaluesoutsidethoseobserved

Whatdoes“outside”mean?

betweentransects?

outside“surveyarea”?

Page 16: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

TemporalextrapolationModelsaretemporallyimplicit(mostly)

Dynamicvariableschangeseasonally

Migrationcanbeanissue

Needtounderstandwhatthepredictionsare

Page 17: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

ExtrapolationExtrapolationisfraughtwithissues

Wanttobepredicting“insidetherug”

Ingeneral,trynottodoit!

(Thinkaboutvariancetoo!)

Page 18: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

RecapUsingpredictGetting“overall”abundance

Subsetting

PlottinginR

Extrapolation(anditsdangers)

Page 19: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Estimatingvariance

Page 20: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

NowwecanmakepredictionsNowwearedangerous.

Page 21: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Predictionsareuselesswithoutuncertainty

Page 22: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Wheredoesuncertaintycomefrom?

Page 23: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

SourcesofuncertaintyDetectionfunction

GAMparameters

Page 24: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Let'sthinkaboutsmoothsfirst

Page 25: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

UncertaintyinsmoothsDashedlinesare+/-2standarderrors

Howdowetranslateto ?N̂

Page 26: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

BacktobasesBeforeweexpressedsmoothsas:

Theorytellsusthat:

where isabitcomplicated

(derivedfromthesmoothermatrix)

s(x) = (x)∑Kk=1 βkbk

β ∼ N( , )β̂ Vβ

Vβ

Page 27: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Predictionstopredictionvariance(roughly)

“map”dataontofittedvalues

“map”predictionmatrixtopredictions

Here needtotakesmoothsintoaccount

pre-/post-multiplyby to“transformvariance”

linkscale,needtodoanothertransformforresponse

Xβ

βXp

Xp

Xp

⇒ XTp VβXp

Page 28: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Addingindetectionfunctions

Page 29: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

GAM+detectionfunctionuncertainty(Gettingalittlefast-and-loosewiththemathematics)

( ) ≈ (GAM) +CV2 N̂ CV2

(detectionfunction)CV2

Page 30: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Notthatsimple...AssumesdetectionfunctionandGAMareindependent

Maybethisisokay?

(Probablynottrue?)

Page 31: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

VariancepropagationIncludethedetectabilityasterminGAM

Randomeffect,meanzero,varianceofdetectionfunction

Uncertainty“propagated”throughthemodel

Detailsinbibliography(toomuchtodetailhere)

Underdevelopment

(Cancoverinspecialtopic)

Page 32: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Thatseemedcomplicated...

Page 33: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Rtotherescue

Page 34: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

InR...Functionsindsmtodothis

dsm.var.gamassumesspatialmodelanddetectionfunctionareindependent

dsm.var.proppropagatesuncertaintyfromdetectionfunctiontospatialmodel

onlyworksforcountmodels(moreorless)

Page 35: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

VarianceofabundanceUsingdsm.var.gam

dsm_tw_var_ind <- dsm.var.gam(dsm_all_tw_rm, predgrid, off.set=predgrid$off.set)summary(dsm_tw_var_ind)

Summary of uncertainty in a density surface model calculated analytically for GAM, with delta method

Approximate asymptotic confidence interval: 2.5% Mean 97.5% 1539.018 2491.864 4034.643 (Using log-Normal approximation)

Point estimate : 2491.864 CV of detection function : 0.2113123 CV from GAM : 0.1329 Total standard error : 622.0389 Total coefficient of variation : 0.2496

Page 36: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

VarianceofabundanceUsingdsm.var.prop

dsm_tw_var <- dsm.var.prop(dsm_all_tw_rm, predgrid, off.set=predgrid$off.set)summary(dsm_tw_var)

Summary of uncertainty in a density surface model calculated by variance propagation.

Probability of detection in fitted model and variance model Fitted.model Fitted.model.se Refitted.model1 0.3624567 0.07659373 0.3624567

Approximate asymptotic confidence interval: 2.5% Mean 97.5% 1556.898 2458.634 3882.646 (Using log-Normal approximation)

Point estimate : 2458.634 Standard error : 581.0379 Coefficient of variation : 0.2363

Page 37: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Plotting-dataprocessingCalculateuncertaintyper-cell

dsm.var.*thinkspredgridisone“region”Needtosplitdataintocells(usingsplit())(Couldbearbitrarysetsofcells,seeexercises)

Needwidthandheightofcellsforplotting

Page 38: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Plotting(code)predgrid$width <- predgrid$height <- 10*1000predgrid_split <- split(predgrid, 1:nrow(predgrid))head(predgrid_split,3)

$`1` x y Depth SST NPP DistToCAS EKE off.set126 547984.6 788254 153.5983 8.8812 1462.521 11788.97 0.0074 1e+08 LinkID Nhat_tw height width126 1 0.01417657 10000 10000

$`2` x y Depth SST NPP DistToCAS EKE off.set127 557984.6 788254 552.3107 9.2078 1465.41 5697.248 0.0144 1e+08 LinkID Nhat_tw height width127 2 0.05123483 10000 10000

$`3` x y Depth SST NPP DistToCAS EKE off.set258 527984.6 778254 96.8199 9.6341 1429.432 13722.63 0.0024 1e+08 LinkID Nhat_tw height width258 3 0.01118858 10000 10000

dsm_tw_var_map <- dsm.var.prop(dsm_all_tw_rm, predgrid_split, off.set=predgrid$off.set)

Page 39: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

CVplotp <- plot(dsm_tw_var_map, observations=FALSE, plot=FALSE) + coord_equal() + scale_fill_viridis()print(p)

Page 40: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

InterpretingCVplotsPlottingcoefficientofvariation

Standardisestandarddeviationbymean

(percell)

Canbeusefultooverplotsurveyeffort

CV = se( )/N̂ N̂

Page 41: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Effortoverplotted

Page 42: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

BigCVsHereCVsare“wellbehaved”

Notalwaysthecase(hugeCVspossible)

Thesecanbeapaintoplot

Usecut()inRtomakecategoricalvariable

e.g.c(seq(0,1, len=100), 2:4, Inf)orsomesuch

Page 43: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

RecapHowdoesuncertaintyariseinaDSM?

Estimatevarianceofabundanceestimate

Mapcoefficientofvariation

Page 44: Making predictions - GitHub · Predictions to prediction variance (roughly) “map” data onto fitted values “map” prediction matrix to predictions Here need to take smooths

Let'strythat!


Related Documents
Depth Map Prediction from a Single Image using a Multi ...papers.nips.cc/paper/5539-depth-map-prediction... · Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Depth Map Prediction from a Single Image using a Multi...
Category: Documents
RIVER MODELLING FOR FLOOD RISK MAP PREDICTION: …eprints.usm.my/15638/1/RIVER_MODELLING_FOR_FLOOD_RISK_MAP_PREDICTION.… · i RIVER MODELLING FOR FLOOD RISK MAP PREDICTION: CASE
RIVER MODELLING FOR FLOOD RISK MAP PREDICTION:...
Category: Documents
Statistical landslide prediction map as a basis for a risk map · 2014-05-18 · Statistical landslide prediction map as a basis for a risk map M. Komac Geological Survey of Slovenia,
Statistical landslide prediction map as a basis for a risk.....
Category: Documents
APPENDIX 1 SUMMARY OF TRAFFIC INFORMATION. 2005 … · SUMMARY OF TRAFFIC INFORMATION. 2005 URBAN PREDICTION Predictions from 1998 Predictions from 1999 Differences? Road No. ...
APPENDIX 1 SUMMARY OF TRAFFIC INFORMATION. 2005 … ·...
Category: Documents
TMT Predictions 2019 - Deloitte UK · TMT Predictions – Charts and graphics January 2019 1 Contents Prediction 1: 5G 2 Prediction 2: AI 3 Prediction 3: Smart Speakers 4 Prediction
TMT Predictions 2019 - Deloitte UK · TMT Predictions –.....
Category: Documents
Data Mining Protein Structures' Topological Properties to Enhance Contact Map Predictions
Data Mining Protein Structures' Topological Properties to.....
Category: Technology
Dennis D. McCarthy Elements of Prediction. Why are we here? IERS Working Group on Predictions IERS Working Group on Predictions Definitive user requirements.
Dennis D. McCarthy Elements of Prediction. Why are we here?....
Category: Documents
Advancements in NMR Predictions- Neural Network vs. HOSE ... · PDF fileAdvancements in NMR Predictions-Neural Network vs. HOSE Code Algorithms ... Prediction Accuracy ... (ppm)-40-20
Advancements in NMR Predictions- Neural Network vs. HOSE ......
Category: Documents
Comparison of polar motion prediction results supplied by the IERS Sub-bureau for Rapid Service and Predictions and results of other prediction methods.
Comparison of polar motion prediction results supplied by...
Category: Documents
Some Thoughts about Learning Predictions Onlinewhitem/presentations/...Some Thoughts about Learning Predictions Online Martha White TTT, 2019 Online Prediction Learning • Constant
Some Thoughts about Learning Predictions...
Category: Documents
Reliability predictions in product developmentproofengineering.com/images/Reliability Prediction Overview.pdf · Parts count reliability prediction ... to use NSWC 2011 Handbook of
Reliability predictions in product...
Category: Documents
Computational Prediction of Pressure and Thermal ... · Computational Prediction of Pressure and Thermal Environments in ... mal load predictions in aerospace ... and commercial vehicles.
Computational Prediction of Pressure and Thermal ... ·...
Category: Documents
ANL – BSFC map prediction Engines 22-26
ANL – BSFC map prediction Engines 22-26
Category: Documents
ENSO: Recent Evolution, Current Status and Predictions · 2016-05-06 · ENSO: Recent Evolution, Current Status and Predictions Update prepared by: Climate Prediction Center / NCEP
ENSO: Recent Evolution, Current Status and Predictions ·.....
Category: Documents
Reconciling Predictions in the Regression Setting: An ......Reconciling Predictions in the Regression Setting: An Application to Bus Travel Time Prediction Joao Mendes-Moreira˜ 1,2
Reconciling Predictions in the Regression Setting: An...
Category: Documents
Sea Ice Prediction Network (SIPN)...Predictions by method! SIO predictions with contributor-supplied uncertainty 2012 Observed extent lies outside the intervals for 11 of 15 predictions,
Sea Ice Prediction Network (SIPN)...Predictions by method!.....
Category: Documents
Centrifugal Compressor Map Prediction and Modification_new
Centrifugal Compressor Map Prediction and Modification_new
Category: Documents
Prediction Markets: Theory and Applications · an accurate prediction. Chapter 7 extends the classical prediction elicitation setting to describe decision markets, where predictions
Prediction Markets: Theory and Applications · an accurate....
Category: Documents
Functional Capacity Evaluations (FCEs): Medical Evidence ... · NIOSH predictions Matheson/Epic: Controlled with prediction module Key : Controlled with prediction module Occupro:
Functional Capacity Evaluations (FCEs): Medical Evidence...
Category: Documents
Aggregating Algorithm for prediction of packs · 2019. 8. 1. · prediction under delayed feedback. Under the prediction of packs protocol, the learner must make a few predictions
Aggregating Algorithm for prediction of packs · 2019. 8......
Category: Documents
Prediction in Antitrust is Hard (But Some Predictions are Harder than Others) · Prediction in Antitrust is Hard (But Some Predictions are Harder than Others) Washington Bar Association’s
Prediction in Antitrust is Hard (But Some Predictions are...
Category: Documents
Beyond MAP: Making Multiple Predictions: Diversity, DPPs and more.
Beyond MAP: Making Multiple Predictions: Diversity, DPPs.....
Category: Documents
Accurate future prediction: Know about personal future predictions
Accurate future prediction: Know about personal future...
Category: Spiritual
Supplementary Online Material - busco.ezlab.org · Augustus predictions using generic parameters may produce sub-optimal gene predictions. Training the Training the prediction parameters
Supplementary Online Material - busco.ezlab.org · Augustus...
Category: Documents
Research Department - ECMWF · climate change predictions (Collins and Knight, 2007). For hydrological prediction in general, the Hydrologic Ensemble Prediction Experiment (HEPEX)
Research Department - ECMWF · climate change predictions.....
Category: Documents
Experimental Climate Monitoring and Prediction Reports/2011/Dec/WMS_Report...predictions (delivered via IRI map tool) predict 25mm- over 120mm rainfall from 29th Nov – 04th December
Experimental Climate Monitoring and Prediction...
Category: Documents
ENSO: Recent Evolution, Current Status and Predictions...Current Status and Predictions Update prepared by: Climate Prediction Center / NCEP 1 May 2017 . Outline ... with increasing
ENSO: Recent Evolution, Current Status and...
Category: Documents
Bioinformatics Gene detection and prediction Gene predictions in prokaryotes Gene predictions in eukaryotes Difficulties of gene prediction Statistical.
Bioinformatics Gene detection and prediction Gene...
Category: Documents
Chapter 4 Prediction and Bayesian Inference 4.1 Estimators versus predictors 4.2 Prediction for one-way ANOVA models –Shrinkage estimation, types of predictions.
Chapter 4 Prediction and Bayesian Inference 4.1 Estimators.....
Category: Documents
Scaling Ab Initio Predictions of 3D Protein Structures in ... · Keywords Bioinformatics ·Proteins ·3D protein structure ·Protein structure prediction ·Tertiary structure prediction
Scaling Ab Initio Predictions of 3D Protein Structures in...
Category: Documents