Top Banner
The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie
20

The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

Jan 13, 2016

Download

Documents

Diana Wright
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Use of Cloud computing in impact assessment of climate change

Kwang Soo Kim and Doug MacKenzie

Page 2: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Outline

• Introduction• Cloud computing• Case study:

Calculation of rainfall frequency in the 21st century

• Results• Conclusions

Page 3: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Climate Change

Page 4: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Impact of climate change

• Geographical distribution of species across a wide range of ecosystems (Walther et al. 2002)

• The timing of blooming for temperate zone species (Root et al. 2003)

• Crop production in positive or negative ways in different regions in the 21st century (Rosenzweig et al. 2001).

Simulation models has been used to assess the impact of Climate change

Page 5: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Computational scale of impact assessment study

• The spatial resolution of GCMs has increased » The typical resolution used in the IPCC TAR was about 250 km

(Houghton et al. 2001).

» In the AR4, many of those models had higher spatial resolution (Miller et al. 2006).

» NCAR-CCSM3 had spatial resolution of 150 km.

• An ensemble prediction system has been used » Probabilistic forecasts of climatic events are generated

» Murphy et al. (2004) used a 53-member ensemble of models to determine the range of climate changes.

• Climate change scenarios» The Special Report on Emission Scenarios (SRES)

» B1, A1B and A2

Page 6: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Cloud computing

• A paradigm of computing in which virtualized resources are provided as a service over the Internet (Gruman & Knorr, 2008)

• Computing resources “As a Service”» Infrastructure as a service (IaaS) » Platform as a service (PaaS) » Software as a service (SaaS)» Data storage as a service (dSaaS)

• Utility computing

• Distributed computing

Page 7: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Amazon web service – Elastic Compute Cloud (EC2)

• Customers can rent computers on which to run their own computer applications

• Scalable deployment of applications by creating virtual machines• A customer can create, launch, and terminate server instances

as needed• Customers are charged by the hour for active servers

» Hourly charge per virtual machine ($0.10 to $1.2 per hour)» Data transfer charge ($0.10 to $0.17 per gigabyte)» Allocated and unused Elastic IP address » Storage using Amazon Elastic Block Store (EBS) » Additional transfer charges using Elastic Load Balancing » Using Amazon's CloudWatch service to monitor your virtual machine » Using Amazon's Elastic Load Balancing which distributes load

among selected virtual machine

Page 8: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Science Clouds

• The Science Clouds project was initiated by the University of Chicago (UC) and the University of Florida (UFL)

(http://workspace.globus.org/clouds/)» EC2-style cloud computing

» members of the scientific community to lease resources for short amounts of time

• The Science Clouds do not require users to directly pay for usage» Verify the person asking for an allocation is indeed a member of the

scientific community

» Ask for a short writeup of the scientific project.

» Based on the project the individual is allocated a small (testing), middle (development), or large (science) hour credit on the Science Clouds.

Page 9: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Objective

• Calculate monthly rainfall frequency in the 21st century» Daily precipitation (P) was calculated

P = RF * 86400» RF = daily rainfall flux

» It was assumed that rainfall occurred on a day when P > 0.254 mm

Page 10: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Climate projection data

• The daily sets of GCM outputs were obtained from the WCRP CMIP3 multi-model database (https://esgcet.llnl.gov:8443).

Page 11: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

System architecture

Page 12: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

System configuration

• Amazon web service EC2 » Small instances for Server and Client images

• Server» Fedora 8» MySQL database server» Network file system (NFS) service

• Clients» Ubuntu 8.10» A script to download a daily climate change dataset from the internet.» The data process program

» using NetCDF file format» search and extract subsets of the original dataset.

Page 13: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Results

• The running time using 10 client instances was about 32 hr.

• Downloading the climate data» Between 4.8 hr and 9.1 hr

» Total time was 66 hr

• Database transaction» Between 15.2 hr and 24.8 hr

» Total time was 209 hr

Page 14: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Running time

Page 15: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

File size and transfer rate

Page 16: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Database transaction

Page 17: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Computation cost (USD)

• CPU: » Downloading/Processing data: (275 hr + 32 hr) x $ 0.1 = $ 30.7

» Processing/Downloading results: 42 hr x $ 0.1 = $ 4.2

• Transfer: » 70 GB x $ 0.1 = $ 7

• Storage: » Climate data storage: 30 GB x $ 0.1 = $ 3

» Database storage: 10 GB x $ 0.1 = $ 1

• Total cost» $ 46

Page 18: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Conclusions

• Cloud computing could provide inexpensive and temporary computing resources to analyse large-scale scientific data for the climate change impact assessment.» In a 10 processor-core configuration, our approach would be up to 10

times faster than the calculation on a single processor core machine.

» The costs for processor core use, data transfer and temporary storage were about $35, $7 and $4, respectively.

• Cloud computing have benefits» Running time

» Local storage resources

» Network resources.

Page 19: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Acknowledgement

• New Zealand’s Foundation for Research,Science and Technology through contract CO2X050, Better Border Biosecurity (B3) (www.b3nz.org).

Page 20: The New Zealand Institute for Plant & Food Research Limited Use of Cloud computing in impact assessment of climate change Kwang Soo Kim and Doug MacKenzie.

The New Zealand Institute for Plant & Food Research Limited

Questions ?

Email: [email protected]