Ecogrid update and future plan: from Local, Regional to Global 2005 NSC-NSF Annual Meeting Taipei, 8, July, 2005 Fang-Pang Lin National Center for High-performance.

Ecogrid update and future plan: from Local, Regional to Global

2005 NSC-NSF Annual MeetingTaipei, 8, July, 2005

Fang-Pang LinNational Center for High-performance Computing

National Applied Research Laboratory

Collaboratory for Ecological Research in Taiwan

Fang-Pang LinNational Center for High-performance Computing

National Applied Research Laboratory

22

New Frontier: Merging of Environmental Science and

Information Technology

PreviouslyUnobtainable

Observations andUnderstanding

EnablingTechnology

• Advance science

Science Drivers

• Focus development

PersistentInfrastructure

• Broaden impact

Education & Capacity

Building

• Develop human resources

Sustained Collaboration

• Build teams and trust

Source: Tim Kratz, Peter Arzberger et al.at NSC-NSF 2005 annual meeting

33

“The World is Flat”*

Fundamental changes are underway in how: Researchers interact with each other and with resources Resources are distributed and accessed on the network Science and education are conducted

This is leading to new Research and educational paradigms Discoveries and innovations Networks of people

• Multidisciplinary, multi-institutional, international

Ecology is a global science Demanding global approaches Providing future opportunities for creating previously

unobtainable observations and understanding

* Thomas Friedman

44

“Bandwidth” and “degree of connectivity” are the new measures of power …

• Three distinguishing factors to harness power– Culture to exploit & share knowledge– Competitive setting that embraces change– Ability to partner

Thomas Friedman, New York Times, P.11 11/Apr/1998

55

The plan for the cyber-Infrastructure

Commercial network

Profits

Educational Network Education

Innovation Research Network

Grids

Source: Whey Fone Tsai, 2003

66

77

88Frequency of measurement

Spatial extent

Annual

100 km

Monthly Weekly Daily Hourly Min. Sec.

10 km

1 km

100 m

10 m

1 m

10 cm

Existing Sensor Networks

random selection from Ecology 2003

Source: John Porter et al., Bioscience, July, 2005

Sensor networks allow high frequency observations over broad spatial extents

99

The Vision:

• Nerve ending of Taiwan in terms of Network (Last-Mile for TWAREN)

• Focus on National Ecological Observation for supporting sustainable development of the island.

The Plan ：• 2003: Fushan, Nan-Jen-Shan, Kenting, Tatachia, Guan-Dai-Shi• 2004: Yuan-yan Lake, Remote islands• 2005: Coastal areas and Agricultural areas

The Technology：• Grid-based system design• Hierarchical Network structure. • Hybrid comm. Protocols:

GSM, GPRS, WLAN, P2P, RF• Remote data communication (sensor-based)• Remote control & Grid-based automation (sensors & robots)

TERN sites

KING sites

1010

1111

Date Time CONDMSG

FROMAll

Network transmission status

05/07/08 01:05:30 S GVC "05/07/08 01:05:30", S GVC connected (CONNECT 9600/V42).

05/07/08 01:06:21 S YYL_Met "05/07/08 01:06:21", S YYL_Met connected.

05/07/08 01:06:22 W YYL_Met "05/07/08 01:06:22", W YYL_Met E08 error count: 2.

05/07/08 01:06:26 S YYL_Met "05/07/08 01:06:26", S YYL_Met Area1 at location 16687, collected 504 FS words.

05/07/08 01:06:28 S YYL_Met "05/07/08 01:06:28", S YYL_Met disconnected.

05/07/08 01:35:29 S GVC "05/07/08 01:35:29", S GVC connected (CONNECT 9600/V42).

05/07/08 01:36:26 S YYL_Buoy "05/07/08 01:36:26", S YYL_Buoy connected.

05/07/08 01:36:49 S YYL_Buoy "05/07/08 01:36:49", S YYL_Buoy Area1 at location 58573, collected 1528 FS words.

05/07/08 01:36:50 F YYL_Buoy "05/07/08 01:36:50", F YYL_Buoy area 2 does not exist.

1212

1313

12

14

16

18

20

22

22-Aug 23-Aug 24-Aug 25-Aug 26-Aug 27-Aug 28-Aug

Date

Wat

er T

emp

erat

ure

(°C

)

0

2

4

6

8

10

12

14

16

18

20

Pre

cip

ita

tio

n(m

m p

er 5

min

ute

in

terv

al)

Surface

0.5 meters

1 meter

1.5 meters

2 meters

2.5 meters

3 meters

Precipitation

Typhoon

Yuan Yang Lake, Taiwan – August 2004

Part of a growing global lake observatory network - http://lakemetabolism.org

An example of episodic events and threshold dynamicsAccess can be

difficult during the most interesting times

Photo by Peter Arzberger, October 2004

Used

by N

SF

Directo

r Feb

2005

1414

Plan for Global Lake and Coral Reef Observation

AS/NCHC

Meeting 7-0, Mar 2005 at San Diego supported by NSF, Moore foundation and PRAGMA

10 Lakes: New Zealand, Australia, Canada, US, Korea, UK, Finland,Taiwan4 Coral Reefs sites: Breat Barrier Reef, World Bank Coral Reef efforts, US LTER, NOAA, TW LTER

1515

1616

1717

1st camera

2nd camera

3rd camera

Optical connector

Kenting Ecosite ~2.4G WiFi

Field server

Taipower Admin office

Taipower Exibition Center

1818

Architecture (I)

1919

Architecture (II)

TaiPowerIntranet

Internet

WLAN 802.11b

Optical Fiber

Cat. 5e cable

Outdoor Equipment Box

Video Server, IP =192.168.1.21 (Site A)192.168.1.22 (Site B)192.168.1.23 (Site C)

RPM, IP =192.168.1.11 (Site A)192.168.1.12 (Site B)192.168.1.13 (Site C)

Switch Hub

WLAN Bridge, IP =192.168.1.52 (Site A)192.168.1.54 (Site B)192.168.1.56 (Site C)

WLAN Bridge, IP =192.168.1.51 (to .52)192.168.1.53 (to .54)192.168.1.55 (to .56)

Display Server, IP= 192.168.1.201

Buffer Server, IP= 192.168.1.202

Control Room 2F

Switch Hub

Administration Building

Ethernet Router #1,IP=192.168.1.3 <>

211.22.250.202:8080

Switch Hub

Ethernet Router #2,IP=192.168.1.2 <>

210.241.251.138:8080

ADSL Modem

Electrical-Opticaltransceiver

Electrical-Opticaltransceiver

Network Topology

2020

2121

2222

Basic Planning for IMS

Stakeholders:Managers (Funding/Manpower/Strategy)

Executive Managers (plan/coordination/auditing)

Scientists (e.g. Ecologists)Information ManagersIT Developers

2323

1.     Information Manager (IM) 1.1             IM Definition and Responsibility 1.2             QA/QC 1.3             Training courses 2. Portal 2.1 Basics 2.1.1 Scenarios: Ecologists => login => upload => registration 2.1.3 Data description guidelines 1) Data/Information        composite attributes: numerics->place/time/objectives/people/equipment/owners…etc        cross references (Database layer)        Data Format First row Row should be Definition according to EML EML, Usage tables for cross references between the version of Chinese and that of English. 3. Sensors/equipment 4. Access Points 5. Network           Backbone issues: GSN and TANET2/TWAREN           Last miles: ADSL/GPRS/RF           Local sensor network: Local Sensor Network            RF (~900MHz)            RF (~150MHz) <data modem: Chin>            IEEE 802.11 (2.4GHz 54/11Mbps, 100 meters depending on power)            IEEE 802.16 (WiMAX, 75Mbps, 30 miles)            Others           Hook-up Mechanism           Equipments            Pairing Structure: One to One, One to many            Details listed 6. Data format conversion Excel -> saved as ascii (csv=> mysqls, oracles)

2424

Need to build a project registration

interface

Ecologist• Project registration • Define requirement• Device procurement

Site IM• Create project db schema• Register new schema to

common data interface

IT executive manager• coordinate IT resources

IT Specialist • Design infrastructure• Device procurement• Link new schema to

common interface

YesNo Site IMStream Data?

Site IM, IT Specialist, Ecologist• Test and deploy

Site IM same data collected already?

NO

yes

Site IMOperation

Need to write operation manual Made by Hsiu-Mei, May 2005

2525

GLEON Site Architecture (by Tony Fountain)

Common Interface Layer

DataAcquisition

DataManagement

DataAnalysis and Modeling

DataLogger

Database ModelsModelsModels

Sensor NetMiddleware

Comm. & Ctl.Web Services

Sensor SensorSensor

QA/QCAgent

Data-relatedWeb Services

AnalyticalWeb Services

. . .. . .

The components and the relationships are only tentative to start the discussion.

2626

GLEON Network Architecture (by Tony Fountain)

Common Interface Layer

DataAcquisition

DataManagement

DataAnalysis and Modeling

DataLogger

Database ModelsModelsModels

Sensor NetMiddleware

Comm. & Ctl.Web Services

Sensor SensorSensor

QA/QCAgent

Data-relatedWeb Services

AnalyticalWeb Services

. . .. . .

Common Interface Layer

DataAcquisition

DataManagement

DataAnalysis and Modeling

DataLogger

Database ModelsModelsModels

Sensor NetMiddleware

Comm. & Ctl.Web Services

Sensor SensorSensor

QA/QCAgent

Data-relatedWeb Services

AnalyticalWeb Services

. . .. . .

Common Interface Layer

DataAcquisition

DataManagement

DataAnalysis and Modeling

DataLogger

Database ModelsModelsModels

Sensor NetMiddleware

Comm. & Ctl.Web Services

Sensor SensorSensor

QA/QCAgent

Data-relatedWeb Services

AnalyticalWeb Services

. . .. . .

Portal

Lake Site 1

Lake Site 2

Lake Site 3

Desktop & Mobile Users

Note that each site implements the Interface Layer, but the actual site architectures are independent and site-specific.

2727

2828

Fishery Observatory in Open Ocean (Fishery sites in plan: Pintong, Ilan and/or Penhu)

Fishery Observatory in Open Ocean (Fishery sites in plan: Pintong, Ilan and/or Penhu)

(After Tim Acker et. Al., BioSonics Inc.)

Wireless network

antenna

antenna

Fishery Platform/Observatory

in Ocean

Production management system

Observational servers

Source: Chi-Yuan Lin, TFRIN

2929(2005.03.20, photo by 周秀美 , NCHC)

3030

Sensor Grid Portal (T.H.Chen and C.K. Sun, NCHC)

Provide Basics of a General Grid Platform and allow further development into a domain specific platform, such as IM for ecological research

Need a component which is aware of the status of resources in the grid environment, and be able to dispatch/redirect jobs to the most appropriate resources,

Integrate computing resource and storage resource using Grid middleware:

Computing middleware Storage middleware

The ultimate goal is to have a complete grid platform and a user portal, so that NCHC could maximize the value of its hardware and knowledge center, enable end users to access various NCHC grid resources through the single entry, with user friendly Web interfaces.

Portal Server

3131

Solutions Part of Computing

Use Globus as computing middleware Provides both Web Services (utilizing Axis engine) and Java APIs as

the interfaces for job submission and other functions. Utilize a RDB (MySQL) to store Meta Scheduler data. Provides design information for the “MetaScheduler”.

Part of Storage Use SRB as storage middleware SRB client and the Data mover component Authentication and authorization

Part of Portal Server LifeRay Portal server Portal user management and user profile integration with Globus a

nd SRB. Integrate with GT2 CoG Integrate with SRB JARGON

3232

NCHC Portal Architecture

3333

Configuration Hardware environment and Software function

Service machine 2

SRB Server

Globus serverSimple CA

SRB UtilitiesGRIS

Service machine 1

Grid PortalMeta-scheduler

Globus serverSRB UtilitiesCertificate

GIIS

TCP/IP Network

Other Grid service machines

Globus serverSRB UtilitiesCertificate

GRIS

Other Grid service machine

Globus serverSRB UtilitiesCertificate

GRIS

3434