Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002) 1 An Architecture for Access To A Compute Intensive Image Mosaic Service in the NVO G. B. Berriman, D. Curkendall, J. Good, J. Jacob, D.S. Katz, M. Kong, S. Monkewitz, R. Moore, T.Prince, R. Williams
12
Embed
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)1 An Architecture for Access To A Compute Intensive Image Mosaic Service.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
1
An Architecture for Access To A Compute Intensive Image Mosaic
Service in the NVO
G. B. Berriman, D. Curkendall, J. Good, J. Jacob,D.S. Katz, M. Kong, S. Monkewitz, R. Moore, T.Prince,
R. Williams
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
2San Diego Supercomputer Center
Compute Resources Catalogs Data Archives
InformationDiscovery
Metadatadelivery
Data Discovery
Data Delivery
Catalog Mediator Data mediator
1. Portals and Workbenches
Bulk DataAnalysis
CatalogAnalysis
MetadataView
DataView
4.GridSecurityCachingReplicationBackupScheduling
2.Knowledge & ResourceManagement
Standard Metadata format, Data model, Wire format
Catalog/Image Specific Access
Standard APIs and Protocols Concept space
3.
5.
6.
7. Derived Collections
National Virtual ObservatoryData Grid
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
3
Montage - Custom Image Mosaics
http://montage.ipac.caltech.edu
• User specified size, WCS projection, coordinates, spatial sampling, rotation
• Rectification of backgrounds in images
• Supports drizzle algorithm
Delivery:• Semi-annual deliveries from Feb 2003 • Final Delivery Jan 2005• Available for download
Science Drivers:• Science Grade Images• Analyze diverse images as if part of same “multi-wavelength image”
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
In p u t p ixelsp r o jected o nceles tia l s p h er e
O u tp u t p ixelsp r o jected o nceles tia l s p h er e
To ta l F lu x
S k y A re a C o v e ra g e(s te ra d ia n s )
M o n ta g e R ep ro jec tio n M o d u le
C en tra l to th e a lg o rith m is a ccu ra te ca lcu la tio no f th e a rea o f sp h erica l p o lyg o n in tersectio nb etw een tw o p ixels (a ssu m es g rea t circle seg m en tsa re a d eq u a te b etw een p ixel v ertices)
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
5
C o o rdina te d E xe c utio n A c ro s sM ultiple G R ID P ro c e s s o rs
Inp utIm ages
S imp le c o a d d f o r f in a l mo sa ic(sk y a re a s u se d a s w e ig h ts )
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
6
M o n ta g e B a c k g ro u n d C o rre c tio n Pro c e d u re
A co rre ctio n is ca lcu la te d fo r e a ch im a g e b a se d o n a ll th e d iffe re n ce s b e tw e e n it a n d itsn e ig h b o rs (a n a p p ro x im a tio n to a le a s t sq u a re s fi t to th e d if fe re n ce d a ta w ith b r ig h tn e s s o u tlie rp ix e ls e x clu d e d ). Th e co r re ctio n is cu r re n tly a p la n e b u t co u ld b e a h ig h e r o rd e r su r fa ce .
Th is is d o n e fo r a ll im a g e s , th e n h a lf th e co r re ctio n d e te rm in e d is a p p lie d (to a p a ra m e te rd a ta b a se ; e q u iv a le n t n u m e r ica lly to a p p ly in g it to th e im a g e s ).
Th e p ro ce s s is i te ra te d u n til s te p d iffe re n ce s fo r a ll im a g e s b e co m e s sm a ll.
D iffe r e n ce s in o v e r la p a r e a s
E xam p le: T h ree overla p p in grep rojected 2 M AS S im a g es
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
7
M o n ta g e B a c k g ro u n d C o rre c tio n R e su lts
R ep rojectedB a ckg rou n d C orrectedIm a g es
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
8
Deployment of Montage
Performance Goal:Sustain a throughput of at least 30 square degrees per min on 1024 x 400 Mhz R12K O3000 or equivalent
Deploy operationally on the Teragrid by January 2005
Teragrid will be used by NVO for compute intensive services
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
9
Managing Requests For Compute Intensive Services
•Access compute intensive service or bulk data delivery service from existing clients • Apache has no memory of requests, and so little control of multiple time intensive requests:
• restart automatically after failed request• stop large requests as needed• monitoring of requests• load balancing
Solution:Request Object Management Environment
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
10
• Application of Enterprise Java Bean e-business technology• Used everyday by e.g. banks to manage secure financial transactions• Mature technology based on specialized Java Virtual Machine, the EJB server• EJB’s manage and persist transactions, perform load balancing, handle security
• Middleware that sits between client and processor whose role is straightforward
•Accepts requests from clients through standard interfaces•Registers them in database•Sends them for processing•Capable of managing very large numbers of requests
What Is ROME?
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
11
MONTAGE ProcessingModules
RequestDB
ROME
GRIDClient
Application
Image Cacheand
Catalog
Requests
CustomClients
BrowserForms
Clients
ImageSurveyData
ImageReprojection
ImageReprojection
ImageReprojection
ImageReprojection
ImageReprojection
ImageReprojection
ImageReprojection
BackgroundModelling
Web Services
SurveyMetadata
RequestManagement
ROMEMONTAGEInteraction
Virtual Observatories, Astronomical Telescopes and Instrumentation (August 2002)
12
RE Q UE S T M A NA G E R (A P P L ICA T IO N S E RV E R)
JV M
RE Q UE S T P RO CE S S O R
JV M
RE Q UE S T P RO CE S S O R
JV M
RE Q UE S T P RO CE S S O R
JV M
W orker ThreadW orker Thread
Reques tUpdate
W orker Threads
ApplicationApplicationApplication
s tdin s tdout
o p ti o n a l m e ssa g e so cke t
Status
Update
Reques t
In terruptEx ec
Reques t Mes s ages
Us erRegis tration
Us er
GetReques t
CurrentReques t
A pplic ationMes s age
Mes s age
Ma in
Proc es s orRegis tration
P ro ce sso r
R O M E R eq u est O b ject M a n a g em en t E n viron m en t