ESA Scientific Archives and Virtual Observatory Systems
Post on 19-Jan-2016
27 Views
Preview:
DESCRIPTION
Transcript
ESA Scientific Archives andVirtual Observatory Systems
Christophe.Arviset@esa.intScience Archives and VO Team
Research and Scientific Support DepartmentESA – ESAC – Madrid, Spain
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 2 Research & Scientific Support Department
European Space Astronomy Centre
ESAC default location for: Science operations,
long history with astronomical missions,
now expanding with solar system missions.
Science archives, Astronomy Planetary
ESA VO activities, ESAC to be the European VO
node for space-based astronomy.
http://www.esa.int/SPECIALS/ESAC/Located near Madrid, Spain
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 3 Research & Scientific Support Department
Archives at ESAC ESAC is the Centre where most of ESA Scientific Directorate’s Scientific
Archives are developed, maintained and operated.
ESAC Science Archives Team is giving support to various projects
ISO Data ArchiveSince December 1998
XMM-Newton Science Archive
Since April 2002
Integral SOC Science Data Archive
Since July 2005
Planetary Science ArchiveGiotto, Mars Express
Rosetta, Venus ExpressSmart-1, Huygens
Since March 2004
Herschel, Planck, GAIA, … in the future
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 4 Research & Scientific Support Department
Archives at ESAC (cont’d) Scientific Archives geared to various users
Scientific Community (public access) PI team and observers (controlled access) Science Operations Team (privilege access)
Common Architecture and Look and Feel Better corporate image for ESA Faster, cheaper, better development Re-use of expertise
User friendly access Eg PSA Mars Map Browser
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 5 Research & Scientific Support Department
Science Archives Team @ ESAC A core Science Archives and VO Team (~15 people) in ESAC
Support many projects (horizontal support vs vertical organization)• ISO, XMM-Newton, Integral, Planetary missions, Herschel, • Virtual Observatory• Soho, Planck, Gaia in the future…
Organized by functions: database, user interface, data distribution, inter-operability
Close collaboration with Archive Scientist (1 per project) Expert of the science field, close to the community Consolidate user requirements Set development priorities Final acceptance tests
Strong re-use of design/system/code between projects
Not responsible for writing data reduction software (pipeline) nor data production
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 6 Research & Scientific Support Department
Code re-use across Archives
Manpower shared between projects Knowledge transfer between projects Small fraction of FTE for “old” projects
All Archives share a common part of the code Same functionalities across projects But still some very specific projects functionalities Faster and reliable development of new projects Easier and cheaper maintenance of existing systems
Same Look & Feel for ESA Science Archives User Interface Powerful and easy to use Better corporate image for ESA Easy learning curve for users
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 7 Research & Scientific Support Department
Data Repository
All Data from all archives are stored on magnetic disks Standard formats (FITS for astronomy, PDS for planetary) Save various levels of data IDA (0.5TB), XSA (3TB), ISDA (6TB), PSA (4TB) Small amount still
Unique disk repository for proprietary and public data Access controlled by the application server
Access to data is immediate Via the Archive User I/F or for the VO Except for on-the-fly reprocessing requests
Clever FTP server using links instead of real files Same disk space for FTP Higher quotas for users
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 8 Research & Scientific Support Department
DataBase
Strong effort of Data Modeling before starting any project
Unique database for all uplink and downlink metadata
All metadata is systematically extracted from uplink AND downlink data Done when products are generated Can be re-run with for new metadata is user reqs change
Sybase for some projects ISO (10GB), XMM-Newton (6GB), Planetary (2GB)
Oracle for others Integral (6GB)
PostGres for future projects (Herschel, Soho, …)
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 9 Research & Scientific Support Department
Existing Access – GUI
Project
Business
Logic
TranslationLayer
Meta Data in
Database
Data Productson hard disks
ProductRequest
Products
GUI Database/Product Request
GUI Database Result / Product
DB/Product Request
DB Result / Product
User friendly web access : Java Applet Powerful queries, organized by panel,
hundreds of queriable and displayable parameters
Public data accessible to all, proprietary data accessible only to privilege users
Images preview (icons, full image) Quick download (1 click) at various processing
levels, shopping basket retrieval Remote access to other archives (ADS, and
tools (Aladin, VOSpec)
Project prior to the VO Existing data model, data format, access
mechanisms
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 10 Research & Scientific Support Department
Existing Access : Scriptable I/F
Project
Business
Logic
TranslationLayer
Meta Data in
Database
Data Productson hard disks
ProductRequest
Products
GUI Database/Product Request
GUI Database Result / Product
DB/Product Request
DB Result / Product
Scriptable
Interface
Project Product Request
Project Productforeach obs in (observation_list) AIOgetfile (obs, file_required) analyse_process file_required save results delete file_requiredend
DB/Product Request
DB Result / Product
Access to metadata (via XML scripting) Access to data through socket (direct data stream) or
FTP (several files or individual file)
Own Interoperability standards and needs as required by the project
Not using VO protocols, but ready for it…
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 11 Research & Scientific Support Department
VO compliant Architecture
Project
Business
Logic
TranslationLayer
Meta Data in
Database
Data Productson hard disks
ProductRequest
Products
GUI Database/Product Request
GUI Database Result / Product
DB/Product Request
DB Result / Product
Project Product Request
Project Product
DB/Product Request
DB Result / Product
SIAP/SSA Request
VOTable / Product
TranslationLayer
Scriptable
Interface
Project Product Request
Project Product
(Scriptable Interface)foreach obs in (observation_list) AIOgetfile (obs, file_required) analyse_process file_required save results delete file_requiredend
DB/Product Request
DB Result / Product
Existing scriptable I/F already carried all the functionalities required by the new VO standards Access to metadata (database) Access to products (data repository)
By using “Translation Layer”, easy to convert the VO standards interfaces into our existing I/F Little effort required Translation Layer easy to code using XML
As VO standards evolve, our scriptable I/F system remains, but we just have to adapt our translation layers
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 12 Research & Scientific Support Department
ESA-VO Main Goals ESA part of the EURO-VO ESA part of the IVOA (International VO Alliance)
RSSD astronomical archives being VO compatible VO in mind when building archives Archives and VO work done by the same team
Develop VO applications eg VOSpec, VOQuest, …
Develop VO publishing services Registry of VO resources, DALToolkit, DMMapper, …
ESAC to disseminate VO technology Link to existing and new ESA astronomy
projects VO outreach (workshop, conferences, …)
http://esavo.esac.esa.int/
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 13 Research & Scientific Support Department
VO in action : VOSpec application
Select Source
Find/Select in the VORegistry
VO Spectral Servers
IUEspectra
ISOspectra
VEGATheoretical
Model
DisplayTheoretical Spectra
and Observational Spectra
IUEspectra
ISOspectra
VEGATheoretical
Model
NormalizeTheoretical
Spectra
IUEspectra
ISOspectra
VEGATheoretical
Model
Theoretical SpectraFits
Observational Spectra
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 14 Research & Scientific Support Department
Interoperability ESA PSA – NASA PDS
(Prototype in development) in the context of the IPDA (International Planetary Data Alliance)
From Mars Map Browser, Select region of interest
Contact PSA and PDS using the common data (image) access protocol
Display NASA PDS and ESA PSA images
Christophe ARVISETScience Archives in the 21st Century, 26/04/2007, page 15 Research & Scientific Support Department
Conclusion
ESA Scientific Archives built and operated within a same team
ESA Archives flexible architecture allows easier and long-term maintenance
ESA strongly believes in the Virtual Observatory, both for astronomy and for planetary missions
Strong link between Archives development and Virtual Observatory work
ESAC is becoming the Data Centre for ESA space-based archives
ESAC is becoming the VO node for European space-based astronomy
top related