ICSTI Workshop, 2012/03/05, F. Genova The astronomical Virtual Observatory
Jan 08, 2016
ICSTI Workshop, 2012/03/05, F. Genova
The astronomical Virtual Observatory
ICSTI Workshop, 2012/03/05, F. Genova
Sharing astronomical data : why
• Major scientific objectives– Long term observations of variable natural phenomena– A large number of objects, complex interactions, many scales
• Observations with different techniques, at different scales (ground- and space-based observatories, large surveys)Multi-wavelength observations make a significant and increasing
fraction of publications• Re-using data for scientific objectives different from the
original ones, i.e. optimize the science return of large ground- and space-based instruments and of large surveys
IUE (1978-1996): five times more publications from data retrieved in the archive than from the selected observing teams (Wamsteker, Griffin, 1995) – a major precursor
ICSTI Workshop, 2012/03/05, F. Genova
Astronomy data landscape
• On-line information and services: heterogeneous, distributedobservatory archives, value-added services and tools (CDS), electronic journals and the NASA ADS bibliographic database, modelling data, tools, etc
• Lots of work behind the scene: re-using data requires that–it is properly described–users trust its QUALITY
• On-line services are widely used by the astronomical community
ICSTI Workshop, 2012/03/05, F. Genova
Data policy• Based on data sharing• Observational data obtained through competitive process,
available to all in observatory archives (after a 1 year proprietary period)
• Academic journals– A small number of major journals often managed by Astronomical
Societies or international agreement (Astronomy & Astrophysics, 25 member countries from Europe and South America)
– Table of contents and abstracts freely available, as well is ‘long’ tables at CDS (« attached data »)
– Open access after 1-3 years, immediate open access to a significant part of A&A
– Wide use of arXiv e-print archive
ICSTI Workshop, 2012/03/05, F. Genova
The VO: We did not start from nothingA network of on-line services
• The community is used to define exchange standards in international partnership – More than 30 years ago: FITS (observational data)
– Since 1983: bibcode (1999A&A…447…89T)
• Even before the Web: remote query to services• Since 1993, implementation and networking of
on-line services covering the whole range of ‘data’ – in particular for bibliographic information
A unified access to tables published in journals (CDS & journals)
• Common description– Improved quality: Checks
complementary to the referee’s– Data discovery: VO-enabled content
description– Data retrieval
• Also more general ‘additional data ’
ICSTI Workshop, 2012/03/05, F. Genova
http://vizier.u-strasbg.fr/viz-bin/vizHelp?cats/cats.htx
ICSTI Workshop, 2012/03/05, F. Genova
The astronomical Virtual Observatory• Enable seamless access to the wealth of on-line astronomy
resourcesall the world’s astronomical data should feel like they sit on the astronomer’s desktop workspaceAn ambitious goal and no pre-existing organisational model to follow (~2000)
• Pragmatic approach with a few basic principles– A global VO– Always keep in mind science usage and implementation by data centres– Fulfil astronomy’s needs (disciplinary VRE) BUT when possible use
generic building blocks to allow wider interoperability
• Global interoperability requires international agreementThe definition of interoperability standards isoverseen by the International Virtual Observatory Alliance
ICSTI Workshop, 2012/03/05, F. Genova
The IVOA
• Precursor: a Work Package of the European OPTICON network led by CDS (international already)
• IVOA created in 2002• Alliance of VO projects• Procedure adapted from W3C
(WD, PR, REC)• Working Groups and interest
groups on the different aspects • Now in implementation phase but
maintenance and new topics require sustainability
• Any data centre can join by implementing the interoperability layer
http://www.ivoa.net/Documents/
VO architecture
ICSTI Workshop, 2012/03/05, F. Genova
http://www.ivoa.net/Documents/Notes/IVOAArchitecture/index.html
Interoperability: current status
Continuing to work on standards remains mandatory–Feedback from implementation and scientific usage–Evolution of astronomy – new facilities, new science–Evolution of technology
From C. Arviset
Passage to maintenancemode for many standards
VO-enabled access to VizieR (in blue)
VO evolution• The VO has never been solely a technology
development• Scientists and data providers have been participating
from the beginning in the VO development• Things had to be made in the proper order
– The basic building blocks (standards and tools) had to be – and have been – built, with in mind take-up by data centres and science users
– Now towards operational phase – The focus is moving towards more support to take-up
by scientists and data providers, plus outreach towards education
Science requirements
• Science requirements have been present from the beginning– Scientists in VO projects– Science Advisory Committees of individual projects– Science demos
• When the basic standards have been developed, IVOA set up a Committee, then a Standing Committee for Science Priorities to identify high priority science cases, then performs a gap analysis to identify the lacking standards
ICSTI Workshop, 2012/03/05, F. Genova
Impact on Data Providers:The Euro-VO census
• Census of European Data Centres (EuroVO-DCA, EuroVO-AIDA, 2009, 2010)
• Inclusive definition : Data Centres populate the VO with data and services, service to the community, added-value, sustainability, quality
• 69 ‘data centres’ answeredData archives, services, theory data and services
Data centres in Europe (and elsewhere!)• A huge diversity in aims
– large archives of ground- and space-based telescopes provided by national or international agencies
– large systematic surveys of the sky, results of large simulations
– generalist data bases and services– smaller contributions of scientific teams which
share their expertise• Huge diversity in size and organisations• An ecosystem of data and service providers
willing to share data and knowledge - a distributed, heterogeneous system with no a central point nor hierarchical organisation
Strands of work during operational phase• Support to take-up by data providers• Support to take-up by the scientific community• Continuous technical development
– Standards (update of existing standards and new standards because of feedback/evolutions) – VO teams + IVOA
– Tools• Outreach towards education and the general
public • Interdisciplinarity a must in the current
« political » context
Interdisciplinary aspects
• IVOA had in mind to use generic components when possible. e.g. for two critical components for « wide » interoperability– Registry of Resources: OAI-PMH, Dublin Core (with
extensions)– Vocabulary: RDF + SKOS (semantic web)
• Re-use/adaptation by other disciplines: pragmatic approach through dissemination of knowledge by knowledgeable staff in EU projects in « nearby » disciplines (HELIO et al., VAMDC)
ICSTI Workshop, 2012/03/05, F. Genova
The astronomy knowledge infrastructure• Science driven information network widely used by the
scientific community• A model based on open access to data and services
(pragmatic open access strategy)• A fully distributed model
– Agencies responsible for large infrastructures provide data archives
– Established data centres provide value-added services and tools– Now smaller, motivated actors are appearing – labs willing to
share their knowledge• Links, portals, access tools• Mid-term sustainability
– Support to archive/data centres– Support to national projects which work on interoperability and
tools– Support to the International (and European) layers
ICSTI Workshop, 2012/03/05, F. Genova
European strategic exercisefor astronomy:
Astronet Roadmap (2008)
The data/service infrastructure isan important part of the disciplinary infrastructure
Riding the Wave report of the EU High Level Expert Group
on Scientific Data
Data is an infrastructureCollaborative Data Infrastructure
European strategies
ICSTI Workshop, 2012/03/05, F. Genova
Conclusion
• Astronomy : a global, heterogeneous, distributed, open data infrastructure widely used by the scientific community
• Multipolar, with no central point• Open to large organisations as well as to labs willing to share
their knowledge
• The sharing of knowledge is a global challenge• No single model – the way disciplines organize themselves to
share their data is strongly dependent on their culture• Any Collaborative Data Infrastructure should be able to
accommodate disciplinary « pillars » when they exist