ESA Archives and VO activities - obspm.frEuropean Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) ESA and the VO, page 2 Christophe Arviset IVOA SPM, Moscow,

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 1

IVOA SPM, Moscow, 21/09/2006

ESA Archives and VO activities

http://esavo.esac.esa.int/

Christophe ArvisetESA-VO, Spain

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 2

IVOA SPM, Moscow, 21/09/2006

Presentation Overview

ESA Archives at ESAC

ESA VO Project

Some ESA VO Tools

Conclusion

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 3

IVOA SPM, Moscow, 21/09/2006

ESACEuropean Space Astronomy Centre• Located near Madrid,

Spain

ESAC default location for:• science operations,

• long history with astronomical missions, starting with IUE,

• now expanding with solar system missions.

• science archives

ESAC to be the main European VO node for space-based astronomy http://www.esa.int/SPECIALS/ESAC/

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 4

IVOA SPM, Moscow, 21/09/2006

Astronomy Archives at ESACISO Data Archive,

• Available since Dec. 1998 : http://iso.esac.esa.int/ida/• Last main version, IDA 10.0 on 29/06/2006

XMM-Newton Science Archive• Available since April 2002: http://xmm.esac.esa.int/xsa/• Active development, XSA 3.5 on 28/08/2006

Integral Science Data Archive• Available internally since Dec. 2003, • Open publicly since July 2005 , http://integral.esac.esa.int/isda/• Active development, ISDA 2.5.2 on 27/06/2006

Herschel, Planck, GAIA, … to come in the future

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 5

IVOA SPM, Moscow, 21/09/2006

Same Look & Feel for ESA Science Archives

Similar User Interface• Powerful and easy to

use• Panels aimed at

general users and expert users

Same functionalities across projects

Better corporate image for ESA

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 6

IVOA SPM, Moscow, 21/09/2006

ESA Planetary Science Archive

All Planetary missions into a single archive (PSA)• Available since March 2004, http://www.rssd.esa.int/PSA• Active development, PSA 2.7 on 01/08/2006• Mars Express, Smart-1, Giotto, Huygens, Rosetta data• Venus Express, BepiColombo and future missions to be included in

the future

Also based on ESA astronomy archive infrastructure

SOHO Archive to be developed in 2006/2007

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 7

IVOA SPM, Moscow, 21/09/2006

PSA User Interfaces

Classical User Interface Map Based Interface (MEX)

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 8

IVOA SPM, Moscow, 21/09/2006

One Team, Several ProjectsA core Science Archives and VO Team (~14 people) in ESAC• Support many projects (horizontal support vs vertical organization)• Organized by functions: database, user interface, data distribution,

inter-operability

Advantages• Manpower shared between projects, small fraction for old projects• Knowledge transfer between projects• Re-use of existing systems for new projects• Easier and cheaper maintenance of existing systems• Ensure for long term preservation of data holdings

1 Archive Scientist per project to consolidate User requirements• Expert of the science field, Close to the community

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 9

IVOA SPM, Moscow, 21/09/2006

One Team, Several ProjectsNot responsible for data pipeline development, nor for the data quality

Responsible for Archive “accessibility tools” and VO access

Main people involved (so far) in the VO• Christophe Arviset (team leader)• Pedro Osuna (ESA VO Technical

Coordinator)• Matteo Guainazzi (ESA VO Scientist)• Jesus Salgado, Inaki Ortiz, Aurelien

Stebe, Isa Barbarisi

Other people working mainly on archives

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 10

IVOA SPM, Moscow, 21/09/2006

Existing Access – GUI

Proj

ect

Busine

ss L

ogic

TranslationLayer

Meta Data in Database

Data Productson hard disks

ProductRequestProducts

GUI Database/Product Request

GUI Database Result / Product

DB/Product Request

DB Result / Product

User friendly web access : Java Applet Powerful queries, organized by panel, hundreds of queriable and displayable parametersPublic data accessible to all, proprietary data accessible only to privilege usersImages preview (icons, full image)Quick download (1 click) at various processing levels, shopping basket retrievalRemote access to other archives (ADS, and tools (Aladin, VOSpec)

Project prior to the VO• Existing data model, data format, access

mechanisms

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 11

IVOA SPM, Moscow, 21/09/2006

Existing Access : Scriptable I/F

Proj

ect

Busine

ss L

ogic

TranslationLayer

Meta Data in Database

Data Productson hard disks

ProductRequestProducts

GUI Database/Product Request

GUI Database Result / Product

DB/Product Request

DB Result / Product

Script

able

Inte

rfac

e

Project Product Request

Project Productforeach obs in (observation_list)

AIOgetfile (obs, file_required)analyse_process file_requiredsave resultsdelete file_required

end

DB/Product Request

DB Result / Product

Access to metadata (via XML scripting)Access to data through socket (direct data stream) or FTP (several files or individual file)

Own Interoperability standards and needs as required by the projectNot using VO protocols, but ready for it…

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 12

IVOA SPM, Moscow, 21/09/2006

VO compliant Architecture

Proj

ect

Busine

ss L

ogic

TranslationLayer

Meta Data in Database

Data Productson hard disks

ProductRequestProducts

GUI Database/Product Request

GUI Database Result / Product

DB/Product Request

DB Result / Product

Project Product Request

Project Product

DB/Product Request

DB Result / Product

SIAP/SSA Request

VOTable / Product

TranslationLayer

Script

able

Inte

rfac

e

Project Product Request

Project Product

(Scriptable Interface)foreach obs in (observation_list)

AIOgetfile (obs, file_required)analyse_process file_requiredsave resultsdelete file_required

end

DB/Product Request

DB Result / Product

Existing scriptable I/F already carried all the functionalities required by the new VO standards• Access to metadata (database)• Access to products (data repository)

By using “Translation Layer”, easy to convert the VO standards interfaces into our existing I/F• Little effort required• Translation Layer easy to code using XML

As VO standards evolve, our scriptable I/F system remains, but we just have to adapt our translation layers

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 13

IVOA SPM, Moscow, 21/09/2006

Presentation Overview

ESA Archives at ESAC

ESA VO Project

Some ESA VO Tools

Conclusion

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 14

IVOA SPM, Moscow, 21/09/2006

ESA VO Main ObjectivesESAC to be the main European VO node for space-based astronomy • Link to all the astronomy projects at ESAC (ISO, XMM-Newton,

Integral, Herschel, Planck, Gaia, …)

All ESAC Archives to be VO-compliant

Development of VO-compliant tools:• VOSpec, VOQuest• Registry, DMMapper, DALtoolkit, …

Active member of the EURO-VO and IVOA

Science research through the VO

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 15

IVOA SPM, Moscow, 21/09/2006

ESA is an active member of EURO-VO

Member of EURO-VO Data Centre Alliance• Development of the ESAC Astronomical Archives• Make these archives accessible through VO standards• VO Support to other European Data Centre• Data Processing Centre, through the ESA-VO GRID

• Processing the data where they reside, instead of moving the data around

Member of EURO-VO Technology Centre• Technology research and prototyping on VO standards• Link with VOTECH project• Link with IVOA corresponding working groups

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 16

IVOA SPM, Moscow, 21/09/2006

EURO-VO Facility CentreFunded by ESA and ESO

WebPortal, user friendly for Scientists• VO tools, User recipes and User manuals, …

WebPortal, user friendly for Technical people• VO development tools, tutorials, links to VO

standards, …

Support for VO technology take-up in Europe• Scientific and technical VO Workshops

Registry of VO resourceshttp://www.euro-vo.org/

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 17

IVOA SPM, Moscow, 21/09/2006

VO Recipes

http://www.euro-vo.org/pub/fc/recipes.html

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 18

IVOA SPM, Moscow, 21/09/2006

Presentation Overview

ESA Archives at ESAC

ESA VO Project

Some ESA VO Tools

Conclusion

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 19

IVOA SPM, Moscow, 21/09/2006

ESA VO RegistryFull Searchable Registry • implemented in Java with Sybase RDBMS• Being upgraded to IVOA RI 1.0 specs

Fully compliant OAI interface for harvesting

Services accessible by Web Service, HTTP POST or GET

User friendly Interface with Web Pages for Search and Insert/Update

http://esavo.esac.esa.int/registry

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 20

IVOA SPM, Moscow, 21/09/2006

VOSpecA tool to retrieve, display, manipulate spectra and lines coming from various VO resources

Main functionalities:• automatic conversion of units via dimensional

equation analysis• Fitting, de-redening capabilities• interface with theoretical models• interface with atomic and molecular databases

and on-the-fly identification on SEDs• Some more analysis functions

http://esavo.esac.esa.int/vospec

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 21

IVOA SPM, Moscow, 21/09/2006

VOSpec : Real and Theoretical Spectra

Select Source

Find/Select in the VORegistry

VO Spectral Servers

IUEspectra

ISOspectra

VEGATheoretical

Model

DisplayTheoretical Spectra

and Observational Spectra

IUEspectra

ISOspectra

VEGATheoretical

Model

NormalizeTheoretical

Spectra

IUEspectra

ISOspectra

VEGATheoretical

Model

Theoretical SpectraFits

Observational Spectra

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 22

IVOA SPM, Moscow, 21/09/2006

VOSpec and Line AccessVOSpec implements the SLAP (Simple Line Access Protocol) VO standard, which allows to publish atomic and molecular line databases in the VO

This means that – for these databases – VOSpec allows you to:• List the atomic and molecular transitions which may occur in a given

wavelength band• Superpose the closest atomic or molecular transition to any line

features in an observed spectrum displayed with VOSpec

Demo

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 23

IVOA SPM, Moscow, 21/09/2006

VOQuest (=== Still Prototype ===)

VOQuest allows you to search for catalogues, which support certain type of queries, such as:• Coordinates, Fluxes, Photometry• Still a prototype, interface to be improved

The same ADQL query is sent to several VO-compliant catalogue servers and query results come back in the same format• Using Source Catalogue Data Model

Further actions are then feasible using other VO tools• Plotting, column manipulation, X-matching with

TOPCAT(Astrogrid)

See talk at IVOA @ Victoria• http://www.ivoa.net/internal/IVOA/InterOpMay2006Applications/VOQuest.pdf

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 24

IVOA SPM, Moscow, 21/09/2006

DAL ToolkitPurpose : help astronomers to easily publish their data through VO protocols

DALingestor• Takes FITS spectra as inputs• Create necessary metadata for SSA• In the future, will also support images

(SIAP) and lines (SLAP)

DALserver• Publish data through SSAP, SIAP,

SLAP

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 25

IVOA SPM, Moscow, 21/09/2006

ConclusionsESAC is a major Data Centre and its goal is to become the VO node in Europe for space astronomical data• ISO, XMM-Newton, Integral, Herschel, Planck, Gaia, …

ESA is putting a significant effort into the VO• At a European (EURO-VO) and worldwide level (IVOA)

VO tools developed at ESA• ESA VO Registry – a registry of VO resources• VOSpec – a tool to consume SSAP, TSAP, SLAP aware services • VOQuest – a tool to consume Source Catalog DM aware services• DALToolkit – a toolkit to publish basic resources through SIAP, SSAP,

SLAP

European Space Astronomy Centre (ESAC)Villafranca del Castillo, MADRID (SPAIN)Christophe ArvisetESA and the VO, page 26

IVOA SPM, Moscow, 21/09/2006

Acknowledgements

ESAC Science Archives and VO Team