Prologue Major VO objective: “Data at your fingertips” (Lawrence) VO-user’s desire: “going backstage” –Data quality assessment by user (talk Gilmore)

Prologue

• Major VO objective: “Data at your fingertips” (Lawrence)

• VO-user’s desire: “going backstage”– Data quality assessment by user (talk Gilmore)

• “how did the data-provider arrive at its final result exactly?”

– Diamond-in-haystack searches • Upper-limits, swamping by false positives, homogenizing psf-

dependent photometry (talk Kuijken)

– Statistics of large datasets• Interaction with raw/intermediate data (e.g., weak lensing:

Kuijken)

– Comparing different routes of analysis (talk Wolk)

The Astro-Wise systemA Federated Information Accumulator for Astronomy

Gijs Verdoes Kleijn OmegaCEN, Kapteyn Groningen, NL

for the

Sternwarte(+MPE)Munich

TerapixParis

OAC Naples

Astro-Wise Consortium

Kapteyn InstituteGroningen

Coo

rdin

ator

(National WideField data centers D,F,I,NL and ESO)

The Astro-Wise systemA Federated Information Accumulator for Astronomy

Gijs Verdoes Kleijn OmegaCEN, Kapteyn Groningen, NL

for the

Astro-Wise Consortium

• Has built the Astro-Wise system: – combining {archiving+processing+scientific analysis} of optical wide-field imaging– EU FP5 RTD 5 year programme (parallel to AVO) – Funding dec 2006, other funding until 2009

• And has started using it for science

Motivation Astro-Wise project

• Wide Field Imaging now and future :• NOAO-MOSAICI, CTIO-MOSAICII, ESO-WFI/VIMOS, Magellan-IMACS,

LaPalma-WFC, CFHT-MegaCAM, Subaru-SuprimeCAM,…, VISTA, OmegaCAM, Pan-Starrs, LSST,…

• Optimize scientific return from Wide Field Imaging: • Facilitate (un)forseen uses of calibrated and raw Tera/Petabyte data sets by

global pools of collaborators and resources• This requires enabling changes at any time in:

1. Physical parameters (e.g., instrument/atmosphere)2. Improved algorithms3. Improved insight in 1. or 2.

• And requires a single integrated information system

Quality controlQuality controlUser tuned researchUser tuned research

Archiving (raw+final)Archiving (raw+final)Dissemminating resultsDissemminating resultsCalibrationCalibrationIntegrated Information systemIntegrated Information system

The Astro-Wise approach:• Objects with

– Attributes– Methods

The Astro-Wise approach:• Objects with

– Attributes– Methods

• Link all dependencies of objects

The Astro-Wise Environment

CPUs

DATABASE•LINKS•METADATA images•CATALOGSContains “ALL input/output”

DATA SERVER:•Images:

•Calibration & science•Raw & final

Methods/AlgorithmsIn on-line repository

Python (wrapped)

Benefits from integrated dynamic database relevant for VO

• Both final and raw products accessible• Objects are linked, all bits are traced:

– on-the fly re-processing– Built–in workflow

• Easy publishing for VO (to be implemented) – Public data already viewable via web tools

• Own compute GRID operational• A federated integrated-system:

– Naturally serves global collaborative projects – Scalable: image server/database/compute GRID resources can grow via federation

• Mydb environment operational for individual research• Quick scripting: many tasks are “5 Line Scripts”• Fully user tunable – own provided algorithms/scripts allowed• Python: existing software can be wrapped (implemented for Sextractor, swarp,

being implemented for GALPHOT, GALFIT)

A flavor of available tools• Data processing: web & command-line

A flavor of available tools

A flavor of available tools

• Cut-out server

Status

• Astro-Wise system operational in :Groningen, Leiden, Nijmegen, Munich, Bonn, Bochum, Paris, Napoli, Santiago

• Astro-Wise will serve:– OmegaCAM surveys

• KIDS (Kuijken)• VESUVIO (Galaxy cluster imaging)• OmegaTrans (planet transits)• OmegaWhite (close binary stars)

• And plans to support– MegaCAM@CFHT, – Coma legacy Survey ACS– LOFAR

Prologue revisited

• Major VO objective: “Data at your fingertips” (Lawrence)

• VO-user’s desire: “going backstage”– Data quality assessment by user

• “how did the data-provider arrive at its final result exactly?”

– Diamond-in-haystack searches • Upper-limits, swamping by false positives, homogenizing psf-

dependent photometry (Gilmore, Kuijken)

– Statistics of large datasets• Interaction with raw/intermediate data (e.g., weak lensing:

Kuijken)

– Comparing different routes of analysis (Wolk)

SummaryAstro-Wise: a federated information accumulator for astronomy

• “Raw and final data at your fingertips”– traces every bit of final data back to raw data bits

• Raw data are sacred– processed data are derivatives continuously adding

information to the system • The system is scalable: can be done via

federationFind me at IVOA stand to see Astro-Wise working

or

Contact me for personal demoor

Visit www.astro-wise.orgor

Next astro-wise workshop20-23 November 2006