Google Sky

Google Sky

VO-DAS, Uniform Resolution for Astronomical Data Access

Chenzhou CUIChinese Virtual Observatory Project

National Astronomical Observatory of China

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Contents

• Brief introduction of Virtual Observatory• Data access in Virtual Observatory• VO-DAS architecture• VO-DAS use cases• Prospects and concerns

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

VO concept

• Virtual Observatory (VO) is a data-intensively online astronomical research and education environment, taking advantages of advanced information technologies to achieve seamless, global access to astronomical information.

• The power of the World Wide Web is its transparency. It is as if all the documents in the world are inside your PC. The idea of the Virtual Observatory is to achieve the same transparency for astronomical data and other related information (Quinn et al. 2004).

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

IVOA in Growth• International Virtual Observatory Alliance (IVOA) • formed in June 2002• with a mission to facilitate the international

coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory.

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

IVO, Global Astronomy GRID

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Data Access in the VO• The task of the IVOA Data Access working

groups is to define and formulate VO standards for remote data access.

• Client data analysis software will use these services to access data via the VO framework; data providers will implement these services to publish data to the VO.

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Online Astronomical Resources

• Catalogs (tabular data, database tables)– CDS VizieR, SDSS DAS, ...

• Image archives– NASA Centers, CDS Aladin, SDSS, DSS, …

• Spectrum archives– SDSS, LAMOST, …

• Bibliographic archives– ADS, arXiv::astro-ph, …

• Information services• Software, services, etc

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

IVOA Data Access Specifications

• Simple Cone Search • Simple Image Access • Simple Spectral Access• IVOA Astronomical Data Query Language • IVOA SkyNode Interface• VOTable Format Specification • VOSpace service specification • …

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

VO Data Access System (VO-DAS)

• An OGSA-DAI based service system to provide uniform access to astronomy data, including catalogs, images and spectra.• Goals of VO-DAS

– Supporting high volume data query– Interlinking distributed and heterogeneous archives

• catalogs, images, spectrums– Providing a software that works for astronomersworks for astronomers

• OGSA-DAI Supported Asynchronous Data Access Service– OGSA-DAI is a middleware product which supports the exposure of data resources, such as relational or XML databases, on to grids.– Work with Globus Toolkit 4– WSRF interface– Good wrapper for variant databases– Asynchronous query supported – Available for large dataset

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

VO-DAS Architecture

• VO-DAS Server– WSRF based web service– Integrates OGSA-DAI services– Job controlling and monitoring

• DataNode– OGSA-DAI wrapped data resources

• Interfaces– Resource Metadata Interface– Data Query Interface– Data Delivery Interface– Data Analysis Interface– Management Interface

Data Node

VO-DAS Server

Interfaces

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

VO-DAS Use Cases• Query Workflow

– Synchronous Query– Asynchronous Query– Asynchronous Query on multiple DataNodes

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Asynchronous Query on Multiple DataNodes

Interoperability

• Resource Registry: VO Registry• Archive Integration: OGSA-DAI WSRF• Query Language: ADQL and extension• Data Format: VOTable, ASCII, CSV, FITS

etc.• Storage: FTP, (VOSpace?)• Interfaces: ConeSearch, SIA, SSA, etc.

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Status• An Alpha version of VO-DAS was ready

– Simple ADQL access by a GUI java client– ConeSearch and XMatch between multiple OGSA-DAI services– VOTable, ASCII, CSV data format– Simple job controlling and monitoring– FTP storage server– Automatic resource discovery on NVO Registry

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Performance

• High Volume Data Query Test– Long time to query and retrieve big data file

• Frequently Data Query Test– Multiple user simultaneously query– A user frequently access data from server

• Conclusion– A million rows can be handled successfully

• Depend on memory size– It is not stable when simultaneously query (high loaded case)

• Reason: OGSA-DAI’s defect? Or database problem?– It is complicated to handle failure

Science Demo• Galaxy red shift• Galactic Tidal Stream

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Future Work

• Short Term– Image and spectra support– A stable system

• Used by astronomers in a small community• Long Term

– VO-DAS + VOSpace?– VO-DAS + VO applications?– VO-DAS + Advanced Workflow?

• Document and Q&A

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

LAMOST

Prospects and Concerns

• New projects– Data resources– Requirements

• Hard time for GRID• Non-technical obstacles

– Limited funding for• Manpower• Hardware, esp. data storage • Bandwidth

– Isolated management– Coordination is time consuming

2.4m@LiJiangHXMT

50m@Miyun

23/4/22CANS 2007, Aug. 25-26, 2007, Xi'an

Q & A

?www. .org

Thank You