Key Activities. MND sections

Key Activities. MND sections

Group meeting 13.08.08

Overview• Yesterday section meeting showed substantial progress in many

directions:

- Central repository of monitoring metrics for LHC experiments (Elisa and Dashboard team, intersection development )

- GridMap view for site administrators (Elisa, Max, Dashboard team)- Testing of LB notification system (Olga Kodolova and LB Team)- Adapting CMS SAM availability applications for other LHC VOs (William)- Development of the Grid Messaging System and support of pilot users and applications (Daniel)- New features of ATLAS DDM and ProdSys monitoring. (Ricardo, Benjamin, Lu, Markus)- Creating of ATLAS Grid Information System (AGIS) and integration of Dashboard applications

with it (Raquel, Ricardo, Benjamin)- A lot of improvements to the CMS Site Status Board (Pablo)- Improvements of the CMS Interactive Job Monitoring application, following the requests of the

CMS community (Anastasia, Julia)- Improvements to the CMS schema and queries, fixing problems causing high DB load during

CCRC08 (Irina, Julia)- Use of GMS for propagating of job status reports from the worker node to the Dashboard DB

(Bernd)- Improvements for Data Transfer Reliability for ALICE experiments (Spyridon)

Summer students

• We have four summer students in the MND section this year: Lu feng, Bernd Schodel, Markus Huber, Spyridon Koulozis

• Very good contribution to the sections work!

• See details about student presentation at yesterday section meeting:

http://indico.cern.ch/conferenceDisplay.py?confId=39049

Central repository for common metrics and site view

• Objective of this activity• ● Provide a comprehensive monitoring tool which, from one unique

console,• gives an overview of the overall status of the services in their site. This• should be a tool easy to use, also for persons external to the VO, and

which• does not require a particular knowledge of each experiment.• ● This tool will extract information from the experiment specific tools• (Dashboard, MonALISA, Dirac, Phedex) and will display it in a consistent• way.• ● It will display the information using the gridmap technology and will

provide links to the source of the information• ● An additional requirement from sites is to have a clear definition of

the• targets from the experiments: this information should be displayed by

this• tool (but it has to be clearly defined and provided by the experiments)

Testing of LB notification

• We are first pilot users

• In case of success LB notification should replace RGMA and IC XML sources for job status changes information for jobs submitted via LCG WMS.

Adapting of CMS SAM availability application for other VOs

• UI to follow the results of SAM tests is already in production for ATLAS

• Is coming for LHCb and ALICE

• Creating the collectors for topology info for LHC experiments to be introduced to SAM DB (William for ATLAS, Julia for CMS)

ATLAS Data Management• Integration with AGIS• Collection of OSG scheduled downtimes• Notification messages for cloud activity / performance

– Delivery using the dashboard messaging API• Part of maintenance now given to ATLAS shifters and experts• Activities for dataset subscriptions (in development)

– Delivery by the end of August– New mandatory ‘activity’ field when issuing subscriptions– Transition to a single instance of the DDM dashboard – new

filters by activity instead of multiple instances• New statistics in the main display (Lu Feng)

ATLAS Production System

• Integration with AGIS• RSS feed exposing problematic sites /

tasks• Cloud / site admin display (Lu Feng)• Grouping error messages by similarity

– Better than proddb error codes which are not always useful

• Improved shifter’s display• Reporting of pilot job details via python

API (issuing HTTP requests)

AGIS• ATLAS Grid Information System• Built using the dashboard framework• Collection of site information

– From GOCDB for EGEE and OIM for OSG• Collection of service information

– From the LCG and OSG BDIIs• Collection of ATLAS cloud and tier information

– From the TiersOfATLAS and Panda database– Stopped after a transition period when update of info directly in

AGIS is fully tested• Querying available via a python API, command line tools,

direct HTTP with multiple output formats (including ToA)• Updating via secure HTTP calls• Prototype: http://agis.cern.ch/dashboard/request.py/agis

ATLAS Tier0 Dashboard

• Developed in collaboration with the ATLAS T0 team (Guido Negri)

• T0 data exposed via a dashboard application

• Prototype display built using HTML canvas elements

Grid Messaging System

CMS Site Status Board