INFSO-RI-508833 Enabling Grids for E-sciencE Using of GANGA interface for Athena applications A. Zalite / PNPI.

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

Using of GANGA interface for Athena applicationsA. Zalite / PNPI

Enabling Grids for E-sciencE

INFSO-RI-508833

Overview

• General introduction to ATLAS software framework - Athena: structure, applications, user interface.

• Introduction to GANGA.

• Running Athena using Ganga interface.

Enabling Grids for E-sciencE

INFSO-RI-508833

Athena framework

• In practice framework is a skeleton of the application into which developers plug in their code and provides most of the common functionality.

• Athena - Atlas software framework based on GAUDI. Athena provides common services such as the transient data store, (interactive) job configuration and auditing, data(-base) access, message streams, etc. for Atlas software. The idea is to improve the coherency of the different software domains within Atlas, and thereby the ease of use for end-users and developers, by having them all use the same well-defined interfaces.

Enabling Grids for E-sciencE

INFSO-RI-508833

Athena framework

• The overall principle of the Athena framework is separation of algorithm and data.

• Each algorithm implements 3 methods for invocation by framework: an initialize() function which is executed at the start of the job, an execute() function which is executed every event and a finalize() function which is executed at the end of the job.

• An algorithm has a number of control and data parameters which allow run-time configuration. They can be set in the startup jobOptions files.

• The idea is you can write data to the Transient Store (TS) from one algorithm. You can then read in this data in another algorithm and do some processeing on that data and then write resulting data to the TS and so forth.

Enabling Grids for E-sciencE

INFSO-RI-508833

CVS, CMT

• All ATLAS offline code is stored in Concurrent Versions System (CVS), which manages the evolution of the source files.

• The build of the binaries (compilation option, include files, libraries…) as well as the run-time environment are managed by Configuration Management Tool (CMT).

• The software is grouped into separate packages, where some packages may be related or "depend" upon each other.

Enabling Grids for E-sciencE

INFSO-RI-508833

Running Athena

• First step is setup of Atlas software environment (controlled by CMT)

• > athena By default, it looks for jobOptions.py

• > athena myJobOptions.py Athena will use myJobOptions.py for input

configuration

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga

• Ganga (Gaudi / Athena and Grid Alliance) is an interface to the Grid that is being developed jointly by Atlas and LHCb.

• It is optimised for the configuration and management of jobs that use the Gaudi / Athena Framework which is common to the both of these experiments.

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga

• Ganga aims to allow a user to run a job at any site where the user has authorisation, and in particular at remote sites to be accessed through the Grid. To this end, Ganga creates a site-independent job representation, based on information supplied by the user, and takes care of any file transfers necessary between the user's machine and the remote site.

• In the current Ganga model, each job entails running a single executable. This may correspond to a single binary, for example to perform event reconstruction, or may be a script that runs in sequence any number of binaries, for example event generation followed by detector simulation followed by event reconstruction.

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga

• To run from account on lxplus, or over afs, perform setup with: > source /afs/cern.ch/sw/ganga/install/etc/setup.sh

• After setup, the Ganga Graphical User Interface (GUI) is started by giving the command:

> ganga

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga GUI

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga GUI

• The Ganga Graphical User Interface (GUI), as shown at startup, has the following features: – Panel for browsing job and directory trees– Drop-down menus and toolbar – Panel for job configuration and monitoring– Embedded python shell

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga GUI (Job tree)

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga

• To run from account on lxplus, or over afs, perform setup with: > source /afs/cern.ch/sw/ganga/install/etc/setup.sh

• After setup, the Ganga Graphical User Interface (GUI) is started by giving the command:

> ganga

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga GUI (File tree)

Enabling Grids for E-sciencE

INFSO-RI-508833

Ganga: examples

Let’s try to perform practical examples

• Run /bin/hostname• Run /bin/hostname with output to the file

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

• The job-creation window

Batch system to submit: Local, LSF, EDG/LCG, PBS, Glite, DIRAC.Applications are divided into two classes: Generic and known applications, for example Athena.

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Submitting

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Submitting

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Submitting

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: outputs

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: outputs

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: outputs

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job Creation

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Submitting

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: output

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: output

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: NewJob3.output

Enabling Grids for E-sciencE

INFSO-RI-508833

Running Athena• Ganga uses Athena application handler to run Athena

within the Ganga framework.• The handler benefits from Ganga's job submission and

job monitoring capabilities• The handler has the following features:

– AthASK is used to hide away Atlas software configuration complexities. The user simply provides AthASK commands needed to run Athena job from within the GUI.

– User packages (i.e. binary builds, source code) can be specified and these directory trees will be compressed and transported (together with the rest of the job) to the worker node ready to be used by the submitted job.

– Integrated with AMI browser. Job option files can be updated with selected files from AMI using POOL.

– Integrated with JOE. Job option files can be edited using JOE before submission.

Enabling Grids for E-sciencE

INFSO-RI-508833

Athena Startup Kit (ASK)

• The Athena Startup Kit (ASK) integrates the ATHENA framework, and the software based on it, with the official releases and the various supporting tools in order to provide an easier, more consistent, and more robust user interface to Atlas offline software.

• ASK operates completely in user-space: it performs only those tasks that are otherwise left to the end-user by the "standard" Atlas tools.

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job for Athena

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job for Athena

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job for Athena

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job for Athena

•In case of local instalation

Enabling Grids for E-sciencE

INFSO-RI-508833

New Job for Athena

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Submitting

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Run

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: output

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: output

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: output

Enabling Grids for E-sciencE

INFSO-RI-508833

Job Finished: output

Enabling Grids for E-sciencE

INFSO-RI-508833

LHCb prototype: GANGA

• Framework for job creating-submitting-monitoring

• Can be used with different physical applications and different submission back-ends

• Was also chosen as a prototype component by the Atlas experiment

GANGAGUI

JobOptionsAlgorithms

Collective&

ResourceGrid

Services

Submitting jobsMonitoring

Retrieving results

GAUDI Program

GAUDI – LHCb analysis framework

ExperimentBook-keeping

DB

Gaudi/Athena aNd Grid Alliance