Category 7 Summary Organizers: Torre Wenaus and NK Nobu Katayama KEK.

Category 7 Summary

Organizers: Torre Wenaus and NK

Nobu Katayama

KEK

HENP software architecture and software engineering

• In our category, we have discussed mainly about tools and/or mechanisms to deal with– Large number of detector components– Large number of people– Multiple choices– Integrationat active development stages

• There are two different scales– One person trying to help 10 software developers– 10 people trying to organize development work by

100 people

Goals• For users, tools should just work; they should be

– Simple to use– Easy to understand– Error prone– Stable and reliable– Supported well

• Toward generic, non-experiment specific tools?– Standardization

• Common “data” format

– Coordination among experiments/laboratories– Good tools become being used (slowly) by more and

more people• Goodness itself doesn’t always make it happen, though

Talk/poster statistics

• LHC– ATLAS(6)– CMS(3)– LHCb(2)– ALICE(2)

• Other exp.– CDF(2)– Icecube, GLAST(2), CLEO-III(2), LC

• Accelerator/control– DESY(2), BaBar/PEP-II

• Non-exp. specific (Root, freeHEP)• LCG projects (SEAL, SCRAM)

Subcategories

• Event model, data definition(4)

• Package/project building, project management and release tool (7)

• Geometry/detector definition(3)

• Framework and library (8)

• Others (6)

Event model/data definition

• Storegate: a Data Model for the Atlas Software Architecture (ATLAS)

• Event Data definition in LHCb• Athena Data Dictionary and Description Langua

ge (ATLAS) (as I understand it), we are waiting for LCG versi

on of it• Event as an Object-Relational Database; Avoidin

g the Dependency Nightmare (CLEO-III)– Dependencies kept in one “library”

Object Relational Approach

Track:1Track:1 EM Shower:1EM Shower:1

Track:2Track:2

Hit:1Hit:1

Hit:2Hit:2

Hit:3Hit:3

Hit:4Hit:4

Hit:5Hit:5

Hit:6Hit:6

Hit:7Hit:7

Hit:8Hit:8

EM Shower:2EM Shower:2

EM Shower:3EM Shower:3

1:data:1

2:data:1

4:data:1

7:data:1

3:data:2

5:data:2

6:data:2

8:data:2

1:data:1,2

2:data:2

Describe N-M relations

Package/project building, project management, release tool

• OVAL: CMS testing tool• User-friendly tools for managing projects

(VCMT, GLAST)• Athena Startup Kit (ASK, ATLAS)

– User need to know only few commands when jumping into analysis

• GLAST Nightly Package Builds and Code Release System Tests

• BOA: framework for automated builds (CMS)• NICOS system of nightly build for distributed

development (ATLAS, LCG)• SCRAM (poster, LCG)

General Description

• Original goal : help to detect automatically the changes in the physics performances of the CMS software used for data production.

• Current features of the tool : compile the provided test programs, run them in specified runtime conditions (shell variables + auxiliary files), compare the output with references.

• User interface is CVS-like, configuration files are XML-like. It is written in PERL and can be configured for any build system (scram, make).

From OVAL

Configuration : OvalFile

<diffline expr="^OVAL:"><diffnumber expr="^OVAL NUM: .* is (.*)$" tolerance="5%">

<file name= ".orcarc">MaxEvents = 500Random:Seeds = 0 3</file>

<var name= "FEDERATION" value= "cmsuf01…">

<environment name= "pt15"> <var name= "DATASET" value= "eg_ele_pt15"> < program name= "Clusters.cpp"> < program name= "Electrons.cpp" args="-geo detailed"></environment>

<environment name= "flow"> < var name= "DATASET" value= "jm_minbias"> < program name= "EnergyFlow.cpp"></environment>

From OVAL

GLAST CHEP 03 March 24-28 2003

T.Burnett 11/12

VCMT for WindowsVCMT for Windows

GLAST CHEP 03 March 24-28 2003

K.Young 12/22

Web Interface(Carrot, JAS,…,

Scripts)

Architecture of System TestsArchitecture of System Tests

Executive Script(Perl)

Suite of SimulationsRun on

Supported Platforms

Oracle DB

Root Files,Stdio Dumps Root Scripts

Root HistogramFiles

Detector/geometry definition

• Detector Description Framework in LHCb• GraXML – Modular Geometric Modeler (A

LICE)Waiting for LCG decision• Use of a generic identification scheme con

necting events and detector description in the ATLAS experiment– IDs for 100 million channels in the ATLAS det

ector!

14/18CHEP 2003 - 3/21/03Sébastien Ponce

Data Diagram

Structure

MaterialGeometry

DetElem GeometryInfo

CalibrationConditionAlignmentConditionReadoutCondition

MuonStation

EcalClusterCondition

Conditions

EcalClusterCondition

MuonStationAlignment

VeloReadout

Lvolume

Pvolume

Solid

Box

Sphere

*

Points to

Inherits fromResolved on demand Material

Isotope Mixture

Element

15/18CHEP 2003 - 3/21/03Sébastien Ponce

Transient Store

● Tree-like structure● Items identified by a logical name

● Load/update on demand● Automatic update when new event

Geometry

LvolumeLvolumeLvolumeLvolume GeometryDb

DetectorData Service

PersistencyService

Algorithm

Ask forObject

Retrievepointer

Checkpresence LoadAsk creation

Cnv

16/18CHEP 2003 - 3/21/03Sébastien Ponce

CondDB Implementation

Geometry

LvolumeLvolumeLvolumeLvolume

DetectorData Service

PersistencyService

Algorithm

Ask forObject

Retrievepointer

Checkpresence Ask creation

Cnv

ConditionDb

ConditionService

Ask for data Retrieve themfrom Database

Abstract interface

● Additional service of the framework● Independent of the type of data contained (XML)● Based on an abstract interface with several

possible backends (Objectivity, ORACLE, MySQL)

Framework/library

• Root: Status and Future (See slides by yourself)• The FreeHEP Java Library• SEAL: Common core libraries and services for LHC appli

cations• The AliRoot framework (ALICE) (See slides by yourself)• Dynamically configurable system for fast simulation and r

econstruction for CMS (FAMOS)• Java Physics Generator and Analysis Modules• The Virtual Monte Carlo (ALICE)• Reconstruction and Analysis on Demand: A Success Sto

ry (CLEO-III)

Mark Dönszelmann, SLAC CHEP, La Jolla, 24-28 March 2003

FreeHEPJava Library

HEP Specific Components

Physics package 3-vector, 4-vector’s and utilities Jet Finding, Event Shape routines Diagnostic Event Generator

HEP IO Access to a number of common HEP

formats: StdHEP MCFIO HBook Root

HEP3D Extensions to Java3D specific for Ge

ant4YaPPI – Yet another Particle Property Interface

XML Particle Property Database

AIDA - Abstract Interfaces for Data Analysis

Interfaces and JAIDA - Java reference implementation

AIDAJNI – C++ wrapper around AIDA More on this in Max Turri’s talk on T

uesdayHepRep - HEP Representables for Event Display

Interfaces and JHepRep - Java reference implementation

HepRepJNI – C++ wrapper around HepRep

see several other talks and postersFuture

Plot Widget (JAS)

Mark Dönszelmann, SLAC CHEP, La Jolla, 24-28 March 2003

FreeHEPJava Library

General Components

Application Framework General framework for Java applicati

ons using plugin and service technology.

AID – Abstract Interface Definition Compiler to make C++ headers and

Java Interfaces from aid files.Graphics3D

Non-Java3D implementation of 3D graphics, for WIRED.

FreeHEP IO Extensions to the JavaIO system to h

andle specific encodings, ASCII85, …

PSViewer PostScript Viewer to embed viewing

of EPS in applications, or for previews.

Swing ExtensionsFreeHEP Tools

JNeeds – java package dependency Ant – extensions to the build system

VectorGraphics Low level Input and Output of binary

vector graphics formats (CGM, EMF, PDF, SVG)

Vector output files, extension to Java2D

XML XMLIO – classes to store/restore st

ate of an applicationFuture

RecordLoop/EventLoop (IceCube)and more…

Domain Coverage

EventGeneration

Core Services

Dictionary

Whiteboard

Foundation and Utility Libraries

DetectorSimulation

Engine

Persistency

StoreMgr

Reconstruction

Algorithms

Geometry Event Model

GridServices

I nteractiveServices

Modeler

GUIAnalysis

EvtGen

Calibration

Scheduler

Fitter

PluginMgr

Monitor

NTuple

Scripting

FileCatalog

ROOT GEANT4 DataGrid Python Qt

Monitor

. . .MySQLFLUKA

EventGeneration

Core Services

Dictionary

Whiteboard

Foundation and Utility Libraries

DetectorSimulation

Engine

Persistency

StoreMgr

Reconstruction

Algorithms

Geometry Event Model

GridServices

I nteractiveServices

Modeler

GUIAnalysis

EvtGen

Calibration

Scheduler

Fitter

PluginMgr

Monitor

NTuple

Scripting

FileCatalog

ROOT GEANT4 DataGrid Python Qt

Monitor

. . .MySQLFLUKA

SEAL

Simulation/Reconstruction Chain

MC 4-vector

SimHit

RecHit / Digi

Analysis object

FAMOS/ORCA

Reconstruction: ORCA

FAMOS/ORCA

Simulation: OSCAR

FAMOS/OSCAR Start with fastest modeAIM: 1000 times faster

MC 4-vector

Analysis object

100-200 sec

1-10 sec

10-100 sec

DetectorSimulationEvent Generators

Interoperability

Pythia

Pandora

Whizard

EventSelection

LCD FastMC

TESLASimDet

JLCQuickSim

AnalysisCode

Java

Fortran

C, C++

VMC Concept

UserCode

VMC

Geant4 VMC

ParticlesHits

Geant4

Geant3

Output

Fluka VMC

Fluka

Input

Geant3 VMC

Other

• Revision control in the Grid Era – the unmet challenge (CDF)

• Commissioning the CDF Offline• Concrete uses of XML in software development

and data analysis (IceCube)• Improving the Security and Performance of the B

aBar Detector Controls System (poster)• Concepts for Integrating Information systems (po

ster, DESY)• UML and explorative prototyping for developing i

nteractive applications (DESY)

Ominous clouds in the LHC era

• The detector components, computer systems, software, and people are– Complex– Inter dependent– Rapidly changing– Globally distributed

• How can we be dealing with the situation?• How can we integrate all software to work

together?

Trends

• XML is coming (+ RDMS is back)– Describe, describe, describe

• Plug-ins– Handle multiple choices (how lucky!)

• Nightly builds– Just to see if anything has broken yesterday

• On demand (just in time?) reconstruction• Ever expanding roots?

– VMC, AliEn, Geometry ,(PROOF, AliEnFS on other sessions)

KEKB upgrade strategy

Present KEKB L=1034

2002

03 04 05 080706 09 10 11

L=103

5

L~1036

dt =500fb1~1PB/year

One year shutdown to: replace vacuum chambers double RF power upgrade inj. linac g C-band

larger beam currentsmaller y*long bunch optioncrab crossing

ILER=1.5A2.6A

ILER=9.4A

ILER=20A

Constraint:8GeV x 3.5GeVwall plug pwr.<100MWcrossing angle<30mrad

dt =3000fb1

~10PB/year beforeLHC!!

My personal favorites• For Belle, when I go home, I would investigate m

ore about– OVAL, GraXML, FreeHEP Java library– Root and related works

• For Super KEKB planning and work,– Virtual Monte Carlo and other MC frameworks– Definition tools– Nightly build tools– On demand reconstruction, analysis processing

• I am eager to see progresses of the LCG projects and adopt