BOSS Development Environment BOSS--BESIII Offline Software System Ma Qiumei 2006.03.01.

BOSS Development Environment BOSS--BESIII Offline Software System

Ma Qiumei

2006.03.01

2

Outline

Simple introduction of BOSS How to use CMT in BOSS Simple introduction of CVS Common questions and solutions

3

History of BesIII Offline Software System Besf/v1 : The original version completely based on belle Besf/v2 : The version improving with other frameworks like babar, gaudi, etc. Offline/3.0.0 : The version with the new building tools—CMT Boss/1.0.0 : The testing version based on Gaudi Boss/2.0.0 : The modified version based on Gaudi Boss/2.1.0 、 2.2.0 、 2.3.0 、 3.0.0 、 4.0.0 、 4.1.0 、 4.2.

0 5.0.0 、 5.1.0 、 6.0.0

4

Boss development environment(Now)

Machine: lxplus.ihep.ac.cn OS : Linux-7.3 Compiler : gcc-3.2 Framework :Based on GAUDI-v12r0 External Libs: CERNLIB 、 ROOT 、 CLHEP 、 Geant4 、 and etc. Developing language : C++ ， some FORTRAN Database : PostgreSql,MySql Version management tool : CVS Browsing files in CVS repository : ViewCVS Configuration management tool : CMT

5

Boss development environment(future)

Machine: lxplus.ihep.ac.cn OS : Scientific Linux Cern (SLC) Compiler : gcc-3.2.3 Framework :Based on GAUDI_v16r4 External lib: lcg/external; geant4 and etc. Developing language : C++ ， some FORTRAN Database : PostgreSql,MySql Version management tool : CVS Browsing files in CVS repository : ViewCVS Configuration management tool : CMT

6

BOSS Structure

Analysis -- Analysis software and some analysis toolsBesExamples -- Examples for using some packagesBesPolicy -- define some standard patterns of Bes BesCxxPolicy -- define some standard patterns only for c++ compilier BesFortranPolicy -- define some standard patterns only for fortran compilier BesRelease -- for all the packages release BesROOT -- convert data from ASCII to DSTCalibration -- for calibration framework and common too

lsControl -- Common tools and framework of bossDatabase -- the geometry data IO

7

DetectorDescription -- Detector Description packages Emc -- for emcEvent -- different event modelEventDisplay -- the interface for event displayEventFilter -- External -- interfaces of external libsGenerator -- for generators MagneticField --Mdc -- for mdc

8

Muc -- for MuonCounterPhyTool -- some tools for physics analysis Reconstruction -- all reconsrtuction algorithmsRootPolicy -- Simulation -- simulationTestRelease -- for some test releaseTof -- tofUtilities -- some common tools More details please see the website: http://koala.ihep.ac.cn/cgi-bin/viewcvs.cgi/BossC

vs

9

How to use CMT in BOSS 　　

Simple introduction of CMT

Setting up your CMT environment Working on a package

Working with boss

10

What is CMT （ Configuration management tool ）

A set of tools and conventions structures software development or

production concepts of areas, packages, versions, constituents

organises software into packages describes package properties describes package constituents operates the software production

(management, build, import/export, etc...) by transparently configuring and driving the various

conventional tools (CVS, make, MSDev, Web, tar, compilers, linkers, archivers, etc...)

11

The goals

Organise software development from a single person up to teams in large projects (scalability)

Organise relationships between entire software bases or between simple packages (scalability)

Manage site, platform or product specific properties

Automate complex production tools for non software experts (simplicity)

Factor out know-how from project to project, from package to package or from team to team (process improvement)

12

Who are using CMT

The physics research experiments using CMT as the following:

a.. Virgo b.. LHCb c.. GLAST d.. Atlas

e.. Nemo f.. Opera g.. Planck h.. Auger i.. GANIL j.. GAM

13

Types of Package

Packages may have particular semantics

Normal packages provide software or documents

Container packages manage other packages

Policy packages hold bes specific conventions, policies, patterns

Interface convert conventions and policies of external packages or libs

14

Structure of a package 　 /src source files (*.cxx) /<package-name> header files (*.h) /cmt requirements file and some scripts /share jobOptions and other running scripts /test some app’s source files /i386_linux24 Automatically created at build time

(libs. Applications and etc. under this

directory)

/ChangeLog a log file

15

Global software organization

use

use useuse

use

use

use use

Release area

User area MDCGeomSvcMDCGeomSvc-01-00-01

BesReleaseBesRelease-00-00-01

MDCGeomSvcMDCGeomSvc-00-00-01

BesGeoMdc BesGeoMdc-00-*

External/CLHEPCLHEP-00-01-*

BesPolicyBesPolicy-01-*

CLHEP

External area

use

16

几个概念 Release （发布；软件版本）：某个 releas

e 对应于所有软件包的某一个固定版本，它是稳定的，一旦发布基本上就不再更改。

nightly_build ：不需要发布一个新的 release ，但是用户希望使用某些软件包的新改变，就建立了这么一个 release ： nightly_build 。这个 release 是不稳定的，经常更新。

TestRelease ：运行 boss 的接口程序，可以选择需要运行的程序。

17

Setting up your CMT environment

首先，连接 CMT $mkdir cmthome $cp /afs/ihep.ac.cn/bes3/offline/template/cmthome/setupCMT.csh ~/cmthome 注：如果你使用的是 bash ，则 copy setupCMT.sh 到你的 cmthome 目录下。

然后，修改 login requirements ，设置符合自己要求的配置 $cp /afs/ihep.ac.cn/bes3/offline/template/cmthome/requirements ~/cmthome 编辑 requirements 文件，使其符合你的要求 $vi requirements注：需要修改的可能有两个地方： set BES_RELEASE 5.0.0 macro WorkArea "/ihepbatch/bes/maqm/workarea"

18

最后，设置环境完成 source setupCMT.csh cmt config source setup.csh

注：最好把以下语句放到 .tcshrc 文件中： source /ihepbatch/bes/maqm/cmthome/setupCMT.csh source /ihepbatch/bes/maqm/cmthome/setupCVS.csh source /ihepbatch/bes/maqm/cmthome/setup.csh source /ihepbatch/bes/maqm/workarea/TestRelease/TestReleas

e-00-00-14/cmt/setup.csh

19

Working on a package Basic scenarios

Creating a package Checking out a package alone Modifying the package

More: working on several packages Apply same set of commands via cmt

broadcast

20

Creating a new package

1 、 Choosing a location i.e. Which container? … e.g.

Database2 、 Choosing a name

Must be a unique package name in boss (should not either just differentiate by case of letters)

3 、 Choosing a version4 、 The command cmt create BesGeoDB BesGeoDB-00-00-01

21

Examples… The simplest one

One new application with (some) source file(s)

>cmt create A A-00-00-01>cd A/A-00-00-01/cmt>vi requirements>vi A.cxx >gmake>source setup.csh>cd ../i386-linux24>./A.exe

package Ause BesPolicy BesPolicy-01-*application A A.cxx

1) Create a new packageand move to it

2) Describe the package

3) Create the package items

and build them

4) set the environment

and run

22

The requirements file

The requirements file

The requirements file centralizes all configuration parametersAuthor(s),

manager(s)

Structural information •specialized directory structure •used packages•links to external packages

Constituents•Libraries•Applications•generated documents

Make macros and environment variables

and their possible values on various platforms, sites, environments Customization for

new languages, or document generators

Definition of conventional behavioural

patterns

Sufficient for transparently generating all required makefiles and setup scripts, or for

launching test procedures. They are the sources of all queries.

23

Example of requirements

package BesGeoDBauthor maqiumei [email protected] version BesGeoDB-00-00-01branches cmt src BesGeoDB use BesPolicy BesPolicy-*use PGSQL PGSQL-* External library BesGeoDB *.cxx

apply_pattern package_linkopts

24

Checking out a package Getting a package out from CVS to modify it

Go to your private development area Checkout the package (installing the proper

CMT directory structure)cmt co TestRelease

Get the HEAD revision, and make use of the most recent tag

cmt co –r TestRelease-00-00-15 TestRelease Get the required revision, and make use of that tag

The directory version is created with the appropriate CVS tag applied to the package

25

Modifying and building a package

The basic development cycle Modify

> vi *** Build

> source setup.csh

> gmake Test (if available)

26

Working on several packages

Checking out a complete containercmt co Databasecmt co –requirements Database/*/cmt/requirements

Checking out all packages cmt co BesRelease cmt co –requirements

BesRelease/*/cmt/requirements

Applying a command to all checked out packages

cmt broadcast <shell command> Cmt broadcast cmt config Cmt broadcast gmake

27

Working with boss The principle of the TestRelease package

This package is the basic interface to run a job Running boss only requires to “select” a set of

already built components and to locally install a set of runtime files Checking out TestRelease

Customizing it : selecting components Before running : install runtime files

This generally creates symlinks towards runtime files (shared libraries, jobOptions, data files, …)

Running boss

28

Running boss #running a simple example -- ExHelloWorld cd <yourworkarea> cmt co TestRelease cd Testrelease/*/cmt vi requirements Add “use ExHelloWorld ExHelloWorld-* BesExamples” to requiremen

ts cmt broadcast cmt config cmt broadcast gmake source setup.csh cd ../run boss.exe HelloWorldOptions.txt boss.exe jobOptions_rec.txt

29

Simple introduction of jobOptions

////####################################// Boss job options file for MdcRec//======================================

ApplicationMgr.DLLs += {"MdcTrkRecon"}; ApplicationMgr.TopAlg += { "MdcTrkRecon" }; ApplicationMgr.ExtSvc += { "MdcGeomSvc" }; ApplicationMgr.DLLs += { "RootHistCnv" };

//--------------------------------------------------------------// Set output level threshold (2=DEBUG, 3=INFO, 4=WARNING, 5=ERROR, 6=FATAL )// MessageSvc.OutputLevel = 2;

30

// Number of events to be processed (default is 10)ApplicationMgr.EvtMax = 200;

EventSelector.InputFiles ="AsciiDmp_out.dat";

ApplicationMgr.HistogramPersistency = "ROOT";

NTupleSvc.Output = { "FILE101 DATAFILE='Mdc.root' OPT='NEW' TYP='ROOT'“};

//===============================// End of job options file// ===============================

31

Running the reconstruction algorithms

#include "Ascii.txt“ #include "PartPropSvc.txt" //zhangy #include "$MDCTRKRECONROOT/share/jobOptions_MdcPatRec.txt“ //zangsl //#include "$TRKRECOROOT/share/jobOptions_TrkReco.txt" #include "$MDCDEDXALGROOT/share/job_dedx_all.txt" #include "$EMCRECROOT/share/EmcRecOptions.txt" #include "$TRKEXTALGROOT/share/TrkExtAlgOption.txt" #include "$TOFRECROOT/share/jobOptions_TofRec.txt" #include "$MUCRECALGROOT/share/jobOptions_MucRecTrkExt.txt" #include "$ROOTIOROOT/share/jobOptions_Save2Root.txt"

32

// Set output level threshold (2=DEBUG, 3=INFO, 4=WARNING, 5=ERROR, 6=FATAL ) MessageSvc.OutputLevel = 5; // Number of events to be processed (default is 10)ApplicationMgr.EvtMax = 200;EventSelector.InputFiles="AsciiDmp_out.dat";ApplicationMgr.HistogramPersistency = "ROOT";RootCnvSvc.digiRootOutputFile ="digi.root";

NTupleSvc.Output = { "FILE101 DATAFILE='Mdc.root' OPT='NEW' TYP='ROOT'", "FILE104 DATAFILE='kalfit.root' OPT='NEW' TYP='ROOT'", "FILE103 DATAFILE='DeDx.root' OPT='NEW' TYP='ROOT'", "FILE201 DATAFILE='Tof.root' OPT='NEW' TYP='ROOT'", "FILE301 DATAFILE='Emc.root' OPT='NEW' TYP='ROOT'", "FILE401 DATAFILE='Muc.root' OPT='NEW' TYP='ROOT'" };

33

CVS 是什么 ? 1989 年 4 月， Brian Berliner 设计了 CVS 并

编写了代码，之后 Jeff Polk 帮助 Brian 设计了 CVS 模块发行分支。

CVS(Concurrent Versions System ) 版本控制系统是一种 GNU 软件包，主要用于在多人开发环境下的源码的维护。 它可以记录源文件开发的历史。

34

CVS and CMT

The BESIII Offline Software sources are stored in CVS and can be accessed using the CVS command.

The BESIII Offline Software libraries have been built using the CMT. Therefore, using the CMT tool is the recommended way to modify existing packages or re-build the examples included in the release.

35

如何使用 CVS ？ 首先，你需要申请一个 CVS 账号 然后，需要设置变量： export CVSROOT=':pserver:[email protected]:/bes/bes'

或 setenv CVSROOT ‘:pserver:[email protected]:/bes/bes

‘ 最后，你需要知道一些常用的 CVS 命令（见下

页）

36

常用的 CVS 命令 (1) Checking in to CVS cvs import –m “**” test Bes test-00-00-01 查看源程序的变化 cvs –n update 添加一个源文件 cvs add 源文件名（路径要正确） 删除一个源文件 cvs remove 源文件名（路径要正确）

37

常用的 CVS 命令 (2) 提交所作的修改 cvs commit –m “**” 给一个包做 tag cvs tag 版本号 查看当前工作的源文件与 cvs库里源文件的区别 cvs diff

38

Viewcvs简介 ViewCVS 是一个使用 Python书写的查看 CVS 代

码库中的 所有数据 的软件。它已经被很多的配置管理系统所使用， 也是现在开放源代码产品中与 cvsweb 一样可以选择的 CVS辅助工具。

ViewCVS架设了一个友好的 WEB界面，通过WEB界面可以非常直观的浏览代码 , 查看图形化的修改记录。

http://koala.ihep.ac.cn/cgi-bin/viewcvs.cgi/BossCvs/

39

常见问题及解决方法 编译的时候：

$ cmt config > Warning : package CERNLIB CERNLIB External not

found (requested by tester) $ gmake >ihepbatch/bes/maqm/workarea/Calibration/CalibDat

a/CalibData-00-00-04/CalibData/CalibTime.h: 14:34: facilities/Timestamp.h: No such file or directory

>/usr/bin/ld: cannot find -lxerces-c

40

运行的时候： boss.exe jobOptions_sim.txt

libGaudiSvc.so: cannot open shared object file: No such file or directory

boss.exe jobOptions_sim.txt

undefined symbol: ****

41

Documents

http://www.cmtsite.org http://atlas.web.cern.ch

/Atlas/Groups/software/oo/tools/cmt http://boss.ihep.ac.cn http://koala.ihep.ac.cn/cgi-bin/viewc

vs.cgi/BossCvs/

42

Thank you!