Understanding its Feasibility P. Mato/CERN, September 8 th , 2010
Jan 20, 2016
Understanding its Feasibility
P. Mato/CERN, September 8th, 2010
CMT (has became) is too complex◦ Too many concepts: macros, sets, tags, patterns,
fragments, actions, projects, packages, etc.◦ Relatively simple build operations require
complicated interactions between these concepts CMT is too slow
◦ The added complexity has caused CMT to became slower and slower
◦ Big improvements in speed are not in horizon Missing support for new IDEs
◦ Eclipse, XCode, VisualStudio9, etc.
Using CMake in GAUDI based Projects - P. Mato/CERN 2
Generates native build environments. It does not do the build itself◦ UNIX/Linux->Makefiles◦ Windows->VSProjects/Workspaces◦ Apple->Xcode
Open-Source, Cross-Platform, Docs, etc. Can cope with complex, large build environments Flexible & Extensible
◦ Scripting language (turing-complete)◦ Modules for finding/configuring software◦ Extensible to new platforms and languages◦ Create custom targets/commands◦ Run external programs
Using CMake in GAUDI based Projects - P. Mato/CERN 3
1. Can we build GAUDI (and other LHCb projects) with CMake?
◦ What about the build performance? ◦ What about the support for IDEs?◦ Is anything missing?
2. What would be the impact for end-users?◦ Can we reproduce the same functionality?◦ How difficult would be to learn the new system?
3. Can we ‘automatically’ migrate from CMT to Cmake?
◦ Would it be possible the co-existence of both systems in parallel?
Using CMake in GAUDI based Projects - P. Mato/CERN 4
Exercised with GAUDI and LHCb projects◦ Easy to learn and produce high-level macros that
build the major products (linker/component libraries, dictionaries, test programs, etc.)
◦ Product installation can be made identical to CMT◦ CMake does not runtime environment setting
Needed to generate setup file at build/install time Tested on SLC5, MacOSX and Windows
◦ Verified that generation for Xcode and Eclipse works nicely (not completely finished)
◦ Make it work also for Windows (nmake, vcexpress) Converted all tests to Ctest
◦ CppUnit, QMtest
Using CMake in GAUDI based Projects - P. Mato/CERN 5
#-------------------------------------------------------------------#---GAUDI_LINKER_LIBRARY ( <name> src1 src2 ... LIBRARIES lib1 ...)#-------------------------------------------------------------------function(GAUDI_LINKER_LIBRARY library sources ) PARSE_ARGUMENTS(ARG "LIBRARIES" "" ${ARGN}) foreach( fp ${ARG_DEFAULT_ARGS}) file(GLOB files src/${fp}) if(files) set( lib_srcs ${lib_srcs} ${files}) else() set( lib_srcs ${lib_srcs} ${fp}) endif() endforeach() add_library( ${library} ${lib_srcs}) set_target_properties(${library} PROPERTIES COMPILE_FLAGS –DGAUDI_ target_link_libraries(${library} ${ARGN}) if(TARGET ${library}Obj2doth) add_dependencies( ${library} ${library}Obj2doth) endif() #----Installation details----------------------------------------- install(TARGETS ${library} LIBRARY DESTINATION ${lib})endfunction()
#-------------------------------------------------------------------#---GAUDI_LINKER_LIBRARY ( <name> src1 src2 ... LIBRARIES lib1 ...)#-------------------------------------------------------------------function(GAUDI_LINKER_LIBRARY library sources ) PARSE_ARGUMENTS(ARG "LIBRARIES" "" ${ARGN}) foreach( fp ${ARG_DEFAULT_ARGS}) file(GLOB files src/${fp}) if(files) set( lib_srcs ${lib_srcs} ${files}) else() set( lib_srcs ${lib_srcs} ${fp}) endif() endforeach() add_library( ${library} ${lib_srcs}) set_target_properties(${library} PROPERTIES COMPILE_FLAGS –DGAUDI_ target_link_libraries(${library} ${ARGN}) if(TARGET ${library}Obj2doth) add_dependencies( ${library} ${library}Obj2doth) endif() #----Installation details----------------------------------------- install(TARGETS ${library} LIBRARY DESTINATION ${lib})endfunction()
Using CMake in GAUDI based Projects - P. Mato/CERN 6
Comparison between cmt and cmake◦ Same code version, some machine◦ “cmt br make –j8” vs. “make –s –j8” ◦ LHCB has ~ 100 packages, REC ~ 50 packages
cmt cmake
GAUDI (noop) 50 s 7 s
GAUDI (full) 613 s 148 s
LHCB (noop) 480 s 17 s
LHCB (full) 2700 s 356 s
REC (noop) 335 s 13 s
REC (full) 1594 s 332 s
X ~4
X ~7
X ~5
Using CMake in GAUDI based Projects - P. Mato/CERN 7
projectA
cmake packA hat
src docpackA
CMakeList
CMakeList
src docpackBCMakeList
packB
Using CMake in GAUDI based Projects - P. Mato/CERN 8
Package dependencies◦ CMake [only] takes care of the library dependencies
within a project and across projects (export/import)◦ Invented GAUDI_USE_PACKAGE() to emulate CMT use
statement (mainly used to set the –I and –L) Project dependencies
◦ Invented GAUDI_USE_PROJECT() to emulate CMT project use statement (CMake module path, LD_LIBRARY_PATH, etc.)
Runtime environment◦ Generation of setup files possible
Windows DLL libraries exporting all symbols◦ Generation of .DEF files
Using CMake in GAUDI based Projects - P. Mato/CERN 9
GAUDI_USE_PACKAGE(Boost)GAUDI_USE_PACKAGE(CppUnit)GAUDI_USE_PACKAGE(ROOT)#---Libraries------------------------------------------------------------------GAUDI_LINKER_LIBRARY(GaudiKernel Lib/*.cpp LIBRARIES ${ROOT_LIBRARIES} ${Boost_LIBRARIES})#---Executables and Tests------------------------------------------------------GAUDI_EXECUTABLE(genconf Util/genconf.cpp LIBRARIES GaudiKernel)
GAUDI_UNIT_TEST(DirSearchPath_test ../tests/src/DirSearchPath_test.cpp LIBRARIES GaudiKernel)GAUDI_UNIT_TEST(test_SerializeSTL ../tests/src/test_SerializeSTL.cpp LIBRARIES GaudiKernel cppunit)#---Dictionaries---------------------------------------------------------------REFLEX_BUILD_DICTIONARY(GaudiKernel dict/dictionary.h dict/dictionary.xml LIBRARIES GaudiKernel)
#--Installation----------------------------------------------------------------GAUDI_INSTALL_HEADERS(GaudiKernel)GAUDI_INSTALL_PYTHON_MODULES()GAUDI_INSTALL_SCRIPTS()
GAUDI_USE_PACKAGE(Boost)GAUDI_USE_PACKAGE(CppUnit)GAUDI_USE_PACKAGE(ROOT)#---Libraries------------------------------------------------------------------GAUDI_LINKER_LIBRARY(GaudiKernel Lib/*.cpp LIBRARIES ${ROOT_LIBRARIES} ${Boost_LIBRARIES})#---Executables and Tests------------------------------------------------------GAUDI_EXECUTABLE(genconf Util/genconf.cpp LIBRARIES GaudiKernel)
GAUDI_UNIT_TEST(DirSearchPath_test ../tests/src/DirSearchPath_test.cpp LIBRARIES GaudiKernel)GAUDI_UNIT_TEST(test_SerializeSTL ../tests/src/test_SerializeSTL.cpp LIBRARIES GaudiKernel cppunit)#---Dictionaries---------------------------------------------------------------REFLEX_BUILD_DICTIONARY(GaudiKernel dict/dictionary.h dict/dictionary.xml LIBRARIES GaudiKernel)
#--Installation----------------------------------------------------------------GAUDI_INSTALL_HEADERS(GaudiKernel)GAUDI_INSTALL_PYTHON_MODULES()GAUDI_INSTALL_SCRIPTS()
Library nameLibrary name SourcesSources
Dependent libraries
Dependent libraries
GAUDI_USE_PACKAGE(Kernel/LHCbKernel)GAUDI_USE_PACKAGE(GSL)GAUDI_USE_PACKAGE(Boost)
INCLUDE(GaudiObjDesc)
#---GOD Headers-----------------------------------------------------GOD_BUILD_HEADERS(xml/*.xml)
#---Libraries-------------------------------------------------------GAUDI_LINKER_LIBRARY(TrackEvent *.cpp LHCbKernel GaudiKernel ${ROOT_LIBRARIES} MathCore GenVector ${GSL_LIBRARIES})
#---GOD Dictionary--------------------------------------------------GOD_CUSTOM_DICTIONARY(dict/lcgDict.h xml/lcgdict/lcg_selection.xml)GOD_BUILD_DICTIONARY(xml/*.xml)
#---Installation----------------------------------------------------GAUDI_INSTALL_HEADERS(Event)
GAUDI_USE_PACKAGE(Kernel/LHCbKernel)GAUDI_USE_PACKAGE(GSL)GAUDI_USE_PACKAGE(Boost)
INCLUDE(GaudiObjDesc)
#---GOD Headers-----------------------------------------------------GOD_BUILD_HEADERS(xml/*.xml)
#---Libraries-------------------------------------------------------GAUDI_LINKER_LIBRARY(TrackEvent *.cpp LHCbKernel GaudiKernel ${ROOT_LIBRARIES} MathCore GenVector ${GSL_LIBRARIES})
#---GOD Dictionary--------------------------------------------------GOD_CUSTOM_DICTIONARY(dict/lcgDict.h xml/lcgdict/lcg_selection.xml)GOD_BUILD_DICTIONARY(xml/*.xml)
#---Installation----------------------------------------------------GAUDI_INSTALL_HEADERS(Event)
CMAKE_MINIMUM_REQUIRED(VERSION 2.4.6)INCLUDE($ENV{CMAKEGAUDIPROJECT})#-------------------------------------------------------------GAUDI_PROJECT(LHCB v31r4)GAUDI_USE_PROJECT(GAUDI v21r10p1)
INCLUDE(GaudiPolicy)GAUDI_USE_PACKAGE(Boost)GAUDI_USE_PACKAGE(ROOT)GAUDI_USE_PACKAGE(AIDA)GAUDI_USE_PACKAGE(CLHEP)
GAUDI_GET_PACKAGES(packages)GAUDI_SORT_PACKAGES( packages ${packages})foreach( package ${packages}) add_subdirectory(${package})endforeach()
GAUDI_PROJECT_VERSION_HEADER()GAUDI_BUILD_PROJECT_SETUP()
CMAKE_MINIMUM_REQUIRED(VERSION 2.4.6)INCLUDE($ENV{CMAKEGAUDIPROJECT})#-------------------------------------------------------------GAUDI_PROJECT(LHCB v31r4)GAUDI_USE_PROJECT(GAUDI v21r10p1)
INCLUDE(GaudiPolicy)GAUDI_USE_PACKAGE(Boost)GAUDI_USE_PACKAGE(ROOT)GAUDI_USE_PACKAGE(AIDA)GAUDI_USE_PACKAGE(CLHEP)
GAUDI_GET_PACKAGES(packages)GAUDI_SORT_PACKAGES( packages ${packages})foreach( package ${packages}) add_subdirectory(${package})endforeach()
GAUDI_PROJECT_VERSION_HEADER()GAUDI_BUILD_PROJECT_SETUP()
Project Declaration
Project Declaration
Project Dependencies
Project Dependencies
Find dynamically all packages and sort them out (not really
needed)
Find dynamically all packages and sort them out (not really
needed)
Common commands and package dependencies
Common commands and package dependencies
Project build artifactsProject build artifacts
In the prototype I have ‘copied’ the same structure of CMT, project organization and main procedures
Some exceptions:◦ The build is done out-of-source◦ The installation step is done after the build (make
install) We could take advantage of the new
functionality◦ Managing ‘build’ options and cache them
Using CMake in GAUDI based Projects - P. Mato/CERN 13
Developed a small python script that is able to ‘translate’ CMT packages to CMake◦ It makes use of the knowledge CMT has of the
package (cmt show uses; cmt show constituents; cmt show <library>_use_linkopts; etc.)
◦ It works almost 100% for LHCB and REC projects◦ The results can be committed to SVN and keep
them in sync either by hand or re-run the script Building projects with CMake can produce
exactly the same output◦ In principle is easy to test the build results
Using CMake in GAUDI based Projects - P. Mato/CERN 14
1. Can we build GAUDI (and other LHCb projects) with CMake?YES. It has been quite easy. Scripting has been essential.
◦ What about the build performance? Much better than CMT◦ What about the support for IDEs? Very good so far ◦ Is anything missing? Few things: dependencies, runtime env.
I could easily implement so far the missing functionality
2. What would be the impact for end-users?I think is small. The paradigm is the same
◦ Can we reproduce the same functionality? I think so E.g. user package overriding an existing released one, etc.
◦ How difficult would be to learn the new system? Trivial ‘translation’ of requirements to CMakeLists files Good native CMake documentation
3. Can we ‘automatically’ migrate from CMT to CMake? YES◦ Would it be possible the co-existence of both systems in parallel?
I think so
Using CMake in GAUDI based Projects - P. Mato/CERN 15