19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 1 Using GraphViz in Dependancy Analysis of BaBar Software Igor A. Gaponenko LBL / NERSC.

19-MAY-2006 GraphViz in Dependancy Analysis of BaBar Software Page: 1

Using GraphViz in Dependancy Analysis of BaBar Software

Igor A. GaponenkoLBL / NERSC / HENPC


Contents

• What is GraphViz and why here?– A problem of migrating CDB (Condition/DB of BaBar)

from Objectivity/DB to ROOT I/O and MySQL– Dependency analysis of BaBar software

• BaBar SRT (Software Release Tools)– A short introduction into relevant (in a context of this talk)

aspects of SRT– Generating graphs

• More on GraphViz and its applications


What is GraphViz?

And why here?


So, What is GraphViz?

• WWW.GRAPHVIZ.ORG

• Free Graph Visualization Software– Developed at AT&T (to visualize telephone networks?)

• (Relatively) simple graph description language: DOT

• Graph presentation is (sort of) separated from its (graph) description

• More details to come in a dedicated section… But first, why we got so interested in it.

http://www.graphviz.org/


Migrating CDB of BaBar from Objectivity

• A complex problem involving– >200 persistent classes– A few more hundreds of relevant transient classes– The Objectivity API specific code: ~100 software packages– Direct or indirect package dependencies onto Objectivity API from physics code– 1400 binary applications built in latest releases (many of them need CDB)

• “Certain restrictions apply”– Working on a frozen snapshot of code is not an option– The migration must be incremental

• Facilitate testing• Do things in parallel with other developments

– The BaBar software must keep functioning– Both technologies should co-exist w/ no code duplication at a level of physics code

• Hence– A need in major code re-factoring, reorganizing packages to take out technology-

specific code and confine it in specialized packages– A need in instruments helping to analyze code dependencies, monitor an evolution

of dependencies in a course of the migration

tough

tough

transition period


CONCRETE FACTORY

What’s wrong with these packages?

WHAT WE HAVE WHAT WE NEED

CLIENT

CONCRETE FACTORY

OBJECTIVITY

main()

CLIENT

CONCRETE FACTORY

OBJECTIVITY

main()

ABSTRACT FACTORY

ROOT

instantiateuse

instantiateuse


PICTURE

GENERATED BY GraphViz


More Examples from BaBar

• A bit more complex package dependency diagram:

– http://www.slac.stanford.edu/~gapon/TALKS/2006_May_LBNL_GroupMeeting/MooseApp_BdbCond.png

• A dependency diagram for configuration files of BaBar Framework based applications:

– http://www.slac.stanford.edu/~gapon/TALKS/2006_May_LBNL_GroupMeeting/mod2tcl_CdbBdbInit.png

– Comments:• BaBar Framework uses the TCL scripting language for that

• The above shown diagram presents how TCL files depend on some Framework module. Modules are specially designed configuration objects Framework application are linked with. TCL provides an interactive interface to these objects. A module shown on the diagram is used to control general parameters of CDB.


BaBar SRT


Generating Graphs from BaBar code

A

• For packages, we’re not using source (C++) code! A structure of BaBar SRT is used instead.

• SRT is based on GNU Make. Plus shell and

Perl. SRT has a flat and static dependency model for packages.

ALLOWED LIBRARY DEPENDENCIES

B

C

.

Y

Z

ACTUAL LIBRARY DEPENDENCIES

A

C

Y

ZD

Cyclic dependencies for libraries are not allowed!


SRT release and packages

• A release is a consistent collection of packages

• A package defines:

• one library• (optionally) binary(-s)• (optionally) exe.scripts

• Source of package dependencies (defined through special files included into the package makefile):

• link_A.mk• [ bin_A.mk ]• [ bin_BinApp.mk ]

A

C

Y

ZD

A / BinApp.cc

E

- For binary applications dependency restrictions are relaxed. They’re allowed to have upstream dependencies!

- Though, binary always depends on its package’s library.


Generating Graphs for GraphViz

• Developed the scanners - Perl scripts to scan special (.mk) files in packages– A compact database of dependencies is build – one per SRT release

• Developed analyzers – Perl scripts taking queries, using the databases and generating answers as:

– Human readable dependency printouts– Graph descriptions (in the DOT language) for GraphViz

• Types of queries supported by analyzers:

“..I have an application MyApp. Tell me how my application depends on Objectivity/DB?..”

“..I have three applications: App1, App2 and App3. What’s a combined dependency diagram for all of them on these packages: A, B, C and Z? ...”

“..I have a package called Pravda. Which packages depend directly or indirectly on my package? Asking this because I’m going to do som eserious changes to my packages and I wish to know which applications I may potentially break..”


Automate Everything!

• The following ideas exist :

– Build dependency database at a time when an SRT release is built.

– Set up a Web service (CGI, whatever) to provide an interface (HTML forms?) for making queries like those one mentioned on the previous slide.

– Generate full dependency diagrams for each application and for each package at a release built time. Publish this information (in a form of DOT programs) in a documentation section of the release.

– Generate “interactive” graphs with a URL behind each package box. That URL will bring me into a Web page with…

• No time to implement them


An Overview of GraphViz


The DOT Graph Description Language• A trivial example of the DOT program and its effect :

graph G {

run -- intr;

intr -- runbl;

runbl -- run;

run -- kernel;

kernel -- zombie;

kernel -- sleep;

kernel -- runmem;

sleep -- swap;

swap -- runswap;

runswap -- new;

runswap -- runmem;

new -- runmem;

sleep -- runmem;

}


Sub-graphs• Nested graphs (sub-graphs) can easily be defined as well :

digraph G {

subgraph cluster_0 {

label = "hello world";

a -> b;

a -> c;

color = hot_pink;

}

subgraph cluster_1 {

label = "MSDOT";

style = "dashed";

color = purple;

x -> y;

x -> z;

y -> z;

y -> q;

}

top -> a;

top -> y;

y -> b;

}


Example: ER Graph


Example: Memory References Graph


Graph Layouts

• The same graph can be presented by 5 algorithms :– dot

• makes ``hierarchical'' or layered drawings of directed graphs. The layout algorithm aims edges in the same direction (top to bottom, or left to right) and then attempts to avoid edge crossings and reduce edge length.

– neato, fdp• make ``spring model'' layouts. neato uses the Kamada-Kawai algorithm, which is

equivalent to statistical multi-dimensional scaling. fdp implements the Fruchterman-Reingold heuristic including a multigrid solver that handles larger graphs and clustered undirected graphs.

– twopi• radial layout, after Graham Wills 97.

– circo• circular layout, after Six and Tollis 99, Kauffman and Wiese 02. Suitable for certain

diagrams of multiple cyclic structures.

• Lots of examples in here:– http://www.graphviz.org/webdot/demo.html


Algorithm: DOT


Algorithm: NEATO


Algorithm: TWOPI


Presentation Attributes

• In the DOT language a graph definition is combined with its presentation parameters. All attributes:

– http://www.graphviz.org/doc/info/attrs.html

• Nodes attributes (shapes) :

– http://www.graphviz.org/doc/info/shapes.html

• Edges attributes :

– http://www.graphviz.org/doc/info/arrows.html

• Colors :

– http://www.graphviz.org/doc/info/colors.html


What else can be tuned?

• Output formats

– http://www.graphviz.org/doc/info/output.html

• In fact, all known graphic formats supported

• I’ve also heard that it’s also possible to split an output picture of a complex graph onto smaller ones for printing purposes. Never found how to do it though. Had to use a separate tool.

19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 1 Using GraphViz in Dependancy Analysis of BaBar Software Igor A. Gaponenko LBL / NERSC.

Documents