CMS Software Documentation Systems

CHEP 2001CHEP 2001

April, 2001 Lassi A. Tuura, Northeastern Universityhttp://cmsdoc.cern.ch/cmsoo.html

CMS Software CMS Software DocumentationDocumentation

SystemsSystems

Ianna OsborneLucas TaylorLassi A. TuuraJohannes-Peter WellischStephan Wynhoff

For the CMS Collaboration

2Lassi A. Tuura, Northeastern UniversityApril, 2001http://cmsdoc.cern.ch/cmsoo.html

OverviewOverview

Web Pages Code Cross-Reference User Guides Tutorials Architectural Documents Project Planning Documents

Processing Systems Interactive Web Servers To Come Thoughts


Web PagesWeb Pages

Project web pages / templates are part of the project’s documentation subsystem All project web pages (not just documentation) versioned per release Apply a common look to all web pages when building the

documentation Permanent repository of essential development knowledge/hints Installation instructions, references to project download servers, etc. Release notes and such come very easily Links to rest of the documentation

Simple but very useful A simple perl script that copies HTML files, makes simple substitutions

and applies a common structure (itself a template that is read in)

Not yet used in all projects Very successful in the projects in which it is used


Code Cross-ReferenceCode Cross-Reference

Code documentation is embedded into the source code Versioned documentation for each release Process code to the produce a web: as up to date as it can get Most aspects of source code covered: subsystems, directories, files,

classes, methods, macros, …

Having considered a number of systems, chose doxygen Documentation overhead is very small High-quality output, very complete, easy to navigate Capable, used widely, lively development Allows documentation of independent projects to be merged

(we are now working on generation/merging docs for our externals)

Successful, used and liked a lot Need consistent encouragement for code authors to write

meaningful comments and to maintain existing ones up to date


ExampleExample


User GuidesUser Guides

In the documentation subsystem of each project Versioned documents for each release

Documentation subsystem provides Document assembly logic Prefix and initial sections Trailer Glossary, bibliography

Other packages provide “section inserts” Merged into the shell at to provide the meat in the middle Simple but functional convention

Produces both a web and a printed version (~150 pages)


TutorialsTutorials

Have organised a few tutorials on CMS software ~A week long, ~30 people trained each time

A fair amount of preparation, but very successful Getting essential feedback—not just on training, but software as

well Good for recruiting users/developers: lowers doubt barriers

The following format seems to have worked best so far Gradually from simple usage—how to find the software, how to

run commands—to more, even doing real analysis and visualisation No long lectures: teach a subject, work on it, teach some more,

…– Everybody in front of the computer all the time– Teachers are available all the time, including the exercises– Make sure everybody gets somewhere (= good/right results)

Planning 3-4 tutorials each year, changing subjects


Architecture DocumentsArchitecture Documents

The CMS CAFÉ project maintains A database of requirements, use cases, scenarios, constraints,

designs… A collection of document templates for describing various views A collection skeleton documents that describe a particular aspect, and

pull in or link to the various “shell” fragments (requirement, …) (See also J.-P. Wellisch’s paper!)

Gave up on LaTeX—too hard, too many processings required Use XML instead: most content in simplified DocBook, own DTD to

describe the shell fragments (whose actual text content is DocBook) Agressively hyperlinked: the entire document set always has

consistent structure, embedded fragments, cross-references and hyperlinks

Environment provides common functionality Automatically generated glossary, bibliography (used subset of global) Indexing, image conversions, … everything one generally needs


Architecture Documents…Architecture Documents…

CVS repository structure Common stuff (templates, biblio, glossary, XML definitions, processing) Shells: requirements, use cases, scenarios, constrains, …

(directory/kind) Documents (directory/document)

Very powerful documentation concept and a capable system We can now actually guarantee that all references to requirement X-Y-

Z have all the same number, the same text, and all point to the same, unique source where the user can find out more about the requirement

Can slice and dice descriptions when importing/linking to shells: can import only the parts that relevant to the importing context

Easy to build all sorts of summaries

But only beginning to make use of it for real documents If successful, parts of the model may see use in the other projects But see “Thoughts” slide as well!

10

Lassi A. Tuura, Northeastern UniversityApril, 2001http://cmsdoc.cern.ch/cmsoo.html

Architecture Documents…Architecture Documents…

11


Project Planning DocumentsProject Planning Documents

Starting to build a project planning database not unlike CAFÉ Task coordinators provide the necessary information Used in scheduling and resource planning

Collecting and maintaining information about tasks Task timing Resources required Task relationships

Currently producing simple reports Full task and deliverable descriptions, resource summaries,… Trivial CSV file for schedule: can import into MS Project, FastTrack,… Evolving a project management DTD; no well accepted standard

An interesting prototype, may integrate or cross-link with the CAFÉ documents and perhaps with XProject (by Torre Wenaus)

12


Processing SystemsProcessing Systems

Code cross reference (source: C++) Doxygen outputs web pages and class inheritance diagrams with

graphviz (could also produce LaTeX and man pages, we don’t need those)

User guides (source: LaTeX) LaTeX + dvips (PS) + ps2pdf (PDF); latex2html (web pages) Whines: latex2html is… uh, very limited

Tutorials Normal presentation tools (PowerPoint, StarOffice) Example code + documentation released with the projects

Architectural documents (source: XML) Norman Walsh’ DocBook XML/XSL stylesheets + local

customisations, Saxon 6.x (XSL processor; to web pages and XSL-FO), PassiveTeX from TeXlive 6 (XSL-FO to PS/PDF with xmltex + passivetex + latex + dvips or dvipdfm); image formats PNG/EPS (gimp for conversions)

13


Interactive Web ServersInteractive Web Servers

SCRAM has its own servers for release download and information on external requirements

LXR (Linux Cross-Reference) for browsing code Very impressive tool for browsing and searching code Complements Doxygen nicely Updated automatically

– Twice a day to re-index snaphots– Nightly to index new releases– Weekly to rebuild everything

Installed and operational for all projects; not yet publicly available

Other services under discussion cvsweb and bonsai: browsing the CVS repository/history Continuous snapshot build system (already have source and docs) Problem tracking system

14


To ComeTo Come

Web workbooks/guides Recognise the need for more than just the current user guide Lacking manpower

FAQs Maybe use FAQ-O-Matic?

Moving the infrastructure into a separate project Working towards “cookie-cutter” projects, documentation included Generating web pages automatically for all projects

A cool acronym !

15


ThoughtsThoughts

It is now fairly clear that XML/XSL is the future way to go… XML processing systems are maturing

– Outstanding HTML/Web output– Print quality improving—getting close or better than LaTeX– DocBook is in many ways more sophisticated than LaTeX

Shows as use of the tools in many large software projects– Linux & FreeBSD document projects, KDE, GNOME, …

Concerned about authoring cost in our community– Almost everybody is familiar with LaTeX– How to train writers to use Simplified DocBook?– XML is certainly reasonable possibility in projects that have fewer

developers who are keen on trying things out—but will it work in large projects with dozens or 100+ developers? (Or when will it work?)

Assessing quality and authoring ease in smaller projects first If successful, will discuss among developers in the larger projects