SLAC PEP-II
BABAR
SLACRAL
Babar logo
SPIRES E S & H Databases PDG arXiv
Organization Detector Computing Physics Documentation
Personnel Glossary Sitemap Search Hypernews
Unwrap page!
Home
Workbook
 1. Introduction
 2. Accnt. Setup
 3. QuickTour
 4. Packages
 5. Modules
 6. Event Info.
 7. Tcl Cmds.
 8. Editing
 9. Comp.+Link
 10. Run the Job
 11. Debugging
 12. Parameters
 13. Tcl Files
 14. Find Data
 15. Batch
 16. Analysis
 17. ROOT I
 18. Kanga
Additional Info.
 Other Resources
 BABAR
 Unix
 C++
 SRT/CVS Cmds.
 SRT/CVS Dev.
 Sim/Reco
 CM2 NTuples
 Root II, III
 PAW I, II
 tcsh Script
 perl Script
Check this page for HTML 4.01 Transitional compliance with the
W3C Validator

(More checks...)

Packages and Releases

About BaBar releases and packages: what they are, how to check them out, BaBar file organization, and SRT/CVS.

Contents:


Packages

BaBar analysis software is organised in terms of packages. A package is a collection of files intended to perform a well-defined task. For example, a package might find calorimeter clusters, or simulate the drift chamber response. Each package has a unique name and its own library and include files. Some packages are independent, while others work only as a group. For example, the GEANT simulation of BaBar is made up of individual subsystem simulation packages that only work together. There are several different types of packages. The core reconstruction packages translate the data from raw detector signals into particle candidates. The simulation packages generate simulated events in the same format as real data. Finally, user analysis packages take the reconstructed and simulated data provided by the core reconstruction and simulation packages, and organize and store it in a format that is useful for the user's analysis.

BaBar's software is organized so that most users need only two packages: an analysis package, and the workdir package. The analysis package performs the analysis: it takes the data, analyzes it, and stores it (usually as a histogram or ntuple in a ROOT file). The analysis package "knows" where the other packages are, and works with them to perform the analysis, but the user rarely has to deal with the other packages directly. The workdir package is needed to run the analysis package's application. For example, in the Quicktour, you used the analysis package BetaMiniUser. You also needed the workdir package to run BetaMiniUser's application, BetaMiniApp.

There are many different analysis packages available at BaBar, each optimized for a different type of analysis: packages for studies of semileptonic decays, B-decays, charm decays, tau decays, 3-body decays, and many more. All the analysis packages access the same basic database of event information, but they do different things with it, depending on the needs of the analysis. For example, packages for studies of semileptonic decays store lots of information about the lepton, and compute lots of lepton-related quantities. Meanwhile, packages for studies of 3-body decays compute and store variables relevant to 3-body decays. Most user analysis packages are called SomethingUser: for example, BetaMiniUser, CharmUser, PenguinUser, RhoGammaUser, SemiExclUser, and so on. Most Analysis Working Groups (AWGs) maintain their own analysis package or packages. Once you are ready to perform your own analysis, your AWG will advise you as to what is the best analysis package to choose.


Software Structure

BaBar File Organization

BaBar software and other files are currently located on a NFS (Network File System) or AFS (Andrew File System) file system, which are mounted on all the public UNIX workstations at SLAC. This means that most BaBar file locations begin with "/afs/" or "/nfs/". However, most references to the BaBar file system should be made through the environment variable $BFROOT. In this way users are protected from changes in the physical location of the disks.

The following illustrates the $BFROOT directory and its subdirectories.

                                      $BFROOT/
                                         |
   +-------------------------------------+-------------------------------------+
   |                                                                           |
  bin data demo dist doc etc fonts include lib man package prod repo src www  ...
  

Quick guide to the subdirectories of $BFROOT.

The bulk of BaBar software is organised and maintained in a single, self consistent file system under $BFROOT/dist (for distribution). This directory may also be accessed through the BFDIST environment variable. Packages are stored in $BFROOT/dist/packages, and releases are stored in $BFROOT/dist/releases. The following tree illustrates how these files are organized. (There was not nearly enough room in the tree to include all the files in each directory, so only a sample is shown at each level.)


                                   $BFROOT/
                                      |
                                     dist
                                      |
                    -----------------------------------------
                    |                                       |
                 packages                                releases
                    |                                       |
        --------------------------          --------------------------------------------
         |             |        |             |       |         |           |        |
      BetaMiniUser  workdir  HepTuple      16.1.1.   18.6.3  analysis-42  current  newest
                       |                                        |
       ----------------------------------            --------------------------
       |          |          |          |            |             |          |
   V00-04-15  V00-04-17  V00-04-20  V00-04-21    BetaMiniUser   workdir    HepTuple
                             |                                     |
       ---------------------------------               --------------------------------
      |            |        |          |               |          |        |          |
   GNUmakefile   README  RooAlias.C  RooLogon.C   GNUmakefile  README  RooAlias.C  RooLogon.C

  

Or, to summarize:

  $BFROOT/dist/packages  -->  packages  -->  versions  -->  package files
  $BFROOT/dist/releases  -->  releases  -->  packages  -->  package files
  

Look first at the packages directory. Within each package subdirectory, there are usually several "versions" of the package, labeled with tags like "V00-04-17". When new, improved versions of packages are developed, the old versions are not deleted. Instead, the new packages are just assigned a different tag. That way, users who depend on the old package do not have to worry. The higher the tag number, the more recent the version. Each version subtree then contains all of the files that you would get if you checked out that version of the package.

The release directory is a bit different. As mentioned above, a BaBar release is a self-consistent set of packages. So each release contains exactly one version of each package, no more. Therefore, there is no need for a subtree for different versions.

The files in $BFROOT/dist/ are actually just reference copies. The actual development code is maintained using two systems called SRT and CVS. In order to use BaBar code (as opposed to just reading it in $BFROOT/dist), you need to check out packages and releases using SRT commands, as described below. Every package has a Package coordinator who is responsible for maintaining the package.

Release Numbers and Nicknames

If you look in $BFROOT/dist/releases, you will see a long list of release names. Most of the names are just numbers, such as 14.3.2a, 15.8.0, 16.1.1d, 17.1.0, and 18.0.4b. The higher the number, the newer the release. Releases with the same first number (in the same series) are very similar. For example, releases 18.3.3 and 18.6.1 in both 18-series releases, so the differences between them are probably small. But releases in different series, like 14.5.5 and 16.1.3, are probably very different. Every release has a number. It is the release's "real name". But some of the names listed in $BFROOT/dist/release are words, not numbers. These names are actually soft links to a numbered release. These symbolic releases are defined as follows:

newest
The most recently built release. The only quality requirement is that the release (mostly) compiled.
current
This release is the most recent to have reached an acceptable level of quality. "Acceptable" is determined by a combination of code and physics checks as defined by the QA/QC group with input from the physics groups and detector subgoups.
analysis-xx
These releases are successful physics analysis releases that have been tested and found stable for analysis jobs. analysis-42, the release used in the Workbook, is one such example.
Other symbolic release names exist, such as "online" and "database", are defined as and when required. The "nightly" is the culmination of everyone's submitted changes to individual packages. Each night SLAC attempts to compile the very latest version of the code with all changes and emails are automatically sent to package coordinators with details of what crashes. The release analysis-42,
  > ls -l $BFDIST/releases/analysis-42
  lrwxr-xr-x    1 gpdf     bfactory        6 Sep  4 12:36 /afs/slac.stanford.edu/g/babar/dist/releases/analysis-42 -> 22.3.0
  
is a actually a link to release 22.3.0.

What analysis release should YOU use?

The best way to determine what release you should use is to consult other BaBar people, like your advisor or AWG. Otherwise, you can check the Physics Releases page to see what the currently recommended analysis release is. Or you can check what release is at the top of the Extra Tags Page. Release analysis-42, was the best release to use for a full analysis job at the time of writing (Jan 2008). In general, it is usually safe to use the most recent analysis-xx release.


SRT and CVS: Babar's Software Release System

Babar's software release system is called SRT, for Software Release Tools. It is based on a popular non-BaBar system called CVS (Concurrent Versions System). Developers use the SRT/CVS system to develop and maintain code. Users performing a physics analysis use the SRT/CVS system to check out and run BaBar code.

Using SRT

The basic principle behind SRT, from a user's point of view, is that users create their own private copies of the releases in $BFDIST/releases. In the user's case, however, a release is typically a very lightweight collection of directories and is tied to a full BaBar release. With the initialisation and creation of a user's release comes a GNUmakefile which "knows" where to look for binaries, libraries, and include files of the official BaBar release.

For example, in the Quicktour you created a test release based on release analysis-50, which contains many packages. However, the only packages that you need to check out are the workdir package (needed to run the application) and any other package for which the default tag (the tag in $BFDIST/releases/analysis-50) is not good enough. In the Quicktour you added BetaMiniUser, because you wanted to modify some of the code in BetaMiniUser. The release "knows" that when it is looking for code, it is supposed to look in your personal release first, and in $BFDIST/releases/analysis-50 second.

SRT commands

The following sections describe the most important SRT commands for users: newrel to create a new release; srtpath to set up the SRT path, and addpkg to add a package.

Creating a Test Release: newrel

One of the first steps taken in the sample analysis job run in the Quicktour was to create a new release. The SRT command to create a new release is newrel. There are two types of releases supported: test and production. Production releases are created by advanced BaBar code-writers, for eventual inclusion in $BFDIST/releases. A test release is a private copy of a release for the user's own analysis. At this time we will use and discuss test releases only. The syntax to create a new test release is:

  newrel -t <baserel> <testrel>
  
where baserel is name of one of the base releases listed in $BFROOT/dist/releases, and testrel is an arbitrary name that you choose for your test release. This creates a new subdirectory called testrel in the current directory. This new release will contain several empty directories (bin, doc, include, lib, man, results and tmp). It also has links to the base release, and a GNUmakefile. The GNUmakefile has been copied from the base release and is the master one for the release. It is this make file that "knows" where all the libraries and include files are. That is, when using gmake to compile an executable in testrel, the appropriate include and lib directories of the base release will be searched for any code not in found in testrel. For example, in the Quicktour analysis, the command
  newrel -t analysis-50 ana50  
created a test release named ana50 based on the analysis-50 release (also known as 24.3.2).

Scratch Directories

The full command used in the Quicktour was:

  newrel -s $BFROOT/build/<first_initial>/<username> -t analysis-50 ana50
  
The command-line option for newrel, -s <scratchdir>, puts the lib, bin, and result directories in a "scratch" directory scratchdir, with symbolic links from their normal places in the release directory. The complete syntax is:
  newrel -s <scratchdir> -t <baserel> <testrel>
  
The advantage of using a scratch directory is that the bulky lib, bin and result directories will not take up space in your personal home directory. The disadvantage is that the contents of these directories are deleted periodically (about once a week, depending on the scratch directory). If this happens, then you will not lose the C++ code in your test release, but you will have to rebuild (re-gmake) your libraries and binaries.

The recommended scratch area for BaBar builds is:

$BFROOT/build/<first_letter>/<username>
For example, if your username is "kiwi", you would use:
$BFROOT/build/k/kiwi
This scratch area is attached to a fast server to allow it to handle a high volume of I/O. You also have a scratch directory at:
$BFROOT/work/<first_letter>/<username>
but this one is not as good for builds. There are other non-BaBar-specific NFS disk areas which can also be used, but they aren't as good as $BFROOT/build, either. If you want, you can define the environment variable MYWORK to this directory and pass it an argument to the newrel command. For example:
  setenv MYWORK $BFROOT/work/k/kiwi
  newrel -s $MYWORK -t analysis-50 ana50
  
The first command sets the MYWORK environment variable to the pathname of the user's scratch space (a directory called ${USER} in the babar work space).

Setting The Path: srtpath

The first thing you should do after creating a new release is issue the srtpath command from the release directory:
  > cd testrel
  testrel> srtpath <rel> <arch>
  
This command sets up several important variables for BaBar's Software Release environment:
  • BFCURRENT = Release number.
  • BFARCH = Architecture of current machine.
  • PATH = Search path for executables.
  • MANPATH = Search path for man pages.
In the Quicktour, after you checked out your release, you did:
  > cd ana50
  ana50> srtpath
  
The system responded:
  enter release number (CR=22.4.3):
  
When you pressed <enter>, you accepted the default release offered, namely 22.4.3. The system then offered you a choice of architectures:
  Select/enter BFARCH (CR=1):
  1) Linux24SL3_i386_gcc323                     [prod][test][active][default]
  2) Linux24SL3_i386_gcc323-noOptimize-Debug    [prod]
  3) Linux24RHEL3_i386_gcc323                   [default2]
  
When you pressed <enter> again, you accepted the default rchitecture, which was "1) Linux24SL3_i386_gcc323". In general it is a good idea to accept srtpath's default, because it usually knows what it is doing. Now the environment variables should be set:
  > echo $BFCURRENT
  22.4.3
  > echo $BFARCH
  Linux24SL3_i386_gcc323
  > echo $PATH
  /usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/etc:/usr/afsws/bin:/usr/afsws/etc:/usr/bin/X11:
  /cern/pro/bin:.:/afs/slac.stanford.edu/g/babar/bin:/afs/slac.stanford.edu/g/babar/package/objy8.0.9
  /babar/linux86gcc3/bin:./bin/Linux24SL3_i386_gcc323:/afs/slac.stanford.edu/g/babar/dist/releases
  /22.4.3/bin/Linux24SL3_i386_gcc323
  > echo $MANPATH
  ./man:/afs/slac.stanford.edu/g/babar/dist/releases/22.4.3/man:::/afs/slac.stanford.edu/g/babar/man
  
So srtpath has ensured that you are using the correct binarires and man pages for the given release, and the correct architecture for the machine that you are using. The primary use of srtpath is to set up your working environment to access the appropriate code. The syntax of this command also supports developers who want to switch between architectures, and allows for explicit architecture assignment in locations where Hepix scripts do not exist. Incidentally, the srtpath command is actually an alias for running a script. We can see this by issuing the command:
   > which srtpath
     srtpath: aliased to set dialog=1; set argument=(!*);
                         source $BFROOT/bin/SRTstartup.csh;
                         unset dialog argument
  
To ease the network access, a remote site can install srtpath.csh locally and set an alias pointing to this C-Shell script.

IMPORTANT: The srtpath command applies only to your current session. If you logout, and then log back in, you will have to issue srtpath again.

Checking Out Packages: addpkg

When you check out a test release (newrel -t), no actual packages are included. There are links to the packages in the base release, but you cannot modify the base packages. If you want to make even small changes to a package, you have to add a copy of the package to your test release. The SRT command to check out packages is addpkg. The syntax is:

addpkg <pkgname>
where pkgname is the name of the package. A directory with the same name and contents as the package will be created in your test releasedirectory. If you addpkg a package which already exists in your release, then addpkg will, after confirming with you, update the package directory to the version appropriate for the release and will then check that the required links (in the include directory) are present. The addpkg command also issues the gmake installdirs command. This command creates architecture-specific directories for storing the packagebinaries. The addpkg command has an optional argument that allows you to specify a version tag:
addpkg <pkgname> [<version_tag>] 
If the version tag is not specified, then you will get the default version of the package that comes with your release. (Recall that each release has exactly one version of each package.) One of the more common situations in which the <version_tag> argument is used is when a fix is needed. When developers discover a bug or problem in one of the important stable releases, they create a new bug-free package, and post the tag version number on the following page:

Extra Tags Page Adding the bug-free package fixes the bug. Whenever you check out a new release, make sure that you check this page to find out what extra tags are needed for your release. To see a list of packages, issue the showtag command from your test release:

  ana50> showtag
  BetaMiniUser V00-04-05
  workdir V00-04-21
  

More SRT/CVS commands

Additional useful SRT/CVS commands for tracking code changes, keeping track of your tags, and updating packages and releases can be found in the Useful SRT commands section of the Workbook.


workdir: The Work Directory

In the Quicktour analysis a work directory was created from within the release directory with the command:
addpkg workdir
workdir is the package from which all BaBar applications are run. No matter what other packages you are using, you will always need workdir as well. In the Quicktour analysis job you used the workdir to run BetaMiniApp, the executable for BetaMiniUser. (That was why you had to cd to workdir before you typed "BetaMiniApp snippet.tcl".) Before you can use workdir, you have to set it up with the command:
  testrel> gmake workdir.setup
  

Or, from workdir itself,

  workdir> gmake setup
  
"setup" is one of the targets defined in the GNUmakefile of the workdir package. The make process is initiated with the gmake command. Targets and make commands will be discussed in more detail in the Compile and Link chapter of the Workbook. The main result of gmake setup is that many symbolic links are created. A few of the key links are:
RELEASE
By default this is a link to the directory one level up in the tree, the release directory: ../
PARENT
This is a link to the production release directory under BFDIST that corresponds to the base release with which the new release was created: $BFROOT/dist/releases/<baserel>
bin
This link points to the bin directory of the RELEASE directory: RELEASE/bin/
The workdir package will be further discussed in the Run the Job section of the Workbook. At this point the new release and work directories have been created and ready for the user to begin using standard analysis tools, modifying or writing code, and other forms of work.

A Note About User Analysis Packages

One last thing you should know about user analysis packages is that many of them are NOT included in the releases. This means that they are listed in $BFROOT/dist/packages but not in $BFROOT/dist/releases. For example, BetaMiniUser is included in releases, but CharmUser is not. The reason for this is that many user analysis packages are of interest to only a few users, so the team that maintains them is too small and busy to keep up with BaBar's demanding standards for release-quality packages. You can still add and use any of these packages, but they have not been tested by BaBar Central to make sure that they are compatible with a given release.


General Related Links:
Online directory of BaBar packages
Online directory of BaBar releases

Back to Workbook Front Page


[Workbook Author List] [Old Workbook] [BaBar Physics Book]

Valid HTML 4.01! Page maintained by Adam Edwards

Last modified: August 2008