Beta/Kanga Tutorial -- May 15, 2000

This tutorial demonstrates how to run a very simple Beta job in two modes:
  1. reading data from the Objectivity event store and writing histograms to an Hbook file (Exercise 1); and
  2. reading data from a Kanga file and writing histograms to a ROOT file (Exercise 2).
and then, how to run a longer, more complicated job in batch (Exercise 3). The output of this last job can be used with ROOT. See the ROOT tutorial documentation for more information.

Note: This tutorial is most likely to be helpful to you if you have not run any Beta job in many months (or at all). If you have run Beta successfully in the last two or three months, you probably already know most of what's here.

This tutorial is created partially from material found in the BaBar Offline Workbook, the Kanga Home Page, and the Physics Analysis Recipes.

To execute this tutorial successfully, you will need:

Exercise 1. Reading from Objy and writing an Hbook file

In Exercise 1 we will:

  1. Logon to a Unix platform. At SLAC this is usually shire or tersk.

  2. We will be using the first of a new series of releases of BaBar software specifically designed for people doing physics analysis. It will incorporate only those changes that affect physics and will therefore tend to be more stable than the production releases.

    To initialize the software release tools for this release, called 8.6.2d-physics-1, and your platform, type:

       > srtpath 8.6.2d-physics-1
       Select/enter BFARCH (CR=1):
       1) SunOS5
       2) SunOS5-noOptimize-Debug
    (Text in brown indicates output from the shell or other program. Also, a carriage-return is always implied at the end of each line of user input.)

    Choose SunOS5 (or OSF1V4 or Linux2, depending on your platform) by typing "1" and hitting return.

    The srtpath command modifies your unix path so that any files you need while you work with BaBar code will obtained from the right place. Different files may be required for different architectures (Sun, Linux, OSF) or different configurations of the software (versions with or without debugging information, for example). We have chosen the version of the code that is optimized and contains no debug info, for speed and compactness.

  3. Select a directory to contain your test releases. (A test release is a directory tree on disk that will contain the necessary subdirectories to edit, compile, link, and run a BaBar analysis job.) Choose a location with at least 10 MB free space. We have chosen the directory mybabar, but you may choose any other, even your home directory. cd to this directory now.
       > cd mybabar
  4. If you haven't already, create your scratch directory, $BFROOT/work/<first_letter>/<your userid>. For example, if your userid was rfc:
       > mkdir $BFROOT/work/r/rfc
    Create your test release directory using the newrel command. Use the option
       -s $BFROOT/work/<first_letter>/<userid>
    to specify your scratch area. For example, user rfc would type:
       mybabar> newrel  -s $BFROOT/work/r/rfc -t 8.6.2d-physics-1 Beta_tt
    The last argument provides the (arbitrary) name for the new directory that will contain the test release.

    (The unix prompt will be indicated by ">" unless the command must be issued from a particular directory, in which case, the deepest subdirectory will appear, in this case, "mybabar>.")

    It is possible to incorporate several executable versions of the code in the same test release, for different architectures, for example. For more information on this click here.

  5. Switch to your new test release directory:
       mybabar> cd Beta_tt
  6. Now we add the packages we will need:
       Beta_tt> addpkg workdir
       Beta_tt> addpkg BetaUser
  7. Next, select the Objectivity federation where the data you want to use resides. To use recent BaBar data, you would ordinarily type:
      Beta_tt> analboot2
    But, if the production federations are blocked, as they would ordinarily be during the hour of this tutorial, type instead:

       Beta_tt> setenv OO_FD_BOOT /nfs/objyserv6/objy/databases/test/physics-demo/V1/BaBar.BOOT
  8. This will specify a private federation that has been prepared for this tutorial.

  9. Switch to the BetaUser directory and copy in a simple example from the BaBar offline workbook, then switch back to your test release:
       Beta_tt> cd BetaUser
       BetaUser> cp $BFROOT/www/doc/workbook/examples/NTrkExample/* .
       BetaUser> cd ..
    (Note: the "." at the end of the second line is part of the copy command!)

  10. Now we can compile and link. Always use batch for this and always do this from your test release top-level directory and always use the bldrecoq queue:
       Beta_tt> bsub -q bldrecoq -o Exercise1.log gmake lib ROPT=-noDebug-Shared
    (The procedure at non-SLAC sites may be different; check with your local guru.)

    "-noDebug" turns off debugging symbol generation, for speed and code size, and "-Shared" allows you to use shared libraries, which means that if you modify your own code, you need only re-compile it, and re-linking is not necessary. The ROPT variable may be set to achieve the same effect:

       > setenv ROPT -noDebug-Shared
  11. We can monitor the progress of the batch job with:
       > bjobs      (use option -l for more info)
    (Text in green is informational; you don't type it.)

  12. When the compile has finished successfully, submit the link job:
       Beta_tt> bsub -q bldrecoq -o Exercise1.log gmake BetaUser.bin ROPT=-noDebug-Shared
  13. When the link has completed, switch to the workdir from which the job will run:
       Beta_tt> cd workdir
  14. Now prepare the workdir directory for use:
       workdir> gmake setup
    This need be done only once, before the first time you run from it.

  15. We have chosen to analyze run 12125. We need to determine the collection name for the event collection containing the reconstructed physics data for this run. There are several ways to do this. We are going to prepare a .tcl file containing the necessary instructions. To do this for run 12125, type:
       workdir> getdata -t allevents -b12125 -e12125 -v
    This will produce a file called allevents_12125-12125.tcl, containing the tcl commands which can be source'd at run time to choose your file(s). To examine the contents of this file, type:
       workdir> cat allevents_12125-12125.tcl
       # allevents collections from analboot2
       mod talk BdbEventInput
         input add /groups/isPhysicsEvents/0001/2100/P8.6.1aV00fb/00012125/cb001/allevents
       # No. of allevents events is 50564 for runs 12125-12125
       # The production federation for these collections is analboot2
    If the production database is blocked, we instead do the following:
       Beta_tt> setenv BdbInputCollection /groups/isMultiHadron/0001/2100/P8.6.1aV00fb/00012125/cb001/allevents
    (If you'd like more information on finding and setting input collections, click here.)

  16. We're ready to go. We start the run by invoking the program betarun with the name of the relevant .tcl file as an argument:
       workdir> betarun ../BetaUser/bdbMicro.tcl
  17. If we are using the special private federation, you have already specified the correct collection by setting the BdbInputCollection variable. Otherwise, do this now by typing:
       > source allevents_12125-12125.tcl
  18. Now process three events:
       > ev beg -nev 3
  19. Now let's continue with the next 47 events for a total of 50:
       > ev continue -nev 47
  20. When it finishes and leaves you at the tcl prompt again, type:
       > exit
  21. We can now look at the Hbook output. Enter:
       workdir> pawX11
    Hit a carriage return to select the default Xwindows output.

  22. Open the Hbook file and display its contents:
       PAW > h/fil 1 framework.hbook
       PAW > h/lis
  23. Set some optional statistics display parameters and then display the momentum histogram:
       PAW > opt stat
       PAW > set stat 111111
       PAW > h/pl 1
  24. Exercise 1 is complete. Exit from paw:
       PAW > exit

Exercise 2. Reading from Kanga and writing a ROOT ntuple file

In Exercise 2 we will:

A bit of introduction:

Kanga (a Kinder ANd Gentler Analysis) provides files containing the same information as that in the Objectivity event store and conditions database, but without the complexities of Objectivity. User code is exactly the same in each case, but the executable must be remade for Kanga. (It's also possible to make a dual-purpose executable that can be used for both types of data, depending on the settings of some environment variables.) The advantages of using Kanga are increased throughput and access to data when the production databases are blocked.

Kanga files can be used in any analysis that depends only on variables contained in the micro-dst. If you need to study hits in the drift chamber, for example, you must get your events from Objectivity raw data.

ROOT is a histogram and analysis package from CERN, and is supposed to be a modern replacement for HBook/PAW (although HBook files can be converted to be used in ROOT). The data in Kanga files are stored in ROOT trees (similar to HBook ntuples), but reading Kanga files doesn't commit you to using ROOT to make your histograms. You can read Kanga files and produce HBook output, and you can read Objectivity files and produce ROOT output.

Let's begin:

  1. Go back to the directory you selected in Exercise 1 to contain that test release (the one where you did the first newrel command). For example:
     > cd ~/mybabar
  2. We assume the software release tools are still set up for your platform and release 8.6.2d-physics-1. If not (for instance, you have logged out and back in since doing Exercise 1), initialize the software release tools for release 8.6.2d-physics-1 and your platform again:
       mybabar> srtpath 8.6.2d-physics-1
    Choose SunOS5 (or OSF1V4 or Linux2) by typing "1" and hitting return.

  3. Create a new test release directory and cd to it. (Remember to replace r/rfc in the example below with your userid and first letter.)
       mybabar> newrel -s $BFROOT/work/r/rfc -t 8.6.2d-physics-1 Kanga_tt 
       mybabar> cd Kanga_tt
  4. Add packages and initialize workdir:
       Kanga_tt> addpkg BetaUser
       Kanga_tt> addpkg workdir
       Kanga_tt> gmake workdir.setup
  5. Now set some environment variables. (Again, don't enter the text in green--it just explains the purpose of each variable.)
       > setenv BetaKanga         yes  (allows reading from Kanga File)
       > setenv BetaBdbMicro      no   (disables reading from the database)
       > setenv BetaRootTuple     yes  (make a ROOT, rather than an Hbook, file)
    As before, for convenience and speed:
       > setenv ROPT -Shared-noDebug
  6. Copy in the example from the workbook:
       Kanga_tt> cp $BFROOT/www/doc/workbook/examples/NTrkExample/* BetaUser/
  7. Compile and link. Since we know the code will compile and link, we can combine the two steps and save a bit of time:
       Kanga_tt> bsub -q bldrecoq -o Exercise2.log gmake lib BetaUser.bin
    (The ROPT option has been taken care of in by setting the environment variable.)

  8. Make the .tcl to select the input collection (we will use run 12125 again):
       Kanga_tt> cd workdir
       workdir> getdata -t kanga -b12125 -e12125 -v
    This will produce the file kanga_12125-12125.tcl, which will be source'd as above.)

  9. Now we're ready to run a few events:
       workdir> betarun ../BetaUser/kanga.tcl
  10. When we get the tcl prompt, we must select the run, and change the magnetic field setting (this is a temporary fix). Enter the following:
       > source kanga_12125-12125.tcl
       > module talk RooBuildEnv 
       RooBuildEnv>   FixedFieldStrength set 1.51007 
       RooBuildEnv> exit 
    Also, to avoid a bunch of annoying messages you can type:
       > module disable DefaultKlongMicroSelection
    but there's no harm if you don't.

  11. Now we can continue. Let's try 100 events this time:
       > ev begin -nev 100
       > exit
  12. Now you can inspect the ROOT output file:
       workdir> bbrroot
       root [0] in = TFile("framework.root");      (read in the file)
       root [1];                           (list the histograms and ntuples)
       root [2] h1d1->Draw();                      (display a histogram; those are "ones," not lower-case "L"'s.)
       root [3] .q                                 (exit Root)

Exercise 2 is done.

Exercise 3. Producing a more complicated and longer output file

In this exercise we will:

  1. We will be picking up the sample analysis from the BetaExamples package. By adding the package, you will be able to examine the routine.
       workdir> cd ..
       Kanga_tt> addpkg BetaExamples
  2. We need to modify four routines in BetaUser:, myAnalysis.tcl, kanga.tcl (which we will rename kanga_batch.tcl), and We will do this below.

    However, you can skip this if you want: already modified versions of these files may be found in $BFROOT/www/doc/tutorials/15May2000_BetaKanga_Tutorial/kanga_batch. If you'd rather, just copy the contents of this directory into BetaUser and then skip to the compile and link step below. Otherwise switch to the BetaUser directory

     >  cd BetaUser
    and begin editing:

    The edits are done. Switch back to the main test release directory:
     BetaUser>  cd ..
  3. Now compile and link just as before:
       Kanga_tt> bsub -q bldrecoq -o Exercise3.log gmake lib
    when it completes, do
       Kanga_tt> bsub -q bldrecoq -o Exercise3.log gmake BetaUser.bin
    or, again, if you're brave, you can do both at once:
       Kanga_tt> bsub -q bldrecoq -o Exercise3.log gmake lib BetaUser.bin
    and save yourself one "PENDING".

  4. Now go the the workdir directory.
       Kanga_tt> cd workdir
    To run the job, type:
       workdir> bsub -q kanga -o Exercise3.log ./betarun ../BetaUser/kanga_batch.tcl
    The batch queue kanga is specially set up to have access to a local copy of the kanga datasets, and therefore runs more efficiently.

    Note: To run a batch job reading an Objectivity collection, follow a similar procedure, but the job must be submitted to the bfobjy batch queue. Also the collection name must be specified using one of the methods suitable for the BdbEventInput module as in Exercise 1.

  5. The output of this job is, again, the file framework.root. If we don't have time today to actually run the job, a copy of this output file can be found in $BFROOT/www/doc/tutorials/15May2000_BetaKanga_Tutorial/Exercise3.root.

    To look at the output:

       workdir> bbrroot
       root [0] in = TFile("framework.root");      (read in the file)
       root [1];                           (list the histograms and ntuples)
       root [2] h1d3->Draw();                      (display a 1-d histogram)
       root [3] h2d4->Draw();                      (display a 2-d histogram)
       root [n] .q                                 (exit Root)
You can use this file in the ROOT tutorial to learn more about ROOT.

Ray F. Cowan
Leon Rochester
Last modified: Thu Jul 6 11:21:20 PDT 2000