Workbook for Offline Users: New Packages: Making a
new Analysis package based on BetaMiniUser
This chapter describes how to create a new analysis package based on
the BetaMiniUser package. This is useful for users wanting to write their
own Beta analysis and to use the powerful CVS tools to maintain the
package. In this section step-by-step instructions are provided to
help users make all the necessary changes required to get their new
package into a state that it can be included in the BaBar CVS
repository as a unique analysis package.
At this stage, the analysis package you will create will belong to
you only, and will not yet exist in the BaBar CVS repository.
The following workbook chapter New Packages: Getting your package
into CVS describes how to get your new, tested and operational,
package into the BaBar framework and into official releases.
Contents
The general procedure to set up a new package and integrate it into
the BaBar framework is a two-step process:
- Create and test your new analysis package. You may have
been modifying BaBar packages for your own use for a while now, in
which case the instructions below will just give step-by-step instructions
of how tochange the package to something uniquely named so that it can be put
into the BaBar framework. Or you may be relatively new to
editing code, in which case this example may teach you a bit about
how a BaBar package works.
- Add your package to the BaBar repository
by testing it carefully, requesting the creation of a new package
(basically telling the computing gurus that you have a new package
called TopUser) and starting to use your new package in CVS
The first step is described in this chapter, while the second step
is explained in the next WorkBook chapter New Packages: Getting your package
into CVS. Making a new analysis package is usually a good idea as
it allows several people to work together on analyses and commit their
changes using all the tools of CVS, and also makes available the tools
developed by the group for the whole collaboration. Additionally,
making source code available can be useful at the review stage to make
all steps in the analysis transparent and testable. Additionally, CVS
allows your code to keep in step with different releases, allowing you
to tag specific versions of your code that work with specific BaBar
code releases. Note also that you should NEVER check code back into
BetaMiniUser (or any other package) without specific permission of the
package coordinators.
You should name your package to give a clear indication of its role,
and all the files within that package should be similarly
named. Three-letter acronyms are favoured within BaBar.
Analysis packages written for specific physics groups are commonly
named as SomethingUser, for example, BetaMiniUser,
TauUser, CharmUser, etc.
Packages that are related to detector subsystems should be named
with the three-letter acronym that is appropriate to the subsystem,
such as Emc, Drc, Dch, etc.
To allow specific examples to be used throughout this chapter, we
shall describe the procedure to set up a test release called top-a30,
based on analysis-30 and create a new analysis package that we shall
call "TopUser", based on the BetaMiniUser package.
To start, follow the WorkBook QuickTour and check out a BaBar
software release: newrel -t etc., with the test release name top-a30 ,
(see the QuickTour section
of the Workbook). It is recommended at this stage that you base your
test release on analysis-30.
Now check out the workdir and BetaMiniUser packages in the usual way
and set up the workdir package:
~> cd top-a30
top-a30> srtpath <enter> <enter>
top-a30> addpkg workdir
top-a30> gmake workdir.setup
top-a30> addpkg BetaMiniUser
Now copy the files in BetaMiniUser into a new directory called TopUser:
top-a30> mkdir TopUser
top-a30> cp BetaMiniUser/* TopUser
The system will probably respond,
cp: omitting directory `BetaMiniUser/CVS
That is because this command copies only the files (not the subdirectories
or their contents) to TopUser. But that is what you want, because
BetaMiniUser's only directory is the CVS directory, which holds information
about the CVS repository where the BetaMiniUser package resides. Our new
package isn't yet in CVS (it doesn't actually exist yet), so it would be
inappropriate to have a CVS directory at this point.
To check what went in there:
top-a30> cd TopUser
TopUser> ls
AppUserBuild.cc MyReadList.hh
AppUserBuildBase.cc MyReadList.tcl
AppUserBuildBase.hh MyReadListSnippet.tcl
AppUserBuildRoo.cc MyReskimStreamPhysics.tcl
BetaMiniPatches.tcl MyTimePointAnalysis.cc
BetaMiniPidKilling.tcl MyTimePointAnalysis.hh
BetaMiniUserProduction.tcl MyTimePointAnalysis.tcl
GNUmakefile NamedScalers.cc
History NamedScalers.hh
MyDstarAnalysis.cc README
MyDstarAnalysis.hh RewriteMini.tcl
MyDstarAnalysisSnippet.tcl bdbMini.tcl
MyDstarMicroAnalysis.tcl bdbMiniPhysProdSequence.tcl
MyDstarMiniAnalysis.tcl bdbMiniPhysics.tcl
MyDstarPhysics.tcl bdbMiniQA.tcl
MyHistManager.cc bin_BetaMiniApp.mk
MyHistManager.hh bin_BetaMiniRooApp.mk
MyK0Analysis.cc bin_BetaMiniUser.mk
MyK0Analysis.hh binlist
MyK0MicroAnalysis.tcl btaMini.tcl
MyK0MiniAnalysis.tcl btaMiniPhysProdSequence.tcl
MyMiniAnalysis.cc btaMiniPhysics.tcl
MyMiniAnalysis.hh btaMiniQA.tcl
MyMiniAnalysis.tcl link_BetaMiniUser.mk
MyReadList.cc
If you have followed the workbook up until this point, then
you will recall the PExample module that you made in the
Editing Code and
Compile,link,run sections of the Workbook.
In the following example you will add the PExample module
to your new TopUser package. So you need to copy the PExample files
to your TopUser directory:
TopUser> cp $BFROOT/www/doc/workbook/NewExamples/PExample/* .
(Don't forget the dot "." at the end.)
Now you will have two new files in TopUser - PExample.cc
and PExample.hh - as well as modified versions of AppUserBuildBase.cc
and MyMiniAnalysis.tcl. You may also have a
file named header.inc. This is a Workbook file that makes Workbook
pages look nice - it has nothing to do with PExample, so you should
remove it.
The next step is to get rid of all of the files that are not needed in
your new package. Your goal is to produce a package with just one module -
the PExample module - plus any modules needed to run PExample. As you
saw in the Quicktour, you need the MyMiniAnalysis module, but the others
can go:
TopUser> rm MyDstar*
TopUser> rm MyK0*
TopUser> rm MyTime*
TopUser> rm MyRead*
You also need to edit AppUserBuildBase.cc and MyMiniAnalysis.tcl to remove
all reference to these modules. For example, for the MyDstarAnalysis
module, you will need to remove the lines
#include "BetaMiniUser/MyDstarAnalysis.hh"
and
theBuild->add(new MyDstarAnalysis("MyDstarAnalysis", "Dstar analysis module"));
from AppUserBuildBase.cc, and remove
module disable MyDstarAnalysis
from MyMiniAnalysis.tcl. The lines for the other modules are similar (or
nonexistant).
It is probably a good idea to remove the History and README files
as well. These files are not part of the code, they just provide
information about the package. But as they contain many references to
files in the package, they may get in the way when you are tracking down
dependencies with grep.
TopUser> rm History
TopUser> rm README
With regard to the other files, it is not obvious whether they need to be
removed or not. To find out, use grep (the unix search utility) to
see where they are used. For example, let's grep all the files in
TopUser for BetaMiniPatches:
TopUser> grep BetaMiniPatches *
BetaMiniPatches.tcl:# $Id: BetaMiniPatches.tcl,v 1.3 2003/04/08 18:13:04 brownd Exp $
BetaMiniPatches.tcl:# puts "BetaMiniPatches: Setting DchBuildEnv::dedxCalibBkgStrategy to false"
btaMini.tcl:set patches BetaMiniPatches.tcl
The first two lines are in BetaMiniPatches.tcl itself, so you do not
need to worry about them. However, the third line shows that btaMini.tcl
uses BetaMiniPatches.tcl. So if you are going to keep btaMini.tcl, then you
will also have to keep BetaMiniPatches.tcl.
Similar investigations of the other files will reveal that you can
remove the following:
TopUser> rm BetaMiniPidKilling.tcl
TopUser> rm BetaMiniUserProduction.tcl
TopUser> rm MyReskimStreamPhysics.tcl
TopUser> rm RewriteMini.tcl
TopUser> rm bdb*
TopUser> rm btaMiniQA.tcl
TopUser> rm NamedScalers*
TopUser> rm header.inc
TopUser> rm MyHistManager*
Now you are left with the following files:
TopUser> ls
AppUserBuild.cc MyMiniAnalysis.hh binlist
AppUserBuildBase.cc MyMiniAnalysis.tcl btaMini.tcl
AppUserBuildBase.hh PExample.cc btaMiniPhysProdSequence.tcl
AppUserBuildRoo.cc PExample.hh btaMiniPhysics.tcl
BetaMiniPatches.tcl bin_BetaMiniApp.mk link_BetaMiniUser.mk
GNUmakefile bin_BetaMiniRooApp.mk
MyMiniAnalysis.cc bin_BetaMiniUser.mk
As you edit, your system may automatically save backup copies of
some files. These files have the same name as the original, except
for a tilde (~) at the end. I find it easier to delete these files as they
appear, so that they do not cause problems when I use grep
to check for dependencies.
The first thing you should now consider doing is renaming your source code
files so that the names are indicative of belonging to a package called
TopUser. You also need to do a bit of renaming to let the files know they
are now in the package TopUser, not BetaMiniUser any longer.
To do this, grep for all instances of BetaMiniUser in the source code.
TopUser> grep BetaMiniUser *.cc
TopUser> grep BetaMiniUser *.hh
TopUser> grep BetaMiniUser *.tcl
Doing so reveals that there are six files that you need to
edit, and change each instance of BetaMiniUser to TopUser:
AppUserBuild.cc
AppUserBuildBase.cc
AppUserBuildBase.hh
AppUserBuildRoo.cc
PExample.cc
MyMiniAnalysis.cc
MyMiniAnalysis.tcl
btaMiniPhysProdSequence.tcl
btaMiniPhysics.tcl
The TopMain module
BaBar packages generally have source files named with three-letter
acronyms (TLAs), which identify those files as belonging to the
package. In our case, "Top" is appropriate to use.
Now we should rename PExample.* so that the cc and hh files
look like they belong to a package called TopUser. For now, let's just
go with TopMain, since this is the main analysis
source code for our package:
mv PExample.cc TopMain.cc
mv PExample.hh TopMain.hh
You now must go through TopMain.* and change all instances of PExample
to TopMain. You also need to change the two lines with PEXAMPLE in
capitals to TOPMAIN in TopMain.hh.
The last step is to check for and edit any other files that
depend on PExample:
TopUser> grep PExample *
This will reveal that you have two files to edit: AppUserBuildBase.cc
and MyMiniAnalysis.tcl. In both files you need to change all
instances of PExample to TopMain.
Taking these steps has renamed the module PExample to be now a
module called TopMain.
Other files
There are a few other files that should be renamed to reflect
their new roles as members of the TopUser package:
mv BetaMiniPatches.tcl TopPatches.tcl
mv btaMini.tcl topMini.tcl
mv btaMiniPhysProdSequence.tcl topMiniPhysProdSequence.tcl
mv btaMiniPhysics.tcl topMiniPhysics.tcl
A check for references to these files
TopUser> grep BetaMiniPatches *
TopUser> grep btaMini *
reveals that you have five files to edit:
TopPatches.tcl
MyMiniAnalysis.tcl
topMini.tcl
topMiniPhysProdSequence.tcl
topMiniPhysics.tcl
and change BetaMiniPatches to TopPatches, and btaMini to topMini.
AppUserBuild.cc
AppUserBuild*.cc are the files used the build your executable.
Since they are such important files, it is a good idea to check them
to make sure you did not forget anything before you continue.
Many BABAR packages come with an AppUserBuild.cc file. The
setup in the analysis-30 version of BetaMiniUser is a bit unusual:
instead of one AppUserBuild.cc file, there are three:
AppUserBuild.cc for older "bdb" code, AppUserBuildRoo.cc
for newer "roo" code, and AppUserBuildBase.cc for code common
to bdb and roo. In most packages, however, you will have only
one AppUserBuild.cc file.
The original AppUserBuild files had "#include BetaMiniUser/file.hh"
statements for several BetaMiniUser modules. Make sure that you have
changed all instances of BetaMiniUser to TopUser.
PExample also appeared in the original AppUserBuildBase.cc in two places:
in an "include" statement, and in the constructor of the
AppUserBuildBase class. Make sure that you replaced all instances
of PExample with TopMain.
In the constructor, you can also change the description of TopMain if
you like, for example, to "New package example" - the second
argument of this line isn't that important, while the first argument
is what the module will be referred to and must not contain spaces.
Dependency files
The dependency files are:
link_BetaMiniUser.mk
bin_BetaMiniUser.mk
bin_BetaMiniApp.mk
bin_BetaMiniRooApp.mk
link_<package>.mk; files tells the compiler what packages your
package depends on. bin_<package>.mk; files tell the compiler
what packages your binaries depend on. Most packages put all of their
dependencies in one bin_<package>.mk file. However, some packages
(like BetaMiniUser) with multiple binaries have one binXXX.mk file per binary.
From time to time you will need to update these files as your
dependencies change (for example, if you add some code to do a
complicated task, you may require another BaBar package for this
particular code to work correctly. Fortunately help is at hand, as the
SoftRelTools (Software Release Tools) package contains a script that
you can use at any time to update your link_ and bin_
The way to use this script is to go into your release directory and
issue the commands:
gmake installdirs
workdir/PARENT/SoftRelTools/make-linkfiles -p TopUser
This will create two new files:
link_TopUser.mk.NEW
bin_TopUser.mk.NEW
in your TopUser directory with updated dependences. All you need to do
next is to satisfy yourself that these files are sensible and then
overwrite the existing *_TopUser.mk files.
Of course, in order to know if the new versions are sensible,
you need to understand a bit more about how they work. Here
is a brief explanation.
bin_TopUser.mk contains lines of the form:
override LINK_BaBar += $(PACKAGE)GNUmakefile
If you were making this file from scratch, then to find
out which lines are needed in here, you would look through the
.cc file that is compiled to produce an executable. In
most applications, this file will be AppUserBuild.cc. (In BetaMiniUser,
you have to check all 3 AppUserBuild*.cc files.)
There is a LINK_* entry for each package for which there is a file
included with
#include "Package/file.hh"
Although you may get away with only including lines for a subset
of these files, it is likely that it will catch up with you one day
when you change operating system or release.
link_TopUser.mk is also a required file for each
package. For each package from which something was included and will be
compiled into the libraries, there is a line of the form
override LINK_BaBar += TopUserGNUmakefile
If you were making this file from scratch, you would have to go
through all of the .cc files apart from the one that
compiles into your binary (executable) and put a line in the
link_TopUser.mk file for each package that is included with a line like
#include "Package/file.hh"
as it is these lines that tell the compiler that your package needs
information from the other listed packages.
In this case, if you compare the files produced with make-linkfiles
with the older ones, you will find that references to many packages
have been removed. For example, in link.TopUser.mk.NEW, the lines
for TrgTools, TrkBase, and TrkFitter (among others) have been removed.
These are packages that were used by one or more of the modules that you
removed. Only the modules needed by MyMiniAnalysis and TopMain will
remain. This is exactly what you want.
The make-linkfiles command produced only one bin_TopUser.mk file, not
three. But if you compare it to the original 3 bin_XXX.mk files, you
will find that all of the dependencies from the original files have
been included in the single new file. If your PExample module had introduced
new dependencies for the binaries, then these would have been included
as well.
Now that we've confirmed that the new files are OK, let's replace the old files:
mv link_TopUser.mk.NEW link_TopUser.mk
mv bin_TopUser.mk.NEW bin_TopUser.mk
rm link_BetaMiniUser.mk
rm bin_BetaMiniUser.mk
rm bin_BetaMiniApp.mk
rm bin_BetaMiniRooApp.mk
Related documentation
GNUmakefile
Next you should sort out your GNUmakefile. The original BetaMiniUser
package could produce two binaries: BetaMiniApp, and the "extra" binary
BetaMiniRooApp. (An extra binary is one that is compiled only if the
user explicitly requests it via "gmake BinaryName.bin".) Here, we simply
rename the binaries: BetaMiniApp becomes TopMiniApp, and BetaMiniRooApp
becomes TopMiniRooApp.
(You will notice that the GNUmakefile contains references to the
AppUserBuild files. This is where the GNUmakefile tells the compiler
what source files are needed to build the executables.)
In TopUser there is also a one-line file called "binlist". You need
to change BetaMiniApp to TopMiniApp in this file.
When you start writing new code to go into your new package, it is
strongly recommended that you put all of your source code in a single
package directory. Subdirectories should be used only for things like
documentation, auxiliary files, and macros. If you break from this
convention, you will need to modify your GNUmakefile accordingly.
Related documentation
Finishing touches
If you have not already, now is a good time to remove
any backup files (files with a tilde ~ at the end) in your
TopUser package.
If you were really planning to add this package to CVS, you
would also want to go through each file and edit the comments.
For example, there may be references to BetaMiniApp that should
be changed to TopUserApp. These will not harm the program, so
we can leave them in this case. But in a real package you want
your comments to be accurate.
Also, a real package would come with its own History and README
files, to tell the user about the evolution of the package and how
to use it. So if you were planning to add this package to CVS,
you would need to write those files.
Now is also a good time to remove your BetaMiniUser package.
top-a30> rm -r BetaMiniUser
Telling the framework about your package
Now you have a unique package with appropriately named files. All you
need to do is tell the BaBar framework about your package, and you can
start your analysis.
Because yours is a new package, and not part of the release contained
in - and checked out from - cvs, you need to tell the framework that
the package exists. In this preliminary step before putting your
package into the BaBar CVS repository, you do this by checking out the
package PackageList:
top-a30> addpkg PackageList
PackageList contains a file called
link_all_reco.mk. You need to edit this file to include the
existence of your own new package. This is easy - just copy a chunk of
the repeated code in link_all_reco.mk and put your details in
there. Be sure to put your package information above any packages
that yours depends upon. If you are setting up a standard
analysis package, then a good place to start is to look for the entry
for TauUser and put it either above or below that
entry. The syntax you need to introduce into link_all_reco.mk is:
PACKAGELIST += TopUser
ifneq ($(LINK_TopUser),)
LINKLISTDEPENDS += [LINK_TopUser, $(LINK_TopUser)]
override LOADLIBES += -lTopUser
-include TopUser/link_TopUser.mk
endif
If you forget to do this step, or if you subsequently introduce a
dependency on another package, you might get compiler messages about
"symbol referencing errors" - so check that indeed your
package is listed in link_all_reco.mk above all the packages that it
depends upon.
Further information is available in HOWTO-dependency
in the SoftRelTools
package
Before starting, it is useful to issue a
top-a30> gmake installdirs
command from your test release directory, just to make sure that the
required directories for temporary binary and library files
exist.
Also don't forget that in each session, each time you log in,
you must issue the command:
top-a30> srtpath
Finally, as in the Quicktour,
you will need a file called .bbobjy in your test release directory,
containing the single line,
FD_NUMBER = ****
where **** is one of your 4-digit FDID numbers.
To find the FDID numbers assigned to you, use the command
GetFDID.
Compile your code
Now you can compile and link as in the WorkBook Quicktour, with:
top-a30> bsub -q bldrecoq -o all.log gmake all
If there are any error messages, you must deal with them first, but
you can generally ignore warnings.
If you edit any header files you will need to recompile the library
files with gmake lib (remember: all gmake commands should
be issued from the release directory). gmake bin is used
whenever source code is changed. Executables are stored in
/bin/$BFARCH. To check that everything worked,
you can do:
top-a30> ls -l bin/$BFARCH
and make sure the binary file you find there,
TopMiniApp, has a time stamp of only a minute or two
ago (i.e. that it was just made).
From time to time doing gmake clean to clean up files
is a good idea - remember that it removes existing executables, but
also cleans up library files - which can get rid of seemingly
inexplicable errors.
To check that your binary is ok, you can now follow the steps in
the Workbook QuickTour to
see that your job runs ok. (Note that you will need to copy
your snippet tcl file to TopUser, and edit it to change BetaMiniUser
to TopUser. The run command also changes; now it will be
"TopMiniApp snippet.tcl".)
If the resulting ROOT file, myHistogram.root, has only
empty histograms, you should check that you have correctly followed
all the steps in this section, and that you have not put
"TopMain" in the place of "TopUser" or vice
versa. The result of such an error is generally either an error
message, or simply the failure of the BaBar framework to include your
module in the full analysis sequence, thereby resulting in your
histograms being booked, but never filled.
As a last step, we should also check that the "extra" binary,
TopMiniRooApp, links correctly. To force an extra binary to
be compiled, you must call it by name:
ana30> gmake TopUser.TopMiniRooApp
Once again, check the bin/$BFARCH directory to make sure TopMiniRooApp
was produced.
An example of a working TopUser package, produced using the instructions
in this section, is located at:
$BFROOT/www/doc/workbook/NewExamples/TopUser
Related Documents:
Getting help
While attempting to compile and link a new package, many unfamiliar
error messages are likely to occur. The best place to start looking
for help is one of the BaBar Hypernews
groups. Depending on the particular question, one of the following
groups might be most appropriate (it is best not to cross-post to more
than one group):
When you do post to hypernews groups, you should usually state the
packages you are using (mention that your own package is based on
BetaMiniUser if this is correct), state which release your test release is
based on, and what machine architecture also. You should also give the
relevant part of any log files you have if you were running your
analysis and it crashed and/or the traceback from the debugger. It can
also be useful if you are running at SLAC to give the path to your
files so that a helpfully-inclined person can go and get their hands
dirty to help you.
Now your shiny new package is ready to be filled up with powerful
analysis code. Since your package and executable are uniquely named,
you can safely allow others to copy your package and modify files or
simply test what's there. However:
- your package at this stage is only guaranteed to work for the
release analysis-30
(it will work for other releases unless
there has been an incompatible change in the basic BaBar code that is
called by TopUser),
- your package is still not an official BaBar package, so whenever
it is used, the PackageList package must also be checked out and the
lines telling
link_all_reco.mk about your package must be added.
Once you are happy the your package is working correctly, and that
it does something useful, you can release it to the (BaBar) world. To
get your new package included in the BaBar CVS repository and in
future BaBar releases, follow the steps in the next section of the
BaBar WorkBook New Packages: Getting
your package into CVS.
Authors:
Jenny Williams
Last modified: 21 January 2006
Last significant update: 17 July 2005
|
