SLAC	PEP-II
BABAR
SLAC<->RAL

HEPIC	E,S & H	Databases	PDG	HEP preprints
Organization	Detector	Computing	Physics	Documentation
Personnel	Glossary	Sitemap	Search	Hypernews

Workbook - SkimCity

This section will show step by step how to create, run, and test a simple example skim. For more on the overall picture of how skimming fits into in CM2, see the CM2 goals overview.

This section assumes you are familiar with setting up a test release and working with a Beta job in the BaBar computing environment. Throughout this section, command line entries are written in red, and code that needs be added is written in green. If you're already familiar with skims and just want to add new features like persistent candidates, you might want to skip ahead to this section.

What packages to use
Setting up a Mini skim
Some of the above made automatic
Recommended ways to define a skim
Skim documentation
Adding more features
Running the skim
Running over skimmed collections
Your skim in skim production

What packages to use

The skim code lives in a package called "FilterTools". To actually run the skim executible, you will also need to add the "SkimMini" package. Remember to first do (from the release directory, Beta-22.x.x throughout this example):


   Beta-22.x.x> srtpath <enter> <enter>
   Beta-22.x.x> XXXXboot

where XXXXboot is the appropriate conditions database. For example, cond16boot is appropriate for release 16, cond18boot is appropriate for release 18 and cond22boot is appropriate for release 22. Then


   Beta-22.x.x> addpkg FilterTools 
   Beta-22.x.x> addpkg SkimMini

the FilterTools package already contains a number of skims from various analyses. Generic documentation on the skim selection variables, skim rates, releases uses in central skimming is available here.

Setting up a Mini skim

Structure of Mini skims

Skims require that the following line be added to the file FilterTools/defineMiniSkims.tcl:


declareMiniSkim ExampleAWG/MyExample   deepCopyMicro  deepCopyMicro {candLists} {tagBits}

The first argument is the skim name, including (before the "/") the AWG the skim belongs to. The 2nd (for data) and 3rd (for Monte Carlo) arguments define if the skim is to be a pointer or deep copy skim. Small skims (tipically, those with rates below 7%) tend to be deep copy, as are skims that are to be exported to non-Tier-A sites. The last two arguments are optional. If they are not specified, a file called

FilterTools/MyExamplePath.tcl

must exist to actually define how the skim should select events.

An example Mini skim

The preferred way to define a skim is to define it based on the results of SimpleComposition modules. These modules define a list of BtaCandidates, for example the list of all B0 to J/psi KS candidates passing a particular set of selection criteria. It is straightforward to define a skim to be all events that have one or more candidates in this list of BtaCandidates. First, you need to define all of the SimpleComposition modules required to make your list (or lists) of interesting candidates. Documentation on SimpleComposition is here. Next, you should define tag bits based on your list or lists of BtaCandidates. This step is especially useful if you are defining your skim based on multiple lists of BtaCandidates. These bits will let you easily separate out the different modes as you analyze your skim. Simple tcl can be used to create these tag bits:

   
module clone TagFromList TagJpsiKs
talkto TagJpsiKs {
  listsToTag set B0JpsiKs
  tagName set JpsiKs
}

Here, we define the tag bit "JpsiKs" to be set for any event with one or more entry in the "B0JpsiKs" BtaCandidate list. Using these tag bits, it's very simple to define a skim. The following goes into the FilterTools/MyExamplePath.tcl file:

   
global writeUsrDataForSkim

path create MyExamplePath

# only run on MultiHadrons
path append MyExamplePath FilterBGFMultiHadron

module clone TagFromList TagJpsiKs
talkto TagJpsiKs {
  listsToTag set B0JpsiKs
  tagName set JpsiKs
}
module clone TagFilterByName MyExampleFilter
path append MyExamplePath MyExampleFilter
talkto MyExampleFilter {
  orList set JpsiKs
}

# (don't) Write UsrData
set writeUsrDataForSkim 0

There are a few predefined modules that we have used here. In particular, the BGFMultiHadron filtering module is used to select only multihadron events. Also, "TagFilterByName" is a generic filter module that is defined to filter on one or more tag bits. We make a clone of this module and ask it to filter on the "JpsiKs" tag bit (not BtaCandidate list). This is the module that actually defines the skim in this example. Any event passing this filter will be included into the skim. The last few lines specify that this example does not define any user data. How to include user data is decscribed below.

Some of the above made automatic

Some of the methods to define a skim described in the previous sections have been made automatic, through additional arguments to the "declareMiniSkims" proc.


declareMiniSkim ExampleAWG/MyExample   deepCopyMicro  deepCopyMicro {candLists} {tagBits}

candLists is a list of candidate lists. If not empty (e.g. is {candList1 candList2}), and if either candList1 or candList2 has at least one candidate (logical OR), a tagbit named "MyExampleTag" is created and set, and the event is selected for the skim. In other words,
```
pass  =  "at least one candidate in any of these lists".
```
Moreover, the lists candList1 and candList2 are automatically persisted (see below).
tagBits is a list of tagbits. If not empty (e.g. is {tagbit1 tagbit2}) and if either tagbit1 or tagbit2 is set (logical OR), the event is selected. In other words,
```
pass  =  "at least one of these bits set".
```
tagBits and candLists can also be used in conjunction. In this case the skim path will contain both the filter on tagBits, AND the filter on candLists. In other words,
```
pass  =  "at least one of these bits set" AND "at least one candidate in any of these lists".
```

Using these features, implementing the example above would not even require to write a FilterTools/MyExamplePath.tcl file, but just defining the skim in defineMiniSkims.tcl as:


declareMiniSkim ExampleAWG/MyExample   deepCopyMicro  deepCopyMicro {B0JpsiKs} {BGFMultiHadron}

Which is admittedly shorter and less error prone. For people using very old releases: note that these features require at least the following tags:


FilterTools V00-16-02
SkimMini V00-03-06

Another example Mini skim

To get started we'll add .cc, .hh and .tcl files for the new skim to FilterTools. Some empty outlines are already available to be copied into FilterTools. From the release directory do:

   
   Beta-22.x.x> cd FilterTools

Next you should download the three MyExampleSkim* example files from here into the FilterTools package you just checked out.

There should now be three files called "MyExampleSkim.cc", "MyExampleSkim.hh", and "MyExamplePath.tcl" located in FilterTools. The structure of MyExampleSkim.cc looks similar to a standard Beta User job, but there are a couple of differences. First, the constructor is:

MyExampleSkim::MyExampleSkim( const char* const theName, 
			      const char* const theDescription )
  : TagFilterModule( theName, theDescription )

Note the last line uses TagFilterModule instead of the usual:

  : AppModule( theName, theDescription )

Second, MyExampleSkim::event calls setPassed. The argument to this function will determine whether this event passes the skim or not.

   setPassed( false ); // Do not pass this event
   setPassed( true );  // Do pass this event

MyExampleSkim could run as is, but all it does is fail every event, which probably makes for a short and depressing analysis. So we should add some physics code to select interesting events. A more complete version of MyExampleSkim is available by downloading the three MyExampleSkim* files from here into FilterTools.

This example looks for events with a B0 -> D mu nu candidate. Events are chosen which have a D+- and a muon of the opposite charge. In addition, the muon must have momentum > 1.0 GeV. At the end of the event setPassed( true ) is called if all of these conditions are met, or setPassed( false ) if they are not.

Now that all the code is in place, the skim module can be added to the sequence by adding the following lines to the SkimTagSequence.cc file:

   #include "FilterTools/MyExampleSkim.hh"

   forWho->add(new MyExampleSkim("MyExampleSkim", "example filter module"));

Finally you need to define this skim. Not surprisingly, this is done in FilterTools/defineMiniSkims.tcl. Add the following line to the bottom of this file:


declareMiniSkim ExampleAWG/MyExample   deepCopyMicro  deepCopyMicro

For this example, the skim is declared as a "deepCopyMicro" skim. Generally, skims that select more than 5% of events should be declared as pointer skims instead of deepCopy.

Recommended ways to define a skim

As we have just shown, skims can actually be defined in several different ways. Since we aim now at easily switching on/off in centralized skim productions the individual skims and the potentially time-consuming code to produce the related sequences and tagbits, we want that all the relevant code can be included/excluded simply commenting/uncommenting the appropriate line in defineMiniSkims.tcl. Depending on the details of your particular skim, you can:

If the skim only uses predefined tagbits and/or lists of composites, use the "{candLists} {tagBits}" syntax in defineMiniSkims.tcl . This has the advantage of not requiring a dedicated MyExampleSkimPath.tcl file.
If you need to define your own candidate lists, use SimpleComposition, and add these lists to the file FilterTools/SkimCompProdSequence.tcl. Note that this file should only contain definition of SimpleComposition lists, but no tagbit definitions to avoid triggering the execution of the code when the skim is commented out.
If you need to define FilterTools/MyExampleSkim.{cc,hh} files, or if your selection is more complicated than the predefined combination of logical ORs of candLists and tagBits (or logical AND of the two) possible with the "{candLists} {tagBits}" syntax, use the FilterTools/MyExampleSkimPath.tcl file.

Skim documentation

Now that we have defined a beautiful skim it is time to document the skim properly. Since the skim documentation is created automatically by FiterTools/SkimMakeDoc there are some rules to follow. The documentation for all skims resides in FilterTools/doc. If your skim is called MyExampleSkim you should document your skim in FilterTools/doc/MyExampleSkim.readme (alternatively MyExampleSkimPath.readme can be used as well, but the first name is preferred). If you prefer to document your skim in HTML you can do that in FilterTools/doc/MyExampleSkim.html. Please do not include the standard BaBar header in the HTML file.

If you want to document several skims in one file you can set a reference to another skim. For example, if MyExampleSkim is documented in the readme file of MyFirstExampleSkim, your MyExampleSkim.readme should look like this:

 See <skim>MyFirstExampleSkim</skim> documentation.

The <skim> token tells the documentation generator where to look for the documentation. This token must appear in the first line of the file (empty lines and the CVS identifier $Id do not count). If you have your skim documentation on a web page that is not part of FilterTools/doc you can use the following syntax to set a link:

 See <url>http://www.google.com</url>

However, the recommended way is to put the HTML page in the FilterTools package as described earlier. It is the responsibility of the skim author to make sure that the skim has its proper documentation file even if it only contains a link to a web page or BAD note where further information can be found.

If you want to create the global skim documentation for yourself (you do not have to do this) do:

   Beta-22.x.x> gmake FilterTools.binscripts
   Beta-22.x.x> cd workdir
   Beta-22.x.x> SkimMakeDoc

The script will write the documentation to the current directory. Use SkimMakeDoc -h to get help on other options.

Adding more features

It's also possible to add additional information to events which pass the skim in the form of composite candidates and User Data. This section will show an example of adding composite candidates to MyExampleSkim.

Adding persistant composite candidates from SimpleComposition or CompositionTools

If the list of composite candidates you're interested in already exists (for example a list created by SimpleComposition as in the first example skim), you can persist them without adding any C++ code at all. For example, the list of D+- mesons used in MyExampleSkim ("DcLoose") is made in CompositionTools. If you wanted to persist these candidates, you would just add to MyExamplePath.tcl the lines:

   global BtaCandLists
   lappend BtaCandLists DcLoose

Adding persistant composite candidates from MyExampleSkim

You can also persist candidates that are created within your skim. This is done in the final version of MyExampleSkim that can be copied into FilterTools:

   
   Beta-22.x.x> cd FilterTools

Next download the three MyExampleSkim* files (.tcl, .hh, .cc) files from here into FilterTools. (These are slightly different from the ones you may have downloaded above.)

Several changes have been made to this version of MyExampleSkim. First, MyExamplePath.tcl has some additional settings, which define the name of the list of candidates to be stored - "MyExampleCandList" in this case:

  global BtaCandLists
  lappend BtaCandLists MyExampleCandList

Second, the composite candidates are made in MyExampleSkim.cc by adding the 4-vectors of the D and the muon. All that's needed to persist them is to append them to a list with the key "MyExampleCandList". The code needed to do this has been added to the constructor and event method of MyExampleSkim.cc:

   MyExampleSkim::MyExampleSkim( const char* const theName, 
			      const char* const theDescription )
   : TagFilterModule( theName, theDescription )
   , _outputCandList("outputCandlist",this,"MyExampleCandList")
   ...
  
   MyExampleSkim::event( AbsEvent* anEvent )
      {
        // Make composite candidates by adding 4-vectors
	static BtaOpAdd4 candCombiner;
	...
	// Get list of composite candidates that will be persisted
	getTmpAList( anEvent, _compositeList, _outputCandList.value() );
	...
		// Create a composite D mu candidate
		BtaCandidate *compositeCand= 
			candCombiner.create(*muon,*Dc);
		...
		// Add the candidate to the list to be persisted
		_compositeList.append( compositeCand );
	...	
      }

That's all it takes - now "MyExampleCandList" will be available as a list when running a Beta job over the skimmed collections, just like any other list of BtaCandidates.

Adding User Data

To take full advantage of skims in CM2, you can add User Data to the skim- numerical data that can be attached to either an event or a candidate. A complete example of working with User Data is located here.

Running the skim

You'll want to thoroughly check that your skim produces the output you want before trying to commit it. The next two sections will show how to run the skim and examine the output collections. To do this, you'll need to set up a .tcl snippet in your workdir. Cut and paste the following to a tcl snippet file "myskim.tcl" in workdir:

# -------- myskim.tcl: begin -----------
set SkimsToRun MyExample
set SkimOutputDir /work/users/<username>/outputDir
set SkimsToWrite all
set SkimMC yes
set SkimFwkDump yes
set SkimInputCollection /store/SP/R22/001237/200506/22.0.1/SP_001237_012005
set SkimNEvent 100
sourceFoundFile SkimMini/SkimMiniProduction.tcl
# -------- myskim.tcl: end -----------

This file has a number of options which are defined as follows:

SkimsToRun - list of skims to run (e.g. MyExample)
SkimOutputDir - Output of the skim will be written to this directory
SkimsToWrite - # Write the output of all the skims being run
SkimMC - We want to skim Monte Carlo
SkimFwkDump - Print the Framework path before the start of the event loop
SkimInputCollection - name of the input collection (the example is a generic B0B0bar collection)
SkimNEvent - The number of events to run over (100 in this case)
Other available options are documented in SkimMini/SkimMiniProduction.tcl

See the SkimMiniApp documentation for more information on how to configure the application. Remember to change the "<username>" part of the SkimOutputDir setting to match you real username! If you have not setup your /work/users/<username> area, see the page about user output collections.

You now have everything needed to start testing your skim, just compile and link FilterTools and the "SkimMiniApp" application:

   > cd Beta-22.x.x
   Beta-22.x.x> gmake FilterTools.lib
   Beta-22.x.x> gmake SkimMini.SkimMiniApp

and run SkimMiniApp from the workdir using the myskim.tcl file as the argument:

   Beta-22.x.x> cd workdir
   workdir> SkimMiniApp myskim.tcl

The output should look something like this example log file. When the skim is finished you can check that there is a collection called MyExample in the outputDir directory:

   workdir> KanUserAdmin list /work/users/marchior/outputDir
   /work/users/marchior/outputDir/MyExample

This collection contains all the events that passed the MyExample skim, and can now be used as input to a Beta job. You can first quickly check that the skim produced some output using the "KanCollUtil" utility as follows:

   workdir> KanCollUtil /work/users/marchior/outputDir/MyExample
   /work/users/marchior/outputDir/MyExample (9 events)

This means of the 100 events in the original collection, 9 passed the skim requirements and are stored in the MyExampleSkim collection. Note that since the selection rate of this skim is 9%, it should be a pointer skim under the 7% cutoff rule discussed above.

Running on skimmed collections

If you've completed the above steps, you should now have a skimmed collection that's ready to be used in an analysis job. This section will briefly describe how to modify a BetaMiniApp-type job to run over the MyExampleSkim collections.

Reading the input collections

In the tcl snippet file you use with BetaMiniApp, you should specify the following to use your new skimmed collections: (obviously replacing "<username>" as above):

   
   lappend inputList /work/users/<username>/outputDir/MyExampleKan

Accessing persisted candidates

Lists of persisted candidates are accessed like any other list of BtaCandidates. For the example which had a list named "MyExampleCandList", you'll need in your BetaMiniUser's event method something like:

   
   HepAList< BtaCandidate >* myList;
   getTmpAList( anEvent, myList, IfdStrKey(HepString("MyExampleCandList")) );

You can then iterate over "myList" to access the composite candidates

   
   BtaCandidate* theCompCand(0);

   HepAListIterator<BtaCandidate> iterComposite( *myList );

   while( 0 != (theCompCand = iterComposite()) )
    {
       //... Insert Nobel-prize winning analysis here...
    }

You can now perform any operation that you normally would on a BtaCandidate, including iterating over its daughters.

Your skim in skim production

Whenever your skim is run in the central skim production, it will be run over all data samples and generic MC samples. However, your skim will not be run on any signal MC by default. To request the skimming of signal MC you need to define the MC modes in the file FilterTools/skimsSPmodes.tcl:


   declareSkimSPmodes MySkim {1234 5678}

This will run MySkim on the modes 1234 and 5678. The collections should show up in bookkeeping as SP-1234-MySkim-Rxx where xx is the skim cycle.
PLEASE note the space between the skim name and the open bracket, and be careful with the correct spelling (capital and lower-case letters) of declareSkimSPmodes

Author: Jonathan Hollar
Contributors: Giuseppe Finocchiaro, Frank Winklmeier
Last updated: Giovanni Marchiori, 14 Aug 2007