Tcl: Run-Time Job Control
Contents:
The previous section described modules, paths, and sequences, but it did not explain how to create, enable/disable, and append them.
That is the task of this section.
The user interface to the Framework is based on Tcl, Tool Command Language.
BaBar uses standard Tcl with some added BaBar specific commands.
There are two common ways of using Tcl to interface with the
Framework:
- Issue commands directly through the Framwork, one at a time
at the Framework prompt (">");
- Write the commands in a Tcl script, and source the script.
In this section the focus is on the interactive method - issuing commands directly to the Framework as the job is running.
Later, the Workbook's Tcl Files section will teach you more about Tcl and Tcl scripts.
This section uses a few standard conventions when expressing Tcl command syntax:
- Anything contained inside square brackets: '[ ]' is optional:
- mod[ule] indicates that the entire word
need not be typed. The commands mod and module have the same effect.
- [<argument2>] implies that the extra argument is optional.
- Names written <like this> are argument names
for which you will enter the appropriate value.
Quick link: Complete list of BaBar Tcl commands.
In addition to the standard Tcl commands,
the Framework has BaBar-specific commands that you can use at the
Framework prompt '>' in any application. The most important
commands are listed below.
help
Lists the new commands, those beyond the standard Tcl command set which have been defined for use with BaBar code.
exit
Exits the application, and ends the job. The endJob() function of each module is executed.
Note that the exit command will end the job only if it is typed from the Framework prompt (">").
From a module prompt, the exit command will exit the module and return control to the Framework (ie, return you to the Framework prompt).
ev[ents] beg[in] [-nev <N>]
Begins an event processing loop. By default the job begins at the first event in the first input file and continues until all input events are exhausted.
The optional -nev N switch will cause the job to end after N events.
mod[ule] disable/enable <mod1> [<mod2> <mod3> ...]
seq[uence] disable/enable <seq1> [<seq2> <seq3>...]
path disable/enable <path1> [<path2> <path3>...]
Disables/enables the module, sequence or path. Modules are enabled by default.
A disabled module will be bypassed during event processing, even if it is on the analysis path.
The module enable command re-enables a disabled module or list of modules. The same holds for sequences and paths.
mod[ule] list,
seq[uence] list,
path list
Lists the modules, sequences, or paths that make up the analysis job.
mod[ule] talk
Allows the user to "talk to" the module, and set or reset its run-time parameters.
The mod talk command is perhaps the most important interactive framework command, and is the main subject of the next section.
One of the most useful module commands is the module talk command which allows interactive changing of the module's run-time parameters.
This means that the user can alter analysis parameters without having to re-compile the code.
Issuing the module talk command transfers command input to the specified module.
> module talk MyModule
MyModule>
The Framework prompt '>' changes to the module prompt 'MyModule>' to indicate that all commands are now issued to the module, not the Framework.
This will continue until the user enters the exit command,
MyModule> exit
>
which returns job control to the Framework and restores the Framework prompt '>'.
Here is the standard set of commands avaiable in a module talk session:
(These can be displayed by typing help during a talk with the module.)
| Command |
Definition |
| echo <arg1> [<arg2> ...]
|
echos the arguments to the standard output.
|
| exit |
return control to the Framework. |
| help |
lists the commands available. |
| show |
lists the parameters available and their current value. |
| sourceFoundFile <file>
|
source the tcl file in the path given by $BFDEFAULTSEARCHPATH or $BFSEARCHPATH.
|
| verbose |
a boolean parameter that controls standard output. |
| production |
a boolean parameter to indicate processing in Production.
|
| enableFrames |
a boolean to enable frames. |
Every analysis job needs an active input module (usually KanEventInput) to receive input data files.
In addition to the basic module commands of help and exit, an input module has a special set of input commands.
The most important input command is input add, which you have already used in the Quicktour:
> mod talk KanEventInput
KanEventInput> input add /store/SP/R14/001237/200309/14.3.1c/SP_001237_000533
The best way to learn about the framework is to interact with it yourself.
In this section, we will re-run the Quicktour Analysis, but we will take the time to investigate some of the new Framework commands you just learned.
As you follow the example, feel free to experiment with other commands.
Any parameters or settings that you alter will affect only the job that is running - they cannot harm your BetaMiniApp executable or any future jobs.
If you get stuck, remember that the "help" command will always tell you what commands are available, and the "exit" command will always get you out.
Ready? Okay, let's begin!
A Framework session with the Quicktour's BetaMiniApp
To begin your Framework session, go to workdir as before, and
run BetaMiniApp:
workdir> BetaMiniApp snippet.tcl
Eventually (after a lot of output to the screen) given a prompt: '>'. This indicates that your are now "in the Framework".
>
The long output to the screen before the framework prompt was in fact the output of the "path list" command, which was issued in MyMiniAnalysis.tcl.
The path list command shows all of the structure in the paths by using indentation.
Whether or not a module is included in a defined path is seen only from the path list command.
The module list command lists all
modules that have been loaded into the Framework.
> mod list
You can see that there are a lot of modules involved in your analysis job!
The help command always gives you a list of available commands:
> help
Command(s) available in the "AppFrame" module:
echo Send text argument to stdout (useful in scripts).
exit Leave the current menu, module, or process
help Bring up help text for the current context.
show Display the value of any parameters or statistics.
sourceFoundFile Source file in $BFDEFAULTSEARCHPATH or $BFSEARCHPATH.
verbose bool parameter: set or list.
production bool parameter: set or list.
enableFrames bool parameter: set or list.
events Command used to control the looping over events
module Command used to control and specify modules
path Command used to create, modify and control paths
sequence Command used to create, modify and control sequences
config Dump and restore configuration.
action Command used to control module actions
strictModuleDisable bool parameter: set or list.
For more information on a particular command type '<command> help'.
A minimal set to select an input file and run 5 events is:
module talk FileInput
input file <your_file>
exit
ev begin -nev 5
exit
A script can also be specified as the first argument when running the program.
In addition the following command aliases can also be used.
Alias: Command Equivalent:
mod module
cont events continue
talk module talk
seq sequence
begin events begin
up exit
clone module clone
logout exit
ev events
quit exit
Here you see the familiar help, show, and exit commands.
Other commands, like echo, sourceFoundFile, and config, are more useful in tcl scripts.
You'll learn about tcl files in the Workbook's Tcl Files section.
Now let's use the mod talk command to interact with some of the modules you saw on the path list.
EvtCounter is a useful module that prints out the number of events that have been processed at regular intervals.
It can also be used to flag or skip bad events.
Let's talk to it:
> mod talk EvtCounter
EvtCounter> help
Command(s) available in the "EvtCounter" module:
echo Send text argument to stdout (useful in scripts).
exit Leave the current menu, module, or process
help Bring up help text for the current context.
show Display the value of any parameters or statistics.
sourceFoundFile Source file in $BFDEFAULTSEARCHPATH or $BFSEARCHPATH.
verbose bool parameter: set or list.
production bool parameter: set or list.
enableFrames bool parameter: set or list.
print bool parameter: set or list.
printFreq general parameter: set or list.
skipStart general parameter: set or list.
skipStop general parameter: set or list.
badEvents Vector parameter: set <value> [at <position>] or list.
eventId string key parameter: set or list.
exitOnDuplicate bool parameter: set or list.
The first six commands (from echo to enableFrames) are standard commands that every module has.
But you can see that EvtCounter also has its own run-time parameters.
The parameters skipStart and skipStop are used to skip events in a given range.
First, let's use the list command to find out the current values:
EvtCounter> skipStart list
Value of skipStart for module EvtCounter is 0
EvtCounter> skipStop list
Value of skipStop for module EvtCounter is 0
Right now, both skipStart and skipStop are set at their default values: zero. No events will be skipped.
Now imagine we found out the that there is some problem with events 10-20.
We use the set command to change the run-time parameters:
EvtCounter> skipStart set 10
EvtCounter> skipStop set 20
EvtCounter> skipStart list
Value of skipStart for module EvtCounter is 10
EvtCounter> skipStop list
Value of skipStop for module EvtCounter is 20
EvtCounter> exit
>
The exit command takes you out of EvtCounter and puts you back in the Framework.
Now let's talk to another module:
> mod talk RooTupleEnv
RooTupleEnv> show
Current value of item(s) in the "RooTupleEnv" module:
Value of verbose for module RooTupleEnv: f
Value of production for module RooTupleEnv: t
Value of enableFrames for module RooTupleEnv: f
Value of useScalers for module RooTupleEnv: f
Value of globalHist for module RooTupleEnv: f
Value of ntupleFile for module RooTupleEnv: t
Value of histFileName for module RooTupleEnv is myHistogram.root
RooTupleEnv is the module that manages the root output files of the analysis job.
The default filename for the output file is myHistogram.root. Now you can change it to something else:
RooTupleEnv> histFileName set NewName.root
RooTupleEnv> show
Current value of item(s) in the "RooTupleEnv" module:
Value of verbose for module RooTupleEnv: f
Value of production for module RooTupleEnv: t
Value of enableFrames for module RooTupleEnv: f
Value of useScalers for module RooTupleEnv: f
Value of globalHist for module RooTupleEnv: f
Value of ntupleFile for module RooTupleEnv: t
Value of histFileName for module RooTupleEnv is NewName.root
RooTupleEnv> exit
>
The GoodTrackLooseSelection module is the module that makes the GoodTracksLoose list of charged particle candidates.
Let's talk to it:
> mod talk GoodTrackLooseSelection
GoodTrackLooseSelection> show
Current value of item(s) in the "GoodTrackLooseSelection" module:
Value of verbose for module GoodTrackLooseSelection: f
Value of production for module GoodTrackLooseSelection: t
Value of enableFrames for module GoodTrackLooseSelection: f
Value of inputBtaCandidateList for module GoodTrackLooseSelection: IfdStrKey(ChargedTracks)
Value of outputBtaCandidateList for module GoodTrackLooseSelection: IfdStrKey(GoodTracksLoose)
Value of outputMap for module GoodTrackLooseSelection: IfdStrKey(GoodTracksLoose)
Value of histoFlag for module GoodTrackLooseSelection: f
Value of maxEventTuple for module GoodTrackLooseSelection is 0
Value of criteria for module GoodTrackLooseSelection is loose
Value of debugHistograms for module GoodTrackLooseSelection is f
Value of doGhost for module GoodTrackLooseSelection is f
Value of ghostThr1 for module GoodTrackLooseSelection is 0.0025
Value of ghostThr2 for module GoodTrackLooseSelection is 0.05
Value of ghostThr3 for module GoodTrackLooseSelection is 0.365
Value of pMin for module GoodTrackLooseSelection is 0
Value of pMax for module GoodTrackLooseSelection is 10
Value of ptMin for module GoodTrackLooseSelection is 0.1
Value of nHitsMin for module GoodTrackLooseSelection is 5
Value of nHitsDchMin for module GoodTrackLooseSelection is 12
Value of nHitsSvtMin for module GoodTrackLooseSelection is 0
Value of spanMin for module GoodTrackLooseSelection is 0
Value of spanDchMin for module GoodTrackLooseSelection is 0
Value of chisqProbMin for module GoodTrackLooseSelection is 0
Value of docaMax for module GoodTrackLooseSelection is 1.5
Value of zMax for module GoodTrackLooseSelection is 10
Value of zMin for module GoodTrackLooseSelection is -10
Value of phiMax for module GoodTrackLooseSelection is 10
Value of phiMin for module GoodTrackLooseSelection is -10
Value of thetaMax for module GoodTrackLooseSelection is 10
Value of thetaMin for module GoodTrackLooseSelection is -10
Value of pMinHit for module GoodTrackLooseSelection is -10
Value of fracMinHit for module GoodTrackLooseSelection is -10
Value of SvtOnlyPtMax for module GoodTrackLooseSelection is 10
Value of ImposeCalculatedPOCA for module GoodTrackLooseSelection is f
Value of RejectGhostsVisibleE for module GoodTrackLooseSelection is f
Value of GhostsVisibleETrkList for module GoodTrackLooseSelection is ChargedTracks
Value of nDchMinGhost for module GoodTrackLooseSelection is 1
Value of ptMaxGhost for module GoodTrackLooseSelection is 0.35
Value of deltaPhiMaxGhost for module GoodTrackLooseSelection is 0.3
Value of deltaThetaMaxGhost for module GoodTrackLooseSelection is 0.3
Value of deltaPtMaxGhost for module GoodTrackLooseSelection is 0.15
Value of RejectLoopersVisibleE for module GoodTrackLooseSelection is f
Value of LoopersVisibleETrkList for module GoodTrackLooseSelection is ChargedTracks
Value of nSvtMinLooper for module GoodTrackLooseSelection is 1
Value of cosThetaMaxLooper for module GoodTrackLooseSelection is 0.2
Value of ptMaxLooper for module GoodTrackLooseSelection is 0.25
Value of deltaPtMaxLooper for module GoodTrackLooseSelection is 0.12
Value of deltaThetaMaxSameChargeLooper for module GoodTrackLooseSelection is 0.2
Value of deltaThetaMaxOppositeChargeLooper for module GoodTrackLooseSelection is 0.18
Value of deltaPhiMaxSameChargeLooper for module GoodTrackLooseSelection is 0.18
Value of deltaPhiMaxOppositeChargeLooper for module GoodTrackLooseSelection is 0.16
This module has a lot of run-time parameters! As you may be able to guess from the names, most of these parameters are related to track reconstruction.
The values of these parameters are the values of the cuts used to select candidates for the GoodTracksLoose list.
Let's tighten the selection for this list. Already the module demands a minimum of 12 hits in the DCH (nHistDchMin = 12);
let's require at least 1 hit in the SVT as well. And let's refuse to accept tracks with momentum less than 0.2 GeV.
GoodTrackLooseSelection> nHitsSvtMin set 1
GoodTrackLooseSelection> pMin set 0.2
GoodTrackLooseSelection> nHitsSvtMin list
Value of nHitsSvtMin for module GoodTrackLooseSelection is 1
GoodTrackLooseSelection> pMin list
Value of pMin for module GoodTrackLooseSelection is 0.2
GoodTrackLooseSelection> exit
>
Now the GoodTracksLoose list will be made only from candidates that satisfy the new, tighter selection criteria.
IMPORTANT NOTE: In a real analysis you probably would NOT want to change the parameters for GoodTracksLoose or any of the other "standard" run-time lists,
because this would make it harder to compare your results to those of other BaBar users who use the standard lists.
This was just an example. (You will learn about the standard lists in the Event Information section.)
Incidentally, it is often the case that a list named SomeList is produced by a module called SomeListSelection - although it is not an absolute rule. In fact, it's not
even 100% true here -- there's an "s" missing in the module name.
It's almost time to run the job. But first, let's investigate our two BetaMiniUser modules, MyMiniAnalysis and QExample:
> mod talk MyMiniAnalysis
MyMiniAnalysis> show
Current value of item(s) in the "MyMiniAnalysis" module:
Value of verbose for module MyMiniAnalysis: f
Value of production for module MyMiniAnalysis: f
Value of enableFrames for module MyMiniAnalysis: f
Value of eventInfoList for module MyMiniAnalysis: IfdStrKey(Default)
Value of btaMicroList for module MyMiniAnalysis: IfdStrKey(BtaMicroCandidates)
Value of btaTruthMap for module MyMiniAnalysis: IfdStrKey(GHit)
Value of btaListToPrint for module MyMiniAnalysis: IfdStrKey(NoListToPrint)
The verbose parameter determines whether or not detailed warning/error messages are printed.
All modules have a verbose parameter (although some may not use it).
The default value is false.
In MyMiniAnalysis, there are several warning messages that will not be printed unless verbose is set to true.
So let's reset it:
MyMiniAnalysis> verbose set t
MyMiniAnalysis> verbose list
Value of verbose for module MyMiniAnalysis: t
MyMiniAnalysis> exit
>
The command "verbose set true" would also work.
Finally, let's talk to QExample - the new Workbook user module that you added in the Quicktour section:
> mod talk QExample
QExample> show
Current value of item(s) in the "QExample" module:
Value of verbose for module QExample: f
Value of production for module QExample: f
Value of enableFrames for module QExample: f
QExample>
You will notice right away that QExample is different from the other modules.
The only run-time parameters that is has are the three default module parameters - verbose, production, and enableFrames.
And yet, QExample has a parameter that you might want to reset at run time.
QExample's job is to make a histogram of the number of charged tracks per event.
It uses the standard list ChargedTracks.
You might want to choose a different list of charged particle candidates, like GoodTracksLoose.
But right now, you can't, because the name of the list is not a run-time parameter.
To make those changes, you would have to edit QExample.cc, and then recompile and relink your code.
The solution is to make the list name into a run-time parameter.
You will learn how to do this in the Workbook's Run-Time Parameters section.
For now, there is not much that you can do in QExample, so let's exit the module and get on with the analysis job.
QExample > exit
Before the job can run, it needs some data. To input data to the framework, you have to talk to an input module and use its special input add command.
The module will almost always be KanEventInput. As in the Quicktour:
> mod talk KanEventInput
KanEventInput>
As mentioned above, input modules have their own special set of input commands:
KanEventInput> help
Command(s) available in the "KanEventInput" module:
echo Send text argument to stdout (useful in scripts).
exit Leave the current menu, module, or process
help Bring up help text for the current context.
show Display the value of any parameters or statistics.
sourceFoundFile Source file in $BFDEFAULTSEARCHPATH or $BFSEARCHPATH.
verbose bool parameter: set or list.
production bool parameter: set or list.
enableFrames bool parameter: set or list.
Kan_Verbose symbolic parameter: set or list.
ROOT_debug general parameter: set or list.
input (Command) : Type 'input help' for details.
first general parameter: set or list.
printDataStats bool parameter: set or list.
useFileCache bool parameter: set or list.
eidKey string key parameter: set or list.
KanEventInput> input help
input add <collection1> [<collection2> ...]
Add collection(s) to list
input list Lists the current collection list
input nosilent Sets verbosity level
input remove <collection1> [<collection2> ...]
Remove collection(s) from list
input silent Sets verbosity level
input wipe[-out] Empty the list
input addcomponent >comp1> [<comp2> ...] Adds a component to the input stream
Only input modules have these special input commands. The most important is input add. As in the Quicktour:
KanEventInput> input add /store/SP/R18/001237/200309/18.6.0b/SP_001237_013238
KanEventInput> input list
Collections:
/store/SP/R18/001237/200309/18.6.0b/SP_001237_013238
Components:
hdr
tag
tru
aod
cnd
KanEventInput> exit
>
Finally, you are ready to run the job. Again, as in the Quicktour,
you use the Framework's event commands to begin a loop over 3 events,
> ev beg -nev 3
Notice the multiple warning messages from MyMiniAnalysis.
You did not see these in the Quicktour,
but you see them now, because you set the verbose parameter to true.
Now continue the loop over the next 37 events:
> ev cont -nev 37
Again, you get the messages you saw in the Quicktour, plus the extra MyMiniAnalysis warning messages.
Finally you're done, so you can end the job and exit the Framework:
> exit
Now look at the histogram.
workdir> bbrroot
root [0] TFile f("NewName.root"); // Note that the file has the new name you gave it.
root [1] f.ls();
TFile** NewName.root Created for you by RooTupleManager
TFile* NewName.root Created for you by RooTupleManager
KEY: TH1F h1d1;1 MC reco abs mtm difference
KEY: TH1F h1d2;1 Reco track momentum
KEY: TH1F h1d3;1 Tracks per Event
KEY: TH1F h1d4;1 TagInspector Status
h1d3->Draw();
The following figure shows two Tracks per Event histograms:
the old Quicktour histogram on the left, and the new histogram that you just made on the right.
As you would expect, the histograms are different. The old histogram has 40 entries, whereas the new histogram has only 30 (29 + 1 "underflow").
This is because you told EvtCounter to skip events 10-20.
Note: The module talk command will tell you what parameters are available, and allow you to reset them, but it won't tell you what they are for.
So in order to make wise choices for the module parameters, you need to know a bit about the module: how it works, what it is for, and how resetting the parameters will affect your analysis.
Fortunately, most modules already come with acceptable default settings, so you do not have to worry about them.
Part of working with BaBar code is identifying which modules you do need to know about.
Usually, this will include your main analysis module, and possibly a few (maybe 1-5) more.
Review of Main Points
- Run-time control of analysis jobs is implemented with Tcl commands.
- BaBar has its own extra set of Tcl commands for job control.
- Important Framework commands: help, exit, module talk
- Important module commands: help, show, set, list
- The module talk command is used to set a module's run-time parameters.
Page maintained by Adam Edwards
Last modified: January 2008
|