Framework: the Environment for Running
Describes the BaBar Framework and how an application is built from it using
tcl commands.
BaBar's framework is a system that allows user-developed code to be
combined with central BaBar code to perform analysis. Its "top-down"
approach allows the user (you) to create or change low-level code without
harming the overall structure of the code. But although you will probably
supply only the low-level code, it is useful to know how your code fits
in with the rest of the BaBar code.
In the quick tour analysis you created a Beta executable by
compiling the code in the BetaMiniUser package.
When you issued the command:
workdir> BetaMiniApp snippet.tcl
you were eventually (after a lot of output to the screen) given a prompt
: '>'. In a sense, this prompt indicates that your are now
"in the Framework". From the Framework you can interact with your application.
In the quick tour analysis you issued the commands:
> ev beg -nev 3
and
> ev cont -nev 37
which were commands to the Framework to run the analysis application on the
first forty events.
> exit
was the command you used to end your Framework session and return to the
unix prompt.
Running Beta and BetaMiniUser executables is one of the most popular
ways to interact with the framework.
The framework approach provides many benefits,
including the ability to re-configure and enable or disable certain parts
of your analysis on the fly and, even better, it lets the user change analysis
parameters, the data source, and data destination without time consuming
re-compilation. You can even change analysis parameters halfway through
a job. So it is useful to understand how the framework is implemented
so that can you use it correctly.
Framework Implementation
So what exactly is the framework? Physically, the framework is a class
library, located in BaBar's framework package. The code is stored in
$BFDIST/packages/Framework. You can examine it there, or check it out
in the usual way (that is, cd to your release directory, and type
"addpkg Framework").
The framework class library defines the form of all user classes and the
interactions between these classes. It provides the mechanism for job control
to be passed from the operating system (i.e., the computer) to BaBar code.
(Note: The main() function that all C++ programs must have
is found inside Framework/AppMain.cc.)
The Framework allows four types of processes to occur:
- INPUT, from the database, generated events, other files,
etc,
- OUTPUT, to the database, to the screen, to another filesystem,
etc,
- PROCESSING of the data,
- FILTERING, to terminate or redirect subsequent data processing.
Modules
In the Offline world, BaBar code is built from well defined objects called
modules. Technically, a module is a C++ class that inherits from
a generic module class called AppModule. Any analysis code written to
interface to the Framework must be written as a module.
The framework passes data from module to module. Each module uses
the data to perform one well-defined task. There are many types of modules.
There are general-purpose modules that perform tasks like reconstruction,
and special-function modules exist; these are input modules
and output modules to control the input and output of data
and filter modules to control subsequent data processing.
In addition to the "ready made" modules, users can, and will
need to, write or modify their own analysis modules. User-defined modules
are made in a user's private test release. In the QuickTour analysis,
a user-defined module (the NTrkExample.cc and .hh files) was placed in the
BetaMiniUser package directory. In this directory there are already some
example and template module files.
Writing or modifying a user analysis module involves writing
or modifying C++ code. If this is unfamiliar, don't worry. This topic will
be covered with example exercises in
Booking
a New Histogram as an Example of Editing Code.
A Framework sequence is a ordered list of related modules (and
sometimes other sequences) that provide a defined task that is too big
for just one module. For example, reconstruction of the data from a given
sub-detector is a very big task, so a sub-detector reconstruction package
will typically include a sequence of modules to deal with the
reconstruction setup (e.g., loading of calibration constants),
selection of alogrithms, and actual reconstruction.
A path is an ordered list of modules and sequences. As they are created,
paths are strung together to form the main execution path.
This execution path starts at the active input module and
terminates at the active output module. Between the two is
everything related to an analysis, from (for example)
the data source, particle identification and reconstruction setups to
the analysis proper, and finally to the ntuple/database/file
output of results.
Each module, sequence, or path can be in one of two states: enabled
(the default) or disabled. Only enabled modules, sequences and paths
are seen by the framework and executed.
Uniqueness of Modules, Sequences and Paths
Legal module, sequence, and path names may contain the characters a-z, A-Z,
0-9 and '_'. Names are case sensitive. They are all global in
scope must be unique to your test release environment. However, once
instantiated, a module or a sequence can be appended to more than one sequence
or path. In this case, enabling or disabling a module or sequence will
affect every sequence or path that has the embedded module. Also, when the
application runs, the framework will only execute each enabled module once per
event (the first time the module is encountered in a path or sequence) along
its execution path.
Diagram of Framework Entities
The following diagram illustrates the relationships between modules, sequences,
paths. Filled-in boxes represent enabled modules (or sequences or paths),
and striped boxes represent disabled modules. The arrows represent paths,
and only boxes that the arrow passes through are appended to the path.
The execution path at the top of this figure shows
what happens for every event. The execution path begins
with the input module and terminates with the output module, both of
which have to be enabled and active. All enabled paths are executed.
The next level down in the figure shows that the individual path B (for
example) consists of several sequences and some individual modules.
Finally, the bottom part of the figure shows that sequence 1 (for
example) is in turn constructed out of modules.
You can see why it important to make sure that modules that you want to
execute are appended and enabled. Modules α, χ and
δ are connected in a sequence but the middle module has not been enabled,
so the framework ignores it. Also, the user-defined MyModule at the bottom has been
compiled successfully, but it hasn't been included in a sequence or path, so the
framework never sees it. Now, if this was your plan, then that is fine. For example,
if module χ is buggy (though it compiled) you may disable it
on purpose. On the other hand, if you expected module χ or MyModule to be
executed, then you are in trouble, because it won't be.
Application Execution
An application like BetaMiniApp is built from one or more paths, which in
turn are built from sequences and compiled modules. The
Framework must have access to an object of each module type that will
be used in the analysis path. In the BetaMiniUser package there are file
named AppUserBuild*.cc. These files contain the
implementation of the AppUserBuild* classes, which are the classes
responsible for creating module objects and making them available to
the Framework. It will be discussed further in the next chapter, Beta: the
Analysis Package.
Once the Framework has a list of module objects to work with,
Tcl scripts and commands can be used to create
and define the framework's sequences and execution path. In the
QuickTour analysis the snippet.tcl script file was executed in the Framework
by passing the tcl script as an argument to the BetaMiniApp executable.
workdir> BetaMiniApp snippet.tcl
Either more scripts, or the user directly, tells the framework what
to do next.
The framework will not work unless each module
has the uniform structure of the class AppModule. The class AppModule
and interface definition file is located in
$BFROOT/dist/releases/analysis-30/Framework/AppModule.cc
An analysis module, inheriting from AppModule, must provide
the following methods:
beginJob ( AbsEvent *
)event ( AbsEvent * ) endJob ( AbsEvent * )
In addition, input and output modules may provide
beginRun(AbsEvent *) and endRun(AbsEvent *)
member functions which BaBar's offline version of Framework also calls
at beginJob(AbsEvent *) and endJob(AbsEvent
*), respectively. These functions are only needed by OPR
(online prompt reconstruction), and are not used in analysis jobs.
A job is basically an entire Framework
session. In the Quicktour analysis, the Job began when we
ran the Beta executable (BetaMiniApp), and ended when we typed
exit at the Framework prompt. The beginJob()
member function is executed once (at the very beginning of a job) Similarly,
endJob() is executed once at the end.
The event() method is where analysis code goes. This
is executed once for each event in a run.
The above figure shows the order of execution in a job. A job
begins with a call to the beginJob() of each module,
starting with the active input module and ending with the active
output module. Then event() is called for each module,
and then endJob(). (Filter modules may cause some paths
to be skipped.)
For example, in the interactive job in the Quicktour, the
beginJob() method is initiated when you type the "ev beg"
("events begin") command.
As an example, imagine an analysis job run over 20,000
events. The user begins with the ev beg -nev 10000 command,
which causes the code beginRun() to be executed for the input and
output modules. Then the events are processed, meaning that event()
gets executed 10000 times. Then the user decides to modify the controlling tcl so as
to change some analysis parameters before the remaining 10000 events are
processed. (For example, maybe something happened to the detector at event 10001
so that different parameters are needed.) Then the user continues with ev
cont -nev 10000, causing the event() member function to be
called again 10000 times. Finally, when the controlling tcl exits,
the endjob() method cleans up.
The previous section described modules, paths, and sequences, but it did
not explain how to create, enable/disable, and append them. That is the
task of this section. The user interface to the Framework is based on Tcl,
Tool Command Language. BaBar uses standard Tcl with some added BaBar specific
commands.
There are two common ways of using Tcl to interface with the
Framework. One is through a script. Also you can enter one command at a
time from a Framework prompt (">"), entered with a carriage return (that is, by
pressing "enter" after each command).
We make use of a few standard conventions when expressing Tcl
command syntax. Anything contained inside square brackets: '[ ]' is optional.
This takes two forms. mod[ule] indicates that the entire word
need not be typed. It is as legal and proper to issue mod as
it is to issue module as a command. Also,
[<argument2>] implies that the extra argument is optional.
Names written <like this> are argument names
for which you will enter the appropriate value.
In addition to the standard Tcl commands,
the Framework has the following BaBar specific commands that can be displayed
at the '>' prompt in any application:
help- The help command lists the new commands, those
beyond the standard Tcl command set which have been defined for use with
BaBar code.
echo <arg1> [<arg2>
... ]- The echo command simply echos the argument to
standard output (usually your screen). Arguments that contain embedded
space characters should be enclosed in double quotes ("").
exit- The exit command exits the application. The end
of job callbacks for all modules that have been enabled will be executed.
Note that the exit command only has this meaning within the Framework itself.
This comand is also defined automatically for each module (module talk),
under which circumstances it returns control to the Framework.
Here is a complete list of general
commands. These can also be displayed by typing "help"
at the '>' prompt in a Framework application.
Event Processing Commands
ev[ents] beg[in] [-nev <nev>]
- The events begin command begins an event processing
loop. Beginning the event processing loop includes resetting counters and
rewinding input data files. By default the begin processing loop will continue
until all input events are exhausted. The optional -nev switch may be used
to specify an event limit.
ev[ents] cont[inue] [-nev <nev>]
- The events continue command continues the event
processing loop without issuing any resets. This continues analysis in the
same run as the previous event loop.
Here is a complete list of event commands. These
can also be displayed by typing "events help" at the '>'
prompt in a Framework application.
Module Commands
The commands related to modules all begin with the key word mod[ule].
It is through this command that you can set the enabled/disabled status and
also set the active input and output modules.
mod[ule] disable <mod1/all> [<mod2>
..]- The module disable command disables a module or
list of modules. A disabled module will be bypassed during event processing
even though it appears in a path or sequence. Modules are enabled by default.
mod[ule] enable <mod1/all> [<mod2>
..]
The module enable command re-enables a disabled module or list
of modules.
mod[ule] in[put] <mod>
- The module input command makes active the input
module for subsequent event processing loops. The specified module must also
be enabled.
mod[ule] out[put] <mod>
- The module output command makes active the output
module for subsequent event processing loops. The specified module must also
be enabled.
Here is a comprehensive list of
module commands. These can also be displayed
by typing "module help" at the '>' prompt in
a Framework application.
module talk
One of the more useful module commands is the module talk command
which allows interactive changing of certain module parameters. This is the
main feature that allows a user to alter analysis parameters without needing
to re-compile the code.
Issuing the module talk command transfers command
input to the specified module.
>module talk MyModule
MyModule>
At this point, any commands issued go through the module MyModule. (Note
that the '>' prompt changes to 'MyModule>'
to indicate that Framework commands are being issued to the module.)
Some standard module commands are help, to get a list of available
commands, show, to list the analysis parameters, and exit,
which returns control back to the Framework.
This is a very time-saving and useful feature. It is the
main subject of
Framework Continued.
module talk[To] input modules
The Framework is capable of receiving input from the Event Server for on-line
purposes, a data file for general offline analysis, or from the Event Generator
for analysis of simulated events. These have to be configured in the
active input module (usually KanEventInput) through the module talk command.
In addition to the basic module
commands of help and exit, this module has an input
command, which you have already used in the QuickTour:
> mod talk KanEventInput
KanEventInput> input add /store/SP/R14/001237/200309/14.3.1c/SP_001237_000533
Here is a comprehensive list of the
input commands.
A useful command, in the input module, that you might like
to use is described in the QuickTour. Basically it skips a specified number
of events - useful if you are bored with running over the same 40 events in
the Quicktour! You can skip, say 2000, events with:
module talk KanEventInput
first set 2001
exit
before you do the ev beg ... or ev cont ...commands.
This, and other available commands, can be displayed by typing "help"
at the '...EventInput>' prompt.
module talk[To] output modules
Output from a Framework application can be generated to file systems, displays,
to the database, etc. As in the case of the input module, these have to be
set by talking to the active output module (BdbEventOutput for the database
and RooEventOutput for Kanga.) The output command, issued
after talking to the output module, allows for the streaming of the event
from paths to destinations. Output streams must first be created, enabled,
the stream bound to a destination at one end, and bound to one or more paths
at the other end.
Here is a comprehensive list of the
output commands.
Sequence Commands
The commands related to sequences all begin with the key word seq[uence].
The first step in defining a sequence is to create it. Once
a sequence is created modules need to be added to it. An example
(from BetaMiniUser/MyDstarMicroAnalysis.tcl) is as follows:
sequence create MyDstarSequence
sequence append MyDstarSequence MyDstarAnalysis
This creates a sequence in the regime of the Framework named MyDstarSequence
and then appends the MyDstarAnalysis module to it. The final result is a
sequence consisting of one module. More modules could have been appended in
the initial append command or with multiple append commands.
Path Commands
The commands related to paths all begin with the key word path.
Creating a path and appending sequences and modules is analagous to creating
a sequence and appending modules. One useful command, is path list
(an analagous command exists for modules and sequences).
>path list
This command will list all of the paths available, their component
sequences and modules, and the enabled/disabled status of each. Here
is the output of the path
list command with respect to the executable created in the quick
tour analysis. Here is the output of the module list command with respect to the
quick tour analysis. The path list command shows all of the structure
in the paths by using indentation. The module list command lists all
modules that have been loaded into the Framework. Whether or not a
module is included in a defined path is seen only from the path list
command.
You may recognize the output of path list from the
Quicktour. That's because this command was included in MyMiniAnalysis.tcl,
so the first part of the output to the screen was just the output of the
path list command.
Here is a comprehensive list of
path commands.
Action Commands
Actions are are another useful feature of the Framework;
they are run just before and after each module's beginJob, beginRun, event,
endJob and endRun member functions. Hence, for example, they are useful
for measuring the amount of time spent processing in each module. Framework
uniformity of actions are accomplished by having all action classes inherit
from AppAction. Action commands apply to all modules; some common commands
are:
action disable <act1/all> [<act2>
..]
- Disables the action(s).
action enable <act1/all> [<act2>
..]
- Enables the action(s). (Actions are enabled by default.)
action list
- List all defined actions.
Here is a comprehensive list of
actioncommands.
Config Commands
(This is a feature of Framework that is not widely used; nevertheless we list
it here for completeness.) Very often, tcl scripts that run an application
source many secondary scripts which, in turn, setup paths, sequences and
enables or disables them and modules. The config commands allows you
to dump the entire application configuration into one tcl file so that it
can be rerun from just this one file.
Here is a comprehensive list
of config commands.
Tcl Scripts
Tcl scripts are used extensively to set up sequences, paths, and to perform
general Framework setup such as input and output files. Many types of sequences
are worthless if the order of the modules is wrong or if some modules of
the sequence are not enabled. To maintain this coherence there is often a
script file for each sequence. Although there is normally a one to one correspondence
between scripts and sequences it is important to realise that the two are
not the same.
Tcl scripts are files with names having a '.tcl' suffix.
Much of the syntax for these files is the same as the command line use. Each
line of the file may contain either a comment, a command, or some general
language constructs such as an if statement. Comment lines are indicated
by a pound character: '#'.
You should be aware that an initial tcl file may, and often
does, source many secondary scripts.
When the Beta executable was run in the quick tour analysis,
a tickle file was passed as an argument. Recall the command:
workdir> BetaMiniApp snippet.tcl
The job passes from the two setup tcl files,
snippet.tcl and MyMiniAnalysis.tcl, to
the main physics analysis file, btaMini.tcl. Here is
an annotated version of MyMiniAnalysis.tcl, with comments
in blue.
Related documents:
This has been a brief description of the Framework. You
will interact with it more in another section of this workbook: Framework
Continued.
General Related Documents:
- Overview, Reference Manual, and Users Guide postcript documents
are found in the
doc directory of the Framework package. Note
that these are outdated material but may still be illuminating.
Author:
Tracey Marsh
Contributors:
Daniel E. Azzopardi
Joseph Perl
Leon Rochester
Marc Turcotte
Asoka De Silva
Jenny Williams
Last modification: 13 June 2005
Last significant update: 24 Oct 2002
|
