SLAC PEP-II
BABAR
SLAC<->RAL
Babar logo
HEPIC E,S & H Databases PDG HEP preprints
Organization Detector Computing Physics Documentation
Personnel Glossary Sitemap Search Hypernews
Workbook Home Introduction Account Setup QuickTour Packages Modules Unwrap page!
Event Information Tcl Commands Editing Compile and Link Run the Job Debugging
Check this page with the
W3C Validator
(More checks...)
Parameters Tcl Files Find Data Batch Analysis ROOT Tutorial

Compile and link

To turn C++ code into a runnable program, you need to compile and link the code. Compiling the code puts it together into things called libraries (lib files). Linking the code uses the information in these libraries to build a program.

The program is the ultimate goal of the compile-and-link stage. The programs are usually called executables, binaries, or applications. BaBar application names usually end in "App". For example, when you compile and link the code in the BetaMiniUser package, it produces an application called BetaMiniApp.

In the last section, you made some changes to C++ code in the BetaMiniUser package. But in order for these changes to be implemented, you have to recompile and relink your code to make a new BetaMiniApp that includes your changes.

This section begins with an introduction to gmake, BaBar's compile-and-link utility. Then you will compile and link your code from Example 2. The section also includes suggestions for how to deal with compile-and-link problems, and an optional, more detailed discussion of gmake's GNUmakefiles.


Contents:


Introduction to gmake

BaBar's compile-and-link utility is called gmake. The basic gmake command is:

   > gmake <target> 

This tells gmake to "make" or "build" the target. Different targets correspond to different gmake actions. Some important targets include lib to compile code, and bin to link code.

The available targets and the instructions for building them are defined in a file called GNUmakefile. Whenever you issue a gmake command, gmake looks in the current directory for a GNUmakefile. Then it follows the instructions for that target.

There is a GNUmakefile in every release, including the test release you created. GNUmakefiles are also included in any package that you check out. The GNUmakefile of the release is the master GNUmakefile. This is why you usually issue gmake commands from the test release directory. Commands issued from a package in a release will only have access to targets defined in the package's GNUmakefile.

For general use of BaBar software, you will use gmake commands often, but you will probably not need to write or modify a GNUmakefile. Even users writing new analysis modules rarely need to make significant modifications to a GNUmakefile.

Important gmake targets

The all target

In the above example, you used the command "gmake all" to compile and link your code. The all target is a "smart" target that makes all of the targets you need (and some that you may not need) in the correct order. The targets made by gmake all are: Some of these targets are needed for a sucessful compile and link. "gmake include" takes care of dependencies. "gmake lib" compiles the code, and "gmake bin" links it. And the first command, "gmake installdirs", creates the $BFARCH directories needed to store the files created by the include, lib and bin targets.

On the other hand, as mentioned in the Quicktour, many of these targets are not needed for your simple BetaMiniUser job. For example, the database commands are needed only if you are planning to set up your own federated database, which is unlikely unless you are an advanced user running simulation software. Also, the javalinks and python targets are not really needed here. They are included in gmake all to ensure that the targets are made in the right order for users who do need them.

The installdirs target

Different machine architectures have different compiled formats. To keep object files of different architecture formats separate the lib and bin directories have a subdirectory for each architecture that you compile code on.

The command that makes these subdirectories is gmake installdirs. A new test release comes with lib, bin, tmp and test directories, but they are all empty. The command gmake installdirs creates a subdirectory called $BFARCH in each one of these subdirectories. Here $BFARCH is the architecture that you chose when you entered the srtpath command; for example, Linux24SL3_i386_gcc323.

The include target

The include target creates any include files needed. These files record the dependencies among your source code files. Their names have the same root as the source file and a '.d' suffix. In BaBar software these files are stored in the tmp directory of your release.

Like the lib and bin directories, the tmp directory has a subdirectory for each architecture that you compile code on.

The lib target

The command to compile your code is gmake lib.

Compiling a C++ file converts it into a re-locatable, architecture-dependent version of the source code. The result of this step is called an object. The object files also have the same root name as the source file with a '.o' suffix. Object files are also stored in the tmp directory of your test release.

These object files are swept into archive libraries. Libraries are stored in the lib directory of your release and have a '.a' suffix.

The bin Target

Once object modules have been created they need to be linked together to build the executable. The command to link your code is gmake bin. The new executables are placed in your bin/$BFARCH directory. If you list this directory, you will see a '*' after the names of most of the files, indicating that they are executables.

Starting clean: the clean target

When issued from your test release directory, gmake clean will delete the files in the directories: bin, lib, tmp, test, and results for all architectures. This removes all object files, libraries, and the executable from your release so that you may restart the compilation and building process of the executable from a "clean" state.

A related useful target is cleanarch. The command gmake cleanarch is like gmake clean, but it removes only library, binary and temporary files saved under the current architecture (i.e. the architecture set when you most recently issued an srtpath command in the current session). It leaves the libraries and/or binaries built with other architectures intact.

A shortcut: Compiling or linking only one package

So far, you have always used the command "gmake all" to compile and link your code. But if all you want to do is make the BetaMiniApp executable, then you need only some of the gmake commands invoked by "gmake all":

gmake lib
gmake BetaMiniUser.bin

Or you could put them together to make a single command:

gmake lib BetaMiniUser.bin

You will notice that the link command, gmake BetaMiniUser.bin, is a bit different from the gmake bin command that you used before. In general, commands of the form "gmake package.target" perform the task "gmake target" on the files in "package" only. The command "gmake BetaMiniUser.bin" links only the code in BetaMiniUser. This is useful if you have a lot of packages checked out, because instead of making a bunch of executables that you don't need, you make only the BetaMiniUser executables.

Let's look at another example. Assume that you have checked out:

BetaMiniUser
KanEvent
RecoDataK
and have made substantial changes to files in BetaMiniUser and a few in at least one of the other two packages. Assume also, that there are two binaries (i.e. executable files) that can possibly be built in the BetaMiniUser package, FirstApp and SecondApp, and that it takes a long time to compile the binaries. In the case that you care only about the SecondApp executable, a common series of commands would be:
gmake clean     //remove pre-editing libraries and other temporary files
gmake lib       //rebuild the libraries for all the packages
gmake BetaMiniUser.SecondApp  //build only the SecondApp executable
The gmake clean command removes the library and binary files and other auxilliary and machine-generated files that existed before the package was edited to avoid possible conflicts. The gmake lib command need remakes all the needed library files. Then gmake BetaMiniUser.SecondApp builds the SecondApp executable.

Using only the gmake commands that you need can save you some compile time, and avoids the creation of lots of unnecessary files. A word of caution, however: if you are going to skip parts of gmake all, make sure that you know what you are doing. If your job keeps crashing, try gmake all instead.

gmake options

The gmake command comes with a number of useful options. The command "gmake -n TARGET" doesn't actually run commands, it just prints them. For example, if you enter the command "gmake -n lib" from your ana41 directory, then you will get the following output:
GNU Make version 3.79.1,
Build OPTIONS = Linux24SL3_i386_gcc323-Debug-native-Objy-Optimize-Fastbuild-Ldlink2-SkipSlaclog-Static-Lstatic
Linux yakut06 2.4.21-47.ELsmp #1 SMP Thu Jul 20 10:30:12 CDT 2006 i686 athlon i386 GNU/Linux  [uname -a]
echo "-> lib";
for package in xx BetaMiniUser database workdir; do if [ "$package" != "xx" -a "$package" != "BASE" ]; 
then gmake  $package.lib NOPKG=yes; fi; done
If you can understand it, this message gives you an idea of what the gmake lib command would do if you entered it without the "-n" switch. However, if you check in your lib directory:
   ana41> ls -l lib/$BFARCH
you will not find any new lib files; only old lib files (if you have compiled before) or none at all.

Another useful gmake command is "gmake -k TARGET". This command tells gmake to keep going even when some targets can't be made.

To see a full list of options and a description of each one, enter "gmake --help" at the command line.


Example 2, Part 2: Compile and link your new code

Quick links:

Now it is time to compile and link your new code from Example 2 of the Editing Code section.

First, make sure you have done:

ana41> srtpath <enter> <enter>
ana41> cond22boot

If you look in your lib and bin directories, you will see your old lib and bin files from the Quicktour: ana41> ls -l lib/$BFARCH total 2520 -rw-r--r-- 1 penguin br 2577960 Apr 20 21:17 libBetaMiniUser.a drwxr-xr-x 5 penguin br 2048 Apr 20 21:05 templates ana41> ls -l bin/$BFARCH/ total 69009 -rwxr-xr-x 1 penguin br 70663891 Apr 20 21:21 BetaMiniApp -rw-r--r-- 1 penguin br 72 Apr 20 21:23 Index

Let's clean out these old files before we begin, just to be on the safe side:

ana41> gmake clean
The system will respond with something like this:
GNU Make version 3.79.1,
Build OPTIONS = Linux24SL3_i386_gcc323-Debug-native-Objy-Optimize-Fastbuild-Ldlink2-SkipSlaclog-Static-Lstatic
Linux yakut03 2.4.21-47.ELsmp #1 SMP Thu Jul 20 10:30:12 CDT 2006 i686 athlon i386 GNU/Linux  [uname -a]
-> clean:
-> cleanarch: Linux24SL3_i386_gcc323
'tmp/Linux24SL3_i386_gcc323' is soft link to /afs/slac.stanford.edu/g/babar/build/p/penguin/ana41/tmp/Linux24SL3_i386_gcc323
'shtmp/Linux24SL3_i386_gcc323' is soft link to /afs/slac.stanford.edu/g/babar/build/p/penguin/ana41/shtmp/Linux24SL3_i386_gcc323
rm -f -r  bin/Linux24SL3_i386_gcc323 lib/Linux24SL3_i386_gcc323 shlib/Linux24SL3_i386_gcc323 
/afs/slac.stanford.edu/u/br/penguin/ana41/shlib/Linux24SL3_i386_gcc323/ldlink 
/afs/slac.stanford.edu/u/br/penguin/ana41/shlib/Linux24SL3_i386_gcc323/LDLINK 
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana41/tmp/Linux24SL3_i386_gcc323 
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana41/shtmp/Linux24SL3_i386_gcc323 
test/Linux24SL3_i386_gcc323 results
mkdir  bin/Linux24SL3_i386_gcc323 lib/Linux24SL3_i386_gcc323 shlib/Linux24SL3_i386_gcc323 
/afs/slac.stanford.edu/u/br/penguin/ana41/shlib/Linux24SL3_i386_gcc323/ldlink 
/afs/slac.stanford.edu/u/br/penguin/ana41/shlib/Linux24SL3_i386_gcc323/LDLINK 
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana41/tmp/Linux24SL3_i386_gcc323 
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana41/shtmp/Linux24SL3_i386_gcc323 
test/Linux24SL3_i386_gcc323 results
-> installdirs:
(Here I have split the lines so the output would fit on the page.)

The gmake messages are actually quite useful - gmake tells you what it is doing. You can see from the above message that gmake clean removes your old xxx/$BFARCH directories, and then does a "gmake installdirs" to put back new, empty ones.

If you check your lib and bin directories again, you will find that they are now (almost) empty:

ana41> ls -l lib/$BFARCH
total 2
drwxr-xr-x    5 penguin  br           2048 Apr 21 20:44 templates
ana41> ls -l bin/$BFARCH
total 0
ana41>

Now you can compile and link your code. Compile and/or link jobs should always be send to the bldrecoq queue. From your test release issue

  
ana41> rm all.log
ana41> bsub -q bldrecoq -o all.log gmake all

A feature of the batch system (bsub part of the command) is that log files are NOT overwritten. Instead, if you do not remove your old all.log file, then the output for the current job will be appended to the bottom of the old log file. To avoid that, you either have to delete the old log file, or choose a new name for your new log file. That is why you removed the old log file above.

The gmake all command compiles and links the code.

The first thing to do after compiling and linking is to check your bin directory to make sure that your binary was produced and that it is brand new:

ls -l bin/$BFARCH
total 69014
-rwxr-xr-x    1 penguin  br       70669051 Apr 21 20:56 BetaMiniApp
-rw-r--r--    1 penguin  br             72 Apr 21 20:57 Index
Also, check your log file to make sure there are no error messages. If your compile/link was successful, your log file will look something like this: all.log.

Problems with Compile and Link

If you make a mistake when you edit BaBar code, then you will get an error messages on your terminal or in your log file. There are two types of problems that you could encounter:

  1. Compile and link errors: The gmake command(s) fail
  2. Run-time errors: The compile and link is successful, but when you try to run the executable, the job crashes.

This section will focus on the first type of error: compile and link errors. Later, the debugging section will teach you how to deal with run-time errors.

Compile and link errors: an example

You may not have a successful compilation the first time. The more code you edit, the more likely you are to make a small mistake that could cause the compile or link process to fail.

The first thing you should do after every compile or link is check your bin directory to make sure your binary was produced and that it is brand new. If it was not produced, then you need to look in your log file to find out why gmake failed.

As an example, suppose you accidentally removed the line

#include "BetaMiniUser/QExample.hh"
from QExample.cc. Then you try to compile and link with "gmake all" as usual. (You don't need to do this, you can just read this example. Though of course you are welcome to try it if you like.)

When this is done, you check the bin directory:

ana41> ls -l bin/$BFARCH
total 1
-rw-r--r--    1 penguin  br             72 May 15 04:40 Index
There is no BetaMiniApp in the bin directory. Something has gone wrong.

You investigate the log file for further clues. Here is the log file: all.log with errors. Note that the target that gmake is working on is indicated by the little arrows in the log file, "->BetaMiniUser.lib" and "->BetaMiniUser.bin".)

Scrolling down the log file, you find several gmake error messages. There's one in the BetaMiniUser.lib stage:

gmake[3]: *** [/afs/slac.stanford.edu/u/br/penguin/ana41/lib/Linux24SL3_i386_gcc323/l
ibBetaMiniUser.a(QExample.o)] Error 1
gmake[2]: *** [BetaMiniUser.lib] Error 2
and one in the BetaMiniUser.bin stage:
gmake[4]: *** [/afs/slac.stanford.edu/u/br/penguin/ana41/lib/Linux24SL3_i386_gcc323/l
ibBetaMiniUser.a(QExample.o)] Error 1

Whenever you see those stars (***) and the word Error, that means you are in trouble.

Now look at the lines just above the ***/Error line to find out what was the last thing gmake did before it failed. Right before the lib-stage error, the message is:

Compiling QExample.cc [libBetaMiniUser.a] [cc-1]
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:25: syntax 
   error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:37: syntax 
   error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:51: syntax 
   error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:57: warning: ISO
   C++ forbids declaration of `_numTrkHisto' with no type
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:57: `manager
   ' was not declared in this scope
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:58: warning: ISO
   C++ forbids declaration of `_pHisto' with no type
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:58: `manager
   ' was not declared in this scope
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:60: syntax 
   error before `return'
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:66: syntax 
   error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:75: syntax 
   error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:83: syntax 
   error before `->' token
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:86: `trkList
   ' was not declared in this scope
/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:88: syntax 
   error before `while'

(In this case, the error messages for the lib and bin stages are identical. This does not always happen. In any case you should look at the errors from the earliest target first, because later targets depend on earlier targets. Fixing the lib-stage error might fix the bin-stage error automatically.)

One message that is repeated over and over again is "syntax error before '::' token." What does that mean? You're not sure. So you decide to look at the first error message that occurs, since that is the first place that gmake ran into trouble:

/afs/slac.stanford.edu/u/br/penguin/ana41/BetaMiniUser/QExample.cc:25: syntax 
   error before `::' token
Here is line 25 of QExample.cc:
QExample::QExample( const char* const theName, 

The job fails at the beginning of the QExample constructor. Not only that, but according to the error message, it fails before the '::' token.

Now, there is only one thing in front of the '::' token: the word QExample. But QExample is a perfectly valid class - why should gmake think that it is a syntax error? It seems that for some reason, gmake does not recognize the QExample class. And if gmake does not recognize a class, then that is probably because it has not read the header file for that class.

Sure enough, when you check QExample.cc, you find that you have forgotten to include QExample.hh. This would explain the other error messages as well: every time gmake sees "QExample", it is confused because it does not know what QExample is. Furthermore, it also does not recognize _numTrkHisto, because this object was defined in QExample.hh as a private member object.

You put back the #include statement, and send your "gmake all" job again,

ana41> rm all.log
ana41> bsub -q bldrecoq -o all.log gmake all

This time, everything worked fine:

ana41> ls -l lib/$BFARCH

total 2408
-rw-r--r--    1 penguin  br        2462826 May 15 04:43 libBetaMiniUser.a
drwxr-xr-x    5 penguin  br           2048 May 15 04:36 templates

ana41> ls -l bin/$BFARCH

total 48073
-rwxr-xr-x    1 penguin  br       49225543 May 15 04:44 BetaMiniApp
-rw-r--r--    1 penguin  br             72 May 15 04:44 Index

There is a brand new BetaMiniApp! And you can tell from the time that BetaMiniUser's lib file has also been recompiled.

Normally, gmake is pretty good at figuring out what needs to be recompiled and what doesn't. But to be safe, you can issue a "gmake clean" to clean out all the old lib and bin files before you recompile.

The example above of course shows you only one example of the many, many possible error messages that can appear in your log file. Other typical compile errors include:

Compile/link error messages can be confusing. It is not always obvious what the problem is. In the end, all you can do is try your best to decipher the log file. If you can't figure it out, then your best bet is to ask someone who has had more practice with gmake errors, or, if you are all alone, submit your problem to the prelimbugs Hypernews forum. When you submit your question, be sure to indicate:

For example, if you could not figure out what was wrong in the above example, you would first explain your problem, and then provide the following details:

You will get an answer much faster if you provide complete information.

If your compile-and-link problem occurs in one of the Workbook examples, then you can email me, the Workbook editor, and I'll see what I can do. Perhaps you have discovered a bug in the Workbook! (But please try to solve it yourself, first.)

More about GNUmakefiles (optional)

Now you have learned most of what you need to know to use gmake to compile and link. But you may find it helpful to learn a bit more about how GNUmakefiles work. One of the virtues of GNUmakefiles is their ability to handle the many dependencies among the many, many lines of C++ code that must be put together to make an executable. This (optional) section revisits the GNUmakefile, with a focus on how these dependencies are managed.

An executable is built from the code defined in multiple files. When changes are made in one or more of these files the code in the modified files and all dependent files will need to be re-compiled and re-linked. Compile and link commands often involve many options, such as directory search paths for included files, names for compiled files, warning messages, debugging information and so forth. Compile and link commands quickly become quite lengthy, and in large software projects the dependencies amongst files is usually rather involved. gmake's job is to manage this complicated process.

GNUmakefiles revisited

GNUmakefiles define a set of instructions for how to compile and assemble code segments into the executable. The file also defines many variables such as search paths, which compiler to use, etc. The instructions specify the dependencies of the code segments so that the gmake utility can recognize only those components that need to be reprocessed. A well-written makefile can reduce unnecessary compilation and save much time.

The gmake facility looks in the current directory for a file named GNUmakefile, makefile, or Makefile (in that order). The first one of these files found is used to define the dependency/build rules for the given target.

Targets, Dependencies, and Build Rules

The general structure of an instruction in the GNUmakefile consists of a target, a dependency, and a rule.

Associated with each target is a set of dependent files. When any of the dependent files have been modified or if the target file does not exist, the target will be rebuilt (or built) in accordance with the rule. For example, a line in a GNUmakefile might be:

   foo.o : foo.cc foo.hh bar.hh
   	g++ -c foo.cc 
The target is foo.o, the dependent files are foo.cc, foo.hh, and bar.hh, and the rule is "g++ -c foo.cc". If any of foo.cc, foo.hh or bar.hh have been modified, or if the foo.o does not yet exist, issuing gmake foo.o will result in the recompilation of foo.cc in accordance with the rule. The default target is the first target listed in the GNUmakefile. It is the rule that will be executed when gmake is invoked without an argument.

Sometimes targets are used to perform tasks not related to compiling and linking code. If you define a target for which there is no file of the same name, and the rule commands do not create the target file, then a file of the target name will not exist in the directory. Because the target does not have a corresponding file it is always considered out of date; the rules will be executed every time the target is made. These targets are referred to as phony targets. Phony targets do not usually have dependent files.

An example of a phony target is workdir's setup target. The command gmake workdir.setup does not compile or link code. Instead, it creates some links in workdir: PARENT, RELEASE, bin, and shlib.

Time stamps

gmake relies on time stamps to determine what needs to be redone. The Makefile (or GNUmakefile) for a given package knows about the dependences between all the files. If a given file is being compiled with a gmake command, any files that it depends upon and that are newer than it will also be compiled. Therefore, if file X is newer than file Y, but Y depends on X, recompiling Y will force a recompilation of X, and any other files that might have been edited since Y was last touched.

As such, to force a file to be recompiled, you may need to manually "touch" one of the relevant files. An example of such a command is:

  touch BetaMiniUser/AppUserBuildBase.cc
This changes the time stamp on AppUserBuild.cc, so that gmake will identify it as a new file, and recompile it. In this case, this will cause almost the entire package to recompile as almost all the other files in the package depend on AppUserBuildBase.cc.

Link files

In BaBar software each package includes a link_PACKAGE.mk file in addition to the GNUmakefile. The link file is used by the GNUmakefile. The use of this file is mainly for overriding previous definitions of linking rules defined by the test release's GNUmakefile. This overriding mostly serves to append the new package directory to the linking list. More information on this can be found in the HOWTO package in HOWTO-dependency.
General Related Documents:

Back to Workbook Front Page

Send comments to Workbook Team.