Compile and Link
To turn C++ code into a runnable program, you need to compile and
link the code. Compiling the code puts it together into things
called libraries (lib files). Linking the code uses the information
in these libraries to build a program.
The program is the ultimate goal of the compile-and-link stage.
The programs are usually called executables, binaries,
or applications. BaBar application names usually end in "App".
For example, when you compile and link the code in the BetaMiniUser
package, it produces an application called BetaMiniApp.
In the last section, you made some changes to C++ code in the
BetaMiniUser package. But in order for these changes to be implemented,
you have to recompile and relink your code to make a new BetaMiniApp
that includes your changes.
This section begins with an introduction to gmake, BaBar's
compile-and-link utility. Then you will compile and link your
code from Example 2.
The section also includes suggestions for how to deal with
compile-and-link problems, and an optional, more detailed discussion
of gmake's GNUmakefiles.
Contents:
BaBar's compile-and-link utility is called gmake.
The basic gmake command is:
> gmake <target>
This tells gmake to "make" or "build" the target.
Different targets correspond to different gmake actions.
Some important targets include lib to
compile code, and bin to link code.
The available targets and the instructions for building them
are defined in a file called GNUmakefile.
Whenever you issue a gmake command, gmake looks in the
current directory for a GNUmakefile. Then it follows the
instructions for that target.
There is a GNUmakefile in every release, including the test release you
created. GNUmakefiles are also included in any package that you check
out. The GNUmakefile of the release is the master GNUmakefile.
This is
why you usually issue gmake commands from the test release
directory. Commands issued from a package in a release will only
have access to targets defined in the package's GNUmakefile.
For general use of BaBar software, you will use gmake commands
often, but you will probably not need to write or modify a GNUmakefile.
Even users writing new analysis modules rarely need to make significant
modifications to a GNUmakefile.
In the above example, you used the
command "gmake all" to compile and link your code. The all target
is a "smart" target that makes all of the targets you need (and some that
you may not need) in the correct order. The targets made by gmake all are:
- installdirs - Creates the required architecture-specific
directories (for example, lib/$BFARCH and bin/$BFARCH).
- database.import - A database command. Creates a local copy of a
database and its bootfile. (But it will not work unless you have made your
.bbobjy file.)
- schema - A database command. Sets up class definitions for a
database.
- include - Makes any needed include files.
- lib - Compiles the code.
- bin - Links the code.
- javalinks - Support for java.
- python - Support for python.
Some of these targets are needed for a sucessful compile and link.
"gmake include" takes care of dependencies. "gmake lib" compiles the
code, and "gmake bin" links it. And the first command, "gmake installdirs",
creates the $BFARCH directories needed to store the files created by the
include, lib and bin targets.
On the other hand, as mentioned in the
Quicktour,
many of these targets are not needed for your simple BetaMiniUser job.
For example, the database commands are needed only if you are planning
to set up your own federated database, which is unlikely unless you are
an advanced user running simulation software. Also, the javalinks and
python targets are not really needed here. They are included in gmake all
to ensure that the targets are made in the right order for users who
do need them.
Different machine architectures have different compiled formats. To
keep object files of different architecture formats separate the
lib and bin directories have a subdirectory for each architecture
that you compile code on.
The command that makes these subdirectories is gmake installdirs.
A new test release comes with lib, bin, tmp and test directories, but they
are all empty. The command gmake installdirs creates a subdirectory called
$BFARCH in each one of these subdirectories. Here $BFARCH is the
architecture that you chose when you entered the srtpath command; for
example, Linux24SL3_i386_gcc323.
The include target creates any include files needed.
These files record the dependencies among your source
code files. Their names have the same root as the source
file and a '.d' suffix. In BaBar software these files
are stored in the tmp directory of your release.
Like the lib and bin directories, the tmp directory has a subdirectory
for each architecture that you compile code on.
The command to compile your code is gmake lib.
Compiling a C++ file converts it into a re-locatable, architecture-dependent version of the
source code. The result of this step is called an object. The object
files also have the same root name as the source file with a '.o'
suffix. Object files are also stored in the tmp directory
of your test release.
These object files are swept into archive libraries. Libraries are
stored in the lib directory of your release and have a '.a' suffix.
Once object modules have been created they need to be linked together
to build the executable. The command to link your code is gmake bin.
The new executables are placed in your bin/$BFARCH directory.
If you list this directory, you will see a '*' after the names of most of
the files, indicating that they are executables.
When issued from your test release directory,
gmake clean will delete the files in the
directories: bin, lib, tmp, test, and results for all
architectures. This removes all object files, libraries, and the
executable from your release so that you may restart the compilation
and building process of the executable from a "clean" state.
A related useful target is cleanarch. The command gmake cleanarch
is like gmake clean, but it removes only library, binary and temporary files saved under
the current architecture (i.e. the architecture set when you most
recently issued an srtpath command in the current session).
It leaves the libraries and/or binaries built with other architectures
intact.
So far, you have always used the command "gmake all" to compile
and link your code. But if all you want to do is make the
BetaMiniApp executable, then you need only some of the gmake commands
invoked by "gmake all":
gmake lib
gmake BetaMiniUser.bin
Or you could put them together to make a single command:
gmake lib BetaMiniUser.bin
You will notice that the link command, gmake BetaMiniUser.bin,
is a bit different from the gmake bin command that you used before.
In general, commands of the form "gmake package.target" perform
the task "gmake target" on the files in "package" only.
The command "gmake BetaMiniUser.bin" links only the code in
BetaMiniUser. This is useful if you have a lot of packages
checked out, because instead of making a bunch of executables
that you don't need, you make only the BetaMiniUser executables.
Let's look at another example. Assume that you have checked out:
BetaMiniUser
KanEvent
RecoDataK
and have made substantial changes to files in BetaMiniUser and a few in
at least one of the other two packages. Assume also, that there are
two binaries (i.e. executable files) that can possibly be built in the
BetaMiniUser package, FirstApp and SecondApp, and that it takes a long time
to compile the binaries. In the case that you care only about the
SecondApp executable, a common series of commands would be:
gmake clean //remove pre-editing libraries and other temporary files
gmake lib //rebuild the libraries for all the packages
gmake BetaMiniUser.SecondApp //build only the SecondApp executable
The gmake clean command removes the library and binary files and other
auxilliary and machine-generated files that existed before the package was
edited to avoid possible conflicts. The gmake lib command need remakes all
the needed library files. Then gmake BetaMiniUser.SecondApp builds
the SecondApp executable.
Using only the gmake commands that you need can save you
some compile time, and avoids the creation of lots of unnecessary
files. A word of caution, however: if you are going to skip parts of
gmake all, make sure that you know what you are doing. If your
job keeps crashing, try gmake all instead.
The gmake command comes with a number of useful options.
The command "gmake -n TARGET" doesn't actually run commands, it
just prints them. For example, if you enter the command
"gmake -n lib" from your ana42 directory, then you will get the
following output:
GNU Make version 3.79.1,
Build OPTIONS = Linux24SL3_i386_gcc323-Debug-native-Objy-Optimize-Fastbuild-Ldlink2-SkipSlaclog-Static-Lstatic
Linux yakut06 2.4.21-47.ELsmp #1 SMP Thu Jul 20 10:30:12 CDT 2006 i686 athlon i386 GNU/Linux [uname -a]
echo "-> lib";
for package in xx BetaMiniUser database workdir; do if [ "$package" != "xx" -a "$package" != "BASE" ];
then gmake $package.lib NOPKG=yes; fi; done
If you can understand it, this message gives you an idea of
what the gmake lib command would do if you entered
it without the "-n" switch. However, if you check in your
lib directory:
ana42> ls -l lib/$BFARCH
you will not find any new lib files; only old lib files
(if you have compiled before) or none at all.
Another useful gmake command is "gmake -k TARGET". This
command tells gmake to keep going even when some targets
can't be made.
To see a full list of options and a description of each one,
enter "gmake --help" at the command line.
Quick links:
Now it is time to compile and link your new code from
Example 2
of the Editing Code section.
First, make sure you have done:
ana42> srtpath <enter> <enter>
ana42> cond22boot
If you look in your lib and bin directories, you will
see your old lib and bin files from the
Quicktour:
ana42> ls -l lib/$BFARCH
total 2520
-rw-r--r-- 1 penguin br 2577960 Apr 20 21:17 libBetaMiniUser.a
drwxr-xr-x 5 penguin br 2048 Apr 20 21:05 templates
ana42> ls -l bin/$BFARCH/
total 69009
-rwxr-xr-x 1 penguin br 70663891 Apr 20 21:21 BetaMiniApp
-rw-r--r-- 1 penguin br 72 Apr 20 21:23 Index
Let's clean out these old files before we begin, just to be on the safe side:
ana42> gmake clean
The system will respond with something like this:
GNU Make version 3.79.1,
Build OPTIONS = Linux24SL3_i386_gcc323-Debug-native-Objy-Optimize-Fastbuild-Ldlink2-SkipSlaclog-Static-Lstatic
Linux yakut03 2.4.21-47.ELsmp #1 SMP Thu Jul 20 10:30:12 CDT 2006 i686 athlon i386 GNU/Linux [uname -a]
-> clean:
-> cleanarch: Linux24SL3_i386_gcc323
'tmp/Linux24SL3_i386_gcc323' is soft link to /afs/slac.stanford.edu/g/babar/build/p/penguin/ana42/tmp/Linux24SL3_i386_gcc323
'shtmp/Linux24SL3_i386_gcc323' is soft link to /afs/slac.stanford.edu/g/babar/build/p/penguin/ana42/shtmp/Linux24SL3_i386_gcc323
rm -f -r bin/Linux24SL3_i386_gcc323 lib/Linux24SL3_i386_gcc323 shlib/Linux24SL3_i386_gcc323
/afs/slac.stanford.edu/u/br/penguin/ana42/shlib/Linux24SL3_i386_gcc323/ldlink
/afs/slac.stanford.edu/u/br/penguin/ana42/shlib/Linux24SL3_i386_gcc323/LDLINK
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana42/tmp/Linux24SL3_i386_gcc323
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana42/shtmp/Linux24SL3_i386_gcc323
test/Linux24SL3_i386_gcc323 results
mkdir bin/Linux24SL3_i386_gcc323 lib/Linux24SL3_i386_gcc323 shlib/Linux24SL3_i386_gcc323
/afs/slac.stanford.edu/u/br/penguin/ana42/shlib/Linux24SL3_i386_gcc323/ldlink
/afs/slac.stanford.edu/u/br/penguin/ana42/shlib/Linux24SL3_i386_gcc323/LDLINK
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana42/tmp/Linux24SL3_i386_gcc323
/afs/slac.stanford.edu/g/babar/build/p/penguin/ana42/shtmp/Linux24SL3_i386_gcc323
test/Linux24SL3_i386_gcc323 results
-> installdirs:
(Here I have split the lines so the output would fit on the page.)
The gmake messages are actually quite useful - gmake tells you what it is doing.
You can see from the above message that gmake clean removes your old xxx/$BFARCH
directories, and then does a "gmake installdirs" to put back new, empty ones.
If you check your lib and bin directories again, you will
find that they are now (almost) empty:
ana42> ls -l lib/$BFARCH
total 2
drwxr-xr-x 5 penguin br 2048 Apr 21 20:44 templates
ana42> ls -l bin/$BFARCH
total 0
ana42>
Now you can compile and link your code. Compile and/or link jobs
should always be send to the bldrecoq
queue. From your test release issue
ana42> rm all.log
ana42> bsub -q bldrecoq -o all.log gmake all
A feature of the batch system (bsub part of the command)
is that log files are NOT overwritten. Instead, if you
do not remove your old all.log file, then the output for the current
job will be appended to the bottom of the old log file.
To avoid that, you either have to delete the old log file, or choose
a new name for your new log file. That is why you removed the
old log file above.
The gmake all command compiles and links the code.
The first thing to do after compiling and linking is to
check your bin directory to make sure that your binary
was produced and that it is brand new:
ls -l bin/$BFARCH
total 69014
-rwxr-xr-x 1 penguin br 70669051 Apr 21 20:56 BetaMiniApp
-rw-r--r-- 1 penguin br 72 Apr 21 20:57 Index
Also, check your log file to make sure there are no error messages.
If your compile/link was successful, your log file will look something
like this: all.log.
If you make a mistake when you edit BaBar code, then you
will get an error messages on your terminal or in your log file.
There are two types of problems that you could encounter:
- Compile and link errors: The gmake command(s) fail
- Run-time errors: The compile and link is successful, but when
you try to run the executable, the job crashes.
This section will focus on the first type of error: compile
and link errors. Later, the
debugging section
will teach you how to deal with run-time errors.
You may not have a successful compilation the first time.
The more code you edit, the more likely you are to make a
small mistake that could cause the compile or link process
to fail.
The first thing you should do after every compile or
link is check your bin directory to make sure your binary was
produced and that it is brand new. If it was
not produced, then you need to look in your log
file to find out why gmake failed.
As an example, suppose you accidentally removed the line
#include "BetaMiniUser/QExample.hh"
from QExample.cc. Then you try to compile and link with
"gmake all" as usual. (You don't need to do this, you can just
read this example. Though of course you are welcome to try it
if you like.)
When this is done, you check the bin directory:
ana42> ls -l bin/$BFARCH
total 1
-rw-r--r-- 1 penguin br 72 May 15 04:40 Index
There is no BetaMiniApp in the bin directory. Something has gone wrong.
You investigate the log file for further clues.
Here is the log file: all.log with errors.
Note that the target that gmake is working on is indicated by the little
arrows in the log file, "->BetaMiniUser.lib" and "->BetaMiniUser.bin".)
Scrolling down the log file, you find several gmake error messages.
There's one in the BetaMiniUser.lib stage:
gmake[3]: *** [/afs/slac.stanford.edu/u/br/penguin/ana42/lib/Linux24SL3_i386_gcc323/l
ibBetaMiniUser.a(QExample.o)] Error 1
gmake[2]: *** [BetaMiniUser.lib] Error 2
and one in the BetaMiniUser.bin stage:
gmake[4]: *** [/afs/slac.stanford.edu/u/br/penguin/ana42/lib/Linux24SL3_i386_gcc323/l
ibBetaMiniUser.a(QExample.o)] Error 1
Whenever you see those stars (***) and the word Error,
that means you are in trouble.
Now look at the lines just above the ***/Error line to find out what was the last
thing gmake did before it failed. Right before the lib-stage error, the message is:
Compiling QExample.cc [libBetaMiniUser.a] [cc-1]
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:25: syntax
error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:37: syntax
error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:51: syntax
error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:57: warning: ISO
C++ forbids declaration of `_numTrkHisto' with no type
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:57: `manager
' was not declared in this scope
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:58: warning: ISO
C++ forbids declaration of `_pHisto' with no type
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:58: `manager
' was not declared in this scope
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:60: syntax
error before `return'
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:66: syntax
error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:75: syntax
error before `::' token
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:83: syntax
error before `->' token
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:86: `trkList
' was not declared in this scope
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:88: syntax
error before `while'
(In this case, the error messages for the lib
and bin stages are identical. This does not always happen.
In any case you should look at the errors from the earliest target
first, because later targets depend on earlier targets. Fixing
the lib-stage error might fix the bin-stage error automatically.)
One message that is repeated over and over again is "syntax error before '::' token."
What does that mean? You're not sure. So you decide to look at the first error message
that occurs, since that is the first place that gmake ran into trouble:
/afs/slac.stanford.edu/u/br/penguin/ana42/BetaMiniUser/QExample.cc:25: syntax
error before `::' token
Here is line 25 of QExample.cc:
QExample::QExample( const char* const theName,
The job fails at the beginning of the QExample constructor. Not only that,
but according to the error message, it fails before the '::' token.
Now, there is only one thing in front of the '::' token: the word
QExample. But QExample is a perfectly valid class - why should gmake think
that it is a syntax error? It seems that for some reason, gmake does not
recognize the QExample class. And if gmake does not recognize a class,
then that is probably because it has not read the header file for that class.
Sure enough, when you check QExample.cc, you find that you have forgotten
to include QExample.hh. This would explain the other error messages as well:
every time gmake sees "QExample", it is confused because it does not know what
QExample is. Furthermore, it also does not recognize _numTrkHisto, because
this object was defined in QExample.hh as a private member object.
You put back the #include statement, and send your "gmake all" job again,
ana42> rm all.log
ana42> bsub -q bldrecoq -o all.log gmake all
This time, everything worked fine:
ana42> ls -l lib/$BFARCH
total 2408
-rw-r--r-- 1 penguin br 2462826 May 15 04:43 libBetaMiniUser.a
drwxr-xr-x 5 penguin br 2048 May 15 04:36 templates
ana42> ls -l bin/$BFARCH
total 48073
-rwxr-xr-x 1 penguin br 49225543 May 15 04:44 BetaMiniApp
-rw-r--r-- 1 penguin br 72 May 15 04:44 Index
There is a brand new BetaMiniApp! And you can tell from the
time that BetaMiniUser's lib file has also been recompiled.
Normally, gmake is pretty good at figuring out what needs to
be recompiled and what doesn't. But to be safe, you can issue
a "gmake clean" to clean out all the old lib and bin files
before you recompile.
The example above of course shows you only one example of the
many, many possible error messages that can appear in your log file.
Other typical compile errors include:
- Errors with incorrect use of gmake. For example, issuing the gmake
command from a package instead of your test release.
- Forgot
srtpath. (If you're not sure, it never
hurts to issue this command again.)
- Forgot to make your .bbobjy file for gmake all.
- Forgot to #include a header file (as in the above example).
- Declarations of variables: multiple and lack of.
- Loops within loops within loops. It is easy to forget to close
a loop (missing closed brace "}"). "Smart" editors like emacs can help you to avoid
or track down these errors.
- Simple typos. If the compiler doesn't recognize something you are sure you
defined, look closely: did you spell it right?
- Treating an object like a pointer, or a pointer like an object.
- Cascading errors: Often, later errors are caused by earlier ones.
Don't try to 'correct' everything before recompiling.
- Not enough disk space for your executable. Executables can be big.
Check to make sure you have not exceeded your quota.
- Try again, but issue "gmake clean" and "gmake installdirs" first, to be
safe. (Normally should not be necessary, but if things don't work, this
sometimes fixes them.)
Compile/link error messages can be confusing. It is not always obvious
what the problem is. In the end, all you can do is try your best to decipher
the log file. If you can't figure it out, then your best bet is to ask
someone who has had more practice with gmake errors, or, if you are all
alone, submit your problem to the prelimbugs Hypernews forum. When you submit your
question, be sure to indicate:
- What release you are using.
- What tags you have checked out.
- What parts of the code you have changed.
- The error message - or the location of your log file.
For example, if you could not figure out what was wrong in the
above example, you would first explain your problem, and then
provide the following details:
- Release: analysis-42
- Tags: BetaMiniUser V00-04-00, workdir V00-04-20
(You could also mention that workdir and BetaMiniUser tags are analysis-42's
default tags.)
- Code changed: In BetaMiniUser, added QExample module, and
edited AppUserBuild.cc to include it.
- The error message: See ~username/ana42/all.log
You will get an answer much faster if you provide complete information.
If your compile-and-link problem occurs in one of the Workbook examples,
then you can email me, the
Workbook editor, and I'll see what I can do. Perhaps you have
discovered a bug in the Workbook! (But please try to solve it yourself,
first.)
Now you have learned most of what you need to know to
use gmake to compile and link. But you may find it helpful
to learn a bit more about how GNUmakefiles work.
One of the virtues of GNUmakefiles is their ability to
handle the many dependencies among the many, many
lines of C++ code that must be put together to make an
executable. This (optional) section revisits the GNUmakefile,
with a focus on how these dependencies are managed.
An executable is built from the code defined in multiple files. When
changes are made in one or more of these files the code in the
modified files and all dependent files will need to be
re-compiled and re-linked. Compile and link commands often involve
many options, such as directory search paths for included files, names
for compiled files, warning messages, debugging information and so
forth. Compile and link commands quickly become quite lengthy, and in
large software projects the dependencies amongst files is usually
rather involved. gmake's job is to manage this complicated process.
GNUmakefiles revisited
GNUmakefiles define a set of instructions for how
to compile and assemble code segments into the executable. The
file also defines many variables such as search paths, which compiler
to use, etc. The instructions specify the dependencies of the code
segments so that the gmake utility can recognize only
those components that need to be reprocessed. A well-written makefile
can reduce unnecessary compilation and save much time.
The gmake facility looks in the current directory for a file named
GNUmakefile, makefile, or Makefile (in that order). The first one of
these files found is used to define the dependency/build rules for the given
target.
Targets, Dependencies, and Build Rules
The general structure of an instruction in the GNUmakefile consists of
a target, a dependency, and a rule.
Associated with each target is a set of dependent files. When any of the
dependent files have been modified or if the target file
does not exist, the target will be rebuilt (or built)
in accordance with the rule. For example, a line in a
GNUmakefile might be:
foo.o : foo.cc foo.hh bar.hh
g++ -c foo.cc
The target is foo.o, the dependent files are foo.cc, foo.hh, and bar.hh,
and the rule is "g++ -c foo.cc". If any of foo.cc, foo.hh or bar.hh have
been modified, or if the foo.o does not yet exist, issuing
gmake foo.o will result in the recompilation of foo.cc in
accordance with the rule.
The default target is the first target
listed in the GNUmakefile. It is the rule that will be executed when gmake
is invoked without an argument.
Sometimes targets are used to perform tasks not related to compiling
and linking code. If you define a target for which there is no file of
the same name, and the rule commands do not create the target file,
then a file of the target name will not exist in the
directory. Because the target does not have a corresponding file it is
always considered out of date; the rules will be executed every time
the target is made. These targets are referred to as phony
targets. Phony targets do not usually have dependent files.
An example of a phony target is workdir's setup target.
The command gmake workdir.setup does not compile or link code.
Instead, it creates some links in workdir: PARENT, RELEASE, bin, and shlib.
gmake relies on time stamps to determine what needs to be
redone. The Makefile (or GNUmakefile) for a given package knows about
the dependences between all the files. If a given file is being
compiled with a gmake command, any files that it depends upon and that
are newer than it will also be compiled. Therefore, if file
X is newer than file Y, but Y
depends on X, recompiling Y will
force a recompilation of X, and any other files that
might have been edited since Y was last touched.
As such, to force a file to be recompiled, you may need to
manually "touch" one of the relevant files. An example of
such a command is:
touch BetaMiniUser/AppUserBuildBase.cc
This changes the time stamp on AppUserBuild.cc, so that gmake
will identify it as a new file, and recompile it. In this
case, this will cause almost the entire package to recompile as
almost all the other files in the package depend on AppUserBuildBase.cc.
In BaBar software each package includes a link_PACKAGE.mk file in
addition to the GNUmakefile. The link file is used by the
GNUmakefile. The use of this file is mainly for overriding previous
definitions of linking rules defined by the test release's
GNUmakefile. This overriding mostly serves to append the new package
directory to the linking list. More information on this can be found
in the HOWTO package in
HOWTO-dependency.
General Related Documents:
Page maintained by Adam Edwards
Last modified: January 2008
| 