SCons for Fermi LAT: An Overview

SConscript (SCons) was chosen for its cross-platform
capabilities and replaces CMT as the Science Analysis Systems build tool. SCons is Open Source software written in Python, and our Fermi-specific extensions are also written in Python. (For a more detailed introduction, go to http://scons.org/.)

Details of the build system used to produce releases will, for the most part, not be of interest to end-users, but the file organization within an SCons-built release and tools associated with SCons releases, such as the Installer, are. This Overview will concentrate on those aspects of the Fermi LAT software environment as it describes how to acquire a release and run programs belonging to it. Experienced CMT users may wish to consult the CMT – SCons Comparison chart.

Accessing a Release

From a Central SLAC Linux machine

SCons builds of ScienceTools or other containers may be found on the u35 disk:

/nfs/farm/g/glast/u35/ReleaseManagerBuild

They are further organized by platform-compiler (e.g. redhat4-i686-32bit-gcc34, redhat5-x86_64-64bit-gcc41, etc.), Debug or Optimized, container name, and finally the top-level directory of a single build. For tagged releases the top-level directory is of the form xx-yy-zz (e.g., 09-18-06). The path to a typical build looks like this:

/nfs/farm/g/glast/u35/ReleaseManagerBuild/redhat4-i686-32bit-gcc34/Optimized/ScienceTools/09-18-06

The general form for the full absolute path for container container-name and name build-name is:

/nfs/farm/g/glast/u35/ReleaseManagerBuild/platform-compiler/dbg-opt/container-name/build-name

Such a release may be used in-place (details to be covered later) or may be copied to your private disk space. Alternatively, you may use one of the Installers to fetch and install a release in your private disk space. See more on Installers in the next section.

From your desktop or laptop

Use one of the Installers for SCons builds, depending on your taste in interfaces: GUI or command-line.

There is a slight difference in the way the GUI and the Command Line installers operate. The GUI installer will create a subdirectory of the directory you specify named after the release (e.g., ScienceTools-3153 for LATEST build 3153) and install there. RMViewer (a tool primarily of interest to developers which also includes an install function) and the command line installer install directly in the directory specified.

With any of these methods you will have a choice of three types of distributions: source, developer or end-user. If you expect to use the release as is, choose end-user. developer will also do, but it includes many more files which you won't need.

Anatomy of a Release Manager Build

An end-user build obtained by using one of the installers has subdirectories lib/build-variant and exe/build-variant for binaries; bin/build-variant for wrapper and set-up scripts. It has several other subdirectories, such as data, xml and python, for platform-independent files. Some of the latter themselves have per-package subdirectories; others don't.

Note: Here build-variant is similar in form to platform-compiler above but unfortunately not identical; it has an additional component to distinguish between debug and optimized builds. Typical examples of build-variant names are redhat4-i686-32bit-Optimized and Windows-i386-32bit-vc90-Debug.

Developer builds have everything included in end-user builds plus per-package subdirectories for sources just as they appear in CVS.

A build in the SLAC SCons build area on u35 has everything a developer build has and in addition, intermediate build products under the per-package subdirectories.

External Libraries: Directory Structure

SCons builds expect the following organization of external libraries:

• A top-level directory, commonly known as GLAST_EXT. Typically the environment variable GLAST_EXT is defined to point to this directory.

• A subdirectory of GLAST_EXT for each supported external, named after the external (e.g. CLHEP, cfitsio, python, ...

• Within each external subdirectory, subdirectories for each available version.

• Within each version subdirectory, a subdirectory named after the compiler.

• Within the compiler subdirectory, whatever collection of directories is required for that particular external. Typically they have names like include, lib, bin, doc.

External Libraries at SLAC

To access external libraries from a SLAC Linux machine, define GLAST_EXT as:

/afs/slac/g/glast/ground/GLAST_EXT/platform-compiler

e.g.,

/afs/slac/g/glast/ground/GLAST_EXT/redhat4-i686-32bit-gcc34

As of November, 2010, for ScienceTools platform-compiler may be any of:

redhat4-i686-32bit-gcc34
redhat4-x86_64-64bit-gcc34
redhat5-i686-32bit-gcc41
redhat5-x86_64-64bit-gcc41

GlastRelease builds exist only for redhat4 32-bit Linux.

Windows users with access to the SLAC V disk may set GLAST_EXT to:

V:\Glast_Software\Toaster\GLAST_EXT\Windows-i386-32bit-v71
V:\Glast_Software\Toaster\GLAST_EXT\Windows-i386-32bit-v90

for VS 2003 or VS 2008, respectively. As of November, 2010, the Windows Release Manager builds only ScienceTools (both compilers).

Running Programs from an SCons Build

There are two ways to run executables from an SCons build: by invoking wrapper scripts (in which case a new process is spawned for each program run) or by setting up the environment for a process by means of a special script (_setup.sh or _setup.csh for unix; _setup.bat for Windows) generated as part of the RM build, then invoking executables directly from that process.

Using Wrapper Scripts

Wrapper scripts are in the bin/build-variant subdirectory of the build. A typical session using this approach involves the following steps:

• Set GLAST_EXT environment variable
• (optional) Do any other set-up required for your session; e.g., establish path to ftools
• Run one or more wrapper scripts.
  
  Note: wrapper scripts change (add onto) standard path variables so should be run in a child process. From the unix (Linux or Mac) command line just invoke the file directly. On Windows use the command exec.

unix	myST/bin/redhat4-i686-32bit-gcc34/test_astro
Windows	exec myST\bin\XP-i386-32bit-vc71\test_astro.bat

Using the _setup script

The idea here is to establish the proper environment in a process, then run one or more programs from that process by invoking the executables directly. Executables are located in the exe/build-variant subdirectory of the build. The basic steps for this method are:

• set environment variable GLAST_EXT as above.
• set environment variable INST_DIR to the top-level directory of the build.
• run the setup file.

In this mode the setup file is typically run in the same process in which you plan to run the executable(s). On unix (Linux or Mac) use the source command. Depending on your shell the command will be one of

source $INST_DIR/bin/build-variant/_setup.sh
source $INST_DIR/bin/build-variant/_setup.csh

On Windows simply run the setup file:

%INST_DIR%\bin\build-variant\_setup.bat

Here build-variant is used in the same sense as above; i.e., has a value like redhat4-i686-32bit-gcc34-Optimized on Linux or Windows-i386-32bit-vc71-Debug on Windows.

Optionally do other environment setup needed for your analysis before or after running the SCons-generated setup file.

• Run one or more executables:

unix	myST/exe/redhat4-i686-32bit-gcc34/test_astro
Windows	myST\exe\XP-i386-32bit-vc71\test_astro.exe