SLAC	PEP-II
BABAR
SLAC<->RAL

HEPIC	E,S & H	Databases	PDG	HEP preprints
Organization	Detector	Computing	Physics	Documentation
Personnel	Glossary	Sitemap	Search	Hypernews

Comp. Search

Who's who?

Meetings

FAQ Homepage

Introduction

Currently, site-specific build customizations for BABAR code releases may be located in $BFOVERRIDE/SoftRelTools (or in the SoftRelTools area of a personal test release). These locations contain, if needed, makefile fragments (e.g. arch_spec.mk) which are included into the GNUmakefile to selectively override variables ("macros") and/or targets in the public release. This scheme has a number of problems; it is not release-dependent, it is hard to track down problems at remote sites, and difficult to maintain.

A new scheme beginning with release 4.0.5 introduces a new location for saving site-specific customizations, refines the standard locations where site-specific information is stored to provide a release-dependent and more flexible scheme, and changes the order in which these locations are searched. A key element is a new package called SiteConfig which is managed by CVS. Another element is the introduction of the $BFOVERRIDE/<release> directory.

The SiteConfig package contains a subdirectory for each and every site that requires customizations (e.g. SiteConfig/slac contains those for SLAC). A new environmental variable, $BFSITE, is defined to support this mechanism. It is expected that all site-specific customizations will ultimately be maintained in the SiteConfig package.

However, because it may not be possible to anticipate needed changes for the SiteConfig files in advance of a new release, the $BFOVERRIDE mechanism has been retained with one change: customizations are kept in a release-dependent subdirectory (i.e. $BFOVERRIDE/<relelase>, e.g. $BFOVERRIDE/4.0.7). It is expected that the $BFOVERRIDE area be used only to contain those customizations needed for retroactive support of a release, i.e. those changes that were not committed to the SiteConfig package in time for a given release. Changes stored in $BFOVERRIDE for the current BABAR release should then be saved in the SiteConfig package by a responsible site contact for the benefit of future releases.

Note that old releases (4.0.4 and before) expecting *.mk files to reside in the top level of the $BFOVERRIDES directory will continue to function, but that releases starting with 4.0.5 will require site customizations to reside in $BFOVERRIDES/<release>. (Sites that wish to continue using their current BFOVERRIDE directory need only to make links like "ln -s . 4.0.5" in $BFOVERRIDE - see details below.)

The SiteConfig package

This package contains site-specific configuration files and is structured as a collection of subdirectories, one each for each BABAR site, e.g. SiteConf/slac and SiteConf/ral. It is expected that all users define the $BFSITE environmental variable to the appropriate value for a given site. (This can be automated, for example, using a site-specific login script.)

Configuration files can be of two forms: base or replacement files, e.g. arch_spec.mk, or an "add-on file", e.g. arch_spec.mk+. Replacement files, if present, will completely replace identically named files in the search path (see below). Add-on files, if present, will simply be included into the corresponding .mk file as yet another level of customization as discussed later. The SiteConfig mechanism is the primary and preferred method to accomplish site-specific customizations. All other methods should be used only for testing or when absolutely needed for compatibility with old releases.

Terry Hung is the package coordinator for SiteConfig and will be responsible for version tagging. Tagging a new version will be done as necessary immediately prior to the build of a new release.

Figure 1. Production release directory. The SiteConfig area is where all site-specific customizations should be stored. As sites discover a need for such an area, they should create a new subdirectory for their site, then create the necessary .mk+ files and commit the changes back to the CVS repository.

The $BFOVERRIDE/<release> directory

A BABAR software release is generally considered frozen once it has been successfully built on all reference machines. However, remote sites may need to make further adjustments to their configurations for a given release after it is frozen. Even though these configuration changes should be committed to CVS, there will be a mismatch between the tagged version in the public release and the version needed to make the release work properly. Thus, a release-dependent site-wide configuration area in $BFOVERRIDE/<release> is available for this purpose. Files in this area should be retained indefinitely to ensure continued compatibility with affected releases. Note that if all changes are committed to SiteConfig in time, then $BFOVERRIDE/<release> is not needed and should be empty.

$BFOVERRIDE, therefore, serves as a permanent location for stablized fixes for a given release which were committed too late to be part of that release.

It is expected that all sites will have a single, officially-maintained $BFOVERRIDE area. However, individual users are free to use the $BFOVERRIDE mechanism for their own purposes.

Figure 2. This is an example override area for an entire site. Generally, this area is needed if and only if changes are required to make a particular release function. The appropriate location for site-specific overrides is in $BFDIST/SiteConfig in the production release area. Note that if files must be placed in $BFOVERRIDE, then they should be left indefinitely or until all work with that particular release is truely finished. (Note that the "(*.mk)" files are shown for backward compatibility with old releases prior to 4.0.5.)

Base (or replacement) files (`*.mk`)

Base files contain makefile fragments and are identified by the file extension ".mk". Base files are intended to represent the best guess of the release coordinator as to what is needed to build a particular release (e.g. which compiler, what linker options, etc.). The copy in the production release may be overridden by others as governed by the search path described below. Replacing the base file is the way the old BFOVERRIDE system worked. Starting with release 4.0.5, it is assumed that this will be largely unnecessary and that site customizations may be done exclusively with add-on files. However, the base file replacement mechanism will still function.

Add-on files (`*.mk+`)

An add-on file is a makefile fragment identified by the file extension .mk+. Add-on files are appended to the end of the corresponding base .mk file with the -include <pkg>/<file>+ directive, e.g. -include SoftRelTools/arch_spec.mk+. A typical site is likely to differ from that of the reference platform upon which releases are built. Examples may include different compiler names (e.g. xlC -> xlCnew), or the need to include a different set of compiler or linker options.

Add-on files provide several types of customization:

redefine a variable ('='), e.g. CC = xlC313
add parameter(s) to a variable ('override +='), e.g. override CCFLAGS+=-g
remove parameter(s) from a variable ('<var>:=$(<var>:<parm>=)'), e.g. CCFLAGS := $(CCFLAGS:-g=)
add a completely new variable and/or target

Note: These operations should provide sufficient customization for gmake variables and targets. If not, one may completely replace the original file with a replacement .mk file. This should be considered an unusual measure.

Search Path

The following search path is used both for the base (.mk) and add-on (.mk+) files. A search is first made for the base .mk file; the first one found is used. Then a separate search is made for the add-on .mk+ file; the first one found, if any, is then included into the selected .mk file. The search path is as follows (note that within makefiles, environment variables are dereferenced with a special syntax, e.g. "$(BFSITE)"):

$(PWD)/SiteConfig/$(BFSITE)

This is s local copy of the SiteConfig package created by addpkg SiteConfig for a personal test release. If you wish to commit changes back, there are three scenerios:

(most likely) if the change is required for both current and future production releases, update both CVS and $BFOVERRIDE/<release>.
if the change is for this production release only, update the site's local $BFOVERRIDE/<release>.
if the change is only for future production releases, commit the changes back ito the CVS repository.

Figure 3. Sample personal test release directory. This is where .mk and .mk+ files are developed and tested. Generally, site contacts need only modify the SiteConfig/<their-site>/SoftRelTools/*.mk+ files. Once these files have been debugged, they should be committed back to the CVS repository. If the changes are needed for the current production release, then the modified files should also be copied into the site's $BFOVERRIDE/<release>/SoftRelTools directory. Typically, site-specific customizations need come only from the production release area (i.e. $BFDIST/...).

$BFOVERRIDE/<release>

This is the release-specific site configuration directory. It is needed only when configuration changes were made too late to get into the SiteConfig release. Files in this directory should be kept for all releases in use, but they should be also merged into CVS for future releases.

$(BFDIST)/releases/$(BFCURRENT)/SiteConfig/$(BFSITE)

The set of site configuration files in the public production release.

$(PWD)/include

This is a local directory that contains specified include files for a package or a symbolic link that points to the local copy of the package created by addpkg <pkg> (note that currently SoftRelTools is the only package to use this mechanism).

$(BFDIST)/releases/$(BFCURRENT)/include

The collection of original files in the public production release.

Warning: The $BFOVERRIDE directory will no longer be in the search path for releases starting with 4.0.5, but site-specific files in the top level of that directory should be kept to build earlier releases.

Site Preparation

Here are general instructions for making your site compatible with the new BFOVERRIDE scheme. This is only an example and assumes the site is "slac" and that the csh shell is being used. Modifications to this sequence will obviously be necessary for your site. If you have questions about this procedure, please contact Terry Hung.

Convert any site-wide $BFOVERRIDE files at your site into the add-on style, i.e. leave only the needed modifications then convert those to the add-on format, and store as .mk+ files in a temporary directory, <temp-dir>. SLAC, for example has the following situation. $BFOVERRIDE = $BFROOT/etc, and within $BFOVERRIDE/SoftRelTools one finds:
```
arch_spec.mk
arch_spec_f77.mk
arch_spec_RW.mk
```
For each .mk file, a corresponding .mk+ file should be created. Please see the files within the slac/SoftRelTools directory of the SiteConfig package for examples.
Modify the SiteConfig package.
checkout the SiteConfig package

invent a simple, short, unique, lowercase site identifier for your site ($BFSITE)

create a $BFSITE subdirectory under SiteConfig

create a SoftRelTools subdirectory under SiteConfig/$BFSITE

copy the new .mk+ files into the SiteConfig/$BFSITE/SoftRelTools directory

commit your changes
For example,
```
    $ cvs co SiteConfig
    $ cd SiteConfig
    $ mkdir slac
    $ cvs add slac
    $ cd slac
    $ mkdir SoftRelTools
    $ cvs add SoftRelTools
    $ cd SoftRelTools
    $ cp <temp-dir>/*.mk+ .
    $ cvs add *
    $ cd ../..
    $ cvs commit    
```
Define the $BFSITE environment variable in your site-specific user login script so that all users will have this defined in subsequent logins, e.g.
```
    $ setenv BFSITE slac
```
You should be finished - test the system

Alternative Site Preparation

This method is discouraged as sites will not benefit from the added functionality in the new scheme, and is mentioned only as a temporary expedient. The minimal actions a site must make to run any new release starting with 4.0.5 are as follows.

    $ cd $BFOVERRIDE
    $ ln -s 4.0.5 .

In other words, create a link with the name of the new release which points to the top of the $BFOVERRIDE directory. All files currently in the top level of $BFOVERRIDE (i.e. *.mk files) will be used as replacement files during subsequent makes for this release.

Again, it is worth stressing that this short-circuiting of the new scheme prevents you from getting a stable base makefile,and no release dependencies. Please do not consider this a long-term solution for your site.

Environment Variables

BFSITE: This variable is set to the name of the site, e.g. slac. It is used to obtain site-specific configuration files in $BFDIST/SiteConfig/$BFSITE.
BFOVERRIDE: Location of site-specific configurations discovered too late to be committed to the SiteConfig package. Subdirectories identify the release-specific configurations to be used. Individuals may also use BFOVERRIDE for personal purposes. Note: the top directory in $BFOVERRIDE will no longer be in the search path for releases starting with 4.0.5, but continues to provide configuration information for older releases.

Files

./SiteConfig/$BFSITE/<pkg>/<file>+: Add-on configuration files for a given package created by addpkg SiteConfig for this test release.
$BFOVERRIDE/<release>/<pkg>/<file>, <file>+:: Site-specific (or personal) add-on configuration files for a given package, including, potentially, both base (replacement) and add-on files.
$BFDIST/releases/<release>/SiteConfig/$BFSITE/<pkg>/<file>+: Site-specific add-on file(s) for a given package in this production release. Generally, only <pkg> = SoftRelTools is present, but the mechanism works for any package. In unusual circumstances, a base (replacement) file may be found here.
./<pkg>/<file>: The contents of a package created by addpkg in a test release. These files are included through symbolic links in ./include/<pkg>.
$BFDIST/releases/<release>/<pkg>/<file>: The contents of a package for this production release.

Credits

Many people contributed ideas and criticism to the evolving BFOVERRIDE concepts over the past several months. The detailed conceptual design and implementation for the new scheme are due to Terry Hung and Tom Pavel.

This page maintained by Tom Glanzman

Contents

Alternative Site Preparation