Porting the SLAC IEPM-BW Version 2 Code
Connie Logg, Les Cottrell, and Jerrod Williams
Please send comments on this writeup to Jerrod
2/28/03

Definitions ($abc is shorthand which may be used in the document):

Monitoring Host - ($mh) The node that the tests are running from
Target Node - ($tn) A node that the tests are running to
Probe - ($prb) A script which runs a measurement from a $mh to a $tn and returns the results.
Monitoring Node Configuration File - ($mncf) - The configuration file which specifies the directories for the source code and data & analysis, the various paths for the binaries required by the code, and other $mn specific definitions.
Alias File - ($ALIASFILE) - The file which contains the specifications for each of the destination nodes in the SLAC format.

Requirements:

Perl and perl modules:
Date::Calc qw(Add_Delta_Days Delta_Days) and includes the file Date/Calc.pm.
Date::Manip qw(ParseDate UnixDate DateCalc) and includes the file Date/Manip.pm.
Time::localtime
Sys::Hostname
getopts.pl
getopt.pl
IO::Handle
Proc::ProcessTable
SOAP::Lite

Step 1: Define the Monitoring Node Configuration File

A Sample Monitoring Host Configuration File is available. Note that ALL parameters not under optional, must be defined. This file must be created and saved with the naming convention [monitoring host name].cfg, before porting the IEPM-BWtests code.

Step 2: Porting the Code to a Monitoring Host

"make-kit" has been developed to facilitate the loading of necessary code on the new monitoring node. This script is run from a slac node upon request and takes only a few minutes to execute. "make-kit" reads the monitoring node's configuration file and creates a tar file containing all source files needed for loading onto the monitoring host. Also created is an HTML file, unique to each new monitoring host, that is accessed to explain how to obtain and de-tar the file for use. For your convience, the HTML file includes links to sites that provide the sources for BBCP, IPERF, and BBFTP. It will be up to the user to obtain those packages himself and create them on his host. The URL for your particular site follows the convention http://www.slac.stanford.edu/comp/net/bw-html/distribution/[monitoring host name].html. If this URL does not work after you have received confirmation that it should be up, please contact Jerrod Williams.

Notes:

Step 3: Summary of the IEPM-BW Code

The Program Logic Manual gives a brief descriptions of the scripts needed to run the tests, analyze code performance, and perform analysis of the results.
CONVENTIONS: files ending in ".pl" are code fragments and samples of how to do certain manipulations such as dates.

Please refer to the PLM to understand what each script does and how they relate to one another.

Step 4: Setting up the Destination Nodes and defining the $ALIASFILE parameters

Included in the download from step 2 is you own starter ALIASFILE located in $SRCDIR/toolkit. We've provided you with an intitial node at slac to perform tests but it will be up to you, the Admin at the new monitoring site, to acquire and setup your own pool of sites to monitor.

Please reference our documentation for setting up the remote nodes. If you have any questions on setting up remote nodes, please contact Jerrod Williams or Les Cottrell.

The $ALIASFILE specification contains several parameters, many of which are SLAC specific. A Sample Destination Node Configuration File is available.

Step 5: Running the Tests

run-bw-tests has the following calling sequence:

run-bw-tests -n [nodename] -c $mncf -s [SRCDIR] -t "[list of tests]" [-w1] -D [0|1|2]

-n is the specific destination node to test. If not provided, all nodes are tested.
-t is the list of tests
-w n This is for web100 invocation, which is currently under development. The number is provided for future expansion to faciliate specifying various levels of web100 variable dumping.
-c fully qualified filename - is the fully qualified MHCF. If the environment variable 'BW_CFG' is defined to be this name, it is not necessary to provide it via '-c'.
-s 'fully qualified directory specification' - is the source code directory. If the environment variable 'BW_SRC' is defined to be this name, it is not necessary to provide it via '-s'.
-D n - This is the debug flag and is particularly important for run-bw-tests when running it from the command line. '-D 0' indicates that the measurements are to be run, but that no data is to be written to the data files, only the terminal. '-D 1' indicates that the measurements are to be run and the data is to be written to the data files, and not the terminal. '-D 2' indicates that the measurements are to be run and the data is to be written to the data files, and the terminal. If called from a crontab, '-D 2' can be used in production, and the output will be printed to STDOUT.

[list of tests] is currently iperf, bbcpmem, bbcpdisk, bbftp, ping, and trace

For example: to run through all your nodes from the command line for all tests:

run-bw-tests -c $mncf -s [SRCDIR] -t "iperf,bbcpdisk,bbcpmem,bbftp"

To run just iperf for one node:

run-bw-tests -n $dn -c $mncf -s [SRCDIR] -t "iperf"

To run ping and traceroute to just two nodes:

run-bw-tests -n "$dn1,$dn2" -c $mncf -s [SRCDIR] -t "ping,trace"

If the $mncf file is specified by the environment variable "BW_CFG", -c does not need to be provided.
For running from the command line, -n and -D 0 must always be provided. This causes all results to be written to the terminal and not the raw data log file.

Individual tests can also be run by calling do_[test]. [test] contains either ping, trace, iperf, bbcpdisk, bbcpmem, or bbftp. do_[test] is called by run-bw-tests but, we find it's easier to debug specific problems by running the specfic tests this way.

do_[test] has the following calling sequence:
do_test -n [nodename] **NOTE: Environment variables BW_CFG and BW_SRC must be defined.
Options are defined above.

Notes:

Step 6: Converting the Old Plotdata

Step 7: Saving Your Old Rawdata

Step 8: Setting up the Production Testing

Please provide Feedback to the Designing Author: Jerrod Williams