Computing at SLAC
Search SLAC      
SLAC Home | Computing Home | Networking Group| Help

About

Installation

Documentation

Database API's

DB Retrieval Scripts
DB Validation Scripts
Data Fetching Scripts
Tools Page

Adding a New Probe

Moving IEPM-BW

Troubleshooting Hints

Results

Original PLM

Database API's

Database Retrieval Scripts

These scripts use used to fetch information out of the mysql data base.

monhost-getcfg

$iepmSrcDir/monhost-getcfg [-h monitoring host alias] reads the monitoring host information out of the MONHOST and NODES mysql tables. This is a particularly useful tool for validating that the data base and configuration files are set up correctly. If called from the unix command line, it prints its results on the terminal. The '-h' is optional and defaults to aliasName. For example: $iepmSrcDir/monhost-getcfg will return a string of the form:
("SRCDIR='/afs/slac/package/netmon/bandwidth-tests/v3src'", "REPORTSDIR='/var/www/html'", "PRIVATEDIR='/nfs/slac/g/net/iepm-bw/bandwidth-tests/litetest/slaconly'", "WEBPATH='http://nettest2.slac.stanford.edu'", "DAYSTOANALYZE='30'", "DATATOANALYZE='ping:avg=w i lt 1;ping:min=w i lt 0;abwe:avdbcap=w p lt 5;abwe:avabw=w p lt 12;abwe:avxtr=w p lt 8'", "MASTERTIME2RUN='10'", "GNUPLOTPATH='/usr/local/bin/gnuplot'", "PLOTICUSPATH='/afs/slac/package/netmon/bandwidth-tests/ploticus/pl'", "PINGPATH='/bin/ping'", "TRACEPATH='/usr/sbin/traceroute'", "GREPPATH='/bin/grep'", "CGIPATH='http://nettest2.slac.stanford.edu/cgi-bin'", "CGIDIR='/var/www/cgi-bin'","SITENAME='SLAC IEPMLITE'")

It is primarily called from perl scripts by:
@ans= `$iepmSrcDir/monhost-getcfg `;
chop @ans;
#now define the parms
eval("\@parms = $ans[0];");
grep (eval("\$$_;"),@parms);
and the following perl variables are defined.

  • $SRCDIR - the code source directory (usually the same as iepmSrcDir)
  • $REPORTSDIR - which is the directory path to the top of the directory where the reports and webpages are stored.
  • $PRIVATEDIR - which is to hide information that you want ot keep from the user
  • $WEBPATH - which is the http string for getting to the top of the report and results directory
  • $DAYSTOANALYZE - how many days worth of data to analyze with the report scripts
  • $DATATOANALYZE - which is a character string that specifies what data values are to be analyzed and how they are supposed to be plotted. For example: $DATATOANALYZE = 'ping:avg=w i lt 1;ping:min=w i lt 0;abwe:avdbcap=w p lt 5;abwe:avabw=w p lt 12;abwe:avxtr=w p lt 8' indicates to the system that it is to analyze:
    • ping "avg" data and plot it with a gnuplot spec of "w i lt 1"
    • ping "min" data and plot it with a gnuplot spec of "w i lt 0"
    • abwe "avdbcap" data and plot it with a gnuplot spec of "w p lt 5"
    • abwe "avdbw" data and plot it with a gnuplot spec of "w p lt 12"
    • abwe "avxtr" data and plot it with a gnuplot spec of "w p lt 8"
    Note that the value to plot is actually the column heading in the data file. The data fetch scripts are described shortly.
  • MASTERTIME2RUN - is the longest in seconds a probe is to be allowed to try to execute
  • GNUPLOTPATH - is the path to gnuplot
  • PLOTICUSPATH - is the path to ploticus plot command
  • PINGPATH - is the path to the ping command
  • TRACEPATH - is the path to the traceroute command
  • GREPPATH - is the path to the grep command
  • CGIPATH - is the http path to the cgi-bin directory
  • CGIDIR - is the directory path to the cgi-bin directory
  • SITENAME - is the site name from the MONHOST table

    • get-nodelist

    $iepmSrcDir/get-nodelist returns a string which when evaluated, defines an associative array %TESTLIST. For example:
    @ans = $iepmSrcDir/get-nodelist
    eval("$ans[0]";)
    where %TESTLIST=('node2.rhic.bnl.gov','abwe,ping,traceroute','node1.clrc.ac.uk','abwe,ping,traceroute')
    Upon evaluation, $TESTLIST{'node2.rhic.bnl.gov'} = "abwe,ping,traceroute" is the tests performed for the particular node. The list of active nodes can be extracted by @nodes = keys(%TESTLIST). This script is used heavily in the code for going through all the nodes.

    create-abw-list

    $iepmSrcDir/create-abw-list creates the IEPMLITE configuration file for ABWE monitoring processes. It is saved as $iepmSrcDir/config/IEPMLITE.conf. There are 3 columns which are legacy, and can be ignored although they must be provided for parsing purposes. They are "db", "poll", and "shortname". It must be manually stored into the ABWE source directory at: $iepmSrcDir/abw/Conf/IEPMLITE.conf. This is the table the abwed daemon will use to perform abwe measurements. When new nodes are added to this process, this manual store must be repeated, and the daemon must be restarted. The ABWE daemon is started by $iepmSrcDir/abwed IEPMLITE. The abwe_rfl is started by executing $iepmSrcDir/keep-servers-alive. However, the keepalive system generally starts both of these actions and keeps it running, so the issuance of this manual command should not be needed.

    fetch-monhosts

    return a list of the monitoring hosts which have been defined in the database.

    fetch-table-descriptions -t tablename

    returns a list of the field names in a table. This can be useful for populating CGI web pages with the list of selectable fields. Tablenames currently include: NODES, MONHOST, PINGDATA, ABWEDATA, ROUTEDATA, TOOLSPECS, SCHEDULE, ROUTENO, BWDATA, COMMENTS

    fetch-table-data-into-csv -t tablename

    returns a CSV list of all current database entries for the specified tablename. This is used in conjunction with an Update tablename script to easily update missing database data. In particular, fetch-table-data-into-csv -t NODES is run before executing $iepmSrcDir/mysql/update-NODES-info-from-csv to quickly update the NODES table.

    fetch-target-hosts -t datatable -h monhost -s starttime -e endtime

    "fetch-target-hosts" returns the aliasnames of the hosts monitored by monhost which have data in the data table (PINGDATA, ABWEDATA, ROUTEDATA) for the period of time. It is returned as a list (node1.binp.nsk.su=45,node1.cacr.caltech.edu=45,node1.cesnet.cz=42, node1.desy.de=38,node1.dl.ac.uk=41,node1.ece.rice.edu=43,node1.fnal.gov=39, node1.fzk.de=41,node1.indiana.edu=46,node1.internet2.edu=47,node1.kek.jp=42) where the number after the "=" is the number of points found. "-h" is optional, and if not provided the "aliasName" parameter in /etc/iepm.cnf is assumed.

Database Validation Scripts

These scripts create html pages of the information they fetch from the mysql database. They can be useful for validating and verifying what is loaded in the data base. Note that many need some more work on them. Please note that the links provided here are to a version of the package which is in heavy development, and they may fail. Sorry for any inconvenience. A temporary web page linking to some of the output of these scripts is available.

report-logging

"report-logging" generates a of all the test and logging activity.

report-nodes

generates a report of all the nodes listed as target nodes on the monitoring host.

report-monhost [-h monitoring-host-alias]

generates a report of all the nodes listed as monitoring nodes for the specified monitoring host.

report-toolspecs

generates a report of all the probe specifications for the monitoring host.

Database Fetching Scripts

These all have the same arguments, and defaults are provided. "-t" is the testtype, and the default is "background"
"-o" is the directory where the output is written, and /tmp is thye default
"-n" is the aliasname of the desired node, but all are fetched if it is not provided.
"-s" is the starttime, either in epoch seconds or standard date format
"-e" is the endtime, either in epoch seconds or standard date format
"-h" is the monitoring host alias name and defaults to the host the code is executed on "-t" is the testype. This is "background" for the normal data acquisition. If "-e" and "-s" are not provided, the data for the last 24 hours is returned. If the scripts are being run on the monitoring host, only -n, -s, and -e are required.

fetch-ping-data [-n aliasname] [-o output_directory] [-s starttime] [-e endtime] [-t testtype] [-h monitoring_host_alias]

The only testtype currently is "background". If "-o" is not provided, /tmp is assumed. "fetch-ping-data" returns the name of the file where the output is stored.

fetch-abwe-data [-n aliasname] [-o output_directory] [-s starttime] [-e endtime] [-t testtype] -d ['to' or 'from']

The only testtype currently is "background". If "-o" is not provided, /tmp is assumed. "fetch-abwe-data" returns the name of the file where the output is stored. There are 3 types of ABWE data stored in the data base. There is the overall throughput measurement (fetch with NO -d parameter), there is the raw measurement from the monitoring host to the target hosts (fetch with '-d to') and the raw measurement from the target host to the monitoring host (fetch with '-d from'). The 'to' data has the string 'abweto' in the name, the 'from' data has the name 'abwefrom' in the name, and the averaged data has the string 'abwe' in the name.

fetch-trace-data [-n aliasname] [-o output_directory] [-s starttime] [-e endtime] [-t testtype] [-h monitoring_host_alias] -a [0 | 1]

The only testtype currently is "background". If "-o" is not provided, /tmp is assumed. "fetch-trace-data" returns the name of the file where the output is stored. The raw traceroute data is also fetched currently, and is stored also in the "-o" directory. Note the -a parameter. The tracelogs are anonomized is -a is not provided or is 1. For the raw tracelogs -a 0 must be specified. The text of the traceroute output in in the file ending with ".tracelog".

fetch-routeno-data -n aliasname [-o output_directory] [-h monitoring host alias]

- Creates a blank separated file of the route numbers and paths for the specified host and target nodes. The file name is returned. Note that -o can be used to specify the directory where the file should be stored.

fetch-routeno-data-html -n aliasname [-o output_directory] [-h monitoring host alias]

- Creates an html page of the route numbers and paths for the specified host and target, The file name is returned. Note that -o can be used to specify the directory where the file should be stored. nodes.
  Send Us Feedback  
  SLAC