IEPM
SLAC site

Correlation Table for Active and Passive Monitoring

Connie Logg, Les Cottrell and I-Heng Mei
IEPM-BW results | Bulk throughput measurements | Bulk throughput simulation | Windows vs. streams | Effect of load on RTT and loss | Bulk file transfer measurements | QBSS measurements | SC2001 Bandwidth Challenge

Description

The active measurements are obtained by adding up the the Bytes and times reported by iperf or whatever other application tool we are using (bbcpmem (from dev/zero to /dev/null), bbcpdisk (disk to disk), bbftp (disk to disk)) to give the active throughput for all the streams. We make a measurement each 90 minutes, we go back for 28 days.

At the same time we are capturing the passive NetFlow records and we identify which flows go with which active application call. We add up the throughputs for all the streams for a given application call. We calculate the passive throughput using three methods:

Where s is the stream number.

R is the correlation coefficient. The Correlation Coefficient R is defined as:
R = COV(x,y) / (Std(x) * Std(y)), where:
COV(x,y) = (1/(n-1)) * SUM((xi-Avg(x))*(yi-Avg(y))), is the covariance,
and the SUM is over the n paired Samples (i=1..n) , also
Avg(x) = (1/n)) * SUM(xi)) and Avg(y) = (1/n) * SUM(yi)) are the averages, and
Std(x) = sqrt((1/(n-1)) * SUM((xi-xm)2)) and Std(y) = sqrt((1/(n-1)) * SUM((yi-ym)2)). are the standard deviations.
The x coordinate is for the Active measurements, and the y coordinate is for the Passive measurements.
The Error = Avg(x) - Avg(y) / Avg(x);

You can click on the Node to view the time series and scatter plots, and on the Type to get a space separated table of the data for that host. We are currently investigating in more detail the differences in Passive vs. Active Measurements by using tcpdump.
Anomalies

Download a comma-separated-values (CSV) table for further analysis with tools such as Excel

Node (click for plots) Active Tool (click for extracted data)Passive RSamples Avg(x)Avg(y) Std(x)Std(y) Error min(x)max(x)min(y)max(y) Cov
Node (click for plots)Tool (click for extracted data) PassiveRCount Avg(x)Avg(y) Std(x)Std(y) Error min(x)max(x)min(y)max(y) Cov

Anomalies


Page created on Wed Dec 11 23:15:27 PST 2002 by /afs/slac/package/netmon/bandwidth-tests/new/src/passive/make-active-passive-plots-with-R for iepm