We will develop tools to reduce the raw data into forms suitable for analysis. Statistical analysis of the data will be made utilizing existing commercial and public domain tools to identify the critical long term trends, and short term exceptions to expected performance. The reports will be accessible via WWW forms that will allow users to dynamically customize what information more statically (for reports that cannot be generated sufficiently quickly to be interactive) by browsing the project's WWW pages. We will produce reports in tabular, graphical and other forms. We expect to be monitoring several hundred remote sites. Each will be monitored at a roughly 30 minute frequency. Reports will be generated showing various metrics including response time, packet loss, unreachability, at 30 minute intervals as well as averaged over longer periods such as daily and monthly. We will also create alerts identifying unreachable hosts, and major departures from expected daily response and packet loss performance, for network service people and interested collaborators. The alerts will be enunciatable via email, pager, or simply by highlighting some part of a WWW report with a hyperlink to more information.

The reports will be accessed via WWW and hence accessible worldwide (with appropriate restrictions) using any common platform. In general the reports will be vendor neutral. Care will be taken for selected reports to sanitize them to reduce liability issues. In some cases, access to the reports may be restricted to certain sites such as those involved in the relevant distributed collaboration. Updating of the reports will be automated and based on appropriate time schedules.

To make the tools and information available to a wide range of sites in an orderly and timely fashion, we will investigate setting up an hierarchy of sites in a distributed collaboration. The first level "remote sites" will simply need to provide the name of a suitable pingable host, which by design is pingable all the time. The second level "collection sites" will install the data monitoring and data collection tools to be provided by the proposed project. The collection sites will monitor the other sites in the distributed collaboration, save the data and make it available to the third level "analysis sites". The analysis sites will collect the data from the collection sites, analyze the data, create the reports and make them available via WWW. This hierarchy matches the hierarchy of sites seen in many distributed collaborations such as those in High Energy Physics experiments. Besides making the monitoring more manageable, it also helps limit the monitoring traffic by helping eliminate the need or desire for every host to monitor every other host. It also ensures that the data is measured with a common methodology and is archived in acommon format.

The work will include documenting and making the data collection tools available for distribution and installation at remote sites.

We recognize that an important but hard problem is distinguishing which aspects of end-to-end measurement reflect limitations in the application; the local-area networks at the endpoints; the Internet connectivity of the source and destination sites; and the wide-area network path between them. By its nature, end-to-end measurement is not conducive to separating out performance factors beyond the split between the application and the network path.

However, we picture addressing this issue as follows. First, we could use a packet filter to record the "wire times" associated with the measurement traffic. These timings then allow us to distinguish application effects from network path effects. Second, to distinguish between different network elements, we could add use of "traceroute", and its newly-announced successor, "pathchar", (both written by Van Jacobson of LBNL) to the measurement suite.

Finally, if IEPM measurements can interact with NIMI measurements (see below), then NIMI cloud measurements could provide additional localization of network path problems.

We view pursuing these approaches as beyond the limited scope of the initial IEPM effort, but important to consider as we develop the IEPM architecture, to ensure that it can ultimately accommodate better fault localization.

A major HEPNRC/FNAL contribution (the chief investigator at HEPNRC is David Martin) is currently in the area of defining the distributed architecture for monitoring, and preparing the monitoring and collection code for public distribution. The work with LBNL will be to better understand the Internet dynamics and the available tools. This will help to improve the details of the monitoring and to evaluate how to incorporate new tools into the monitoring. SLAC is developing new ways to analyze and report the information and preparing tools to facilitate this.

We envision the work in this proposal as dovetailing with that of the NSF-sponsored "National Internet Measurement Infrastructure" (NIMI) project. (Vern Paxson is one of the principle investigators of NIMI; the others are Matt Mathis, Jamshid Mahdavi, and Gwendolyn Hunt, of the Pittsburgh Supercomputing Center.) The goal of NIMI is to pilot a scalable measurement system for probing Internet clouds and assessing the performance they deliver. A key facet of NIMI is the development of a modular architecture that can accommodate numerous, different measurement techniques.

The two projects mesh very well. NIMI will provide the general mechanisms for scheduling measurements and retrieving results. IEPM will provide NIMI with access to ESnet and HEP sites (the initial deployment of systems will be to LBNL, SLAC and FNAL, this will be followed by systems at ORNL, BNL and/or CERN) as a testbed for developing and refining the measurement architecture (in particular, the mechanisms for scheduling individual measurements and returning the results) in the context of a large, but not unduly large, network. Both projects will contribute measurement and analysis tools to one another; and IEPM will be assured of compatibility with the NIMI infrastructure if/when the latter becomes widely deployed.

The initial deployment of NIMI systems will be at LLBL, SLAC, and FNAL, to be followed by ORNL and BNL. An alternative would be to place one of the systems at CERN. After the initial deployment is successfully completed, for a future project, we expect to deploy NIMI systems at one to two dozen sites in N. America, Europe and Japan.

The key differences between the work we propose here and that of NIMI are:

SLAC & LBNL have a long history of Wide Area Network support going back to the original creation of HEPnet (the predecessor of ESnet), the creation of the first Internet link to Mainland China, and most recently with the collocation of ESnet management at LBNL. More recently SLAC & LBNL have assumed a leadership role in wide area network monitoring (for a list of presentations and papers see: http://www/xorg/nmtf/nmtf.html#present and

http://www-nrg.ee.lbl.gov/nrg-papers.html.) Vern Paxson of LBNL has published ground breaking work on Internet performance, and Les Cottrell of SLAC is the chairman of the ESnet Network Monitoring Task Force, and the Network Monitoring Focal Group which has made many presentations and publications in this area.

End-to-End Internet Performance, Les Cottrell, C. Granieri & C. Logg, presentation to EOWG/FEPG Meeting, DOE/Germantown, Nov 14, 1996.

Internet Monitoring & NMTF Futures, Les Cottrell, ESCC Meeting, Oct 29, 1996, Princeton, NJ. Report from the Network Monitoring Focus Group, D.E. Martin, Energy Sciences Network Site Coordinating Committee Meeting, Oct 29, 1996, Princeton, NJ,   postscript and PDF

Internet Gridlock and WAN Monitoring, Les Cottrell, invited talk given at the Babar Collaboration Meeting, Dresden, July 1996.

Network Monitoring for the LAN and WAN, invited talk given at ORNL, June 24, 1996 by Les Cottrell.

Distributed Computing Environment Monitoring and User Expectations Les Cottrell & Connie Logg, SLAC-PUB-95-7008, Contributed to the International Conference on Computing In High Energy Physics '95 (CHEP95) Conference, Rio De Janiero, Brazil September 18-22, 1995. See also the html and Postscript transparencies.

Network Monitoring Les Cottrell & Connie Logg, presented at the 1995 DOE Telecommunications Conference, Portland Oregon, July 1995.