SciDAC Proposal - SLAC's involvement

SLAC will focus on the areas of network performance measurement and monitoring and also the roles of network transport applications at both application level (e.g. FTP, GridFTP etc) and protocols level (e.g. TCP, UDP etc).

Extra network provisioning (through increased resources or advanced technologies) is only useful when the SciDAC applications are able to take advantage of it.
The monitoring of network usages is important to identify potential network related problems in both the host and network services and to plan for future upgrades.

Through extending the DOE funded DWMI proposal to fully support an end-to-end monitoring solution involving both end-systems and intermediate routers and switches, a full depiction of the performance bottlenecks can be determined.
Currently, the DWMI solution provides network monitoring with regular active and passive measurement methods in order to provide a detail representation (and trends) of Internet usage patterns and performances.
We currently configure the intrusiveness of active network measurement (e.g. iperf, ping, traceroute, pipechar etc.) based on known network capabilities, and use passive means such as SNMP, Netflow and host level solutions such as Web100 extensively.
We have studied and evaluated the host effective network monitoring tools for each network and believe that as end-host performance and network utilisation increases, it will become essential to be able to capture information from reliable passive means such a real GridFTP transfers.

With the network monitoring information available, we are developing innovative event detection facilities using multiple techniques (neural networks, ... and principal component analysis) whereby network performance variations are captured quickly and automatically and email notification is generated for relevant parties of such changes.
However, much more research needs to be conducted in order to determine the best algorithms for such event detection.
The implementation of such notifications have vastly improved the ability to both determine the occurence of such problems (from days to hours) and to diagnose such problems through manual means.

A natural extension to the labour intensive manual diagnosis and cross correlation is to automate such problem solving through mathematical and systems based design.
We are currently pursuing such goals by working closely with institutions such as Internet2, ESnet and Geant to automatically pinpoint the exact cause of network performance degradations; whether it be on the end-hosts or across the wide area network.
We are working with projects such perfSONAR and AMP in order to federate and utilise such information on wide-area networks.

An important limitation to be able to utilise the vast network resources available is often caused by bottlenecks at host end systems.
Low level operating system based parameters such as queue sizes and TCP congestion control algorithms are often the cause of low throughput performances experienced - especially when 10Gbit/sec and beyond network capacities are available end to end.

SLAC has extensive research and real-life experience with both identifying and resolving the host based bottlenecks for high throughput network transfers.
In particular we have extensively tested and published on the performances of new TCP algorithms that promise to both utilise network resources effectively and fairly.
We have also experimented with UDP-based transport algorithms that can fully utilise lambda and QoS/DiffServ network paths and are currently working closely with BNL to monitor QoS paths effectively.

Whilst the application of such new algorithms have bought much benefit to High Energy Physics experiments at Babar, we feel that they can also have a huge benefit on other network intensive applications and we wish to explore this futher with other science groups.
As the introduction of these new TCP algorithms gather speed, we feel that it will become very important to measure both the positive and negative effects on production services and well as the Internet as a whole.

SLAC is also a leader in the field of new transport applications.
Programs such as bbcp and xrootd have been proven in both high-performance experiments (at the recent SuperComputing 2005 conference) and in large scale production systems.
As networks become more intelligent with advanced technologies, it will be even more useful to be able to interface with the network services available.
Through close colloboration with both SciDAC projects and internal SLAC initiatives, it shall be possible to integrate the two into one cohesive and effective package.