May 4-5, 1998

Rough notes taken by Les Cottrell

There were 6 reviewers including major users from HENP (Richard Mount), Fusion (Tom Casper), the weapons program, plus people from supercomputing centers, and Internet engineering (Guy Almes). There were about 22 attendees, many of whom (about 11) made presentations. Several (~10) of the attendees were also members of the ESSC. The presentations were from the ESnet management and ESnet funded folks, from the major user programs (HENP, BES, Fusion) and from the DOE MICS program. There were about 12 laptops in use.

The ESnet budget is about $14M and has been pretty flat for the last 5 years. About $500K is used for international connectivity, $270K for video conferencing, $6M to Sprint, $5M for Jim Leighton's group, and $730K for equipment. The ESnet community includes 28 major user facilities, about 20 National Labs, 278 Colleges & Universities, 265 companies and 47 Federal Labs. The Sprint contract expires August 1999, and a new contract will need to be defined and set up before then. They are hoping to be involved in NGI and are looking at pursuing Class Based Queuing (CBQ). In the longer term they want to incorporate new technologies needed by the DOE mission, and to improve International & University access. DOE HQ regards ESnet as a major success. There were questions on how preferred Sprint is for the follow on contract, how the ESnet budget fraction of the total MIC/DOE budget was determined, how closely coupled the ESnet & NASA are on the current Sprint contract and whether that will continue with the next contract. ESnet also gets about $1M/year from other programs such as DoD to provide them with connectivity.

The idea of a steering committee for policy and a site coordinating committee (ESCC) to represent sites and to provide technical expertise (often by working groups or task forces) has been very successful. ESnet was originally set up to support the Energy Research (ER) program. More recently it has added support for Energy Efficiency, Nuclear Energy, the Energy Information Agency, Human Resources and the Defense Programs with the appropriate MOU. The ESSC has representatives from each of the ER programs and several of the added programs, plus ESnet management.

The ESCC makes good connections with the technical people who are running the sites. Their meetings are well attended. The people involved in the ESCC are volunteers. They are an excellent complement to the ESnet management people.

The international requirements are addressed via the ESnet International meetings which are informal, unofficial meetings of international partners of ESnet. Sometimes these meetings are held in conjunction with the ESSC meetings. The HEP community in the form of ICFA will probably recommend improved international connectivity.

There is a significant requirement for good connectivity to Universities with DOE programs. ESnet is responsible for providing connectivity for all DOE/ER programs. Advice is provided on how to connect to vBNS/I2 and ISPs and ESnet peers effectively with vBNS. Occasionally connectivity is provided directly between ESnet and a University, usually with the University paying the connection cost. There is a requirement that the University CIO has to make a written request to ESnet (to avoid intra-University conflicts). In the earlier days a few Universities became ESnet backbone sites and the connectivity was funded by ESnet. There were questions on how large the university component is for the ER program. Over 242 Universities are funded by the DOE. 96 universities are funded out of HEP for about $90M (520 students, 410 post-docs & 580 faculty.

The last review in 1994 asked for more monitoring & measurements, a strategic plan, an operating plan and broad interconnection of networks & ISPs.

One remark Jim made was that PIM is not ready for rolling out into the backbone. Someone at SLAC may wish to follow up on this.

Jim made a big point that network planning requires an exceptional degree of cooperation by its nature "…it’s the sociology and politics, not the technology" and requires cooperative efforts between peers with no one generally "in charge".

Planning is an art and very dynamic, so one has to stay flexible. For example, one has to understand the politics, the sociology, and procurement, budgetary and AUP constraints, one has to be ready to jump on new opportunities such as HPCC, NREN, NGI, I2, and new technologies, pricing and architecture.

There were many questions on International link strategy. Such a strategy is very difficult to come up with due to cost asymmetries, program requirements, national policies etc. Will probably have to come back to this later.

They are developing a highly available, production quality set of high-level services with a common look and feel. The choices, where possible, are based on COTS, only do otherwise if there are no products available. Examples of the services/applications they are supporting include:

• video (VCSS with about 1400 hours/month or 600 conferences) conferencing,
• audio (MeetingPlace with a 48 port audio bridge with phone quality audio - out of band, i.e. phone lines) conferencing, and
• data (PictureTalk with 10 user license, will have MS NetMeeting and T.120 support coming soon) conferencing.

For the future they are looking at audio-video record-play on demand, using ATM for video conferencing, providing support for continuous presence (e.g. all 9 presenters can be visible simultaneously), Web casting, and live video distribution (e.g. by using ATM). Allen made no mention of X.500 or white page support. In response to a question he said they are looking to drop X.500 and are looking for a commercial LDAP server, but have not found one suitable yet, also the standard for distributed LDAP services is not finalized.

I talked to Guy Almes afterwards. He was very supportive of our measurements and feels it is very valuable and essential for ESnet. We also talked about analyzing the Surveyor data, and he says now is a good time to discuss this. They have many Gbytes of data. They have converted it from Oracle to a flat Unix ASCII files to simplify things. He also mentioned the new Abilene project which is to build a new OC12/OC48 backbone similar to vBNS using fibers donated by Qwest. This may provide a competitor to vBNS that would be helpful.

Fusion relies on the network, work comes to a halt without the basic network services. ESnet is essential and effective tool for the community. The higher level services are emerging with the hope, by users, that this will make their life easier, rather than more complex.

Providing feedback and collecting requirements is done via the ESSC program representatives. This is an informal route, there is no formal meeting with the representatives. In general, in the fusion community, users are quite satisfied, they like the high quality and availability. The complaints were on connections to universities and international links. People are looking for help on the security infrastructure, and are looking for long term support for collaboration services (i.e. they want to know they are there for the long haul before committing to using them.

University issues:

• The connectivity is weak compared to DOE Labs;
• The DOE/ER & university organizations are not parallel
• Where is the ER labs/PI;s place within university departments, conflicts between the ESnet local subnet and the university network in general.
• There are differences in funding both in amounts and in separation of overheads
• There are differences in culture, the university's mission is to provide support for education/departments/students, they have a strong NSF orientation and there are AUP conflicts.

A large fraction of the BES facility usage is from within the U.S. (over 80%), i.e. less than say the HENP field, so they are less interested in international links. Ray presented the example (that Peter Kuhn of SSRL and Les Cottrell made available to Ray) of a new crystallography beamline that gathers 12 Gbytes per 8 hour shift, plus 10Mbps for high performance 3D graphics, plus 6 Mbps for full-motion video, plus 300Kbps per beamline for videoconferencing. There will be 4 such beamlines at SSRL, one starting summer 1998, 3 more in 1999, and 16 nationwide.

Universities cannot access foreign Labs via Esnet. Walter showed several slides showing the ESnet connectivity to relevant sites together with the quality of the connections. He said the monitoring is being done very well and is extremely useful. Networking is indispensable to HEP. Esnet provides high quality service between Labs and to several universities. VBNS (& ESnet hubbing/peering) is solving problems of universities trying to reach national labs. HEP research requires connectivity to Europe and Asia. Future HEP experiments will exercise networking with the use of distributed databases => networking needs will increase.

Key concerns:

• Lack of strategy for universities and international connections. Need strategy for tactical problem tracking. International may need more attention. It is unclear what ESnet's responsibility is in this area (at the level of $500K it cannot be very great). The DOE ER community has an increasing need for International connectivity that so far has been considered only a small part of the ESnet requirements.
• Universities are not in such an "unattended state". The future success of ESnet requires that it has excellent connectivity to the Universities. Need to work with the Universities in a visibly organized, coordinated way.
• Maybe the ESnet charter should be expanded or re-interpreted. At the moment it is kind of constrained to providing good connectivity between ESnet sites. It provides connectivity in support of DOE ER programs. With current funding not a lot can be done, maybe working more with NSFnet. Another more positive suggestion was to encourage ESnet to have excellent connectivity to the networks that the Universities connect to. Initially this would be the vBNS/I2 GigaPoPs.

They like the user involvement in the committees and working groups.

Larry Price pointed out that there was an ESSC led attempt to provide better connectivity to several universities about a year ago, however this ran into political opposition as universities focussed on NSF funding for providing university connections.