SLAC logo

International Internet connectivity from SLAC Network logo

Les Cottrell. Page created: March 28, 2001.

Central Computer Access | Computer Networking | Network Group | ICFA-NTF Monitoring
SLAC Welcome
Highlighted Home
Detailed Home


To provide information to physicists on International connectivity for a talk scheduled at SLAC for April 5th, 2001, and to help understand the needs for improvements to meet the needs of BaBar regional computing centers at IN2P3 Lyon France, INFN Rome Italy and RAL Oxford UK, and other HENP experiments, I put together this page to provide a repository of information concerning major International links (mainly as seen from SLAC).


Where possible I contacted and gathered information from networkers at other sites and from ESnet to understand the network configurations. In addition I used the SLAC reverse traceroute server to find the routes from SLAC to the remote site and to identify the Autonomous Systems (AS) along the path. I also used pipechar a tool for reporting dynamic network characteristics in particular the bottleneck bandwidth to get estimates of bandwidths. Finally I used iperf, as described in Bulk throughput measurements to measure the maximum TCP data rate achievable at the time. I used a Sun Solaris 5.7 host at SLAC with a Gbit Ethernet connection to the network and thence to the SLAC site border to make the pipechar and iperf measurements.


The SLAC LAN provides connectivity to the outside world at up to Gbit speeds via its so-called DMZ. SLAC connects to all the above sites via Esnet. The ESnet link to SLAC is at OC3 (155Mbps) to the Sunnyvale GigaPoP. The GigaPoP connects to the QWest ATM cloud by an OC12 (622Mbps) link. SLAC also has a 622Mbps link to Internet 2 via a Gigabit Ethernet link from SLAC to Stanford University.

The RENATER (France), DFN (Germany), GARR (Italy), and UKERNA (UK) academic and research networks have also signed the MoU with UCAID for the development, together with other worldwide research networks, of Internet2, so this might be an alternative way to reach some of the sites from SLAC.

The main exchange points between ESnet and other ISPs are in New York and Chicago (STARTAP), with the trans-Pacific networks connecting via the Sunnyvale hub. The New York exchange points are in two locations as seen in the figure below prepared in September 2000 by Jim Leighton of ESnet.
New York IXPs, Sep-2000

CERN, Geneva, Switzerland

The traceroute indicates that path from SLAC goes via ESnet to Chicago and thence to CERN. The pipechar indicates that the bottleneck is in ESnet and is about 65Mbps. Looking at Netperf throughput on transatlantic-line CERN (sunstats) & CERN's PoP in Chicago it appears that throughput achievable from CERN to the US increased by about a factor of 2 in mid-January 2001. The plans indicate that CERN is looking to double the bandwidth in the short term future.

IN2P3, Lyon France

The traceroute from SLAC to IN2P3 indicates the route goes over ESnet to Chicago and then crosses the Atlantic on the French PHYnet. The traceroute from IN2P3 to SLAC indicates that the routes are symmetric. The pipechar indicates the bandwdith across the Atlantic and between ESnet and PHYnet is about 43Mbps. This is in rough agreement with the plans which also call for upgrades. The iperf measured maximum TCP throughput was 29Mbps. More information on the French academic and research networks can be found in NRD's and Renater's map.

RAL, Oxford, UK

The traceroute from SLAC to RAL indicates that the route goes from SLAC to ESnet at Sunnyvale, to New York where it joins JANET and crosses the Atlantic to RAL. The pipechar indicates that there are bottlenecks starting in JANET around 45Mbps. A pipechar from Glasgow to SLAC also confirms botlenecks in JANet around 45Mbps. A bulk throughput iperf measurement made on March 29 2001, around 2:00pm PST indicates that the maximum TCP throughput available at the time was about 40Mbps. The current trans Atalantic link to the UK was upgraded to 6 * OC3s on March 23, 2001. The peering to ESnet was upgraded to 85Mbps early February 2001. The JANet Backbone map shows the topology and link speeds.

INFN, Rome, Italy

The traceroute from SLAC to INFN shows the route goes from SLAC via the ESnet Sunnyvale hub to New York then via DANTE to the European TEN-155 network and thence to the Italian GARR academic research network and to Rome. The pipechar indicates that the bottleneck starts on the ESnet and DANTE/TEN-155 and is 45Mbps. The iperf measurements indicate a maximum TCP bandwidth attainable of 26Mbps. The TEN-155 link to the GARR backbone in Milan is at 155Mbps, the GARR backbone is fully meshed at 155Mbps, and the link from GARR to INFN Rome is also at 155Mbps. The GARR-B Network Map and Statistics gives more details on the links.

DESY, Hamburg, Germany

The traceroute shows the route from SLAC to DESY goes via the ESnet Sunnyvale hub to New York, across the Atlantic onto DANTE and onto the TEN-155 European backbone to the German academic research network (WIN) and thence to DESY. The pipechar measurement indicates that the links between ESnet and DANTE and from DANTE through WIN to DESY are at ~45Mbps (T3). From other sources we know the link between ESnet and DANTE is actually 50Mbps, and the Transatlantic link is 90Mbps.

KEK, Tokyo, Japan

The traceroute from SLAC to KEK indicates the route goes from SLAC to the ESnet Sunnyvale Hub and thence to Japan and KEK. The pipechar indicates the bandwidth from ESnet to KEK is limited to 10Mbps.

IHEP, Beijing, China

Information on the link from SLAC to IHEP can be found at: Comparison of with The traceroute from SLAC to IHEP indicates the route goes via KEK (see above) and then via HEPnet-J to IHEP. This latter link is a 128kbps fiber link. The pipechar measurement shows the 10Mbps link from Esnet to KEK, and then a lower speed link via HEPnet-J to IHEP.

BINP, Novosibirsk, Russia

Information on the Novosibirsk link can be found at Network connectivity between SLAC and BINP, Novosibirsk. The traceroute shows the route goes from SLAC to the ESnet Sunnyvale hub to KEK and thence visa HEPnet-J to Novosibirsk. The pipechar indicates that the bottlneck is is 1.5Mbps from HEPnet-J to BINP. This does not agree with our understanding that this link is a fiber 128kbps link.


The table below shows a summary of the information gathered above, plus some historical meassurements made at earlier times. The column labelled AS gives the Autonomous Systems (almost the same as the Internet Service Providers) traversed by the route between SLAC and the destination site (in column 1), apart from KEK it does not provide the AS at the end sites. The Peering @ gives the location where ESnet peers with the next AS providing connectivity to the destination site. The Bandwidth Summer 2000 was the bandwidth between SLAC and the destination site. The Bandwidth March 2001 is the bottleneck bandwidth between the SLAC site and the destination site. If there is an asterisk (*) then the bandwidth is as measured by iperf if a tilde (~) then it is measured by pipechar, otherwise it is by knowing the stated link speeds. The bandwidth between two nodes one at each site may be lower than the bandwidth between the borders o the two sites, so in general we would expect the pipechar (or prior to March 2001 the pchar) measured bandwidth from host to host to be less than the bandwidth from site boundary to site boundary. We minimized this effect at the SLAC end by using a measurement host with high speed (Gigabit Ethernet) connectivity to the SLAC core network and thence the border. Also the maximum iperf TCP bandwidth that can be achieved through a link is typically 80-95% of the unloaded link speed (see Bulk Throughput measurements), so the iperf bandwidths would be expected to be the lowest.
Site City Country AS Peering @ Bandwidth Feb 2000 Bandwidth Sep 2000  Bandwidth Mar 2001 Bandwidth Jul 2001
CERN Geneva Switzerland ESnet, CERN Chicago 12Mbps* 22Mbps~, 27Mbps*, 34Mbps 65Mbps~, 63Mbps*, 155Mps 80Mbps~, 85Mbps*, 155Mbps
IN2P3 Lyon France ESnet, PHYnet Chicago 6Mbps~, 4.5Mbps* 21Mbps* 45Mbps~, 29Mbps* 80Mbps~, 90Mbps*, 155Mbps
RAL Oxford UK ESnet, JANet New York   8.9Mbps~, 6.5Mbps*,  10Mbps 45Mbps~, 40Mbps* 35Mbps~, 45Mbps*, 50Mbps
INFN Rome Italy ESnet, TEN-155, GARR New York 1.5Mbps~, 1.5Mbps* 7Mbps* 45Mbps~, 26Mbps* 45Mbps~, 27.5Mbps*
DESY Hamburg Germany ESnet, TEN-155, WIN New York     45Mbps~
KEK Tokyo Japan ESnet, KEK Sunnyvale     10Mbps
IHEP Beijing China ESnet, KEK, HEPnet-J Sunnyvale     128kbps
BINP Novosibirsk Russia ESnet, KEK, HEPnet-J Sunnyvale     128kbps
It can be seen that there is considerable improvement (factors of 75% to 1600%) in the last year in the maximum TCP throughput achievable from SLAC to the four major international sites of interest to Babar. This is illustrated in the figure below.
Performance for BaBar sites
Page owner: Les Cottrell