|Impact of routing on End-to-end Internet performance Les Cottrell Last Update: March 29, 1999 Central Computer Access | Computer Networking | Internet Monitoring | Tutorial on Internet Monitoring||
An example of a short term routing change can be seen in the figure below that shows the response time in milliseconds measured each half hour between 4 monitoring sites in the US to NIST in Gaithersburg Maryland. It can be seen that around 21:00 on November 20th until around 18:00 on November 21st there was a sudden increase in the response time of an additional 150-250 milliseconds. The cause is believed to be a change in the routes that diverted them via the NIST campus in Boulder, Colorado.
An example of the change (in this case an improvement) obtained when a site (Brown University) changed its Network Service Provider, in this case moving to vBNS can be seen below. The measurements are for each half hour, the black indicates the response time in milliseconds, and the red the percentage packet loss. The dramatic improvement in response time and packet loss as Brown moves onto vBNS in June 1998, is immediately apparent.
The figure below shows the response time in milliseconds
and % packet loss, for each half hour from
September 1998 thru November 1998,
between SLAC (Silicon
Valley/San Francisco Peninsular) and KEK (near Tokyo Japan).
The black line is for the response time (Round Tript Time - RTT)
and the red line is for packet loss.
Around the end of October 1998, KEK's (Japan) traffic to SLAC was
rerouted via STAR-TAP in
Chicago instead of going more directly via the Sprint Oakland POP. As can be seen
in the figure this increased the response time by about 80-100 milliseconds. Also
shown in this figure is the detailed behavior at the time the changeover was
made, and the green line shows the ITU G114 300 millisecond threshold for
RTT under which toll quality voice is possible.
The figure below shows the median % monthly packet loss measured from KEK to
various remote sites which have been grouped by area. The areas are indicated in
the legend together with the number of monitor-remote site pairs in parenthesese
that contribute to each group. It can be seen that the direcyt connection of
the NACSIS (Japan) line to Europe at the start of July 1998
made a dramatic improvement in the packet loss between KEK and Europe while
having little effect on the N. American remote sites.
The figure below shows the median % packet loss for the Budker Institute
of Nuclear Physics (BINP) in Novosibirsk measured from various monitoring
sites in Canada, the U.S. and Japan. The top two curves (red and blue) are for
Canadian and .edu monitoring sites (the number of monitoring sites are
shown in the
legend in parentheses), and are seen to have performances several times worse
than when measured from ESnet monitoring and Japanses sites. This can be
understood by looking at the routing. The routes from Canada and .edu cross the
Atlantic and go via the Deutsches Forschungs Network (DFN) to DESY (Hamburg,
Germany) where they go via a single satellite hop to Novosibirsk. The
ESnet go across the pacific to KEK Japan and then via a fiber optic link to
Novosibirsk. The trans-Pacific/fiber route is seen to be much better than the
The figure below shows the % packet loss (green) and RTT (red) in milliseconds
measured from DESY to Novosibirsk between March 8th and 10th 1999. the
measurements are for ten 100 byte pings made each half hour. It can be seen that
each day from about 17:30 thru 19:00 one observes 100% packet loss. When one
examines the routes most of the time the route is very direct (via satellite with
a minimum round trip delay of about 1 second)
from DESY to Novosibirsk, however, between 17:30 and 19:00 the route travels
twice across the Atlantic and goes via DFN, UUNET
(Washington, NY, Stockholm) and
Relcom. Obviously this backup route is very poor with ~ 100% packet loss.
Further investigation reveals that the satellite is in the shadow of the
earth between 17:30 and 19:00 at this time of year, and the operators shut it
down to save the batteries.
The figure below shows the median % monthly packet loss measured at European
sites for remote sites in N. America. Roughly speaking the curves are divided
into two sets, those with good to acceptable (< 2.5%) loss and those with
very poor to unusable (>> 5%) loss. In the former set, CERN has dedicated
bandwidth (4Mbps at the time) to N. America, and DESY and CNAF/Italy had
dedicated (1.5Mbps) to ESnet. It can be seen that this dedicated bandwdith leads
to good to acceptable performance. The other routes even though typically
having much more bandwdith (e.g. the DESY/DFN link to N. America was 90Mbps,
the INFN/GARR lnk was 45Mbps) shared this bandwidth with a large Research and
Education (R&E) community. The improvement around the end of the year
was partially due to the holiday season, and also KFKI/Hungary started to
share the TEN-155 155Mbps trans-Atlantic link.
The figure below shows the monitoring sites grouped by Top Level Domain (TLD) across the top and the remote monitored sites grouped by TLD along the left column. Each cell in the table shows the median monthly % ping packet loss for February 1999, colored according to: white (good) <=1% loss, green (acceptable) > 1% & <=2.5%, yellow (poor) >2.5% & <=5%, pink (very poor) > 5% & <=12%, red (un-usable) > 12%. One can notice that the diagonal values (i.e. the traffic stays within a TLD) are generally better than the off-diagonal values. This is a consequence of the fact that performance is generally improved by reducing the number of Network Service Providers (NSPs) or Internet Service Providers (ISPs) routes need to pass through. This is strongly advocated by the Automobile Nextwork eXchange (ANX) consortium who have a very stringent Service level Agreement (SLA) with their providers.