Througput Performance between SLAC and CaltechLes Cottrell. Page created: September 10, 3000.Central Computer Access | Computer Networking | Network Group | ICFA-NTF Monitoring |
|
traceroute to lugh.cacr.caltech.edu (131.215.145.183): 3-25 hops, 38 byte packets 3 RTR-DMZ.SLAC.Stanford.EDU (134.79.111.4) [AS3671 - SU-SLAC] 1.85 ms (ttl=253) 4 ESNET-A-GATEWAY.SLAC.Stanford.EDU (192.68.191.18) [AS32 - Stanford Linear Accelerator Center] 1.66 ms (ttl=252) 5 gac2-atms.es.net (134.55.24.6) [AS293 - Energy Sciences Network (ESnet)] 18.0 ms (ttl=251) 6 UCSD--esnet.ATM.calren2.net (198.32.248.69) [AS11422 - NET-C2-SOUTH] 18.3 ms (ttl=250) 7 USC--UCSD.POS.calren2.net (198.32.248.33) [AS11422 - NET-C2-SOUTH] 22.5 ms (ttl=249) 8 QAnh--USC.POS.calren2.net (198.32.248.17) [AS11422 - NET-C2-SOUTH] 22.2 ms (ttl=248) 9 UCI--QAnh.POS.calren2.net (198.32.248.126) [AS11422 - NET-C2-SOUTH] 23.8 ms (ttl=247) 10 UCR--UCI.POS.calren2.net (198.32.248.13) [AS11422 - NET-C2-SOUTH] 24.5 ms (ttl=246) 11 CIT--UCR.POS.calren2.net (198.32.248.9) [AS11422 - NET-C2-SOUTH] 26.5 ms (ttl=245) 12 media-converter.caltech.edu (192.41.208.50) [AS31 - CIT-SSDP] 25.1 ms (ttl=244) 13 SFL-border.dmz.caltech.edu (192.12.19.252) [AS226 - California Institute of Technology] 26.2 ms (ttl=243) 14 CACR-rtr.ilan.caltech.edu (131.215.254.145) [AS226 - California Institute of Technology] 25.8 ms (ttl=242) 15 lugh.cacr.caltech.edu (131.215.145.183) [AS226 - California Institute of Technology] 25.3 ms (ttl=240!)On September 11, 2000 the routing from SLAC to Caltech was changed to go via Stanford University andCalREN2 rather than via ESnet. the new route appears below:
traceroute to lugh.cacr.caltech.edu (131.215.145.183): 1-25 hops, 38 byte packets 3 RTR-DMZ.SLAC.Stanford.EDU (134.79.111.4) [AS3671 - SU-SLAC] 1.53 ms (ttl=253) 4 RTR-SUNET47.SLAC.Stanford.EDU (192.68.191.34) [AS32 - Stanford Linear Accelerator Center] 2.11 ms (ttl=252) 5 Core-gateway.Stanford.EDU (171.64.1.115) [AS32 - BN-CIDR-171.64] 1.83 ms (ttl=251) 6 i2-gateway.Stanford.EDU (171.64.1.209) [AS32 - BN-CIDR-171.64] 1.76 ms (ttl=250) 7 STAN.POS.calren2.NET (171.64.1.213) [AS32 - BN-CIDR-171.64] 1.69 ms (ttl=249) 8 SUNV--STAN.POS.calren2.net (198.32.249.73) [AS11423 - NET-C2-NORTH] 2.39 ms (ttl=248) 9 abilene--QSV.POS.calren2.net (198.32.249.62) [AS11423 - NET-C2-NORTH] 7.62 ms (ttl=247) 10 losa-scrm.abilene.ucaid.edu (198.32.8.18) [AS226 - NETBLK-RA] 13.3 ms (ttl=246) 11 USC--abilene.ATM.calren2.net (198.32.248.85) [AS11422 - NET-C2-SOUTH] 52.6 ms (ttl=249!) 12 QAnh--USC.POS.calren2.net (198.32.248.17) [AS11422 - NET-C2-SOUTH] 58.2 ms (ttl=248!) 13 UCI--QAnh.POS.calren2.net (198.32.248.126) [AS11422 - NET-C2-SOUTH] 61.9 ms (ttl=247!) 14 UCR--UCI.POS.calren2.net (198.32.248.13) [AS11422 - NET-C2-SOUTH] 68.0 ms (ttl=246!) 15 CIT--UCR.POS.calren2.net (198.32.248.9) [AS11422 - NET-C2-SOUTH] 68.8 ms (ttl=245!) 16 media-converter.caltech.edu (192.41.208.50) [AS31 - CIT-SSDP] 74.0 ms (ttl=244!) 17 SFL-border.dmz.caltech.edu (192.12.19.252) [AS226 - California Institute of Technology] 69.7 ms (ttl=243!) 18 CACR-rtr.ilan.caltech.edu (131.215.254.145) [AS226 - California Institute of Technology] 60.7 ms (ttl=242!) 19 lugh.cacr.caltech.edu (131.215.145.183) [AS226 - California Institute of Technology] 62.4 ms (ttl=240!)There appears to be no reverse traceroute server at Caltech, so Juan C. Barayoga [juan@hep.caltech.edu] rann a traceroute for me.
traceroute to WWWSLUG.SLAC.Stanford.EDU (134.79.18.131), 30 hops max, 38 byte packets 1 Lauritsen-RSM-126.caltech.edu (131.215.126.253) 1.339 ms 0.617 ms 0.614 ms 2 Booth-border.ilan.caltech.edu (131.215.254.254) 2.181 ms 0.948 ms 1.086 ms 3 media.dmz.caltech.edu (192.12.19.249) 1.783 ms 0.975 ms 0.765 ms 4 c2-gsr.caltech.edu (192.41.208.49) 2.820 ms 0.877 ms 1.084 ms 5 UCR--CIT.POS.calren2.net (198.32.248.10) 2.345 ms 2.419 ms 2.623 ms 6 UCI--UCR.POS.calren2.net (198.32.248.14) 4.736 ms 3.803 ms 3.539 ms 7 QAnh--UCI.POS.calren2.net (198.32.248.125) 4.156 ms 4.997 ms 4.325 ms 8 USC--QAnh.POS.calren2.net (198.32.248.18) 4.206 ms 3.985 ms 4.009 ms 9 UCSD--USC.POS.calren2.net (198.32.248.34) 8.531 ms 8.230 ms 7.779 ms 10 esnet--UCSD.ATM.calren2.net (198.32.248.70) 45.771 ms 7.938 ms 8.790 ms 11 slac1-atms.es.net (134.55.24.13) 26.194 ms 26.757 ms 25.554 ms 12 * * * 13 * * * 14 * * * 15 WWWSLUG.SLAC.Stanford.EDU (134.79.18.131) 21.827 ms 25.976 ms 22.115 ms
2cottrell@pharlap:~>pingroute.pl -c 100 lugh.cacr.caltech.edu Sun Sep 10 14:59:30 2000 Architecture=SUN5, commands=traceroute -q 1 and ping -s node 1400 100, pingroute.pl version=1.4, 5/16/00, debug=1 pingroute.pl version 1.4, 5/16/00 using traceroute to get nodes in route from pharlap to lugh.cacr.caltech.edu traceroute: Warning: ckecksums disabled traceroute: Warning: Multiple interfaces found; using 134.79.16.102 @ qfe1 traceroute to lugh.cacr.caltech.edu (131.215.145.183), 30 hops max, 40 byte packets pingroute.pl version 1.4, 5/16/00 found 15 hops in route from pharlap to lugh.cacr.caltech.edu 4 ESNET-A-GATEWAY.SLAC.Stanford.EDU (192.68.191.18) 0.938 ms 5 gac2-atms.es.net (134.55.24.6) 48.453 ms 6 UCSD--esnet.ATM.calren2.net (198.32.248.69) 34.704 ms 7 USC--UCSD.POS.calren2.net (198.32.248.33) 54.358 ms 8 QAnh--USC.POS.calren2.net (198.32.248.17) 54.803 ms 9 UCI--QAnh.POS.calren2.net (198.32.248.126) 52.124 ms 10 UCR--UCI.POS.calren2.net (198.32.248.13) 57.540 ms 11 CIT--UCR.POS.calren2.net (198.32.248.9) 65.949 ms 12 media-converter.caltech.edu (192.41.208.50) 59.489 ms 13 SFL-border.dmz.caltech.edu (192.12.19.252) 69.309 ms 14 CACR-rtr.ilan.caltech.edu (131.215.254.145) 62.714 ms 15 lugh.cacr.caltech.edu (131.215.145.183) 69.011 ms Wrote 15 addresses to /tmp/pingaddr, now ping each address 100 times from pharlap pings/node=100 100 byte packets 1400 byte packets NODE %loss min max avg %loss min max avg from pharlap 192.68.191.18 ESNET-A-GATEWAY.SLAC.STANFORD. 0% 0.0 194.0 6.0 0% 2.0 162.0 7.0 Sun Sep 10 15:09:25 PDT 2000 134.55.24.6 GAC2-ATMS.ES.NET 0% 17.0 178.0 23.0 0% 19.0 276.0 35.0 Sun Sep 10 15:12:43 PDT 2000 198.32.248.69 UCSD--ESNET.ATM.CALREN2.NET 0% 18.0 20.0 18.0 0% 20.0 22.0 20.0 Sun Sep 10 15:16:02 PDT 2000 198.32.248.33 USC--UCSD.POS.CALREN2.NET 0% 18.0 21.0 18.0 0% 20.0 23.0 20.0 Sun Sep 10 15:19:20 PDT 2000 198.32.248.17 QANH--USC.POS.CALREN2.NET 0% 18.0 22.0 18.0 0% 21.0 24.0 21.0 Sun Sep 10 15:22:38 PDT 2000 198.32.248.126 UCI--QANH.POS.CALREN2.NET 0% 19.0 22.0 19.0 0% 22.0 24.0 22.0 Sun Sep 10 15:25:56 PDT 2000 198.32.248.13 UCR--UCI.POS.CALREN2.NET 0% 20.0 22.0 20.0 0% 23.0 25.0 23.0 Sun Sep 10 15:29:15 PDT 2000 198.32.248.9 CIT--UCR.POS.CALREN2.NET 0% 21.0 23.0 21.0 0% 24.0 26.0 24.0 Sun Sep 10 15:32:33 PDT 2000 192.41.208.50 MEDIA-CONVERTER.CALTECH.EDU 0% 24.0 25.0 24.0 0% 27.0 28.0 27.0 Sun Sep 10 15:35:51 PDT 2000 192.12.19.252 SFL-BORDER.DMZ.CALTECH.EDU 0% 26.0 104.0 27.0 0% 29.0 243.0 34.0 Sun Sep 10 15:39:09 PDT 2000 131.215.254.145 CACR-RTR.ILAN.CALTECH.EDU 0% 26.0 49.0 26.0 0% 29.0 33.0 30.0 Sun Sep 10 15:42:27 PDT 2000 131.215.145.183 LUGH.CACR.CALTECH.EDU 0% 26.0 28.0 26.0 0% 30.0 33.0 30.0 Sun Sep 10 15:45:45 PDT 2000The pingroute measurement from SLAC to Caltech after the route was changed to send traffic from SLAC to Caltech via the Stanford campus and CalREN2 rather than by ESnet.
9cottrell@flora01:~>bin/pingroute.pl -c 100 lugh.cacr.caltech.edu Mon Sep 11 11:07:26 2000 Architecture=SUN5, commands=traceroute -q 1 and ping -s node 1400 100, pingroute.pl version=1.4, 5/16/00, debug=1 pingroute.pl version 1.4, 5/16/00 using traceroute to get nodes in route from flora01 to lugh.cacr.caltech.edu traceroute: Warning: ckecksums disabled traceroute to lugh.cacr.caltech.edu (131.215.145.183), 30 hops max, 40 byte packets pingroute.pl version 1.4, 5/16/00 found 19 hops in route from flora01 to lugh.cacr.caltech.edu 4 RTR-SUNET47.SLAC.Stanford.EDU (192.68.191.34) 2.742 ms 5 Core-gateway.Stanford.EDU (171.64.1.115) 1.276 ms 6 i2-gateway.Stanford.EDU (171.64.1.209) 1.916 ms 7 STAN.POS.calren2.NET (171.64.1.213) 2.133 ms 8 SUNV--STAN.POS.calren2.net (198.32.249.73) 2.438 ms 9 abilene--QSV.POS.calren2.net (198.32.249.62) 7.411 ms 10 losa-scrm.abilene.ucaid.edu (198.32.8.18) 12.620 ms 11 USC--abilene.ATM.calren2.net (198.32.248.85) 66.815 ms 12 ISI--USC.POS.calren2.net (198.32.248.26) 74.395 ms 13 UCLA--ISI.POS.calren2.net (198.32.248.30) 73.344 ms 14 JPL--UCLA.POS.calren2.net (198.32.248.2) 69.756 ms 15 CIT--JPL.POS.calren2.net (198.32.248.6) 72.575 ms 16 media-converter.caltech.edu (192.41.208.50) 75.949 ms 17 Booth-border.dmz.caltech.edu (192.12.19.254) 38.460 ms 18 CACR-rtr.ilan.caltech.edu (131.215.254.145) 26.945 ms 19 lugh.cacr.caltech.edu (131.215.145.183) 47.004 ms Wrote 19 addresses to /tmp/pingaddr, now ping each address 100 times from flora01 pings/node=100 100 byte packets 1400 byte packets NODE %loss min max avg %loss min max avg from flora01 192.68.191.34 RTR-SUNET47.SLAC.STANFORD.EDU 0% 1.0 32.0 2.0 0% 3.0 116.0 5.0 Mon Sep 11 11:17:26 PDT 2000 171.64.1.115 CORE-GATEWAY.STANFORD.EDU 0% 1.0 5.0 1.0 0% 3.0 10.0 3.0 Mon Sep 11 11:20:44 PDT 2000 171.64.1.209 I2-GATEWAY.STANFORD.EDU 0% 1.0 18.0 2.0 0% 3.0 6.0 3.0 Mon Sep 11 11:24:03 PDT 2000 171.64.1.213 STAN.POS.CALREN2.NET 0% 1.0 4.0 1.0 0% 3.0 20.0 3.0 Mon Sep 11 11:27:21 PDT 2000 198.32.249.73 SUNV--STAN.POS.CALREN2.NET 0% 3.0 8.0 3.0 0% 5.0 18.0 5.0 Mon Sep 11 11:30:39 PDT 2000 198.32.249.62 ABILENE--QSV.POS.CALREN2.NET 0% 6.0 13.0 6.0 0% 8.0 21.0 9.0 Mon Sep 11 11:33:57 PDT 2000 198.32.8.18 LOSA-SCRM.ABILENE.UCAID.EDU 0% 11.0 23.0 12.0 0% 14.0 20.0 14.0 Mon Sep 11 11:37:15 PDT 2000 198.32.248.85 USC--ABILENE.ATM.CALREN2.NET 0% 16.0 19.0 16.0 0% 18.0 25.0 19.0 Mon Sep 11 11:40:33 PDT 2000 198.32.248.26 ISI--USC.POS.CALREN2.NET 0% 16.0 19.0 17.0 0% 19.0 21.0 19.0 Mon Sep 11 11:43:51 PDT 2000 198.32.248.30 UCLA--ISI.POS.CALREN2.NET 0% 17.0 24.0 17.0 0% 20.0 22.0 20.0 Mon Sep 11 11:47:10 PDT 2000 198.32.248.2 JPL--UCLA.POS.CALREN2.NET 0% 17.0 21.0 18.0 0% 20.0 27.0 21.0 Mon Sep 11 11:50:28 PDT 2000 198.32.248.6 CIT--JPL.POS.CALREN2.NET 0% 19.0 32.0 19.0 0% 22.0 24.0 22.0 Mon Sep 11 11:53:46 PDT 2000 192.41.208.50 MEDIA-CONVERTER.CALTECH.EDU 0% 21.0 23.0 21.0 0% 23.0 31.0 24.0 Mon Sep 11 11:57:05 PDT 2000 192.12.19.254 BOOTH-BORDER.DMZ.CALTECH.EDU 0% 18.0 82.0 20.0 0% 22.0 170.0 24.0 Mon Sep 11 12:00:23 PDT 2000 131.215.254.145 CACR-RTR.ILAN.CALTECH.EDU 1% 20.0 22.0 20.0 0% 24.0 58.0 24.0 Mon Sep 11 12:03:42 PDT 2000 131.215.145.183 LUGH.CACR.CALTECH.EDU 0% 20.0 91.0 22.0 0% 24.0 61.0 24.0 Mon Sep 11 12:07:01 PDT 2000
pathchar to lugh.cacr.caltech.edu (131.215.145.183) mtu limitted to 1500 bytes at FLORA01.SLAC.Stanford.EDU (134.79.16.29) doing 32 probes at each of 64 to 1500 by 44 0 localhost | 25 Mb/s, 210 us (0.91 ms) 1 router1.SLAC.Stanford.EDU | 77 Mb/s, 166 us (1.39 ms) 2 router2.SLAC.Stanford.EDU | 118 Mb/s, 57 us (1.61 ms) 3 router3.SLAC.Stanford.EDU | 114 Mb/s, -107 us (1.50 ms) 4 ESNET-A-GATEWAY.SLAC.Stanford.EDU (192.68.191.18) | 28 Mb/s, 8.16 ms (18.3 ms) 5 gac2-atms.es.net (134.55.24.6) | 36 Mb/s, 59 us (18.7 ms) 6 UCSD--esnet.ATM.calren2.net (198.32.248.69) | 337 Mb/s, 1.76 ms (22.3 ms) 7 USC--UCSD.POS.calren2.net (198.32.248.33) | 321 Mb/s, 268 us (22.8 ms) 8 ISI--USC.POS.calren2.net (198.32.248.26) -> 198.32.248.26 (1) | 336 Mb/s, 240 us (23.3 ms) 9?UCLA--ISI.POS.calren2.net (198.32.248.30) | 112 Mb/s, 285 us (24.0 ms) 10 JPL--UCLA.POS.calren2.net (198.32.248.2) | 354 Mb/s, 759 us (25.6 ms) 11 CIT--JPL.POS.calren2.net (198.32.248.6) | 284 Mb/s, -2 us (25.6 ms) 12 media-converter.caltech.edu (192.41.208.50) | 48 Mb/s, -267 us (25.3 ms) 13 Booth-border.dmz.caltech.edu (192.12.19.254) | 65 Mb/s, 613 us (26.7 ms) 14 CACR-rtr.ilan.caltech.edu (131.215.254.145) 15 * 1 504 528 32 ->131.215.254.145 (1)The pathchar after the routing was changed to go via CalREN2 appears below (measured 9/20/00):
pathchar to lugh.cacr.caltech.edu (131.215.145.183) mtu limitted to 1500 bytes at FLORA05.SLAC.Stanford.EDU (134.79.16.59) doing 32 probes at each of 64 to 1500 by 44 0 FLORA05.SLAC.Stanford.EDU (134.79.16.59) | 25 Mb/s, 213 us (0.91 ms) 1 RTR-CORE1.SLAC.Stanford.EDU (134.79.19.2) | 83 Mb/s, 187 us (1.42 ms) 2 RTR-CGB6.SLAC.Stanford.EDU (134.79.135.6) | 110 Mb/s, 73 us (1.68 ms) 3 RTR-DMZ.SLAC.Stanford.EDU (134.79.111.4) -> 134.79.111.4 (3) | 53 Mb/s, 102 us (2.11 ms) 4?RTR-SUNET47.SLAC.Stanford.EDU (192.68.191.34) | 167 Mb/s, -58 us (2.07 ms) 5 Core-gateway.Stanford.EDU (171.64.1.115) | 171 Mb/s, -1 us (2.13 ms) 6 i2-gateway.Stanford.EDU (171.64.1.209) | 219 Mb/s, 28 us (2.25 ms) 7 STAN.POS.calren2.NET (171.64.1.213) | 101 Mb/s, 145 us (2.65 ms) 8 SUNV--STAN.POS.calren2.net (198.32.249.73) | ?? b/s, 1.63 ms (5.84 ms) 9 abilene--QSV.POS.calren2.net (198.32.249.62) | 71 Mb/s, 3.50 ms (13.0 ms) 10 losa-scrm.abilene.ucaid.edu (198.32.8.18) | ?? b/s, 122 us (13.2 ms) 11 USC--abilene.ATM.calren2.net (198.32.248.85) | 282 Mb/s, 427 us (14.1 ms) 12 QAnh--USC.POS.calren2.net (198.32.248.17) | 103 Mb/s, 309 us (14.8 ms) 13 UCI--QAnh.POS.calren2.net (198.32.248.126) | ?? b/s, 654 us (16.1 ms) 14 UCR--UCI.POS.calren2.net (198.32.248.13) | 109 Mb/s, 787 us (17.8 ms) 15 CIT--UCR.POS.calren2.net (198.32.248.9) | 275 Mb/s, -633 us (16.6 ms) 16 media-converter.caltech.edu (192.41.208.50) | 48 Mb/s, 370 us (17.6 ms) 17 SFL-border.dmz.caltech.edu (192.12.19.252) | 82 Mb/s, -48 us (17.6 ms) 18 CACR-rtr.ilan.caltech.edu (131.215.254.145) 19 * 1 812 527 17 ^C: 13 328 323 17
lugh% ./iperf -c pharlap.slac.stanford.edu -p 5007 -w 256k -P 6 ------------------------------------------------------------ Client connecting to pharlap.slac.stanford.edu, TCP port 5007 TCP window size: 256 KByte ------------------------------------------------------------ [ 9] local 131.215.145.183 port 38117 connected with 134.79.16.102 port 5007 [ 6] local 131.215.145.183 port 38114 connected with 134.79.16.102 port 5007 [ 5] local 131.215.145.183 port 38112 connected with 134.79.16.102 port 5007 [ 4] local 131.215.145.183 port 38113 connected with 134.79.16.102 port 5007 [ 7] local 131.215.145.183 port 38115 connected with 134.79.16.102 port 5007 [ 8] local 131.215.145.183 port 38116 connected with 134.79.16.102 port 5007 [ ID] Interval Transfer Bandwidth [ 9] 0.0-11.1 sec 776 KBytes 561 Kbits/sec [ 6] 0.0-11.2 sec 776 KBytes 556 Kbits/sec [ 5] 0.0-11.3 sec 520 KBytes 367 Kbits/sec [ 7] 0.0-12.1 sec 1.0 MBytes 682 Kbits/sec [ 4] 0.0-12.3 sec 1.3 MBytes 846 Kbits/sec [ 8] 0.0-13.1 sec 264 KBytes 161 Kbits/sec
On September 11, 2000 SLAC and Stanford changed the routing from SLAC to Caltech to use Stanford/CalREN2 rather than ESnet. The new route is given above. With the new route the performance was about doubled. See below:
Sometime between 9/14/00 and 9/18/00 the routing of packets from
UCSD to SLAC was changed to go via CalREN2 rather than via ESnet.
We were unable to provide a traceroute
from lugh since we could not find traceroute on lugh.
Following
the route change we used iperf with a 256kByte window and got
19Mbits/s with 1 stream and 25Mbps with 10 streams from lugh.cacr.caltech.edu
and pharlap.slac.stanford.edu.
This is still below the 50Mbits/s fmeasured from SLAC to
Caltech. We requested the isntallation of pathchar on lugh to assist
in finding nthe bottleneck from Caltech to SLAC.