|
|
Througput Performance between SLAC and IN2P3
Les Cottrell.
Page created: June 12, 2000, last update: June 12, 2000.
|
|
From datamove3 : - 2 threads No Comp 7.5 Mb/s - 4 threads No Comp 12.4 Mb/s - 5 threads No Comp 13.6 Mb/s - 6 threads No Comp 19.1 Mb/s - 8 threads No Comp 20.1 Mb/s - 10 threads No Comp 19.9 Mb/s - 5 threads with Comp 13.6 Mb/s - 8 threads with Comp 6.3 Mb/s - 10 threads with Comp 7.4 Mb/s From tersk02 : - 10 threads with Comp 26.4 Mb/s - Second try : 25.1 Mb/s From shire01 : - 10 threads with Comp 37.9 Mb/sSo it appears that the throughput is limited by the machine itself when we use compression. Dominique has seen the load on datamove3 increasing from ~5 up to ~17 when he was trying to use 10 threads with compression. The normal load on datamove3 stays below 1.3, and typically it is much lower. Randy Melen sees CPU usage typically less than 10%. He concludes that nothing is being done on this system whenever he watches it!
Since bulk data transfers from SLAC to other sites are critical to BaBar it is important to understand and resolve this.
29cottrell@tersk01:~>bin/pingroute.pl -c 100 ccobsn04.in2p3.fr
Architecture=SUN5, commands=traceroute -q 1 and ping -s node 1400 100, pingroute.pl version=1.4, 5/16/00, debug=1
pingroute.pl version 1.4, 5/16/00 using traceroute to get nodes in route from tersk01 to ccobsn04.in2p3.fr
traceroute: Warning: ckecksums disabled
traceroute to ccobsn04.in2p3.fr (134.158.104.144), 30 hops max, 40 byte packets
pingroute.pl version 1.4, 5/16/00 found 11 hops in route from tersk01 to ccobsn04.in2p3.fr
1 RTR-MSFC-SCS-IR2A.SLAC.Stanford.EDU (134.79.127.7) 0.622 ms
2 RTR-CGB6.SLAC.Stanford.EDU (134.79.135.6) 0.859 ms
3 RTR-DMZ.SLAC.Stanford.EDU (134.79.111.4) 1.158 ms
4 ESNET-A-GATEWAY.SLAC.Stanford.EDU (192.68.191.18) 0.821 ms
5 chicago1-atms.es.net (134.55.24.17) 108.832 ms
6 206.220.243.32 (206.220.243.32) 107.531 ms
7 cernh9-s5-0.cern.ch (192.65.184.142) 222.656 ms
8 in2p3-cernh9.cern.ch (192.65.184.46) 216.639 ms
9 192.70.69.29 (192.70.69.29) 235.880 ms
10 Lyon-ANDA.in2p3.fr (134.158.240.1) 237.299 ms
11 ccobsn04.in2p3.fr (134.158.104.144) 220.581 ms
Wrote 11 addresses to /tmp/pingaddr, now ping each address 100 times from tersk01
pings/node=100 100 byte packets 1400 byte packets
NODE %loss min max avg %loss min max avg from tersk01
134.79.127.7 RTR-MSFC-SCS-IR2A.SLAC.STANFOR 0% 0.0 22.0 0.0 0% 0.0 22.0 0.0 Mon Jun 12 13:26:50 PDT 2000
134.79.135.6 RTR-CGB6.SLAC.STANFORD.EDU 0% 0.0 3.0 0.0 0% 1.0 26.0 1.0 Mon Jun 12 13:30:08 PDT 2000
134.79.111.4 RTR-DMZ.SLAC.STANFORD.EDU 0% 0.0 27.0 1.0 0% 1.0 9.0 1.0 Mon Jun 12 13:33:26 PDT 2000
192.68.191.18 ESNET-A-GATEWAY.SLAC.STANFORD. 0% 0.0 287.0 13.0 0% 1.0 412.0 19.0 Mon Jun 12 13:36:44 PDT 2000
134.55.24.17 CHICAGO1-ATMS.ES.NET 0% 56.0 427.0 92.0 0% 58.0 217.0 111.0 Mon Jun 12 13:40:02 PDT 2000
206.220.243.32 206.220.243.32 0% 58.0 182.0 92.0 0% 60.0 161.0 61.0 Mon Jun 12 13:43:21 PDT 2000
192.65.184.142 CERNH9-S5-0.CERN.CH 0% 168.0 227.0 169.0 0% 171.0 293.0 191.0 Mon Jun 12 13:46:39 PDT 2000
192.65.184.46 IN2P3-CERNH9.CERN.CH 0% 169.0 308.0 203.0 0% 173.0 319.0 216.0 Mon Jun 12 13:49:57 PDT 2000
192.70.69.29 192.70.69.29 0% 172.0 302.0 186.0 0% 177.0 290.0 179.0 Mon Jun 12 13:53:16 PDT 2000
134.158.240.1 LYON-ANDA.IN2P3.FR 0% 172.0 211.0 177.0 0% 177.0 221.0 180.0 Mon Jun 12 13:56:34 PDT 2000
134.158.104.144 CCOBSN04.IN2P3.FR 0% 172.0 321.0 192.0 0% 177.0 367.0 188.0 Mon Jun 12 13:59:53 PDT 2000
3cottrell@flora01:~>sudo /afs/slac/g/scs/bin/pathchar ccobsn04.in2p3.fr
Password:
pathchar to ccobsn04.in2p3.fr (134.158.104.144)
mtu limitted to 1500 bytes at FLORA01.SLAC.Stanford.EDU (134.79.16.29)
doing 32 probes at each of 64 to 1500 by 44
0 FLORA01.SLAC.Stanford.EDU (134.79.16.29)
| 24 Mb/s, 222 us (0.94 ms)
1 RTR-CORE1.SLAC.Stanford.EDU (134.79.19.2)
| 105 Mb/s, 178 us (1.41 ms)
2 RTR-CGB6.SLAC.Stanford.EDU (134.79.135.6)
| 83 Mb/s, 51 us (1.65 ms)
3 RTR-DMZ.SLAC.Stanford.EDU (134.79.111.4)
| 145 Mb/s, -69 us (1.60 ms)
4 ESNET-A-GATEWAY.SLAC.Stanford.EDU (192.68.191.18)
-> 192.68.191.18 (1)
| 23 Mb/s, 28.0 ms (58.1 ms)
5?chicago1-atms.es.net (134.55.24.17)
| 33 Mb/s, 563 us (59.6 ms)
6 206.220.243.32 (206.220.243.32)
-> 206.220.243.32 (3)
| 27 Mb/s, 55.2 ms (171 ms)
7?cernh9-s5-0.cern.ch (192.65.184.142)
| 31 Mb/s, 398 us (172 ms)
8 in2p3-cernh9.cern.ch (192.65.184.46)
| 26 Mb/s, 1.48 ms (175 ms)
9 192.70.69.29 (192.70.69.29)
-> 192.70.69.29 (1)
| 117 Mb/s, -9 us (175 ms)
10?Lyon-ANDA.in2p3.fr (134.158.240.1)
-> 134.158.240.1 (1)
| 61 Mb/s, -133 us (175 ms)
11?ccobsn04.in2p3.fr (134.158.104.144)
11 hops, rtt 172 ms (175 ms), bottleneck 23 Mb/s, pipe 510907 bytes
This indicates that the bottleneck is about 23Mbps, which is well above the
measured 6Mbps bbftp thruput.