Tips&Tricks for ATLAS at SLAC


Transferring large files from CERN to SLAC (or SLAC to CERN, etc.)

See the relevant discussion on hypernews.

bbcp is much faster than scp (~15 MB/s compare to ~0.3 MB/s). 

bbcp is not installed at CERN. Copy it from SLAC (/usr/local/bin/bbcp) in your
own CERN area (in my xrootd directory in the following examples).
Note, you can find the Mac binary at
/afs/slac.stanford.edu/package/bbcp/prod/bin/ppc_darwin_80/bbcp

To transfer a single file from CERN to SLAC:
~/xrootd/bbcp -s 64 -P 2 -f notes.txt grenier@noric09.slac.stanford.edu:.

To transfer a whole directory:
~/xrootd/bbcp -s 64 -P 2 -f -r data grenier@noric09.slac.stanford.edu:atlasdata

To transfer from SLAC to CERN:
~/xrootd/bbcp -z -T /usr/local/bin/bbcp -s 64 -P 2 -f grenier@noric09.slac.stanford.edu:notes.txt notes.txt

Some notes from Wei:
> 1. bbcp uses ssh. So one can avoid typing password by adding your CERN ssh public key to SLAC's authorized key file (~/.ssh/authorized_keys)
> 2. I am using 64 streams, which is the maximum.
> 3. -P 2 is not necessary unless you want to see the process bar.

Another note from Gary Buhrmaster, regarding the number of streams:

> The number of "optimal" streams varies, and is influenced
> by the size of the tcp buffers. More streams can actually
> hurt performance. There are some tuning suggestions
> available at:

> http://pcbunn.cithep.caltech.edu/bbcp/using_bbcp.htm

> Usually the default number of streams of ~4 is
> reasonable, but you want to increase the window
> size (i.e. eliminate the '-s 64' and instead
> add in '-w 2M'). As always, your results will
> vary.


Last edited by Andy Haas on Dec. 15, 2008