skimImport V1.7 - Kanga import using scp or sfcp or bbftp Usage: skimImport [OPTIONS] [[[user@]node:]root-source [root-dest]] Transfers files flagged for import in the skimData database, and updates the database to make the files visible to the skimData user. Defaults: skimImport --cipher=blowfish --copy --ftp-type=scp --local-program=scp --multiple=1 --retry-interval=240 --sort=priority,jobid,streamid --timestamps --update-sql tersk.slac.stanford.edu:/afs/slac.stanford.edu/g/babar/kanga/EventStore/groups /localdisk/kanga/EventStore/groups OPTIONS are -? --help Show this help -V --version Show version information -v --verbose Verbose running of skimImport and scp/sfcp/bbftp (specify multiple times for more detail) --quiet Quiet running of skimImport --test Test - do not actually run scp/sfcp/bbftp --node NODE Remote node name (or user@node) --remote-user USER Remote user name --include SPEC Include only files that match the specified wildcard. Wildcards are *, ?, and **; trailing / for root dir; no / or ** does filename match. --exclude SPEC Exclude files that match the specified wildcard --filelist FILE Only import files listed in FILE (must also be in DB) --xfilelist FILE Don't import files listed in FILE --priority X Select only imports of priority X (X=A-D) --config FILE File listing where to make symlinks --noupdate-sql Don't update SQL database with new status --nocopy Don't import files - just check for files that are already here --sort [-]TYPE[,[-]TYPE...] Order import by: priority/remote, path, lpath (case-blind), jobid/requestid, streamid, size. Use -TYPE for reverse sort. Default=priority,jobid,streamid. --multiple N Copy N files in a single scp/sfcp/bbftp command --maxcopy N Maximum total number of files to copy --split N Maximum files before database rescan --retries N Number of retries on failure (***Retries probably don't yet do what you want***) --retry-interval SEC Interval between retries (default=240 sec) --notimestamps Disable timestamps --freeminimum SIZE Skip writing files to disk with less space (MBytes) --ftp-type NAME Use scp or sfcp or bbftp (default=scp) or SQL database connection options: --host DB_HOST Host where SQL server is run --user DB_USER SQL user name to use --db DB_NAME SQL database to use --driver DRV Driver default --pwdfile FILE File containing user's SQL password or scp options (with --ftp-type=scp): -c --cipher NAME Selects the cipher to use for encrypting the data transfer. This option is directly passed to ssh. -i --identity-file FILE Selects the file from which the identity (private key) for RSA authentication is read. This option is directly passed to ssh. -L --nopriviledge-port Use non privileged port. With this you cannot use rhosts or rsarhosts authentications, but it can be used to bypass some firewalls that dont allow privileged source ports to pass. Same as saying "-o UsePriviledgePort=no" or -P to ssh. -1 --remote-scp1 Force scp to use command "scp1" on the remote side instead of "scp". This may be necessary in some situations, if the remote system has "scp2" symlinked to "scp". -o --ssh-options OPT Options passed to ssh. -p --preserve Preserves modification times, access times, and modes from the original file. --preserve-time Preserves modification times and access times from the original file. --preserve-mode Preserves access modes from the original file. -Q --statistics Turn on statistics display. -B --batch Selects batch mode (prevents asking for passwords or passphrases). -C --compress Compression enable. Passes the -C flag to ssh to enable compression. -P --port N Specifies the port to connect to on the remote host. -S --ssh-command PATH Specifies the path to ssh program. or sfcp options (with --ftp-type=sfcp): -i --identity-file FILE Specifies ssh identity file --ssh-flag FLAG Specifies other ssh options (can be repeated) -b --diskbuffer BYTES Specifies the disk I/O buffer size in bytes (default=262144, i.e. 256K) -d --nbuffers N Specifies the number of disk I/O buffers (default=nstreams+1) -e --execute --remote-program FILE Specifies the locations of the ooss_fget_mt program at the remote site. -h --ssh-command CMD ssh command use to contact remote host (default=ssh) -p --port N Service port number -s --nstreams N Specifies the number of TCP/IP streams to use for the copy operation (default=3) -S --bind-retry-delay SECS Delay between bind attempts (seconds) -t --socketbuffer BYTES Specifies the socket buffer size (i.e. TCP/IP window size; default=524288, i.e. 512K). -T --bind-retries N Maximum number of bind attempts or bbftp options (with --ftp-type=bbftp): -a --use-authfile Use bbftpc with auth file /tmp/adye/user@node.bbm -A --authfile FILE Use bbftpc with the specified auth file --ssh-flag FLAG Specifies other ssh options (can be repeated) --dir-local PATH Directory where to find bbftp/bbftpc -b --background Run bbftp in background after the password request -c --compress Compress the data during transmission. Compression and uncompression are done "on the fly" and do not require any additionnal disk space. Do not use it if the files to transmit are not compressible, as this will only lead to a waste of CPU and time. -E --execute --remote-program FILE Specifies the location of the bbftpd program at the remote site. -i -I --identity-file FILE Specifies ssh identity file -L --ssh-command CMD ssh command use to contact remote host (default=ssh) -l --localcos N Local file class of service (not used by skimImport) -m --magic-number N Use bbftpc with magic number -o --outputfile FILE Write logfile to FILE (default=stdout) -p --nstreams N Number of parallel streams (default=1, maximum=10) -r --bbftp-retries N Number of retries on transfer failure (default=5) --nouse-ssh Don't use ssh for authentication and startup -w --controlport N Control port of bbftpd Author: skimImport was written by Tim Adye . It uses SQL access routines written by Alessandra Forti . skimImport is a controlling script for any of the following file transfer programs:- scp, by Timo Rinne and Tatu Ylonen , is part of the ssh1 distribution from http://www.ssh.com/products/ssh/ sfcp by Artem Turnov and Andrew Hanushesvky http://www.slac.stanford.edu/~abh/sfcp/ bbftp by Gilles Farrache http://ccweb.in2p3.fr/bbftp/ Version: skimImport V1.7 (Id: skimImport,v 1.10 2001/03/09 16:26:26 adye Exp ) ImportUtils.pm V1.7 (Id: ImportUtils.pm,v 1.10 2001/03/09 16:26:14 adye Exp ) ImportUtils_scp.pm V1.0 (Id: ImportUtils_scp.pm,v 1.2 2001/02/22 14:47:27 adye Exp ) SSH Version 1.2.21 [sparc-sun-solaris2.5.1], protocol version 1.5. Standard version. Does not use RSAREF. ImportUtils_sfcp.pm V1.3 (Id: ImportUtils_sfcp.pm,v 1.8 2001/02/22 14:47:30 adye Exp ) /afs/rl.ac.uk/user/a/adye/packages/ftp-tng/ooss_fget_mt Version: 2.5 ImportUtils_bbftp.pm V1.3 (Id: ImportUtils_bbftp.pm,v 1.8 2001/02/22 14:47:24 adye Exp ) bbftp version 1.9.4-tja1 using default port 4021 compiled with 5 tries