Commands used to "stage" data to and from Silo

1. Simple way of getting data from Silo to disk --

   Use the strun command found in $E154_COM directory:

   strun -s SOURCE RUN

   where:
   SOURCE = "raw" for raw data, "dst1" for DST1, etc
   RUN = run number, eg 1337
   

   After the data have been copied to staging disk, a symbolic link will be created in your current working directory. Its name will be r RUN.SOURCE, eg r1337.raw

2. A more masochistic way of getting data from Silo to disk --

   The E154 data will be all on Silo tapes, and there will be a catalog of tape number, file number, run number and data type (e.g. raw, DST, MC). You need to move the data to a disk file before using it. The command to do this is:

   stagein -VTAPENAME -qFILENUM -FU -L32768 -sMMM FN

   where:
   TAPENAME = Silo tapename, eg qc0100
   FILENUM = file number on tape, eg 1
   MMM = maximum disk file size in MB, eg 300
   FN = your filename for referring to this data

   Please note that it is -FU and not -FF as in E143. You will not be able to read the data properly if you get it wrong! There is a catalog of tapes in the Silo with the approximate size of each file.

   Data will be copied from tape to the staging disk, and a link will be set up from FN to the disk file. This command may take some time to execute, so it's probably a good idea to put it in background. If the data is already on disk (eg from an earlier stagein command by you or somebody else), no copying is needed and a link will be set up immediately. Data will stay on disk until the staging system determines that there is insufficient space for new requests. You should always issue the stagein command for every job, even though the disk files have often stayed on disk for many days at a time.

   If the actual data exceeds MMM MB in size, transfer will proceed until the limit is reached. If the disk file exists, but is smaller than your request and is not the complete tape file, it will be replaced. If the actual data is less than MMM MB in size, the unused space will be automatically released.

   Since each tape contains no more than 1 GB, specifying 1000 MB should be sufficient in all cases. However, this is probably not a great idea. Not knowing how much will really be used, the staging system will dutifully find 1 GB of space even if it means purging other disk files, including files that you might want to use in the near future. You should look up in the catalog how much space is required for this run.

   The filename FN may point to any disk or partition, even one with practically no free space (eg your home directory), because stagein does not actually put the data there. It will simply contruct a symbolic link from FN to wherever the file really is on the staging disk. When stagein returns, you can direct the analysis program to use FN as input.

   There is/will be a command which integrates stagein with the analysis program.

3. There is usually no need to get rid of a staged disk file manually. When space runs low, the system will remove stale staged files. However, there are occasions where it is important to remove the file explicitly, e.g. if the tape file has changed since the old file was staged in. The command is
   

   stageclr -VTAPENAME -qFILENUM


   Note that you can stageclr files originally staged in by anyone-- be careful what you type. Please do not use the standard Unix command rm for this purpose. The space freed up may not be known to the staging system.

4. Reserving staging space for writing

   This is useful when making DST, or writing out a large ntuple. Note that the staging system may delete your file without warning when space runs low. Do not use this space for anything you cannot afford to lose.

   stageout -VTAPENAME -qFILENUM -lsl -FU -L32768 -sMMM -poutput FN ...

   where:
   TAPENAME = (arbitrary) tapename, e.g. FOO
   FILENUM = (arbitrary) file number, eg 1
   MMM = space to be reserved in MB, eg 300
   FN = filename associated with it, e.g. BAR

   When this command returns successfully, it will have created a link from FN to an actual disk location. You can then simply write to FN. If you do not use up all the reserved space, it is not released until you say:

   stageupdc -VTAPENAME -qFILENUM

   If you try to write more than the reserved space and if there is indeed more space available on that physical disk, it will continue to write. Unfortunately, the staging manager does not know about it, and subsequent space allocation may be affected.