This page is still under construction. The old documentation, despite being rather out of date, may still be of some help.

Importing BaBar Data to a Local Site

This page describes the procedures for importing data from SLAC to your local site, as well as a few hints for managing the files on your local disk. It assumes that you have installed a local bookkeeping database. It is also possible to use the SLAC bookkeeping database to manage the import (not documented here), but without any record of which collections are available locally, so this option is only suitable for small sites which make one-off transfers.

There are many ways of selecting which collections to import. Here we outline a fairly general procedure that should simplify a setup where you wish to import data as is becomes available at SLAC (eg. from skimming).

In summary, we select which datasets are required, and use that to select files. The selected files are imported to local disk, prepared for analysis use (eg. inserted into xrootd), and then the collection is marked as local in the bookkeeping.

Summary

Selecting Datasets

BbkUser --site=MY_SITE --dbuser=bfactory --dbname=bbkr18 \
        --ds_name=A0-Run6-OnPeak-R22d --set ds_is_local=1

BbkUser --site=MY_SITE --dbuser=bfactory --dbname=bbkr18 \
        --ds_is_local=0 --ds_type=PRskims,SPskims -i ds_skim=local_skims.lis \
        ds_name --set ds_is_local=1

Datasets can be selected by ds_name, ds_type, ds_skim (or ds_stream), ds_on_peak, ds_run_cycle, ds_skim_cycle, ds_modenum, or ds_is_signal (run BbkUser --list-columns ds for a full list). Most of these can be specified as a list (like the --ds_type selection above) with wildcards (eg. --ds_name='A0-Run*-R22d', though the same selection could be obtained with --ds_type=PRskims --ds_skim=A0 --ds_skim_cycle=R22d, which is faster and probably more robust).

Selecting Files

BbkUser --site=MY_SITE --dbuser=bfactory --dbname=bbkr18 \
        --file_status=2 --ds_is_local=1 --components=E --set file_status=-1

BbkUser --site=MY_SITE --dbuser=bfactory --dbname=bbkr18 \
        --file_status=2 --ds_is_local=1 --components=^E --set file_status=1M \
        dse_type skim dse_modenum components \
        dse_id dse_lumi events file_id gbytes --summary

Most analysis users don't require the mini information, so we mark files with only mini information (--components=E) with --set file_status=-1. NB. it is important not to leave the mini files with status 2 here, otherwise the system won't later know to ignore these files when determining whether the collection is complete. If the mini files are required (eg. for PR skimming (not SP), or Event Display detail), then just leave out this line and remove the --components=^E on the second command.

The remaining files (with header, tag bit, and micro information) in our datasets are selected with the second command. It uses --set file_status=1M. Files with statuses beginning with 1 are flagged for import, with the second character specifying the priority (A=top, Z=bottom).

This BbkUser command also prints a summary of the files selected for the cron job's log file.

File Status Codes

file_status values
2	Initial status - file disposition to be determined
1	to import
1A	to import with top priority
...
1Z	to import with lowest priority
0	on disk (or at least availably locally)
-1	don't import

but you can also use other values for other purposes, eg.

Other suggested uses of file_status
3	file imported - to be checked
4	file imported OK - to be swept into xrootd system
0T	file is on tape

Any file_status value starting with 0 indicates that the file is local.

Importing

Importing with Multiple Processes

Importing to Multiple Disks