BdbServer 
A Data Distribution Server

The server itself runs under the objysrv account, this avoids any privilege problem.

The server is activated by a cron job running every 15 minutes. At the moment it is only capable to run one request at a time, so please be patient (a multi-threaded version is foreseen in the future).
 

BdbServer Commands

• Each line of the mail will be composed of a single word command and a list of operands
• The order does not matter
• A command can be split in different lines by repeating the command name

• mailto username@node - Mail address where the mails will be sent (by default reply to the sender)
• federation federation name (analboot2 or sp3analboot) - Select the federation
• dbid list of database ID - Will extract all the databases corresponding to the list of ID
• extract Sweep_directory_name/date - Trigger a new sweep export. At the momment this command is used to export Objy data to in2p3, it will be extended to cope with exports to other location (command reserved for central Data Distribution team)
  
  The Sweep_directory_name can be one of the following:
• analboot2
• sp3analboot
• Skims
• Skims-SP
• clean Sweep_directory_name/date - Clean disk space when an export has been imported (restricted command)
• CollExport List of Collections - Export the databases corresponding to one or more collections
• DataType Type of Data - In conjunction with "CollExport", restricts the export to given set of data types.
  
  At the momment the following DataTypes are allowed:
• tag
• micro
• mini
• rec
• raw
• Any combination allowed by BdbCollExport, for instance: raw:tag:aod

The BdbServer/Results directory will be cleaned-up on a regular basis (not yet implemented) and when room should be made on the export disk.

 
This server is still experimental, suggestions are welcomed - Please report any problems
 

Technicalities :

The server is using the directory /objy/databases/export2/BdbServer on datamove3.slac.stanford.edu as a working space.

     Requests : contains the request files
     Run : is used by the server while it is processing a request
     Results : contains the extracted databases, the tdf file and a copy of the request
     Logs : contains the server log-file : job.log

A special file BdbServer.lock is created while the server is running. It prevents another instance of the server to run at the same time. The lock file is deleted when a request has been processed.

The server is using 3 perl scripts currently maintained in ~boutigny/DataExport, they will eventually be committed to CVS :
 

1. CheckMail.pl is run by a cron job each 10 minutes. It checks the content of the file /var/mail/bbrdist and analyze its content. Then it builds a request file and put it in /objy/databases/export2/BdbServer/Requests. The request file is named with the mail-id of the requestor appended with a number in case we have more than one request.
2. BdbServer.pl is the server itself, it is run by a trscron job each 15 minutes. It is doing the following:
• Create a lock file to prevent another instance of BdbServer to run
• Check if a new request file is created and process it
• Check if the federation is available
• Move the request file to the Run directory
• Process the request file
• Move the files from the Run to the Results directory
• Send an E-mail to the requestor
• Delete the lock file
3. BuildTdf.pl is an auxilliary script used with the "dbid" command, it compares the list of DBID to the federation catalog and build the corresponding tdf file