DAQ and Monitoring Procedure

Basic procedure

• The desk for the shifter is in the central lab, R249, in the grad office, extension 2016.
• Run the monitoring as described here.
• Keep an eye on:
• the hodoscope hit map and hit probability
• the start counter hit probability (should be 20-25%)
• the trigger rate (should be 9Hz or higher)
• Call MCC if the trigger rate is below 10Hz for an unusually long time or if the hit probabilities get too high or too low, indicating a problem with the beam.
• End runs and start runs when necessary, as required by change of conditions or time expired. The procedure for manual start/stop of runs is described here
• Keep a detailed log of any and all activities in the testbeam logbook at http://kminus.slac.stanford.edu/cgi-bin/dircrd/logbooktestbeamelogbook.pl
• At the end of your shift write a brief shift summary.
• Layout, specs of the detectors and a run plan are sumamrized in Jerry's "How-to" document
• In case of problems call or page the relevant expert. The numbers are listed here

Simple checks

• Is the DAQ running? What is the file name?
  Check the NewDAQWobbler script - is it updating?
  Or, on the dircrd/dircrd-priv desktop type ls -latr /u2/data | tail and see what the modification date on the most recent file is.
• I want to stop a run but it is owned by someone else.
  On the dircrd-priv desktop type ps -elafww | grep readTestBeam. Note the PID of the process (the number behind the user name of the process owner) and kill it with the command sudo kill -9 process.pid. Remember to gzip the .dat file so that the copy robot process can pick up the file and create the root file.
• Is the automatic DAQ job submitter script running?
  On the dircrd-priv desktop type ps -elafww | grep run.pl. If you want to kill it, note the PID of the process and type sudo kill -9 process.pid

There's a problem - what do I do?

• I cannot log in on kosmic/ksizero because someone locked the screen.
  Switch to a console (press Control+Alt+F3) and log in. Kill the other person's sessions by typing
  sudo -s
killall kdm
  However, this will not kill the other person's DAQ run and may leave messages in the queue. To stop the run or clean the message queue, see the documentation.
• Something fishy is going on with the monitoring. Is it real or an artifact of the mirroring/wobbler/analysis software?
  When there seems to be a problem with the data in online monitoring and/or on the web monitoring, make sure you check the actual data file on dircrd/dircrd-priv before panicking.
  On the dircrd/dircrd-priv desktop type ls -latr | tail. Then type DAQWobbler.sh latest.data.file. Do all TDCs and ADCs look normal or fishy?
  If the problem is also visible here, try to end the run and start a new run. If that doesn't fix it, it's finally time to panic - call the appropriate expert.

Last significant update: Mar 31, 2008,	by Joe