The Liaison Shift Manual

(14 January 03)
(TH, 28 April 03, added two links for Jerry's diodes)

The Liaison Shift Manual is a work in progress and at times may not be completely up-to-date.
The on-line version is generally more up-to-date than any printed version.

Comments welcome: hertzbac@slac.stanford.edu
Quick Index:
The Job: general, to-do list, admin. tasks, training.
Connections: Phone/pager numbers, Further useful links.
Workplace: practical details, Terminals, Starting from scratch. standard setup,
Sensors: SVT diodes, EMC & DCH diodes, trigger, DCH current, DIRC scalers & diodes, IFR current, BLMs.
Tools: EPICS, Liaison Alarm Handler, StripTool, e-logbook, ambient db, XV, OPR plots, MCC SCP, PEP-II StripTool, MCCERRLOG.
BaBar & PEP Channel Names in EPICS & Stripcharts


This manual can be found at /BFROOT/www/Detector/Backgrounds/liaisons/manual/.
The old manual is still available at /BFROOT/www/Detector/Backgrounds/liaisons/shift_manual.html.

How to print this guide. (we would very strongly prefer that you read this document online) 

CONTENTS

******************************************************
****         Diode calibration is now done automatically!         ****
**** Please ignore any references to calibrating the diodes. ****
******************************************************


Your Job

Liaison responsibilities

General job description
Concrete list of things to do
Administrative tasks
Liaison training info
.

Your Connections

Be informed, keep in touch!

Phone/pager numbers
Further useful links
.

Your Workplace

The liaison desk at MCC

Practical details
Computer terminals
Starting from scratch
The standard setup and how to get there
.

Your Senses

Background sensors in BaBar and PEP-II

radiation protection/
beam abort
SVT abort diodes
EMC and DCH sensors
crucial for BaBar operation
(EPICS alarms)
Trigger rates
Drift chamber current
DIRC scalers

additional sensors
SVT monitoring diodes
Jerry's DIRC diodes
IFR currents (forward layer 18)
PEP-II beam loss monitors
PEP-II beams
.

Your Tools

Monitoring and archiving software

online: BaBar EPICS
Alarm Handler
StripTool
offline: Electronic Logbook
BaBar ambient database
SVT diode analysis tools
other software: XV, OPR plots
.

PEP Tools

PEP Tools for Liaison Use

MCC SCP
SCP history buffer
PEP-II StripTool
MCC ERRLOG
.

Channel Names

BaBar & PEP Channel Names in EPICS & Strip Charts

.

Your Job - Liaison responsibilities

General job description

Your two main priorities are to:

Maximizing integrated luminosity

We must get as much data on tape during each shift as we possibly can. As a liaison, you have significant impact on the efficiency of the B-factory, mainly in the following ways:
  1. As the communication link between IR-2 and MCC: You are the BaBar's representation at MCC. You may not realize this instantly, but your physical presence is actually appreciated by the operators. If they have a question or a problem related to BaBar, it is much easier and more convenient for them to just walk up to the liaison desk and ask you than to call into the anonymity of IR-2. Similarly, it comforts the shift leader to know that he can always talk to a BaBarian at MCC, somebody who speaks his language but knows a little more about the accelerator and its current status. (Yes, you may have to make an effort to learn about PEP.) The impact of a frictionless flow of communication on our efficiency should not be underestimated. In the long run, it is clear that we can only profit from the flow of knowledge between BaBar and PEP-II if the knowledge flows in both directions.
  2. As a background radar anticipating or reacting to situations that can hamper BaBar operation: Often we get backgrounds that are well within tolerable radiation limits but, nevertheless, prevent BaBar from running efficiently. Examples are drift chamber current, the L1 trigger rate, and the number of photons seen in the photomultipliers of the DIRC SOB. It is your job to keep an eye on these signals and to alert the operators as soon as they approach the limits. (That does not mean you have to stare at stripcharts all the time, just check the values every now and then, and look for dangerous trends.)
  3. You are in a position to coordinate, on a short term, activities at BaBar and PEP. BaBar should run at 95% to 99% efficiency; the loss of 5 minutes of your shift (1%) is noticeable!

Protecting BaBar

We must protect BaBar from excessive machine radiation, and not allow machine radiation to permanently damage our detector (more than a tolerable limit).
  1. The first level of protection is provided automatically by the beam abort system that is triggered by BaBar's SVT PIN diodes. Your job is to ensure the flawless operation of this system around the clock, i.e. to calibrate it (now automatic!), to check if it performs as expected, and, in case of problems, investigate and, if needed, alert the experts.
  2. Another level of protection is provided during injection by the EMC PIN diodes, which limit the rate of injection if the radiation level at the EMC is too high. Your job is to check that the system is working correctly.
  3. You can help understand and mitigate backgrounds: Sitting at the liaison desk, you are in an ideal position to spot interesting phenomena, unexpected correlations between signals, etc., as they occur. A background expert can always retrieve almost all the information you see online from history buffers, but there just isn't enough time to look at all that data continuously in a systematic way. Thus the background folks rely on you, the liaison, to find and point out patterns in background signals.

Concrete list of things to do on Liaison Shift

Administrative tasks

Meetings:

Each Liaison should arrive at least 15 minutes before the hour, and attend the shift-change meeting where the outgoing PEP EOIC (Engineering Operator In Charge) reviews the past 2 shifts with the incoming PEP shift crew. This is held in the MCC meeting room before the owl shift, and in the next building for the others.

If the situation allows, the Liaison should be present at the MCC morning meeting at 08:00. Similarly, swing shift Liaison should try to attend the shift-change meeting at 23:45.

Obviously, shifters should pass on any relevant information to their immediate successors. This includes the next IR-2 shift when there is no Liaison following you.

Hypernews:

For information that is relevant for several shifts to come we have a dedicated Liaison hypernews. Be sure to check the postings of the last two months or so before starting a shift block. This is also the place where you would ask any questions, make suggestions and complain about issues related to liaison work.

The minutes of the daily IR-2 BaBar Operations Meeting are posted in the Detector Operations hypernews. Read this daily before and during your shift block, so you are aware of the current concerns in IR-2.

When you write a hypernews posting from the liaison desk, please make sure you are using your Netscape login so that the posting will appear under your name. It is annoying for the previous shifter to receive postings written by somebody else under his name, and wrong names make it difficult to search for a posting later.

Your Connections

Be informed, stay in touch!

Phone / pager numbers

Further useful general links

Note: for system-specific links (like SVTRAD information), go to the respective system.

Your Workplace

The liaison desk at MCC

Practical details

Computer terminals

There are two computer terminals at the liaison desk in MCC, both of which are Sun workstations on the IR-2 subnet. You can run EPICS, Netscape, the MCC SCP, and xterms for your email, etc. on either. The Workspace Manager bar at the bottom of the screen has icons to start Netscape, XV, and EPICS. NOTE: Because of the secure logon, these workstations can access off-site url's only if you ssh to an scs machine, such as flora, and run Netscape there.

The names of the machines are bbr-con13 and bbrmcc, and your username is babarpep. These are normal Sun workstations as used in IR-2 and many other places at SLAC.. Unlike most Sun Ultra5 workstations BaBar has, these can run in TrueColor mode. The liaison Sun workstations have graphics cards which have largely solved the colormap problem. However, Netscape, if run improperly, is by far the worst colormap hog; to start Netscape, just click on the Netscape icon in the bar at the bottom of the screen.

The window manager on the Sun workstations (and IR-2 workstations) only allows you to place each window in either a single workspace (open circle on the upper window-bar) or in all workspaces (closed circle, called "sticky"). Click on the circle to change it, or click on the small bar in the upper left corner of the window to open the menu that also allows you to "send" the window to another workspace. If you want a stripchart to occupy two workspaces on the Sun, you will have to fire up a second copy. (A future upgrade will provide more flexibility.)

Starting from scratch

If the set of liaison shift EPICS panels is not already in place when you arrive, or there is some other problem with the terminal, you should execute the following procedure step by step to start from scratch.

Standard setup and how to get there

There is a recommended way of distributing various EPICS panels and strip charts among the workspaces on the liaison terminals. Of course, background experts and experienced liaisons will have their own preferences but for the beginner liaisons it may be a good idea to stick to this setup so they will easily find things again when returning on shift. Generally it is easier to navigate if your windows are logically distributed among the workspaces, rather than all jammed into one.

The recommended set-up and procedure are the following:

If your predecessor liaison leaves you with a muddle of windows, we strongly recommend that you restore the standard set-up. If needed, you can quit EPICS from the main ODC panel, but leave stripcharts running to retain the history, although you should close extraneous stripcharts. If you need help setting up windows, see starting from scratch. If you simply keep accumulating new windows and strip charts, and wait for your successor to clean up the mess, either the workstation or the host all these windows run on will conk out on you and you will find yourself doing the starting-from-scratch thing anyway, but possibly at a most inconvenient time. If it is really messed up, you may have to reboot.

It is a good general practice to close EPICS windows and StripTools you no longer need. Be especially careful about starting more than one Alarm Handler because each alh process opens hundreds of EPICS connections, which drains IOC resources (DCH-MON and CEN-MON have been the most sensitive in the past). Also, you really do not want more than one beeping at you.

Your Senses

Background sensors in BaBar and PEP-II

Excessive backgrounds in any BaBar detector can affect the quality of the data. The only BaBar system that can actually abort the beam is the SVTRAD that protects the SVT. In addition, SVTRAD and the EMC diodes can limit the rate of beam injection in order to protect the detector.
 
Summary Table Normal range
taking
data
Trip/alarm 
limit
Maintenance
required
by liaison
Strip
Chart
Person
(most)
responsible
Radiation
protection
=>
beam abort/
injection inhibit
SVT abort
diodes
< 50 mR/s depending
on diode/
timespan etc.
calibration
AUTOMATIC!
SVT SVT Expert on-call 
page 849-9407 
cell 255-4190 
Radiation
protection
=>
injection inhibit
EMC
diodes
0.2 V, 
higher
during
injection
0.7 V  none BKG EMC Commissioner 
page 849-9105 
cell 255-4189 
BaBar
operation/
efficiency
=> EPICS
alarms/
warnings
L1 trigger rate
L1 dead time
1.5 kHz
0.4%
(at 4.0E33)
2 kHz 
5% 
none FCT Valerie Halyo
page 940-8277
cell 743-7062
DCH
current
200 - 300 µA typically
~1000 µA
none BKG DCH Shifter
page 846-0981
cell 255-4188
BKG
trigger rates
1M and 1B
25 kHz
50/injection
50 kHz
200/injection
none none Inform the
Pilot
DIRC
scaler rates
< 150 kHz 200 kHz (warn.)
300 kHz (alm.)
none BKG DIRC Shifter
page 846-0545
cell 387-8437
addi-
tional
sensors
SVT monitor.
diodes
< 5 mR/s none SVT
monitor
SVT Expert on-call 
page 849-9407 
cell 255-4190 
SVT diamonds < 50 mR/s - none SVT SVT Expert on-call 
page 849-9407 
cell 255-4190
DCH 
diodes
0.3 V 1.0 V none BKG DCH Shifter
page 846-0981
cell 255-4188
Jerry's DIRC
diodes
0.3 mR/s none BKG Jerry Va'vra
926-2658
IFR
currents
< 500 µA 1000 µA none BKG IFR shifter
page 849-9117
cell 255-4191
PEP-II
BLMs
arbitr.
scales
none MCC

color code:


red: watch out! this stuff can abort beams or prevent injection!
orange: careful! this stuff won't kill the beams but can prevent BaBar from taking useful data
yellow: relax and have fun watching these signals

Strip Charts:

SVT, BKG, FCT are the three standard strip charts accessed from the BaBar EPICS main background panel (BaBar Online Detector Control -> BKG)

The SVT monitor strip chart is accessed via BaBar Online Detector Control -> SVT -> Background => History plots -> Monitoring Diodes Dose Rates
(where "->" stands for normal click, "=>" stands for right-click and drag down; Windows and Macintosh users are reminded that EPICS and almost all other UNIX applications are navigated by single, not double, mouse clicks).

General note about sensors and signals

As a liaison, you are dealing with two pretty much independent online control systems: the EPICS of BaBar, and the SCP/EPICS of PEP-II. Every PEP-II EPICS channel is available in BaBar EPICS by prefixing it with "ACC:".

Example: The DCH diode signal is read out via PEP-II hardware and goes into (PEP-II) EPICS channel PR00:ASTS:DCH_TOTL. It is accesible in BaBar EPICS as ACC:PR00:ASTS:DCH_TOTL.
On the other hand, not every BaBar channel is available in PEP-II EPICS. Since the machine guys want to be able to correlate machine parameters with background signals, some of BaBar channels relevant to backgrounds have been made available to SCP and PEP-II EPICS. Therefore, many background signals have two channel names, one for access via a BaBar stripchart, and a second one for access from a PEP-II stripchart. As a liaison you have full access to BaBar EPICS, and therefore to all channels. Nevertheless, it is sometimes useful to know the PEP-II names of signals. In the following we give both names (where both exist).

SVT PIN Diodes

[Recent Changes | Known Problems]
Primary Documentation: SVTRAD Users Guide
Plots: Daily SVTRAD Plots
FastHistory Plots
Further Documentation: SoftInjection system overview
Troubleshooting Information
Tim Meyer's original user guide
SVT expert operational information and tools
SVTRAD design, history, and status
On-call Experts: SVT Commissioner, (650) 849-9407 (pager)
SVTRAD expert, Shane Curry, (650) 846-0842 (pager) 

Please see the SVTRAD Users Guide for more information on operating the SVTRAD system.

System Description

The SVT uses 12 PIN diodes to monitor background radiation conditions near the interaction region. The complete system of diodes, readout electronics and monitoring software is known as SVTRAD. Recently (Summer 2002) two diamond radiation sensors were also installed in the backward regiond of the SVT.

The SVTRAD system serves three important functions:

  1. Protect the SVT and the BaBar detector from instantaneous background radiation on short time-scales (microseconds)

  2. (hardware ABORT system)
  3. Enforce the long-term radiation budget of the SVT

  4. (SoftAbort and SoftInjection systems)
  5. Monitor short (minutes) and long (months) time-scale background changes and assist the PEP operators tuning the machine

  6. (Monitoring system)


SVTRAD is actually two independent systems which utilize the same 12 sensors:
The physical locations of the 12 PIN diodes are shown in this CAD drawing. The two diamonds are sitting about 5 cm further away from IP than the BW:MID and BE:MID diodes and also in the horizontal plane.

  1. Hardware Abort system
  2. Monitoring system
The SoftAbort and SoftInjection systems are implemented on top of the Monitoring functionality. (In other words, they are software based.)

Instructions for Liaison

  1. Calibrate the hardware abort system once per day. (AUTOMATIC!) The ABORT calibration takes about 5 min and requires that there is no beam in the machine. The best opportunity for the ABORT calibration is during a global calibration of BABAR, which is also done once per day. If it has been more than 30 hours since the last calibration, ask MCC for permission to do it after the next beam loss.
  2. Instructions for performing the ABORT Calibration (when there are no beams!)
                  What should happen: after ~10 seconds, all 4 SVT MID plane diodes should register red ALM on the Background panel. The Calibrate ABORT threshold panel should agree by setting the CFG_TRIP lights red. An xterm window should pop up and begin spewing information, and then pauses. After a few seconds, it should print the status of each of the 4 diode's calibration tasks when completed. Then all of the CFG_TRIP lights should have turned green again.
  3. Watch the background rates for the 4 midplane diodes and the 8 diodes out of the midplane. There are two strip charts you should have displayed at all times: SVT Monitoring Diodes Dose Rates and SVT ABORT Diode Averages.
  4. Try to correlate background spikes among the various diodes with each other, the diamonds  and with other background signals in BaBar. The diamonds are shown in the strip charts as SVT:C1:DM:MID_RINST (BW diamond) and SVT:C2:DM:MID_RINST (BE diamond) - these should have a high correlation with the BW:MID and BE:MID diodes. Warn the PEP operators if there is a danger of the SoftAbort system tripping the beams.
  5. Understand any beam aborts caused by the SVTRAD system.
  6. Page the SVT Commissioner for any concerns, strange behavior which is not a known problem, or problems which cannot be solved.

EMC and DCH CsI-diode sensors

This section contains criteria for bypassing the EMC diode injection inhibit. (updated 04 OCT 2001)

The CsI crystals in the EMC are subject to radiation damage over time, and crystals in the (forward) endcap, especially those nearest the beam pipe are expected to receive the largest radiation dose. There are four sensors around the beam pipe near the calorimeter endcap (EMC), which provide a measure of the radiation exposure of the EMC endcap.

Similarly, there are four sensors at the backward end of the drift chamber (DCH) which provide a measure of the radiation exposure of the readout electronics on the drift chamber backward end plate.

Each sensor consists of a small CsI(Tl) crystal read out by a PIN diode (like a mini-calorimeter). The signals of four diodes are added together, so there is one EMC signal and one DCH signal. Read-out is based on Tom Mattison's CBAM module. The output signals are voltages. No effort is made to convert to mR/s because we are looking at the sum of four diodes and don't know much about the uniformity of the signal.

EMC diode thresholds (injection inhibit and pain)

The utilization of these signals has evolved over the lifetime of the experiment. Currently (since midyear 2000?) the EMC diode signal is connected to the injection inhibit chain, but not to the abort chain. The DCH diode signal is monitored, but is not used in either the abort or inhibit chain. .

The EMC and DCH diode signals are in the default BKG stripchart as EMC_TOTL and DCH_TOTL, and are at the bottom of the EPICS BKG panel as PIN DIODES. The values on the BKG panel turn yellow at 0.7 V, and red at 1.0 V, the "pain threshold." If either the EMC or DCH diode signal remains above 1.0 V for a substantial time, you should make a notation in the logbook, inform the PEP operators that the radiation level at the corrsponding location is too high, and ask that they try to reduce it. If they cannot reduce the level, page the EMC or DCH commissioner for advice. 

The EMC diodes inhibit injection when their signal reaches 0.7 V. This has the effect of limiting the rate of injection. (The pain threshold of 1.0 V was previously used as the abort threshold. The system hardware inhibits injection at a fixed 70% of the abort threshold, i.e., 0.7V.) If injection or topoff is significantly delayed by the EMC inhibit, please note this in the logbook.

WHAT TO DO IF DIODE SIGNAL PREVENTS INJECTION:

When the EMC diode signal inhibits injection, it usually just extends the time required, while limiting the instantaneous exposure of the EMC. However, the inhibit can be so persistent that it becomes impossible to inject at all. In this case you will have to determine if the EMC diode system is malfunctioning, or if there is really a problem with the beam. The following is a new procedure, as of October 2001, based on experience and measurements in September 2001.

It is important to realize that the HER can generate radiation at small angles, which is seen by the EMC diodes, but is not seen by any other BaBar background sensors. The EMC endcap is in the forward direction as seen by the HER beam, and this is precisely one type of background the EMC diodes protect against. The absence of signals in other BaBar sensors does not invalidate the information from the EMC diodes. PEP cannot bypass the EMC diode inhibit without also bypassing the SVT inhibit, which we definitely do not want to do. You can determine if the EMC diodes are malfunctioning, and take them out of the inhibit chain by following these instructions.

  1. If there is no beam, or a small beam current, in the machine, and PEP cannot fill because of the injection inhibit, look at the EMC diode signal with NO beam in PEP.
  2. If there is normal beam in the machine, and PEP cannot top off because of the injection inhibit, have them try to inject one beam at a time.
  3. Any time there is a serious problem with injection, so that you have to go through this decision process, make a clear entry in the logbook, describing the severity of the inhibit problem, and the conclusion reached.
  4. As stated above, PEP cannot bypass the EMC diodes without also bypassing the SVT inhibit protection. (A BIG NO-NO!) Here is how you can bypass the EMC inhibit from the BaBar side, if the above procedure concludes there is a problem with the EMC inhibit. It is done in hardware:
  5. Now, inform the EMC Commissioner, and make your logbook entry after returning to MCC. 

    Calibration / pedestal adjustment

    The diodes run with zero bias now, so that leakage current should no longer be an issue. However (for reasons unknown so far) we do observe pedestal drifts on the time-scale of weeks or months. If you notice that a signal more than 0.1 V away from zero when there are no beams in the machine, please notify the expert.

    Contacts

    More detailed information

    Look below for details of channel names in strip charts & panels.
    Detailed guide for liaisons
    The BaBar Radiation Protection System
    (Information provided by Tim Meyer)
    Some older information about the CBAM system (may still be useful to learn more about the CBAM logic).
    (Information provided by Thomas Schietinger)

    Trigger rate and dead time

    Trigger rates are very sensitive to backgrounds, with ~90% of the L1 rate due to background. High backgrounds increase the deadtime and have a severe impact on the detector efficiency.
    The following are readily available to the liaison: It is recommended that you watch the L1 figures, as the raw triggers can be quite sensitive to PEP and detector problems (EMC hot towers, for instance). With the exception of the 1M trigger, the trigger rates are only available when BaBar is ramped up and taking data.

    With a luminosity of 4x1033 cm-2s-1, the typical L1 rate is 1.5 kHz (October 2001). If the L1 rate goes above 2 kHz, and the L3 trigger is running on 32 nodes, the deadtime increases rapidly to an untolerable level, because L3 cannot keep up. At times L3 runs on 60 nodes, in preparation for higher luminosity, in which case the dead time is OK to much higher L1 rates. However, we do not want to use up all the L3 capacity at this luminosity, so Liaison should watch the L1 trigger rate.

    If the L1 trigger rate increases noticeably from one fill or top off to the next, this should be pointed out to the EOIC, even if the dead time is acceptable. Similarly, a step increase in L1 and/or dead time may be due to a trapped dust particle, and should be pointed out to the EOIC. An L1 rate of 2 kHz (at ~4x1033 cm-2s-1), or a dead time of 5%, is a sure sign of a problem.

    More information

    See the Trigger Home Page for an introduction to the BaBar trigger and L1 trigger object definitions.
    Look below for details of channel names in strip charts & panels.
    Detailed guide for liaisons
    Sibylle's trigger background remediation page

    Drift chamber current

    The current drawn by the drift chamber wires most accurately reflects background conditions. This very sensitive background sensor is of course only available when DCH is ramped up. Liaisons are encouraged to watch the total current, as well as current per superlayer and current per quadrant (all these signals are available on the BKG EPICS window; the total DCH HV current, DCH::HV:PS_1:I_TOT.VAL, is available on the standard BaBar BKG strip chart). Note that historically superlayer 6/inner quadrant has been involved in roughly 75% of all DCH HV trips, although the pattern of backgrounds seen by the Drift Chamber varies noticeably with time.

    Trip level

    The drift chamber consists of 44 HV segments (one HV segment per superlayer per quadrant, with the exception of the inner-most superlayer, which consists of two HV segments per quadrant). Whenever the current in any one segment reaches the trip level (currently [February 2001] 30 µA at the working voltage V1 = 1930V), HV to that segment is cut off and the rest of the drift chamber is ramped down to V0 = 800V and has to be brought up again, with obvious consequence for the data taking efficiency. Mechanical stress on the wires that occurs when HV trips also causes irreversible damage to the drift chamber, the possible extent of which has not yet been studied in detail.

    The HV segment superlayer 6/inner quadrant is involved in 75% of all DCH trips. Its current is therefore a good measure of how close to tripping we are. For the total current, the trip level depends, of course, on the shape of the background distribution. During a trip it is typically at 400 - 500 µA.

    Prevention of ramp up

    The DCH HV EPICS software monitors the radiation rates registered by the four MID-plane SVT PIN diodes, and will refuse to ramp up the chamber to V1 if any of them is higher than an individually chosen "pain threshold." The default is for the DCH to use the thresholds for the SVT 10-minute soft abort timer, currently (2002/12/13) 50 mRad/s for FW and BE, 80 mRad/s for FE, and 1000 mRad/s for BW. These are the SVT abort diode monitoring rates displayed in the SvtAbortAverages strip chart. The chamber will not be tripped or ramped down if these levels are exceeded; they just prevent ramping up with high background.

    Channel names

    Look below for details of channel names in strip charts & panels.

    DIRC scaler rates

    One PMT in each of the 12 azimuthal DIRC sectors is connected to a scaler counting the incoming photon rate as a measure of backgrounds and sending this information to BaBar EPICS.

    The signal from sector 0 is part of the standard BKG strip chart. All 12 sectors are represented as dials on the BaBar BKG EPICS panel. It is recommended that you watch these scalers in BaBar EPICS, where the signals directly and accurately give the photon counting rate.

    In addition, sectors 0, 3, 6, and 9 have a second PMT connected to a scaler which is followed by a frequency-to-voltage converter for read-out by PEP-II software (access from SCP). The conversion is about 1 V = 170 kHz and only known to 10-20% accuracy. 

    In the past, we have used DRC scaler rates, along with the EMC background trigger rate (1M) and EMC leakage currents, as a measure of how likely the DCH is to ramp up without tripping. Some day, we may have to start using these signals in this way again.

    EPICS warning and alarm levels

    BaBar EPICS gives a yellow warning for rates exceeding 200 kHz and a red alarm for rates exceeding 300 kHz. These warnings and alarms are to be taken seriously, as the efficiency drops considerably at such rates. Tell the operators that DIRC backgrounds are too high!

    More information

    Look below for details of channel names in strip charts & panels.
    Joe Schwiening's hypernews posting

    Jerry's DIRC diodes

    In February 2000 we installed a set of 7 radiation sensors around the two beam pipes inside the DIRC SOB. The sensors are built after the DCH and EMC sensors (see above), i.e. also consist of CsI(Tl) crystals and PIN diodes. These sensors are best viewed from the dedicated EPICS panel: BaBar ODC -> DRC => SOB/RAD -> CsI-Rad (enjoy the design!) - strip chart also available from there. Signal #2 (usually the highest) is loaded into the standard BaBar BKG strip chart.

    More recently, Jerry installed an arsenal of new sensors (the total is now 20) along the beam pipes in an attempt to elucidate the origin of the so-called "trapped events" and other spikes. Locations of the sensors can change - liaisons need not know about this in detail, but someone will eventually be using them for background studies.

    More information

    Introduction to Jerry's Background Monitor. Short PDF version and long PDF version.
    Look below for details of channel names in strip charts & panels.
    Jerry's Feb 2000 hypernews posting;
    Jerry's Feb 2001 hypernews posting;
    Jerry's another Feb 2001 hypernews posting

    IFR currents (forward endcap, layer 18)

    In March 2000, Jim Johnson has made available to SCP a read-out the IFR currents measured in the outermost layer of the forward IFR endcap. See his hypernews posting for more info.
    The 4A signal is loaded into the standard BaBar BKG strip chart. Not much experience yet with this signal.
    Look below for details of channel names in strip charts & panels.

    PEP-II Beam Quantities

    PEP-II beam variables

    Very often it is convenient to have the luminosity, beam currents, and/or the lifetimes running along in a strip chart for reference. Look below for the channel names of HER, LER, and luminosity variables.

    PEP-II beam loss monitors

    Sometimes these beam loss monitors (BLM) are useful background indicators. For the channel names of Witold's top ten list (actually only nine), look below

    Your Tools

    Monitoring and archiving software

    BaBar EPICS

    EPICS (Experimental Physics and Industrial Control System) is a very user-friendly (at least for the end-user!) online control and monitoring program. Its windows and point-and-click properties are pretty much self-explanatory. For more info on how to navigate it, go to this page.

    How to start EPICS

    Simply click the yellow "EPICS" icon on the bar at the bottom of the screen on the Sun. Make the EPICS main panel occupy all workspaces, so you can pop up EPICS windows everywhere.

    A few things to note:

    Mouse buttons:

    Various panels on various workspaces:

    The window manager on the Sun workstations only allows you to place each window in either all workspaces or a single workspace of your choice.
    See Computer terminals above for more details.

    Color scheme:

    ...and a warning: If an EPICS window is occluded by other windows on your terminal, clicking on an active part of the window (e.g., a button) when clicking to bring the EPICS window to the top, executes the action. Be careful where you click in an EPICS window.

    Printing:

    The EPICS main background panel

    Clicking on BKG on the BaBar main EPICS panel takes you to our main background panel. It gives you a quick overview on the background situation, featuring From this panel you also start the three recommended standard strip charts, BKG, SVT, and FCT. It's a good idea to have this panel always ready on one of the workspaces, appropriately on BkgPanel

    Liaison Alarm Handler

    A dedicated Liaison Background/PEP-II alarm handler has been created to help alert you when something's going on. The Alarm Handler is another standard EPICS tool, and is typically loved and hated by all. The Alarm Handler watches a number of EPICS channels, and will beep and flash when any of them cross into an alarm state. The only way to make the application SHUT UP is for the user to "acknowledge" the alarm by clicking the appropriately flashing little square.

    Starting the Liaison Alarm Handler

    To start the Liaison Alarm Handler, use the YELLOW button on the upper left hand corner of the EPICS main BKG panel labelled Liaison ALH. Once you place the window on your screen, clicking the Liaison button will call up the Main Window. The Main Window is where all the action is -- it shows the monitored channels in an hierarchial fashion. Most of the channel names should be self-explanatory.

    Make the Liaison Alarm Handler occupy all workspaces. Since you will be typing at the workstation with the logbook open, it is suggested that you activate the Alarm Handler on that Sun. (You do not want it beeping at you from both machines.) If you reduce the Alarm Handler window to an icon, and place it in a convenient location, it will be readily available when you need it, but otherwise not in the way. Try placing the icon in a lower corner, next to the workspace bar.

    Using the Liaison Alarm Handler

    For directions on how to make the most of your Alarm Handler, please see the BaBar Care&Feeding Manual section on the alh and the older documentation over here.

    The main contact for the Liaison Alarm Handler (right now) is TIMeyer (meyertim@slac.stanford.edu, page at 570-0173, phone at x2889). Please direct comments, questions, and suggestions to him. The Liaison alarm handler's configuration files in the svt-mon/alh/ application directory as LiaisonBkgs.alhConfig and EMC_DIO.alhConfig

    StripTool

    The software you use to bring up Strip Charts, the monitoring tool.

    Start a strip chart

    There are several ways to bring up a strip chart:

    General StripTool information

    If some (PEP-II related) channels no longer appear on the strip chart, but are still readable in SCP, chances are that the channel access server (the EPICS interface to the MCC database) has a problem and needs to be reset. Do the following: go to any MCC operator and friendly ask her/him the following question: "Could you please warm-slic the channel access server?" If the operator doesn't remember the command, remind him/her that it's warmslcx slccas/restart. (Note: you can tell whether the channel access server is up by looking at the main BaBar EPICS panel under BaBar/PEP-II CommLink and checking that PEP-II ACC is CONNECTED.) 

    Recommended strip charts

    We recommend to have the following three standard strip charts up and running:

    All three strip charts are easily accessible from the BaBar EPICS main background panel (BaBar Online Detector Control -> BKG, upper left corner).

    The contents of the three strip charts change as needs change. A basic set of variables includes:

    This is to be considered a basic set of channels to watch. You are free to carpenter your own personal strip charts according to your preferences, using the channel names given above. (The current background situation may suggest looking closer at some channels than others, etc.) 

    Electronic logbook

    The official, Oracle/web-based BaBar electronic logbook is our primary logbook. Note that you can easily attach any type of file (a captured strip chart for instance) to the e-logbook.

    Make all logbook entries in the Main Shift Logbook.(Liaison had its own logbook page, but it is no longer used.) Write all interesting information in the electronic logbook, for example:

    Most Liaison entries naturally belong under "Beam Conditions," and it is generally assumed that entries there are from the Liaison. If you make an entry elsewhere, preface it with a notation, e.g., [LIA], so that it is obvious that it is from the Liaison. Items of general or lasting interest should also be posted to the Liaison Hypernews forum. It doesn't really matter much where you write your notes, but write them, and be sure to click on "Update" or "Commit Changes" whenever you make an entry!

    The BaBar shift leader is responsible for starting a new entry for each shift. Do not do it yourself. If the BaBar main web server is down for some reason, try logging on to the www-ir2.slac.stanford.edu server instead of www.slac.stanford.edu!

    Liaison assistant script
    In September 2001, Alberto Lusiani wrote a script that keeps track of the PEP-II status, currents, and luminosity. To start this, open an xterm in the logbbook workspace, and type bbStat. You can copy lines from this into the logbook to record beam conditions. Caution: There is a tendency to over-use this facility, resulting in cluttered logbooks that are difficult to follow. Please use this sensibly. If you make multiple entries for a topoff, making them all under one timestamp provides organization and a readable logbook. (For examples, see the logbook entries under Beam Conditions on day shift 22-25 October 2001. Here is 24 October.

    BaBar ambient database

    All the BaBar background signals (and a lot more) are continuously written to the ambient database. It may happen that you want to look at the history of a certain signal (to check, for instance, if the current behaviour is very unusual or not).

    The easiest way to access ambient data is by means of the ODC Browser. You bring it up by clicking on the correspionding button on the BaBar main EPICS panel. In the browser, click the right dot next to the tree at the bottom and there you are. (There is a rumour that this browser will no longer be supported and is going to be replaced by JAS... does anybody know anything more?)

    If you want to dive into a serious offline analysis, you may want to use the very handy OdcNtupleMaker

    SVT diode analysis tools

    The SVT radiation folks maintain their own archiving system, to which you have access. See the SVT PIN diodes detailed guide for more information. 

    Using XV to capture and print windows 

    XV version 3.10a can be used to capture, save and print windows of applications that do not have a functional printing facility. The variable delay before the capture mode is activated, the ability to capture an arbitrary rectangular area of the screen, and the ability to save the pictures on disk in a variety of formats, including GIF and PostScript, make xv a superior alternative to the xwd/xwud/xpr utilities envoked by StripTool and other applications.

    To start XV on the Sun, just left-click the XV icon in the workspace manager bar at the bottom of the screen. To get the XV controls window, right-click anywhere in its main window. To capture a window, choose Grab and select the grab delay time, which lets you locate the window you want to capture. Unless the speaker on your computer is silenced, you will hear a beep when XV goes into the capture mode. If you experience problems with "Grab", try "AutoGrab". The rest is easy and intuitive. 

    OPR plots

    From time to time, accelerator guys may ask you whether there was any change in beamspot position and size after, say, a machine development day, a change of bunch pattern, or some other change to the machine. The beamspot value is determined on a run-by-run basis in OPR (which normally runs 8 to 24 hours behind data taking), and the details of the process are documented in BADs 13 and 102. You can make plots of the beam spot parameters yourself:
    1. Determine the run number range you are interested in: go to the Electronic Logbook, click "List Runs", enter the date range and select "Short listing" and "Show OPR status".
    2. Run the following script in a directory that contains an emptypawlogon.kumac file (you can create one by typing "touch pawlogon.kumac"):
    3. ~philiph/perl/subsys_chart.pl -R -s b_s 10000-11000
      (in the above example 10000-11000 is the run range).
    4. In the file b_s_chart.ps that has been created by the script, you are interested in pages 1 ("Beamspot position") and 3 ("Beamspot size along beam axis"). Note that determination in OPR of the Y size of the beamspot and of the XY tilt angle is unreliable and should not be trusted.
    5. If you see any peculiarities, such as step-like changes, on these plots, you can determine the time of the change by reading the run numbers corresponding to the changes off the plots and looking up the date and time of these runs in the Electronic Logbook.

    PEP Tools for Liaison Use

    MCC SCP

    SCP stands for "SLAC Control Program" (formerly SLC Control Program) and is pronounced "skip". 

    How to start SCP

    To start from bbrmcc on the Sun:


    The SCP touch panel window:

    SCP windows

    SCP comes up with four windows:

    General SCP information

    The BaBar Panel

    INDEX -> BaBar Panel takes you to the BaBar Index Panel, under which all the BaBar-relevant information is organized. The buttons should be fairly self-explanatory.

    A good panel to look at is (from the BaBar Index Panel) BaBar Protec Panel -> BaBar Digitl Inputs. It tells you which of the BaBar inputs aborted the beams or inhibited injection. Note that the SVT PIN diodes are summarized under SVTDIG (you have to go to the BaBar SVT Background EPICS panel to determine which of the four SVT diodes tripped). SVT A1 and SVT A2 are channels meant for SVT diode signals processed by the (analog) CBAM module. This part of the system was never realised and we ended up with two empty channels...

    The Director's Display

    Go to the "Director's Display" (from the main index panel) to obtain a quick overview of currents and luminositites in the past 24 hours or any other period of time (Burt still looks at it every morning).

    Other panels

    If you are curious and want to explore other panels in SCP, please keep in mind that the SCP panels actually control the machine! Always think before hitting a button!

    The safest way to explore other panels in SCP is by using HELP button. By clicking once on the help button you will put SCP in the help mode. In this mode clicking on any other panel or button will provide help information in the graphics window and also it will open any panel, which is safe to open. This will prevent unintentional pushing on expert's buttons. Make sure that there is a white stripe in the HELP box, which indicate that you are in the help mode:


    History buffer

    Each of the variables in the MCC database is saved to an MCC history buffer approximately every six minutes (a few are saved more often). This information is very useful and can be easily accessed and plotted via the SCP.

    Correlation plots

    SCP is an excellent and frequently used tool for data acquisition during all kinds of machine/background experiments. It can write up to 160 channels with a sampling rate of seconds to a file for easy inspection and analysis later with SCP or some other tool.

    In the early days of BaBar when background conditions changed from day to day, liaisons heavily participated in such studies and running correlation plots was one of their standard tasks during a shift. Now that we're approaching "factory mode" with backgrounds stabilizing, we don't really do that anymore, unless it is a dedicated background shift.

    This link will take you to instructions on how to run and analyse correlation plots with SCP (from the old liaison manual). 

    PEP-II StripTool

    You can also fire up a strip chart from SCP with access to PEP-II EPICS channel names. To do so, click INDEX -> BaBar Panel -> EPICS Strip Chart. By loading in, for instance, the configuration file /u1/pepii/strip/strip_cud_BKGD.cnf you will get the MCC standard background strip chart, the one the operators actually tune on! This strip chart, as well as other standard stripcharts mentioned in this document, are maintained by the Liaison Coordinator.

    Note that PEP-II EPICS runs not on the MCC VAX, but on a UNIX machine at the MCC, and, depending on what terminal you log in from, you may or may not have permission to start PEP-II EPICS from SCP (error messages appear in the SCP's "Input the Local Messages" window). Another thing to note is that the overhead monitors in MCC can only display 32 colors, so some colors (for example, yellow) should not be used in PEP-II StripTool.

    The following channels are currently present in the standard PEP-II BaBar backgrounds stripchart:

    MCC ERRLOG

    Beam abort incidents, injection inhibits as well as the resets (and tons of other stuff) are logged in the MCC error log. Sometimes it might be useful for the liaison to have a look at it to find out what exactly occured at which time. See also Getting Error Reports from SCP

    You can do that from the OpenVMS node MCC (see computer information below) by typing ERRLOG. Then type HELP to learn more about the commands at your disposal. The most important ones are SET and LIST.

    Here's a typical example:

    MCC>ERRLOG
    ERRLOG>HELP
    ... (carriage return to exit HELP)
    ERRLOG>SET/SINCE=26-MAY-1999:12:30/BEFORE=27-MAY-1999:10:30
    ERRLOG>SET/STRING=DCH_
    ERRLOG>LIST

    This will churn out all DCH incidents during the specified time period. Other choices for string might be

    ERRLOG>SET/STRING=DCH_LABT

    for all DCH LER aborts and resets, or

    ERRLOG>SET/STRING="DCH_HABT,FAULT"

    for all DCH HER aborts, or

    ERRLOG>SET/MATCH=OR
    ERRLOG>SET/STRING=(DCH_,EMC_,SVT_,SVT1,SVT2,BBR_,BBHV,DASY)

    for all BaBar-related incidents. You can also do things like SET/DATE=YESTERDAY with obvious effect.

    BaBar & PEP Channel Names in EPICS & Strip Charts

    EMC and DCH CsI-diode sensor channel names

                          BaBar EPICS             PEP-II EPICS
    EMC diodes            ACC:PR00:ASTS:EMC_TOTL  PR00:ASTS:EMC_TOTL
    DCH diodes            ACC:PR00:ASTS:DCH_TOTL  PR00:ASTS:DCH_TOTL
    (PEP-II channels are simply mirrored to BaBar) 

    DIRC channel names

                           BaBar EPICS             PEP-II EPICS
                                               (converted to voltage)
    sector 0 (12 o'clock)  DRC::ENV:VSM:WRM0       PR02:ASTS:CH100
           1 (1 o'clock)   DRC::ENV:VSM:WRM1                      
           2 etc.          DRC::ENV:VSM:WRM2                      
           3               DRC::ENV:VSM:WRM3       PR02:ASTS:CH101
           4               DRC::ENV:VSM:WRM4                      
           5               DRC::ENV:VSM:WRM5                      
           6               DRC::ENV:VSM:WRM6       PR02:ASTS:CH102
           7               DRC::ENV:VSM:WRM7                      
           8               DRC::ENV:VSM:SCL8                      
           9               DRC::ENV:VSM:SCL9       PR02:ASTS:CH103
           10              DRC::ENV:VSM:SCL10                     
           11              DRC::ENV:VSM:SCL11
    Note that the BaBar signals have physically different sources than the PEP-II signals!

    Jerry's DIRC Diode channel names

                          BaBar EPICS             PEP-II EPICS
    #1 (mid-plane, LER)   DRC::WTR:VSM:CALC10     PR00:ASTS:CSI_1     
    #2 (top, LER)         DRC::WTR:VSM:CALC11     PR00:ASTS:CSI_2
    #3 (top, center)      DRC::WTR:VSM:CALC12     PR00:ASTS:CSI_3
    #4 (top, HER)         DRC::WTR:VSM:CALC13     PR00:ASTS:CSI_4
    #5 (mid-plane, HER)   DRC::WTR:VSM:CALC14     PR00:ASTS:CSI_5
    #6 (bottom, HER)      DRC::WTR:VSM:CALC15     PR00:ASTS:CSI_6
    #7 (bottom, LER)      DRC::WTR:VSM:CALC16     PR00:ASTS:CSI_7
    ...                   ...                     ...
    #20                   DRC::WTR:VSM:CALC29

    Drift Chamber channel names

                          BaBar EPICS             PEP-II EPICS
    total                 DCH::HV:PS_1:I_TOT      PB60:DCH:HV:PS_1I_TOT
    SL6, out. quad.       DCH::HV:PS_1:IMON_6     not available

    IFR Channel names

                     BaBar EPICS                PEP-II EPICS
    east A           IFR:SPR:HV:PS_5:IMON_4_A   BBR:IFR:HVI:L18FEA
         B           IFR:SPR:HV:PS_5:IMON_4_B   BBR:IFR:HVI:L18FEB
    west A           IFR:SPR:HV:PS_5:IMON_3_A   BBR:IFR:HVI:L18FWA
         B           IFR:SPR:HV:PS_5:IMON_3_B   BBR:IFR:HVI:L18FWB
    (BaBar channels are simply mirrored to PEP-II) 

    Trigger channel names

                          BaBar EPICS             PEP-II EPICS (out-of-date)
    total L1 rate         FCT:BBR::GPR            BBR:FCT:TRG:PSC
    L1 dead time          FCT:BBR::DT             BBR:FCT:L1:DEAD
    1M trigger rate       FCT:BBR::TR_23          PR02:ASTS:CH104
    1B trigger rate       FCT:BBR::TR_22          PR02:ASTS:CH105

    Injection related Trigger channel names

                               BaBar EPICS             PEP-II EPICS
    HER injection rate         FCT:INJRATEH            BBR:FCT:INJRATEH
    LER injection rate         FCT:INJRATEL            BBR:FCT:INJRATEL
    1M cts / HER injection     FCT:INJTRCTH            BBR:FCT:INJTRCTH
    1M cts / LER injection     FCT:INJTRCTL            BBR:FCT:INJTRCTL
    1B cts / HER injection     FCT:INJTRCT1            BBR:FCT:INJTRCT1
    1B cts / LER injection     FCT:INJTRCT2            BBR:FCT:INJTRCT2

    PEP-II Beam Variable channel names

                          BaBar EPICS             PEP-II EPICS
    
    HER current [mA]      CEN:PEP:HER:I           HB60:DCCT:SUMY
    HER lifetime [min]    CEN:PEP:HER:LIFETIME    HB60:DCCT:LIFETIME
    HER Inverse lifetime                          HB60:DCCT:INVLIFE
    HER dI/dt [µA/s]      CEN:PEP:HER:DIDT        HB60:DCCT:DIDT
    
    LER current           CEN:PEP:LER:I           LB60:DCCT:SUMY
    LER lifetime          CEN:PEP:LER:LIFETIME    LB60:DCCT:LIFETIME
    LER Inverse lifetime                          LB60:DCCT:INVLIFE
    LER dI/dt [µA/s]      CEN:PEP:LER:DIDT        LB60:DCCT:DIDT
    
    Luminosity            CEN:PEP:BBRCO:LUMVAL    PB60:LUMCOR
    Specific Luminosity                           PB60:SPLUMVAL

    Witold's favorite PEP-II Beam Loss Monitor channel namess

    location                PEP-II EPICS
    
    beginning of S2A        PR02:ASTS:BLSC7017
    after adj. coll.        PR02:ASTS:BLSC7044
    nose Q2L, LER-sens.     PR02:ASTS:BLSC7052
    nose Q2R, HER-sens.     PR02:ASTS:BLSC8012
    HER x-coll. #1          PR12:ASTS:BLHC7102
    HER y-coll. #1          PR12:ASTS:BLHC8012
    HER y-coll. #2          PR12:ASTS:BLHC8072
    LER x-coll. #1          PR04:ASTS:BLLC2081
    LER y-coll. #1          PR04:ASTS:BLLC3011
    For access from BaBar EPICS simply prefix channel names by ACC:.

    How to print this guide:

    This document is a compromise between a fully self-contained, booklet style manual (hardly maintainable), and a bunch of links to various documents scattered throughout the BaBar Web (hardly printable).

    We prefer that you read this online, but if you want a hardcopy, print this page.

    If you want additional figures, and supplementary material covered by this guide, also print the following documents:

    1. the figures linked to the SVT Abort Diode section,
    2. the liaison guide to the SVT diodes


    This document was created in April 2000 by Thomas Schietinger,

    and updated and reorganized by hertzbac@slac.stanford.edu (last edit 14 Jan 03)

    Parts of it are based on the old liaison shift manual initially written by Terry Geld with major additions by TI Meyer, Sasha Telnov, and others.
    Last significant update:  Expiry date: