SLAC PEP-II
BABAR
SLAC<->RAL
Babar logo
HEPIC E,S & H Databases PDG HEP preprints
Organization Detector Computing Physics Documentation
Personnel Glossary Sitemap Search Hypernews
Unwrap page!
Det. Search
Who's who?
Meetings
FAQ
Images
Archive
Systems
Performance
Intern. region
Vertex Tracker
Drift chamber
DIRC
Calorimeter
IFR
LST
Magnet
Electronics
Trigger
Operations
Run Coordination
Contact Experts
Shift Takers Info
Operations Manual
Electronic Logbook
Ops Hypernews
Shift Signup
Check this page for HTML 4.01 Transitional compliance with the
W3C Validator
(More checks...)

Trigger Trouble/Operations History

Trigger Configuration History, L1 event damages log since June 2005

Start/Affected 
Run(s)
Start 
Time
Comments
67707 August 17, 2006 The last physics run for Run 5b.
67202- July 31, 2006 The EMT prescale is changed back from 3 to 1.
64894- May 17, 2006 The old TSF firmware is installed back to both of the TSF-X and TSF-Y crates.
64106- April 22, 2006 The old DCT crate is again taken out of the data-taking.
64105-64885 April 22, 2006 Both of the TSF-X and TSF-Y crates with version_9 firmware in this run period.
64104 April 22, 2006 TSF-Y crate with version_9 firmware while X crate still has the user-reset-problem-fixed version.
64100-64103 April 22, 2006 The TSF-X and TSF-Y crates have the user-reset-problem-fixed version. There are TSF->ZPD transmission errors. ~5% ZPD damages, so we decided to roll back to the version we took data in January-March. The old DCT crates are back on for testing purpose.
63595- April 5, 2006 The old DCT system is not in data-taking from now on.
63527 April 3, 2006 First physics run after PEP recovered from the shutdown for Q2 chamber/bellow replacement.
63354- Mar.27, 2006 A new and improved version of the TSF firmware is installed to both of the TSF-X and TSF-Y crates. No power cycling is required anymore if timing signal was lost unexpectedly.
62240- Feb.14, 2006 DCH front-end FEX mode enabled
61369- Jan.16, 2006 Changed the EMT prescale from 1 to 3.
61147- Jan.10, 2006 Swapped the boards for TSF-X8 and X9. What seems to be a neighbour data problem between TSF X8-X9 in the new TSF system (since run 60947) disappeared after the board swapping. Run 61147 is the first good physics run after board swapping.
60996 Jan.6, 2006 about 200 TSF damaged events. See HN posted.
60947-63353 Jan.4, 2006 The first colliding physics run after the new TSF-X firmware (test_9 version) went in.
60597-60600 Dec.26, 2005 Old TSF-Y4 has trouble, see screen shots for the TSF Comparison plots and the Old TSF occupancy plots. A reconfigure solved the problem. This only happens to the old TSF, and has no impact on the data quality.
60166 Dec.12, 2005 L1T damage for TSF-X 0,1,2,3, 12, 13, 14, and 15 and also for ZPD 0, 1, 2, 5, 6, 7. A reconfigure solved the problem.
60043-60044 Dec.9, 2005 The mis-match between the new and old TSF is for TSF-X7 (or wedge number 7), and they were either no match at all or location matched only between the new and old TSF's. A dip in the BLT phi map was seen too. It turns out to be a problem with the old TSF-X7. A reconfigure solved the problem.
59743 Nov 29, 2005 First physics run with new TSF firmware (test_8 version) in TSF-Y only (TSF-X still uses the old firmware).
59565-59575 Nov 24, 2005 A dip in SL10 for both new and old TSF's. It went away after the detector was ramped down and up.
59485 Nov 19, 2005 First colliding beam run for Run 5b after DCH Phase 2 electronics installation. Note that the old DCT system has been back on since the October shutdown.
58384-58421 Sept 27, 2005 This run range is where the TSF and ZPD data got affected by the bit corruptions in TSF-Y. A DCC board was replaced after 58421. We know for sure that run 58385 is completely bad while only the last 9 minutes of run 58384 contained the damages. The rest of the runs might be okay. However, runs 58384-58399 are marked unusable due to DIRC problem.
58036-58039 Sept 15, 2005 Background runs
57991- Sept 13, 2005 The old DCT system is out of data-taking from now on.
57921 Sept 10, 2005 PHYSICS_L1PASSTHRU run with 1Z for trigger rate studies (on-resonance data).
57920 Sept 10, 2005 PHYSICS_1E_L1PASSTHRU run with 1Z replaced by 1E for single photon studies (on-resonance data).
57874-57885 Sept 9, 2005 affected by a timing shift (by 1 15MHz clock tick) in the DCH electronics. See HN posted.
57689-57690 Sept 2, 2005 Both runs are physics_L1passthru runs (off-resonance data) even though ConfigKey is 1698 for run 57689 and 169f for 57690 (due to problem with TCF leaf node).
57515 Aug 27, 2005 One physics run without the DCZ in the partition because it caused failure in configuring ORC. A power cycle of the DCZ crates solved the problem.
57211- Aug 16, 2005 Physics runs with a new version (v2.2) of GLT FC firmware. We see some EMT->GLT and DCT->GLT transmission errors, as well as a small fraction of events with GLT-FC word corruptions.
57087 Aug 12, 2005 First physics run with a testing version (v2.0) of GLT FC firmware. We see some EMT->GLT and DCT->GLT transmission errors, as well as a small fraction of events with GLT-FC word corruptions. The original firmware was back in place after this run.
56968-56969 Aug 10, 2005 EMC has 24 (mistakenly marked) dead channels.
56797 Aug 3, 2005 First run after the bug in the DCH feature extraction code was fixed .
56074-56078 July 12, 2005 HER single beam data taken for background rate study. Note that all these runs have DCZ in the configuration, except run 56074.
55680-55682 June-29 2005 TSF damages showed up and then disappeared after reconfigured. The problem seemed to happen to TSF-X15. The new TSF-X ROM logfile indicated that there were out-of-frame errors. See HN posted here.
55386 June-17 2005 The parasitic (old) TSF-X15 has dips in the occupancy plots. See HN posted here. This doesn't affect new TSF, and it's still a good run.
55381 June-17 2005 TSF damages showed up and lasted about 10 minutes and then went away on its own even without reconfigure. The problem seemed to happen to TSF-X15. The new TSF-X ROM logfile indicated that there were out-of-frame errors from 1:35-1:46am. See HN posted here.
55259-55266 June-13 2005 L1 rate was about 20kHz and deadtime > 70%. GLT output lines dropped due to configuration problem with createFileTable. Bad runs!
54999-55000 June-5 2005 The parasitic (old) TSF-X15 has dips in the occupancy plots. See the new vs old TSF comparision plots. The problem went away after a reconfigure. The new TSFs are okay, so it doesn't affect data quality. No impact on real physics trigger! These are still good runs.
54988 June-4 2005 The parasitic (old) TSF-X14 has dips in the occupancy plots. See the new vs old TSF comparision plots. The plots here showed that the new TSF system was in fact okay. The problem went away after a reconfigure. The new TSFs are okay, so it doesn't affect data quality. No impact on real physics trigger! This is still a good run.
54709 May-25 2005 The TSF-X0 in both new and old TSFs showed a big gap in the occupancy plots. This is due to fibers damaged by a gogher. Bad run!
54217-54224 May-5 2005 L1 rate went crazy to 5 kHz, caused by corrupted TC and the GLT objects that were used in the GLT line cuts were all shifted up by one. See the associated TrigOper and DetOper HN postings. Data in this run range are NOT usable.
53460 Mar-28 2005 The DCZ configuration is now all from the database, including TSF 6 bit LUT, BLT mask, all DCT DAQ latency, and ZPD LUT. The ZPD configuration test was the last one put in and tested on March 28, 2005.
50581-2 Jul-30 2004 Eunil : We had dips/spikes on TSF SL5,6,7 around 4:00 am. It looks like the entire TSF-Y3 board was in trouble. Re-configure solved the trouble as usual. hn associate with is available and see this for plots.
50254 Jul-19 2004 Eunil : TSY SL7 cell distribution showed dips and spikes. Entire Y7 board seemed to be affected. Reconfigure solved the trouble.
50247 Jul-19 2004 Eunil : DCT generated 100% deadtime. TSFY rom log showed out of frame error all the time. Went to IR2, and confirmed that TSFY0 SYNC* LED was on. As reconfigure by pilot did not solve the trouble, power recycled the TSFY crate and the trouble went away.
49759 - 49776 Jul-04 2004 Eunil: A single dip in TSF SL1. This is my first time to see this single dip from TSF cell distributions. See the hn posting. The cause of this is unknown but mysteriously disappeard.
- Jun-19 2004 Eunil: EMT fan tray died. Ray Rodriguez came in and replaced it with the one in the teststand (central lab). The cause of this is unknown.
49287 Jun-19 2004 Eunil : DCT caused 100 % deadtime. The TSFY rom log showed
19-Jun-2004 22:24:14 odf-dct-3-> L1DTsfFexEngine::L1Accept(): Out of frame for 
1 time(s) with damage 0x0
for 20 minutes. Reconfiguration solved this trouble. According to the log, 1) rebooting of the rom (due to dch TDC reset storm), 2) odf-dct-3 Out of frame, and 3) reconfiguration happened consecutively. See hn for more details.
- Jun-07 2004 Eunil : TSFY-26 board on Y5 slot (ID:20002205) now formally has the tag describing the RUN/LOAD LED is broken. See hn and all threads associated with.
48814 Jun-04 2004 Eunil : GLT-FCT word corruption histogram fired in the bin 18. Also FCTS epics showed a flat rate of ~300 Hz on lines of 17-23. Two troubles believed to be independent. This appeard on one cosmic run. The source of the latter is unknown but there is much better understanding on the former. See hn and all threads associated with.
48733 Jun-02 2004 Eunil : Got paged due to same problem as happened on May-24 (See below). Y-7 data were corrupted. Reconfigure solved the trouble.
48680-98 May-31 2004 Eunil : Got paged due to a dip in SL1 x5. Dip appeared to be in a group of last 4 cells in x5. This time, multiple reconfigurations did not solve the problem. We added dcz in x5 and saw the dip in the new tsf as well, indicating the trouble is not from the trigger but likely from the DCH. In parallel, we power-recycled the DCT tsf crate but did not work. We also swapped x5 and x4 boards. Unfortunately after x5 and x4 were swapped, the dip went away. Now in x5 slot, we have TSFX18 (20002192) and in x4 slot, we have TSFX15 (20002199).
48588 May-28 2004 Eunil : Got paged due to a problem with JAS HTML configuration for DCT/BLT/GLT monitoring histogram. The cause was the incompatibility between new L1TFMon histograms directory structure and JAS XML expectcation. Gregory fixed the problem immediately.
48423-8 May-24 2004 Eunil : Got paged due to spikes in distributions for TSF SL5,6,7 and PTD SL7. Plots can be found for 48428. Page 4 and 5 contain TSF and PTD distributions. It looked like Y-7 data got completely corrupted. Again, reconfiguration removed the trouble.
48129 May-14 2004 Eunil : Got paged due to _slightly_ unusual distribution for TSF SL2 and SL4. E-log (day shift) has two histograms attached. It disappeared as run continued and I don't think it is a real trouble but post here for the record. Plots can be found for SL2 and SL4 respectively. Later (19/May) Su Dong noticed other oddities in PTD and TSF SL9,10. See the hn.
48037,35 May-11 2004 Eunil : Got paged due to 1) no L3 tracks in event display and 2) empty FastMon plots for entire TSF/PTD/BLT/GLT. 1) had to do with a new L3 build installed earlier afternoon and 2) had to do with the update in FastMon GUI. Both problems went away when we backed out to old versions. Data quality should be ok for both cases. Details can be found in the hn ( 816 , 817) and associated threads.
Cosmic Run May-09 2004 Eunil : There was a TSF board failing configuration. It was Jamie who got paged, as Eunil was out of town. It appeared that the board died for good. Details can be found in the hn and associated threads.
47719-21 May-02 2004 Eunil : Got paged due to dips and spikes in TSF SL 5,6,7 and PTD A7 cell distributions. Location of dips and spikes corresponded to the entire TSFY1 board. An immediate reconfiguration was asked to the pilot and the problem disappeard afterward. Details can be found in the hn and associated threads.
47385 Apr-20 2004 Eunil : Installed 10.18.0 version of libL1DctApp.o (1: cp libL1DctApp.o from $BFDIST/online_releases/10.18.0/lib/segment/vxworks/, to dataflow/dct 2: make a soft link to libL1DctApp.o 3: edit productionStartup/startup and add dctsegtest 14) A DAQTEST run went allright.
47257-60 Apr-16 2004 Eunil : Got paged due to high deadtime for the BACKGROUND runs. L3 rate was high but unable to find the cause. What should have been done is to take a look at L3 OEP trigger lines and identify the line(s) causing the high rate. (In ir2, "Level 3 Trigger Output lines rates" window) It turned out that for this time L3InputPulser(=sourcecalpulser)/L3OutPulser(prescale=1) was the line caused the high deadtime.
46995-9 Apr-08 2004 Eunil and Jamie: Tested online release 10.17.0 with cosmic runs and backed out 10.12.0. Details can be found in the hn. Later noticed BaBar kept running 10.17.0 and worried incompatibility between dct loadable and rest of the online env but turned out to be ok.
46927 Apr-06 2004 Eunil and Jamie: Tried to run with PHYSICS_L1PASSTHRU_REDUCED but gave us more than 50 % deadtime so we stopped. Lost 1 minuite. Rainer informed us that the configuration was not quite ready.
46544 Mar-26 2004 Eunil: FastMon showed a dip in TSF SL5 cell distribution. Looks like the are cells 45-48, corresponding the first four cells in TSFY2. Appears to be an Elephant chip fault. Dip disappeared after the reconfigure. Runs 46544-46546 appear to have this dip. See here for a hn posting.