*******
DRAFT 0.1 - for discussion only

3 August 1997


Level 3 Trigger Algorithm Qualification Procedure - UNOFFICIAL
-------------------------------------------------

AUTHORS: The Level 3 Trigger Group



Recall Requirement E.3.1:

     An administrative procedure shall be established for qualification
     testing of new Level 3 filter algorithms for use in production.
     The necessary mechanisms for implementing it shall be provided.
     The procedure should include:

     * testing of the algorithm in the offline environment against
       Monte Carlo and, when available, real physics and background data;
     * publication of algorithms and test results to the collaboration;
     * operation of the new filter in increasingly realistic test modes
       in the online environment, with collection of additional
       qualification data; and
     * a final acceptance decision made by a designated responsible
       individual or group.

Software is easy to change, but the consequences of changes and the
actual function of code in the real-world environment may be difficult
to predict.  In the matter of Level 3 software, which is of critical
importance to the experiment's physics mission, and of irreversible
effect, this gap must be closed.

A procedure is needed to ensure that new filters are appropriate to the
physics and other requirements on them, reliable, and implement the 
intended algorithms correctly.

Elements of such a procedure have been drafted.  It should include:

     a) testing of new algorithms in the offline environment
        against Monte Carlo and, once the experiment is running, real
        physics and background data, using samples of prescaled random
        triggers and Level 1 Accepts previously collected in addition
        to ordinary physics data; 

     b) detailed comparison of results with those from existing 
        algorithms, including studies of correlations;

     c) publication of test results as a \babar\ Note or equivalent,
        with an opportunity for collaboration comments and/or review;

     d) code walk-throughs or other final quality control procedures,
        at the discretion of responsible management;

     e) operation of the new filter in a test mode in parallel with the
        production online environment, not permitted to filter or modify
        production data, and probably operating on a prescaled basis,
        but otherwise in as close a configuration as possible to
        production, using online configuration tools, and with storage,
        analysis and review of the results; 

     f) to the extent permitted by available CPU and logging resources, 
        operation of the new filter in production, initially with no
        direct effect on the logging decision, then subsequently as an
        OR in parallel with any existing production Level 3, but
        defining a separate set of Level 3 output lines;

     g) a final acceptance decision made by a designated responsible
        individual or group, possibly at a high level if the proposed
        new configuration is expected to change significantly the
        physics performance of the experiment.

Item e) presents some implementation challenges in OEP as well as, 
potentially, in DataFlow.  It may be seen as less firm than the
others; it tests a fairly narrow problem space wedged between a/b)
and f).

Thanks to P. Sinervo for useful discussions which served to frame the
need for ideas such as these.