==============================================================================
==============================================================================
SLAC CMLOG Modifications
==============================================================================
==============================================================================
Author:
Michael P. Laznovsky [ESD]
lazmo@slac.stanford.edu
Original: 17-Aug-2001
Last Modified: 20-Sep-2001
==============================================================================
Table of Contents
-----------------
1.0 Introduction
2.0 Filter Types
2.1 Null
2.2 JLAB
2.3 SLAC
3.0 APIs
3.1 C++ API
3.1.1 C++ Filter Creation & Installation
3.1.2 C++ Throttle Setup
3.1.2.1 C++ JLAB Filter Throttle Setup
3.1.2.2 C++ SLAC Filter Throttle Setup
3.1.3 C++ SLAC Throttle On/Off
3.1.4 C++ SLAC Throttle Show
3.1.5 C++ SLAC Throttle Remove
3.2 C API
3.2.1 C Filter Creation & Installation
3.2.2 C Throttle Setup
3.2.2.1 C JLAB filter throttle
3.2.2.2 C SLAC filter throttle
3.2.3 C SLAC Throttle On/Off
3.2.4 C SLAC Throttle Show
3.2.5 C SLAC Throttle Remove
4.0 Sample Applications
4.1 Sample C++ Application
4.2 Sample C application
4.3 Shortcuts
5.0 Adding New Filters
5.1 Files to modify
==============================================================================
1.0 Introduction
==============================================================================
This document describes the changes made to CMLOG client code at SLAC to
support a different style of message throttling than the one originally
built in to the CMLOG client.
First off, old JLAB calls should still work the same as before, which is
to maintain a history of unique field values and throttle on each of those
seperately. No provision was provided to specify specific values of
special interest to throttle; the system would just allow a certain
number of *each* value through.
The new implementation reformulates the old cmlogFilter:: class into
a base class extended by the new classes cmlogFilterNull, cmlogFilterJLAB,
and cmlogFilterSLAC (the non-filter, the JLAB-style filter, and the new
SLAC filter respectively). The application must now explicitly create the
desired filter, as described below, and pass it to the cmlog client. If no
filter is explicity set up, it defaults to the old JLAB filter, or whatever
is specified in "-D_DEFAULT_FILTER=" in the client makefile (set to
"cmlogFilterSLAC" for SLAC).
The new filter supports filtering on specific contents of tags (fields)
of a message. A throttle "limit" specifies the number of messages desired
per time period ("deltaTime"), and throttles may be re-started at any time
with different limit and time-period parameters.
In addition, summary messages are output when throttles are started & stopped,
and after throttling has resulted in messages being dropped, and a function
is provided for displaying the current list of throttles.
==============================================================================
2.0 Filter Types
==============================================================================
--------
2.1 Null
--------
The "Null" filter is the empty filter, ie. no filtering. All messages are
allowed through.
--------
2.2 JLAB
--------
The original JLAB filter is the default if setFilter() is not called. It
operates by maintaining a running history of values seen for the specified
throttle tags, and throttling each unique value independent of the others.
--------
2.3 SLAC
--------
The new SLAC filter is in way the opposite of the JLAB filter; it allows
one to specify that only messages with certain values in particular tags
should be subject to throttling. Messages with values not specified in
throttles are passed through unfiltered.
==============================================================================
3.0 APIs
==============================================================================
Unless otherwise noted, non-void functions return either CMLOG_SUCCESS or
CMLOG_ERROR;
-----------
3.1 C++ API
-----------
----------------------------------------
3.1.1 C++ Filter Creation & Installation
----------------------------------------
cmlogFilterNull* cmlogFilterNull::newFilterNull();
cmlogFilterJLAB* cmlogFilterJLAB::newFilterJLAB();
cmlogFilterSLAC* cmlogFilterSLAC::newFilterSLAC();
These calls create a new filter of the specified type: Null, JLAB, or SLAC.
A pointer to the filter object is returned; this pointer should be passed
to the "setFilter()" function described below, and should be saved by the
application for making later calls to throttle functions.
----------------------------------------------------------------
cmlogFilter* cmlogClient::setFilter(cmlogFilter *filter)
This function takes a pointer obtained from one of the filter-creation
calls described above and installs the filter in the cmlog client. All
subsequent cmlog messages will be run through this filter. Any previous
filter is removed from the system. If no call to setFilter() is made,
the cmlog client will use the default JLAB filter.
Although cmlogClient::setFilter() is declared to take an argument of
type "cmlogFilter*", pointers to any of the above filters will be accepted,
as they are all derived from cmlogFilter.
This function returns its argument, thus filter creation & installation may
be combined into one statement:
cmlogFilterSLAC*
filter = client -> setFilter (cmlogFilterSLAC::newFilterSLAC());
------------------------
3.1.2 C++ Throttle Setup
------------------------
--------------------------------------
3.1.2.1 C++ JLAB Filter Throttle Setup
--------------------------------------
The original JLAB set/get throttle calls are still supported; see SLAC
filter throttle (below) for argument description.
int cmlogClient::setThrottle (char *tag, int size, int limit, double delta)
int cmlogClient::getThrottle (char **tag, int& size, int& limit, double& delta)
Most arguments are the same as for SLAC filter calls (below) with the
exeption of:
int size
Specifies how many different throttles the JLAB filter is
allowed to maintain. Since this filter could potentially create
a virtually unbounded number of unique throttles (due to its
design), this parameter is necessary to keep it from blowing up.
The SLAC version on the other hand requires that each throttle
be explicitly specified and is thus limited to however many
throttles the user wants to set, up to a compile-time limit
(currently 100; see MAX_LIST_SIZE in Common/cmlogConfig.h).
--------------------------------------
3.1.2.2 C++ SLAC Filter Throttle Setup
--------------------------------------
The following functions set up specific throttles within the SLAC filter.
Throttles take effect as soon as they are set up, and may be temporarily
disabled as a group with the cmlogFilterSLAC::throttleOff() function, or
individually permanently removed by repeating the same setThrottle() call
with a "limit" value of -1, or by calling throttleRemove with an appropriate
index obtained via throttleShow().
They take the form:
int cmlogFilterSLAC::setThrottle(char *ctag,
int limit,
double deltaTime,
<CTYPE> val)
where supported <CTYPE> values are:
BYTE -- unsigned 8-bit ints/chars
short -- signed 16-bit ints
unsigned short -- unsigned 16-bit ints
long -- signed 32-bit ints
unsigned long -- unsigned 32-bit ints
float -- 32-bit floats
double -- 64-bit floats
char * -- char array (ie. string)
Arguments are:
char* ctag
A string containing a tag name. In the SLAC CMLOG environment,
message fields have tags such as "facility", "verbosity",
"severity", etc (see Common/cmlogUtil.h). Message strings that
are passed to functions like logMsg() etc. all end up in the
field labeled "text", so if you want to throttle something in
that field, specify tag "text". See the sample code below.
int limit
double deltaTime
These parameters specify how many messages (limit) we want to
allow through in a specified time period (deltaTime seconds).
Only "limit" number of messages with matching "tag" and "val"
fields will be logged during a given deltaTime period, and a
count is maintained of how many messages were dropped. After
the period is up, a summary message is output to the log and
another "limit" messages are allowed through. "Limit" may
equal zero, in which case only summary messages are output.
A "limit" value of -1 will remove a throttle from the system.
Note that you still need to supply valid "tag" and "val" fields
in order to uniquely identify the specific throttle to remove.
An alternative is to call throttleOn() or throttleOff() to
temporarily suspend *all* throttling. An alternative is to
use throttleShow() to obtain a numbered list of current throttles,
and then calling throttleRemove() with the appropriate index.
<CTYPE> val
This is the actual value to throttle on. <CTYPE> must match the
type of data in the field; a string "5" won't match an integer 5,
for instance. For this reason, for example, field "text" can
only be throttled with a "char* val"; the SLAC filter will treat
this "val" as a substring, and any message that contains "val"
as a part will be subject to throttling. For instance, the call:
filter -> setThrottle ("text", 1, 10.0, "abc");
will subject all of the following messages to potential throttling:
"abc"
"myApp: flabcounter overflow (399)"
"Danger, Will Robinson! Frabcous snark!"
While int- and string-valued throttles use exact matches to determine if
specific messages are to be filtered out, float and double include a fudge
factor in their comparisons. This "epsilon" is currently computed to be
10^(-5) of the specified throttle value... for instance, for a float or
double "val" of 100.0, a message will need to match within +/- 0.001 to be
considered for filtering, while for 1.e+8 an absolute difference less than
1000.0 will do.
Return values:
CMLOG_SUCCESS
CMLOG_ERROR
------------------------------
3.1.3 C++ SLAC Throttle On/Off
------------------------------
void cmlogFilterSLAC::throttleOn (void) // global ENABLE
void cmlogFilterSLAC::throttleOff (void) // global DISABLE
These 2 calls act to temporarily disable or re-enable *all* throttling
in the SLAC filter. They don't affect individual throttle settings, but
just set a global flag.
----------------------------
3.1.4 C++ SLAC Throttle Show
----------------------------
void cmlogFilter::throttleShow ()
This function writes a summary of currently set throttles, including all
of the arguments originally used to set up each throttle.
------------------------------
3.1.5 C++ SLAC Throttle Remove
------------------------------
void cmlogFilter::throttleRemove (int n)
This function removes the n'th throttle, as listed in
throttleShow() output. Note that this will renumber all throttles after
this one, so removing a series of throttles with this function requires
keeping track of the changing throttle numbers via throttleShow().
An alternative way to remove a throttle is to set "limit" to -1 in
setThrottle().
==============================================================================
3.2 C API
==============================================================================
The straight-C calls are largely front-ends for the C++ equivalents,
and thus share many of the same arguments.
In the C versions, it's not necessary for the application to save a
pointer to the current filter, as it's saved internally by the client
code. The makes it possible to use the C calls from, say, the VxWorks
shell without having to create new global variables.
--------------------------------------
3.2.1 C Filter Creation & Installation
--------------------------------------
These calls create the specified filter and install it in the cmlog
client, thus combining the C++ newFilterXXX() and setFilter() calls.
Returned value is either CMLOG_SUCCESS or CMLOG_ERROR.
int cmlog_new_null_filter()
int cmlog_new_jlab_filter()
int cmlog_new_jlab_filter3(int maxSeverity,
int maxVerbosity,
int maxListSize)
int cmlog_new_slac_filter()
int cmlog_new_slac_filter3(int maxSeverity,
int maxVerbosity,
int maxListSize)
----------------------
3.2.2 C Throttle Setup
----------------------
------------------------------
3.2.2.1 C JLAB filter throttle
------------------------------
These calls are analogous to the C++ versions:
int cmlog_jlab_set_throttle (char *ctag,
int valuerange,
int limit,
double interval)
int cmlog_jlab_get_throttle (char **tag,
int *valuerange,
int *limit,
double *interval)
------------------------------
3.2.2.2 C SLAC filter throttle
------------------------------
These calls take the form:
int cmlog_set_slac_throttle_<TYPE> (cmlog_filter_t filter,
char *ctag,
int limit,
double deltaTime,
<CTYPE> val)
where supported <TYPE> and <CTYPE> are:
<TYPE> <CTYPE>
------ -------
byte unsigned char
short short
ushort unsigned short
long long
ulong unsigned long
float float
double double
string char *
See cmlogFilterSLAC::setThrottle().
----------------------------
3.2.3 C SLAC Throttle On/Off
----------------------------
void cmlog_slac_throttle_on () /* global ENABLE */
void cmlog_slac_throttle_off () /* global DISABLE */
See cmlogFilterSLAC ::throttleOn() / ::throttleOff(), above.
--------------------------
3.2.4 C SLAC Throttle Show
--------------------------
void cmlog_slac_throttleShow () /* show throttles */
See cmlogFilterSLAC::throttleShow(), above.
----------------------------
3.2.5 C SLAC Throttle Remove
----------------------------
void cmlog_slac_throttleRemove (int n) /* remove throttle #n */
See cmlogFilterSLAC::throttleRemove(), above.
==============================================================================
4.0 Sample Applications
==============================================================================
--------------------------
4.1 Sample C++ Application
--------------------------
#include <cmlog.h>
#include <cmlogFilter.h>
#include <cmlogFilterSLAC.h>
cmlogClient *client = cmlogClient::logClient(); // set up client connection
cmlogFilterSLAC *filter = new FilterSLAC(); // create SLAC filter
client -> setFilter (filter); // install SLAC filter
// set up first throttle
filter -> setThrottle ("text", // in field "text"
2, // allow 2 messages
10.0, // every 10 seconds
"agv"); // matching substring "agv"
// set up second throttle
filter -> setThrottle ("foo", // in field "foo"
1, // allow 1 message
60.0, // every 60 seconds
(long)101); // when its value == 101
...
filter -> throttleShow (); // display list of throttles
...
filter -> throttleOff (); // temporarily disable all throttles
...
filter -> throttleOn (); // re-enable all throttles
------------------------
4.2 Sample C application
------------------------
#include <cmlog.h>
cmlog_client_t client; /* client pointer */
client = cmlog_open (argv[0]); /* set up client connection */
cmlog_new_slac_filter(); /* create/install SLAC filter */
/* first throttle... */
cmlog_slac_set_throttle_string ("text", /* in field "text" */
2, /* allow 2 messages */
10.0, /* every 10 seconds */
"agv"); /* matching substring "agv" */
/* second throttle... */
cmlog_slac_set_throttle_long ("foo", /* in field "foo" */
1, /* allow 1 message */
60.0, /* every 60 seconds */
(long)101); /* when its value == 101 */
cmlog_slac_throttle_show (); /* display list of throttles */
cmlog_slac_throttle_off (); /* disable all throttles */
cmlog_slac_throttle_on (); /* re-enable all throttles */
-------------
4.3 Shortcuts
-------------
The following "shortcuts" are provided to reduce the amount of typing
necessary to invoke some of the throttling functions from the VxWorks shell.
They take the same arguments as the corresponding C calls, and they also
write their output to stdout in addition to the error log.
original shortcut
-------- -------
cmlog_slac_set_throttle_byte cml_byte
cmlog_slac_set_throttle_short cml_short
cmlog_slac_set_throttle_ushort cml_ushort
cmlog_slac_set_throttle_long cml_long
cmlog_slac_set_throttle_ulong cml_ulong
cmlog_slac_set_throttle_float cml_float ** [see note below]
cmlog_slac_set_throttle_double cml_double
cmlog_slac_set_throttle_string cml_string
cmlog_slac_throttle_on cml_on
cmlog_slac_throttle_off cml_off
cmlog_slac_throttle_show cml_show
cmlog_slac_throttle_remove cml_remove
** There is currently a problem passing floats to functions from the
VxWorks shell, thus this call should not be used. This also goes
for any other functions that expect float arguments. "Double" is
OK.
==============================================================================
5.0 Adding New Filters
==============================================================================
-------------------
5.1 Files to modify
-------------------
...TBD............
refer to cmlogFilter<JLAB,SLAC>.<h,cc>
cmlogFilterXYZ.<c,h> as templates
implement filter() method, optionally others (setThrottle, etc)
==============================================================================
Future Improvements
-------------------
Allow string <-> numerical matches?
Support numerical ranges? 10-20, 3.5-4.5, etc.
dropped_<severity,verbosity>? in cmlogFilterJLAB.cc
::removeThrottle(index)?
==============================================================================
==============================================================================
==============================================================================
Out of memory.
We wish to hold the whole sky,
But we never will.
==============================================================================
==============================================================================