LCLS Controls

SLC-Aware IOC Design

James Silva – ESD Software

February 15, 2005

CMLOG Service

1 Scope

The following document describes a software design for the Input-Output Controller (IOC) Common Message Logging (CMLOG) (iocCmlog) service which can be used by both SLC-Aware IOCs and non-SLC IOCs. The document includes a description of the logic and design of the iocCmlog service.

2 Introduction

2.1 Background

The CMLOG software package was developed by Jie Chen at Jefferson Lab (chen@jlab.org) and is used by several laboratories, including SLAC.

The CMLOG client is currently used by non-SLC vxWorks 3.13 IOCs to log messages. Messages are sent to a CMLOG server running on one of the UNIX machines via a client daemon running on the IOC, and the message log can be browsed from any UNIX session by starting the CMLOG browser.

CMLOG was chosen to be the message logging service for SLC-Aware IOCs for various reasons. It is already in use by the vxWorks IOCs and, by programming the iocCmlog service, we do not have to port the SLC micro errlog code to accommodate the new SLC-Aware IOCs.

The iocCmlog service replaces cmlogVxLogMsg delivered as part of the JLAB CMLOG package. It is separate from the iocLog service delivered as part of EPICS base, which uses a different mechanism to store and browse messages. Both iocCmlog and iocLog may run on the same IOC at the same time with EPICS-only messages going to both the iocLogServer and the cmlogServer. The SLC-Aware IOC messages will ONLY go to the cmlogServer.

2.2 References

CMLOG Programmer’s Guide

http://www.jlab.org/~chen/cmlog/cmlog-2-1-b/index.html

EPICS App Dev Guide

http://www.aps.anl.gov/epics/base/R3-14/6-docs/AppDevGuide.pdf

SLC-Aware IOC Utility Routines

http://www.slac.stanford.edu/grp/lcls/controls/global/sw/slc_ioc/exec/slcUtil.htm

SLC-Aware IOC Functional Requirements

http://www.slac.stanford.edu/grp/lcls/controls/global/sw/slc_ioc/requirements/funcReqts.html

LCLS C Coding Standards

http://www.slac.stanford.edu/grp/lcls/controls/global/standards/software/codeStdsC.html

2.3 Requirements

The iocCmlog service must be usable by both the SLC-Aware and EPICS IOC tasks. It must also run Solaris, Linux, vxWorks and RTEMS (CMLOG is currently not built for RTEMS, and messages logged on RTEMS platforms will simply be printed to the console until the RTEMS build is complete).

The iocCmlog service uses the EPICS message queue service to store and retrieve messages for logging. The EPICS message queue service is used for concurrency control and platform independence.

Code must be C and follow the LCLS C coding standards.

3 Service Design

3.1 Service Description and External Interfaces

3.1.1 Service Block Diagram

Please refer to Section 3.2.1 for a description of logical flow of data and control of the iocCmlog service.

3.1.2 External Interfaces

iocCmlogStart – start the iocCmlog service

iocCmlogDStart – start the cmlog client daemon (vxWorks and RTEMS only)

iocCmlogHdlrQSize – set the iocCmlog message queue size

iocCmlogStop - stop the iocCmlog service

iocCmlogLogMsg – send log message to iocCmlog message queue

iocCmlogVLogMsg – send log message to iocCmlog message queue

3.1.3 Data Flow

Messages enter the iocCmlog message queue from two sources:

- IOC applications call iocCmlogLogMsg() directly. The iocCmlogLogMsg() function places the message on the message queue, and it is then picked up by iocCmlogHdlr().

- An EPICS applications calls errlogPrintf(), and the EPICS errlog task calls iocCmlogListener() whenever an EPICS message is issued. The iocCmlogListener() function places the message on the message queue, and it is then picked up by iocCmlogHdlr().

Once a message is retrieved from the queue, the iocCmlogHdlr() function calls cmlog_logmsg() to send the message to the CMLOG client daemon running on the IOC, which sends the message to the CMLOG server. Once the message reaches the server, it becomes viewable by the CMLOG browser applications.

3.2 Service Detailed Design

3.2.1 Functional Flow

The CMLOG client daemon, which receives messages from all IOC CMLOG applications and forwards them on to the CMLOG server, must be started before the iocCmlog service. For vxWorks and RTEMS (hard) IOCs, the daemon is started early in the IOC boot script, before EPICS databases are loaded, hardware configured, and IOC initialization. For hard IOC, the daemon may be started using the function iocCmlogDStart(). For soft IOCs, the daemon is started at machine bootup and remains up across IOC restarts.

The iocCmlog service is started by the function iocCmlogStart(), which can be started in the IOC boot script or at the console command line. Once it is started, IOC applications can log messages directly to CMLOG using iocCmlogLogMsg(), and EPICS applications can log messages via errlogPrintf(). The function iocCmlogHdlr() task registers the function iocCmlogListener() with the EPICS errlog service to log errlog messages to CMLOG.

Messages are then queued by the function iocCmlogLogMsg(). The function iocCmlogHdlr(), spawned by iocCmlogStart(), processes incoming messages and sends them to the CMLOG client daemon, which then sends the messages to the CMLOG server. Messages are optionally printed to the console and the printout includes the process name, status string, and formatted text string.

The iocCmlog service is stopped by calling iocCmlogStop(), which sends a “stop” message to the iocCmlog message queue.

Please refer to the diagram in Section 3.1.1 for a visual representation of this functional flow.

3.2.2 File-Scope Data

- iocCmlog message queue

- iocCmlog message queue size

- number of missed messages due to full queue

3.2.3 Message Logging

The iocCmlog service logs only 3 messages on its own:

- iocCmlogHdlr startup (may be disabled using a compiler switch)

- iocCmlogHdlr exit

- Messages discarded due to full queue

3.2.4 Major Routines

The following functions will reside in and be documented as iocCmlog.c, iocCmlog.h, and iocCmlog.html under $EPICS/site/src/iocCmlog. A library called libiocCmlog will be built by a standard EPICS Makefile in this directory and installed under $EPICS/site/lib/<target> for all EPICS targets. IOC applications will include this library in their builds. The iocCmlog.html file will be a standard EPICS distribution web page for iocCmlog and will have build and usage instructions plus a link to this design document. The iocCmlog.html file will be installed in the web directory by the Makefile. Once iocCmlog is finished, a link to iocCmlog.html will be added to the EPICS collaboration soft support page:

http://epics.aps.anl.gov/epics/modules/soft.php

3.2.4.1 iocCmlogHdlr(no args)

At initialization, iocCmlogHdlr:

(a) Creates a message queue by calling epicsMessageQueueCreate. It uses a file-scope value (default to 100, changeable by iocCmlogHdlrQSize) for message queue size. The queue ID is updated in file-scope space which is initially set to 0. If an error occurs or the queue is already created, printf is called and the thread exits.

(b) Logs a startup message (disabled using a compiler switch) by calling iocCmlogLogMsg with facility_c = "EPICS", status = COS_ALARM, severity = NO_ALARM, doPrintf = 1, status_c = NULL, destination_c = null, filename_c = null, lineno = 0, format_c = "iocCmlogHdlr Startup".

(c) Registers the errlog listener callback by calling errlogAddListener with iocCmlogListener as the listener and a null private pointer.

(d) Finds the IOC name and uses it as host for later message logging. The result of getenv("IOC_NAME") is used. If the "IOC_NAME" environment variable does not exist, the host tag is not included in the call to cmlog_logmsg and cmlog_logmsg will set it.

(e) Calls cmlog_open to get a client handle and sets it to 0 if an error is returned.

It then loops waiting on a message using epicsMessageQueueReceive. For each message, it:

(a) Calls cmlog_logmsg with inputs from the message. The format string and arguments depend on which tags are available for this message. At a minimum, the tags include text, status, and severity. If the compiler switch is on that requires the code tag to be a string instead of an integer, then the code tag is included. If host could be determined during initialization, then the host tag is included. Other optional tags are provided if they could be determined for the specific message being logged and include codeI (for VMS style status codes), process, dest, file, and lineno.

(b) If the doPrintf flag is set for this message, a printf is also done with process name, status string, and formatted text string.

(c) If the status is COS_ALARM and the text is “iocCmlogHdlr Exit”, calls errlogRemoveListener, epicsMessageQueueDestroy, and cmlog_close. It then sets the queue ID to 0 in file-scope space and exits.

(d) If there are no messages pending in the queue (epicsMessageQueuePending) and there have been discarded messages by iocCmlogVLogMsg due to a full queue, then a message is logged by calling iocCmlogLogMsg with facility_c = "EPICS", status = WRITE_ALARM, severity = MINOR_ALARM, doPrintf = 1, status_c = NULL, destination_c = null, filename_c = null, lineno = 0, format_c = "iocCmlogHdlr = %ld messages were discarded", and the number of missing messages as the argument. The number of missed messages is then set to zero.

3.2.4.2 iocCmlogHdlrQSize(const int queueSize)

This is called from the startup file before iocCmlogStart and sets the file-scope queue size for the iocCmlogHdlr message queue. It cannot be less than 32 or greater than 10000. Note that if iocCmlogHdlr is already started, it will need to be manually stopped and restarted for the value to take effect.

3.2.4.3 iocCmlogStart(no args)

This is called from the startup file or from IOC shell. If the iocCmlogHdlr queue ID is 0 in file-scope space, it starts iocCmlogHdlr using epicsThreadCreate and calls printf if error. Arguments to epicsThreadCreate are:

(a) Name = "iocCmlogHdlr"

(b) Priority = epicsThreadPriorityLow

(c) Stack Size = epicsThreadStackSizeMedium

(d) Funptr = iocCmlogHdlr

(e) parm = null

3.2.4.4 iocCmlogDStart(no args)

This is called from the vxWorks or RTEMS startup file. If the cmlogClientD is not already started (epicsThreadGetId), it starts cmlogClientD using epicsThreadCreate and calls printf if error. Arguments to epicsThreadCreate are:

(a) Name = "cmlogDaemon"

(b) Priority = epicsThreadPriorityLow

(c) Stack Size = epicsThreadStackSizeMedium

(d) Funptr = cmlogClientD

(e) parm = null

3.2.4.5 iocCmlogStop(no args)

This is called from IOC shell. If the iocCmlogHdlr queue ID is not 0 in file-scope space, it calls iocCmlogLogMsg with facility_c = "EPICS", status = COS_ALARM, severity = MAJOR_ALARM, doPrintf = 1, status_c = NULL, destination_c = null, filename_c = null, lineno = 0, format_c = "iocCmlogHdlr Exit".

3.2.4.6 iocCmlogListener(void *notused_p, const char * const text_c)

This is called by the EPICS errlog task for every EPICS message. It calls iocCmlogLogMsg with facility_c = "EPICS", status = NO_ALARM, severity = ALARM_NSEV (to default to INFO), doPrintf = 0, status_c = “INFO”, destination_c = null, filename_c = null, lineno = 0, format_c = input text.

3.2.4.7 iocCmlogRegistrar(no args)

This is called during EPICS and registers iocCmlogHdlrQSize, iocCmlogStart, iocCmlogDStart, iocCmlogMsg and iocCmlogStop using iocshRegister so that they can be called from IOC shell. This routine requires a database definition file called iocCmlog.dbd with one line:

registrar(iocCmlogRegistrar)

IOC applications will need to include iocCmlog.dbd to use the iocCmlog service.

3.2.4.8 iocCmlogLogMsg

iocCmlogLogMsg(const char * const facility_c, const unsigned long status, const int severity, const int doPrintf, const char * const status_c, const char * const destination_c, const char * const filename_c, const unsigned long lineno, const char * const format_c, …)

Calls iocCmlogVLogMsg() with the va_list argument determined by va_start(). See section 3.2.4.9 for detailed description of arguments.

3.2.4.9 iocCmlogVLogMsg

iocCmlogVLogMsg(const char * const facility_c, const unsigned long status, int severity, const int doPrintf, const char * const status_c, const char * const destination_c, const char * const filename_c, const unsigned long lineno, const char * const format_c, va_list format_args_p)

This method is called by slcCmlogVLogMsg() and by iocCmlogLogMsg().

It creates a structure with the following elements and, if the iocCmlogHdlr queue ID in file-scope space is not zero, adds it to the iocCmlogHdlr message queue by calling epicsMessageQueueTrySend. If epicsMessageQueueTrySend returns an error, the number of missed messages is incremented. If the queue ID is zero and the doPrintf argument is set, printf is called with the process name, status string, and formatted message string. The status from the queuing is returned.

(a) Facility:

Same as the input facility string. If null, set facility to "EPICS". Maximum size is 8 characters.

(b) Destination:

Input destination name string. If destination name is null, set it to a zero-length string. Maximum size is 8 characters.

(c) CodeI:

Input status.

(d) Text:

Constructed using epicsVsnprintf (see $EPICS/base/include/epicsStdio.h) with the input message format string and input variable arguments. Maximum size is 256 characters.

(e) Process:

Set to epicsThreadGetNameSelf(). If not available, set to a zero length string. Maximum size is 16 characters.

(f) File:

Input filename. If filename is null, set it to a zero-length string. Maximum size is 96 characters.

(g) Line:

Input lineno.

(h) Verbosity:

If the input severity is -1, verbosity is determined from the lower 3 bits of the input status which is assumed to be a VMS condition code:

Lower 3 bits of Condition Code	Verbosity	CMLOG Severity	Severity Tag
0 = warning	3	1	WARN
1 = success	5	0	SUCC
2 = error	2	2	EROR
3 = informational	4	0	INFO
4 = fatal	0	2	FATL
>= 5	4	0	INFO

If the input severity is positive, it is assumed to be EPICS and verbosity is determined from this table:

Input EPICS Severity	Verbosity	CMLOG Severity	Severity Tag
0	5	0	NO_ALARM
1	3	1	MINOR_ALARM
2	2	2	MAJOR_ALARM
3	1	3	INVALID_ALARM
>=4	4	0	INFO

(i) Severity String:

See severity tag from tables above. $EPICS/base/include/alarmString.h has the table of EPICS severity strings. Maximum size is 16 characters.

(j) CMLOG Severity:

See table above. Defines for severity are provided in $EPICS/base/include/alarm.h.

(k) Status String

For a VMS status, the status string is constructed by concatenating Subsystem, '-', first character of severity tag, '-', input status string. If the input status string is null, default it to the severity tag. Subsystem is determined from the lower 12 bits of the top 2 bytes of the VMS condition code and matches one of the facility values in the SLC control system message files. If subsystem cannot be determined from the condition code, use facility.

For an EPICS status, the status string is either the input status string, if non-NULL, or the string from the alarmStatusString table in alarmString.h. If the EPICS status is >= ALARM_NSTATUS, it is set to “INFO”.

Maximum size is 32 characters.

(l) Bit Mask

Bit mask with bits set for doPrintf, EPICS (don’t use the codeI tag) or VMS (do use the codeI tag), process name available, destination available, and file name available.