LCLS Controls

SLC-Aware IOC Design

---

Async Utilities Design

Quick links:

• Introduction
• Async Utilities

1. Scope

This document describes the software design for the SLC-aware IOC Async Utilities. These utilities are for use by cstrAsync, cstrHdlr, and mgntHdlr.

2. Introduction

2.1 Background

Job Global data (including slcJobFunc_as) in file scoped within the .c file containing these utilities. That data is protected by being accessible only by calling these utilities. Again, access to the global area by async handler threads is always through an Async utility.

Much of the functionality here is implemented for the SLC Micros in utilities whose source lives in REF_RMX_ASYNC.

The following table shows the names of Functions on the SLC RMX micros and the Functions on the SLC IOC which accomplish roughly the same functionality.

SLC Micro Function(s)	SLC IOC Function
ASYNCDBINIT INIT_CYC_GLBCOM CYCLE_INIT	slcAsyncInit
INIT_MICR_STS	cstHdlrInitMicrSts
GET_CYCLE_TIME	None
GET_CYCLE_FCN	slcAsyncGetCycleFcn
METER_DBSEND	slcAsyncMeterDbsend
PROC_MICR_STS	cstrHdlrProcMicrSts
LOCK_STATS	slcAsyncLockStats
UNLOCK_STATS	slcAsyncUnlockStats
SET_FCN_STATS	slcAsyncSetFcnStats

2.2 References

1. SLC Asynch Database Update Design Spec by T Lahey, N Spencer 1989

2. Improving Control of Auto-Checking Functions by T Lahey,N Spencer, R Hall 1990

2.3 Requirements

See the SLC-Aware IOC Functional Requirements by Stephanie Alison.

3. Async Utilities Design

General sketch for all Async Handler Threads that perform cycling

This is the general pattern followed by All Async Handler Threads including (but not limited to) cstrHdlr and mgntHdlr.

Prior to an Async Handler thread running, slcAsyncInit() is called by cstrHdlr at it’s initialization time to set up the Async Global area once for all async Job/functions.

These are the calls that Async Handlers should make for handling cycling functions (Handlers may do other things like receiving VMS messages. Those things are not listed):

LOOP:

· If slcThreads_as[thisHdlr].stop ==epicsTrue, clean up and return.

· Receive an IOC_ASYNC message from cstrAsync which indicates the cycling function to perform. The function index and name is part of the message. The time of receipts is received as part of that call too.

· Call a local cycling function based on the name from the message. The cycling function returns dblputDone=epicsTrue if any dbupdates were done. It returns dblputDone=epicsFalse otherwise.

· Call slcAsyncMeterDbSend(funcIx, requestType=slcAsync, dblputDone). This is called whether or not the cycling function did any dblputs. It does dbupdate() if needed.

· Call slcAsyncSetFcnStats(funcIx, epicsTime, requestType=slcAsync) to update CSTR related statistics in job global.

Exists Flag and Mutex Used by Async Tasks

1. Boolean flags.

· asyncExists is a flag in file scope. It is set to False by the executive before cstrAsync is run using an slcAsync utility function.

At the end of it’s processing, slcAsyncInit() sets the asyncExists flag to True to indicate that the slcJobFunc global table creation is complete. If slcAsyncInit fails, it sets the asyncExists flag to “false.

Public slcAsync utilities check if the async global area has been set up successfully by calling slcAsyncReady() which checks the asyncExists.

slcAsyncExit() sets the asyncExists flag to “false”.

· The other flags of interest are the active and stopped flags in slcThread_as.

2. Mutex

There is a single mutex (asyncRWMutex) which is used to protect access to the Global Data by utility functions which write to that area. The mutex insures that the utilites are thread-safe. slcAsyncLockStats() and slcAsyncUnlockStats() are utilities for use obtaining and releasing the mutex.

Message Logging

Messages to be logged are stated below under the individual utility descriptions.

Function Return Status

Most utilities return int unix-style status set to 0 (for success) or non-zero (for failure).

Resource Management

Writes to the SLC database will be metered in a manner similar to the SLC Micro Test Job by calling the utility function slcAsyncMeterDbsend().

Global Data

slcJobFunc_as is an array of structures in file scope, with one structure (array element) per cycling function across the entire IOC. Each Job may have zero or more cycling functions. The array is calloc'ed by slcAsyncInit() to have one entry per cycling function across the current slc ioc. The pointer to this array is static and file scoped.

slcJobFunc_as and the associated set of dblist pointers are often referred to as “Global Data” within this document. It is called “Job Global” to correspond with the SLC Micro code.

There is a file scoped Boolean flag called asyncExists and a mutex that both provide protected access to Job Global (see descriptions above).

It is important to note that in the CSTR database, all functions for a given Job must be grouped together (example: CSTR:micr:1,CNAM may NOT indicate a mixed grouping like this: KLYS-TRMP CRAT-WATC KLYS-FCHK...)
slcAsyncInit issues an error message and returns bad status if the functions are not all grouped together for all jobs.

Two new entries will be added to slcJob_ts.

· The first is "first cycling function index". This is an integer index into the array slcJobFunc_as. This integer is passed to async functions as an input argument.

· The second is "number of cycling functions". This is also an integer.

The following table describes the association between slcJob_as and slcJobFunc_as.

Async Utilities below look at all cycling functions for a given job.
That is done by looking in slcJobs_as[jobEnum] for the current job, and picking up the first cycle function index and number_of_functions. Those two values are used to inspect the slcJobFunc_as and associated arrays of secondary values.

The following structure is slcJobFunc_ts. To the right is the initial value for each element as set by slcAsyncInit.

INITIAL VALUE

--------------

/* Async cycling function information */

typedef struct {

slcJob_te slcJobId_e; /* enum into slcJob_as for this job */ COMPUTED based on CNAM

char funcName_c[5]; /* null terminated 4-character func name */ READ FROM CSTR:CNAM

unsigned long cyclTimeSec; /* cycle time from CSTR:CYCL */ READ FROM CSTR:CYCL

unsigned long mtrlSec; /* dpupdate meter length in seconds */ READ FROM CSTR:MTRL

unsigned long mtrcSec; /* dbupdate meter max # dbupdates in mrtl */ READ FROM CSTR:MTRC

unsigned long mtrtSec; /* dbupdate meter max time between dbupdates */ READ FROM CSTR:MTRT

epicsTimeStamp meterStartTime_s; /* time metering started */ current time

unsigned long meterCount; /* metering counter */ 0

vmsStat_t lastDbupdateStatus; /* last database update status */ DB_OKOK

epicsTimeStamp updateTimeStamp_s; /* timestamp of last dbupdate */ current time

epicsTimeStamp cycleTimeStamp_s; /* last time this cycl function was executed */ current time

float betweenCycle; /* time between last 2 cycles */ 0.0

unsigned long numFuncExe; /* total # function executions in calc interval */ 0

unsigned long numVmsMsg; /* # of function executions from vms msg interv */ 0

unsigned long numDbUpdates; /* # of dbupdates (for metering and %) interval */ 0

unsigned long numDbUpdateFail; /* # of dbupdates that failed in calc interval */ 0

epicsTimeStamp sendTimeStamp_s; /* NEW time when last msg sent attempt by cstr to hdlr */ current time

unsigned long sendErrCount; /* NEW number of failures in sending to hdlr. Not zeroed except at init. */ 0

} slcJobFunc_ts;

Diagnostics

Write Async job/function table for an input job name (can be ALL*) to the console.

Provide a way for the user to change a subset of cycling parameters for a job/function pair. Use utilities slcAsyncSetXXXX described below to set them. They must lock the global area before writing it.

· One utility will zero the sendErrCount in job global so the user can track send failures.

Async Utility Function descriptions

The Async utilities all exist in a single file called libsrc/util/slcAsync.c. Prototypes are in slcAsync.h

Utilities are thread-safe. This is accomplished by using a single mutex to protect access to the global data.

It is necessary to restart the SLC IOC in order to add a new cycling function. But, all other cycling parameters (besides CNAM) from the database like CYCL and MTRx are kept in job global and used by Async utilities. Every time those parameters are used, they are read from Job Global. Thus, for an existing cycling function, those items are changeable from the ioc console.

· int slcAsyncInit(void)

Called by cstrAsync at initialization time to set up the Async Global area before any Handler threads are run. If slcAsyncInit returns bad status, cstrAsync exits. The global area is actually a job/function called slcJobFunc_as in file scope (see above for more on this table). slcAsyncInit is not called from anywhere else.

slcAsyncInit corresponds to several functions on the slc micros as shown in the table above.

slcAsyncInit returns 0 if successful. It returns -1 (error) if there are errors returned by memory allocation, database calls, or the CSTR data fetched from the SLC database.

Call slcAsyncInitVerify() to get cycling parameters from the database, check them, and allocate/initialize slcJobFunc_as. If it returns bad status, goto egresss and do the flag processing at the end of this function which will cause us to return bad status and cause the thread to exit..

Allocate the mutex (asyncRWMutex_ps) for use in accessing the Global Area by calling epicsMutexCreate

If everything above went OK,

Set asyncExists local flag to “true” using slcAsyncSetAsyncExists(epicsTrue);.

· int slcAsyncInitVerify(void)

Get cycling parameters from the database, check them, and allocate/initialize slcJobFunc_as for all job/functions.

Return 0 for success, -1 for failure.

A failure is considered fatal and the caller should cause cleanup and exit to occur. Any of the following cases (and probably more) are considered fatal.

· Invalid job name in CNAM

· Invalid ordering in CNAM

· Any error from any db utility.

· Any error from getting the mutex. Or memory allocation error.

· Sizes of dblists and dbdatas are not as expected.

· Problem initializing a timestamp

This is a static function (called only by another function within this .c file).

The following secondaries values are read (one time only) from the CSTR SLC database primary.

Call dblistalloc(), dblist() and then dblget(CSTR,[this_micro],1,SECN) which, in one call (per secondary) returns lists (arrays) of each value of size SLC_MAX_N_ASYN. Calloc slcJobFunc_as based on how many functions are in the CNAM list. These are the secondaries obtained from the database:

§ CNAM - contains Job Name and Cycle Function.

§ CYCL - is the cycle length in seconds.

§ MTRL, MTRC, and MAXT - database write metering parameters.

If any of the DB calls fail, then return bad status.

Loop through the dblists and do the following steps

1. As explained above, the slcJobFunc_as array is calloc'ed by this function to have one entry per cycling function across the current slc ioc.. Cycling functions for a given Job are grouped together in slcJobFunc_as Check that the job name in CNAM is valid and matches a job name in the slcJob_ts array. Log a fatal error if there's no match and break out of loop.

2. Store CNAM (job and function) into slcJobFunc_as. Compute the index to slcJob_as for each functions job and store it in slcJobFunc_as[].slcJobEnum.

3. Check CYCL, MTRC, MTRL, MAXT (dblist/dblget was already done ),
against reasonable limits. Any value which is outside reasonable limits is reset to an appropriate default (see the supertype I table in the requirements doc for defaults). Store values in slcJobFunc_as using the slcAsyncSetXXX() functions below (for example call slcAsyncSetCYCL()). Log a message if the values are set to defaults stating DB value and default it was set to. Return good status in that case.

4. Increment function counter in slcJob_as for this job. Set function index in slcJob_as if first function for job.

Call dbListFree() to free the one-time use lists obtained above for all the DB lists.

Call dblistalloc() and dblist() to get MMSK, CMSK, HSTA, FMSK, and SCAN Supertype II lists. Don’t do dblget() yet. That will be done by slcAsyncUpdSt2() which is called by the main thread right after slcAsyncInit().

· int slcAsyncGetCycleFcn (int 1stCycleFuncIx, int numCycleFunc, int* cycl2Exe, char * cycl2ExeName_c, epicsTimeStamp currentTime_s);

This is a public function called by cstrAsync. It returns (in cycl2Exe and cycl2ExeName_c) the slcJobFuncIx (index and name) of the cycling function which is “most ready to run” for the current job. It does this by going through all entries in Job Global for the current job as indicated by 1stCycleFuncIx and numCycleFunc. If one is not found, then -1 is returned in cycl2Exe.

Call slcAsyncReady() and return 0 if it returns 0.

Get current time by calling eicsTimeGetCurrent() and set into output parameter currentTime_s. Return if error.

Call slcAsyncLockStats() to get a mutex on the Global Area. Return if it returns an error.

Go through entries in slcJobFunc_as (using the input arguments to index it) and see if there an async function for this job that is ready to run based on current time, CSTR: CTIM, CYCL, SCAN, HSTA, CMSK, and MMSK. Return an indication as to which of these async functions is “most ready to run” to the caller in cycl2Exe and cycl2ExeName_c.

For a cycling function to be ready to run, the following conditions must be true.

§ Cycling time (slcJobFunc_as[].cyclTimeSec) must be > 0

§ Current_time – last_execution_time > cycling_period

§ Current_time – time_of_last_msg_send > cycling_period (don’t keep trying if q full)

§ The cycling_period is either CYCL or SCAN, if the bit for the function is set in MMSK and SCAN is less than CYCL.

§ The function must be enabled as determined by checking the appropriate bit in CMSK.

§ HSTA is only checked for the magnet job. The TEST job is not HSTA’able. The magnet job must be enabled as determined by checking the appropriate bit in HSTA. Note: because there is no direct correspondence between CycleFuncIx (input parameters) and HSTA, it will be necessary to use a hard codeed mask to know which Bit to check in HSTA.

Egress: Call slcAsyncUnlockStats() to release the mutex on the Global Area.

· void slcAsyncSetCYCL (jobFuncIx, value)

set CYCL in slcJobFunc_as[jobFuncIx] to the value passed in.

Check for valid values. And use default if needed. Defaults listed in requirements document.

· void slcAsyncSetMTRC (jobFuncIx, value)

set MTRC in slcJobFunc_as[jobFuncIx] to the value passed in.

Check for valid values. And use default if needed. Defaults listed in requirements document.

· void slcAsyncSetMTRL (jobFuncIx, value)

set MTRL in slcJobFunc_as[jobFuncIx] to the value passed in.

Check for valid values. And use default if needed. Defaults listed in requirements document.

· void slcAsyncSetMAXT (jobFuncIx, value)

1. set MAXT in slcJobFunc_as[jobFuncIx] to the value passed in.
2. Check for valid values. And use default if needed. Defaults listed in requirements document.

· int slcAsyncProcChk1(CTIMdblist_ps, CTIMdbdata_ps, UTIMdblist_ps, UTIMdbdata_ps, ELPSdblist_ps, ELPSdbdata_ps )

This is a public function called by cstrHdlr only. It processes UTIM, CTIM, and ELPS.

These secondaries are always sent (updated) without checking if they changed because UTIM and CTIM change when this function runs.

Call slcAsyncReady() and return if it returns an error.

Call slcAsyncLockStats() to get a mutex on the Global Area. Return if it returns an error.
For each cycling function in the Global Area do the following and dblput the data:

Put the last database update time from Global into the UTIM dblist.
Put the last cycling function update time from Global into the CTIM dblist.
Put elapsed time from Global in the ELPS dblist.

Timestamps are converted to VMS format using timeEpicsToVms() before putting them into the slc database.

cstrHdlr will do the dbupdate.
Call slcAsyncUnlockStats() to release the mutex on the Global Area.

Any failures in the calls above logic cause a -1 (error) return status. Otherwise 0 is returned (success).

· int slcAsyncProcChk2(NRUNdblist_ps, NRUNdbdata_ps, FAILdblist_ps, FAILdbdata_ps, PUPDdblist_ps, PUPDbdata_ps, PVAXdblist_ps, PVAXdbdata_ps, epicsBoolean * dbUpdateNeeded_ps)

This is a public function called by cstrHdlr only. It processes NRUN, FAIL, PVAX, and PUPD.

dbUpdateNeeded is an output parameter.

Call slcAsyncReady() and return if it returns an error.

Call slcAsyncLockStats() to get a mutex on the Global Area.

For each cycling function in the Global Area:

If Number of Executions (NRUN) has changed in Global vs what’s in the previous used dblist, put the new value from Global into the NRUN dblist and set dbUpdateNeeded flag that something changed

If number of failures (FAIL) has changed in Global vs what’s in the previous used dblist, put the new value from Global into the FAIL dblist and set dbUpdateNeeded flag that something changed.

Calculate Percent executions triggered by Vax Messages (PVAX). If it has changed, store it in the PVAX dblist and set flag.

 Calculate Percent executions triggering successful database updates (PUPD). If it has changed, store it in the PUPD dblist and set flag.

Do dblputs as needed for data listed above.

Zero the counters for NRUN, FAIL, DBUPDATES, and VAXFUNCS in global.
Call slcAsyncUnlockStats() to release the mutex on the Global Area.

Any failures in the calls above logic cause a -1 (error) return status. Otherwise 0 is returned (success).

· int slcAsyncUpdSt2(void )

This is a static function (called only by another function within this .c file).

Call slcAsyncReady() and return error if not ready.

Call slcAsyncLockStats() to get a mutex on the Global Area. Return and error if it returns an error.

Get MMSK, CMSK, HSTA, FMSK and SCAN from database using dblget()

Pointers to the dblget lists are already set up in file scope.

Return -1 if a dblget fails. Return 0 for success.

Call slcAsyncUnlockStats() to release the mutex on the Global Area. Return an error if it returns an error.

· epicsBoolean