Review of Epics
Libraries and Resource Usage
The LCLS slc-Iocs are being
developed with a combination of independently programmed slc threads and Epics
Operating System Independent libraries (libCom and libcom OSI). The independent
slc threads may be stopped and started repeatedly while the IOC itself remains
up. Concern was expressed in controlling
and managing resources used by these threads, such as heap memory, message
queues, semaphores, socket and file descriptors, and threads; including those
allocated within the Epics libraries.
The libCom and libCom OSI
libraries are documented in the IOC Application
Developer's Guide. The documentation,
in most cases, is a quick summary of the library utilities usage and the
interface. Generally the documentation
does not cover resource usage within the libraries.
This report is an overview
of resource management within the Epics Libraries used (or may be used) by the
slc-Ioc developers. The report ends with
a set of suggestions for resource management within the slc-Ioc threads.
LIbCom Utility Libraries
The epics libraries have
been written well in terms of resource management. Those utilities that allocate resources such
as heap memory and mutexes usually have a ‘create’ or ‘init’ routine to perform
the allocations, and a ‘destroy’, ‘free’ or ‘release’ routine that
de-allocates. It would be good practice
for the programmer to review the source of any epics library utilized, and
ensure that the library routines are used properly and the de-allocation
routines are called.
The epics libcom and LibCom
OSI sources are in the directories
$EPICS_BASE/src/libCom, and
$EPICS_BASE/src/libCom/osi
The following sections give
an overview of the utilities that may be useful for the slc-Ioc threads. They are listed by their directory under
/libCom.
bucketLib
Hash facility for integers,
pointers and strings. Used by Channel access. This library uses the freeList
and epicsAssert utilities (see below) which protect the heap memory allocated
by the bucketLib routines, and will ultimately suspend the calling thread if
any memory allocation errors occur.
dbmf
memory manager for
applications that allocate and free memory frequently. dbmf uses ellLib as list utility, and uses
epicsMutex to protect it’s list and nodes during the function calls, and will
suspend the calling thread if any memory allocation errors occur.
ellLib
A general purpose linked
list library. This library uses
epicsAssert (see OSI below). It DOES NOT
use mutexes, or any method, to protect the list or nodes in a multi-threaded
environment.
freeList
A general purpose freemem
list library that uses epicsMutex to protect its list during function calls.
The thread uses cantProceed, which will suspend the thread if a memory
allocation error occurs.
gpHash
A general purpose hash
utility. It uses ellLib for the list,
and epicsMutex to protect the list during function calls. It uses cantProceed and epicsAssert which
will suspend the calling thread for memory allocation and assert errors.
It should be noted that the
mutex does not protect the memory linked into the list by the calling thread
once the gpHash function has completed. This is true for any of the hash, list,
or freelist libraries the slc threads may use.
As an example problem:
Some method of protecting
the structure must be added, possibly similar to the message queue technique of
copying the data, rather than just returning a pointer. See the Resource
Management … section below for further suggestions.
misc
A set of miscellaneous
include files, functions, and definitions. Documented in the IOC Application
Developer’s Guide. Some that look useful for slc threads:
osi
This directory includes the
operating system independent libraries (libCom OSI) that ultimately have
different implementations depending on the operating system being used. Mutexes, threads, timers etc. are implemented
dependent on the capabilities of the underlying operating system. The subdirectories, by OS name, contain the
OS dependent implementations.
The following libraries are included:
Ring
See documentation in the IOC
Application Developer’s Guide. epicsRingBytes
and epicsRingPointer do not employ any method to protect the ring buffer in a
multi-threaded environment. Calling threads should implement with read and
write locking Mutexes, unless there is a single reader and / or a single
writer.
test
Contains test programs for
several of the libCom utilities.
timer
a Timer facility. This is very well documented in the IOC
Application Developer’s Guide.
Resource management, thread safety,
and the C programming language
On most operating systems, in
a multi-threaded environment, global variables are available to all threads
within a process, unless defined ‘static’.
A static variable is global to all functions below the definition point
within the file it is defined. Heap
memory is shared by all threads within a process. Other resources, such as
sockets, file descriptors, mutexes, and message queues are accessible by all
threads if their references are assigned to a global structure. The following
are some suggestions when programming for thread safety and resource management.
Globals
Threads
Shared resources
Heap memory allocations
Other recommendation should
be added here as the program evolves.