Communication with the SLC Control System passes either through the Database Service or through the SLC Message Service. The SLC IOCs must mimic the iRMXmicros in control and timing, to minimize changes in the SLC Control System. This document discusses the design of the Linac Coherent Light Source (LCLS) Stanford Linear Collider (SLC) Input/Output Controller (IOC) Database Service (DBS). Specifically, the Database Service design, Database Input/Output Utilities, and Console commands are discussed. The database service implementation described herein could be incorporated into any SLAC subsystem that communicates with the SLC Control System and utilizes the EPICS toolkit for device control.

2 Introduction

The SLC IOC emulates many aspects of the SLC iRMX micro (micro), including maintaining its part of the VMS SLC database. Note that “database” is used to describe a “flat file architecture” throughout this document, and does not refer to a commercial or other relational database. Like a SLC micro, the SLC IOC DBS receives a piece of the entire SLC operational database at initialization time whose functionality is configured for it by the dbgen/dbinstall utilites running on the Alpha. Data is transmitted in a series of byte streams, called supertype (ST) blocks, or superblocks, over a TCP/IP LAN from the DBEX process running on the Alpha.

The data sent to the SLC IOC (and micro) is divided into four sections, physically in memory, according to its function and read/write attributes. Supertype 0 consists of pointer information (actually 4 byte offsets into the other ST 1-3 data blocks) by which each primary/unit/secondary name gets translated into memory address. (Primary/unit/secondary names are part of a naming convention which identifies a particular device.) Supertype 0 information gets written on the Alpha only at dbgen/dbinstall time. The other three Supertypes (1-3) of concern to the SLC IOC contain actual data. Supertype 1 is assumed to be stable, only changeable by the Alpha, and consists of configuration values. Supertype 2 data consists of setpoint values; any update of a Supertype 2 item in the control system originates only in VMS and causes both an update in memory in VMS and a message to the micro/ SLC IOC to do the same update in the micro/ SLC IOC copy of same memory. For the SLC IOC (as opposed to a micro), the EPICS database can also update Supertype 2 data, causing the SLC IOC to update both its local copy in its SLC database as well as send the Supertype 2 data to the Alpha. Any update of a Supertype 3 item originates in the micro/SLC IOC and causes an update in the micro's memory. A separate call sends accumulated updates to the Alpha to be stored in the corresponding VMS memory by DBEX. Any update of a supertype 4 item originates only in VMS and is never seen by the micro/SLC IOC (i.e., any micro/SLC IOC’s supertype 4 data resides only in VMS memory for use by application code in VMS).

Those familiar with the LINAC and PEPII iRMX micro computer code will recognize that most all “dbmain” functionality has been ported to the SLC IOC DBS to run on top of an EPICS OSI layer. The dbmain functionality has been split between dbRecv, dbSend, and dbHdlr threads, responsible for receiving, sending and handling data, respectively, from/to the Alpha.

2.1 Background

The Database Service provides most, if not all, of the functionality contained in the iRMX micro’s REF_:[RMX.DBS] DBMAIN.C module.

2.1.1 General DB Service Design Concepts

· Amount of change to VMS should be minimal.

· Use existing VMS include files wherever possible; refer to VMS directory REF_:[RMX.INCLUDE] *

· Port existing iRMX application code (c, fortran, assembly) wherever applicable.

· Goal – generate same amount of network traffic as SLC micros

· More memory is available for use on SLC IOC – use to advantage for database storage to make database access quicker, if possible.

· Higher performance CPUs are being used, compared with micros – use to advantage

· Use EPICS libcom OSI libraries and platform independent libCom facilities to reduce coding time and to benefit from collaboration knowledge

· Account for different platform byte orientation. VMS packs; solaris and vxWorks (and many 32 bit architectures) byte align on 4-byte boundaries.

o The Supertype byte stream header (supertypehdr_ts), shared between VMS and all micros/SLC IOCS, is problematic, as it has three 2-byte words followed by four 4-byte words. Supertype header sent to/from VMS contains no padding.

· Messages from VMS control system are little-endian, VMS format, and packed

· Support ASCII representations for primary, secondary names and units

· SLC IOC needs supplementary data definition:

o primary ASCII to #

o secondary (secn) ASCII to # mappings for given primary

o datum width (for I or Z)

2.2 References

References for various areas are listed below.

DBEX:

(References for handling data traffic between the SLC IOC and DBEX process executing on the Alpha control system):

· Use VMS procedures in REF_:[RMX.DBS] DBMAIN.C and REF_:[RMX.SLCLIB] exnet.c as the design basis for interface to the DBEX Server process on the Alpha.

· Use VMS assembly code REF_:[RMX.DBS] DBLISTU.A38 as the design basis for dblist and dbunits for accessing SLC supertype 0 database (used in dbIPL.c)

· Also, talk to the experts – Tony Gromme, Ken Underwood, Nancy Spencer, Ed Miller, Ron McKenzie, Ron Chestnut, Stephanie Allison, Terri Lahey.

EPICS references :

· IOC Application Developer's Guide, Release 3.14.6

· Where $LCLS = /afs/slac.stanford.edu/g/lcls/vol1/epics/site/src/slc, use $LCLS/msg/dbSend.c as a design reference for a queued thread and $LCLS/msg/msgRecv.c as design reference for a socket reading thread.

Also see SLC Executive and Message Service Design Presentations

Legacy SLC Database:

POOP: Database Internals
BUG: Database
PG: Database
Introduction to the SLC Database by Ken Underwood, 1988
Notes on Existing SLC Operational Database by Tony Gromme (includes notes on TCP Proxy, 3/2004)

SLC-Aware IOC:

SLC-Aware IOC webpage

SLC IOC Message Service Design

SLC IOC Utilities

2.3 DB Service Functional Requirements

Please refer to: SLC Database Service (DBS) Functional Requirements , by Stephanie Allison, for the complete list of Database Service requirements.

2.3.1 DB Service Use Cases

DBEX

“Look like” a tcp micro – SLC IOC has the same functional requirements as an iRMX micro
Same message formats, traffic, use proxy
Req and Accept database (“IPL”)
Accept updated setpoints from Alpha
Provide updated data from IOC

Supertype 3 (ST3) readbacks
Supertype 2 (ST2)setpoints (new)

Accept and reply to diagnostic inquiries
Accept and Process DBEX up/down messages (up contains db version) (new)

Console Users

Utilities to edit, dump, get type of various slc data

Application code

Utilities to find, get, put, update slc data
Coded “data flow dependancies”

“Job” threads updating Alpha with ST data should wait for round trip confirmation that DBEX received it prior to acknowledging back to Alpha via the message service

This dependency surfaced in the design review

??? What else

As each “job” specification (bpm, magnet) becomes available, make sure that DBS meets its needs

3 Design

3.1 Service Description and Block Diagram

The primary function of the DBS is to keep its portion of the SLC database - shared data between the Alpha Control System and the SLC IOC – up to date. It accomplishes this by communicating to the DBEX process on the Alpha Control System. All of the database message communications between the SLC IOC and Alpha Control System is handled by the dbSend and dbRecv threads on the SLC IOC, and the DBEX process running on prod or dev Alpha. The SLC database transactions between VMS and SLC IOC occur through a TCP proxy.

There are two TCP/IP sockets per SLC IOC that handle DBS communications (“dbex” and “ctl”). These sockets are connections to a proxy, which also maintains two socket connections with DBEX. The proxy forwards data from DBEX to the proper SLC IOC after interpreting a proxy message “forward” header. Thus, the proxy allows DBEX to make two single socket connections in order to communicate with many micros/SLC IOCs.

The dbex socket is bi-directional and handles initial database downloads, updated data traveling in both directions, diagnostic messages, acknowledgements, and dbex up/down messages. The ctl socket is uni-directional and exists solely for DBEX to acknowledge back to the SLC IOC/micro that it received an update to the database.

DB Service consists of the dbSend thread, dbHdlr thread, dbRecv thread, database Input/Output utilities, database Console Commands, and DBS Utilities. The DB Input/Output utilities are global to all threads (refer to Specification for Database Input/Output Routines), while the DBS Utilities are used only by the db* threads and generally contain conversion utilities.

The Device Service function is shown in the DBS Block Diagram

3.1.1 DB Service Block Diagram

The functionality contained in file REF_:[RMX.DBS] DBMAIN.C has been split into dbRecv and dbSend threads, while dbHdlr thread handles new data manipulations for the SLC IOC. The dbSend and dbHdlr threads are “queued message” threads, primarily taking direction from messages sent to their respective message queues. The dbRecv thread is considered a “socket” thread, as it primarily takes direction from incoming network messages from DBEX on the dbex socket.

dbRecv’s prime responsibility is to wait at the dbex socket for network messages from DBEX. It responds to incoming data messages by storing data, in VMS format, as well as by sending queued messages to dbSend to acknowledge DBEX back (when DBEX requests, via a flag in the supertype header) on the dbex socket. Note that currently, acknowledges back to DBEX are only requested by DBEX during IPL, and not during ST 2 data updates. After the SLC IOC receives a database download at startup, it sends a message to dbHdlr to convert the data.

dbSend takes direction from queued messages sent from the slcExec, dbRecv, and “device service” threads. As part of carrying out its functions, it sends all data to DBEX via the dbex socket. For outgoing network messages consisting of supertype 2 or 3 updated data, it requests DBEX to acknowledge back and waits for that acknowledgement on the (dedicated) CTL socket immediately after sending the message.

dbHdlr’s prime responsibility is to offload (large amounts of) data handling from dbRecv or dbSend. It initializes DBS globals and waits at its queue for the message from dbRecv to convert data (compressed ST0 data to string-based format). Upon DBS restart or exit, it resets globals and releases resources. It is desired to be able to stop/restart the DBS and redownload the database while the EPICS application code, co-existing in the SLC IOC, continues to run. dbHdlr coordinates these efforts.

3.1.2 External Interfaces

The following sections describe the ways in which to access the SLC IOC DBS.

3.1.2.1 DBS Sockets

The SLC IOC communicates with DBEX, the Database Executive process that runs on the SLC Control System under VMS on the Alpha. It uses a byte stream composed of a proxy header (intended for proxy use) followed by data service header (or supertype header, interpreted by DBS) followed by data (for data-carrying messages). These messages are tcp/ip, and are managed by a proxy host. The proxy host establishes TCP/IP connections with DBEX in order to communicate with many micro/SLC IOCS using their socket connections. The two DBS socket connections on each SLC IOC are:

Dbex socket
Ctl socket

Both dbRecv and dbSend threads are responsible for the bi-directional communications on the dbex socket, while dbSend communicates with the uni-directional Ctl socket.

3.1.2.2 dbSend Queue

When any job thread or other utility decides to update the Alpha with a job’s updated data, it calls the reentrant utility, “dbupdate(job)”, which sends the “DB_UPDATE” message to dbSend message queue.

dbRecv also sends messages to dbSend Queue in order to have it send DBEX acknowledgements for supertype data, as well as diagnostic requests.

3.1.2.3 DBS Input/Output (I/0) Utilities

General database input/output utilities direct the reading and writing of data from / to the slc database and application code (other job threads). See Database I/O Utilities.

3.1.2.4 DBS Console commands

Console commands entered at the EPICS iocsh, RTEMS Cexp shell, or vxWorks shell, access the database (eventually via the DBS I/O utilities above) for the user.

See Console Commands API.

3.1.3 Data Structures

For the SLC IOC, it is desired to be able to stop and restart the SLC-only threads while the EPICS applications continue running. Furthermore, it is desired to redownload the SLC database upon command, after which device threads reinitialize and clear their dblists. In order to prevent heap memory fragmentation, most data structures which are allocated by DBS will be under control of a memory pool manager, namely the EPICS freeListLib.

3.1.3.1 Headers : Proxy and Supertype

The byte stream messaging between DBEX and the SLC IOC always contains a fwd_hdr_ts proxy header which the proxy interprets, and a dbsuptypehdr_ts supertype header, which the Alpha DBEX process and SLC IOC db* threads use to communicate back and forth. (The supertype header is equivalent to the Message Service message header.)

Name	Data type	Represents
fwd_hdr_ts
ip_port_u len user cmd crc	ip_port_tu int4u user_field_ts int1u int1u	Lower half of the ip address, and the port number Message bytecount - this fwdheader User defined; chunk count for large buffers; Fwd_server command, e.g. 8 bit crc over header; currently set to 0x55
dbsuptypehdr_ts	or	dbsuptype_tu = union
network netlsn id <<<<< Note len micr blkbeg blkend	int2u int2u int2u potential byte int4u Int4u int4u int4u	Type of data alignment mistmatch here >>>>> Size of superblock Microname Copy to location in superblk Copy from location in superblk
datw[ ]	Array of int2u	type-specific data; packed; max size is NETBUFLEN minus sizeof(dbsuptypehdr_ts)/2

The format of the data (datw[]) following the supertype header is dependant upon flags in the supertype header. The dbsuptype_tu definition makes a union of the supertype header (dbsuptypehdr_ts) and the data, such that any element of the supertype header or the data following can be accessed as a 16-bit data word. (This is important when unpacking the packed VMS formatted supertype data into native, structured format.)

3.1.3.2 Supertype 0 Byte Stream

The legacy VMS ST 0 byte stream, comprised of pointers to primary/unit/secondary data, contains a hash table followed by a series (X number) of unit/primary/secondary (UPS) blocks. Each UPS block is followed by its associated nsub number of secondary blocks (secn). In order to save memory on the legacy micros, numbers were associated with each primary, unit, and secondary name in lieu of storing the 4-char ASCII representations for each. In the SLC IOC, ASCII names will be stored instead of numbers. The ASCII representation facilitates programmers and is necessary for console users when typing in primary/unit/secondary names. The ASCII name representations, along with its associated secondary data information (from ST0 block and file on NFS) is translated into a “dictionary” for the SLC IOC (refer to Dictionary).

fwd_hdr_ts proxy header

Dbsuptypehdr_ts supertype header

hash length

array listhead ptrs

array node counts

UPS block 0

secn block [0 – nsub]:

supn | subn

fmt | slen

sptr =

offset in supn

…

….

UPS block X

secn block [0 – nsub]

Where a UPS block contains:

3 links – next UPS, unit,

collided nodes

len - ups + nsub’s

catn - prim #

unit - unit #

nsub - # of secns

3.1.3.3 Supertype 1-3 Byte Stream

Supertype 1-3 byte stream data consists of actual data values (pointed to by ST 0).

fwd_hdr_ts proxy header

Dbsuptypehdr_ts supertype header

Data

data

Data

data

….

3.1.3.4 Send/Receive Buffer Structure

The dbRecv receive buffer and the dbSend send buffer, both allocated in their respective threads, have similar structure to the overall byte streams. They maintain both headers and contain data, all of which are in VMS format. Upon receiving the data stream into the receive buffer, dbRecv converts the incoming dbsuptypehdr_ts into native format so that it can check the contents of the message. The native format is stored in the thread’s queued message variable, the structure of which is shown in the next section.

Name

Data type

Represents

Dbsndrcvbuf_ts

Receive and send buffers

proxy_hdr

supblk

Fwd_hdr_ts

dbsuptype_tu

Proxy header

Superblock header + data

3.1.3.5 DB Queued Message Structure

The dbmsgmail_ts contains all information required when communicating from db thread to db thread. The msgheader_ts contains the message ID (function). The nativeSB contains the native supertypehdr_ts, converted in dbRecv. The diagnostic data field is to service the diagnostic message received by dbRecv from DBEX; the reply is formatted and sent to DBEX from dbSend.

Name

Data type

Represents

Dbmsgmail_ts

Q messages

Msgheader

nativeSB

dbdiag

msgheader_ts

dbsuptypehdr_ts

dbdiagmsg_ts

Msg header

Native superblk hdr

Diagnostic data

3.1.3.6 Dictionary

The dictionary is a string based hash table whose entries contain exemplary information (meta data) about each primary/unit/secondary (PUS) data, namely:

Supertype number
Format (I,A,R,Z,S,T)
Length, or number of data elements (V= variable)
Data width (1,2,4,8 bytes)
Offset into supertype block (when added to that supertype block’s memory pointer, this becomes the address of the data locally)
Secondary number (secn), this is no longer used by SLC IOC, but is copied from the ST0 subblock and stored anyway

3.1.3.6.1 PRIMARY.MAP file

The PRIMARY.MAP ASCII file is created as part of the dbGen/dbInstall process on the Alpha. It is then released to a TBD production area on NFS in order for the SLC IOC to read it. On each line it contains primary name, primary number (catn), secondary name, secondary number (secn), format, width of data (bytes), and number of data items. Note that for each primary, secondary numbers are relative to their associated primary. A secondary of one primary may have a different number than the same secondary name of another primary.

The PRIMARY.MAP file is created so that the SLC IOC can a) convert primary and secondary numbers into their ASCII name representation, and b) have access to the data width information for each secondary. Primary names that do not exist in the SLC IOC database are still entered in the dictionary for name validation.

The native dictionary structure is created from a) the converted number-oriented, compressed, VMS formatted superblock 0, and b) by reading the PRIMARY.MAP file, available on an nfs mount point at location pointed to by environment variable.

PRIMARY.MAP format:

Primary catn Secondary secn format width number

QUAD 1 BMON 135 R 4 1

QUAD 1 KTIM 31 T 8 1 …

NOTE: file lines containing any given primary name must be contiguous in the file; the algorithm used by dbST0CreateDict() depends on this contiguousness.

3.1.3.6.2 Dbhash from gphash

For hash facilities, the dictionary uses EPICS libCom gphash (general purpose hash) as a base for its own dbhash facility. gphash sets up an internal hash table of ‘tablesize’ and returns a pointer to it. It also maintains a mutex which is not needed during a) single-threaded write-only initialization while other threads are locked out waiting for downloadEvent, and b) read-only access during operations. (This mutex requires less than 2 usecs when the mutex is available, so it doesn’t warrant modifying the gphash facility just for selectable mutex locking, unless we call it more than 100 times / second.)

However, gphash doesn’t use memory management facilities, thus it was decided to copy gphash to dbhash and modify it to use routines available in freeListLib. Dbhash will use memory pools for the hash array (ELLIST[tablesize]), and for each GPHENTRY. It will remove the mutex locking and it will substitute the strcmp call with strncmp such that hash strings will not need to be null-terminated.

3.1.3.6.3 Four use cases

When organizing the way data is accessed in the dictionary, there are four possible ways to arrange the data because there are four use cases that fall out of the various database input/output and shell routines. A primary (P) must always be specified, while the other two arguments, unit (U) and secondary (S), may be any combination of ALL*, or specified.

The four use cases are outlined in the bullets, and figure below.

PUS - prim, unit, sec all known; ie dblist(lp, p, u, s)
PU* - prim, unit, all secs; ie dbdump(p,u)
P** - prim, all units; ie dblunits(lp, prim) or dbdump(p)
P*S - prim, all units, sec; ie dblist (lp, p, ALL*, s)

To implement all four optimizations would be costly in memory and may be overkill. At a minimum, the P** case should be implemented, as it can handle all four use cases. Shell commands do not need to be optimized, as they occur at “human speed”, so PU* does not need to be implemented. The P*S case can be handled by P** (not quite as efficiently), and will be re-evaluated for implementation when we address how often the magnet and bpm jobs receive SCP requests with ALL* for units. For those threads that respond to a specific SCP request, they will be implementing the PUC use case during runtime. This means that the P** and PUC cases will be implemented, at a minimum. Every “primary” in PRIMARY.MAP will be entered into the hash table, regardless of its presence in this particular SLC IOC for primary name error checking. Each “primary secondary” doublet from PRIMARY.MAP could also be entered for ease of secondary name error checking; at the moment this is a TBD.

3.1.3.6.4 Planned Dictionary Structure

The following figure shows the minimum Dictionary implementation:

3.1.3.6.5 DBHENTRY structure – dictionary entry

Each dbhash table entry is of type DBHENTRY. The DBHENTRY is defined as:

Name

Data type

Represents

DBHENTRY

Hash table entry

node

name

userPvt

ELLNODE

Const char *

Void *

Link node

“primary unit secondary”

Pointer to either:

dbdictPUS_ts or dbdictP_ts

‘name’ in the above structure maintains our “hashing string” and points to a dictNameTriplet_ts (for PUS name) or points to a dbdictNameSgl_ts (for a single P, U, or S name), while userPvt points to either each dbdictUPS_ts secondary meta data entry or each dbdictP_ts primary entry.

3.1.3.6.6 dbdictNameTriplet_ts

For PUS, the “hashing string” consists of the unique “primary unit secondary” string. Memory is allocated for the dbdictNameTriplet_ts and a const char * pointer is established with the following:

Name	Data type	Represents
dictNameTriplet_ts		Triplet PUS name string
Name[12]	Char	Non- null-terminated “primary unit secondary” string name; length = 4-4-4 = 12 bytes

Note that gphash uses “strcmp” which requires use of null termination for the name string. This will be changed to “strncmp” in dbhash, such that we have char[12].

3.1.3.6.7 dbdictNameSgl_ts

For the hashing string used with primary, unit, or secondary name only, the following structure is used:

Name	Data type	Represents
dbdictNameSingle_ts		Single P-U- or S name string
Name[4]	Char	Non null-terminated “primary” or “unit” or “secondary” string name; length = 4 bytes

3.1.3.6.8 PUS entry – dbdictPUS_ts

Each DBHENTRY entry that points to a dbdictNameTriplet_ts also has its userPvt pointer pointing to a dbdictPUS_ts structure that contains the secondary meta data. The “primary unit secondary” meta data has the following structure definition:

Name

Data type

Represents

dbdictPUS_ts

SLC Secondary data type

supn

count

sptr

width

format

short

unsigned long

short

char

Supertype number

Word count

Word offset into supn block

1,2,4,8 bytes; data width

(I,A,R,Z,S,T)

3.1.3.6.9 dbdictP_ts entry

Entries for unique “primary” name will be entered into the hash table of the dictionary. When adding these, a pointer to dbdictNameSgl_ts, containing the primary - only name string is entered in the DBHENTRY name field, and the userPvt pointer of DBHENTRY will point to a dbdictP_ts structure. If the primary is not present in the SLC IOC, the userPvt pointer of DBHENTRY is null.

The dbdictP_ts structure represents a linked list head of its respective units present from the ST0 data for that primary.

Each primary entry has the following structure:

Name	Data type	Represents
dbdictP_ts		Primary linked list head for units
listUnits	ELLLIST	List head for linked units

Note: listUnits must be first element in structure.

3.1.3.6.10 dbdictU_ts entry

Each unit node in the units linked list is in monotonically increasing order by unit number, and is the linked list head for all secondaries contained in that primary/unit.

Each unit in the linked list has the following structure:

Name

Data type

Represents

dbdictU_ts

Unit linked list node for each primary

node

listSecns

name

ELLNODE

ELLLIST

Const dbdictNameSgl_ts *

Unit Linked list node

List head for linked secondaries

Unit (number) name

Note: node must be first element of structure.

3.1.3.6.11 dbdictS_ts entry

Each secondary node in the secondary linked list (which has a dictU_ts as its linked list head), has the following structure:

Name

Data type

Represents

dbdictS_ts

Secn linked list node for each unit

node

dictEntry

name

ELLNODE

DBHENTRY *

Const dictNameSingle_ts *

Secondary linked list node

Pointer to dict entry containing PUS

Secondary name

Note: node must be first element of structure.

Each secondary entry contains a pointer to the “primary unit secondary” DBHENTRY where the dbdictPUS_ts secondary meta information is pointed to.

The Dictionary figure is repeated below, with above structures labeled.

3.1.3.6.12 Dictionary memory management

The epics libCom memory manager utilities in freeListLib will be used to pool memory for storage of the dictionary structures. The freeList library of routines allocates a large pool of memory, and manages this large pool of memory as many smaller blocks of memory all of the same size. Thus, there must exist a memory pool manager for each different structure type. Dbhash will be rewritten to manage the memory pool for the hash table array and the GPHENTRY structures. File dbdict.c, whose utilities are called from dbHdlr, will mange the other six data structures defined above in support of the dictionary.

The following is an analysis of the different structures in support of the dictionary, as well as the estimated byte count of each structure. Note that byte count has been adjusted to the next largest 4 byte boundary to account for byte alignment on 4 byte boundaries.

dbhash allocates the following different structures:

dbhInitPvt –

dbhPvt - the table structure (tableSize, nShift, ELLIST ** paplist, mutex)

12 bytes
no memory manager - there is only one dbhPvt, and it is small in size

paplist array – array of [ELLIST *] – hash table

tableSize x 4 bytes ; array of ELLIST *
paplistPool

dbhAdd –

paplist list heads pointed to by each paplist[index]– ELLIST linked list head – if no previous entry exists

12 bytes x (number of PUS entries + number of P entries) (assuming no collisions)

GPHENTRY(* pvtid, *name, *userPvt, ELLNODE (=* next, * prev, cnt) )

24 bytes x Number of hash entries (points to all dictPUS, dictP, dictU, dictS entries)

dbHdlr, who creates and manages the dictionary allocates the following:

dictNameTriplet_ts : char * name “primary unit secondary” for every PUS –

12 bytes x every PUS

dictPUS_ts for every PUS

12 bytes x every PUS

dictNameSingle_ts: char * name “primary “ for every P** -

4 bytes x every primary

dictP_ts for every primary ( ELLLIST )

12 bytes x every primary

dictU_ps for every U of every P** - unit linked list (ELLNODE, ELLLIST, dictNameSingle_ts *)

number of units for this primary (P/U) x 24 bytes

dictNameSingle_ts: char * name “unit” for every unit of this primary

4 bytes x every unit/primary

dictS_ts for every S of every unit in primary –sec linked list – (ELLNODE, GPHENTRY*, dictNameSingle_ts*)

number of secondaries for this unit x 16 bytes

dictNameSingle_ts: char * name “secondary” for every secondary of this unit

4 bytes x every PUS

3.1.3.7 DBS Globals

DBS Global data is shared between all DBS threads and initialized by dbHdlr thread. Even though the data is available in global space to all threads, only DBS threads or utilities use and manage this data.

Name	Type	Represents
dbnodes_p[4]	dbsuptype_tu	Pointers to superblocks 0-3 dbRecv mallocs, writes pointers & data while other threads are blocked at db download event. After download event pointers read-only (ptr to ST0 & data deleted; rest never change)
dict_p	void *	Pointer to dictionary; new char-based gphash table created for housing supertype 0 data (secondary data & location) and valid ASCII names. gphash has its own mutex protection for add/delete/find dictionary utilities.
dbex_up dbVersion	epicsBoolean char *	Status of dbex up (T) or down (F) Db major / minor version w dbex_up msg
dbRWMutex	epicsMutexId	Activiated for reads or writes from/to supertype 1-4 data. dbRecv (for Alpha VMS updates), and dbl* I/O utilities access
dbhilo_updates [N_JOBS]	Dbhilo_job_ts	Each “job service” array contains pairs of offsets to upper and lower bounds of modified ST2/3 data. Acc’ed by utility rte dbhilo_update called from dblput.
dbhiloMutex [N_JOBS]	epicsMutexId	Activiated for reads or writes from/to dbhilo_updates[ ]. Acced by dbSend, and private utility dbhilo_update called by dblput
dbAckEvent [N_JOBS]	epicsEventId	Job service threads wait at this event until their updated data has been acknowledged by Alpha
dbAckStatus [N_JOBS]	vmsstat_t	Job service threads receive this status after waiting at dbAckEvent until their updated data has been acknowledged by Alpha

3.1.3.8 SLC IOC Globals

3.1.3.8.1 dbExists, downloadEvent

There are two other globals, dbExists and downloadEvent, which are created prior to DBS thread creation, and thereafter are partially maintained by DBS (and partially by slcExec). These globals, dbExists, and downloadEvent, are initialized to dbExists = F and downloadEvent semaphore to blocking such that any threads who want to access data after they are created and before the database is downloaded, are blocked. Similarly, any console users who want to access data before the database is downloaded have an error immediately returned from their console command after the underlying code checks the dbExists flag. The dbHdlr thread is responsible for setting the dbExists flag and resetting the downloadEvent (after download) during initialization.

<Note, we decided not to implement a dbstop or dbinit:

Upon a “dbstop”, dbHdlr resets the dbExists flag. Upon a “dbinit” and after a re-download, dbHdlr sets the dbExists flag and unblocks the downloadEvent.>

3.1.3.8.2 dbStopEvent –Not implementing

The dbStopEvent is set by slcExec when a user types in ‘dbstop’ at the console. A
”DB_STOP” message is also sent to dbHdlr. This causes the DBS to clear the database, clear the dbhilo update lists, and to reset the dbExists flag. dbRecv throws away updates coming from the Alpha. If an IPL occurs during this state, an error is logged. Updates going to the Alpha are also thrown away by dbSend in this state. Diagnostic requests, as well as DXUP, DXDN messages from the Alpha are processed normally. Dbl* utilities should not be called, as job threads who access the db are blocked at the dbStopEvent.

Upon the user typing in ‘dbinit’ at the console, slcExec sets the downloadEvent to block, resets the dbStopEvent, and sends a “DB_DOWNLOAD” message to dbSend in order to initiate the IPL process. After download, dbHdlr sets the dbExists flag and sets the downloadEvent to unblock.

3.1.3.8.3 slcThreads

slcThreads[thread enum] is a global array of thread information. Each DBS thread must maintain the fields in its array.

Name	Type	Represents
tid	EpicsThreadId	Thread identifier
active	EpicsBoolean	True if thread is active / looping; set by thread
stop	EpicsBoolean	True if thread should stop; set by slcExec
mqId	epicsMessageQueueId	Message queue identifier

During thread initialization, all of the DBS threads set :

· active = T

· mqId = message queue id from create call

slcExec sets tid = thread id from create call; it also sets stop = F

During operations, all threads must periodically check their stop flag, and if it is set T, they must exit gracefully. Exiting consists of

releasing any resources that the thread manages

set slcThreads[thread enum].active = F
set slcThreads[thread enum].tid = NO_THREAD

3.1.3.8.4 slcSockets

dbRecv maintains slcSockets[slcDbex] and dbSend maintains slcSockets[slcCtl].

3.1.4 Data Flow

3.1.4.1 Introduction

3.1.4.2 DB Service Initialization

slcExec starts Phase 1 message Q’ed threads dbSend, dbHdlr
slcExec starts Phase 2 dbRecv after all Phase 1 active flags are set T
All db threads :

Create cmlog handle
Init resources

Message queue
Sockets
Memory heap
Timers, semaphores
Set active = T

Wait (at message Q or socket)

dbHdlr thread initializes shared db thread globals
slcExec sends “DB_DOWNLOAD” to dbSend
dbSend requests a download (IPL) from DBEX

3.1.4.3 DB Service Download (IPL)

The following describes, at a high level, the threads, data, and routines involved with downloading the database.

3.1.4.3.1 DB Service Download Block Diagram

3.1.4.3.2 dbSend Thread DB_DOWNLOAD Q request

Receives “DB_DOWNLOAD” message from slcExec

(dbdownloadme:)
Formats msg.nativeSB with supertype header info indicating download request
Converts msg.nativeSB to VMS format, copying into sndbuf.supblk (htoVMSSupHdr(sndbuf.supblk, msg.nativeSB) )
Pre-fixes appropriate proxy_hdr in sndbuf
Sends sndbuf to dbex socket

3.1.4.3.3 dbRecv Thread receives supertype 0-3 data

rcvbuf receives Supertype 0-3 messages from DBEX at dbex socket
Converts VMS supertype header in rcvbuf to msg.nativeSB using utility VMStohSupHdr(msg.nativeSB, rcvbuf.supblk)
Sends “DB_DBEX_ACK” queued message in message header id to dbSend Q to ack DBEX (if requested by DBEX ; flag in id field if supertype header)

Msg.nativeSB contains native superblock

Validates incoming data and proper sequence of download (dbmicromail)
For first piece of a super block (if any super blocks 0-3 are larger than 8K, they are downloaded in pieces) - allocates super block memory (dbsuperalloc) from heap according to len (for all pieces) in supertype header; populates dbnode_p[superblock number] with pointer
Copies rcvbuf (dbmsgcopy) to alloc’ed memory; drops proxyhdr

Lock / unlock dbRWMutex around memcpy to alloc’ed memory

dbRecv sends a DB_CONVERT queued message to dbHdlr after it receives the last piece of supertype 3 data (all data has been downloaded).

3.1.4.3.4 dbSend Thread DB_DBEX_ACK Q request

Receives “DB_DBEX_ACK” messages from dbRecv for each ST 0-3 block received
Sends “ack” to DBEX socket

uses msg.nativeSB.id;
htoVMSSupHdr(sndbuf.supblk, msg.nativeSB)
Pre-fixes proxy_hdr
Sends sndbuf to dbex socket

DBEX does not currently request an ack when it updates the micro with ST3 data; so this dbex acking only occurs during IPL.

3.1.4.4 DB Hdlr Thread Transfers ST0 to Dictionary

dbRecv sends a DB_CONVERT queued message to dbHdlr after it receives the last piece of supertype 3 data. It is up to the dbHdlr thread to implement all steps necessary to transfer the ST0 data into the dictionary format.

3.1.4.4.1 DB Hdlr Thread DB_CONVERT request

Receives “DB_CONVERT” Q message from dbRecv
Replace ST0 num-oriented hash table with string based hash table dictionary
Step 1 VMS Conversion:

Walk thru all ST0 byte stream structures to swap words and longs in place (if architecture is not little endian) (VMStohST0)

Step 2 Transfer structures:

Initialize dictionary memory
Read PRIMARY.MAP file and make entries into dictionary, each combining

Existing secondary data from ST0
File data – data width, ASCII names

into dictPUS_ts structure

Hash on “primary unit secondary” (PUS case)

add dictPUS_ts entry to dictionary
optimized for SCP commands to job threads that make dblists, etc on the fly

Hash on “primary”

Links “ALL*” units

Every unit in primary is a node in linked list of monotonically increasing unit numbers
Each unit in primary is a linked list head for linked list of secondaries

Each secondary node points to its PUS entry

For primary name error checking

Enter “primary” even if it doesn’t exist in ST0

Hash on “primary secondary” - TBD

For faster secondary name error checking
For P*S use case in dblist(lp, P, ALL*, S)

Step 3, Finalize:

Delete ST0
Set dbExists flag
Signal downloadEvent

3.1.4.4.2 Correlation of PRIMARY.MAP and ST0 data

The following is PDL for creating dictionary entries based on file data and ST0 data.

For each unique primary name in PRIMARY.MAP file

Make “primary” entry in dictionary

call dbunitsST0 (ld_p, catn)

if (status OK and ld_p.len>0)

for each unit# in list

dbgetUpsST0(uptr, catn, unit)

link unit in “primary” (for ALL*)

for every secondary in file

dbgetSecST0(sptr,uptr,secn)

store “primary unit sec” with

combined ST0 & file data;

+ link sec in unit entry (for ALL*)

3.1.4.5 DB Service Receives Updates from Alpha

During normal operations, tasks on the Alpha may issue a dbput which cause an update of supertype 2 data to be sent to micros/SLC IOCs. The byte stream is similar to the supertype 2 data received during IPL. The id of the supertype header is masked for:

3.1.4.5.1 DB Service Updates Block Diagram

3.1.4.5.2 dbRecv Thread Receives Update from Alpha

· Receives Supertype 2 (setpoint) update messages from DBEX

· Note: Currently DBEX does not set the request ack flag in the supertype id header which would cause dbRecv to send “DB_DBEX_ACK” msg to dbSend.

· Lock dbRWMutex

· Copies super block recv’ed (dbmsgcopy) to SLC IOC ST2 block

· Unlock dbRWMutex

3.1.4.5.3 dbSend Thread – no action

3.1.4.6 DB Service Sends Updates (originating in EPICS) to Alpha

Updated data flowing from the SLC IOC to the Alpha is dicussed in this section.

3.1.4.6.1 Updates originate from dblput calls

Supertype 2 and 3 data is updated in the EPICS database by various ioc tasks. The SLC IOC has various asynch and handler jobs that read the data from the EPICS database and then update the slc database using dblput calls. Dblput reentrant utility stores the data to the slc database (locking and unlocking dbRWMutex around the memcpys) and then calls dbhilo_update utility in order to store the upper and lower bounds of the data offset (hi/lo offset pair). These hi/lo offset pairs are the actual offsets into the supertype 2 or supertype 3 data memory and are stored in a dbhilo_update array unique for each job and for each supertype (2, or 3). The dbhilo_mutex[job] is locked and unlocked around the memcpy when updating the dbhilo_update array.

3.1.4.6.2 Updates to Alpha Data Flow

When a job thread has determined that it wants to send its updated data to the Alpha, it calls the reentrant utility dbupdate(). Dbupdate() determines the calling job’s job id from slcThreads[thread enum] global and then sends the message “DB_UPDATE”, along with job id to dbSend queue. Dbupdate() then waits on the ackEvent[job],

3.1.4.6.3 dbSend Thread Sends Updates to Alpha

Receives DB_UPDATE messages from various “job threads” to send update

Jobs call dbupdate utility which ultimately sends the DB_UPDATE Q message and job id

Lock dbhilo_mutex[job]

Prevents writing to dbhilo_update[job] table while reading & writing

Compress job’s list of pending update pairs, if possible (microdbsendc)

For each update pair in dbhilo_update[job], do the following:

Lock dbRWMutex
Read dbhilo_update[job] sptr pair
memcpy data from ST block indicated by sptr pair into sndbuf (microdbsendb)
Unlock dbRWMutex
Increment msg seq # (1-256), write in proxyhdr.user_byte

store msg seq #, “job id” locally

Send sndbuf (already in VMS format) to dbex socket
After sending data to dbex socket, loop waiting at CTL socket for Ack from DBEX (CTL socket is non-blocking read)*

Further updates for this job are blocked; other job updates q’ed
Timeout after ~15 sec (less if !dbex_up)

Wait 1 sec at ctl socket
Check for slcExec “stop flag”

CTL loop exit conditions:

1) Successful Ack

Clear pair from dbhilo_update[job] that was just sent
If (! dbex_up), set dbex_up flag
dbAckStatus[job]= success

2) Invalid msg seq #

If (! dbex_up), set dbex_up flag
no clear
dbAckStatus[job]= invalid; error

3) Timeout

no clear
dbAckStatus[job]= timeout ; error

4) Socket error

Set “lost connection flag” ; no clear
Loop waiting for connection

Check for slcExec “stop flag”

dbAckStatus[job]= socket error

When all updates for job have been sent, or there was an exit condition:

Unlock dbhilo_mutex[job]
Signal ackEvent[job]
Upon dbex_up / connection up

Send updates for all jobs

*Note that if it is too hard to configure the CTL socket with non-blocking reads, a new thread will be created, dbCtrl, which will synchronize with dbSend. dbSend will issue sleeps() of 1 second each and check the ctlEvent[job] flag each time, for up to timeout value of 15 seconds. It will also compare the msg seq # received by dbCtrl to determine whether it was a valid ack or not.

3.2 dbRecvThread – Detailed Design

3.2.1 Functional Flow

Also see: Also refer to DB Service Download (IPL)

And DB Service Receives Updates from Alpha

3.2.1.1 Initialization

The dbRecv thread performs the following at initialization

Allocate rcvbuf receive buffer for DBEX messages
Create the sdDbex socket and connect with the proxy
Set its slcThreads[dbRecv].active = epicsTrue;

3.2.1.2 Normal Processing Loop

While slcThreads[msgRecv].stop != epicsTrue

Receive the message into rcvbuf buffer (dbRead()) using a getBuffer utility.
interpret the fwdheader
converts supertype header into native format by placing it in queued message nativeSB field – uses DBS utility VMStohSupHdr()
if supertype header requests ack to DBEX (true for all ST 0-2 downloads), send queued message to dbSend with message id = DB_DBEX_ACK
switch on supertype header id

diagnostic request: SUPERHDR_FUNC_DGINQ

copy incoming data to queued message and send to dbSend with message id = DB_DIAG (dbmdiag_sendrpy())

dxup message: SUPERTYPE_ID_OVERRIDE | SUPERHDR_FUNC_NOTFY followed by “DXUP”

copy dbVersion
if dxup was previously epicsFalse

call dbupdate(ALL*)

· dbupdate sends DB_UPDATE(ALL*) to dbSend; doesn’t block on any ackEvent for ALL* case

set dxup flag = epicsTrue

IPL block or ST 2 Alpha update: SUPERTYPE_ID_OVERRIDE

dbmicromail() validates message using incoming headers and ensures specific download sequence is followed
dbsuperallo() creates memory for each superblock
dbmsgcpy() copies from rcvbug to superblock memory

Also see: Also refer to DB Service Download (IPL)

And DB Service Receives Updates from Alpha

3.2.1.3 Termination

When the slcThreads[msgRecv].stop flag == epicsTrue

Set slcThreads[msgRecv].active = epicsFalse
Destroy the sdDbex socket
Set slcThreads[dbRecv].tid = NO_THREAD
Return

3.2.2 Data

The dbRecv thread uses the following globals:

slcThreads[dbRecv].tid = is set to NO_THREAD just before dbRecv exits.
slcThreads[dbRecv].active – is set to epicsTrue when the socket is created and connected, set to epicsFalse just before dbRecv releases resources and exits.
slcThreads[dbRecv].stop – is polled within the dbRecv main loop
slcThreads[dbRecv].mqId - NULL always, as the dbRecv thread has no msg queue.
slcThreads[dbRecv].msgSource – the source of the current incoming message.
slcSockets[sdDbex] = socket descriptor returned when the socket connection is made, it is set to INVALID_SOCKET whenever a failure is detected with the socket, or when the socket is destroyed.

3.2.3 Resource management

3.2.3.1 Connection management

The dbRecv thread manages the slcSockets[sdDbex] Database Service socket. Whenever a failure is detected the dbRecv thread is responsible for re-connecting with the Proxy (dbSockConns()).

3.2.3.2 Memory Pool

The epics libCom memory manager freeList will be used to pool memory for local storage of the supertype data received. This library of routines allocates a large pool of memory, and manages this large pool of memory as many smaller blocks of memory (all of the same size). The granularity of “smaller blocks” will most likely be the maximum DBS message size (from DBEX) of 8k bytes.

3.2.4 Message Logging

The slcCmlog utilities are used for logging the following status and error messages:

TBD

3.2.5 Diagnostics

TBD

3.2.6 Major routines

epicsShareFunc void epicsShareAPI dbRecvThread(void)

local vmsstat_t dbInitGlob(void);

local void cleanup(void);

local vmsstat_t dbSockConns(void);

local void dbSockClose (void);

local vmsstat_t dbRead (

dbsndrcvbuf_ts *inmsg_ps,

const unsigned short *max_bytecount_p,

int *actual_bytecount_p,

int *sbbytecnt);

local void dbmicromail(const dbsndrcvbuf_ts *mail2_p,

const unsigned long actrcvbc,

const int sbbytecnt);

local void dbmsgcopy(const dbsndrcvbuf_ts *mail2_p,

unsigned long supblkx,

unsigned long nwords);

local vmsstat_t dbsuperallo (const dbsndrcvbuf_ts *mail2_p,

unsigned long supblkx);

local void dbmdiag_sendrpy(dbsndrcvdiagmsg_ts *diagmsgbuf_p,

unsigned short dgmsgbc);

local vmsstat_t formST2Ack (const unsigned short id);

3.3 dbSendThread – Detailed Design

3.3.1 Functional Flow

Also refer to DB Service Download (IPL)

And DB Service Sends Updates (originating in EPICS) to Alpha

3.3.1.1 Initialization

The dbSend thread performs the following as initialization

Create the message queue
Set its slcThreads[dbSend].mqId = message queue id
Set its slcThreads[dbSend].active = epicsTrue;

3.3.1.2 Normal Processing Loop

While (slcThreads[dbSend].stop) != epicsTrue

Read from message queue
Switch on message id

DB_DOWNLOAD_REQ

dbdownloadme()

DB_ACK_DBEX

dbAck()

DB_DIAG_DBEX

dbDiagReply()

DB_UPDATE

micro_dbsend(job)
Refer to dbSend Thread Sends Updates

Inside each specific routine :

Check/wait for valid socket
Convert the nativeSB header to VMS;
Convert any message data content from native to VMS
Fill in the forward header
Send message to Proxy

3.3.1.3 Termination

When the slcThreads[dbSend].stop flag == epicsTrue

Set slcThreads[dbSend].active = epicsFalse
Destroy the message queue
Set slcThreads[dbSend].mqId = NULL
Set slcThreads[dbSend].tid = NO_THREAD
Return

3.3.2 Data

The dbSend thread uses the following globals:

slcThreads[dbSend].tid = is set to NO_THREAD just before dbSend exits.
slcThreads[dbSend].active – is set to epicsTrue when the socket is valid and connected, set to epicsFalse just before dbSend releases resources and exits.
slcThreads[dbSend].stop – is polled within the dbSend main loop
slcThreads[dbSend].mqId – is set to the dbSend message queue id. Set to NULL when exiting.
slcThreads[dbSend].msgSource – the source of the current incoming message.
slcSockets[sdMsg] = socket descriptor returned when the Message Service socket connection is made, checked for failure indicated by INVALID_SOCKET.

3.3.3 Resource management

3.3.4 Message Logging

3.3.5 Diagnostics

3.3.6 Major routines

epicsShareFunc void epicsShareAPI dbSendThread(void)

static void cleanup(void);

static void chkSockConnection(void);

static vmsstat_t dbdownloadme(void);

static vmsstat_t dbWriteDbs(int sd,

const unsigned short id,

dbsndrcvbuf_ts *outmsg_ps,

const unsigned short *max_bytecount_p,

unsigned short *actual_bytecount_p);

static vmsstat_t dbAck (fwd_hdr_ts *pFwd, dbsuptypehdr_ts *pSup);

static vmsstat_t dbDiagReply (fwd_hdr_ts *pFwd, dbsndrcvdiagdata_tu *pDiag);

3.4 dbHdlrThread – Detailed Design

3.4.1 Functional Flow

3.4.1.1 Initialization

The dbHdlr thread performs the following at initialization

Create the message queue
Set its slcThreads[dbHdlr].mqId = message queue id
Set its slcThreads[dbHdlr].active = epicsTrue;
Create DBS globals: - allocate memory (from pools), initialize; see Data section

3.4.1.2 Normal Processing Loop

While (slcThreads[dbHdlr].stop) != epicsTrue

Read from dbHdlr message queue
If message = DB_CONVERT

dbST0Convert (void)
dbExists = epicsTrue
unblock downloadEvent

3.4.1.3 Termination

When the slcThreads[dbHdlr].stop flag == epicsTrue, or any other detected “terminal” failure:

If dbdownloadEvent is set, clear (to unblock waiting tasks)

Set slcThreads[dbHdlr].active = epicsFalse
Destroy the message queue
Set slcThreads[dbHdlr].mqId = NULL
Set slcThreads[dbHdlr].tid = NO_THREAD
De-allocate and release DBS global memory and resources
Return

3.4.2 Data

The following DBS global variables are initialized by dbHdlr:

dbnodes_p[4]	dbsuptype_tu	Pointers to superblocks 0-3 dbHdlr mallocs, writes pointers & data while other threads are blocked at db download event. After download event pointers read-only (ptr to ST0 & data deleted; rest never change)
dict_p	void *	Pointer to dictionary
dbex_up dbVersion	epicsBoolean char *	Status of dbex up (T) or down (F) Db major / minor version w DXUP messagesents on IPL and when DBEX comes up
dbRWMutex	epicsMutexId	Activiated for reads or writes from/to supertype 1-4 data. dbRecv (for Alpha VMS updates), and dbl* I/O utilities access
Dbhilo_updates [N_JOBS]	Dbhilo_job_ts	Each “job service” array contains pairs of offsets to upper and lower bounds of modified ST2/3 data. Acc’ed by utility rte dbhilo_update called from dblput.
dbhiloMutex [N_JOBS]	epicsMutexId	Activiated for reads or writes from/to dbhilo_updates[ ]. Acced by dbSend, and private utility dbhilo_update called by dblput
dbAckEvent [N_JOBS]	epicsEventId	Job service threads wait at this event until their updated data has been acknowledged by Alpha
dbAckStatus [N_JOBS]	vmsstat_t	Job service threads receive this status after waiting at dbAckEvent until their updated data has been acknowledged by Alpha

3.4.3 Resource management

Refer to Dictionary memory management

3.4.4 Message Logging

The slcCmlog utilities are used for logging the following status and error messages:

TBD

3.4.5 Diagnostics

TBD

3.4.6 Major routines

epicsShareFunc void epicsShareAPI dbHdlrThread(void)

File dbIPL.c contains routines to transfer (or convert, but here convert does not include endian or byte alignment) the VMS ST0 structure to the dictionary structure. See DBS Utilities section below.

3.5 DBS Utilities

This section describes utilities that only the DB Service threads call.

3.5.1 dbUtil

dbUtil.c contains conversion and debugging utilities that print to the console.

int VMStohSupHdr(dbsuptypehdr_ts * suphdr, dbsuptype_tu * const supblk);
int htoVMSSupHdr (dbsuptype_tu * supblk, dbsuptypehdr_ts * const suphdr);
int4u VMStohST0 (void);
void printhSupHdr (char *bufname, dbsuptypehdr_ts *pBuf, int bytecnt);
void printVMSSupHdr (char *bufname, dbsuptype_tu *pBuf, int bytecnt);
void printdblist (dbheader_ts * *const dbdata_pp);

3.5.2 dbST0

File dbST0.c contains routines to transfer (or convert, but here convert does not include endian or byte alignment) the VMS ST0 structure to the dictionary structure. Since the routines operate on VMS formatted data from DBEX, some “legacy-style” data types are used (int2u, int4u, etc) for compatibility.

The utility :

vmsstat_t dbST0Convert (void);

called from dbHdlr

uses the following local static procedures to process the downloaded ST0 meta data, along with reading PRIMARY.MAP file:

LOCAL vmsstat_t dbST0alloc(dbST0header_ts **const dbdata_pp,

const int2u bytes_in_element,

const int2u num_elements);

LOCAL vmsstat_t dbST0list (dbST0header_ts **const dblist_pp,

const int2u prim,

const int2u unit,

const int2u secn);

LOCAL vmsstat_t dbST0units (dbST0header_ts **const dbdata_pp, const int2u prim);
LOCAL int4u dbST0GetUpsBlk (unsigned short catn, unsigned short unit, suptype0_ts *sb0);
LOCAL vmsstat_t dbST0GetSubBlk (int4u ups_offset, char secNum, suptype0_ts *sb0);
LOCAL vmsstat_t dbST0ChkClist (dbST0header_ts * * const dbdata_pp, int4u orgCnt, int4u maxCnt );

Above uses suptype0_ts as the definition of the ST0 structue:

typedef struct

{

int2u headerw[11]; /* packed supertype header */

int2u hshl; /* size of hash array */

int4u hTbl[DBHASH]; /* Hash table - array of offsets */

int2u hcTbl[DBHASH]; /* Counts of nodes in above array */

} suptype0_ts;

Above uses dbST0header_ts as the ST0-type of list populated by dbST0list() and dbST0units().

typedef struct

{

int4u max; /* max length of list (words; includes this hdr) */

int4u cur; /* current length of list */

} dbST0header_ts;

All ST0-type lists used in discovering which primaries and units are configured in ST0 are currently malloc’ed and free’ed. This design should be modified to use the new-style dictionary free lists.

3.5.3 dbdict

dbdict.c contains utilities (mainly used by dbST0) to insert entries from ST0 into the dictionary. Please reference the Dictionary section for dictionary structure and data structures used. The dbdict.c routines use dbhash.c utilities for hash table management.

void dbdictInitFreeLists ( void);
void dbdictDelFreeLists ( void);
dbhEntry_ts * dbdictInsP(const char *prim_p, const epicsBoolean hasunits );
vmsstat_t dbdictInsAllUnits (const dbdictP_ts * unitList_ps,const dbST0header_ts * dbunits_ps);
vmsstat_t dbdictInsSecnInAllUnits ( const dbdictP_ts * unitList_ps,

const dbST0header_ts * dbunits_ps,

const char * prim_p,

const char * secn_p,

const dbST0header_ts * dblist_ps,

const short supn, const short count,

const short width, const char fmt);

Utility dbdictInsSecnInAllUnits() uses local static dbdictInsPUS() procedure to insert a “prim secn unit” entry in hash table as it inserts a linked list of units belonging to a given prim:

LOCAL dbhEntry_ts * dbdictInsPUS (const char * prim, const char *unit, const char *secn,

const short supn, const short count,

const long sptr, const short width, const char fmt);

3.5.4 dbhash

gphash* has been copied to dbhash and the following modifications have been made to dbhash.c (and dbhash.h):

DBHENTRY entries are allocated and released from a freeList manager.
Paplist hash array is allocated and released from a freeList manager.
epicsMutexId lock has been removed from dbhPvt structure
pvtid has been removed from DBHENTRY
to save memory, strcmp has been changed to strncmp everywhere and namelen arg has been added to dbhFind(), dbhAdd(), dbhDelete(). The const char * name element of DBHENTRY points to non-null terminated free list blocks of the correct size.

3.5.5 dblist

Dblist.c contains utilities for managing the two free lists used by the DB I/O utilities – dblistSmFreeList and dblistBigFreeList. The dblist routines use epics utility freeListLib.c; refer to freeList.h for the prototypes.

void dblistInitFreeLists (void);
void dblistDelFreeLists (void);
void dblistNew (const unsigned long bytes_to_allocate, dbheader_ts * dbdata_p );
void dblistEnlarge (const unsigned long bytes_to_allocate, dbheader_ts * dbdata_p);

When a small list runs out of room, it gets a big list, copies the small list contents to the big list, and frees the small list. Similarly, when a big list runs out of room it gets more space directly from the heap, copies its contents to the new memory, and frees the big list. If the dbheader_ts (refer to Database Input/Output Utilities) list size is other than DBL_SZBIG or DBL_SZSM, it is assumed the list came from the heap for later freeing purposes (logic in dbfree()).

3.5.6 dbShell for Callbacks (and dbshellUtil for utilities)

dbShell contains the implementation (callbacks) for the Shell Commands.

The callbacks for the shell commands have the name “<shell command>Callback” . The callbacks will, in general, first check the dbExists flag. If the database exists, they call underlying routines which return vmsstat_t and pass the desired return data in the first argument (as a 2-byte integer). If the underlying routine had an error, the error is output, otherwise the callback routine converts desired value(s) to ASCII prior to output to the console.

Any support utility is contained in dbshellUtil.c. If it is desired to use these utilities from other threads, they can be moved.

3.5.6.1 dbgetformatCallback(const char * const prim, const char * const secn)

Returns the native data type (R,I,Z,A,S,T) format as a single character

if (! dbExists) print error; return
if (verifyPUS(prim,*,secn) )

o dbhEntry_p = dbhFind(“P”)

o get first unit from unit linked list

o fmt = dbgetformat(P,U,S)

o if error, print

o output fmt

3.5.6.1.1 dbshellUtil:

epicsBoolean dbverifyPUS(const char * const prim, const char * const unit, const char * const secn, )

· dbhEntry_p = dbhFind(“PUS”)

o if dbhEntry_p != null, return true

o else dbhEntry_p = dbhFind(“P”)

· if dbhEntry_p->userPvt = null, then it’s a primary name error

o log error (console or cmlog)

o return false

· Else

o get first unit from unit linked list

o find match for secn – return true

o if no match, log error (console or cmlog)

§ return false

3.5.6.1.2 dbshellUtil:

char * dbgetformat (const char * const prim, const char * unit, const char * const secn )

Returns the single character format of PUS; returns ‘/0’ (null) if not found in dictionary.

o dbhFind(“PUS”)

o return dbhEntry_p ->userPvt->format

3.5.6.2 dbgetwidthCallback(const char * const prim, const char * const secn)

Returns the data type data width: 1,2,4 or 8 bytes.

if (! dbExists) print error; return
if (dbverifyPUS(prim,*, secn) )

o dbhEntry_p = dbhFind(“P”)

o get first unit from unit linked list

o width = dbgetwidth (P,U,S)

o scanf (width) and output ASCII

3.5.6.2.1 dbshellUtil:

int dbgetwidth(const char * const prim, const char * const unit, const char * const secn)

Returns the integer byte width of PUS; returns zero if not found in dictionary.

o dbhFind(“PUS”)

o get dbhEntry_p ->userPvt->width

o byte swap, return value

3.5.6.3 dbgetcountCallback(const char * const prim, const char * const unit, const char * const secn)

Returns the number of elements for the secondary.

if (! dbExists) print error; return
if (dbverifyPUS(prim,*, secn) )
if (dbverifyPUS(prim,*, secn) )

o dbhEntry_p = dbhFind(“P”)

o get first unit from unit linked list

o count = dbgetcount (P,U,S)

o scanf (count) and output ASCII

3.5.6.3.1 dbshellUtil:

int dbgetcount(const char * const prim, const char * const unit, const char * const secn)

Returns the integer number of elements for the secondary; returns zero if not found in dictionary.

o dbhEntry_p = dbhFind(“PUS”)

o if null, then its an error

o get dbhEntry_p ->userPvt->count

o byte swap, return value

3.5.6.4 dbgetmetaCallback(const char * const prim, const char * const unit, const char * const secn)

Returns the meta data for the secondary – count (number of elements), followed by format (R,I,Z,A,S,T), followed by width ie “23A4”

The callback routine calls

dbgetcount(PUS); convert to ASCII

dbgetformat(PUS), and

dbgetwidth(PUS); convert to ASCII

Then concatenates the three strings for output to console of the form <count><format><width>

3.5.6.5 dbdumpCallback(const char * const prim, const char * const unit,

const char * const secn)

Dump the primary/unit/secondary value.

One of the following fields can be ALL* (not both): unit, secn.

TBD: have both unit and secn be ALL*

if (dbverify(PUS))

o dballoc(lp)

o dblist (lp, P, U, S)

o dblget (lp, ld)

o convert to ASCII and output

3.5.6.6 dbdumpunitsCallback(const char * const prim)

Dump all the units, if any, for given primary.

if (dbverify(PUS))

o dballoc(ld)

o dblunits(ld, P)

o for each unit in ld list, output

3.5.6.7 dbeditCallback(const char * const prim, const char * const unit,

const char * const secn,

char * const value1, char * const value2, ...)

Converts ASCII values to proper format and puts them into the

SLC database. User will need to know native data type in order to input proper values.

Dballoc(dblist)

if (SUCCESS( iss = dblist (dblist, P,U,S))

dblget(dblist, dbdata)
with dblist :
if (ptr->supn = 1) and (! editST1) output error message, return
if number of values != sptr->count output error, return
convert each ASCII value to proper format, based on ptr->fmt, ptr->width
convert data to packed VMS
form address (dbnode[ptr->supn] + ptr->sptr) lo and hi (lo + count * width)
lock dbRWMutex
write data
unlock dbRWMutex
log the edit to cmlog
log success to console and return

3.5.6.8 dbupdateCallback(const char * const jobName)

Force update of any outstanding supertype 2 and 3 data changes

for jobName to DBEX. If jobName is "ALL*", forces update for all jobs.

If (SUCCESS (iss = dbupdate(jobName))

Output “successfully queued”
Else, output proper error message

3.5.6.9 dbdumphashCallback(void)

Dump the hash table.

Call dbhDump()

3.5.6.10 dbdumpdatabaseCallback(no args)

Dump the entire SLC database.

TBD: Database meta data as well as data values are output.

Note that output can be piped to a filename.

Expand on dbhDump to display data along with hash meta data.

3.5.6.11 dbexistsCallback(void)

if dbExists() output True (1)
else output False (0)

3.5.6.12 dbgetversionCallback(no args)

Returns the current database version in an 8 character string.

Output dbgetVersion();

3.5.6.12.1 dbshellUtil:

char * dbgetVersion(void)

Return (dbVersion) global variable.

3.5.7 dbio

dbio.c contains implementation for the Database Input/Output Utilities . Please refer to the document for full details.