[TEG.LCLSDOC]DB.STUFF 16 Mar 2004 rev. 06 May 2004 Database-related questions, and question-like statements, and statement-like questions: 1. "SLC-aware IOC" (just IOC for short) will at its bootup time pull down from wherever its copy of its piece of each of Epics native database and SLC operational database, right? Following questions assume yes. 2. In the IOC there will be (will there not?) a utility smart enough to update both, whatever mixture of names the update request specifies? For the SLC operational database part thereof, IOC will imitate sends to DBEX (as done by iRMXmicro procedures microdbsendb, dbdownloadme, and dbmdiag_sendrpy in module REF_RMX_DBS:DBMAIN.C86). SLC operational database itself has never provided any "callback" feature or automatic notification to anyone of any update. In VMS as well as in iRMXmicro, any fetch from the database simply fetches whatever is sitting in memory at the given moment for the specified primary.micro.unit.secondary, and in itself provides no active device-reading capability. 3. IOC will have code to imitate the update receiver in the "forever loop" of iRMXmicro procedure DBMAIN in module REF_RMX_DBS:DBMAIN.C86. I mean imitate loosely, not slavishly. 4. SLC operational database transactions between VMS and IOC will go through TCP proxy. Rest below is descriptive information about existing SLC operational database, followed by a little information about TCP proxy, followed by a little information about SLC message service (though not so relevant to database): Any individual item (terminal node, one might say) in SLC operational database is identified by quadruplet: primary.micro.unit.secondary, where unit is a 16-bit number and each of the other three is a 4-character identifier beginning with a letter. Grossest division of database (as it is held in memory) is by micro, in the sense that each micro owns one piece which does not overlap with any other micro's piece, and any item anywhere must belong to some (one) micro. (Note that in general no two iRMXmicro's can directly communicate with each other, though KISnet provides a very special-purpose point-to-point exception used by fast feedback.) Names of micros representable in database are listed in REF_DBSFILE:MICROS.DBS. Names of micros with which message service communication is supported are listed in REF_DBSFILE:MICRONAME.DAT (not quite the same list of names). Note that when a new micro (physically existing, not just a VMS process pretending to be a micro) is added, it has to be added to each of REF_DBSFILE:MICROS.DBS and REF_DBSFILE:MICRONAME.DAT (the latter says for each micro whether that micro is prod or dev and what kind of communication it supports (e.g., SLCnet or Ethernet by preference or by necessity) and how it gets booted). MICRONAME.DAT also matches each micro's 4-character name with 8-bit "bitid", each unique. Source file REF_RMX_SLCLIB:MICRONAME.C86 must in identically the same way match 4-character micro names with 8-bit "bitid's". Each micro's piece of database is most grossly divided (physically in memory) into five supertypes (as they are known): supertype 0 consists of pointer information by which name (primary.unit.secondary) gets translated into memory address. Supertype 0 information gets changed only at dbgen/dbinstall time. Other four supertypes contain actual data. Of them, supertype 1 is assumed to be least changeable, and is updated only with relative difficulty. Any update of a supertype 2 item originates only in VMS and causes both an update in memory in VMS and a message to the micro to do the same update in the micro's copy of same memory. Any update of a supertype 3 item originates only in the micro and causes an update in the micro's memory; a separate call sends accumulated updates (maybe plus a little more) to be sent by micro & stored in corresponding VMS memory by DBEX. Any update of a supertype 4 item originates only in VMS and is never seen by the micro (in other words, any micro's supertype 4 resides only in VMS memory for use only by application code in VMS). There is a known set of 4-character primary names (known as "primaries"), defined in REF_DBSFILE:PRIMARY.DBS. Each possible primary is defined in that file to have a fixed set of 4-character secondary names (known as "secondaries"). For any primary, each secondary's definition includes supertype number and data type. Possible data types are: 32-bit float, 16-bit integer, 32-bit integer, char string. Any integer secondary has either default decimal representation or default hex representation. Any float secondary or integer secondary can be a singleton or an array. Char strings are defined in a somewhat bewildering variety of ways. For any given primary, any unit of that primary has space for the same set of secondaries as any other unit of that primary. Any single instance of a given primary is identified by which micro it resides in, and, within that micro, its unit number. No micro can access any other micro's database. There are numerous files defining existing units of primaries by supplying (default) data for those units (where unit number is a 16-bit unsigned integer, variously restricted; generally 0 is disallowed; sometimes upper limit is 255), each unit belonging to a specified micro. All these files are in REF_DBSFILE:*.DBS. In any supertype block of any micro, all information is packed sequentially, and there is no way to add new unit definitions in the "running" database. Changing database storage allocation in any way requires "dbgen" (on dev Alpha) and "dbinstall" (on dev Alpha and subsequently on prod Alpha). "Dbgen" takes as input most of REF_DBSFILE:*.DBS and a few more files. More detail: all database message communication between micro and VMS is to/from process DBEX running in (prod or dev) Alpha. At micro bootup time, micro explicitly asks DBEX for download of its database. Download messages have same format as subsequent update messages, though at bootup time the downloaded pieces must arrive in a known sequence. Download messages of course cover all of supertypes 0,1,2,3 for given micro; subsequent or concurrent DBEX -> micro update messages are only of supertype 2, and micro -> DBEX update messages are only of supertype 3. At micro bootup time, no application code is allowed to begin executing until (at least) database download is complete. Any message traveling through the TCP proxy in either direction begins with (non-database-specific) fwd_hdr_ts as described below. For database download messages and database update messages in either direction, next header is database-specific, and is defined in lots of languages; I prefer structure dbsuptypehdr_ts in includefile REF_RMX_INCLUDE:DB_MICRO_STRUC.HM. Note that this structure also gives the beginning of each of any micro's supertype blocks as held in memory. This header (for example, dbsuptypehdr_ts) is followed by at most one (large or small) piece of strictly contiguous memory-image data, the quantity and beginning relative address within supertype block being specified in the header. You can also look at struct sbhdr definition in includefile REF_C_INC:CACHE.H. You can also look at REF_RMX_INCLUDE:SUPSTR.INC and REF_SLCTXT:DB_SUPSTR.TXT. I regard iC386 (Intel C) and Compaq Alpha C as distinct languages, non-interchangeable code, partly because iC386 has 48-bit pointers as opposed to 32-bit pointers. Likewise for FTN386 (Intel Fortran, which has no structures, lord save you) and Compaq Alpha Fortran. For structure of supertype blocks as held in micro, and structure of unit definitions contained in supertype 0, you can also look at comments in REF_RMX_DBS:DBLISTU.A38 (also this may give the best picture of data lists and pointer lists as created or expected by lowest-level (iRMX & VMS) database access procedures DBLIST, DBLGET, DBLPUT). Lowest-level iRMXmicro procedures are in REF_RMX_DBS:*.A38. Any message transmitted in either direction through the TCP proxy must begin with a "TCP proxy forward header", defined by C structure fwd_hdr_ts in includefile REF_C_INC:MSGHEAD.HC. (Note that fwd_hdr_ts header is absent from messages transmitted via SLCnet.) The TCP proxy (PX00 in prod network and PX01 in dev network, each running Linux) supports a TCP connection to each interested process in the (prod or dev) Alpha, and four TCP connections to each micro that has Ethernet. The proxy itself is as passive as possible, in the sense that none of the TCP connections just mentioned is established by the proxy. From a micro, to establish such a connection with the (prod or dev) proxy, you connect specifying proxy TCP port 6060 (named PROXIES_HOST_MSG_PORT below), and since you probably cant choose TCP port number for your own end of this connection, you send a first message specifying one of following four "MICROS_HOST_" values (as seen in includefile REF_C_INC:SLC_IP_PORTS.HC): PROXIES_HOST_MSG_PORT 6060 TCP port on proxy for all four types. MICROS_HOST_MSG_PORT 6060 TCP port on micro for SLC "message service" (the ubiquitous phrase "message service" almost always means just this one). MICROS_HOST_DBEX_PORT 6070 TCP port on micro for DBEX messages, including ack message from micro to DBEX acknowledging data message from DBEX to micro. MICROS_HOST_TIMEX_PORT 6080 TCP port on micro for TIMEX messages. MICROS_HOST_CTL_PORT 6090 TCP port on micro for control messages, including ack message from DBEX to micro acknowledging data message from micro to DBEX. For the given connection, this initial "registration" fwd_hdr_ts also contains the least significant 16 bits of the micro's own (your own) IP address; the proxy assumes that IP addresses truncated to least significant 16 bits are unique across all its client micros. After receiving this initial "registration" message on this connection, the proxy can recognize messages from the Alpha destined for this connection. For clarity here is a simplified restatement (without unions) of structure fwd_hdr_ts: typedef struct { unsigned short ipaddr_lo, conn_id; unsigned long datalen; unsigned char user[2], cmd, crc; } fwd_hdr_ts; This initial "registration" fwd_hdr_ts the micro sends (you send) to the proxy (of course goes no further than the proxy and) must contain: ipaddr_lo = least significant 16 bits of micro's own IP address, in network (big-endian) byte order. conn_id = one of above four "MICROS_HOST_" values, in network (big-endian) byte order. datalen = bytecount of data if any following this header, in network (big-endian) byte order. Zero for first message as mentioned above. user = anything. cmd = PX_REGISTER_PORT_FUNC = 5 as defined in REF_C_INC:MSG_IP_TYPES.H. crc = 0x55. For subsequent messages which the micro sends (you send) to the proxy, ipaddr_lo and conn_id become one 32-bit destination field, which is the 4-byte ascii Alpha process name unique abbreviation (in network byte order in the sense that most significant character of name is transmitted first). Field cmd must contain value PX_FORWARD_ALIAS_FUNC = 4, and field crc always contains 0x55. For each message stream which the micro receives from the proxy (i.e., for each connection), the micro assumes the datalen ( = bytecount not including fwd_hdr_ts itself) in each message's fwd_hdr_ts will in effect point exactly to the next incoming fwd_hdr_ts, and if a message received does not begin with a recognizable & correct fwd_hdr_ts then the micro should terminate that connection, reconnect, and send another initial "registration" fwd_hdr_ts, and hope for the best. Unlike MICROS_HOST_MSG_PORT connection or MICROS_HOST_TIMEX_PORT connection, the micro's (or IOC's) database code should establish two connections, MICROS_HOST_DBEX_PORT and MICROS_HOST_CTL_PORT. If sending to DBEX (destination Alpha process name unique 4-byte abbreviation = "V018"), you can request an ack by setting the PX_REQUEST_APP_RECEIPT = 0x80 bit in the cmd field (iRMXmicro generally does this), and wait right then and there for DBEX to send an ack via MICROS_HOST_CTL_PORT connection (separate connection allows you to wait without worrying right at that point about other messages DBEX is already sending to you). In fwd_hdr_ts of data message which the IOC receives (you receive) from DBEX, cmd field will contain PX_FORWARD_FUNC. If 16-bit id field in dbsuptypehdr_ts has SUPERTYPE_ID_ACKREQ bit (defined in REF_RMX_INCLUDE:DB_MICRO_STRUC.HM and in REF_C_INC:CACHE.H) set, then you must send an acknowledgement consisting of fwd_hdr_ts followed by a dbsuptypehdr_ts containing in id field the value (SUPERHDR_FUNC_ACK | SUPERTYPE_ID_OVERRIDE) with the SUPERTYPE_ID_IPLDWN bit also set if set in the same field in the message you are ack'ing. DBEX has its own delay queues allowing DBEX to continue transactions with other micros while waiting for this micro (you) to ack; hence you use same MICROS_HOST_DBEX_PORT connection to send this ack. "SLC message service" is primarily designed to support the following kind of transaction: a process in the Alpha sends a message to the primary task of a subjob in the micro (or sends one message each to the primary task in the given subjob in a number of micros) and waits for the reply (if one micro) or replies (if more than one micro). While waiting for the replies, the Alpha process can do nothing else. In any micro, any subjob's primary task is generally structured to send its reply before seeing any subsequent incoming message service message. In short, "speak only when spoken to," though there are some exceptions, one of which is the logging of error messages. Abovementioned Alpha process name unique abbreviation is necessarily of the form "Vnnn" where characters nnn are digits of decimal representation of number known as SLCnet interrupt id. A few SLCnet interrupt id's are defined permanently in includefile REF_C_INC:PROCESS_PARM.H; in particular, V018 = DBEX. (By the way, all success/informational/warning/error/fatal messages to be logged by calls to err_send0/1/2/3... from the micro are sent to Alpha process ERR_INT = destination V017.) Any Alpha process sending a message to the primary task of a subjob in a micro puts its own name abbreviation in longword source in structure msgheader_ts as defined in includefile REF_C_INC:MSGHEAD.HC, and puts ascii name of micro in dest longword in msgheader_ts, and puts destination iRMXmicro "subjob" number in 7 bits masked by 0x7F00 in 16-bit func word in msgheader_ts; these "subjob" numbers are defined in includefile REF_C_INC:MSGFUNC.HC and other REF_C_INC:*FUNC.HC. Header msgheader_ts appears only in "SLC message service" messages transmitted in either direction between a process in Alpha and a "subjob" in micro; msgheader_ts does not appear in database messages or in TIMEX messages.