SLAC ESD Software Engineering Group
	Integrated Relational Model of Installed Systems IRMIS @ SLAC	SLAC Detailed SLAC Computing Software Home Software Detailed topic-homepage, e.g. ESD Standards

• IRMIS Overview

• IRMIS at SLAC
• User Interface and querying
• IRMIS for LCLS
• IOC Report (updated daily after crawls)
• Operational Details
  • Oracle schemas
  • Crawler scripts
  • Crawler cron jobs and logs
  • Source code directories
  • IRMIS source directories
  • IOC Boot directories
  • Set IOC(s) to be crawled
  • Schema diagram

IRMIS Overview

"IRMIS is a collaborative effort between several EPICS sites to build a common Relational DataBase schema and a set of tools to populate and search an RDB that contains information about the operational EPICS IOCs installed at that site."  IRMIS (the schema, crawler programs and UI) was developed by Don Dohan and Claude Saunders at APS.  For general information and distributions see the IRMIS home page.

IRMIS at SLAC

IRMIS is used (or will be used) at SLAC to query for PVs and their attributes, and associated resources and PV clients, by a variety of criteria.  Querying can be done using the IRMIS desktop UI program, or by writing sql scripts.   PV lists obtained from IRMIS can be used in the construction of config files for pv clients (alh, car, cw, possibly displays...)

Elements of IRMIS have been adopted and adapted for the controls software group at SLAC:

• The PV schema
• Oracle schema
• PV and ALH crawlers
• the IRMIS Desktop, a java UI

Other elements of IRMIS include cabling, device and application schemas.  We are not populating these now...but may in the future.

Elements modified or created at SLAC:

• Some small adaptations in the PV crawler and schema
• Addition of config file tree structure crawling in the ALH crawler
• New crawlers for Channel Watcher and Channel Archiver
• New PV Client viewer addition to the IRMIS Desktop
  

top

UI and DATABASE QUERYING

IRMIS DESKTOP

This is a java UI for the IRMIS Oracle database, and can be invoked in 3 ways:

1.  from XP:

for the Production database double click on v:\cd\soft\Projects\IRMIS\IRMIS_APP\irmis.jar

for the Development database double click on v:\cd\soft\Projects\IRMIS\IRMIS_APP_DEV_DB\irmis.jar

2.  from a Solaris or Linux workstation:

run this script (Production database only):

irmisUI

** If you're on XP, please run as shown in #1 above, the v: drive.  Running the solaris or linux version in a teraterm window is slow.

3.  from the lclshome edm display:

click the IRMIS button in the Tools section (Production database only)

So far at SLAC, only the idt::pv and the idt::pvClient document types are populated.

The IRMIS Desktop comes up with the idt::pv document displayed.

To see the pvClient document, select File/New Document/idt::pvClient

Query results can be saved to an ascii file for further processing.

SQL querying

Two views have been created to ease sql querying.  These views combine data from the IOC_BOOT, IOC, REC and REC_TYPE tables, and the FLD and FLD_TYPE tables.  They select currently loaded PVs, where IOC_BOOT.CURRENT_LOAD = 1.

• CURR_PVS has all currently loaded PVs
• CURR_PVS_PLUS_FIELDS has currently loaded PVs and their fields
  

Query scripts for formatted Channel lists for PV Clients (interim)

The plan is to develop a really cool and spiffy way to generate PV Client channel lists for ALH, CW and CAR from the IRMIS db, but for now we have a first pass at a really simple-minded script version. It uses our new irmis_ro account.

The script is:
~jrock/work_lcls_irmis/pvClientQueries/getClientList.csh

You have to cd to ~jrock/work_lcls_irmis/pvClientQueries first in order to run it.

The script currently has no error handling or parameter checking.

The script's parameters are:
PV Client (ALH, CAR or CW)
(optional W) - use this if you want to enter a freeform where clause

For example:
getClientList.csh CAR
or
getClientList.csh ALH W

The script will prompt you for output filename, search criteria and desired client flag values.
The script queries the database based on system (e.g. LCLS), ioc (e.g. XL02) and record name. In any of these, the wildcard character % can be used. For example, %BDES will result in all the BDES values for the given system(s)/IOC(s). XL% will result in all the XL IOCs, % will result in all IOCs, etc.

If you specify "W" as the optional second parameter, you will be prompted for a freeform where clause, instead of being prompted for ioc and record name. For example you could enter
ioc like 'XL%' and (rec_nm like %BDES or rec_nm like %IDES)

top

OPERATIONAL DETAILS

Oracle schemas

The IRMIS database schema is installed in 2 SLAC Oracle instances:

• SLACPROD contains production data, crawled daily
• SLACDEV contains the schema which is used for testing new features and crawling new boot directories.

The owner username in both instances is irmisdb.  For the password see Judy Rock, Bob Hall or Ken Underwood.

There is a read-only user as well: irmis_ro, password irmisread.

see schema diagram below.

Crawler scripts

The PV crawler is run once for each IOC boot directory structure.  The SLAC PV crawler (runSLACPVCrawler.csh) runs the crawler 3 times to accommodate the various CD IOC directory structures.  The LCLS PV crawler (runLCLSPVCrawler.csh) runs the crawler only once (so far!).   It is separate to enable it to be run on a different schedule if necessary. 

PV crawler operation summary

For the location of the crawler scripts, see Source code directories below.

The SLAC scripts that run the crawlers (including the LCLS crawler script) live in the CD software distribution path.  The IRMIS crawler perl programs themselves are run by the scripts , which are not gmake-d nor distributed.

runSLACPVCrawler.csh and runLCLSPVCrawlerLx.csh set up for and run the IRMIS pv crawler multiple times to hit all the boot structures and crawl groups.  Environment variables set in the crawl scripts, and in pvCrawlerSetup.csh, point the crawler to IOC boot directories, log directories, etc.   Throughout operation, errors and warnings are written to the log file.

1. First, source pvCrawlerSetup.csh/pvCrawlerSetupLx.csh to set up crawler env vars.
2. Do any necessary setup specific to the crawl group (e.g., copy TR01 files, rdist LCLS files, etc.)
3. Set the Oracle IOC table ACTIVE column to 1 for the set of IOCs to be crawled.  This is done in the set*IOCsActive.csh script for the current crawl operation, based on the IOC's System (crawl group).  e.g., set all LCLS IOCs ACTIVE to 1.
4. Next, run /afs/slac/package/epics/vers2/tools/irmisV2_SLAC/db/src/crawlers/pv/pv_crawler.pl:
   1. Call /afs/slac/package/epics/vers2/tools/irmisV2_SLAC/db/src/crawlers/pv/SLACBootScan.pm to get list of IOCs to be crawled (Active = 1).
   2. For each IOC in the list, check its boot date.  (Most PEPII IOCs have a TIMEOFBOOT PV- this is checked to determine actual boot date, which is compared to the last boot date crawled by IRMIS. LCLS IOCs for the most part have a STARTTOD PV).  If the boot date has changed, or if there is no boot date found, or if sequence crawling is not disabled, crawl the IOC, otherwise skip this IOC.
   3. Read through the IOC's startup file, and parse each command.
   4. dbd, db and template files are parsed IF their modification date or size has changed.  PVs from the parsed files are stored in the Oracle database. 
   5. Sequence .o are also parsed.  Sequence parsing is NOT done for soft IOCs, nor for LCLS IOCs, because the .o file is not available.
5. Reset IOC.ACTIVE columns are all reset to their normal state.
6. Send Error and Warning e-mails.

** For PV Crawlers: the crawler group for any given IOC is determined by its row in the Oracle IOC table.  The system column refers to the boot group for the IOC, as shown below.

** The PV client crawlers load all client directories in their config files; currently includes both CD and LCLS. 

Log filenames are created by appending a timestamp to the root name shown in the tables below.

top

Crawler cron schedule and error reporting

Crawlers are launched by trscrontab on slcs2, user cddev.  All run on slcs2, except runLCLSPVCrawlerLx.bash, which runs on lcls-daemon2.

Following the crawls, the calling scripts grep for errors and warnings, and send lists of these to Judy Rock, Bob Hall and Ernest Williams and Jingchen Zhou.  To track down the error messages in the e-mail, refer to the log file du jour.

(for soft IOCs, sequences are not crawled because they are not available as .o files)

• all crawls: "Use of uninitialized value in string at PVCrawlerDBLayer.pm line 409"

(this happens when the boot date is 0 - for those IOCs without a TIMEOFBOOT PV - this is OK)

• AFS crawl: "Could not locate /afs/slac.stanford.edu/g/cd/soft/ioc/PB61/envPaths or setEnvEpics"

(we don't need this for crawling)

• some crawls: unable to reference TOP or IOC variables - in general these are OK.  However, if they are followed by other directories that can't be found or referenced, this may indicate a problem that needs fixing.
  

top

Source directories

In the SLAC scripts area, each crawler has certain files and directories:

• config : list of files or directories to crawl, distributed to $CD_CONFSYS
• script : scripts that run the corresponding crawler, and any other auxiliary scripts

The LCLS script area contains all LCLS-specific scripts, config files, etc.

In the IRMIS software area, each crawler has its own source code directory.  The SLAC/LCLS crawler run scripts cd into these directories to run the crawlers, where db.properties can be found.  The database instance is set here, along with other properties.

The UI is run from a button on lclshome which invokes the irmisUI script. The jar file is here: /usr/local/lcls/package/irmis/irmisV2_SLAC/apps/deploy/irmis.jar

/afs/slac/package/epics/tools/irmisV2_SLAC/README has info on the applications and also instructions for building the IRMIS UI.  (although work on the UI is much better done in Eclipse!!)

top

IOC Boot directories

IOC boot directories are:

top

PV Crawler: What gets crawled

The IOC table in the IRMIS schema contains a list of IOCs.  The system column designates which crawl grouping the IOC belongs to.  For LCLS IOCs, the group is "LCLS", not surprisingly!

An IOC will be hit by the pv crawler if its ACTIVE column is 1 in the IOC table. 

An IOC will not be crawled unless its STARTTOD (boot time) has changed since the last crawl.

Specific files will not be crawled unless their mod date or size has changed.  Crawling specific files can be triggered by changing the date (e.g. touch).

IOCs without a STARTTOD PV will be crawled every time.

post-crawl emails: what to look for

• Comparison of number of IOCs in 2 log files
  If the numbers don't match, the PV Crawler terminated early, before it processed all the IOCs. Go to the logfile to determine the cause. This is rare.
• Duplicate PV report
  Forward these to the relevant IOC Engineer.
• IOCS not found in IRMIS, added to IOC table
  If an IOC directory in $IOC is not in the IRMIS IOC database table yet, IRMIS will automatically add it, and produce a message which you'll see here.
• PV Crawler messages
  Usually Don't worry about "expecting 3 arguments to nfsMount"!
  Quickly check "Could not locate" - usually an IOC dir that isn't quite in production yet.
  You can send a message to the IOC engineer if you see " unreachable" - it could be a typo, or it could be intentional.
  A "Could not locate" an st.cmd file will create "Parse error: 2" messages, which can be ignored if the IOC isn't in production.
  

where IRMIS PVs go

Qualifying IRMIS PVs populate the bsa_root_names table, which is joined in with Elie's complex views to device data in lcls_infrastructure. For details on the bsa_root_names load, please have a look at the code for the stored procedure which loads it (see one of the tables above)

Crawling a new directory structure

• start off by testing in the SLACDEV instance!!  To do this, must check the pv crawler dir out of cvs into a test directory, and modify db.properties to point to SLACDEV instance PLUS cvs checkout crawl scripts and setenv TWO_TASK to SLACDEV for set*IOCsActive scripts.
• all directories have to be visible from each the IOC boot directory
• the host running the crawler must be able to "see" Oracle (software and database) and the boot directory structure.
• set up env vars in run*PVCrawler.csh and pvCrawlerSetup.csh.
• if necessary, create crawl group in IOC table, and set*IOCsActive.csh to activate.
• add call to pv_crawler.csh to run with the new env vars...
• hope it works!

top

SCHEMA DIAGRAM

/afs/slac/package/epics/tools/irmisV2_SLAC/db/ddl/irmis-pv-schema.png

Added table:  the ioc_alias table contains aliases for LCLS IOCs.  For example, ioc-sys0-al00 alias is AL00.

top

Programmers' Guides, Users' Guides, Requirements, Design, Papers, Administration, How-To, Hardware, IOC, Database

[SLAC ESD Software Engineering Group][ SLAC Home Page]

Author: Judy Rock 27-Nov-2006
Modified by: 17-May-2007, jrock, modified to point to new LCLS crawler run script location for Linux. Note, all modifications are listed on same line!