NLCTA Archiver System

Guidelines For The Use Of This Document

This document provides a general description of the NLCTA Archiver system.

Overview

The NLCTA Archiver system was built under Solaris 9 and runs on the Solaris 10 opi00gtw04 machine. It is the oldest SLAC archiver system in active use and contains NLCTA archiver data as old as 2002. It is based on an older version of the Channel Archiver software than the LCLS, FACET, and Test Facility archiver systems. The NLCTA archiver system is the smallest SLAC archiver system with only one archive engine storing data for currently less than 1,300 PVs. The engine stores data in a NFS area.

The NLCTA archiver system libraries and executables were built using EPICS version 3.14.6. The NLCTA archiver system, like the other SLAC archiver systems, is an EPICS Channel Archiver system and an EPICS extension. The NLCTA archiver sources are under the /afs/slac.stanford.edu/package/epics/R3.14.6/extensions/src/archiver directory.

Archive Engine Subsystem

The NLCTA archive engine process obtains EPICS data and stores it in archive engine directories organized by months. Unlike the other SLAC archiver systems, the engine does not run under the control of a Archive Daemon process. The engine is started by a script and may be stopped by a UNIX "kill" command. The engine provides a web interface that provides information about the engine process.

The NLCTA archive engine process (as well as the index update process described later in this document) is started by the opi00gtw04 "Type 1" startup file /etc/init.d/st.archiver when the opi00gtw04 system is rebooted. To start this engine process manually:

  1. Logon to the opi00gtw04 machine using the "cddev" account.
  2. cd /afs/slac.stanford.edu/g/cd/soft/ref/ext/ChannelArchiver/script
  3. ./st.ChannelArchiver nlcta
    Respond with anything to the prompts (e.g., user name and reason for starting engine. This log information stored for these responses is no longer used.). Ignore messages about missing PVs since they are no longer used.

While the NLCTA archive engine process is running a URL is available that provides information about the engine. This information is contained on the main NLCTA archive engine web page: NLCTA Archive Engine Main Web Page. The information displayed includes the date/time the engine was last started, the archive index pathname (specifying the path where the engine is currently storing data and its associated index), the number of channels (PVs) being requested, the number of channels currently connected, and the next time archiver data will be written by the engine.

The NLCTA Archive Engine URL can also be used to obtain information about the connection status of each PV for an engine. For example, to obtain the connections status for all PVs of an archiver group for the NLCTA archive engine do the following from the NLCTA Archive Engine main web page:

  1. Select the "Groups" web link.
  2. On the resulting displayed groups web page for the engine select the group name web link for the engine group whose PV connection status you want to check.

The NLCTA archive engine writes its data and current index in a directory that includes the current year and month in its path. For March 2012 this path is:

The index name is not simply "index" as is the case for the other SLAC archiver systems. It is a name with the letter "i" followed by the current year, an underscore, and the current month number. For example, the full directory path for the March 2012 NLCTA archive engine index is: The current NLCTA archive engine index is displayed on the NLCTA Archive Engine main web page.

The directory to which the NLCTA archive engine writes its data and current index will not change until the engine is restarted. When the engine is restarted (either by reboot of the opi00gtw04 system or restarting manually using the st.ChannelArchiver script referenced previously) the engine startup script determines the current year and month, which is used to determine the directory in which the engine will store data and the name of the current index. Therefore if the NLCTA archive engine is running and storing data in the March 2012 directory it will not store data in the April 2012 directory until the engine is restarted in April. The NLCTA monthly archiver maintenance activities include starting the engine for the first time during a month. Please see the "NLCTA Monthly Archiver Maintenace" section of this document for more details.

IMPORTANT: The NLCTA archiver administrator must create the year/month data directories before they are used-- the NLCTA archiver scripts will not create them as needed. As of March 2012, all monthly directories have been created for the year 2012. Before the next year (e.g., 2013), the NLCTA archiver administrator needs to create the monthly directories for the new year under the following directory:

Archiver Server Subsystem

The NLCTA archiver server subsystem enables archiver data to be retrieved in response to requests by the NLCTA Archive Viewer.

There is a top-level NLCTA archiver index that is used to retrieve archiver data from both archive engine data areas:

This index is continually rebuilt (with a 30 second delay between rebuilds) by the following script: This script is started by the same "Type 1" startup file used to start the NLCTA archiver engine when the opi00gtw04 system is rebooted: /etc/init.d/st.archiver. It may also be started manually by performing the following:
  1. Logon to the opi00gtw04 machine using the "cddev" account.
  2. cd /afs/slac.stanford.edu/g/cd/soft/ref/ext/ChannelArchiver/script
  3. ./st.update_archiver_indexes_nlcta_top

The NLCTA archiver data may be displayed by invoking this system's Archive Viewer on a machine such as opi00gtw04:

  1. Logon to the opi00gtw04 machine using the "cddev" account.
  2. taarchview

The NLCTA Archive Viewer obtains archiver data and information by a request to the opi00gtw04 Archiver Data Server executable (invoked for each Archive Viewer request). This executable script may be found on the opi00gtw04 system here:

This script invokes the ArchiveDataServer C++ executable located in this same directory.

The following file on the opi00gtw04 system specifies the indexes that may be selected in the Test Facility Archive Viewer:

Unlike the LCLS, FACET, and Test Facility archiver systems the NLCTA archiver system does not read text files with lists of top-level index paths and an associated start/end time for each path to retrieve archiver data. The Archive Data Server uses the serverconfig.xml file to determine the location and name of the sole NLCTA top-level index file.

Monthly Archiver Maintenance

As previously explained, when the engine is restarted (either by reboot of the opi00gtw04 system or restarting manually using the st.ChannelArchiver script referenced previously) the engine startup script determines the current year and month, which is used to determine the directory in which the engine will store data. Near the first of a new month before the NLCTA archiver engine is restarted during the new month, maintenance work should be done including restarting the engine and insuring that data stored in the new month archiver data directory is accessible in the NLCTA top-level archiver index (and therefore accessible to NLCTA Archive Viewer users). The following procedure describes this monthly maintenance activity:

  1. Logon to the opi00gtw04 machine using the "cddev" account in three separate windows (refereded to in this procedure as the update index activity window, the engine activity window, and the edit window).
  2. In the edit window do the following:
    1. cd /nfs/slac/g/cd_archiver/tarf/data
    2. Edit the indexconfig.xml file by adding three lines just above the last line (</indexconfig>). The easiest method is to first copy the existing three lines just before the last line and then paste these three lines just below them. For an example of these three new lines:
      <archive>
        <index>2012/February/i2012_02</index>
      </archive>
      Then edit the index tag line: (1) changing the month to the previous month, (2) the month index to the previous month index, and (3) possibly the year (twice) if the new previous month was January. For example, the three new lines shown above would be transformed by editing to the following if the previous month was March:
      <archive>
        <index>2012/March/i2012_03</index>
      </archive>
      After these editing changes save the changes.
  3. In both the update index activity window and the engine activity window issue the following command:
  4. In the update index activity window stop the update index process by first determining the process id and then issuing the "kill" command:
  5. In the engine activity window stop the engine process by first determining the process id and then issuing the "kill" command:
  6. In the engine activity window immediately restart the archive engine:
    1. ./st.ChannelArchiver nlcta
    2. In response to questions regarding your name and reason for restarting the engine any response is fine (the log information resulting from this input is no longer used).
    3. Ignore all error messages referring to non-existant PVs. These PVs are no longer used.
  7. Verify that the engine has been restarted successfully by viewing the NLCTA archive engine main web page NLCTA Archive Engine Main Web Page. Also verify that the "Archive Index" information shows a path that includes the current year, month, and month index.
  8. In the update index activity window run the following command to update the subsystem_master_index index in the /nfs/slac/g/cd_archiver/tarf/data directory with engine data in the now closed previous month data directory: Wait until this script completes and the /nfs/slac/g/cd_archiver/tarf/data/subsystem_master_index file last modify time as shown by a "ls -l" command reflects the current date.
  9. In the edit window do the following:
    1. cd /nfs/slac/g/cd_archiver/tarf/tarf_top
    2. Edit the indexconfig.xml file (NOTE: this file has the same name as the file edited previously but is a different file in a different directory). Edit the last index tag line: (1) changing the month to the current month, (2) the month index to the current month index, and (3) possibly the year (twice) if the new current month is January. For example if the previous last tag line was:
      <index>../data/2012/March/i2012_03</index>
      If the new month is April the result of editing this line would be:
      <index>../data/2012/April/i2012_04</index>
      After these editing changes save the changes.
  10. In the update index activity window run the following command to create a new version of the NLCTA top-level index file in the /nfs/slac/g/cd_archiver/tarf/tarf_top directory named current_and_master_index2: Wait until this script completes.
  11. In the edit window make the new version of the NLCTA top-level index file the current version:
  12. In the update index activity window restart the update index process:

Verify that the above procedure was performed successfully by plotting any PV for a recent time period in days that spans archiver data for the previous month and the current month. As described previously, the NLCTA Archive Viewer can be invoked by the following command on opi00gtw04 under the cddev account:

There is a concern that the opi00gtw04 will be rebooted after the start of a new month and before the NLCTA archiver maintenance is performed near the start of this new month. This would cause the the new month data not to be available for retrieval until the monthly maintenance could be performed. This has never occurred but is possible. If the NLCTA archiver administrator peforms the monthly maintenance on the first workday after the start of a new month this should not pose even a potential serious problem since NLCTA runs only during normal working hours and the maintenance can be done early on this first workday before a problem is detected.

If the NLCTA archiver administrator is not available to perform the monthly maintenance on the first workday at the start of a new month an editing change to the st.ChannelArchiver script can be made to ensure that archiver data is stored in a specified month directory even if the engine is restarted by a reboot of the opi00gtw04 system during the next month. To do this perform the following procedure:

  1. Checkout the st.ChannelArchiver from the ext/ChannelArchiver/script branch of the old "SLAC-Controls Software" CVS repository:
    1. export CVSROOT=/afs/slac/g/cd/soft/cvs
    2. cvs co ext/ChannelArchiver/script
  2. cd ext/ChannelArchiver/script
  3. Edit the st.ChannelArchiver file. Search for a string such as "setting the month", which will be found towards the bottom of the file. The comments associated with this string describe how to enable the normalled commented out line "set nmonth =" line to set the month index to that for any desired month. For instance "set nmonth = 3" would set the archiver data month directory to the 3rd month (March).
  4. cvs commit st.ChannelArchiver
    This automatically updates the new version of st.ChannelArchiver in the /afs/slac.stanford.edu/g/cd/soft/ref/ext/ChannelArchiver/script directory.

When it is desired to revert back to the normal st.ChannelArchiver behavior of setting the current engine data directory to the current month the above process can be reversed by commenting out the "set nmonth =" again and CVS committing the changed version of st.ChannelArchiver.

Author:  Bob Hall 23-Mar-2012