Test Facility Archiver System

Guidelines For The Use Of This Document

This document provides a general description of the Test Facility Archiver system.


The Test Facility Archiver system was built under RedHat 5 and runs on the the RedHat 5 testfac-archeng machine. It is similar to the LCLS and FACET archiver systems but both the "sampling" (engine) and "server" functionality runs on one system (testfac-archeng) rather than separate systems. Also each engine (currently only one) stores data to a NFS area rather than local disk.

The archiver system libraries and executables were built in AFS space under /afs/slac/g/acctest. The EPICS Channel Archiver is an EPICS extension and the Test Facility Archiver system was built using EPICS version 3.14.12. The archiver sources are under the /afs/slac/g/acctest/epics/extensions/extensions-R3-14-12/src/ChannelArchiver directory.

Archiver Sampling Subsystem

The Test Facility archiver sampling subsytem obtains EPICS data and stores it in current archiver engine directories. The archive engine processes (currently only one) provides this functionality. Engine processes run under the control of the ArchiveDaemon process, which will restart any engine process that is stopped or dies. The ArchiveDaemon process also provides a web interface that allows engine processes to be stopped (which the ArchiveDaemon process will subsequently restart) as well as providing information about each engine process.

The Test Facilities ArchiveDaemon process is started by the testfac-archeng "Type 1" startup file st.ArchiveDaemon when the testfac-archeng system is rebooted. Starting the ArchiveDaemon process will start all engine processes. To start the ArchiveDaemon process manually:

  1. Logon to the testfac-archeng machine using the "acctf" account.
  2. cd /nfs/slac/g/cd/tf_archiver/acctfeng
  3. scripts/start_daemons.pl -p [Note: the "-p" option is only needed if there may be engine processes already running, which must be killed first]

While the ArchiveDaemon process is running a URL is available that provides information about each engine it controls. This information includes the current number of channels (PVs) connected and the number of channels requested to be archived for each engine. This information is contained on the main ArchiveDaemon web page: Test Facility ArchiveDaemon Main Web Page. Connection information is under the "Status" heading (e.g., "1/4 channels connected" would mean one channel is currently connected out of the four that are requested to be archived for the associated engine).

The ArchiveDaemon URL can also be used to obtain information about the connection status of each PV for an engine. For example, to obtain the connections status for all PVs of an archiver group for engine "acctf_1" do the following from the Test Facility ArchiveDaemon main web page:

  1. Select the ACCTF_1 web link.
  2. On the resulting displayed web page select the "Groups" web link.
  3. On the resulting displayed groups web page for the engine select the group name web link for the engine group whose PV connection status you want to check.

If it is desired to restart an archive engine:

  1. On the Test Facliity ArchiveDaemon main web page note the port number (under the "Port" heading) for the engine you want to restart. For example, the first engine will have the port number 4901.
  2. Stop the engine by entering a URL of the form: "http://testfac-archeng:XXXX/stop", where "XXXX" is the engine's port number.
  3. Return to the Test Facility ArchiveDaemon main web page and note that the "Status" column will eventually show "Not running" and then later the connection status for the restarted engine during the process of the ArchiveDaemon stopping the engine process and then restarting it. This process is typically done after an engine's configuration file has been changed (e.g., adding or removing PVs) since the engine must be restarted to read the modified configuration file.

If it is desired to stop the running Archive Daemon process and all of the engines it controls:

  1. Logon to the testfac-archeng machine using the "acctf" account.
  2. cd /nfs/slac/g/cd/tf_archiver/acctfeng
  3. scripts/stop_daemons.pl -p

An engine writes its data and current index file in a subdirectory of the engine's archive engine directory (for example, for engine "acctf_1" this directory is /nfs/slac/g/cd/tf_archiver/acctf_1). The soft link "current_index" points to the index file relative directory pathname (e.g., 2012/02_10/index). This path changes after the engine has been restarted on a day after that indicated in the path (e.g., 2/10/2012, for the previous example relative directory pathname).

The engine's configuration file is located in the engine's archive engine directory (e.g., /nfs/slac/g/cd/tf_archiver/acctf_1). For engine "acctf_1" this file is named "acctf_1-group.xml". As previously explained, the associated engine must be restarted after its configuration file has changed in order for the engine to read the modified file.

Archiver Server Subsystem

The Test Facility archiver server subsystem enables archiver data to be retrieved in response to requests by the Test Facility Archive Viewer. It also performs activities necessary to support this activity, such as copying data from the archive engine data areas to a long-term data storage area and rebuilding indexes.

There is a top-level Test Facility archiver index that is used to retrieve archiver data from both archive engine data areas and the long-term data storage area:

This index is continually rebuilt (with a 30 second delay between rebuilds) by the following script: This script is started by the testfac-archeng "Type 1" startup file /etc/init.d/st.updateIndex when the testfac-archeng system is rebooted. It may also be started manually by performing the following:
  1. Logon to the testfac-archeng machine using the "acctf" account.
  2. cd /nfs/slac/g/cd/tf_archiver/arch_acctf
  3. scripts/st.update_indicies_not_server

The archiver administrator runs the following script after all engines are restarted to copy archive engine data areas data/index files to the long-term data storage area and to rebuild the long-term data storage area indexes (the top-level index master_index in /nfs/slac/g/cd/tf_archiver/acctf and master_index files for engines in subdirectories below it):

  1. Logon to the testfac-archeng machine using the "acctf" account.
  2. cd /nfs/slac/g/cd/tf_archiver/arch_acctf
  3. scripts/update_server.pl

Test Facility archiver data may be displayed by invoking this system's Archive Viewer on a machine such as testfac-srv01:

  1. Logon to the testfac-srv01 machine using the "acctf" account.
  2. acctfarch

The Test Facility Archive Viewer obtains archiver data and information by a request to the testfac-archeng Archiver Data Server executable (invoked for each Archive Viewer request). This executable script may be found on the testfac-archeng system here:

This script invokes the ArchiveDataServer C++ executable located in this same directory.

The following file on the testfac-archeng system specifies the indexes that may be selected in the Test Facility Archive Viewer:

The ArchiveDataServer C++ executable uses the acctf_indexes_info.txt and acctf_sparce_indexes_info.txt files in the /nfs/slac/g/cd/tf_archiver/acctf_aida_indexes directory to retrieve Test Facility archiver data. These files specify the top-level indexes used to retrieve data for specified time spans. These files are edited by the archiver administer whenever new top-level indexes are added.

Author:  Bob Hall 09-Feb-2012