CHANNEL_ARCHIVER_SERVICE__README.txt This is the README file of the ChannelArchiver service. Auth: Dave Hickin, 20-Mar-2012 (david.hickin@diamond.ac.uk) SUMMARY ------- The ChannelArchiver service is an EPICS V4 client and server which can be used to query the data archived by an EPICS Channel Archiver. It includes a bash script, gethist, which can parse a wide range of dates and times and process the corresponding data. EXAMPLES -------- The server is run as follows % ./start_server % ./start_server data/DCCT/index dccthist start_server calls the ArchiverService executable (which should be bin/$EPICS_HOST_ARCH/ArchiverServiceRPC) supplying two arguments. The first is the service name, the second the location of the index file. By default start_server supplies the index file in data/fredjanet and the service name archiveService. Should you wish to use other data or another service name then these should be supplied as arguments or the defaults changed. The following are examples of running the client using the bash script gethist: 0) Use the -h or --help options to get help % ./gethist -h % ./gethist --help 1) Retrieve archived data for the channel "fred" with the default start and end times, which are 1 hour ago and now. % ./gethist fred 2) Retrieve archived data for the channel "janet" between Jan 1 1990 and now. % ./gethist -s "Jan 1 1990" -e "now" -f out.txt -x janet 3) Long options are also supported. % ./gethist --start "Jan 1 1990" --end "now" --file out.txt --scientific janet 4) The options without argument can preceed another: % ./gethist -xn fred 5) A wide variety of date and time formats are supported (those supported by date -d). The following are all valid: % ./gethist -s "31 Jan 1990" janet % ./gethist -s "Jan 31 1990" janet % ./gethist -s "January 31 1990" janet % ./gethist -s 1990-01-31 janet % ./gethist -s 1990-01-31Z janet You can also supply a time, for example % ./gethist -s "30 Jun 2001 17:34" fred % ./gethist -s "30 Jun 2001 5:34 p.m." fred % ./gethist -s "30 Jun 2001 5:34 p.m." fred % ./gethist -s "30 Jun 2001 17:34:56" fred % ./gethist -s "30 Jun 2001 17:34 56 seconds" fred % ./gethist -s "30 Jun 2001 5:34 p.m. 56 seconds" fred The following are also valid % ./gethist -s yesterday -e now janet % ./gethist -s "last week" -e today janet % ./gethist -s "last year" -e "last month" janet % ./gethist -s "5 minutes ago" -e "10 seconds ago" janet % ./gethist -s "2 weeks ago" -e "1 hour ago" janet % ./gethist -s "4 years ago" -e "3 months ago" janet 6) Multiple channels can be queried with a given set of parameters as follows: % ./gethist -s "4 years ago" -e "3 months ago" fred janet gethist by default queries the service with name "archiveService". Services with other names can be queried using the -S option: % ./gethist -S a_service -s "yesterday" -e "now" a_channel The gethist script calls the archive service client (ArchiveClient), which can be called directly. It takes the same short options as gethist, except that the start and end times are in seconds past EPICS epoch and the service must always be specified with the -S argument. For example % bin/linux-x86/ArchiverClient -S archiveService -s 405012094 -e 720631294 fred ArchiveClient does not support long opts. Both gethist and ArchiveClient support a number of options see the help for each function for details: % ./gethist -h % ./gethist --help % bin/linux-x86/ArchiverClient -h The service can be also queried using the eget utility. The service name is specified using the -s argument and the query parameters using -a. For example % eget -s archiveService -a "starttime=405012094" -a "endtime=720631294" -a "entity=fred" (See below for an explanation of the query parameters). SERVICE INTERFACE AND RESPONSE FORMAT ------------------------------------- Requests to the service can be made using the NTURI normative type (see the EPICS V4 website: http://epics-pvdata.sourceforge.net/alpha/normativeTypes/normativeTypes.html). The query parameters are "entity", "starttime", "endtime" and "maxrecords", which are respectively the channel name, the start and end times between which to query (in seconds past the EPICS epoch, 01-01-1990 00:00:00 UTC") and the maximum number of entries to retrieve. The "entity" field is required. The server will error the request if it is absent. The others have default values of 0, 2,147,483,647 and 1,000,000,000 respectively. Thus the service can be queried using eget by supplying the parameters above. For example % eget -s archiveService -a "starttime=600825600" -a "endtime=600912000" -a "maxrecords=1000"-a "entity=SR-DI-DCCT-01:SIGNAL" retrieves the first 1000 archived values of SR-DI-DCCT-01:SIGNAL between the start and end times specified in seconds past the EPICS epoch - in this case between 15-01-2009 00:00 UTC and 16-01-2009 00:00 UTC - from the service "archiveService" and formats the result in columns. The service also accepts queries in the pure query form, i.e. a structure containing the string fields specified and thus the service can be queried using the eget -q option. The service's response is in the form of an NTTable normative type with the columns "value", "secPastEpoch", "nsec", "status" and "severity", which are arrays of respectively doubles, longs, ints, ints and ints. FILES THAT COMPRISE THE CHANNEL ARCHIVER SERVICE ------------------------------------------------ ArchiverServiceRPC.cpp C++ source code of the server side of ArchiverServiceRPC.h the channel archiver service ArchiverServiceRPCMain.cpp ArchiverClient.cpp C++ source code of the client application (top level) ArchiverClientResponseHandler.cpp C++ source code for the client handling ArchiverClientResponseHandler.h of the response from the server ServiceClient.cpp C++ source code for the client sending ServiceClient.h the query and receiving the response common.cpp C++ source code of types and utilities common.h common to client and server start_server bash script to start server gethist bash script to query service data directory containing sample Channel Archiver data test directory containing a script for testing the service. Type run_test -h for details of how to use this. SUPPLIED DATA ------------- By default the server queries some supplied data which has two channels ("fred" and "janet") and which covers the period on the 21st September 2005 between 17:49:57 and 19:33:16. This is located in the directory data/fredjanet. The data directory also includes some real data (storage ring beam current). The channel name is SR-DI-DCCT-01:SIGNAL and has values between 12/01/2009 11:20:41 and 21/01/2009 07:30:12. This is located in data/DCCT. PREREQUISITES ------------- 1. EPICS V3 Base - for a large number of EPICS types 2. pvDataCPP - for pvData 3. pvAccessCPP - for pvAccess 4. The Channel Archiver - for Channel Archiver access and types In addition, depending on your target/compiler, you may need 5. pvCommonCPP - for (Boost) shared pointers SETUP ----- 1. EPICS V3 Base, the Channel Archiver and the EPICS V4 prerequisites (pvData, pvAccess and, if required, pvCommon) should be built. 2. The Channel Archiver service needs to know the location of these. This can be achieved by creating a RELEASE.local file in the configure directory of the ChannelArchiverService directory. For example if your EPICS V3 Base and Extensions code is in /epics and the directories of the EPICS V4 prerequisites you wish to link to are in /epics4 then it could be of the form EPICS_BASE=/epics/base PVDATA=/epics4/pvDataCPP PVACCESS=/epics4/pvAccessCPP ARCHIVER=/epics/extensions ARCHIVER should point to the directory containing src/ChannelArchiver. This will depend on the layout of your EPICS installation. If you need pvCommonCPP you will also need include its location: PVCOMMON=/epics4/pvCommonCPP 3. The Archiver server and client are built by the command "make" issued from the ChannelArchiverService directory. 4. The start_server script starts the ChannelArchiverService server using the data supplied in the directory "data". Should you wish to use other data you need have data in the ususal format created by the ChannelArchiver application, including the index file, and will need to modify the start_server script to point to this as described below (see the ChannelArchiver manual, for example, http://www.slac.stanford.edu/grp/lcls/controls/global/sw/epics/extensions/ ChannelArchiver/manual.pdf). EXECUTION --------- This section describes how you start server and client sides of the Channel Archiver Service. Start the server side first. To start the Channel Archiver server ------------------------------------ * cd to the directory containing start_server E.g. % cd ~/Development/epicsV4/exampleCPP/ChannelArchiverService * Start the channel archiver service using the start_server command. The command can take two arguments or use default values. The first argument is the name of the index file and the second is the service name. The script calls the ArchiverService executable for the host architecture (which should be bin/$EPICS_HOST_ARCH/ArchiverServiceRPC). By default it queries the index file in data/fredjanet with service name archiveService. * Start the channel archiver service using the start_server command as descibed above using default arguments or supplying the index file and name service E.g. % ./start_server % ./start_server data/DCCT/index dccthist * The server can be terminated with a SIGTERM (like CTRL-C in its process window). To run a Channel Archiver Service Client using gethist ------------------------------------------------------ With the server started, * In another window from the server cd to the directory containing gethist (both client and server are in the same dir for demo purposes) E.g. % cd ~/Development/epicsV4/exampleCPP/ChannelArchiverService * Execute the gethist as described above E.g. % ./gethist --start "Jan 1 1990" --end "now" --file out.txt --scientific janet % ./gethist -S dccthist --start "10 years ago" --end "yesterday" SR-DI-DCCT-01:SIGNAL Querying the service using eget ------------------------------- With the server started, assuming eget is in your path: * Execute the eget utility: eget -s archiveService -a "starttime=600825600" -a "endtime=600912000" -a "entity=SR-DI-DCCT-01:SIGNAL"