The History of CHADS and The Porting of CHADS to dpTestHist under the GrenAIDA-newLabour AIDA Release.

SLAC Detailed SLAC Computing ESD Software AIDA RONM Home

This page presents the history of CHADS and also
the work that was done to port CHADS to run under the lastest release of AIDA.

What's this project all about?

In the beginning, I created a program called CHADS (Channel Archiver Data Source). Chads was without form, and void; and darkness was upon the face of the deep. Then I said, let me port Chads to dpTestHist under the lastest release of AIDA.

The porting of Chads to dpTestHist provided a c++ server to replace the java server DpTestHistServer.java which George/Greg wrote. That new c++ server is called dpTestHistServer.cc and runs under the GrenAIDA-newLabour AIDA Release.

This new DpTestHistServer optionally calls the EPICS Channel Archiver to return real history data instead of just returning static hard-coded test data like DpTestHistServer.java does.

The next phase of this project is to provide the actual History Server called dpChads.cc. It will also run under the GrenAIDA-newLabour AIDA Release.

Here's what George/Greg wrote about returning history data through GrenAIDA newLabour:
With the latest release of AIDA, it is now possible for AIDA data providers to choose to return a "raw" Any, for instance, one that has been populated by streaming data into an Any from a IDL defined struct via the insert() Helper routine of that struct. An example case is if the Data Provider is returning data that is already available through CORBA as struct. Another example (as right now) is when the c++ implementation of DaValue is not yet available, so the utility routines for populating it, and for converting it to an Any (toAny) are not yet available, but we still need to return a structure.

For these reasons this Aida Data Provider (DpTestHistServer.java) has been written to test the raw CORBA any handling of Aida, and illustrates the use of an IDL defined structure, histData.

(the comments above are from George/Greg's file $AIDA/idl/dpTestHist.idl).

dpTestHistServer.java is here: $AIDA/edu/stanford/slac/aida/dp/dpTestHist

The History of Chads

• Chads had three project phases up-to and including the port to dpTestHist.
  
  
  1. The package/aida ChadServer release (early 2002). This release was doucmented on the Aida Server's SysAdmin guide and resides in /afs/slac/package/aida. This release is pretty much independent of other aida utilites and consists of a client and server. The server (ChadServer) for this release calls EPICS Channel Access to fetch History data for a hard-coded PV. This release was not well integrated with the rest of AIDA.
  2. The ~ronm/dev/chads ChadServer version (July 2002). This version, which was based on the previous release, was never officially released. But, it was an important milestone because George/Greg made it work a little bit with the newLabour AIDA release. They were able to make a da.getX call (through AIDA da) and enter into the ChadsServer. ChadServer then called EPICS Channel Access to fetch History data for a hard-coded PV. The thing that didn't work was the return of the history data through AIDA da.
  3. The $AIDA dpTestHist release (January 2003). This release is what the first part of this web page is about. The code is going to be released to the EPICS extension tree under $AIDA/src/epics.... The release works with the GrenAIDA NewLabour Release of AIDA. ChadServer was renamed to dpTestHistServer for this release to be consistent with AIDA server naming convensions. This release supports requesting data throught AIDA da, calling EPICS Channel Access for a hard coded PV, and returning the data through AIDA da.

The dpTestHist Release

The dpTestHist release provides a History Server written in C++ with a method: Any = get(target, args). The existing idl file that George/Greg wrote is used as a basis for the code ($AIDA/idl/dpTestHist.idl). This release uses an IDL structure (called struct histData) that we define in dpTestHist.idl for holding the history data. That IDL structure is streamed into a Corba Any which is returned by the get call. The dpTestHist server registers with the Da Name Server through the interface defined in $AIDA/daNameServer.idl





The makefile for this release (in the sub-directory under package/aida/src/epics) is executed by typing gmake. It does not use George/Greg's scheme of automatially generating the IDL file because, for one reason, the IDL file for dpTestHist must contain the struct histData for holding the history data and the automatic IDL generation scheme does not support creating a struct in an IDL file





Gregs Data Providers page describes how Chads fits in with all the other data providers.

What was done initially for the dpTestHist Server release? This list is in no particular order.

• (Done)Clean up the C++ code in the server and impl. Add standard headers (comments).
• (Done)Fix the makefile. It needs to do it's IDL commands. Also, Clean it up. Use more variables and make it more readable.
• (Done)Do initial release to CVS and see it all work from the reference area. Make sure you can view the source from cvsweb.
• (Done)Cause George/Greg's top level makefile invoke the dpTestHist makefile as it traverses the tree. (see Greg's email). This requires a change to makefileAida.sun4 and other stuff. Test that it works with aidamake.
• (Done)Rewrite the server so it doesn't instanciate the archive for every get() operation like it does now (see dpTestHist_impl.cc).
• (Done)Do Performace testing (like test 11 in Tests.java), and Performance tuning of the code.
• (Done)Rewrite the server to only fetch the Channel Archiver data once. Currently, we fetch once to count how much. Then fetch again to actually store it into the unbounded sequence. That gives better performance for the sequence, but the Channel archiver fetches are file operations. So, either fetch once into memory, then, copy over to the sequence. Or, better yet, rely on the sequence expansion (like re-alloc, I imagine) to expand the sequence.

What were the final steps for the dpTestHist server release? This list is in no particular order.

• (Done)Add repeat count to the IDL defined History Data structure and return the repeat count along with time and values currently being returned
• (Done)The client needs be able specify time ranges before getAny(). We need to use AIDA query parameters, see Tests.java for examples. Use r.setParm()
• (Done)The client needs to be able to pass an actual PV in the target and the server needs to fetch data for that target
• Populate Names Db for history data. This lets the client request data for a given PV. Right now, the test client asks for test//val.hist. Greg favors the syntax being NAME//hist.
  (I added 3 pv's to the AIDA database)
• (Done)When the archiver is converted to use Oracle instead of a file based storage system, the archive_name in dpTestHist_impl() will need to be changed to indicate Oracle username/password/db_instance via a utility. Create an environment variable, which, when set will determine if dpTestHist connects to the file or Oracle based Archiver (ie support both types of connections).

Performance Issues:

• (Done)Reduce size of timestamp returned to client. At least take off all the trailing zeros to save bandwidth.
• (Done>Write a standalone test program which fetches Channel Archiver data from either the Oracle or File based Archiver. It should accept the PV and Time Range in command line arguments. This program will enable us to determine how much of the time that is taken by an AIDA fetch of history data is really Archiver time. It will also enable us to determine if any PV's or time ranges are problematic.

Author: Ron MacKenzie 08-Jan-2003
Modified by: