presents the work involved in porting dpTestHist to dpChads under the lastest release of AIDA. It also presents work that remains to be done in CHADS.
This project provides the actual History Server called dpChads.cc. It will also run under the GrenAIDA-newLabour AIDA Release.
See the Chads History and dpTestHist page. DpTestHist was the predicessor to dpChads.
- (Done)Create CVS directories for dpChads
- (Done)Take dpTestHist code and rename things to make it into dpChads. Put new code in CVS and re-test it.
- (Done)Write a test client (see Greg's email for where it goes). It can be in java. It could prompt like dev/chads/.../ChadsClient.cc does for various parameters (even if we don't support all parameters yet). Those params include PV and time range for the get.
- Do the Corba Properties or environment variable settings necessary to allow running three incarnations of dpChads (one for each archiver database: PepII, NLC, and 8-Pack). See the design section bullet below entitled: How will CHADS get the data for any PV in the system?
- (Done)Add Exception throwing and logging
- (Done)Add process startup message.
- (Done)Always log through a local error logging class. That way, message throttling can be added later easily.
- (Done)Add exception catch and throw where needed. See testChArch.cc for an example of catching an ArchiveException object which is thrown by the Value Iterator operations.
- NOTES FOR LATER: DA does not support the throwing of excepitons by data providers. I got strange results when I threw as follows. Test this again after Exception handling is implemented in DA.
- Throw in DpTestServer.java. In daAIDATest 1, no exception is seen or caught. DA returns 0.0 to the test program.
- Throw in dpChadsServer.cc. In daDpChads 1, we catch an exception but it is null.
- In both cases above, DA does internally print out the exception (so it is catching it). But, it doesn't rethrown correctly to the test program application.
- LATER: Currently, dpChads_impl does not throw an exception if an error happens at instanciation time. Neither does dpCa or dpSLC. But, those others do throw in a method called init (but there in no logic in there in dpCa). So, is dpChads supposed to implement and init() method? If so, what goes in there, the stuff I have in the constructor now for instanciating the archive object? The way it is now, if an error happens in dpChads_impl at instanciation time, and then, the client tries to use the object, the client application might crash.
- (Done)Two new classes were created. These classes should be moved to some general purpose location if another C++ data provider is ever written.
- webUtil - general purpose web utilities for C++ programs. openURL() is one such method.
- aidaUtil - general purpose utilities for use by AIDA programs. register_with_name_service() is one such method.
- When we start CHADS, we use the Exception Logging configuration file to set the properties that tell where to connect to the event service for exceptions. This should be incorporated into the overall AIDA PROD/DEV scheme for setting properties. I wonder if an http URL works on -ORBconfig (on the command line). Or in the ORBACUS_CONFIG environemnt variable (probably not?).
- Speaking of PROD/DEV issues... Bob's Aida Prod/dev Requirements/Design Document (2/5/202) says that we need to run prod and dev incarnations of Chads. There are some issues with that. First, the channel archiver currently only really runs in prod mode. Second, if we did have a dev Chads Service, how would we tell it to connect to some other archive? Currently dpTestHistServer instanciates an Archive object with this value: $ARDATAFILES/multi_archive.txt.
- The CHADS server needs to handle exceptions thrown by the archiver access library routines (LibIO). But, it's not clear what those exceptions are and how to recover. The current archiver browser (written for NLCTA) in python does not worry about exceptions. The users are told to restart the browser if it is not getting data.
- Following in Bob Sass' footsteps (they're big footsteps, I know, but I'll give it a shot), the general exception scheme for a data provider is as follows. There will be a limited number of exceptions thrown back across the corba boundry. Bob's dpSlc just throws one exception from the get method called,"unableToGetDataException". He logs and throws all in one call the exceptions as they occur. He appends specific text to the exception (in the reason field) which tells what was being fetched and what error was returned by the SLC database. You can look at the SLC DB code to see that there are relatively few exceptions there too. If an exception occurs, the data provider returns no data and the client is expected to try again after correcting the problem.
- Consider creating an initialization function within dpChads_impl (currently dpTestHist_impl). It could be called from the constructor, and after the appropriate exceptions (to re-initialize).
- Related to exceptions, is the issue of connections. What if the connection fails to the oracle db? What if the database goes down? Is an exception thrown back to CHADS? What does a client do to recover in that case (call the distructor and then reinstanciate archive object?).
- Add metering to exception logging (so we don't overflow the log). This might need to be done in the cmlog logger (downstream from the event service).
- How do we handle the case of too much data requested. Currently, if more than DP_HIST_MAX_ELEMENTS is requested, then, we log a message and just return what we got to the user. Should we throw an exception?
- Populate Names Db for history data. This lets the client request data for a given PV. Right now, the test client asks for test//val.hist. Greg favors the syntax being NAME//hist.
(I added 3 pv's to the AIDA database purposes already)
- The .h file needs to be a dependency in the makefile. I currently isn't.
- Create seperate areas under src and/or bin for c++ daNameServer and aidaObjectRef code. Remember that code is all generated from IDL files, none of it is written by a programmer. All the IDL commands in dpTestHistServer's makefile don't all go into that one makefile. For example, we need the daNameServer generated code to be in it's own directory, not in dpChads' directory. Write a makefile for it there. Incorporate into top level makefile (tree traversal). Do the same for aidaObjectRef.
Greg adds that those modules (daNameServer and aidaObjectRef) should be just built into a common shared library that other c++ modules (like the history server) would use. dpChads would be in it's own library.
- Work with Greg and Jingchen on making it so dpChads (and ERR) startup under CDDEV on slcs6. Later needs to start on PROD too.
- dpTestHistServer.cc reads the nameserver IOR from a file on AFS. dpChads should use a URL to find the SOR instead. Several issues here. C++ does not have a URL class for opening a URL. So, I could use a 3rd party package like http://curl.haxx.se/libcurl/. Could I use corbaloc::? That would require a server, right? One option is to leave it as it is. The restriction would be that the aida name server must run on an AFS machine, as must the Chads Server (OK?). Another option is to put the SOR into Oracle. Then, the java and c++ programs would get it from there.
- Greg wonders if the real epics extension history server (dpChads) should go under EPICS extensions instead of under AIDA/src. Where ever it goes, it will be called dpChads as stated on the "Data Providers"AIDA web page.
- It's Kristi's feeling that the high-level EPICS/base/config Makefiles should contain the modifcations to support IDL compiling. I put the modifications in the low-level Makefile.Host in the dpTestHist directory. See 15 Jan 2003 email.
- Look at further optimization of the Channel Archiver calls. Can more be moved to the constructor to make the get method faster?
- Run code profiler (see GNU gCov). That only gives line counts. Do some timing tests too. Tools for it?
- The dpChads server will only receive a request for one PV at a time. Greg agrees. What does the server receive after the Db is populated? Is it the PV that's then passed to Channel Archiver? (yes) Is it ever more than one PV? (no) Like a range or list? (no). Are there Da utilities that are to be used in this process by the server? (no, Chads just gets the PV and passes it in the Channel Arch. request). In the future, Bob (Ken?) point out that the data providers will need to be able to handle lists to save bandwidth (see Turkey project).
- For the time being, there will be no other IDL methods implemented (just the one get() we now implement.
- The idl structure that history data is returned in will look like the structure of EPICS Chan. Arch. data.
- How will CHADS get the data for any PV in the system? The way things are now, CHADS gets data from the PEPII archive only as determined by the path to multi_archive.txt (or Oracle username/password) specified in the environment variable ARDATAFILES (or Archiver Oracle environment settings). Currently dpTestHistServer instanciates an Archive object with values determined from those environment settings. To support more than just PEPII data, we will have three (or more) Chads data providers running (one for each archiver database: PepII, NLC, and 8-Pack). Then, DA would be responisble for directing the request to the correct Chads data provider. (greg agrees).
An alternative approach would have been to just have one CHADS data provider running and have it create and manage three archive objects itself. That one server would receive requests for all PV's and figure out itself which archive to access. It's not clear that LibIO supports having three archive objects in one client. That support would need to be determined. The first way sounds best (multiple servers).
Long Term (later) Design Issues
- Consider multi-threaded server. See the email from me with reply from IONA entitled "Concurrency Model Question". With a threaded server (which ours are), we need to do thread syncronization within our servers. I did a poor-man's version of syncro. in dpChads, but it's a time consuming kludge. Threaded server is an Orbacus property. See Orbacus manual "Concurrancy" chapter. Orbacus provides a Jthreads class library for implementing the Java thread class in C++. It's in ORBACUS/jtc directory. Use it!
- Consider setting Orbacus Properties for a 'different' type of server.
[SLAC Controls Software Group][ SLAC Home Page]
Author: Ron MacKenzie 03-Mar-2003