Migration Procedures to incorporate Object Persistence
into the Reconstruction environment

David Quarrie

BABAR Computing

Version Information

Draft: 29th June 1997

This document is still under development. If you have any questions or comments, please address them to the author.

Table of Contents

• Introduction
• Common Assumptions
• Container Classes
• Migration for the Persistent Strategy
• Migration for the Transient Strategy

Introduction

This document is a draft for detailing the migration procedures that will have to be performed in order to incorporate object persistence into the reconstruction environment. Two alternative strategies are being pursued for this, and the migration procedures for each are detailed here. The different strategies are:

1. Perform a direct conversion of the existing transient objects that are to be made persistent, and modify any necessary usage of them. This is the Persistent Strategy.
2. Create a set of new persistent classes and hide them behind modified versions of the transient classes, maintaining the existing interfaces if possible. This is the Transient Strategy. This is described in detail in The BABAR Event Application Programmer Interface.

Common Assumptions

Both strategies are based on use of the RD45 header files and macros. These are detailed in the draft BABAR DDL Coding Guidelines and Hints. Some relevant information is:

• Persistent-capable classes have to inherit from a persistent base class (BdbPersObj or d_Persistent_Object)
• The BdbNew(A) macro replaces the new operator and must be used to create objects of persistent-capable class A.
• The BdbRef(A) macro replaces the object pointer (A*) and allows other objects to reference persistent objects. Unfortunately, as currently implemented by RD45, the semantics of BdbRef(A) are different depending on whether it appears in a transient object or in a persistent object. In a transient object, BdbRef(A) has the identical semantics to a normal pointer, thus allowing use of the * and -> operators etc. In a persistent object it behaves like an association, which has the following implications:

B* theB;        // Transient
    theB = <something>;   // setting the pointer
    theB->aFunction; // Accessing a function of B

    #include "PkgB/B.hh"  // New
    #include "PkgB/PB.hh
    BdbPtr<B, BP> theB;

• The BdbDelete(A) macro replaces the delete operator for persistent objects.
  

• In order to be machine-independent, the ODMG datatypes d_Long, d_Float, etc. should be used instead of int, float, etc.
• Persistent-capable objects cannot be embedded in other objects (transient or persistent). Thus, if class A is persistent-capable, then class B cannot directly contain an object of class A, it can only contain a reference BdbRef(A) to an object of class A.
• RD45 provide a limited set of container classes that can contain persistent objects (or references to persistent objects). HepVector(A) contains persistent objects whereas HepRefVector(A) contains references to persistent objects. Neither of these container classes is directly persistent-capable, but must be embedded in another persistent-capable class. They have interfaces that are similar to the equivalent Rogue Wave vector classes.

The advantage of the RD45 headers is that the macros will function in either a fully transient environment (in which case BdbNew(A) just converts to "new A", BdbRef(A) to A*, etc.) or to a persistent environment (where they map to the corresponding Objectivity operators). Thus they can potentially play an important role as part of the migration strategy and also in allowing classes to be used in an environment where Objectivity is not present (which might be the case for Online Event Processing).

The following discussion also makes the assumption that persistent-capable classes cannot be created transiently. This is in fact not strictly correct, but severe restrictions apply to which persistent-capable classes can usefully be created transiently. For example, they cannot contain any references to other classes. In addition, consider the situation where persistent-capable class A has a reference to persistent-capable class B (BdbRef(B)). If B is created transiently, then there is no way by which the BdbRef in A can be set to reference B.

Container classes

Both strategies depend on a set of persistent container classes. Currently there are 4 possibilities:

1. Use the basic Objectivity ooVArray (extensible vector) and ooMap (dictionary) classes. The ooVArray class is not itself persistent-capable, but can be embedded into either transient or persistent-capable objects. RD45 supplies versions of ooVArray called HepVector (containing either transient or persistent-capable objects) and HepRefVector (containing references to persistent-capable objects) that have RogueWave-based versions in a fully transient environment and so can be used as part of a migration strategy.
2. Use the Objectivity-supplied persistent versions of the Rogue Wave Tools.h++ classes. For each transient class within Tools.h++ there are two classes within this library, both of which can contain persistent-capable objects. However, one is itself persistent-capable while the other is not and can therefore be embedded within other objects. Thus in principle a simple mapping from a transient Tools.h++ class (e.g. RWTPtrVector<T>) to the corresponding embeddable (RWTEmbRefVector<T>) or persistent-capable (RWODTRefVector<T>) class may be established. The problems with this approach is that the persistent Tools.h++ classes are based on the transient version 6, whereas the transient classes are based on version 7. Furthermore, the continued long-term support of the persistent classes by Objectivity is uncertain since their strategic direction is towards STL-based classes.
3. Use the new STL-based persistent classes that will be available with Objectivity Version 5. These will be available in beta-form on some platforms in July, but will not be shipped on all BABAR platforms until early 1998.
4. Write our own container classes based upon the basic Objectivity classes, but mapping closer to the interfaces of the existing transient Tools.h++ classes.

Migration for the Persistent Strategy

1. Decide on persistent container classes

This is the major decision that has to be made for this strategy and has a major impact on the recoding and re-design of the existing reconstruction packages.

2. Create a persistent-capable base class

• Make the AbsEvtObj class persistent-capable. Create a file AbsEvent/AbsEvtObj.ddl, based on AbsEvtObj.hh. This will cause all classes that inherit from it to be persistent-capable.

class AbsEvtObj : public d_Persistent_Object {
 };

3. Modify each class that is to be made persistent-capable

• Create file A.ddl, deriving it from file A.hh. Ensure that class A inherits from AbsEvtObj by the following lines:

#include "AbsEvent/AbsEvtObj.hh"
    [...]
    class A : public AbsEvtObj {
    [...]

• For each persistent-capable class B that is contained in A (theB say), replace the occurrence of B by BdbRef(B), and replace all "." operations on theB (e.g. theB.aFunction( )) by -> operations (e.g. theB->aFunction( )). This also requires that any forward declaration of B is replaced by inclusion of the header file for B. [It might be possible to #include "PkgB/B_ref.hh" instead of #include "PkgB/B.hh". This would reduce the dependency problem]. Finally, the instance of B must be created by use of the BdbNew(B) macro, presumably in the constructor for A and be deleted in the destructor for A using BdbDelete(A). For example:

class B;
    class A : public AbsEvtObj {
        [...]
        B theB;
        [...]
    };

    is replaced by:

    #include "PkgB/B.hh"
    [...]
    class A; public AbsEvtObj {
        [...]
        BdbRef(B) theB;
        [...]
    };

• Convert all data members of A that are simple C data types to ODMG machine-independent data types:

d_Char      8-bit ASCII
    d_Octet     8-bit byte
    d_Short     Signed 16-bit integer
    d_UShort    Unsigned 16-bit integer
    d_Long      Signed 32-bit integer
    d_ULong     Unsigned 32-bit integer
    d_Boolean   Boolean
    d_Float     32-bit float
    d_Double    64-bit double

4. Create Proxy classes

The role of the proxy classes in this strategy is to:

1. Locate the persistent information within the persistent event.
2. Insert navigational information into the persistent event so that the information may later be accessed.

In general there needs to be a proxy for each collection object that can be accessed from the event. Thus there might be a proxy for the list of tracks within the evbent, but not necessarily for the list of "Hits on Track" if such lists are only accessed via the track list and not directly from the event itself.

The role of the ProxyDict event is to "flatten" the event structure and to aid in allowing new information to be added to the event within a minimum of recompilation. Unfortunately BdbRef(A) cannot directly be stored & retrieved from a ProxyDict and therefore the templated class BdbIfdRef<A> must be used to store and retrieve persistent information.

• When using Ifd<A>::put(), replace IfdDataProxyTemplate<A> proxies by IfdDataProxyTemplate<BdbBdbRef<A> >

  BdbRef(A) theA;
    BdbIfdRef<A>* theBdbIfdRef = new BdbIfdRef<A>( theA );
        IfdDataProxyTemplate<BdbIfdRef<A> >* theProxy =
            new IfdDataProxyTemplate<BdbIfdRef<A> >( theBdbIfdRef );
        Ifd<BdbIfdRef<A> >::put( anEvent, theProxy );

• When using Ifd<A>::get(), replace A*, by BdbIfdRef<A>*

BdbRef(A) theA;
    BdbIfdRef<A>* theBdbIfdRef = Ifd<BdbIfdRef<A> >::get( theDict );
        if ( NULL != theBdbIfdRef ) {
            theA = *theBdbIfdRef;
        }

 [we're still hoping to make this simpler]

5. Modify each class that references a persistent-capable class

• Find all occurrences of A* and replace them by BdbRef(A).
• Find all forward declarations of A & replace them by inclusion of the header file for class A.
• Find all embedded instances of class A and replace them by BdbRef(A). Replace all "." operators by "->" operators. Create the instance via BdbNew(A) and delete it via BdbDelete(A).
• Find all occurrences of "new A" and replace them by BdbNew(A).
• Find all occurrences of "delete A" and replace them by BdbDelete(A).

Migration for the Transient Strategy

1. Decide on the mapping from transient Tools.h++ classes to persistent containers

This strategy is not so heavily dependent on the choice of the persistent container classes. Life is certainly made easier if the set closely maps to the transient Tools.h++ classes, but it is possible to map to a simpler set of persistent container classes. For example, a transient sorted doubly-linked list can be mapped to an unsorted vector (ooVArray) by maintaining the ordering of the insertion and extraction of the elements in the unsorted vector. Similarly, a Hash Table can be mapped to an ooMap or even to several ooVArrays containing the elements & their keys. The impact is in the complexity of the proxy classes.

Note that the proxy classes for this strategy are inherently more complex than those for the Persistent strategy since they have to perform the creation of the transient objects (in their faultHandler( )) or the persistent objects (in their storeHandler()) and check that these objects don't already exist.

2. Modify each class that is to be made "persistent-capable"

In this strategy, a class is made "persistent-capable" by pairing it with a persistent-capable sibling.

• Create a class AP, having DDL file AP.ddl

AP.ddl

    #include "RD45/Hep.h"
    class A;
    class BdbObjectCache;

    class AP : public d_Persistent_Object {
    public:
        BdbHintDeclare;
    public:
        AP( );
        AP( A* theTransient );
        virtual ~AP( ) {}

        A* transient( BdbObjectCache* theCache ) const;

    private:
        [data members]
    };

    AP.cc

    #include "PkgA/A.hh"
    #include "BdbEvent/BdbObjectCache.hh"

    BdbHintInit(AP);

    AP::AP( A* theA ) {
        [copy data members]
    }

    A*
    AP::transient( BdbObjectCache* theCache ) {
    A* theA = new A;
    [set data members of A from those of this.]

    // Couple the transient & persistent objects
    theA->_persistent = ooThis( );
    return theA;
}

• Modify the transient class A to include a persistent() function member and a reference to the persistent counterpart (AP). Add AP as a friend class.

A.hh

    #include "PkgA/AP.hh"

    class A {
    public:

        virtual BdbRef(AP) persistent( );

    private:
        friend class AP;
        BdbRef(AP) _persistent;
    };

    A.cc

    BdbRef(AP)
    A::persistent( ) {
        if ( ! _persistent.isValid( ) ) {
            _persistent = BdbNew(AP)( this );
        }
        return _persistent;
    }

• Replace all data members within class A that reference other "persistent-capable" classes B (B*) by BdbPtr<B, BP>. This acts as a smart-pointer, ensuring that the objects of class A and B are properly synchronized with their siblings AP & BP.

A.hh

    #include "BdbEvent/BdbPtr.hh"
    class B;

    class A {
    [...]
    private:
        BdbPtr<B, BP> _theB;
    };

3. Create Proxy classes

The role of the proxy classes in this strategy is to:

1. Locate the persistent information within the persistent event and convert it to its transient form.
2. Create the persistent information from the transient form and insert navigational information into the persistent event so that the information may later be accessed.

Thus these proxy classes are more complex than their equivalents in the Persistent strategy. However, the numbers of such proxies are identical. In general there needs to be a proxy for each collection object that can be accessed from the event. Thus there might be a proxy for the list of tracks within the event, but not necessarily for the list of "Hits on Track" if such lists are only accessed via the track list and not directly from the event itself. The proxy for the list of tracks can perform a dual role in this case.

The role of the ProxyDict event is also to "flatten" the event structure and to aid in allowing new information to be added to the event within a minimum of recompilation.

DB Home | BaBar Home | Computing | Reconstruction | Simulation | Search

DRQuarrie@LBL.Gov