1. Chunking 2. MOOD design
2.1 Primary tables
2.2 Secondary tables
3. Use cases
3.1 Activate config
3.2 Analyze config
3.3 Find config by time
3.4 Add parm file
3.5 Create a config
4. Next steps
4.1 Refine db
4.2 Refine parm list
4.3 LATC issues
4.4 Storing parameter files
4.5 Closing the loop
moon cow

MOOT Report for Week of 12 August 2005

1. Configuration chunking

At the lowest level, a configuration consists of a large number of register settings and a smaller, but still formidable, collection of values input to elements of Flight Software. This representation of the information is inconvenient for everyone and should be avoided whenever possible, but there is no single right way to chunk the information. When writing to the registers it is natural to partition them according to the hardware path used to access them (leading to categories like Tracker front-end registers); operators wishing to configure the LAT for a particular application will tend to arrange the information hierarchically and will draw the boundaries in different places. Part of MOOT's job is to translate between these points of view.

In the end, FSW choices about which information is bound together for upload must be respected, but the application side is free to subdivide further. Both points of view can be represented in an xml file. Here the hierarchical application view is explicitly represented by the xml hierarchy. The FSW chunking may be inferred by looking at the values of the out attributes. These values must refer to one of the <outfile> elements described at the end of the file.

2. MOOD design

Many details still need to be resolved, but, roughly speaking, MOOD will have primary tables and secondary tables. One primary table keeps track of the configuration history. The others describe the fundamental objects of interest: configs, files to be input to FSW, and (application view) parameters. The secondary tables keep track of relations between these objects. There may be additional supporting tables of other kinds, to be determined during implementation.

2.1 Primary tables

2.1.1 History

A row is added to History each time a request to load a configuration is made. The row is last modified when a newer configuration is successfully loaded. Rows are never deleted from this table.

History fields
Field name Explanation, typical contents
history_keyPrimary key, auto_increment; unique identifier for the row
config_fkForeign key, referencing an entry in Configs.
request_timeTimestamp for request to activate
ack_timeTimestamp when notification was received of success or failure of load
active_startSame as ack_time for successfully loaded configurations; else null.
active_endTimestamp when config was overwritten by the next one
activate_statusOne of a list like 'requested', 'active', 'superseded', 'failed',etc.
requestorName (or username?) of person requesting activation.

2.1.2 Configs

A row is added to Configs when someone chooses to define a new configuration. Rows are never deleted from this table. At the end of a successful creation all required FSW files have been entered into the FMX Logical table.

Configs fields
Field name Explanation, typical contents
config_keyPrimary key, auto_increment; unique identifier for the row
modeMajor mode, one of an enumerated set (data, calibration,..)
nameConvenient short name for interactive use
descriptionLonger description of what the configuration is supposed to do.
creatorWho asked to create it
request_timeTime of start of process to create config
creation_timeTime of creation (or of end of attempt)
statusAt what stage of processing is this config? E.g., 'OK', 'abort', 'in_progress', ...
instrumentWhich LAT-like instrument does it refer to?

2.1.3 FSW input files

There will be three FSW services which transform configuration source information to binary: LATC, LCI and CDM. In MOOT's use of LCI and CDM, expect that the transformation from input files to output files will be one-to-one. LATC is more complicated. It can accept inputs which specify only a few registers or a very large number or something inbetween. However LATC has a fixed collection of about 15 possible output files, determined largely by register access boundaries (e.g. one for AEM, one for GEM, one for ARC, and so forth). As much as possible, will attempt to construct input files for LATC so that a single input file produces just one of the LATC standard outputs (perhaps not counting a special output file known as default) and no two inputs produce the same type of output.

FSW_inputs fields
Field name Explanation, typical contents
FSW_input_keyPrimary key, auto_increment; unique identifier for the row
sourcereference which may be used to find the file; e.g. file path or path and version in CVS archive
source_fmtinformation for reader of source; e.g., might have value 'xml'
class_fkDenotes type of file; e.g., CAL gains or GEM input to LATC. Note this value is not unique within the table, but it refers to a primary key in the FSW_class secondary table.
statusKeeps track of processing stages. May make row before file has actually been created. Once created, it probably gets committed to CMX. Finally, it is processed by FMX and put into the Logical table.
FSW_idIf the input file is transformed to binary successfully and inserted into the FMX Logical table, it will it will have an id assigned to it.
creatorWho asked to create it
creation_timeWhen the row was created

2.1.4 Parameters

Parameters are in application space rather than hardware space. The fine-granularity hardware configuration information is partitioned one way by these parameters and another way by the files managed by FMX. The Parameters table will include the following fields (and probably several more):

Parameters fields
Field name Explanation, typical contents
parameter_keyPrimary key, auto_increment; unique identifier for the row
class_fkRefers to primary key of row in Parameter_class table, which in turn can be translated to some name like tkr_thresholds. Note this value is not unique within the table.
qualityOne of an enumerated set, e.g. {'PROD', 'DEV', 'TEST', 'SUPSED', 'INVALID'}
descriptionCould just be "default" if it's the standard set of values for the parameter; otherwise describe purpose of this set.
sourceReference to a file containing the values belonging to this parameter class instance
source_fmtInformation which may be useful to readers of the file referenced by source
mode_listOne or more modes (e.g., 'Physics', 'CI') for which this file is appropriate
creatorWho asked to create it
creation_timeWhen was this row was created?
instrumentWhich LAT-like instrument does it refer to?
vstartStart of time interval during which the data in this parameter set is valid or applicable
vendEnd of time interval during which the data in this parameter set is valid or applicable
flavorNormal value is something like 'default' or 'vanilla'.

Wildcard which can be used to identify a special set of values. Probably should not be allowed to have anything other than the normal value for standard data-taking or calibration configurations.

2.2 Secondary tables

2.2.1 Linking tables

Some secondary tables link together primary tables, using keys. Will have one linking Configs to FSW_inputs and another linking FSW_inputs to Parameters. Probably will need a third to keep track of LATC master files and the other LATC files belonging to them. In all cases it's a many-to-one relationship.

Configs_to_FSW fields
Field name Explanation, typical contents
config_fkReference to element in Configs
FSW_fkReference to element in FSW_inputs

FSW_to_Parameters fields
Field name Explanation, typical contents
FSW_fkReference to element in FSW_inputs
parm_fkReference to element in Parameters

Master_to_LATC fields
Field name Explanation, typical contents
master_fkReference to element in FSW_inputs of type LATC master
latc_fkReference to other element in FSW_inputs used as input to creation of the master file.

2.2.2 Class-defining tables

One secondary table keeps track of the kind of processing needed for each FSW input:

FSW_class fields
Field name Explanation, typical contents
class_keyPrimary key, auto_increment; unique identifier for the row
from_precursorsreference to program or script which turns parameter input into the source for FSW. Goal is to automate the transformation from parameter files to FSW input file. May require additional fields in this table.
to_binaryIndicates FSW service to invoke to create binary

Yet another secondary table will keep track of information specific to each parameter class. The parameter classes correspond to <parm> elements in the xml delegate description. May require additional fields in this table.

Parameter_class fields
Field name Explanation, typical contents
class_keyPrimary key, auto_increment; unique identifier for the row
nameMore pleasant way to refer to the row for humans, e.g something like "Housekeeping schedule", "ACD defaults", "ACD exception masks", etc. These names will be unique within the table.
descriptionAny other information about the class for human consumption.

2.3 Filling the tables

The secondary tables are filled "behind the scenes": the two class-defining tables are essentially configuration information for MOOT which should be entered before start-up and only very rarely changed, most typically by adding a new row to define a new class in one or the other table. The linking tables will be filled automatically in the process of creating a configuration or new FSW_input, as will (some day, though perhaps not to start) FSW_inputs.

For now all entries to Parameters will be made by hand, though, since some of them will just be output products from other processes, it should be possible to automate somewhat. In the meantime, it may be possible without very much work to enhance rdbGUI to help with manual entry. There will be a lot of rows in this table, so some sort of aid will be a practical necessity.

New rows in Configs will be made upon explicit request, presumably as part of the invocation of some script or program which will do the actual creation, typically involving as well the creation of new rows in one or more of the linking tables. Similarly, a new row will be added to History upon operator request to activate a config; filling the fields in the row will be a relatively minor part of the process.

3. Use cases

3.1 Activate pre-existing config

Pre-existing means there are entries in Configs, FSW_inputs, Parameters, Config_FSW and FSW_Parm describing the config and FSW services have been invoked to create binaries (entries have been made in the FMX Logical table).

  1. Query FMX to determine if all binaries belonging to the config have been uploaded.
  2. If not,
    1. report this back to LICOS (or similar interface)
    2. LICOS invokes MOOT service to identify binaries needing to be uploaded
    3. LICOS reformats into packets, etc.; executes upload
    4. When acknowledge comes back, LICOS informs FMX that files have been uploaded, causing FMX to make entries in its Physical table. Alternatively, MOOT provides a wrapper if we want to avoid direct contact between LICOS and FMX.
  3. LICOS initiates telecommands to select all files belonging to the config
  4. When acknowledge comes back, MOOT service should be called to update History. New row should be marked as active. Previously active config should be marked inactive.

3.2 Analyze config

Given something in Config table, can search table relating it to FSW files for list of files. For each file, search table relating file to parameters to retrieve all relevant rows in Parameters table.

3.3 Find config by time

Search History.

3.4 Add parm file

The file has to be stored in some appropriate place (CVS? dedicated directory accessible to MOOT?) and registered in the Parameters table.

3.5 Create a config

For the time being, this is an entirely offline process (that is, not done just prior to upload and not automated), though MOOT may provide some help with some phases of it.

  1. Register all parameter files that will be needed for the config.
  2. Select mode. This determines collection of FSW inputs, hence also parameters, required. MOOT can make a new row in Configs, marking status as in progress.
  3. For each of the FSW input files, operator selects collection of parameter files which are its precursors. See if this collection has already been used to produce the FSW input. If not, invoke process to build a new FSW input file and update FSW_Inputs and FSW_Parm accordingly. In the short term the "process" invoked above is likely to consist of someone copying a file or editing by hand.
  4. If any of the LATC inputs is new, will have to make a new LATC master configuration file (and MOOT has to keep track of it somewhere).
  5. Make entries in Config_FSW to record collection of FSW inputs for this config
  6. Mark config as complete

4. Next steps

4.1 Refine database design

Design will evolve during implementation phase.

4.2 Refine parm list, FSW input file list

4.3 LATC issues

There is another level of processing needed for the collection of LATC files belonging to a particular config which hasn't been fully accounted for here, but it should be possible to accommodate it with relatively modest enhancements. The relational table LATC_master should help.

4.4 Storing parameter files

Need a well-defined secure place for them.

4.5 Closing the loop

Much of the process described in section 3.5 Create a config above needs to be automated. Once piece of this will be to add new parm files derived from data analysis (e.g., noisy strips, CAL pedestals and gains).


Initial draft: 11 August 2005
Last revised: Friday, 09-Sep-2005 11:32:06 PDT
J. Bogart