Matlab and ChannelArchiver Data

As a result of exporting data from the WinBrowser or other ChannelArchiver tools, a Matlab command file is generated. When using the CGIExport web interface, you can download such a command file and save it to the local disk. Is should be saved under a name with the ".m" extension, for example "archive_data.m" so that Matlab recognizes it via the dot-m ending.
Within Matlab, change the current directory to where the this file was saved/created and execute it by typing "archive_data" or whatever name you chose for the data file. Now the exported data is available within Matlab. To be of any use, you have to understand how this data is formatted.

Matlab Command File

The file that's created is an ASCII command file. It is neither a data file nor a binary file in order to be most portable. The first few lines in this command file are actually comments, providing a brief description of the format. Matlab can read and execute this file format: Run Matlab, cd to where the file is and type the file name (without the .m ending). As a result, one structure per channel is available in Matlab. The channel name equals the structure name. You can now use Matlab to modify the data, plot it, ...

Structure Format

For each channel, a structure with these elements is created:

• t - time string
• v - value
• s - status
• d - date number
• l - length of data
• n - name

Example:

  fred.t(1)={'03/22/2000 17:02:28.700986000'};
  fred.v(1)=0.0718;
  fred.s(228)={''};
  ...
  fred.t(228)={'03/22/2000 17:10:10.231108000;'};
  fred.v(228)=nan;
  fred.s(228)={'Archive_Off'};
  ...
  fred.d=datenum(char(fred.t));
  fred.l=size(fred.v,2);
  fred.n='fred';
  

The t field contains the full time information from the archive. This can rarely be used in Matlab as is, however, so it's converted into a Matlab date/time number (supported since at least MatLab V.4.2).
The l and n fields allow the definition of generic Matlab functions, see the following examples.

Matlab Examples

These are generic Matlab scripts that usually accept one or more archive data structures as arguments:

• Plot one channel over time
• Plot two channels over time, different axes
• X/Y Plot: plot X vs. Y over time
• Waterfall: plot several channels over time
• How to show raw data, fft and comments in the same plot

Design Decisions

There are several options for exporting data to Matlab. The described one offers:

  + Human readable
  + comments can be included
  + full archive information
  + no Matlab libraries required
  - slow
  

It is slow because Matlab has to interpret and execute the commands in the file. After that, you can use the Matlab "save" command to create a binary Matlab data file. This binary file usually loads faster, but it is specific to your version of Matlab, it might also be bigger, which is why the ChannelArchiver tools do not directly create such a binary data file.
In one example, the original command file had a size of 90kB and the corresponding binary data file was 174kB. Loading (=executing) the command file took about 5 seconds, while the binary data file loads in virtually no time. A Matlab profile of the command file reveals that 90% of the time were spent in datenum calls.

The most interesting part is the handling of time/date.
From what I understand, Chris Larrieu @ JLab implemented this format:

     start, end time  - string

     channel data: one array for the values,
                   another array with a normalized time number, meaning:
                   0.0  == start time
                   1.0  == end time.
  

This format, then written in binary format, offers:

  + speed
  + bigger?
  - not easily Human readable, especially the time stamp
    is now channel specific, relative to the start and end time
    chosen for this export run
  - no comments
  - might require Matlab libraries