How to get data from Striptool

HVS 5/28/04

It is possible to get data from striptool into a file which can be analyzed offline using matlab or some other data analysis tool. I'll describe here a technique by which this can be done, and discuss some ways to deal with a large number of complications which arise due to the nature of the way data gets saved.

The first thing to do is recognize that on your striptool graphics window there is an option to "dump the data". In the striptool graphics window use the right mouse button to pop up a menu bar. Scroll the mouse down to "dump data". A dialog box appears and in the "selection" field you can type in a file name and then "OK".

This action writes a (text) file to the unix file system. The location is (presently) specified by /nfs/mccfs0/u1/pepii/strip/data/.

To get to this file you can get onto one of the SLAC unix machines and move yourself to the directory specified above.

I prefer to deal with the information on the VMS MCC machine rather than on unix. One way to copy the data from unix to MCC is to use ftp (there are many other methods too). In the example I show, we'll presume that I have used striptool to dump data to a file called "junkdata". Here is how you can move the data to MCC:

From one of the SLAC unix machines move to the directory where the data file lives. Run ftp by typing "ftp". In ftp type "open" and at the (to) prompt give the name of the remote machine you wish to connect to (in this case MCC.SLAC.STANFORD.EDU). Then ftp will prompt for a username and password. Once connected use the ftp "put" command and ftp will prompt you for a local file name (in this case "junkdata") and a remote (destination) file name.

I have not found a way to have the file get written to any directory other than my own home directory. Similarly, I haven't had success extracting a file from a directory other than the current directory at the time ftp was started. I have found that being aware of this constraint is useful when using ftp.

The nature of the file that striptool dumps is that of a text file. This is both good news and bad news. The good news is that being a text file it is fairly easy to manipulate. The bad news is that the text file format is not immediately ready for use by matlab or other similar data analysis tools. Below is an example of what happens when I look at my "junkdata" file with my VMS text editor:

As you can see the file has a header of text information for the variables sampled. It contains vertical columns of data below. The line width is longer than the text editor can display, and the time column has date,hour...(down to a millisecond) in an arbitrary format. As shown only the time and the first sampled variable are visible.

At this point things look a little bit frustrating since most of the data is not even visible. One way around that is to use your text editor skills to "clean up" the file. For most things I would want to do with this data I am willing to throw away the timestamp information since I know that the data interval is very close to one second per update. Additionally, I am generally willing to sacrifice the header text with information about the variables since I generally know which data column contains which sampled variable independently since I am generally familiar with the data I want to save.

Having agreed to these sacrifices I can then use my handy text editor skills to "clean up" the file to more like this:

What I am doing is striping away all the information which does not fit the pattern of a set of pure columns of numeric values. In the example I show here there were only three things sampled by the striptool. If there were more variables sampled their information would appear in columns further off to the right. In this case I know that the three sampled devices are HER current, LER current, and PEP luminosity.

What is going on with the "???" values in some of the columns? I included this for a reason. When I started the striptool acquisition I typed in first one variable to sample (HER current was first). As soon as I did that the striptool started acquiring. It took me ~10 seconds to tell striptool to acquire LER current, and another ~10 seconds to tell it to acquire PEP luminosity. The "???" values correspond to time points at which this column's data had not been acquired. This is a rather common condition of striptool data, and you will want to deal with it.

The way I deal with it is to use the text editor to remove all data for any time samples for which this occurs. Remember; you want a clean pattern of pure vertical columns of numeric values. Once I have this, I can save the file. I'll call mine "junkdata.txt".

The reason I get so bent about the "clean pattern of pure vertical columns of numeric data" is that a text file of that form can be loaded easily into matlab. Observe the following:

After saving a file as described above I launch matlab from the same directory as where lives the cleaned up file. In matlab type "load junkdata.txt". This loads the contents of the file into a single matrix called junkdata, and this can be seen by using the matlab "whos" command after loading the data. The junkdata matrix has size 70x3 corresponding to three sampled variables and seventy points.

I can then plot the first column (HER current) of data (vs. it's index) using standard matlab plotting syntax which yields this picture:

Once you've got the data into matlab your analysis possiblities are nearly endless!