INDEX
*  Sequential Record Processing in SPIRES: Global FOR
+  Introduction
+1  Introduction to this Manual
+2  Notation Conventions
1  An Introduction to Global FOR
2  The "FOR class" Commands
2.1  The "FOR class" Command and the Various Classes
2.2  The FOR SUBFILE (FOR SUB) Command
2.3  The FOR TREE (FOR TRE) Command
2.4  The FOR DEFQ (FOR DEF) Command
2.5  The FOR ADDS (FOR ADD) Command
2.6  The FOR UPDATES (FOR UPD) Command
2.7  The FOR REMOVES (FOR REM) Command
2.8  The FOR TRANSACTIONS (FOR TRA) Command
2.9  The FOR RESULT (FOR RES) Command
2.10  The FOR STACK (FOR STA) Command
2.10a  The FOR STORED Command
2.11  The FOR SET Command
2.12  The FOR RESIDUAL DATA (FOR RES DAT) Command
2.13  The FOR GOAL (FOR GOA) Command
2.14  The FOR INDEX command
2.15  The FOR PATH Command
2.16  The FOR STORED Command
2.17  The FOR LOAD Command
2.18  The FOR * Command
2.19  The FOR ** Command
3  The VIA Clause and Access Classes
3.1  The VIA RESULT (VIA RES) Clause
3.2  The VIA STACK (VIA STA) Clause
3.3  The "VIA SET setname" Clause
3.4  The VIA TRANSACTIONS (VIA TRA) Clause
3.5  The VIA RESIDUAL DATA Clause
3.6  The VIA TREE (VIA TRE) Clause
3.7  The "VIA INDEX index-name" Clause
3.8  The "VIA PATH pathname" Clause
3.9  The FOR TRANSACTIONS VIA "Access-Class" Commands
4  The WHERE Clause
4.0.1  (WHERE clause limits)
4.1  The Simple WHERE Clause
4.1.1  (The LENGTH and OCCURS Options)
4.1.2  (The NOT Option)
4.1.3  (Using Structures in a WHERE Clause)
4.2  The Compound WHERE Clause
4.2.1  Parentheses and Compound WHERE Clauses
4.2.2  The NOT Option and Compound WHERE Clauses
4.2.3  The Order of Compound WHERE Clauses
4.2.4  Specifying the Same Element Twice in a Compound WHERE Clause
4.3  Processing Rules and WHERE Clauses
4.4  Special Cases Handled by WHERE Clauses
4.4.1  Specifying Same-Structure Processing
4.4.2  Binding Element Retrieval to the Same Structural Path
4.4.3  Specifying a Particular Element by "Path"
4.4.4  Inter-element Relations in WHERE Clauses
4.4.4.1  (Details on Inter-element Relations)
5  Record-Processing Commands Available in Global FOR Mode
5.1  Command Processing in Global FOR
5.2  Command Options in Global FOR
5.2.1  The FIRST (FIR) Option
5.2.2  The "*" Option
5.2.3  The NEXT (NEX) Option
5.2.4  The "n" Options
5.2.5  The REST (RES) Option
5.2.6  The LAST (LAS) Option
5.2.7  The ALL Option
5.3  The DISPLAY (DIS) Command under Global FOR
5.4  The TRANSFER (TRA) and UPDATE (UPD) Commands under Global FOR
5.5  The MERGE (MER) Command in Global FOR
5.6  The REMOVE (REM) Command in Global FOR
5.7  The DEQUEUE (DEQ) Command in Global FOR
5.7a  The UNQUEUE (UNQ) Command in Global FOR
5.8  The STACK (STA) Command in Global FOR
5.8a  The UNSTACK (UNS) Command in Global FOR
5.9  The SHOW KEYS (SHO KEY) Command
5.10  The SKIP (SKI) Command
5.11  The REFERENCE (REF) Command in Global FOR
5.12  The DEFINE SET (DEF SET) and GENERATE SET (GEN SET) Commands
5.13  The GENERATE LOAD (GEN LOA) Command
6  Other Global FOR Commands
6.1  The SHOW LEVELS (SHO LEV) Command
6.2  The SET SCAN (SET SCA) Commands
6.2.1  SET SCAN FORWARD (SET SCA FOR) and SET SCAN BACKWARD (SET SCA BAC)
6.2.2  SET SCAN START (SET SCA STA)
6.2.3  SET SCAN STOP (SET SCA STO)
6.2.4  SET SCAN PREFIX (SET SCA PRE)
6.2.5  SET SCAN LIMIT (SET SCA LIM)
6.2.6  SET SCAN SKIP (SET SCA SKI)
6.2.7  SET SCAN INCREMENT (SET SCA INC)
7  Leaving Global FOR Mode
7.1  The ENDFOR (ENDF) Command
7.2  Leaving Global FOR Mode Automatically
7.3  The END Clause: Special Command Execution for Protocols
8  Global FOR as a Mode of Operation
8.1  Global FOR as a Mode of SPIRES
8.2  System Variables Associated with Global FOR
8.3  Global FOR as a Searching Tool
8.3.1  (Comparison of Sequential Searching Techniques)
8.4  Global FOR as an Updating Tool
8.5  Global FOR as a Data Base Management Tool
8.6  Some Common Situations Requiring the Use of Global FOR
8.7  How SPIRES Accesses Records in Global FOR Mode
:29  SPIRES Documentation

*  Sequential Record Processing in SPIRES: Global FOR

******************************************************************
*                                                                *
*                     Stanford Data Center                       *
*                     Stanford University                        *
*                     Stanford, Ca.   94305                      *
*                                                                *
*       (c)Copyright 1994 by the Board of Trustees of the        *
*               Leland Stanford Junior University                *
*                      All rights reserved                       *
*            Printed in the United States of America             *
*                                                                *
******************************************************************

        SPIRES (TM) is a trademark of Stanford University.

+  Introduction

+1  Introduction to this Manual

Users working with SPIRES subfiles often need to access records in ways not covered by the standard searching and updating facilities. For instance, you may need to locate records using some record criterion that is not indexed. Or you may need to update all the records having a certain non-indexed element value but do not necessarily know the keys of the records. Or you may just want to examine all the records that have been added or updated in the subfile since the file was last processed in SPIBILD.

One common link between these three situations is that you do not know or have the keys to the records you wish to handle. If you did, you could use the standard "DISPLAY key", "TRANSFER key" and "UPDATE key" commands to work with these records one at a time (though that method would be very tedious if many records were involved). These situations can all be handled with Global FOR procedures, which allow you to work with groups of records based on criteria other than their keys or those element values that appear in indexes.

Global FOR allows you to sequentially process records in a particular class, whether it be the class of records added to the subfile today, the class of records in the current search result, the class of records that have been removed today, and so on. Each of the classes can be further subsetted, depending on values in the records themselves. Many commands can be used to work with these records, the primary ones being the familiar DISPLAY and TRANSFER commands, used with new options in Global FOR.

This manual will teach you about using Global FOR in your SPIRES activity. Global FOR is discussed briefly in the manual "SPIRES Searching and Updating", sections B.5.3.3, D.3 and D.4. That material is sufficient for many users; this manual is a reference of the complete capabilities, especially written for advanced users, file owners and programmers. Because you should understand the material in that manual to understand the information here, reading that manual is a prerequisite. From time to time, there are sections here that are more advanced (say, a section requiring some knowledge of SPIRES protocols) or generally more useful to file owners but these sections have been marked accordingly. Remember that if there is a term or subject you do not understand, you can issue the EXPLAIN command in SPIRES for information.

Many of the concepts and tools used to sequentially process records under Global FOR are also used in the sequential processing of parts of a record. Partial FOR processing is the subject of Chapter 12 of the manual "SPIRES Technical Notes".

Two points of terminology need to be explained. First, when the term "process" is used in this manual (most often appearing with the word "records"), it means "working with" records using the SPIRES commands available in Global FOR; when you process records, you may be displaying, transferring, removing or in some other way working with the records. On the other hand, the phrase "processing a file in SPIBILD" occasionally appears, referring to that procedure where records in the deferred queue are moved into the goal record tree and the indexes are updated accordingly. Thus, processing records in Global FOR and processing a file in SPIBILD are two different activities. In this manual, "processing" refers to records, not files, unless stated otherwise explicitly.

Related to the previous point, the second point of terminology concerns a shortcut used throughout SPIRES documentation. Most files are processed in SPIBILD every night automatically; the only records in the deferred queue on any given day are those placed there that day. Other files are processed more often (say, twice a day) or less often (once a month) by their owners. In all cases, for any file, the records in the deferred queue have been put there "since the file was last processed". In SPIRES documentation, the word "today" is often substituted for the quoted phrase in the last sentence. So if a discussion mentions "processing all the records added today", it is talking about the records added since the file was last processed, which may or may not mean only records added today.

+2  Notation Conventions

In SPIRES manuals, examples of sessions are shown with prompts and messages from SPIRES as they actually appear on the terminal, usually in uppercase; commands you type are shown in lowercase. You may use upper or lower case interchangeably when you are actually using SPIRES. For example:

Here SPIRES types "-OK to clear?" and you type "ok".

In formal command syntax descriptions, uppercase letters denote command verbs or other command elements to be entered exactly as shown; a value for lowercase terms and characters must be supplied by you. For example:

To use this particular command, you type the command verb "SELECT" with the name of the desired subfile, for instance, "Restaurant":

where "->" is the prompt from SPIRES.

Brackets ([]) denote optional elements. Braces ({ }) indicate that you must specify one (and only one) of the alternatives within the braces. Within the braces or brackets, a vertical line (|) separates possible choices. Neither brackets nor braces are to be typed as part of the command. For example:

could be entered as

depending on whether you want the CLEAR option. Here is another example, using braces:

This command must be entered either as

where you supply the "record-key-value," or as

with the choice again governed by the desired result.

Cross references to other sections of the manual are denoted by brackets, as in "[See 4.]", which refers you to chapter 4 for additional information.

1  An Introduction to Global FOR

To use commands in Global FOR, you must first enter the Global FOR environment by issuing a "FOR class" command, where "class" specifies a group of records to be processed sequentially. There are about a dozen main classes, such as the class of added records or the class of records in a search result, and such classes can be further subsetted in order to find records in the class fitting some particular criterion.

Taking one of the sample classes above, suppose you want to work with all the records in a subfile that have been added today, that is, since the file was last processed. After selecting the subfile, you would issue the command

"Adds" is a Global FOR class -- "FOR ADDS" tells SPIRES that you want to work with the group of records that have been added recently and which are now in the deferred queue, that area housing updates to the file, including added and updated records as well as removal requests. Most Global FOR commands issued after this "FOR class" command will affect a record or records in this class. The "FOR class" command itself does not access any records; the real work is done by record processing commands issued under Global FOR, as shown below.

When you enter the Global FOR mode, the SPIRES prompt changes from "->" to "+>". The plus sign ("+") indicates that you are in Global FOR mode and reminds you that some SPIRES commands act differently -- these commands are the Global FOR record processing commands discussed later. [See 5.] However, most other SPIRES commands are unaffected and can still be issued, as can any other system commands, such as WYLBUR or MILTEN commands.

Now that you have told SPIRES which records you want, you can begin to sequentially process them. For instance, you can DISPLAY the records, or TRANSFER and UPDATE them, or REMOVE them, DEQUEUE them, STACK them, or use some combination of these commands. Several other commands are also available for processing records in Global FOR. Many of these commands, when used outside of Global FOR, require the key value of the record, e.g., "DEQUEUE 191" or "TRANSFER GA.JNK". Under Global FOR, you will not specify the key at all, because you have told SPIRES in the "FOR class" command exactly which records you want to process. However, in Global FOR, there are other new options available for those commands, some of which are demonstrated below.

You now want to process the added records in the subfile:

The ENDFOR command tells SPIRES to remove you from the Global FOR mode. SPIRES may automatically eject you from Global FOR: suppose that instead of issuing the ENDFOR command after the "SHOW KEYS REST" you had done the following:

Because your previous command had processed all the "REST" of the records in the class, there were no more to process when the DISPLAY command was issued. SPIRES tells you that there are no more records to process by removing you automatically from Global FOR. You can start over again with another "FOR class" command, or simply continue with other commands.

The remainder of this document discusses each aspect of Global FOR in detail. The "FOR class" commands, along with various options, are discussed in the next three chapters. The Global FOR commands that process records, such as DISPLAY, SHOW KEYS and TRANSFER, are explained in chapter 5. Two other Global FOR commands, SET SCAN and SHOW LEVELS, are explained in chapter 6. Returning from the Global FOR environment, either by an ENDFOR command or an automatic ejection, is discussed in chapter 7. Further general information about using Global FOR is included in the last chapter, chapter 8.

2  The "FOR class" Commands

Outside of Global FOR, most SPIRES commands used to process records of a subfile are key-dependent; that is, you must know the record's key value to process it. For instance, the MERGE command requires the key value of the record you wish to update. The beauty of Global FOR is that you do not need to know the key values of the records you want to process. Instead, the "FOR class" command defines a group of records to be processed by subsequent commands under Global FOR, and SPIRES determines the record's key values for you. All Global FOR processing begins with a "FOR class" command.

A "FOR class" command can define a very large, general set of records (such as FOR ADDS, which accesses all records in the subfile's deferred queue that have been added) or a smaller subset of records from a larger set that fit very specific criteria. (You can, for instance, examine only added records that have more than one occurrence of a telephone number element.) WHERE clauses, VIA clauses, and SET SCAN commands can all be used to subset a "FOR class" command.

This chapter will discuss the "FOR class" command without WHERE or VIA clauses, explaining each of the classes that can be specified in the command. Chapter 3 will discuss the VIA clause, which provides an alternate source for record keys. The WHERE clause, which is a very powerful option for modifying the class of records, is discussed in Chapter 4. The SET SCAN commands, which normally restrict the class of records to be processed, are issued after a "FOR class" command, and will not be discussed until Chapter 6.

2.1  The "FOR class" Command and the Various Classes

A "FOR class" command can be very simple or very complex, depending on whether or not VIA or WHERE clauses are used. Here is the general syntax of the FOR command:

There are six retrieval classes, ten access classes, and two special classes, thus making a total of eighteen different values for "class" in the "FOR class" command. The first sixteen are known as the main classes. Only the first five access classes are valid in the VIA clause, but all of them may also be specified as "class". [See Chapter 3 for an explanation of the VIA clause, and Chapter 4 for a discussion of the WHERE clause.]

Here are the different values for "class":

Retrieval Classes:

Access Classes:

Special Classes:

Each of these classes is discussed completely in the rest of this chapter.

2.2  The FOR SUBFILE (FOR SUB) Command

When records are added or updated in a subfile, SPIRES places them in the deferred queue, where they stay until the file is next processed, usually sometime overnight. During processing, the records in the deferred queue are moved into the goal record data set, or "tree", where the goal records already in the subfile are kept. Because the FOR SUBFILE command prepares SPIRES to examine records in both the tree and the deferred queue, it allows you to sequentially process all records currently in the subfile in their most recent form. Thus, if a record has not been updated since the file was last processed, it will come from the tree; if it has been updated, it will come from the deferred queue; if it has been added, it will come from the deferred queue; and if it has been removed, it will not be processed at all.

The FOR SUBFILE command has the following syntax:

Both the VIA and WHERE clauses are discussed later in this manual. [See Chapter 3 and Chapter 4.]

Records processed under FOR SUBFILE are processed in ascending key order, the order in which they are stored in the subfile. For example, if SPIRES assigns "slot numbers" as keys for the subfile (each added record receives the next available number in the series 1, 2, 3, etc.), then record 15 is stored in the tree (and in the deferred queue if they are there) before record 16, but after record 14. Similarly, if the keys of the records are dates stored in their special internal form by SPIRES, then "March 1980" would be stored before "April 1980" and after "February 1980" in the tree or deferred queue. Records processed under FOR SUBFILE are thus processed in ascending key order unless a VIA clause or a SET SCAN BACKWARD command is in effect. [See 3, 6.2.1.]

For example, with the SHOW KEYS command, which displays the key values of records in the class, you do the following processing:

After issuing the FOR SUBFILE command, you receive the Global FOR prompt ("+>") telling you that SPIRES is ready for any Global FOR commands you want to issue. The SHOW KEY FIRST command, a Global FOR command, processes the first record in the class, whose key is "AA.CSB". Then SHOW KEY NEXT processes the next record in the class, whose key is "AC.JAM", and so forth. [See 5.9 for details on the SHOW KEYS command, used in many of the examples of this chapter.]

FOR SUBFILE should be used when you must guarantee that record updates are taken into account when records are examined in Global FOR. Since SPIRES is examining both the tree and the deferred queue, more processing occurs in FOR SUBFILE operations than in FOR TREE ones.

Because FOR SUBFILE will cause SPIRES to process all the records in the file, you might be processing thousands of records, if the subfile is a very large one. A WHERE clause, which causes SPIRES to examine each record for particular information, can increase the processing costs substantially. [See 4.] You can limit the subset of records examined by using a SET SCAN command. [See 6.]

2.3  The FOR TREE (FOR TRE) Command

Under FOR SUBFILE, SPIRES examines both the deferred queue and the tree simultaneously, processing the latest copy of all records in the subfile. On the other hand, the FOR TREE command prepares SPIRES to sequentially process records in the subfile without looking at records in the deferred queue. No added records will be processed, nor will the updated copies of any records; any records that have been removed since the file was last processed will still be in the tree, so they will be processed by commands under FOR TREE.

In the tree, records are stored in ascending key order, so records processed under FOR TREE are processed in that order unless a VIA clause or a SET SCAN BACKWARD command is in effect. [See 3, 6.2.1.]

The syntax of the FOR TREE command is

The VIA and WHERE clauses are discussed later in this manual. [See 3, 4.]

Here you are examining the keys of records in the tree beginning with the first one. If you compare this example to the example discussed under FOR SUBFILE [See 2.2.] you will notice that the first key shown under FOR SUBFILE was "AA.CSB", which is the second key here. Assuming that these two examples were created the same day, that indicates that the record whose key is "AA.AJM", which is in the tree, has been removed since the file was last processed and thus is not processed under FOR SUBFILE. Since the removed record remains in the tree until overnight processing, it is processed under the FOR TREE command.

The fact that FOR TREE processing does not access the deferred queue is not a disadvantage, since other Global FOR commands can. It is very useful that it does not, in fact. Occasionally you may need to examine the old version of a record that has been updated or removed since the file was processed. If the FOR TREE command did not exist, you could only access the tree copy by dequeuing the update or removal request, which would not be very convenient. [See 3.2, which describes a procedure for processing the old version of a record.]

If you do need to use the latest copy of records in the subfile, taking added, updated and removed records into account, you can use FOR SUBFILE. [See 2.2.] However, processing records under FOR SUBFILE is slightly more expensive than under FOR TREE and is unnecessary if there are no records in the deferred queue. Generally, unless many records have been added, updated or removed, FOR TREE will suffice. If you frequently process all or a number of records in a subfile for report generation or other reasons, you should coordinate that activity with the processing of the file in SPIBILD so that you can take advantage of FOR TREE processing.

Because FOR TREE means that you will be processing the entire goal record data set, then if the subfile is a very large one, you might be processing thousands of records. That can be especially expensive if you have a WHERE clause on the FOR TREE command, causing SPIRES to examine each record for some particular criteria. [See 4.] You may be able to use the SET SCAN command to subset the class of records being processed at less cost than a WHERE clause. [See 6.]

If the records you are processing under FOR TREE retrieve data from records in other subfiles or record-types (through phantom elements or subgoal processing), SPIRES will examine deferred queues for the latest copy of those records. In other words, FOR TREE means "tree-only" processing for the goal records of the selected subfile, not any subgoal records.

2.4  The FOR DEFQ (FOR DEF) Command

With the FOR DEFQ command, you can sequentially process all records that have been added or updated since the file was last processed. It is useful when you want to review the major updating activity of the day.

Here is the syntax of the FOR DEFQ command:

The VIA and WHERE clauses are discussed later in this manual. [See 3, 4.]

For example,

Note that the DISPLAY command here operates differently than outside of Global FOR. The ALL option, available under Global FOR, specifies that all records in the class should be processed. [See 5.2.7, 5.3.]

Although each record processed under FOR DEFQ is either an added or an updated record, SPIRES does not tell you which type each is. If it is important that you know, then FOR ADDS and FOR UPDATES can be used separately instead. [See 2.5, 2.6.]

Records are processed in key order under FOR DEFQ:

The records are not processed in the order in which they were modified or added but in the order in which they are stored in the deferred queue, which is ascending by key.

It may at first seem unusual that FOR DEFQ, which by name would seem to access all the record information in the deferred queue, actually accesses only the added and updated records. After all, the deferred queue also contains information about removed and merged records. As it happens, merged records are treated as updated ones in the deferred queue. Though you only have part of a record in your active file when you issue a MERGE command, SPIRES gets the latest version of the record, merges your new information into that copy, and puts the entire record into the deferred queue as an updated record. So actually the FOR DEFQ command accesses all added, updated and merged records.

Records being removed are treated quite differently, however. Unlike "adds" and "updates", where SPIRES processes an entire record, you specify only a key with the REMOVE command to cause a record to be removed. So for a removal request, the deferred queue has only the record's key. "Removes" are thus quite different from adds or updates and so are handled separately. [See 2.7.]

2.5  The FOR ADDS (FOR ADD) Command

The records accessed by a FOR ADDS command are a subset of the records accessed by the FOR DEFQ command -- specifically, the records that have been added to the subfile since the file was last processed.

The syntax of the command is:

The VIA and WHERE clauses are discussed in later sections of this manual. [See 3, 4.]

The following example demonstrates the use of the FOR ADDS command:

After placing copies of all of the records into the active file, the user has dequeued all the added records. The records are still retrievable, however, using the FOR TRANSACTIONS command. [See 2.8.]

What about records that are added and then updated later that day? They are still considered added records to SPIRES. Whether a record is considered an add or an update depends on its relationship to the tree, not on what commands you have issued to place it in the deferred queue. A record in the deferred queue with a key matching that of a record in the tree is an update; if there is no tree version, it is an add.

Records that are added and then dequeued or removed can not be accessed under the FOR ADDS command, since they are no longer being added to the subfile, nor under the FOR REMOVES command. They can be accessed through the FOR TRANSACTIONS command. [See 2.7, the FOR REMOVES command, 2.8, the FOR TRANSACTIONS command.]

Remember that the records will be processed by key order, not by the order in which they were added. The two orders would presumably be the same if the subfile were one in which SPIRES assigned the keys numerically (a slot subfile).

2.6  The FOR UPDATES (FOR UPD) Command

Like the FOR ADDS command, the FOR UPDATES command accesses a subset of records accessible through the FOR DEFQ command -- in this case, all the updated records.

Like other "FOR class" commands, the syntax for this command is:

The VIA and WHERE clauses are discussed in later chapters. [See 3, 4.]

An "update" is the latest copy from the deferred queue of a record that already exists in the tree. A record that is removed and added again later that day (with the same key as before) is also considered an updated record, not an added one or a removed one.

Records that have been added and then updated later in the day can be accessed by the FOR ADDS command, not the FOR UPDATES command. Records that have been updated and then either removed or dequeued will not be accessed by the FOR UPDATES command; FOR TREE can be used to access the original copy. The dequeued, updated copy or copies can be accessed with the FOR TRANSACTIONS command. [See 2.8.]

The records accessed by the FOR UPDATES command are processed in key value sequence (as shown in the example above), not the order in which the records were updated.

2.7  The FOR REMOVES (FOR REM) Command

Records in the tree that have been removed by a REMOVE command are treated quite differently than adds or updates. While at least one entire record resides in the deferred queue for each record being added or updated, only the record key is in the deferred queue for a removal request. Since only limited information is available, those Global FOR commands that work with whole records cannot be used under FOR REMOVES. The REMOVES class contains the keys of only those records in the tree that have been removed. Keys of records that have been added and then removed that day are not included. Such records can be resurrected using the FOR TRANSACTIONS command. [See 2.8.]

The syntax of the FOR REMOVES command is:

VIA and WHERE clauses are discussed in detail elsewhere. [See 3, 4.]

The only Global FOR record processing commands that can be issued under FOR REMOVES are DEQUEUE, SHOW KEYS, SKIP and STACK. [See 5.] All other Global FOR commands require access to the record data itself; all that is available here under Global FOR is the record key. If you try to use a "forbidden" command, such as DISPLAY, under FOR REMOVES, you will be ejected from Global FOR mode, as shown above. There is no next record that can be displayed, so SPIRES ejects you from Global FOR mode.

WHERE clauses are allowed on a FOR REMOVES command; however, the only element they can refer to is the record key. For example, if the key of a record is an element called ID, and you want to know the keys of removed records that are higher than 50:

However, if the record also has a DATE-ADDED element, it could not be used in the WHERE clause:

There is a way to examine and process records being removed, which will be explained later. [See 3.2.] You might want to compare the SET SCAN commands with the WHERE clause in a case such as the previous example. [See 6.2.]

2.8  The FOR TRANSACTIONS (FOR TRA) Command

Suppose you have just added a record to a subfile and now decide to update it. You make changes, issue an UPDATE command and then realize you should not have made the changes. Can you access the original added record? The obvious answer would seem to be that you could use the FOR ADDS command. However, as you learned in the previous two sections, the FOR ADDS command would provide you with the latest copy of the added record, which is the updated copy that you do not want.

The command that is useful here is FOR TRANSACTIONS, which accesses all the different versions of added and updated records since the file was last processed.

The syntax of the FOR TRANSACTIONS command is:

A special group of VIA clauses are allowed on the FOR TRANSACTIONS command. [See 3.9.] The WHERE clause is discussed later. [See 4.]

Under FOR TRANSACTIONS, the records are generally processed in the order that the transactions were made, from oldest to newest, and not in key order as under most of the other "FOR class" commands. (Records may not be in exact chronological order if more than one user is updating the subfile at one time.) However, the FOR TRANSACTIONS VIA "access-class" do process the records in key order. [See 3.9.]

The example above shows that there are three different versions of record 19 and one of record 25 in the deferred queue. The file owner can see the sequence of subfile updating commands that created the records processed under FOR TRANSACTIONS by issuing the SHOW SUBFILE TRANSACTIONS command. It tells the file owner what time a record was processed, what account processed it, and so forth, so it is very handy to use with the FOR TRANSACTIONS command. It is possible that none of the three copies of record 19 is the latest version -- the record may have since been dequeued or removed, a possibility not shown under FOR TRANSACTIONS. However, if any version is the latest copy of a record, it will generally be the last version processed under FOR TRANSACTIONS.

That fact can lead to a problem. Suppose only one record has been updated in a subfile:

The example shows that record 19 was the only record in the transactions class when FOR TRANSACTIONS processing began. However, once record 19 was transferred and updated, the new version also joined the class of transactions, becoming the "next" record to be processed. Thus, each time you update a record in the class of transactions, there is one more record put into the class. Because of this, you should beware of processing the same records over and over under FOR TRANSACTIONS. [See 5.4 for details on the TRANSFER and UPDATE commands in Global FOR.]

The FOR TRANSACTIONS command is particularly useful when you have added records during the day and then accidentally dequeued, updated or removed them. Because you can easily track all the record modifications made during the day, you can use FOR TRANSACTIONS processing to create audit trails. It is also useful on the rare occasion when the deferred queue's tree has been damaged.

The FOR TRANSACTIONS command is actually a short form of the command FOR DEFQ VIA TRANSACTIONS. Since the DEFQ class does not contain the keys of removed records, they are not processed under FOR TRANSACTIONS. However, the keys of removed records later dequeued can be accessed using the FOR REMOVES VIA TRANSACTIONS command. [See 3.4.]

2.9  The FOR RESULT (FOR RES) Command

The FOR RESULT command lets you process records in the current search result. Its syntax is:

No VIA clause is allowed on the FOR RESULT command, which is actually a short form of the command FOR SUBFILE VIA RESULT. [See 3.1.] The WHERE clause is discussed in a later chapter. [See 4.]

Suppose you want to remove all records in the RESTAURANT subfile that were added before 1984 and that have not been updated since then:

The above method is much more efficient than another way, using a WHERE clause:

Although the second method took one less command than the first, it required considerably more computing resources. In the first example, searching an index (which is very efficient) saves SPIRES from having to examine each record in the tree individually. The difference is similar to the difference between using an index in a book to find certain information and looking page by page through the book to find the same information. Obviously it is more efficient to use the index first, if possible. Remember that in the FIND command in the first example, DATE-ADDED is an index name; in the WHERE clause in the second, it is an element name. Obviously, if there were no DATE-ADDED index, the index search could not be done.

The FOR RESULT command is useful not only as an updating tool but also as a searching tool. If you have a search result that you want to subset further using unindexed information, you can use a WHERE clause with the FOR RESULT command. Remember that the ALSO command can often be used in this situation more easily, but not, in many cases, as efficiently. [See 8.3.]

Under FOR RESULT, records are processed in the order in which they would appear if the TYPE command were issued, an order which may seem haphazard. If you want the records to be processed in a particular order (key order, for instance), you can use the SEQUENCE command to arrange the result. The stack created by the SEQUENCE command can then be processed under the FOR STACK command. [See 2.10.]

The FOR RESULT command always accesses the latest copy of each record in the result. Therefore, if a record in the result has been removed since the file was last processed, the record will not be processed under the FOR RESULT command.

It is not necessary that a result exist when you issue the FOR RESULT command; a search result can be established or changed under a FOR RESULT command. If you change the result, subsequent Global FOR commands will process the new result, starting from the "top" of the result.

A stored result can often be processed directly (without having to issue the "FIND @RESULT.name" command) by using the FOR SET command. [See 2.11.]

2.10  The FOR STACK (FOR STA) Command

When you want to sequentially process records in a stack, you use the FOR STACK command:

In the example, after creating a stack of sequenced records, you begin using the TRANSFER command [See 5.4.] so that you can update the records in the stack one at a time.

The syntax of the FOR STACK command is:

The WHERE clause is discussed later. [See 4.] No VIA clause is allowed on the FOR STACK command. [See 3.] The FOR STACK command operates similarly to the FOR RESULT command, except that a stack rather than a result will be processed. The stack does not have to exist prior to the FOR STACK command, but may be created (or changed) after the FOR STACK command is issued. If the stack is changed by creating or restoring a new stack, then subsequent Global FOR commands will begin processing the new stack from the "top", with one exception. The STACK command when issued while a FOR STACK command is in effect does begin creating a new stack, but Global FOR commands continue processing the original stack until that stack is exhausted or until an ENDFOR command terminates the Global FOR processing. (The TYPE or SEQUENCE commands will end Global FOR processing automatically in such cases, and process records in the new stack.) [See 5.8.]

The FOR STACK command is equivalent to FOR SUBFILE VIA STACK, meaning that all records processed will be in their latest version. [See 3.2.] A stored stack can be processed directly, without restoring it first, by using the FOR SET command. [See 2.11.]

2.10a  The FOR STORED Command

The syntax of the FOR STORED command is:

where "setname" is a stored result or stack.

This Global FOR command can only be used within SPIBILD. Refer to the document "File Management" or online, type EXPLAIN FOR STORED.

2.11  The FOR SET Command

When you use a generated set of records (which may have been sorted by the SPISORT program), you can access that set later with a FOR SET command:

where "setname" is the name of the ORVYL data set created by the DEFINE SET and GENERATE SET commands [See 5.12.] or is a stored result or stack. If the set is stored as an ORVYL data set under some account other than your own, then use the form "FOR SET ORV.gg.uuu.setname" where "gg.uuu" is the account number under which the set is stored. A WHERE clause can be appended to the FOR SET command but not a VIA clause. [See 3, 4.]

The DIRECT option tells SPIRES not to retrieve the goal record that a sort entry points to, but instead to use the data within the sort entry itself. You would use this option for processing a "direct set" previously created with the "DEFINE SET setname DIRECT" command. (Direct sets are a way of increasing the efficiency of large sorting applications, and of applications where the same set might be sorted multiple times for multiple reports. For more information see Chapter 1 of "SPIRES Technical Notes" or EXPLAIN DIRECT SET.)

The UNFILTERED option may be used to tell SPIRES not to use the "path information" in the set to filter out undesired element occurrences. [See the manual "SPIRES Technical Notes" for more details on the use of path information with the FOR SET command.]

The FOR SET command can be issued after the set has been defined and generated (and then usually sorted by the SPISORT program). Under FOR SET, records will normally be processed in their sorted order.

The "setname" can also be a stored result or stack, in either the form "RESULT.name" or "STACK.name" where name is the given name of the stored result or stack. (Again, the prefix "ORV.gg.uuu." can be added if you are using a result or stack stored under another account. If you are using a temporary result or stack, use the "TEMPORARY RESULT.name" or "TEMPORARY STACK.name" form.) Thus, you do not have to restore the stack or "find" the stored result again before issuing the "FOR class" command. The equivalent command in SPIBILD is: FOR STORED setname. [EXPLAIN FOR STORED COMMAND.]

Not all stored results can be used by the FOR SET command. Only a result stored from non-structured pointer groups is allowed. The file definition must be examined to determine whether pointer groups are structured or not, though the presence of sub-indexes and qualifiers, identified by the SHOW INDEXES command, are a good indication that they are.

Here is an example:

The FOR SET command is equivalent to FOR SUBFILE VIA SET, meaning that you will get the latest copies of records when you process them under this command. To access the tree copies rather than the latest ones in the deferred queue, you can issue the command "FOR TREE VIA SET setname". [See 3.3.]

Because stored stacks and results do not contain element path information, be sure that you do not try to process them under a FOR SET using a format requiring element path information. [See the manual "SPIRES Formats" for information on the PATH and NPATH options.]

2.12  The FOR RESIDUAL DATA (FOR RES DAT) Command

The FOR RESIDUAL DATA command is a file owner tool that is designed for use on rare occasions in order to facilitate recovery when there has been file damage. It can be used only if the records in the subfile have been REMOVED to a residual data set. [See "SPIRES File Definition".] Generally it is not useful for regular file activity. It provides an alternate way to retrieve goal records without going through the tree.

The syntax of the command is like that of other "FOR class" commands:

A WHERE clause can be appended to the command, but not a VIA clause. [See 3, 4.]

This command is not recommended for general use because it is possible that two versions of the same record may be in the residual data set. Usually this does not happen, but there is no guarantee that it will not. When you access records through the tree, the tree knows where the correct version is. Avoiding the tree by using the FOR RESIDUAL DATA command means that the same record in two versions could be processed twice.

If you do need to use this command, you can use the alternate form FOR SUBFILE VIA RESIDUAL DATA to insure that you get the latest version of each of the records processed, though duplicates may still occur. [See 3.5.] If you issue the command for a subfile whose records are not REMOVED, subsequent Global FOR commands will fail.

2.13  The FOR GOAL (FOR GOA) Command

The FOR GOAL command is actually a short form of the command FOR SUBFILE VIA TREE. [See 3.6 for a detailed description.]

2.14  The FOR INDEX command

When you want to sequentially process all the goal records having elements with values in a particular index, use the FOR INDEX command:

where "index-name" is a search term from the list displayed by the SHOW INDEXES command. The WHERE clause is discussed later. [See 4.] The SEQUENCE clause is discussed below. No VIA clause is allowed. [See 3.]

Record sorting is often done in SPIRES with the SEQUENCE command or with the SPISORT program. For both of these processes, SPIRES must examine each goal record being sorted, extract the element or elements being sorted upon, and sort the values, creating a stack or set of pointers to the goal records. The stack or set can then be processed under Global FOR (or with the TYPE command for stacks). A typical command sequence, using the SEQUENCE command, might be:

But if the CONFERENCE subfile already has a STATE index, the sorting of the goal records by state is always readily available, through the FOR INDEX command:

This sequence of commands probably provides the same result (see the "Notes" section below for exceptions) and is more efficient, since now SPIRES does not have to sort all the goal records by state.

Let's consider this process in more detail. Suppose the CONFERENCE subfile has goal records and index records with the following structure:

Each goal record has a key element called ID, a number, as well as a person's NAME and home STATE. In the STATE index, each index record has the name of a state as the key element, and a GOALREC.ID, which is the number of a goal record containing that state name. Suppose the following index records existed:

Now you issue the following commands:

When the DISPLAY command is issued, SPIRES examines the first index record, retrieves the goal record keys, and begins retrieving the "named" goal records, one at a time.

Note that if you use an option on a record processing command that requests an individual record (DISPLAY FIRST, for example), you are requesting an individual goal record, not all the goal records in a single index record. In other words, DISPLAY FIRST would only display record 5 in the above example, because the DISPLAY command is requesting the processing of goal records, not index records. The index records are only used to provide an access path to the goal records.

You might imagine the index to be a giant stack of records, and the clusters of record pointers in a single index record (the two for ALABAMA, for instance) to be "sub-stacks". Using the SEQUENCE clause, the goal records in a sub-stack can be further sorted, if desired. For example, the table above can be further sorted by NAME:

The goal records are still arranged in order by state, but within each state they are arranged by name as well.

The syntax of the SEQUENCE clause is similar to that of the SEQUENCE command:

where "element-name" is the name of or a valid alias for a goal record element by which further sorting is to be done. The SEQUENCE clause must be placed in parentheses. The "(D)" option may be used to request that the values be sorted in descending order rather than ascending, for that particular element. The "(X)" option may be used to request that the external form rather than the internal form of the element be used for sorting (that is, the form in which the value is displayed, rather than the form in which it is stored). The two options may be combined as shown in the syntax: "(DX)". Other element names to be used in the sorting may follow, if desired. EXPLAIN SEQUENCE COMMAND, or see the SPIRES manual "Searching and Updating", section B.4.5 for more information about the SEQUENCE command.

Notes and Considerations:

2.15  The FOR PATH Command

Path processing in SPIRES is a facility that allows you to have multiple subfiles selected at once, or multiple "views", through multiple formats, of a single subfile. Preliminary documentation for this facility can be found in the SPIRES manual "Technical Notes", section 14.

The FOR PATH command is part of the path processing facility in SPIRES. The FOR PATH command is similar to the FOR INDEX command, except that the pointers to the desired goal records are not found in an index to the goal records but in another subfile. Also, the FOR PATH command may be considered an extension of the FOR INDEX command in that it can perform the same function as FOR INDEX in many cases but can also restrict the index records retrieved based on element criteria. That second purpose is discussed later in this section.

Suppose that you have two subfiles, COURSES and ROOMS:

The goal record elements for the ROOMS subfile include a multiply occurring element called COURSE.NUMBER, which represents the numbers of courses taught in that room, the same type of value found as the goal record key for the COURSES subfile (BIO 239, for example).

Suppose you want a report on the classes taught, listed in order by room number; courses using the same room should be listed in order by day and time. The easiest way to solve the problem is with this group of commands:

However, a more efficient method is to take advantage of the ROOMS subfile, which already has the room numbers sorted.

The Global FOR command THROUGH 1 FOR SUBFILE specifies which records in the ROOMS subfile are to be examined. The command THROUGH 1 FOR SUBFILE establishes which records in the ROOMS will be used as "access records" to the goal records in the COURSES subfile. In this case, FOR SUBFILE indicates that all the records in the ROOMS subfile will be used. The following command, "FOR PATH 1...", indicates that goal records in the COURSES subfile will be retrieved using the records available through "path 1", and that the element COURSE.NUMBER in those records should be used as the pointer to indicate which goal record in COURSES should be retrieved. In other words, the goal records in the ROOMS subfile, arranged in key order by room, are used as index records to goal records in the COURSES subfile, even though two different subfiles, perhaps not even in the same file, are involved.

Most of the Global FOR classes can be used in the "THROUGH pathname FOR class" command; the exceptions are RESULT, STACK, INDEX and PATH. (You can circumvent the prohibition of FOR RESULT and FOR STACK here by storing a result or stack with the STORE RESULT.name or STORE STACK.name command, and then using the stored result or stack as a set: THROUGH pathname FOR SET RESULT.name, for example.) The command may have a WHERE clause or SET SCAN commands associated with it in order to limit the "index records" processed or examined. [See 4, 6.2.]

The syntax of the FOR PATH command is similar to that of the FOR INDEX command:

where "pathname" is the name or number of a path established by a previous THROUGH... SELECT command and "elemname" is the name of the element in the path-selected subfile that contains the key values of records in the primary subfile. The SEQUENCE clause has the same form as it has on the FOR INDEX command and has the same uses. [See 2.14.] The WHERE clause is discussed in a later chapter. [See 4.]

Perhaps a more common use of the FOR PATH command is in situations where you could otherwise use FOR INDEX except that you want to restrict the index records accessed based on some criteria within them. (For example, you want to have a WHERE clause on the index records accessed rather than on the goal records.)

Suppose that both COURSES and ROOMS are subfiles in the same file:

Suppose also that the goal records for ROOMS are index records for COURSES (i.e., we could issue the command sequence SELECT COURSES and FOR INDEX ROOMS). Here are two possible reports that could be produced:

In example 1, you request a list of all classes in the Terman building (i.e., room numbers prefixed by TERMAN) in order by room number. SPIRES only accesses those index records with keys beginning with the string TERMAN when it picks up pointers to the COURSE goal records. [See 6.2 for information on the SET SCAN commands.]

In example 2, you request a list of all classes held in rooms that can hold more than 100 people, arranged in order by room number. In both examples, you tell SPIRES to use criteria in the index records to determine which goal records to process.

Note that example 1 could probably have been done with a different method: a search command (FIND ROOM PREFIX TERMAN) and a SEQUENCE command. The second example could not have been done that way, however, since the COURSES goal record did not contain the needed criteria (the size of the room). For the first example, the method shown above is at least as efficient, and possibly somewhat more efficient, than the FIND and SEQUENCE method.

The same "Notes and Considerations" discussed under FOR INDEX apply to FOR PATH. [See 2.14.] The FOR PATH command is equivalent to the command FOR SUBFILE VIA PATH, meaning that all records processed under FOR PATH will be in their latest versions. [See 3.8.]

2.16  The FOR STORED Command

The FOR STORED command differs from the other Global FOR commands in that it is only available in SPIBILD. It is available for file owners to use when processing files. The syntax of the commmand is:

where "RESULT" is a stored result and "STACK" is a stored stack.

For more detailed information on the use of this command, refer to the document "SPIRES File Management" or online, type EXPLAIN FOR STORED.

2.17  The FOR LOAD Command

The FOR LOAD command processes records that have been copied into a load file using the GENERATE LOAD command. It is available for file owners and users with See access to the file containing the selected subfile.

The syntax of the commmand is:

where "loadname" is the name of the ORVYL file containing the load data. If the load is stored under a different account, be sure to use the fully qualified form: "ORV.gg.uuu.loadname".

Using FOR LOAD is a handy way to see what records are in a load; once it is generated, the only other way to examine its contents is to load it back into the file or into another file. For that reason, the SHOW KEYS command is particularly useful with FOR LOAD.

For more detailed information on the GENERATE LOAD command, refer to the document "SPIRES File Management" or online, type EXPLAIN GENERATE LOAD.

2.18  The FOR * Command

The "FOR *" command is one of two commands that are special cases in Global FOR, the other being "FOR **". The "FOR *" command usually has a WHERE clause, and is used to change WHERE clauses in the middle of a Global FOR stream. It can only be issued when some other "FOR class" command is in effect.

The syntax of the command is:

The WHERE clause is discussed later in this manual. [See 4.] No VIA clause is allowed on the "FOR *" command. [See 3.]

In the example, you first ask to process the records in the tree with more than 6 occurrences of the INGREDIENTS element and then ask for the key values of the first two records that fit the WHERE clause criterion. Next, you issue the "FOR *" command, indicating a different WHERE clause to be used for subsequent record processing. However, your current position "within the class" remains the same: you will not start processing records at the beginning of the class again, but will continue from the last record processed while the previous WHERE clause was in effect. The example makes that point clear. Records 47 and 138 had already been processed when the "FOR *" command was issued, and so the SHOW KEYS REST command does not show them again. But when you ask to see the keys of all the records in the tree with more than four ingredients ("SHOW KEYS ALL") SPIRES begins processing again "from the top" and records 47 and 138 are included in the list. Thus, the "FOR *" command issued in the example is not equivalent to "FOR TREE WHERE INGREDIENTS OCCURS > 4", since that command would restart you at the top of the class. [See 5.1 for a discussion of your position within the class and the "current record pointer".]

If you issue a "FOR *" command without a WHERE clause, the current WHERE clause is no longer in effect, and processing continues from the same position.

Any SET SCAN commands in effect when a "FOR *" command is issued will remain in effect under the "FOR *" command unless you issue a CLEAR SCAN command. [See 6.2.]

2.19  The FOR ** Command

The "FOR **" command simply reissues the "FOR class" command that is currently in effect. It can only be issued when you are already in Global FOR mode. Its syntax is simply:

No VIA or WHERE clause is allowed on this command. [See 3, 4.]

In the example, "FOR **" is equal to "FOR TREE WHERE INGREDIENTS OCCURS > 6". Thus, it starts you over again without having to reissue the identical "FOR class" command. In most cases, using the FIRST or ALL options on the subsequent Global FOR commands will probably be as effective as the "FOR **" command. [See 5.1.] For instance:

If the two commands on the right are substituted for the two commands on the left in the previous example, the result will be about the same. (The difference is that the resulting active file from the commands on the left will include a line of four asterisks, "****", before each record and a semicolon, ";", after each one, which will not happen from the single-record processing commands on the right.)

Once you have left Global FOR mode, either because you issued an ENDFOR command or because you were automatically ejected, you cannot issue the "FOR **" command. Any SET SCAN commands in effect when a "FOR **" command is issued will remain in effect under the "FOR **" command, unless you issue a CLEAR SCAN command. [See 6.2.] Alternatively, you can reissue the original "FOR class" command, which will clear any "scans" in effect.

3  The VIA Clause and Access Classes

Suppose that you want to see the original tree copies of all the updated records in the deferred queue. How would you do it? The FOR UPDATES command accesses the latest copies; the FOR TREE command would access the tree copies, but would give you all the records in the tree rather than just the updated ones. The solution requires a VIA clause on a "FOR class" command.

The VIA clause on the "FOR class" command allows you to specify that only the records in the Global FOR class whose keys (or pointers) are determined by the access class of the VIA clause are to be processed by subsequent Global FOR commands. The records in the Global FOR class are then processed in the order of the keys in the access class. In other words, the "access-class" in the VIA clause determines the keys of the records to be processed as well as the order in which they should be processed; the class in the "FOR class" part of the command determines where SPIRES should get each record. (The "class" in a "FOR class" command not having a VIA clause specifies all of that at once: the keys of the records to be processed, the order and the source.) Of course, even with a VIA clause, the "FOR class" command does not cause SPIRES to start processing records (looking for records in the "class" that are also in the "access class"). That activity does not begin until a record-processing command is issued. [See 5.]

Only some access classes discussed in the previous chapter can be used in a VIA clause (plus TREE in a special case), and only seven retrieval classes can be the Global FOR class when a VIA clause is used. [See 2.]

The syntax of the "FOR class" command using a VIA clause is:

where "access-class" can be STACK, RESULT, "SET setname", TRANSACTIONS, RESIDUAL DATA, "INDEX index-name", "PATH pathname" or TREE, and where "class" can be TREE, SUBFILE, DEFQ, ADDS, UPDATES, REMOVES, or TRANSACTIONS.

There is also a separate set of VIA "access-classes" that can be used with the FOR TRANSACTIONS command. [See 3.9.]

There are over fifty combinations of "class" and "access class", not all of which are useful or meaningful. Here are some examples of "FOR class" commands with VIA clauses:

For instance, FOR TREE VIA STACK, as discussed at the beginning, indicates that we want to begin processing the tree copy of records in the current stack. (Of course, strictly speaking, there are no records in a stack or result or set -- only keys or pointers. However, because accessing a result, stack or set in Global FOR usually leads you to records, we can take the liberty of referring to "the records in a result", for instance.) FOR SUBFILE VIA STACK indicates that we want to process the latest copy of each record in the stack. (FOR SUBFILE VIA STACK is equivalent to FOR STACK, so it is seldom issued in this longer form.) FOR UPDATES VIA RESULT means that we want to process the latest copy of all records being updated that are found by the search result.

Here is a chart showing the allowed combinations of VIA clauses:

  +-----------+------------------------------------------------------------+
  |           |               VIA access-class                             |
  | FOR class | RESULT | STA | SET | TRANS | RES DAT | INDEX | PATH | TREE |
  |-----------+--------+-----+-----+-------+---------+-------+------+------|
  | TREE      |        |     |     |   X   |    D    |       |      |  X   |
  |-----------+--------+-----+-----+-------+---------+-------+------+------|
  | SUBFILE   |   D    |  D  |  D  |   X   |         |   D   |   D  |      |
  |-----------+--------+-----+-----+-------+---------+-------+------+------|
  | DEFQ      |        |     |     |   D   |    X    |       |      |  X   |
  |-----------+--------+-----+-----+-------+---------+-------+------+------|
  | ADDS      |        |     |     |       |    X    |       |      |  X   |
  |-----------+--------+-----+-----+-------+---------+-------+------+------|
  | UPDATES   |        |     |     |       |    X    |       |      |  X   |
  |-----------+--------+-----+-----+-------+---------+-------+------+------|
  | REMOVES   |        |     |     |       |    X    |       |      |  X   |
  +-----------+--------+-----+-----+-------+---------+-------+------+------+

The positions marked with an "X" indicate combinations that are not allowed. Those marked with a "D" indicate the default combination when only the class that would be used in the VIA clause is used in a "FOR class" command (e.g., FOR SUBFILE VIA RESULT is the actual result when you issue the FOR RESULT command). The positions without a mark indicate combinations that are allowed.

The following chart shows the special set of access classes that can be used with the FOR TRANSACTIONS command:

  +------------------+----------------------------------------+
  |                  |               VIA                      |
  | FOR TRANSACTIONS |  DEFQ    ADDS    UPDATES    REMOVES    |
  +------------------+----------------------------------------+

The sections in this chapter will describe each access class as it is used in a VIA clause. Further information about the classes can be found in the previous chapter. [See 2.]

3.1  The VIA RESULT (VIA RES) Clause

The VIA RESULT clause is used to specify that the records in the Global FOR class that are also in the current search result are to be processed by subsequent Global FOR commands. The records will be processed in the order in which their keys or pointers occur in the search result. [See 2.9.] If you change the search result during processing, the next Global FOR command will begin processing records in the new result from the top.

Below are the six possible "FOR class" commands using the VIA RESULT clause:

FOR SUBFILE VIA RESULT

This is equivalent to FOR RESULT. [See 2.9.] When processing records under this command, SPIRES examines the deferred queue. The latest copy of an updated record is processed; records being removed are not processed at all.

FOR TREE VIA RESULT

This command is used to examine the tree versions of all the records in the search result. Thus, any records removed or updated during the day would still be processed in their tree versions. Unless the indexes are immediate indexes, remember that the indexes reflect the TREE rather than the SUBFILE copies of records, since records but not indexes may have been updated since the file was last processed.

FOR ADDS VIA RESULT

This command will access only the records in the search result that have been added since the file was last processed. Note that this command will only yield a meaningful result if the indexes used to create the result are immediate indexes.

FOR UPDATES VIA RESULT

This command will access only the records in the search result that are being updated. The latest copy of each will be processed under this command.

FOR DEFQ VIA RESULT

Because the class DEFQ combines ADDS and UPDATES, this command is equivalent to FOR UPDATES VIA RESULT, described above.

FOR REMOVES VIA RESULT

This command is useful when you need to determine whether any records in the current search result are being removed, and if so, which ones. Remember that only a few Global FOR commands can be issued under a FOR REMOVES command. [See 2.7.] Suppose for instance that you had removed several hundred records from a subfile during the day and you needed to know whether any of them had been updated since 1979:

You could then use a FOR TREE VIA STACK command to see which records they were. [See 3.2.]

3.2  The VIA STACK (VIA STA) Clause

To specify that the records in a Global FOR class whose keys or pointers are in the current stack are to be processed, use the VIA STACK clause on a "FOR class" command. You can then process the records in the order in which they are stacked. The stack can be changed during your processing, but subsequent Global FOR commands will start processing from the start of the new stack rather than continuing with the old one, unless you alter the stack by issuing the STACK command. [See 5.8 for a discussion of this exception.] A stored stack can be processed using the FOR SET command or by using the VIA SET clause on a "FOR class" command.

Below are the six possible "FOR class" commands using the VIA STACK clause:

FOR SUBFILE VIA STACK

This command is equivalent to FOR STACK. [See 2.10.] The latest versions of records will be processed, meaning that records being removed will not be processed at all.

FOR TREE VIA STACK

This command prepares SPIRES for processing copies of records from the tree that are in the stack. It is often used when you are trying to get the original tree copies of records being removed or updated:

FOR ADDS VIA STACK

If any records in the stack have been added since the file was last processed, this command could be used to retrieve only them. If the stack was created by sequencing a search result, this command would not be very useful. [See 3.1, the FOR ADDS VIA RESULT command.]

FOR UPDATES VIA STACK

This command would be useful if you wanted to process only the records in a stack that had been updated.

FOR DEFQ VIA STACK

This command accesses records being added or updated that are in the current stack. It should not be confused with the FOR SUBFILE VIA STACK command (FOR STACK) which accesses the latest copy of all records in the subfile, including adds and updates, that are also in the stack. FOR DEFQ VIA STACK accesses only added and updated records in the stack.

FOR REMOVES VIA STACK

Similar to FOR REMOVES VIA RESULT, this command is useful when you need to determine whether any records in the current stack are being removed.

3.3  The "VIA SET setname" Clause

Once you have created a set of records, you process those records with either the "FOR SET setname" command or a "FOR class" command that has a "VIA SET setname" clause, where "setname" is the name you chose for the set when you issued the DEFINE SET command. (Remember that the set can also be a stored stack or result.) [See 2.11.] The records in the Global FOR class that are also in the set can then be processed in the order in which their keys are arranged in the set.

The "FOR class VIA SET setname" commands are discussed in detail below:

FOR SUBFILE VIA SET setname

This command works identically to the "FOR SET setname" command, providing access to the latest copies of the records specified in the set. [See 2.12.]

FOR TREE VIA SET setname

Using this command, you will be able to access tree copies of all records in the set. As with all FOR TREE commands, this command allows you to access records slightly more efficiently than a FOR SUBFILE command, since processing under FOR TREE does not involve checking the deferred queue for the latest version of each record. Hence, FOR TREE would be preferable to FOR SUBFILE (see below) if the file had recently been processed and there were few or no updated or removed records in the deferred queue.

The remaining four possibilities, FOR ADDS, FOR UPDATES, FOR DEFQ, and FOR REMOVES, all with "VIA SET setname" clauses are similar to their counterparts with VIA RESULT or VIA STACK clauses. [See 3.1, 3.2.] The difference is that it is the records in the named set, rather than those in the current result or stack, that are accessed in the class.

3.4  The VIA TRANSACTIONS (VIA TRA) Clause

So far, each of the access classes in a VIA clause has specified a subset of records in the Global FOR class that you want to process. The VIA TRANSACTIONS clause, however, has a different effect on "FOR class" processing. A "FOR class VIA TRANSACTIONS" command specifies that subsequent commands will process the specified copies of the records in the transactions list that are in the Global FOR class. Details and examples given below for each of the commands should make this clear. Note too that the information discussed in the section on the FOR TRANSACTIONS command is relevant here. [See 2.8.]

FOR DEFQ VIA TRANSACTIONS

This command is equivalent to FOR TRANSACTIONS. It accesses all the versions in the deferred queue of records that have been added or updated that day, even if they were later dequeued. Thus, if a record was added and then updated later, both copies could be accessed by this command. The copies are accessed in the order in which they were added or updated, oldest to newest. You should read about FOR TRANSACTIONS for more details. A similar command is FOR TRANSACTIONS VIA DEFQ. [See 3.9.]

FOR ADDS VIA TRANSACTIONS

This command allows you to access all copies of records added to the subfile during the day, even if they have since been dequeued. If a record is added and updated later that day, it is still considered an "add" by SPIRES; both copies can be accessed under the FOR ADDS VIA TRANSACTIONS command and neither copy with the FOR UPDATES VIA TRANSACTIONS command (discussed below). Suppose for instance that today record 17 was added, record 18 was added, record 17 was dequeued, record 18 was updated, and finally record 17 was added again:

The FOR ADDS command accesses only the latest copy of the records being added, meaning that any dequeued adds will not be processed. The FOR ADDS VIA TRANSACTIONS command shown above accesses the original added versions of records 17 and 18, then the updated copy of 18 and then the newly added copy of 17. You may also find the FOR TRANSACTIONS VIA ADDS command useful. [See 3.9.]

FOR UPDATES VIA TRANSACTIONS

Similar to FOR ADDS VIA TRANSACTIONS, this command allows you to access all deferred queue copies of records in the tree that have been updated during the day, even if they have since been dequeued. As noted before, FOR UPDATES does not include copies of added records that were updated later the same day. Under FOR UPDATES VIA TRANSACTIONS, records will be processed by Global FOR commands in the order in which they were updated. See also FOR TRANSACTIONS VIA UPDATES. [See 3.9.]

FOR REMOVES VIA TRANSACTIONS

Using this command you can determine which records have been removed during the day, including those which have been dequeued:

In the above example, the SHOW KEYS ALL command issued under FOR REMOVES shows that records 5 and 17 are being removed. However, the SHOW KEYS ALL command issued after FOR REMOVES VIA TRANSACTIONS shows that record 17 was removed first, then 25, then 25 again (that is, it was probably dequeued and later removed again) and then 5, with record 25 dequeued again at some point. Thus, the records are processed in the order in which the original transactions occurred. The FOR TRANSACTIONS VIA REMOVES command may also be useful. [See 3.9.]

3.5  The VIA RESIDUAL DATA Clause

Like the FOR RESIDUAL DATA command, the VIA RESIDUAL DATA clause is useful primarily to file owners. Only two "FOR class" commands have any meaning with this clause; the other four are not legal with it. The two that are meaningful are FOR TREE and FOR SUBFILE.

FOR TREE VIA RESIDUAL DATA

This is the equal of FOR RESIDUAL DATA -- both commands have the same effect. [See 2.11.] This type of processing actually does not involve the tree but since it does retrieve "tree copies" (the "original" copy in the subfile before any of today's updating activity is taken into account), the command syntax reflects the general meaning rather than the strict one.

FOR SUBFILE VIA RESIDUAL DATA

This command gives you access to the latest copy of each of the records in the residual data set. If the record has been updated today, the most recent version will be processed; if the record has been removed today, the record will not be processed. Otherwise, the "tree copy" will be processed. No added records are processed since they are not in the residual data set.

Remember that the same record could be processed twice if it occurs twice in the residual data set; for that reason, these commands are not recommended for general use. [See 2.11.]

3.6  The VIA TREE (VIA TRE) Clause

The VIA TREE clause is allowed only on a FOR SUBFILE command.

FOR SUBFILE VIA TREE

FOR SUBFILE VIA TREE is similar to FOR TREE in that you can sequentially process all the records in the tree under both commands. However, FOR SUBFILE VIA TREE requests that, for every record in the tree, SPIRES examine the deferred queue to see if the record has been updated or removed. Then the most recent version of the record should be used in processing. Hence, if a record has not been updated since the file was last processed, it will come from the tree; if it has been updated, it will come from the deferred queue; if it has been added or removed, it will not be processed at all.

This command replaces (and is equivalent to) the command FOR GOAL, which is still allowed. If FOR GOAL is issued with a VIA clause, such as FOR GOAL VIA ADDS, it is equivalent to FOR SUBFILE with the same VIA clause.

3.7  The "VIA INDEX index-name" Clause

Taking advantage of the record sequencing inherent to an index, you can process the goal records in a particular order without having to create a set or stack first. Of course, the element by which you want the records sorted must be indexed. If this is the case, you can use the FOR INDEX command or a "FOR class" command with the "VIA INDEX index-name" clause appended. The records in the Global FOR class that are also indexed in the named index can then be processed in the order in which they are indexed.

The VIA INDEX clause may have a SEQUENCE clause to indicate further sequencing of the goal records having the same indexed value:

where the SEQUENCE clause, which must be enclosed in parentheses, has the same syntax as the SEQUENCE command. It is discussed in the section on the FOR INDEX command. [See 2.14.]

The "FOR class VIA INDEX index-name" commands are discussed below:

FOR SUBFILE VIA INDEX index-name

This command works identically to the "FOR INDEX index-name" command, providing access to the latest copies of records specified in the index.

FOR TREE VIA INDEX index-name

This command allows you to access tree copies of records in the index. It is slightly more efficient than FOR INDEX, and because the tree copies more accurately reflect the contents of the indexes than the "subfile" copies might, this form may be preferable in many cases. (FOR INDEX and the VIA INDEX clause do not examine the deferred queue of the index record.)

The other four possibilities, FOR ADDS, FOR UPDATES, FOR DEFQ and FOR REMOVES, used with "VIA INDEX index-name" clauses, all work similarly to their counterparts with VIA STACK or VIA RESULT. [See 3.1, 3.2.]

3.8  The "VIA PATH pathname" Clause

If the goal records of another subfile can be considered as index records to a subfile, path processing techniques in conjunction with Global FOR processing can be used to process the subfile's records via the second subfile. This concept was discussed in detail in the section on the FOR PATH command. [See 2.15.]

The FOR PATH command is a shortcut for FOR SUBFILE VIA PATH. The "VIA PATH pathname" clause can be added to a "FOR class" command to indicate that the records in the Global FOR class that are specified by the named path will be processed by subsequent Global FOR commands.

The VIA PATH clause may have a SEQUENCE clause to indicate further sequencing of the goal records:

where the SEQUENCE clause, which must be enclosed in parentheses, has the same syntax as the SEQUENCE command. The "elemname" is the name of the element in the path-selected subfile that is the pointer to the goal records in the primary subfile. [See 2.15 for examples.] It is discussed in the section on the FOR INDEX command. The "FOR class VIA PATH pathname" commands are described briefly below:

FOR SUBFILE VIA PATH pathname

As mentioned above, this is equivalent to the FOR PATH command, discussed earlier in this manual. It provides access to the latest copies of records specified in the set.

FOR TREE VIA PATH pathname

Using this command, you can access the tree copies of all goal records in the primary subfile that have pointers provided by the path-selected subfile.

The other four possible classes, ADDS, UPDATES, DEFQ and REMOVES, work similarly with a VIA PATH clause to the way the work with a VIA STACK or VIA RESULT clause. [See 3.1, 3.2.]

3.9  The FOR TRANSACTIONS VIA "Access-Class" Commands

The FOR TRANSACTIONS command accesses different versions of added and updated records since the file was last processed. If a record has been added, updated, or removed, the FOR TRANSACTIONS VIA class command is another way to keep track of the multiple copies of records in the DEFQ.

The FOR TRANSACTIONS command allows a subset of access classes that are only allowed with the FOR TRANSACTIONS command. These are: VIA DEFQ, VIA ADDS, VIA UPDATES, and VIA REMOVES.

FOR TRANSACTIONS VIA DEFQ

This command differs from the simple FOR TRANSACTIONS command in a number of ways.

1) The FOR TRANSACTIONS command presents the records in the order in which they were added or updated. However, FOR TRANSACTIONS VIA DEFQ presents each record in sequence by record key. For example:

2) The FOR TRANSACTIONS VIA DEFQ command includes the TREE copy of each record that is accessed -- that is, the original copy of the record in the database before any changes were made.

3) In the FOR TRANSACTIONS command, records are presented that never were added to the file (e.g. a DECOMITted transaction). These do not show up in the FOR TRANSACTIONS VIA DEFQ command.

For more detail about the transactions, it is recommended that the file owner use the SHOW SUBFILE TRANSACTIONS command.

FOR TRANSACTIONS VIA ADDS

This command allows you to access the transactions for the records that have been ADDed to the subfile in key sequence. If a record is added and updated that day, it is still considered an "add" by SPIRES; both copies can be accessed under the FOR TRANSACTIONS VIA ADDS (but not if the record has since been removed)

FOR TRANSACTIONS VIA ADDS differs from FOR ADDS VIA TRANSACTIONS in the following ways:

1) The records in FOR TRANSACTIONS VIA ADDS are presented in key sequence.

2) If the record has been added and then removed, it is not included in the FOR TRANSACTIONS VIA ADDS (but it is listed under FOR ADDS VIA TRANSACTIONS).

In the above example, FOR ADDS VIA TRANSACTIONS shows that record 9 was added to the database, then record 10, and then record 9 was updated two more times -- the keys are presented in the order that the adds and updates were made. On the other hand, FOR TRANSACTIONS VIA ADDS processes the records in key order. [See 3.4.]

FOR TRANSACTIONS VIA UPDATES

This command allows you to access the transactions for the records that have been UPDATEd in the subfile in the key sequence. If a record is added and updated that day, it is still considered an "add" by SPIRES; both copies can be accessed under the FOR TRANSACTIONS VIA ADDS (but not if the record has been removed) FOR TRANSACTIONS VIA UPDATES does not include copies of added records that were updated later the same day. [See 3.4.]

FOR TRANSACTIONS VIA REMOVES

This command allows you to access the transactions for the records that have been REMOVEd from the subfile in key sequence. However, if a record has been added and removed that day, it can only be accessed through the FOR TRANSACTIONS or FOR TRANSACTIONS VIA DEFQ commands.

4  The WHERE Clause

So far in this manual, Global FOR has been used either to retrieve entire classes of records or else to retrieve subsets of records from one class that are also contained in another "access" class. For instance, using FOR ADDS, you can access all the added records in a subfile. But sometimes you need more focused criteria for record retrieval than a class of records (or intersection of classes) can provide.

For example, suppose you are looking for an added record in the DRINKS subfile and you do not remember the key of the record, though you do remember that the record has "gin" as one of the values for the element CONSTITUENT. If only a few records have been added to DRINKS since the file was last processed, you can easily retrieve the record you want by examining one record after another in the ADDS class:

But if quite a few records have been added since the file was processed, the method above will be time-consuming, since you will have to look at all the added records, whether or not they have a CONSTITUTENT of gin.

If you include a WHERE clause on the FOR ADDS command, you can have SPIRES do this investigative work for you:

After you have issued a Global FOR command with a WHERE clause, your subsequent Global FOR commands will only process the subset of records that have passed this WHERE filtering. Thus, in the second example above, DISPLAY commands will only display the subset of added records with a CONSTITUENT occurrence of gin. (Note that SPIRES generally still "examines" the entire class, even though it only "processes" the filtered subset.) In general, a WHERE clause on a "FOR class" command specifies criteria that records in the class must meet in order to be processed by subsequent Global FOR commands.

Efficiency of WHERE Clauses

A WHERE clause sometimes makes Global FOR processing of records more expensive for the following reason. When you process records under a "FOR class" command without a WHERE clause, SPIRES knows instantly which records are to be processed (i.e., all the records in the class). But when a WHERE clause is in effect, SPIRES usually must examine each record individually to see which records fit the criteria and should be processed. Because the WHERE clause causes more work for SPIRES, it costs more to use than an unadorned "FOR class" command does. On the other hand, it is certainly more efficient for SPIRES to examine the records than for you to do so. In addition, you can often reduce expenses by using a smaller class (such as a result instead of the tree), or a VIA clause, or a SET SCAN command.

Other Capabilities of WHERE Clauses

The example above shows the simplest WHERE clause available, but your WHERE clause can be both more complex and more precise. For instance, the WHERE clause can name more than one criteria, thereby specifying that records retrieved must satisfy all of the conditions you name (or must satisfy any -- or even none -- of the conditions you name). Or your criteria clause can use the OCCURS option to request records that merely have a value for an element, regardless of what that value is. With the "inter-element relations" capability, your criteria clause might request only records in which one element equals (or is less than or greater than) another element in the record. [See 4.5.]

The WHERE clause offers other capabilities besides these, and most of them will be discussed in detail within this chapter. The upcoming section discusses the syntax and power of the "simple" WHERE clause, in which you specify only one criterion clause. [See 4.1.] The section following that one discusses the "compound" WHERE clause, which lets you specify multiple criteria for greater precision. [See 4.2.] The final sections discuss more advanced WHERE clause capabilities for specialized uses, such as binding element retrieval to a structural path. [See 4.4.]

The ALSO Command

The ALSO command is another mechanism, in addition to Global FOR commands, for searching on unindexed elements. The ALSO command only operates on a preceding search result; for example:

The ALSO command takes an existing result or stack and creates another result or stack that satisfy the conditions named in the criteria clause. The ALSO command can use WHERE-clause terms, such as OCCURS, LENGTH, "same structure" processing, and inter-element relations. Therefore, keep in mind that the discussion in the rest of this chapter that pertains to WHERE clauses also applies to ALSO criteria clauses.

4.0.1  (WHERE clause limits)

A WHERE clause has two significant limits that are worth knowing if you work with many or long values: it can have only 256 bytes of values and only 30 expressions.

So, for example, if you have this command:

 FOR SET WHERE SUBJECT = RAIN OR SNOW OR SLEET OR TITLE HAIL CAESAR

The total number of bytes is the total of the lengths of the strings RAIN, SNOW, SLEET and HAIL CAESAR: 24 bytes. (Note: The total is based on the lengths of the converted, i.e., internal values of the strings.)

The example has 4 expressions:

If your WHERE clause exceeds either of these limits, SPIRES gives you the error message "WHERE clause overflow".

4.1  The Simple WHERE Clause

A simple WHERE clause, when used on a "FOR class" command, has the following syntax:

The simple "criteria-clause" has the following syntax:

Here "element" is the name (or a valid alias) of the element to be examined by SPIRES, "relational-operator" is one of the standard SPIRES relational operators listed below, and "value" is the value to be compared with the element value or values. OCCURS and LENGTH are discussed in detail later in this section. A dynamic element name can be specified for "element". [See Chapter 20 of "SPIRES Technical Notes" for an explanation of dynamic elements.]

How Criteria Values Are Compared to Stored Element Values

When you issue a Global FOR command with a WHERE clause, SPIRES will process all records in the class that fit the criteria specified. If a WHERE clause specifies that an element must match a particular criterion, any record with an occurrence of that element matching the criterion will be processed, even though other occurrences of the same element within the record might not match the criterion at all. (You can use a filter to mask extraneous element occurrences. See the description of filters in the manual "SPIRES Technical Notes"; see also Section 4.4 later in this chapter.)

In comparing criteria clause values to stored element values in a record, SPIRES ignores the case (upper or lower) of both values, so you do not have to worry about case in your criteria clause. However, since SPIRES does process the criterion value through any INPROCs (processing rules for record input) that have been coded for the element, the value you specify in a WHERE clause must generally be valid as an element value. This topic is covered in more detail in Section 4.3.

Relational Operators in a WHERE Clause

All of the standard SPIRES relational operators, in the four categories of equality, range, inequality and content, are available in WHERE clauses and are listed below. If you do not use a relational operator, SPIRES will assume you want the equality operator. [See "SPIRES Searching and Updating", section B.3.2.3 for details on relational operators.]

When a WHERE clause is in effect, all relational operators are equally efficient, since SPIRES will examine the entire element value under Global FOR in any case. Thus, for example, even operators like WORD or STRING, which can be quite inefficient in an indexed search, become just as efficient as the equality operator in a WHERE clause.

Simple WHERE Clauses: Examples

Here are some examples of "FOR class" commands with simple WHERE clauses:

In the example marked (a) above, it is important to note that the command is asking to examine values of the CHILDREN element, not the number of occurrences of the element. The command would only be meaningful if the CHILDREN element contained integer values (e.g., for number of children). If the element has textual values such as "John", "Jerry", "Carolyn" and "Ellen", the record would still be processed, but the processing would convey false information, since in sorting, alphabetic characters are always considered "less than" numerical ones. Command (b), on the other hand, uses the OCCURS option, discussed further below, to process by the number of occurrences of a textual element.

4.1.1  (The LENGTH and OCCURS Options)

You can use a WHERE clause to access records that have a particular number of occurrences of an element or that have occurrences of a particular length. Using the OCCURS option, for example, you can specify that you want to process records that do not have any occurrences of a particular element or structure:

Using the LENGTH option, you can ask to process records with at least one occurrence of a particular element that is a particular length:

LENGTH refers to the number of bytes of storage used by the element value in internal storage within the file. Because elements with numerical values are often converted from string values into some other type (dates are stored internally as a four-byte HEX value, for instance), the LENGTH operator is generally most useful with textual data, such as an address or a comment.

In WHERE clauses using either LENGTH or OCCURS, a value must be supplied. Thus, FOR UPDATES WHERE PHONE OCCURS is not valid; the equivalent command "FOR UPDATES WHERE PHONE OCCURS > 0" is valid. Remember that if no relational operator is used, the equality operator is assumed. If any occurrence of the element being examined fits the criterion specified, that record will be processed.

When you use the OCCURS option for a dynamic element, the number of occurrences of the primary element in the dynamic element definition controls the number of occurrences of the dynamic element. [See Chapter 20 of "SPIRES Technical Notes" for an explanation of dynamic elements.]

The LENGTH and OCCURS options can also be used with elements that are structures, as described further below.

4.1.2  (The NOT Option)

The NOT option, when used in a simple WHERE clause, tells SPIRES that you want to process records only when the stated criterion does not occur as any of the occurrences of the named element. For example, a record with a ZIP-CODE value of 94305 would be processed under the command "FOR TREE WHERE ZIP-CODE 94305" but would not be processed under the command "FOR TREE WHERE NOT ZIP-CODE 94305". The set of records processed under one of these commands would exactly complement the set processed under the other command.

Note that the NOT option operates differently from the inequality operator (~=) with records having multiple occurrences of the named element. For example suppose a record contains two addresses and hence two zip-codes, 94305 and 94010. The record will be processed under the command "FOR TREE WHERE ZIP-CODE 94305" because, after all, there is one occurrence of the ZIP-CODE element that did fit the specified criterion; but it will not be processed under the second command, "FOR TREE WHERE NOT ZIP-CODE 94305", even though one of the occurrences is not 94305. In other words, the NOT option will only process records in which there were NO occurrences whatsoever of the value named.

On the other hand, after a command using the inequality operator "~=" instead of the NOT option the record containing both zip-codes would be processed, because it does have one occurrence of ZIP-CODE that is not equal to 94305.

Suppose a subfile has the following three records:

The records would be processed under "FOR class" commands as follows:

4.1.3  (Using Structures in a WHERE Clause)

Although simple elements are most often named in WHERE clauses, structures can also sometimes be used. In the examples below, PHONE is a structure consisting of the elements AREA-CODE and LOCAL-NUMBER.

With that command you could begin processing records in a search result that had more than one occurrence of the PHONE structure.

Structures can also be used in WHERE clauses with the LENGTH operator, but not as easily. Because the stored value of a structure may contain additional bytes of information (such as the length and number of occurrences of each element in the structure) these bytes become part of the total length of the structure. Though you may be able to determine exactly how many extra bytes will be added [See "SPIRES File Definition", section B.6.2.] it is easier to use the inequality operators such as ">" or "<=" than to try working with exact lengths.

There are cases where the "value" of a structure (when that value has been assembled from the values of separate elements within the structure) can be used in a WHERE clause just as though it were a conventional, single element value. In the example below, once again the value for PHONE is in fact built out of values for AREA-CODE and LOCAL-NUMBER:

Such a construction is only allowed when the structure is processed by $STRUC or $STRUC.IN as an INPROC. The general rule is that, if the structure can be entered into the record as a single value (which is then broken up into structural elements by SPIRES), such a value can be used in a WHERE clause.

4.2  The Compound WHERE Clause

Sometimes you might wish to specify more than one criterion in a WHERE clause. For instance, suppose you are searching for records with an "only child" named John -- you don't want to retrieve any records where John is just one of several children. To satisfy multiple conditions, you issue a compound WHERE clause -- that is, a WHERE clause that combines two or more criteria clauses, using one of the three logical operators, AND, AND NOT and OR:

There are only two relatively broad limits to compound WHERE clauses: 1) only 30 relational operators maximum are allowed; and 2) the entire "FOR class" command cannot be longer than 235 characters. You can circumvent the second limit either by using short element aliases and command abbreviations, or by assigning parts of the WHERE clause to SPIRES variables for substitution when the command is issued. [For more on variables, see the manual "SPIRES Protocols".]

The complete syntax of a criteria clause is:

where "criteria-clause" is another criteria clause with the same syntax as the first.

In a simple WHERE clause, the "element" is not optional. However, if there are two or more criteria clauses, the name is optional in the second or later clauses -- the last named element will be the default. Generally, this shortcut should only be used with the logical operator OR, because the presence or absence of the element name in the second clause is significant to the meaning of the clause when used with the AND or AND NOT operator. [See 4.2.4.] These two commands are equivalent:

(It is usually more efficient to repeat the element name, particularly if the goal record has phantom structures.)

The same shortcut is not allowed for relational operators, however. In the first command below, the PREFIX operator is only in effect for the first criteria clause. The default equality operator is assumed to apply to the second clause. Thus, these two commands are equivalent:

To find all phone numbers with a prefix of 497 or 324, the PREFIX operator should be repeated, as in this command:

Compound WHERE Clauses: Examples

Here are several other examples of "FOR class" commands using compound WHERE clauses:

Though unnecessary, the element name ADDRESS is repeated in the last example to avoid confusion. You would only need to repeat it if there were an element in the subfile named LENGTH that you did not want used.

4.2.1  Parentheses and Compound WHERE Clauses

Parentheses can sometimes help clarify a compound WHERE clause:

In this example, you are requesting records added in 1984 that were updated in 1984 or 1985. Omitting parentheses would create an entirely different request:

Here you would be requesting records added in 1984 that were updated in 1984 or any records updated in 1985, even if they were added in some year other than 1984. The rules about using parentheses in a WHERE clause are the same as those for using them in a search request. [See "SPIRES Searching and Updating", section B.3.4.2.]

4.2.2  The NOT Option and Compound WHERE Clauses

In a compound WHERE clause, the NOT option will only apply to the first criterion:

That command will let you process records having a value of CALIFORNIA for STATE but which contained no ZIP-CODE of 94025. The NOT option does not carry over into the other criteria of the WHERE clause unless parentheses are used:

4.2.3  The Order of Compound WHERE Clauses

Criteria clauses may be placed in any order in a WHERE clause. When SPIRES examines each record to determine whether it fits the WHERE clause criteria, it first checks the record against the first criteria clause, then the second, and so forth. For maximum efficiency then, you should arrange the criteria clauses in order starting with those most likely to fail -- that way SPIRES can more quickly proceed to the next record to be examined. For example, here are two slightly different ways to scan a student subfile for freshmen whose home-state is Virginia:

In either case, the same number of records will be examined and rejected when records are processed. However, if there are more Freshmen than Virginians (likely everywhere but in Virginia) then more records will be rejected after only one element has been examined if you use the first compound WHERE clause. Using the other form, more records will have to be examined for two criteria, which is slightly less efficient. This efficiency gap will scarcely be noticeable unless the difference between the number of records rejected by each criteria clause is large, and unless the total number of records being examined is also large.

4.2.4  Specifying the Same Element Twice in a Compound WHERE Clause

Compare these two seemingly nonsensical commands:

At first glance, it seems that the two commands are identical, and that neither command makes very much sense, since a single date cannot be both before 1983 and after 1984.

In fact, the two commands have different meanings and the first command does indeed make some sense in certain situations. The first command states that records to be processed must have at least one occurrence of DATE with a value after 1984 and at least one occurrence (not necessarily the same one) with a value before 1983. This is a logical request if more than one occurrence of DATE is allowed in the record. On the other hand, the second command states that records to be processed must have at least one occurrence of DATE with both a value after 1984 and value before 1983. Thus the second command really is as contradictory as it looks, as this example shows:

The rules for repeating element names are as follows:

 - in a compound WHERE clause, you should repeat the element name in the second clause, if you
 wish to specify that any occurrence can fit either criterion as long as each  criterion  is
 satisfied by at least one occurrence.
 - you should not repeat the element name in the  second  criteria  clause,  if  you  wish  to
 specify  that  the  same  occurrence  of an element should meet more than one criterion (or
 should not, in the case of the AND NOT logical operator);

One additional wrinkle: using the OR logical operator to join the two parts means that any occurrence can fit either criterion as long as at least one criterion is satisfied by at least one occurrence, whether or not the element name is repeated.

4.3  Processing Rules and WHERE Clauses

File owners often use processing rules to control element values stored within a record during input (INPROCs) and the way that those values may look when the record is displayed during output (OUTPROCs). Such processing rules are chosen by the file owner, who puts them in the file definition as part of the "description" of each element. [See "SPIRES File Definition", section B.4.]

When you issue a "FOR class" command that includes a WHERE clause of the form "WHERE element relational-operator value", SPIRES first takes the value and processes it through any INPROCs defined for the specified element. For example, if your command is "FOR TREE WHERE DATE-ADDED = 7/1/85", then the value "7/1/85" is processed through the INPROCs chosen for the DATE-ADDED element. It follows that a value given in a WHERE clause must generally be a legitimate value for the element if the element value were within a record being added to the subfile. In other words, error processing for element values in a WHERE clause tries to parallel element value error processing during record input.

Thus if SPIRES would not allow you to use a value when adding a record (i.e., if the error handling parameter for the INPROC were "E" or "S"), then you would not be allowed to use that value in a WHERE clause either. On the other hand, if SPIRES would respond to the element value during record-input with a warning message (of parameter "W"), without actually rejecting the value, you can use the value in a WHERE clause, but will receive a warning that there is a problem with the input.

The process is clear-cut when the relational operator in the WHERE clause is an equality, inequality or range operator. But it is slightly less clear-cut when the WHERE clause contains a content operator, such as STRING or WORD, because in these situations SPIRES still runs the value through the processing rules as if it were a complete value. For example, if only one of four values, FRESHMAN, SOPHOMORE, JUNIOR or SENIOR, was allowed as a value for the CLASS element (i.e., because $INCLUDE was used to limit allowed values), then the following command would fail:

The value FRESH was not one of the four allowed, so that when the value went through the processing rules for the CLASS element, the WHERE clause was rejected (just as an added record with that value would be), even though "fresh" is a valid prefix for the value FRESHMAN. In all situations like the one above, the file owner will be able to give you all the information (e.g., on which INPROCS are coded) that you will need to use Global FOR successfully.

Some processing rules that may be present in an INPROC string are ignored when a WHERE clause is issued -- that is, even if the value being processed would cause the processing rule to fail, the "FOR class" command will not fail. Generally, they are processing rules that verify, rather than change, a value. Here are the rules that are ignored, both in WHERE clause and SET SCAN processing:

4.4  Special Cases Handled by WHERE Clauses

The next few pages discuss two different ways to guarantee that multiple criteria you specify in a WHERE clause (or in an ALSO command) are retrieved along an identical structural path:

 - the '@'-sign, used as a prefix to element names in complex WHERE clauses,  to  ensure  that
 element  occurrences  come  from  the  same  "nested" level of the same structure
 occurrence; [See 4.4.1.]
 - the TYPE = SCAN option on the SET FILTER command, which (among other uses) can ensure  that
 element  occurrences  are retrieved along the same structural path (even though they may be
 nested at different levels).  [See 4.4.2.]

Finally, the chapter will describe how to specify a unique element when two elements in a subfile share the same name [See 4.4.3.] and will show how the values of two different elements can be compared in a WHERE clause. [See 4.4.4.] These topics are all fairly complex; if your uses of WHERE clauses will be more straight-forward, you may be able to skim these sections or skip them completely.

4.4.1  Specifying Same-Structure Processing

Using the "@"-Sign for Same-Structure Processing

Suppose a subfile of businesses has a multiply occurring structure for addresses of branch offices, each occurrence containing STREET-ADDRESS, CITY, STATE, and so on. Next suppose that you want to process records of businesses with branch offices in Menlo Park, California:

That command would indeed retrieve the records you wanted, However, if a business with an office in California also had an office in, say, Menlo Park, New Jersey, that record too would be processed, because SPIRES examines all occurrences of the elements specified in WHERE clauses in each record, so that element values fitting the criteria may not necessarily come from the same structure.

To ensure "same-structure processing", i.e., that all values are found in the same structure, at the same level, you would insert the symbol "@" (the "at" sign) directly before the name of the second and subsequent elements to which the limitation should apply:

SPIRES will then process only records having a value of MENLO PARK for CITY and a value of CALIFORNIA for STATE, both of the values occurring in the same ADDRESS structure.

There are four restrictions in using "@"-sign clauses:

 1) the logical operator preceding the "at" sign may only be AND or AND NOT;
    it may not be OR;
 2) the element preceded by the "@" sign must not use the OCCURS option;
 3) the element preceded by the "@" sign must be an element in the same
    structure as the preceding element; and,
 4) the "@" sign may not be preceded by parentheses:

The OCCURS Option and Same-Structure Processing

In WHERE clauses using the "at" sign, the OCCURS option should not be used by the element that preceeds the @element in the clause. For example, compare the following two commands:

As written, the first command is contradictory, in requesting records with a total of two occurrences of SEMESTER in the record, but also implying (with the "at" sign) that the occurrences should come from the same structure -- SPIRES reconciles this contradiction by ignoring the "at" sign and not performing any structural binding at all. The second command makes its intentions much clearer: "Find records having a value of MUS100 for COURSE that have two occurrences of SEMESTER in that same structure." So for the second command, structural binding is maintained.

If you need to do more than one OCCURS test in the same structure, and one or more of them simply must occur (OCCURS > 0), then you can substitute LENGTH >= 0 for OCCURS > 0, such as:

You can't use OCCURS > 0 with SEMESTER, but you can use LENGTH >= 0. Note that the @element (COURSE) can use OCCURS since it is not followed by another @element condition.

4.4.2  Binding Element Retrieval to the Same Structural Path

In same-structure retrieval using the "at" clause, as described in the preceding section, the element preceded by an @-sign must not only come from the same structure as the first-named element, but must come from the same depth of that structure as well. In other words, the @-element or elements cannot be nested at a deeper structural level than the level of the first-named element.

If you want to ensure that element values you retrieve lie along the same structural path, but you don't care whether they are nested at the same level or not, you can use the SCAN option on the SET FILTER command, naming a structural element as the element to be filtered, and naming an element nested within that structural element as part of the SET FILTER command's WHERE clause. [For a full description of the syntax of the SET FILTER command, see the manual "SPIRES Technical Notes".]

For example, consider a student subfile that contains a number of nested structures like Chinese boxes: a multiply-occurring structure for "year", each occurrence of which contains multiple occurrences for "quarter", each occurrence of which contains multiple occurrences for "course":

If you want to process occurrences of YEAR.STRUC only when they contain particular values for QUARTER and COURSE, the "at" clause will not be helpful, because QUARTER and COURSE, although they occur along the same structural path as YEAR, lie in "different" (more deeply nested) structures. The solution might be to set filters on the structures as in the following example:

Once the filters are set, you would need to activate them by naming the structural path to be filtered as part of the WHERE clause on your Global FOR command:

The WHERE clause on the last command is important: if you did not include it on your Global FOR command, the filtering would not occur.

Perhaps the trickiest part of this process is keeping the two different WHERE clauses straight in your mind: the SET FILTER command can have a WHERE clause just as a Global FOR command can have one, and the syntax allowed for the two WHERE clauses is identical. However, the WHERE clause on the SET FILTER command lets you solve a problem that might be impossible to solve using Global FOR alone. [Complex same-structure processing can also be handled by Partial FOR techiniques, but Partial FOR is more complicated and laborious than setting filters. [See the manual "SPIRES Technical Notes."]]

4.4.3  Specifying a Particular Element by "Path"

A few subfiles use the same element name within several structures, especially when the element is a common type, such as a name, phone-number or comment. Suppose a subfile of businesses has an address structure for both the MAIN office and the BRANCH offices, each structure containing at least a CITY element. If you issue the command "FOR SUBFILE WHERE CITY PALO ALTO", SPIRES will examine either the MAIN or BRANCH city for each record, but not both (the first one encountered in the list shown by the SHOW ELEMENT NAMES command would be the one examined), and would then process records where that particular type of office was in Palo Alto. Rather than have to figure out which one SPIRES will use, you would want to be more specific (e.g., you only want records where the main office is in Palo Alto). You can specify the particular "path" for SPIRES to follow through the record to find the appropriate occurrences of the element, again using the "at" sign ("@"):

This form is common throughout SPIRES in situations where you need a particular structural element. Its syntax looks like this:

where, from left to right, each structure named is nested within the previous one. There must be no blanks around the "@". Note that you only need to specify as many structure names from the right as are needed to uniquely identify the desired path.

Remember that specifying the path is only necessary if the element name is used within several different structures in the goal record. File definers often assign different aliases for the element in different structures to avoid the problem altogether. For example, in this case, the file definer might allow MCITY and BCITY for aliases.

4.4.4  Inter-element Relations in WHERE Clauses

You can compare the values of one element in a record to those of another element in the same record using an option on the criteria clause. This capability allows "inter-element relations".

The syntax of the "FOR class" command using inter-element relations is

where "element1" and "element2" are the names of the two elements being compared. The elements must be of the same type, for instance, string or integer. The "@" sign indicates that what follows it is an element name rather than a value. (If your value in a criteria clause actually begins with "@", the entire value must be surrounded by quotation marks.) Neither the OCCURS nor the LENGTH option can be used with inter-element relations. Any of the relational operators can be used, including "BETWEEN... AND..." and "FROM... TO...", which both use two values rather than one -- either or both values can be replaced by the "@element" option.

For example, to find records in a subfile that have been updated since the day they were added, you could type

assuming that those elements existed in the subfile.

Special care should be taken with "multiply occurring" elements (elements that may have several different values in a record, such as the office-phone element in the example below). Suppose you are using a subfile for an answering service, where record elements include a name, home phone number and office phone numbers, and you have these three records:

   NAME = Dr. Adams;      NAME = Dr. Bax;        NAME = Dr. Copland;
   HOME-PHN = 555-1111;   HOME-PHN = 555-2222;   HOME-PHN = 555-3333;
   OFF-PHN = 555-1111;    OFF-PHN = 555-2222;    OFF-PHN = 555-4444;
                          OFF-PHN = 555-2223;

If you issue the command "FOR TREE WHERE HOME-PHN = @OFF-PHN" you will be able to use Dr. Adams' and Dr. Bax's records. But if you issue the command "FOR TREE WHERE HOME-PHN ~= @OFF-PHN" you will be able to use Dr. Copland's record, but also Dr. Bax's, since his home phone is not the same as one of his office phones. Thus, when SPIRES examines a record for an inter-element relation, it compares all values of the first element to all values of the second one; if any one comparison fits the criterion requested, the record is retrieved. If you wanted only records where no office phone number matched any home phone number, you could say "FOR TREE WHERE NOT HOME-PHN = @OFF-PHN" which would retrieve all records except those where the two numbers matched.

If the elements being compared are not at the record level, but instead are structurally bound, then SPIRES compares the elements within individual occurrences of those structural bounds. So if NAME, HOME-PHN and OFF-PHN were elements within a multiply occurring structure, HOME-PHN in one occurrent of the structure would only be compared to OFF-PHN in that same structural occurrence. For example, Dr. Copland's HOME-PHN would NOT be compared to the OFF-PHNs of either Dr. Adams or Dr. Bax.

Dynamic Elements and Inter-element Relations

You may use dynamic elements in WHERE clauses expressing inter-element relationships, as long as the dynamic element precedes the relational operator:

"Element2" is the name of the element to which the dynamic element is to be compared. The type of "element2" must be the same as the type of the dynamic element. [See Chapter 20 of "SPIRES Technical Notes" for an explanation of dynamic elements.]

Phantom Elements and Inter-element Relations

You can use phantom elements in WHERE clauses expressing inter-element relationships, and test a primary element against any phantom element:

[See Chapter 23 of "SPIRES Technical Notes" for an explanation of phantom elements.]

4.4.4.1  (Details on Inter-element Relations)

This section will cover some of the more complex details of using inter-element relations that would not be particularly useful to the average searcher.

Depending on the file definition, there may be situations where not all element values with the same element name are compared to all the values of another element.

Assume the following record structure:

Assume further that all structure and simple elements shown above can be multiply occurring, and all simple elements are of the same type.

Any element could be described by the structural path to that element. For example: the path of S12B could be described as "S1@S12@S12B". Such a definition contains all structure names except for the last name, which is a simple element name.

Now, consider the WHERE clause: "WHERE elem1 relational-operator @elem2" where "elem1" and "elem2" represent any of the simple elements shown in the sample. To determine which occurrences of "elem1" are compared to which occurrences of "elem2", first write "elem1" and "elem2" in their full structural form. For example: "S12B > @S1A" would yield "S1@S12@S12B" and "S1@S1A". Next, choose the form with the least number of structure names (if they have the same number, choose either). Let's call this one form X and the longer one form Y.

Rules:

If either "elem1" or "elem2" (or both) are "record level" elements, then all occurrences of "elem1" are compared to all occurrences of "elem2".

If all the structure names of X match all the structure names of Y through the same number of structure names as are contained in X, then all the occurrences of "elem1" contained within the single occurrence of X's structures are compared to all the occurrences of "elem2" contained within that same occurrence of X's structures.

If all the structure names of X do not match all the structure names of Y through the same number of structure names as are contained in X, then determine the "common root" as being the structure names which do match (beginning with the first structure name of both X and Y). For the first occurrence of the structural path from the common root leading to "elem2", compare all occurrences of "elem2" in that singular occurrence to all the occurrences of "elem1" belonging to the same occurrence of the common root.

Here is an example to help you understand this last rule:

WHERE "S12A relational-operator @S11A" would make the following comparisons:

WHERE "S11A relational-operator @S12A" would make the following comparisons:

5  Record-Processing Commands Available in Global FOR Mode

About a dozen commands are used to process records in Global FOR mode. Some of them are available exclusively in Global FOR and have no meaning outside of it (SHOW KEYS, for example); others are valid outside of Global FOR but operate quite differently inside of it (e.g., DISPLAY and REMOVE). Of course, these are not the only commands that can be issued or that are useful in Global FOR. As mentioned previously, all SPIRES commands and commands of other systems can also be issued.

Here are the commands that will be discussed in this chapter:

The commands SHOW LEVELS and SET SCAN, which do not actually process records but which are useful tools under Global FOR, will be discussed in the next chapter. [See 6.]

5.1  Command Processing in Global FOR

Consider the "FOR class" command which begins Global FOR processing: the first part is the actual "FOR class" command, followed by the second part, the optional WHERE clause. It is important to bisect the command here because the two parts affect record processing at different times. The first part, "FOR class", tells SPIRES that Global FOR commands which follow will process records in the particular class. Thus, SPIRES determines which records are in the class and prepares to process them. Also at this point, if there is a WHERE clause, SPIRES determines whether the WHERE clause is syntactically correct, but not whether any records in the class actually fit its criteria. SPIRES then returns the Global FOR prompt quickly after the "FOR class" command is issued because no record processing has occurred. Only some preliminary internal setup and verification have been done. The real processing, including examination of records to see which ones fit any WHERE clause in effect, only occurs when one of the Global FOR commands listed above is issued.

These points are illustrated below in the diagram and example. You have just issued the command FOR ADDS in your FRIENDS subfile. Imagine that you will be examining the following records in the order shown:

NAME is the key element of the goal record; PHONE is an optionally occurring element. In Global FOR, records are processed in the order in which they are stored in the class; since records are stored in the deferred queue in key order, the records will be processed in the order shown in the diagram.

After issuing the FOR ADDS command, you can begin working with the records. Suppose that since you only want to examine the records, you will only need the DISPLAY command. Outside of Global FOR, using the DISPLAY command requires that you tell SPIRES the key of the record you wish to display; otherwise, SPIRES would not know what record to show you. Here you do not even need to know the record key, for you have told SPIRES in the "FOR class" command which records you want to see. You can simply issue the command

and the first record is displayed. If you then issue the command again:

then the next record is displayed to you. Eventually of course, there will be no more records:

Perhaps it seems unusual to you that each time you issue the DISPLAY command you see the next record rather than the same one over and over again. By way of comparison, for example, the TYPE command issued alone always shows you the same records, starting over from the first one each time you issue it. Remember though that the purpose of Global FOR is to simplify sequential record processing, that is, one record after another. Usually in Global FOR, you want to process one record after another and not keep starting over again (although that can be done, using the FIRST or ALL options described later or the FOR ** command). What allows the DISPLAY command and, in fact, most other Global FOR commands to process records one by one and not have to keep starting over again with each new command are the "current record pointer" and the "examination pointer" maintained in Global FOR.

The current record pointer keeps track of which record is currently being processed. As records are processed, it moves down through them, always pointing at the record currently being processed. When the "FOR class" command is issued, the pointer is "above" all the records, in the position shown in the diagram as position A. When the first DISPLAY command in the sample session is issued, the pointer moves to position B, pointing at the record being displayed. The pointer moves to position C when the next DISPLAY command is issued, and so on.

How does using a WHERE clause affect processing?

As pointed out previously, no record processing is done by the "FOR class" command, even with an appended WHERE clause. The pointer is established at position A, and SPIRES awaits a Global FOR record processing command:

SPIRES is told to display the next record of the class of added records that have exactly one occurrence of the PHONE element. Because of the WHERE clause, SPIRES cannot simply access the next record it finds and display it; instead, SPIRES must examine the next record, checking to see whether the record has a PHONE element. This is where the examination pointer becomes important. The examination pointer indicates the record currently being examined for WHERE clause conformity. Like the current record pointer, it begins at position A. When the DISPLAY command is issued, it moves to position B, and the first record, for Adams, is examined. (The current record pointer is still at position A.) Since that record does not fit the WHERE clause criteria, because it does not have a PHONE element, the examination pointer proceeds to C, pointing at the record for Buchanan. That record is then examined. When SPIRES determines that the WHERE clause is satisfied, the current record pointer is moved from A to C (bypassing B), and the record is processed according to the command. Thus, the examination pointer moves ahead of the current record pointer, which catches up to the examination pointer only when the examination pointer finds a record meeting the WHERE clause criteria. (The examination pointer also exists when there is no WHERE clause in effect, but it is really inseparable from the current record pointer in that case, since no records will actually be examined and then excluded from processing due to a WHERE clause.)

As processing continues, SPIRES keeps a count of how many records have been examined and how many of them have fit the WHERE clause criteria and been processed. You can see that information at any time by issuing the SHOW LEVELS command. [See 6.1.]

5.2  Command Options in Global FOR

You are not limited to processing only "the next record" with Global FOR. Most of the Global FOR commands discussed in this chapter have several options that allow you to specify particular records in the class that you want to process.

Here are the standard options:

If no option is used, the default is NEXT, except for the GENERATE commands, in which cases the default is ALL. Not all the options shown here are meaningful for all the Global FOR commands. The exceptions are shown in the chart below and described in the next section of this chapter.

+---------------+--------------------------------------------------+
|               |                  Command Option                  |
| Command       | FIRST |  *  | NEXT |   n   | REST | LAST | ALL   |
|---------------+-------+-----+------+-------+------+------+-------|
| DISPLAY       |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| TRANSFER      |       |     |      |  nth  | NEXT |      | FIRST |
|---------------+-------+-----+------+-------+------+------+-------|
| MERGE         |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| REMOVE        |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| DEQUEUE       |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| UNQUEUE       |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| STACK         |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| SHOW KEYS     |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| SHOW REC INFO |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| SKIP          |       |     |      |  nth  | NEXT |      | FIRST |
|---------------+-------+-----+------+-------+------+------+-------|
| REFERENCE     |       |     |      |  nth  | NEXT |      | FIRST |
|---------------+-------+-----+------+-------+------+------+-------|
| GENERATE SET  |       |     |      |   n   |      |      |       |
|---------------+-------+-----+------+-------+------+------+-------|
| GENERATE LOAD |       |     |      |   n   |      |      |       |
+---------------+-------+-----+------+-------+------+------+-------+

All boxes left blank in the chart signify options allowed on the command. The "n" indicates that SPIRES processes the next "n" records; the "nth" indicates that the "nth" record from the current one is processed. (If a WHERE clause is present, then "n" indicates that the next "n" records that fit the criteria are processed, while "nth" indicates that the "nth" record that fits the criteria from the current one is processed.)

For the "nth" commands, SPIRES substitutes NEXT for REST and FIRST for ALL, as shown in the chart.

5.2.1  The FIRST (FIR) Option

This option indicates that the first record in the class be processed. (Naturally, if a WHERE clause is in effect, this means the first record in the class that fits the WHERE clause criteria.) The current pointer is set to this record. FIRST is occasionally used with a command to reset the pointer at the beginning of the group of records so that they can all be processed again. FIRST can be specified as an option at any time while under Global FOR.

Here is an example that uses the FIRST option:

The diagram shows the position of the current record pointer after the DISPLAY FIRST command shown above is issued:

After the SHOW KEYS ALL command was issued, the current record pointer was at position F (the last record fitting the WHERE clause criterion), and the examination pointer was at position G, the last record examined. The DISPLAY FIRST command first reset both the pointers to position A; SPIRES then examined each record from the beginning of the class, finally processing (and setting the current record pointer to) the first record in the class that has one occurrence of a PHONE element; that record is the "current" record being processed. (By the way, after the last command shown, the pointers are at position D.)

5.2.2  The "*" Option

To process the current record, the "*" option can be used:

The "*" option is quite useful, for example, when you want to determine whether or not to update a record by examining it first:

Since the current record is being processed again, the current record pointer remains in the same place, indicating the "*" record.

Changing the WHERE clause in the midst of Global FOR processing by issuing a "FOR *" command will not change the current record pointer, even if the current record does not fit the WHERE clause:

Records processed by commands with options other than "*" will conform to the WHERE clause requirements.

5.2.3  The NEXT (NEX) Option

The next record past the current record (and which fits any WHERE clause in effect) is processed when this option is used. This is the default if no option is indicated for all except the GENERATE commands, as shown by many earlier examples. The current record pointer is set to the processed record.

5.2.4  The "n" Options

For most of the Global FOR commands described in this chapter, more than one record can be processed at a time. For example, more than one record can be examined with the DISPLAY command. Using some positive integer as "n", you can specify that the next "n" records be processed:

If a WHERE clause is present, all "n" records processed will naturally fit the criteria specified:

Not all the commands discussed in this chapter can be used to process more than one record at a time. For instance, it would not be very valuable to TRANSFER two records at once, since a TRANSFER command prepares SPIRES to accept exactly one record for updating. (Only one record can be updated at a time in SPIRES.) Thus, several commands (TRANSFER, MERGE, SKIP and REFERENCE) interpret the "n" option differently than "the next n records". For them, it means "the nth record past the current one". So "TRANSFER 5" tells SPIRES to transfer the fifth record from the current one, not to transfer the next five records. Of course, if a WHERE clause is in effect, then "TRANSFER 5" means that SPIRES should examine records for the WHERE clause criteria and transfer the fifth one that fits.

Whether "n" means "n" or "nth", a command using the "n" option resets the current record pointer to the last record processed by the command.

When the option used means "n", then Global FOR mode is terminated if no record can be found; otherwise, from one to "n" records will be processed:

However, if the command means "nth", Global FOR mode is terminated if the "nth" record cannot be found:

Since there was no tenth record in the class, no records can be processed. [See 7.2.]

5.2.5  The REST (RES) Option

The remaining records in the class are processed sequentially. If a WHERE clause was used in the "FOR class" command in effect, only records that fit it will be processed. The current record pointer is moved to the last record processed.

The current record pointer then indicates the last record processed; in the example above, the pointer would be at position F in the diagram. The examination pointer, since it has examined all the records in the class, is now at position G. It is only at the end of a class under WHERE clause processing that the two pointers would end up at different locations. When the EISENHOWER record was processed, both pointers were at position F. Then the examination pointer moved to position G, and SPIRES examined the next record in the class, which happened to be the last one in the class. Since that record, FILLMORE, did not have a PHONE element, the current record pointer did not move from position F.

5.2.6  The LAST (LAS) Option

The last record in the class that fits the WHERE clause criteria (if any) is processed using the LAST option. The current record pointer then indicates the last record processed.

Both pointers are now at position G in the diagram. Note that if a WHERE clause is in effect, the current record pointer indicates the last record that fits the criteria -- not necessarily the last record in the class.

The current record pointer is now at position F, and the examination pointer is at position G. SPIRES has actually examined all the records in the class (otherwise, how could SPIRES know that the Eisenhower record was the last one to fit the criteria?); this fact is reflected in the statistics provided by the SHOW LEVELS command. [See 6.1.]

Warning: Like the ALL option below, the LAST option causes SPIRES to examine all the records in the class, from the first one to the very last one, looking for the last one which fits the criteria of any WHERE clause in effect. It does not start looking for the last record from the current record; it begins all over again. As a result, the LAST option can be very expensive if you have a large subfile and a WHERE clause -- it will examine all the records in the subfile but will only process the last one it finds that fits the criteria specified.

5.2.7  The ALL Option

If you use the ALL option, all the records in the class are processed, beginning with the first record and continuing through the last. If a WHERE clause exists, all the records that fit its criteria are processed. The ALL option can be issued with a command at any time under Global FOR. The current record pointer points to the last record processed that fits any WHERE clause in effect, which is not necessarily the last record in the class, as discussed above.

The current record pointer works similarly for the REST, LAST and ALL options -- SPIRES examines all of the remaining records from the point specified (from the beginning of the class for the ALL and LAST options; from the examination pointer position for the REST option). As SPIRES examines each record, the current record pointer remains pointing at the last record examined that fits the criteria of the WHERE clause. Thus when SPIRES has examined the final record in the class, the current record pointer indicates the last record examined that fits, and this is the last record processed by the command; the examination pointer indicates the last record in the class. If you issue another command at this point, such as TRANSFER NEXT, SPIRES does not have to examine the rest of the records in the class again, since SPIRES begins examining records from the point where the examination pointer is; so you receive an "End of global FOR" message immediately, since there is no "next" record to examine. SPIRES does not automatically remove you from Global FOR mode when you process the "last" record since you may want to process the record again or reprocess the whole class using the FIRST or ALL options. [See 7.2.]

5.3  The DISPLAY (DIS) Command under Global FOR

The DISPLAY command is a very important searching and updating tool in Global FOR. Outside of Global FOR, the DISPLAY command allows you to examine single records individually without having to go through the search procedure; however, you must know the key value, which is issued with the command verb DISPLAY. Under Global FOR, the DISPLAY command works quite differently, though its function, to display records, remains the same. For instance, under Global FOR, the DISPLAY command can show you more than one record at a time. And, of course, you do not need to know the key of the record you want to display since you have already told SPIRES which records you want to process in the previous "FOR class" command.

The syntax of the DISPLAY command in Global FOR is

"DISPLAY n", where "n" is an integer, indicates that the next "n" records should be displayed. DISPLAY issued with no options is equivalent to DISPLAY NEXT. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.] The IN ACTIVE prefix is very useful with the DISPLAY command, causing the requested records to be displayed in the active file.

The DISPLAY command shows you records in their normal output format (whatever format is set), unless a SET ELEMENTS command is in effect.

The following example demonstrates the use of the DISPLAY command:

Occasionally, if you are processing many records, the active file may fill up:

Or the response may look like this:

In the first case, SPIRES has put data into your active file up to about line 99999, which is what it considers to be the limit for the active file. In such a case, the current record, "*", did not get placed into the active file. You can continue processing by clearing your active file (saving the data already there elsewhere) or by renumbering the data with a smaller delta and/or issuing the SET DELTA command. To continue processing, issue the command IN ACTIVE DISPLAY * and then IN ACTIVE DISPLAY REST. Remember that, unless you were using the CLEAN option on the DISPLAY command anyway, the single record "*" will not have a line with the record separator ("****") preceding it and a line with ";" in column 1 following it. If you do not have a WHERE clause in effect, you might want to consider the following sequence instead: FOR class, SKIP n (where "n" is the number of records already processed), IN ACTIVE DISPLAY REST. [See 5.10.]

On the other hand, if the message is the second one, then the active file really is full, according to WYLBUR. In this case, no more records can be placed in the active file at all; you must save the data elsewhere, clear your active file, and then continue processing. Unfortunately, in this case, the record "*" did not make it into the active file but possibly others did not either. (For this reason, it is usually preferable for SPIRES to tell you that there is no more room than WYLBUR, even though SPIRES is probably not absolutely correct.) The only way to tell which records have not made it into the active file is to look at the last few records in the active file. Then you can use the SET SCAN or SKIP commands to get you started with the proper record when you continue. [See 5.10, 6.2.] (Technical note: when the latter case happens, the last 4096 bytes of data did not make it into the active file. Depending on the size of the records, this information may help you figure out where to resume when you begin processing records again.)

5.4  The TRANSFER (TRA) and UPDATE (UPD) Commands under Global FOR

The TRANSFER command is often used under Global FOR when you want to update the records in a subfile. Whether issued in Global FOR or not, the TRANSFER command puts a copy of the requested record into the active file, generally in the SPIRES default format. You can then alter the record in your active file, making any corrections you want to make. SPIRES then awaits an UPDATE command, anticipating the return of the same record that was transferred.

The syntax of the TRANSFER command under Global FOR is

The TRANSFER command can only process one record at a time; the TRANSFER and UPDATE operation is used for single records. Therefore, "TRANSFER n" means to transfer the "nth" record from the current one, rather than the next "n" records. TRANSFER ALL and TRANSFER REST are invalid commands in that the action requested will not take place; however, TRANSFER ALL will transfer the first record and TRANSFER REST will transfer the "next" record. (A message announcing this action is also printed at the terminal.) [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]

The CLEAR option can be specified if there is already text in your active file and you don't want to be prompted "-OK TO CLEAR?" (assuming SET CLEAR is not in effect). The CLEAR option must follow the END clause if both are used. [See 7.3.] The KEEP option opens up a new active file without clearing the old one. (The PICK command lets you restore the previous active file.)

The UPDATE command is not really a Global FOR command, since it works no differently under Global FOR than it does outside of it. Its syntax under Global FOR remains the same:

If you are updating a record which was just transferred (the usual case), you will not need to use the "key" option. However, if you want SPIRES to update a record that you did not transfer first, you will need to use the "key" option. If you issue an UPDATE command without the "key" option, and you have not previously issued a TRANSFER command, SPIRES will ask you which record you want to update. If you issue an UPDATE command without the "key" option and you have preceded it with a TRANSFER command, the key of the record in your active file must match the key of the record last transferred; otherwise, the request will be aborted.

Here are some examples of the TRANSFER and UPDATE commands in Global FOR:

Outside of Global FOR, whenever you transfer a record, you always know that the copy that is placed in the active file is the latest copy of a record -- that is, if the record has been updated earlier in the day, you will be updating the most recent copy from the deferred queue and not the older "out of date" copy from the tree. [See 2.3.] However, the source for the transferred record under Global FOR is the class specified in the "FOR class" command. Be sure that you know where the copy comes from before you update the record; otherwise, you may be redoing or nullifying any updating work done since the file was last processed.

5.5  The MERGE (MER) Command in Global FOR

The MERGE command in Global FOR causes records in the class specified by the "FOR class" to be updated with the contents of the active file, or with data under the control of a merge input format. For information about how a specific merge format works, you should contact the format's designer, or the file owner. [See the manual "SPIRES Formats" for information about writing formats for merging records.]

The format of the command is

MERGE [FIRST|*|NEXT|n|REST|LAST|ALL] [USING line-range] [END='command']

The "n" option signifies that the next "n" records will be processed by the command. By default, MERGE with no option indicates that the next record be processed.

The USING option indicates a range of lines in the active file that are to be merged into the record. If the option is not present, the entire active file is merged. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]

The command also has three other options, including the prefixes WITH FAST and WITH SEARCHONLY, which can be used by the file owner and by users with Master access to the file to speed up the processing. See the descriptions of WITH FAST and WITH SEARCHONLY in the following section on the REMOVE command. [See 5.6.]

Another option is WITH DATA, which says that the data to use for record input is part of the MERGE command itself. This option is explained in detail in the manual "SPIRES Technical Notes"; online, EXPLAIN WITH DATA.

Here is an example of the MERGE command. Using your FRIENDS subfile, you want to merge the same phone number into any records you added earlier without a phone number:

In the example, you have merged the phone number into all the records being added that have no phone number element. Since there were no more, when you tried to merge another record, you were removed from Global FOR. [See "SPIRES Searching and Updating", section D.5.4, for more information about using the MERGE command.]

5.6  The REMOVE (REM) Command in Global FOR

In Global FOR, the "physical" effect of the REMOVE command (i.e., what actually happens to each record that is the object of the command) remains basically the same as outside Global FOR. [See "Searching and Updating", Chapter D.4.] For each record that is the object of the REMOVE command, a removal request is placed in the deferred queue; although the record is still physically stored in the file, normal record display techniques ("DISPLAY key" or TYPE) cannot access it. Whenever the file is next processed, records that were named in a remove request are physically removed from the subfile.

What makes the REMOVE command distinctive (and dangerous for beginners) in Global FOR, is that the command can be used to remove records in groups as well as one by one.

The form of the REMOVE command in Global FOR is as follows:

If no option is specified, NEXT is assumed as the default. The "n" option, if used, indicates that the next "n" records are to be removed. [See 7.3 for an explanation of the END option, 5.2 for more information about the other options.]

[See below for information about the WITH FAST and WITH SEARCHONLY prefixes, also allowed on the REMOVE command.]

For example, suppose you want to remove all the records in the RESTAURANT subfile that have not been updated since 1975:

Since the REMOVE command under Global FOR is so powerful, SPIRES issues an informational message telling you how many records are actually processed by the command. Suppose, for example, that a user issued these commands:

It is sometimes easy to forget that you are under Global FOR and that commands will operate differently because of it. The user wanted to remove the record whose key was 214; but under Global FOR, the user actually requested that the next 214 records in the class be removed. (There were only 65 records in the class left.) It is important to be able to distinguish between the "REMOVE n" command under Global FOR where "n" is an integer, and the "REMOVE key" command outside of Global FOR, especially if the keys of records are integers. By the way, the problem above can be easily corrected, using the FOR REMOVES and the UNQUEUE or DEQUEUE commands. [See 2.7, 5.7, 5.7a.]

(*) The WITH FAST Prefix

If you are either the file owner or a user with Master access to the file, you can add the WITH FAST or WITH SEARCHONLY prefix to the REMOVE command to speed up the removal processing.

WITH FAST removes the records three to four times faster, in terms of processing costs, than the normal REMOVE command. You can use it under these conditions:

 - You must have Master access to the file.
 - The deferred queue must be empty when you issue your WITH FAST request.
 - The file must have a CKPT data set of its own.  (If the file does not, then adding either a
 CHECKPOINT or IMMEDIATE statement to the file definition and recompiling will  create  one.
 See  the manual "SPIRES File Definition" for information about those statements.)
 - You are willing to forego the updating of immediate indexes as part of the removal process;
 if the subfile has immediate indexes, you will have to process the file in SPIBILD  to  get
 the  indexing  done,  and  to  allow  other  users  to  use  the subfile for updates again.

During the processing, the file will be placed in SET SEARCH mode, so that no one else may update the file. Other users attempting to update with file will get S494 errors.

If the subfile does not have any immediate indexes, then the subfile will be ready for all to use again upon completion of the command. That is, other users will again be able to update the subfile.

However, if the subfile has any immediate indexes, then SPIRES sets the NOJOBGEN flag when the REMOVE command completes. This is because no index updates (immediate indexes or not) are done by REMOVE with the WITH FAST option. The file must be processed or the DEFQ zapped before any further update requests will be honored.

(*) The WITH SEARCHONLY Option

The WITH SEARCHONLY prefix for REMOVE in SPIRES also speeds up the removal processing, though not as much as WITH FAST. However, it does not have the same requirements and restrictions. Basically, during the processing, SPIRES will reserve the file's CKPT (checkpoint) data set, so that it is not reserved and released for each removal as normal REMOVE processing is done. (That's why other users are allowed to do updates during regular INPUT BATCH processing.) Like WITH FAST, WITH SEARCHONLY puts the file into SET SEARCH mode, so that other users of the file are limited to searching and reporting.

Any immediate indexing is done as normal, and the file is available for normal updating activity upon completion of the WITH SEARCHONLY REMOVE command.

WITH SEARCHONLY, like WITH FAST, is available only to users with Master access to the file.

5.7  The DEQUEUE (DEQ) Command in Global FOR

The DEQUEUE command is used to remove transaction requests against a subfile. For instance, if a record has been updated and should not have been, the updated version can be dequeued. Like the REMOVE command, the DEQUEUE command gains additional power under Global FOR -- more than one record can be dequeued with a single command.

The DEQUEUE options are the same as those for REMOVE:

Since more than one record can be dequeued at a time, using the "n" option indicates that the next "n" records are to be dequeued. If no option is specified, NEXT is assumed. [See 7.3 for an explanation of the END option, 5.2 for more details about the other options.]

The command also has one other option, the prefix WITH SEARCHONLY, which can be used by the file owner and by users with Master access to the file to speed up the processing. See the description of WITH SEARCHONLY in the preceding section on the REMOVE command. [See 5.6.]

To correct the mistake the user made in the example in the previous section [See 5.6.] where instead of removing record 214, the user removed the next 214 records under Global FOR, you could do the following:

The reason that 65 rather than 67 records were dequeued is because only 65 records in the result had been removed -- the first two records in the result were not. If the 65 records were the only records that had been removed that day, you could instead issue the DEQUEUE ALL command under FOR REMOVES rather than FOR REMOVES VIA RESULT.

5.7a  The UNQUEUE (UNQ) Command in Global FOR

The UNQUEUE command is used to undo the previous transaction in the deferred queue for a given record. For example, if a record is updated and then updated again, the UNQUEUE command would undo the second update, leaving the record as it was updated the first time. Under Global FOR, multiple records can be unqueued with a single command:

Because multiple records can be unqueued at one time, the "n" option indicates that the next "n" records are to be unqueued. If no option is specified, NEXT is assumed. [See 7.3 for an explanation of the END option, 5.2 for more details about the other options.]

The command also has one other option, the prefix WITH SEARCHONLY, which can be used by the file owner and by users with Master access to the file to speed up the processing. See the description of WITH SEARCHONLY in the preceding section on the REMOVE command. [See 5.6.]

The UNQUEUE command provides a better solution to the problem discussed in the previous two sections where the user removed a large group of records by accident. [See 5.6, 5.7.] Here is the UNQUEUE solution:

The result of unqueuing the records could be different from the result of dequeuing them: records that had been updated before being accidentally removed would be restored to their updated state by UNQUEUE but to their tree version by DEQUEUE.

Remember that the UNQUEUE command itself creates a transaction for each record, meaning that a subsequent UNQUEUE command will "unqueue the unqueue", and so on.

5.8  The STACK (STA) Command in Global FOR

The STACK command is used to create stacks of records which can be processed either by the TYPE or SEQUENCE commands or which can be processed by Global FOR commands under a FOR STACK command. Outside of Global FOR, stacks are created by issuing the STACK command followed by the key of the record to be stacked, which means that they are built up one record at a time. (Stacks can also be created outside of Global FOR by sequencing a search result or by using the EXTRACT command.) In Global FOR mode, the STACK command can stack groups of records at one time.

The syntax of the STACK command is

Since the STACK command can process more than one record per request, "STACK n" requests that the next "n" records be stacked. By default, STACK issued alone is equivalent to STACK NEXT. The CLEAR option eliminates any previous stack you have built. [See 7.3 for an explanation of the END option, 5.2 for more information about the other options.]

A stack created under Global FOR always consists of record keys, a fact that is important to remember when you are concerned about processing efficiency. [See 8.7.] Remember that a stack does not contain the records themselves -- only the keys (or pointers, in the case of a some sequenced search results). Thus, anytime you process a stack, SPIRES accesses the latest copy of each record (unless you specify otherwise), not the record as of the time the stack was created.

Remember too that a stack can be saved, using the "STORE STACK.name" command. If you want to process the same subset of records several times over several terminal sessions, you can store the stack and later restore it.

Suppose you need a list of all the drinks in the DRINKS subfile that you have added, and the list needs to be in alphabetical order by drink name:

You must use the STACK command if you want to examine the original versions of records that have been removed since the file was last processed. [See 3.2.]

It is possible to use the STACK command during FOR STACK processing to create a second stack from the original one. SPIRES can basically only maintain one stack at a time. As soon as a STACK command is issued under a FOR STACK command, SPIRES begins building a new stack. The old stack, for most intents and purposes, is gone; any of the commands used to process a stack (e.g., TYPE, SEQUENCE) will end Global FOR processing, replacing the old stack with the new one:

The SEQUENCE command "closed" the new stack being created, replacing the old stack with this new one. If you had issued a STACK NEXT command instead of SEQUENCE, however, SPIRES would process the next record in the original stack, which is still available for Global FOR processing. As soon as you leave Global FOR mode in any way (by issuing an ENDFOR, or say, another FOR STACK command) the newly created stack replaces the old one. (On the other hand, issuing the FOR ** command lets you begin processing the old one again, not the new one.) By the way, issuing the CLEAR STACK command will eliminate both stacks.

5.8a  The UNSTACK (UNS) Command in Global FOR

Within Global FOR processing, the UNSTACK command is used to remove groups of records from an existing stack.

The syntax of the UNSTACK command is

"UNSTACK n" removes the next "n" occurrences of records in the FOR class from the stack. [See 7.3 for an explanation of the END option, 5.2 for more information about the other options.]

For example, suppose you wanted to work with a stack of all the drinks in a subfile except those drinks containing coffee:

In this example, the keys defined by the result are removed from the stack.

5.9  The SHOW KEYS (SHO KEY) Command

The SHOW KEYS command is used to examine the key values of records in the class specified in the "FOR class" command. It operates similarly to the DISPLAY command except that only record keys are shown to you. Like most of the remaining commands in this chapter, it can only be issued when you are in Global FOR mode.

The SHOW KEYS command in Global FOR has the following syntax:

If the "n" option is used, the keys of the next "n" records will be displayed. If no option is used, NEXT is assumed. [See 7.3 for an explanation of the END option, 5.2 for details about the other options.] The IN ACTIVE prefix can be used to place the keys in your active file.

Suppose you just wanted to get an idea of what records had been added to a subfile since the file was last processed:

5.10  The SKIP (SKI) Command

The SKIP command allows you to "skip to" a particular record in the specified Global FOR class -- that is, to set that record as the "current record". [See 5.2.] Because it processes records, it is affected by any WHERE clause in effect: for instance, SKIP LAST moves the current record pointer to the last record that fits the WHERE clause criteria, not necessarily the last record in the class, although all records, including the last one in the class, are examined. The SKIP command is only valid in Global FOR mode.

The SKIP command in Global FOR has this syntax:

The easiest way to understand the SKIP command is to remember that, to SPIRES, SKIP means "set the current record pointer to". Therefore, "SKIP n" tells SPIRES to set the current record pointer to the "nth" record past the current one (the "nth" record that fits the WHERE clause if one is in effect, of course). Obviously, with that definition, SKIP ALL and SKIP REST are meaningless. However, if issued, SKIP ALL is equivalent to SKIP FIRST and SKIP REST is equivalent to SKIP NEXT. (A warning message telling you what action has been taken will be issued by SPIRES in these cases.) "SKIP *" clearly has no effect on processing at all. [See 7.3 for an explanation of the END option, 5.2 for details about the other options.]

The SKIP command gives you extra flexibility in moving the record pointer around. Suppose, for instance, that you have displayed a record under Global FOR and now want to display the third record, and only the third record, from the current one:

If you had said "DISPLAY 3" instead of using the SKIP command, you would have seen the next three records, rather than just the third of the group.

SKIP is perhaps most often used as a searching tool in conjunction with the SHOW LEVELS command. [See 6.1 for an example.]

5.11  The REFERENCE (REF) Command in Global FOR

The REFERENCE command is used to bring a particular record into the main memory of the computer, usually so that partial record processing can be done. Thus, the command is primarily useful for Partial FOR processing. It is occasionally used in protocols as well, usually to access record element values using the $GETUVAL and $GETCVAL functions.

Outside of Global FOR, the REFERENCE command, like TRANSFER, is followed by the key of the record to be referenced. In Global FOR, however, the key is not used, since the records to be processed are determined by the "FOR class" command. Also, like the TRANSFER command in Global FOR, only one record can be referenced at one time.

The syntax of the REFERENCE command in Global FOR is

Since REFERENCE can only process one record per request, "REFERENCE n" references the "nth" record from the current one. REFERENCE ALL and REFERENCE REST are equivalent to REFERENCE FIRST and REFERENCE NEXT respectively. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]

5.12  The DEFINE SET (DEF SET) and GENERATE SET (GEN SET) Commands

Both of these commands, valid only in Global FOR mode, are used to create sets of records that can be sorted by the batch processor SPISORT. (SPISORT is often used when the group of records being sorted is too large for the SEQUENCE command or when more sophisticated sorting requirements are needed than can be provided by the SEQUENCE command. A set (before or after sorting) can be processed by Global FOR commands under FOR SET. [See 2.11.]

These commands are discussed in detail in "SPIRES Technical Notes".

The DEFINE SET command does not actually process any records; like the SET FORMAT $REPORT command, it sets up a particular environment for records when they are processed. The GENERATE SET command, like the other commands discussed in this chapter, does process records. Its syntax is:

Note that ALL is the default option rather than NEXT. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]

Be sure that you issue the ENDFOR command or leave Global FOR in some other way before you run a SPISORT job that uses the set -- leaving Global FOR "closes" the set, and it is not complete until that occurs. If you run a SPISORT job that executes before you have closed the set, the sorting will not be done. [See 7.]

5.13  The GENERATE LOAD (GEN LOA) Command

This command, which directs SPIRES to create an ORVYL file of SPIRES records in their internal form, is fully explained in the manual "SPIRES File Management".

The syntax of the command is:

where "loadname" is the name of the ORVYL file to be created (or replaced) by SPIRES. By default the option ALL is assumed, as is the case with the GENERATE SET command. [See 5.12.] GENERATE LOAD is an "n" command, meaning that if the "n" option is used, the next "n" records will be processed. [See 7.3 for an explanation of the END option, 5.2 for details about the other options shown here.]

6  Other Global FOR Commands

Each of the commands discussed in the previous chapter caused records to be processed under Global FOR mode. The Global FOR commands in this chapter are different from those in the preceding chapter in that they do not process records. The SHOW LEVELS command provides information about the records already processed under Global FOR. The SET SCAN commands give you extra control over the records to be examined and processed under Global FOR by letting you place restrictions on the keys of the records to be examined and processed. The commands of this chapter will help you use other Global FOR commands to process records.

6.1  The SHOW LEVELS (SHO LEV) Command

In Global FOR mode, the SHOW LEVELS command tells you how many records have been processed under the current "FOR class" command. It is very useful as a searching tool. The SHOW LEVELS command does not process any records; it only provides information about current Global FOR processing.

The SHOW LEVELS command (which takes no options) reveals three important pieces of information, as the following example shows:

First, SHOW LEVELS displays the Global FOR class, including the VIA clause, if there is one; second, it tells you the number of records (including the current one) that have been examined, found to fit the WHERE clause criteria, if any, and then processed; and third, it displays the number of records in the class that SPIRES has examined, whether or not the examined records fit the criteria. If no WHERE clause is in effect, both numbers will be the same, reflecting the number of records (through the current one) processed by SPIRES.

The above example tells you that in the class of added records, six records exist and three of them have at least one occurrence of the phone number element. If no records had been processed yet, no numbers would be displayed in response to the SHOW LEVELS command. Note that SKIP LAST tells SPIRES to examine all the records, though the current record pointer is set to the last record found that fits the criterion; thus, the SHOW LEVELS information accurately reflects the fact that all the records (not just the records up to the last one fitting the WHERE clause criteria) have been examined.

The two values shown represent the values in the two SPIRES system variables $GXCOUNT (Global FOR eXamined COUNT) and $GPCOUNT (Global FOR Processed COUNT). [See 8.2.] You should use these system variables, rather than the command IN ACTIVE SHOW LEVELS, to pass on information on Global FOR processing (e.g., to a protocol).

When used together during Global FOR searching of non-indexed elements, SKIP LAST and SHOW LEVELS can display a "result count" analogous to the result count you would get after a FIND [index-name] command. For example, you might use the two commands together in order to search for all occurrences of an address element that are too long for a mailing label:

(Remember that you could look at the single record that fits the criteria by using the "DISPLAY *" command.) If none of the records have an address element that is too long you are automatically removed from Global FOR:

The SHOW LEVELS command is also very important in partial record processing, where it provides different information.

Interrupting Global FOR Processing

Whenever you interrupt ongoing Global FOR processing by pressing the BREAK key, you will see a display of how many records had been processed and examined up to the time of interruption:

The numbers displayed can help you decide whether to continue processing or not.

Global FOR processing will also be interrupted when a specified number of CPU seconds has elapsed if the SET TIMER PAUSE command is in effect. [See B.3.6.2 in "SPIRES Searching and Updating".] For example:

6.2  The SET SCAN (SET SCA) Commands

The SET SCAN commands give you the extra ability to further subset the records being processed in Global FOR mode. Unlike WHERE clauses, which cause SPIRES to examine all records in the given class in order to prevent records not fitting the WHERE clause criteria from being processed, the different SET SCAN commands control how many or even which records are examined. Thus, the SET SCAN commands determine which records are examined; a WHERE clause determines which of the examined records are then processed.

The keys of records in the TREE and DEFQ (and thus ADDS, UPDATES and REMOVES) and SUBFILE (which merges TREE and DEFQ) classes are kept in ascending order. Several of the SET SCAN commands take advantage of this built-in indexing. For example, if you specify a starting point of the record whose key is GQ.JNK (i.e., SET SCAN START GQ.JNK) then SPIRES can pass over all the records in the tree from the beginning to the record with that value when you issue a Global FOR command that processes records, for SPIRES knows that no records before that record will have key values after "GQ.JNK". Because the record keys are readily available, SPIRES can quickly find the starting point without actually examining the records themselves (just as if using an index), which is quite an advantage over WHERE clause processing, where SPIRES must examine the records. Thus, with judicious use of the SET SCAN commands, you can limit the amount of record examination done by SPIRES, thus saving computer resources and time.

The SET SCAN commands, valid only in Global FOR mode, are

Though only one "option" can be issued at a time, several SET SCAN commands can be in effect simultaneously during a Global FOR session. They are not available when a VIA clause is used on a "FOR class" command, except under FOR SUBFILE VIA TREE. [See 3.] Each different form of the command is discussed in sections below.

The CLEAR SCAN command can be used to eliminate a particular SET SCAN command in effect:

Generally, it is not advisable to change a SET SCAN command in effect (except to CLEAR it) when you are in the middle of processing records. Instead, starting the processing over again, by issuing a "FOR **" command and then the new SET SCAN commands, is preferable. Issuing a SET SCAN START or SET SCAN PREFIX, however, will reinitialize the Global FOR processing, as if a "FOR **" had been issued. Changing a SET SCAN command in effect after records have already been processed may not produce the desired result. This caution does not apply to SET SCAN BACKWARD and SET SCAN FORWARD, described below.

Here is a chart showing the SET SCAN commands allowed under various "FOR class" commands. The SET SCAN commands generally cannot be issued when an access class or VIA clause occurs in the "FOR class" command. (There are two exceptions: SET SCAN LIMIT is allowed under any "FOR class" command; SET SCAN SKIP is allowed only under FOR RESULT or FOR STACK.)

+----------+----------------------------------------------------+
|          |                Global FOR Class                    |
| SET SCAN | TREE, SUBF, DEFQ, | RESULT, | SET, PATH, RES DATA, |
|  Option  | ADDS, UPDS, REMS  | STACK   | STORED, INDEX, LOAD  |
|----------+-------------------+---------+----------------------|
| FORWARD  |                   |   NA    |    NA                |
| BACKWARD |                   |   NA    |    NA                |
| START    |                   |   NA    |    NA                |
| STOP     |                   |   NA    |    NA                |
| PREFIX   |                   |   NA    |    NA                |
| LIMIT    |                   |         |                      |
| SKIP     |      NA           |         |    NA                |
+----------+-------------------+---------+----------------------+
 - "NA" indicates that the named SET SCAN option  is  not  available  for  these
 Global FOR classes.

6.2.1  SET SCAN FORWARD (SET SCA FOR) and SET SCAN BACKWARD (SET SCA BAC)

By default, when SPIRES processes the records in a retrieval class, it proceeds forward through key order -- that is, through the order in which the keys are kept in the class. In the TREE and DEFQ, for example, keys are kept in ascending order; in a STACK they are in the order in which they were stacked or sequenced.

There are no options on the commands:

SPIRES normally "scans forward" through the class of records during Global FOR processing; SET SCAN FORWARD is the default. However, you can also process the records in the reverse order, going "backward" through the class. To do this, issue the SET SCAN BACKWARD command:

These two commands can be used with any of the tree-structured classes (TREE, SUBFILE, DEFQ, ADDS, UPDATES, REMOVES).

As the example indicates, these commands start from the current position. If you don't have a current position established, SET SCAN BACKWARD normally starts at the end of the class, and SET SCAN FORWARD starts at the beginning of the class. But if you've done SET SCAN START, they both start from that starting point and move in the appropriate direction. Normally SET SCAN BACKWARD stops at the beginning of the class, and SET SCAN FORWARD stops at the end of the class. But if you've done SET SCAN STOP, both will stop at that stopping point. Therefore, SET SCAN BACKWARD treats SET SCAN START and SET SCAN STOP as beginning and ending points respectively, but in reverse keys order.

6.2.2  SET SCAN START (SET SCA STA)

The SET SCAN START command tells SPIRES that no records are to be processed by subsequent Global FOR commands until a key of "value" or higher is found:

For example,

The command can be used with those classes that are tree-structured (TREE, SUBFILE, DEFQ, ADDS, UPDATES, REMOVES) using any subfile. The value need not be an actual value in the class. SPIRES will process the next record after the key specified, if the specified record does not exist.

The value is processed through the key's INPROCs. If that processing fails, the command will fail too. In some cases, such as check digit processing, an INPROC may be ignored. [See 4.1.1, Processing Rules and WHERE Clauses.]

6.2.3  SET SCAN STOP (SET SCA STO)

The SET SCAN STOP command tells SPIRES that no records are to be processed by Global FOR commands after a key of "value" or higher has been reached. If it exists, the record whose key is "value" will be processed.

The value is processed through the key's INPROCs, just as it is with the SET SCAN START command. [See 4.1.] Both commands can be used with those classes that are tree-structured, using any subfile. They are often used together to specify a range of records to be examined for processing by Global FOR commands.

6.2.4  SET SCAN PREFIX (SET SCA PRE)

The SET SCAN PREFIX command specifies a value that must begin the key value of all records being examined for Global FOR processing:

It can be used with any tree-structured class.

For example,

The SHOW KEYS ALL command processed only records with keys beginning with the string "GA".

Generally, the SET SCAN PREFIX command is useful only with subfiles having keys stored as string values. You will probably not find it useful with a slot subfile, which stores the key as a binary value, for example. It is not useful with subfiles having augmented keys or record keys that are structures.

The value is processed through the key element's INPROCs (if any) when the command is issued; the command can thus fail if the value cannot be processed by the rules. [See 4.1.]

6.2.5  SET SCAN LIMIT (SET SCA LIM)

This command can be used to limit the number of records examined by SPIRES during Global FOR processing. It does not specify the number of records that will be processed; if there is a WHERE clause in effect, that number could be much smaller.

The syntax of the command is:

where "n" is an integer. Unlike all the other SET SCAN commands, the SET SCAN LIMIT command can be issued with any subfile under any "FOR class" command.

Here is an example of the command:

Though the subfile has 107 records, only ten are examined by SPIRES when the SHOW KEYS ALL command is issued. Only four keys are displayed since only four of the ten records examined fit the WHERE clause criteria, as shown in the response to the SHOW LEVELS command. Since SPIRES is allowed to examine only ten records, requesting more (SHOW KEYS REST) causes an "-End of global FOR". Note that if SHOW KEYS ALL had been issued again rather than SHOW KEYS REST, then SPIRES would begin again to do the same ten records.

6.2.6  SET SCAN SKIP (SET SCA SKI)

This command allows you to set the starting position of a Global FOR process at the "nth" entry of the result or stack. Records before the "nth" entry are skipped before any Global FOR processing, including WHERE clause handling, begins.

The syntax of the command is:

where "n" is an integer specifying the number of record pointers to be skipped. The command may only be issued under FOR RESULT and FOR STACK.

Compare these two examples:

In the first example, the SET SCAN SKIP 10 command tells SPIRES to skip the first ten records in the stack. The SKIP command in the second example tells SPIRES to skip the next ten records in the stack that fit the WHERE clause criteria. As the SHOW LEVEL commands show, the SET SCAN SKIP command does not process or examine any records; the SKIP command does. [See 5.10.]

Like some other SET SCAN commands, the SET SCAN SKIP command does not have any effect if issued after Global FOR record processing commands have been issued. For example, in this command sequence:

the SET SCAN SKIP command would be ignored.

If the stack or result contains duplicate record pointers next to each other in the stack, the extras (i.e., those after the first one) will be skipped but not included in the count.

6.2.7  SET SCAN INCREMENT (SET SCA INC)

The SET SCAN INCREMENT command gives you another option for subsetting records being processed in GLOBAL FOR mode.

The command syntax is:

As an example, if you want to Display every 10th record from a database you can issue the following commands:

The result of this process is the output of records 1, 21, 31, 41 ... from the database -- or ten percent of the file.

This command is designed to work in conjunction with other GLOBAL FOR subset processes such as WHERE Clauses, SET SCAN START, STOP, PREFIX and LIMIT options, and SPIRES SCAN Filters. The record incrementing activity takes place after all the preceding subset activity is complete.

The SHOW LEVELS command and the $GPCOUNT variable will reflect this activity in that the number of records processed is the incremental number produced. In the example above the PROCESSED count will be one tenth the value of the EXAMINED count.

This command is valid only for SPIRES Tree processing activity such as FOR SUBFILE, FOR TREE, FOR DEFQ. It is not valid during FOR STACK, RESULT, TRANSACTIONS etc. [See 6.2.6.]

The CLEAR SCAN INCREMENT command may be issued to turn the incrementing process off, or you can SET SCAN INCREMENT with an increment of 0 or 1 to clear any existing increment.

7  Leaving Global FOR Mode

Eventually, when you have finished your record processing, you will want to leave Global FOR mode. Sometimes this occurs automatically, when you have asked SPIRES to process a record that does not exist (the NEXT record after ALL have been processed, for instance). Another way is to issue the ENDFOR command, available anytime during a Global FOR session. Logging off, exiting from SPIRES or selecting another subfile will also automatically remove you from Global FOR mode. This chapter will discuss the first two ways and also explain the END clause, a useful feature in SPIRES protocols.

7.1  The ENDFOR (ENDF) Command

The ENDFOR command is used to leave Global FOR mode directly.

Note that the smallest allowed abbreviation is ENDF rather than END, which is a WYLBUR command.

The command can be issued at any time during a Global FOR session. Its effect is to remove you from Global FOR. The same subfile remains selected, any stack or search result still exists (with one exception: if you have issued the STACK command under FOR STACK, the new stack replaces the old one when you leave Global FOR mode) and any records processed (updated, for example) remain that way. SPIRES does not change the status of any records when you leave Global FOR; even if you have a record transferred when you leave Global FOR, SPIRES still awaits its return via updating:

Remember that you do not have to leave Global FOR mode (by issuing the ENDFOR command, for instance) before issuing a new "FOR class" command.

If you have to leave Global FOR mode temporarily (perhaps overnight) but would like to return later to the exact point where you leave off, you can use the SHOW LEVELS, SKIP and "FOR *" commands to save yourself time and money. For example, suppose you have issued these commands and then ended your session:

The SHOW LEVELS command tells you that SPIRES has examined 186 records, nineteen of which fit the WHERE clause, causing them to be processed. Later, when you return, you can continue from the same point by issuing these commands:

This solution is useful whether or not a WHERE clause is in effect. If no WHERE clause is in effect, you do not have to issue a "FOR *" command. (Of course, if you were working with one of the deferred queue classes, such as ADDS, or if the file has been processed since you were last working with it, new records may have been placed in the class or others removed when you return to the subfile. In this case, returning to the same point to continue processing will be more difficult.)

A more efficient method for returning to the same position can be used if the Global FOR class with which you are working allows you to use the SET SCAN START command:

Later, to return to where you were:

7.2  Leaving Global FOR Mode Automatically

As you have seen, sometimes you can be ejected from Global FOR automatically, by requesting a record that does not exist:

After you have skipped to the last record in the class, no "next record" exists, so SPIRES ejects you from Global FOR mode, thereby telling you that the requested record does not exist. If there are no records at all in the class (or no records that fit the WHERE clause, if present), the first Global FOR command to process records would cause an automatic ejection:

There is no "last" record in the class to skip to, since there are no records in the class.

Compare the two examples below. In both cases, there are only two more records to be processed in the class (although you, the user, are not aware of this).

In the left example, the two records are processed; in the right example, no record is transferred and you are ejected from Global FOR. What causes the different treatment?

The SHOW KEYS command, like DISPLAY, REMOVE and others, is an "n" command, meaning that more than one record can be processed by the command under Global FOR. The TRANSFER command, as well as REFERENCE, SKIP and MERGE, is an "nth" command, meaning that only one record can be processed by the command. The example thus shows that if the command is an "n" command, then SPIRES will process from one to "n" records in the class, depending on how many records are left to be processed in the class. As long as there is at least one, those records will be processed and SPIRES will not eject you from Global FOR. Presumably, from looking at the results, you would know that there are now no more records to be processed since only two (and not five) keys were displayed.

On the other hand, when you issue an "nth" command, you are requesting a specific, single record to be processed. If that record does not exist (the fifth of two records, in the example), then SPIRES ejects you from Global FOR, just as described above. Again, automatic ejection from Global FOR tells you that the record you requested does not exist.

7.3  The END Clause: Special Command Execution for Protocols

File owners often write protocols that use Global FOR processing. They are faced with the problem of handling automatic ejection from Global FOR mode, as described in the previous section. [See 7.2.] The users of the protocol are often unaware that they are using Global FOR; they could be quite puzzled by the "-End of global FOR" message. (In fact, from a protocol, that message is replaced by the message "-End clause missing", even more puzzling to a user.) Moreover, the file owner needs to detect that there are no more records to be processed so that subsequent commands that process the records can be skipped.

If you are familiar with the two SPIRES variables $YES and $NO, which detect the success or failure or the previous command, you might try to use them to solve the problem:

Thus, even though no record was transferred, $YES remained true ($NO remaining false) meaning that the TRANSFER command did not "fail", according to SPIRES. (The command would fail if there were no records in the class at all, or if some processing error occurred with the record -- that would set $NO to true.)

The solution which SPIRES provides is the END clause, available with all the Global FOR record processing commands:

where "command1" is a Global FOR record processing command and "command2" is any command that can be issued when you are in SPIRES, including another Global FOR command. The apostrophes are optional if "command2" is a single word; the equal sign ("=") is optional.

For example,

When a command such as one of the above is issued, the END clause is ignored unless "command1" fails because there are no more records in the class to be processed. When that happens, SPIRES ejects you from Global FOR, suppresses the "-End clause missing" message, and executes the command specified in the END clause.

The "*" command, which writes the message which follows the "*" to the user's terminal, is executed only if there are no records to be displayed.

Though the END clause can be used in command mode, it is primarily used in protocols. For example, if you want to merge all the records in a class (with or without a WHERE clause), a simple "one-line" protocol such as the one shown below can be used. (Remember that the data to be merged must be in your active file.)

In the second line of the example, you are giving SPIRES instructions to merge a record, then go back and merge the next one, and so forth; when there are no more records to merge, SPIRES returns from the protocol. [The "one-line protocol" is discussed fully in "SPIRES Protocols".]

In protocol mode, if you are in Global FOR and a record-processing command without an END clause causes you to get an "-End clause missing" message, then an ENDFOR command must be issued before any other Global FOR commands, including any "FOR class" command, can be successfully issued. This is what the situation described above looks like:

After you issue the one-line protocol (the command line beginning with the colon ":"), SPIRES takes over, issuing the commands for you. (Note that in protocol mode, when SPIRES is issuing the commands, the Global FOR prompt did not appear until you were prompted again.) Because the SHOW KEYS ALL command lacked an END clause, the subsequent FOR UPDATES command was blocked -- an ENDFOR command must be issued before any more Global FOR activity can occur.

Sometimes in a protocol you might want to protect yourself from the "-End clause missing" and "-Endfor command required" messages but not want to issue a command in an END clause. One way to solve this problem is by using the END clause:

where "-" is the SPIRES protocols comment command, which is a command that, in effect, does nothing. In this case, however, it would suppress the "-End clause missing" message if the command caused an automatic ejection from Global FOR without having any effect of its own.

Be aware that when an error really does occur, such as a processing error on a record being transferred that prevents the transfer from occurring, the $NO flag will be set by that failure and will remain set regardless of the outcome of the command listed in the END clause, which would have been executed. That way, you can test for a command failure on the last record without having to worry that the END clause command reset the value of $NO.

8  Global FOR as a Mode of Operation

Though the preceding chapters have shown you the commands of Global FOR, this chapter will cover Global FOR as a mode -- when you might want to use it, how to use combinations of commands for greater efficiency, and other information and techniques not fully discussed previously.

8.1  Global FOR as a Mode of SPIRES

When you enter Global FOR mode, as you have seen, new SPIRES commands are available (e.g., SHOW LEVELS, SKIP) while others work differently than the way they work outside of Global FOR (TRANSFER, DISPLAY). However, all other SPIRES commands, as well as those of other systems, are available under Global FOR, just as they are outside of Global FOR. For example, under a FOR TREE command you can still issue search requests (FIND command) and examine the retrieved records (TYPE). (Of course, a FIND command will affect the result being processed if you are under a FOR RESULT, and it will eliminate any stack in effect, which would affect you if you were under a FOR STACK command.) You can issue any SHOW command to display information, change the record display format (with SET ELEMENTS or SET FORMAT), add records, and so forth. None of these commands will change the status of your Global FOR processing.

You can also issue other "FOR class" commands when already in Global FOR mode. You do not have to leave Global FOR by issuing an ENDFOR command first. Of course, the "FOR *" and "FOR **" commands can be issued only when you are in Global FOR.

You cannot issue SPIRES commands that are followed by a key value, except UPDATE. For instance, if you are under Global FOR, you cannot issue the command "DISPLAY GA.JNK" if GA.JNK is the key of a record. Under Global FOR, the DISPLAY command has other options, none of which are the record key. Thus, to display (or transfer, merge, remove, dequeue or stack) a specific record by specifying its key value, you must leave Global FOR mode.

The ENDFOR command will explicitly remove you from Global FOR. There are other commands that will also remove you from Global FOR mode, even though they are not specifically Global FOR commands:

If you execute a protocol in SPIRES, it may or may not end your Global FOR processing, depending on the presence or absence of any of the above commands. Also remember that the SEQUENCE or TYPE commands will also end FOR STACK processing when you have been creating a second stack. [See 5.8.]

8.2  System Variables Associated with Global FOR

There are four useful system variables related to Global FOR processing that can be used in SPIRES formats or protocols. The variable $FORTYPE is an integer variable that tells whether or not a "FOR class" command is in effect and what class is being processed. $PRTCNT is also an integer variable. It tells you how many or what records the last Global FOR command was requested to process. $GPCOUNT and $GXCOUNT are integer variables that can give you the same values shown by the SHOW LEVELS command. In many situations, it is preferable to use them rather than a command such as IN ACTIVE SHOW LEVELS. [See 6.1.]

$FORTYPE has a value of zero when no "FOR class" command is in effect, and some other value, as shown below, when one is in effect. Thus, if a protocol needs to determine whether the user is in Global FOR mode (and leave it, if so), the following command could be included:

meaning "If $FORTYPE has any value other than "0" (which represents 'false') then execute the ENDFOR command."

Here are the values that $FORTYPE can contain:

If you issue a "FOR class" command with a VIA clause (FOR retrieval-class VIA access-class), $FORTYPE represents the retrieval class, not the access class. Remember that the other classes not shown above are really "shortcuts" representing two classes connected by a VIA clause. For instance, if FOR STACK is in effect, then $FORTYPE is "4", since FOR STACK is really FOR SUBFILE VIA STACK. [See 3.]

When commands are being issued under Global FOR (or Partial FOR), such as DISPLAY NEXT, TRANSFER or REMOVE 7, $PRTCNT (for "Partial Count") reflects the number of records to be processed. For example, if the command issued were "REMOVE 7", then $PRTCNT would contain the value "7". Here is a chart of special values $PRTCNT can contain; under the heading "Records Requested" are the values given in the command which would result in the $PRTCNT value shown, such as DISPLAY FIRST.

If $PRTCNT is any other positive number, it is the number of records that were requested to be processed, such as "7" above. Note that $PRTCNT does not reflect how many records were successfully processed; it reflects how many or which records were requested to be processed. When an ENDFOR command is issued, $PRTCNT is reset to "1" by SPIRES; otherwise, the variable is only changed when a Global FOR command, such as DISPLAY, is issued.

$PRTCNT is also very useful in partial record processing.

$GPCOUNT (for "Global Processed COUNT") and $GXCOUNT ("Global eXamined COUNT) contain the same numbers displayed by the SHOW LEVELS command:

8.3  Global FOR as a Searching Tool

Global FOR techniques will probably not be used by the novice who searches in SPIRES subfiles. Generally, using the standard searching techniques (using indexes and the FIND command) on subfiles carefully designed for such, the SPIRES newcomer can find the information wanted. If non-indexed searching is additionally necessary, the novice often can use the ALSO command. [See "SPIRES Searching and Updating", section B.5.3.2.]

Naturally the most efficient searching in SPIRES is done using indexes. Just as it is faster and easier for you to find a book on Beethoven by looking through the subject index of a library's card catalog instead of looking through every book in the library, it is more efficient for SPIRES to examine an index than to examine the goal records themselves. So, before using Global FOR with a WHERE clause to find the records to be processed, be sure that they cannot be found by an indexed search instead.

Remember that if you can do even part of your searching using indexes, you may be able to use the FOR RESULT command with a WHERE clause that specifies the non-indexed portion of the search. This is considerably more efficient than placing all of the search criteria in a WHERE clause under FOR TREE or FOR SUBFILE.

When you are searching in a subfile and want to be certain that you have accessed all the records, you might want to examine the added records. For instance, you might just display all the added records one at a time to see whether any of the records are of interest, or you could place a WHERE clause on the FOR ADDS command to insure that only the records most valuable to you are shown. (If there are many records in which you will not be interested, it is more efficient to process only those in which you will be, i.e., to use a WHERE clause.) A very simple and efficient way to see how many new records there are is:

The example shows that 27 records have been added to the subfile since the file was last processed in SPIBILD. SPIRES did very little work to provide you with this information. (Using a WHERE clause would have increased the work somewhat, of course.)

Finally, it is interesting to note that the command sequence FOR RESULT, DISPLAY ALL is equivalent to TYPE. And in fact, if updated and removed records are of no concern to you, then the command sequence FOR TREE VIA RESULT, DISPLAY ALL is slightly more efficient than the TYPE command, which does not bypass the deferred queue.

8.3.1  (Comparison of Sequential Searching Techniques)

In most cases the Global FOR command "FOR RESULT WHERE" will be more useful and efficient than an "ALSO" command. Some general rules can be stated.

 - If you have no preceding search result and want to  subset  the  goal  records  based  on
 unindexed  information,  you  must use Global FOR, but be aware that this is an expensive
 process for a large file.
 - If you have a preceding search result and want to subset the goal records  further  based
 on  unindexed  information,  you  may  use  either Global FOR RESULT or the ALSO command.

But:

 - If the search result obtained  by  specifying  unindexed  criteria  must  be  manipulated
 further  by  indexed  search  commands  (FIND,  AND,  AND  NOT,  OR),  you must use ALSO.
 - If the result obtained by specifying unindexed criteria will not be  manipulated  further
 by indexed search commands, but will be displayed, it will be about twice as efficient to
 use Global FOR.

Unlike the FIND command, the ALSO command cannot initiate a search; it must always operate on a preceding search result--in this respect it is like a qualifier. Unlike the ALSO command, the Global FOR commands need not operate via a search result, though they can. However, the Global FOR commands do not automatically provide you with a count of the number of records meeting the criteria specified. Such a count can be obtained, however, and a new set of criteria specified if the number is too small or too large. To get this count, you use the SKIP and SHOW LEVELS commands, as shown in the example below.

The following example shows this process, which involves several examinations of the goal records in the search result, and is therefore rather time-consuming and expensive:

The system's response to the SHOW LEVELS command gives two numbers: the second indicates the number of records examined--here it is 22, the same as the number of records in the search result. The first number indicates how many of the records examined met the criteria specified in the WHERE clause -- 4 for the first WHERE clause and 11 for the second.

The same results are more directly obtained by the use of the ALSO command, which gives an indication of the number of records meeting the criteria immediately, just as a FIND or other index search command does.

For example:

If further index search commands (for example, FIND, AND) are necessary, then the ALSO command must be used, since the "result" of a Global FOR command is not a set of pointers in a search result. The pointers in a search result can be combined logically with the pointers meeting the criteria specified in subsequent search commands. If, however, the records meeting the WHERE criteria are to be displayed at the terminal or placed in the active file, then Global FOR is a more efficient way to do this than the ALSO command.

Compare the two search scenarios following:

The second series of search commands is almost twice as efficient as the first. With the ALSO command, the system must read the goal records to examine the "eyes" element, then read the records meeting the criteria a second time when a TYPE command is given. With the FOR RESULT command, the record is read to examine the "eyes" element, then, while the record is still in main memory, it is displayed on the terminal. The net effect is that a record is accessed only once when FOR RESULT is used.

Finally, it should be noted that the ALSO command does not allow OCCURS or LENGTH tests, "same structure" processing, or inter-element relations. [See 4.1.1, 4.4.2, 4.5.] Only the WHERE clause on a "FOR class" command provides those capabilities.

8.4  Global FOR as an Updating Tool

Global FOR can be a very useful tool for people who maintain the data in a subfile. It is often used when a group of records fitting certain criteria must all be updated or removed. The records can perhaps be found by an indexed search and then sequenced if necessary. Then, under either FOR RESULT or FOR STACK, you could transfer and update each of the records (or merge or remove, as appropriate). (Unless you are the only person updating the subfile and have a good memory, you probably want to use FOR RESULT rather than FOR TREE VIA RESULT, for instance, to be certain that records already updated are processed in their most recent form.)

If you are updating many records under this method, your work may be interrupted, perhaps for lunch or a meeting. Two methods for "saving your place" were discussed earlier in this manual. [See 7.1.] By storing the result or stack and using the SHOW LEVELS and SKIP commands, you can usually return to the point where you left off.

One problem you might occasionally face is updating a record twice, the second time being a mistake. Using FOR TRANSACTIONS, you can retrieve the "middle" version of the record (the first version being the original, the last being the mistake). If there has been a lot of updating activity, you might want to use a WHERE clause that specifies the key element and its value:

This sequence would correct the problem as stated. Of course, if you had updated the record several times in a single day, you would want to examine each version of the record under FOR TRANSACTIONS until you found the proper one, then transfer and update that one. [See 2.8, 6.2.]

Naturally, to review your updating work, you can use the FOR DEFQ, FOR ADDS, FOR UPDATES and FOR REMOVES commands. Reviewing such work is considered part of data base management, discussed in the next section.

8.5  Global FOR as a Data Base Management Tool

One of the major tasks of the data base manager is to review the updating activity on the subfile. As data base manager, you would use FOR ADDS to examine the records being added. You could compare the updated records to their original versions by displaying them first under FOR UPDATES and then stacking the record keys and then examining the records under FOR TREE VIA STACK. You could examine the records being removed by stacking the records under FOR REMOVES and then examining them under FOR TREE VIA STACK. [See 3.2.] In any of these situations, you could dequeue any records in which you detected an error, knowing that if your "correction" were in error, you could recover from that as well, since all copies of added and updated records can be processed under FOR TRANSACTIONS and removed records that are dequeued can always be removed again later. Remember that, for file owners, the SHOW SUBFILE TRANSACTIONS command is also useful in regard to managing the updating activity.

Global FOR is also useful when you need to work directly with other record-types of a file, usually indexes. For example, suppose you wanted to have a list of the values in an index. Using the BROWSE command you would only be able to see a few at a time:

On the other hand, the file owner can directly access the index records by issuing the ATTACH command and then examining the index records as desired using Global FOR:

In the above example, after attaching the appropriate index record, you were able to treat the attached record-type as a subfile. Of course, since no indexes are defined for an index (usually), the only way to work with these records is by working with them individually, using the key of a record (DISPLAY ALTO, for instance), or by using Global FOR commands to process records, as shown above.

Working with the index records this way is often necessary in creating reports. In such cases you take advantage of the indexes maintained by SPIRES to produce a report displaying the records in the order in which they are indexed. That is, you can write a report format that displays information from each goal record in the subfile for which there is a pointer (and hence a record) in the index. For example:

Once you have created the report format, you attach the index, set the format, issue the SET REPORT command (if necessary) and then, using Global FOR commands (such as FOR TREE, IN ACTIVE DISPLAY ALL) produce the report. [See "SPIRES File Definition", section B.5.6, for information about the ATTACH command. See "SPIRES Formats", section I.3.2.2 for information about report formats.]

8.6  Some Common Situations Requiring the Use of Global FOR

This section demonstrates particular methods for handling some standard problems that are solved using Global FOR. Though some of them have been discussed already, you may find it useful to have this information together in this section as well.

To see all the records added or updated in the subfile today:

or issue the IN ACTIVE DISPLAY ALL command to place copies of the records in your active file.

To see the original records that have been updated today:

To see the actual records being removed from the subfile today:

To sequence records in a Global FOR class:

First create a stack of the records desired, and then issue the SEQUENCE command to sort them.

To process a particular record in a class:

You may be able to shrink the number of records examined by using a WHERE clause and SET SCAN commands. Then examine the records one by one, using the DISPLAY command. (You may want to make the record display shorter by issuing the SET ELEMENTS command, so that you only see certain elements of the record.) When you have found the desired record, issue the desired record-processing command with the "*" option.

8.7  How SPIRES Accesses Records in Global FOR Mode

When you issue commands to process records in Global FOR, SPIRES tries to handle your request as efficiently as possible. Knowing what information is most easily available to SPIRES in the various classes will help you choose the most efficient way of processing.

In fact, if you want to maximize efficiency, you should consider many different aspects of Global FOR processing. The class being processed, the access class of the VIA clause, the content of the WHERE clause, the Global FOR processing command, and the file structure itself are all important aspects, and each should be considered carefully if you want to save the most money. This section will present useful information about and helpful suggestions for increasing Global FOR efficiency.

Generally commands that need to access only the record key are more efficient than those that need the entire record. Thus SHOW KEYS, SKIP, REMOVE, DEQUEUE and STACK are usually more efficient than DISPLAY, TRANSFER, MERGE, REFERENCE, GENERATE SET and GENERATE LOAD. (They are always at least as efficient, and usually more efficient.) All the record processing commands are equally efficient if a WHERE clause is in effect, since SPIRES must get the whole record anyway for examination. (An exception: if the WHERE clause specifies the key element, SPIRES does not need to fetch the record, so the distinction between the two types of commands would exist. Note in this regard though that if you can use a SET SCAN command using the key rather than a WHERE clause, you are being more efficient.)

In some subfiles, goal records are not REMOVED to a residual data set but are actually stored completely in the tree. In some other subfiles, the elements that are declared in the FIXED section of the file definition, as well as the key, are in the tree while the rest of each record is stored in the residual data set. In such cases, a WHERE clause that specifies an element that is in the tree is more efficient than one that specifies an element in the residual data set. And if the entire record is in the tree, all the record processing commands are equally efficient, regardless of whether the command needs the entire record or only the key.

In regard to FOR SUBFILE processing, remember that the larger the deferred queue, the more expensive it will be to process records. However, processing the file in SPIBILD in such a case and then using FOR TREE instead would probably be more expensive than simply using FOR SUBFILE, unless you are going to process the file anyway.

Global FOR Efficiency with the Access Classes

When you create a search result, it is comprised either of keys (if the goal record data set is not REMOVED) or of pointers to the goal records (if it is REMOVED). In either case, the information contained in the result lets SPIRES access the records in the quickest and most direct way. If the goal records are in the tree, SPIRES goes directly to the tree (using the keys) to access them. If they are REMOVED, then SPIRES goes directly to them in the residual data set using the pointers, bypassing the TREE entirely. Since in either case there is no quicker way to get the key than to get the whole record (as there is for some cases of FOR TREE), all the record processing commands are equally efficient, even if a WHERE clause is in effect.

FOR DEFQ VIA RESULT, FOR UPDATES VIA RESULT and FOR REMOVES VIA RESULT are not as efficient as the same commands without the VIA clause if the goal records are REMOVED to a residual data set. Why? Because in that case the result contains pointers, not keys, and the deferred queue records are stored by key. For each record, SPIRES must use the result pointer to find the original record in the residual data set, get the key from the record, and then get the record from the deferred queue, if it exists there. This rule also applies to VIA STACK and VIA SET used with FOR DEFQ, FOR UPDATES, FOR ADDS and FOR REMOVES, if the stack or set (the set being a stored result or stack) contains pointers rather than keys.

FOR TREE VIA RESULT is somewhat more efficient than FOR RESULT (which is really FOR SUBFILE VIA RESULT) since it bypasses the deferred queue. It is important to remember that the indexes reflect the contents of the tree versions of the goal records, so that FOR RESULT may let you access records whose contents have changed so that they no longer fit the search criteria. In general, "FOR TREE VIA access-class" is more efficient than "FOR SUBFILE VIA access-class".

A stack that is created by sequencing a search result containing pointers (i.e., the goal records are REMOVED to a residual data set) is thus made up of pointers. The same rules described above for processing a result apply to such stacks.

A set created by the DEFINE SET and GENERATE SET commands always contains keys, not pointers. Hence, if the goal records processed under "FOR SET setname" are REMOVED to a residual data set, commands that process the record key rather than the entire record will be more efficient, and using only the record key in the WHERE clause, rather than other elements, is also more efficient.

Remember that a set can also be a stored stack or result, which may contain pointers instead of keys. In such cases, the record processing commands are equally efficient, whether they process the key or the entire record.

:29  SPIRES Documentation

I. Primers

II. User Language

III. Application Development

IV. Reference Guides (Cards)

V. Prism

VI. SPIRES Aids and Study Materials

VII. Other Related Documents

(The following documents are not SPIRES documents per se, but describe utilities and programs that may be useful in developing SPIRES applications.)

Obtaining Documentation

The above documents (except any marked "in preparation") may be obtained through the PUBLISH command on the Forsythe computer at Stanford University. If you do not use SPIRES at Stanford, contact your local system administrator to find out how SPIRES documents are made available there.

Updates to SPIRES Manuals

SPIRES manuals are updated regularly as changes are made to the system. This does not mean that all manuals are out of date with each new version of SPIRES. The changes to the documentation match those made to SPIRES: they are usually minor and/or transparent. Not having the most current version of a manual may mean you do not have all the most recent information about all the latest features, but the information you do have will usually be accurate.

A public subfile, SPIRES DOC NOTES, contains information about changes to SPIRES manuals. Using this subfile, you can determine whether the manual you have has been updated and if so, how significant those updates are. You need to know the date your manual was published, which is printed at the top of each page. For details on the procedure, issue the command SHOW SUBFILE DESCRIPTION SPIRES DOC NOTES.


INDEX


$FORTYPE VARIABLE   8.2
$GPCOUNT   8.2
$GXCOUNT   8.2
$NO   7.3
$PATHKEY VARIABLE   2.14
$PRTCNT VARIABLE USAGE   8.2
$YES   7.3
* CLASS IN GLOBAL FOR   2.18
                        2.1
* OPTION   5.2.2
* RECORD IN GLOBAL FOR   2.18
                         5.2.2
                         1
** CLASS IN GLOBAL FOR   2.19
                         2.1
ACCESS CLASS   3
               2.1
ACTIVE FILE IS TOO BIG   5.3
ACTIVE FILE LIMIT EXCEEDED   5.3
ADDED RECORDS   3.9
                8.6
                8.3
                3.4
                3.3
                3.2
                3.1
                2.6
                2.5
                2.4
                2.3
                2.2
ADDS CLASS   3.9
             8.3
             3.4
             3.3
             3.2
             3.1
             2.8
             2.5
             2.4
             2.1
ALL OPTION   5.2.7
             5.2.6
             2.4
ALSO COMMAND   4
               2.9
ALSO COMMAND, AND GLOBAL FOR   8.3.1
AND NOT OPERATOR   4.2.1
                   4.2
AND OPERATOR   4.2.1
               4.2
ASCENDING KEY ORDER   6.2.1
                      2.2
ATTACH COMMAND   8.5
BROWSE COMMAND   8.5
CLASS   2.1
CLEAR SCAN COMMAND   2.19
                     6.2
CLEAR STACK COMMAND   8.6
                      5.8
COMMAND PROMPT   1
COMPOUND WHERE CLAUSE   4.2
COMPOUND WHERE CLAUSE, NOT OPTION   4.2.2
COMPOUND WHERE CLAUSE, PARENTHESES   4.2.1
CPU TIME, MONITORING   6.1
CRITERIA CLAUSE   4.1
CRITERIA CLAUSE, COMPOUND   4.2
CRITERIA CLAUSE, COMPOUND, NOT OPTION   4.2.2
CRITERIA CLAUSE, COMPOUND, PARENTHESES   4.2.1
CURRENT RECORD   2.18
                 5.2.2
                 1
CURRENT RECORD POINTER   5.2.7
                         5.2.6
                         5.2.5
                         5.2.4
                         5.2.3
                         5.2.2
                         5.2.1
                         5.1
DEFERRED QUEUE   8.7
                 3.4
                 2.8
                 2.7
                 2.6
                 2.5
                 2.4
                 2.2
DEFINE SET COMMAND   5.12
                     2.11
DEFQ CLASS   3.9
             3.4
             3.3
             3.2
             3.1
             2.6
             2.5
             2.4
             2.1
DEQUEUE COMMAND   8.7
                  5.6
                  2.7
DEQUEUE COMMAND, IN GLOBAL FOR   5.7
DEQUEUED RECORDS   8.5
                   2.8
                   2.5
DISPLAY COMMAND   8.7
                  5.1
                  2.4
                  1
DISPLAY COMMAND, IN GLOBAL FOR   5.3
DOCUMENTATION, BIBLIOGRAPHY   :29
DYNAMIC ELEMENTS, IN WHERE CLAUSES   4.1
DYNAMIC ELEMENTS, IN WHERE CLAUSES WITH OCCURS OPTION   4.1.1
DYNAMIC ELEMENTS, INTER-ELEMENT RELATIONS   4.4.4
ELAPSED CPU TIME, MONITORING   6.1
ELEMENT ALIASES   4.4.3
                  4.1
ELEMENT FILTER, IN WHERE CLAUSES   4.4.2
ELEMENT PATH INFORMATION, FOR FORMATS   2.11
ELEMENT STRUCTURES   4.1.3
END CLAUSE   7.3
END CLAUSE MISSING   7.3
END OF GLOBAL FOR   7.2
ENDFOR COMMAND   2.19
                 8.1
                 7.1
                 5.12
                 1
EXAMINATION POINTER   5.2.7
                      5.2.6
                      5.2.5
                      5.2.1
                      5.1
FILE OWNER   4.3
             3.5
             2.12
             2.8
             2.3
             +1
FILE PROCESSING   2.3
                  2.2
                  +1
FILTERS, ON WHERE CLAUSES IN GLOBAL FOR   4.4.2
FIRST OPTION   5.2.1
FOR * COMMAND   2.18
                5.2.2
FOR ** COMMAND   2.19
FOR ADDS COMMAND   3.9
                   8.3
                   3.4
                   3.3
                   3.2
                   3.1
                   2.8
                   2.5
                   2.4
                   1
FOR CLASS COMMAND   8.1
                    2.1
                    1
FOR CLASS COMMAND, AND ALSO COMMAND   8.3.1
FOR CLASS COMMAND, VIA CLAUSE   3
FOR CLASS COMMAND, WHERE CLAUSE   4.1
                                  4
FOR DEFQ COMMAND   3.9
                   3.4
                   3.3
                   3.2
                   3.1
                   2.6
                   2.5
                   2.4
FOR GOAL COMMAND   3.6
                   2.13
FOR INDEX COMMAND   2.14
FOR LOAD COMMAND   2.17
FOR PATH COMMAND   2.15
FOR REMOVES COMMAND   3.9
                      5.7
                      3.4
                      3.3
                      3.2
                      3.1
                      2.7
FOR RESIDUAL DATA COMMAND   3.5
                            2.12
FOR RESULT COMMAND   8.4
                     8.3
                     2.9
FOR SET COMMAND   2.11
                  2.9
FOR STACK COMMAND   7.1
                    5.8
                    2.10
                    2.9
FOR STORED COMMAND   2.10a
                     2.16
FOR SUBFILE COMMAND   8.7
                      3.6
                      3.5
                      3.3
                      3.2
                      3.1
                      2.3
                      2.2
FOR TRANSACTIONS COMMAND   3.9
                           8.4
                           3.4
                           2.8
                           2.5
FOR TRANSACTIONS VIA CLAUSE   3.9
FOR TREE COMMAND   8.7
                   3.5
                   3.3
                   3.2
                   3.1
                   3
                   2.3
                   2.2
FOR UPDATES COMMAND   3.9
                      3.4
                      3.3
                      3.2
                      3.1
                      3
                      2.6
                      2.4
GENERATE LOAD COMMAND   8.7
                        5.13
GENERATE SET COMMAND   8.7
                       5.12
                       2.11
GLOBAL FOR   +1
GLOBAL FOR COMMANDS   1
GLOBAL FOR COMMANDS, * OPTION   5.2.2
GLOBAL FOR COMMANDS, ALL OPTION   5.2.7
                                  5.2.6
                                  2.4
GLOBAL FOR COMMANDS, END CLAUSE   7.3
GLOBAL FOR COMMANDS, FIRST OPTION   5.2.1
GLOBAL FOR COMMANDS, LAST OPTION   5.2.6
GLOBAL FOR COMMANDS, N OPTION   5.2.4
GLOBAL FOR COMMANDS, NEXT OPTION   5.2.3
GLOBAL FOR COMMANDS, NTH OPTION   5.2.4
GLOBAL FOR COMMANDS, REST OPTION   5.2.5
GLOBAL FOR, AND OTHER COMMANDS   8.1
GLOBAL FOR, AS DATA BASE MANAGEMENT TOOL   8.5
GLOBAL FOR, AS SEARCHING TOOL   8.3
GLOBAL FOR, AS UPDATING TOOL   8.4
GLOBAL FOR, CLASSES OF RECORDS   2.1
GLOBAL FOR, COMBINING CLASSES   8.6
GLOBAL FOR, COMMAND DEFAULTS   5.2
GLOBAL FOR, COMMAND OPTIONS   5.2
GLOBAL FOR, COMMAND PROCESSING   5.1
GLOBAL FOR, DEQUEUING RECORDS   5.7
                                5.6
GLOBAL FOR, DISPLAYING RECORDS   5.9
                                 5.3
GLOBAL FOR, EFFICIENCY CONSIDERATIONS   8.7
GLOBAL FOR, ENTERING   2.1
GLOBAL FOR, ERROR MESSAGES   7.3
                             4.3
GLOBAL FOR, EXAMINING INDEXES   8.5
GLOBAL FOR, EXAMINING REMOVED RECORDS   8.6
GLOBAL FOR, FOR CLASS COMMAND   2.1
GLOBAL FOR, KEY PROCESSING COMMANDS   8.7
GLOBAL FOR, LEAVING   2.19
                      8.1
                      7.2
                      7.1
                      7
                      5.12
                      1
GLOBAL FOR, LEAVING AUTOMATICALLY   2.19
                                    8.1
                                    7.3
                                    7.2
                                    6.2.5
                                    5.8
                                    5.2.7
                                    5.2.4
                                    2.7
                                    1
GLOBAL FOR, LIMITING PROCESSING   6.2
GLOBAL FOR, MERGING MULTIPLE RECORDS   7.3
GLOBAL FOR, OVERVIEW   1
GLOBAL FOR, PROMPT   2.2
                     1
GLOBAL FOR, RECORD EXAMINATION   5.1
GLOBAL FOR, RECORD PROCESSING COMMANDS   5
GLOBAL FOR, REMOVING RECORDS   5.6
GLOBAL FOR, RETURNING TO SAME POSITION   7.1
GLOBAL FOR, SKIPPING RECORDS   6.2.7
                               6.2.6
                               5.10
GLOBAL FOR, STACKING RECORDS   5.8
GLOBAL FOR, SYSTEM VARIABLES   8.2
GLOBAL FOR, UNQUEUING RECORDS   5.7a
GLOBAL FOR, UNSTACKING RECORDS   5.8a
GLOBAL FOR, UPDATING RECORDS   5.5
                               5.4
GOAL CLASS   2.13
             2.1
IN ACTIVE PREFIX   5.3
INDEX CLASS   3.7
              2.14
INDEXES   3.8
          3.7
          2.15
          2.14
INPROCS   6.2.4
          6.2.3
          6.2.2
          4.3
INTER-ELEMENT RELATIONS   8.3.1
INTER-ELEMENT RELATIONS IN WHERE CLAUSES   4.4.4
INTERRUPTING GLOBAL FOR PROCESSING   6.1
KEY ORDER   6.2.1
            5.1
            2.2
LAST OPTION   5.2.6
LENGTH OPTION   8.3.1
                4.1.1
LIMITING NUMBER OF OCCURRENCES WITH LENGTH OPTION   4.1.1
LOAD CLASS   2.1
LOGICAL OPERATORS   4.2.1
                    4.2
MAIN CLASS   2.1
MERGE COMMAND   8.7
MERGE COMMAND, IN GLOBAL FOR   5.5
MERGED RECORDS   5.5
                 2.4
MILTEN COMMANDS   1
N OPTION   5.2.4
NEXT OPTION   5.2.3
NOT OPTION   4.1.2
NOT OPTION IN COMPOUND WHERE CLAUSES   4.2.2
NOTATION   +2
NTH OPTION   5.2.4
OCCURS OPTION   2.19
                8.3.1
                4.1.1
OR OPERATOR   4.2.1
              4.2
OVERNIGHT FILE PROCESSING   2.2
                            +1
PARENTHESES IN WHERE CLAUSES   4.2.1
PARTIAL FOR   8.2
              6.1
              5.11
              +1
PATH CLASS   3.8
             2.15
PATH INFORMATION, FOR FORMATS   2.11
PATH PROCESSING   3.8
                  2.15
PATH, STRUCTURAL ELEMENT   4.4.4.1
                           4.4.3
PROCESSING RULES   6.2.4
                   6.2.3
                   6.2.2
                   4.3
PROTOCOLS   8.2
            8.1
            7.3
PROTOCOLS, ONE-LINE   7.3
RECORD PROCESSING   +1
RECORD PROCESSING, PARTIAL   8.2
                             6.1
                             5.11
                             +1
RECORD SEPARATOR   2.19
RECORDS, ADDED   3.9
                 8.6
                 8.3
                 3.4
                 3.3
                 3.2
                 3.1
                 2.6
                 2.5
                 2.4
                 2.3
                 2.2
RECORDS, DEQUEUED   8.5
                    2.8
                    2.5
RECORDS, MERGED   5.5
                  2.4
RECORDS, REMOVED   3.9
                   8.6
                   5.7
                   3.4
                   3.3
                   3.2
                   3.1
                   2.9
                   2.7
                   2.6
                   2.4
                   2.3
                   2.2
RECORDS, SORTING   8.6
                   2.15
                   2.14
                   2.11
RECORDS, UPDATED   3.9
                   8.6
                   8.4
                   5.4
                   3.4
                   3.3
                   3.2
                   3.1
                   2.6
                   2.5
                   2.4
                   2.3
                   2.2
REFERENCE COMMAND   8.7
REFERENCE COMMAND, IN GLOBAL FOR   5.11
RELATIONAL OPERATORS   4.1
RELATIONAL OPERATORS, INEQUALITY   4.1.3
                                   4.1.2
REMOVE COMMAND   8.7
REMOVE COMMAND, IN GLOBAL FOR   5.6
REMOVED RECORDS   3.9
                  8.6
                  5.7
                  3.4
                  3.3
                  3.2
                  3.1
                  2.9
                  2.7
                  2.6
                  2.4
                  2.3
                  2.2
REMOVES CLASS   3.9
                3.4
                3.3
                3.2
                3.1
                2.7
                2.1
REPORT FORMATS   8.5
RESIDUAL DATA CLASS   3.5
                      2.12
                      2.1
RESIDUAL DATA SET   8.7
REST OPTION   5.2.5
RESULT CLASS   8.4
               8.3
               3.1
               2.9
               2.1
RETRIEVAL CLASS   6.2.1
                  3
                  2.1
SAME STRUCTURE PROCESSING   4.4.1
SAME-STRUCTURE PROCESSING, IN WHERE CLAUSES   4.4.1
SEARCH RESULT   8.3
                3.1
                2.9
SEARCH RESULT, STORED   8.7
                        2.10a
                        2.11
                        2.9
SEQUENCE COMMAND   8.6
                   5.8
                   2.15
                   2.14
                   2.10
                   2.9
SEQUENTIAL SEARCHING, EFFICIENCY   8.3.1
SEQUENTIAL SEARCHING, TECHNIQUES, COMPARISON OF   8.3.1
SET CLASS   3.3
            2.11
            2.1
SET ELEMENT COMMAND   8.6
SET FILTER COMMAND, FOR WHERE CLAUSES   4.4.2
SET SCAN BACKWARD COMMAND   6.2.1
                            2.3
                            2.2
SET SCAN COMMANDS   2.19
                    2.18
                    6.2
                    5
                    4.3
                    2.3
                    2.2
SET SCAN FORWARD COMMAND   6.2.1
SET SCAN INCREMENT COMMAND   6.2.7
SET SCAN LIMIT COMMAND   6.2.5
SET SCAN PREFIX COMMAND   6.2.4
SET SCAN SKIP COMMAND   6.2.6
SET SCAN START COMMAND   7.1
                         6.2.2
SET SCAN STOP COMMAND   6.2.3
SET TIMER PAUSE COMMAND   6.1
SET, RECORD   8.7
              5.12
              3.3
              2.11
SHOW ELEMENT NAMES COMMAND   4.4.3
SHOW KEYS COMMAND   2.18
                    8.7
                    5.9
                    2.7
                    2.2
SHOW LEVELS COMMAND   6.2.7
                      8.3.1
                      7.1
                      6.2.6
                      6.1
                      5.10
                      5
SHOW SUBFILE TRANSACTIONS COMMAND   2.8
SIMPLE WHERE CLAUSE   4.1
SKIP COMMAND   6.2.7
               8.7
               8.3.1
               7.1
               6.2.6
               6.1
               5.10
               2.7
SLOT KEYS   6.2.4
            6.2
            2.5
            2.2
SPECIAL CLASS   2.1
SPIBILD   2.10a
SPIBILD PROCESSOR   8.7
                    2.16
                    2.3
                    +1
SPIRES PROMPT   2.2
                1
SPISORT UTILITY   5.12
                  2.14
                  2.11
STACK   8.7
        8.6
        3.2
        2.10
STACK CLASS   7.1
              3.2
              2.10
              2.1
STACK COMMAND   8.7
                8.6
                7.1
                2.10
STACK COMMAND, IN GLOBAL FOR   5.8
STACK, STORED   8.7
                2.10a
                2.11
                2.10
STORE STACK.NAME COMMAND   5.8
STORED CLASS   2.1
STRUCTURAL ELEMENT PATH   4.4.4.1
                          4.4.3
SUBFILE CLASS   8.7
                3.6
                3.5
                3.3
                3.2
                3.1
                2.3
                2.2
                2.1
SYSTEM VARIABLES   8.2
                   7.3
THROUGH PREFIX   3.8
                 2.15
TIMER FOR CPU TIME   6.1
TRANSACTIONS CLASS   3.9
                     8.4
                     3.4
                     2.8
                     2.5
                     2.1
TRANSFER COMMAND   8.7
                   7.1
                   2.8
                   1
TRANSFER COMMAND, IN GLOBAL FOR   5.4
TREE   8.7
       2.2
TREE CLASS   8.7
             3.6
             3.5
             3.3
             3.2
             3.1
             2.3
             2.2
             2.1
TYPE COMMAND   5.8
               2.10
UNQUEUE COMMAND, IN GLOBAL FOR   5.7a
UNSTACK COMMAND   5.8a
UNSTACK COMMAND, IN GLOBAL FOR   5.8a
UPDATE COMMAND   7.1
                 5.4
                 2.8
UPDATED RECORDS   3.9
                  8.6
                  8.4
                  5.4
                  3.4
                  3.3
                  3.2
                  3.1
                  2.6
                  2.5
                  2.4
                  2.3
                  2.2
UPDATES CLASS   3.9
                3.4
                3.3
                3.2
                3.1
                2.6
                2.1
VIA CLAUSE   8.2
             6.2
             2.12
             2.11
             2.10
             2.9
             2.8
             2.1
VIA CLAUSE ON FOR CLASS COMMAND   3
VIA CLAUSE, EFFICIENCY   8.7
VIA INDEX CLAUSE ON FOR CLASS COMMAND   3.7
VIA PATH CLAUSE ON FOR CLASS COMMAND   3.8
VIA RESIDUAL DATA CLAUSE ON FOR CLASS COMMAND   3.5
VIA RESULT CLAUSE   8.7
VIA RESULT CLAUSE ON FOR CLASS COMMAND   3.1
VIA SET CLAUSE   8.7
VIA SET CLAUSE ON FOR CLASS COMMAND   3.3
VIA STACK CLAUSE   8.7
VIA STACK CLAUSE ON FOR CLASS COMMAND   3.2
VIA TRANSACTIONS CLAUSE ON FOR CLASS COMMAND   3.4
VIA TREE CLAUSE   6.2
VIA TREE CLAUSE ON FOR CLASS COMMAND   3.6
WHERE CLAUSE   5.2.6
               4
               2.9
               2.3
               2.2
WHERE CLAUSE OVERFLOW   4.0.1
WHERE CLAUSE, AND SET SCAN COMMANDS   6.2
WHERE CLAUSE, AND THE SET FILTER COMMAND   4.4.2
WHERE CLAUSE, COMPOUND   4.2
WHERE CLAUSE, COMPOUND, ORDER OF CLAUSES   4.2.3
WHERE CLAUSE, EFFICIENCY   8.7
WHERE CLAUSE, ERROR MESSAGES   4.3
WHERE CLAUSE, INTER-ELEMENT RELATIONS   4.4.4
                                        8.3.1
WHERE CLAUSE, LENGTH OPTION   8.3.1
                              4.1.1
WHERE CLAUSE, LIMITS   4.0.1
WHERE CLAUSE, NOT OPTION   4.1.2
WHERE CLAUSE, NOT OPTION, COMPOUND CRITERIA   4.2.2
WHERE CLAUSE, OCCURS OPTION   2.19
                              8.3.1
                              4.1.1
WHERE CLAUSE, ON FOR REMOVES COMMAND   2.7
WHERE CLAUSE, PARENTHESES   4.2.1
WHERE CLAUSE, SAME ELEMENT OCCURRENCE   4.2.4
WHERE CLAUSE, SAME STRUCTURE PROCESSING   8.3.1
WHERE CLAUSE, SAME-STRUCTURE PROCESSING   4.4.1
WHERE CLAUSE, SIMPLE   4.1
WHERE CLAUSE, VERIFICATION OF   5.1
WHERE CLAUSE, WITH KEY ELEMENT   8.7
WHERE CLAUSE, WITH PROCESSING RULES   4.3
                                      4.1
WHERE CLAUSE, WITH STRUCTURES   4.1.3
WITH FAST PREFIX   5.6
WITH SEARCHONLY PREFIX   5.6
WYLBUR COMMANDS   1