****************************************************************** * * * Stanford Data Center * * Stanford University * * Stanford, Ca. 94305 * * * * (c)Copyright 1994 by the Board of Trustees of the * * Leland Stanford Junior University * * All rights reserved * * Printed in the United States of America * * * ****************************************************************** SPIRES (TM) is a trademark of Stanford University.
Users working with SPIRES subfiles often need to access records in ways not covered by the standard searching and updating facilities. For instance, you may need to locate records using some record criterion that is not indexed. Or you may need to update all the records having a certain non-indexed element value but do not necessarily know the keys of the records. Or you may just want to examine all the records that have been added or updated in the subfile since the file was last processed in SPIBILD.
One common link between these three situations is that you do not know or have the keys to the records you wish to handle. If you did, you could use the standard "DISPLAY key", "TRANSFER key" and "UPDATE key" commands to work with these records one at a time (though that method would be very tedious if many records were involved). These situations can all be handled with Global FOR procedures, which allow you to work with groups of records based on criteria other than their keys or those element values that appear in indexes.
Global FOR allows you to sequentially process records in a particular class, whether it be the class of records added to the subfile today, the class of records in the current search result, the class of records that have been removed today, and so on. Each of the classes can be further subsetted, depending on values in the records themselves. Many commands can be used to work with these records, the primary ones being the familiar DISPLAY and TRANSFER commands, used with new options in Global FOR.
This manual will teach you about using Global FOR in your SPIRES activity. Global FOR is discussed briefly in the manual "SPIRES Searching and Updating", sections B.5.3.3, D.3 and D.4. That material is sufficient for many users; this manual is a reference of the complete capabilities, especially written for advanced users, file owners and programmers. Because you should understand the material in that manual to understand the information here, reading that manual is a prerequisite. From time to time, there are sections here that are more advanced (say, a section requiring some knowledge of SPIRES protocols) or generally more useful to file owners but these sections have been marked accordingly. Remember that if there is a term or subject you do not understand, you can issue the EXPLAIN command in SPIRES for information.
Many of the concepts and tools used to sequentially process records under Global FOR are also used in the sequential processing of parts of a record. Partial FOR processing is the subject of Chapter 12 of the manual "SPIRES Technical Notes".
Two points of terminology need to be explained. First, when the term "process" is used in this manual (most often appearing with the word "records"), it means "working with" records using the SPIRES commands available in Global FOR; when you process records, you may be displaying, transferring, removing or in some other way working with the records. On the other hand, the phrase "processing a file in SPIBILD" occasionally appears, referring to that procedure where records in the deferred queue are moved into the goal record tree and the indexes are updated accordingly. Thus, processing records in Global FOR and processing a file in SPIBILD are two different activities. In this manual, "processing" refers to records, not files, unless stated otherwise explicitly.
Related to the previous point, the second point of terminology concerns a shortcut used throughout SPIRES documentation. Most files are processed in SPIBILD every night automatically; the only records in the deferred queue on any given day are those placed there that day. Other files are processed more often (say, twice a day) or less often (once a month) by their owners. In all cases, for any file, the records in the deferred queue have been put there "since the file was last processed". In SPIRES documentation, the word "today" is often substituted for the quoted phrase in the last sentence. So if a discussion mentions "processing all the records added today", it is talking about the records added since the file was last processed, which may or may not mean only records added today.
In SPIRES manuals, examples of sessions are shown with prompts and messages from SPIRES as they actually appear on the terminal, usually in uppercase; commands you type are shown in lowercase. You may use upper or lower case interchangeably when you are actually using SPIRES. For example:
-OK to clear? ok
Here SPIRES types "-OK to clear?" and you type "ok".
In formal command syntax descriptions, uppercase letters denote command verbs or other command elements to be entered exactly as shown; a value for lowercase terms and characters must be supplied by you. For example:
SELECT subfile-name
To use this particular command, you type the command verb "SELECT" with the name of the desired subfile, for instance, "Restaurant":
-> select restaurant
where "->" is the prompt from SPIRES.
Brackets ([]) denote optional elements. Braces ({ }) indicate that you must specify one (and only one) of the alternatives within the braces. Within the braces or brackets, a vertical line (|) separates possible choices. Neither brackets nor braces are to be typed as part of the command. For example:
SHOW SPIRES MAIL [CLEAR|NOCLEAR]
could be entered as
SHOW SPIRES MAIL or SHOW SPIRES MAIL CLEAR or SHOW SPIRES MAIL NOCLEAR
depending on whether you want the CLEAR option. Here is another example, using braces:
DEQUEUE {record-key-value|ALL}
This command must be entered either as
DEQUEUE record-key-value
where you supply the "record-key-value," or as
DEQUEUE ALL
with the choice again governed by the desired result.
Cross references to other sections of the manual are denoted by brackets, as in "[See 4.]", which refers you to chapter 4 for additional information.
To use commands in Global FOR, you must first enter the Global FOR environment by issuing a "FOR class" command, where "class" specifies a group of records to be processed sequentially. There are about a dozen main classes, such as the class of added records or the class of records in a search result, and such classes can be further subsetted in order to find records in the class fitting some particular criterion.
Taking one of the sample classes above, suppose you want to work with all the records in a subfile that have been added today, that is, since the file was last processed. After selecting the subfile, you would issue the command
-> for adds +>
"Adds" is a Global FOR class -- "FOR ADDS" tells SPIRES that you want to work with the group of records that have been added recently and which are now in the deferred queue, that area housing updates to the file, including added and updated records as well as removal requests. Most Global FOR commands issued after this "FOR class" command will affect a record or records in this class. The "FOR class" command itself does not access any records; the real work is done by record processing commands issued under Global FOR, as shown below.
When you enter the Global FOR mode, the SPIRES prompt changes from "->" to "+>". The plus sign ("+") indicates that you are in Global FOR mode and reminds you that some SPIRES commands act differently -- these commands are the Global FOR record processing commands discussed later. [See 5.] However, most other SPIRES commands are unaffected and can still be issued, as can any other system commands, such as WYLBUR or MILTEN commands.
Now that you have told SPIRES which records you want, you can begin to sequentially process them. For instance, you can DISPLAY the records, or TRANSFER and UPDATE them, or REMOVE them, DEQUEUE them, STACK them, or use some combination of these commands. Several other commands are also available for processing records in Global FOR. Many of these commands, when used outside of Global FOR, require the key value of the record, e.g., "DEQUEUE 191" or "TRANSFER GA.JNK". Under Global FOR, you will not specify the key at all, because you have told SPIRES in the "FOR class" command exactly which records you want to process. However, in Global FOR, there are other new options available for those commands, some of which are demonstrated below.
You now want to process the added records in the subfile:
-> for adds +> display <--- The first record in the class of added records is displayed. +> display <--- The next record is displayed. +> transfer * <--- You transfer the record you've just seen ("*" indicates "the current record being processed") and modify it. +> update +> show keys rest <--- The key values of all the rest of the records in the class are displayed. +> endfor -End of global FOR ->
The ENDFOR command tells SPIRES to remove you from the Global FOR mode. SPIRES may automatically eject you from Global FOR: suppose that instead of issuing the ENDFOR command after the "SHOW KEYS REST" you had done the following:
+> display <--- asking to see the next record in the class -End of global FOR ->
Because your previous command had processed all the "REST" of the records in the class, there were no more to process when the DISPLAY command was issued. SPIRES tells you that there are no more records to process by removing you automatically from Global FOR. You can start over again with another "FOR class" command, or simply continue with other commands.
The remainder of this document discusses each aspect of Global FOR in detail. The "FOR class" commands, along with various options, are discussed in the next three chapters. The Global FOR commands that process records, such as DISPLAY, SHOW KEYS and TRANSFER, are explained in chapter 5. Two other Global FOR commands, SET SCAN and SHOW LEVELS, are explained in chapter 6. Returning from the Global FOR environment, either by an ENDFOR command or an automatic ejection, is discussed in chapter 7. Further general information about using Global FOR is included in the last chapter, chapter 8.
Outside of Global FOR, most SPIRES commands used to process records of a subfile are key-dependent; that is, you must know the record's key value to process it. For instance, the MERGE command requires the key value of the record you wish to update. The beauty of Global FOR is that you do not need to know the key values of the records you want to process. Instead, the "FOR class" command defines a group of records to be processed by subsequent commands under Global FOR, and SPIRES determines the record's key values for you. All Global FOR processing begins with a "FOR class" command.
A "FOR class" command can define a very large, general set of records (such as FOR ADDS, which accesses all records in the subfile's deferred queue that have been added) or a smaller subset of records from a larger set that fit very specific criteria. (You can, for instance, examine only added records that have more than one occurrence of a telephone number element.) WHERE clauses, VIA clauses, and SET SCAN commands can all be used to subset a "FOR class" command.
This chapter will discuss the "FOR class" command without WHERE or VIA clauses, explaining each of the classes that can be specified in the command. Chapter 3 will discuss the VIA clause, which provides an alternate source for record keys. The WHERE clause, which is a very powerful option for modifying the class of records, is discussed in Chapter 4. The SET SCAN commands, which normally restrict the class of records to be processed, are issued after a "FOR class" command, and will not be discussed until Chapter 6.
A "FOR class" command can be very simple or very complex, depending on whether or not VIA or WHERE clauses are used. Here is the general syntax of the FOR command:
FOR class [VIA access-class] [WHERE criteria-clause]
There are six retrieval classes, ten access classes, and two special classes, thus making a total of eighteen different values for "class" in the "FOR class" command. The first sixteen are known as the main classes. Only the first five access classes are valid in the VIA clause, but all of them may also be specified as "class". [See Chapter 3 for an explanation of the VIA clause, and Chapter 4 for a discussion of the WHERE clause.]
Here are the different values for "class":
Each of these classes is discussed completely in the rest of this chapter.
When records are added or updated in a subfile, SPIRES places them in the deferred queue, where they stay until the file is next processed, usually sometime overnight. During processing, the records in the deferred queue are moved into the goal record data set, or "tree", where the goal records already in the subfile are kept. Because the FOR SUBFILE command prepares SPIRES to examine records in both the tree and the deferred queue, it allows you to sequentially process all records currently in the subfile in their most recent form. Thus, if a record has not been updated since the file was last processed, it will come from the tree; if it has been updated, it will come from the deferred queue; if it has been added, it will come from the deferred queue; and if it has been removed, it will not be processed at all.
The FOR SUBFILE command has the following syntax:
FOR SUBFILE [VIA access-class] [WHERE criteria-clause]
Both the VIA and WHERE clauses are discussed later in this manual. [See Chapter 3 and Chapter 4.]
Records processed under FOR SUBFILE are processed in ascending key order, the order in which they are stored in the subfile. For example, if SPIRES assigns "slot numbers" as keys for the subfile (each added record receives the next available number in the series 1, 2, 3, etc.), then record 15 is stored in the tree (and in the deferred queue if they are there) before record 16, but after record 14. Similarly, if the keys of the records are dates stored in their special internal form by SPIRES, then "March 1980" would be stored before "April 1980" and after "February 1980" in the tree or deferred queue. Records processed under FOR SUBFILE are thus processed in ascending key order unless a VIA clause or a SET SCAN BACKWARD command is in effect. [See 3, 6.2.1.]
For example, with the SHOW KEYS command, which displays the key values of records in the class, you do the following processing:
-> select people -> for subfile +> show key first AA.CSB +> show key next AC.JAM +> show key next AD.MQC +>
After issuing the FOR SUBFILE command, you receive the Global FOR prompt ("+>") telling you that SPIRES is ready for any Global FOR commands you want to issue. The SHOW KEY FIRST command, a Global FOR command, processes the first record in the class, whose key is "AA.CSB". Then SHOW KEY NEXT processes the next record in the class, whose key is "AC.JAM", and so forth. [See 5.9 for details on the SHOW KEYS command, used in many of the examples of this chapter.]
FOR SUBFILE should be used when you must guarantee that record updates are taken into account when records are examined in Global FOR. Since SPIRES is examining both the tree and the deferred queue, more processing occurs in FOR SUBFILE operations than in FOR TREE ones.
Because FOR SUBFILE will cause SPIRES to process all the records in the file, you might be processing thousands of records, if the subfile is a very large one. A WHERE clause, which causes SPIRES to examine each record for particular information, can increase the processing costs substantially. [See 4.] You can limit the subset of records examined by using a SET SCAN command. [See 6.]
Under FOR SUBFILE, SPIRES examines both the deferred queue and the tree simultaneously, processing the latest copy of all records in the subfile. On the other hand, the FOR TREE command prepares SPIRES to sequentially process records in the subfile without looking at records in the deferred queue. No added records will be processed, nor will the updated copies of any records; any records that have been removed since the file was last processed will still be in the tree, so they will be processed by commands under FOR TREE.
In the tree, records are stored in ascending key order, so records processed under FOR TREE are processed in that order unless a VIA clause or a SET SCAN BACKWARD command is in effect. [See 3, 6.2.1.]
The syntax of the FOR TREE command is
FOR TREE [VIA access-class] [WHERE criteria-clause]
The VIA and WHERE clauses are discussed later in this manual. [See 3, 4.]
-> select people -> for tree +> show key first AA.AJM +> show key next AA.CSB +> show key next AC.JAM +>
Here you are examining the keys of records in the tree beginning with the first one. If you compare this example to the example discussed under FOR SUBFILE [See 2.2.] you will notice that the first key shown under FOR SUBFILE was "AA.CSB", which is the second key here. Assuming that these two examples were created the same day, that indicates that the record whose key is "AA.AJM", which is in the tree, has been removed since the file was last processed and thus is not processed under FOR SUBFILE. Since the removed record remains in the tree until overnight processing, it is processed under the FOR TREE command.
The fact that FOR TREE processing does not access the deferred queue is not a disadvantage, since other Global FOR commands can. It is very useful that it does not, in fact. Occasionally you may need to examine the old version of a record that has been updated or removed since the file was processed. If the FOR TREE command did not exist, you could only access the tree copy by dequeuing the update or removal request, which would not be very convenient. [See 3.2, which describes a procedure for processing the old version of a record.]
If you do need to use the latest copy of records in the subfile, taking added, updated and removed records into account, you can use FOR SUBFILE. [See 2.2.] However, processing records under FOR SUBFILE is slightly more expensive than under FOR TREE and is unnecessary if there are no records in the deferred queue. Generally, unless many records have been added, updated or removed, FOR TREE will suffice. If you frequently process all or a number of records in a subfile for report generation or other reasons, you should coordinate that activity with the processing of the file in SPIBILD so that you can take advantage of FOR TREE processing.
Because FOR TREE means that you will be processing the entire goal record data set, then if the subfile is a very large one, you might be processing thousands of records. That can be especially expensive if you have a WHERE clause on the FOR TREE command, causing SPIRES to examine each record for some particular criteria. [See 4.] You may be able to use the SET SCAN command to subset the class of records being processed at less cost than a WHERE clause. [See 6.]
If the records you are processing under FOR TREE retrieve data from records in other subfiles or record-types (through phantom elements or subgoal processing), SPIRES will examine deferred queues for the latest copy of those records. In other words, FOR TREE means "tree-only" processing for the goal records of the selected subfile, not any subgoal records.
With the FOR DEFQ command, you can sequentially process all records that have been added or updated since the file was last processed. It is useful when you want to review the major updating activity of the day.
Here is the syntax of the FOR DEFQ command:
FOR DEFQ [VIA access-class] [WHERE criteria-clause]
The VIA and WHERE clauses are discussed later in this manual. [See 3, 4.]
For example,
-> select restaurant -> for defq +> display all <--- All updated and added records are displayed. +>
Note that the DISPLAY command here operates differently than outside of Global FOR. The ALL option, available under Global FOR, specifies that all records in the class should be processed. [See 5.2.7, 5.3.]
Although each record processed under FOR DEFQ is either an added or an updated record, SPIRES does not tell you which type each is. If it is important that you know, then FOR ADDS and FOR UPDATES can be used separately instead. [See 2.5, 2.6.]
Records are processed in key order under FOR DEFQ:
-> select restaurant -> for defq +> show keys all 10 24 39 76 +>
The records are not processed in the order in which they were modified or added but in the order in which they are stored in the deferred queue, which is ascending by key.
It may at first seem unusual that FOR DEFQ, which by name would seem to access all the record information in the deferred queue, actually accesses only the added and updated records. After all, the deferred queue also contains information about removed and merged records. As it happens, merged records are treated as updated ones in the deferred queue. Though you only have part of a record in your active file when you issue a MERGE command, SPIRES gets the latest version of the record, merges your new information into that copy, and puts the entire record into the deferred queue as an updated record. So actually the FOR DEFQ command accesses all added, updated and merged records.
Records being removed are treated quite differently, however. Unlike "adds" and "updates", where SPIRES processes an entire record, you specify only a key with the REMOVE command to cause a record to be removed. So for a removal request, the deferred queue has only the record's key. "Removes" are thus quite different from adds or updates and so are handled separately. [See 2.7.]
The records accessed by a FOR ADDS command are a subset of the records accessed by the FOR DEFQ command -- specifically, the records that have been added to the subfile since the file was last processed.
The syntax of the command is:
FOR ADDS [VIA access-class] [WHERE criteria-clause]
The VIA and WHERE clauses are discussed in later sections of this manual. [See 3, 4.]
The following example demonstrates the use of the FOR ADDS command:
-> for adds +> in active display all +> dequeue all -Dequeued: 17 records +>
After placing copies of all of the records into the active file, the user has dequeued all the added records. The records are still retrievable, however, using the FOR TRANSACTIONS command. [See 2.8.]
What about records that are added and then updated later that day? They are still considered added records to SPIRES. Whether a record is considered an add or an update depends on its relationship to the tree, not on what commands you have issued to place it in the deferred queue. A record in the deferred queue with a key matching that of a record in the tree is an update; if there is no tree version, it is an add.
Records that are added and then dequeued or removed can not be accessed under the FOR ADDS command, since they are no longer being added to the subfile, nor under the FOR REMOVES command. They can be accessed through the FOR TRANSACTIONS command. [See 2.7, the FOR REMOVES command, 2.8, the FOR TRANSACTIONS command.]
Remember that the records will be processed by key order, not by the order in which they were added. The two orders would presumably be the same if the subfile were one in which SPIRES assigned the keys numerically (a slot subfile).
Like the FOR ADDS command, the FOR UPDATES command accesses a subset of records accessible through the FOR DEFQ command -- in this case, all the updated records.
Like other "FOR class" commands, the syntax for this command is:
FOR UPDATES [VIA access-class] [WHERE criteria-clause]
The VIA and WHERE clauses are discussed in later chapters. [See 3, 4.]
-> for updates +> show keys 3 17 113 298 +>
An "update" is the latest copy from the deferred queue of a record that already exists in the tree. A record that is removed and added again later that day (with the same key as before) is also considered an updated record, not an added one or a removed one.
Records that have been added and then updated later in the day can be accessed by the FOR ADDS command, not the FOR UPDATES command. Records that have been updated and then either removed or dequeued will not be accessed by the FOR UPDATES command; FOR TREE can be used to access the original copy. The dequeued, updated copy or copies can be accessed with the FOR TRANSACTIONS command. [See 2.8.]
The records accessed by the FOR UPDATES command are processed in key value sequence (as shown in the example above), not the order in which the records were updated.
Records in the tree that have been removed by a REMOVE command are treated quite differently than adds or updates. While at least one entire record resides in the deferred queue for each record being added or updated, only the record key is in the deferred queue for a removal request. Since only limited information is available, those Global FOR commands that work with whole records cannot be used under FOR REMOVES. The REMOVES class contains the keys of only those records in the tree that have been removed. Keys of records that have been added and then removed that day are not included. Such records can be resurrected using the FOR TRANSACTIONS command. [See 2.8.]
The syntax of the FOR REMOVES command is:
FOR REMOVES [VIA access-class] [WHERE criteria-clause]
VIA and WHERE clauses are discussed in detail elsewhere. [See 3, 4.]
-> for removes +> stack all -Stack: 6 RECORDS +> for removes +> display next -End of global FOR ->
The only Global FOR record processing commands that can be issued under FOR REMOVES are DEQUEUE, SHOW KEYS, SKIP and STACK. [See 5.] All other Global FOR commands require access to the record data itself; all that is available here under Global FOR is the record key. If you try to use a "forbidden" command, such as DISPLAY, under FOR REMOVES, you will be ejected from Global FOR mode, as shown above. There is no next record that can be displayed, so SPIRES ejects you from Global FOR mode.
WHERE clauses are allowed on a FOR REMOVES command; however, the only element they can refer to is the record key. For example, if the key of a record is an element called ID, and you want to know the keys of removed records that are higher than 50:
-> for removes where ID > 50 +> show keys all 78 96 145 +>
However, if the record also has a DATE-ADDED element, it could not be used in the WHERE clause:
-> for removes where date-added after july 1979 +> dequeue all -End of global FOR ->
There is a way to examine and process records being removed, which will be explained later. [See 3.2.] You might want to compare the SET SCAN commands with the WHERE clause in a case such as the previous example. [See 6.2.]
Suppose you have just added a record to a subfile and now decide to update it. You make changes, issue an UPDATE command and then realize you should not have made the changes. Can you access the original added record? The obvious answer would seem to be that you could use the FOR ADDS command. However, as you learned in the previous two sections, the FOR ADDS command would provide you with the latest copy of the added record, which is the updated copy that you do not want.
The command that is useful here is FOR TRANSACTIONS, which accesses all the different versions of added and updated records since the file was last processed.
The syntax of the FOR TRANSACTIONS command is:
FOR TRANSACTIONS [VIA access-class] [WHERE criteria-clause]
A special group of VIA clauses are allowed on the FOR TRANSACTIONS command. [See 3.9.] The WHERE clause is discussed later. [See 4.]
-> for transactions +> show keys all 19 19 25 19 +>
Under FOR TRANSACTIONS, the records are generally processed in the order that the transactions were made, from oldest to newest, and not in key order as under most of the other "FOR class" commands. (Records may not be in exact chronological order if more than one user is updating the subfile at one time.) However, the FOR TRANSACTIONS VIA "access-class" do process the records in key order. [See 3.9.]
The example above shows that there are three different versions of record 19 and one of record 25 in the deferred queue. The file owner can see the sequence of subfile updating commands that created the records processed under FOR TRANSACTIONS by issuing the SHOW SUBFILE TRANSACTIONS command. It tells the file owner what time a record was processed, what account processed it, and so forth, so it is very handy to use with the FOR TRANSACTIONS command. It is possible that none of the three copies of record 19 is the latest version -- the record may have since been dequeued or removed, a possibility not shown under FOR TRANSACTIONS. However, if any version is the latest copy of a record, it will generally be the last version processed under FOR TRANSACTIONS.
That fact can lead to a problem. Suppose only one record has been updated in a subfile:
-> for transactions +> show keys all 19 +> transfer * <--- "*" refers to the current record processed +> update +> show keys next 19 +> show keys all 19 19 +>
The example shows that record 19 was the only record in the transactions class when FOR TRANSACTIONS processing began. However, once record 19 was transferred and updated, the new version also joined the class of transactions, becoming the "next" record to be processed. Thus, each time you update a record in the class of transactions, there is one more record put into the class. Because of this, you should beware of processing the same records over and over under FOR TRANSACTIONS. [See 5.4 for details on the TRANSFER and UPDATE commands in Global FOR.]
The FOR TRANSACTIONS command is particularly useful when you have added records during the day and then accidentally dequeued, updated or removed them. Because you can easily track all the record modifications made during the day, you can use FOR TRANSACTIONS processing to create audit trails. It is also useful on the rare occasion when the deferred queue's tree has been damaged.
The FOR TRANSACTIONS command is actually a short form of the command FOR DEFQ VIA TRANSACTIONS. Since the DEFQ class does not contain the keys of removed records, they are not processed under FOR TRANSACTIONS. However, the keys of removed records later dequeued can be accessed using the FOR REMOVES VIA TRANSACTIONS command. [See 3.4.]
The FOR RESULT command lets you process records in the current search result. Its syntax is:
FOR RESULT [WHERE criteria-clause]
No VIA clause is allowed on the FOR RESULT command, which is actually a short form of the command FOR SUBFILE VIA RESULT. [See 3.1.] The WHERE clause is discussed in a later chapter. [See 4.]
Suppose you want to remove all records in the RESTAURANT subfile that were added before 1984 and that have not been updated since then:
-> select restaurant -> find date-added before 1984 and date-updated before 1984 -Result: 3 RESTAURANT(S) -> for result +> remove all -Removed: 3 records +>
The above method is much more efficient than another way, using a WHERE clause:
-> select restaurant +> for tree where date-added before 1984 and date-updated before 1984 +> remove all -Removed: 3 records +>
Although the second method took one less command than the first, it required considerably more computing resources. In the first example, searching an index (which is very efficient) saves SPIRES from having to examine each record in the tree individually. The difference is similar to the difference between using an index in a book to find certain information and looking page by page through the book to find the same information. Obviously it is more efficient to use the index first, if possible. Remember that in the FIND command in the first example, DATE-ADDED is an index name; in the WHERE clause in the second, it is an element name. Obviously, if there were no DATE-ADDED index, the index search could not be done.
The FOR RESULT command is useful not only as an updating tool but also as a searching tool. If you have a search result that you want to subset further using unindexed information, you can use a WHERE clause with the FOR RESULT command. Remember that the ALSO command can often be used in this situation more easily, but not, in many cases, as efficiently. [See 8.3.]
Under FOR RESULT, records are processed in the order in which they would appear if the TYPE command were issued, an order which may seem haphazard. If you want the records to be processed in a particular order (key order, for instance), you can use the SEQUENCE command to arrange the result. The stack created by the SEQUENCE command can then be processed under the FOR STACK command. [See 2.10.]
The FOR RESULT command always accesses the latest copy of each record in the result. Therefore, if a record in the result has been removed since the file was last processed, the record will not be processed under the FOR RESULT command.
It is not necessary that a result exist when you issue the FOR RESULT command; a search result can be established or changed under a FOR RESULT command. If you change the result, subsequent Global FOR commands will process the new result, starting from the "top" of the result.
-> show result -No result exists -> for result +> find composer beethoven -Result: 93 SELECTIONS +> remove all -Removed: 93 records +> find composer brahms -Result: 2 SELECTIONS +> remove all -Removed: 2 records +>
A stored result can often be processed directly (without having to issue the "FIND @RESULT.name" command) by using the FOR SET command. [See 2.11.]
When you want to sequentially process records in a stack, you use the FOR STACK command:
-> select restaurant -> find cuisine chinese -Result: 16 RESTAURANTS -> sequence name -Stack: 16 RESTAURANTS -> for stack +> transfer first +>
In the example, after creating a stack of sequenced records, you begin using the TRANSFER command [See 5.4.] so that you can update the records in the stack one at a time.
The syntax of the FOR STACK command is:
FOR STACK [WHERE criteria-clause]
The WHERE clause is discussed later. [See 4.] No VIA clause is allowed on the FOR STACK command. [See 3.] The FOR STACK command operates similarly to the FOR RESULT command, except that a stack rather than a result will be processed. The stack does not have to exist prior to the FOR STACK command, but may be created (or changed) after the FOR STACK command is issued. If the stack is changed by creating or restoring a new stack, then subsequent Global FOR commands will begin processing the new stack from the "top", with one exception. The STACK command when issued while a FOR STACK command is in effect does begin creating a new stack, but Global FOR commands continue processing the original stack until that stack is exhausted or until an ENDFOR command terminates the Global FOR processing. (The TYPE or SEQUENCE commands will end Global FOR processing automatically in such cases, and process records in the new stack.) [See 5.8.]
The FOR STACK command is equivalent to FOR SUBFILE VIA STACK, meaning that all records processed will be in their latest version. [See 3.2.] A stored stack can be processed directly, without restoring it first, by using the FOR SET command. [See 2.11.]
The syntax of the FOR STORED command is:
FOR STORED setname [WHERE criteria-clause]
where "setname" is a stored result or stack.
This Global FOR command can only be used within SPIBILD. Refer to the document "File Management" or online, type EXPLAIN FOR STORED.
When you use a generated set of records (which may have been sorted by the SPISORT program), you can access that set later with a FOR SET command:
FOR SET setname [DIRECT|UNFILTERED] [WHERE criteria-clause]
where "setname" is the name of the ORVYL data set created by the DEFINE SET and GENERATE SET commands [See 5.12.] or is a stored result or stack. If the set is stored as an ORVYL data set under some account other than your own, then use the form "FOR SET ORV.gg.uuu.setname" where "gg.uuu" is the account number under which the set is stored. A WHERE clause can be appended to the FOR SET command but not a VIA clause. [See 3, 4.]
The DIRECT option tells SPIRES not to retrieve the goal record that a sort entry points to, but instead to use the data within the sort entry itself. You would use this option for processing a "direct set" previously created with the "DEFINE SET setname DIRECT" command. (Direct sets are a way of increasing the efficiency of large sorting applications, and of applications where the same set might be sorted multiple times for multiple reports. For more information see Chapter 1 of "SPIRES Technical Notes" or EXPLAIN DIRECT SET.)
The UNFILTERED option may be used to tell SPIRES not to use the "path information" in the set to filter out undesired element occurrences. [See the manual "SPIRES Technical Notes" for more details on the use of path information with the FOR SET command.]
The FOR SET command can be issued after the set has been defined and generated (and then usually sorted by the SPISORT program). Under FOR SET, records will normally be processed in their sorted order.
The "setname" can also be a stored result or stack, in either the form "RESULT.name" or "STACK.name" where name is the given name of the stored result or stack. (Again, the prefix "ORV.gg.uuu." can be added if you are using a result or stack stored under another account. If you are using a temporary result or stack, use the "TEMPORARY RESULT.name" or "TEMPORARY STACK.name" form.) Thus, you do not have to restore the stack or "find" the stored result again before issuing the "FOR class" command. The equivalent command in SPIBILD is: FOR STORED setname. [EXPLAIN FOR STORED COMMAND.]
Not all stored results can be used by the FOR SET command. Only a result stored from non-structured pointer groups is allowed. The file definition must be examined to determine whether pointer groups are structured or not, though the presence of sub-indexes and qualifiers, identified by the SHOW INDEXES command, are a good indication that they are.
Here is an example:
-> select albums -> for set stack.brahms +> in active show keys all +>
The FOR SET command is equivalent to FOR SUBFILE VIA SET, meaning that you will get the latest copies of records when you process them under this command. To access the tree copies rather than the latest ones in the deferred queue, you can issue the command "FOR TREE VIA SET setname". [See 3.3.]
Because stored stacks and results do not contain element path information, be sure that you do not try to process them under a FOR SET using a format requiring element path information. [See the manual "SPIRES Formats" for information on the PATH and NPATH options.]
The FOR RESIDUAL DATA command is a file owner tool that is designed for use on rare occasions in order to facilitate recovery when there has been file damage. It can be used only if the records in the subfile have been REMOVED to a residual data set. [See "SPIRES File Definition".] Generally it is not useful for regular file activity. It provides an alternate way to retrieve goal records without going through the tree.
The syntax of the command is like that of other "FOR class" commands:
FOR RESIDUAL DATA [WHERE criteria-clause]
A WHERE clause can be appended to the command, but not a VIA clause. [See 3, 4.]
This command is not recommended for general use because it is possible that two versions of the same record may be in the residual data set. Usually this does not happen, but there is no guarantee that it will not. When you access records through the tree, the tree knows where the correct version is. Avoiding the tree by using the FOR RESIDUAL DATA command means that the same record in two versions could be processed twice.
If you do need to use this command, you can use the alternate form FOR SUBFILE VIA RESIDUAL DATA to insure that you get the latest version of each of the records processed, though duplicates may still occur. [See 3.5.] If you issue the command for a subfile whose records are not REMOVED, subsequent Global FOR commands will fail.
The FOR GOAL command is actually a short form of the command FOR SUBFILE VIA TREE. [See 3.6 for a detailed description.]
When you want to sequentially process all the goal records having elements with values in a particular index, use the FOR INDEX command:
FOR INDEX index-name [(SEQUENCE element-list)] [WHERE clause]
where "index-name" is a search term from the list displayed by the SHOW INDEXES command. The WHERE clause is discussed later. [See 4.] The SEQUENCE clause is discussed below. No VIA clause is allowed. [See 3.]
Record sorting is often done in SPIRES with the SEQUENCE command or with the SPISORT program. For both of these processes, SPIRES must examine each goal record being sorted, extract the element or elements being sorted upon, and sort the values, creating a stack or set of pointers to the goal records. The stack or set can then be processed under Global FOR (or with the TYPE command for stacks). A typical command sequence, using the SEQUENCE command, might be:
-> select conference -> for tree +> stack all -Stack: 53 PEOPLE +> sequence state name -Stack: 53 PEOPLE +> in active type +>
But if the CONFERENCE subfile already has a STATE index, the sorting of the goal records by state is always readily available, through the FOR INDEX command:
-> select conference +> for index state (sequence name) +> in active display all
This sequence of commands probably provides the same result (see the "Notes" section below for exceptions) and is more efficient, since now SPIRES does not have to sort all the goal records by state.
Let's consider this process in more detail. Suppose the CONFERENCE subfile has goal records and index records with the following structure:
Goal record STATE index record ID (key) STATE (key) NAME GOALREC.ID STATE
Each goal record has a key element called ID, a number, as well as a person's NAME and home STATE. In the STATE index, each index record has the name of a state as the key element, and a GOALREC.ID, which is the number of a goal record containing that state name. Suppose the following index records existed:
STATE = ALABAMA; GOALREC.ID = 5; GOALREC.ID = 2; STATE = CALIFORNIA; GOALREC.ID = 1; GOALREC.ID = 7; GOALREC.ID = 6; STATE = HAWAII; GOALREC.ID = 9;
Now you issue the following commands:
-> select conference +> for index state +> set format $report State ID Name +> display all June 26, 1984 Page 1 State ID Name -------------------- ---------------- ----------------- Alabama 5 Withaban, Joe Alabama 2 Tusca, Lou Sue California 1 Heric, Ome California 7 Santa, Barbara California 6 Ricer, Ronny Hawaii 9 Aloha, Roy +>
When the DISPLAY command is issued, SPIRES examines the first index record, retrieves the goal record keys, and begins retrieving the "named" goal records, one at a time.
Note that if you use an option on a record processing command that requests an individual record (DISPLAY FIRST, for example), you are requesting an individual goal record, not all the goal records in a single index record. In other words, DISPLAY FIRST would only display record 5 in the above example, because the DISPLAY command is requesting the processing of goal records, not index records. The index records are only used to provide an access path to the goal records.
You might imagine the index to be a giant stack of records, and the clusters of record pointers in a single index record (the two for ALABAMA, for instance) to be "sub-stacks". Using the SEQUENCE clause, the goal records in a sub-stack can be further sorted, if desired. For example, the table above can be further sorted by NAME:
-> for index state (sequence name) +> display all June 26, 1984 Page 1 State ID Name -------------------- ---------------- ----------------- Alabama 2 Tusca, Lou Sue Alabama 5 Withaban, Joe California 1 Heric, Ome California 6 Ricer, Ronny California 7 Santa, Barbara Hawaii 9 Aloha, Roy +>
The goal records are still arranged in order by state, but within each state they are arranged by name as well.
The syntax of the SEQUENCE clause is similar to that of the SEQUENCE command:
(SEQUENCE element-name [(D)|(X)|(DX)] element-name ...)
where "element-name" is the name of or a valid alias for a goal record element by which further sorting is to be done. The SEQUENCE clause must be placed in parentheses. The "(D)" option may be used to request that the values be sorted in descending order rather than ascending, for that particular element. The "(X)" option may be used to request that the external form rather than the internal form of the element be used for sorting (that is, the form in which the value is displayed, rather than the form in which it is stored). The two options may be combined as shown in the syntax: "(DX)". Other element names to be used in the sorting may follow, if desired. EXPLAIN SEQUENCE COMMAND, or see the SPIRES manual "Searching and Updating", section B.4.5 for more information about the SEQUENCE command.
Notes and Considerations:
Path processing in SPIRES is a facility that allows you to have multiple subfiles selected at once, or multiple "views", through multiple formats, of a single subfile. Preliminary documentation for this facility can be found in the SPIRES manual "Technical Notes", section 14.
The FOR PATH command is part of the path processing facility in SPIRES. The FOR PATH command is similar to the FOR INDEX command, except that the pointers to the desired goal records are not found in an index to the goal records but in another subfile. Also, the FOR PATH command may be considered an extension of the FOR INDEX command in that it can perform the same function as FOR INDEX in many cases but can also restrict the index records retrieved based on element criteria. That second purpose is discussed later in this section.
Suppose that you have two subfiles, COURSES and ROOMS:
subfile COURSES subfile ROOMS COURSE.NUMBER (key) ROOM.NUMBER (key) COURSE.NAME COURSE.NUMBER DAYS NUMBER.OF.SEATS TIME ROOM.NUMBER (etc.)
The goal record elements for the ROOMS subfile include a multiply occurring element called COURSE.NUMBER, which represents the numbers of courses taught in that room, the same type of value found as the goal record key for the COURSES subfile (BIO 239, for example).
Suppose you want a report on the classes taught, listed in order by room number; courses using the same room should be listed in order by day and time. The easiest way to solve the problem is with this group of commands:
SELECT COURSES FOR SUBFILE STACK ALL SEQUENCE ROOM.NUMBER DAYS TIME SET FORMAT $REPORT ROOM.NUMBER COURSE.NUMBER DAYS TIME TYPE
However, a more efficient method is to take advantage of the ROOMS subfile, which already has the room numbers sorted.
-> - First select the primary subfile. -> select courses -> - Next establish the path for the second subfile. -> through 1 select rooms -Path established: 1 -> - Choose the records in the 2nd subfile to be examined. -> through 1 for subfile -> - Establish Global FOR mode for goal records in primary. -> for path 1, pointer=course.number (sequence days time) +> set format $report room.number course.number days time +> in active display all
The Global FOR command THROUGH 1 FOR SUBFILE specifies which records in the ROOMS subfile are to be examined. The command THROUGH 1 FOR SUBFILE establishes which records in the ROOMS will be used as "access records" to the goal records in the COURSES subfile. In this case, FOR SUBFILE indicates that all the records in the ROOMS subfile will be used. The following command, "FOR PATH 1...", indicates that goal records in the COURSES subfile will be retrieved using the records available through "path 1", and that the element COURSE.NUMBER in those records should be used as the pointer to indicate which goal record in COURSES should be retrieved. In other words, the goal records in the ROOMS subfile, arranged in key order by room, are used as index records to goal records in the COURSES subfile, even though two different subfiles, perhaps not even in the same file, are involved.
Most of the Global FOR classes can be used in the "THROUGH pathname FOR class" command; the exceptions are RESULT, STACK, INDEX and PATH. (You can circumvent the prohibition of FOR RESULT and FOR STACK here by storing a result or stack with the STORE RESULT.name or STORE STACK.name command, and then using the stored result or stack as a set: THROUGH pathname FOR SET RESULT.name, for example.) The command may have a WHERE clause or SET SCAN commands associated with it in order to limit the "index records" processed or examined. [See 4, 6.2.]
The syntax of the FOR PATH command is similar to that of the FOR INDEX command:
FOR PATH pathname, POINTER = elemname [(SEQUENCE element-list)]... ... [WHERE criteria-clause]
where "pathname" is the name or number of a path established by a previous THROUGH... SELECT command and "elemname" is the name of the element in the path-selected subfile that contains the key values of records in the primary subfile. The SEQUENCE clause has the same form as it has on the FOR INDEX command and has the same uses. [See 2.14.] The WHERE clause is discussed in a later chapter. [See 4.]
Perhaps a more common use of the FOR PATH command is in situations where you could otherwise use FOR INDEX except that you want to restrict the index records accessed based on some criteria within them. (For example, you want to have a WHERE clause on the index records accessed rather than on the goal records.)
Suppose that both COURSES and ROOMS are subfiles in the same file:
subfile COURSES subfile ROOMS COURSE.NUMBER (key) ROOM.NUMBER (key) COURSE.NAME COURSE.NUMBER DAYS NUMBER.OF.SEATS TIME ROOM.NUMBER (etc.)
Suppose also that the goal records for ROOMS are index records for COURSES (i.e., we could issue the command sequence SELECT COURSES and FOR INDEX ROOMS). Here are two possible reports that could be produced:
1) -> select courses -> through 1 select rooms -> through 1 for subfile -> through 1 set scan prefix TERMAN -> for path 1, pointer=course.number (sequence days time) +> set format $report room.number (heading = Room) +> set format * + Course.number Days Time +> display all Feb. 3, 1984 Page 1 Room Course Days Time -------------- --------------- ------------ ------------- TERMAN 011 ENG 119 M,W,F 0900-0950 TERMAN 011 ENG 223 M,W,F 1000-1150 TERMAN 113 PHY 475 T,H 0930-1045 (etc.) 2) -> select courses -> through 1 select rooms -> through 1 for subfile where number.of.seats > 100 -> for path 1, pointer=course.number (sequence days time) +> set format $report room.number (heading = Room) +> set format * + Course.number Days Time +> display all Feb. 3, 1984 Page 1 Room Course Days Time -------------- --------------- ------------ ------------- CERAS 204 EDU 114 M,W,F 1100-1200 CERAS 210 EDU 103 M,W,F 1300-1400 (etc.)
In example 1, you request a list of all classes in the Terman building (i.e., room numbers prefixed by TERMAN) in order by room number. SPIRES only accesses those index records with keys beginning with the string TERMAN when it picks up pointers to the COURSE goal records. [See 6.2 for information on the SET SCAN commands.]
In example 2, you request a list of all classes held in rooms that can hold more than 100 people, arranged in order by room number. In both examples, you tell SPIRES to use criteria in the index records to determine which goal records to process.
Note that example 1 could probably have been done with a different method: a search command (FIND ROOM PREFIX TERMAN) and a SEQUENCE command. The second example could not have been done that way, however, since the COURSES goal record did not contain the needed criteria (the size of the room). For the first example, the method shown above is at least as efficient, and possibly somewhat more efficient, than the FIND and SEQUENCE method.
The same "Notes and Considerations" discussed under FOR INDEX apply to FOR PATH. [See 2.14.] The FOR PATH command is equivalent to the command FOR SUBFILE VIA PATH, meaning that all records processed under FOR PATH will be in their latest versions. [See 3.8.]
The FOR STORED command differs from the other Global FOR commands in that it is only available in SPIBILD. It is available for file owners to use when processing files. The syntax of the commmand is:
FOR STORED {RESULT.name|STACK.name} [WHERE clause]
where "RESULT" is a stored result and "STACK" is a stored stack.
For more detailed information on the use of this command, refer to the document "SPIRES File Management" or online, type EXPLAIN FOR STORED.
The FOR LOAD command processes records that have been copied into a load file using the GENERATE LOAD command. It is available for file owners and users with See access to the file containing the selected subfile.
The syntax of the commmand is:
FOR LOAD loadname [WHERE clause]
where "loadname" is the name of the ORVYL file containing the load data. If the load is stored under a different account, be sure to use the fully qualified form: "ORV.gg.uuu.loadname".
Using FOR LOAD is a handy way to see what records are in a load; once it is generated, the only other way to examine its contents is to load it back into the file or into another file. For that reason, the SHOW KEYS command is particularly useful with FOR LOAD.
For more detailed information on the GENERATE LOAD command, refer to the document "SPIRES File Management" or online, type EXPLAIN GENERATE LOAD.
The "FOR *" command is one of two commands that are special cases in Global FOR, the other being "FOR **". The "FOR *" command usually has a WHERE clause, and is used to change WHERE clauses in the middle of a Global FOR stream. It can only be issued when some other "FOR class" command is in effect.
The syntax of the command is:
FOR * [WHERE criteria-clause]
The WHERE clause is discussed later in this manual. [See 4.] No VIA clause is allowed on the "FOR *" command. [See 3.]
-> select drinks -> for tree where ingredients occurs > 6 +> show keys 2 47 138 +> for * where ingredients occurs > 4 +> show keys rest 141 144 +> show keys all 47 72 111 138 141 144 +>
In the example, you first ask to process the records in the tree with more than 6 occurrences of the INGREDIENTS element and then ask for the key values of the first two records that fit the WHERE clause criterion. Next, you issue the "FOR *" command, indicating a different WHERE clause to be used for subsequent record processing. However, your current position "within the class" remains the same: you will not start processing records at the beginning of the class again, but will continue from the last record processed while the previous WHERE clause was in effect. The example makes that point clear. Records 47 and 138 had already been processed when the "FOR *" command was issued, and so the SHOW KEYS REST command does not show them again. But when you ask to see the keys of all the records in the tree with more than four ingredients ("SHOW KEYS ALL") SPIRES begins processing again "from the top" and records 47 and 138 are included in the list. Thus, the "FOR *" command issued in the example is not equivalent to "FOR TREE WHERE INGREDIENTS OCCURS > 4", since that command would restart you at the top of the class. [See 5.1 for a discussion of your position within the class and the "current record pointer".]
If you issue a "FOR *" command without a WHERE clause, the current WHERE clause is no longer in effect, and processing continues from the same position.
Any SET SCAN commands in effect when a "FOR *" command is issued will remain in effect under the "FOR *" command unless you issue a CLEAR SCAN command. [See 6.2.]
The "FOR **" command simply reissues the "FOR class" command that is currently in effect. It can only be issued when you are already in Global FOR mode. Its syntax is simply:
FOR **
No VIA or WHERE clause is allowed on this command. [See 3, 4.]
-> for tree where ingredients occurs > 6 +> show keys all 47 138 141 +> for ** +> in active display 2 +>
In the example, "FOR **" is equal to "FOR TREE WHERE INGREDIENTS OCCURS > 6". Thus, it starts you over again without having to reissue the identical "FOR class" command. In most cases, using the FIRST or ALL options on the subsequent Global FOR commands will probably be as effective as the "FOR **" command. [See 5.1.] For instance:
+> for ** +> in active display first +> in active display 2 +> in active continue display next
If the two commands on the right are substituted for the two commands on the left in the previous example, the result will be about the same. (The difference is that the resulting active file from the commands on the left will include a line of four asterisks, "****", before each record and a semicolon, ";", after each one, which will not happen from the single-record processing commands on the right.)
Once you have left Global FOR mode, either because you issued an ENDFOR command or because you were automatically ejected, you cannot issue the "FOR **" command. Any SET SCAN commands in effect when a "FOR **" command is issued will remain in effect under the "FOR **" command, unless you issue a CLEAR SCAN command. [See 6.2.] Alternatively, you can reissue the original "FOR class" command, which will clear any "scans" in effect.
Suppose that you want to see the original tree copies of all the updated records in the deferred queue. How would you do it? The FOR UPDATES command accesses the latest copies; the FOR TREE command would access the tree copies, but would give you all the records in the tree rather than just the updated ones. The solution requires a VIA clause on a "FOR class" command.
The VIA clause on the "FOR class" command allows you to specify that only the records in the Global FOR class whose keys (or pointers) are determined by the access class of the VIA clause are to be processed by subsequent Global FOR commands. The records in the Global FOR class are then processed in the order of the keys in the access class. In other words, the "access-class" in the VIA clause determines the keys of the records to be processed as well as the order in which they should be processed; the class in the "FOR class" part of the command determines where SPIRES should get each record. (The "class" in a "FOR class" command not having a VIA clause specifies all of that at once: the keys of the records to be processed, the order and the source.) Of course, even with a VIA clause, the "FOR class" command does not cause SPIRES to start processing records (looking for records in the "class" that are also in the "access class"). That activity does not begin until a record-processing command is issued. [See 5.]
Only some access classes discussed in the previous chapter can be used in a VIA clause (plus TREE in a special case), and only seven retrieval classes can be the Global FOR class when a VIA clause is used. [See 2.]
The syntax of the "FOR class" command using a VIA clause is:
FOR class VIA access-class [WHERE criteria-clause]
where "access-class" can be STACK, RESULT, "SET setname", TRANSACTIONS, RESIDUAL DATA, "INDEX index-name", "PATH pathname" or TREE, and where "class" can be TREE, SUBFILE, DEFQ, ADDS, UPDATES, REMOVES, or TRANSACTIONS.
There is also a separate set of VIA "access-classes" that can be used with the FOR TRANSACTIONS command. [See 3.9.]
There are over fifty combinations of "class" and "access class", not all of which are useful or meaningful. Here are some examples of "FOR class" commands with VIA clauses:
FOR TREE VIA STACK FOR SUBFILE VIA STACK FOR UPDATES VIA RESULT
For instance, FOR TREE VIA STACK, as discussed at the beginning, indicates that we want to begin processing the tree copy of records in the current stack. (Of course, strictly speaking, there are no records in a stack or result or set -- only keys or pointers. However, because accessing a result, stack or set in Global FOR usually leads you to records, we can take the liberty of referring to "the records in a result", for instance.) FOR SUBFILE VIA STACK indicates that we want to process the latest copy of each record in the stack. (FOR SUBFILE VIA STACK is equivalent to FOR STACK, so it is seldom issued in this longer form.) FOR UPDATES VIA RESULT means that we want to process the latest copy of all records being updated that are found by the search result.
Here is a chart showing the allowed combinations of VIA clauses:
+-----------+------------------------------------------------------------+ | | VIA access-class | | FOR class | RESULT | STA | SET | TRANS | RES DAT | INDEX | PATH | TREE | |-----------+--------+-----+-----+-------+---------+-------+------+------| | TREE | | | | X | D | | | X | |-----------+--------+-----+-----+-------+---------+-------+------+------| | SUBFILE | D | D | D | X | | D | D | | |-----------+--------+-----+-----+-------+---------+-------+------+------| | DEFQ | | | | D | X | | | X | |-----------+--------+-----+-----+-------+---------+-------+------+------| | ADDS | | | | | X | | | X | |-----------+--------+-----+-----+-------+---------+-------+------+------| | UPDATES | | | | | X | | | X | |-----------+--------+-----+-----+-------+---------+-------+------+------| | REMOVES | | | | | X | | | X | +-----------+--------+-----+-----+-------+---------+-------+------+------+
The positions marked with an "X" indicate combinations that are not allowed. Those marked with a "D" indicate the default combination when only the class that would be used in the VIA clause is used in a "FOR class" command (e.g., FOR SUBFILE VIA RESULT is the actual result when you issue the FOR RESULT command). The positions without a mark indicate combinations that are allowed.
The following chart shows the special set of access classes that can be used with the FOR TRANSACTIONS command:
+------------------+----------------------------------------+ | | VIA | | FOR TRANSACTIONS | DEFQ ADDS UPDATES REMOVES | +------------------+----------------------------------------+
The sections in this chapter will describe each access class as it is used in a VIA clause. Further information about the classes can be found in the previous chapter. [See 2.]
The VIA RESULT clause is used to specify that the records in the Global FOR class that are also in the current search result are to be processed by subsequent Global FOR commands. The records will be processed in the order in which their keys or pointers occur in the search result. [See 2.9.] If you change the search result during processing, the next Global FOR command will begin processing records in the new result from the top.
Below are the six possible "FOR class" commands using the VIA RESULT clause:
This is equivalent to FOR RESULT. [See 2.9.] When processing records under this command, SPIRES examines the deferred queue. The latest copy of an updated record is processed; records being removed are not processed at all.
This command is used to examine the tree versions of all the records in the search result. Thus, any records removed or updated during the day would still be processed in their tree versions. Unless the indexes are immediate indexes, remember that the indexes reflect the TREE rather than the SUBFILE copies of records, since records but not indexes may have been updated since the file was last processed.
This command will access only the records in the search result that have been added since the file was last processed. Note that this command will only yield a meaningful result if the indexes used to create the result are immediate indexes.
This command will access only the records in the search result that are being updated. The latest copy of each will be processed under this command.
Because the class DEFQ combines ADDS and UPDATES, this command is equivalent to FOR UPDATES VIA RESULT, described above.
This command is useful when you need to determine whether any records in the current search result are being removed, and if so, which ones. Remember that only a few Global FOR commands can be issued under a FOR REMOVES command. [See 2.7.] Suppose for instance that you had removed several hundred records from a subfile during the day and you needed to know whether any of them had been updated since 1979:
-> find date-updated after 1979 -Result: 196 PEOPLE -> for removes via result +> stack all -Stack: 5 PEOPLE
You could then use a FOR TREE VIA STACK command to see which records they were. [See 3.2.]
To specify that the records in a Global FOR class whose keys or pointers are in the current stack are to be processed, use the VIA STACK clause on a "FOR class" command. You can then process the records in the order in which they are stacked. The stack can be changed during your processing, but subsequent Global FOR commands will start processing from the start of the new stack rather than continuing with the old one, unless you alter the stack by issuing the STACK command. [See 5.8 for a discussion of this exception.] A stored stack can be processed using the FOR SET command or by using the VIA SET clause on a "FOR class" command.
Below are the six possible "FOR class" commands using the VIA STACK clause:
This command is equivalent to FOR STACK. [See 2.10.] The latest versions of records will be processed, meaning that records being removed will not be processed at all.
This command prepares SPIRES for processing copies of records from the tree that are in the stack. It is often used when you are trying to get the original tree copies of records being removed or updated:
-> for updates +> stack all -Stack: 5 PEOPLE +> for removes +> stack all -Stack: 13 PEOPLE +> for tree via stack +> in active display all +>
If any records in the stack have been added since the file was last processed, this command could be used to retrieve only them. If the stack was created by sequencing a search result, this command would not be very useful. [See 3.1, the FOR ADDS VIA RESULT command.]
This command would be useful if you wanted to process only the records in a stack that had been updated.
This command accesses records being added or updated that are in the current stack. It should not be confused with the FOR SUBFILE VIA STACK command (FOR STACK) which accesses the latest copy of all records in the subfile, including adds and updates, that are also in the stack. FOR DEFQ VIA STACK accesses only added and updated records in the stack.
Similar to FOR REMOVES VIA RESULT, this command is useful when you need to determine whether any records in the current stack are being removed.
Once you have created a set of records, you process those records with either the "FOR SET setname" command or a "FOR class" command that has a "VIA SET setname" clause, where "setname" is the name you chose for the set when you issued the DEFINE SET command. (Remember that the set can also be a stored stack or result.) [See 2.11.] The records in the Global FOR class that are also in the set can then be processed in the order in which their keys are arranged in the set.
The "FOR class VIA SET setname" commands are discussed in detail below:
This command works identically to the "FOR SET setname" command, providing access to the latest copies of the records specified in the set. [See 2.12.]
Using this command, you will be able to access tree copies of all records in the set. As with all FOR TREE commands, this command allows you to access records slightly more efficiently than a FOR SUBFILE command, since processing under FOR TREE does not involve checking the deferred queue for the latest version of each record. Hence, FOR TREE would be preferable to FOR SUBFILE (see below) if the file had recently been processed and there were few or no updated or removed records in the deferred queue.
The remaining four possibilities, FOR ADDS, FOR UPDATES, FOR DEFQ, and FOR REMOVES, all with "VIA SET setname" clauses are similar to their counterparts with VIA RESULT or VIA STACK clauses. [See 3.1, 3.2.] The difference is that it is the records in the named set, rather than those in the current result or stack, that are accessed in the class.
So far, each of the access classes in a VIA clause has specified a subset of records in the Global FOR class that you want to process. The VIA TRANSACTIONS clause, however, has a different effect on "FOR class" processing. A "FOR class VIA TRANSACTIONS" command specifies that subsequent commands will process the specified copies of the records in the transactions list that are in the Global FOR class. Details and examples given below for each of the commands should make this clear. Note too that the information discussed in the section on the FOR TRANSACTIONS command is relevant here. [See 2.8.]
This command is equivalent to FOR TRANSACTIONS. It accesses all the versions in the deferred queue of records that have been added or updated that day, even if they were later dequeued. Thus, if a record was added and then updated later, both copies could be accessed by this command. The copies are accessed in the order in which they were added or updated, oldest to newest. You should read about FOR TRANSACTIONS for more details. A similar command is FOR TRANSACTIONS VIA DEFQ. [See 3.9.]
This command allows you to access all copies of records added to the subfile during the day, even if they have since been dequeued. If a record is added and updated later that day, it is still considered an "add" by SPIRES; both copies can be accessed under the FOR ADDS VIA TRANSACTIONS command and neither copy with the FOR UPDATES VIA TRANSACTIONS command (discussed below). Suppose for instance that today record 17 was added, record 18 was added, record 17 was dequeued, record 18 was updated, and finally record 17 was added again:
-> for adds +> show keys all 17 18 +> for adds via transactions +> show keys all 17 18 18 17 +>
The FOR ADDS command accesses only the latest copy of the records being added, meaning that any dequeued adds will not be processed. The FOR ADDS VIA TRANSACTIONS command shown above accesses the original added versions of records 17 and 18, then the updated copy of 18 and then the newly added copy of 17. You may also find the FOR TRANSACTIONS VIA ADDS command useful. [See 3.9.]
Similar to FOR ADDS VIA TRANSACTIONS, this command allows you to access all deferred queue copies of records in the tree that have been updated during the day, even if they have since been dequeued. As noted before, FOR UPDATES does not include copies of added records that were updated later the same day. Under FOR UPDATES VIA TRANSACTIONS, records will be processed by Global FOR commands in the order in which they were updated. See also FOR TRANSACTIONS VIA UPDATES. [See 3.9.]
Using this command you can determine which records have been removed during the day, including those which have been dequeued:
-> for removes +> show keys all 5 17 +> for removes via transactions +> show keys all 17 25 25 5 +>
In the above example, the SHOW KEYS ALL command issued under FOR REMOVES shows that records 5 and 17 are being removed. However, the SHOW KEYS ALL command issued after FOR REMOVES VIA TRANSACTIONS shows that record 17 was removed first, then 25, then 25 again (that is, it was probably dequeued and later removed again) and then 5, with record 25 dequeued again at some point. Thus, the records are processed in the order in which the original transactions occurred. The FOR TRANSACTIONS VIA REMOVES command may also be useful. [See 3.9.]
Like the FOR RESIDUAL DATA command, the VIA RESIDUAL DATA clause is useful primarily to file owners. Only two "FOR class" commands have any meaning with this clause; the other four are not legal with it. The two that are meaningful are FOR TREE and FOR SUBFILE.
This is the equal of FOR RESIDUAL DATA -- both commands have the same effect. [See 2.11.] This type of processing actually does not involve the tree but since it does retrieve "tree copies" (the "original" copy in the subfile before any of today's updating activity is taken into account), the command syntax reflects the general meaning rather than the strict one.
This command gives you access to the latest copy of each of the records in the residual data set. If the record has been updated today, the most recent version will be processed; if the record has been removed today, the record will not be processed. Otherwise, the "tree copy" will be processed. No added records are processed since they are not in the residual data set.
Remember that the same record could be processed twice if it occurs twice in the residual data set; for that reason, these commands are not recommended for general use. [See 2.11.]
The VIA TREE clause is allowed only on a FOR SUBFILE command.
FOR SUBFILE VIA TREE is similar to FOR TREE in that you can sequentially process all the records in the tree under both commands. However, FOR SUBFILE VIA TREE requests that, for every record in the tree, SPIRES examine the deferred queue to see if the record has been updated or removed. Then the most recent version of the record should be used in processing. Hence, if a record has not been updated since the file was last processed, it will come from the tree; if it has been updated, it will come from the deferred queue; if it has been added or removed, it will not be processed at all.
This command replaces (and is equivalent to) the command FOR GOAL, which is still allowed. If FOR GOAL is issued with a VIA clause, such as FOR GOAL VIA ADDS, it is equivalent to FOR SUBFILE with the same VIA clause.
Taking advantage of the record sequencing inherent to an index, you can process the goal records in a particular order without having to create a set or stack first. Of course, the element by which you want the records sorted must be indexed. If this is the case, you can use the FOR INDEX command or a "FOR class" command with the "VIA INDEX index-name" clause appended. The records in the Global FOR class that are also indexed in the named index can then be processed in the order in which they are indexed.
The VIA INDEX clause may have a SEQUENCE clause to indicate further sequencing of the goal records having the same indexed value:
FOR class VIA INDEX index-name [(SEQUENCE element-list)]
where the SEQUENCE clause, which must be enclosed in parentheses, has the same syntax as the SEQUENCE command. It is discussed in the section on the FOR INDEX command. [See 2.14.]
The "FOR class VIA INDEX index-name" commands are discussed below:
This command works identically to the "FOR INDEX index-name" command, providing access to the latest copies of records specified in the index.
This command allows you to access tree copies of records in the index. It is slightly more efficient than FOR INDEX, and because the tree copies more accurately reflect the contents of the indexes than the "subfile" copies might, this form may be preferable in many cases. (FOR INDEX and the VIA INDEX clause do not examine the deferred queue of the index record.)
The other four possibilities, FOR ADDS, FOR UPDATES, FOR DEFQ and FOR REMOVES, used with "VIA INDEX index-name" clauses, all work similarly to their counterparts with VIA STACK or VIA RESULT. [See 3.1, 3.2.]
If the goal records of another subfile can be considered as index records to a subfile, path processing techniques in conjunction with Global FOR processing can be used to process the subfile's records via the second subfile. This concept was discussed in detail in the section on the FOR PATH command. [See 2.15.]
The FOR PATH command is a shortcut for FOR SUBFILE VIA PATH. The "VIA PATH pathname" clause can be added to a "FOR class" command to indicate that the records in the Global FOR class that are specified by the named path will be processed by subsequent Global FOR commands.
The VIA PATH clause may have a SEQUENCE clause to indicate further sequencing of the goal records:
FOR class VIA PATH pathname, POINTER=elemname [(SEQUENCE element-list)]
where the SEQUENCE clause, which must be enclosed in parentheses, has the same syntax as the SEQUENCE command. The "elemname" is the name of the element in the path-selected subfile that is the pointer to the goal records in the primary subfile. [See 2.15 for examples.] It is discussed in the section on the FOR INDEX command. The "FOR class VIA PATH pathname" commands are described briefly below:
As mentioned above, this is equivalent to the FOR PATH command, discussed earlier in this manual. It provides access to the latest copies of records specified in the set.
Using this command, you can access the tree copies of all goal records in the primary subfile that have pointers provided by the path-selected subfile.
The other four possible classes, ADDS, UPDATES, DEFQ and REMOVES, work similarly with a VIA PATH clause to the way the work with a VIA STACK or VIA RESULT clause. [See 3.1, 3.2.]
The FOR TRANSACTIONS command accesses different versions of added and updated records since the file was last processed. If a record has been added, updated, or removed, the FOR TRANSACTIONS VIA class command is another way to keep track of the multiple copies of records in the DEFQ.
The FOR TRANSACTIONS command allows a subset of access classes that are only allowed with the FOR TRANSACTIONS command. These are: VIA DEFQ, VIA ADDS, VIA UPDATES, and VIA REMOVES.
This command differs from the simple FOR TRANSACTIONS command in a number of ways.
1) The FOR TRANSACTIONS command presents the records in the order in which they were added or updated. However, FOR TRANSACTIONS VIA DEFQ presents each record in sequence by record key. For example:
-> for transactions +> show keys all 25 24 25 17 19 +> for transactions via defq +> show keys all 17 19 24 25 25 +>
2) The FOR TRANSACTIONS VIA DEFQ command includes the TREE copy of each record that is accessed -- that is, the original copy of the record in the database before any changes were made.
3) In the FOR TRANSACTIONS command, records are presented that never were added to the file (e.g. a DECOMITted transaction). These do not show up in the FOR TRANSACTIONS VIA DEFQ command.
For more detail about the transactions, it is recommended that the file owner use the SHOW SUBFILE TRANSACTIONS command.
This command allows you to access the transactions for the records that have been ADDed to the subfile in key sequence. If a record is added and updated that day, it is still considered an "add" by SPIRES; both copies can be accessed under the FOR TRANSACTIONS VIA ADDS (but not if the record has since been removed)
FOR TRANSACTIONS VIA ADDS differs from FOR ADDS VIA TRANSACTIONS in the following ways:
1) The records in FOR TRANSACTIONS VIA ADDS are presented in key sequence.
2) If the record has been added and then removed, it is not included in the FOR TRANSACTIONS VIA ADDS (but it is listed under FOR ADDS VIA TRANSACTIONS).
-> for adds via transactions +> sho keys all 6 7 8 9 10 9 9 +> for transactions via adds +> show keys all 6 7 8 9 9 9 10 ->
In the above example, FOR ADDS VIA TRANSACTIONS shows that record 9 was added to the database, then record 10, and then record 9 was updated two more times -- the keys are presented in the order that the adds and updates were made. On the other hand, FOR TRANSACTIONS VIA ADDS processes the records in key order. [See 3.4.]
This command allows you to access the transactions for the records that have been UPDATEd in the subfile in the key sequence. If a record is added and updated that day, it is still considered an "add" by SPIRES; both copies can be accessed under the FOR TRANSACTIONS VIA ADDS (but not if the record has been removed) FOR TRANSACTIONS VIA UPDATES does not include copies of added records that were updated later the same day. [See 3.4.]
This command allows you to access the transactions for the records that have been REMOVEd from the subfile in key sequence. However, if a record has been added and removed that day, it can only be accessed through the FOR TRANSACTIONS or FOR TRANSACTIONS VIA DEFQ commands.
So far in this manual, Global FOR has been used either to retrieve entire classes of records or else to retrieve subsets of records from one class that are also contained in another "access" class. For instance, using FOR ADDS, you can access all the added records in a subfile. But sometimes you need more focused criteria for record retrieval than a class of records (or intersection of classes) can provide.
For example, suppose you are looking for an added record in the DRINKS subfile and you do not remember the key of the record, though you do remember that the record has "gin" as one of the values for the element CONSTITUENT. If only a few records have been added to DRINKS since the file was last processed, you can easily retrieve the record you want by examining one record after another in the ADDS class:
-> select drinks -> for adds +> display
But if quite a few records have been added since the file was processed, the method above will be time-consuming, since you will have to look at all the added records, whether or not they have a CONSTITUTENT of gin.
If you include a WHERE clause on the FOR ADDS command, you can have SPIRES do this investigative work for you:
-> select drinks -> for adds where constituent = gin +> display
After you have issued a Global FOR command with a WHERE clause, your subsequent Global FOR commands will only process the subset of records that have passed this WHERE filtering. Thus, in the second example above, DISPLAY commands will only display the subset of added records with a CONSTITUENT occurrence of gin. (Note that SPIRES generally still "examines" the entire class, even though it only "processes" the filtered subset.) In general, a WHERE clause on a "FOR class" command specifies criteria that records in the class must meet in order to be processed by subsequent Global FOR commands.
A WHERE clause sometimes makes Global FOR processing of records more expensive for the following reason. When you process records under a "FOR class" command without a WHERE clause, SPIRES knows instantly which records are to be processed (i.e., all the records in the class). But when a WHERE clause is in effect, SPIRES usually must examine each record individually to see which records fit the criteria and should be processed. Because the WHERE clause causes more work for SPIRES, it costs more to use than an unadorned "FOR class" command does. On the other hand, it is certainly more efficient for SPIRES to examine the records than for you to do so. In addition, you can often reduce expenses by using a smaller class (such as a result instead of the tree), or a VIA clause, or a SET SCAN command.
The example above shows the simplest WHERE clause available, but your WHERE clause can be both more complex and more precise. For instance, the WHERE clause can name more than one criteria, thereby specifying that records retrieved must satisfy all of the conditions you name (or must satisfy any -- or even none -- of the conditions you name). Or your criteria clause can use the OCCURS option to request records that merely have a value for an element, regardless of what that value is. With the "inter-element relations" capability, your criteria clause might request only records in which one element equals (or is less than or greater than) another element in the record. [See 4.5.]
The WHERE clause offers other capabilities besides these, and most of them will be discussed in detail within this chapter. The upcoming section discusses the syntax and power of the "simple" WHERE clause, in which you specify only one criterion clause. [See 4.1.] The section following that one discusses the "compound" WHERE clause, which lets you specify multiple criteria for greater precision. [See 4.2.] The final sections discuss more advanced WHERE clause capabilities for specialized uses, such as binding element retrieval to a structural path. [See 4.4.]
The ALSO command is another mechanism, in addition to Global FOR commands, for searching on unindexed elements. The ALSO command only operates on a preceding search result; for example:
-> find ingredient rum -Result: 34 BEVERAGES -> also contributor john smith -Result: 16 BEVERAGES
The ALSO command takes an existing result or stack and creates another result or stack that satisfy the conditions named in the criteria clause. The ALSO command can use WHERE-clause terms, such as OCCURS, LENGTH, "same structure" processing, and inter-element relations. Therefore, keep in mind that the discussion in the rest of this chapter that pertains to WHERE clauses also applies to ALSO criteria clauses.
A WHERE clause has two significant limits that are worth knowing if you work with many or long values: it can have only 256 bytes of values and only 30 expressions.
So, for example, if you have this command:
FOR SET WHERE SUBJECT = RAIN OR SNOW OR SLEET OR TITLE HAIL CAESAR
The total number of bytes is the total of the lengths of the strings RAIN, SNOW, SLEET and HAIL CAESAR: 24 bytes. (Note: The total is based on the lengths of the converted, i.e., internal values of the strings.)
The example has 4 expressions:
SUBJECT = RAIN SUBJECT = SNOW SUBJECT = SEX TITLE = HAIL CAESAR
If your WHERE clause exceeds either of these limits, SPIRES gives you the error message "WHERE clause overflow".
A simple WHERE clause, when used on a "FOR class" command, has the following syntax:
FOR class WHERE [NOT] criteria-clause
The simple "criteria-clause" has the following syntax:
element [OCCURS|LENGTH] relational-operator value
Here "element" is the name (or a valid alias) of the element to be examined by SPIRES, "relational-operator" is one of the standard SPIRES relational operators listed below, and "value" is the value to be compared with the element value or values. OCCURS and LENGTH are discussed in detail later in this section. A dynamic element name can be specified for "element". [See Chapter 20 of "SPIRES Technical Notes" for an explanation of dynamic elements.]
When you issue a Global FOR command with a WHERE clause, SPIRES will process all records in the class that fit the criteria specified. If a WHERE clause specifies that an element must match a particular criterion, any record with an occurrence of that element matching the criterion will be processed, even though other occurrences of the same element within the record might not match the criterion at all. (You can use a filter to mask extraneous element occurrences. See the description of filters in the manual "SPIRES Technical Notes"; see also Section 4.4 later in this chapter.)
In comparing criteria clause values to stored element values in a record, SPIRES ignores the case (upper or lower) of both values, so you do not have to worry about case in your criteria clause. However, since SPIRES does process the criterion value through any INPROCs (processing rules for record input) that have been coded for the element, the value you specify in a WHERE clause must generally be valid as an element value. This topic is covered in more detail in Section 4.3.
All of the standard SPIRES relational operators, in the four categories of equality, range, inequality and content, are available in WHERE clauses and are listed below. If you do not use a relational operator, SPIRES will assume you want the equality operator. [See "SPIRES Searching and Updating", section B.3.2.3 for details on relational operators.]
equality range inequality content -------- ------------- ---------- ------- = BEFORE ~= LIKE (blank) AFTER > PREFIX BETWEEN...AND >= SUFFIX FROM...TO < WORD <= STRING HAVING MASK WITH
When a WHERE clause is in effect, all relational operators are equally efficient, since SPIRES will examine the entire element value under Global FOR in any case. Thus, for example, even operators like WORD or STRING, which can be quite inefficient in an indexed search, become just as efficient as the equality operator in a WHERE clause.
Here are some examples of "FOR class" commands with simple WHERE clauses:
FOR ADDS WHERE CONTRIBUTOR = GQ.JAN FOR DEFQ WHERE CONTRIBUTOR LIKE G_.JAN FOR RESULT WHERE DATE-ADDED BEFORE 1985 FOR TREE WHERE COMMENTS STRING FOOD FOR TREE VIA RESULT WHERE CHILDREN < 3 (a) FOR TREE VIA RESULT WHERE CHILDREN OCCURS < 3 (b)
In the example marked (a) above, it is important to note that the command is asking to examine values of the CHILDREN element, not the number of occurrences of the element. The command would only be meaningful if the CHILDREN element contained integer values (e.g., for number of children). If the element has textual values such as "John", "Jerry", "Carolyn" and "Ellen", the record would still be processed, but the processing would convey false information, since in sorting, alphabetic characters are always considered "less than" numerical ones. Command (b), on the other hand, uses the OCCURS option, discussed further below, to process by the number of occurrences of a textual element.
You can use a WHERE clause to access records that have a particular number of occurrences of an element or that have occurrences of a particular length. Using the OCCURS option, for example, you can specify that you want to process records that do not have any occurrences of a particular element or structure:
FOR TREE WHERE CONTRIBUTOR OCCURS = 0
Using the LENGTH option, you can ask to process records with at least one occurrence of a particular element that is a particular length:
FOR TREE WHERE CONTRIBUTOR LENGTH = 30
LENGTH refers to the number of bytes of storage used by the element value in internal storage within the file. Because elements with numerical values are often converted from string values into some other type (dates are stored internally as a four-byte HEX value, for instance), the LENGTH operator is generally most useful with textual data, such as an address or a comment.
In WHERE clauses using either LENGTH or OCCURS, a value must be supplied. Thus, FOR UPDATES WHERE PHONE OCCURS is not valid; the equivalent command "FOR UPDATES WHERE PHONE OCCURS > 0" is valid. Remember that if no relational operator is used, the equality operator is assumed. If any occurrence of the element being examined fits the criterion specified, that record will be processed.
When you use the OCCURS option for a dynamic element, the number of occurrences of the primary element in the dynamic element definition controls the number of occurrences of the dynamic element. [See Chapter 20 of "SPIRES Technical Notes" for an explanation of dynamic elements.]
The LENGTH and OCCURS options can also be used with elements that are structures, as described further below.
The NOT option, when used in a simple WHERE clause, tells SPIRES that you want to process records only when the stated criterion does not occur as any of the occurrences of the named element. For example, a record with a ZIP-CODE value of 94305 would be processed under the command "FOR TREE WHERE ZIP-CODE 94305" but would not be processed under the command "FOR TREE WHERE NOT ZIP-CODE 94305". The set of records processed under one of these commands would exactly complement the set processed under the other command.
Note that the NOT option operates differently from the inequality operator (~=) with records having multiple occurrences of the named element. For example suppose a record contains two addresses and hence two zip-codes, 94305 and 94010. The record will be processed under the command "FOR TREE WHERE ZIP-CODE 94305" because, after all, there is one occurrence of the ZIP-CODE element that did fit the specified criterion; but it will not be processed under the second command, "FOR TREE WHERE NOT ZIP-CODE 94305", even though one of the occurrences is not 94305. In other words, the NOT option will only process records in which there were NO occurrences whatsoever of the value named.
On the other hand, after a command using the inequality operator "~=" instead of the NOT option the record containing both zip-codes would be processed, because it does have one occurrence of ZIP-CODE that is not equal to 94305.
Suppose a subfile has the following three records:
NAME = Acme; NAME = Brentwood; NAME = Co-op; ZIP = 94010; ZIP = 94010; ZIP = 94305; ZIP = 94305; ZIP = 94123; ZIP = 94305;
The records would be processed under "FOR class" commands as follows:
FOR class WHERE ZIP = 94305 FOR class WHERE ZIP ~= 94305 Acme, Co-op Acme, Brentwood FOR class WHERE NOT ZIP = 94305 FOR class WHERE NOT ZIP ~= 94305 Brentwood Co-op
Although simple elements are most often named in WHERE clauses, structures can also sometimes be used. In the examples below, PHONE is a structure consisting of the elements AREA-CODE and LOCAL-NUMBER.
-> for result where phone occurs > 1 +>
With that command you could begin processing records in a search result that had more than one occurrence of the PHONE structure.
Structures can also be used in WHERE clauses with the LENGTH operator, but not as easily. Because the stored value of a structure may contain additional bytes of information (such as the length and number of occurrences of each element in the structure) these bytes become part of the total length of the structure. Though you may be able to determine exactly how many extra bytes will be added [See "SPIRES File Definition", section B.6.2.] it is easier to use the inequality operators such as ">" or "<=" than to try working with exact lengths.
There are cases where the "value" of a structure (when that value has been assembled from the values of separate elements within the structure) can be used in a WHERE clause just as though it were a conventional, single element value. In the example below, once again the value for PHONE is in fact built out of values for AREA-CODE and LOCAL-NUMBER:
-> for result where phone 415-497-4420 +>
Such a construction is only allowed when the structure is processed by $STRUC or $STRUC.IN as an INPROC. The general rule is that, if the structure can be entered into the record as a single value (which is then broken up into structural elements by SPIRES), such a value can be used in a WHERE clause.
Sometimes you might wish to specify more than one criterion in a WHERE clause. For instance, suppose you are searching for records with an "only child" named John -- you don't want to retrieve any records where John is just one of several children. To satisfy multiple conditions, you issue a compound WHERE clause -- that is, a WHERE clause that combines two or more criteria clauses, using one of the three logical operators, AND, AND NOT and OR:
FOR TREE WHERE CHILDREN = JOHN AND CHILDREN OCCURS = 1
There are only two relatively broad limits to compound WHERE clauses: 1) only 30 relational operators maximum are allowed; and 2) the entire "FOR class" command cannot be longer than 235 characters. You can circumvent the second limit either by using short element aliases and command abbreviations, or by assigning parts of the WHERE clause to SPIRES variables for substitution when the command is issued. [For more on variables, see the manual "SPIRES Protocols".]
The complete syntax of a criteria clause is:
element [OCCURS|LENGTH] relational-operator value [{AND|[AND] NOT|OR} criteria-clause]
where "criteria-clause" is another criteria clause with the same syntax as the first.
In a simple WHERE clause, the "element" is not optional. However, if there are two or more criteria clauses, the name is optional in the second or later clauses -- the last named element will be the default. Generally, this shortcut should only be used with the logical operator OR, because the presence or absence of the element name in the second clause is significant to the meaning of the clause when used with the AND or AND NOT operator. [See 4.2.4.] These two commands are equivalent:
FOR TREE WHERE ZIP 94305 OR 94306 FOR TREE WHERE ZIP 94305 OR ZIP 94306
(It is usually more efficient to repeat the element name, particularly if the goal record has phantom structures.)
The same shortcut is not allowed for relational operators, however. In the first command below, the PREFIX operator is only in effect for the first criteria clause. The default equality operator is assumed to apply to the second clause. Thus, these two commands are equivalent:
FOR TREE WHERE PHONE PREFIX 497 OR 324 FOR TREE WHERE PHONE PREFIX 497 OR PHONE = 324
To find all phone numbers with a prefix of 497 or 324, the PREFIX operator should be repeated, as in this command:
FOR TREE WHERE PHONE PREFIX 497 OR PREFIX 324
Here are several other examples of "FOR class" commands using compound WHERE clauses:
FOR SUBFILE WHERE NAME OCCURS = 0 AND NOT STATUS = OPEN FOR TRANSACTIONS WHERE START-DATE FROM 1976 TO 1978 OR FROM 1981 TO 1984 FOR * WHERE ADDRESS OCCURS > 2 OR ADDRESS LENGTH > 30
Though unnecessary, the element name ADDRESS is repeated in the last example to avoid confusion. You would only need to repeat it if there were an element in the subfile named LENGTH that you did not want used.
Parentheses can sometimes help clarify a compound WHERE clause:
-> for tree where date-added 1984 and (date-updated 1984 or 1985)
In this example, you are requesting records added in 1984 that were updated in 1984 or 1985. Omitting parentheses would create an entirely different request:
-> for tree where date-added 1984 and date-updated 1984 or 1985
Here you would be requesting records added in 1984 that were updated in 1984 or any records updated in 1985, even if they were added in some year other than 1984. The rules about using parentheses in a WHERE clause are the same as those for using them in a search request. [See "SPIRES Searching and Updating", section B.3.4.2.]
In a compound WHERE clause, the NOT option will only apply to the first criterion:
FOR TREE WHERE NOT ZIP-CODE 94025 AND STATE CALIFORNIA
That command will let you process records having a value of CALIFORNIA for STATE but which contained no ZIP-CODE of 94025. The NOT option does not carry over into the other criteria of the WHERE clause unless parentheses are used:
FOR TREE WHERE NOT (ZIP-CODE 94025 AND STATE CALIFORNIA)
Criteria clauses may be placed in any order in a WHERE clause. When SPIRES examines each record to determine whether it fits the WHERE clause criteria, it first checks the record against the first criteria clause, then the second, and so forth. For maximum efficiency then, you should arrange the criteria clauses in order starting with those most likely to fail -- that way SPIRES can more quickly proceed to the next record to be examined. For example, here are two slightly different ways to scan a student subfile for freshmen whose home-state is Virginia:
-> for tree where home-state virginia and class freshman --- or --- -> for tree where class freshman and home-state virginia
In either case, the same number of records will be examined and rejected when records are processed. However, if there are more Freshmen than Virginians (likely everywhere but in Virginia) then more records will be rejected after only one element has been examined if you use the first compound WHERE clause. Using the other form, more records will have to be examined for two criteria, which is slightly less efficient. This efficiency gap will scarcely be noticeable unless the difference between the number of records rejected by each criteria clause is large, and unless the total number of records being examined is also large.
Compare these two seemingly nonsensical commands:
-> for subfile where date after 1984 and date before 1983 -> for subfile where date after 1984 and before 1983
At first glance, it seems that the two commands are identical, and that neither command makes very much sense, since a single date cannot be both before 1983 and after 1984.
In fact, the two commands have different meanings and the first command does indeed make some sense in certain situations. The first command states that records to be processed must have at least one occurrence of DATE with a value after 1984 and at least one occurrence (not necessarily the same one) with a value before 1983. This is a logical request if more than one occurrence of DATE is allowed in the record. On the other hand, the second command states that records to be processed must have at least one occurrence of DATE with both a value after 1984 and value before 1983. Thus the second command really is as contradictory as it looks, as this example shows:
-> set element date -> for subfile where date after 1984 and date before 1983 +> display first DATE = 7/01/85; DATE = 11/20/80; +> for subfile where date after 1984 and before 1983 +> display first -End of Global FOR ->
The rules for repeating element names are as follows:
- in a compound WHERE clause, you should repeat the element name in the second clause, if you wish to specify that any occurrence can fit either criterion as long as each criterion is satisfied by at least one occurrence.
- you should not repeat the element name in the second criteria clause, if you wish to specify that the same occurrence of an element should meet more than one criterion (or should not, in the case of the AND NOT logical operator);
One additional wrinkle: using the OR logical operator to join the two parts means that any occurrence can fit either criterion as long as at least one criterion is satisfied by at least one occurrence, whether or not the element name is repeated.
File owners often use processing rules to control element values stored within a record during input (INPROCs) and the way that those values may look when the record is displayed during output (OUTPROCs). Such processing rules are chosen by the file owner, who puts them in the file definition as part of the "description" of each element. [See "SPIRES File Definition", section B.4.]
When you issue a "FOR class" command that includes a WHERE clause of the form "WHERE element relational-operator value", SPIRES first takes the value and processes it through any INPROCs defined for the specified element. For example, if your command is "FOR TREE WHERE DATE-ADDED = 7/1/85", then the value "7/1/85" is processed through the INPROCs chosen for the DATE-ADDED element. It follows that a value given in a WHERE clause must generally be a legitimate value for the element if the element value were within a record being added to the subfile. In other words, error processing for element values in a WHERE clause tries to parallel element value error processing during record input.
Thus if SPIRES would not allow you to use a value when adding a record (i.e., if the error handling parameter for the INPROC were "E" or "S"), then you would not be allowed to use that value in a WHERE clause either. On the other hand, if SPIRES would respond to the element value during record-input with a warning message (of parameter "W"), without actually rejecting the value, you can use the value in a WHERE clause, but will receive a warning that there is a problem with the input.
The process is clear-cut when the relational operator in the WHERE clause is an equality, inequality or range operator. But it is slightly less clear-cut when the WHERE clause contains a content operator, such as STRING or WORD, because in these situations SPIRES still runs the value through the processing rules as if it were a complete value. For example, if only one of four values, FRESHMAN, SOPHOMORE, JUNIOR or SENIOR, was allowed as a value for the CLASS element (i.e., because $INCLUDE was used to limit allowed values), then the following command would fail:
-> for adds where class prefix fresh -Element=CLASS: -Serious data error, code=E46 ->
The value FRESH was not one of the four allowed, so that when the value went through the processing rules for the CLASS element, the WHERE clause was rejected (just as an added record with that value would be), even though "fresh" is a valid prefix for the value FRESHMAN. In all situations like the one above, the file owner will be able to give you all the information (e.g., on which INPROCS are coded) that you will need to use Global FOR successfully.
Some processing rules that may be present in an INPROC string are ignored when a WHERE clause is issued -- that is, even if the value being processed would cause the processing rule to fail, the "FOR class" command will not fail. Generally, they are processing rules that verify, rather than change, a value. Here are the rules that are ignored, both in WHERE clause and SET SCAN processing:
The next few pages discuss two different ways to guarantee that multiple criteria you specify in a WHERE clause (or in an ALSO command) are retrieved along an identical structural path:
- the '@'-sign, used as a prefix to element names in complex WHERE clauses, to ensure that element occurrences come from the same "nested" level of the same structure occurrence; [See 4.4.1.]
- the TYPE = SCAN option on the SET FILTER command, which (among other uses) can ensure that element occurrences are retrieved along the same structural path (even though they may be nested at different levels). [See 4.4.2.]
Finally, the chapter will describe how to specify a unique element when two elements in a subfile share the same name [See 4.4.3.] and will show how the values of two different elements can be compared in a WHERE clause. [See 4.4.4.] These topics are all fairly complex; if your uses of WHERE clauses will be more straight-forward, you may be able to skim these sections or skip them completely.
Suppose a subfile of businesses has a multiply occurring structure for addresses of branch offices, each occurrence containing STREET-ADDRESS, CITY, STATE, and so on. Next suppose that you want to process records of businesses with branch offices in Menlo Park, California:
-> for subfile where city menlo park and state california +>
That command would indeed retrieve the records you wanted, However, if a business with an office in California also had an office in, say, Menlo Park, New Jersey, that record too would be processed, because SPIRES examines all occurrences of the elements specified in WHERE clauses in each record, so that element values fitting the criteria may not necessarily come from the same structure.
To ensure "same-structure processing", i.e., that all values are found in the same structure, at the same level, you would insert the symbol "@" (the "at" sign) directly before the name of the second and subsequent elements to which the limitation should apply:
-> for subfile where city menlo park and @state california
SPIRES will then process only records having a value of MENLO PARK for CITY and a value of CALIFORNIA for STATE, both of the values occurring in the same ADDRESS structure.
There are four restrictions in using "@"-sign clauses:
1) the logical operator preceding the "at" sign may only be AND or AND NOT; it may not be OR; 2) the element preceded by the "@" sign must not use the OCCURS option; 3) the element preceded by the "@" sign must be an element in the same structure as the preceding element; and, 4) the "@" sign may not be preceded by parentheses:
-> for tree where dept music and (@year > 1975) -Use of @-sign on YEAR ignored +>
In WHERE clauses using the "at" sign, the OCCURS option should not be used by the element that preceeds the @element in the clause. For example, compare the following two commands:
-> for tree where semester occurs = 2 and @course = mus100 -> for tree where course = mus100 and @semester occurs = 2
As written, the first command is contradictory, in requesting records with a total of two occurrences of SEMESTER in the record, but also implying (with the "at" sign) that the occurrences should come from the same structure -- SPIRES reconciles this contradiction by ignoring the "at" sign and not performing any structural binding at all. The second command makes its intentions much clearer: "Find records having a value of MUS100 for COURSE that have two occurrences of SEMESTER in that same structure." So for the second command, structural binding is maintained.
If you need to do more than one OCCURS test in the same structure, and one or more of them simply must occur (OCCURS > 0), then you can substitute LENGTH >= 0 for OCCURS > 0, such as:
-> for tree where semester length >= 0 and @course occurs = 0 -> for tree where semester length >= 0 and @course occurs > 0
You can't use OCCURS > 0 with SEMESTER, but you can use LENGTH >= 0. Note that the @element (COURSE) can use OCCURS since it is not followed by another @element condition.
In same-structure retrieval using the "at" clause, as described in the preceding section, the element preceded by an @-sign must not only come from the same structure as the first-named element, but must come from the same depth of that structure as well. In other words, the @-element or elements cannot be nested at a deeper structural level than the level of the first-named element.
If you want to ensure that element values you retrieve lie along the same structural path, but you don't care whether they are nested at the same level or not, you can use the SCAN option on the SET FILTER command, naming a structural element as the element to be filtered, and naming an element nested within that structural element as part of the SET FILTER command's WHERE clause. [For a full description of the syntax of the SET FILTER command, see the manual "SPIRES Technical Notes".]
For example, consider a student subfile that contains a number of nested structures like Chinese boxes: a multiply-occurring structure for "year", each occurrence of which contains multiple occurrences for "quarter", each occurrence of which contains multiple occurrences for "course":
ID YEAR.STRUC YEAR, YR QUARTER.STRUC QUARTER, QTR COURSE.STRUC COURSE
If you want to process occurrences of YEAR.STRUC only when they contain particular values for QUARTER and COURSE, the "at" clause will not be helpful, because QUARTER and COURSE, although they occur along the same structural path as YEAR, lie in "different" (more deeply nested) structures. The solution might be to set filters on the structures as in the following example:
-> set filter (scan,display) for year.struc where yr = 1985 ... ... and course = mus100 -> set filter (scan,display) for quarter.struc where qtr = fall
Once the filters are set, you would need to activate them by naming the structural path to be filtered as part of the WHERE clause on your Global FOR command:
-> for subfile where year.struc occ > 0 +> display
The WHERE clause on the last command is important: if you did not include it on your Global FOR command, the filtering would not occur.
Perhaps the trickiest part of this process is keeping the two different WHERE clauses straight in your mind: the SET FILTER command can have a WHERE clause just as a Global FOR command can have one, and the syntax allowed for the two WHERE clauses is identical. However, the WHERE clause on the SET FILTER command lets you solve a problem that might be impossible to solve using Global FOR alone. [Complex same-structure processing can also be handled by Partial FOR techiniques, but Partial FOR is more complicated and laborious than setting filters. [See the manual "SPIRES Technical Notes."]]
A few subfiles use the same element name within several structures, especially when the element is a common type, such as a name, phone-number or comment. Suppose a subfile of businesses has an address structure for both the MAIN office and the BRANCH offices, each structure containing at least a CITY element. If you issue the command "FOR SUBFILE WHERE CITY PALO ALTO", SPIRES will examine either the MAIN or BRANCH city for each record, but not both (the first one encountered in the list shown by the SHOW ELEMENT NAMES command would be the one examined), and would then process records where that particular type of office was in Palo Alto. Rather than have to figure out which one SPIRES will use, you would want to be more specific (e.g., you only want records where the main office is in Palo Alto). You can specify the particular "path" for SPIRES to follow through the record to find the appropriate occurrences of the element, again using the "at" sign ("@"):
-> for subfile where main@city palo alto
This form is common throughout SPIRES in situations where you need a particular structural element. Its syntax looks like this:
structure-name@structure-name...@element-name
where, from left to right, each structure named is nested within the previous one. There must be no blanks around the "@". Note that you only need to specify as many structure names from the right as are needed to uniquely identify the desired path.
Remember that specifying the path is only necessary if the element name is used within several different structures in the goal record. File definers often assign different aliases for the element in different structures to avoid the problem altogether. For example, in this case, the file definer might allow MCITY and BCITY for aliases.
You can compare the values of one element in a record to those of another element in the same record using an option on the criteria clause. This capability allows "inter-element relations".
The syntax of the "FOR class" command using inter-element relations is
FOR class WHERE [NOT] element1 relational-operator @element2
where "element1" and "element2" are the names of the two elements being compared. The elements must be of the same type, for instance, string or integer. The "@" sign indicates that what follows it is an element name rather than a value. (If your value in a criteria clause actually begins with "@", the entire value must be surrounded by quotation marks.) Neither the OCCURS nor the LENGTH option can be used with inter-element relations. Any of the relational operators can be used, including "BETWEEN... AND..." and "FROM... TO...", which both use two values rather than one -- either or both values can be replaced by the "@element" option.
For example, to find records in a subfile that have been updated since the day they were added, you could type
-> for subfile where mod-date after @add-date
assuming that those elements existed in the subfile.
Special care should be taken with "multiply occurring" elements (elements that may have several different values in a record, such as the office-phone element in the example below). Suppose you are using a subfile for an answering service, where record elements include a name, home phone number and office phone numbers, and you have these three records:
NAME = Dr. Adams; NAME = Dr. Bax; NAME = Dr. Copland; HOME-PHN = 555-1111; HOME-PHN = 555-2222; HOME-PHN = 555-3333; OFF-PHN = 555-1111; OFF-PHN = 555-2222; OFF-PHN = 555-4444; OFF-PHN = 555-2223;
If you issue the command "FOR TREE WHERE HOME-PHN = @OFF-PHN" you will be able to use Dr. Adams' and Dr. Bax's records. But if you issue the command "FOR TREE WHERE HOME-PHN ~= @OFF-PHN" you will be able to use Dr. Copland's record, but also Dr. Bax's, since his home phone is not the same as one of his office phones. Thus, when SPIRES examines a record for an inter-element relation, it compares all values of the first element to all values of the second one; if any one comparison fits the criterion requested, the record is retrieved. If you wanted only records where no office phone number matched any home phone number, you could say "FOR TREE WHERE NOT HOME-PHN = @OFF-PHN" which would retrieve all records except those where the two numbers matched.
If the elements being compared are not at the record level, but instead are structurally bound, then SPIRES compares the elements within individual occurrences of those structural bounds. So if NAME, HOME-PHN and OFF-PHN were elements within a multiply occurring structure, HOME-PHN in one occurrent of the structure would only be compared to OFF-PHN in that same structural occurrence. For example, Dr. Copland's HOME-PHN would NOT be compared to the OFF-PHNs of either Dr. Adams or Dr. Bax.
You may use dynamic elements in WHERE clauses expressing inter-element relationships, as long as the dynamic element precedes the relational operator:
WHERE dynelem relational-operator @element2
"Element2" is the name of the element to which the dynamic element is to be compared. The type of "element2" must be the same as the type of the dynamic element. [See Chapter 20 of "SPIRES Technical Notes" for an explanation of dynamic elements.]
You can use phantom elements in WHERE clauses expressing inter-element relationships, and test a primary element against any phantom element:
WHERE primary_elem relational-operator @phantom_elem
[See Chapter 23 of "SPIRES Technical Notes" for an explanation of phantom elements.]
This section will cover some of the more complex details of using inter-element relations that would not be particularly useful to the average searcher.
Depending on the file definition, there may be situations where not all element values with the same element name are compared to all the values of another element.
Assume the following record structure:
SA = ...; SB = ...; S1; S1A = ...; S1B = ...; S11; S11A = ...; S11B = ...; S12; S12A = ...; S12B = ...; S2; S2A = ...; S2B = ...; S21; S21A = ...; S21B = ...; S22; S22A = ...; S22B = ...;
Assume further that all structure and simple elements shown above can be multiply occurring, and all simple elements are of the same type.
Any element could be described by the structural path to that element. For example: the path of S12B could be described as "S1@S12@S12B". Such a definition contains all structure names except for the last name, which is a simple element name.
Now, consider the WHERE clause: "WHERE elem1 relational-operator @elem2" where "elem1" and "elem2" represent any of the simple elements shown in the sample. To determine which occurrences of "elem1" are compared to which occurrences of "elem2", first write "elem1" and "elem2" in their full structural form. For example: "S12B > @S1A" would yield "S1@S12@S12B" and "S1@S1A". Next, choose the form with the least number of structure names (if they have the same number, choose either). Let's call this one form X and the longer one form Y.
Rules:
If either "elem1" or "elem2" (or both) are "record level" elements, then all occurrences of "elem1" are compared to all occurrences of "elem2".
If all the structure names of X match all the structure names of Y through the same number of structure names as are contained in X, then all the occurrences of "elem1" contained within the single occurrence of X's structures are compared to all the occurrences of "elem2" contained within that same occurrence of X's structures.
If all the structure names of X do not match all the structure names of Y through the same number of structure names as are contained in X, then determine the "common root" as being the structure names which do match (beginning with the first structure name of both X and Y). For the first occurrence of the structural path from the common root leading to "elem2", compare all occurrences of "elem2" in that singular occurrence to all the occurrences of "elem1" belonging to the same occurrence of the common root.
Here is an example to help you understand this last rule:
S1; S11; S11A = 1; S11A = 2; S11; S11A = 3; S12; S12A = 4; S12; S12A = 5; S1; S11; S11A = 6; S11; S11A = 7; S12; S12A = 8;
WHERE "S12A relational-operator @S11A" would make the following comparisons:
S12A = 4 with S11A = 1 S12A = 4 with S11A = 2 S12A = 5 with S11A = 1 S12A = 5 with S11A = 2 S12A = 8 with S11A = 6
WHERE "S11A relational-operator @S12A" would make the following comparisons:
S11A = 1 with S12A = 4 S11A = 2 with S12A = 4 S11A = 3 with S12A = 4 S11A = 6 with S12A = 8 S11A = 7 with S12A = 8
About a dozen commands are used to process records in Global FOR mode. Some of them are available exclusively in Global FOR and have no meaning outside of it (SHOW KEYS, for example); others are valid outside of Global FOR but operate quite differently inside of it (e.g., DISPLAY and REMOVE). Of course, these are not the only commands that can be issued or that are useful in Global FOR. As mentioned previously, all SPIRES commands and commands of other systems can also be issued.
Here are the commands that will be discussed in this chapter:
DISPLAY TRANSFER and UPDATE MERGE REMOVE DEQUEUE UNQUEUE STACK SHOW KEYS SKIP REFERENCE DEFINE SET, GENERATE SET GENERATE LOAD
The commands SHOW LEVELS and SET SCAN, which do not actually process records but which are useful tools under Global FOR, will be discussed in the next chapter. [See 6.]
Consider the "FOR class" command which begins Global FOR processing: the first part is the actual "FOR class" command, followed by the second part, the optional WHERE clause. It is important to bisect the command here because the two parts affect record processing at different times. The first part, "FOR class", tells SPIRES that Global FOR commands which follow will process records in the particular class. Thus, SPIRES determines which records are in the class and prepares to process them. Also at this point, if there is a WHERE clause, SPIRES determines whether the WHERE clause is syntactically correct, but not whether any records in the class actually fit its criteria. SPIRES then returns the Global FOR prompt quickly after the "FOR class" command is issued because no record processing has occurred. Only some preliminary internal setup and verification have been done. The real processing, including examination of records to see which ones fit any WHERE clause in effect, only occurs when one of the Global FOR commands listed above is issued.
These points are illustrated below in the diagram and example. You have just issued the command FOR ADDS in your FRIENDS subfile. Imagine that you will be examining the following records in the order shown:
(A) <--- Current Record Pointer (B) NAME = Adams; (C) NAME = Buchanan; PHONE = 111-2222; (D) NAME = Coolidge; PHONE = 333-4444; (E) NAME = Dewey; (F) NAME = Eisenhower; PHONE = 555-6666; (G) NAME = Fillmore;
NAME is the key element of the goal record; PHONE is an optionally occurring element. In Global FOR, records are processed in the order in which they are stored in the class; since records are stored in the deferred queue in key order, the records will be processed in the order shown in the diagram.
After issuing the FOR ADDS command, you can begin working with the records. Suppose that since you only want to examine the records, you will only need the DISPLAY command. Outside of Global FOR, using the DISPLAY command requires that you tell SPIRES the key of the record you wish to display; otherwise, SPIRES would not know what record to show you. Here you do not even need to know the record key, for you have told SPIRES in the "FOR class" command which records you want to see. You can simply issue the command
+> display NAME = Adams; +>
and the first record is displayed. If you then issue the command again:
+> display NAME = Buchanan; PHONE = 111-2222;
then the next record is displayed to you. Eventually of course, there will be no more records:
+> display NAME = Fillmore; +> display -End of global FOR ->
Perhaps it seems unusual to you that each time you issue the DISPLAY command you see the next record rather than the same one over and over again. By way of comparison, for example, the TYPE command issued alone always shows you the same records, starting over from the first one each time you issue it. Remember though that the purpose of Global FOR is to simplify sequential record processing, that is, one record after another. Usually in Global FOR, you want to process one record after another and not keep starting over again (although that can be done, using the FIRST or ALL options described later or the FOR ** command). What allows the DISPLAY command and, in fact, most other Global FOR commands to process records one by one and not have to keep starting over again with each new command are the "current record pointer" and the "examination pointer" maintained in Global FOR.
The current record pointer keeps track of which record is currently being processed. As records are processed, it moves down through them, always pointing at the record currently being processed. When the "FOR class" command is issued, the pointer is "above" all the records, in the position shown in the diagram as position A. When the first DISPLAY command in the sample session is issued, the pointer moves to position B, pointing at the record being displayed. The pointer moves to position C when the next DISPLAY command is issued, and so on.
How does using a WHERE clause affect processing?
-> for adds where phone occurs = 1 +>
As pointed out previously, no record processing is done by the "FOR class" command, even with an appended WHERE clause. The pointer is established at position A, and SPIRES awaits a Global FOR record processing command:
+> display NAME = Buchanan; PHONE = 111-2222; +>
SPIRES is told to display the next record of the class of added records that have exactly one occurrence of the PHONE element. Because of the WHERE clause, SPIRES cannot simply access the next record it finds and display it; instead, SPIRES must examine the next record, checking to see whether the record has a PHONE element. This is where the examination pointer becomes important. The examination pointer indicates the record currently being examined for WHERE clause conformity. Like the current record pointer, it begins at position A. When the DISPLAY command is issued, it moves to position B, and the first record, for Adams, is examined. (The current record pointer is still at position A.) Since that record does not fit the WHERE clause criteria, because it does not have a PHONE element, the examination pointer proceeds to C, pointing at the record for Buchanan. That record is then examined. When SPIRES determines that the WHERE clause is satisfied, the current record pointer is moved from A to C (bypassing B), and the record is processed according to the command. Thus, the examination pointer moves ahead of the current record pointer, which catches up to the examination pointer only when the examination pointer finds a record meeting the WHERE clause criteria. (The examination pointer also exists when there is no WHERE clause in effect, but it is really inseparable from the current record pointer in that case, since no records will actually be examined and then excluded from processing due to a WHERE clause.)
As processing continues, SPIRES keeps a count of how many records have been examined and how many of them have fit the WHERE clause criteria and been processed. You can see that information at any time by issuing the SHOW LEVELS command. [See 6.1.]
You are not limited to processing only "the next record" with Global FOR. Most of the Global FOR commands discussed in this chapter have several options that allow you to specify particular records in the class that you want to process.
Here are the standard options:
command-verb [FIRST|*|NEXT|n|REST|LAST|ALL]
If no option is used, the default is NEXT, except for the GENERATE commands, in which cases the default is ALL. Not all the options shown here are meaningful for all the Global FOR commands. The exceptions are shown in the chart below and described in the next section of this chapter.
+---------------+--------------------------------------------------+ | | Command Option | | Command | FIRST | * | NEXT | n | REST | LAST | ALL | |---------------+-------+-----+------+-------+------+------+-------| | DISPLAY | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | TRANSFER | | | | nth | NEXT | | FIRST | |---------------+-------+-----+------+-------+------+------+-------| | MERGE | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | REMOVE | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | DEQUEUE | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | UNQUEUE | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | STACK | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | SHOW KEYS | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | SHOW REC INFO | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | SKIP | | | | nth | NEXT | | FIRST | |---------------+-------+-----+------+-------+------+------+-------| | REFERENCE | | | | nth | NEXT | | FIRST | |---------------+-------+-----+------+-------+------+------+-------| | GENERATE SET | | | | n | | | | |---------------+-------+-----+------+-------+------+------+-------| | GENERATE LOAD | | | | n | | | | +---------------+-------+-----+------+-------+------+------+-------+
All boxes left blank in the chart signify options allowed on the command. The "n" indicates that SPIRES processes the next "n" records; the "nth" indicates that the "nth" record from the current one is processed. (If a WHERE clause is present, then "n" indicates that the next "n" records that fit the criteria are processed, while "nth" indicates that the "nth" record that fits the criteria from the current one is processed.)
For the "nth" commands, SPIRES substitutes NEXT for REST and FIRST for ALL, as shown in the chart.
This option indicates that the first record in the class be processed. (Naturally, if a WHERE clause is in effect, this means the first record in the class that fits the WHERE clause criteria.) The current pointer is set to this record. FIRST is occasionally used with a command to reset the pointer at the beginning of the group of records so that they can all be processed again. FIRST can be specified as an option at any time while under Global FOR.
Here is an example that uses the FIRST option:
-> for adds where phone occurs = 1 +> show keys all Buchanan Coolidge Eisenhower +> display first NAME = Buchanan; PHONE = 111-2222; +> display NAME = Coolidge; PHONE = 333-4444; +>
The diagram shows the position of the current record pointer after the DISPLAY FIRST command shown above is issued:
(A) (B) NAME = Adams; (C) NAME = Buchanan; <--- Current record pointer PHONE = 111-2222; (D) NAME = Coolidge; PHONE = 333-4444; (E) NAME = Dewey; (F) NAME = Eisenhower; PHONE = 555-6666; (G) NAME = Fillmore;
After the SHOW KEYS ALL command was issued, the current record pointer was at position F (the last record fitting the WHERE clause criterion), and the examination pointer was at position G, the last record examined. The DISPLAY FIRST command first reset both the pointers to position A; SPIRES then examined each record from the beginning of the class, finally processing (and setting the current record pointer to) the first record in the class that has one occurrence of a PHONE element; that record is the "current" record being processed. (By the way, after the last command shown, the pointers are at position D.)
To process the current record, the "*" option can be used:
-> for adds +> display NAME = Adams; +> display * NAME = Adams;
The "*" option is quite useful, for example, when you want to determine whether or not to update a record by examining it first:
+> display NAME = Adams; +> transfer * +>
Since the current record is being processed again, the current record pointer remains in the same place, indicating the "*" record.
Changing the WHERE clause in the midst of Global FOR processing by issuing a "FOR *" command will not change the current record pointer, even if the current record does not fit the WHERE clause:
+> display * NAME = Adams; +> for * where phone occurs > 0 +> display * NAME = Adams; +>
Records processed by commands with options other than "*" will conform to the WHERE clause requirements.
The next record past the current record (and which fits any WHERE clause in effect) is processed when this option is used. This is the default if no option is indicated for all except the GENERATE commands, as shown by many earlier examples. The current record pointer is set to the processed record.
For most of the Global FOR commands described in this chapter, more than one record can be processed at a time. For example, more than one record can be examined with the DISPLAY command. Using some positive integer as "n", you can specify that the next "n" records be processed:
-> for adds +> display NAME = Adams; +> display 2 NAME = Buchanan; PHONE = 111-2222; NAME = Coolidge; PHONE = 333-4444; +>
If a WHERE clause is present, all "n" records processed will naturally fit the criteria specified:
-> for adds where phone occurs < 1 +> display 2 NAME = Adams; NAME = Dewey; +>
Not all the commands discussed in this chapter can be used to process more than one record at a time. For instance, it would not be very valuable to TRANSFER two records at once, since a TRANSFER command prepares SPIRES to accept exactly one record for updating. (Only one record can be updated at a time in SPIRES.) Thus, several commands (TRANSFER, MERGE, SKIP and REFERENCE) interpret the "n" option differently than "the next n records". For them, it means "the nth record past the current one". So "TRANSFER 5" tells SPIRES to transfer the fifth record from the current one, not to transfer the next five records. Of course, if a WHERE clause is in effect, then "TRANSFER 5" means that SPIRES should examine records for the WHERE clause criteria and transfer the fifth one that fits.
Whether "n" means "n" or "nth", a command using the "n" option resets the current record pointer to the last record processed by the command.
When the option used means "n", then Global FOR mode is terminated if no record can be found; otherwise, from one to "n" records will be processed:
-> for adds +> show keys 10 Adams Buchanan Coolidge Dewey Eisenhower Fillmore +>
However, if the command means "nth", Global FOR mode is terminated if the "nth" record cannot be found:
-> for adds +> transfer 10 -End of global FOR ->
Since there was no tenth record in the class, no records can be processed. [See 7.2.]
The remaining records in the class are processed sequentially. If a WHERE clause was used in the "FOR class" command in effect, only records that fit it will be processed. The current record pointer is moved to the last record processed.
-> for adds where phone occurs = 1 +> display NAME = Buchanan; PHONE = 111-2222; +> display rest NAME = Coolidge; PHONE = 333-4444; NAME = Eisenhower; PHONE = 555-6666; +>
The current record pointer then indicates the last record processed; in the example above, the pointer would be at position F in the diagram. The examination pointer, since it has examined all the records in the class, is now at position G. It is only at the end of a class under WHERE clause processing that the two pointers would end up at different locations. When the EISENHOWER record was processed, both pointers were at position F. Then the examination pointer moved to position G, and SPIRES examined the next record in the class, which happened to be the last one in the class. Since that record, FILLMORE, did not have a PHONE element, the current record pointer did not move from position F.
The last record in the class that fits the WHERE clause criteria (if any) is processed using the LAST option. The current record pointer then indicates the last record processed.
-> for adds +> display last NAME = Fillmore; +>
Both pointers are now at position G in the diagram. Note that if a WHERE clause is in effect, the current record pointer indicates the last record that fits the criteria -- not necessarily the last record in the class.
-> for adds where phone occurs = 1 +> display last NAME = Eisenhower; PHONE = 555-6666; +>
The current record pointer is now at position F, and the examination pointer is at position G. SPIRES has actually examined all the records in the class (otherwise, how could SPIRES know that the Eisenhower record was the last one to fit the criteria?); this fact is reflected in the statistics provided by the SHOW LEVELS command. [See 6.1.]
Warning: Like the ALL option below, the LAST option causes SPIRES to examine all the records in the class, from the first one to the very last one, looking for the last one which fits the criteria of any WHERE clause in effect. It does not start looking for the last record from the current record; it begins all over again. As a result, the LAST option can be very expensive if you have a large subfile and a WHERE clause -- it will examine all the records in the subfile but will only process the last one it finds that fits the criteria specified.
If you use the ALL option, all the records in the class are processed, beginning with the first record and continuing through the last. If a WHERE clause exists, all the records that fit its criteria are processed. The ALL option can be issued with a command at any time under Global FOR. The current record pointer points to the last record processed that fits any WHERE clause in effect, which is not necessarily the last record in the class, as discussed above.
The current record pointer works similarly for the REST, LAST and ALL options -- SPIRES examines all of the remaining records from the point specified (from the beginning of the class for the ALL and LAST options; from the examination pointer position for the REST option). As SPIRES examines each record, the current record pointer remains pointing at the last record examined that fits the criteria of the WHERE clause. Thus when SPIRES has examined the final record in the class, the current record pointer indicates the last record examined that fits, and this is the last record processed by the command; the examination pointer indicates the last record in the class. If you issue another command at this point, such as TRANSFER NEXT, SPIRES does not have to examine the rest of the records in the class again, since SPIRES begins examining records from the point where the examination pointer is; so you receive an "End of global FOR" message immediately, since there is no "next" record to examine. SPIRES does not automatically remove you from Global FOR mode when you process the "last" record since you may want to process the record again or reprocess the whole class using the FIRST or ALL options. [See 7.2.]
The DISPLAY command is a very important searching and updating tool in Global FOR. Outside of Global FOR, the DISPLAY command allows you to examine single records individually without having to go through the search procedure; however, you must know the key value, which is issued with the command verb DISPLAY. Under Global FOR, the DISPLAY command works quite differently, though its function, to display records, remains the same. For instance, under Global FOR, the DISPLAY command can show you more than one record at a time. And, of course, you do not need to know the key of the record you want to display since you have already told SPIRES which records you want to process in the previous "FOR class" command.
The syntax of the DISPLAY command in Global FOR is
DISPLAY [FIRST|*|NEXT|n|LAST|REST|ALL] [END='command']
"DISPLAY n", where "n" is an integer, indicates that the next "n" records should be displayed. DISPLAY issued with no options is equivalent to DISPLAY NEXT. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.] The IN ACTIVE prefix is very useful with the DISPLAY command, causing the requested records to be displayed in the active file.
The DISPLAY command shows you records in their normal output format (whatever format is set), unless a SET ELEMENTS command is in effect.
The following example demonstrates the use of the DISPLAY command:
-> select drinks -> for tree +> set elements name +> display NAME = Harvey Wallbanger; +> display NAME = Yellow Bird; +> display 3 NAME = Golden Cadillac; NAME = Singapore Sling; NAME = Bee's Knees; +> display * NAME = Bee's Knees; +> display first NAME = Harvey Wallbanger; +> display next NAME = Yellow Bird; +>
Occasionally, if you are processing many records, the active file may fill up:
+> in active display all -Active file limit exceeded. Code=S197 -21317 records processed +>
Or the response may look like this:
+> in active display all Active file is too big. +>
In the first case, SPIRES has put data into your active file up to about line 99999, which is what it considers to be the limit for the active file. In such a case, the current record, "*", did not get placed into the active file. You can continue processing by clearing your active file (saving the data already there elsewhere) or by renumbering the data with a smaller delta and/or issuing the SET DELTA command. To continue processing, issue the command IN ACTIVE DISPLAY * and then IN ACTIVE DISPLAY REST. Remember that, unless you were using the CLEAN option on the DISPLAY command anyway, the single record "*" will not have a line with the record separator ("****") preceding it and a line with ";" in column 1 following it. If you do not have a WHERE clause in effect, you might want to consider the following sequence instead: FOR class, SKIP n (where "n" is the number of records already processed), IN ACTIVE DISPLAY REST. [See 5.10.]
On the other hand, if the message is the second one, then the active file really is full, according to WYLBUR. In this case, no more records can be placed in the active file at all; you must save the data elsewhere, clear your active file, and then continue processing. Unfortunately, in this case, the record "*" did not make it into the active file but possibly others did not either. (For this reason, it is usually preferable for SPIRES to tell you that there is no more room than WYLBUR, even though SPIRES is probably not absolutely correct.) The only way to tell which records have not made it into the active file is to look at the last few records in the active file. Then you can use the SET SCAN or SKIP commands to get you started with the proper record when you continue. [See 5.10, 6.2.] (Technical note: when the latter case happens, the last 4096 bytes of data did not make it into the active file. Depending on the size of the records, this information may help you figure out where to resume when you begin processing records again.)
The TRANSFER command is often used under Global FOR when you want to update the records in a subfile. Whether issued in Global FOR or not, the TRANSFER command puts a copy of the requested record into the active file, generally in the SPIRES default format. You can then alter the record in your active file, making any corrections you want to make. SPIRES then awaits an UPDATE command, anticipating the return of the same record that was transferred.
The syntax of the TRANSFER command under Global FOR is
TRANSFER [FIRST|*|NEXT|n|LAST] [END='command'] [CLEAR|KEEP]
The TRANSFER command can only process one record at a time; the TRANSFER and UPDATE operation is used for single records. Therefore, "TRANSFER n" means to transfer the "nth" record from the current one, rather than the next "n" records. TRANSFER ALL and TRANSFER REST are invalid commands in that the action requested will not take place; however, TRANSFER ALL will transfer the first record and TRANSFER REST will transfer the "next" record. (A message announcing this action is also printed at the terminal.) [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]
The CLEAR option can be specified if there is already text in your active file and you don't want to be prompted "-OK TO CLEAR?" (assuming SET CLEAR is not in effect). The CLEAR option must follow the END clause if both are used. [See 7.3.] The KEEP option opens up a new active file without clearing the old one. (The PICK command lets you restore the previous active file.)
The UPDATE command is not really a Global FOR command, since it works no differently under Global FOR than it does outside of it. Its syntax under Global FOR remains the same:
UPDATE [key]
If you are updating a record which was just transferred (the usual case), you will not need to use the "key" option. However, if you want SPIRES to update a record that you did not transfer first, you will need to use the "key" option. If you issue an UPDATE command without the "key" option, and you have not previously issued a TRANSFER command, SPIRES will ask you which record you want to update. If you issue an UPDATE command without the "key" option and you have preceded it with a TRANSFER command, the key of the record in your active file must match the key of the record last transferred; otherwise, the request will be aborted.
Here are some examples of the TRANSFER and UPDATE commands in Global FOR:
-> select friends -> for adds +> display NAME = Adams; +> transfer * <--- The current record is transferred. ... <--- Changes are made to the record. +> update +> transfer 5 clear <--- The fifth record past the current one is transferred. ... <--- Changes are made to the record. +> update +> transfer clear <--- The next record is requested. -End of global FOR <--- There are no more records in the class. ->
Outside of Global FOR, whenever you transfer a record, you always know that the copy that is placed in the active file is the latest copy of a record -- that is, if the record has been updated earlier in the day, you will be updating the most recent copy from the deferred queue and not the older "out of date" copy from the tree. [See 2.3.] However, the source for the transferred record under Global FOR is the class specified in the "FOR class" command. Be sure that you know where the copy comes from before you update the record; otherwise, you may be redoing or nullifying any updating work done since the file was last processed.
The MERGE command in Global FOR causes records in the class specified by the "FOR class" to be updated with the contents of the active file, or with data under the control of a merge input format. For information about how a specific merge format works, you should contact the format's designer, or the file owner. [See the manual "SPIRES Formats" for information about writing formats for merging records.]
The format of the command is
MERGE [FIRST|*|NEXT|n|REST|LAST|ALL] [USING line-range] [END='command']
The "n" option signifies that the next "n" records will be processed by the command. By default, MERGE with no option indicates that the next record be processed.
The USING option indicates a range of lines in the active file that are to be merged into the record. If the option is not present, the entire active file is merged. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]
The command also has three other options, including the prefixes WITH FAST and WITH SEARCHONLY, which can be used by the file owner and by users with Master access to the file to speed up the processing. See the descriptions of WITH FAST and WITH SEARCHONLY in the following section on the REMOVE command. [See 5.6.]
Another option is WITH DATA, which says that the data to use for record input is part of the MERGE command itself. This option is explained in detail in the manual "SPIRES Technical Notes"; online, EXPLAIN WITH DATA.
Here is an example of the MERGE command. Using your FRIENDS subfile, you want to merge the same phone number into any records you added earlier without a phone number:
-> collect 1. ? phone = 777-8888; 2. ? *** -> select friends -> for adds where phone occurs = 0 +> merge all - MER Key = 19 - MER Key = 27 - MER Key = 35 - MER Key = 43 - Requests/Success: MER 4 4 SUM 4 4 -End of merge input +> merge -End of global FOR ->
In the example, you have merged the phone number into all the records being added that have no phone number element. Since there were no more, when you tried to merge another record, you were removed from Global FOR. [See "SPIRES Searching and Updating", section D.5.4, for more information about using the MERGE command.]
In Global FOR, the "physical" effect of the REMOVE command (i.e., what actually happens to each record that is the object of the command) remains basically the same as outside Global FOR. [See "Searching and Updating", Chapter D.4.] For each record that is the object of the REMOVE command, a removal request is placed in the deferred queue; although the record is still physically stored in the file, normal record display techniques ("DISPLAY key" or TYPE) cannot access it. Whenever the file is next processed, records that were named in a remove request are physically removed from the subfile.
What makes the REMOVE command distinctive (and dangerous for beginners) in Global FOR, is that the command can be used to remove records in groups as well as one by one.
The form of the REMOVE command in Global FOR is as follows:
REMOVE [FIRST|*|NEXT|n|REST|LAST|ALL] [END='command']
If no option is specified, NEXT is assumed as the default. The "n" option, if used, indicates that the next "n" records are to be removed. [See 7.3 for an explanation of the END option, 5.2 for more information about the other options.]
[See below for information about the WITH FAST and WITH SEARCHONLY prefixes, also allowed on the REMOVE command.]
For example, suppose you want to remove all the records in the RESTAURANT subfile that have not been updated since 1975:
-> select restaurant -> find date-updated before 1976 +> for result +> remove all -Removed: 3 records +>
Since the REMOVE command under Global FOR is so powerful, SPIRES issues an informational message telling you how many records are actually processed by the command. Suppose, for example, that a user issued these commands:
-> select restaurant -> find date-updated before 1978 -Result: 67 RESTAURANTS -> for result +> set elements name id +> display 2 ID = 214; NAME = Dutch Goose; ID = 229; NAME = Good Earth; +> remove 214 -Removed: 65 records +>
It is sometimes easy to forget that you are under Global FOR and that commands will operate differently because of it. The user wanted to remove the record whose key was 214; but under Global FOR, the user actually requested that the next 214 records in the class be removed. (There were only 65 records in the class left.) It is important to be able to distinguish between the "REMOVE n" command under Global FOR where "n" is an integer, and the "REMOVE key" command outside of Global FOR, especially if the keys of records are integers. By the way, the problem above can be easily corrected, using the FOR REMOVES and the UNQUEUE or DEQUEUE commands. [See 2.7, 5.7, 5.7a.]
If you are either the file owner or a user with Master access to the file, you can add the WITH FAST or WITH SEARCHONLY prefix to the REMOVE command to speed up the removal processing.
WITH FAST removes the records three to four times faster, in terms of processing costs, than the normal REMOVE command. You can use it under these conditions:
- You must have Master access to the file.
- The deferred queue must be empty when you issue your WITH FAST request.
- The file must have a CKPT data set of its own. (If the file does not, then adding either a CHECKPOINT or IMMEDIATE statement to the file definition and recompiling will create one. See the manual "SPIRES File Definition" for information about those statements.)
- You are willing to forego the updating of immediate indexes as part of the removal process; if the subfile has immediate indexes, you will have to process the file in SPIBILD to get the indexing done, and to allow other users to use the subfile for updates again.
During the processing, the file will be placed in SET SEARCH mode, so that no one else may update the file. Other users attempting to update with file will get S494 errors.
If the subfile does not have any immediate indexes, then the subfile will be ready for all to use again upon completion of the command. That is, other users will again be able to update the subfile.
However, if the subfile has any immediate indexes, then SPIRES sets the NOJOBGEN flag when the REMOVE command completes. This is because no index updates (immediate indexes or not) are done by REMOVE with the WITH FAST option. The file must be processed or the DEFQ zapped before any further update requests will be honored.
The WITH SEARCHONLY prefix for REMOVE in SPIRES also speeds up the removal processing, though not as much as WITH FAST. However, it does not have the same requirements and restrictions. Basically, during the processing, SPIRES will reserve the file's CKPT (checkpoint) data set, so that it is not reserved and released for each removal as normal REMOVE processing is done. (That's why other users are allowed to do updates during regular INPUT BATCH processing.) Like WITH FAST, WITH SEARCHONLY puts the file into SET SEARCH mode, so that other users of the file are limited to searching and reporting.
Any immediate indexing is done as normal, and the file is available for normal updating activity upon completion of the WITH SEARCHONLY REMOVE command.
WITH SEARCHONLY, like WITH FAST, is available only to users with Master access to the file.
The DEQUEUE command is used to remove transaction requests against a subfile. For instance, if a record has been updated and should not have been, the updated version can be dequeued. Like the REMOVE command, the DEQUEUE command gains additional power under Global FOR -- more than one record can be dequeued with a single command.
The DEQUEUE options are the same as those for REMOVE:
DEQUEUE [FIRST|*|NEXT|n|REST|LAST|ALL] [END='command']
Since more than one record can be dequeued at a time, using the "n" option indicates that the next "n" records are to be dequeued. If no option is specified, NEXT is assumed. [See 7.3 for an explanation of the END option, 5.2 for more details about the other options.]
The command also has one other option, the prefix WITH SEARCHONLY, which can be used by the file owner and by users with Master access to the file to speed up the processing. See the description of WITH SEARCHONLY in the preceding section on the REMOVE command. [See 5.6.]
To correct the mistake the user made in the example in the previous section [See 5.6.] where instead of removing record 214, the user removed the next 214 records under Global FOR, you could do the following:
-> select restaurant -> find date-updated before 1978 -Result: 67 RESTAURANTS -> for removes via result +> dequeue all -Dequeued: 65 records +>
The reason that 65 rather than 67 records were dequeued is because only 65 records in the result had been removed -- the first two records in the result were not. If the 65 records were the only records that had been removed that day, you could instead issue the DEQUEUE ALL command under FOR REMOVES rather than FOR REMOVES VIA RESULT.
The UNQUEUE command is used to undo the previous transaction in the deferred queue for a given record. For example, if a record is updated and then updated again, the UNQUEUE command would undo the second update, leaving the record as it was updated the first time. Under Global FOR, multiple records can be unqueued with a single command:
UNQUEUE [FIRST|*|NEXT|n|REST|LAST|ALL] [END='command']
Because multiple records can be unqueued at one time, the "n" option indicates that the next "n" records are to be unqueued. If no option is specified, NEXT is assumed. [See 7.3 for an explanation of the END option, 5.2 for more details about the other options.]
The command also has one other option, the prefix WITH SEARCHONLY, which can be used by the file owner and by users with Master access to the file to speed up the processing. See the description of WITH SEARCHONLY in the preceding section on the REMOVE command. [See 5.6.]
The UNQUEUE command provides a better solution to the problem discussed in the previous two sections where the user removed a large group of records by accident. [See 5.6, 5.7.] Here is the UNQUEUE solution:
-> select restaurant -> find date-updated before 1978 -Result: 67 RESTAURANTS -> for removes via result +> unqueue all -Unqueued: 65 records +>
The result of unqueuing the records could be different from the result of dequeuing them: records that had been updated before being accidentally removed would be restored to their updated state by UNQUEUE but to their tree version by DEQUEUE.
Remember that the UNQUEUE command itself creates a transaction for each record, meaning that a subsequent UNQUEUE command will "unqueue the unqueue", and so on.
The STACK command is used to create stacks of records which can be processed either by the TYPE or SEQUENCE commands or which can be processed by Global FOR commands under a FOR STACK command. Outside of Global FOR, stacks are created by issuing the STACK command followed by the key of the record to be stacked, which means that they are built up one record at a time. (Stacks can also be created outside of Global FOR by sequencing a search result or by using the EXTRACT command.) In Global FOR mode, the STACK command can stack groups of records at one time.
The syntax of the STACK command is
STACK [FIRST|*|NEXT|n|REST|LAST|ALL] [END='command'] [CLEAR]
Since the STACK command can process more than one record per request, "STACK n" requests that the next "n" records be stacked. By default, STACK issued alone is equivalent to STACK NEXT. The CLEAR option eliminates any previous stack you have built. [See 7.3 for an explanation of the END option, 5.2 for more information about the other options.]
A stack created under Global FOR always consists of record keys, a fact that is important to remember when you are concerned about processing efficiency. [See 8.7.] Remember that a stack does not contain the records themselves -- only the keys (or pointers, in the case of a some sequenced search results). Thus, anytime you process a stack, SPIRES accesses the latest copy of each record (unless you specify otherwise), not the record as of the time the stack was created.
Remember too that a stack can be saved, using the "STORE STACK.name" command. If you want to process the same subset of records several times over several terminal sessions, you can store the stack and later restore it.
Suppose you need a list of all the drinks in the DRINKS subfile that you have added, and the list needs to be in alphabetical order by drink name:
-> select drinks -> for subfile where contributor ga.jnk +> stack all -Stack: 74 RECORDS +> sequence name -Stack: 74 RECORDS +> in active clean type +>
You must use the STACK command if you want to examine the original versions of records that have been removed since the file was last processed. [See 3.2.]
It is possible to use the STACK command during FOR STACK processing to create a second stack from the original one. SPIRES can basically only maintain one stack at a time. As soon as a STACK command is issued under a FOR STACK command, SPIRES begins building a new stack. The old stack, for most intents and purposes, is gone; any of the commands used to process a stack (e.g., TYPE, SEQUENCE) will end Global FOR processing, replacing the old stack with the new one:
-> show stack -Stack: 15 RECORDS -> for stack where date-updated > April 1980 +> stack 5 -Stack: 5 RECORDS +> sequence date-updated -Stack: 5 RECORDS ->
The SEQUENCE command "closed" the new stack being created, replacing the old stack with this new one. If you had issued a STACK NEXT command instead of SEQUENCE, however, SPIRES would process the next record in the original stack, which is still available for Global FOR processing. As soon as you leave Global FOR mode in any way (by issuing an ENDFOR, or say, another FOR STACK command) the newly created stack replaces the old one. (On the other hand, issuing the FOR ** command lets you begin processing the old one again, not the new one.) By the way, issuing the CLEAR STACK command will eliminate both stacks.
Within Global FOR processing, the UNSTACK command is used to remove groups of records from an existing stack.
The syntax of the UNSTACK command is
UNSTACK [FIRST|*|NEXT|n|REST|LAST|ALL] [END='command']
"UNSTACK n" removes the next "n" occurrences of records in the FOR class from the stack. [See 7.3 for an explanation of the END option, 5.2 for more information about the other options.]
For example, suppose you wanted to work with a stack of all the drinks in a subfile except those drinks containing coffee:
-> select drinks -> find ingredient coffee -Result: 3 BEVERAGES -> for subfile +> stack all -Stack: 170 BEVERAGES +> for result +> unstack all -Stack: 167 BEVERAGES
In this example, the keys defined by the result are removed from the stack.
The SHOW KEYS command is used to examine the key values of records in the class specified in the "FOR class" command. It operates similarly to the DISPLAY command except that only record keys are shown to you. Like most of the remaining commands in this chapter, it can only be issued when you are in Global FOR mode.
The SHOW KEYS command in Global FOR has the following syntax:
SHOW KEYS [FIRST|*|NEXT|n|REST|LAST|ALL] [END='command']
If the "n" option is used, the keys of the next "n" records will be displayed. If no option is used, NEXT is assumed. [See 7.3 for an explanation of the END option, 5.2 for details about the other options.] The IN ACTIVE prefix can be used to place the keys in your active file.
Suppose you just wanted to get an idea of what records had been added to a subfile since the file was last processed:
-> for adds +> show keys all Adams Buchanan Coolidge Dewey Eisenhower Fillmore +>
The SKIP command allows you to "skip to" a particular record in the specified Global FOR class -- that is, to set that record as the "current record". [See 5.2.] Because it processes records, it is affected by any WHERE clause in effect: for instance, SKIP LAST moves the current record pointer to the last record that fits the WHERE clause criteria, not necessarily the last record in the class, although all records, including the last one in the class, are examined. The SKIP command is only valid in Global FOR mode.
The SKIP command in Global FOR has this syntax:
SKIP [FIRST|NEXT|n|LAST] [END='command']
The easiest way to understand the SKIP command is to remember that, to SPIRES, SKIP means "set the current record pointer to". Therefore, "SKIP n" tells SPIRES to set the current record pointer to the "nth" record past the current one (the "nth" record that fits the WHERE clause if one is in effect, of course). Obviously, with that definition, SKIP ALL and SKIP REST are meaningless. However, if issued, SKIP ALL is equivalent to SKIP FIRST and SKIP REST is equivalent to SKIP NEXT. (A warning message telling you what action has been taken will be issued by SPIRES in these cases.) "SKIP *" clearly has no effect on processing at all. [See 7.3 for an explanation of the END option, 5.2 for details about the other options.]
The SKIP command gives you extra flexibility in moving the record pointer around. Suppose, for instance, that you have displayed a record under Global FOR and now want to display the third record, and only the third record, from the current one:
+> skip 3 +> display * NAME = Dewey; +>
If you had said "DISPLAY 3" instead of using the SKIP command, you would have seen the next three records, rather than just the third of the group.
SKIP is perhaps most often used as a searching tool in conjunction with the SHOW LEVELS command. [See 6.1 for an example.]
The REFERENCE command is used to bring a particular record into the main memory of the computer, usually so that partial record processing can be done. Thus, the command is primarily useful for Partial FOR processing. It is occasionally used in protocols as well, usually to access record element values using the $GETUVAL and $GETCVAL functions.
Outside of Global FOR, the REFERENCE command, like TRANSFER, is followed by the key of the record to be referenced. In Global FOR, however, the key is not used, since the records to be processed are determined by the "FOR class" command. Also, like the TRANSFER command in Global FOR, only one record can be referenced at one time.
The syntax of the REFERENCE command in Global FOR is
REFERENCE [FIRST|*|NEXT|n|LAST] [END='command']
Since REFERENCE can only process one record per request, "REFERENCE n" references the "nth" record from the current one. REFERENCE ALL and REFERENCE REST are equivalent to REFERENCE FIRST and REFERENCE NEXT respectively. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]
Both of these commands, valid only in Global FOR mode, are used to create sets of records that can be sorted by the batch processor SPISORT. (SPISORT is often used when the group of records being sorted is too large for the SEQUENCE command or when more sophisticated sorting requirements are needed than can be provided by the SEQUENCE command. A set (before or after sorting) can be processed by Global FOR commands under FOR SET. [See 2.11.]
These commands are discussed in detail in "SPIRES Technical Notes".
The DEFINE SET command does not actually process any records; like the SET FORMAT $REPORT command, it sets up a particular environment for records when they are processed. The GENERATE SET command, like the other commands discussed in this chapter, does process records. Its syntax is:
GENERATE SET [FIRST|*|NEXT|n|REST|LAST|ALL] [END='command']
Note that ALL is the default option rather than NEXT. [See 7.3 for an explanation of the END option, 5.2 for information about the other options.]
Be sure that you issue the ENDFOR command or leave Global FOR in some other way before you run a SPISORT job that uses the set -- leaving Global FOR "closes" the set, and it is not complete until that occurs. If you run a SPISORT job that executes before you have closed the set, the sorting will not be done. [See 7.]
This command, which directs SPIRES to create an ORVYL file of SPIRES records in their internal form, is fully explained in the manual "SPIRES File Management".
The syntax of the command is:
GENERATE LOAD loadname [REPLACE] [FIRST|*|NEXT|n|REST|LAST|ALL] ... ... [END='command']
where "loadname" is the name of the ORVYL file to be created (or replaced) by SPIRES. By default the option ALL is assumed, as is the case with the GENERATE SET command. [See 5.12.] GENERATE LOAD is an "n" command, meaning that if the "n" option is used, the next "n" records will be processed. [See 7.3 for an explanation of the END option, 5.2 for details about the other options shown here.]
Each of the commands discussed in the previous chapter caused records to be processed under Global FOR mode. The Global FOR commands in this chapter are different from those in the preceding chapter in that they do not process records. The SHOW LEVELS command provides information about the records already processed under Global FOR. The SET SCAN commands give you extra control over the records to be examined and processed under Global FOR by letting you place restrictions on the keys of the records to be examined and processed. The commands of this chapter will help you use other Global FOR commands to process records.
In Global FOR mode, the SHOW LEVELS command tells you how many records have been processed under the current "FOR class" command. It is very useful as a searching tool. The SHOW LEVELS command does not process any records; it only provides information about current Global FOR processing.
The SHOW LEVELS command (which takes no options) reveals three important pieces of information, as the following example shows:
-> for adds where phone occurs > 0 +> skip last +> show levels Processed Examined FOR ADDS 3 6 +>
First, SHOW LEVELS displays the Global FOR class, including the VIA clause, if there is one; second, it tells you the number of records (including the current one) that have been examined, found to fit the WHERE clause criteria, if any, and then processed; and third, it displays the number of records in the class that SPIRES has examined, whether or not the examined records fit the criteria. If no WHERE clause is in effect, both numbers will be the same, reflecting the number of records (through the current one) processed by SPIRES.
The above example tells you that in the class of added records, six records exist and three of them have at least one occurrence of the phone number element. If no records had been processed yet, no numbers would be displayed in response to the SHOW LEVELS command. Note that SKIP LAST tells SPIRES to examine all the records, though the current record pointer is set to the last record found that fits the criterion; thus, the SHOW LEVELS information accurately reflects the fact that all the records (not just the records up to the last one fitting the WHERE clause criteria) have been examined.
The two values shown represent the values in the two SPIRES system variables $GXCOUNT (Global FOR eXamined COUNT) and $GPCOUNT (Global FOR Processed COUNT). [See 8.2.] You should use these system variables, rather than the command IN ACTIVE SHOW LEVELS, to pass on information on Global FOR processing (e.g., to a protocol).
When used together during Global FOR searching of non-indexed elements, SKIP LAST and SHOW LEVELS can display a "result count" analogous to the result count you would get after a FIND [index-name] command. For example, you might use the two commands together in order to search for all occurrences of an address element that are too long for a mailing label:
-> for tree where address length > 30 +> skip last +> show levels Processed Examined FOR TREE 1 1324 +>
(Remember that you could look at the single record that fits the criteria by using the "DISPLAY *" command.) If none of the records have an address element that is too long you are automatically removed from Global FOR:
-> for tree where address length > 30 +> skip last -End of global FOR ->
The SHOW LEVELS command is also very important in partial record processing, where it provides different information.
Whenever you interrupt ongoing Global FOR processing by pressing the BREAK key, you will see a display of how many records had been processed and examined up to the time of interruption:
-> for subfile where phone occurs > 0 +> display ... <--------(ATTN/BREAK or equivalent key pressed) -Processed: 28, Examined: 29 -Continue processing records?
The numbers displayed can help you decide whether to continue processing or not.
Global FOR processing will also be interrupted when a specified number of CPU seconds has elapsed if the SET TIMER PAUSE command is in effect. [See B.3.6.2 in "SPIRES Searching and Updating".] For example:
-> set timer 1 -> set timer pause -> for subfile where phone occurs > 0 +> display -Elapsed CPU time: 1 second -Processed: 5, Examined: 6 -Continue processing records?
The SET SCAN commands give you the extra ability to further subset the records being processed in Global FOR mode. Unlike WHERE clauses, which cause SPIRES to examine all records in the given class in order to prevent records not fitting the WHERE clause criteria from being processed, the different SET SCAN commands control how many or even which records are examined. Thus, the SET SCAN commands determine which records are examined; a WHERE clause determines which of the examined records are then processed.
The keys of records in the TREE and DEFQ (and thus ADDS, UPDATES and REMOVES) and SUBFILE (which merges TREE and DEFQ) classes are kept in ascending order. Several of the SET SCAN commands take advantage of this built-in indexing. For example, if you specify a starting point of the record whose key is GQ.JNK (i.e., SET SCAN START GQ.JNK) then SPIRES can pass over all the records in the tree from the beginning to the record with that value when you issue a Global FOR command that processes records, for SPIRES knows that no records before that record will have key values after "GQ.JNK". Because the record keys are readily available, SPIRES can quickly find the starting point without actually examining the records themselves (just as if using an index), which is quite an advantage over WHERE clause processing, where SPIRES must examine the records. Thus, with judicious use of the SET SCAN commands, you can limit the amount of record examination done by SPIRES, thus saving computer resources and time.
The SET SCAN commands, valid only in Global FOR mode, are
SET SCAN {FORWARD|BACKWARD} SET SCAN START value SET SCAN STOP value SET SCAN PREFIX value SET SCAN LIMIT number SET SCAN SKIP number * * SET SCAN SKIP, available only for result and stack processing, is quite different from the others. See its section below for more details.
Though only one "option" can be issued at a time, several SET SCAN commands can be in effect simultaneously during a Global FOR session. They are not available when a VIA clause is used on a "FOR class" command, except under FOR SUBFILE VIA TREE. [See 3.] Each different form of the command is discussed in sections below.
The CLEAR SCAN command can be used to eliminate a particular SET SCAN command in effect:
CLEAR SCAN {BACKWARD|START|STOP|PREFIX|LIMIT}
Generally, it is not advisable to change a SET SCAN command in effect (except to CLEAR it) when you are in the middle of processing records. Instead, starting the processing over again, by issuing a "FOR **" command and then the new SET SCAN commands, is preferable. Issuing a SET SCAN START or SET SCAN PREFIX, however, will reinitialize the Global FOR processing, as if a "FOR **" had been issued. Changing a SET SCAN command in effect after records have already been processed may not produce the desired result. This caution does not apply to SET SCAN BACKWARD and SET SCAN FORWARD, described below.
Here is a chart showing the SET SCAN commands allowed under various "FOR class" commands. The SET SCAN commands generally cannot be issued when an access class or VIA clause occurs in the "FOR class" command. (There are two exceptions: SET SCAN LIMIT is allowed under any "FOR class" command; SET SCAN SKIP is allowed only under FOR RESULT or FOR STACK.)
+----------+----------------------------------------------------+ | | Global FOR Class | | SET SCAN | TREE, SUBF, DEFQ, | RESULT, | SET, PATH, RES DATA, | | Option | ADDS, UPDS, REMS | STACK | STORED, INDEX, LOAD | |----------+-------------------+---------+----------------------| | FORWARD | | NA | NA | | BACKWARD | | NA | NA | | START | | NA | NA | | STOP | | NA | NA | | PREFIX | | NA | NA | | LIMIT | | | | | SKIP | NA | | NA | +----------+-------------------+---------+----------------------+
- "NA" indicates that the named SET SCAN option is not available for these Global FOR classes.
By default, when SPIRES processes the records in a retrieval class, it proceeds forward through key order -- that is, through the order in which the keys are kept in the class. In the TREE and DEFQ, for example, keys are kept in ascending order; in a STACK they are in the order in which they were stacked or sequenced.
There are no options on the commands:
SET SCAN FORWARD SET SCAN BACKWARD
SPIRES normally "scans forward" through the class of records during Global FOR processing; SET SCAN FORWARD is the default. However, you can also process the records in the reverse order, going "backward" through the class. To do this, issue the SET SCAN BACKWARD command:
-? select people -? for tree +? show keys 6 AA.AJM AA.CSB AA.MQR AA.RSC AA.TEG AC.JAM +? set scan backward +? show keys * AC.JAM +? show keys rest AA.TEG AA.RSC AA.MQR AA.CSB AA.AJM +?
These two commands can be used with any of the tree-structured classes (TREE, SUBFILE, DEFQ, ADDS, UPDATES, REMOVES).
As the example indicates, these commands start from the current position. If you don't have a current position established, SET SCAN BACKWARD normally starts at the end of the class, and SET SCAN FORWARD starts at the beginning of the class. But if you've done SET SCAN START, they both start from that starting point and move in the appropriate direction. Normally SET SCAN BACKWARD stops at the beginning of the class, and SET SCAN FORWARD stops at the end of the class. But if you've done SET SCAN STOP, both will stop at that stopping point. Therefore, SET SCAN BACKWARD treats SET SCAN START and SET SCAN STOP as beginning and ending points respectively, but in reverse keys order.
The SET SCAN START command tells SPIRES that no records are to be processed by subsequent Global FOR commands until a key of "value" or higher is found:
SET SCAN START value
For example,
-> select jokes -> for subfile +> set scan start 100 +> show keys 3 100 102 103 +> show key first 100 +>
The command can be used with those classes that are tree-structured (TREE, SUBFILE, DEFQ, ADDS, UPDATES, REMOVES) using any subfile. The value need not be an actual value in the class. SPIRES will process the next record after the key specified, if the specified record does not exist.
The value is processed through the key's INPROCs. If that processing fails, the command will fail too. In some cases, such as check digit processing, an INPROC may be ignored. [See 4.1.1, Processing Rules and WHERE Clauses.]
The SET SCAN STOP command tells SPIRES that no records are to be processed by Global FOR commands after a key of "value" or higher has been reached. If it exists, the record whose key is "value" will be processed.
SET SCAN STOP value
The value is processed through the key's INPROCs, just as it is with the SET SCAN START command. [See 4.1.] Both commands can be used with those classes that are tree-structured, using any subfile. They are often used together to specify a range of records to be examined for processing by Global FOR commands.
The SET SCAN PREFIX command specifies a value that must begin the key value of all records being examined for Global FOR processing:
SET SCAN PREFIX value
It can be used with any tree-structured class.
For example,
-> select people -> for tree +> set scan prefix ga +> show keys all GA.BPB GA.JNK GA.JSM GA.RHM GA.SPI +>
The SHOW KEYS ALL command processed only records with keys beginning with the string "GA".
Generally, the SET SCAN PREFIX command is useful only with subfiles having keys stored as string values. You will probably not find it useful with a slot subfile, which stores the key as a binary value, for example. It is not useful with subfiles having augmented keys or record keys that are structures.
The value is processed through the key element's INPROCs (if any) when the command is issued; the command can thus fail if the value cannot be processed by the rules. [See 4.1.]
This command can be used to limit the number of records examined by SPIRES during Global FOR processing. It does not specify the number of records that will be processed; if there is a WHERE clause in effect, that number could be much smaller.
The syntax of the command is:
SET SCAN LIMIT n
where "n" is an integer. Unlike all the other SET SCAN commands, the SET SCAN LIMIT command can be issued with any subfile under any "FOR class" command.
Here is an example of the command:
-> select people -> show subfile size -The Subfile has 107 Records. -> for tree where bus-phone prefix 497 +> set scan limit 10 +> show keys all AA.AJM AA.MQC AA.TEG AC.JAM +> show levels Processed Examined FOR TREE 4 10 +> show keys rest -End of global FOR ->
Though the subfile has 107 records, only ten are examined by SPIRES when the SHOW KEYS ALL command is issued. Only four keys are displayed since only four of the ten records examined fit the WHERE clause criteria, as shown in the response to the SHOW LEVELS command. Since SPIRES is allowed to examine only ten records, requesting more (SHOW KEYS REST) causes an "-End of global FOR". Note that if SHOW KEYS ALL had been issued again rather than SHOW KEYS REST, then SPIRES would begin again to do the same ten records.
This command allows you to set the starting position of a Global FOR process at the "nth" entry of the result or stack. Records before the "nth" entry are skipped before any Global FOR processing, including WHERE clause handling, begins.
The syntax of the command is:
SET SCAN SKIP n
where "n" is an integer specifying the number of record pointers to be skipped. The command may only be issued under FOR RESULT and FOR STACK.
Compare these two examples:
1) -> for stack where identifier = SU +> set scan skip 10 +> show level Processed Examined FOR SUBFILE VIA STACK +> 2) -> for stack where identifier = SU +> skip 10 +> show level Processed Examined FOR SUBFILE VIA STACK 11 25 +>
In the first example, the SET SCAN SKIP 10 command tells SPIRES to skip the first ten records in the stack. The SKIP command in the second example tells SPIRES to skip the next ten records in the stack that fit the WHERE clause criteria. As the SHOW LEVEL commands show, the SET SCAN SKIP command does not process or examine any records; the SKIP command does. [See 5.10.]
Like some other SET SCAN commands, the SET SCAN SKIP command does not have any effect if issued after Global FOR record processing commands have been issued. For example, in this command sequence:
FOR RESULT REMOVE 5 SET SCAN SKIP 20
the SET SCAN SKIP command would be ignored.
If the stack or result contains duplicate record pointers next to each other in the stack, the extras (i.e., those after the first one) will be skipped but not included in the count.
The SET SCAN INCREMENT command gives you another option for subsetting records being processed in GLOBAL FOR mode.
The command syntax is:
SET SCAN INCREMENT number
As an example, if you want to Display every 10th record from a database you can issue the following commands:
For Subfile Set Scan Increment 10 Display ALL
The result of this process is the output of records 1, 21, 31, 41 ... from the database -- or ten percent of the file.
This command is designed to work in conjunction with other GLOBAL FOR subset processes such as WHERE Clauses, SET SCAN START, STOP, PREFIX and LIMIT options, and SPIRES SCAN Filters. The record incrementing activity takes place after all the preceding subset activity is complete.
The SHOW LEVELS command and the $GPCOUNT variable will reflect this activity in that the number of records processed is the incremental number produced. In the example above the PROCESSED count will be one tenth the value of the EXAMINED count.
This command is valid only for SPIRES Tree processing activity such as FOR SUBFILE, FOR TREE, FOR DEFQ. It is not valid during FOR STACK, RESULT, TRANSACTIONS etc. [See 6.2.6.]
The CLEAR SCAN INCREMENT command may be issued to turn the incrementing process off, or you can SET SCAN INCREMENT with an increment of 0 or 1 to clear any existing increment.
Eventually, when you have finished your record processing, you will want to leave Global FOR mode. Sometimes this occurs automatically, when you have asked SPIRES to process a record that does not exist (the NEXT record after ALL have been processed, for instance). Another way is to issue the ENDFOR command, available anytime during a Global FOR session. Logging off, exiting from SPIRES or selecting another subfile will also automatically remove you from Global FOR mode. This chapter will discuss the first two ways and also explain the END clause, a useful feature in SPIRES protocols.
The ENDFOR command is used to leave Global FOR mode directly.
+> endfor -End of global FOR ->
Note that the smallest allowed abbreviation is ENDF rather than END, which is a WYLBUR command.
The command can be issued at any time during a Global FOR session. Its effect is to remove you from Global FOR. The same subfile remains selected, any stack or search result still exists (with one exception: if you have issued the STACK command under FOR STACK, the new stack replaces the old one when you leave Global FOR mode) and any records processed (updated, for example) remain that way. SPIRES does not change the status of any records when you leave Global FOR; even if you have a record transferred when you leave Global FOR, SPIRES still awaits its return via updating:
+> transfer next clear +> endfor -End of global FOR -> update ->
Remember that you do not have to leave Global FOR mode (by issuing the ENDFOR command, for instance) before issuing a new "FOR class" command.
If you have to leave Global FOR mode temporarily (perhaps overnight) but would like to return later to the exact point where you leave off, you can use the SHOW LEVELS, SKIP and "FOR *" commands to save yourself time and money. For example, suppose you have issued these commands and then ended your session:
-> for tree where date-updated before 1976 <--- You process numerous records. +> show levels Processed Examined FOR TREE 19 186 +> endfor -End of global FOR ->
The SHOW LEVELS command tells you that SPIRES has examined 186 records, nineteen of which fit the WHERE clause, causing them to be processed. Later, when you return, you can continue from the same point by issuing these commands:
-> for tree +> skip 186 +> for * where date-updated before 1976 <--- You continue processing records
This solution is useful whether or not a WHERE clause is in effect. If no WHERE clause is in effect, you do not have to issue a "FOR *" command. (Of course, if you were working with one of the deferred queue classes, such as ADDS, or if the file has been processed since you were last working with it, new records may have been placed in the class or others removed when you return to the subfile. In this case, returning to the same point to continue processing will be more difficult.)
A more efficient method for returning to the same position can be used if the Global FOR class with which you are working allows you to use the SET SCAN START command:
-> for tree where date-updated before 1976 <--- You process numerous records. +> show key next 117 +> endfor -End of global FOR ->
Later, to return to where you were:
-> for tree where date-updated before 1976 +> set scan start 117 <--- You process records, beginning with 117.
As you have seen, sometimes you can be ejected from Global FOR automatically, by requesting a record that does not exist:
-> for adds +> skip last +> display next -End of global FOR ->
After you have skipped to the last record in the class, no "next record" exists, so SPIRES ejects you from Global FOR mode, thereby telling you that the requested record does not exist. If there are no records at all in the class (or no records that fit the WHERE clause, if present), the first Global FOR command to process records would cause an automatic ejection:
-> for adds +> skip last -End of global FOR ->
There is no "last" record in the class to skip to, since there are no records in the class.
Compare the two examples below. In both cases, there are only two more records to be processed in the class (although you, the user, are not aware of this).
+> show keys 5 +> transfer 5 179 -End of global FOR 186 -> +>
In the left example, the two records are processed; in the right example, no record is transferred and you are ejected from Global FOR. What causes the different treatment?
The SHOW KEYS command, like DISPLAY, REMOVE and others, is an "n" command, meaning that more than one record can be processed by the command under Global FOR. The TRANSFER command, as well as REFERENCE, SKIP and MERGE, is an "nth" command, meaning that only one record can be processed by the command. The example thus shows that if the command is an "n" command, then SPIRES will process from one to "n" records in the class, depending on how many records are left to be processed in the class. As long as there is at least one, those records will be processed and SPIRES will not eject you from Global FOR. Presumably, from looking at the results, you would know that there are now no more records to be processed since only two (and not five) keys were displayed.
On the other hand, when you issue an "nth" command, you are requesting a specific, single record to be processed. If that record does not exist (the fifth of two records, in the example), then SPIRES ejects you from Global FOR, just as described above. Again, automatic ejection from Global FOR tells you that the record you requested does not exist.
File owners often write protocols that use Global FOR processing. They are faced with the problem of handling automatic ejection from Global FOR mode, as described in the previous section. [See 7.2.] The users of the protocol are often unaware that they are using Global FOR; they could be quite puzzled by the "-End of global FOR" message. (In fact, from a protocol, that message is replaced by the message "-End clause missing", even more puzzling to a user.) Moreover, the file owner needs to detect that there are no more records to be processed so that subsequent commands that process the records can be skipped.
If you are familiar with the two SPIRES variables $YES and $NO, which detect the success or failure or the previous command, you might try to use them to solve the problem:
+> transfer next -End of global FOR -> if $yes then * The transfer command succeeded. * The transfer command succeeded. ->
Thus, even though no record was transferred, $YES remained true ($NO remaining false) meaning that the TRANSFER command did not "fail", according to SPIRES. (The command would fail if there were no records in the class at all, or if some processing error occurred with the record -- that would set $NO to true.)
The solution which SPIRES provides is the END clause, available with all the Global FOR record processing commands:
command1 END='command2'
where "command1" is a Global FOR record processing command and "command2" is any command that can be issued when you are in SPIRES, including another Global FOR command. The apostrophes are optional if "command2" is a single word; the equal sign ("=") is optional.
For example,
TRANSFER END='JUMP FINISHED' CLEAR DISPLAY ALL END='* No records to display.' IN ACTIVE CONTINUE DISPLAY NEXT END LIST
When a command such as one of the above is issued, the END clause is ignored unless "command1" fails because there are no more records in the class to be processed. When that happens, SPIRES ejects you from Global FOR, suppresses the "-End clause missing" message, and executes the command specified in the END clause.
+> display all end='* No records to display.' * No records to display. ->
The "*" command, which writes the message which follows the "*" to the user's terminal, is executed only if there are no records to be displayed.
Though the END clause can be used in command mode, it is primarily used in protocols. For example, if you want to merge all the records in a class (with or without a WHERE clause), a simple "one-line" protocol such as the one shown below can be used. (Remember that the data to be merged must be in your active file.)
-> for adds where phone occurs = 0 +> :++go//merge end='return'//jump go ->
In the second line of the example, you are giving SPIRES instructions to merge a record, then go back and merge the next one, and so forth; when there are no more records to merge, SPIRES returns from the protocol. [The "one-line protocol" is discussed fully in "SPIRES Protocols".]
In protocol mode, if you are in Global FOR and a record-processing command without an END clause causes you to get an "-End clause missing" message, then an ENDFOR command must be issued before any other Global FOR commands, including any "FOR class" command, can be successfully issued. This is what the situation described above looks like:
-> set echo -> :for removes//show keys all//for updates -> show keys all -End clause missing -> for updates -Endfor command required +>
After you issue the one-line protocol (the command line beginning with the colon ":"), SPIRES takes over, issuing the commands for you. (Note that in protocol mode, when SPIRES is issuing the commands, the Global FOR prompt did not appear until you were prompted again.) Because the SHOW KEYS ALL command lacked an END clause, the subsequent FOR UPDATES command was blocked -- an ENDFOR command must be issued before any more Global FOR activity can occur.
Sometimes in a protocol you might want to protect yourself from the "-End clause missing" and "-Endfor command required" messages but not want to issue a command in an END clause. One way to solve this problem is by using the END clause:
END='-'
where "-" is the SPIRES protocols comment command, which is a command that, in effect, does nothing. In this case, however, it would suppress the "-End clause missing" message if the command caused an automatic ejection from Global FOR without having any effect of its own.
Be aware that when an error really does occur, such as a processing error on a record being transferred that prevents the transfer from occurring, the $NO flag will be set by that failure and will remain set regardless of the outcome of the command listed in the END clause, which would have been executed. That way, you can test for a command failure on the last record without having to worry that the END clause command reset the value of $NO.
Though the preceding chapters have shown you the commands of Global FOR, this chapter will cover Global FOR as a mode -- when you might want to use it, how to use combinations of commands for greater efficiency, and other information and techniques not fully discussed previously.
When you enter Global FOR mode, as you have seen, new SPIRES commands are available (e.g., SHOW LEVELS, SKIP) while others work differently than the way they work outside of Global FOR (TRANSFER, DISPLAY). However, all other SPIRES commands, as well as those of other systems, are available under Global FOR, just as they are outside of Global FOR. For example, under a FOR TREE command you can still issue search requests (FIND command) and examine the retrieved records (TYPE). (Of course, a FIND command will affect the result being processed if you are under a FOR RESULT, and it will eliminate any stack in effect, which would affect you if you were under a FOR STACK command.) You can issue any SHOW command to display information, change the record display format (with SET ELEMENTS or SET FORMAT), add records, and so forth. None of these commands will change the status of your Global FOR processing.
You can also issue other "FOR class" commands when already in Global FOR mode. You do not have to leave Global FOR by issuing an ENDFOR command first. Of course, the "FOR *" and "FOR **" commands can be issued only when you are in Global FOR.
You cannot issue SPIRES commands that are followed by a key value, except UPDATE. For instance, if you are under Global FOR, you cannot issue the command "DISPLAY GA.JNK" if GA.JNK is the key of a record. Under Global FOR, the DISPLAY command has other options, none of which are the record key. Thus, to display (or transfer, merge, remove, dequeue or stack) a specific record by specifying its key value, you must leave Global FOR mode.
The ENDFOR command will explicitly remove you from Global FOR. There are other commands that will also remove you from Global FOR mode, even though they are not specifically Global FOR commands:
SELECT CLEAR SELECT EXIT LOGOFF
If you execute a protocol in SPIRES, it may or may not end your Global FOR processing, depending on the presence or absence of any of the above commands. Also remember that the SEQUENCE or TYPE commands will also end FOR STACK processing when you have been creating a second stack. [See 5.8.]
There are four useful system variables related to Global FOR processing that can be used in SPIRES formats or protocols. The variable $FORTYPE is an integer variable that tells whether or not a "FOR class" command is in effect and what class is being processed. $PRTCNT is also an integer variable. It tells you how many or what records the last Global FOR command was requested to process. $GPCOUNT and $GXCOUNT are integer variables that can give you the same values shown by the SHOW LEVELS command. In many situations, it is preferable to use them rather than a command such as IN ACTIVE SHOW LEVELS. [See 6.1.]
$FORTYPE has a value of zero when no "FOR class" command is in effect, and some other value, as shown below, when one is in effect. Thus, if a protocol needs to determine whether the user is in Global FOR mode (and leave it, if so), the following command could be included:
IF $FORTYPE THEN ENDFOR
meaning "If $FORTYPE has any value other than "0" (which represents 'false') then execute the ENDFOR command."
Here are the values that $FORTYPE can contain:
$FORTYPE FOR 0 (none) 4 SUBFILE 8 TREE 12 DEFQ 16 UPDATES 20 REMOVES 24 ADDS
If you issue a "FOR class" command with a VIA clause (FOR retrieval-class VIA access-class), $FORTYPE represents the retrieval class, not the access class. Remember that the other classes not shown above are really "shortcuts" representing two classes connected by a VIA clause. For instance, if FOR STACK is in effect, then $FORTYPE is "4", since FOR STACK is really FOR SUBFILE VIA STACK. [See 3.]
When commands are being issued under Global FOR (or Partial FOR), such as DISPLAY NEXT, TRANSFER or REMOVE 7, $PRTCNT (for "Partial Count") reflects the number of records to be processed. For example, if the command issued were "REMOVE 7", then $PRTCNT would contain the value "7". Here is a chart of special values $PRTCNT can contain; under the heading "Records Requested" are the values given in the command which would result in the $PRTCNT value shown, such as DISPLAY FIRST.
$PRTCNT Records Requested 1 NEXT 0 * -1 FIRST -2 LAST -3 ALL -4 REST
If $PRTCNT is any other positive number, it is the number of records that were requested to be processed, such as "7" above. Note that $PRTCNT does not reflect how many records were successfully processed; it reflects how many or which records were requested to be processed. When an ENDFOR command is issued, $PRTCNT is reset to "1" by SPIRES; otherwise, the variable is only changed when a Global FOR command, such as DISPLAY, is issued.
$PRTCNT is also very useful in partial record processing.
$GPCOUNT (for "Global Processed COUNT") and $GXCOUNT ("Global eXamined COUNT) contain the same numbers displayed by the SHOW LEVELS command:
-> for adds where phone occurs > 0 +> skip last +> show levels Processed Examined FOR ADDS 3 6 +> /* $GPCOUNT $GXCOUNT * 3 6
Global FOR techniques will probably not be used by the novice who searches in SPIRES subfiles. Generally, using the standard searching techniques (using indexes and the FIND command) on subfiles carefully designed for such, the SPIRES newcomer can find the information wanted. If non-indexed searching is additionally necessary, the novice often can use the ALSO command. [See "SPIRES Searching and Updating", section B.5.3.2.]
Naturally the most efficient searching in SPIRES is done using indexes. Just as it is faster and easier for you to find a book on Beethoven by looking through the subject index of a library's card catalog instead of looking through every book in the library, it is more efficient for SPIRES to examine an index than to examine the goal records themselves. So, before using Global FOR with a WHERE clause to find the records to be processed, be sure that they cannot be found by an indexed search instead.
Remember that if you can do even part of your searching using indexes, you may be able to use the FOR RESULT command with a WHERE clause that specifies the non-indexed portion of the search. This is considerably more efficient than placing all of the search criteria in a WHERE clause under FOR TREE or FOR SUBFILE.
When you are searching in a subfile and want to be certain that you have accessed all the records, you might want to examine the added records. For instance, you might just display all the added records one at a time to see whether any of the records are of interest, or you could place a WHERE clause on the FOR ADDS command to insure that only the records most valuable to you are shown. (If there are many records in which you will not be interested, it is more efficient to process only those in which you will be, i.e., to use a WHERE clause.) A very simple and efficient way to see how many new records there are is:
-> for adds +> skip last +> show levels Processed Examined FOR ADDS 27 27 +>
The example shows that 27 records have been added to the subfile since the file was last processed in SPIBILD. SPIRES did very little work to provide you with this information. (Using a WHERE clause would have increased the work somewhat, of course.)
Finally, it is interesting to note that the command sequence FOR RESULT, DISPLAY ALL is equivalent to TYPE. And in fact, if updated and removed records are of no concern to you, then the command sequence FOR TREE VIA RESULT, DISPLAY ALL is slightly more efficient than the TYPE command, which does not bypass the deferred queue.
In most cases the Global FOR command "FOR RESULT WHERE" will be more useful and efficient than an "ALSO" command. Some general rules can be stated.
- If you have no preceding search result and want to subset the goal records based on unindexed information, you must use Global FOR, but be aware that this is an expensive process for a large file.
- If you have a preceding search result and want to subset the goal records further based on unindexed information, you may use either Global FOR RESULT or the ALSO command.
But:
- If the search result obtained by specifying unindexed criteria must be manipulated further by indexed search commands (FIND, AND, AND NOT, OR), you must use ALSO.
- If the result obtained by specifying unindexed criteria will not be manipulated further by indexed search commands, but will be displayed, it will be about twice as efficient to use Global FOR.
Unlike the FIND command, the ALSO command cannot initiate a search; it must always operate on a preceding search result--in this respect it is like a qualifier. Unlike the ALSO command, the Global FOR commands need not operate via a search result, though they can. However, the Global FOR commands do not automatically provide you with a count of the number of records meeting the criteria specified. Such a count can be obtained, however, and a new set of criteria specified if the number is too small or too large. To get this count, you use the SKIP and SHOW LEVELS commands, as shown in the example below.
The following example shows this process, which involves several examinations of the goal records in the search result, and is therefore rather time-consuming and expensive:
-> find name smith -Result: 22 PEOPLE -> for result where sex = male and eyes = blue +> skip last +> show levels Processed Examined FOR SUBFILE VIA RESULT 4 22 +> for result where sex = male +> skip last +> show levels Processed Examined FOR SUBFILE VIA RESULT 11 22 +>
The system's response to the SHOW LEVELS command gives two numbers: the second indicates the number of records examined--here it is 22, the same as the number of records in the search result. The first number indicates how many of the records examined met the criteria specified in the WHERE clause -- 4 for the first WHERE clause and 11 for the second.
The same results are more directly obtained by the use of the ALSO command, which gives an indication of the number of records meeting the criteria immediately, just as a FIND or other index search command does.
For example:
-> find name smith -Result: 22 PEOPLE -> also sex = male and eyes = blue -Result: 4 PEOPLE -> backup -Result: 22 PEOPLE -> also sex = male -Result: 11 PEOPLE ->
If further index search commands (for example, FIND, AND) are necessary, then the ALSO command must be used, since the "result" of a Global FOR command is not a set of pointers in a search result. The pointers in a search result can be combined logically with the pointers meeting the criteria specified in subsequent search commands. If, however, the records meeting the WHERE criteria are to be displayed at the terminal or placed in the active file, then Global FOR is a more efficient way to do this than the ALSO command.
Compare the two search scenarios following:
-> find name smith -> find name smith -> also eyes string blue -> for result where eyes string blue -> type +> display all <--- Records displayed <--- Records displayed -> +>
The second series of search commands is almost twice as efficient as the first. With the ALSO command, the system must read the goal records to examine the "eyes" element, then read the records meeting the criteria a second time when a TYPE command is given. With the FOR RESULT command, the record is read to examine the "eyes" element, then, while the record is still in main memory, it is displayed on the terminal. The net effect is that a record is accessed only once when FOR RESULT is used.
Finally, it should be noted that the ALSO command does not allow OCCURS or LENGTH tests, "same structure" processing, or inter-element relations. [See 4.1.1, 4.4.2, 4.5.] Only the WHERE clause on a "FOR class" command provides those capabilities.
Global FOR can be a very useful tool for people who maintain the data in a subfile. It is often used when a group of records fitting certain criteria must all be updated or removed. The records can perhaps be found by an indexed search and then sequenced if necessary. Then, under either FOR RESULT or FOR STACK, you could transfer and update each of the records (or merge or remove, as appropriate). (Unless you are the only person updating the subfile and have a good memory, you probably want to use FOR RESULT rather than FOR TREE VIA RESULT, for instance, to be certain that records already updated are processed in their most recent form.)
If you are updating many records under this method, your work may be interrupted, perhaps for lunch or a meeting. Two methods for "saving your place" were discussed earlier in this manual. [See 7.1.] By storing the result or stack and using the SHOW LEVELS and SKIP commands, you can usually return to the point where you left off.
One problem you might occasionally face is updating a record twice, the second time being a mistake. Using FOR TRANSACTIONS, you can retrieve the "middle" version of the record (the first version being the original, the last being the mistake). If there has been a lot of updating activity, you might want to use a WHERE clause that specifies the key element and its value:
-> for transactions where id=100 +> transfer +> update +>
This sequence would correct the problem as stated. Of course, if you had updated the record several times in a single day, you would want to examine each version of the record under FOR TRANSACTIONS until you found the proper one, then transfer and update that one. [See 2.8, 6.2.]
Naturally, to review your updating work, you can use the FOR DEFQ, FOR ADDS, FOR UPDATES and FOR REMOVES commands. Reviewing such work is considered part of data base management, discussed in the next section.
One of the major tasks of the data base manager is to review the updating activity on the subfile. As data base manager, you would use FOR ADDS to examine the records being added. You could compare the updated records to their original versions by displaying them first under FOR UPDATES and then stacking the record keys and then examining the records under FOR TREE VIA STACK. You could examine the records being removed by stacking the records under FOR REMOVES and then examining them under FOR TREE VIA STACK. [See 3.2.] In any of these situations, you could dequeue any records in which you detected an error, knowing that if your "correction" were in error, you could recover from that as well, since all copies of added and updated records can be processed under FOR TRANSACTIONS and removed records that are dequeued can always be removed again later. Remember that, for file owners, the SHOW SUBFILE TRANSACTIONS command is also useful in regard to managing the updating activity.
Global FOR is also useful when you need to work directly with other record-types of a file, usually indexes. For example, suppose you wanted to have a list of the values in an index. Using the BROWSE command you would only be able to see a few at a time:
-> select albums -> browse instrument ALTO CLAVICHORD DOUBLE HAMMERKLAVIER MARTENOT PERCUSSION SYNTHESIZER TYPEWRITER ->
On the other hand, the file owner can directly access the index records by issuing the ATTACH command and then examining the index records as desired using Global FOR:
-> attach 5 of recordings -> show element names INSTRUMENT (key) POINTER -> for tree +> show keys 3 ALTO BARITONE BASS +> in active show keys all +>
In the above example, after attaching the appropriate index record, you were able to treat the attached record-type as a subfile. Of course, since no indexes are defined for an index (usually), the only way to work with these records is by working with them individually, using the key of a record (DISPLAY ALTO, for instance), or by using Global FOR commands to process records, as shown above.
Working with the index records this way is often necessary in creating reports. In such cases you take advantage of the indexes maintained by SPIRES to produce a report displaying the records in the order in which they are indexed. That is, you can write a report format that displays information from each goal record in the subfile for which there is a pointer (and hence a record) in the index. For example:
GRADUATE-1 Amanda B. Reckonwith Music Lilac A. Rugg French Lit. GRADUATE-2 Dan DeLyon German Studies Barbie Kew Computer Science Elmer Sklue English (etc.)
Once you have created the report format, you attach the index, set the format, issue the SET REPORT command (if necessary) and then, using Global FOR commands (such as FOR TREE, IN ACTIVE DISPLAY ALL) produce the report. [See "SPIRES File Definition", section B.5.6, for information about the ATTACH command. See "SPIRES Formats", section I.3.2.2 for information about report formats.]
This section demonstrates particular methods for handling some standard problems that are solved using Global FOR. Though some of them have been discussed already, you may find it useful to have this information together in this section as well.
-> for defq +> display all
or issue the IN ACTIVE DISPLAY ALL command to place copies of the records in your active file.
-> clear stack -> for updates +> stack all -Stack: 23 RECORDS +> for tree via stack +> display all
-> clear stack -> for removes +> stack all -Stack: 8 RECORDS +> for tree via stack +> display all
-> clear stack -> for tree where contributor ga.jnk +> stack all -Stack: 74 RECORDS +> sequence name -Stack: 74 RECORDS +> type
First create a stack of the records desired, and then issue the SEQUENCE command to sort them.
-> for subfile where date-added after 6/1/1980 +> display <--- The first record is displayed. +> display ... +> display <--- The desired record is displayed. +> remove * +> display <--- The next record is displayed. ...
You may be able to shrink the number of records examined by using a WHERE clause and SET SCAN commands. Then examine the records one by one, using the DISPLAY command. (You may want to make the record display shorter by issuing the SET ELEMENTS command, so that you only see certain elements of the record.) When you have found the desired record, issue the desired record-processing command with the "*" option.
When you issue commands to process records in Global FOR, SPIRES tries to handle your request as efficiently as possible. Knowing what information is most easily available to SPIRES in the various classes will help you choose the most efficient way of processing.
In fact, if you want to maximize efficiency, you should consider many different aspects of Global FOR processing. The class being processed, the access class of the VIA clause, the content of the WHERE clause, the Global FOR processing command, and the file structure itself are all important aspects, and each should be considered carefully if you want to save the most money. This section will present useful information about and helpful suggestions for increasing Global FOR efficiency.
Generally commands that need to access only the record key are more efficient than those that need the entire record. Thus SHOW KEYS, SKIP, REMOVE, DEQUEUE and STACK are usually more efficient than DISPLAY, TRANSFER, MERGE, REFERENCE, GENERATE SET and GENERATE LOAD. (They are always at least as efficient, and usually more efficient.) All the record processing commands are equally efficient if a WHERE clause is in effect, since SPIRES must get the whole record anyway for examination. (An exception: if the WHERE clause specifies the key element, SPIRES does not need to fetch the record, so the distinction between the two types of commands would exist. Note in this regard though that if you can use a SET SCAN command using the key rather than a WHERE clause, you are being more efficient.)
In some subfiles, goal records are not REMOVED to a residual data set but are actually stored completely in the tree. In some other subfiles, the elements that are declared in the FIXED section of the file definition, as well as the key, are in the tree while the rest of each record is stored in the residual data set. In such cases, a WHERE clause that specifies an element that is in the tree is more efficient than one that specifies an element in the residual data set. And if the entire record is in the tree, all the record processing commands are equally efficient, regardless of whether the command needs the entire record or only the key.
In regard to FOR SUBFILE processing, remember that the larger the deferred queue, the more expensive it will be to process records. However, processing the file in SPIBILD in such a case and then using FOR TREE instead would probably be more expensive than simply using FOR SUBFILE, unless you are going to process the file anyway.
When you create a search result, it is comprised either of keys (if the goal record data set is not REMOVED) or of pointers to the goal records (if it is REMOVED). In either case, the information contained in the result lets SPIRES access the records in the quickest and most direct way. If the goal records are in the tree, SPIRES goes directly to the tree (using the keys) to access them. If they are REMOVED, then SPIRES goes directly to them in the residual data set using the pointers, bypassing the TREE entirely. Since in either case there is no quicker way to get the key than to get the whole record (as there is for some cases of FOR TREE), all the record processing commands are equally efficient, even if a WHERE clause is in effect.
FOR DEFQ VIA RESULT, FOR UPDATES VIA RESULT and FOR REMOVES VIA RESULT are not as efficient as the same commands without the VIA clause if the goal records are REMOVED to a residual data set. Why? Because in that case the result contains pointers, not keys, and the deferred queue records are stored by key. For each record, SPIRES must use the result pointer to find the original record in the residual data set, get the key from the record, and then get the record from the deferred queue, if it exists there. This rule also applies to VIA STACK and VIA SET used with FOR DEFQ, FOR UPDATES, FOR ADDS and FOR REMOVES, if the stack or set (the set being a stored result or stack) contains pointers rather than keys.
FOR TREE VIA RESULT is somewhat more efficient than FOR RESULT (which is really FOR SUBFILE VIA RESULT) since it bypasses the deferred queue. It is important to remember that the indexes reflect the contents of the tree versions of the goal records, so that FOR RESULT may let you access records whose contents have changed so that they no longer fit the search criteria. In general, "FOR TREE VIA access-class" is more efficient than "FOR SUBFILE VIA access-class".
A stack that is created by sequencing a search result containing pointers (i.e., the goal records are REMOVED to a residual data set) is thus made up of pointers. The same rules described above for processing a result apply to such stacks.
A set created by the DEFINE SET and GENERATE SET commands always contains keys, not pointers. Hence, if the goal records processed under "FOR SET setname" are REMOVED to a residual data set, commands that process the record key rather than the entire record will be more efficient, and using only the record key in the WHERE clause, rather than other elements, is also more efficient.
Remember that a set can also be a stored stack or result, which may contain pointers instead of keys. In such cases, the record processing commands are equally efficient, whether they process the key or the entire record.
(The following documents are not SPIRES documents per se, but describe utilities and programs that may be useful in developing SPIRES applications.)
The above documents (except any marked "in preparation") may be obtained through the PUBLISH command on the Forsythe computer at Stanford University. If you do not use SPIRES at Stanford, contact your local system administrator to find out how SPIRES documents are made available there.
SPIRES manuals are updated regularly as changes are made to the system. This does not mean that all manuals are out of date with each new version of SPIRES. The changes to the documentation match those made to SPIRES: they are usually minor and/or transparent. Not having the most current version of a manual may mean you do not have all the most recent information about all the latest features, but the information you do have will usually be accurate.
A public subfile, SPIRES DOC NOTES, contains information about changes to SPIRES manuals. Using this subfile, you can determine whether the manual you have has been updated and if so, how significant those updates are. You need to know the date your manual was published, which is printed at the top of each page. For details on the procedure, issue the command SHOW SUBFILE DESCRIPTION SPIRES DOC NOTES.