INDEX
*  SPIRES Formats
A  Introduction
A.1  Introduction to this Manual
A.2  Notation Conventions
A.3  An Overview of Formats
A.3.1  An Introduction to Formats
A.3.2  The Format Creation Procedure and Some Sample Formats
A.3.3  Basic Concepts of Formats
A.3.4  The Outline of the Manual
B  Output Formats
B.1  Introduction to Output Formats
B.1.1  Do You Need a Custom-Designed Output Format?
B.1.2  Output Formats in General
B.1.3  Format Definitions as Records in the FORMATS Subfile
B.2  The Identification Section
B.2.1  The ID Statement
B.2.2  The COMMENTS (COM) Statement
B.2.3  The AUTHOR Statement
B.2.4  The DEFDATE, MODDATE and MODTIME Statements
B.2.5  The FILE Statement
B.2.6  The RECORD-NAME (REC) Statement
B.2.7  A Sample Identification Section
B.3  Instructions for Formatting: the Frame Definitions Section
B.3.1  The FRAME-ID Statement
B.3.1.1  The COMMENTS (COM) Statement
B.3.2  The DIRECTION (DIR) Statement
B.3.3  The FRAME-DIM Statement
B.3.3.1  (*) The SET NROWS and SET NCOLS Uprocs
B.3.4  The USAGE Statement
B.3.5  The Rest of the Frame Definition
B.4  Label Groups: The Basic Building Blocks of Formats
B.4.1  The LABEL Statement
B.4.2  The GETELEM Statement
B.4.2.1  (*) Other Forms of the GETELEM Value
B.4.3  The VALUE (VAL) Statement
B.4.3.1  (*) Special Characters in the VALUE Statement
B.4.4  The PUTDATA Statement
B.4.5  Label Group Statements that Control the Value
B.4.5.1  The DEFAULT Statement
B.4.5.2  The OUTPROC (OUT) and INPROC (IN) Statements
B.4.5.3  The INSERT (INS) Statement
B.4.5.4  The SET CVAL Uproc
B.4.5.5  The UPROC (P) Statement
B.4.5.6  The ENTRY-UPROC Statement
B.4.5.7  The SET BUILD Entry-Uprocs
B.4.6  Label Group Statements that Control the Placement of the Value
B.4.6.1  The START Statement
B.4.6.2  The SET STARTROW and SET STARTCOL Uprocs
B.4.6.3  The MARGINS (MAR) Statement
B.4.6.4  The BREAK (TRUNC) Statement
B.4.6.5  The LENGTH (LEN) Statement
B.4.6.6  The MAXROWS (ROWS) Statement
B.4.6.7  The SET PADCHAR Uproc and the PADCHAR Statement
B.4.6.8  The SET ADJUST Uproc
B.4.6.9  The SET REPEAT Uproc
B.4.6.10  The SET SKIPROW Uproc
B.4.7  Element Headings: the TITLE and TSTART Statements
B.4.8  Statements that Control Frame Execution; Procedural Statements
B.4.8.1  Condition Tests: The IF... THEN, THEN and ELSE Uprocs
B.4.8.1a  The Block-Construct Uprocs
B.4.8.2  The JUMP and GOTO Uprocs
B.4.8.3  The CASE Uproc
B.4.8.4  The LOOP Statement
B.4.8.5  The SET STARTOCC and SET LOOP BACKWARDS Entry-Uprocs
B.4.8.5a  The SET FILTER and CLEAR FILTER Uprocs
B.4.8.6  Subroutines within Frames: The XEQ PROC and RETURN Uprocs
B.4.8.7  Subroutines that are Frames: the IND-FRAME Statement
B.4.8.8  Leaving a Frame: the RETURN, ABORT, STOPRUN and BACKOUT Uprocs
B.4.8.9  Passing Commands to WYLBUR from a Format
B.4.8.10  Variable Manipulation: The LET and SET Uprocs
B.4.8.11  Terminal Input: The ASK Uproc
B.4.8.12  Terminal Output: The * (Star) Uproc
B.4.8.13  (*) Terminal Input/Output and Buffer Processing
B.4.8.14  Comments as Uprocs: the "-" (Hyphen) Uproc
B.4.8.15  (*) The SITE Statement
B.4.9  The COMMENTS (COM) Statement in Label Groups
B.5  The Format Declaration Section of the Format Definition
B.5.1  Naming the Format: The FORMAT-NAME Statement
B.5.2  Identifying the Frames: the FRAME-NAME and FRAME-TYPE Statements
B.5.3  Limiting Account Access: The ACCOUNTS (ACCT) Statement
B.6  Compiling Your Format Definition
B.6.1  The FORMATS Subfile
B.6.2  Compiling the Format
B.6.3  Recompiling a Format: the RECOMPILE Command
B.6.4  Destroying a Format: the ZAP FORMAT (ZAP FOR) Command
B.7  Using Your Format
B.7.1  Setting a Format: the SET FORMAT (SET FOR) Command
B.7.2  Debugging a Format
B.7.2.1  The SET FTRACE (SET FTR) Command
B.7.2.2  SET FTRACE JUMP
B.7.2.3  SET FTRACE SNAPSHOT
B.7.2.4  SET FTRACE VARIABLES
B.7.2.5  SET FTRACE FRAMES
B.7.2.6  SET FTRACE BRIEF
B.7.2.7  Directing Tracing Output to a Trace Log
B.7.2.8  Using the $FTRACE Variable
B.7.3  A Troubleshooter's Checklist for Format Debugging
B.8  Handling Structures in Output Formats
B.8.1  The PUTDATA Statement and Frame Dimensions for the Indirect Frame
B.8.2  Statements for the Indirect Frame: the SUBTREE Statement
B.8.3  Statements in the Calling Label Group: the IND-STRUCTURE Statement
B.9  User and System Variables in Output Formats
B.9.1  Using Variables in Formats
B.9.2  System Variables that are Useful in Output Formats
B.9.3  User Variables in Formats: Vgroups
B.9.3.1  The Vgroups Section of the Format Definition
B.9.3.2  The ALLOCATE Statement
B.9.3.3  Resetting Variables and Vgroups; the EVAL Uproc
B.10  Report Formats
B.10.1  The Fundamentals of Report Formats
B.10.1.1  Execution Order of Frames in a Report Format
B.10.2  Enabling Report Mode Processing (SET REPORT and WITH REPORT)
B.10.3  Controlling Page Layout for Reports
B.10.3.1  Header Frames
B.10.3.2  Footer Frames
B.10.3.3  Carriage Control in Report Formats
B.10.3.4  The SET NOBREAK Uproc
B.10.3.5  The EJECT PAGE, SET NEWPAGE and SET FRONTPAGE Uprocs
B.10.3.6  The FLUSH Uproc
B.10.3.7  The HOLD Uproc
B.10.3.7a  (*) The SET NOPUTFLAG Uproc and The $PUTFLAG Variable
B.10.3.8  Multiple Column Output
B.10.4  Using the Initial Frame to Provide Prefatory Material
B.10.5  Using the Initial Frame to Initialize System Variables
B.10.6  The Ending Frame
B.10.7  The Break Frame-Types: Group-Start and Group-End Frames
B.10.7.1  The BREAK-ELEM (BRK) and BREAK-LEVEL (LVL) Statements
B.10.7.2  The Group-Start and Group-End Frame Definitions
B.10.7.3  Sorting Records for Report Processing
B.10.7.4  The SET MAXLEVELS Uproc
B.10.8  Sorting Values in Formats
B.10.9  Using Multiple Character Set Fonts in Report Formats
B.10.10  System Variables that are Useful in Report Formats
B.10.11  A Sample Report Format Definition
B.10.12  Sample Output from the Sample Report Format
B.11  Invoking Formats from within Formats: Load Formats
B.11.1  The Load Format Itself
B.11.2  The LOAD-FORMAT (LOAD) and USING-FRAME (USING) Statements
B.11.3  The LOAD-OPTIONS Statement
B.12  Accessing Data in Other Record Types from Output Formats
B.12.1  Phantom Structures
B.12.2  Same-File Subgoal Processing
B.12.2.1  The Indirect Frame for the Subgoal: the SUBTREE Statement
B.12.2.2  The Label Group that Invokes Subgoal Processing
B.12.3  Subfile Subgoal Processing in Output Formats
B.12.4  Accessing Subtrees Within a Subgoal
B.12.5  SUBFILE versus TREE Accessing of Subgoal Records
B.12.6  A Complete Example of a Format Using Subgoal Processing
B.12.7  (*) Subgoal Processing Without Data
B.12.8  Subgoal Access to Transaction Records
B.13  Full Screen Output Formats
B.14  General File Formats for Output
B.15  Output Formats for Partial Processing
B.16  Output Formats with Device Services
B.16.1  Directing Output Frames to Multiple Areas: The IN-AREA Statement
B.16.2  The IN-AREA Statement with Report Formats
B.16.3  Creating Reports During SPIBILD Input
B.16.4  Directing Output Data to Another Subfile as Input
C  Input Formats
C.1  Introduction to Input Formats
C.1.1  Do You Need a Custom-Designed Input Format?
C.1.2  General Differences Between Input and Output Formats
C.1.3  The Input Format Definition
C.2  The Frame Definitions Section of an Input Format Definition
C.2.1  The FRAME-ID Statement
C.2.2  The DIRECTION (DIR) Statement
C.2.3  The FRAME-DIM Statement
C.2.4  The USAGE Statement
C.3  Label Groups for Input Formats
C.3.1  The GETDATA Statement
C.3.2  The VALUE (VAL) Statement
C.3.3  The PUTELEM Statement
C.3.3.1  The SET PUTOCC Uproc
C.3.3.1a  The SET STARTOCC Entry-Uproc
C.3.3.2  The SET SKIPEL Uproc
C.3.4  Changing or Testing the Label Group's Value
C.3.4.1  The INPROC (IN) Statement
C.3.4.2  The SET CVAL Uproc
C.3.4.3  The SET UVAL and DOPROC Uprocs
C.3.4.4  The BUILD RECORD Uproc
C.3.5  Statements that Control the Retrieval Location of the Value
C.3.5.1  The START and XSTART Statements
C.3.5.2  The LENGTH (LEN) Statement
C.3.5.3  The MAXROWS (ROWS) Statement
C.3.5.4  The MARGINS (MAR) Statement
C.3.5.5  The SCAN Statement
C.3.5.6  The SET SCANROW Uprocs
C.3.6  Error Handling in Input Formats
C.3.6.1  The Error Flags: $PROCERR, $APROCERR, $WPROCERR and $GPROCERR
C.3.6.2  The SET TESTFAIL Uproc or Entry-Uproc
C.3.6.3  The SET ERROR Uproc
C.3.6.4  The SET MESSAGES Uprocs
C.3.6.5  The ATTN/BREAK Key and the $ATTN Flag
C.3.7  Execution Statements Used in Input Formats
C.3.8  System Variables used with Input Formats
C.3.9  Designing and Coding Input Frames
C.3.9.1  Multiply-Occurring Elements
C.3.9.2  Structures
C.3.9.3  Guidelines for Data Retrieval with the GETDATA Statement
C.3.9.4  Using the Position Variables
C.3.9.5  The Order of Element Processing in an Input Format
C.3.9.6  (*) Record Keys
C.3.9.7  Formats for the ADDUPDATE and ADDMERGE Commands
C.3.10  A Sample Input Format Definition
C.4  Prompting Input Formats
C.4.1  The Frame Identification Statements
C.4.2  Label Groups for Prompting Input Formats
C.4.2.1  Error Handling
C.4.2.2  Null and Attention Responses
C.4.2.3  Providing Help to the User
C.4.3  A Sample Prompting Input Format Definition
C.5  Merge Formats
C.5.1  Merge Frames
C.5.1.1  The USAGE Statement
C.5.1.2  Handling Elements in Merge Frames: PUTELEM and REMELEM Statements
C.5.1.3  Occurrence Numbers in Merge Frames
C.5.1.4  The SET REMOVEL Uproc
C.5.1.5  (*) Formats Used with the ADDMERGE Command
C.5.1.6  Filtering Merge Input
C.5.2  Referenced Record Processing
C.5.3  (*) The SET PUTSTRUC Uproc
C.6  The Format Declaration Section
C.7  Compiling and Testing Your Format Definition
C.8  Retrieving Data in Other Record-Types from Input Formats
C.8.1  Subgoal Processing for Input Frames
C.9  Multiple-Record Input Formats for SPIRES and SPIBILD
C.9.1  General Rules about Batch Input Formats
C.9.2  Multiple-Record Input Formats for Adding Records
C.9.3  Multiple-Record Merge Formats Outside of Global FOR
C.9.4  SPIBILD Merge Formats Under Global FOR Processing
C.10  General File Formats for Input
C.11  Input Formats for Partial Record Processing
C.12  Input Formats for Full-Screen Applications
C.13  Special Format Techniques for Updating Records
C.13.1  Updating the Same Record Being Displayed
C.13.2  Updating Other Records from an Output Format
C.13.3  Updating Other Records During Record Input
D  Assorted Topics of Formats
D.1  Combining Frames into Formats: Multiple-Purpose Format Definitions
D.1.1  Input and Output Frames in a Single Format
D.1.1.1  The "USING frame" Command Prefix
D.1.1.2  The SHOW FRAMES Command
D.1.1.3  Formats for TRANSFER/UPDATE Processing
D.1.2  Sharing Subroutines Between Input and Output Frames
D.1.3  Multiple Formats in a Single Format Definition
D.2  XEQ Frames
D.2.1  The XEQ FRAME Command
D.2.2  XEQ Frame Definitions
D.2.2.1  XEQ Frames for Output
D.2.2.2  XEQ Frames for Input
D.2.3  The SHOW FORMAT INFORMATION (SHO FOR INF) Command
D.2.4  The SHOW FORMAT INPUT (SHO FOR INP) Command
D.2.5  Global Formats
D.3  Startup Frames
D.4  Generating Formats Code from System Formats
D.4.1  The GENERATE FORMAT Command
D.4.2  Customizing a Generated Format Further
E  Appendices
E.1  Annotated List of Elements (Statements) in a Format Definition Record
E.2  SPIRES System Variables Specific to Formats
E.2.1  Flag Variables
E.2.1.1  $NEWPAGE and $FRONTPAGE
E.2.1.2  $NOBREAK
E.2.1.3  $FREC
E.2.1.4  $GPROCERR and $WPROCERR
E.2.1.4a  $GCHANGED and $GNCHANGED
E.2.1.5  $LRGNXT, $LRGLCTR and the SET LARGE Uproc
E.2.1.6  $SORTKEND
E.2.1.7  $SORTDEND
E.2.1.8  $NEWCOL
E.2.1.9  $REPORT and $NOREPORT
E.2.1.10  $ENDATA
E.2.1.11  $ABORT
E.2.1.12  $REPROMPT
E.2.1.13  $PROTECT
E.2.1.14  $TESTFAIL
E.2.1.15  $ATTN
E.2.1.16  $SKIPF
E.2.1.17  $SUPPRESS
E.2.1.18  $FLUSH
E.2.1.19  $PROCERR
E.2.1.20  $DEFAULT
E.2.1.21  $SKIPEL
E.2.1.22  $APROCERR
E.2.1.23  $OVERFLOW
E.2.1.24  $CHANGED
E.2.1.25  $TESTREF and $REFERENCED
E.2.1.26  $LREC
E.2.1.27  $NODATA
E.2.1.28  $FTRACE
E.2.1.29  $PUTFLAG
E.2.1.29a  $NOACCESS
E.2.2  Integer Variables
E.2.2.1  $RECNO ($RECNUM)
E.2.2.2  $PAGECTL
E.2.2.3  $LLEFT
E.2.2.4  $LINEPP
E.2.2.5  $HDRLEN
E.2.2.6  $FTRLEN
E.2.2.7  $MARGIN
E.2.2.8  $PAGENO ($PAGENUM)
E.2.2.9  $COLUMNS
E.2.2.10  $COLWDTH
E.2.2.11  $CURCOL
E.2.2.12  $ULEN
E.2.2.13  $CLEN or $VLEN
E.2.2.14  $CROW
E.2.2.15  $CCOL
E.2.2.16  $LOOPCT
E.2.2.17  $NROWS
E.2.2.18  $NCOLS
E.2.2.19  $SROW
E.2.2.20  $SCOL
E.2.2.21  $LROW
E.2.2.22  $LCOL
E.2.2.23  $EROW
E.2.2.24  $ECOL
E.2.2.25  $FLINES
E.2.2.26  $ELOCC and $LASTOCC
E.2.2.27  $CURROCC and $TRUEOCC
E.2.3  Variables of other Types
E.2.3.1  $UVAL
E.2.3.2  $CVAL
E.2.3.3  $PVAL
E.2.3.4  $SORTKEY
E.2.3.5  $SORTDATA
E.2.3.6  $STARTCHAR
E.2.3.7  $ENDCHAR
E.2.3.8  $LABEL
E.2.3.9  $PARM
E.2.3.10  $ELEMID
E.2.3.11  $FORMAT, $SETFORMAT, and $GLOFORMAT
E.2.3.12  $FORMATID and $GLOFORMATID
E.2.3.13  $FRAME
:
:29  SPIRES Documentation

*  SPIRES Formats

******************************************************************
*                                                                *
*                     Stanford Data Center                       *
*                     Stanford University                        *
*                     Stanford, Ca.   94305                      *
*                                                                *
*       (c)Copyright 1994 by the Board of Trustees of the        *
*               Leland Stanford Junior University                *
*                      All rights reserved                       *
*            Printed in the United States of America             *
*                                                                *
******************************************************************

        SPIRES (TM) is a trademark of Stanford University.

A  Introduction

A.1  Introduction to this Manual

How data is arranged is an integral part of any computer application. The input data must be presented to the computer in a form it can interpret; similarly, the data output by the computer must be arranged so that it can be read and understood by the user.

In the SPIRES data base management system, input and output data is frequently processed through "formats", programs that may gather and arrange the data for the computer to place in the SPIRES file or for the computer to display to the user. SPIRES system formats, such as the standard format and the prompting input format called $PROMPT, are available for use with any SPIRES data base. However, many users want or need to create custom formats designed especially for the records in a particular subfile.

As a SPIRES user, you may want to develop custom formats for many different reasons, but each reason can probably be generalized to one of the following:

This manual is designed to teach you how to create your own custom SPIRES formats. Before reading it, however, you should be familiar with the material in the SPIRES primer "A Guide to Data Base Development" or in the reference manual "SPIRES Searching and Updating". Although reading the SPIRES primer "A Guide to Output Formats" is also highly recommended as a good background to this manual, it is not essential. That document introduces the SPIRES formats language and teaches you how to design output formats with the most popular features, following one example step-by-step through the format design process.

Though this manual will also teach you how to create formats, it also serves as the primary reference source on formats -- material from it is displayed when you issue the SPIRES command EXPLAIN to find out more information online about formats topics. If you are just learning formats, the size of this manual might be overwhelming (another reason why the primer serves as a better introduction to the topic). Note that sections whose titles are preceded by "(*)" usually consist of specialized material describing how to handle rare and unusual formatting problems. The (*) prefix indicates that reading the section is not necessary for most users.

Acknowledgments

The SPIRES Data Base Management System is developed and maintained by the Data Base Management Division of the Center for Information Technology, Stanford University. The formats language and processor were designed by Bill Kiefer. This manual was written by John Klemm, who thanks Becky Morton, Dick Guertin, Jack Hamilton, June Genis and especially Bill Kiefer, John Sack, Lynn McRae and Sandy Laws for their help in its preparation.

A.2  Notation Conventions

In this manual, examples of sessions are shown with prompts and messages from SPIRES as they appear on the terminal, often in uppercase; commands you type are shown in lowercase. (You may use upper- or lowercase interchangeably when you are actually using SPIRES). For example:

Here SPIRES types "-OK TO CLEAR?" and you type "ok".

In formal command syntax descriptions, uppercase letters denote command verbs or other command elements to be entered exactly as shown; a value for lowercase terms and characters must be supplied by you. For example:

To use this particular command, you type the command verb "SELECT" with the name of the desired subfile, for instance, "Restaurant":

where "->" is the prompt from SPIRES.

Brackets ([]) denote optional parts of a command's syntax. Braces ({ }) indicate that you must specify one (and only one) of the alternatives within the braces. Within the braces or brackets, a vertical line (|) separates possible choices. Neither brackets nor braces are to be typed as part of the command. For example:

could be entered as

depending on whether you want one of the options.

Sections and subsections of this manual whose titles are preceded by an asterisk in parentheses -- (*) -- may be considered optional reading at that point. Usually the material provides details that most users will not need about how to handle uncommon situations, or technical information about how SPIRES internally handles some particular piece of formats code.

A.3  An Overview of Formats

This chapter, in a general way, describes formats conceptually and technically. A couple of simple examples of formats and their "format definitions" will also be examined, to show some of the capabilities of SPIRES formats. This chapter will also describe the structure of the rest of the manual.

A.3.1  An Introduction to Formats

Conceptually, a format is a design for the arrangement of data. The primary purpose of the design is to facilitate the interpretation of the data, whether it is a person or a computer doing the interpreting. For example, the SPIRES standard format, in which goal record data is presented in the form

tells you (on output) or SPIRES (on input) the name of each piece of data, where the data begins, where it ends, and thus, what it is. The format is a template for mapping data, specifying what data elements go where and suggesting relationships between the elements. You cannot make sense out of any data presented to you unless you understand how it is "formatted", i.e., how it is arranged.

Technically, in SPIRES a format is a program that processes data, usually as the data is placed inside the data base ("input formats") or is displayed from the data base ("output formats"). In its most basic form, the program tells SPIRES the source of the data and its destination. However, many other capabilities are based on that foundation. The program may, for example, modify or test the data, if desired. In addition, it may take advantage of standard programming facilities, such as looping, branching or subroutines.

The SPIRES formats language has a very rich and eclectic vocabulary, including pieces from file definition (e.g., processing rules), protocols (e.g., labels, variable groups and procedural statements) and other parts of SPIRES, such as system variables and functions.

A format program, written in the formats language, is called a "format definition". Just as a SPIRES file definition is a goal record in the public subfile FILEDEF, a format definition is a goal record in the subfile FORMATS, entered in the standard SPIRES format. Because of this, the format program structure is guided by the structure of FORMATS goal records; and at the detail level, much of the program statement syntax is affected by the rules for data entry via the standard SPIRES format.

In the next sections, examples of input and output formats will be briefly discussed in order to demonstrate some of these points.

A.3.2  The Format Creation Procedure and Some Sample Formats

Before stating the specific capabilities of formats, let's examine the procedure for creating two simple formats (one input, one output) for a subfile whose goal records consist of names and addresses. This procedure will introduce some concepts and terminology that will simplify the discussions of formatting capabilities later. [See B.1.1, C.1.1.]

The steps involved in creating a SPIRES format are:

     1) design the format (both the layout and the program);
     2) write the format definition;
     3) add the definition to the FORMATS subfile;
     4) compile the format definition; and
     5) test, modify and use the format.

Each of those steps is discussed below as we create an output format for the subfile.

1) Design the Format

The first step is the conceptual one. Generally, the first question to ask is, what is the purpose of the format? In this particular case, the purpose of the format is to put the names and addresses into our active file so that a LABELER program can read the data, converting it into mailing labels. Often a specific need, such as mailing labels, creates the purpose, but a more general goal, such as the desire to display the data more attractively or to make data entry easier, can certainly be a reasonable purpose.

The LABELER program we want to use has certain requirements about the data input to it:

So a record to be displayed in this LABELER format would look like this:

Our SPIRES file was designed partially around these requirements -- for example, each line of the address as it is to appear on a mailing label is a separate occurrence of the ADDRESS.LINE element. The NAME element is only allowed to have one occurrence, while the ADDRESS.LINE element may have up to three. Also, both elements are limited to 36 characters in length per occurrence. These goal record characteristics, specified in the file definition, suggest that file designers often create files with specific formats in mind from the beginning -- format design requirements may affect some aspects of the file design.

At this point, it is a good idea to look at the format as a template laid out on a grid, such as a piece of graph paper. The grid will represent one goal record displayed via the format. The size of the grid is determined here by the LABELER program: 5 rows by 36 columns is the maximum size of a record. Below is a representation of that grid, each "." representing one position in the grid:

In formats terminology, this 5x36 grid is called a "frame". (Later, the term "frame" will also mean the part of the program that moves the data to and from the grid.) Within the frame, we can position element values and textual strings as we like: they can be centered or left- or right-adjusted within the frame, or they can be restricted to certain columns, or they can wrap around from one row to the next, and so forth. (The "and so forth" will be discussed later.) We will position the exclamation point and the elements in a very simple manner, all left-adjusted within the frame:

The strings in parentheses indicate the elements, whose values would appear there instead.

External specifications guided the design of this particular format; for another format, you might have more creative freedom. Whatever the situation, however, it is recommended that you lay out your format design in a grid such as the one above, especially when you are just beginning to write format definitions. Some more specific guidelines for designing formats will be presented later.

2) Write the Format Definition

The next step is to use the format definition language to write a format definition. The format definition is both a program and a goal record, and the format language reflects this dual role. When we treat it as a program, we say it consists of instructions, or "statements"; when we treat it as a goal record, it consists of "elements", which must follow the standard rules for record input. (In general, we will refer to the elements in a format definition as "statements", to avoid confusion with the elements in records of your subfile to be handled by the format.) Below is the format definition for the LABELER format, as collected in the active file:

     1.    ID = GQ.JNK.ADDRESS.LABELS;
     2.    FILE = GQ.JNK.ADDRESS.BOOK;
     3.    RECORD-NAME = REC01;
     4.    FRAME-ID = LABEL.OUT;
     5.      DIRECTION = OUTPUT;
     6.      FRAME-DIM = 5,36;
     7.      USAGE = DISPLAY;
     8.      LABEL = DELIMITER;
     9.        VALUE = '!';
    10.        PUTDATA;
    11.      LABEL = NAME;
    12.        GETELEM;
    13.        PUTDATA;
    14.      LABEL = ADDRESS.LINE;
    15.        GETELEM;
    16.        PUTDATA;
    17.        LOOP;
    18.    FORMAT-NAME = LABELER;
    19.      FRAME-NAME = LABEL.OUT;
    20.        FRAME-TYPE = DATA;

We can break that format definition into three main sections:

1) Identification Section (lines 1-3)

These statements provide information about the format definition itself and the file to which it applies. The ID element serves as the key of the format definition goal record when it is placed in the FORMATS subfile. The FILE and RECORD-NAME statements name the file and the particular record-type within the file for which the format is being created.

2) Frame Definitions Section (lines 4-17)

This section is usually the largest section in the format definition; most, if not all, of the instructions on how the goal record is to be processed are stated here. Although a format may have multiple frames, often one will suffice. Here, the single "frame definition" begins with some statements about the frame itself, including its name and dimensions. Then the remainder of the frame definition consists of "label groups", which specify the work to be done.

Each label group begins with a LABEL statement (which, by the way, has nothing to do with the fact that this format will be used to create mailing labels) and handles one value or one element. For example, the label group in lines 11 through 13 GETs the ELEMent named by the LABEL statement (NAME) and PUTs that DATA into the grid into a default location (the first column of the next row). Format execution within a frame begins with the first label group and proceeds from one to the next, executing the statements within each.

Label groups are often more complex, using more statements than are shown in this example. For instance, they may test the accessed element value and choose not to put it into the frame, or they may be used entirely to control the execution flow of the frame without accessing data elements or values at all.

3) Format Declaration Section (lines 18-20)

In this section, the frames, which are the building blocks of the format, are put together and given a name used in the SET FORMAT command (the name specified in the FORMAT-NAME statement). When that command is issued, SPIRES fetches the frames under that FORMAT-NAME; SPIRES will execute them later when a command that uses this format is issued. (Remember that more than one frame definition may be specified in the format definition.)

Though only three sections were shown in this definition, many formats contain a fourth section, called "Vgroups", that defines user variables to be used during frame processing. Statements in the Frame Definitions and Format Declaration sections may assign, test, change and display variable values; these capabilities are used frequently in more complicated format applications.

Granted, the above presentation does not teach you how to write a format definition, that is, it does not teach you specifically how to convert a design on a grid into a format definition, but it does introduce some of the statements allowed and show you that some format definitions can be rather simple. The rest of the manual will teach you what formats statements to specify for a given design feature of your formats.

3) Add the Format Definition to the FORMATS Subfile

To compile a format definition, you must first place it in the FORMATS subfile. If the format definition shown above is in your active file, the procedure is simple:

SPIRES examines the format definition in your active file, and if it follows the rules for goal records in the FORMATS subfile (e.g., it has a value for the key element ID, and it has the proper elements in the Frame Definitions section), then the record is added to the subfile.

4) Compile the Format Definition

Once SPIRES has accepted your format's blueprint (that is, once you have successfully added the format definition to the FORMATS subfile), you may compile it:

The COMPILE command names the format definition in the FORMATS subfile that is to be compiled. SPIRES in effect "builds" the format from the definition, checking the syntax of the statements and creating the compiled code to be used by SPIRES when the format is invoked. If a syntax error is found by the compiler, an error message is issued, and you must correct the format definition record in the FORMATS subfile, and then try compiling again. When the format definition is compiled, the format may be used.

5) Test, Modify and Use the Format

You may now select the appropriate subfile, and set the format:

One of the record output commands, probably DISPLAY or TYPE, may now be used to display records through the format. If the format is not satisfactory, adjustments can be made to the format definition, which you then "recompile".

The procedure for format creation consists of the same steps regardless of the type of format. Next we will create an input format for the ADDRESS BOOK subfile.

We want to make data entry into the subfile very easy. Only two elements are collected: the single occurrence of NAME and the multiple (up to 3) occurrences of ADDRESS. An input format may prompt the user for the appropriate data (as the system format $PROMPT does) or may read the data from a data set such as your active file (as the standard SPIRES format does). For our simple input format, let's assume we will collect the data for each record in our active file, in a manner similar to the way the data is displayed by our output format above -- that is, the name will be on the first line, and the address lines will be found on the next one to three lines.

Again, we lay out a grid, representing a frame. This time, since we will not need the exclamation point delimiter required by the LABELER program, we can eliminate one row, leaving the grid as 4 rows by 36 columns:

Below is the appropriate input format definition. You will notice several new statements, but basically this format definition is similar to the output format definition above:

     1.    ID = GQ.JNK.ADDRESS.INPUT;
     2.    FILE = GQ.JNK.ADDRESS.BOOK;
     3.    RECORD-NAME = REC01;
     4.    FRAME-ID = INPUT.DATA;
     5.      DIRECTION = INPUT;
     6.      FRAME-DIM = 4,36;
     7.      USAGE = FULL;
     8.      LABEL = NAME;
     9.        GETDATA;
    10.        PUTELEM;
    11.      LABEL = ADDRESS.LINE;
    12.        GETDATA;
    13.        UPROC = IF $ENDATA THEN RETURN;
    14.        PUTELEM;
    15.        LOOP;
    16.    FORMAT-NAME = INPUT;
    17.      FRAME-NAME = INPUT.DATA;
    18.        FRAME-TYPE = DATA;

This format definition has the same three sections; the major differences are in the statements within the Frame Definition section:

The format definition would be added to the FORMATS subfile and compiled, just as shown earlier for the output format. Then you would probably test the format like this:

With this basic outline of the process for creating SPIRES formats, we are now better equipped to consider the capabilities of particular types of formats.

A.3.3  Basic Concepts of Formats

Let's examine the process of how formats work and how they are used in more detail.

All formats are created to process data. In almost all cases, a format maps data from a record-type within a SPIRES file to a character array or vice versa. (A special type of format called a "global format" [See D.2.5.] is not associated specifically with any data base.)

No format can be used until a SET FORMAT command (or an equivalent, such as automatic format selection when a subfile is selected) has been issued. When SPIRES receives a SET FORMAT command, it looks for the record containing the compiled characteristics of the named format in a subfile called FORCHAR (for FORmat CHARacteristics). FORCHAR records are created by SPIRES when a format definition is compiled; users, including format definers, rarely need to access this subfile themselves. The compiled code is then in user memory for use by subsequent commands.

Also at this time, SPIRES initializes any user-defined variable groups ("vgroups") that can be used by frames that will be executed. Any user variables used during format execution (e.g., to hold element values for testing later in the format) must be defined either in the Vgroups section of the format definition, or in a separate record for the VGROUPS subfile. [See B.9.3.] When the compiled format is set, room in user memory is reserved for these variables, and initial values, if any, are assigned to them.

Sometimes a format may have a special type of frame called a "startup frame", which is executed as soon as SPIRES "loads" the format (i.e., when the SET FORMAT command is issued). The startup frame is typically used to send information to the terminal about how the format is used or to set certain variables or make certain tests, just like "select-commands" may be used when a subfile is selected. [See D.3.]

All of the commands listed below may cause format execution if a format is set:

When any of the above commands is issued and a format is set, SPIRES checks to see whether execution of any frames within that format is appropriate. This decision is based on the values of several statements within the format definition, specifically DIRECTION and USAGE in the Frame Definitions section and FRAME-NAME and FRAME-TYPE within the Format Declaration section. [See B.3.2, B.3.4, B.5.2.]

A single format may have frames to be used when the ADD command is executed and different frames to be used when the DISPLAY command is executed. Similarly, a single format may have two different "sets" of output frames, one set of which is invoked by a DISPLAY or TYPE command, and the other which is invoked by a TRANSFER command, or even by a TYPE or DISPLAY command that is preceded by the "USING frame" prefix. [See D.1.1.1.] The ability to have several contrasting uses for a format can make applications simple -- you may not have to keep switching formats back and forth to alternately add and display records, for instance. On the other hand, it creates a minor terminology problem, because a format used for adding records, which we are tempted to call an "input format", may be defined so that it can also be used to display records, in which case it also qualifies as an "output format".

That description of the terminology problem also suggests its solution. A given format may be both an input and an output format, so calling one an "input format" will not rule out the possibility that it may also be defined for use as an output format as well. The term "input format" simply implies that the format definition contains frames that can be executed when data is being mapped into the data base. Similarly, "output format" implies that the format definition contains frames that can be executed when data is being mapped from the data base. A format then may be either or both. [See D.2 to see how it can be neither.]

For example, the two formats defined in the previous section can be combined into a single format:

Identification section

      1.     ID = GQ.JNK.ADDRESS.FORMATS;
      2.     FILE = GQ.JNK.ADDRESS.BOOK;
      3.     RECORD-NAME = REC01;

Frame Definitions section

      4.     FRAME-ID = LABEL.OUT;
      5.       DIRECTION = OUTPUT;
      6.       FRAME-DIM = 5,36;
      7.       USAGE = DISPLAY;
      8.       LABEL = DELIMITER;
      9.         VALUE = '!';
     10.         PUTDATA;
     11.       LABEL = NAME;
     12.         GETELEM;
     13.         PUTDATA;
     14.       LABEL = ADDRESS.LINE;
     15.         GETELEM;
     16.         PUTDATA;
     17.         LOOP;
     18.     FRAME-ID = INPUT.DATA;
     19.       DIRECTION = INPUT;
     20.       FRAME-DIM = 4,36;
     21.       USAGE = FULL;
     22.       LABEL = NAME;
     23.         GETDATA;
     24.         PUTELEM;
     25.       LABEL = ADDRESS.LINE;
     26.         GETDATA;
     27.         UPROC = IF $ENDATA THEN RETURN;
     28.         PUTELEM;
     29.         LOOP;

Format Declaration Section

     30.     FORMAT-NAME = INPUT.AND.LABELS;
     31.       FRAME-NAME = LABEL.OUT;
     32.         FRAME-TYPE = DATA;
     33.       FRAME-NAME = INPUT.DATA;
     34.         FRAME-TYPE = DATA;

Though only the single format INPUT.AND.LABELS is set, different frames, each with a different purpose, would be executed when different commands are issued. When a TYPE or DISPLAY command is issued, for instance, the output frame LABEL.OUT would be executed; when an ADD command is issued, however, the input frame INPUT.DATA would be executed. Hence, a single format serves as both an input and an output format.

Suppose now that a command is issued that will cause one or more frames of the set format to be executed. Again, for example, suppose that the set format has frames that will be executed when a TYPE command is issued. SPIRES will then begin executing the appropriate data frames, following the instructions in the label groups for each frame, beginning at the first label group and continuing straight through (unless some branching instruction is encountered) for each record. Data frames are executed once per record. Certain types of frames, available in "report mode", are executed once per "group" of records or at the end of all records when multiple records are output.

If we return to the conceptual notion of a frame being a grid in which data is positioned, we can think of a format being a collection of such grids. Many formats may have only one grid, but a format may consist of multiple grids; the grids may appear one after the other, or they may be grids superimposed upon grids. The latter capability is particularly important when data elements within structures are being processed. Thus, just the way you can position a data element within a frame, you can construct a frame and position it within another frame.

As a program then, a format can proceed from frame to frame, or a frame may invoke another frame, like a subroutine, that gets control, executes and then returns control back to the "calling" frame. Such a frame is called an "indirect frame" because it is invoked not directly by a user command but instead by another frame.

When SPIRES begins executing a frame, it first establishes a "buffer" in user memory -- the buffer can be considered the internal equivalent to a grid. For output, the data is positioned individually in the buffer according to the directions given in the label-groups. For input, input data is placed in the buffer until it is full and then individual pieces are read from the buffer, according to the label-groups. (The exception to this is the prompting input format, in which the data being input is not read from a data set but is retrieved by prompting the user for data values at the terminal.)

A.3.4  The Outline of the Manual

This manual will show you how to write formats, based on the concepts shown in this section. In Part B, we will concentrate on output formats. In a task-related approach, we will begin constructing simple output formats, adding to our capabilities and skills as we need them. Our format creation procedure a few sections back will be covered in detail; from that point of view, Part B is worth studying even if you just want to write input formats, since the procedure, as you saw, is very similar for both types of formats.

In Part C, we will examine input formats. Input formats for adding and modifying (either via the UPDATE or MERGE command) records will be covered, including discussions on the major methods of providing the input data -- reading input data sets, prompting the user, and providing the data in the format itself (normally done in merge processing). Though the direction of data flow has changed, the basic format definition concepts do not, and the chapters of Part C build on the statements and techniques introduced in Part B.

In Part D, some miscellaneous topics will be covered, including formats used for both input and output and formats used for neither. Also discussed there will be single format definitions that define multiple formats.

The concluding section, Part E, provides several appendices to the manual, including descriptions of all SPIRES system variables associated with formats (they are used from time to time throughout the manual) and explanations for error messages that you might receive when you compile a format definition.

B  Output Formats

B.1  Introduction to Output Formats

Output formats can be designed to display data from a SPIRES data base. The format may be designed for displaying individual records or for reporting on groups of records (providing subtotals and totals across records, for example).

The first few chapters of this part of the manual will lead you step-by-step through the procedure of creating output formats. Later in this part, special features and capabilities of output formats will be discussed, such as the ability to call other formats from within a format, the ability to access records in other data bases from within a format, and even the ability to update records while displaying them.

It is important to realize, especially if you will be writing input formats, that most of the topics discussed in this part concerning output formats also relate to input formats. Statements discussed here in regard to output often are identical to, or have their opposite counterpart in, statements in input formats. Please be aware that the details of such topics covered in this part may not be repeated in the part on input formats -- cross references to this part will be provided there instead.

One final reminder before beginning this chapter: Remember that the term "output formats" is misleading in that a single format can be used for many different purposes (input and output) by many different commands. The term is used to identify formats that can be used for output, but not necessarily exclusively for output.

B.1.1  Do You Need a Custom-Designed Output Format?

Before you decide to create an output format (and go to the trouble of learning how if you have never done so before), you should seriously consider whether a custom-designed format is necessary. SPIRES allows you to display record data in various ways without forcing you to learn the formats language. If you are not acquainted with these "system formats" (general-purpose formats that can often be used by any data base) and features, which are individually discussed below, you are encouraged to read about them in the references cited.

The data in records to be displayed may be controlled in two ways: 1) you may control the specific elements to be shown; or 2) you may control specific element values to be shown, based on their values. In the first case, the control is handled by element lists; in the second, it is handled by element filters. An example of element filters will be shown later in this section.

The Standard SPIRES Format

The most commonly seen system format is the standard SPIRES format, whose general form is:

Here are some notes on the standard SPIRES format:

The $PROMPT Format

This format also displays element names and values, but in a different form from the standard format. The element names are shown on the left; their values appear several blanks later on the right. Structures are labeled, and the format makes it clear which elements are in them. In general, it is more expensive to use than similar, custom-designed output formats, though you can generate a custom format from $PROMPT. Not only is one of these generated formats as cheap to use as a custom-designed one, it is also simple to create, saving you from writing a complex format definition. [See D.4.1.]

If you are not familiar with the format, select a subfile, issue the SET FORMAT $PROMPT command, and display some records with it. The format is also useful for input, as its name suggests. It is described fully in the SPIRES manual "Searching and Updating", section D.3.

The $REPORT Format

This format displays records in a tabular form. The elements are arranged horizontally in columns across the page. (Because the $REPORT format is a "report" format [See B.10.] it is designed primarily to produce output for printed reports, though by no means is it limited to that.) The documentation for it can be found in part C of "Searching and Updating".

Here are some additional notes on the $REPORT format:

Filtered Record Processing

This feature allows you to restrict elements displayed to particular occurrences or to occurrences with particular values. Filters affect the output regardless of what format is set. Although formats can restrict the occurrences displayed of a particular element, filters provide a more general facility.

Here is an example of a SET FILTER command:

This command tells SPIRES that later record-display commands should process only the occurrences of the CHILDREN structure that have an AGE element value of less than ten for any given record. Any records displayed after the command has been issued will be affected -- no occurrences of the structure that have an AGE value greater than or equal to ten will be displayed, no matter what format is used.

A command such as "SET FILTER FOR CHILDREN (1/3) WHERE AGE < 10" will limit displays to only the first three occurrences of the structure that represent children less than ten years old. Another form of the command, such as "SET FILTER FOR CHILDREN IN 2/4", identifies the specific occurrences (the second through fourth) that should be processed.

Filters are a very powerful feature that can be used instead of output formats or in conjunction with them, depending on the application. They have no effect on output formats used for the TRANSFER command [See B.3.4.] nor do they generally affect input formats. If your application involves both formats and protocols, you may find it easier to limit the displayed values with SET FILTER commands in the protocol than with specific code in the format definition. Complete documentation for filters is in chapter 21 of "SPIRES Technical Notes". [See B.4.8.5a for information about how filters may be set within an output format.]

Dynamic Elements

Dynamic elements can be used to show element values in different forms than the way in which they would be output by the standard format. For a very simple example, suppose an element were displayed like this:

You might issue a DEFINE ELEMENT command to create the dynamic element RPM:

The RPM element is based on the value of the SPEED element for any given record. If you issue the command "TYPE SPEED, RPM", you might get the following result:

Dynamic elements may be defined by anyone with access to the selected subfile; they only last for that user as long as he or she has the subfile selected. They may concatenate or perform computations with several element values. They may use SPIRES functions and system variables. They may use either the internal (i.e., pre-OUTPROC) or external (post-OUTPROC) form of the elements.

Dynamic elements are a very helpful tool; like filters, they may be used instead of or in conjunction with output formats, though if you intend to use them in a format, they must be represented by variables in GETELEM statements. [See B.4.2.1.] For more information about them, see "SPIRES Technical Notes", section 20.

Why You Might Need a Custom-Designed Format

Despite the power of all of these system formats and features, considered separately or in combinations, they do not handle all the situations that custom formats do. Moreover, a customized format has certain advantages over them.

In regard to the system formats described above, a customized format does exactly what you want -- you do not have to compromise your needs with generalized capabilities. In regard to the features such as dynamic elements, a customized format has the advantage of being compiled, and compiled code is more efficient to execute than uncompiled equivalents.

Generally, the individual capabilities described above for both the formats and features can be coded in output formats as well. But specifically, here are a few other capabilities of customized formats that may not be as easy to handle by other methods:

B.1.2  Output Formats in General

Output format record-processing can be considered a two-step operation. First, the data is retrieved from the record according to the instructions given in the format definition (specifically, in the definition of the frame being executed) and placed, after any specified alterations, in the "buffer". The buffer, an area in main memory, is associated with the frame currently executing.

Second, when execution of the frame is complete, the buffer is "flushed" (sent) to the output device, usually the terminal screen or active file. If the format contains multiple frames to be executed, SPIRES will go on to the next one, starting this process over again. Similarly, if multiple records are being processed and the last frame has been executed for a given record, SPIRES will go on to the next record, starting the process over again.

The concept of the buffer as an intermediate storage place for data is important to keep in mind, especially for output formats. For example, if an instruction within a frame tells SPIRES to immediately display a message at the terminal using the star ("*") "Uproc" [See B.4.8.13.] you might be surprised at first to see the message on your screen ahead of the data already processed by the format. But again, the formatted data is being held in the buffer until the frame executes completely, whereas the message from the star Uproc is sent to the terminal immediately. [See B.3.3.]

Output formats can be designed to display individual records or to produce reports processing multiple records. However, regardless of whether the format is designed for single record displays or multiple-record reports, only one record is processed per execution of the format -- that is, the frames that process records are written to process individual records. Commands that cause multiple records to be displayed (e.g., DISPLAY ALL) cause the format to be executed one time per record.

Many output formats are written with a single frame to process the data. The "data frame" can handle all record-level elements in the records, but elements within structures must usually be handled by separate frames that are invoked from the data frame. Within a frame, each element is usually "processed" (i.e., retrieved from the record, modified if desired and placed in the buffer) by a group of statements called a "label group"; although several label groups may be used and are sometimes necessary to handle an element, the norm is one label group per element.

B.1.3  Format Definitions as Records in the FORMATS Subfile

All format definitions must be placed in the public subfile FORMATS before they can be compiled and used. Hence they have a dual role: a format definition is a collection of instructions in a particular language, and it is a goal record in a subfile. The rest of this manual discusses the format definition language, but this section will discuss the format definition as a goal record.

Generally, FORMATS records are entered in the standard SPIRES format. Almost all format definitions are entered into the FORMATS subfiles as records in the standard SPIRES format, so the syntax of statements in a format definition will be shown as they would be entered in that format. For example, the ID statement's syntax is:

Thus, in the FORMATS subfile, the value for the element ID must have the form "gg.uuu.anyname". If you enter the format definition in the standard format, you must begin the statement with the element name and end it with the semicolon. The equals sign, though not shown as such in the syntax statements, is always optional.

The maximum length for any single element value in a FORMATS goal record is 3072 bytes. That means, for example, that no single occurrence of the COMMENTS statement may be longer than 3072 characters (about 40 72-character lines). However, COMMENTS statements are allowed to occur multiple times wherever they appear in a format definition, so very long comments could be split into multiple occurrences. [See B.2.2.]

The rules for data entry using the standard SPIRES format are discussed in detail in the SPIRES manual "Searching and Updating", section D.1.2. The most important ones to remember are the following:

The last rules are important to stress. Many of the statements in a format definition are elements within structures. For example, a frame definition is an occurrence of a structure; each label group in a frame is an occurrence of a structure within the frame structure.

All structures in a FORMATS goal record have key elements, such as VARIABLE in the VARIABLES structure shown above. Because a statement introducing or ending a structure may thus look like any other statement, you can easily lose your bearings in the format definition hierarchy.

Also important to remember is that SPIRES allows you to enter the elements in a structure (or the record-level elements, for that matter) in almost any order. However, they will be rearranged into the order shown by the SHOW ELEMENTS command for the FORMATS subfile. Thus the order in which you enter the statements may not be the order in which they are stored and compiled and executed. That is occasionally a problem for people learning to write label groups. [See B.4.]

An appendix in the back of this manual shows the proper order of statements in a format definition record. [See E.1.] In addition, the examples throughout will show you the order in which to code the statements.

A format definition has four major parts: the Identification section, the Vgroups section, the Frame Definitions section and the Format Declaration section. The first section represents most of the record-level elements in a format definition record; the others represent multiply-occurring structures. For example, there may be more than one frame definition, each one representing one occurrence of the FRAME-DEF structure.

Each of the sections of the definition are covered in separate chapters of Part B. [See B.2, B.3, B.5.] In addition, one chapter covers label groups, the multiply-occurring structure within the FRAME-DEF structure. [See B.4.] Other chapters in Part B will discuss how to compile the created format definition, how to use it, and how to handle other special situations in output formats.

B.2  The Identification Section

This chapter describes the format definition statements that relate to the format definition as a whole. The key of the format definition record is the first statement, ID. Other statements, FILE and RECORD-NAME, identify the file and the record-type within the file to which the formats defined in this record will apply. Other statements (such as AUTHOR and DEFDATE) may contain other information about the record that will be useful to you.

In terms of the FORMATS goal record, these statements are record-level elements. Following the RECORD-NAME statement come the VGROUPS, FRAME-DEF and FORMAT-DEC structures, discussed in later chapters. Several other record-level elements, such as GEN-FILE, follow these structures, but they are discussed in later chapters. [See B.4.5.2, C.10.]

B.2.1  The ID Statement

The ID statement specifies the key element value of the format definition for the FORMATS subfile -- no other format definition in the FORMATS subfile may have the same value.

The form of the ID statement is:

where "gg.uuu" is your account number and "anyname" is a character string containing any characters other than blanks.

Usually, to make identification easier, a format definition's ID references the file to which it applies. For example, if a file is named XR.RMN.TAPES, a format definition for one of its record-types might be specified as:

The ID value is not the value used in the SET FORMAT command; that value is specified in the FORMAT-NAME statement in the format declaration section. [See B.5.1.] The ID value is used when you want to update the record in the FORMATS subfile and when you compile the format definition, so it is usually advisable to keep it relatively short and easy to type. A reasonable limit to suggest here is that the entire value be no longer than 30 characters, but the absolute limit is over 100.

You may replace the "gg.uuu:" portion of the value with a period or an asterisk:

Both are equivalent to the first form shown above for account number XR.RMN.

B.2.2  The COMMENTS (COM) Statement

This free-form, multiply occurring statement is the first of several COMMENTS statements allowed throughout the definition, appearing in most every section. The COMMENTS statement here is often used to describe the purpose of the format(s) being defined, though of course you may use it for anything you want.

For example,

Be aware that, if this were collected in your active file, the extra blanks on the second line of the COMMENTS value (preceding "of the RECORDS subfile") would be retained in the value. It is shown this way here, and in similar examples throughout the document, to make long values easier to read than the way you probably should enter them:

     ID = GQ.JNK.RECORDS.LISTING;
     COMMENTS = This format definition is for the LISTING format
of the RECORDS subfile.;
     AUTHOR = John Klemm, DBMG, 497-4420.;

That style of presentation, though more accurate, can make examples more difficult to read.

Here is an example demonstrating "block comments", which are often easier to read because you enter them the way you want to see them:

Certainly the asterisk border draws your attention to the comments inside.

Some users maintain their format definitions in WYLBUR or ORVYL data sets. Rather than transferring the definition from the FORMATS subfile, making changes, and then updating it, they make changes to their own copy of the definition and then update the copy in the FORMATS subfile. This procedure lets them take advantage of the "-" element, sometimes called the "dash element" or the "throwaway element". If you add a record that includes throwaway element values, those values are indeed thrown away, and not included in the stored record; the throwaway element is available for all SPIRES subfiles. People who maintain their own copies of a format definition can use the throwaway element to place comments anywhere they please within the definition, since SPIRES will ignore them.

Yet another type of comment is allowed in UPROC statements. [See B.4.8.14.]

B.2.3  The AUTHOR Statement

Like the COMMENTS statement, this is an optional, multiply occurring, free-form statement, meant to include your name, as the definition's author, along with a phone number in case the SPIRES system programmers need to get in touch with you. Such a need seldom arises, of course, but when it does, the appearance of this statement is very helpful.

Here is an example of an AUTHOR statement:

B.2.4  The DEFDATE, MODDATE and MODTIME Statements

These three statements, automatically supplied when you add or update your format definition in the FORMATS subfile, provide the date the record was first added to the subfile (DEFDATE), as well as the date (MODDATE) and time (MODTIME) that it was last updated. They are provided for your convenience.

B.2.5  The FILE Statement

This singly occurring statement contains the full name (including the account number prefix) of the file to which the format definition applies.

For example,

If the file belongs to your account, you may replace the "gg.uuu." portion of the file name with either a period or an asterisk:

SPIRES will replace those characters with your account number for record storage.

You can write a format definition for a record-type within any file; however, to use the format, you must be able to access the record-type, usually by selecting the subfile for which it is the goal record-type.

If you are not the file owner, you may not know the file name. That information may be obtained by selecting the appropriate subfile and issuing the SHOW SUBFILE INFORMATION command.

This statement and the RECORD-NAME statement [See B.2.6.] may be omitted if you are writing a general file format or a global format. [See B.14, D.2.5.]

B.2.6  The RECORD-NAME (REC) Statement

This singly occurring statement identifies the record-type in the previously named file to which the format definition applies. If you do not know the name of the record-type, it may be discovered by selecting the subfile for which the format is being written and issuing the SHOW SUBFILE INFORMATION command.

Here is an example of the RECORD-NAME statement:

The record name will never be longer than six characters.

A format definition and all the formats it defines can be used with only one record-type of one file (unless it is a general file format, described later in this manual). Usually a format is written for the goal record-type of a subfile, but not always -- the format definition is tied to a specific record-type of a file, not to a subfile, via the FILE and RECORD-NAME statements. It is possible to access other record-types from a format, however, through subgoal processing. [See B.12, B.14.]

B.2.7  A Sample Identification Section

Below is a sample identification section, combining the statements described in this chapter:

Remember that this definition is a goal record in the standard SPIRES format. Hence, in the second COMMENTS statement, the inclusion of quotation marks around the title "SPIRES Formats" means that the entire element value must be enclosed in quotation marks, and the internal quotation marks must be doubled. Similarly, if a semicolon (;) appears within a value, the value must be enclosed in quotation marks. Within the value, using apostrophes (') instead of quotation marks (") and avoiding semicolons are alternatives to consider as well. [See B.1.3.]

The next section of the format definition, Vgroups, will be discussed later. For the time being, just remember that we will be able to use variables declared in the Vgroups section later in the frames that we write. The next two chapters will discuss the Frame Definitions section.

B.3  Instructions for Formatting: the Frame Definitions Section

The Frame Definitions section, which contains sets of formatting instructions, does most and often all of the work for a format. The section is comprised of one or more "frame definitions". Most frame definitions consist of two parts: frame identification statements, that provide the name of the frame and put limits on how the frame may be used (for example, for input or for output), and label groups, the individual subroutines that specify the processing to occur.

Though many formats have only one frame to be executed, formats commonly have multiple ones. There are several reasons why this is so. The most common reason is that the goal record-type for which you are writing the output format has structures in it. In most such cases, the processing of the elements within the structure must be specified in a separate frame definition. How to handle structures is the subject of a later chapter. [See B.8.] Other reasons for using multiple frames, such as the ability to share the same code between multiple formats, will be discussed in later chapters of Part B.

The remainder of this chapter will focus on the basic statements of frame identification. Some others, such as SUBTREE and LOAD-FORMAT, are involved with the above-mentioned reasons for having multiple frames, and thus will be covered in later chapters. Discussion of the label groups in a frame definition will appear in the next chapter.

B.3.1  The FRAME-ID Statement

The FRAME-ID statement signals the beginning of a frame definition. It provides a name for the defined frame that will be necessary in various situations later. In particular, this name is used in the format declaration section to tell SPIRES which frames can be used when a format is executed. [See B.5.2.]

The frame name may be from one to sixteen characters long. No blanks are allowed in the name. Few special characters (i.e., not alphabetic or numeric) are allowed in the name either, though the useful exceptions are periods, hyphens and underscores, which are commonly used as substitutes for blanks.

For example,

B.3.1.1  The COMMENTS (COM) Statement

Just after the FRAME-ID statement, you can add another COMMENTS statements in the same form as the one described earlier. [See B.2.2.] The COMMENTS statement here will most likely describe how the frame will be used, or, if it is an indirect frame, which other frame or frames called it. [See B.4.8.7.]

B.3.2  The DIRECTION (DIR) Statement

This statement specifies the direction of the data mapping. A frame used for data output will have the value OUTPUT for this statement.

For example,

Frames for input will have the value INPUT. A third value, INOUT, is used primarily in formats used in full-screen applications [See D.1.2.] though it may also be specified for frames containing code that is shared between input and output formats. [See D.1.]

In other words, the output commands DISPLAY, TYPE, SCAN and TRANSFER may cause execution of frames of DIRECTION = OUTPUT. The input commands ADD, UPDATE, MERGE and BATCH (in SPIBILD) cannot cause execution of such frames; instead, they may cause execution of frames of DIRECTION = INPUT. Neither of these statements is meant to imply that such frames will necessarily be executed when one of those commands is executed; that depends on other factors as well. [See B.3.4, B.5.2.]

By default, if no value is coded for the DIRECTION statement (i.e., the statement is omitted from the frame definition), it is given the value of OUTPUT. Getting into the habit of explicitly coding it is recommended, however, since forgetting to code it properly on an input frame would cause a compilation error.

Generally speaking, an indirect frame has the same direction as the frame that calls it, but that is not always the case. [See B.4.8.7, B.16.3, C.13.3, D.1.2.]

B.3.3  The FRAME-DIM Statement

The FRAME-DIM statement defines the size of the two-dimensional array (also known as the format "buffer") into which the data for an output frame is placed. In other words, the values given are the dimensions of the frame being defined. This statement also specifies whether SPIRES will process the frame "line by line" or as a "fixed frame" (see below). This statement is very important -- it may only be omitted when the frame being defined is not placing data in or reading data from the buffer. [See B.4.8.7.]

The syntax of the FRAME-DIM statement is:

where "nrows" and "ncols" are integers representing the number of rows down and the number of columns across the frame respectively. The only restriction on the size of these two values is that their product (nrows times ncols) must be less that 65,536 (64K). (For purposes of comparison, a standard terminal screen, 24 rows by 80 columns, would have a product of 1,920, and a printer page of 60 rows by 150 columns would have a product of 9,000.)

If the value of "nrows" is given as "0" (zero) or omitted, as in the last form shown above, then "line by line processing" goes into effect (see below). If the value of "ncols" is 0, then the width of the destination area (such as the active file) will be used; in other words, the width is dynamically set, based on the final destination of the data. For the active file and the terminal, both in SPIRES and batch SPIRES, the value used for "ncols" is the value of the system variable $LENGTH, which is 72 by default. If the format is used to display records in other device services areas, the width of the area will be used for "ncols". [See the note below about SET HCLIP, and see the SPIRES manual "Device Services" for more details.]

When the frame is executed, SPIRES will construct a buffer in memory having the dimensions given. Subsequent instructions within label groups may reference any position within the frame by citing its row and column numbers. References to positions outside the frame dimensions (too high a row or column number, for instance) may cause an S808 error when the frame is executing -- they will not cause a compilation error. [See B.6.2.] When the frame finishes executing, the buffer is "flushed" (that is, released from main memory and sent to the specified output device, such as your terminal).

(*) The SET HCLIP Uproc in Device Services Formats

The SET HCLIP Uproc allows you to display records in a device services area when the right margin of your format's FRAME-DIM would otherwise be too wide for the area. HCLIP stands for "horizontal clipping", and the effect is that when records are displayed in a device services area, the right portion of the data output by the format will simply not be displayed, rather than giving you S808 or S825 errors. You will see only what will fit in the dimensions of the device services area.

A primary use of this Uproc is in report formats coded for use in Prism. A report might be too wide to fit on Prism's screen, so one option is simply to forbid users to display the report online (i.e. they must print the report in order to see it). Alternatively, if the SET HCLIP Uproc is used in the format, you can allow the application users to display the portion of the report that will fit on the Prism screen. (The entire wider report would still be generated for printing.)

Code SET HCLIP in an initial frame, so that the setting can be established before frame dimensions are set. The HCLIP setting is reinitialized each time a multi-record output command is issued; there is also a SET NOHCLIP Uproc to turn it off.

Choosing Frame Dimensions: Fixed Frame vs. Line-by-Line

The following principles should be considered when setting frame dimensions:

These guidelines seem to suggest that you should get the dimensions exactly right -- "too large" means inefficiency, "too small" means an error if the record being formatted is larger than anticipated. Ideally that is true, but it is somewhat impractical when you are dealing with records whose sizes vary. Given only these principles on "fixed frame dimensions", you should probably risk erring on the side of "too large".

However, there are other possibilities to consider. A neat alternative is to take advantage of "flush processing". You set "nrows" to a low, reasonable number and then specify the SET FLUSH Uproc. [See B.4.8.10, E.2.1.18.] Then, if a label-group tries to place data in a row beyond the last row of the frame, the partially completed buffer is flushed, as described above, and format processing continues, constructing and releasing rows of formatted data one by one. Once "flush processing" begins, each row is sent to the output device as soon as some value is positioned in the next row.

For example, if a data record that would require 35 rows is placed in a buffer of 30 rows that allows flush processing, as soon as SPIRES tries to place data in row 31, the first 30 rows would be flushed to, say, the active file. Then, when SPIRES tries to place data in row 32, row 31 is flushed, and so on. This process can continue indefinitely; the record may require just a few extra rows beyond the fixed frame dimensions, or hundreds. The SET FLUSH Uproc, which is specified in the format declaration section [See B.5.] grants you this flexibility.

One significant limitation of "flush processing" should be kept in mind: Once a frame or a row has been flushed, you cannot put any more data into it. Compare the two "frames" shown below. The string AAAAAAA represents an occurrence of the element A, the string BBBBBBBB represents an occurrence of the element B, and the strings made up of periods represent blanks. Both frames have fixed dimensions, but Frame 2 also has SET FLUSH. For both frames, you want to do the same thing: place all the occurrences of element A on the left, and all the occurrences of element B on the right, both sets of occurrences beginning on the first line. Note that all occurrences of A will be processed before any occurrences of B:

Presumably you want both to look like Frame 1. However, because the first row of Frame 2 is flushed when the second occurrence of element A is positioned, the first occurrence of element B cannot be positioned there; the row is already gone. The best that can be done at that point is to begin the occurrences of element B on the same row as the last occurrence of element A, as shown. Summarizing this example, we can say that multiply occurring elements to be positioned side by side with other multiply occurring elements should not usually be done within flush processing.

Line-by-line processing takes flush processing to the extreme. Each row of the frame is constructed and then is flushed when SPIRES tries to put data in the next row. Because SPIRES is working with a smaller frame (a single row at a time), line-by-line processing is slightly more efficient than fixed-frame processing. If you use a format extensively, line-by-line processing could represent a significant savings over time. On the other hand, line-by-line processing shares the same minor restriction as flush processing -- once you have written data into row 2, you cannot put any in row 1.

There may be ways around this restriction (such as another way to design the format or specify the instructions) but it is generally preferable to use fixed frame dimensions and code straightforwardly than to use line-by-line processing with a few kluges. The extra coding you do or the extra processing SPIRES must do in the latter case may far outweigh any efficiency advantages gained by using line-by-line processing. Formats that put out row after row of data down the page are natural candidates for line-by-line processing, though.

As mentioned earlier, line-by-line processing is requested by coding "0" for "nrows" in the FRAME-DIM statement:

Other aspects of frame dimensions are discussed elsewhere as appropriate.

B.3.3.1  (*) The SET NROWS and SET NCOLS Uprocs

You can change the frame dimensions set for a frame by using the SET NROWS and SET NCOLS "Uprocs". (A Uproc is a statement requesting a particular procedure to be executed at that point in the format.) [See B.4.5.5.] Their syntaxes are:

where "nrows" and "ncols" are non-negative integers, integer variables, or expressions whose results can be converted to integers. If SET NROWS or SET NCOLS appears in a label group within the frame, the value of "nrows" or "ncols" must be less than the corresponding FRAME-DIM value. The frame dimensions of the buffer will change immediately. Further changes to the frame dimensions may be made in the frame, as long as they have successively smaller values. (In general, you should not set "nrows" to zero within a frame in an attempt to switch to line-by-line processing. However, you can do it if you have not already put any data into the buffer and if the SET FLUSH Uproc is in effect.)

These UPROCs can thus be used, for example, to change the number of rows for fixed-dimension frames such as initial or header frames once you know how many rows of data they have used. [See B.10.3.1.]

The SET NROWS and SET NCOLS Uprocs can also change the frame dimensions of a frame before it is entered, if they are coded as UPROCs in the frame declaration of the Format Declaration section. (They would be coded after the FRAME-NAME statement for the frame they are to be applied to.) [See B.5.2.] Then they would be executed before the frame itself was executed. Here the value of "nrows" or "ncols" may be larger than the corresponding FRAME-DIM value, but whatever it is, it will override that FRAME-DIM value when the frame is subsequently entered. Setting the value of "nrows" to zero will set line-by-line processing for the frame when it is entered.

B.3.4  The USAGE Statement

The USAGE statement, in combination with the DIRECTION statement, determines which commands can cause execution of the frame being defined. For output frames, the most common usage is DISPLAY, indicating that DISPLAY, TYPE and SCAN commands can cause their execution.

The syntax of the USAGE statement is:

where "value" is one of the usage values shown below, and NAMED is an additional option, which is described below.

For an output frame, four values are allowed:

By default for DIRECTION=OUTPUT frames, the USAGE is DISPLAY. That is, if no USAGE statement is coded, or if USAGE = NAMED is coded by itself, the primary usage value is DISPLAY.

For most simple output formats, USAGE = DISPLAY is coded. Later we will see how the other values, including NAMED, are commonly used. [See D.1.1.1.]

B.3.5  The Rest of the Frame Definition

Next in the frame definition usually come the label groups, which will specify the processing that should be done: which elements should be accessed, where their values should be placed, etc. Alternatively, other statements may appear next, shifting execution control to other formats. [See B.11.]

Label groups are more common, however, and they are the subject of the next chapter. Before that, however, here is the start of a sample frame definition, preceded by the Identification section, using the statements described in this chapter:

Note that there is another statement that may appear between the DIRECTION and FRAME-DIM statements, the SUBTREE statement. SUBTREE is specified for indirect frames that are used to access element structures, and will be discussed later, along with indirect frames. [See B.8.2.]

B.4  Label Groups: The Basic Building Blocks of Formats

The second part of a frame definition is a collection of program instructions arranged in "label groups". Most frames have multiple label groups, at least one for each element being processed -- a single label group generally retrieves only one element.

Label groups basically have two purposes: to handle a single data value, and to control format execution. A single label group may do either or both. Specifically, label group statements have five functions:

More than twenty different statements are available within a label group for an output frame, each one serving at least one of the above functions. The wide variety of possibilities is just barely suggested by their names, shown below:

Several other label group statements are available for input and "inout" frames. [See C.3.]

When SPIRES executes a frame, it begins with the first label group, executes the instructions described therein, and then proceeds to the next one. In general, the statements within a label group are executed independently, but some of them will have an effect on others -- as an extreme example of this, a UPROC = JUMP statement could possibly "undo" all the rest of the statements that preceded it in the label group. So there are two ways to look at a label group: first, as a collection of individual instructions, executed one by one; and second, as a single "super-instruction" that executes all at once, in most cases handling one data element.

The latter view is preferable for several reasons. First, whenever execution branching occurs (e.g., skipping some instructions, or looping back to earlier instructions), execution always resumes at the start of a label group. You cannot jump into the middle of a label group.

Second, a single execution of a label group handles a single value. Each label group has a value (actually in two forms, represented by the system variables $CVAL and $UVAL) and many label groups deal exclusively with their value -- retrieving it from the data base record, testing and adjusting it, and placing it in the buffer. Although these individual activities can be split into multiple label groups, the SPIRES formats language is designed to handle all a value's processing, in most cases, in one label group of statements.

Third, because the statements in a label group are actually elements in the LABEL-GROUP structure of a format definition record, they will be compiled and executed in the order in which they are stored in the FORMATS subfile, which may be different from the order in which you coded them. [See B.1.3, E.1.] In other words, you may code the three label group statements GETELEM, PUTDATA, and UPROC = JUMP in that order, but when the record is added to the FORMATS subfile, the order will be changed to GETELEM, UPROC = JUMP, and PUTDATA, and the statements would be executed in the changed order. (The PUTDATA statement would never execute, because the JUMP Uproc tells SPIRES to jump to the next label group.) Hence, treating a label group as a single, large instruction whose component statements work together in a standard order is more reliable than treating it as just a collection of instructions that will be executed. This subject will come up again in examples later in this chapter and the next.

Below is a list of the statements most commonly found in output format label groups, showing the order in which they are stored in the FORMATS record and are executed.

This chapter covers most of the label group statements listed above; a couple of them used primarily with structures, report formats or full-screen applications are discussed later. [See B.8.3, B.10.8, B.13.] The first few sections cover the most basic statements: LABEL, GETELEM, VALUE and PUTDATA. The remaining statements are then discussed in more or less the order of the categories shown above.

B.4.1  The LABEL Statement

The LABEL statement is the first and only required statement in a label group. It identifies the beginning of the label group and, if given a value, identifies the label group itself.

The syntax of the LABEL statement is:

where "label.name" is a string of one to sixteen characters. Like the FRAME-ID statement, the LABEL value should contain only alphanumeric characters and not special characters, with the common exceptions of periods, hyphens or underscores. [See B.2.1.] No blanks are allowed in the label name either.

If no label name is given (i.e., the LABEL statement is given a null value), the syntax is:

Label names are specified for several different reasons. Assigning a name lets you "jump" to that particular label group explicitly, using the XEQ PROC and JUMP Uprocs. The name will also be used as the default value for a subsequent GETELEM, PUTELEM or REMELEM statement in the label group. [See B.4.2, C.3.3, C.5.1.2.] The name will also be used by SPIRES to identify the label group in error messages if an error within the label group is detected during compilation. (If no label name is given, the label group is identified by a count from the last named label group.) And if you compile your format with the LABELS option, your label names will be used in SET FTRACE output. (SET FTRACE is a formats tracing and debugging command.) [See B.6.2, B.7.2.1.]

Here are some examples of LABEL statements:

B.4.2  The GETELEM Statement

The GETELEM statement tells SPIRES to retrieve an occurrence of the named element from the record being processed for handling by the current label group. In general, other statements within the label group will then position the element value in the frame, though that is not required -- the GETELEM statement simply retrieves the occurrence, and the rest of the label group determines what is done to it.

The most explicit form of the GETELEM statement is:

where "element.name" is the name of the element to be retrieved. Another form takes advantage of the label name supplied in the LABEL statement:

Here the name given in the previous LABEL statement is presumed to be the name of the element to be retrieved by GETELEM. [See B.4.1.]

For example, the two sets of LABEL and GETELEM statements below are equivalent:

Either set could be the beginning of a label group that is to retrieve the PHONE.NUMBER data element. If both the LABEL and the GETELEM statements are given values, the value given in the GETELEM statement is the element that will be retrieved.

When the element has multiple occurrences and you want to retrieve all of them, one by one, you must use the LOOP statement. [See B.4.8.4.] However, if you only want to retrieve one of the occurrences, you may request it explicitly:

where "n" represents an integer, either 0 (zero) or positive. (Beware: the first element occurrence is numbered 0, the second is 1, the third 2, etc.) Another form, "GETELEM = element.name::n", is now considered obsolete, though it may still be used; it was replaced by the form shown above, because the obsolete form is confusing in regard to variables, whose subscripts are indicated similarly. [See B.4.2.1 for information on using variables for the element name or the occurrence number.]

If no occurrence number is given, then the next occurrence (usually the first occurrence, number 0) is retrieved by GETELEM. Note, however, that the two statements below:

are not exactly equivalent. If you use the LOOP statement to retrieve multiple occurrences of the element, code the first statement rather than the second, for the second tells SPIRES to always retrieve the first occurrence. With the first statement, a LOOP statement will cause SPIRES to always retrieve the "next" occurrence. [See B.4.8.4.] Alternatively, in some situations you can use the SET STARTOCC Entry-Uproc to cause a loop to begin with a specified occurrence. [See B.4.8.5.]

Several other, less common forms for specifying the element to be retrieved are discussed later. [See B.4.2.1.]

The GETELEM statement is not usually coded for elements that are structures, though it can be. A structure is most often processed with an indirect frame. [See B.8.]

When a GETELEM statement fails to retrieve a value (that is, no occurrences or, in the case of a loop, no more occurrences of that element exist), the rest of the label group is skipped, and execution resumes with the next label group. The DEFAULT statement can be used to force SPIRES to continue executing the current label group in such cases. [See B.4.5.1.] Do not confuse "no occurrence" with a "null occurrence" where the element occurs but has no value. A null occurrence will not cause the rest of the label group to be skipped; however, the retrieved value is null.

When a GETELEM statement is executed, values for several important system variables are established. The most important are $UVAL ("Unconverted VALue") and $CVAL ("Converted VALue"). These two variables represent two forms of the element value retrieved, and they may be used to test or alter the value. [See B.4.5.4.] In fact, it is $CVAL that will be placed in the output buffer by the PUTDATA statement. [See B.4.4.]

B.4.2.1  (*) Other Forms of the GETELEM Value

Some other forms of the value for the GETELEM statement, though not commonly used, are discussed in this "optional" section. The forms are:

Each of these forms is discussed below. In addition, a use of the GETELEM statement with structures is discussed.

1) The first form may be used when the name of the element to be retrieved is stored in the given string variable. Alternatively, if it is a four-byte value of type HEX, then it represents the "element ID" ($ELEMID) of a particular element. [See E.2.3.10.] In either case, SPIRES uses the variable to determine the element whose values are to be retrieved.

Although a LOOP statement may cause the label group to be executed repeatedly, SPIRES will only do the variable substitution the first time through. In other words, changing the value of the variable within the label group will not cause the label group to retrieve a different element if the re-execution of the label group is caused by the LOOP statement. [See B.4.8.4.] However, if you leave the label group and return to it later, the current value of the variable will be used.

2) When "@n" (where "n" is an integer) is given as the value of GETELEM, SPIRES uses that integer as the absolute element number within the record (or within the structure, if the frame is an indirect frame processing a structure). Elements are numbered from 0 (zero) both at record-level and within a structure; the slot number key of a slot record-type is always number 0. This form may not be specified using variables.

3) The "@element.name" form is related to the "@n" form discussed above. It may be coded in an indirect frame that processes structures if the frame definition contains multiple SUBTREE statements. [See B.8.2.] This form tells SPIRES to convert the element name to the absolute element number, as in "@n". Then, this indirect frame can be called to process other structures, as listed in the SUBTREE statements, even though the element names in the GETELEM statements do not match the element names in the structure being processed. SPIRES will change the element names to the element numbers for the first structure listed in the SUBTREE statement and then use those element numbers when retrieving elements in the other structures.

As mentioned earlier, the GETELEM statement is seldom used to retrieve an entire structure. [See B.4.2.] In most cases, an indirect frame is coded to process the elements within the structure individually. However, a structure may be retrieved all at once with a GETELEM statement, usually if the structure is defined with the $STRUC or $STRUC.OUT system proc (A33) for an OUTPROC. [See B.8, C.5.3.]

You may also use GETELEM with a structure to find out how many occurrences of the structure exist:

The user variable NUM.ADDRESSES is set to the value of the system variable $ELOCC, which contains the number of occurrences of the data element processed by the GETELEM statement. [See B.9.3, E.2.2.26.]

(*) Specifying Element and Variable Occurrences

Because an element occurrence can be specified in the same way as on occurrence of a variable in an array, using the symbol '::', confusion can arise. For instance, examine the following statement:

Does this statement refer to the "Nth" occurrence of the element represented by #ELEMENT or to the "Nth" value of the variable array represented by #ELEMENT? SPIRES assumes the latter case -- #N represents the occurrence number of the variable, not of the element.

To specifically request the occurrence of the element rather than the variable:

Thus, these forms are equivalent:

In both cases, #N represents the occurrence number of the element ELEM. However, when the element name is in a variable, these two forms are not equivalent:

To specify both variable and element occurrence:

You can also specify the element occurrence number with an indexed variable:

The letter "I" as an occurrence number for the variable indicates that the definition of the variable included the INDEXED-BY statement, pointing to another variable whose value is to be used as the occurrence number for the first variable. [See the discussion of the INDEXED-BY statement in the manual "SPIRES Protocols" for further information.]

B.4.3  The VALUE (VAL) Statement

Sometimes in an output frame you have other values that are not elements that you want to be placed in the frame. The VALUE statement can be used to give the label group a value to process, just as it processes an element value accessed by the GETELEM statement. In fact, the GETELEM and VALUE statements are mutually exclusive: you may not code both of them in a single label group.

Specifically, the VALUE statement can be used:

The syntax of the VALUE statement is:

where "expression" is an expression following the same rules as expressions in a LET command or LET Uproc. [See B.4.8.10.] The type of the value (e.g., string, integer) depends on the result of the evaluation of the expression; it is not by default converted to a string value (see below).

Each individual part of the expression must not exceed 256 characters in length. The VALUE statement follows the LABEL statement in a label group definition.

For example, here is a VALUE statement specifying a string value to be placed in the frame:

This value might appear at the top of a personnel record, identifying the data that will follow. Note that string values should usually be enclosed in apostrophes. [See B.4.3.1.]

Values and Types

Values that are expressions or whose type is not string may require extra care in handling:

During format execution, SPIRES will evaluate the expression. By default, arithmetic is done using packed decimal values, so the pieces of this particular expression, "3" and "4", are converted to packed numbers, and the result, "7", is also a packed decimal. However, before the value is output, i.e., placed in the format, it needs to be converted to a string value. The easiest way to accomplish this is to apply the $STRING function to the expression:

Another alternative would be to code an OUTPROC, such as $PACK.OUT, that would convert the value to a string. [See B.4.5.2.] But regardless of your method, it is important to know the type of the evaluated expression and, if it is not a string and you are going to position the value in an output frame, to convert it to a string. [See B.4.5.4 for a further discussion of type conversions in this regard.]

The value of the evaluated expression is assigned to the system variable $UVAL. When SPIRES executes the VALUE statement (just like the GETELEM statement), the values of several important system variables are established, in particular $UVAL and $CVAL. [See B.4.5.]

B.4.3.1  (*) Special Characters in the VALUE Statement

Although it is not absolutely required, character strings in the VALUE statement should be enclosed in apostrophes:

If a string value is not enclosed in quotation marks, blanks within the value will be ignored when it is evaluated. For example,

would become ENDOFDATA for processing. It is interpreted as three separate strings to be concatenated together. (Blanks not surrounded by apostrophes or quotation marks are considered concatenation operators if no specific operator is given.)

It is possible, though not recommended, to use quotation marks (") instead of apostrophes (') around a string. However, because the VALUE statement is an element in a FORMATS goal record written in the standard SPIRES format, the quotation marks around the string must be doubled, and the entire value must be enclosed in quotation marks.

Here are some examples of values, the first being a mixed expression:

Step A shows the original value. Step B shows the quotation marks doubled, and step C shows the entire value being placed in quotation marks, as the value is assigned to the VALUE statement.

But compare that method to the apostrophe method:

Apostrophes are much easier to use than quotation marks in this context.

Special characters in strings do not require much special handling. As always, the characters to be careful with are the apostrophe, the quotation mark, and the semicolon. If the string contains an apostrophe and is to be surrounded by apostrophes, the internal apostrophe must be doubled:

This is also true for quotation marks, except that once again, the rules for quotation marks within element values for standard format record input must be followed too:

The original value is shown in step A. Step B shows the value in quotation marks to indicate that it is a string value, and step C shows the value as given in a VALUE statement, showing the entire value in quotation marks and all other quotation marks doubled.

If a value contains a semicolon, the entire value of the VALUE statement must be enclosed in quotation marks:

This example shows that the value of the VALUE statement is the string 'END OF DATA; END OF OUTPUT', including the apostrophes.

To summarize, when you have a character string that is either part of or the entire value of the VALUE statement:

These rules also apply to strings in LET and SET Uprocs. [See B.4.8.10.]

B.4.4  The PUTDATA Statement

The PUTDATA statement tells SPIRES to place the current value of the system variable $CVAL in the frame. That value usually represents the value created when the GETELEM or VALUE statement in the frame is executed.

The syntax of the PUTDATA statement is:

or

where "n" is an integer. The most common form is the first; the second form is discussed at the end of this section. [See B.8.1 for its use with indirect frames.]

The data value will be positioned in the frame starting in the row and column designated by the START statement [See B.4.6.1.] or, if no START statement is coded in the label group, at the default starting position.

A label group (or even all the label groups in a frame) may be as simple as the following:

In the left example, the value of element TITLE is retrieved (GETELEM) and placed in the frame starting in the default position (PUTDATA). In the right example, the string value "*" is placed in the frame in the default position. In both examples, the default position would be column 1 of the next row of the frame (the equivalent of START = X,1).

Several system variables are reset by the successful execution of the PUTDATA statement, in particular $CROW (current row) and $CCOL (current column). [See E.2.2.14, E.2.2.15.]

(*) The PUTDATA Statement and the Type of Value

When SPIRES executes a PUTDATA statement, it assumes that $CVAL is a string value; in other words, if $CVAL is not a string value but is, for instance, a packed decimal, SPIRES will not convert it to a string value before placing it in the frame. Instead, SPIRES will effectively "retype" the $CVAL variable to a string, as if the $RETYPE function were used. [See B.4.5.4 for a further discussion of data types and the PUTDATA statement.]

For example, suppose you have a label group that looks like this:

The value of $CVAL is "1.5", but it is a packed decimal value. Internally, it is stored as the hexadecimal characters "01 5C FF", along with the information that the value should be interpreted as a packed decimal. However, SPIRES ignores that last piece of information when the PUTDATA is executed, interpreting the value as a string. Hence the value that should be interpreted as a packed decimal is interpreted as a string when it is placed in the buffer, and the result is "garbage", data that is useless to you, in the format. (Note: There are situations where you might want packed decimal or integer data to be output without conversion to character strings, so that the data can be submitted to some other program that can read it in that form. In such cases, you would use the technique shown above intentionally, and the results would not be garbage to you.)

(*) Options on the PUTDATA Statement

As shown above, the PUTDATA statement can be coded with an integer value to specify different processing for special circumstances.

"PUTDATA = 1" has an effect when all of the following conditions are true:

When this situation arises and "PUTDATA = 1" is coded, the part of the value that fit in the buffer is blanked out, the system flag variable $OVERFLOW is set, and execution of the current frame stops. If the frame is an indirect frame, control returns to the calling frame; if the frame is a data frame, execution control continues with the next data frame, or if none exists, returns to command level. [See B.4.8.7, B.4.8.8, B.5.2.] The flag $OVERFLOW can be checked to determine whether those actions have taken place. [See E.2.1.23.]

If the situation occurs when the "1" option is not specified on the PUTDATA statement, the frame will overflow, causing the format processing to stop for the current record; error message S808 will be displayed.

A TITLE statement appearing in a label group having "PUTDATA = 1" will not be affected by the "1" option, meaning that the title could cause an S808 error. You should consider handling the title in a separate label group (positioning it with a VALUE statement rather than TITLE) with its own "PUTDATA = 1" if you want to use both the overflow-handling option and titles. [See B.4.7.]

"PUTDATA = 2" has an effect when the label group's value (that is, $CVAL) is null (i.e., has a length of zero). Normally, the value would not be placed in the buffer, meaning that the current row and column numbers would not reflect the placement of that value -- they would still reflect the placement of the last value placed in the buffer. If "PUTDATA = 2" is coded, then the current row and column numbers will be updated as if the null value had actually been placed in the buffer. [See B.4.6.1.]

"PUTDATA = 3" ensures that a value's length does not change if it wraps to other rows. With this option, if values are to be broken on blanks, succeeding blanks will not be stripped from the value when it wraps to the next line.

"PUTDATA = 4" acts like HOLDATA/FLUSHDATA on the containing label group.

"PUTDATA = 5" to be used if HOLDATA processing is to take place and the $CVAL being output is large (several lines) and would force the current screen out as a blank or nearly blank screen. PUTDATA = 5 tells the FORMATS processor to flush the beginning lines of the value onto the page despite the HOLDATA process.

B.4.5  Label Group Statements that Control the Value

Although a frame definition may contain label groups with only LABEL, GETELEM and PUTDATA statements, most label groups have more. The rest of this chapter will discuss the three other categories of label group statements, beginning with the statements that control the value that will be placed in the frame. Section B.4.6 will discuss those statements used to position the value within the frame, and B.4.7 will cover the statements used to control program execution.

In all of these sections, references will be made to the value being positioned in the frame. We will call that value "$CVAL", which is the name of a SPIRES system variable containing the value. It represents the "Converted VALue", that is, the value derived from a GETELEM or VALUE statement after it has been processed by INPROC, OUTPROC or INSERT statements. [See B.4.5.2, B.4.5.3.]

To be more specific, when SPIRES executes a label group containing either a GETELEM or VALUE statement, it establishes values for, among others, the two system variables $CVAL and $UVAL. For an element, $UVAL represents the internal, stored form of the element; $CVAL initially represents the external form, the form derived by processing the value through the OUTPROCs coded in the file definition or in the label group.

For a value or expression given in the VALUE statement, $UVAL and $CVAL are originally established with the same value, which is the result of the expression. Then, if an OUTPROC or INPROC statement appears in the label group, $UVAL is processed through that to establish a new $CVAL.

For either the GETELEM or VALUE value, if the INSERT statement appears, strings are then inserted in or appended to $CVAL, giving it a new value. Finally, the value of $CVAL can still be changed before it is placed in the frame, using the SET CVAL Uproc. [See B.4.5.4.]

One other statement allows you to set the value of $CVAL in some cases: the DEFAULT statement, which can be used to provide a default value when a GETELEM statement fails to find an element occurrence. [See B.4.5.1.]

B.4.5.1  The DEFAULT Statement

When SPIRES executes a GETELEM statement and no element occurrence is retrieved, the remainder of the label group is skipped, and execution resumes with the next label group. However, when the DEFAULT statement is coded and the GETELEM statement fails:

The DEFAULT statement has no effect when $CVAL is established by a VALUE statement rather than a GETELEM statement. It also has no effect in input frames.

The syntax of the DEFAULT statement is:

where "value" is a character string representing the value to be used when no value is retrieved by the GETELEM statement. This section will discuss the use of the DEFAULT statement's value. Later, the use of the DEFAULT statement to control label group execution will be discussed. [See B.4.8.4.]

The value supplied must be either a literal character string or a string variable; no other expressions are allowed. Like character strings given in the VALUE command, this value should usually be enclosed in apostrophes. The same rules given for strings in the VALUE command apply here. [See B.4.3.]

Here is an example of a DEFAULT statement:

If no value is retrieved by the preceding GETELEM statement, the value "No address" will be provided instead.

When a default value is accessed, it becomes the value of both $UVAL and $CVAL system variables. The default value will not be processed through any OUTPROC for the element, whether the OUTPROC is given in the file definition or in the label group itself. [See B.4.5.2.] The INSERT statement, if coded, will be applied to the default value, however. [See B.4.5.3.]

The system flag variable $DEFAULT is always set when default value processing occurs; you can test it if you need to know whether the default value is being provided. [See B.4.8.1, E.2.1.20.]

If no value is given in the DEFAULT statement (meaning that "DEFAULT;" was coded), then the value of $UVAL will be null. However, other value-control statements, such as INSERT, will still be applied to the null value to create $CVAL. [See B.4.5.3.] Unless "PUTDATA = 2" is coded, a null value for $CVAL will not be "placed" in the buffer, meaning that the current row and column numbers will not be updated to reflect the positioning of the null value. [See B.4.4.] It is important to consider how a null element value or a null DEFAULT value will be processed by each label group where it might occur, since how it is handled may affect the placement of other values later.

B.4.5.2  The OUTPROC (OUT) and INPROC (IN) Statements

When a GETELEM statement is executed and a value is retrieved, its internal value is processed through any OUTPROC rules given in the file definition for that element. The OUTPROC statement is used in an output frame for one of two purposes:

The syntax of the OUTPROC statement is the same in a format definition as it is in a record definition:

where "rule string" contains one or more processing rules (actions, system procs, or user-defined processing rules). If multiple rules are given, they are separated by single slashes (/), optionally surrounded by blanks.

Another form of the OUTPROC statement can be coded if you want to override file definition OUTPROC processing but not replace it with something else:

With this form, no OUTPROC processing will occur at all, and $CVAL and $UVAL will be equivalent.

A typical use of the OUTPROC statement is to display a different form of a stored date than the one chosen in the file definition:

The OUTPROC string usually consists of actions and system procs. A user-defined proc can be included, but it must be defined at the end of the format definition, or in an EXTDEF subfile record that is referenced in the EXTDEF statement at the end of the format definition (see below).

The A62 or A124 actions or the $CALL system proc, used to invoke USERPROCs, may be coded in the OUTPROC string. However, the actual USERPROC definition must be in the record definition of the record named by the RECORD-NAME statement.

The $STRUC, $STRUC.IN and $STRUC.OUT system procs (action A33) can be coded to retrieve a structure element as if it were a single element. [See B.8.1.]

The $LOOKUP proc (A32 rule) has an important security limitation worth noting here: If someone other than the file owner writes a format for one record-type in a file, and if that format contains $LOOKUP procs or A32 actions in either INPROC or OUTPROC statements, then users of the format must have been granted subgoal access to the accessed record-type; otherwise, an error during format processing will occur. The access may be granted only by the file owner in the file definition, through either the SUBGOAL statement in the subfile section or the FILE-ACCESS statements, where the account numbers of the format users must be given an access level of SEE (or a level incorporating SEE access).

Similarly, the file owner may require that access by some or all accounts to an element be limited to the external form of the element as determined by the file definition's processing rules. (This restriction is made with the combination of the OUTPROC-REQ and PRIV-TAG statements in the file definition.) If a label group containing an OUTPROC statement tries to retrieve such an element, no value is retrieved and, unless the DEFAULT statement appears in the label group, the remainder of the label group is skipped. [See B.4.5.1.] If default processing is requested, the value of $UVAL will be the same as $CVAL, i.e., the value after the file definition's OUTPROC rules are executed and any INSERT statements in the label group are applied.

(*) The EXTDEF-ID and PROC Statements

SPIRES allows you to create your own procs (collections of system procs and actions) where each string of processing rules is identified by a single name. That name can then be used in place of the string in OUTPROC and INPROC statements, for example, to identify the processing rule string that should replace it. The proc facility is discussed in detail in section C.10 of the manual "SPIRES File Definition".

Procs may be coded in OUTPROC and INPROC statements in format definitions. A proc may be used in a format if its definition appears in one of the following places:

          EXTDEF-ID = gg.uuu.idname;

The EXTDEF-ID statement also appears at the end of the format definition, following any proc definitions. (Both proc definitions and EXTDEF-ID statements are allowed in a single format definition.)

EXTDEF-ID used to be called PROCDEF, which is allowed as an alias.

(*) The INPROC Statement in Output Frames

Sometimes a processing rule that you want to use is only available as an INPROC. For example, you might want to verify that a value created by an expression in a VALUE statement is 5 characters long, and the $LENGTH system proc is only available as an INPROC. You can code an INPROC instead of an OUTPROC if the following three conditions are met:

(*) The OUTPROC Statement and Virtual Elements

Because the value of a virtual element is created by its processing rule strings, understanding how the presence or absence of an OUTPROC statement will affect it is important. If no OUTPROC statement is coded in a label group retrieving a virtual element, then the OUTPROC statement in the file definition will be used to create the $CVAL form of the value for the label group, while the OUTPROC followed by the INPROC will be executed to create $UVAL.

However, if an OUTPROC statement is coded in the label group, that OUTPROC string will be used to create the value of $CVAL, but the value of $UVAL varies; specifically:

This is the only situation in which the presence of an OUTPROC statement in the format definition will change the value of $UVAL -- in other situations, $UVAL is not affected by OUTPROC statements.

(*) Handling Processing Rule Errors in Output Formats

Processing rule errors can occur during record output, though they occur more frequently during input processing. The techniques used for handling them in input formats also work in output formats. [See C.3.6.] Several system variables are set when a processing rule error occurs, and they may be tested in the label group to determine whether an error has occurred. For example,

In this example, SPIRES verifies that the retrieved CODE value contains only numerals (the $VERIFY system proc). If it contains other characters, the error flag is set. The Uproc tests the flag $APROCERR, which is set if any processing rule executed during the execution of the current label group has caused an error. If that flag is set, SPIRES is to proceed to the label group ALPHA.CODE, preventing any further execution of the NUMERIC.CODE label group.

The "D" parameter in the $VERIFY proc represents the error level for the rule -- it overrides the default "S" level for that particular proc, which would cause an error message to be displayed at the terminal. [See C.3.6.4.] Note however that "S" level errors that occur during format processing do not stop the execution of the format -- this is true for both input and output formats. For output, the record will be completely processed by the format, even though a serious-error message may be displayed on the terminal screen. Remember that "S" level errors that occur during output processing when no format is set (i.e., when the standard SPIRES format is set) will prevent any further output of the record.

B.4.5.3  The INSERT (INS) Statement

The INSERT statement can be used to add a character string to the beginning, middle or end of $CVAL. Typically, it is used to identify the data, usually in a more attractive way than the standard format "element-name =" prefix, which could be considered a type of "insert".

For example,

On output, the value should look something like this:

There are three forms available for the INSERT statement:

The first form requests that the given string expression be inserted in front of the current value of $CVAL. The second form requests that it be appended to the end of $CVAL. The third requests that the string be inserted in front of the "nth" character of $CVAL; that is, the first character of the inserted string will begin at the "nth" character position in $CVAL, with the remainder shifted to the right; "n" is an integer. If "n" is used, and the current value of $CVAL has fewer than "n" characters, the insert string will be appended directly to the end of the value.

The string expression may have one of the following forms or a concatenation of them:

Note that END or "n" may not be expressed by variables but must be directly coded.

Because special characters are often used as insert characters, be careful to follow the data entry rules for the INSERT statement (i.e., putting a literal value in apostrophes) and for the record as a whole. [See B.1.3.] For example, to request that a semicolon be inserted at the end of a value, you would code the following INSERT statement:

Multiple INSERT statements are allowed, each one changing the value of $CVAL:

Even if $CVAL is null, the insertion will occur. This situation might arise when the DEFAULT statement is coded without a default value, setting $CVAL to null. [See B.4.5.1.] When the insertion is then applied, $CVAL becomes the insertion string.

Unlike the TITLE statement, which is only applied once in a label group, the INSERT statement will apply to all occurrences of an element processed in the label group with a LOOP statement. [See B.4.7 for a comparison of the TITLE, INSERT and VALUE statements, B.4.8.4 for the LOOP statement.]

B.4.5.4  The SET CVAL Uproc

One final way to alter the value of $CVAL is the SET CVAL Uproc. (Uprocs are command-like statements allowed in a label group.) [See B.4.5.5.] Since a label group's Uprocs are executed after the GETELEM, VALUE, DEFAULT, OUTPROC, INPROC and INSERT statements, the SET CVAL Uproc can completely override the processing of these statements. For example,

No matter what value $CVAL had before the Uproc was executed, it has the value "Author Unknown" afterwards, and that is the value that would be placed in the frame. That label group might just as well have been written as:

The syntax of the SET CVAL Uproc is:

where "expression" can consist of literal strings, user or system variables (including $CVAL itself, referring to its value before this Uproc is executed), or system functions. For example,

which is equivalent to the final example in the previous section, which used multiple INSERT statements. [See B.4.5.3.]

Though quotation marks around the "value" of the UPROC statement are usually unnecessary, they are added below to help show what the statement means:

That is, the value of the UPROC statement is "SET CVAL = expression". Note that both equals signs are optional, though the second one is usually included for readability's sake.

The SET CVAL Uproc is often used in conjunction with other Uprocs, in particular the IF...THEN Uproc:

Remember that the SET CVAL Uproc is the last chance to change the value of $CVAL before it is placed in the frame. It is easy to forget, for example, that INSERT statements will be applied to $CVAL before the SET CVAL statement is executed.

(*) Data Type Considerations for the SET CVAL Uproc

In an output frame, the SET CVAL Uproc will convert the result of the expression to a string. In contrast, the VALUE statement will not do that but will leave the type of the expression alone. However, for output, it is assumed that you will convert the "VALUE value" to a string somewhere before the PUTDATA with an OUTPROC or SET CVAL Uproc. [See B.4.3, B.4.5.2.]

For example, these two label groups do not produce equivalent results:

In both cases, the result of the expression is a packed decimal value, but the SET CVAL statement converts the result to a string; the VALUE statement does not make that conversion. However, the PUTDATA statement assumes that $CVAL is a string, and so it tries to read the packed decimal $CVAL as a string, producing "garbage characters" from the label group on the left. The character string "7" would be produced from the label group on the right.

The label group on the left could be repaired in several ways:

If you used one of these methods, $CVAL would be properly converted to a string for placement in the frame. In terms of efficiency, the second way is best -- using processing rules is more efficient than using functions. (Note that the fourth method exploits the fact that the SET CVAL Uproc automatically converts the value to a string.)

B.4.5.5  The UPROC (P) Statement

The SET CVAL Uproc (pronounced "YOU-prock") is one of many Uprocs to be discussed in this manual. [See B.4.5.4.] A Uproc is a statement specifying a procedure to be executed at that point in the frame execution.

Uproc statements closely follow the syntax of protocol statements (but see the notes below); in fact, many of them have the same names and purposes. Uproc statements may appear in label groups containing GETELEM and/or PUTDATA statements, or in label groups by themselves. Some of the tasks of Uprocs are:

Uproc statements are of the form:

Uprocs are executed in the order in which they appear in the label group definition. The allowable statements fall into several categories. Below is a list of all the Uprocs allowed in a format, though not all are allowed in every frame or every label group. Details on most of them are provided in other sections of this manual; consult the index for details on specific ones. (Some, such as REPROMPT and the SET DISPLAY Uprocs, are discussed in the manual "SPIRES Device Services".)

Procedural Language Statements Allowed

Special "Formats-Only" Uprocs

SET Uprocs

SET Entry-Uprocs

The following statements can only be used in Entry-Uprocs [See B.4.5.6.]

     SET BUILDEND     SET LOOP BACKWARDS     SET SUBGOAL
     SET BUILDSEP     SET STARTOCC

Formats Uprocs and Protocol Commands

The major differences between Uprocs in the Formats language and commands available in the Protocol language are:

          UPROC = *'The time is ' $TIME;
      (Command:)  /*The time is $TIME

Comments for Uprocs

Comments for Uprocs are usually handled by the "-" Uproc. You may code a Uproc statement without a value, in order to provide spacing in your format definition:

Though they are more difficult to create and read, comments may also be appended to Uprocs by using the semicolon delimiter to separate the Uproc from the comment. Remember though that because the UPROC statement is an element in a FORMATS goal record, the entire value must be placed in quotation marks if internal semicolons appear:

B.4.5.6  The ENTRY-UPROC Statement

An Entry-Uproc is a special kind of Uproc statement [See B.4.5.5.] with a significant difference in timing: the Entry-Uproc is executed at the beginning of the label group, where a standard Uproc is executed near the end of the label group. Specifically, Entry-Uprocs are executed before any GETELEM statement (or, in an input format, a GETDATA statement), whereas Uprocs are executed after any GETELEM statement, though before a PUTDATA statement.

As a reminder, here is the order in which statements discussed so far in this chapter would be executed within a given label group:

Entry-Uprocs Compared to Uprocs

As with Uprocs, you can have more than one Entry-Uproc in a label group if you wish. In fact, much of what was said about Uprocs also applies to Entry-Uprocs, with these differences:

Uses for Entry-Uprocs

An Entry-Uproc can be especially useful for setting up an initial condition or environment in its label group. Because a Uproc executes only near the end of its label group, it cannot affect initial conditions (e.g., before an element occurrence is first retrieved) unless it is coded in a previous label group. The two sets of statements below accomplish the same task, but the second is slightly more compact:

 LABEL = SET.INITIAL;
   UPROC = * 'The following locations stock item ' #ITEM;
   UPROC = * '------------------------------------------------';
 LABEL = STORE.LOC;
   GETELEM;
   INSERT = 'Location: ';
   PUTDATA;
   LOOP;

 LABEL = STORE.LOC;
   ENTRY-UPROC = * 'The following locations stock item ' #ITEM;
   ENTRY-UPROC = * '------------------------------------------------';
   GETELEM;
   INSERT = 'Location: ';
   PUTDATA;
   LOOP;

When in doubt which of the two types of statement you need, ask yourself at what point in the label group you want the statement to take effect: if it affects a retrieved or created element value, then you probably want a Uproc. If, on the other hand, it should take effect before a value is retrieved (or created), and if it does not need to be reset for multiple iterations of the label group, then perhaps it should be an Entry-Uproc. [For the sake of economy, this manual sometimes uses the term Uproc in cases where either a Uproc or Entry-Uproc could be equally appropriate.]

A section later in this chapter will describe two procedural statements, SET STARTOCC and SET LOOP BACKWARDS, that must always be coded as Entry-Uprocs, not Uprocs, because they must take place within the label group where the occurrence is retrieved, but before the GETELEM statement is executed. [See B.4.8.5.]

B.4.5.7  The SET BUILD Entry-Uprocs

The SET BUILD collection of Entry-Uprocs provides a handy way to display multiple occurrences of an element as though all the occurrences really formed a single occurrence. The two statements let you concatenate multiple occurrences with a minimum of effort. Additionally, they provide more flexibility in preparing a value for output, with their effects on the value taking place "at the last second" -- they don't affect the value of $CVal as, say, the INSERT statement does.

Here are the 6 SET BUILD Entry-Uprocs:

     SET BUILD SEPARATOR 'string'       (or SET BUILDSEP)
       - concatenates multiple occurrences within a single
         looping label group, gluing them together with the
         specified separator string (e.g., a comma and space).
     SET BUILD END 'string'             (or SET BUILDEND)
       - used with SET BUILDSEP, appends the specified string
         to the end of the value (e.g., an ending period).
   * SET BUILD PREFIX 'string'          (or SET BUILDPRE)
       - used to add a string to the beginning of the value
   * SET BUILD CONTAINER 'string'       (or SET BUILDCON)
       - specifies a container character used to surround
         the entire value if the value contains that character
         or others you specify
   * SET BUILD SURROUND 'string'        (or SET BUILDSUR)
       - specifies a character or pair of characters used to
         surround the entire value; compare with SET BUILD
         CONTAINER
   * SET BUILD ESCAPE 'string'          (or SET BUILDESC)
       - specifies an escape character and other characters
         that should be preceded by the escape character if
         they appear in the value
  (* = available as Uprocs as well as Entry-Uprocs, but does not
       work as efficiently)

SET BUILD SEPARATOR and SET BUILD END

SET BUILD SEPARATOR concatenates multiple occurrences within a single looping label group, appending a specified character set (such as a comma and space) at the end of each occurrence but the last one. [See B.4.8.4 for information on looping.] SET BUILD END, issued within the same label group, lets you append a different character set (perhaps a period) after the last concatenated occurrence.

These Entry-Uprocs create a new mode of operation within the label group, causing positioning statements such as START, MARGINS, and ADJUST to be executed only once, after the looping is finished. (The XSTART statement would not be needed or used at all.) Since the PUTDATA statement will also execute only after all occurrences have been concatenated, end-users of the format will probably never even know that the "single" value displayed is stored as multiple occurrences in the data base.

Syntax of SET BUILDSEP and SET BUILDEND

The syntax of these Entry-Uprocs is as follows:

   SET BUILD SEPARATOR 'string'
   SET BUILD END 'string'

where "string" consists of the character(s) you wish to append to the occurrences as they are displayed.

Thus, the Entry-Uprocs might look like this:

   ENTRY-UPROC = SET BUILDSEP ', ';
   ENTRY-UPROC = SET BUILDEND .;

   ENTRY-UPROC = "SET BUILDSEP '; '";

You can use SET BUILDSEP without SET BUILDEND, but SET BUILDEND makes no sense except as a partner to SET BUILDSEP.

Example of SET BUILDSEP and SET BUILDEND

For an example of how these Entry-Uprocs work, consider a "restaurant" record with multiple values for CUISINE, as below:

 NAME = Castle Grand;
        :
 CUISINE = Continental;
 CUISINE = Crepes;
 CUISINE = Quiche;
        :

A looping label group could retrieve all the occurrences, separating each occurrence by a comma and space, and appending a period at the end:

 LABEL = CUISINE;
   ENTRY-UPROC = SET BUILDSEP ', ';
   ENTRY-UPROC = SET BUILDEND '.';
        :                           <---(Statements such as
   GETELEM;                              TSTART, TITLE,
        :                           <--- and START not shown)
   PUTDATA;
   LOOP;

The resulting display would show CUISINE as a "single" occurrence:

 Name: Castle Grand
 Cuisine: Continental, Crepes, Quiche.

Some Cautions and Considerations

It is important to remember that the concatenated occurrences only seem to be a single occurrence -- technically, they remain multiple occurrences. Thus, if the label group included an INSERT Uproc (or a SET CVAL Uproc) to insert or append a string value, the specified string would be inserted or appended within every one of the multiple occurrences. To insert a single value at the beginning of the concatenated value, use the TITLE statement instead. [See B.4.7.]

Note that if a file definition uses the $BUILD proc to concatenate element values, the $BUILD processing will be overridden by a format label group that uses the SET BUILDSEP Entry-Uproc -- the Entry-Uproc takes precedence.

SET BUILD PREFIX

The SET BUILD PREFIX Uproc, as well as the remaining ones described here, does not cause multiple occurrences to be concatenated. The "building" applies to a single element value, unless SET BUILDSEP is also in effect for the label group. If SET BUILDSEP is in effect, then any of these other SET BUILD Uprocs that are included in the label group will be executed only once, when the final concatenated value is being produced for output.

SET BUILD PREFIX adds a prefix to the value in $CVal just prior to output. This is similar to INSERT except that SET BUILD PREFIX does not actually change the value of $CVAL as INSERT does.

SET BUILD CONTAINER

The SET BUILD CONTAINER Uproc is similar in principle to the way the standard format works in SPIRES. There, for example, if a value contains a semicolon, then the entire value is surrounded by quotes. And if the value contains quotes, the quotes are doubled, and then the entire value is surrounded by quotes.

With SET BUILD CONTAINER, you can specify the characters that get this type of treatment in the label group.

If only one character is specified, that is the character used to surround the value if the value contains other occurrences of the character. Additionally, those other occurrences of that character will be doubled.

If multiple characters are specified, the first is treated as described above. Also, if any of the other characters appear within the value, then the entire value will be surrounded by the first character.

If you specified:

that would be equivalent to the way the standard format in SPIRES works. Note the doubling of the quotation marks with the Uproc's value, and that the entire Uproc value must be put in quotation marks.

SET BUILD SURROUND

The SET BUILD SURROUND Uproc is similar to the SET BUILD CONTAINER Uproc except that SPIRES will always surround the value with the surround character(s), no matter what characters may appear in the value.

If only one character is specified, that is the character used to surround the value on each side. If two characters are specified, the first is used on the left side of the value, and the second on the right.

If you specified:

then the value "ABC" would come out as "<ABC>".

SURROUND and CONTAINER should not both be specified; if they are, SURROUND will be given precedence.

SET BUILD ESCAPE

The SET BUILD ESCAPE Uproc will precede each of a set of characters in the output value with an "escape" character.

The first character is the escape character. Other characters may follow.

SPIRES will scan the value for occurrences of any of the characters, including the escape character. If any appear in the value, SPIRES precedes them with the escape character.

For example:

If the value contains any back slashes or asterisks, SPIRES will precede them with a back slash.

B.4.6  Label Group Statements that Control the Placement of the Value

Unless told otherwise, the PUTDATA statement places the current value of $CVAL starting in the first column of the next row of the frame. The statements discussed in the sections of B.4.6 provide you with much flexibility in positioning a value within a frame.

The first ones to be discussed relate to the starting row-column positioning of the value: START, XSTART, SET STARTROW Uproc and SET STARTCOL Uproc. Just as the value-control statements were involved with the system variable $CVAL, the value-placement-control statements are involved with the system variables $CCOL (Current COLumn) and $CROW (Current ROW). The values of these two variables represent the current position in the frame, that is, the location where the last data placed in the frame ended. [See B.4.6.1 for an example using the current position.]

The subsections of B.4.6 cover how the value is to be positioned, such as how many rows long it can be, whether it should be centered in the row, and so forth: MARGINS, MAXROWS, LENGTH, BREAK, SET ADJUST Uproc, SET JUSTIFY Uproc and so forth. These statements establish or interact with the value's "field", that is, the area in which the value will or can be placed.

B.4.6.1  The START Statement

The START statement specifies the starting row and column for the positioning of $CVAL in the frame.

The START statement may be specified in any of the ways below:

where "row" is one of the following:

and where "column" is one of the following:

The row specifications are described above as they are used in fixed-dimension frames, as opposed to line-by-line ones. See the discussion below for details on how they apply to line-by-line processing.

The "row" or "column" value may not be an expression, other than those explicitly allowed above, such as "X+1". If you must use an expression, use the SET STARTROW or SET STARTCOL Uprocs instead. [See B.4.6.2.]

Here is an example of a frame definition using START statements, followed by a picture demonstrating the outcome of its execution:

Below is the produced frame, surrounded by a box, with column and row numbers shown on the outside:

          ....v....1....v....2....v....3
         +------------------------------+
       1 |1        fiv6                 |
       2 |two 3                         |
       3 |     4                        |
       4 |                              |
       5 |7  9                          |
         +------------------------------+

Here are some notes on each label group:

In the example above, we could have replaced all the Xs and "*+3"s and so forth with explicit numbers, because we knew the exact lengths of all the values we were positioning. The * and X values are most often useful when you are handling values of varying or not easily determined lengths. For example, suppose you want the two elements REVIEWER.NAME and REVIEW.DATE to appear like this:

You do not know how long the reviewer's name will be, but you do not need to:

In the REVIEW.DATE label group, $CVAL is positioned two columns to the right of where the REVIEWER.NAME value ended, regardless of its length. Even if the name had been so long that it wrapped around into the next row, the REVIEW.DATE value would have been placed properly following it. [See B.4.6.3 for a discussion of how long values wrap around.]

Two other types of START statements having the same syntax shown above are allowed in a label group: XSTART and TSTART. XSTART is used when multiple occurrences of an element are being processed by the label group with the LOOP statement. [See B.4.8.4.] TSTART is used when a TITLE statement appears in the label group to indicate the starting position for the TITLE. [See B.4.7.]

The START Statement with Line-by-line Processing

When a frame is being processed line-by-line, either because the FRAME-DIM statement so declares [See B.3.3.] or because SET FLUSH processing is in effect [See B.3.3.] the "row" options cited above for the START command have slightly different meanings and uses. This is because once you have started putting data on a new row in line-by-line processing, you cannot return to an earlier one.

That might seem to make row numbers useless, but that is not the case. Each time data is placed in a new row, that row ($CROW) is equivalent to row 1. In other words, in line-by-line processing:

Similarly,

So, in line-by-line processing, remember that if one label group specifies a starting row of 3, and the next label group specifies a starting row of 5, the second value positioned will start on the fifth row from the end of the first value, counting the row where the first value ends as "1". Also in line-by-line mode, the row "*+1" is always equal to "X". (Though that is often true in a frame with fixed dimensions, it is not true when the last data positioned was placed above, i.e., in a lower-numbered row than, other data already positioned.)

B.4.6.2  The SET STARTROW and SET STARTCOL Uprocs

The two Uprocs SET STARTROW and SET STARTCOL can be used instead of or in conjunction with the START statement. [See B.4.6.1.] Like the START statement, these Uprocs override the default starting position for a value being placed in the frame. Unlike the START statement, they may contain expressions for values and may be conditionally controlled, using the IF... THEN Uproc. [See B.4.8.1.] These statements override the START statement when there is a conflict.

The syntax of these two Uprocs is:

where "expression" is an expression that evaluates as an integer. The expression may not contain the "*" or "X" characters allowed in the START statement; instead, you should use the $CROW and $CCOL variables as appropriate.

Also like the START statement, the SET STARTROW and SET STARTCOL Uprocs do not change the values of $CROW and $CCOL. Those values change only when data is actually placed in the frame by a PUTDATA statement. [See B.4.4.] Hence, if no PUTDATA occurs in the label group in which a SET STARTROW or SET STARTCOL Uproc appears, the Uproc has no effect whatsoever.

These two Uprocs have no effect in input frames.

B.4.6.3  The MARGINS (MAR) Statement

SPIRES, by default, uses the column specification in the FRAME-DIM statement to establish margins for values placed in the frame. [See B.3.3.] For example, if "FRAME-DIM = 3,68;" representing a frame three rows high by 68 columns across, SPIRES will not place data any further to the left than column 1 and no further to the right than column 68. If a value would spill over the right margin, by default it "wraps around" into the next row, starting in column 1 and running toward column 68. If it is still too long, it wraps around again into the next row, and so forth. When the value wraps around, SPIRES looks for a blank character (by default) at which to split the value (see example below).

The MARGINS statement lets you change the default margins for the current value. Its syntax is:

where "lcol" and "rcol", representing the leftmost and rightmost column margins respectively into which data will be placed, are each one of the following:

Generally speaking, the starting position of a value is controlled by the START statement; how it wraps around into other rows is handled by the MARGINS statement. Specifically, the right margin of the first row's worth of the value is controlled by the "rcol" value in the MARGINS statement, as is the right margin for subsequent rows. The left margin of the second row, where the wrap-around resumes, is controlled by the "lcol" value of the MARGINS statement.

However, if no START statement appears, or if the starting column is given as "0" for default, then the default starting column is "lcol" of the MARGINS statement. In other words, placement of a value begins where START (or MARGINS, if no specific starting column is given in the START statement) specifies; the remainder of the row and subsequent rows are controlled by MARGINS.

For example:

Here is a value processed by that label group (the column numbers are shown for clarity):

The value began in the position given by the START statement (row 1, column 5); for the left margin of subsequent rows, the MARGINS value "1" was used; for the right margin, the value "55" was used. Had the START statement been omitted, the quotation would have started in column 1, since "lcol" in the MARGINS statement was "1".

Note that a MARGINS statement only affects the current label group. It does not carry over into subsequent label groups.

B.4.6.4  The BREAK (TRUNC) Statement

The MARGINS statement controls the margins for long values that wrap around into other rows. The BREAK statement controls what character or characters the value can be split upon. By default the value can be split only across rows at blanks. If no blank appears in the value before the end of the row, the value is split with as many characters on the row as possible, that is, with characters up to the right margin.

Here is how the BREAK character is used: If $CVAL will not fit in one row within the given margins, SPIRES temporarily puts as much of the value as will fit, and then scans the value backwards from that point, looking for the break character. The portion of the value up to and including the break character is kept in the row, and the remainder of the value is then started in the next row, and the process begins again. This continues until the value is exhausted, or until the MAXROWS value is exceeded. [See B.4.6.6.] This procedure is slightly different when the break character is a blank; it is described below.

The BREAK statement has several forms:

The character specified will be used as a break character, instead of the default blank. If the character is a special character, the value of the statement should be in quotation marks ("character") not apostrophes ('character').

This form means that either character can be used as the break character. In any given situation, the character allowing the most characters in the row will be used. Like the first form, the second should be used with quotation marks if special characters are specified.

This form means that no break character will be used at all. Values that would wrap around into another row will fill the first row completely and then resume, perhaps "midword", in the next row.

This special form indicates that no wrapping into the next row is to occur at all. If the value would go beyond the right margin, it is truncated there. Compare this method of fixing a value length (i.e., the MARGINS and BREAK statements) with the LENGTH and MAXROWS statement. [See B.4.6.5, B.4.6.6.]

Note that the default blank break character does not apply in label groups that call indirect frames. BREAK = NONE becomes the default for such label groups, but it can be overridden by explicitly coding a BREAK statement. [See B.4.6.4, B.8.1.]

(*) Blanks as Break Characters

When the break character is a blank, the break processing is slightly different from the way described above, though in a manner you would expect. Suppose that "rcol" represents the column of the right margin. If the character in "rcol" is not a blank, but the next character is, the value will be split at that blank, and the next non-blank character will begin the value in the next row. (Only when a blank is the break character can the "rcol+1" character affect the break, and then only when that character is a blank will it be discarded.)

For example, suppose you have the following values, called A and B:

If the margins in effect are 1 and 18 (for "lcol" and "rcol" respectively, and the values start in column 1, here is what they would look like on output (note the use of the character "x" at the end of rows to indicate null characters):

For value B, the blank following "hand," was suppressed -- it did not start the next row. For value A, the blank following "value" did appear on the first row since there was room for it there.

In some situations, such as cases where you want to overwrite a previously written field, it is important to realize that a blank used as a break character in the value will be output at the end of the row, just as a non-blank break character would be, as shown above for value A. (There is an exception to that statement. In the situation described above, where the "rcol+1" character is a blank and causes the break, that blank is suppressed. Similarly, if multiple blanks appear around the point of the break, those blanks up through "rcol" will be output; the rest will be suppressed, and the next row will begin with the next non-blank character.)

For example, here is value C:

Using the same margins and starting column as the above example, SPIRES produces the following:

The internal blanks through column 18 are output; the blanks in columns 19, 20 and 21 are suppressed.

Concerning the nulls displayed as "x"s in the examples above: When the buffer is constructed internally, it is "empty", i.e., full of blanks. When values are positioned in the buffer, they overwrite the blanks. In some situations, such as that described above where values contain blanks, it is necessary to distinguish between the buffer's "empty" blanks and the value's blanks, so the term "nulls" refers to the blanks in the buffer which are not overwritten by any values. Once the frame processing has finished, the differences between the two types are irrelevant -- all blanks in the buffer are treated the same. [See B.4.8.7, B.8.1.]

B.4.6.5  The LENGTH (LEN) Statement

The LENGTH statement can be used to limit the length of $CVAL that will be placed in the output frame. LENGTH is specified as the number of characters allowed in the output value. If $CVAL is longer than that, it will be truncated as it is placed in the frame; if it is shorter, it will be padded with a "pad character" if one is in effect. By default, the pad character is null, meaning no padding occurs. [See B.4.6.7.] Note that $CVAL is not itself changed by the LENGTH statement; that is, $CVAL will not include the additional pad character nor be shortened to "length" characters. The LENGTH statement does not affect anything in the label group until the PUTDATA statement is executed.

The syntax of the LENGTH statement is:

where "length" is an integer or a variable of type INTEGER that may be indexed but not subscripted.

B.4.6.6  The MAXROWS (ROWS) Statement

The MAXROWS statement establishes the maximum number of rows in which the value can be stored. By combining MAXROWS with the margins in effect, SPIRES creates an area in the frame in which the data value will be placed. If the value would exceed the area, it is truncated. If the value is smaller than the given area, the pad character, if set, is used to fill the remainder.

The syntax of the MAXROWS statement is:

where "nrows" is an integer or an integer variable.

A fairly common use for MAXROWS is when you have an element that must begin on a specific row number but which follows a textual element whose length may vary. The MAXROWS statement may be used in that case to limit the size of the textual element.

Note that row X and row *+1 may not be the same after a label group with this statement executes -- X would represent the "next row" after "nrows" from the MAXROWS statement, while *+1 would represent the next row past the row in which the positioned value actually ended, which may be one or more rows shy of "nrows".

B.4.6.7  The SET PADCHAR Uproc and the PADCHAR Statement

Though more commonly used in full-screen applications, the SET PADCHAR Uproc and the PADCHAR statement may be used to provide a pad character to fill out fields established by LENGTH and/or MAXROWS statements. [See B.4.5.5 for more information about Uprocs.] (Remember that, like many Uprocs, SET PADCHAR can also be used as an Entry-Uproc, which would mean that it would be executed at the beginning of its label group, rather than at the end.)

The syntax is quite simple:

Only one character is allowed; if multiple characters are specified, only the first will be used. A string variable may be used in place of 'character' if desired.

As an Entry-Proc, SET PADCHAR will affect the current label group and any subsequent ones. (As a Uproc, it would not affect the label in which it appeared, only subsequent ones.) The pad character remains in effect until another SET PADCHAR Uproc (or Entry-Uproc) is encountered. The pad character is reset to null each time the format is executed, meaning that it should be set in a data frame that will be executed when the pad character is needed.

The PADCHAR statement sets a pad character that applies only to the label group in which it appears. If a pad character has been set by a SET PADCHAR Uproc, it is overridden for the label group's execution, but still applies to all subsequent label groups.

To turn off padding explicitly, set the pad character to null like this:

Unlike most SET Uprocs, SET PADCHAR does not have a system variable directly associated with it -- that is, there is no $PADCHAR variable.

B.4.6.8  The SET ADJUST Uproc

The SET ADJUST Uproc lets you right- and/or left-adjust or center the value within the given margins. By default, values are left-adjusted in their fields.

The SET ADJUST Uproc has the following syntax:

where "adjust" is one of or a variable containing one of the following values:

Any other value has no effect.

Here is a demonstration of the effects of the SET ADJUST Uproc:

     LABEL = REMARKS;
       GETELEM;
       MARGINS = 1,28;
       START = 1,1;
       UPROC = SET ADJUST = adjust;
       PUTDATA;

when "adjust" = LEFT:              when "adjust" = RIGHT:
 ....v....1....v....2....v...       ....v....1....v....2....v...
 Egg covers wide area; has             Egg covers wide area; has
 small amount of thick white;       small amount of thick white;
 yolk is somewhat flattened           yolk is somewhat flattened
 and enlarged.                                     and enlarged.

when "adjust" = JUSTIFY:           when "adjust" = CENTER:
 ....v....1....v....2....v...       ....v....1....v....2....v...
 Egg covers  wide  area;  has        Egg covers wide area; has
 small amount of thick white;       small amount of thick white;
 yolk is  somewhat  flattened        yolk is somewhat flattened
 and enlarged.                             and enlarged.

(*) The ADJUST Statement and the SET JUSTIFY Uproc

Two other statements can be used as well, though neither is as versatile as the SET ADJUST Uproc. The ADJUST statement may appear in a label group following the BREAK statement. It may have the same values as the SET ADJUST Uproc (i.e., RIGHT, JUSTIFY, etc.) although the value cannot be a variable. For example,

The SET JUSTIFY Uproc is an abbreviated form of the "SET ADJUST = JUSTIFY" Uproc -- its syntax is simply:

Again, because it handles all the situations, the SET ADJUST statement is the recommended method.

B.4.6.9  The SET REPEAT Uproc

The SET REPEAT Uproc causes the value of $CVAL to be duplicated to fill the field defined by the LENGTH and/or MAXROWS statements. The value of $CVAL is not changed, only the value as it is placed in the frame.

The syntax of this Uproc is very simple:

This Uproc is frequently used to make borders or division lines:

The result on output would be:

     ----------------------------------------
    (....v....1....v....2....v....3....v....4)

(The columns are shown beneath for clarity.)

If you want blanks between occurrences of the repeated value, you need to add a blank to the end (or perhaps to the beginning) of $CVAL, possibly by using the SET CVAL Uproc or the INSERT statement, or, as in the example above, the VALUE statement:

If no LENGTH or MAXROWS statement appears in the label group, the value will be repeated to fill the row in which it begins, up to the given (or default) right margin.

The SET REPEAT Uproc has no effect in input frames. Also, there is no $REPEAT variable associated with the SET REPEAT Uproc.

Do not confuse the SET REPEAT Uproc with the LOOP statement, which tells SPIRES to begin executing the label group again. SET REPEAT does not affect the execution of the label group; it affects the value being placed in the buffer.

B.4.6.10  The SET SKIPROW Uproc

The SET SKIPROW Uproc may be used in an output format label group to cause double spacing of the value being output to the buffer. That is, if the value being output would wrap into multiple rows, double spacing between rows of output will occur. Note though that double spacing between occurrences of the element or between different element values must be accomplished with the START and XSTART statements. [See B.4.6.1, B.4.8.4.]

The "skiprow" condition is cleared at the start of each label group. Although SET is part of the Uproc's syntax, there is no $SKIPROW variable.

B.4.7  Element Headings: the TITLE and TSTART Statements

The TITLE statement specifies a string to be placed in the frame independently from the value of the label group:

where "string-expression" is an expression consisting of strings and/or variables that evaluate as strings.

The starting position for the TITLE string is specified with the TSTART statement, which immediately precedes the TITLE statement. It has the same syntax as the START and XSTART statements. [See B.4.6.1.]

If no TSTART statement is coded, the title is placed in the same default position as $CVAL is when no START statement is coded: the next row (X), starting in column 1.

A Comparison of TITLE, INSERT and VALUE Statements for Headings

All three of these statements can be used to create headings for values being placed in the frame. However, in most situations, one of them will clearly be preferable to the other two, depending on your particular requirements. Relevant points concerning each statement are shown below:

The TITLE statement

The INSERT Statement

The VALUE Statement

In summary, headings and titles not directly related to element occurrences are best handled by the VALUE statement. Headings for elements, especially multiply occurring ones, are best handled by the TITLE statement. If the heading is to appear on the same row as the element value and the element will have only one occurrence, the INSERT statement may be a bit easier to use than TITLE and TSTART. [In full-screen applications, other considerations regarding protected and unprotected fields may apply to these statements. See the manual "SPIRES Device Services" for further details.]

B.4.8  Statements that Control Frame Execution; Procedural Statements

It is important to remember that a format is a program to be executed, and as such, it needs to have standard procedural capabilities, such as loops, subroutines, and branching. This section will discuss the label group statements, some of which have been introduced already, that control the execution of a frame. Although most of them are Uprocs (i.e., coded as values in UPROC statements), or Entry-Uprocs, some of them are statements themselves.

For example, one of the most important of these statements is the LABEL statement, discussed earlier. [See B.4.1.] Whenever branching occurs, execution continues at a LABEL statement, the beginning of a label group; you cannot jump into the middle of a label group. The rest of section B.4.7 will discuss other execution-control statements.

Most of the procedural statements are particular Uprocs. [See B.4.5.5.] Many of them, particularly the LET, SET and IF... THEN Uprocs, are commonly used with system or user variables. Variables are discussed in detail in a later chapter. [See B.9.]

The procedural statements for output frames are divided into seven major categories: condition tests, subroutine calls, branches, exits, variable manipulation, terminal input/output, error handling and comments. Error handling is covered in detail in the input portion of this manual -- the techniques described there are generally applicable to output formats also. [See C.3.6.]

B.4.8.1  Condition Tests: The IF... THEN, THEN and ELSE Uprocs

Using the IF... THEN Uproc, you can specify that a particular Uproc be executed only if a given "simple" condition (or series of simple conditions) is met. Each simple condition usually involves the testing of a variable, perhaps comparing it to some specific value.

The syntax of the IF... THEN Uproc is:

where "uproc" is another Uproc allowed in the format, and "condition" is either a simple condition (defined below) or the result of a series of simple conditions combined with the logical operators AND or OR. [Note that the word THEN in an IF... THEN Uproc may be replaced by a colon.]

A "simple" condition may be one of the following:

     =     (is equal to)
     ~=    (is not equal to)
     >     (is greater than)
     >=    (is greater than or equal to)
     <     (is less than)
     <=    (is less than or equal to)

The complete condition that is tested in an IF... THEN Uproc may consist of more than one "simple" condition, combined using the AND or OR logical operators:

   UPROC = IF $CVAL = '94305' OR = '94306' THEN LET CODE = 'Local';

You can negate the condition by putting the condition in parentheses and preceding it with the word NOT:

   UPROC = IF NOT ($CVAL = '94305' OR '94306') THEN BEGINBLOCK;

The rules for coding and combining conditions in a formats Uproc are the same as the rules described in the "SPIRES Protocols" manual for the IF... THEN command; see the Protocols manual for details on such topics as when parentheses are necessary and when a repeated "term1" may be omitted in the compound condition.

The THEN and ELSE Uprocs

After an IF... THEN Uproc, two other Uprocs may be used whose outcomes depend on the result of the IF... THEN comparison. They are the THEN and ELSE Uprocs:

If the previous IF... THEN comparison is true, the Uproc given in the THEN Uproc will be executed; otherwise, it will not. Vice versa, if the previous IF... THEN comparison is false, the Uproc given in the ELSE Uproc will be executed; otherwise it will not.

Examples

Here are some examples showing how these Uprocs might be used. In the first example below, if the value of $CVAL is "N/A", it will be changed:

Below, a function is used in the comparison:

This Uproc declares that whenever $CVAL is longer than sixty characters, the user variable LONGVAL will be assigned the value of $CVAL.

In the next example, a user-defined flag variable, #NOPHONE, is tested. Both of these Uprocs are equivalent:

In other words, if the NOPHONE flag is set, then the RETURN Uproc should be executed. [See B.4.7.7.]

Here is a series of Uprocs, using the THEN and ELSE Uprocs:

The above example is interesting in that it shows how several IF... THEN Uprocs can interact. The sample values for $CVAL and their impact on the Uprocs show how the two variables $CVAL and #SMALLCLASS are set. Note that the ELSE in the third Uproc refers to the IF... THEN Uproc in line 2 when $CVAL is less than 10 but to the one in line 1 when $CVAL is not less than 10. (This example was presented here to illustrate how IF... THEN Uprocs can interact, not to recommend using them this way -- such coding is admittedly confusing.)

(*) Using the IF... THEN Uproc Across Label Groups

The THEN and ELSE Uprocs almost always refer to the last executed IF... THEN Uproc, even if it is in an earlier label group or even an earlier frame or calling frame. However, if the Uproc in an IF... THEN Uproc is an XEQ PROC Uproc, e.g.,

the result of the IF-test will be remembered by SPIRES when it returns (if it went there) from the FIRST.RECORD procedure. In other words, even though IF... THEN Uprocs may appear in the FIRST.RECORD proc (and THEN and ELSE Uprocs in the proc will be affected by them), any THEN and ELSE Uprocs executed after SPIRES returns from the proc will be controlled by the IF... THEN Uproc shown above. [See B.4.7.5.]

B.4.8.1a  The Block-Construct Uprocs

This section describes the block-construct Uprocs:

Block-Construct Uprocs in General

[This information is condensed from the reference manual "SPIRES Protocols", which contains more detailed information about block constructs.]

A block construct is a set of Uprocs delimited by two Uprocs, one at the start and one at the end, that define the type of block. The Uprocs within the block execute in order, one by one, but they can also be regarded as a self-contained entity that executes as a whole or not at all:

In this example, if the value of $CVAL is NAME, then all the Uprocs between BEGINBLOCK and ENDBLOCK will be executed; if $CVAL's value is something else, none of those Uprocs will be executed.

There are three sets of block constructs, each of which has three parts:

BEGINBLOCK and ENDBLOCK open and close a simple block of Uprocs. The other two pairs are called "looping block constructs". WHILE and ENDWHILE open and close a block that loops as long as (WHILE) conditions stated at the beginning of the block are true. REPEAT and UNTIL open and close a block that loops until conditions stated at the end of the block are true.

In the diagram, you can see how similar the constructs are. Statement 1 opens the block of Uprocs (2), and statement 3 closes it. Statement 1 must always be paired with its corresponding statement 3; otherwise a compilation or execution error will occur. Statement 3 cannot be the object of an IF, THEN or ELSE Uproc; for example, "ELSE ENDBLOCK" is an illegal Uproc.

IMPORTANT: In formats, you may use these constructs in both UPROC and ENTRY-UPROC statements. However, a block construct must be entirely contained within the Uprocs or within the Entry-Uprocs of a single label-group. In other words, it cannot continue from one label-group to another, nor can it continue from the Entry-Uprocs into the Uprocs of a single label-group.

These constructs are very powerful. Besides their effectiveness in organizing code, they also make it possible to loop within a label-group. Otherwise, looping (with the LOOP statement) can be done only for the entire label-group.

The Uprocs within the block can be any other Uprocs, including other block constructs. However, block constructs inside of others must be completely nested therein; they cannot overlap.

The JUMP and GOTO Uprocs should not be used indiscriminately within block constructs, either. [Many programmers would contend that JUMP and GOTO have no place within block-construct programming at all.] Jumping out of a block construct can lead to possible logic problems with the $ELSE system flag, which controls THEN/ELSE processing, and is not generally recommended. Be aware that if you do use JUMP to leave a block, $ELSE will retain its value from within the block; it will not revert to the value it had when the block began execution, which it would do if you left the block normally.

Looping block constructs provide another way to leave from inside: the LEAVE Uproc. LEAVE causes execution to continue with the next Uproc outside of the looping block, just past the ENDWHILE or UNTIL Uproc. LEAVE is not available in BEGINBLOCK...ENDBLOCK constructs.

Here is an example demonstrating use of the LEAVE Uproc:

The idea behind these Uprocs is to place the first ten occurrences of the CROSSREF element into the first ten occurrences of the #XREF variable. However, when no value exists for an occurrence, the $GETUVAL function returns the value "No value", which in the next Uproc is treated as the signal to leave the block construct.

Another useful Uproc for looping block constructs is the ITERATE Uproc, which transfers execution to the last statement in the block, either ENDWHILE or UNTIL.

Block constructs may be nested to 31 deep in a label-group.

B.4.8.2  The JUMP and GOTO Uprocs

By default, of course, execution of a frame begins with the first label group and proceeds from one to the next until the end of the frame. Several statements and Uprocs are available to alter the standard flow, and are discussed in this and the next few sections of B.4.8.

Certainly one of the most commonly used is the JUMP Uproc. It tells SPIRES to stop executing the current label group immediately and begin executing the next label group or another named label group. Its syntax is:

where "label.name" is the name of the label group at which execution is to resume. If no label name is given, execution continues with the next label group, which is equivalent to 'CASE 1'.

The GOTO Uproc is the same as the JUMP Uproc, except that a label name is required:

Variables may not be used for "label.name" for either statement (cf. the CASE Uproc).

Here is an example using the JUMP Uproc. As is often the case with Uprocs, JUMP may be the object of an IF...THEN or ELSE Uproc:

If no value is retrieved for the RECEIVED element, the $DEFAULT flag is set and thus the other elements relating to the receiving are skipped. [See B.4.5.1, B.4.8.1.]

Once again it is important to remember the order in which statements in a label group are executed. Uprocs always precede PUTDATA statements, so in the following label group, if the JUMP is executed, the PUTDATA will not be:

The named label group may precede or follow the current label group. Of course, if it precedes the current one, beware of creating an infinite loop. Normally you would not name the current label in the JUMP Uproc; if you do need to execute the same label group again, you would probably use the LOOP statement. [See B.4.8.4.]

The CASE Uproc, discussed next, provides an alternate method for jumping to label groups. [See B.4.8.3.]

B.4.8.3  The CASE Uproc

The CASE Uproc has the same purpose as the JUMP Uproc. However, instead of providing a label name, you give an integer value "n", and execution will resume at the "nth" label group from the current one. Also unlike JUMP, a variable can be used as the value.

The syntax of the CASE Uproc is:

where "n" is an integer (positive, negative, or 0) or an integer variable. If "n" is positive, execution will jump to the "nth" label group from the current one. If "n" is negative, execution will jump to the "nth" label group before the current one. If "n" is zero, SPIRES will begin executing the current label group again -- unlike LOOP, however, this statement would cause the first element occurrence to be processed again, if none is specified in the GETELEM statement. (For 0 and negative values, beware of infinite loops.)

Like the JUMP Uproc, CASE may be the object of an IF...THEN or ELSE Uproc, and 'CASE 1' is equivalent to 'JUMP' with no label.

B.4.8.4  The LOOP Statement

The LOOP statement tells SPIRES to begin executing the current label group again. Basically it has two purposes: 1) to let you retrieve multiple occurrences of elements; and 2) to provide a general looping capability that includes "counter variables". The two purposes are discussed separately below.

The syntax of the LOOP statement is:

where "n" is an integer or an integer variable representing the maximum number of times SPIRES should restart the execution of the label group. That is, if "n" is 2, the label group will execute once and then twice again for the loop, for a total of three times.

An alternate form, most common for element processing, is:

This form tells SPIRES to continue looping until something happens within the label group to cause the label group processing to stop (see below).

Several system variables are commonly accessed during loop processing, including $LOOPCT, $ELOCC, $LASTOCC, $CURROCC, and $TRUEOCC. [See E.2.2.16, E.2.2.26, E.2.2.27.]

The LOOP Statement and Multiple Element Occurrences

When you want to retrieve all occurrences of an element, you generally use the LOOP statement. Here is a simple example:

The first occurrence of PHONE.NUMBER is placed in the default starting position, and then the looping occurs. The second time through the label group, SPIRES retrieves the second occurrence of the element, placing it in the default XSTART position (which is START = X,n; where "n" is the starting column for the first occurrence positioned), and so forth. The LOOP statement, when used in label groups having GETELEM statements, causes SPIRES to retrieve the next occurrence of the element each time a loop occurs; no specific reference to an element occurrence needs to be made. [See B.4.2.]

When SPIRES loops back to the start of the label group and the GETELEM retrieves no more occurrences, then the loop is broken. SPIRES then skips the rest of the label group, just as it does when there are no occurrences of an element at all; processing resumes at the next label group. That is the most common method for ending a loop.

When multiple occurrences of elements are being positioned in a frame, the START statement can affect the placement of the second through "nth" occurrences. As noted above, the default positioning for those occurrences is in row X (which is not necessarily *+1), starting in the same column given in the START statement.

The XSTART Statement in a Looping Label Group

The XSTART statement, following a LOOP statement, may be used to override the default starting position for the "extra" occurrences:

Subsequent occurrences of the COMMENTS element will be positioned beginning two rows from the end of the previous occurrence, starting in column 5. The XSTART statement has the same form and allows the same values as the START statement. [See B.4.6.1.] You can also use the Uprocs SET STARTROW and SET STARTCOL when you are working with multiple element occurrences. [See B.4.6.2.]

Note that the XSTART statement would not be used in a looping label group concatenating occurrences with SET BUILDSEP and SET BUILDEND Entry-Uprocs. [See B.4.5.7.]

Other Statements in a Looping Label Group

The other statements in a label group will be applicable to multiple element occurrences. The INSERT statement, for example, is applied to all occurrences. As a result, it is usually replaced by the TITLE and TSTART statements when only the first occurrence should have the inserted string. [See B.4.8.]

The DEFAULT statement interacts with LOOP in an interesting way.

If there are no occurrences of the COURSES1 element, the default value will be displayed once, and no looping will occur. However, if there are no occurrences of the COURSES2 element, the default value will be displayed six times, because of the value given for the LOOP statement. SPIRES realizes in the first situation that you want the default value to be used only once, rather than to create an infinite loop. On the other hand, since a specific number is given on the right, SPIRES keeps looping that many times.

Similarly, if one or more occurrences of COURSES1 are retrieved, the default value will never be displayed. But if less than six occurrences of COURSES2 are retrieved, the default value will be used to bring the number up to that total.

In other words, when a loop count is given and a default value is in effect, SPIRES will loop that many times, regardless of the number of element occurrences that actually exist, supplying the default value each time that the next occurrence does not exist. If you want to supply a loop count and a default value, but you do not want the default value used if there are any values at all, you should test for default processing in a Uproc, testing the value of the $DEFAULT flag.

LOOP vs. JUMP

The LOOP statement and the JUMP statement are very different in effect:

The label group on the right is practically useless. All Uprocs are executed before a PUTDATA statement, and a PUTDATA statement is executed before a LOOP statement. That means that the JUMP statement on the right occurs before any value is placed in the frame. Moreover, in the example on the right, the GETELEM statement would always retrieve the first occurrence of the element; only the LOOP statement tells SPIRES to retrieve the next element occurrence when no specific occurrence is requested. Since the first occurrence is retrieved over and over again, the GETELEM statement would never fail (unless there were no occurrences of CHILDREN in the first place) and hence the label group would become an "infinite loop", which SPIRES would stop executing after 32,767 loops.

The LOOP Statement as a Programming Tool

The LOOP statement can also be handy when you need a general looping capability for a label group. By checking the system variable $LOOPCT (for "loop count") you can keep track of how many loops have occurred:

In this example, the variable TOTAL is augmented each time through the loop with the next occurrence of the ADDEND variable. The particular occurrence added is controlled by $LOOPCT, beginning with occurrence number 0. In other words, the label group adds together the first twenty occurrences of the ADDEND variable.

Because the LOOP statement only restarts the label group it is in, it cannot be used when the loop must include several label groups. In such cases, you may use the JUMP or CASE statements, probably augmented with a user-declared variable to serve as a counter. [See B.4.8.2, B.4.8.3.] Alternatively, you could put the label groups into a separate indirect frame called from a label group with a LOOP statement. [See B.4.8.7.]

B.4.8.5  The SET STARTOCC and SET LOOP BACKWARDS Entry-Uprocs

In addition to controlling or converting an element's value, a label group can also control which occurrence of the element it retrieves, using an Entry-Uproc, which is executed at the beginning of a label group before a GETELEM or IND-STRUCTURE statement. [See B.4.5.6 for a general description of Entry-Uprocs; the IND-STRUCTURE statement, which retrieves occurrences of structures, is discussed later in this manual in chapter B.8.]

The next few paragraphs discuss two Entry-Uprocs, SET LOOP BACKWARDS and SET STARTOCC, which can be used (together or separately) to control which element occurrences a label group with a LOOP statement will retrieve.

The SET LOOP BACKWARDS Entry-Uproc

The SET LOOP BACKWARDS Entry-Uproc lets you retrieve element occurrences in reverse order from how they are stored in the record: last becomes first and first becomes last. The syntax of the Entry-Uproc is simplicity itself:

Thus, if a record contained eight occurrences of the NAME element, the label group below would first retrieve the eighth occurrence, then the seventh, then the sixth, and so on:

     LABEL = NAME;
       ENTRY-UPROC = SET LOOP BACKWARDS;
       GETELEM;
       PUTDATA;
       LOOP;

The SET STARTOCC Entry-Uproc

The SET STARTOCC Entry-Uproc lets you specify the starting occurrence that the GETELEM (or IND-STRUCTURE) statement in a label group should retrieve. The syntax is straightforward:

Here "n" is a number corresponding to the occurrence's actual position in the record, where "0" represents the element's or structure's first occurrence, "1" represents its second occurrence, and so on.

The label group containing a SET STARTOCC Entry-Uproc will almost certainly contain a LOOP statement as well; each iteration of the loop retrieves the next occurrence of the element beginning with the occurrence named in the Entry-Uproc. Thus the label group below would retrieve occurrences of the COURSES element beginning with the fourth occurrence of the element (assuming there was one). It would skip the first three occurrences altogether, and would loop until there were no more occurrences to retrieve:

     LABEL = COURSES;
       ENTRY-UPROC = SET STARTOCC = 3;
       GETELEM;
       PUTDATA;
       LOOP;

The two Entry-Uprocs can be used together to cause occurrence retrieval to begin at an "nth" occurrence and loop backwards from that point. Remember to include a DEFAULT statement [See B.4.5.1, B.4.7.4.] if there is a chance that a specified occurrence will be absent from the record; otherwise, the format will skip the rest of the label group without retrieving "subsequent" occurrences, even though those occurrences come first in the actual record:

     LABEL = COURSES;
       ENTRY-UPROC = SET LOOP BACKWARDS;
       ENTRY-UPROC = SET STARTOCC = 3;
       GETELEM;
       DEFAULT = '---';
       PUTDATA;
       LOOP = 3;

Some additional comments on SET LOOP BACKWARDS and SET STARTOCC:

B.4.8.5a  The SET FILTER and CLEAR FILTER Uprocs

The SET FILTER and CLEAR FILTER Uprocs (or Entry-Uprocs) let you set and clear filters as a format executes, in order to control which data is processed by the format. The syntax parallels that of the interactive SET FILTER and CLEAR FILTER commands:

Filters set with the Uproc act as overlay filters (even though the term OVERLAY is not used in the command). That is, they are additive to other filters that may be in effect for the type of operation the format is executing (e.g. DISPLAY). As with the SET FILTER OVERLAY command, the SEQUENCE, OCC, and IN LIMIT options of the SET FILTER command are not allowed in SET FILTER Uprocs.

The CLEAR FILTER FOR elem-name and CLEAR FILTERS Uprocs clear only the filters set in your format with SET FILTER Uprocs, and not other filters that may have already been in effect.

The SET FILTER and CLEAR FILTER Uprocs may only be used in data or indirect frames. There is one restriction to note: these Uprocs are not allowed if your format uses the FORMAT-OPTIONS = GENELEM or FORMAT-OPTIONS = GENVIRT statement. [See B.6.2.]

All filters established during execution of a format will be cleared when the format is exited, after each record is processed. In other words, you must make sure the filter you want is set for each record processed by the format. For example, don't set your filters in a startup or initial frame.

If you are setting merge filters in an input format, note that you must also execute the BUILD RECORD Uproc in order for the merge filter to work. That is, if the merge filter is set in the format, the record must be built while still under format control. In the absence of BUILD RECORD, the format is exited, the filter set in the format is cleared, and so when the record is built the filter will not be in effect. [See C.3.4.4.]

If your WHERE clause in the SET FILTER Uproc contains a variable, the variable is evaluated at the time SET FILTER is executed. The element's Inprocs are also executed at that time. If there is a mismatch between the variable value and the element's processing rules, you may receive processing rule errors.

Here is a very simple example of how you might use SET FILTER and CLEAR FILTER Uprocs. The effect here is that when these output frames are executed, only DONATION structures containing DATE values greater than 1985 will be displayed.

B.4.8.6  Subroutines within Frames: The XEQ PROC and RETURN Uprocs

In formats, SPIRES lets you handle subroutines basically in two ways: with the XEQ PROC Uproc, in which the called subroutine is within the current frame definition, and with the IND-FRAME (for INDirect FRAME) statement, in which the called subroutine is a different frame altogether. Generally, you use the IND-FRAME procedure when the same subroutine needs to be called from several different frames; if the subroutine will be called only from one frame, the XEQ PROC Uproc is typically used. This section will discuss XEQ PROC; the IND-FRAME statement is discussed in the next section and later in Part B with regard to structure processing. [See B.4.7.6, B.8.2.]

The syntax of the XEQ PROC Uproc is:

where "label.name" is the name of a label group in the same frame. SPIRES will jump to the named label group and begin executing it. If "label.name" is not specified, SPIRES will jump to the next label group in the frame.

At the end of the label group or label groups that comprise the "proc" (subroutine), you place the RETURN Uproc:

When SPIRES encounters this statement, it returns to the next label group after the label group containing the XEQ PROC Uproc that caused the subroutine to be executed. (Note then that it does not return to the statements after the XEQ PROC Uproc in that label group.) There is one exception to that rule: when the XEQ PROC is in the NULL or ATTN clause of an ASK Uproc, SPIRES will return to the ASK Uproc from the subroutine and execute the "ask" again. [See B.4.7.9.]

The RETURN Uproc may be used in other situations besides returning from a subroutine. [See B.4.7.7.]

Here is an example of a label group that calls a subroutine:

Note that the PUTDATA is not executed in MONTHLY.QUANTITY if the XEQ PROC is executed. Control is "returned" to the NEXT.ONE label group when the RETURN statement is encountered in the SMALL.ORDER proc.

Within the subroutine itself, if you do not code a RETURN statement, SPIRES will continue executing one label group after another until it reaches the end of the frame (or some branching statement), at which point the XEQ PROC will be cancelled and execution of the current frame will stop.

By convention, subroutines are placed at the end of a frame. To prevent SPIRES from executing them when it gets to the end of the frame, people generally code a RETURN Uproc in the last label group before the subroutines begin:

Here, the COMMENTS label group is the last "mainstream" label group of the frame. The EXIT label group tells SPIRES to stop processing the frame and "return" one level, halting execution of this particular frame. [See B.4.7.7.] The next label group, the first of the subroutines, does not execute unless an XEQ PROC Uproc (or some other branching Uproc) elsewhere invokes it.

The XEQ PROC Uproc is thus involved with nested procedures, unlike JUMP, which involves simple branching. A subroutine called by XEQ PROC may even call another subroutine, etc., to a maximum depth of eight subroutines.

B.4.8.7  Subroutines that are Frames: the IND-FRAME Statement

As the previous section suggested, you may want a subroutine that can be called from several frames. Since the XEQ PROC Uproc can only specify a label group within the current frame, it is not suitable for this situation. Instead, you must call an "indirect frame". Note that indirect frames are more often used when element structures are being formatted; they are discussed in more detail later. [See B.8.]

Indirect frames are a general tool in formats, useful in a variety of situations. Though the discussion in this section specifically concerns their use in output formats as subroutines, most of the discussion applies to their use in general, with cross references provided when that is not the case. When an indirect frame is invoked from a "calling frame", format execution control is transferred to the indirect frame. Execution control returns to the calling frame when the indirect frame finishes executing, either because the end of the frame has been reached, or because a RETURN Uproc is encountered.

The indirect frame may retrieve record elements, but not elements in structures unless the appropriate IND-STRUCTURE statement is coded. [See B.8.2.]

An indirect frame is invoked by the IND-FRAME statement:

where "frame.name" is the name of another frame defined in the format definition. Thus "frame.name" must follow the rules for frame names given for the FRAME-ID statement. [See B.3.1.]

The IND-FRAME statement usually appears right after the LABEL statement when structure processing is not involved. Neither GETELEM nor VALUE statements (except in subgoal processing) are allowed in the same label group as an IND-FRAME statement.

When the GO.TO.SUB1 label group begins executing, control is given immediately to the frame SUB1.

Besides coding the IND-FRAME statement, you must also declare the frame as an indirect one in the format declaration section. [See B.5.2.]

To some extent, you can think of an IND-FRAME statement as replacing a GETELEM or VALUE statement in a label group. (Note however that in subgoal processing [See B.12.] both a VALUE and an IND-FRAME statement will appear in the calling label group.)

Generally speaking, when an indirect frame is used for a subroutine, no frame dimensions are coded in it, indicating that the frame dimensions of the calling frame will be in effect for any data placement done by the indirect frame. [See B.8.1.] Other frame-identification statements that are often coded in the indirect frame definition are DIRECTION and USAGE, usually with the same values as those of the calling frame. [See B.3.2, B.3.4, D.1.2.]

When execution is transferred to an indirect frame, it executes as any other frame does, one label group after another, unless some type of branching occurs. The indirect frame may itself call subroutines (using either XEQ PROC Uprocs or IND-FRAME statements). As you continue to "descend" into nested subroutines, remember that the RETURN Uproc will return you one level each time it is encountered. You will also return from the indirect frame to the calling frame automatically when the end of the indirect frame is encountered.

The indirect frame is allowed to call itself, but care must be taken to avoid "infinite nesting"; nesting is limited to about ten levels, beyond which a serious error occurs. (You may be able to descend deeper than ten levels in some circumstances.)

The calling label group may have other statements in it, for SPIRES returns to the calling label group once it has finished executing the indirect frame. A frequent companion of the IND-FRAME statement is the LOOP statement, which causes the indirect frame to be executed again. [See B.4.7.4.] Uproc statements may also appear; note that they are executed each time that SPIRES returns from an indirect frame before any looping occurs:

The above example contrasts the manner in which the two types of subroutines handle returns. SPIRES returns to the next statement in the calling label group when it returns from the indirect frame, but it will "return" to the next label group when it finishes executing the subroutine invoked by the XEQ PROC Uproc. The XEQ PROC Uproc here in effect breaks the loop.

B.4.8.8  Leaving a Frame: the RETURN, ABORT, STOPRUN and BACKOUT Uprocs

By default, when SPIRES is through executing label groups in a frame, execution of that frame stops. SPIRES may then continue to other frames of the format, if appropriate, or stop format processing completely, returning you to command level, if all appropriate frames have been executed. (Exception: For indirect frames invoked by calling frames, SPIRES will return to the calling frame when finished with the indirect one.)

The formats language provides other means of escape from a frame besides the default method. Perhaps the most common, and least dramatic, has already been introduced:

When the RETURN Uproc is encountered and SPIRES is not in the middle of an XEQ PROC "proc" within the executing frame, no further processing of that frame occurs. [See B.4.8.6 for its use with the XEQ PROC Uproc.] If it is an indirect frame, control returns to the calling frame. If not, the next appropriate frame as declared in the format declaration section, if any, begins executing. [See B.5.2.]

Two other available Uprocs will stop format processing completely.

The ABORT Uproc causes SPIRES to stop format processing for the current record; no more frames will be executed for that record. However, if multiple records are being processed (e.g., you have issued a TYPE command), SPIRES will begin format processing of the next record.

The STOPRUN Uproc is similar to the ABORT Uproc, except that when multiple records are being processed, a STOPRUN will also stop SPIRES from processing any further records for the issued command. (In a report, however, STOPRUN will execute any ending frames that are present, in order to provide totals up to the point where execution stopped.) [See B.10.6.]

After either an ABORT Uproc or a STOPRUN Uproc, the system flag variable $ABORT is set. [See E.2.1.11.]

For either Uproc, the QUIET option prevents SPIRES from displaying the error message normally associated with an ABORT or STOPRUN. The NOERROR option not only suppresses the error messages (like QUIET) but also does not cause the setting of $NO or $SNUM. (In the absence of the NOERROR option, ABORT and STOPRUN turn on the $NO flag and set $SNUM to 847 and 848, respectively.)

The RETURN, ABORT and STOPRUN Uprocs often appear in THEN clauses of IF... THEN Uprocs, where some critical value is being tested:

If the user's account number is not the same as the account number in the record, a few messages are displayed and no further format processing occurs for the issued command.

Another Uproc, BACKOUT, causes an immediate exit from the current format processing, telling SPIRES to behave as if a frame of the type required for the issued command did not exist in the set format. For example, if the BACKOUT Uproc is encountered during format processing under a DISPLAY command, the record would be displayed in the standard SPIRES format. Note though that if the BACKOUT Uproc occurs after the buffer has already been flushed (e.g., in line-by-line processing), the flushed data has already been output and the BACKOUT Uproc will not affect it; the current contents of the buffer will be discarded, however.

B.4.8.9  Passing Commands to WYLBUR from a Format

You cannot "pause" during the execution of a frame to return to command mode -- once you return to command mode, the current execution of the format has stopped. It is possible, however, to pass a command to WYLBUR from inside a format, using the $ISSUECMD function. For example,

The EVAL Uproc causes the $ISSUECMD function to be evaluated, which passes the SEND command to WYLBUR to execute immediately. [See B.9.3.3.] The format pauses while WYLBUR executes the command, but as soon as the WYLBUR command is executed, the format resumes execution.

The syntax of $ISSUECMD is:

where "command" is a string expression. The function will return a "1" ($TRUE) if it succeeds, "0" ($FALSE) if it fails.

The command specified in the $ISSUECMD function must not be a SPIRES command; only ORVYL and WYLBUR commands are allowed. Be aware that some commands may affect your session such that the format cannot continue executing: some examples are CALL SPIRES and LOGOFF.

B.4.8.10  Variable Manipulation: The LET and SET Uprocs

Although variable handling will be covered in detail in a later chapter [See B.9.] it is such an important part of the procedural language that it is worth introducing here. This brief discussion specifically introduces the LET and SET Uprocs, which can be used to assign values to system and user variables.

Variables are used for a number of different programming reasons. In a SPIRES format, some of the likely reasons include:

Examples of such uses will be shown throughout the manual. [See B.10.11, for instance.]

Values may often be assigned to system variables, identified by the dollar sign that begins their names (e.g., $CVAL, $PROMPT). In a format, a SET Uproc is used for this purpose:

Not all system variables may be set. A list of those that may be set from a format appears elsewhere. [See B.4.5.5.]

You may also assign values to user variables with the LET Uproc:

In that example, the current value of the system variable $CVAL is concatenated to the current value of the user variable #VALUE to make a new #VALUE. [See B.9.3 to learn how user variables are declared and allocated in a format.]

In general, the LET and SET statements follow the same rules as the LET and SET commands in command mode. For example, the variable name right after the LET or SET verb does not have a dollar sign or pound sign. On the other hand, two differences are: 1) in formats, a SET Uproc allows expressions for the value; and 2) the LET and SET Uprocs are element values, which must follow standard format entry rules, meaning, for example, that they must end with semicolons. [See B.2.7 for data entry rules.]

B.4.8.11  Terminal Input: The ASK Uproc

In some circumstances the format design may require the user at the terminal to provide information during format execution. The ASK Uproc, which requests and handles user input from a format, allows the user to interact with the format as it executes.

When the ASK Uproc is executed, SPIRES sends a prompt to the terminal, stopping format execution until a response is sent back. The value given as the response is assigned to the system variable $ASK, unless the response was a null response (usually just a carriage return) or an attention (ATTN/BREAK key), in which cases the format designer may decide how to handle the response. The value in the $ASK variable may be extracted or used through other Uprocs, as desired.

The syntax of the ASK Uproc is:

If both EXACT and NOTRIM are specified, only EXACT will be in effect (see below). The UPPER option indicates that the response should be converted to uppercase when it is assigned to the $ASK variable (see examples below); otherwise, the case of the response remains unchanged. The value "string" is a string of text for SPIRES to use as the prompt to the user; "uproc" is one of the following Uprocs:

The Uproc given in the NULL clause will be executed if a null response (a simple carriage return, or all blanks and a carriage return) is given to the prompt. The Uproc given in the ATTN clause will be executed if the user presses the ATTN/BREAK key in response to the prompt. Though each of those Uprocs is described elsewhere in this manual, several points concerning how some of them may be used are covered in the examples below.

The NOECHO option prevents SPIRES from echoing the response back to the terminal as it is typed. What the user types thus does not show on the screen.

Here is the simplest form of the ASK Uproc:

If no PROMPT clause is given in the ASK Uproc, the current value of the system variable $PROMPT will be used for the prompt. (That variable may be set using the SET PROMPT Uproc. Unlike the PROMPT clause, the SET PROMPT Uproc can take a string expression, as opposed to merely a string, for its value.) Regardless of what the prompt is, it will always be preceded by a colon (:) and followed by a blank, unless the EXACT option is added, which tells SPIRES to display the prompt without the colon and trailing blank.

Here is what might appear on the terminal screen if a user displays a record through a format that has the above Uproc:

The colon is the prompt for input from the user. (The $PROMPT variable had apparently not been set.)

There are four possible types of responses by the user and counter-responses from SPIRES for that Uproc:

The third and fourth situations are the default reactions when SPIRES receives a null or attention response and no NULL or ATTN clauses are coded. Coding one of them will cause the appropriate Uproc to be executed in that situation instead:

Here, if SPIRES receives an attention response, no further format processing is done. If SPIRES receives a null response to the prompt, execution control will go to the EXPLAIN.DATE label group, presumably for more information about the prompt to be displayed to the user. (Remember that a RETURN from an XEQ PROC that is part of a NULL or ATTN clause will return SPIRES to the ASK prompt for reprompting. [See B.4.7.5.] This is the only situation in which RETURN will not return to the next label group.) In both cases, $ASK will be set to null.

Note that if JUMP or XEQ PROC are not followed by a label name, the next label group in the frame is assumed.

The PROMPT clause on the ASK Uproc does not set or clear the value of the $PROMPT variable, which exists outside of formats. Hence, if you issue an ASK Uproc without setting $PROMPT or without using a PROMPT clause, SPIRES may prompt the user with some prompt left over from an earlier format or protocol. You may want to set $PROMPT to a null string first. [See B.5.2 for an example.]

Be aware that the maximum length for a useful prompt is 158 characters, not including the colon or blank that SPIRES will add (160 if the EXACT option is coded). A prompt longer than that will automatically cause an ATTN response to the ASK Uproc.

Technical note: The total length of the prompt and the user's response may not exceed 168 characters. The allowed length for the user's response can be determined by subtracting the length of the prompt (including the colon or blank that SPIRES would add, if present) from 168. If the user's response exceeds that length, it is truncated. An exception: if the prompt's length is zero, the user's response may not exceed 167 characters.

(*) A Comparison of the $ASK and $PARM System Variables

Two system variables, $ASK and $PARM, are commonly used to provide information to the format. The $ASK value is normally set by the user's response to an ASK Uproc; the $PARM value is set by a SET FORMAT or SELECT command. [See B.7.1, E.2.3.9.] The values of both variables are accessible by the format.

The $PARM variable is generally used to provide information about the format. For instance, consider this sample command:

The string of element names following "$prompt" is the value assigned to $PARM. In this case, it names the elements to be prompted for and displayed by the format. The value of $PARM is examined by the startup frame of the $PROMPT format when that command is issued. [See D.3.] (Because the $PARM variable can change between the time the SET FORMAT command is issued and the time the format is used, i.e., via a SET GLOBAL FORMAT command, you should retrieve the $PARM value in a startup frame.) Since the $PARM value is set when a SET FORMAT command is issued, it is seldom used to provide data to the format on a record-by-record basis.

On the other hand, $ASK is often used for that purpose:

This label group might appear in an output frame that displays part of a record and then asks whether the user needs to see more of it. The label group would be executed for each record displayed.

Of course, an important difference in the usage of $ASK and $PARM that the above examples demonstrate is in the ways they are set. One, $PARM, is set because the user voluntarily supplies it; the other is set because the format prompts the user for it. That difference is markedly shown by the example below, which might be part of a startup frame:

If $PARM is given a value in the SET FORMAT command, the user variable #USERPARM is assigned its value. If it is not given a value in the SET FORMAT command, the format prompts for a value, which SPIRES assigns to $ASK, and then the format assigns the value of $ASK to #USERPARM. In other words, when using this format, you can supply the value of #USERPARM on your own via $PARM or through prompting via $ASK.

Both $ASK and $PARM can also be set directly, using the SET ASK and SET PARM Uprocs and commands.

B.4.8.12  Terminal Output: The * (Star) Uproc

Sometimes you may want to send data to the terminal screen that is informational about the format execution, rather than part of the data to be formatted. For example, in debugging situations, you might want to look at the value of a variable as the format executes. The star Uproc can be used for those situations:

where "string-expression" may consist of character strings (surrounded by apostrophes), user or system variables that can be converted to strings, or some combination thereof.

For example:

Here is what the terminal session might look like:

The formatted record is placed in your active file but the message from the * Uproc is sent to your terminal screen. Note that the appropriate value for the system variable $KEY was substituted by SPIRES before the message was displayed; $KEY represents the key of the current record. [EXPLAIN $KEY VARIABLE.] for more details on the $KEY system variable. You do not need to precede the * Uproc with a slash (/) to force evaluation of the expression, as you would with the * command; in fact, the "/" is not allowed in formats.

Note that the terminal message may appear in the middle of the formatted record if you display the record at your terminal. [See B.1.2, B.3.3.]

Note that if you want to direct custom messages to the trace log (with the SET TLOG command) you should use the $ISSUEMSG function rather than the * Uproc, as in this example:

For most tracing situations, $ISSUEMSG is preferable to using the "*" command, since it will only appear on the terminal screen and never as part of the trace log. [EXPLAIN SET TLOG COMMAND.] for information about trace logs. [EXPLAIN $ISSUEMSG FUNCTION.] for an explanation of the function.

The * command and the * Uproc in formats are similar but they are not identical. When a * command (without an "IN area" prefix) is executed by SPIRES, the asterisk will be displayed when the string is sent to the terminal; the asterisk is not displayed when a * Uproc is executed. Also, the * command may be prefixed by "IN area":

From a format, the * Uproc's expression will always be displayed at the terminal. Note too that string expressions do not have to be placed in apostrophes in the command; in the Uproc they almost always do.

B.4.8.13  (*) Terminal Input/Output and Buffer Processing

"Timing problems" with the terminal input and output Uprocs ASK and * sometimes arise because of the way SPIRES handles the frame buffer. These problems are often solved with the FLUSH Uproc.

For example, consider the following frame definition:

If a DISPLAY command is issued, the terminal display from that frame would be:

The reason that these appear out of order from what one would expect is that SPIRES does not "flush" the buffer, i.e., write the buffer to the terminal, until the format attempts to place data onto the next line. Hence, label group THREE executes before the buffer containing the data placed by label group TWO is flushed because another label group may be placing more data onto that same line. But as soon as another label group tries to place data into a later line (as FOUR does), the buffer is flushed.

To circumvent such timing problems, the FLUSH Uproc is available:

This Uproc causes the current contents of the buffer to be flushed immediately. So if you add it to label group THREE:

then the buffer will be flushed to the terminal before the ASK Uproc is executed when a DISPLAY command has been issued:

The FLUSH Uproc is particularly important in report formats. [See B.10.3.6.]

B.4.8.14  Comments as Uprocs: the "-" (Hyphen) Uproc

Comments are allowed as Uprocs, written similarly to the form they take in protocols:

They are often used within groups of Uprocs to comment on the processing being done, step by step. Like COMMENT elements in a format, these are ignored by the compiler. Keep in mind the rules for special characters if your text includes quotation marks or semicolons. [See B.2.7.] Do not confuse the "-" Uproc with the "-" throwaway element. The "-" Uproc is not discarded from the stored record as the throwaway element is. [See B.2.2.]

Unlike COMMENT elements, which are gathered together at the beginning of a label group, the "-" Uproc can be interspersed throughout a series of Uprocs. [See B.4.9.]

B.4.8.15  (*) The SITE Statement

An additional statement within label groups, the SITE statement, makes it more convenient to transport a SPIRES format from one site to another, by ensuring that only label groups appropriate to a particular site will be compiled and executed there.

The syntax of the SITE statement is as follows:

Possible values for "sitecode" are MTS, CMS, TSO, and STS -- the last value stands for Stanford and other ORVYL sites.

When the SITE statement is included in a label group, that label group will only execute at the site(s) named:

At sites other than STS a label group with the above SITE value would be ignored by SPIRES, and would not even become part of the compiled format. This means that the same format could include more than one label group with the same name, without invoking an error message, assuming each of these label groups pointed to a different site:

In a way, the SITE statement acts like a procedural statement in that it causes its label group to be executed only conditionally, depending on the "sitecode" value.

Multiple Sitecodes

You can include more than one sitecode in the SITE statement, and can precede the sitecode(s) with a ~ character to indicate that the label group should NOT be compiled and executed at the named sites:

Site-specific Variables

Note that variables declared either in the VGROUPS subfile or in the VGROUPS section of a format definition can also be filtered by site. [See B.9.3.1 and the manual "SPIRES Protocols".]

B.4.9  The COMMENTS (COM) Statement in Label Groups

As you probably expect by now, each of your label groups can have one or more occurrences of the COMMENTS statement to document the function or purpose of the label group, or mention any special features it contains. Below is an example of such a comment.

To place more detailed comments within a group of Uprocs in a label group, you can use the "-" Uproc. [See B.4.8.14.]

B.5  The Format Declaration Section of the Format Definition

So far, your format definition consists of two sections: the Identification section and the Frame Definitions section, consisting of one or more frame definitions. The final section required in a format definition is the Format Declaration section. Here you specify one or more format names to be used in a SET FORMAT command, and then identify the frames that can be executed under that format name. In other words, the Format Declaration section tells the compiler how to "package" a set of logically related frames that are used together. Hence you can consider this section as the place where a format is actually defined. (The Frame Definitions section defines the specific building blocks of a format, but unless those frames are specified for a specific format in the Format Declaration section, the frames are not used.) You may define multiple formats in a single format definition; a single frame may be used in multiple formats.

As part of a FORMATS subfile goal record, the Format Declaration section consists of one or more occurrences of the FORMAT-DEC structure, each occurrence describing a single format:

Each occurrence of the structure begins with the FORMAT-NAME statement, the key of the structure. Several statements regarding sort space and variables may follow the FORMAT-NAME. [See B.10.8 for information on the SORT-SPACE and ALLOCATE statements.] Next come the "frame declarations", where the frames that comprise the particular format are listed. They may be followed by an ACCOUNTS statement, which lets you restrict access to the named format.

You may have up to 16 occurrences of the FORMAT-DEC structure in a format definition, meaning that you can define up to 16 different formats in a single definition.

B.5.1  Naming the Format: The FORMAT-NAME Statement

The FORMAT-NAME statement, which begins the Format Declaration section, specifies the name of the format being defined here. Its value is used in the SET FORMAT command to identify the format to be set. Its syntax is:

where "format.name" is a name consisting of one to sixteen alphanumeric characters. The special characters "." (period), "_" (underscore) and "-" (hyphen) are also allowed in the name. Although the format name may contain internal blanks (unless it is a general file format or a global format), choosing names without them is preferable. [See B.7.1, B.14, D.2.5.]

Here are some sample FORMAT-NAME statements:

The FORMAT-NAME must be unique among all formats for the record-type. When you try to compile the format definition record and SPIRES finds another format already existing with that name, you will be asked if it is okay to replace it with the new one, if the old one is yours. If it is not yours, SPIRES will tell you to change the FORMAT-NAME statement. [See B.6.2.]

B.5.2  Identifying the Frames: the FRAME-NAME and FRAME-TYPE Statements

The FRAME-NAME and FRAME-TYPE statements identify the frames that collectively constitute the format named by the FORMAT-NAME statement. Specifically, the FRAME-NAME statement names a frame defined in the Frame Definitions section of the format definition, and the FRAME-TYPE statement specifies the circumstances under which the frame is to be executed. Up to 100 frames can be specified in a single format, i.e., per FORMAT-NAME.

The syntax of the FRAME-NAME statement is:

where "frame.name" is the name given in a FRAME-ID statement earlier in the format definition.

The syntax of the FRAME-TYPE statement is similarly simple:

where "frame.type" is either:

There are several other frame types as well, most of which are used in report processing: INITIAL, ENDING, HEADER, FOOTER and SUMMARY. [See B.10.] Another type is used under partial record processing: STRUCTURE. [See B.15.] The XEQ type is most commonly used in global formats and full screen processing. [See D.2.]

Another type, STARTUP, is executed as soon as the SET FORMAT command is issued. Because no records are being processed at that point, a startup frame may not contain data base references (such as GETELEM statements) -- it is generally used to set or test various system and user variables. [See D.3.]

When coding FRAME-NAME and FRAME-TYPE statements, you should remember two rules:

Some Uprocs are also allowed for each frame in the "frame declaration":

Any Uprocs here will be executed before the label groups of the frame to which these apply. Also allowed at this point are COMMENTS statements, usually concerning the particular frame. [See B.2.2.]

Here is an example group of FRAME-NAME and FRAME-TYPE statements for a format of four frames: the two indirect frames CHECK1 and CHECK2 and the two data frames PART1 and PART2.

Note that if CHECK2 were called from CHECK1, it would have to be declared before CHECK1 here. Note also that no startup frame was actually defined; however, the SET PROMPT Uproc will be executed when the format is set. In other words, at the time when the startup frame would be executed, the Uprocs associated with that frame are executed, even though no frame specifically exists. This technique is only available for startup, initial, group-start, group-end, and summary frames. [See B.10.5, D.3.]

(*) Arranging the Frames in Very Large Formats

In formats containing many thousands of lines of code, the compiled code of the last compiled frames is slightly less efficient than the code of earlier ones. In other words, you may see some efficiency improvement if you move more frequently used frames higher in the list of declared frames, and move the less frequently used ones to the end.

For example, in a large report format that has numerous data frames, an initial frame, an ending frame, etc., you might want to declare the data frames first (after any indirect frames they call, of course), since these will be executed many times during the report -- presumably once for each record. The initial and ending frames, since they are executed only once at the beginning and end of the report respectively, would be declared at the end. This way, the most frequently executed frames are declared first, and their code is compiled more efficiently.

Remember, this difference in compiled-code efficiency only affects large formats. None of the examples shown in this manual, for example, would be affected. [See B.10.11.]

B.5.3  Limiting Account Access: The ACCOUNTS (ACCT) Statement

By default, anyone with access to a set of goal records (e.g., through the SELECT command) can set any formats available for those goal records. However, you can limit the use of a format to accounts specified in the optional ACCOUNTS statement, which follows the frame declarations. Its syntax is like the ACCOUNTS statement in a file definition:

where "accounts" consists of one or more account forms, separated by commas. The commonly used account forms are:

      FORM            DESCRIPTION                       EXAMPLE
     gg.uuu     a specific account                       GQ.DOC
     gg....     all accounts in a specific group         GQ....
     g.....     all accounts in a specific community     G.....
     PUBLIC     all accounts                             PUBLIC

(In a sense, the default, i.e., when no ACCOUNTS statement is coded, is PUBLIC, so it is seldom used here.)

When a user tries to set a format to which access has not been allowed, the message "-Privileged command" is issued.

The account specified in the ID statement will always have access to the format, whether or not it is listed in the ACCOUNTS statement. Note that the file owner does not automatically have such access. In summary, if an ACCOUNTS statement is present, only the format "owner" and the accounts listed may use the format.

Although a format may not be available to all users because of its ACCOUNTS statement, its name will still appear in the list of formats displayed by the SHOW FORMATS command for all users.

A Sample Format Declaration Section

Here is a sample Format Declaration section, perhaps slightly more complex than most:

Two formats are defined here. The first, SHORT, consists of the single data frame START.OUTPUT, which calls the indirect frame SUBSTR1. The second, LONG, can only be set by account GQ.DOC. It consists of the two data frames START.OUTPUT and END.OUTPUT, which are executed in that order. Each data frame may call at least one indirect frame: SUBSTR1 from START.OUTPUT and SUBSTR1 and SUBSTR2 from END.OUTPUT.

B.6  Compiling Your Format Definition

At this point you may have a complete format definition, having an Identification section, a Frame Definitions section and a Format Declaration section. On the other hand, you may still need other formats features, such as a way to handle structures or to create and use your own variables. Those and other slightly more advanced topics will be covered later in this part of the manual.

However, at some point your format definition will be completely written. The next steps are to add it to the SPIRES public subfile FORMATS, compile it, and test it. This process is covered in detail in this chapter. Regardless of the complexity of your format, you will follow the procedure described here to make it usable.

This chapter also discusses how to recompile a format definition when you want to make changes to it and how to get rid of it completely ("zap" it) when you are done with it.

B.6.1  The FORMATS Subfile

Before it can be compiled, your format definition must be added to the public subfile FORMATS. As suggested by all the examples in this manual, you should collect the format definition in your WYLBUR active file using the standard SPIRES format, i.e.,

or, as it has been presented here:

To add the definition in your active file to the FORMATS subfile, you issue the SPIRES commands SELECT FORMATS and ADD:

Just as with any other subfile, SPIRES will check your record against the information in the file definition for the FORMATS subfile, making sure that required elements such as the key ID are supplied, that quotation marks around statement values are correctly matched, and so forth. If no errors are detected, the prompt will be returned with no error messages, as shown in the example above. However, if an error in the record is detected, an error message will be displayed; use the EXPLAIN command to find out more information about the specific error if necessary.

A typical error detected at this point is shown below in this portion of a format definition:

Here is what SPIRES' response might look like if you tried to add that record:

This common mistake, omitting the "UPROC =" in front of the Uproc itself, must be corrected using WYLBUR text editing commands before you issue the ADD command to try again.

If you want a printed listing of your format definition, a nicely formatted version can be obtained with the PERFORM PRINT command:

You must first select the FORMATS subfile and then issue the PERFORM PRINT command, followed by the ID of the format definition desired. You may only print copies of your own format definitions, of course. The format definition will be printed on a high-speed printer, with each section of the definition (each frame definition, each vgroup definition, each format declaration section, etc.) printed on a separate page. For more information about the PERFORM PRINT command, issue the command EXPLAIN PERFORM PRINT.

Determining the ID of a Format

There are several ways to track down a format definition when you have forgotten its ID value.

The easiest approach is to set the format and then issue the SHOW FRAMES command. The SHOW FRAMES display includes the format ID for the format that is currently set. [See D.1.1.2.] (When the format is set, the format ID is also contained in the $FORMATID system variable, or in $GLOFORMATID if a global format.)

The file owner can find all the compiled formats that are defined for a file by issuing the PERFORM FORMAT LIST command:

If the "filename" option is omitted, the display will show the formats for the file containing the selected subfile. Then if no subfile is selected, you will be prompted for the "filename". The IN ACTIVE prefix may be added to the command so that the display will be placed in your active file.

For more information about this command, EXPLAIN PERFORM FORMAT LIST.

(*) Searching the FORMATS Subfile

Instead of using SHOW FRAMES or PERFORM FORMAT LIST, you may use the FORMAT subfile's indexes to help find the ID of your format definitions, if the name of the file to which the format applies is known:

You could also use the sub-index or qualifier if you also knew the value of the RECORD-NAME or FORMAT-NAME statements respectively. For example,

If just seeing the ID of the format definition record would refresh your memory, you can use the SHOW KEYS command under Global FOR -- only the IDs of your own format definitions will be shown:

Of course, you could alternatively issue a SET ELEMENTS command such as SET ELEMENTS ID FILE FORMAT-NAME and then DISPLAY ALL under Global FOR, allowing you to see more information about each of your format definitions. (Though not necessary, the SET SCAN PREFIX account helps SPIRES find your records more efficiently; omitting it will not allow you to see format definitions belonging to other accounts, however.)

Finding Formats that Reference an Element

Occasionally you may need to find the formats for a file that reference a particular element. Since elements may be referred to in GETELEM, PUTELEM, REMELEM, IND-STRUCTURE and LABEL (when one of the other statements appears in the label group but with no value), not to mention within Uprocs, Inprocs and Outprocs, searching for the appropriate formats can be challenging.

A virtual element called ELEM-USE in the LABEL-GROUP structure of a FORMATS record contains any element named in the GETELEM, PUTELEM, REMELEM, IND-STRUCTURE and LABEL statements. [There's not much that can be done to help you find elements named elsewhere, except to look through the format definitions with text-editor commands.] This element only contains legitimate element-name values; other types of values that may appear in those statements, such as "@n" (referring to an element by number) or "#variable", do not appear. In addition, any occurrence number following an element name is stripped off.

Here's how you might use ELEM-USE. Suppose you need to find all the formats for a file that refer to the TITLE-ARTICLE element. You might try this way:

Besides the other limitations already noted, be aware that the value in ELEM-USE is the name of the element as it appears in the label group, which could be an alias rather than the primary mnemonic.

B.6.2  Compiling the Format

To compile your format, select the FORMATS subfile and issue the COMPILE command:

where "idname", with or without the account prefix, is the value given in the ID statement. [See B.2.1.]

The STATISTICS option displays data about the size of various internal tables created by SPIRES as it creates the compiled code for the format. This option, seldom used unless the format is very large (say, over 1000 lines), is described in detail at the end of this section. If you use it, you may also use the IN ACTIVE option to direct the statistics to your active file rather than your terminal.

Note: Whether or not you include the STATISTICS option, SPIRES will by default warn you if any table is over 90 percent full. You can suppress those warning messages by appending the NOWARN option to your COMPILE command.

Compiling With Label Group Names

The LABELS option tells SPIRES to store label group names in the compiled format record, for use in SET FTRACE output. If the format is compiled without the LABELS option, SET FTRACE will identify label groups by positional numbers rather than by names.

Alternatively, you can include a FORMAT-OPTIONS = LABELS; statement in your format definition to specify that the compiled format should always include the label group names. In this case, you don't need to include the LABELS option every time you issue the COMPILE or RECOMPILE command.

Note that the FORMAT-OPTIONS statement also takes values other than LABELS, to control how SPIRES stores element references in the compiled format. See the explanation at the end of this section.

The informational messages indicate that the format whose FORMAT-NAME is DOCUMENT.LIST was compiled successfully.

What Happens During Compilation?

When SPIRES compiles a format definition, it first examines the first format defined in the Format Declaration section. There, where each frame to be used is identified, SPIRES finds the appropriate frame names, gets each frame definition from the Frame Definitions section in turn, and compiles it.

Compiling a frame involves two simultaneous processes: 1) verifying that the statements in the frame are syntactically and factually correct (e.g., whether the named file and record-type exists, whether the Uprocs specified are allowed in a given situation, etc.); and 2) converting the high-level formats language into a lower-level SPIRES-internal language. The result of the conversion is called the "compiled characteristics" of the frame. Indirect frames must be compiled before the frames that call them, which is why the indirect ones are declared before their calling frames in the Frame Definitions section.

SPIRES will compile each format defined in the Format Declaration section individually, sending the "Compiled format:" message back to the terminal for each. When all formats are compiled, the message "Format definition compiled" will be displayed.

When all the frames for all the formats are compiled, SPIRES places the compiled characteristics for each format in a public subfile called FORCHAR (for FORmat CHARacteristics). Then when you set a format, SPIRES retrieves the format characteristics from that subfile, reading them into the main memory. You yourself will probably not ever use the FORCHAR subfile directly. [The key of the FORCHAR record includes the names of the file and record-type as well as the format name, which prevents format definitions for the same record-type from having the same format name.]

If Errors are Found

If SPIRES finds errors in the format definition, it will report them to you with error messages, like this:

The example shows a common error message. Probably the element name COPIES.LEFT was mistyped as COPIES.LEFR. SPIRES tells you where the error is by preceding the message with starred lines of information naming the format, frame and label group involved. More details about the error (or errors) can be obtained through the EXPLAIN command (EXPLAIN INVALID ELEM MNEMONIC, for example).

If multiple formats are being compiled in a single format definition, SPIRES will immediately place the compiled characteristics of a successfully compiled format into the FORCHAR subfile, display the message "Compiled format: ..." and move on to the next format. If an error is detected during the compilation, then all subsequent formats will not be moved into FORCHAR. The message "Format ... was not compiled" is your signal that the compilation was not successful and that the format in question was not moved into FORCHAR.

If an error is detected and the format definition does not compile, you should change the format definition appropriately. The procedure is:

Of course, when the format does compile, you may try using it, which is a topic discussed in the next chapter. [See B.7.1.] However, successfully compiling a format definition does not necessarily mean having a format that works the way you want. You may decide to make changes to the format definition, which means recompiling it. And, when you are all done with it, you will want to destroy the format definition and the FORCHAR record. Those subjects are discussed in the next sections. [See B.6.3, B.6.4.]

The STATISTICS Option

At some point, a format may become too large for one or more of the internal tables that are constructed by SPIRES when it is compiled, in which case you may need to move parts of it to "load formats", compiled separately. [See B.11.] By monitoring the table sizes as you compile and recompile a growing format, you can avoid the surprise (if not the heartbreak) of having to create load formats, perhaps at a time when you have the least amount of time to do so.

If you add the STATISTICS option to the COMPILE or RECOMPILE command, SPIRES will display data like this:

The data shows that the "Primary Label Groups" table is nearing its limit; those tables over 90 percent full are marked with an asterisk.

In most cases where SPIRES attempts to fill a table beyond its capacity, the format compilation will fail. However, some tables may have "soft" limits that can be exceeded without causing SPIRES to stop compiling. If a table fills beyond 100 percent without stopping the compilation, the percentage shown will be "**". Unfortunately, there is no definitive way to tell whether the compiled code in those tables is defective; instead, you are advised to divide the format definition into separate load formats if any of the tables exceed 100 percent.

The FORMAT-OPTIONS Statement and its Effect on Compilation

The FORMAT-OPTIONS statement may be added to your format definition in order to control how SPIRES stores label group names or element references in your compiled format. The syntax is:

The possible "values" are:

There are currently two restrictions on use of the GENELEM or GENVIRT options. First, you may not use those options if the format contains SET FILTER or CLEAR FILTER Uprocs. And second, you may not use them if the format has multiple SUBTREE statements in a frame.

If format size and efficiency are vital concerns for you, note that GENELEM and GENVIRT cause some tables in the compiled format to grow larger. These options also cause some drop in efficiency at execution time, since the element names will be converted to numbers whenever the label group is executed.

Even though GENELEM and GENVIRT eliminate one situation in which changes in file definitions require recompilation of formats, you should still be vigilant about when formats need to be recompiled. For example:

B.6.3  Recompiling a Format: the RECOMPILE Command

Once your format is compiled, any changes you make to the format definition in the FORMATS subfile will require you to recompile it in order for those changes to take effect. The RECOMPILE is used to recompile a format definition that is already compiled:

where "idname" is the value given in the ID statement, the same value you provided in the COMPILE command. [Some users may be able to compile and recompile formats belonging to other accounts, in which case the "gg.uuu." prefix must appear before the "idname".]

The STATISTICS option can be added to request information about the sizes of internal tables being constructed by SPIRES as the format is compiled. It can be important information to monitor for large applications. The option is described in detail in the previous section on the COMPILE command. [See B.6.2.] You may use the IN ACTIVE prefix in conjunction with the STATISTICS option to direct the statistics to your active file instead of your terminal.

By default, SPIRES will warn you if any of the internal tables exceed 90 percent of their capacity. If you add the NOWARN option to the command, you suppress these messages.

The LABELS option tells SPIRES to store label-group names in the compiled format record, for use in SET FTRACE output. If the format is compiled without the LABELS option, SET FTRACE will identify label groups by positional numbers rather than by names. If the format definition includes a FORMAT-OPTIONS = LABELS; statement, then the LABELS option on the RECOMPILE command is unnecessary. [See B.6.2.]

Recompiling Because of File Definition Changes

You may also need to recompile a format if the order or names of elements in the record-type of the file changes, or if the order or name of the record-type changes. Strictly speaking, it is only necessary to recompile if the format references elements whose positions are affected by such changes.

If your format definition includes a FORMAT-OPTIONS = GENELEM; or FORMAT-OPTIONS = GENVIRT; statement you may not need to recompile the format when a new element is added to a file definition. But see the previous section for caveats about use of these statements. [See B.6.2.]

B.6.4  Destroying a Format: the ZAP FORMAT (ZAP FOR) Command

When you no longer need the formats in a particular format definition (e.g., you are destroying the file), you should let SPIRES discard the compiled characteristics in the FORCHAR subfile and the format definition in the FORMATS subfile. That will happen when you issue the ZAP FORMAT command.

where "idname" is the value given in the ID statement of the format definition, with or without the account number prefix. Without the SOURCE option, only the compiled code in FORCHAR will be removed; if you include the SOURCE option, the format definition in the FORMATS subfile will also be removed. If you believe it is possible you will want to use the format definition again, you may want to omit that option.

Only the format definer (i.e., the account given in the ID statement), the file owner, and any accounts that the file owner has given METACCT access to the FORCHAR subfile, may zap a particular format.

If you zap a format and later realize you made a mistake, the format definition and possibly the compiled characteristics can be recovered if the system file containing the FORMATS and FORCHAR subfiles has not been processed (i.e., you must realize the mistake the day you zapped it). Try selecting the FORMATS subfile in SPIRES and dequeuing the format definition, or alternatively, processing the records under FOR TRANSACTIONS. That can help you recover the format definition itself, which you can compile. If you need more help, contact your SPIRES consultant before that system file is processed overnight.

You are not charged for storage of format definitions or their compiled characteristics, and you are not required to eliminate them when you no longer need them. Still, to keep the system files from becoming cluttered with unused formats, you are asked to zap formats that you will not be using again.

B.7  Using Your Format

Once you have compiled your format, you may return to SPIRES and try using it. After selecting the appropriate subfile, you issue the SET FORMAT command, discussed in the next section. If the named format has data frames with DIRECTION = OUTPUT and USAGE = DISPLAY (the only ones covered so far), then output commands such as TYPE and DISPLAY will display the requested record or records via those frames.

If your format does not work properly, you will need to debug it, which is the topic of the second section in this chapter.

B.7.1  Setting a Format: the SET FORMAT (SET FOR) Command

To use a compiled format, you must first issue the SET FORMAT command, which looks like this in its most basic form:

where "format.name" matches one of the names given in the FORMAT-NAME statements of the format definition. These names will also appear in the list of formats displayed by the SHOW FORMATS command. (New format names will not show up until the day after they have first been compiled.)

The "parm" option may be added to the end of the command, letting you pass information to the format dynamically (via the $PARM variable). The "parm" option is used, for example, by the system format $PROMPT:

The character string "name city phone.number" would be assigned to the system variable $PARM for processing within the format. Note that the parameter was delimited from the format name with a blank, a form of the SET FORMAT command not shown in the syntax statement above. A blank as delimiter is not allowed in general, but is allowed for system formats such as $PROMPT.

Other forms of the SET FORMAT command are necessary when you are setting a global format or a general-file format. [See B.14, D.2.5.] For a complete list of other forms: [See B.11.2.]

You may also "reset" a format by issuing the SET FORMAT command with the "*" option:

This causes the "startup" frame of the format currently set to be executed again, if there is one. [See B.5.2.] However, static variables are not reset to their original allocated values by the "SET FORMAT *" command. You can do that by setting the format again, identifying the format by name in the SET FORMAT command. [See B.9.3.]

To return to the SPIRES standard format, issue the command CLEAR FORMAT.

B.7.2  Debugging a Format

When you first try your format, it may not work exactly as you expected, especially if it is a complicated one. A data value may have been positioned in the wrong place, or a title may not have appeared, for example. You will need to re-examine your format definition, make changes as appropriate, and recompile it. [See B.6.3.] In most cases, the problem is caused by a minor error or omission -- it may be that a simple typographical error is the cause of what appears to be a major problem.

Here's one important tip to keep in mind: When testing a format, be sure that you are familiar with the data in the sample record or records you use. You may be expecting a certain element to appear that does not even exist in the record, for example. Try displaying the record in the standard SPIRES format if you have any doubts about its contents.

A simple "troubleshooter" list for debugging formats appears at the end of this chapter. [See B.7.3.]

B.7.2.1  The SET FTRACE (SET FTR) Command

The format tracing facility, invoked in its simplest form by the SET FTRACE command, is a very useful tool when you are trying to debug complicated formats. You may set format tracing for any format you can use.

As a record is processed through a format when the tracing facility is turned on, SPIRES will display messages on the terminal screen describing its progress through the format. Events noted include: entry and exit to the format; entry and exit to each frame; call to and return from load formats; subgoal access (including the key of the accessed record and information about the record-type); and errors as they occur. [See B.11, B.12 for information about load formats and subgoal access.]

A number of options are available on the SET FTRACE command, to provide detailed or specialized tracing. Here are the variations:

The VARIABLES and FRAMES options also take lists of variable and frame names:

SET FTRACE alone enables the basic tracing mechanism. You may type one or more of the other forms of the command in order to turn on additional trace information. Each variation except SET FTRACE ALL is described below. For online explanations, type EXPLAIN SET FTRACE followed by the option you are interested in (e.g. EXPLAIN SET FTRACE SNAPSHOT).

Basic Format Tracing

Here is an example of output from a simple SET FTRACE command:

In the above example, the record is placed in the active file; the tracing information is displayed at the terminal, line by line, as SPIRES executes the format. (You may instead direct the tracing information to a "trace log" for later examination.) [See B.7.2.7.]

The format DISPLAY is comprised of the data frame DRINK, which contains the indirect frame INGREDIENTS. INGREDIENTS is an indirect-structure frame that is executed several times, as the example shows. [See B.8.] When SPIRES leaves a frame, the name of the frame is displayed along with the number of the last label-group executed in that frame (the first label-group being number 1). [This format is created and explained in detail in the SPIRES manual "A Guide to Output Formats".]

This is just a simple example. More complex formats will have more complex tracing messages.

Note that you can compile your format with the LABELS option (on the COMPILE or RECOMPILE commands) or include a FORMAT-OPTIONS = LABELS; statement in your format definition, in order to see label-group names in your FTRACE output rather than just the positional numbers shown in the example above. [See B.6.2.]

Seeing What Tracing is in Effect

The SHOW FTRACE command will display the formats tracing that is currently in effect:

Turning Off Formats Tracing

To turn off all format tracing, issue the CLEAR FTRACE or CLEAR FTRACE ALL command. Setting a format or selecting a subfile will also turn it off.

You may instead turn off specific forms of tracing with one or more of these commands:

(The basic format tracing will remain on, even if you turn off all the specialized tracing mechanisms.)

Tracing Global Formats and SPIBILD Formats

You can set tracing for global formats in effect by issuing the command SET GLOBAL FTRACE, and turn it off with CLEAR GLOBAL FTRACE. All of the FTRACE variations described above may also be used for global formats, e.g. SET GLOBAL FTRACE SNAPSHOT, SHOW GLOBAL FTRACE. [See D.2.5.]

Format tracing may also be used to help debug formats used in SPIBILD. [See C.9.]

B.7.2.2  SET FTRACE JUMP

SET FTRACE JUMP produces a trace of all JUMP (GOTO), CASE, and XEQ PROC commands executed (plus RETURNs from XEQ PROC). Here is a simple example of SET FTRACE JUMP output:

B.7.2.3  SET FTRACE SNAPSHOT

SET FTRACE SNAPSHOT displays selected information at certain control points of a format's execution, e.g. whenever PUTDATA, PUTELEM, REMELEM, or IND-STRUCTURE statements are executed. Here is an example:

B.7.2.4  SET FTRACE VARIABLES

If you would like to see the values of static variables as they change during format execution, issue the SET FTRACE VARIABLES command. Here is a very simple example, involving only one variable:

The command SET FTRACE VARIABLES traces all your static variables. You may also include a variable list on the command, to limit the tracing to specific variables. In subsequent SET FTRACE VARIABLES command, you can add to or subtract from the list with a plus or minus sign. For example, this series of commands establishes tracing for all static variables except those named USERCODE and USERNAME:

And here is another example showing how you can add to or subtract from variable lists:

(After these commands, you end up tracing the LONGPHONE and AREACODE variables.)

B.7.2.5  SET FTRACE FRAMES

If you have turned on JUMP, VARIABLES, or SNAPSHOT tracing, the SET FTRACE FRAMES command will limit the detailed tracing to the specified frames. The basic tracing still occurs for other frames, unless you have also issued the SET FTRACE BRIEF command (see below). SET FTRACE FRAMES has no effect if JUMP, VARIABLES, or SNAPSHOT tracing are not in effect. [See B.7.2.2, B.7.2.3, B.7.2.4.]

In most cases, your SET FTRACE FRAMES command would include a frame list. In subsequent SET FTRACE FRAMES commands, you can add to or subtract from the frame list with a plus or minus sign.

SET FTRACE FRAMES without a frame list usually has no effect. (If no formats tracing at all is on when the command is issued, it will turn on the basic tracing, for all frames.) But you can use that form of the command to specify an "exclusion list" for tracing of frames. For example, this series of commands would let you see snapshot tracing for all frames except those named HEADER and FOOTER:

B.7.2.6  SET FTRACE BRIEF

SET FTRACE BRIEF is useful in conjunction with SET FTRACE FRAMES. [See B.7.2.5.] It suppresses all tracing information for frames other than those requested in your SET FTRACE FRAMES commands. In the absence of SET FTRACE BRIEF, you see detailed tracing (JUMP, VARIABLES, or SNAPSHOT tracing) for the frames you selected with SET FTRACE FRAMES, and basic tracing for other frames.

B.7.2.7  Directing Tracing Output to a Trace Log

Normally, the SET FTRACE output is simply displayed at the terminal as the format executes. For complicated debugging tasks, you may find it convenient to send the tracing information to a "trace log" for later examination. The SET TLOG command accomplishes this task. To look at the tracing, issue the command SHOW TLOG. Or, to place the tracing information in your active file, type IN ACTIVE SHOW TLOG. For example:

For details about using trace logs, EXPLAIN SET TLOG.

B.7.2.8  Using the $FTRACE Variable

The format definer can request that specific information be displayed at the terminal during format execution by using the $FTRACE system variable, which indicates whether the tracing facility is on or off. For instance, you can code a statement such as:

This Uproc, coded in a label group for the DATE element presumably, would only send the message to the terminal when the tracing facility was on. Hence, you can keep track of element and variable values as they are changed by coding Uprocs such as that.

Unlike protocols, which may have BREAK XEQ commands to temporarily halt their execution, formats cannot be stopped between label groups or between frames in order to test variable values. SET FTRACE SNAPSHOT and SET FTRACE VARIABLES may provide detailed enough tracing of changing values, but if not, you may find it helpful to use $FTRACE as described here.

Note that if you want to direct custom messages to the trace log (with the SET TLOG command) you should use the $ISSUEMSG function rather than the * Uproc, as in this example:

For most tracing situations, $ISSUEMSG is preferable to using the "*" command, since it will only appear on the terminal screen and never as part of the trace log. [EXPLAIN SET TLOG for information about trace logs. EXPLAIN $ISSUEMSG for an explanation of the function.]

B.7.3  A Troubleshooter's Checklist for Format Debugging

Whenever you are debugging a format, make sure you have not disabled the display of system diagnostics via a SET MESSAGES = 0 command. You should at least use the default setting, SET MESSAGES = 2. This applies to the SET MESSAGES Uproc allowed in the format definition too. [See C.3.6.4.]

If nothing appears except a prompt when you display a record:

If the record is displayed in the standard SPIRES format rather than through yours:

If a data element you are expecting to see does not appear at all:

If only one of several occurrences of an element appears:

If a value appears in the wrong position:

If "garbage" appears somewhere in the display:

The above is only a list of suggestions; it does not represent a complete listing of all possible explanations for an error.

B.8  Handling Structures in Output Formats

The previous chapters of Part B have discussed the basics of output format creation. The remaining chapters of this part will cover other aspects of output formats that are certainly important but perhaps not as fundamental. In this chapter, you will learn how to handle structures in an output format.

Generally, a data frame (i.e., a frame of FRAME-TYPE = DATA) can retrieve only record-level elements, that is, elements not within structures. To retrieve all the elements within a structure requires the use of an indirect frame. [See B.4.8.7.] (There are some situations where you do not need an indirect frame; they are described at the end of this section.) The indirect frame is called by the IND-FRAME statement; additionally, the IND-STRUCTURE statement identifies the structure that will be processed by the indirect frame. [See B.8.2.]

The elements within the structure are individually processed in the indirect frame. That frame contains label groups with GETELEM, PUTDATA and other statements to handle the individual element occurrences just as the data frame does for the record-level elements. The indirect frame also includes a SUBTREE statement that specifies which structure of the goal record is being processed by the frame. [See B.8.1.]

Two important factors to consider are the relationship between the frame dimensions of the calling frame and the indirect frame, and the impact of the PUTDATA statement when it is placed on the label group that calls the indirect frame. They are discussed in the next sections.

Situations where Indirect Frames Are Not Necessary for Structures

There are two situations where formats do not need to have indirect frames to process structures. One is a general situation, and the other is quite specific.

First the general case. You may be able to treat the structure as if it were a single element value, using the system procs $STRUC.OUT or $STRUC (action A33) in the OUTPROC string for the structure. These processing rules are often coded for multiply occurring structures whose individual elements are singly occurring. EXPLAIN A33 RULE or EXPLAIN $STRUC PROC for more information about them.

Suppose for instance that the PHONE structure contains the elements AREA.CODE, PREFIX and SUFFIX:

The OUTPROC tells SPIRES to get the first occurrence of each of three elements in the structure and put them together, separated by hyphens, into a single value like this: "415-497-4420".

Note that the structure is treated as a single element value when the structure processing rules ($STRUC.OUT, A33, etc.) are in effect. Hence, in the format you do not have the flexibility of individually positioning each element of the structure if you retrieve the structure this way, unless you then split the value into its component parts yourself.

The second situation, though not common, does give you positioning flexibility, in that it permits you to position each element of the structure individually, each being handled in its own label group. The second situation arises when the structure containing the elements you want to process is singly occurring, or you want to process only the first occurrence of the structure.

Using the PHONE example above and assuming only the first PHONE occurrence is needed, you might code the following label groups:

Here no mention of the structure PHONE appears. Only the elements within the structure are specified. (If they were not unique, i.e., if other elements having the same element names were in the record-type, the element names could be preceded by "PHONE@", as in "GETELEM = PHONE@PREFIX", to specify a structural path for SPIRES to follow to the desired element.)

All occurrences of the elements within that occurrence of the structure may be retrieved (e.g., using the LOOP statement). Remember though that you cannot use this method to retrieve elements within occurrences of the PHONE structure other than the first in the record.

The rest of this chapter describes the format definition code needed when you must use indirect frames to process structures.

B.8.1  The PUTDATA Statement and Frame Dimensions for the Indirect Frame

Basically an indirect frame that formats data can work in one of two ways. Either it can place its data directly into the calling frame, just as if its label groups were in the calling frame's definition, or it can place its data into a buffer of its own, so that the entire buffer is positioned in the calling frame all at once when the indirect frame has completed execution. Which method SPIRES uses depends on whether a PUTDATA statement is coded on the calling label group, whether or not it is used to handle structures.

If no PUTDATA statement appears there, then the first method is used. Although execution control switches to the indirect frame, all data processed by it is placed directly into the calling frame. The current row and column numbers from the calling frame are thus carried into the indirect frame. In other words, the main impact of the indirect frame for structure processing with this method is that it now allows you access to elements within some structure, but otherwise acts as if you were still in the calling frame. In fact, any FRAME-DIM statement you code in the indirect frame will be ignored, since the frame dimensions of the calling frame are used when the PUTDATA is omitted from the calling label group.

For example, here is a calling label group on the left, and the indirect frame it calls on the right.

The starting positions declared in the indirect frame refer to locations in the calling frame. Since there is no PUTDATA statement in the calling label group, no FRAME-DIM statement is necessary in the indirect frame -- it would be ignored anyway.

Note the use of the TITLE and TSTART statements to provide a title for the values processed by the indirect frame. This is one of the few situations in a label group where the statements are not executed in the order shown -- the TITLE and TSTART statements are executed before the indirect frame is called.

Finally, note the LOOP statement on the calling label group, which directs SPIRES to execute the indirect frame again to process the next occurrence of the structure. Remember that if there were no occurrences of the CLASSES structure at all, the calling label group would not be executed, and hence the indirect frame would not be executed either.

Coding the DEFAULT statement on the calling label group will cause the indirect frame to be executed even though the structure does not occur; all GETELEM statements in the indirect frame will fail, unless those label groups have their own DEFAULT statements. [See B.4.5.1.] If SPIRES is executing an indirect frame under DEFAULT processing, it sets the system variable $NODATA. [See E.2.1.27.]

On the other hand, if a PUTDATA does appear on the calling label group, then a buffer is built apart from the calling frame, using the frame dimensions given for the indirect frame. As it executes, the indirect frame places data into the buffer, which has its own row and column numbers independent of those in the calling frame. When the indirect frame is finished executing, the calling label group regains control, handling the placement of the indirect frame's buffer in the calling frame. The rows of the buffer are put together end to end, so that it becomes a long string value that is assigned to $CVAL. Then, using the PUTDATA and other value placement statements in the calling label group, SPIRES positions $CVAL in the calling frame just like any other value. Any DISPLAY attributes set in the indirect frame are lost because $CVAL only contains data, not attributes.

Here is another calling label group and the indirect frame, but this calling label group has a PUTDATA statement on it. The result is the same as that of the example above:

The structure data is placed in the DO.CLASSES buffer just as it was placed in the calling frame's buffer before. The FRAME-DIM statement assigns 60 columns to each row of the DO.CLASSES frame, which matches the number of columns allotted to it by the calling label group in the START and MARGINS statements (column 5 to column 64 is 60 columns). It is usually crucial that these numbers match; otherwise, after the indirect frame is converted to a long one-dimensional string value, it will not be placed properly into the calling frame.

Note that blank rows at the end of the indirect frame are not returned, just as blank rows at the end of a data frame are not displayed. However, any blank character positions at the end of the last row having data will be included at the end of $CVAL. (That will affect the value of "X" for column positioning for the next START statement, for example.) Generally they will not cause any problems, but you may code an OUTPROC of system proc $TRIM (action A51) in the calling label group to remove them.

You can see that this second example is much more complicated than the first, so usually the first method is preferable. However, there may be times when you must use the latter method to get the desired results. For example, suppose structure ALPHANUM contains singly occurring, optional elements ALPHA and NUM. Your format design positions them like this:

Each occurrence of the structure appears on a new row, except for the first. In this situation, it is easiest to treat the structure as a single value to be positioned in the data frame:

Why was the PUTDATA in the calling label group necessary? First of all, without the PUTDATA there, each new occurrence of the structure would have overridden the previous one -- they would have all started on the same row with the record key. (Remember, the START and XSTART statements in the calling label group would be ignored.) Changing the starting row for the elements in the indirect frame to anything else would still cause similar problems. Problems could also arise if there were occurrences of the structure that did not have the ALPHA element. Try changing the starting rows for the indirect frame elements and adding or removing the PUTDATA statement on the calling label group in various combinations -- you will clearly see why this is the best (though not only) solution to this design problem.

The rest of this chapter will discuss the other statements needed for coding indirect frames.

B.8.2  Statements for the Indirect Frame: the SUBTREE Statement

The first few statements in an indirect frame definition, that is, the frame identification statements, generally look very much like those of the calling frame. The differences usually involve the SUBTREE and FRAME-DIM statements. The FRAME-DIM statement, whose values (or even whose inclusion) depends on the presence or absence of a PUTDATA statement in the calling label group, has already been discussed. [See B.8.1.]

The SUBTREE statement names a structure of the goal record, indicating the hierarchical path from the record level that SPIRES should follow to access the data elements processed by this frame.

SUBTREE may also name a phantom structure. [See B.12.1.]

If the structure does not have a unique name in the goal record-type, then you must indicate the path SPIRES must take to get to that specific structure, beginning with the record-level structure or a unique structure name within that path:

The SUBTREE statement tells SPIRES that all elements referenced in GETELEM and IND-STRUCTURE statements in the frame will be in that structure. Any elements not in the structure named by those statements will cause a compilation error.

The SUBTREE statement is not required for indirect frames that process structures, though it is recommended unless you need to retrieve other elements from outside the structure in such a frame. If you omit the SUBTREE statement, you can use GETELEM statements to retrieve other record-level elements from within the indirect frame, which normally allows access only to the elements within the "subtree".

Warning: If you do retrieve "extra-structural" elements from within the indirect frame with GETELEM statements, the frame may not correctly retrieve occurrences of the elements within the structure (other than the first occurrences) from that point on in the indirect frame. In other words, once an extra-structural element is accessed, the indirect frame no longer has the structural-access power it did have. Moreover, if such an indirect frame is being executed repeatedly because of a LOOP statement, "infinite loop" problems can arise -- SPIRES may access the same structural occurrence repeatedly.

If you need to retrieve extra-structural elements from within the indirect frame, use one of the $GETxVAL functions (like $GETCVAL) instead of the GETELEM statement in order to retain the structural-access power of the frame.

Occasionally a goal record-type contains a "floating structure", i.e., two or more structures from different subtrees that have identical definitions. (An ADDRESS structure containing STREET.ADDRESS, CITY, STATE and ZIP elements might appear in several places within a record-type, for instance.) A single indirect frame may be used to process all of the floating structure's appearances if multiple SUBTREE statements are coded. Each one must define a unique hierarchical path to the floating structure. The IND-STRUCTURE statement in the calling label group tells SPIRES which subtree to access.

The "@n" option on the GETELEM statement is often useful when you want to use the same frame to retrieve from multiple subtrees. [See B.4.2.1.] In this case, the structures processed by the indirect frame do not have to be identical (that is, they do not necessarily have the same elements with the same names) though they would probably be very similar.

The SUBTREE statement may also have the form:

which indicates that the frame is a multiple-subtree frame, designed to handle element retrieval from any structure or at the record level. It allows a frame of FRAME-TYPE = STRUCTURE to be a general frame for any subfile. [See B.15, B.14.]

B.8.3  Statements in the Calling Label Group: the IND-STRUCTURE Statement

The calling label group needs to have an IND-STRUCTURE statement if elements within the desired structure are to be processed in an indirect frame. What other statements are necessary in the label group depends on whether a PUTDATA statement will be included as well. [See B.8.1.]

The IND-STRUCTURE statement has this syntax:

where IND-STRUCTURE may be abbreviated to IND-STRUC and where "structure.name" has the same form as "element.name" in a GETELEM statement. You may, for example, request a specific occurrence of the structure by using the form: "structure.name(n)", where "n" is an integer representing the occurrence number desired, counting from 0 (zero) for the first. [See B.4.2, B.4.2.1.]

A calling label group often has a LOOP statement to cause SPIRES to return to the indirect frame to process the next occurrence of the structure.

Here is a label group that calls an indirect frame for structure processing:

The Uproc, which is presumably keeping a running total of some values retrieved in the indirect frame, is executed for each occurrence of the structure, just before the LOOP statement tells SPIRES to re-execute the indirect frame. When no more occurrences of the structure exist (which SPIRES would determine when trying to execute the IND-STRUCTURE statement), the loop is broken and SPIRES would continue to the next label group.

B.9  User and System Variables in Output Formats

As you have seen in earlier examples, SPIRES variables can be indispensable in many format coding situations. System variables such as $CVAL keep track of the value to be placed in the frame, or its starting position in the frame, or its current length in characters, or some other useful information. User variables can be created to hold values to be used in other label groups, other frames, or even outside of the format.

The system variables always exist; that is, memory is always allocated for them. User variables must be defined, compiled and then "allocated", which means telling SPIRES to reserve memory for them and perhaps assign initial values to them. Both types of variables typically make appearances in Uprocs and VALUE statements, though they are allowed in many other statements as well.

Basically variables may be used the same way in formats as they can be in protocols; most of the syntax rules are the same. The next section will briefly cover those basics and describe the differences from protocol use in detail. [See B.9.1.] The following section will discuss the use of system variables in output formats, briefly describing the most useful ones. The remainder of the chapter will cover user variables, explaining how to define, compile and allocate them. [See B.9.2.] Local vgroups ("variable groups") defined within the format definition and global vgroups defined and compiled separately will both be discussed. [See B.9.3.]

B.9.1  Using Variables in Formats

This section will review basic syntax rules concerning the use of system and user variables in format definition code. More information about variables in general can be found in the manual "SPIRES Protocols".

System variables, whose names each begin with a dollar sign, usually contain values set by SPIRES. Some of them, however, may be set explicitly by you in the format, such as $PROMPT, $ASK and $CVAL, by issuing a SET Uproc:

The SET Uproc in a format, unlike the SET command, allows expressions:

All of the system variables explicitly related to formats are explained in detail later in this manual. [See E.2.] Note that other system variables are available within a format too (for instance, $SELECT, which contains the name of the selected subfile), though they may not be settable.

User-defined variables are identified by the pound sign that begins each of their names, e.g., #ANSWER. A value may be assigned to a user variable in one of two ways: either in a LET Uproc or command, or in a VALUE statement executed when the variable is allocated. [See B.4.8.10, B.9.3.1.] During format execution, only the LET Uproc can do it:

Note that the pound sign (or the dollar sign for system variables) is omitted when the variable is on the left side of the equals sign in a LET or SET Uproc or command. Any variables on the right side of the equality operator must have the pound or dollar sign, as appropriate, if substitution of their values is desired.

In regard to variable substitution, note that the "/" prefix, used in commands to force substitution, is not available in formats; appropriate substitution occurs automatically as long as the variable is not part of a string enclosed by apostrophes or quotation marks. For example, compare these two statements, the first a command, the other a formats Uproc:

Both would have the same effect (though the latter display would not include the "*" at its beginning). In a command, the literals do not have to be enclosed by quotation marks or apostrophes, while they almost always do in Uprocs. On the other hand, you do not need (and in fact cannot use) the "/" to force variable substitution for $TIME in the Uproc.

Sometimes you may want to check or use variable values assigned during format execution outside of the format. However, most of the system variables directly related to formats have no meaningful value outside of format execution. For example, if you issue the command SHOW EVAL $CVAL to see the last value of $CVAL inside the format that just executed, the value returned will probably be garbage; it is certainly not reliable. User variables and some system variables settable from within a format, such as $PARM, $PROMPT and $ASK, can be used to pass values into and out from formats. [See B.9.3 for global vgroups.]

Variable Types and Conversions

Generally speaking, you must be somewhat more conscious of variable types when you work with variables in formats. All system variables (except $CVAL, $UVAL and $PVAL) and user variables in formats have a specific type associated with them, such as string or integer or flag (the most common ones). (The three exceptions may vary in type.) Similarly, all user variables available in a format have a type associated with them; user variables may not be created dynamically in formats in the same way as they can in command mode, so they are defined as having some type or another. [See B.9.3.1.]

Remember then that when you assign a value to a variable, that value must be convertible to the type of the variable; otherwise a conversion error will occur and format processing will stop. For example, if INTEGER is an integer variable, the following Uproc will stop the format from processing the current record further:

Though that particular error is easy to see, it becomes somewhat more obscure when the value being assigned is another variable. For instance, now suppose that the value of the string variable VALUE is "123.45":

That still fails, but it would not fail if the value of VALUE were "123", because that value could be converted to an integer.

Some format statements allow variables as values, such as MAXROWS. The type of the variable used for such statements often does not matter, as long as the substituted value when the statement is executed is convertible to the type the statement expects. For example, a string variable may be given for the value of the MAXROWS statement as long as the variable's value is convertible to an integer when the statement is executed. [See B.4.6.6.]

B.9.2  System Variables that are Useful in Output Formats

Below is a list of system variables that are useful in output formats, each with a brief description. Variables used primarily within report formats are listed later. [See B.10.10.] Details about all the variables are provided later in the manual. [See E.2.]

The variable type is given in parentheses after the variable name.

The first group of variables is reset each time a new label group is entered:

  $UVAL (varies)  - the unconverted (i.e., before OUTPROCs or
                     INSERTs) value of the label group.
  $CVAL (varies)  - the converted (i.e., after OUTPROCs or INSERTs)
                     value of the label group.
  $PVAL (varies)  - the unconverted value of the previous label group.
  $ULEN (int)     - the length of the unconverted value.
  $CLEN (int)     - the length of the converted value.
  $ELOCC (int)    - the number of occurrences of the element specified
                     by GETELEM.
  $LASTOCC (int)  - the occurrence number of the final occurrence of
                     an element specified by GETELEM (usually $ELOCC-1)
                     numbered from 0.
  $LOOPCT (int)   - value of the LOOP counter for the current element,
                     beginning at "0" for the first time through.
  $DEFAULT (flag) - set when a DEFAULT statement is coded and the
                     GETELEM or IND-STRUCTURE statement fails to
                     retrieve an element in the record.
  $PROCERR (flag) - set when an S- or E- level error is returned
                     from OUTPROC or INPROC execution.
  $APROCERR (flag)- set when any error is returned from OUTPROC
                     or INPROC execution.
  $REPEAT (flag)  - can be set, causing SPIRES to duplicate the
                     output value to fill the available space for it.
  $SKIPROW (flag) - can be set, causing SPIRES to double space
                     the value as it is output.
  $LABEL (string) - the name of the currently executing label group.

The variables below are not reset when a new label group starts executing:

  $CROW (int)     - the current row position in the frame, i.e., "*".
  $CCOL (int)     - the current column position, i.e., "*".
  $SROW (int)     - the starting row of the most recently positioned
                     value in the frame.
  $SCOL (int)     - the starting column of the most recently
                     positioned value in the frame.
  $LROW (int)     - the number of the highest row used in the frame
                     (i.e., X-1).
  $LCOL (int)     - the number of the last column processed (X-1).
  $NROWS (int)    - the number of rows assigned to the current frame.
  $NCOLS (int)    - the number of columns assigned to the frame.
  $RECNO (int)    - the number of records processed so far, including
                     the current record, under this command.
  $FREC (flag)    - set when the first record is being processed
                     during a multiple-record processing command.
  $GPROCERR(flag) - set when an S- or E-level error has occurred
                     during INPROC or OUTPROC execution.
  $ABORT (flag)   - set when an ABORT or STOPRUN Uproc is executed;
                     useful only outside of the format.
  $PARM (string)  - the parameter list from the SET FORMAT (or SET
                     GLOBAL FORMAT) command.

Many more variables may be useful within output formats -- this is just a list of those that are particularly useful there.

B.9.3  User Variables in Formats: Vgroups

As soon as your formats involve more complicated processing than just "GETting an ELEMent" and "PUTting the DATA" into the output grid, you will find that user variables are indispensable. Within the format, they are useful for holding values across label groups and frames. (For example, you might want to concatenate elements together and position them as a single value in the frame, requiring you to hold the earliest elements in a variable as you retrieve the later ones.) In complex applications where protocols and formats must work together, user variables may pass values back and forth between the protocols and formats.

Although you can create variables dynamically in command mode, you must usually predefine, compile and allocate them if you want to use them in a format. This process, called creating a "vgroup" (for "variable group"), can be specified completely within the format definition. Such vgroups are called "local vgroups". Alternatively, you may define and compile the vgroup separately, with only the allocation statement appearing within the format definition. That type is called a "global vgroup", because its variables could be referenced by other format definitions or by protocols.

Whether the vgroups you want to use in a format are local or global or a mixture (you may use multiple vgroups in a format), the vgroup definitions are similar. Complete information on the statements in a global vgroup record is given in the manual "SPIRES Protocols", section 4.2.1.1. Since the statements for a local vgroup in the format definition are practically all the same as for a global vgroup, only the basic rules for them are given in this manual, appearing in the next section. [See B.9.3.1.] The brief section that follows will discuss the ALLOCATE statement. [See B.9.3.2.]

B.9.3.1  The Vgroups Section of the Format Definition

To create a local vgroup, you must first define it in the Vgroups section of the format definition. That section begins after the identification section, immediately following the RECORD-NAME statement.

Multiple vgroups may be defined in this section, each vgroup beginning with the VGROUP statement (see below). Any variable in any vgroup in the format definition may be used in a format, as long as the format has an ALLOCATE statement naming that variable's vgroup. [See B.9.3.2.]

A vgroup definition contains from 1 to 256 variable definitions under an umbrella name:

where "vgroupname" is a name from 1 to 16 characters long, which may include alphanumeric or the special characters period (.), hyphen (-) or underscore (_). It may be prefixed by the format definer's account number, which can extend its length to 23 characters, but that is not considered good practice. For example,

After the VGROUP statement and COMMENTS statements (optional) come the individual variable definitions, each of which begins with a VARIABLE statement that names the variable:

Here "variablename" is a name from 1 to 16 characters long. Generally speaking, it should contain only letters and numbers.

Other statements describing the variable may follow:

OCC = number.occurrences;

This statement declares the number of occurrences that the variable may have. The default, if no OCC statement is coded, is one occurrence for each variable. (Multiple occurrences of a variable are individually referenced by an index: the variable name is followed by "::n" where "n" is a number from 0 to 32,767 representing the desired occurrence, counting from 0, or "n" is an integer variable representing such a number. Alternatively, the "::INDEX" feature can be used to handle such references.) The form of the OCC statement will be different for two- or three-dimensional arrays. [See "SPIRES Protocols", Section 4.3.]

LEN = length;

This statement specifies the length in bytes of each occurrence. It must conform to the restrictions inherent in TYPE. Default lengths are provided for each type, as shown in the TYPE chart.

TYPE = type;

This statement, which is probably the most important of them, describes the type of value that the variable will represent. The allowed types (and their allowed and default lengths per occurrence) are:

     Type      Allowed lengths      Default lengths
     STRING      1-32,765*            80
     INTEGER     1, 2, 4               4
     REAL        4, 8                  4
     PACKED      1-16                  4
     FLAG        1                     1
     CHAR        1-256                16
     LINE        4                     4
     HEX         1-32,767*             4
     DYNAMIC     1-32,765              0

     * The upper limit applies to singly occurring variables; if the
       variable occurs more than once, the upper limit  is  253  for
       string variables and 255 for hex.

The total size of ALL variables in a single vgroup cannot be greater than 65,536 bytes. You can define multiple vgroups if you need more variable storage than that.

VALUE = value0, value1, value2, ...;

This statement lets you assign initial values to the occurrences of the variable, one value per occurrence as given in the OCC statement. The given values, each a string, will be converted to the appropriate type during compilation. If special characters appear in any strings, including blanks or commas, those strings should be enclosed in apostrophes. No single value should exceed 255 characters in length.

For flag variables, you may use the values $TRUE and 1 to represent "true", and $FALSE and 0 to represent "false".

To assign the same value to multiple occurrences in a row, you can put the value in parentheses, preceded by the number of occurrences to be assigned that value. For example,

The first occurrence of the variable will have a null value, but the next 26 will have the value ABC.

Other useful statements include COMMENTS (just like other COMMENTS statements in a format definition), SITE (limiting the vgroup to a particular type of site, such as CMS or STS), REDEFINE, INDEXED-BY and DISPLAY-DECIMALS. For more information about the last four statements, see section 4.1.1 of the manual "SPIRES Protocols", or issue an EXPLAIN command.

A typical vgroup definition might look like this:

(*) Using Dynamic Variables with Formats

Dynamic variables may be used in a format if they are handled in either of the following ways:

You cannot create a dynamic variable in a format simply by referencing it in a LET Uproc, as you might do with a LET command in a protocol. That is, the following Uproc will not compile properly unless the variable TOTAL is in a defined vgroup:

Dynamic variables in formats may be useful in the following situations:

[See the manual "SPIRES Protocols" for more information on using dynamic variables.]

B.9.3.2  The ALLOCATE Statement

To use a vgroup in a format you must allocate it, i.e., tell SPIRES to allocate memory for its variables. The ALLOCATE statement is placed in the format declaration section, after the FORMAT-NAME statement:

For "vgroupname", if the vgroup is a local vgroup (that is, its definition appears in the format definition), then the name given in the VGROUP statement should be specified here. If the vgroup is a global vgroup, you must specify the vgroup name including the account number in one of the following forms:

If the name appearing in the ALLOCATE statement is the name of both a local vgroup and a global vgroup, the local vgroup will be used.

If you have multiple vgroups to be allocated for a single format, you can code multiple ALLOCATE statements. As many as 16 vgroups are allowed per format.

The HIDDEN, TEMPORARY, and CONTROLLED options are explained at the end of this section.

The ALLOCATE Statement and The ALLOCATE Command

The vgroup that you name in your ALLOCATE statement is allocated when the format is set, and cleared from main memory when the format is cleared. If you want the global vgroup that was allocated in your format to remain in memory when the format is cleared (e.g., to use the values of the variables with another format) you must issue an ALLOCATE command sometime before clearing the format, perhaps before setting the format in the first place. SPIRES then realizes that the global vgroup is not completely tied to the format, and will let it remain when the format is cleared. [See the manual "SPIRES Protocols" for information on the ALLOCATE command.]

(*) HIDDEN Vgroups

In most cases, when a format is set, a user of the format may examine or change variable values in the format's allocated vgroups -- the variables are not reserved exclusively for the format's use. The user may examine variable values by issuing SHOW STATIC VARIABLES or SHOW EVAL commands, for example; the first command displays the values of all variables in all allocated vgroups, while the second can be used to show the value of individual occurrences of variables. [See "SPIRES Protocols" for details on these commands.]

However, you can use the HIDDEN option on the ALLOCATE statement if you need to "hide" a vgroup from users of the format. When a vgroup is hidden, it cannot be seen with commands such as SHOW ALLOCATED or SHOW STATIC VARIABLES -- not even by the format's owner. Furthermore, its variable values can't be accessed or changed by any protocol or interactive command. The only access to the variables is through the format containing the ALLOCATE statement, or (for global vgroups) through load formats associated with the format.

Perhaps the main benefit of the HIDDEN option is that it allows two or more formats to be set simultaneously on different paths, and to access the same global vgroup, without any of the variables clashing. This is useful for system formats such as $REPORT and $PROMPT, and for formats code generated from these system formats. Thus, if you generate a report format from $REPORT statements, for instance, [See B.10.] you may find a hidden vgroup allocated in the format definition. If you decide you need to alter that vgroup, you may remove the HIDDEN option, if you prefer.

(*) TEMPORARY and CONTROLLED Vgroups

The TEMPORARY and CONTROLLED options on the ALLOCATE statement let you define working areas for local vgroups dynamically, and offer you ways to more effectively manage core for your application. These two options are allowed only for local vgroups.

If you use the TEMPORARY option, the vgroup will be allocated and initialized each time the format is called (or each time the format is entered if the vgroup is in a load format). The vgroup will then be discarded when the format or load format is exited. Because a temporary vgroup is cleared from memory when the format is left, it will not be shown when you issue a command such as SHOW STATIC VARIABLES.

A CONTROLLED vgroup will be assigned only when an ALLOCATE Uproc is executed. And it will be discarded by a DEALLOCATE Uproc (or when the format is exited, if the vgroup is also TEMPORARY). The syntax of these Uprocs is simply:

For example, you may want to define a vgroup that is used only during format startup. Your ALLOCATE statement could include the CONTROLLED option, and your startup frame could include ALLOCATE and DEALLOCATE Uprocs. In this case, the vgroup's memory space would only be assigned during startup.

(*) Using Global Variables not Allocated by the Format

Variables in global vgroups that were allocated but not by the format can be accessed, though only indirectly, via the $STATPUT and $STATGET functions. The $STATPUT function assigns a value to a static variable, and the $STATGET function retrieves one. More information about them can be found in the online manual "SPIRES Protocols". [EXPLAIN $STATPUT FUNCTION.]

By the way, $STATPUT and $STATGET cannot be used to find values of variables in hidden vgroups (see above).

B.9.3.3  Resetting Variables and Vgroups; the EVAL Uproc

Sometimes during format execution, you may want to reset individual variables or even entire vgroups back to their original allocated values. For a single variable, the LET Uproc is commonly used:

Some system functions are particularly useful in this regard:

These functions are discussed in detail in the manual "SPIRES Protocols".

To use these functions from within a format, you must either assign their returned value to a variable with a LET Uproc, e.g.,

or use the EVAL Uproc:

The EVAL Uproc tells SPIRES to evaluate the expression that follows but to discard the returned value. Its syntax is:

where "expression" may consist of functions, variables and strings. Usually functions whose purpose is to execute some process (such as resetting a vgroup) rather than return a value comprise the expression in an EVAL Uproc. Since the action of the $VINIT function (rather than the value returned from it) is what is important to us, the EVAL Uproc lets us execute the function while avoiding unnecessary variable assignments. The other functions named above may also be used with the EVAL Uproc.

B.10  Report Formats

Uses for Reports and Report Formats

Using the material discussed so far in this manual, you could construct a format to display an entire set of goal records, perhaps for a printed copy of your data. However, as attractive as the format might be, you would probably still notice that its "hardcopy" was not likely to be confused with, say, a hand-typed report.

For instance, your output would not have page numbers and would probably not have introductory information on the first page; records would be split across page boundaries; there would be no summary information reporting total or average values across the entire set of records.

A "report format" can add these and other extra features, making your output a more attractive and informative document. Report formats have even been used to create camera-ready copy for catalogs and bibliographies.

In general, reports are created (using SPIRES formats language or one of the easier methods listed elsewhere in this section) to accomplish one or more of the following tasks:

Methods of Creating a Report Definition in SPIRES

There is more than one way to create a report definition in SPIRES, and only the most complex reports require you to code your own SPIRES format by hand. Below, the different methods to create a report in SPIRES are listed in order from the easiest to the most complex (where the most complex method is also the method offering the greatest freedom to customize the way that the report executes):

Report Definer

The utility Report Definer lets you create an effective tabular report complete with simple summaries and page-layout control, simply by filling in a series of screens with your report specifications -- only the most basic SPIRES knowledge is needed. [See Part C of "SPIRES Searching and Updating" for details.]

$REPORT

The system format $REPORT lets you make table-like reports of more complexity than Report Definer offers. Initial, ending, header, footer and summary information are all relatively easy to specify, in commands issued interactively or stored online. [Like Report Definer, $REPORT is discussed in Part C of "Searching and Updating".]

Although you must learn a special subcommand vocabulary to use $REPORT, you do not need to code a report format -- thus for many (probably most) reporting needs, you can use Report Definer or $REPORT and bypass the rest of this chapter entirely.

$REPORT Code Generation

You can also generate a $REPORT definition into a SPIRES report format, by issuing the GENERATE FORMAT command when your $REPORT format is set on its selected subfile, and following the instructions given. The GENERATE FORMAT command creates a format record for you, using your $REPORT commands, and adds the record to the FORMATS subfile -- all you need do then is compile it. [Like the utilities discussed above, $REPORT code generation is discussed in Part C of "Searching and Updating".]

$REPORT definitions usually execute more efficiently in generated form, so even users who have no interest in the report format language should benefit from generating their $REPORT definition. In addition, a $REPORT-generated format provides a good starting point if you decide you want to customize the report format further.

A Report Format Coded From Scratch

You can also code a report format from scratch if you prefer, using the techniques described in the upcoming pages.

The rest of this chapter assumes that you want either to write your own report format from scratch or, preferably, to modify a format generated from $REPORT. In other words, this chapter is for developers who need a more customized format than even $REPORT provides.

B.10.1  The Fundamentals of Report Formats

A report format is basically an output format whose additional facilities (e.g., headers) you provide by coding some additional frames. When a report format is set and you have established "report mode" by issuing the SET REPORT command or Uproc (or by using the WITH REPORT prefix), [See B.10.2.] any commands causing multiple record output will produce more sophisticated results than in a standard output format.

Specifically, any frame whose frame-type is initial, group-start (summary), group-end, or ending, will be executed at the appropriate time. Also, if the command is prefixed with the IN ACTIVE or "IN file" prefix, header and footer frames will be executed, and automatic page control, via carriage control characters in column 1, will be provided.

All of these new frame-types are optional; you should use only those you need. Below is a brief description of each one:

The diagram below shows how these frames might appear in a typical report for five records:

                        Page 1
             +--------------------------+
             |                          |
             |          Initial         |
             |           Frame          |
             |                          |
             +--------------------------+

                        Page 2
             +--------------------------+
             |          Header          |
             |                          |
             | - - - - - - - - - - - - -|
             |        Group-Start       |
             | - - - - - - - - - - - - -|
             |      Data (Record 1)     |
             | - - - - - - - - - - - - -|
             |      Data (Record 2)     |
             | - - - - - - - - - - - - -|
             |         Group-End        |
             | - - - - - - - - - - - - -|
             |        Group-Start       |
             | - - - - - - - - - - - - -|
             |      Data (Record 3)     |
             |                          |
             | - - - - - - - - - - - - -|
             |          Footer          |
             +--------------------------+

                        Page 3
             +--------------------------+
             |          Header          |
             |                          |
             | - - - - - - - - - - - - -|
             |      Data (Record 4)     |
             | - - - - - - - - - - - - -|
             |         Group-End        |
             | - - - - - - - - - - - - -|
             |        Group-Start       |
             | - - - - - - - - - - - - -|
             |      Data (Record 5)     |
             | - - - - - - - - - - - - -|
             |         Group-End        |
             | - - - - - - - - - - - - -|
             |          Ending          |
             | - - - - - - - - - - - - -|
             |          Footer          |
             +--------------------------+

The diagram shows that the initial frame is put on a page by itself. The initial page is the only one that does not by default allow a header and/or footer frame on it.

The group-start and group-end frames are each shown here three times, presumably because in the five records, the first two have the same value for the break element of these frames, the next two have a different value and the last has yet a different value. The group-start frame is also executed before the first data frame, that is, when the first record is retrieved for processing. Similarly, the group-end frame is executed after the last record is processed.

Many system variables relate to report formats, from the flag variable $REPORT (which tells whether report processing is in effect) to the integer variable $LLEFT (which tells how many lines are remaining on the page). The values of some of these variables are maintained by SPIRES; others are set by you, often in the startup or initial frames. Most of the variables will be discussed as they are needed; all will be listed in a later section. [See B.10.10.]

In addition to system variables, almost all report formats employ user variables. If you are not familiar with their use in formats, please read the earlier chapter. [See B.9.3.]

Other tools useful for report formats, though not necessarily restricted to them, include subgoal processing and phantom structures [See B.12.] the FOR INDEX command and the $PATHKEY system variable [See the manual "Sequential Record Processing in SPIRES: Global FOR", section 2.14.] the SPISORT facility and element occurrence paths [See the manual "SPIRES Technical Notes", section 1.] and sort arrays. [See B.10.7.3, B.10.8.]

Those are the basic additional tools that may be useful in a report format. Section B.10.2 provides details on the SET REPORT command, but the remainder of the sections in this chapter discuss these tools in detail. A complete report format definition and some pages of sample output appear at the end of this chapter. [See B.10.11, B.10.12.]

B.10.1.1  Execution Order of Frames in a Report Format

"Timing problems" are probably the major concern in creating a report format -- the order in which the frames were shown in the diagram is not necessarily the order in which they would be executed. Timing problems may include frames splitting across pages when that is not desired, or counter variables that are not incremented properly because they are not coded in the right place. Understanding how SPIRES determines which frames should be executed when can be helpful in solving such problems.

Suppose that the report diagrammed in the last section is being produced. Five records will be processed, and they are sequenced by CITY. The first two records have the value ALTOONA, the second two have BILOXI, and the fifth has CHATTANOOGA. The records are in a stack, and the command IN ACTIVE TYPE has just been issued. Both the SET FORMAT and SET REPORT commands had already been issued.

First, assume the frames all have fixed frame dimensions. The first event triggered by the command is the execution of the initial frame. Besides creating title page type information, the initial frame in this example sets some system variables, including the number of lines per page, the number of lines that the header will need and the number of lines that should be reserved for the footer. [See B.10.4, B.10.5.] When the initial frame is finished executing, the buffer in which it was constructed is flushed; the output goes to the active file.

Although the first data page begins with a header, the header frame does not execute next, because no data for that page has been created yet. Instead, SPIRES accesses the first record, retrieving the CITY element to determine whether its value is different from that of the previous record. There was no previous record, but the retrieved value is different from no value, so the group-start frame executes next. (Note though that the group-start frames will always execute before the first record is processed, even if the retrieved break-element value is null.)

As it executes, SPIRES keeps track of the number of rows left on the page, lowering the number as the row represented by "X" (the "next row" or $LROW+1) comes down the page. Although the header frame has not been executed yet, SPIRES has subtracted the number of lines allowed for it (specified in the initial frame) from the number of lines left on the page so that it has an accurate reflection of the data lines left.

Because SPIRES has accessed the first record before executing the group-start frame, the group-start frame may retrieve element values from that record. For example, the CITY value could be retrieved so that the group-start frame could announce which city is represented by the following records. More than one group-start frame could execute at this point, depending on how many "break elements" and their respective group-start frames you have coded. [See B.10.7 for the order in which they would execute.]

After the first group-start frame is executed, SPIRES realizes that it cannot be put on the page until the header has been placed there, so execution of the header frame begins. (SPIRES can only place data at the end of the active file, so the header must be placed there before the group-start buffer.) A separate buffer is established for the header frame, and the header data is placed within it. When the header frame has executed, its buffer is flushed to the active file, followed by the group-start frame buffer. It is not common for SPIRES to have multiple format buffers in memory at once, but in this situation it is necessary.

(Why isn't the header executed before the group-start frame so that multiple buffers are not necessary? For one, the header frame may need data from the first record to display as a guide word at the top of the report page. If it is executed before the group-start or a data frame, no record would be available from which to provide such values to the format. More importantly though, a new header is only created when a buffer of data for the new page is waiting to be output.)

Next the data frame is executed, processing the first record. Presumably this format has only one data frame, but there could be more -- each is flushed to the active file when execution completes, while SPIRES keeps track of the number of lines left on the page. (See below for what happens when that number becomes 0.) At this point, the first data record has been completely processed.

The second record is retrieved, SPIRES notes that its CITY element has the same value as the previous record's, and the data frame is executed again. When the third record is retrieved, the CITY element is found to be different, which triggers the group-end, followed by the group-start frame again. Next the data frame is executed for the third record.

At some point, the end of the lines on the page that are available for data will be reached. This will happen when SPIRES is flushing the buffer after completing execution of a data, group-start or group-end frame. Either the buffer will just fit with no lines left over or the buffer will be too large and only part of it can be put on the page.

In the case in which it just fits, SPIRES will execute the footer frame and place it on the page. It will then retrieve the next record and continue almost as it did at the start of the first page: it will execute the data, group-start or group-end frame (whichever is next), but the header frame will not be executed until one of those frames is about to be put on the page.

In the case in which the buffer is too large, SPIRES will place as many rows of the buffer in the remainder of the page as possible, unless you have told SPIRES not to place a buffer at the bottom of a page when it will not completely fit there. [See B.10.3.4.] When as much has been placed there as possible, SPIRES executes the footer for the page; then, knowing that it has more data to put on the next page, SPIRES executes the header frame for the new page and then finishes emptying the buffer.

This procedure continues from one record to the next, from one page to the next. When the final record has been processed by the data frame, the group-end frame executes, followed by the ending frame. When the final ending frame is flushed to the page, followed by the final footer, the report processing ends.

If you press the ATTN/BREAK key during report format processing, SPIRES will stop processing records, but the next group-end, footer and ending frames will still be executed to finish the report.

The Effects of Line-by-Line Processing

Any or all of the report frames may be established as line-by-line except for headers and footers. [See B.3.3.] The procedure described above is the same for line-by-line frames except that the buffer is made up of only one row, which is flushed when data is placed in the next row, meaning that the buffer is flushed much more often. Also, footer and header frames may execute when only part of a data, group-start or group-end frame has been executed.

Line-by-line processing restricts your ability to have SPIRES prevent records from splitting across pages. By the time SPIRES knows a record is too large to fit, some of it is already on the page. Line-by-line processing also prevents you from holding groups of records in the buffer at one time to see whether the entire group will fit on the remainder of the page, if that is a consideration. [See B.10.3.7.]

B.10.2  Enabling Report Mode Processing (SET REPORT and WITH REPORT)

Report mode processing can occur only when report mode has been enabled. Report mode is enabled in one of the following ways:

The SET REPORT (SET REP) Command and the SET REPORT Uproc

The syntaxes of the SET REPORT command and SET REPORT Uproc are:

The command may be issued whenever a format is set, though it is most often issued after a format designed as a report format is set (see below). The Uproc may only be issued in a startup frame or in the format declaration section Uprocs associated with the startup frame. [See B.5.2, D.3.]

Report mode affects only those record output commands that can process multiple records: TYPE, OUTPUT, SCAN and some forms of DISPLAY under Global FOR (DISPLAY ALL, DISPLAY REST and "DISPLAY n", where "n" is greater than 1). Even if these commands only process one record, report mode will be in effect; the number of records that are actually processed is irrelevant. When the record-processing command is issued, the frames related to report processing will be executed appropriately, in addition to the data frames.

Other record output commands (other forms of the DISPLAY command, TRANSFER) are not affected by report mode -- only the data frames in the format will be executed for them.

If the format does not have output frames, the SET REPORT command will have no effect. If no format is set (i.e., the SPIRES standard format is in effect), the SET REPORT command will return the message NO FORMAT SET, and report mode will not be set.

To clear report mode, so that the multiple-record processing commands will not cause report mode processing, you can issue one of the following commands:

Note that the "SET FORMAT *" command, which causes the startup frame to be re-executed, will not clear report mode, though it may be reset if the SET REPORT Uproc appears within the startup frame.

The WITH REPORT [WIT REP] Prefix

As an alternative to using SET REPORT to turn on report mode processing for all subsequent record output commands, the WITH REPORT prefix may be added to a single command. WITH REPORT will cause frames related to report processing in the currently set format to be executed appropriately for the specified record-processing command or XEQ FRAME command only.

The WITH REPORT prefix may be added to the TYPE or DISPLAY command, or to the XEQ FRAME command, or to the GENERATE SET command, when a display set is being created. [See 1.9 in SPIRES Technical Notes for an explanation of display sets.] The WITH REPORT prefix may be used in conjunction with other command prefixes, such as IN ACTIVE. Commas may be used to separate prefixes, for clarity, if desired, e.g.:

Effect of Report Mode When No Report Frames Exist

What effect does report mode processing have when none of the special report frames exist? The primary result is the appearance of carriage control to cause page ejects when the output is directed to your active file (or some other file data area). The format data is shifted one column to the right (though you do not account for this shift in your data placement) and a "1" (one) is placed in column 1 in the first line and, by default, every sixtieth line thereafter. Nothing other than carriage control characters will be placed in column one. These characters will be used by the printer to cause a new page to start with that line if you include the CC option on the PRINT command. Another effect is that the "****" and ";" lines that normally appear between records in multiple-record displays are suppressed.

The number of lines that will be output per page is controlled by the system integer variable $LINEPP (lines per page). By default, its value is 60, the most common number of lines per page for printers. It can be set to a different value with the SET LINEPP command or Uproc.

where "n" is an integer from 0 to 32767. The Uproc is normally coded in the initial frame.

In most cases, the carriage control put out by default for non-report output formats is not very useful. If you are printing your data on one of the printers that by default prints 60 lines per page, the printer does not need carriage control every 60 lines to tell it to start a new page -- it would do that anyway. However, if you want to print a different number of lines per page, say, less than 60, it might be handy.

Note that the "1" carriage control symbol will be placed every $LINEPP lines, whether or not that line is in the middle of a record. That is, we have not yet told SPIRES that we do not want records split across pages. [See B.10.3.4 for information about SET NOBREAK.]

B.10.3  Controlling Page Layout for Reports

One of the most useful features of report formats is your ability to control not only the layout of numerous elements for a record but also the layout of numerous records on a page. When designing a report, most people begin with the design of the data frame or frames, keeping in mind how the design would look on a page of paper. Once that is done, consideration is given to the layout of an entire page:

In the previous section, you were introduced to the carriage control capabilities established by the SET REPORT command. [See B.10.2.] In this section and its subsections, that topic, as well as the others suggested by the above questions, will be discussed in detail.

B.10.3.1  Header Frames

A report format may have one or more header frames. A frame is identified as a header frame in the FRAME-TYPE statement in the format declaration section. Whenever SPIRES is ready to flush the frame's buffer and that buffer would be placed on a new page, any header frames are executed in order of declaration, with their output going to the top of the new page. (If part of the buffer would go on the previous page, then the footer frames, if any, would be executed first.) The exception to the rule is that no header is placed on the initial frame's page. [See B.10.4.]

The header frame usually places one or more of the following on the report page:

As with any frame, you must add code to two places in the format definition: 1) you must write a frame definition for the header frame; and 2) you must declare it a header frame to be used by the format in the format declaration section.

The second task is the easier -- you just add two statements in the appropriate format declaration section at the end of the format definition:

where "header.frame.name" is the name of the header frame as given in its FRAME-ID statement.

The frame definition of course depends on your needs. Its frame dimensions must be fixed -- they may not specify line-by-line processing. Note however that the number of rows specified will be placed on the page, whether or not you place data in the last ones. That is different from fixed-dimension data frames, whose last blank lines are not output. All of the report-specific frames (header, footer, initial, ending, group-start and group-end) are the same way. If you do not intend to put data in one or more of the last rows of the header frame, you can use the SET HDRLEN Uproc to prevent SPIRES from putting out the extra blank rows. [See E.2.2.5 for more details on its use.] Alternatively, you may use the SET NROWS Uproc to change the number of rows in the current frame, a technique that will work for the other report frame-types too. [See B.3.3.1.]

Some of the header functions listed above can be carried out by label groups such as those shown in the example below. For example, the values in the $UDATE (or $DATE) and $PAGENO variables contain the current date and page number respectively. Other values to be positioned in the header may be character strings, either literals or variables. In any case, they are generally placed into the frame by means of VALUE and PUTDATA statements. Other statements, such as START, MARGINS, TITLE, LOOP and UPROC, are also allowed.

A header frame may also contain GETELEM and IND-STRUCTURE statements, allowing access to elements within the record currently being processed (i.e., the record whose placement on the page causes the header frame to be executed). You could use this feature, for example, to retrieve element values for guide words that appear at the top of the page, like those in a dictionary or phone book. If there is no "current" record (for instance, during execution of an ending frame), then NODATA processing is in effect, meaning that any values specified with a VALUE, TITLE or DEFAULT statement will be displayed, and the $NODATA flag will be set.

Warning: There can be timing considerations that can cause problems if you use GETELEM within header frames. Be careful to honor any structure processing going on in the data frame at the time the header frame is executed, just as you would in the data frame itself. [See B.8.2.] For instance, if a structure is being processed by an indirect frame called by the data frame at the time the buffer is flushed to a new page, then using the GETELEM statement in the header frame to retrieve elements from another structure or from record-level may result in SPIRES losing its position within the processing of the data-frame structure. If this is a problem, you may want to use one of the $GETxVAL functions (like $GETXVAL) to retrieve the values in the header instead, or perhaps even better, do the GETELEM within the data frame and store it in a variable for later positioning in the header frame.

Note that you can also call indirect frames for structure processing or for subgoal processing if you want to access data records from a header frame. [See B.12.]

Below is a typical header frame definition. Parenthesized letters to the left refer to the notes that follow:

     FRAME-ID = HEADER;
       FRAME-DIM = 3,60;
       DIRECTION = OUTPUT;
 (A)   LABEL = GUIDE.WORD;
         GETELEM = TITLE;
         START = 1,1;
         LENGTH = 20;
         OUTPROC = $CAP;
         PUTDATA;
       LABEL = DATE;
         VALUE = $UDATE;
         START = 1,30;
         OUTPROC = $DATE.OUT(MONTH,,UPLOW,SQU);
         PUTDATA;
       LABEL = PAGE.NUMBER;
 (B)     VALUE = 'Page ' $PAGENO;
         START = 1;
         UPROC = SET ADJUST RIGHT;
         PUTDATA;
       LABEL = BORDER;
         VALUE = '-';
         START = 2,1;
         UPROC = SET REPEAT;
         PUTDATA;

That definition might produce a header that looked like this (row and column numbers are shown for clarity):

    (....v....1....v....2....v....3....v....4....v....5....v....6)
(1)  FLAPJACKS ON PARADE          Aug. 19, 1982          Page 114
(2)  ------------------------------------------------------------
(3)

Line 3 is output as part of the header frame even though no value was placed there.

Notes on the frame definition:

A) This label group shows the technique used when you want an element value from the first record on each page to be displayed in the header. The record being output to a new page, which causes the execution of the header frame, will be accessible by the GETELEM statement.

B) This demonstrates the point made earlier in the manual that VALUE statements must account for the variable type. If "Page" were put in an INSERT statement, then $PAGENO would need to be converted to string by the $STRING function; otherwise, that integer variable would be read as a string, and the proper conversion to string would not take place. As shown, the VALUE statement concatenates a literal string to $PAGENO, which forces $PAGENO to be converted to a string first. [See B.4.3.]

Below are some miscellaneous notes on headers:

Multiple header frames are allowed. They will be executed in the order in which they are declared in the format declaration section.

Often situations arise where you want some text to appear on the first data page of the report, before the data and possibly before the header for that page. One possible solution to this problem is to process that text in a separate header frame that is only executed for the first record. The format declaration section might contain a Uproc such as the following for that frame:

In other words, if the first record of the report is not being processed, skip this frame. Setting $SKIPF is the most obvious way to prevent a header frame from being placed on the page if one is defined. [See E.2.1.17 for another way, using the SET SUPPRESS Uproc.] If you do not use the SET SKIPF or SET SUPPRESS technique, the buffer established for the frame by the frame dimensions would be placed on the page, even if no data would be positioned in the header frame. [See E.2.1.3, E.2.1.16.]

Whenever the flushing of the buffer would cause data to be placed on a new page, the header frames are executed. If the data frame is doing line-by-line processing or if it has fixed dimensions but the Uproc SET NOBREAK has not been set (meaning that the frame is allowed to split across pages), then the record may continue after the header on the top of the next page. Be aware that if you retrieve an element value to place in the header, it will come from that record (the continuing one) rather than from the first complete record on the page.

B.10.3.2  Footer Frames

At the other end of the page from the header frame(s) can come the footer frame(s). Footer frames are used for similar purposes as headers. [See B.10.3.1.] They are handled similarly as well -- you must write a frame definition for them and you must add their names, along with the proper FRAME-TYPE statement, to the format declaration section. However, the most important difference (besides their opposite orientations) is the importance of the variable $FTRLEN (for "footer length"), as opposed to the relative insignificance of $HDRLEN.

Like a header frame, a footer frame is identified as such in the format declaration section:

Multiple footer frames are allowed; they will be executed in the order in which they are declared here.

The frame definition itself might look very similar to the one shown for a header in the previous section. One possible difference might be that the border row of hyphens would be placed before the row of information, rather than after it as it appears in the sample header frame.

When constructing a page, SPIRES needs to know the number of lines to reserve for the footer frame or frames. Thus you must issue the SET FTRLEN Uproc, usually in the initial frame, to tell SPIRES how many lines to save:

That Uproc would be used if the footer frame would require three lines, or if three different footer frames required a total of three lines. The value of $FTRLEN should reflect the total number of lines needed for all footer frames. [See E.2.2.6 for more about $FTRLEN.]

The frame dimensions of a footer frame may not specify line-by-line processing. The number of rows specified in the FRAME-DIM statement will be placed on the page, whether or not data has been placed in the last ones. All the report-specific frame-types work this way. However, you can use the SET NROWS Uproc to delete the last unused rows of a report-specific frame-type, if desired. [See B.3.3.1.]

Values are generally placed in a footer frame with the VALUE and PUTDATA statements, with assistance from START, MARGINS, TITLE, and other statements. Just as they are in header frames, GETELEM statements are also allowed, and subgoal access to other records and record-types is allowed through indirect frames. [See B.12.]

Element values to be displayed in the footer are best retrieved in a data frame and saved in a variable to be used later by the footer frame. The GETELEM statement, used successfully in header frames, may not work as well here, since the record being processed when the footer is executed may be starting on the top of the next page.

B.10.3.3  Carriage Control in Report Formats

A major part of report mode processing is the creation of carriage control in column 1 of the output when the output is directed to the active file (or some other file data area). Rudimentary carriage control is provided in report mode by default and was discussed earlier. [See B.4.2.] However, SPIRES provides you with several options in carriage control, giving you much more flexibility than the default of starting a new page every sixty lines. Those options will be discussed in this and the next few sections. [See B.10.3.4, B.10.3.5, B.10.3.6, B.10.3.7.]

Several important system variables will be mentioned throughout these sections, in particular $LINEPP and $LLEFT. The variable $LINEPP represents the number of lines allowed per page of printed output; the default is 60. The other variable, $LLEFT, represents the numbers of lines left on the page that can be used for output; its value is derived by subtracting the number of lines written on the page from $LINEPP. It does not include the current row. Thus, if $LLEFT is 1, then one more row past the current one can be used for output. [See E.2.2.3.] Whether report mode processing will create carriage control at all depends on the value of the integer variable $PAGECTL ("page control"). Though any of four values may be specified, the two discussed here are 0 (the default when data goes to the active file) and 4.

When $PAGECTL is 0, carriage control symbols are placed in column one of the data; the formatted data is shifted to the right one column. Header and footer frames, if defined and declared, will be executed and positioned on the pages as appropriate. New pages will be triggered by a "1" in column 1. The other carriage control symbol used by SPIRES is "-" (hyphen) to cause triple spacing.

When $PAGECTL is 4, no carriage control symbols are output automatically. No right-shifting of your data takes place. Header and footer frames, if defined and declared for the format, will be executed and positioned on the pages as appropriate, using the values in the variables $LINEPP, $HDRLEN and $FTRLEN to compute their positions. Extra blank lines, rather than hyphens in column 1, will be used to ensure that the footer appears in the right place.

There are two reasons why you might want to use $PAGECTL = 4. First, you might not want any carriage control at all, and possibly you do not have header or footer frames either. However, you want to take advantage of other report mode features, such as group-start frames, without having to have a reserved column for carriage control.

Second, you might want to create your own carriage control, reserving the first column of your frames for carriage control symbols you place there with VALUE or TITLE statements in your label groups. This also allows you to use some of the other symbols, such as a "+" for overstrike, or "8" for the bottom of the current page. You could use the "+" symbol with underscore characters to underline values, for example. Warning: The variable $LLEFT will not be aware of any carriage control you generate and will be counting each line of output as producing a single line. This may cause problems if you also have headers and footers that SPIRES is trying to place in the proper page positions. Some of these problems may be solved by increasing $LINEPP accordingly. Be sure, if you do that, to restore $LINEPP to its original value in the header frame.

Further details on $PAGECTL, including its effect on initial frames and its other values, appear elsewhere. [See E.2.2.2.]

B.10.3.4  The SET NOBREAK Uproc

The SET NOBREAK Uproc can be issued at any point within a fixed-dimension frame to tell SPIRES not to split the frame across page boundaries. Specifically, when the $NOBREAK flag is set, the buffer currently being filled will not be placed on the current page unless enough room for it is available there, according to the value of $LLEFT. Instead, the remainder of the page (except for the footer frame, which is then executed and placed there) will be left blank, and the contents of the buffer will follow the header at the top of the next page.

The SET NOBREAK Uproc takes no value or options:

This Uproc sets the $NOBREAK flag variable. [See E.2.1.2.] It is often placed in the frame declaration in the format declaration section, though it can be placed anywhere in a frame. It applies only to the current buffer being written. For example, if it is set for a data frame, it will not be in effect for any other data frames in the format, unless it is set for them individually.

Setting $NOBREAK will not guarantee that the entire frame will be placed on a single page, merely that if it cannot fit on the remainder of the current page, it will start at the top of the next one. If the frame is too long for the next page, it will break onto the following page. To prevent a frame from being too large to fit on a single page, you need to keep the number of rows low in the FRAME-DIM statement. Note however that a serious error (S808) will occur if SPIRES tries to put data in rows beyond those of the assigned frame.

Another Uproc, HOLD, can be used to keep several frames together from splitting across page boundaries. For example, if your format has multiple data frames, and you do not want to allow a page break between frames, you must use the HOLD Uproc. [See B.10.3.7.] If you want to allow your data frame to split across pages but only at certain rows, define it as several data frames instead, with each one having $NOBREAK set.

B.10.3.5  The EJECT PAGE, SET NEWPAGE and SET FRONTPAGE Uprocs

The EJECT PAGE and SET NEWPAGE Uprocs give you even more control over paging than SET NOBREAK. [See B.10.3.4.] The SET NEWPAGE Uproc tells SPIRES to place the buffer currently being written onto a new page when it is output. What has previously been written into the buffer together with what will be written into it before it is output will be placed on the next page.

The EJECT PAGE Uproc operates somewhat differently. What has already been placed in the buffer previous to the EJECT PAGE Uproc is flushed, being placed on the current page. Any footer frames would then be executed. The $NEWPAGE variable is automatically set and the remainder of the frame is then executed; data resumes being placed in the now empty buffer. When that buffer is output, it will appear on a new page. Note that when the EJECT PAGE Uproc is executed, the header frames are not -- it is the second writing of the buffer, which will appear on the new page, that triggers them.

The SET FRONTPAGE Uproc specifies that not only do you want a new page, but that the output should print on the front of the paper. When $FRONTPAGE is on and a page eject occurs, the "1" carriage control character is replaced with an "F" and $FRONTPAGE is turned off. SET FRONTPAGE is used in conjunction with (and not instead of) SET NEWPAGE and/or EJECT PAGE Uprocs.

The "F" carriage control character has an effect when you are printing in duplex mode on the 4090 printer in Forsythe. SET FRONTPAGE may be abbreviated to SET FRONT or SET FPAGE, and you can turn $FRONTPAGE off explicitly with a SET NOFRONTPAGE Uproc.

None of these Uprocs has any options:

Beware of using absolute row numbers in frames with EJECT PAGE Uprocs. If a START statement before the EJECT PAGE Uproc places data in row 9, and a START statement after it places data in row 10, the new page will begin with nine rows of blanks after the header. This happens because the row 10 data goes into row 10 of the empty buffer; there are no longer nine rows of data ahead of it, but instead nine rows of blanks. In this situation, it is better to place values using "X" and "*" for row numbers; after flushing, $CROW is no longer 9 but is now 1.

Note that the $NEWPAGE flag is set in three ways: 1) by SPIRES during processing of the first record, i.e., when $FREC is set; 2) by SPIRES when the EJECT PAGE Uproc is encountered; and 3) by you when a SET NEWPAGE Uproc is explicitly encountered.

B.10.3.6  The FLUSH Uproc

The FLUSH Uproc causes the buffer to be sent immediately to the appropriate device (usually the terminal or active file). The buffer is then emptied and subsequent label groups can resume placing data in it. The same warning given about absolute row numbers for the EJECT PAGE Uproc applies to the FLUSH Uproc. [See B.10.3.5.]

In most circumstances, SPIRES automatically flushes the buffer. For instance, the buffer is flushed whenever a frame finishes executing (unless it is an indirect or XEQ frame). So only a few circumstances require the FLUSH Uproc.

One situation is when the HOLD Uproc is used to hold a group of formatted records in the buffer to be placed on the page as a group. The HOLD Uproc tells SPIRES not to flush the buffer when a new frame begins but to wait until the FLUSH Uproc is given. [See B.10.3.7.]

The syntax of the FLUSH Uproc is quite simple:

B.10.3.7  The HOLD Uproc

The SET NOBREAK Uproc tells SPIRES to place the current buffer onto the current page only if there is room for all of it there; otherwise, it is to be placed on a new page. The HOLD Uproc lets you extend this principle to groups of records.

When the HOLD Uproc is in effect, the current buffer is not flushed to the page when SPIRES finishes processing a data frame; instead, the buffer is "held" in memory, and the next data frame is processed (perhaps for the next record), with its data following that of the previous data frame in the buffer. At some point the buffer must be flushed (this may be accomplished in several ways), but you can determine whether the buffer can fit onto the remainder of the given page and, if it will not, you can set the $NEWPAGE flag to force the entire buffer to be placed on a new page.

The most direct way to flush the buffer is to issue the FLUSH Uproc. [See B.10.3.6.] The current buffer is immediately flushed to the page, and execution of the frame continues with the "holding" resumed.

The HOLD Uproc affects all initial, data, group-start and group-end frames that begin execution after the HOLD Uproc is executed. If it is executed within a frame definition, it will not affect that frame.

The HOLD Uproc is "turned off" and the buffer is flushed just before the ending frame begins executing -- in other words, all frames executed after the HOLD Uproc is executed and before the ending frame is executed are affected by HOLD.

The HOLD Uproc does not have any options:

The HOLD Uproc may only be used with frames of fixed dimensions. However, the number of rows specified in the FRAME-DIM statement should be high enough to allow for the maximum number of records you expect to be held, and the row positions in START statements should be specified in relative positions ("*" and X values) rather than absolute ones (such as 5 or 10). Because the buffer is not flushed between records, row 1 for the first record is the same row as row 1 for the second record. Coding relative values for starting positions will prevent overlaying records on each other.

If the HOLD Uproc will affect multiple frames (e.g., data and group-start and group-end frames), the FRAME-DIM statement of only the first frame executed after the HOLD statement appears will apply. The other frames must have FRAME-DIM statements, but the particular values will be ignored (see the example below).

Several different events may cause the buffer to be flushed:

After the buffer is flushed, HOLD processing automatically resumes.

Here is the type of situation in which HOLD processing is most useful: Suppose you have a stack of personnel records sequenced by DEPARTMENT. You want to get as many records on a page as possible, but you want all of the records for any given department on a single page. In other words, a page may have records for two or more departments, but all records for those departments should appear on that page. This could be accomplished with group-start and/or group-end frames [See B.10.7.] and the HOLD and FLUSH Uprocs.

At least three frames would be held in the buffer: the group-start frame, the group-end frame and one or more data frames for the records. Because the group-start frame executes before any of the data frames when a new DEPARTMENT value is detected, that frame definition contains the large frame dimensions:

The FRAME-DIM statements of the other frames are ignored by SPIRES when a HOLD Uproc is in effect, so their values are not important, but they must not be omitted.

The HOLD Uproc will apply to the next frame executed, not the current one, so it may be added either to the initial frame or to the group-start frame declaration in the format declaration section:

Thus, HOLD gets set right before the group-start frame is executed.

The buffer should be flushed before the group-start frame is executed again, probably at the end of the group-end frame:

Though you might have considered placing the FLUSH Uproc just ahead of the HOLD Uproc in the group-start frame declaration, FLUSH is not allowed there; it must be within a frame definition.

The only other requirement is to tell SPIRES to begin the buffer on a new page if it is too large to fit in the rest of the current page; this is best done by setting the $NOBREAK flag.

Of course, this implementation does not guarantee that all records for one department will fit on one page. However, records for such departments will each begin on a new page. The point of this design is to prevent a group of records from starting at the bottom of a page (i.e., after other groups of records) if they will not all fit on that page.

The HOLD Uproc is also useful in input formats for processing multiple records in SPIBILD. [See C.9.1.]

B.10.3.7a  (*) The SET NOPUTFLAG Uproc and The $PUTFLAG Variable

A report format (and sometimes a display format) occasionally needs to know if any data has been output beginning at a certain point in a record. For instance, a format may want to output a blank line before and after a group of fields, but want to avoid putting out two blank lines if no fields in the group occur.

The Uproc SET NOPUTFLAG and the variable $PUTFLAG together help you determine whether data has been output in a particular frame, beginning at a point in the frame that you yourself specify.

$PUTFLAG is set to $FALSE at the beginning of any frame (except a header frame, an indirect frame, or an overflow frame in Prism). The variable is set to true as soon as any PUTDATA statement or TITLE statement executes within the frame. Equally importantly, you can set its value to $FALSE yourself by issuing the SET NOPUTFLAG Uproc.

Thus, to check a group of fields to see whether any PUTDATA statements were executed for the group, you can code SET NOPUTFLAG before beginning the group and check the value of $PUTFLAG at the end of the group. (This "group of fields" is often coded as an indirect frame, [See B.8.1.] which is why the value of $PUTFLAG is not reset at the beginning of an indirect frame.)

Another use of the SET NOPUTFLAG/$PUTFLAG duo is to communicate to a header frame (or an overflow frame in Prism) as to whether data has been output beginning from some specified point. For instance, consider a bibliographic report where author, title, and publisher are treated as a unified data segment. You don't care if part of this segment is output on a page following the rest of the segment, but you do want your header to know, when it executes, whether the segment is beginning or is continuing from a previous page.

To declare that the data segment is just beginning (and that no part of it has been output), you can set $PUTFLAG to false with the SET NOPUTFLAG Uproc, coding it to execute just before the series of label groups defining your segment of data will begin to execute. If the header or overflow frame executes after SET NOPUTFLAG and before the label groups in the segment, it will know from $PUTFLAG that the segment has not yet begun.

For instance:

  FRAME-ID = AUTHORTITLE;
      :
    LABEL = PRE.SEGMENT;
      UPROC = Set NoPutFlag;   <--At this point, $PUTFLAG is false
    LABEL = AUTHOR;
        :
      PUTDATA;                 <--After this executes, $PUTFLAG is true
    LABEL = TITLE;
        :
    LABEL = PUBLISHER;
       etc.

As always in reports, the timing of your test can be crucial. For instance, TITLE and PUTDATA statements within a header or overflow frame will reset $PUTFLAG to be true.

B.10.3.8  Multiple Column Output

SPIRES allows you to have reports with multiple columns of data on a single page, like a dictionary or a magazine. Several records may appear in the first column, a few more in the next, and so forth. (Warning: be aware of the terminology distinction between multiple columns of data on a page and columns within a frame. The intended meaning of the word at any given point in this section should be clear by the context.)

The number of columns to appear on the page is controlled by the $COLUMNS variable. To set the number of columns on the page, code the SET COLUMNS Uproc in the initial frame:

where "n" is an integer from 1 up.

Think of the pages of a report with a single, narrow column of data (without headers or footers) running end to end. The FRAME-DIM statement might look like this:

meaning it is a line-by-line frame with thirty columns per row.

Next, shift the pages so that some are side by side rather than end to end, using the number in $COLUMNS to determine how many to place next to each other. Then add a header and footer stretching across all the columns at the top and bottom, and you have a simple idea of how SPIRES constructs a page of multiple columns.

Actually, when multiple column processing is requested, SPIRES creates a special buffer for the page image. The regular buffer in which the frame data is placed is in turn placed in the page buffer. SPIRES determines the size of the page buffer by multiplying the number of columns requested ($COLUMNS) by the assigned width of each column ($COLWDTH). Generally you should set $COLWDTH to be slightly more than the number of columns in the FRAME-DIM statement, so that a good margin between the columns of data will be left.

where "n" is an integer.

Both the SET COLWDTH and the SET COLUMNS Uprocs should appear prior to the entry into the frames to which they apply. Note that you cannot have separate header and footer frames for each column, though you may design your headers and footers so that particular data appears at the top or bottom of each column.

When SPIRES is placing data into the page buffer, it keeps track of $LLEFT as before. When no more lines are left, it begins the next column, rather than the next page. If it finishes the last column on the page, the footer for the entire page is executed, followed by the header for the next page, and then it begins placing data in the first column of the next page.

Just as you can keep records from breaking across pages, you can prevent them from breaking across columns by using the SET NOBREAK Uproc. [See B.10.3.4.] You may also use the SET NEWCOL Uproc, which is similar to the SET NEWPAGE Uproc, causing the current buffer to be placed in the next column. [See B.10.3.5.] Also, you may use the EJECT COLUMN Uproc to cause SPIRES to flush the buffer into the page buffer immediately and set the $NEWCOL flag, which is similar to the effect of the EJECT PAGE Uproc. [See B.10.3.5.]

When multiple column processing is set (i.e., when $COLUMNS is greater than 1), it will apply to group-start, group-end and ending frames as well. You can turn it off for them by changing the value of $COLUMNS before entering them. Executing an EJECT PAGE Uproc at that point is probably advisable too.

Note: Though we are generally reluctant to say that any given task is impossible to do in SPIRES, there is apparently no general way to balance the columns on the final page of data so that all the columns end on the same row in the middle of the page.

B.10.4  Using the Initial Frame to Provide Prefatory Material

As soon as a multiple-record output command is issued when report mode is set, the initial frame is executed. The initial frame generally has two purposes: as a grid containing data to be placed on a page, it is used to provide introductory material for the report, such as a cover sheet or preface; as a frame to be executed, it is used to initialize or reset variable values for report processing. The second purpose is discussed later. [See B.10.5.]

An initial frame is identified as such in the frame declaration part of the format declaration section, using the INITIAL value for the FRAME-TYPE statement:

Multiple initial frames may be specified. They will be executed in the order in which they are specified in the format declaration section.

The frame definition for an initial frame usually contains label groups using VALUE statements to assign a value for placement within the frame. For example, a simple initial frame definition might look like this:

That definition would put the title on line 10, followed by the date on line 12, with both values centered.

GETELEM statements are not allowed in initial frames. However, an initial frame may access elements in records via an indirect frame, using subgoal processing. [See B.12.]

If the initial frame has fixed frame dimensions, the number of rows specified in the FRAME-DIM statement will be placed on the page, whether or not data has been placed in the last ones. All the report-specific frame-types work this way. However, you can use the SET NROWS Uproc to delete the last unused rows of a report-specific frame-type, if desired. [See B.3.3.1.]

Pages produced by initial frames will not have headers or footers provided for them. Headers and footers are not generated until data frames begin executing. If you want headers and footers on the initial page, you will have to create them yourself in the initial frame, or have the prefatory material appear on the first page of data (see below), or call the header and footer frames as indirect frames from the initial frames. (They would also need to be declared indirect frames in the format declaration section.)

By default, the carriage control provided by SPIRES puts each initial frame on a page itself -- the data frames begin on the next page. To handle the carriage control yourself, you must change the value of the $PAGECTL variable. [See E.2.2.2.]

If you want to put prefatory material on the same page as the first page of data, you could have label groups to handle that material in a group-start frame or in the data frame, but you would tell SPIRES to skip those label groups unless $FREC (the flag variable set when the first data record is being processed) is set. [See B.10.3.1.]

B.10.5  Using the Initial Frame to Initialize System Variables

Besides its use for prefatory materials, the initial frame can be used to set system and user variables to particular values before the report processing begins. Setting the page height ($LINEPP) and telling SPIRES the number of lines to reserve for the header or footer ($HDRLEN, $FTRLEN) are typically done using the initial frame. User variables that will be used in other frames may also be re-initialized by the initial frame. [See B.10.6, B.10.7.2 for information on variables in group-start, group-end and ending frames.]

Generally the Uprocs used to set such variables are placed in the format declaration section rather than in the initial frame definition, since they often do not relate to initial-frame data placement at all. In fact, the use of the initial frame to initialize variables is more common than its use to display prefatory material, so SPIRES lets you request initial frame processing without your having to define an initial frame:

No initial frame is named in the FRAME-NAME statement, so no initial frame needs to be defined. However, the Uprocs will be executed right before the point where an initial frame would be executed during format processing.

It might seem strange to re-initialize user variables in an initial frame when presumably they have not even been used, but it may be necessary for several reasons. First of all, you may indeed have already used them, i.e., you have already used the format to produce a report and are now producing a different one without resetting the format. The user variables in the vgroup may still have the values they had at the end of the last report; a counter variable might now be set to 35 instead of 0, for instance. The initial frame, which is executed whenever a multiple-record processing command is issued, could thus reset the variables.

Second, you might be using a global vgroup whose variables have pre-assigned values. The initial frame could reset some or all of them, if desired. Setting and resetting vgroup variables can be done with several functions, $VGROUPINIT, $ASET and $DYNASET, which are described in the manual "SPIRES Protocols".

Using the startup frame to initialize system variables is not generally recommended for two reasons. First, since it is only executed when the SET FORMAT command is issued, variables will not be reset if several reports are created without resetting the format each time. Second, the user may change the values between the time the format is set and the time it is executed, which would probably defeat the purpose of the format initializing them. [See D.3.]

B.10.6  The Ending Frame

Ending frames are most commonly used to present information gathered during report processing, such as a count of the number of records displayed or the average value of an element across all the records. They may also be used, of course, to present ending material such as indexes or footnotes or "legends" that explain data values in the records.

A frame is declared an ending frame in the format declaration section, using the FRAME-TYPE statement:

Multiple ending frames are allowed; they will be executed in the order in which they are declared here.

Unlike initial frames, ending frames are not by default placed on their own pages -- they follow the final record of the data, beginning on the same page (if there is room, of course). An EJECT PAGE Uproc can be added to the frame declaration section if you want the ending frame to begin on its own page. That page will also have headers and footers, however. If you want those suppressed, set a user flag variable in the ending frame, and check that flag before entering the header and footer frames, i.e., in the frame declaration section for those frames. Then, if the flag is set, execute the SET SKIPF Uproc. [See E.2.1.16.]

Like the initial frame, the ending frame may not include GETELEM statements, although subgoal processing through indirect frames is allowed. [See B.12.]

If the ending frame has fixed frame dimensions, the number of rows specified in the FRAME-DIM statement will be placed on the page, whether or not data has been placed in the last ones. However, you can add the SET NROWS Uproc to delete the last unused rows of an ending frame, if desired. [See B.3.3.1.]

Statistics based on the data presented during the report (such as index information) are usually collected in user variables during report processing, or in the $SORTKEY and $SORTDATA system variables. [See B.10.8.] SPIRES does not automatically keep statistical information about the data in the report, the only notable exception being the variable $RECNO, in which SPIRES keeps a count of the records processed since the multiple-record output command was issued. [See E.2.2.1.]

So, for example, if the data records represent families and you want to know the total number of children for all the records in the report, you might establish an integer variable called TOTALCHILDREN. First, you would define them in the Vgroup section of the format. [See B.9.3.1.]

Then, you might set TOTALCHILDREN to 0 in the initial frame, probably at its declaration in the format declaration section:

Then, in the data frame, you would increment the variable for each child. How this is done would depend on the children's element. For example, if the element CHILDREN were an integer value representing the number of children in the family, the label group in the data frame might look like this:

On the other hand, if CHILD were a multiply occurring element whose values were the names of the children, the label group might look like this:

Each time the label group is executed and another child is processed, the counter variable is incremented by one. If you were not displaying the element, the label group could be much simpler:

where the $ELOCC variable represents the number of occurrences of the accessed element.

The ending frame would report the total number of children. Other computations to that value could be made in that frame as well:

The example shows how the average number of children per record processed might be computed in the ending frame. Note that the TOTALCHILDREN value was converted to a packed decimal variable to provide greater accuracy in the division for the average than the division of two integers would provide.

B.10.7  The Break Frame-Types: Group-Start and Group-End Frames

Group-start and group-end frames ("break frames") give you the capability of providing summary information about groups of records within the set of records processed in the report. For comparative purposes, if you think of an initial frame as being useful for providing introductory information about the report that follows, a group-start frame is useful for introductory information about the group of records that follows. Similarly, if you think of an ending frame as being useful for report totals and other information about the preceding report, a group-end frame is useful for subtotals and other information about the preceding group of records in the report.

After arranging the records for your report in order by some element (using the SEQUENCE command, for example), you can tell SPIRES to execute a pair of group-start and group-end frames whenever the element value changes from one record to another (an event called a "summary break" or simply a "break"). Thus, if the records were sequenced by element SEX, a group-end frame could be executed when that value changed, reporting the total number of women's records just processed, followed by a group-start frame announcing that the next group of records in the report will be about men.

Specifically, the group-start frame is executed before each new group of records begins processing, including the very first group of records (i.e., right after the initial frame is executed). It can retrieve element values from the first record in each group, a feature which is most often used to retrieve the new value of the break element, so it can be displayed at the beginning of the group of records. (Note: the obsolete frame-type "summary" is equivalent to "group-start".)

The group-end frame is executed after each group of records has been processed, including the very last group (i.e., just before the ending frame is executed). It can retrieve element values from the last record in each group, a feature which is most often used to retrieve the "old value" of the break element, so it can be displayed at the end of the group. Like the ending frame, a group-end frame is often used to display totals, counts, averages or other statistics gathered during record processing. Variables containing such values are often reset to zero in the group-end frame after they have been placed in the buffer so that the gathering of statistics for the next group will start properly.

As you can see, both a group-end and a group-start frame may execute between two groups of records. If you created a blood donors report where the break element was SEX, the report might look like this between the two groups (the frame-types are indicated at the right):

                    ...
   Martha Washington   Menlo Park      5          <-- Data frames
   Edith Wilson        Palo Alto       8              for FEMALE
   Ellen Wilson        Menlo Park     13
   ** Number of Women Donors: 35                  <-- Group-End
   ** Average number of pints donated: 3.54
   ----------------------------------------
   Blood Donor Records for Men:                   <-- Group-Start

   John Adams          Sunnyvale       5          <-- Data frames
   John Q. Adams       San Mateo       1              for MALE
                    ...

That example shows a fairly typical use of these two frame-types.

The group-start and group-end frames are generally used in pairs, though they do not have to be. You may not need to display data at the end of a group, for example, so you may decide not to use group-end frames. If you do use both, you should be sure to specify the same break element and the same "break level" for both, so that they will be handled as a pair. [See B.10.7.1.] But even if you do not use both, it is useful to think of them in pairs, especially if you request multiple break frames.

You can request multiple group-start and/or group-end frames for multiple break elements. For example, if you have sequenced the records by two elements (for example, SEQUENCE SEX CITY), you can request a group-end and a group-start frame be executed when the CITY value changes and another pair when the SEX value changes. When SEX changes, both pairs will be executed -- since a group of records for a particular sex is then broken into subgroups by city, SPIRES assumes that when the SEX element value changes, a pair of frames for CITY is desired (even if the CITY value does not actually from one record to the next).

For instance, if the example report above also included break frames for the element CITY, the report might look like this at the point where the SEX element changed:

                    ...
   Claudia Johnson     Palo Alto       2          <-- Data frames
   Edith Wilson        Palo Alto       8

   * Number of Women Donors from Palo Alto: 5     <-- Group-End
   * Average number of pints donated: 3.9             for CITY

   ** Number of Women Donors: 35                  <-- Group-End
   ** Average number of pints donated: 3.54           for SEX
   -------------------------------------------
   Blood Donor Records for Men:                   <-- Group-Start
                                                      for SEX
   * in Atherton:                                 <-- Group-Start
                                                      for CITY
   Chester Arthur      Atherton        5          <-- Data frames
   James Garfield      Atherton        6

   * Number of Men Donors from Atherton: 2        <-- Group-End
   * Average number of pints donated: 5.5             for CITY

   * in Burlingame:                               <-- Group-Start
                                                      for CITY
   Bill Taft           Burlingame      2          <-- Data frames
                    ...

Note the nesting of the break frames as shown in the example: pairs of break frames for CITY are surrounded by pairs of break frames for SEX. The way to request such a hierarchy will be discussed in the next section.

Remember that the format will not arrange the records in the order shown -- they must have already been arranged in order, by a command such as SEQUENCE SEX CITY.

Group-start and group-end frames, like all other frames, require code in the format definition in two separate places: they must be defined in the frame definitions section and declared in the format declaration section. In the next section, the statements in the format declaration section will be discussed, followed by a section on break frame definitions. [See B.10.7.1, B.10.7.2.] After that will be a brief discussion of sorting techniques that can be used to arrange the records for processing, including information on the SPISORT program and the path information it provides. [See B.10.7.3.]

B.10.7.1  The BREAK-ELEM (BRK) and BREAK-LEVEL (LVL) Statements

In the format declaration section, each break frame (at least one for each break element) is declared individually. For each break frame, two statements in addition to the FRAME-NAME and FRAME-TYPE statement, must be added:

The BREAK-ELEM statement names the element in the record whose value is to be monitored as records are processed. When that element value changes, the named break frame is executed. The named element may be a virtual element. (For an element within a phantom structure, use the "#variable" form shown below, setting the variable to the name of the phantom element.)

Several different forms besides the one shown above are allowed:

The first form may be used if the name of the break element will be contained in the named string variable. If the variable is a four-byte hex variable, it may contain the element's $ELEMID rather than its name. [See E.2.3.10.]

The second form should be used if the element name is not unique in the record-type. It gives the hierarchical path to the right element.

The third form names a variable whose value is examined between records; if it changes, it triggers the execution of the summary frame. This provides a method for creating summaries when the controlling value is not an element in the records. Note that the pound sign (for user variables) or the dollar sign (for system variables) should precede the variable name.

The fourth form, which was often used when the SPISORT program sorted the goal records, is now obsolete in most situations where it was used. That is, under FOR SET processing, if the named break element is a multiply occurring element and the DEFINE SET command that created the set noted that the element was multiply occurring, then the appropriate occurrence of the break element will automatically be used in break frame processing. [See B.10.7.3.]

The BREAK-LEVEL statement helps SPIRES determine which break frames should be executed. The value given, an integer from 1 to 10, generally reflects the position of the break element within the SEQUENCE or DEFINE SET command. For example, if the records have been arranged by the command "SEQUENCE A B C D E", then the break frames whose break element is A would be assigned a break level of "1" (the highest break level), the frames for B would have "2", etc., with E having "5" (the lowest level). Each pair of group-start and group-end frames should specify the same break level. [See B.10.7.]

When a break occurs at a given level, all the break frames at that level and at lower (higher-numbered) levels are executed. For example, if a break occurs at level 2, group-start and group-end frames for levels 2, 3, 4 and 5 are executed, the group-end frames first, and then the group-start frames. Note however, that the group-end frames and then the group-start frames are executed in the order in which they are declared in the frame declaration section, not in break level order. Hence, most formats using this feature declare group-end frames in "5 4 3 2 1" break-level order, so that the lower-level group-end frames will execute before the higher ones when multiple ones will be executed. On the other hand, group-start frames are usually declared in "1 2 3" order.

For example, here is part of the frame declaration section for the format used in the example at the end of the last section. [See B.10.7.] The records were arranged by the command SEQUENCE SEX CITY.

Here, the break frames of level 2 are specified before those of level 1 so that they will be executed first when a break occurs at level 1.

In some situations, you may want to use both group-start and group-end frames but not have a pair of them for each break level. Suppose, for example, that a report about students has student records sequenced by DEPARTMENT, DEGREE.PROGRAM, and GRAD.YEAR, and that group-start frames are defined for all three break elements but a group-end frame is defined for only GRAD.YEAR. What happens when, between two records, the DEPARTMENT and/or DEGREE.PROGRAM changes but GRAD.YEAR does not?

First, SPIRES would check to see whether it should execute the group-end frame for GRAD.YEAR. But the only basis SPIRES has for making that decision is whether the value of GRAD.YEAR changed from one record to the next. Since it did not change, the group-end frame would not be executed. Clearly, however, you would want it to be executed, since a higher-level break is occurring for the group-start frames.

To ensure that execution, you must tell SPIRES that there are higher break levels for the group-end frame too:

     FORMAT-NAME = STUDENT SUMMARY;
       ...
       FRAME-NAME = GRAD.YEAR.END;
         FRAME-TYPE = GROUP-END;
           BREAK-LEVEL = 3; BREAK-ELEM = GRAD.YEAR;
  *    FRAME-NAME;
  *      FRAME-TYPE = GROUP-END;
  *        BREAK-LEVEL = 2; BREAK-ELEM = DEGREE.PROGRAM;
  *    FRAME-NAME;
  *      FRAME-TYPE = GROUP-END;
  *        BREAK-LEVEL = 1; BREAK-ELEM = DEPARTMENT;
       FRAME-NAME = DEPARTMENT.START;
         FRAME-TYPE = GROUP-START;
           BREAK-LEVEL = 1; BREAK-ELEM = DEPARTMENT;
       FRAME-NAME = DEGREE.START;
         FRAME-TYPE = GROUP-START;
           BREAK-LEVEL = 2; BREAK-ELEM = DEGREE.PROGRAM;
       FRAME-NAME = GRAD.YEAR.START;
         FRAME-TYPE = GROUP-START;
           BREAK-LEVEL = 3; BREAK-ELEM = GRAD.YEAR;

The code marked with asterisks tells SPIRES about the other two break elements whose values should be checked between records. Between records, SPIRES examines all the values of the break elements for the group-end frames to determine which group-end frames should be executed. After they have been executed, SPIRES checks the break elements for the group-start frames to determine which of them should be executed.

B.10.7.2  The Group-Start and Group-End Frame Definitions

A group-start frame is generally used to announce the beginning of a new group of records, usually displaying the new element value that caused the summary break. A group-end frame is generally used to display statistical information gathered from the preceding group of records, usually displaying the shared element value that caused the records to be grouped together.

The break frames are different from other report-specific frames (e.g., header, ending) because they may have GETELEM statements to access data values. But which record is used to obtain such values: the last record in the previous group or the first record in the new group? The answer depends on the frame-type: the group-end frame has access to the elements in the last record of the previous group, while the group-start frame has access to those in the first record of the next group.

Suppose you are creating a printed author index to a bibliographic data base. Each time a new author triggered a break, the group-start frame below would be executed, and the new author's name would appear two lines after the preceding data.

The next data frame to execute would process records having the new value for the AUTHOR element.

A group-end frame definition, on the other hand, would be more likely to concentrate on statistical information:

Assuming that the variable AUTHORENTRIES is a counter that is augmented each time a data record is processed for a given author, the above frame definition would produce a line such as this after the bibliographic entries for that author:

Note that the counter was reset to zero for the next group, but not before its value was added to a TOTALENTRIES variable, which presumably will be used in the ending frame to display the total number of entries for all authors in the data base. [See B.10.6.]

Page control problems when data and break frames are involved are often solved by using the system variable $LLEFT. [See E.2.2.3.] For example, you may want to ensure that a group-start frame will not appear by itself at the bottom of a page with the records it heralds starting at the top of the next one. A possible solution is to check the value of $LLEFT before entering the group-start frame (i.e., in the frame declaration portion of the format declaration section:

Thus, if there are fewer than ten lines left on the page (not counting the lines for the footer), the AUTHOR.START frame will be placed at the top of the next one. A better solution, if the group-start frame has fixed dimensions, would be to check whether $LLEFT was less than zero at the end of the frame, and if so, issue the SET NEWPAGE Uproc. Other solutions might involve the HOLD, FLUSH and EJECT PAGE Uprocs. [See B.10.3.5, B.10.3.6, B.10.3.7.]

If a break frame has fixed frame dimensions, the number of rows specified in the FRAME-DIM statement will be placed on the page, whether or not data has been placed in the last ones. However, you can use the SET NROWS Uproc to delete the last unused rows of a report-specific frame-type, if desired. [See B.3.3.1.]

You do not have to put data into a break frame. Instead you may just want to perform calculations. This means you might not need a break frame definition. Instead, you can code a null value for the frame name in the FRAME-NAME statement in the format declaration section, as you can for an initial or startup frame and perform your calculations there. [See B.5.2.] But if you do code a frame definition, it should not contain a FRAME-DIM statement.

B.10.7.3  Sorting Records for Report Processing

There are four basic ways to get records into a particular arrangement by element value in SPIRES:

Each of these particular methods is discussed in detail in other SPIRES manuals. Any of these methods may be used in connection with a report format. Of course, if group-start or group-end frames are involved in the format, you would want to choose a method that arranges the records by the break elements.

The SPISORT and FOR INDEX methods are worth discussing further because both methods allow a record to appear multiple times if it contains multiple occurrences of a break element. For example, if you are processing records under FOR INDEX ACCOUNT.NUMBER, a record with two or more account numbers can be processed several times, once for each number.

However, suppose you were creating a directory by account number. Your format displays the account number followed by the name of the person to whom it belongs. In such a case, you only want the one account number displayed that caused the record to appear at that particular point in the order. Under FOR INDEX ACCOUNT.NUMBER, the following label group, by itself, will not guarantee that the right occurrence of ACCOUNT.NUMBER is accessed:

That label group will retrieve the first occurrence of the ACCOUNT.NUMBER, which may not be the occurrence that caused the record to be displayed at that point. If you use the FOR INDEX method, you should use filter processing in combination with the $PATHKEY variable to filter out the irrelevant element occurrences. An example of this technique is explained in the SPIRES manual "Technical Notes", chapter 21.

If you use the SPISORT program to sort the records and process them under the FOR SET command, the above label group would retrieve the appropriate occurrence of ACCOUNT.NUMBER. The GENERATE SET command, used to create the record set for SPISORT, produces "path information" to tell SPIRES how to retrieve the appropriate occurrence of the element. In other words, when SPIRES is processing records under a FOR SET command, it has access not only to each record but also to directional information telling which occurrence of a sequenced element caused the record to appear at a particular point in the sorted set.

Unless directed otherwise, by the UNFILTERED option on the FOR SET command, SPIRES will automatically apply the path information to the records processed under FOR SET. Then, only the occurrence of the element that caused the record to appear at that position in the record set is available for display.

More information about SPISORT and FOR SET processing can be found in the manuals "Technical Notes" (SPISORT) and "Sequential Record Processing: Global FOR" (FOR SET command).

(*) The PATH and NPATH Options

In most report format situations, the automatic filtering done by FOR SET processing (described above) is desirable -- usually only the occurrence of an element that caused the record to sort in its particular place is needed. However, situations may arise where the automatic filtering is undesirable, so the UNFILTERED option on the FOR SET command tells SPIRES not to automatically filter the occurrences of the sorted elements.

But there may be still other situations where you need both: you want to know which occurrence of the element caused the record to be sorted where it was (perhaps you want to mark it with an asterisk) and you want to display all the occurrences besides. In this situation, you need to use the UNFILTERED option on the FOR SET command and code the PATH or NPATH options described below in the format definition to identify the particular element occurrence.

To tell SPIRES to use the path information to access the appropriate occurrence in a label group, you use the PATH option on the GETELEM statement:

where "element.name" is the name of the element to be accessed. The "element.name" may be replaced by a variable if desired.

Remember that path information is only available for data sets created by the DEFINE SET and GENERATE SET commands. If no path information is available for an element and the PATH option appears on the GETELEM statement, the first occurrence of the element within the record will be used.

The PATH option may also be used on the BREAK-ELEM statement or on the IND-STRUCTURE statement. [See B.10.7.1, B.8.3.] Suppose the set of records is sorted by ZIPCODE, an element within the multiply occurring structure ADDRESS. In your report format, you not only want to access the appropriate ZIPCODE but also the related CITY and STATE elements. So, in the label group that calls the indirect frame to process the ADDRESS structure you replace this:

with this:

SPIRES will use the path information leading to ZIPCODE to find the proper occurrence of the ADDRESS structure. Then the indirect frame can retrieve the other elements in that particular occurrence of the structure. Note that you do not code the structure name but the name of the element that has the path information that is within the structure in the IND-STRUCTURE statement.

One other variation on the PATH option, primarily useful in general file formats where you might not know the names of the elements with path information, is shown below:

where "n" is an integer or an integer variable. This form, also available in BREAK-ELEM and IND-STRUCTURE statements, replaces the element name with a number corresponding to the "nth" element in the sort list of the DEFINE SET command. For example, if the set is defined with "DEFINE SET TEST ELEM A, B, C", then "GETELEM = NPATH, 3;" is equivalent to "GETELEM = PATH, C;". Unlike the PATH option, this method will cause a format error if no path information is available, since SPIRES would not know what element to retrieve. [See B.14.]

B.10.7.4  The SET MAXLEVELS Uproc

The SET MAXLEVELS Uproc may be used to set the maximum break level to be executed by the report format. It allows a report format to execute the summary frames of all levels or no levels or some number in between. In other words, a single report format can be directed to execute all summary frames or just those down to a given break level.

The syntax of the Uproc is:

where "n" is an integer or an integer expression. The number given will be the lowest break level (i.e., the highest number) of summary frame that will be executed. If, for example, the Uproc "SET MAXLEVELS = 3" is executed, summary frames with break levels of 1, 2 and 3 will be executed, while frames with levels of 4 or higher will not. [See B.10.7.1.]

There is no $MAXLEVELS variable, nor is there a SET MAXLEVELS command -- "maxlevels" can only be set within a format definition. If you want the capability of dynamically changing "maxlevels", you will need to prompt for it, or set its value in another variable before executing the format. For example, you might add the following label group to the initial frame:

If the user gives a null response to the prompt, all summary frames will be executed, which is achieved by setting "maxlevels" to 10 or higher (10 is the limit for a break level).

B.10.8  Sorting Values in Formats

Occasionally you may need a way to sort values within a format. For example, during data frame execution you want to gather information that will appear as an index created in an ending frame. So you need a place to store the indexed values and their page numbers, as well as a way to sort them once they are all gathered.

SPIRES provides two methods to do this: triples, which are a general tool, and the sort facility built into the formats processor. Neither one is restricted to report formats, however, although they are more commonly used there than in non-report formats.

Triples are a special type of dynamic variable. Triples have more flexibility and more capabilities than the format sorting method: for example, their values can be accessed outside of the format, which is not true of the other method. However, computer memory is not handled as efficiently by triples, which could cause problems for applications that use a lot of memory already. Triples are discussed in detail in the manual "SPIRES Protocols", section 7a.

The Sort Facility in Formats

The format sort facility uses the system variable arrays $SORTKEY (sort value) and $SORTDATA (sort data). When the format is set, memory is set aside for sorting, according to the SORT-SPACE statement (see below). At some point in the format, you initialize the sort space and begin assigning values to the $SORTKEY and $SORTDATA arrays (for example, an index entry goes to $SORTKEY and its page number goes to $SORTDATA). When all the data has been gathered, you tell SPIRES to sort the values in the $SORTKEY array, in either ascending or descending order. You then can extract the values from the two arrays: the values in $SORTKEY are now in sort order, while the values in $SORTDATA have been rearranged so that each is in the same position in the array as its "partner" in $SORTKEY.

The restrictions on the format sort facility are discussed later. Here are the specific procedural steps to follow in coding it within a format.

First, you code the "SORT = INITIALIZE" statement at some point before you begin gathering the data. This statement initializes the sort space, discarding any values left in it. (INITIALIZE may be abbreviated to INIT in the SORT statement.) If the data will be gathered in a looping process, make sure that the SORT = INITIALIZE statement is outside of the loop. For example, if the values to be sorted are multiple occurrences of a retrieved element, place the SORT statement ahead of the label group that retrieves it; otherwise, the sort space will be reinitialized each time a new value is accessed. Similarly, if the values are gathered from multiple records before the sorting occurs, be sure that the SORT = INITIALIZE statement is not executed for each record; you could put it in the initial frame.

You then begin gathering the values, assigning them either to $SORTKEY or $SORTDATA:

It is the values in $SORTKEY that will be sorted. You do not need to use the $SORTDATA array unless you have data associated with each occurrence of $SORTKEY that should be carried along with it (like the page number for the index entry in the example above).

You do not handle occurrence numbers of the variables in the array; each time you put another value into either array, it goes into the next available position. (However, if you do use both arrays because you are handling the pairs of values, you should place the value to be sorted into $SORTKEY first and then put the corresponding value into $SORTDATA; if you place a value into $SORTDATA first, the values may not match properly.) You cannot remove or change values placed in the array except by reinitializing the sort space.

Once all the data has been gathered, another form of the SORT statement is specified:

This statement tells SPIRES to sort the $SORTKEY array either in ascending or descending order. (The values may be abbreviated to A and D; AU and DU indicate sorting in either ascending or descending order and discarding "extra" pairs of $SORTKEY and $SORTDATA values when $SORTKEY is not unique, i.e., one pair will not be discarded.) From this point on, unless another SORT = INITIALIZE statement is encountered, each reference to $SORTKEY will retrieve the next occurrence in the array. For example,

You can only retrieve a given value from $SORTKEY and $SORTDATA once. If you need to test or manipulate each value, put it into a variable right away. Consider this bad example:

Only every other value of $SORTKEY would be processed by the PUTDATA statement; the others are tested and "discarded" in the Uproc. This method would be a better way to handle the problem:

The flag variable $SORTKEND, tested in the LOOP label group, is set (set to $TRUE, i.e., 1) when the last sort value in the array has been retrieved. Similarly, the flag variable $SORTDEND is set when the last sort data in $SORTDATA has been retrieved. Both flags are cleared (set to $FALSE, i.e., 0) when the SORT = A or SORT = D statement is encountered.

The sample report at the end of this chapter shows another use of the sort facility. [See B.10.11, B.10.12.]

Allocating Sorting Space

The last coding requirement is the SORT-SPACE statement, which appears in the format declaration section:

where "n" is the number of bytes to be allocated, in thousands. It can be any positive integer up to 64. Unless your format is part of a large application that uses much or too much memory (as evidenced by S198 or CORE EXHAUSTED error messages) you can probably set "n" to an arbitrarily high number. If you do need to worry about memory management, try to determine the maximum number of $SORTKEY and $SORTDATA pairs of entries and the average total length of each pair. (Add 8 bytes to the average length of the pair for system data associated with it.) Multiply the number of pairs by the average length of each pair to get an approximation of the number of bytes of sort space needed.

If both a load format and its calling format include a SORT-SPACE statement, the allocation requested by the calling format will be assigned and the SORT-SPACE statement in the load format will be ignored. The load format's SORT-SPACE will be assigned only if the calling format does not include a SORT-SPACE statement. [See B.11.]

Additional Restrictions and Considerations:

B.10.9  Using Multiple Character Set Fonts in Report Formats

When designing report formats, people often want different data values to be printed with different character sets. For example, some elements should be displayed with italic characters while others should be displayed with bold ones. Two different methods are available, one simple, one complicated.

The simple method is, unfortunately, the less general. It is tied to special multiple-font character sets (such as MT10) available on the IBM 3800 printer at Stanford. Values that are to be in an italic font are processed by the $ITALIC system proc as an OUTPROC; values to be boldfaced are processed by the $BOLD system proc as an OUTPROC. These procs can be included as part of the OUTPROC statement in a label group:

Remember that the OUTPROC statement in the format will completely override the OUTPROC statement in the record definition. Note too that the insert text shown in the example above will not be italicized because the OUTPROC applies only to the element value retrieved from the record. [See B.4.5.2.] To have the entire value be italicized, try this method:

The resulting report must be printed on the 3800 laser printer with one of the multiple-font character sets that were designed for this purpose, such as the MT10 or MT12 font.

When the italicized or boldfaced values are displayed on a terminal, they will not print properly -- that is, you will not be able to read them. In fact, they may cause your terminal to ring the bell, blank the screen, and act peculiarly in general. Adding the MIXED option to the LIST command in WYLBUR, which causes unusual characters to be displayed in their hexadecimal equivalent, may be helpful when you want to see the output data at a terminal. For more information about the use of the $BOLD and $ITALIC system procs, see the manual "SPIRES System Procs" or EXPLAIN $BOLD PROC or EXPLAIN $ITALIC PROC. More information about the multiple-font character sets is available in the I.T.S. documents "Character Sets for the 3800 Laser Printer" and "Using the 3800 Laser Printer".

The second method, which allows you to mix various character fonts of your own choosing, can be very complicated when done in a report format -- it is much easier to use in non-report formats where you are not relying on the $LLEFT variable to keep track of the number of lines left on the page. For this method, which produces output for either the IBM 3800 or the Xerox 9700 printer, column 1 of your format should be reserved for carriage control that you will provide yourself, and column 2 will be used for character set selection.

The principle here is that each row of the format can be printed with only one character set, as specified by the number placed in column 2. When you want a line of output to contain more than one character set, you must place the characters to be italicized, for example, on one row of output, and the characters to be boldfaced on the next row of output but with the overstrike carriage control character (+) in column 1:

When printed with the CC (carriage control) option, these two rows of output would be printed as one, with the second row merged into the first. More details on preparing a data set for this type of printing are given in the manual "Using the 3800 Laser Printer".

Writing a report format to create this type of data set is complicated for several reasons:

(*) Overstrikes and Underscores

An impact printer may be able to simulate boldface type by overstriking, that is, by printing the same characters several times on top of each other. The relevant lines of such a data set might look like this:

     (....v....1....v....2....v....3....v....4....v....5)
       This line will be overstruck three times.
      +This line will be overstruck three times.
      +This line will be overstruck three times.
      +This line will be overstruck three times.

Similarly, text may be underlined by printing the text and then printing the underscore characters on the same line, requiring the overstrike carriage control symbol to prevent the printer from going to a new line. The relevant lines of the data set usually look like this:

     (....v....1....v....2....v....3....v....4....v....5)
       The third word in this line is underscored.
      +          ____

The same steps outlined above are necessary for such effects: you must handle the carriage control yourself after setting $PAGECTL to 4, and you must temporarily increase $LINEPP to account for the extra lines added to the page.

B.10.10  System Variables that are Useful in Report Formats

As you have no doubt discovered, system variables are quite important in report formats. A list of system variables useful within output formats in general appeared earlier in the manual. [See B.9.2.] The list below describes variables useful within report formats in particular. Details for each variable are discussed in an Appendix. [See E.2.]

The variable type is given in parentheses after the variable name. An asterisk preceding the name indicates that it can be set explicitly by the format or by the user.

* $REPORT (flag)     - set within a format when report mode processing
                        is in effect.
  $NOREPORT (flag)   - set within a format when report mode processing
                        is not in effect.
* $NEWPAGE (flag)    - can be set to tell SPIRES to start the current
                        buffer on the next page.
* $FRONTPAGE (flag)  - can be set to cause "1" to be replaced by "F"
                        when a page eject occurs (to force printing on
                        the front of a page)
* $NOBREAK (flag)    - can be set to indicate that the current buffer
                        should begin a new page if it cannot fit on
                        the current page.
  $FREC (flag)       - set while the first record is being processed
                        when a multiple-record processing command was
                        issued.
  $LREC (flag)       - set after the last record has been processed
                        when a multiple-record processing command was
                        issued.
* $PAGECTL (int)     - can be set to specify the type of carriage
                        control provided by SPIRES.
  $RECNO (int)       - the number of records processed thus far under
                        the multiple-record processing command.
* $LINEPP (int)      - can be set to indicate the number of lines
                        allowed per page.
  $LLEFT (int)       - represents the number of lines left for data on
                        the current page after the current line.
* $HDRLEN (int)      - can be set to indicate the total number of lines
                        allowed for all the header frames on the page.
* $FTRLEN (int)      - can be set to indicate the total number of
                        lines to be left for all the footer frames
                        on the page.
* $PAGENO (int)      - the current page number of the report.
* $MARGIN (int)      - can be set to the number of blank characters for
                        a default left margin.
* $SKIPF (flag)      - can be set to tell SPIRES to stop further
                        processing of the current frame.
* $SUPPRESS (flag)   - can be set to tell SPIRES to process the data
                        but not put out any data.
* $COLUMNS (int)     - when greater than 1, indicates that multiple
                        column processing will occur; value represents
                        the number of columns.
* $COLWDTH (int)     - the width in bytes of each column in multiple
                        column processing.
  $CURCOL (int)      - the number of the current column on the page
                        during multiple column processing.
* $NEWCOL (flag)     - can be set to tell SPIRES to start the current
                        buffer in the next column.
  $FLINES (int)      - the number of lines in the buffer when it was
                        last flushed.
* $SORTKEY (string)  - the next value in the sorted array.
  $SORTKEND (flag)   - set when the final $SORTKEY value has been
                        retrieved.
* $SORTDATA (string) - the data value associated with the most
                        recently accessed value of $SORTKEY.
  $SORTDEND (flag)   - set when the final $SORTDATA value has been
                        retrieved.

B.10.11  A Sample Report Format Definition

Below is a sample report format definition for the public subfile BLOOD DONORS. (This file was created and described in detail in the SPIRES primer "A Guide to Data Base Development".) The report uses many of the features described in this chapter.

The format is used to create a phone list of blood donors. The donor records are sequenced by the BLOOD.TYPE, CITY and DATE (representing the date the donor last gave blood) elements, and the report is arranged by the first two categories, making it easy to find appropriate donors when necessary. Donors who have given a gallon of blood are treated specially by the blood bank, and their entries are preceded by asterisks to denote their achievement. Records are selected for the report with the search FIND CAN.BE.CALLED = YES, to include only donors who said they could be called.

The report includes a title page with the date of the report as well as a note concerning who may use the list. Each page within the report has a header and footer, including such information as the page number, the date and the blood-type of donors on that page.

Other features of the report include subtotal information for each blood-type as well as grand totals at the end of the report. Also at the end is a summarization by city, a tally created with the sorting facility.

The format definition below does not include comments, except to describe the variables that are defined. Detailed comments follow the definition, along with some sample pages of output. [See B.10.12.]

The Identification Section

   1.  ID = GQ.JNK.DONORS.REPT;
   2.  MODDATE = WED. JAN. 12, 1983;
   3.  DEFDATE = SUN. JAN. 9, 1983;
   4.  FILE = GQ.JNK.BLOOD.DONORS;
   5.  RECORD-NAME = REC01;

The Vgroup Section

   6.  VGROUP = LOCAL;
   7.    VARIABLE = PEOPLEPT;
   8.      OCC = 1;
   9.      TYPE = INT;
  10.      COMMENTS = Number of people per blood-type.;
  11.    VARIABLE = PINTSPT;
  12.      OCC = 1;
  13.      TYPE = INT;
  14.      COMMENTS = Number of pints donated per blood-type.;
  15.    VARIABLE = GALLONPT;
  16.      OCC = 1;
  17.      TYPE = INT;
  18.      COMMENTS = Number of gallon donors per blood-type.;
  19.    VARIABLE = PEOPLETOTAL;
  20.      OCC = 1;
  21.      TYPE = INT;
  22.      COMMENTS = Total number of donors.;
  23.    VARIABLE = PINTSTOTAL;
  24.      OCC = 1;
  25.      TYPE = INT;
  26.      COMMENTS = Total number of pints donated.;
  27.    VARIABLE = GALLONTOTAL;
  28.      OCC = 1;
  29.      TYPE = INT;
  30.      COMMENTS = Total number of gallon donors.;
  31.    VARIABLE = CITY;
  32.      OCC = 1;
  33.      TYPE = STRING;
  34.      COMMENTS = Value of CITY element for sort array.;
  35.    VARIABLE = CHECKCITY;
  36.      OCC = 1;
  37.      TYPE = STRING;
  38.      COMMENTS = Used to check for new city in sort
  39.  array.;

The Frame Definitions

The Data Frame

  40.  FRAME-ID = DONOR;
  41.    DIRECTION = OUTPUT;
  42.    FRAME-DIM = 6,67;
  43.    USAGE = DISPLAY;
  44.    LABEL = NAME;
  45.      GETELEM;
  46.      START = 1,3;
  47.      UPROC = LET PEOPLEPT = #PEOPLEPT + 1;
  48.      PUTDATA;
  49.    LABEL = PHONE.NUMBER;
  50.      GETELEM;
  51.      START = *,40;
  52.      INSERT = 'Ph: ';
  53.      UPROC = SET ADJUST RIGHT;
  54.      PUTDATA;
  55.    LABEL = DATE.GIVEN;
  56.      GETELEM;
  57.      START = 2,4;
  58.      INSERT = 'Last Donated: ';
  59.      PUTDATA;
  60.    LABEL = TOTAL.PINTS;
  61.      GETELEM;
  62.      START = 2,40;
  63.      INSERT = 'Total pints donated: ';
  64.      UPROC = LET PINTSPT = #PINTSPT + $UVAL;
  65.      UPROC = SET SORTKEY = #CITY;
  66.      UPROC = SET SORTDATA = $STRING($UVAL);
  67.      PUTDATA;
  68.    LABEL = GALLON.DONOR;
  69.      VALUE = '*';
  70.      START = 1,1;
  71.      UPROC = IF $PVAL < 8 THEN JUMP;
  72.      UPROC = LET GALLONPT = #GALLONPT + 1;
  73.      PUTDATA;
  74.    LABEL = ADDRESS;
  75.      GETELEM;
  76.      START = 3,4;
  77.      PUTDATA;
  78.      LOOP;
  79.    LABEL = BLANK.LINE;
  80.      VALUE = ' ';
  81.      START = X,1;
  82.      UPROC = SET NOBREAK;
  83.      UPROC = IF $LLEFT = 0 THEN JUMP;
  84.      PUTDATA;

Comments:

47. In this Uproc, we begin to collect the subtotal information that will be processed in the group-end frame (lines 158-182). Each time a data record is processed by this frame, the PEOPLEPT (for "People-Per-Type") counter variable will be incremented here. Presumably every record should have an occurrence of the NAME element, but note that if one did not, the GETELEM would fail (line 45) and execution would immediately continue at the next label group.

65 and 66. In these Uprocs, we collect the data to be used in the ending frame that displays subtotals by city. Each entry into the sort array consists of two values: the donor's city (for $SORTKEY) and the number of pints the user donated (for $SORTDATA). The CITY variable is set in the group-start frame that retrieves the CITY element (line 190).

71. This Uproc tells SPIRES to skip the rest of the label group if the donor has not given at least a gallon of blood (8 pints).

79. This label group places an extra blank line at the end of each record, serving as a record separator line. It is not put there if there are no lines left at the bottom of the page, since no record will follow on that page.

The Initial Frame

  85.  FRAME-ID = INITIAL;
  86.    DIRECTION = OUTPUT;
  87.    FRAME-DIM = 20,67;
  88.    USAGE = DISPLAY;
  89.    LABEL = TITLE;
  90.      VALUE = 'BLOOD DONORS PHONE LIST';
  91.      START = 10,1;
  92.      UPROC = SET ADJUST CENTER;
  93.      PUTDATA;
  94.    LABEL = DATE;
  95.      VALUE = $UDATE;
  96.      START = 12,1;
  97.      OUTPROC = $DATE.OUT(MONTH,,UPLOW,FULL);
  98.      UPROC = SET ADJUST CENTER;
  99.      PUTDATA;
 100.    LABEL = WARNING;
 101.      VALUE = 'The information in this report is for
 102.  internal blood bank use. Only the professional staff
 103.  and phone volunteers are to use it.  Any questions
 104.  about authorization should be directed to Nurse
 105.  Quigley.';
 106.      MARGINS = 8,60;
 107.      START = 17,8;
 108.      UPROC = SET ADJUST JUSTIFY;
 109.      PUTDATA;
 110.    LABEL;
 111.      SORT = INITIALIZE;

The Header Frame

 112.  FRAME-ID = HEADER;
 113.    DIRECTION = OUTPUT;
 114.    FRAME-DIM = 3,67;
 115.    USAGE = DISPLAY;
 116.    LABEL = TITLE;
 117.      VALUE = 'Blood Donors Phone List';
 118.      START = 1,1;
 119.      PUTDATA;
 120.    LABEL = BLOOD.TYPE;
 121.      VALUE = 'Blood-type: ' $GETCVAL(BLOOD.TYPE);
 122.      START = 1,30;
 123.      UPROC = SET ADJUST RIGHT;
 124.      UPROC = IF $LREC THEN JUMP;
 125.      PUTDATA;
 126.    LABEL = BORDER;
 127.      VALUE = '- ';
 128.      START = 2,1;
 129.      UPROC = SET REPEAT;
 130.      PUTDATA;

Comments:

124. This Uproc in effect suppresses the blood-type guide word when the "last-record" flag is set, meaning that all data frame processing is done.

The Footer Frame

 131.  FRAME-ID = FOOTER;
 132.    DIRECTION = OUTPUT;
 133.    FRAME-DIM = 3,67;
 134.    USAGE = DISPLAY;
 135.    LABEL = LEGEND;
 136.      VALUE = '* = Gallon Donor';
 137.      START = 2,1;
 138.      PUTDATA;
 139.    LABEL = DATE;
 140.      VALUE = $DATE;
 141.      PUTDATA;
 142.    LABEL = PAGE.NUMBER;
 143.      VALUE = 'Page ' $PAGENO;
 144.      START = *,30;
 145.      UPROC = SET ADJUST RIGHT;
 146.      PUTDATA;

A Group-Start Frame for Break-Level 1

 147.  FRAME-ID = BLOOD.TYPE.START;
 148.    DIRECTION = OUTPUT;
 149.    FRAME-DIM = 2,67;
 150.    USAGE = DISPLAY;
 151.    LABEL = BLOOD.TYPE;
 152.      GETELEM;
 153.      INSERT = '* * *  Blood-Type: ';
 154.      INSERT = END, '  * * *';
 155.      UPROC = SET ADJUST CENTER;
 156.      UPROC = SET NEWPAGE;
 157.      PUTDATA;

Comments:

156. Each new blood-type group will begin on a new page.

A Group-End Frame for Break-Level 1

 158.  FRAME-ID = BLOOD.TYPE.END;
 159.    DIRECTION = OUTPUT;
 160.    FRAME-DIM = 4,67;
 161.    USAGE = DISPLAY;
 162.    LABEL = PEOPLEPT;
 163.      VALUE = 'Total number of donors with blood-type '
 164.  || $GETCVAL(BLOOD.TYPE) || ' who can be called: ' ||
 165.  #PEOPLEPT;
 166.      PUTDATA;
 167.    LABEL = AVERAGE;
 168.      VALUE = 'Average number of pints per donor: ' ||
 169.  $DECIMAL($PACK(#PINTSPT)/#PEOPLEPT,3);
 170.      PUTDATA;
 171.    LABEL = GALLON.DONORS;
 172.      VALUE = 'Total number of Gallon Donors: ' ||
 173.  #GALLONPT;
 174.      PUTDATA;
 175.    LABEL = ADD.AND.RESET;
 176.      UPROC = LET PEOPLETOTAL = #PEOPLETOTAL + #PEOPLEPT;
 177.      UPROC = LET PEOPLEPT = 0;
 178.      UPROC = LET PINTSTOTAL = #PINTSTOTAL + #PINTSPT;
 179.      UPROC = LET PINTSPT = 0;
 180.      UPROC = LET GALLONTOTAL = #GALLONTOTAL + #GALLONPT;
 181.      UPROC = LET GALLONPT = 0;
 182.      UPROC = SET NOBREAK;

Comments:

163-165. This value consists of four strings concatenated together. The $GETCVAL function retrieves the converted value of the BLOOD.TYPE element from the most recently processed record.

169. This computation returns the average number of pints per donor for a given blood type. Because the two variables PINTSPT and PEOPLEPT are integers, the result of the division would be an integer, which would not be very accurate in many cases. Hence, to get a more accurate packed-decimal result, one of the variables is converted to packed before the division occurs. [See B.10.6.] The $DECIMAL proc rounds the result to three decimal places. The concatenation operator on the previous line (168) changes the result to a string for output.

175. This label group adds the group totals to the grand totals and resets the group totals before the next group begins.

A Group-Start Frame for Break-Level 2

 183.  FRAME-ID = CITY.START;
 184.    DIRECTION = OUTPUT;
 185.    FRAME-DIM = 2,67;
 186.    USAGE = DISPLAY;
 187.    LABEL = CITY;
 188.      GETELEM;
 189.      OUTPROC = $CAP;
 190.      UPROC = LET CITY = $CVAL;
 191.      UPROC = SET CVAL = 'City or town: ' || $CVAL;
 192.      UPROC = IF $LLEFT < 10 THEN SET NEWPAGE;
 193.      PUTDATA;

Comments:

190. The CITY variable is set here for use in the data frame (line 65). Saving the retrieved value in a variable in the break frame prevents you from having to retrieve the value in the data frame for each record.

192. If there are fewer than 10 lines left on the page at this point, the frame will be placed on the next page. That prevents this group-start frame from appearing at the bottom of a page with no data records following it there.

The Ending Frames

 194.  FRAME-ID = ENDING;
 195.    DIRECTION = OUTPUT;
 196.    FRAME-DIM = 7,67;
 197.    USAGE = DISPLAY;
 198.    LABEL = PEOPLETOTAL;
 199.      VALUE = '** Total number of donors who can be
 200.  called: ' #PEOPLETOTAL;
 201.      START = 3,1;
 202.      UPROC = SET NOBREAK;
 203.      PUTDATA;
 204.    LABEL = AVERAGE;
 205.      VALUE = '** Average number of pints donated: ' ||
 206.  $DECIMAL($PACK(#PINTSTOTAL)/#PEOPLETOTAL,3);
 207.      PUTDATA;
 208.    LABEL = GALLONTOTAL;
 209.      VALUE = '** Total number of Gallon Donors: ' ||
 210.  #GALLONTOTAL;
 211.      START = 6,1;
 212.      PUTDATA;

 213.  FRAME-ID = CITIES.SUMMARY;
 214.    DIRECTION = OUTPUT;
 215.    FRAME-DIM = 50,67;
 216.    USAGE = DISPLAY;
 217.    LABEL = SORT.CITIES;
 218.      UPROC = SET NEWPAGE;
 219.      SORT = ASCENDING;
 220.    LABEL = TITLE;
 221.      VALUE = '* * * Summary of Donations by City * * *';
 222.      START = 3,1;
 223.      UPROC = SET ADJUST CENTER;
 224.      PUTDATA;
 225.    LABEL = HEADERS;
 226.      VALUE = 'City';
 227.      START = 5,1;
 228.      PUTDATA;
 229.    LABEL;
 230.      VALUE = '# Donors';
 231.      START = *,18;
 232.      PUTDATA;
 233.    LABEL;
 234.      VALUE = 'Total Pints';
 235.      START = *,30;
 236.      PUTDATA;
 237.    LABEL;
 238.      VALUE = 'Avg. per Donor';
 239.      START = *,45;
 240.      PUTDATA;
 241.    LABEL = COMPUTE.VALUES;
 242.      UPROC = EVAL $VGROUPINIT(LOCAL);
 243.      UPROC = LET CITY = $SORTKEY;
 244.    LABEL = PUT.CITY;
 245.      VALUE = #CITY;
 246.      START = X,1;
 247.      UPROC = LET PEOPLETOTAL = 1;
 248.      UPROC = LET PINTSTOTAL = $SORTDATA;
 249.      PUTDATA;
 250.    LABEL = COUNTER.LOOP;
 251.      UPROC = LET CHECKCITY = $SORTKEY;
 252.      UPROC = IF #CITY ~= #CHECKCITY THEN LET CITY =
 253.  #CHECKCITY;
 254.      UPROC = THEN JUMP;
 255.      UPROC = LET PEOPLETOTAL = #PEOPLETOTAL + 1;
 256.      UPROC = LET PINTSTOTAL = #PINTSTOTAL + $SORTDATA;
 257.      UPROC = IF ~$SORTKEND THEN JUMP COUNTER.LOOP;
 258.    LABEL = PUT.DONORS;
 259.      VALUE = $STRING(#PEOPLETOTAL);
 260.      START = *,18;
 261.      LENGTH = 6;
 262.      UPROC = SET ADJUST RIGHT;
 263.      PUTDATA;
 264.    LABEL = PUT.PINTS;
 265.      VALUE = $STRING(#PINTSTOTAL);
 266.      START = *,30;
 267.      LENGTH = 8;
 268.      UPROC = SET ADJUST RIGHT;
 269.      PUTDATA;
 270.    LABEL = PUT.AVERAGE;
 271.      VALUE = $STRING($DECIMAL($PACK(#PINTSTOTAL) /
 272.  #PEOPLETOTAL,3));
 273.      START = *,45;
 274.      LENGTH = 10;
 275.      UPROC = SET ADJUST RIGHT;
 276.      PUTDATA;
 277.    LABEL;
 278.      UPROC = IF ~$SORTKEND THEN JUMP PUT.CITY;

Comments:

213. This second ending frame sorts and tabulates the data collected in the sort array on blood donations by city (see lines 64, 65). After telling SPIRES to sort the array (line 219), the frame puts out headings for the table that will be printed (lines 221-240).

242. All of the variable values in the LOCAL array are re-initialized here. [See B.9.3.3.]

244. The rest of the frame demonstrates how you might handle a sort array containing multiple values for a given sortkey. That is, there are multiple pairs of sort data values that will have the same value (a given city) for the sortkey, but on output, the sortkey must only appear once, along with the total of all the sortdata values (total number of pints) for that city.

244. The current value of the CITY variable (derived from $SORTKEY in line 243 or 251-253) is placed in the table in this label group. The PEOPLETOTAL and PINTSTOTAL variables are reset to 1 and the number of pints donated by the first donor for a given city respectively.

250. This is a looping label group to retrieve pairs of $SORTKEY and $SORTDATA values, adding the $SORTDATA values to the PINTSTOTAL variable when the $SORTKEY value is not a new city. When it is a new city, the label groups that place the totals into the frame are executed (lines 252-254, 258 on).

271 and 272. Although the basic computation is the same as it was in lines 169 and 206, the $STRING function was added to convert the packed-decimal value to a string for output. In the other lines, the conversion to string happened automatically because of the concatenation operators. [See B.4.3.]

The Format Declaration Section

 279.  FORMAT-NAME = PHONE.LIST;
 280.    SORT-SPACE = 64;
 281.    ALLOCATE = GQ.JNK.LOCAL;
 282.    FRAME-NAME = DONOR;
 283.      FRAME-TYPE = DATA;
 284.    FRAME-NAME;
 285.      FRAME-TYPE = STARTUP;
 286.      UPROC = SET REPORT;
 287.    FRAME-NAME = INITIAL;
 288.      FRAME-TYPE = INITIAL;
 289.      UPROC = SET HDRLEN = 3;
 290.      UPROC = SET FTRLEN = 3;
 291.      UPROC = EVAL $VGROUPINIT(LOCAL);
 292.    FRAME-NAME = HEADER;
 293.      FRAME-TYPE = HEADER;
 294.    FRAME-NAME = FOOTER;
 295.      FRAME-TYPE = FOOTER;
 296.    FRAME-NAME = BLOOD.TYPE.START;
 297.      FRAME-TYPE = GROUP-START;
 298.      BREAK-LEVEL = 1;
 299.      BREAK-ELEM = BLOOD.TYPE;
 300.    FRAME-NAME = BLOOD.TYPE.END;
 301.      FRAME-TYPE = GROUP-END;
 302.      BREAK-LEVEL = 1;
 303.      BREAK-ELEM = BLOOD.TYPE;
 304.    FRAME-NAME = CITY.START;
 305.      FRAME-TYPE = GROUP-START;
 306.      BREAK-LEVEL = 2;
 307.      BREAK-ELEM = CITY;
 308.    FRAME-NAME = ENDING;
 309.      FRAME-TYPE = ENDING;
 310.    FRAME-NAME = CITIES.SUMMARY;
 311.      FRAME-TYPE = ENDING;

Comments:

284-286. Though no startup frame is actually coded, the SET REPORT Uproc will be executed when the SET FORMAT command is issued. [See B.10.2.]

296-307. Remember that the group-start frames should usually be declared in ascending break-level order (1 2 3 ...), while the reverse is true for group-end frames. Also remember that group-start and group-end frames with the same break element usually have the same break level, and vice versa. [See B.10.7.1.]

B.10.12  Sample Output from the Sample Report Format

Below are excerpts from a sample report created by the report definition in the last section. Some minor changes have been made to the output to allow it to fit properly on the pages of this manual. Specifically,

The Title Page (Created by the Initial Frame)

 +----------------------------------------------------------------+
 |1                                                               |
 |                                                                |
 |                                                                |
 |                                                                |
 |                     BLOOD DONORS PHONE LIST                    |
 |                                                                |
 |                        January 11, 1983                        |
 |                                                                |
 |                                                                |
 |                                                                |
 |                                                                |
 |      The information in this report is for internal  blood     |
 |      bank  use.  Only  the  professional  staff  and phone     |
 |      volunteers  are  to  use  it.   Any  questions  about     |
 |      authorization  should  be  directed to Nurse Quigley.     |
 |                                                                |
 +----------------------------------------------------------------+

The First Page of Records

 +----------------------------------------------------------------+
 | 1Blood Donors Phone List                       Blood-type: A+  |
 |  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -   |
 |                                                                |
 |                   * * *  Blood-Type: A+  * * *                 |
 |                                                                |
 |  City or town: MOUNTAIN VIEW                                   |
 |                                                                |
 |    Harvey, William                        Ph: 328-0920 (work)  |
 |     Last Donated: 03/11/81          Total pints donated: 1     |
 |     193 Circulation Drive                                      |
 |                                                                |
 |  City or town: SAN MATEO                                       |
 |                                                                |
 |    William, Harvey                        Ph: 497-4420 (work)  |
 |     Last Donated: 06/19/81          Total pints donated: 2     |
 |     1552 Corpuscle Circle                                      |
 |                                                                |
 |  Total number of donors with blood-type A+ who can be called: 2|
 |  Average number of pints per donor: 1.000                      |
 |  Total number of Gallon Donors: 0                              |
 | -                                                              |
 | -                                                              |
 | -                                                              |
 |                                                                |
 |  * = Gallon Donor                                              |
 |  01/11/83                                              Page 1  |
 +----------------------------------------------------------------+

The Last Page of Records

 +----------------------------------------------------------------+
 | 1Blood Donors Phone List                       Blood-type: O-  |
 |  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -   |
 |                                                                |
 |                   * * *  Blood-Type: O-  * * *                 |
 |                                                                |
 |  City or town: FREMONT                                         |
 |                                                                |
 |    Lood, Rhett B.                                Ph: Unlisted  |
 |     Last Donated: 07/25/81          Total pints donated: 1     |
 |     2350 Oxnard Boulevard                                      |
 |                                                                |
 |  Total number of donors with blood-type O- who can be called: 1|
 |  Average number of pints per donor: 1.000                      |
 |  Total number of Gallon Donors: 0                              |
 |                                                                |
 |                                                                |
 |                                                                |
 |  ** Total number of donors who can be called: 19               |
 |  ** Average number of pints donated: 2.842                     |
 |                                                                |
 |  ** Total number of Gallon Donors: 3                           |
 |                                                                |
 | -                                                              |
 | -                                                              |
 | -                                                              |
 |                                                                |
 |  * = Gallon Donor                                              |
 |  01/11/83                                              Page 8  |
 +----------------------------------------------------------------+

The Last Page of the Report

 +----------------------------------------------------------------+
 | 1Blood Donors Phone List                                       |
 |  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -   |
 |                                                                |
 |                                                                |
 |                                                                |
 |               * * * Summary of Donations by City * * *         |
 |                                                                |
 |  City             # Donors    Total Pints    Avg. per Donor    |
 |  BIG RED              1             1            1.000         |
 |  FREMONT              1             1            1.000         |
 |  MARIN                1             1            1.000         |
 |  MENLO PARK           2             3            1.500         |
 |  MOUNTAIN VIEW        5            13            2.600         |
 |  PALO ALTO            3            16            5.333         |
 |  REDBLOOD CITY        3             8            2.667         |
 |  SAN JOSE             1             1            1.000         |
 |  SAN MATEO            2            10            5.000         |
 |                                                                |
 | -                                                              |
 |                                                                |
 |  * = Gallon Donor                                              |
 |  01/11/83                                              Page 9  |
 +----------------------------------------------------------------+

B.11  Invoking Formats from within Formats: Load Formats

In some circumstances you may want or need SPIRES to execute another format in the midst of executing the current one:

In any case, the feature that these situations require is called a "load format". That term refers to a format that is "loaded" into SPIRES when it is called from within another format. A load format is like an indirect frame in that, once it is executed, control returns to the frame of the original format that called it. In fact, invoking a load format involves coding an indirect frame: the calling label group calls an indirect frame (here called the "accessing frame"), which contains the LOAD-FORMAT and USING-FRAME statements that tell SPIRES which format and frame to load and execute.

There are basically two steps to using load formats. First, you must define and compile the format to be loaded, if that has not already been done. [See B.11.1.] Second, you must add the appropriate code to the "calling format", which means creating the accessing frame and the calling label group that invokes it. [See B.11.2.]

B.11.1  The Load Format Itself

The most important rule to remember in creating load formats is that a load format is a complete format by itself that must be compiled separately from the calling format. In other words, its definition is generally like any other format's. A load format may even call another load format, nesting to a limit of eight load formats.

If you want to use the same variables in the load format as in the calling one (in particular, if you want to carry values across), you must create a global vgroup, which is then allocated by both formats. [See B.9.3, B.9.3.2.] When SPIRES loads the load format, it will see that the vgroup has already been allocated by the calling format, and will not reallocate or reinitialize (that is, reset) the vgroup.

How do you know which frame or frames of the load format will be executed? First, there is a procedure for SPIRES to follow in order to locate the proper frame -- depending on the type of the calling frame, SPIRES will look for a particular frame-type in the load format to execute. Also, a format definer will generally request a particular frame with the USING-FRAME statement in the calling format. [See B.11.2.]

B.11.2  The LOAD-FORMAT (LOAD) and USING-FRAME (USING) Statements

The LOAD-FORMAT statement, which tells SPIRES what format is to be loaded, is coded in an indirect frame that has no label groups. The USING-FRAME statement, which names a particular frame to be executed (but see the rules below), may follow it. Here for example is the complete definition for an accessing frame:

Only one LOAD-FORMAT statement may be coded per frame.

The syntax of the LOAD-FORMAT statement is:

where "format.name" is one of the following:

where "format-name" is the value in the FORMAT-NAME statement. [See B.14, D.2.5.] Note that format-name and **format-name should NOT be used in a #variable. Only system or account-specific forms can be used in a #variable.

The USING-FRAME statement has the following syntax:

where "frame.name" is the FRAME-ID of the frame in the load format to be executed. [See B.3.1.] The value "frame.name" may be a literal string or a string variable.

Some special options regarding SPIRES's handling of load formats may be requested with the LOAD-OPTIONS statement. [See B.11.3.]

SPIRES will look for a load format frame to execute according to the rules described below. If a frame is named in the USING-FRAME statement, SPIRES will execute it only if it would be found according to the rules:

For output formats, if SPIRES cannot find a frame of the appropriate type, it will look for a data frame to use. This allows initial, ending, header, footer, summary, and XEQ frames to do subgoal processing if necessary. [See B.12.]

A different rule applies for input formats. If the accessing frame has a usage of FULL, SPIRES will look for a frame in the load format whose usage is FULL; if no "full" frame is located, the load format is scanned again for one of usage MERGE. Vice versa, if the accessing frame has a usage of MERGE, SPIRES will look for a frame in the load format whose usage is MERGE, but will look for one of usage FULL if a merge frame cannot be found.

The USING-FRAME statement is necessary if the frame of the load format to be executed is a NAMED frame. [See D.1.1.1.]

Generally the USAGE statements in the load format's appropriate frame will match that in the accessing frame, though it does not have to. If two different but compatible USAGE statements are coded, SPIRES will use the most restrictive values indicated. For example, if the accessing frame has USAGE = ALL, and the requested frame of the loaded format has USAGE = DISPLAY, a TRANSFER command will fail -- SPIRES could not find a frame of the correct USAGE in the load format.

If a frame is used as an indirect frame in the load format but will be named in the USING-FRAME statement in the calling format, it must also have a FRAME-TYPE of DATA or STRUCTURE in the load format. (That is in addition to its declaration as FRAME-TYPE = INDIRECT.) That would probably be done by adding the NAMED option to its USAGE statement [See D.1.1.1.] and declaring it twice in the format declaration section:

Assuming that the frame definition of IND.FRAME had a USAGE value of NAMED, IND.FRAME would not be executed as a data frame when the INVOICE format was executed.

The calling label group that invokes the accessing frame follows the same rules as the calling label group for other indirect frames. [See B.4.8.7, B.8.1.]

Similarly, an XEQ frame in the load format must also be declared as a DATA frame in order for a DATA frame inthe calling format to access it.

B.11.3  The LOAD-OPTIONS Statement

The LOAD-OPTIONS statement lets you specify some options for SPIRES to apply to the format named by the LOAD-FORMAT statement.

where "option" is one of the following:

Multiple options may be specified together, as indicated by the syntax. The LOAD-OPTIONS statement is only available when the LOAD-FORMAT statement is coded in the same frame definition. [See B.11.2.]

Details about the UNLOAD option

As mentioned above, you should code the UNLOAD option only when the load format will be used only once by the format. Remember that a loaded format will remain loaded until another format is set or the current format is reset, meaning that it needs to be loaded only the first time the format is executed. If the load format will be used each time the format is executed, you should not code the UNLOAD option for it, since SPIRES will have to load and unload it for every record processed, which is very inefficient.

B.12  Accessing Data in Other Record Types from Output Formats

Sometimes you need your format to retrieve data from a record in another record type. The most simple method for doing this involves the table-lookup feature, using the $LOOKUP or $SUBF.LOOKUP system procs (actions A32 or A65) in an OUTPROC for the data value. The value you have matches the key of a record in another record type; SPIRES accesses that record, usually retrieving the first occurrence of another element in that record. The table lookup feature is easy to use if you can work within its main limitation: these rules can only retrieve the first occurrence of a single record-level element.

If multiple elements, multiple occurrences of elements, or structural elements are to be retrieved from the other record-type, you should use a different method. One method, called "subgoal processing", connects the record-types through formats. The other method, called "phantom structures", connects them through the file definition.

Formats using subgoal processing can be written by anyone with access to both (or all, if more than one subgoal is done) record types. The subgoal linkage is obviously available only through the format; the connection does not exist if the format is not set. Phantom structures, on the other hand, are coded in the file definition by the file owner (at least for the time being), and they exist independently of formats.

The creation and use of phantom structures is discussed in the manual "SPIRES Technical Notes". However, the first section of this chapter will discuss their use in formats. [See B.12.1.] The other sections here will discuss subgoal processing.

It is worth knowing that a format using phantom structures to retrieve data from other record-types will be easier to code than an equivalent one using subgoal processing. The phantom-structure method does not require you to do the "dirty work" of retrieving the pointer and handling it in a variable. But again, remember that it is only available if the file owner has defined phantom structures in the file definition.

B.12.1  Phantom Structures

Phantom structures establish a subgoal connection in the file definition, allowing the goal records of one record-type to appear as structures in another. For example, if a record-type (call it REC02) contains keys or locators to another record-type (say, REC01), which would be the case if REC02 were an index record-type for REC01, then REC02 could be defined to have a "phantom structure", each occurrence of which is actually a record in REC01 that is pointed to. The keys or locators provide the link between the two record-types -- each pointer element value in a REC02 record identifies a record in REC01, and thus that REC01 record is an occurrence of the phantom structure in the REC02 record.

Phantom structures are often coded in index record definitions so that by directly accessing the index records, you can access the appropriate goal records. Only a few extra statements in the file definition are required. General information on phantom structures, including suggestions on other uses, is available in the manual "SPIRES Technical Notes". The rest of this section discusses their use with formats.

Phantom structures may be used in output frames only -- they may not be used for input. [See C.8 to see how you can retrieve phantom structure data from records that already exist during input processing.] Unlike other subgoal processing, phantom-structure subgoal processing does not require you to retrieve the locator or key and establish it in a VALUE statement in the calling label group -- that linkage is handled automatically by the phantom structure. [See B.12.2.]

How to Code for a Phantom Structure

Assume that you are writing a format definition for a record-type (again, call it REC02) that has a phantom structure whose occurrences consist of records in record-type REC01. The simplest technique for retrieving elements in phantom structures is to simply call an indirect frame to process the structure, just like any other structure: [See B.8.]

Here "phantom.struc" and "phantom.process" represent the names of the phantom structure and of the indirect frame that processes it respectively. The LOOP statement is coded so that all occurrences of the structure will be processed.

The called indirect frame would begin like this:

The SUBTREE statement names the phantom structure that will be processed (the same name as in the IND-STRUCTURE statement in the calling label group).

This technique may be used for both same-file and subfile phantom structures.

Note that the phantom structure name is stored in the compiled format as a string that will be resolved at format execution time. This means that you will not necessarily have to recompile the format whenever the phantom structure file is changed (unless the change is radical).

Alternate (Historical) Techniques

There are four other methods available for retrieving elements in phantom structures. They are described here for "historical" purposes, since these techniques were required before SPIRES allowed you to simply name a phantom structure in a SUBTREE statement, like any other structure.

The first three begin as in the example above, with a label group that calls an indirect frame for structure processing:

The differences among the first three methods arise in the processing done by the indirect frame:

1) For same-file phantom structures (i.e., where both record-types are in the same file), the called indirect frame will probably begin like this:

The SUBTREE statement is required here (but see method 3 below). It names the record-type in the file whose records are retrieved as phantom structure occurrences. For instance, in the example, "&REC01" is the value for the SUBTREE statement, since the indirect frame will be retrieving element values from REC01. If desired, the NODEFQ statement may also be coded here to tell SPIRES not to look in the deferred queue but only in the tree for the occurrence of the phantom structure. [See B.12.5.]

The indirect frame may then retrieve element values from the phantom structure, using GETELEM statements and IND-STRUCTURE statements with more indirect frames, if structures within the phantom structure are accessed. [See B.12.4.]

2) For subfile phantom structures, where the two record-types are not in the same file, the indirect frame must call a separately compiled load format [See B.11.1.] to process the elements in the phantom structure:

3) The third method is available for both types of phantom structures (same-file or subfile). In the indirect frame, the elements to be retrieved will be named in variables in the GETELEM statements:

Here the element being retrieved is named in a variable checked during execution, not as a compiled value. Note that no SUBTREE statement appears here and no load format is necessary -- the indirect frame can retrieve all elements, including structures if they are handled in subsequent indirect frames, within the phantom structure.

4) The fourth method is also available for both same-file and subfile phantom structures. You do not need to code an indirect frame at all, but may instead retrieve a phantom structure element directly from a frame used to process other elements in the same subtree as the phantom structure. Like method 3, this method requires you to name the element to be retrieved, using a variable in the GETELEM statement:

The value for "phantom.elem1" should be an element whose name is unique in both the phantom structure and the current record-type. You can preface the element name with "phantom.struc.name@" to explicitly link the desired element to the phantom structure. Alternatively, you may use the 4-byte hex "ELEMID" for the variable value.

The fourth method is very inefficient if two or more different elements are being retrieved; one of the other methods should be used when you want to retrieve multiple phantom structure elements. (Retrieving multiple occurrences of a single element is not inefficient.)

In general, for same-file phantom structures, the first method is preferable to use, in terms of efficiency. For subfile phantom structures, the third and fourth methods are often the easiest to code, since you do not have to write a separate load format. Also, despite the fact that the element names are not compiled, the third and fourth methods may still be more efficient than the second method, since that involves loading a separate format to handle the phantom structure (although the format is only loaded once). In other words, for efficiency, neither method two nor three is obviously preferable to the other.

B.12.2  Same-File Subgoal Processing

Using subgoal processing, you may access records from other record-types in the same file ("same-file subgoal processing") or from the goal record-types of other subfiles, whether they are in the same file or not ("subfile subgoal processing"). Most of the rules are the same for both types; however, this section will cover the coding procedure for same-file subgoals, and the differences involved for subfile subgoals will be covered in the next section. [See B.12.3.] A complete example of a format definition containing same-file subgoal processing code appears at the end of the chapter. [See B.12.6.]

Same-file subgoal processing allows you to read records of any record-type in the same file as the currently selected subfile during either input or output of the current goal record. For example, you might want to publish one of the indexes of a subfile; that is, you would like some information from your goal records published in the sort order of one of your indexes. Using subgoal processing, you can design a format for the index records in the index that will use the locator or key values in the index to access information from individual goal records in the subfile. In such a case, the index record is the main goal; the goal record becomes the subgoal, since it is accessed through the index record, the main goal. [Note that you may be able to achieve the same effect through phantom structures or through the FOR INDEX command in Global FOR. See the manual "Sequential Record Processing in SPIRES: Global FOR", section 2.14, for more information, or EXPLAIN FOR INDEX COMMAND.]

To retrieve the data from the subgoal, you must do three things:

The second and third requirements are discussed in the next two sections. [See B.12.2.1, B.12.2.2.]

B.12.2.1  The Indirect Frame for the Subgoal: the SUBTREE Statement

The subgoal frame must have a SUBTREE statement in the FRAME-ID section of the indirect frame to specify the subgoal record-type:

where "record-name" is the RECORD-NAME of the subgoal record, found in the file definition. The ampersand indicates subgoal processing, as opposed to normal SUBTREE processing. [See B.8.2.]

The SUBTREE statement has a two-fold effect: first, it causes the format definition compiler to understand that the data elements coded within the frame's label groups are elements of the record-type specified by the SUBTREE value, that is, the subgoal record-type; second, the SUBTREE statement triggers the switching to this second record-type during format execution.

For example, suppose your output format is designed for "REC3", using "REC1" as the subgoal record-type. The indirect frame for REC1 would begin like this:

B.12.2.2  The Label Group that Invokes Subgoal Processing

During execution of a format for subgoal processing, you must specify a specific subgoal record to be retrieved. You must have either the key of the particular record to be used (in external or internal form) or a locator to that record if the record is REMOVED. If you are retrieving a goal record through an index (that is, the goal record is the subgoal of the index's format), each index record will have a pointer element containing either keys or locators to the goal records; since most files have REMOVED goal records, your indexes probably would contain locators.

Specifying the key of a subgoal record to be retrieved

To retrieve a specific record of the subgoal by its key, you code the following statements in the label group of the calling frame that invokes the subgoal's indirect frame:

Here "subgoal.frame.name" is the name of the subgoal frame -- NEW.RECORD in the above example. (The frame named here can be the same frame containing the calling label group, as long as that frame includes a SUBTREE statement.) "Keyvalue" would be one of the following:

If you omit the SET SUBGOAL Entry-Uproc shown, and if the SET VIALCTR Uproc is not in effect (see below), then SPIRES will assume that the value is a key in its internal form. However, for clarity's sake (if not sanity's), being explicit with the SET SUBGOAL Entry-Uproc is encouraged.

This example of subgoal access by key uses $PVAL, which was automatically set in the first label group, as the source of the VALUE in the second label group:

Retrieving Subgoal Records by Locator

To retrieve subgoal records by locator rather than by key value, you must announce your intentions in the calling frame; this is best done by setting the locator-value access mode with the SET SUBGOAL LCTR Entry-Uproc as follows:

where "locatorvalue" is a variable of type HEX holding a four-byte locator. The SET SUBGOAL Entry-Uproc here tells SPIRES that the value given in the VALUE statement is a locator, not a key.

Here is an example of an index record-type's data frame that calls an indirect frame for subgoal processing. The index record-type is REMOVED, meaning that the pointer element LOCATOR is a four-byte hex value; the variable LOC is declared a four-byte hex variable. Two other integer variables are used: OCCNUM and LASTOCC.

The frame puts the indexed value into the buffer, then retrieves the value of the first occurrence of the LOCATOR element, assigning it to the LOC variable, which is then used to retrieve the appropriate subgoal record. The OCCNUM variable is used to set the occurrence number of the LOCATOR element to be retrieved, so that each execution of the LOCATOR label group will retrieve a new occurrence.

There is one more type of SET SUBGOAL Entry-Uproc: SET SUBGOAL TRANSACTIONS, which can be used to connect to data about record transactions, which is stored in the deferred queue. [See B.12.8.]

Handling Errors: The DEFAULT Statement and the SET TESTSUBG Uproc

The most common problem that arises during subgoal processing is that the record being retrieved does not exist -- that is, the subgoal record-type has no record for the given key or locator. SPIRES's standard procedure is to skip the indirect frame call as well as the remainder of the calling label group if the subgoal record doesn't exist.

You can change that procedure in two ways. To make sure that the indirect frame is skipped but that the rest of the calling label group is processed, code the DEFAULT statement in the calling label group. [See B.4.5.1.] In some situations, though, you may want the indirect frame to be executed as well. You can guarantee this will happen by preceding the indirect frame call with the SET TESTSUBG Uproc, perhaps as an Entry-Uproc of the calling label group. [See B.4.5.6.]

Here is an example of the DEFAULT statement coded in a calling label group:

If no subgoal record is retrieved, the $DEFAULT variable is set, and the rest of the label group is processed; thus, processing would continue at the label group NOREC in the example. However, if you need the indirect frame to be executed anyway, you add the SET TESTSUBG Uproc to the start of the label group:

The indirect frame will execute as if a record had been found; however, the $NODATA flag will be set, and GETELEM statements within the frame will fail, taking action specified by any DEFAULT statements. [See B.12.7, E.2.1.27.]

The SET TESTSUBG Uproc has no values or options.

In the situation described above, SET TESTSUBG has an effect only if the DEFAULT statement is also coded in the label group. For subfile subgoal processing, described in the next section, it also affects how the format behaves when the specified subfile does not exist.

If SET TESTSUBG is not in effect and the subfile doesn't exist, then the format fails with an S866 error. If SET TESTSUBG is in effect, then the indirect frame is skipped, but the rest of the label group is executed. (So, if you need to test this condition, you must set a flag inside the indirect frame to indicate that it has been entered, checking it in the calling label group.)

(*) The SET VIALCTR and SET VIALLCTR Uprocs

An older way to request locator access is with the SET VIALCTR Uproc. SET VIALCTR tells the format processor that any further subgoal access during processing of the current goal record will consider locator-access as the default mode, not internal-key-access. To switch back to key access as the default mode later in the format, you can code SET NOVIALCTR as a UPROC in a label group before the calling label group. But again, using the SET SUBGOAL Entry-Uproc in all label-groups that call subgoal processing is the recommended course.

If the subgoal record is a large record segment, you can retrieve it using its four-byte hex locator as the subgoal value if you issue the SET VIALLCTR Uproc, similar to the SET VIALCTR Uproc, before attempting the subgoal call. You can use the $LRGLCTR variable [See E.2.1.5.] to retrieve the large record segment. SET VIALLCTR is like SET VIALCTR in that it establishes a new default mode; to change it back to internal-key-access, use SET NOVIALLCTR.

B.12.3  Subfile Subgoal Processing in Output Formats

SPIRES also allows you, through subgoal processing, to retrieve records from any other subfile to which you have access. Thus, if you had a subfile whose goal record contained keys or locators to goal records in any other subfile you could access, you can use subgoal processing to "tie the files together" through formats.

Subfile subgoal processing can be used to access subfiles within the same file, though generally it is more efficient to use same-file subgoal processing for this purpose; however, since subfile subgoal processing does not require that a SUBGOAL statement be coded in the subgoal's file definition (see below), you may be able to use subgoal processing on other record-types in a single file even if you are not the file owner (as long as the other record-types are goal records for some subfile you can access).

Subfile subgoal processing is invoked by SUBFILE and LOAD-FORMAT statements within the indirect subgoal frame. The SUBFILE statement follows the DIRECTION statement and looks like this:

where "subfile.name" is the name of the subfile being accessed through subgoal. The value "subfile.name" can be either a literal string or a string variable.

If there is a possibility that someone using your format may have access to other subfiles with the same name, you can insure that the correct subfile is used by preceding the subfile name with all or as much of the file name (starting from the beginning) as necessary:

where the "&" character tells SPIRES that what follows is all or part of a file name, followed by a subfile name. (Just the group prefix or the whole account prefix of the file name can be given; remember, however, that it is the prefix from the file name of the subfile you are accessing through subgoal, not just an account number that can select the subfile.)

Naturally, anyone using a format with subfile subgoal processing must have access to the "called" as well as the "calling" subfile -- otherwise, a NO SUCH SUBFILE error message will appear during record display, just the same as if the user had tried to select a subfile to which he or she did not have access. [The format will fail with an S866 error in this case, which can be avoided by preceding the indirect-frame call with the SET TESTSUBG Uproc. [See B.12.2.2.]] However, unlike same-file subgoal processing, subfile subgoal processing does not require that the file definition of the subgoal subfile contain a SUBGOAL statement; your ability to access the subfile replaces the SUBGOAL statement.

The LOAD-FORMAT statement follows the SUBFILE statement. The LOAD-FORMAT value must be a valid format name for the subgoal subfile. (Naturally then, the loaded format must be separately compiled.) [See B.11.] During record processing, the proper record-level frame of type DATA of the loaded format will be executed for each record accessed. To cause a specific frame of the loaded format to be executed, a USING-FRAME statement can be coded after the LOAD-FORMAT statement. [See B.11.2.]

Here is an example of a frame using subfile subgoal processing:

Note in the example that NODEFQ has been coded to assure that only the TREE of the subgoal subfile will be accessed. [See B.12.5.]

Remember that you must call this indirect frame from a label group that contains a value (either a key or a locator) determining which subgoal record to access. [See B.12.2.]

Note that:

There is a limit on the number of ORVYL data sets that can be attached by the user at one time. [EXPLAIN LIMITS IN SPIRES online for the current limit. For more information about this subject in general, see "SPIRES File Definition", section B.5.5.] If the sum of data sets attached by the selected subfile and several subfile subgoals exceeds the limit, an error from ORVYL (S20 -- too many devices attached) will result.

Subfile subgoal processing can be used with two subfiles in the same file, but same-file subgoal processing, discussed in the previous section, is more efficient for this purpose. However, it may be desirable to use subfile subgoal processing for this purpose to take advantage of the particular subfile privileges specified for the subfile.

Note that subfiles that have data base charging may not be accessed through subfile subgoal.

B.12.4  Accessing Subtrees Within a Subgoal

If you want to retrieve data elements within structures of the subgoal record, you must code an extended SUBTREE statement, which is a combination of the regular SUBTREE statement [See B.8.2.] and the subgoal SUBTREE statement described earlier. [See B.12.2.]

where "record-name" is the subgoal record name, and "structure.name" is the name of the desired subtree.

Of course, the standard rules for using structures in a goal record apply to using structures within a subgoal record; that is, if a subgoal frame (which is an indirect frame) of an output format accesses elements in the subgoal at the record level, and if elements at a structure level are needed, the subgoal frame may itself need to call an indirect frame.

Remember that the SUBTREE statement in such an indirect frame must include the "&" character followed by the subgoal record name. If the structure name is not unique in the subgoal record-type, the structural path to the desired structure should follow the record name in the SUBTREE statement. [See B.8.2.]

B.12.5  SUBFILE versus TREE Accessing of Subgoal Records

When accessing subgoal records, SPIRES will by default get the latest copy of a record; that is, SPIRES first looks for the subgoal record in the deferred queue. You may not feel that accessing deferred queue records is necessary, or worth the slightly higher cost. One way to bypass this access is to code the NODEFQ statement in the subgoal frame. The NODEFQ statement will cause SPIRES to read records of the subgoal record-type from the TREE only.

Here is an example of a subgoal frame using the NODEFQ statement:

Records of record-type REC3 will be read only from the TREE.

Note that the deferred queue of the subgoal records is not bypassed if the goal records are being processed by a FOR TREE command in Global FOR, unless the NODEFQ statement is coded in the format.

B.12.6  A Complete Example of a Format Using Subgoal Processing

Here is an example of a format that uses same-file subgoal processing:

     1.  ID = GQ.WCK.SUBREC;
     2.  MODDATE = FRI. FEB. 16, 1977;
     3.  DEFDATE = TUES. FEB. 1, 1977;
     4.  FILE = GQ.WCK.IOTEST;
     5.  RECORD-NAME = REC3;
     6.  VGROUP = LOCAL;
     7.    VARIABLE = NAME;
     8.      TYPE = STRING;
     9.    VARIABLE = LCTRVALUE;
    10.      TYPE = HEX;
    11.    VARIABLE = INT;
    12.      TYPE = INT;
    13.  FRAME-ID = REC3;
    14.    FRAME-DIM = 0,80;
    15.    LABEL = NAME;
    16.      GETELEM;
    17.      INSERT = 'NAME = ';
    18.      UPROC = LET NAME = $CVAL;
    19.      UPROC = LET INT = 0;
    20.      PUTDATA;
    21.    LABEL;
    22.      UPROC = SET VIALCTR;
    23.    LABEL = LOOP;
    24.      GETELEM = LOCATOR::#INT;
    25.      DEFAULT;
    26.      UPROC = LET LCTRVALUE = $UVAL;
    27.      UPROC = IF $DEFAULT: JUMP NAME2;
    28.    LABEL;
    29.      VALUE = #LCTRVALUE;
    30.      IND-FRAME = REC1;
    31.      UPROC = LET INT = #INT + 1;
    32.      UPROC = JUMP LOOP;
    33.    LABEL = NAME2;
    34.      VALUE = 'END OF DATA FOR '#NAME;
    35.      PUTDATA;
    36.  FRAME-ID = REC1;
    37.    SUBTREE = &REC1;
    38.    FRAME-DIM = 0,80;
    39.    LABEL = ID;
    40.      GETELEM;
    41.      START = X,10;
    42.      INSERT = #INT ' - ID = ';
    43.      PUTDATA;
    44.    LABEL = S2;
    45.      IND-STRUCTURE;
    46.      IND-FRAME = S2;
    47.      LOOP;
    48.    LABEL;
    49.      GETELEM = STATE;
    50.      START = X,10;
    51.      PUTDATA;
    52.  FRAME-ID = S2;
    53.    SUBTREE = &REC1@S2;
    54.    FRAME-DIM = 0,80;
    55.    LABEL = ITEM;
    56.      GETELEM;
    57.      START = X,15;
    58.      INSERT = 'ITEM - ';
    59.      PUTDATA;
    60.    LABEL = STOCK;
    61.      GETELEM;
    62.      START = *,*+3;
    63.      INSERT = 'STOCK - ';
    64.      PUTDATA;
    65.  FORMAT-NAME = SUBREC;
    66.    ALLOCATE = LOCAL;
    67.    FRAME-NAME = S2;
    68.      FRAME-TYPE = INDIRECT;
    69.    FRAME-NAME = REC1;
    70.      FRAME-TYPE = INDIRECT;
    71.    FRAME-NAME = REC3;
    72.      FRAME-TYPE = DATA;

In this example, the format, designed for a subfile whose goal record is REC3, accesses data in REC1 through subgoal. The REC3 frame calls the indirect subgoal frame from the label-group beginning at line 28. Since the subgoal record is being accessed by locator values, SET VIALCTR is coded at line 22. Each locator, as it is retrieved, becomes the value for the HEX variable LCTRVALUE (line 26); using this variable as a value, the calling label-group can access a particular record of record-type REC3. Several elements are accessed from the subgoal record, including a structure, S2 (line 44) which must be coded as an indirect frame (lines 52-64); note the syntax of the SUBTREE statement in line 53, which includes the subgoal record name.

B.12.7  (*) Subgoal Processing Without Data

Situations arise in which subgoal access to another subfile is desirable, though not for the purposes of retrieving data from a specific record. In these cases, the SET TESTSUBG Uproc or the NODATA statement is useful for allowing subgoal access to be established despite the absence of a subgoal record.

For example, suppose an application needs to retrieve information about elements in a subfile selected dynamically by the user during format execution. The format needs to use subfile subgoal processing to select the subfile, but it has no subgoal record to retrieve. Instead, it needs to use the $ELNOTEST function to find out about the goal record-type's elements. (The Prism application Screen Definer does this.)

Either the SET TESTSUBG Uproc or the NODATA statement will help you solve the problem. If you use the SET TESTSUBG Uproc, your calling label group might look like this:

Notice that you must still include a VALUE statement in the calling label group even though you don't want to retrieve a specific record. It is the concurrence of the VALUE and IND-FRAME statements that triggers subgoal processing in SPIRES, so both must be present. However, by assigning a null value to VALUE, you tell SPIRES not to try retrieving any record. (You could instead specify the key of a non-existent record, but SPIRES would go to the trouble of trying to retrieve it first.)

During execution, SPIRES would then execute the ENTERSUB frame, with the $NODATA flag set. [See E.2.1.27.] Any attempts to retrieve element values through GETELEM statements would fail, and SPIRES would execute any DEFAULT statement processing.

The NODATA Statement

The NODATA statement works similarly, except that it is coded in the called frame, as shown below:

Remember that the calling label group must still contain a VALUE statement as described above; otherwise, subgoal processing will not occur.

When SPIRES begins executing the indirect frame, the NODATA statement tells it not to retrieve any record. SPIRES executes the frame with the $NODATA flag set. Attempts to retrieve element values through GETELEM statements will fail, and SPIRES will execute any DEFAULT statement processing.

B.12.8  Subgoal Access to Transaction Records

The deferred queue keeps important data about each transaction, such as the account that made the transaction, the date and time it was made, the command that was issued, etc. The file owner has the option of creating a special record-type in the file that makes the transaction data available as if the data for each transaction were a goal record in that record-type. That record-type is the system record-type called $TRANSACTIONS. It is discussed in detail in the SPIRES manual "File Management"; online, EXPLAIN $TRANSACTIONS RECORD-TYPE.

Under the Global FOR command FOR TRANSACTIONS, you can work not only with the tree and current goal copies of a record, but with all copies of a record that have gone into the deferred queue since the file was last processed. If you are examining those records through output formats, you may want to retrieve some of the transaction data for each one as well. That can be done from a format with any of the following:

To set up the subgoal processing of the $TRANSACTIONS record-type from the format for the goal record-type, you should have a calling label-group that looks like this:

Notice that no value is given in the VALUE statement. SPIRES is already aware of the $TRANSACTIONS record "key" that it needs to fetch the transactions data. But remember, for subgoal processing to be in effect, the label group must have a VALUE statement.

The indirect frame you call, where you will retrieve and work with the transaction elements, should begin like this:

where "recnm" is the name of the record-type, from the file definition, that is defined by the $TRANSACTIONS system record-type. Its name will begin with a period, as shown here.

The elements in the $TRANSACTIONS record-type that you can retrieve in the indirect frame are described in detail in the "File Management" manual.

If you use the format but are not working under FOR TRANSACTIONS, the subgoal record being retrieved will not occur. If desirable, you can use DEFAULT processing to handle that situation.

B.13  Full Screen Output Formats

In a full-screen application, the entire screen, rather than a single line of it, is transmitted back and forth between the computer and your terminal. For data entry, for instance, the computer might display the outline of a form, and the user might respond by filling record data into appropriate places on the screen. When all the data for the record was typed onto the screen, the user would press the ENTER key to send the contents of the screen back to the terminal for input processing.

Although full-screen applications are most commonly developed to make data entry easier, they usually allow the user to display the stored records as well. For data display, for example, you might type a command into a particular area of the screen, and the rest of the screen would be used to display a record or perhaps a group of records. Portions of the data displayed may be made brighter or dimmer than other portions or made to flash or to appear in "reverse video".

Full-screen applications usually combine several major components of SPIRES that must work closely together, including formats. Formats are used in full-screen applications for the same reasons they are used in non-full-screen applications: to arrange the data on the screen in an attractive design, for example, or to retrieve input data values from locations scattered around the screen.

Because a format for a full-screen application is usually designed and coded simultaneously with protocols and global vgroups, this manual is not the appropriate place to teach you how to design full-screen formats. A special chapter of the manual "SPIRES Device Services" is called "Full-Screen Programming" -- it discusses how to design, code and put together the pieces of such applications. In terms of formats instruction, that chapter relies on your understanding of the basics of formats development: the sections of the format definition, the FORMATS subfile, compiling formats, and so forth. Thus, you should be familiar with the material in the first nine chapters of Part B before you read that chapter in the "Device Services" manual. An understanding of XEQ frames is also useful. [See D.2.]

Note that some format definition statements mentioned in this manual are usually used in full-screen formats, e.g., DISPLAY and ESTART. Details on their use appear in the other manual.

B.14  General File Formats for Output

General file formats allow you to use the same format for separate record-types. There are two types of general file formats:

Type 1: a General File Format for Identical Record-Types

A general file format is most handy when you have a format for a record-type in one file that you would like to use for an identical record-type in another file (or even the same file). For instance, you might have a production file and a parallel test file, and you would like to use the same format for a record-type in each. Trying to maintain two or more format definitions identically can be a chore -- it is far easier to declare one format definition a general file format, which can be set for any selected or attached record-type that has the same definition.

A general file format may also be useful when an application has multiple identical files, such as an online mail system, where each user might have his or her own SPIRES file to store kept mail. If the mail system constructs the files identically, they can all use the same general file format. When changes are made to the format, they are in effect for all the files at once; they do not need to be made individually for each file.

Type 2: General File Formats for Non-Identical Record-Types

General file formats may also be used when the goal record structure is only known at the time of format execution. As an example, consider the system format $PROMPT, which is a general file format. When you select a subfile and set the $PROMPT format, it finds out what elements are in the record-type, whether they are structures, whether they are singly or multiply occurring, whether they are required, etc. The $PROMPT format definition does not have any of that information in it; it must find out all those details when it is set for a given record-type. General file formats of this nature are constructed quite differently from the type described in the first two paragraphs. Because they are not commonly written by users, they will be discussed only briefly at the end of this section.

To declare the formats in a format definition to be general file formats, you add the GEN-FILE statement to the end of the definition:

The GEN-FILE statement applies to all formats of the format definition.

A special form of the SET FORMAT command is used to set a general file format:

where "format.name" is the name of the format as it appears in the FORMAT-NAME statement; the format name may not contain any internal blanks. [See B.5.1.] The fully qualified form, including the account number of the format definer, need only be used if you are not the format definer. Note that the double asterisks must be specified even if you are setting the format for the file and record-type named in the format definition.

The SHOW FORMATS command will list general file formats in only two situations:

For "Type 1" general file formats, the FILE and RECORD-NAME statements contain the names of one file and one of its record-types that can use the general file formats. The compiler will use that record-type as a template when the definition is compiled, verifying the existence of the named file, record-type and elements, etc.

Though "Type 2" general file formats are more common to SPIRES system formats than user formats, they will be briefly discussed here. No FILE or RECORD-NAME statements are coded for this type, since it may be set for any record-type (at least in principle). Also, the format may not name any data elements; all GETELEM (or PUTELEM statements in input formats) must be in the form:

The variable named is often in an array whose values are assigned the names of the elements. The array is usually established in an initial or startup frame, by making use of the $ELEMID variable and $ELIDTEST function. [See E.2.3.10.]

A general-file format may, for example, ask the user what element should be displayed. The label group below, part of a startup frame, asks that question and verifies that the element named is a real one:

If the $ELEMTEST function returns a null value, then the user's response is not a valid element name for the selected subfile.

In the data frame that retrieves the element's value, the $ELEMID variable can be used in the GETELEM statement:

Using the $ELEMID value, established by the earlier $ELEMTEST function, in the GETELEM statement is more efficient than using a string variable containing the element's name.

B.15  Output Formats for Partial Processing

Indirect frames used to process structures during record processing may also be used to process structures under partial record processing. [Partial record processing is introduced in the manual "SPIRES Technical Notes", section 12.]

The indirect frame must be declared in the format declaration section twice, for each of its two purposes: INDIRECT, when it is invoked by a data frame, and STRUCTURE, when used for partial record processing.

When the DISPLAY format is set and partial FOR processing is in effect, SPIRES will look for a structure frame for the structure named in the "FOR element" command. SPIRES must first scan the list of frames declared for the format for any structure frames, and must then check the SUBTREE statements within them to see if any of them are defined for the structure being processed.

Note that a structure frame does not have to be an indirect frame too. It may be defined solely for partial processing purposes. In that case, it would only be declared once in the format declaration section: as a structure frame.

The structure frame may call indirect frames, possibly for the purpose of processing structures within the structure. Those indirect frames do not have to be declared structure frames unless deeper-level partial processing will occur and those frames are meant to be used for it.

No other frame may call a structure frame. If you want that frame called from another frame, it must also be declared as an indirect frame in the format declaration section.

A structure frame may be coded without the SUBTREE statement in a general file format. The frame may be invoked under Partial FOR processing by issuing a command such as DISPLAY with the "USING frame" prefix to name the desired frame. [See D.1.1.1.]

A structure frame may also have a null value for the SUBTREE statement ("SUBTREE;"), indicating that the frame is designed to handle element retrieval from any structure or even at the record level. Under partial processing, that frame could be executed at any level. This procedure is similar to that of omitting the SUBTREE statement, as described above, except that the user would not have to precede the frame-executing command with the "USING frame" prefix.

Formats, Partial Processing and the "FOR *" Command

The "FOR *" command when issued after a REFERENCE command, tells SPIRES to hold the current record in main memory until the "ENDFOR *" command is issued. Multiple DISPLAY commands will display this "in-core" version of the record. If a format is set, the data frames of direction OUTPUT will be used to display the entire record each time. This capability may be used when you want to look at the entire record from time to time during partial processing. It is most often used in full-screen applications.

B.16  Output Formats with Device Services

This chapter introduces capabilities arising from the union of output formats and device services. Reading the SPIRES manual "Device Services" should be considered a prerequisite to reading this section.

Although full-screen output formats were mentioned earlier, they are only one way in which formats work with device services. Formatted output can be directed to other device services areas besides CRT (the "full-screen" area). Using the IN-AREA statement in a format definition, you can also direct formatted output to several different areas during the execution of a single output format. The output can go directly into another subfile as input using the special SBF area. [See B.16.1, B.16.2, B.16.4.] The IN-AREA statement also lets you direct formatted output to one or more areas during the execution of an input format. [See B.16.3.]

B.16.1  Directing Output Frames to Multiple Areas: The IN-AREA Statement

The IN-AREA statement lets you specify a particular device services area for the output from an output frame. The named area must be defined prior to the execution of the format.

where "area.name" is the name of the area in which the frame's output should be placed, specified either as a literal string or in a string variable that will hold the name when the format is executed. The named area may not be ACTIVE -- the active file is not an actual device services area.

The IN-AREA statement may appear in either of two places in a format definition, depending on whether the frame is a data frame or an indirect frame.

Remember that you may have multiple data frames, which will be executed in the order in which they are named in the format declaration section. [See B.5.2.]

Currently you cannot code the HOLD Uproc in a frame having the IN-AREA statement.

The IN-AREA Statement and Input Formats

The IN-AREA statement allows you to output formatted data from an input format as well. It could be useful, for example, if you wanted to store the input changes to a record being updated. You could gather the new data in variables as the input format was executing, and then, in the output frame, format them and send them to a device such as an ORVYL file. This is allowed in both SPIRES and SPIBILD. Of course, remember that you must set up the areas (DEFINE AREA) before you execute the format.

B.16.2  The IN-AREA Statement with Report Formats

The IN-AREA statement also lets you create multiple reports with a single command, sending each report to a different area. When multiple reports are needed, retrieving each record once for all the reports at once is more efficient than retrieving each record for each report. SPIRES will keep separate values of the report variables such as $LINEPP, $PAGENO, $LLEFT, etc., for each report.

The signal to SPIRES to produce multiple reports appears in the initial frame of the report. Either multiple initial frames are declared in the frame declaration section, with those beyond the "first report" having IN-AREA statements there:

or the initial frame for the first report may call indirect frames to initialize each of the other reports, with the IN-AREA statement appearing in the calling label group:

If the latter method with indirect frames is used, the IN-AREA statement must not appear in the declaration sections for those frames -- it must appear in the label group that calls them.

In general, it is best to code the IN-AREA statement in the frame declaration section for the report-specific frames (as well as data frames) for each report rather than code the frames as indirect frames called from the report-specific frames of the first report. This guideline is particularly true for header and footer frames, which will not work properly if they are called as indirect frames. The rules given for coding the IN-AREA statement in the last section must also be followed. [See B.16.1.]

The SET REPORT command or Uproc sets report mode for the first report. Then, if there are any initial frame declarations containing IN-AREA statements, report mode will be set for the data going to those areas as well. In other words, to set report mode for a report going to a secondary area, you must code an initial frame declaration for that area (as shown in the first example above) and then set report mode for the primary with the SET REPORT command or Uproc. A report will not be created in a secondary area unless report mode is set for the primary area.

Here is an example of the format declaration section for a report format that creates multiple reports. The data for "Report 2" will go to the area REPT2, while the data for Report 1 can go to any area desired, depending on the "IN area" prefix added to the record-processing command.

Notice that the indirect frame LINE.ITEM2 does not have the IN-AREA statement coded in its frame declaration; IN-AREA may not be coded there but must be coded in the label group that calls LINE.ITEM2.

How would you create the actual reports? First, you need to define and assign the areas, probably to separate ORVYL data sets. Here is a set of commands that could be used if you wanted Report 1 to go to a file called REPORT1 and Report 2 to the file REPORT2:

Of course, the data for Report 2 will go to the area REPT2 because of the IN-AREA statements for those frames. The data for Report 1 will go to the area REPT1 because no IN-AREA statement was coded for those frames but the DISPLAY ALL command was preceded by the IN REPT1 prefix.

B.16.3  Creating Reports During SPIBILD Input

You can also use the techniques discussed above to create reports during SPIBILD input processing. [See B.16.2.] Coding details are somewhat different, however, and the differences are discussed below.

In SPIBILD, the SET REPORT command and Uproc are not recognized. Report mode for the reports in secondary areas must be set in a more roundabout fashion than the method shown above. You must code an initial frame of DIRECTION = INPUT, which in turn calls an indirect frame of DIRECTION = OUTPUT:

The frame declaration section would then include the following:

SPIBILD will only execute an initial frame if its direction is INPUT. When the INITIAL.IN frame calls the INITIAL.OUT frame, report mode gets set for INITIAL.OUT because INITIAL.IN is an initial frame.

The same technique must be used for ending frames too: because SPIBILD will only execute ending frames whose direction is INPUT, you must code an ending frame of direction INPUT that calls an indirect frame of direction OUTPUT to serve as the ending frame.

This technique is not necessary for header and footer frames; SPIBILD will execute them as necessary, even though their direction is OUTPUT.

Group-start and group-end frames do not work in SPIBILD, and should not be used.

More information about SPIBILD formats appears later. [See C.9.]

B.16.4  Directing Output Data to Another Subfile as Input

A special device services area called SBF (for "SuBFile") can be used in conjunction with the IN-AREA statement (and several other SPIRES constructs that are needed) to direct output data directly into another SPIRES subfile as input. This lets you copy data from one subfile to another in a very simple manner; and since you can write to multiple areas, you could add or update records in multiple subfiles at once using the data in the current output record.

Besides the IN-AREA statement described in the previous section [See B.16.1.] you need the following:

To establish the subfile device, you issue a DEFINE AREA and an ASSIGN area command:

where "areaname" is the area named in the frame's IN-AREA statement. The "columns" parameter is an integer representing the width of the area, which should be at least as wide as the longest value you fetch from the output record or place in the input record.

The "pathnum" is the number of the established path for the subfile to which the input should be sent.

You do not need "srows" or "scols" after the ON SBF part of the DEFINE AREA command; even if you define multiple areas on SBF, only one can be using it at a given instant (the moment of the PUTDATA; see below), so the areas can overlap on SBF without any adverse effect.

To invoke this processing within the format, you code the SET ROW UPDATE Uproc, usually in the frame declaration section of the frame whose output is going to the SBF area, e.g.,

The syntax of the SET ROW UPDATE Uproc is:

You must specify either ADD or ADDUPD, depending on whether the input records are all new records or they include updates. Whichever you choose, note that SPIRES is passing them to the target subfile as entire records; this procedure does not let you merge data into existing records. For ADDUPD, SPIRES will try to add the record as a new record, but if a record with the provided key already exists, SPIRES will treat the data as an update record. For ADDMERGE, SPIRES will try to add the record as a new record, but if a record with the provided key already exists, SPIRES will treat the data as merge data for the existing record.

The FAST option is the same as the WITH FAST prefix for the INPUT BATCH command, described in the manual "SPIRES File Management". Online, [EXPLAIN WITH FAST PREFIX.] It makes the processing much faster, but has severe restrictions.

The frame should be defined with frame dimensions of "1,n" (1 row by at least the number of columns of the longest value you will send to the target subfile).

Within the frame, you move an element value from a label group's $CVAL into the target subfile via the label group's SET COLNAME Uproc and the PUTDATA statement -- together, these two statements are the same as a PUTELEM statement in an input frame:

The HIRE.DATE element is retrieved from the current record in the primary subfile. It is to be the value of the START.DATE element in the new record for the target subfile, as specified in the SET COLNAME Uproc. When the record goes to the target subfile, the input values will be processed through the goal record-type's Inproc rules, so in this label group, you want to create element values in their external form for the target subfile.

Note too that the START statement's row is the current row. You keep putting the data for a single record into the same start row, starting in column 1, each PUTDATA for a new element "overwriting" the previous one in the buffer. Essentially, the SET COLNAME Uproc changes the way SPIRES handles the buffer. When the PUTDATA is executed, the data is put into the buffer, and the buffer is passed to a temporary storage area in memory where SPIRES puts the accumulating data for the record being built. It keeps a separate area for each record being built, until the record is added to the target subfile. Hence, you can keep putting new element values into the buffer with a START value of "*,1" -- the buffer starts out empty each time a PUTDATA occurs in this context.

Putting data into a row beyond the current row "flushes the buffer"; here that has a slightly different meaning than it normally does, in that it signals the device processor to move the record to the subfile. The data being put into that "next row" would then go into the next record. In other words, you can send multiple records to a target subfile in a single execution of a frame.

The FLUSH Uproc or a return from the frame will also cause the record to be moved to the target subfile.

Handling Errors

The process of adding or updating records to a target subfile this way is very similar to the way SPIRES handles adds and updates from the active file using the standard SPIRES format. When a record is moved to the target subfile, it is processed through the goal record-type's Inprocs all at once; you cannot control and verify each element's Inproc processing the way you can in a normal input format.

You can test $SNUM before and after the record has been "flushed" to the target subfile to check for success; and since the format will continue executing, you may have other options for correcting the error or making other arrangements. However, error handling within the output frame for input problems in the target subfile is definitely more limited and more clumsy than it is in a normal input frame.

Another important point to understand is that once a record has been sent to and accepted by the target subfile, it is a committed transaction. That means that subsequent failure of the format while still handling the current primary record (in other words, during the current pass through the format execution) will not dequeue the successful transactions in target subfiles already done in this pass. If that is a concern, you can use transaction group processing to ensure that a failure will cause the preceding target-subfile transactions to be backed out.

Handling Structures

To create structural occurrences in a record you are creating for the target subfile, you follow the same principles as you would for creating them in your active file for input using the standard SPIRES format.

Here are the specifics:

Merging into Specific Occurrences with ADDMERGE

If you need to merge data into specific element occurrences when using SET ROW UPDATE ADDMERGE, you specify the occurrence number by adding a "SET PUTOCC = n" Uproc [See C.3.3.1.] to the label group. Note that the "n" value starts at 0 (zero) for the first occurrence.

Removing element Occurrences with ADDMERGE

If you wish to remove data element occurrences from a particular record you may do so by coding "SET REMOVEL" UPROC. [See C.5.1.4.] A sample lable group would look something like this:

Accessing SBF device area information

You may find that you need to know which subfile is being updated during SBF area usage -- you may need to access that subfile with $LOOKSUBF for example. The subfile name and file name may be retrieved by means of the $GETAREA function using the FILE or SUBFILE codes. [EXPLAIN $GETAREA FUNCTION.]

C  Input Formats

C.1  Introduction to Input Formats

Input formats can be developed to place data into a SPIRES data base. The format may be designed for adding new records or replacing old records or changing parts of old records. The format may gather the data to be used either by prompting the user at the terminal for input (like the system format $PROMPT) or by reading the data from the active file or a device services area. file.

Despite the variety of purposes, the principles and techniques of input formats are easy to learn, especially after learning about output formats. Many of the statements are the same, some are simply reversed (GETELEM and PUTDATA become GETDATA and PUTELEM, for instance), and some have some new values.

This part of the manual will be similar to the preceding part on output formats. The first few chapters will take you step by step through the creation of an input format. Cross-references to the previous part will frequently be made in order to reduce the repetition of details where they are the same for input and output. Later chapters will cover special types of input formats based on the general form discussed in the first chapters, such as SPIBILD input formats and general-file formats for input.

Remember that the colloquial term "input formats" is misleading in that a format can be used for many different purposes (input and output) by many different commands. The FRAME-TYPE and USAGE statements determine which frames of the format will be executed for a given command.

C.1.1  Do You Need a Custom-Designed Input Format?

Without a doubt, output formats are developed more frequently than input formats. One possible explanation for this is that data input is more commonly done by computer professionals, while data output is often generated for laymen. And chances are that anyone who has read this far in the manual already feels relatively comfortable using the standard SPIRES format or the $PROMPT format for data entry. Both of these are worth considering as alternatives when you are deciding whether to design your own input format.

Here are some notes on the standard SPIRES format, described in detail in the SPIRES manual "Searching and Updating", section D.1.2:

Here are some notes on the $PROMPT format, with details provided in the same manual, section D.6:

Certainly if you are creating your own data base and considering input methods, you should determine whether one of these two formats, which require no special coding or design, might fit your requirements.

However, you might want to create an input format for other reasons:

There are other good reasons why you might want to design your own input format, based on the limitations of the standard and $PROMPT formats. But to repeat, consider whether they might be equally effective before you go to the trouble of writing your own.

C.1.2  General Differences Between Input and Output Formats

Input and output formats are different in more ways than just the direction of data transmission. The record-handling commands with which they are used have very different limits and capabilities. Also, although the same format definition structure (and often the same format definition) must support both types, there are some noticeable differences in the statements, statement values and organization between them.

Although SPIRES can process multiple records for output, with or without an output format set, e.g., the DISPLAY ALL command, the SPIRES processor can only process one record per command for input. Multiple records can be added to the data base in a single command in the SPIBILD processor, however, and input formats may be used there. [See C.9.]

An output format may retrieve data from several record-types in a single execution of the format, using phantom structures or subgoal processing. [See B.12.] An input format may retrieve data from other record-types, using those same techniques, but usually updates only one record in one record-type. [See C.8, C.13.]

In terms of data transmission, consider that an output format has one primary source for data (the SPIRES data base) and the data has either of two destinations: the terminal screen, where it appears directly to the user, or to a data set, such as the active file. Conversely, the destination for the data in an input format is the SPIRES data base, but it may come either from the user at the terminal directly (responding to prompts) or from a data set in which the user has placed the data.

For output formats, the same frame may easily serve both destinations, but for input formats, the same frame seldom serves both sources. An input format that reads data from an external data set must read that data into the format buffer, retrieving the individual pieces as directed by the format definition. A prompting input format does not need to use the buffer, since the user's responses to the prompt are placed in variables, which can be directly put into the record being constructed. Because they use the buffer, input formats that read data are more similar to output formats than prompting ones are. An output format that retrieved data elements and simply sent their values to the terminal via the * Uproc rather than put them into the buffer with PUTDATA statements would be a close counterpart to a prompting input format. The two different types of input formats are discussed in two different chapters. [See C.3, C.4.]

General coding techniques for input and output formats remain similar, at least for record-level elements. A label group, for instance, can still only handle one element at a time. Structures, on the other hand, are more easily handled in input formats because indirect frames are not required. [See C.3.9.2.]

One aspect of record input can make coding input formats more complicated than coding output formats, especially input formats that read data: unpredictability. Output formats are usually processing known quantities: SPIRES can tell you how many occurrences of an element appear in a record, how long they are, etc. Moreover, in terms of data values, the records already in the data base waiting to be displayed have already passed through processing-rule checks to ensure, for instance, that an element's value is either YES or NO, or only comprised of digits.

But the user is supplying new data to the input format. If you are as sure about the data coming into an input format as you are about the data going out through an output format, your input format can be as simple, accordingly. (The input data might already be strictly formatted, perhaps coming from another program.) And your input format is certainly able to use the INPROC rules associated with the record elements in the file definition. However, if your format's users might put data in the wrong places in the input data set or forget to signal the end of data values or make some other similar mistake, your format should be able to handle the problem, even if the solution is simply to realize that a mistake was made and abort further format processing. (Sometimes detecting that is not as easy as it sounds.)

Remember who your users are, however. An input format designed for users who are carefully trained to use it probably does not require the data-verification processing that an input format used by the general public might. Remember too that such verification adds overhead in terms of processing costs: if a complicated test performed against each input record finds only one problem in every 1000 records, then 999 records may not have needed it. [See C.3.10.]

C.1.3  The Input Format Definition

An input format definition has the same parts as an output format definition: the Identification section, the Vgroups section, the Frame Definitions section and the Format Declaration section. The statements in the Identification section of an input format are identical to those of an output format:

These statements were discussed earlier. [See B.2.]

User variables may be useful to an input format too. Either global or local variable groups may be used, just as in output formats. If a local vgroup is desired, it is declared in the Vgroups section, exactly as for output formats. [See B.9.3.1.] The statements in the Vgroups section are:

The indentation shows the structural hierarchy of the Vgroups section. Multiple vgroups containing multiple elements, each with their own characteristics, may be defined.

The statements in the Format Declaration section are also identical to those in output formats. [See B.5.] The most common statements are shown below:

As you can see by the inclusion of the SORT-SPACE statement, sorting with the $SORTKEY array is allowed in input formats too, even though it is more commonly associated with output. [See B.10.8.] Other statements are allowed too, notably the PROCDEF and other proc statements. [See B.4.5.2, E.1.]

An input frame definition varies considerably from its output counterpart. For example, the FRAME-DIM statement must be omitted if the data is to be gathered by prompting the user rather than by reading it from a dataset or a full-screen terminal. And although the USAGE statement is present, its values will be different than they were for output frames.

Within a label group, new statements will be introduced, such as GETDATA, PUTELEM, REMELEM and SCAN. Many other statements useful for output are useless during input and will not appear: GETELEM, TITLE, TSTART, DEFAULT and PUTDATA, among others. In addition, some new Uprocs will be useful: SET ERROR, SET PUTSTRUC, etc.

The new label group statements will be covered within the next few chapters. Specifically, the next chapter will cover the statements in frame definitions that are not related to label groups (i.e., USAGE and FRAME-DIM, primarily). [See C.2.] The following chapter will discuss label group statements for input formats that read data from a dataset [See C.3.] while the chapter following that will cover those statements necessary for prompting input formats. [See C.4.] Another type of input format, merge formats, which are used under the MERGE command for record updates, will be discussed in the following chapter. [See C.5.] A brief discussion of the Format Declaration section will follow that. [See C.6.] A chapter reminding you how to compile and use your format definition will follow that. [See C.7.] The remaining chapters in Part C will cover other special types of input formats.

C.2  The Frame Definitions Section of an Input Format Definition

The Frame Definitions section is comprised of one or more frame definitions, each of which consists of two parts: frame identification statements and label groups. The frame identification statements for input frames are the same as those for output frames though they often have very different values. They will be discussed in this chapter; input label groups will be discussed in the next one. [See C.3.]

The frame identification section may contain these statements, most of which are familiar from output frames:

The first four are discussed in this chapter. The others have already been discussed. [See B.3.]

C.2.1  The FRAME-ID Statement

A frame definition must begin with the FRAME-ID statement, which signals the beginning of the frame defintion and provides a name specified in other statements (such as the FRAME-NAME statement in the format declaration section).

The frame name may be from one to sixteen characters long. No blanks are allowed. Few special characters (i.e., not alphabetic or numeric) are allowed in the name either, though useful exceptions include periods, hyphens and underscores, used as substitutes for blanks, which are not allowed.

C.2.2  The DIRECTION (DIR) Statement

This statement specifies the direction of the data mapping (in other words, whether the frame is for input or output). A frame used for data input will have the value INPUT for this statement:

The input commands ADD, UPDATE, ADDUPDATE, MERGE and BATCH (i.e., BATCH in SPIBILD only) may cause the execution of input frames. This does not mean that such frames will necessarily be executed when one of these commands is executed; that depends on other factors as well, primarily the USAGE statement. [See C.2.4, C.6.] A special type of input frame, INOUT, is used primarily in full-screen applications. [See D.1.2.]

For input frames, be sure to code this statement. By default, if no DIRECTION statement appears, SPIRES assumes the frame will be an output frame. [See B.3.2.] A frame may have only one DIRECTION statement.

Remember that the direction of an indirect frame usually matches that of the frame that calls it, though situations occasionally arise in which that is not the case. [See D.1.2.]

C.2.3  The FRAME-DIM Statement

The FRAME-DIM statement in input frames has several functions:

The syntax of the FRAME-DIM statement is the same as for output frames:

where "nrows" is the number of rows in the buffer and "ncols" is the number of columns. If "nrows" is 0 (zero), then line-by-line processing will be in effect (see below). If "ncols" is 0, then the width of the area containing the data (such as your active file) will be used. [See B.3.3.] If the area used is your active file, "ncols" will be the value of $LENGTH. For formats used in online or batch SPIBILD, the default value of "ncols" is 130 columns. [See C.9.1.]

When the frame is executed, SPIRES will construct a buffer in the computer's memory having the dimensions given. Then data will be read from the source area into the buffer, according to the FRAME-DIM statement and the command issued. Subsequent GETDATA statements in the frame's label groups will read values from the buffer. Unless line-by-line processing is in effect (see below), data will only be read into the buffer once per record. The buffer will be discarded after the frame finishes executing, except when the HOLD Uproc is coded in multiple-record input formats. [See C.9.1.]

For example, suppose your active file contains data like this (column numbers are indicated above for reference purposes):

           (....v....1....v....2)
    1.      Emmet Amana Dintlyke
    2.      1935 W. Rogers Way
    3.      Postly, WY  83004

If the FRAME-DIM statement for an input frame were "FRAME-DIM = 3,20" then that data would fit in it. However, if the statement were "FRAME-DIM = 2,13", the buffer from which the data could be read would look like this:

Only two rows worth of data, each 13 characters long, would be read from the active file.

The amount of data in your active file and the syntax of the issued input command may also affect the buffer. For example, rows at the end of the buffer will be left blank when data is not available for them, perhaps because the record is short or because the number of rows read from the active file was restricted by the USING option, as in "ADD USING F/4". Whether the blanks are significant or not depends on the options specified in particular GETDATA statements. [See C.3.1.]

Remember, if an input frame will be prompting the user for data rather than reading it from a data set, do not code a FRAME-DIM statement in it. [See C.4.1.]

Line-by-line Processing of Input Frames

If you code "0" for "nrows" in the FRAME-DIM statement, you are requesting line-by-line processing. The buffer constructed will only be large enough for one row. When such a frame begins executing, SPIRES reads a line of the data into the buffer. When data is requested from another row by a GETDATA statement that requires data from another row, the current row is discarded, and the requested line is read into the buffer. The data set can only be read in a forward direction, meaning that line 2 cannot be read into the buffer after line 3.

Line-by-line processing for input thus works much as it does for output. [See B.3.3.] If element values proceed linearly through the data set, line-by-line processing is the easiest and most efficient method to use for data input. However, if two different element values appearing side by side each wrap into subsequent rows, line-by-line processing will not be possible.

The START, MARGINS and MAXROWS statements can be used to tell SPIRES to read data from the next line of input into the buffer, though it does not actually happen until the GETDATA statement is executed. [See C.3.5.1, C.3.5.3, C.3.5.4.] If SPIRES tries to retrieve the next line from the data set but no more lines of data exist, the system flag $ENDATA is set. Hence, a label group that reads data should test the value of $ENDATA to make sure that the end of the data was not reached, for subsequent GETDATA statements when no more input lines are available will cause the format to abort. Many of the examples in this chapter show such a test. [See C.3.9.3.]

The SET FLUSH Uproc may also be included to cause line-by-line processing to begin if SPIRES tries to access data outside of the buffer. In other words, SPIRES would read input lines beyond those requested by the frame dimensions if SET FLUSH is in effect and if START statements specify locations of data from those lines.

C.2.4  The USAGE Statement

The USAGE statement, in combination with the DIRECTION statement, determines which commands can cause execution of the frame being defined.

The syntax of the USAGE statement is:

where "value" is one (or two separated by a comma, if one of them is REFERENCE) of the usage values shown below, and NAMED is an additional option, which is described below.

For an input frame, four values are allowed:

By default for frames where DIRECTION = INPUT is coded, the USAGE is MERGE. That is, if no USAGE statement is coded, or if USAGE = NAMED is coded by itself, the usage value is MERGE.

C.3  Label Groups for Input Formats

This chapter is a general introduction to label groups for input formats. Specifically, the emphasis will be on label groups for frames that read data from a dataset. The next chapter will cover aspects of label groups specifically for prompting input frames. [See C.4.] Emphasis will also be placed here on label groups for frames of usage FULL. A later chapter will cover those statements used in "merge formats". [See C.5.]

All label groups in any frame, input or output, start with a LABEL statement. [See B.4.1.] Other statements that may appear in the label group carry out the function of that label group. A label group's purpose may be to handle a single data value, to control format execution, or both. The various purposes of input label group statements are:

Below are the various statements most likely to appear in the label group of an input frame:

Most of these statements were allowed in label groups for output frames. In particular, all those in the fifth category, label-group execution control, were discussed for output frames earlier; since they behave no differently for input, they will not be discussed again here. [See B.4.8.]

Execution order within a label group is particularly important to understand in input frames. SPIRES will execute the statements in the order in which they appear within the stored format definition. Remember that this order might be different from the order in which you entered the statements in the label group -- SPIRES will rearrange them in the proper order when the format definition is placed in the FORMATS subfile. [See B.4, E.1.]

A typical label group that processes an element value might look like this:

The label group starts by establishing the criteria by which SPIRES should determine what to retrieve from the buffer. According to this label group, the data to retrieve should be found on the next row (X), beginning in column 1, after the prefix "PHONE:" and before the suffix "-". Then SPIRES gets the data, processing it through the INPROC rule string, which verifies that all its characters are numeric. The value is then placed in the record as the PHONE.PREFIX element.

A label group for an input frame may be more simple or more complicated than this, depending on which statements are chosen for it. The next section will start the explanations of the various input statements, followed by sections showing how they fit together with other format statements. Sections at the end of the chapter will consider how to combine label groups into a frame definition. A sample input format definition will also be presented. [See C.3.9, C.3.10.]

C.3.1  The GETDATA Statement

The GETDATA statement tells SPIRES to retrieve a string value from the buffer for subsequent label group processing. The location of the data retrieved is specified by other statements that precede GETDATA, such as START, SCAN, MARGINS, MAXROWS and LENGTH, all of which are discussed later in this chapter. If none of these are coded, the default is for the GETDATA to retrieve the contents of the next row (i.e., "START = X") for processing.

The syntax of the GETDATA statement is:

where "n1", "n2", etc., represent code numbers telling SPIRES to override various defaults of GETDATA processing. Below are the allowed codes and their meanings -- the default for GETDATA in each case is easy to determine (a summary appears after the list):

[See the SPIRES manual "Device Services" for more information on those items referring to full-screen formats.]

So, by default,

A discussion on handling null values appears later in this chapter. [See C.3.9.3.]

As soon as the value is retrieved, its value is assigned to the $UVAL (unconverted value) variable. The value is then put through INPROC rules (though not INCLOSE rules) if they are coded in either the file or format definition. [See C.3.4.1 for the INPROC statement.] The resulting value is assigned to $CVAL (converted value).

Multiple rows of data may be retrieved at once by a GETDATA statement; however, they are concatenated together (with a blank between rows) into a single value since a label group may process only single values at a time.

C.3.2  The VALUE (VAL) Statement

The VALUE statement in input frames has the same purposes as it does for output frames. [See B.4.3.] In input frames it provides a way to give the label group a value to process just as the label group processes values retrieved by the GETDATA statement.

Its syntax is:

where "expression" is an expression following the same rules as expressions in a LET or SET Uproc. [See B.4.8.10.] The VALUE statement follows the LABEL statement in a label group definition. The value given is immediately evaluated; the evaluated expression is assigned to the system variable $UVAL. Unless the label group contains a DOPROC Uproc, that value is then processed through any INPROC rules, whether they are for the element named in the PUTELEM statement (or in the label name, if a PUTELEM statement appears in the label group) or appear in the INPROC statement in the label group. Note that INCLOSE processing rules in the INPROC string are not executed at this time. [See C.3.4.1, C.3.4.3.]

Usually in input frames, the value is a variable whose value was assigned in an earlier label group:

Be aware that the evaluated expression will be of a particular data type -- that data type must be handled properly if the correct data is to be stored. An example of this problem was discussed earlier. [See B.4.3.] Be aware also that most INPROC rules expect the input value to be a string, so you may need to convert the value accordingly. That is why the multiplied value was converted to a string value in the first example above.

An explanation of how to use special characters in the VALUE statement was also presented earlier. [See B.4.3.1.]

C.3.3  The PUTELEM Statement

The PUTELEM statement tells SPIRES to store the current label group value (established either by a GETDATA statement, a VALUE statement or a SET CVAL or SET UVAL Uproc) as a single goal record element occurrence.

Its syntax is very similar to that of the GETELEM statement of output frames. All of the following forms are allowed, though the first two are the most common:

In the first form, "element.name" represents the name of the element being stored. The second form, in which no element name is given, may be used when the element name is given in the LABEL statement. [See B.4.1.] The third form may be used when the element name is stored in a variable. Information about the last two forms, as well as more details about the first three, appeared in the sections on the GETELEM statement. [See B.4.2, B.4.2.1.]

When the PUTELEM statement is executed, the current value of $CVAL is stored in the record. By default, it is stored as the next occurrence of the element.

Each new line accessed by the GETDATA statement in the loop will be put as the next occurrence of the element ADDRESS.LINE. This principle is parallel to that of output frames, where a LOOP statement causes a GETELEM statement to access the next occurrence of an element each time.

You can specify a particular occurrence number with the form:

where "n" is an integer or a variable with an integer value representing the occurrence number, counting from 0. [See B.4.2.] The occurrence number may also be controlled by the SET PUTOCC or SET STARTOCC Uproc. [See C.3.3.1, C.3.3.1a.]

The occurrence number for input frames with USAGE = FULL should be considered relative rather than absolute. For example, if three occurrences of an element are entered as occurrences numbered 0, 2 and 4, and no other occurrences are provided, those will become occurrences 0, 1 and 2 respectively. The label groups may be given in any order (e.g., the occurrences numbered 4, 0 and 2) but the "zeroth" occurrence will be number 0, the occurrence numbered "2" will be number 1 and the occurrence numbered "4" will be number 2 when the record is stored.

In terms of format definition code, that point is demonstrated below:

If the input data set looks like this:

the values will be stored in 2-3-1 order, i.e.,

If you assign more than one value to a single occurrence number, all values will be kept; the first value will not be replaced. The values after the first for that occurrence number will follow the first one. For example:

Instead of each new value replacing the "second occurrence" of ADDRESS.LINE, each will become a new occurrence following the previous one. If another label group had a "PUTELEM = ADDRESS.LINE(3)" statement, its value(s) would be put after all those from the label group shown above. To ensure that an element value is placed as the last possible occurrence, choose an arbitrarily high number for the occurrence number, such as 999, though no higher than 8190.

Once you have put a value into a record (with a PUTELEM statement), you cannot remove it -- that is, there is no element-level "dequeue" capability in formats. Instead, you will have to decide whether to let the value go into the data base with the rest of the record or to abort the format processing. [See C.3.6.1.]

Occurrence numbers will be somewhat more absolute in frames of USAGE = MERGE. [See C.5.1.3.] Remember that element occurrences are numbered from zero inside a format; they are numbered from one in the standard format and $PROMPT format.

C.3.3.1  The SET PUTOCC Uproc

The SET PUTOCC Uproc may be used in an input label group to specify which occurrence is to be processed by the PUTELEM (or REMELEM) statement in the label group. Its form is:

where "n" can be an integer, a variable whose value is an integer, or an expression (e.g., SET PUTOCC = #N + 1). The value must be greater than or equal to zero; otherwise, an error will occur.

This statement may be used instead of the occurrence number on the PUTELEM or REMELEM statement. However, if both are used in a single label group, the occurrence number on the PUTELEM or REMELEM will override the Uproc. [See C.3.3, C.5.1.2.]

C.3.3.1a  The SET STARTOCC Entry-Uproc

The SET STARTOCC Entry-Uproc may be used in an input label group to specify a starting occurrence number for the elements being processed by the PUTELEM or IND-STRUCTURE statements in the label group. Subsequent PUTELEMs in the LOOP of the label group will take on occurrence numbers in ascending sequence. You can also add the SET LOOP BACKWARDS Entry-Uproc to the label group so that the occurrence numbers will be generated in descending sequence.

The syntax for both Entry-Uprocs is the same as for output formats:

where "n" is the occurrence number you want to start with.

If you use SET PUTOCC, or add an occurrence number in the PUTELEM statement, this occurrence value sequencing will be canceled. Keep in mind that how SPIRES handles the occurrence numbers will vary depending on whether the frame type is FULL or MERGE. [See C.3.3, C.5.1.3.]

C.3.3.2  The SET SKIPEL Uproc

Under some circumstances, you may want to suppress the PUTELEM statement that will be executed at the end of the label group. Suppose, for instance, that after testing the value in an IF...THEN Uproc, you decide the value should not be put into the data base.

One solution to this problem is to issue the JUMP Uproc:

but of course, no further processing of the current label group would occur in such situations.

Another solution is provided by the SET SKIPEL Uproc:

The SET SKIPEL Uproc tells SPIRES to "skip" any element processing specified by a PUTELEM or a REMELEM statement [See C.5.1.2.] later in the label group -- in other words, to treat the PUTELEM or REMELEM as if it did not exist. The remainder of the label group will be executed, however.

The system flag variable $SKIPEL can be tested to determine whether the SET SKIPEL Uproc was executed in the current label group. [See E.2.1.21.]

C.3.4  Changing or Testing the Label Group's Value

Data being input to a data base must often be tested and/or altered. Most data validation is done through processing rules, in particular, INPROCs. An input format may either take advantage of the element processing rules in the file definition or override those with its own INPROC rule strings. The value can also be changed with the SET CVAL Uproc, though in input frames, this method has significant disadvantages, discussed later.

A better method than the SET CVAL Uproc for changing the input value is provided by the SET UVAL and DOPROC Uprocs. The DOPROC Uproc tells SPIRES not to execute the INPROC rule string in the label group until the DOPROC Uproc is reached. Thus, you can set the value of $UVAL as desired (with the SET UVAL Uproc) before the INPROC rules are executed to convert it to $CVAL.

The INPROC statement is discussed first, followed by the SET CVAL Uproc, and then the SET UVAL and DOPROC Uprocs. [See C.3.4.1, C.3.4.2, C.3.4.3.]

C.3.4.1  The INPROC (IN) Statement

As soon as a value is assigned to a label group containing a PUTELEM statement in it, that value ($UVAL) is processed through the appropriate INPROC rule string (though not INCLOSE rules -- see below), creating $CVAL. [See C.3.4.3 describing the DOPROC Uproc, which may delay the execution of the INPROC rules.] The "appropriate INPROC string" is, by default, located in the file definition. Specifically, it is found as part of the definition of the particular element named in the PUTELEM statement. [See C.3.1.]

The non-INCLOSE rules of the file definition's INPROC strings can be overridden by an input format, using the INPROC statement:

where "rule string" contains one or more processing rules (actions, system procs or user-defined processing rules). Multiple rules are each separated by a slash (/), which is optionally surrounded by blanks.

Another form of the INPROC statement can be coded if you want to override file definition INPROC processing but not replace it with something else:

With this form, no INPROC processing (prior to closeout processing, which may involve INCLOSE rules) will occur at all, and $CVAL and $UVAL will be equivalent.

Although INPROC statements are allowed in output frames in some cases, the converse is not true: OUTPROC statements are not allowed in input frames. The INPROC string usually consists of actions and system procs. A user-defined proc may be included, but it must be defined at the end of the format definition, or in an EXTDEF subfile record that is referenced in the EXTDEF-ID statement at the end of the format definition. [See B.4.5.2.]

The $CALL system proc (actions A62 or A124), used to invoke USERPROCs, may be coded in the OUTPROC or INPROC string. However, the actual USERPROC definition must be in the file definition, under the record-type named by the RECORD-NAME statement.

If any of the system procs $BREAK (action A45) or $STRUC or $STRUC.IN (A33) are coded in the element's INPROC string in the file definition, they will work properly in the format. You may also code these system procs or actions in an INPROC statement in the format definition, if desired.

The INPROC statement may be used to override only the non-INCLOSE rules of the element definition's INPROC string. After format processing is completed, the INCLOSE processing specified in the file definition will occur. Any INCLOSE rules appearing in the INPROC statement of a format will be compiled but will not be executed by the format. An exception to this is that the BUILD RECORD Uproc can be used to force SPIRES to do INCLOSE processing while still under format control. [See C.3.4.4.]

The file owner may require that input for an element by some or all accounts be processed through the INPROC rule string for the element as determined by the file definition's processing rules. (This restriction is made with the combination of the INPROC-REQ and PRIV-TAG statements in the file definition.) If a label group containing an INPROC statement tries to process such an element, the $SKIPEL flag is set, meaning that no PUTELEM or REMELEM statement appearing the label group will be executed. [See C.3.3.2.] In that label group, the value of $CVAL will be the same as $UVAL.

(*) The EXTDEF-ID and PROC Statements

How SPIRES reacts to processing rule errors depends on several factors: the error level of the processing rule (D, W, E or S), and the values of some system variables. Because error handling can be complicated in input formats, it is discussed in its own section later in this chapter. [See C.3.6.]

For more information about INPROC rule strings, see the SPIRES manual "File Definition", section B.4.3.

C.3.4.2  The SET CVAL Uproc

As it was in output frames, the SET CVAL Uproc may be used in input frames to change the label group's value prior to the moving of the data, in this case with the PUTELEM statement. Its syntax is:

where "expression" may consist of literal strings, user or system variables (including $CVAL itself, referring to its value before this statement is executed) and system functions.

In an input frame, the SET CVAL proc will not change $CVAL to a string; the variable's type will be the type determined by the expression. For example,

The type of $CVAL will now be a packed decimal. Compare that result with the result if this Uproc were in an output frame: the data would be converted to a string value. [See B.4.5.4.]

The preferred method for testing and changing element values on input is with processing rules. Because of their built-in error handling capabilities and their internal design, they are easier and more efficient to use than the SET CVAL Uproc. And, with the SET UVAL and DOPROC Uprocs, you can delay the execution of the processing rules until desired. [See C.3.4.3.]

Remember the order in which input processing generally occurs:

INPROC rules are often chosen because they prepare the value for storage -- they check its length to ensure that it fits in the allocated space, for example. They may also put the value into a special form for storage that is reconverted to its original form on output by OUTPROCs. So if you use the SET CVAL Uproc on input to change the value after INPROCs have been executed, you must make sure that the changed value matches the appropriate characteristics that would be ensured by the INPROCs.

For example, consider this label group:

This label group is not likely to work the way the designer planned it. The input value is converted to an integer, but if its value is zero, it is replaced by the string "No value".

Two problems with that label group come to mind. First, the file owner might have assumed that the input value would be an integer and limited the storage space for that element to four bytes (with a LEN statement) accordingly. In that case, an error will occur on record closeout when the input value is "No value", since its length is eight.

A second problem is slightly more subtle. Suppose that instead of "No value", the value "None" (a four-byte string) replaced zero. That value would be stored. However, on output, if the element had an OUTPROC that converted the value from an integer to a string (e.g., $INT.OUT), the stored value "None" would be converted to a meaningless integer value (in this case, -711551611). The reason is that the type of the value stored in the data base is almost always determined by the processing rules, not by anything stored with the value. Hence the $INT.OUT rule assumes that the stored value was previously converted to an integer, and thus reads the stored value as if it were one. It has no way of knowing that the stored value should be read as a string value in this particular case.

Postscript to the example: It is far preferable to have an OUTPROC change the stored integer value 0 to "No value" on output than to try to store the string "No value" when the other values are integers. In other words, code an OUTPROC string like this:

Another example involving the SET CVAL Uproc was shown in regard to the VALUE statement. [See C.3.2.]

To summarize, if you do use the SET CVAL Uproc, remember that the non-INCLOSE INPROC rules have already been executed and thus you are responsible for the new value of $CVAL being in the appropriate form for storage and later output.

C.3.4.3  The SET UVAL and DOPROC Uprocs

The DOPROC Uproc has two effects: First, its presence in an input-frame label group prevents the non-INCLOSE INPROC rules that would be executed for the label group's value from executing at the beginning of the label group. Second, it causes those INPROC rules to be executed at the point where the DOPROC Uproc appears in the label group, i.e., somewhere in the UPROC statements. This delay provided by the DOPROC Uproc allows the SET UVAL Uproc to change the label group's value prior to the execution of the INPROC rules.

The syntaxes of the two Uprocs are:

The "expression" may consist of literal strings, user or system variables (including $UVAL itself) and system functions. The type of the evaluated expression determines the type of $UVAL. Remember, however, that most INPROC rules assume the input value is a string, meaning that in most cases, you should be sure $UVAL is a string before the DOPROC Uproc is executed.

The DOPROC Uproc may only appear in input frames (SET UVAL may appear in output frames, but it would seem to be of very little use there.) and may only appear once per label group. Also, it may not be used in label groups containing an IND-STRUCTURE statement. [See B.8.3, C.3.9.2.] Finally, if DOPROC Uproc appears in a label group, it MUST be executed before you execute the associated PUTELEM. Otherwise an S871 error will occur for executing a PUTELEM without executing its associated DOPROC Uproc.

Here is a sample label group using the SET UVAL and DOPROC Uprocs:

Although the INPROC string appears before the Uprocs, the DOPROC Uproc delays the execution of the INPROC rules. If no value is retrieved by the GETDATA statement, the format prompts for a GRADE value, which is then assigned to $UVAL. At that point the DOPROC Uproc tells SPIRES to execute the INPROCs.

As indicated by the appearance of the ASK Uproc, these Uprocs are often used when user-prompting is necessary. In fact, they will be a great convenience in prompting input formats. [See C.4.2.]

C.3.4.4  The BUILD RECORD Uproc

The BUILD RECORD Uproc causes SPIRES to perform all of the INCLOSE processing, merging, and final record build while still under format control. The syntax is simply:

Be sure to do all PUTELEMs before executing the BUILD RECORD Uproc.

Currently, the most useful application of this Uproc is in conjunction with merge filters. If you set merge filters in your format (with the SET FILTER Uproc) then you must execute the BUILD RECORD Uproc in order for the filters to work. The record-building must happen while you're still under format control, since the filter set in the format would be cleared when the format is exited. [See B.4.8.5a.]

At this time, it is not possible to have access to the values built into the memory copy of the record by the BUILD RECORD Uproc. Future enhancements will likely provide ways to do that, as well as tools to control the closing out of structures and records, in conjunction with BUILD RECORD.

C.3.5  Statements that Control the Retrieval Location of the Value

The statements discussed in the next sections tell SPIRES the location of the value to be retrieved by the GETDATA statement. They all precede the GETDATA statement in the label group.

The first four, START, LENGTH, MAXROWS and MARGINS, were first discussed for output frames. They work similarly here.

While the START statement tells SPIRES where to find the value by providing a specific grid location, the SCAN statement tells SPIRES to locate the value based on particular criteria, such as a character string that precedes it. The SCAN and START statements work together -- the START statement tells SPIRES where to begin scanning for the value. The SCAN statement makes the GETDATA statement less efficient, since SPIRES must scan the area defined by START, MAXROWS, LENGTH and MARGINS byte by byte to locate the value.

C.3.5.1  The START and XSTART Statements

By default, a GETDATA statement causes SPIRES to retrieve data starting in the first column of the row following the highest one retrieved from ("X"). The START statement lets you specify the starting location of your choice.

By default, if multiple occurrences of an element are retrieved because of the LOOP statement, SPIRES retrieves the data for these occurrences starting in the "X" row, beginning in the same column as indicated in the START statement. The XSTART statement lets you specify an alternate starting location for them.

The syntax of the START and XSTART statements was given earlier. [See B.4.6.1, B.4.8.4.]

Suppose the input data for goal records about volleyball teams looks like this:

The input label group to retrieve and process those names might look like this:

The XSTART statement is not really necessary here since the default XSTART location is the value given.

Note that the GETDATA statement is executed before any Uprocs. Hence, the SET STARTROW and SET STARTCOL Uprocs, useful in output formats, are not useful in input ones -- the data has already been retrieved. [See B.4.6.2.]

C.3.5.2  The LENGTH (LEN) Statement

In input frames, the LENGTH statement specifies the length in bytes of the area in the buffer from which the data will be extracted. Its syntax is:

where "length" is an integer or an integer variable.

The area of the buffer whose length is given by the LENGTH statement begins in the position defined by the START statement, unless a SCAN statement is present. [See C.3.5.1, C.3.5.5.]

If no SCAN, LENGTH or MAXROWS statement is coded, SPIRES will retrieve the value only from the starting row for the length of the row. But if a LENGTH statement is coded, with a value larger than the amount of space (or data -- see example below) left on the starting row (taking margins into account), then the area from which data can be retrieved will wrap around into the next row, and so forth.

The LENGTH statement describes the area from which the label group may retrieve a value, not necessarily the length of the retrieved value. However, trailing blanks and/or pad characters at the end of the value, and at the end of each row when the value wraps across multiple rows, are not considered part of the value or, hence, part of the LENGTH, unless GETDATA = 4 is coded.

For example, consider this data in a frame of 40 columns width (LENGTH = 80):

As you would expect, when SPIRES retrieves that value, it strips off the extra trailing blanks at the end of the first row (leaving one behind to separate "when" from "stored"). Thus, as the value itself states, the stored value would be 68 characters long. However, if GETDATA = 4 were coded, the extra trailing blanks at the end of each row would not be stripped, and the total value length would indeed be the length given in LENGTH -- 80 characters.

Suppose instead that the value of LENGTH is 60 rather than 80 and "GETDATA = 4" is not coded. The value retrieved would be:

The actual retrieved value is not 60 characters long but 58, again because the LENGTH statement describes the area containing the value, not necessarily the length of the retrieved value. The two extra blanks at the end of the first row were discarded. Had the statement "GETDATA = 4" been coded, the retrieved value would have been:

The value now has the specified length of 60, retaining all the blanks at the end of row 1, since the statement "GETDATA = 4" says that they, and any pad characters, are significant.

When the label group is finished executing, the values of $CROW and $CCOL (the "*" row and column) will reflect the last character position examined for data, not the last row in which data was found. [See C.3.9.4.]

C.3.5.3  The MAXROWS (ROWS) Statement

In input frames, the MAXROWS statement establishes the maximum number of rows from which SPIRES can retrieve the data with a GETDATA statement. If no MAXROWS, SCAN or LENGTH statement is coded, a GETDATA statement can retrieve the data only from the current row. [See C.3.5.2.]

The syntax of the MAXROWS statement is:

where "nrows" is an integer or an integer variable. Setting MAXROWS to zero has the same effect as omitting the MAXROWS statement.

The value of MAXROWS should be large enough to accomodate any data expected. If, for example, MAXROWS = 3 and the data value wraps into a fourth row, that "extra" part of the data will not be retrieved. Moreover, if the next data value begins in row X according to the label group, that remaining piece of the previous data value will be considered the start of the next value. [See C.3.9.3.]

C.3.5.4  The MARGINS (MAR) Statement

As in output frames, the MARGINS statement specifies left and right margins for the label group. For input, those margins in combination with the START, MAXROWS and LENGTH statements will define the area of the buffer from which data may be extracted, if GETDATA = 6 is coded.

By default, if no MARGINS statement appears, the left margin will be column 1 while the right margin will be the last column on the row, according to the FRAME-DIM statement.

The syntax of the MARGINS statement is:

where "lcol" and "rcol" represent the left and right column numbers respectively, given in one of the following forms:

The left margin value is ignored for the first row, since that is controlled by the START statement, unless there is no START statement, in which case the left margin value is applied to that row. The right margin value is used for all rows, including the first.

C.3.5.5  The SCAN Statement

The SCAN statement tells SPIRES to start or stop retrieving a data value based on the value itself rather than on its position. It is probably more often used with the END option, which tells SPIRES to stop retrieving the value when a particular character string or characters are encountered. It may also be used in conjunction with the SET SCANROW Uprocs, which are useful for controlling how SPIRES interprets longer values that may wrap across multiple rows. [See C.3.5.6.] The SCAN capability is very useful for variable length values, i.e., ones whose length cannot be predicted ahead of time.

The SCAN statement has the following possible forms:

The forms on the top left indicate the criteria to be used to find the beginning of the data value; the forms on the top right, the end. The CONTAINER and ESCAPE forms are somewhat different from the others and are discussed further below.

The "string expression" form specifies that the given expression should be located in the data field to indicate the start or end of the data. If the string expression contains non-alphanumeric characters, it should be enclosed in apostrophes (if the character string itself contains apostrophes, those internal ones should be doubled). The string expression may be given as some combination of the following: a literal string, a system variable or a user variable.

The "(characters)" form specifies that any one of the characters within the parenthesized list will indicate the start or end of the data value. If the "not sign" (~) precedes it, then a character that is not one of those characters will indicate the start or end of the value. These values may be given in any of the following forms:

If you must use a special character, such as a semicolon or right parenthesis, you should place the character or characters in a string variable, whose value you give in the SCAN statement:

The value of the UPROC statement was surrounded by quotation marks because a semicolon appeared in it.

SCAN may appear several times in a label group, allowing specification of both beginning and ending criteria and container or escape criteria (see below). If both are specified and the scanning finds only the start of the value, then the GETDATA will retrieve the rest of the area defined by the START, MAXROWS, LENGTH and/or MARGINS statements (but see below).

Because SPIRES must examine the buffer byte by byte to locate the value being scanned for, the SCAN statement is not as efficient as other techniques for finding and/or limiting the value and should only be used when other techniques do not work easily.

The SCAN statement is most often used in combination with the other location statements in this section, such as START and MAXROWS. By default, scanning begins at the default starting position for the value; if a START statement is coded, scanning starts at that point instead. Also by default, MAXROWS and LENGTH will be applied to the value located by the scan. If, however, "GETDATA = 6" is coded, MAXROWS and LENGTH will apply to the area to be scanned instead. [See C.3.1.]

Limiting the area to be scanned with the "GETDATA = 6" statement and/or LENGTH or MAXROW statements is recommended. If "GETDATA = 6" is not coded and SPIRES cannot find the start of the value, an S808 error will occur, and format processing will stop. The same error will occur when no LENGTH or MAXROWS statement is coded if SPIRES does not find the beginning of the value or if SPIRES finds the beginning but not the end.

Note that SPIRES will scan multiple rows looking for the beginning of the value to be retrieved, whether or not MAXROWS or LENGTH are specified. However, by default the value retrieved will only be retrieved from the row on which the start of the value is found. Coding a SCAN statement for the ending of the value or coding a MAXROWS or LENGTH statement with or without "GETDATA = 6" may change that default, allowing multiple rows to be retrieved. Two other option numbers on the GETDATA statement are important to the SCAN statement. By default, the characters or character strings that start or stop scanning are not included as part of the value. "GETDATA = 2" will include the scanned starting data in the value; "GETDATA = 3" will include the scanned ending data. [See C.3.1.] If the "(characters)" form is used, the system variables $STARTCHAR and/or $ENDCHAR contain the particular character that started or stopped the retrieval of the value. [See E.2.3.6, E.2.3.7.]

Here is an example of a label group that uses the SCAN statement to locate the end of a long text value:

When SPIRES during scanning finds a question mark, exclamation point or period, the value ends. Because of the "GETDATA = 3" statement, the ending punctuation mark will be retained as part of the value. [See C.3.9.4 to see how SCAN affects the various variables, such as $CROW.]

The CONTAINER and ESCAPE forms of SCAN are useful when working with input that was created using the SET BUILD CONTAINER and SET BUILD ESCAPE Uprocs. [See B.4.5.7.] These two forms of SCAN have the syntax shown below:

If the characters are special characters, which they usually are, you should enclose them in apostrophes like a text string.

The CONTAINER form lets you specify a set of characters, any of which may surround the input value being accessed. For example, if you have SCAN = CONTAINER,'|"', then if the value is surrounded by either vertical bars (|) or quotation marks ("), then those characters will be stripped from the input value.

If a SCAN = END form is also specified, then if that END character or string is between the container characters, it will not stop the scan.

The ESCAPE form lets you specify a single character to serve as an escape character. That means that the escape character will be stripped from the value, and the character that follows it will be retained as part of the value and not interpreted as a container character or an end character.

Suppose you have these three statements:

Notice that CONTAINER and ESCAPE have no power to locate the start of a value. In other words, the SCAN, CONTAINER statement above will not be enough to provoke the GETDATA to start retrieving data at the next quotation mark it finds. You might still need to specify a starting SCAN too.

The SET SCANROW Uprocs

Two Uprocs that affect how SPIRES will scan for data are

These commands affect all subsequent GETDATA statements executed by the format until they are changed or canceled. To cancel them,

C.3.5.6  The SET SCANROW Uprocs

In some input situations, you need to be scanning for the continuation and ending of both individual elements and the entire record, using different characters to signal different conditions.

For example, suppose your input data contains multiple records, each of which may require multiple rows. The end of a record is signalled by the period character at the end of the final row. Continuation of the record from one row to the next is indicated by a wrap character, a backslash in our example (\). The separate element values of each record are delimited by a special character, like a tab character or a vertical bar:

One of the complications of this example situation is that the end-of-record character, the period, may appear (and does, in the sample data) in the middle of an element value; only when it appears at the end of a row of data is it meant to be a record delimiter.

Two Uprocs, SET SCANROW WRAP and SET SCANROW END, are meant to solve the problem of the example situation:

where both "wrap-character" and "end-character" are single characters. These Uprocs are in effect for all subsequent label groups in the current frame and any other input frames of the format while the current record is being processed; they are reset when the format begins executing for the next record. You can reset them explicitly if and when needed:

In our example, before the label group that picks up the first element, we would code:

One of the individual element label groups might look like this:

With similar label groups for each element, the values would each be picked up, minus the various separator characters, and without any trailing blanks from rows where the value was split across multiple rows.

SPIRES is actually scanning each row from the right end, looking for the first non-blank chaacter, and determining whether it is a "scanrow" character. If it is, then SPIRES sets the row end as the character to the left of the scanrow character, for use by the regular scanning process as described in the label group. If it is not, the scanning ends with that row, and the scanned data will include the last character.

If the scanrow character that ends the row is the "wrap" character, scanning continues on the next row. If the scanrow character is the "end" character, the scanning ends.

Additionally, if SPIRES encounters the scanrow-end character at the end of a row during the regular scanning process, the GETDATA scan ends, even though the regular scan-end character may not have been encountered. Essentially then, the scanrow-end character, if found at the end of the row, is another scan-end character.

C.3.6  Error Handling in Input Formats

Most data errors in goal records are detected on input rather than output. Such errors may, for example, involve processing rules or multiple occurrences of a singly-occurring element or values that are too long for a fixed-length element. Obviously, if a data entry error has occurred, that error should be detected and announced when the person who made the error is updating the subfile, rather than when a user is trying to display records. Thus, error handling facilities described below, though valid in output frames, appear more frequently in input frames. [See B.4.5.2.]

Before the details on error handling in formats are presented, it is important to know how data entry errors are handled by SPIRES in general. Below is a chronological look at how SPIRES processes a record being added through the standard SPIRES format with regard to errors:

An input format added into this scheme generally fits into the first three steps. [An input format with a BUILD RECORD Uproc enters into the realm of step 4, since BUILD RECORD forces INCLOSE processing to happen. [See C.3.4.4.]] It cannot protect the user specifically from the errors in step 4, except insofar as it can test for those situations itself. For example, once the input format has executed, it cannot prevent SPIRES from rejecting the record if it has too many occurrences of an element to be stored. However, the format may count the number of occurrences as it executes, discarding extras ahead of time so that the entire record will not be rejected later.

The order in which the elements are processed is determined by the input format. Note too that the entire format will execute for the record even when a serious error is detected for the very first element read. By testing the error flags, this can be changed.

The subsections of this section will cover the tools and techniques for handling errors, primarily processing rule errors, in formats. [See C.3.6.1, C.3.6.2, C.3.6.3.]

C.3.6.1  The Error Flags: $PROCERR, $APROCERR, $WPROCERR and $GPROCERR

Whenever SPIRES detects a processing rule error in a format, one or more flag variables are set by SPIRES -- they may not be set by the user explicitly. Those variables may be tested in Uprocs with appropriate action taken based on their values.

The $PROCERR (for PROC ERRor) flag is set when a processing rule error occurs with an error level of S or E. [See C.3.6.3 for more information on the error levels.] When the next label group is entered, the error flag is cleared (set to $FALSE).

The $GPROCERR (for Global PROC Error) flag is set when a processing rule error occurs with an error level of S or E (except when the $TESTFAIL flag is set), but is not cleared when the next label group is entered. It remains set until the next record begins processing. If $GPROCERR is set at the end of the format's execution, the record will be discarded.

The $WPROCERR (for Warning PROC ERRor) flag is set when a processing rule error occurs with an error level of W. It, like $GPROCERR, is not cleared when the next label group is entered but remains set till the next record begins processing.

The $APROCERR (for Any PROC ERRor) flag is set when a processing rule error occurs with any error level -- D, W, E or S. Like $PROCERR, it is cleared when the next label group begins executing.

An example of how these variables might be used is shown below:

If a serious error (S) is detected by the $DATE system proc, the $PROCERR flag will be set and the format will stop processing when the Uproc is executed. This prevents SPIRES from executing the format any further when it is known at this point that the record will not be added to the subfile.

Whether you tell SPIRES to "stoprun", abort, or return is up to you. [See B.4.8.8.] It often depends on what type of error message you want the user to get, assuming the user has not turned off error messages. [See C.3.6.4.] For example, here are the terminal displays for three sessions. In each case, the error occurs as given in the example above, but the particular action taken in the Uproc differs.

         The STOPRUN Uproc
   -> add
   -Element=DELIVERY.DATE:  -Serious data error, code=E31
   -Error at or before line 1
   -FORMAT execution error, code=S848
   -Update terminated, code=S2
   ->

         The ABORT Uproc
   -> add
   -Element=DELIVERY.DATE:  -Serious data error, code=E31
   -Error at or before line 1
   -FORMAT execution error, code=S847
   -Update terminated, code=S2
   ->

         The RETURN Uproc
   -> add
   -Element=DELIVERY.DATE: -Serious data error, code=E31
   -Error at or before line 1
   -Update terminated, code=S261
   ->

In all three cases, the format stops when the Uproc is executed. The RETURN Uproc is the least efficient of the three since SPIRES will try to add the record to the data base, testing the $GPROCERR flag first. Of course, since you have already discovered (by testing $PROCERR) that the record should not be added, you might as well tell SPIRES to abort the request. On the other hand, the user's input may have multiple errors and you might want to report all errors before rejecting the record, which would mean allowing SPIRES to try adding the record.

In SPIRES input formats, there is no major difference between the STOPRUN and ABORT Uproc, since only one record is being handled per command. However, in SPIBILD, where a single command may process many records, the difference is very important: ABORT stops the processing of the current record, allowing processing to continue to the next record, while STOPRUN stops all further processing of any records under the current command. [See C.9.]

It may at first seem surprising that when SPIRES detects an S-level processing error, the format continues executing. After all, if the record will be rejected, why should further processing continue? The reason is that by continuing, SPIRES gives you the opportunity to correct the error. However, to take advantage of this capability, you need to use the SET TESTFAIL Uproc. [See C.3.6.2.]

C.3.6.2  The SET TESTFAIL Uproc or Entry-Uproc

Once the error flag $GPROCERR is set, it cannot be unset, meaning that the record will be discarded, no matter what. The SET TESTFAIL Uproc tells SPIRES not to set $GPROCERR when a processing rule error is first detected but instead to set it when the PUTELEM occurs. Since a Uproc that tests the value of $PROCERR and takes some alternate course of action to save the day can be inserted between the INPROC and the PUTELEM statement, the $GPROCERR flag does not have to be set.

The syntax of the SET TESTFAIL Uproc is simple:

SET TESTFAIL can also be issued as an Entry-Uproc rather than a Uproc. After you have SET TESTFAIL, the $TESTFAIL flag remains in effect throughout the execution of the format, unless it is turned off by:

The timing of the SET TESTFAIL request can be quite important. If the statement executes at a point when INPROC rules may already have set $GPROCERR, then it is already too late to be of use; you must code the statement before those INPROCs have a chance to execute, which for a Uproc would mean coding it in the label group preceding the one with the processing rules. As an Entry-Uproc, however, SET TESTFAIL can be coded within the same label group where the processing error might occur. [See B.4.5.5, B.4.5.6 for more on Uprocs and Entry-Uprocs.]

Below is a collection of label groups that takes advantage of SET TESTFAIL as an Entry-Uproc:

The first label group reads the data from the data set; if an error occurs in processing the data, the REPROMPT.KIDDOS label group is executed, prompting the user at the terminal for replacement data, which is processed in the next label group. The SET TESTFAIL Entry-Uproc (which would be in effect for the rest of the format, not just the label group in which it first appears) would help the format recover from non-integer input for CHILDREN without throwing the entire record away.

C.3.6.3  The SET ERROR Uproc

Sometimes you may need to process a value with functions rather than processing rules, or perhaps you want to check an input value against another value from an earlier label group that you have saved in a variable. In such cases, you may need the ability to set the error flag yourself as a result of your processing or tests -- you want to prevent SPIRES from putting the element value into the record.

In label groups containing a PUTELEM statement, you may code the SET ERROR Uproc:

where you choose the appropriate error level.

When the SET ERROR Uproc is executed, the following will occur:

Here is a sample label group that uses the SET ERROR Uproc. The value of the CHOICE1 variable was established in an earlier label group.

Below is the terminal display a user might see when a frame with the label group above is executed:

The error message displayed will always include the error number E221. It will also include the value of $UCODE if that has been set in the current label group.

To set the string value of $UCODE, use the SET UCODE Uproc:

where "message" is a string of text no longer than 128 characters. The $UCODE message will be displayed within the system error message:

The $UCODE variable was set to "Must not match CHOICE1".

The value of $UCODE remains the same until SPIRES begins executing another processing rule string or until another SET UCODE Uproc is encountered. If you do not set the value of $UCODE yourself, it will retain its value from the latest processing rule string executed or the latest SET UCODE Uproc, whichever has been most recently executed.

C.3.6.4  The SET MESSAGES Uprocs

The SET MESSAGES Uprocs provide a way to suppress various types of system messages concerning the format execution. The Uprocs are very similar to the SET MESSAGES commands, even to the point of setting the same system variables. However, the impact of the SET MESSAGES Uprocs is felt only within the executing format; when the format is finished executing, the values of those variables are reset to their pre-format values.

The form of the SET MESSAGES Uproc is:

where "type" is ALL, WARNING or SERIOUS (the default is ALL) and where "level" is an integer from 0 to 4, meaning:

In the sample messages shown, the code number can be explained if desired ("EXPLAIN 217", for example). The letter in parentheses indicates the type of message -- S for Serious, W for Warning.

In formats, there is no difference between levels 3 and 4. However, under the SET MESSAGES command outside of formats, level 4 specifies that the error code explanation from the EXPLAIN subfile should be displayed with the code and text. This option is not available within formats.

Serious messages generally tell you that an error has been detected in processing, e.g., "-Warning data error". Warning errors usually announce a potential problem or tell you that SPIRES has taken some action to avoid a problem or give you more information about a serious error, e.g., "-ERROR AT LINE 1". Be aware that the message type is not connected to the error level of a processing rule. For example, a processing rule error of level S may produce both a Serious and a Warning message, as shown in the sample terminal sessions below. Alternatively, setting Warning messages to 0 does not mean that messages from a W-level processing error will not be displayed.

The integer variables $SERI, $SERX, $WARNI and $WARNX are all affected by the various SET MESSAGES Uprocs. Both $SERI and $SERX are set to the given level by a SET MESSAGES or SET SERIOUS MESSAGES Uproc; both $WARNI and $WARNX are set to the given level by a SET MESSAGES or SET WARNING MESSAGES Uproc. [See the manual "SPIRES Protocols", section 5.9, for more information about these variables.]

The default levels for messages as a format begins executing will vary depending on whether the format was invoked by a protocol or by the user interactively. If invoked by a protocol, the values of $WARNX and $SERX are used; if invoked by the user, the values of $WARNI and $SERI are used. But a SET MESSAGES Uproc within the format will change both variables of a pair, regardless of how format execution was invoked.

Below are examples of the impact of various SET MESSAGES statements. Assume that all levels are set to 2 before format execution. The Uproc above each example indicates a SET MESSAGE Uproc executed within the format before the error occurs.

Note that the UPDATE ABORT message is outside of format control -- it could be suppressed with a SET MESSAGES command before the ADD command.

Typically in the last two cases, programmers include a Uproc such as the following in the label groups containing PUTELEM statements:

to display the message from the $MSG proc (A56 action) from the element's INPROC string, if any. This provides the user with specific information about the error without the serious error messages appearing too.

C.3.6.5  The ATTN/BREAK Key and the $ATTN Flag

Although it is not exactly an error, pressing the ATTN/BREAK key during format execution is usually meant to halt further processing because of some error, either the user's or the programmer's. There are two possible cases:

The most important point to be aware of is that pressing ATTN/BREAK during format execution does not cause the format to immediately stop executing. However, in the second case, SPIRES does set the $ATTN flag, which can be checked within the format, e.g.,

Remember though that such a statement itself adds some overhead to the format. Unless you expect ATTN/BREAK to be pressed frequently during format execution, such a Uproc is probably not worthwhile.

The $ATTN flag is reset to $FALSE when a new command is issued (hence, its value cannot be checked outside of the format, since a command to do so would reset the flag) and when an ASK Uproc is executed within the format.

C.3.7  Execution Statements Used in Input Formats

The same basic execution statements discussed in Part B for output frames are available for input frames. Most of them are Uprocs:

Other execution statements discussed earlier may also be used in input frames:

All of these work exactly the same in input frames as in output frames. [See B.4.8.]

C.3.8  System Variables used with Input Formats

System variables are as important to input formats as they are to output formats. Below are some of the most useful, some of which relate to merge formats. Details are provided later in Part C and in the Appendix. [See E.2.]

Other variables are available, of course, including some discussed in other contexts, such as the sort variables like $SORTKEY, and others that are not specific to formats, such as $KEY or $SLOT.

In the chart below, the variable name is followed by the variable type in parentheses.

The first group of variables is reset when a new label group is entered:

  $UVAL (varies)  - the unconverted (i.e., before INPROCs) value
                     of the label group.
  $CVAL (varies)  - the converted (i.e., after INPROCs) value of
                     the label group.
  $PVAL (varies)  - the unconverted value of the previous label group.
  $ULEN (int)     - the length of the unconverted value.
  $CLEN (int)     - the length of the converted value.
  $LOOPCT (int)   - value of the LOOP counter for the current element,
                     beginning at "0" for the first time through.
  $PROCERR (flag) - set when an S- or E-level error is returned from
                     INPROC (but not INCLOSE) execution.
  $APROCERR(flag) - set when any error is returned from INPROC
                     execution.
  $CHANGED (flag) - set when a data value has been changed (full-screen
                     applications).
  $LABEL (string) - the name of the currently executing label group.
  $SKIPEL (flag)  - can be set to tell SPIRES to bypass execution
                     of a PUTELEM or REMELEM statement.

The following variables are not reset when a new label group begins executing:

  $CROW (int)      - the current row position in the frame, i.e., "*".
  $CCOL (int)      - the current column position, i.e., "*".
  $SROW (int)      - the starting row of the most recently accessed
                      value in the frame.
  $SCOL (int)      - the starting column of the most recently
                      accessed value in the frame.
  $LROW (int)      - the number of the highest row used in the frame
                      (i.e., X-1).
  $LCOL (int)      - the number of the last column processed (X-1).
  $NROWS (int)     - the number of rows assigned to the current frame.
  $NCOLS (int)     - the number of columns assigned to the frame.
  $GCHANGED (flag) - set when the $CHANGED flag is set (any iteration
                      of processing the screen).
  $GNCHANGED(flag) - set when $CHANGED is set (this iteration only
                      of screen processing).
  $GPROCERR (flag) - set when an S- or E-level error has occurred
                      during INPROC or OUTPROC execution.
  $ABORT (flag)    - set when a UPROC = ABORT is executed; useful
                      only outside of the format.
  $PARM (string)   - the parameter list from the SET FORMAT (or SET
                      GLOBAL FORMAT) command.
  $TESTFAIL(flag)  - can be set to indicate that format processing
                      should continue after a serious INPROC error as
                      long as no PUTELEM statement is executed in that
                      label group.
  $ENDATA (flag)   - set when a GETDATA statement tries to access data
                      outside the buffer.
  $STARTCHAR(str)  - the character causing SPIRES to stop scanning for
                      "SCAN = (chars)" or "SCAN = ~(chars)".
  $ENDCHAR (str)   - the character causing SPIRES to stop scanning for
                      "SCAN = END,(chars)" or "SCAN = END,~(chars)".
  $WDSLINE (line)  - the line number of the last line read from the
                      active file.
  $SUPPRESS(flag)  - can be set to prevent data from being read into
                      the buffer; GETDATA statements are ignored.

C.3.9  Designing and Coding Input Frames

The various pieces of input label groups have been covered in the preceding sections of this chapter. Now that you have the ingredients, you need to know the commonly used procedures to combine them into label groups.

Writing input label groups for most elements is straightforward -- using your knowledge of output label groups in combination with the examples in the previous sections you could probably write simple frame definitions.

The elements may be processed in any order that is also allowed by the standard SPIRES format. For instance, the key does not have to be the first element processed; it may even be the last. And not all occurrences of an element need to be processed at the same time. [See C.3.9.2 for details on the order of element processing within structures.]

Some of the topics considered in this section are multiply occurring elements, structures, keys and the order of elements. Remember that, although this chapter specifically addresses the subject of input frames of USAGE=FULL that read data from a data set, many of the concepts apply to other types, such as prompting input formats. Where they do not apply will usually be pointed out.

C.3.9.1  Multiply-Occurring Elements

Multiply-occurring elements are as easily handled in input frames as output frames. The LOOP and XSTART statements are generally used in such situations, with an occasional assist from the SET PUTOCC Uproc. [See C.3.3.1, C.3.5.1.] The PUTELEM statement, usually without occurrence numbers when the LOOP statement also appears in the label group, is also an important part. [See C.3.3.]

Here is a relatively simple example of a label group to process multiple occurrences of an element:

This label group handles any number of occurrences of the element, from 0 to "n". Each time the GETDATA statement is executed, the next row of data is read from the buffer, beginning with row 5. Because no MAXROWS or LENGTH statement is coded, only one row at a time will be retrieved. [See C.3.5.3.] As soon as a blank row is retrieved by the GETDATA statement, no further execution of the label group occurs (as specified by the GETDATA = 5 statement). [See C.3.1, C.3.9.3 for other ways to stop.] If desired, the user may precede each row of cross references with the string "CR:", which is changed to null by the INPROC rule $CHANGE. That, interestingly enough, provides a way to enter a null value for the element:

That row would be retrieved, the rest of the label group would not stop executing since the retrieved value was non-null, and then the value would be converted to null by the INPROC. [See C.3.9.3 for more on null handling.]

Remember that once a PUTELEM occurs, the data placed in the record may not be "backed out" by the format (an INCLOSE rule, such as $MAX.OCC, might discard it, but that is not under format control). Thus, if you do use occurrence numbers in the PUTELEM statement, looping through that label group will not replace that occurrence each time -- each added "second occurrence" (for example) will follow the previously added one. [See C.3.3.]

Another way to handle multiple occurrences is to have the user input the multiple occurrences separated by a character such as a comma or slash (/); the format could then retrieve the complete set of values as a single value and, using the $BREAK proc (A45 action) in the label group's INPROC string, split the value into the multiple element occurrences. [See C.3.4.1.]

C.3.9.2  Structures

Structures are much easier to handle in input formats than in output formats because the structure's elements do not have to be processed within indirect frames. Thus, an input format for a record-type containing structures and even structures within structures ("nested structures") may be considerably simpler, in terms of the different frames involved, than a comparable output format for the same record-type.

On the other hand, if indirect frames are not used and the structure is multiply occurring (as they usually are), you will have to create your own programming loop for the label groups of the structure. If you were using an indirect frame, you would use the LOOP statement to cause the indirect frame of the structure to be re-executed. The example at the end of the section demonstrates this point.

The rules for handling structures in an input format definition are quite similar to the rules for data entry of structures using the standard format:

Here is an example of part of a line-by-line frame that processes a structure called ADDRESS, which has elements STREET.ADDRESS, CITY, STATE and ZIPCODE:

Here is an example of some data that might be processed by those label groups:

Note that opening an occurrence of a structure and then not putting any elements into it does not cause any problems. In fact, doing that has no effect at all -- simply entering and leaving a structure does not advance the counter keeping track of which structural occurrence you are processing. (Keeping track of the proper occurrence is more important in merge processing.)

C.3.9.3  Guidelines for Data Retrieval with the GETDATA Statement

Trying to decide what combination of statements should be used to retrieve a value from the buffer can be trying. You have many statements from which to choose, and of course the combinations of statements make your decision more complex.

Below is a set of questions, each followed by suggestions on what statements to choose to handle the particular situation. They may not handle all situations, but they are designed to give you some ideas. Many of these techniques are used in the sample format definition at the end of this chapter.

The term "fixed" is used several times in the questions, for instance, in the term "fixed starting position". Here, it means that the starting position will always be exactly the same for a particular value, such as row 3, column 2. However, users will make mistakes, and sometimes values are entered in the wrong place by accident. (If the GETDATA statement fails to retrieve a value, or the proper value, from a "fixed" starting position, then stopping the format processing at that point is recommended.) But for the purposes of this discussion, the term "fixed" refers to what the format is expecting, not to what the user actually might enter.

* Element Input Order

Will the elements be entered in a specific order?

Most input formats are designed so that the elements are entered in a specific order -- for example, NAME is always on the row before ADDRESS, which is always before CITY, etc. In that case, the element values may not need labels to identify them, such as "ADDRESS:" or "NAME=", since SPIRES always knows which element value it will retrieve from that row. (Labels may still be useful for data entry purposes, however, and may be stripped off with the $CHANGE function or system proc, or with a combination of the SCAN and GETDATA statements.)

Will the elements be entered in an unknown order?

If the data values may be entered in any order, then labels on them will be necessary, e.g., "NAME = Phillip Tertip" rather than just "Phillip Tertip". In such cases, all elements are usually entered with the same type of label, such as "elemname=" or "elemname:". Label groups are written to retrieve the element-name label and then retrieve the following value, assigning its value to the named element. [See C.3.9.5.]

Another possible method for fixed-dimension frames (though it would be inefficient for anything but a buffer with fewer than, say, five rows) would be to use the SCAN statement to scan for the label of each specific element, e.g., to tell SPIRES to scan the entire buffer for the NAME label in one label group and then for the ADDRESS label in the next, and so forth. However, this method is not generally recommended.

* Starting Position

Is the starting position fixed?

If the value will always start in the same fixed position, code the START statement with numeric values for "row" and "col", e.g.,

Is the starting position relative to other values?

If the value's location depends on the ending location of the previous value, code the START statement with X or * or some other variables as values for "row" and "col". Be sure you understand how the variables $CROW (*), $CCOL (*), $SROW, $SCOL, $LROW (X-1) and $LCOL (X-1) are reset. [See C.3.9.4.]

Is the starting position unknown but within a specified area?

If you know the value will be starting within a given area, you can code the START, MARGINS, MAXROWS and/or LENGTH statements to define the area for searching and then code the SCAN statement to search for the value within the area. Be sure to add the "6" option to the GETDATA statement too.

Is the starting position a complete unknown?

See "Will the elements be entered in an unknown order?" above.

* Is the ending position fixed?

That is, is there a character position in the buffer beyond which the value cannot go but before which no other value can start? If so, then MAXROWS or LENGTH should be coded to tell SPIRES how many rows or how many characters to retrieve. (Note that the value can end earlier than that position, as long as the remainder of the area is left blank and those trailing blanks are stripped from the value at some point, probably by GETDATA.) If the boundary position is the end of the current row, then neither MAXROWS nor LENGTH is needed, since SPIRES will retrieve a value only from the current row if neither they nor a SCAN statement is coded.

If the ending position is not fixed, then some signal must appear in the data to tell SPIRES where the value ends. In the standard SPIRES format, the signal is a semicolon character. Code the SCAN statement with the END option to tell SPIRES what to use as an ending signal when retrieving the value. "GETDATA = 3" can be coded if you want the scanned ending character(s) to be left on the retrieved value. The beginning of the next value could also be the ending signal, if it is uniquely identifiable in the area being scanned.

* Can a single value wrap around into multiple rows?

If the answer is no, then nothing special is required. If it is yes, and the number of rows needed is fixed (again, meaning that no other value can appear in that area), then code the "MAXROWS = n" statement where "n" is the number of rows. Alternatively, code the LENGTH statement. [See C.3.5.2.] Be sure to specify a MARGINS statement if the left and right margins of the area are not to be "1" and the right margin of the frame, respectively.

If the answer is yes, but the number of rows is not fixed, code the SCAN statement with the END option to locate the end of the value within the area. It is also a good idea to code a MAXROWS or LENGTH statement to define a boundary for the value -- otherwise, if the scanning does not find the end of the value, an S808 error will occur in fixed-dimension frames while in line-by-line frames, SPIRES will read line after line until the end of the data is reached (all of the data retrieved to that point will be returned as the value).

* Is there a fixed number of occurrences for the element?

If there will always be one occurrence of an element, nothing special is required. If there will be a fixed number of multiple occurrences, code the "LOOP = n" statement and, if necessary, the XSTART statement.

If the number of occurrences is unknown, you might consider designing the format so that each new occurrence will begin on a new row and that a blank row will signal the end of occurrences.

An easier solution, if it is available, is to have the $BREAK proc (A45) in the element's INPROC, either in the file definition or in the format label group's INPROC statement. With that method, a break character is used within the retrieved value to delimit multiple occurrences of the element. Unlike other methods, this technique does not require the use of the LOOP statement since all occurrences are retrieved at once by the GETDATA statement. [See C.3.4.1.]

Other methods for handling this problem are available, including using two different characters to indicate the end of the scanned value, one to delimit occurrences and the other to signal the end of the last occurrence. The $ENDCHAR variable can be checked to determine which one ended the current occurrence. Another method might involve placing the retrieved value into a variable and then parsing that value for separate occurrences. Both of these methods require multiple label groups, however.

* Can the end of the input data be detected?

There are two aspects of this question to consider here: the end of the buffer, and the end of the data being read into the buffer. It is easy to determine where the end of the buffer is: the system variables $NROWS and $NCOLS contain the frame dimensions. Those values can be compared to $CROW and $CCOL to determine how close retrieval is occurring to the end of the buffer. If you are not keeping track of the current position within the buffer and a GETDATA tries to retrieve data from beyond the buffer's boundaries, an S808 error will occur, and further format processing will stop.

The other aspect occurs in line-by-line processing where data is being read into the buffer one line at a time as requested by GETDATA statements. When SPIRES tries to read a line into the buffer and discovers that the last one has already been read, the $ENDATA flag is set and format execution continues. The next GETDATA statement to be processed, however, causes an ABORT Uproc to be automatically and immediately executed. The trick then is to check the $ENDATA flag after GETDATA statements that are likely to read the last line of input data. Several examples of this technique have already been shown, and it is also used in the sample format at the end of this chapter. [See C.3.3, C.3.5.5, C.3.10.]

* Should null values be stored or be treated as non-occurrences?

If a value is supposed to appear in a given location and only blanks are there, does the null value retrieved by GETDATA indicate that the element does not occur, or does it mean that the element value is to be null?

If you want it to mean a null occurrence will be stored for that element, nothing special is required. The GETDATA statement will establish $UVAL as null, and the remainder of the label group will be executed, including the PUTELEM. (Of course, the element's INPROC rules could make the value non-null.)

If you want the retrieved null value to mean no occurrence of the element, then you must code either "GETDATA = 5" or "GETDATA = 8". In the first case, the remainder of the label group will be skipped; no further processing of the label group will take place. In the latter case, the remainder of the label group will be processed, except that the SET REMOVEL Uproc is in effect. In merge frames, that means that the PUTELEM statement is changed to a REMELEM. [See C.5.1.4.] But in frames of usage FULL, SET REMOVEL is equivalent to the SET SKIPEL Uproc, which indicates that the PUTELEM statement should be skipped. In other words, in frames of usage FULL, "GETDATA = 8" means that the remainder of the label group, except for the PUTELEM statement, should be executed.

* Is the value to be retrieved not a string?

If the input data is coming from another program, the individual values might be non-string data, such as integers or packed decimals. By default, GETDATA statements strip the leading and trailing blanks (hex 40 characters) from strings, but these blanks may be significant if the data is non-string. Add the "1" and "4" options to the GETDATA statement to prevent the blank-stripping.

C.3.9.4  Using the Position Variables

The rules SPIRES uses for input frames to set the position variables such as $CROW and $CCOL are different from those for output frames. The reason why involves the differences between the area defined from which the value should be retrieved, the "scanned" value, and the actual retrieved value.

The variables $SROW and $SCOL, referring to starting row and column, are the easiest to establish rules for. These two variables are always reset when a GETDATA goes to retrieve data from the buffer, whether or not a value is actually retrieved. They represent the starting position given by the START statement or assumed by SPIRES if no START statement is coded in the label group. They do not reflect any SCAN statement in effect, meaning that they will represent the point where SPIRES begins scanning, not where the value actually begins.

Consider this label group example:

The values of $SROW and $SCOL will always be 3 and 1 respectively.

The label group demonstrates an important point about the position variables in input frames -- their values are all reset before Uprocs have any access to them. If you displayed the values of $SROW and $SCOL with a Uproc like the one above in an output frame, they would reflect the starting position of the last value placed in the buffer, not the value being placed by the current label group.

The values of $CROW and $CCOL (*) are based on the character position where the last retrieved value ended. This position may represent a blank or a pad character that was stripped from the value; in other words, it is the last character position from which the value could have been extracted, not necessarily the last position of data in the value. If the SCAN statement is coded with "GETDATA = 3", then the position of the ending scan character is represented by $CCOL and $CROW, since it is part of the value. However, if "GETDATA = 3" is not coded, then the last character position before the ending scan character is used for $CROW and $CCOL.

The variables $LROW and $LCOL (both representing X-1) may be the same as $CROW and $CCOL respectively, though they follow somewhat different rules. The $LCOL variable is closely connected to $CCOL. They are usually the same, except when a SCAN statement with the END option is coded and "GETDATA = 3" is not coded (meaning that the ending scan character is not part of the value). In that case, $LCOL represents the last position of the ending scan character or character string, whereas $CCOL represents the last position of the value preceding the ending scan character (see example below).

The $LROW variable reflects the highest-numbered row from which data has been (or could have been) retrieved. Assume for the remainder of this section that you are retrieving values in one row after another, so that each time you reset $CROW you are probably resetting $LROW too. If no LENGTH or MAXROWS statement is coded, $LROW represents the last row in which the data was retrieved, and if SCAN with the END option is coded, $LROW represents the row where the ending scan character appeared. However, if LENGTH or MAXROWS is coded, $LROW represents the row in which the LENGTH or MAXROWS value ends, not the row in which the value ends.

Consider this example:

Here is the input data set:

     (....v....1....v....2....v....3)
  (1) This is the input data set.
  (2)
  (3)

Here are the variable values after execution of this label group:

If "GETDATA = 3" were coded, the value of $CCOL would change to 27. If MAXROWS were omitted, the value of $LROW would change to 1.

C.3.9.5  The Order of Element Processing in an Input Format

Two different issues come to mind in a discussion of the order of element processing. One is the order in which the data is entered, the other is the order in which the data is processed. In line-by-line frames, the two are often the same, though even there, the frame may process the values in a different order than they appear on the row. But in a fixed-dimension frame, the input order may be completely different from the order in which the elements will be processed by the format, since the format may pick up values from anywhere in the buffer at any given time.

In any case, the important point to remember is that the data elements may be processed in any order by the input format, with the exception that elements within a structure must be processed together if they are to be part of the same structural occurrence. [See C.3.9.2.] Thus, SPIRES gives you an enormous amount of freedom in regard to element processing order, regardless of the type of frame dimensions in effect.

But what about the order of the input data? Is it possible to allow it to be input in any given order and still be processed correctly by the input format, just as data can be entered with the standard SPIRES format? Yes, it is possible. The data values entered by the user generally include labels of some type to indicate to SPIRES which elements are meant, similar to the element names used in the SPIRES standard format. Usually several label groups are necessary to process data this way:

These label groups assume that the input data is all record-level data (i.e., no data in structures), that the element names are separated from their values with blanks and that the values do not wrap around into multiple lines, e.g.,

Executing these label groups, SPIRES retrieves the value up to the first blank, assigning that value to the ELEMNAME variable. It then retrieves the rest of the data on the row and uses that data as the value for the element named in #ELEMNAME. SPIRES then goes back to the GET.MNEMONIC label group and starts again.

This simple set of label groups could be expanded to cover other situations. For instance, the element name could be verified to be a legitimate element mnemonic with the $ELEMTEST function, which could also be used to determine whether the named element is a structure, with appropriate branching to another label group if so.

The example at the end of this chapter also demonstrates how user-supplied labels identifying the values that follow can be used to control format processing. [See C.3.10.]

C.3.9.6  (*) Record Keys

Though it often begins the input data, the record key may appear at any point in the data and may be processed at any point in the format, with the exception that it may not be in the middle of a structure. (Actually that would cause a problem for the structure, not the key.)

Most input formats are designed for use with either the ADD command (usage of FULL) or the MERGE command (usage of MERGE). However, when an input format with frames of USAGE = FULL is set, it may also be invoked by an UPDATE command. For non-slot subfiles, that does not cause any problems -- the record being input, whether being added or updated, will have the record key. But for slot subfiles, a record being added should not usually have the key element (unless you are putting the correct value, found in the system variable $NEXTSLOT), while the opposite is true for a record being updated.

If you want your format to be useful for both adding and updating slot records, you will need to have a label group that reads and processes the key for updates but that is ignored for adds. One method to do this is to test the value of the $CURCMD (current command) system variable -- if its value is UPD, then the record is being updated. However, if its value is ADD, the command issued may be either ADD or ADDUPDATE.

Hence, a better method is to check the value of the key, if given, with the $RECTEST function:

You might want to check the input value against the value of $NEXTSLOT as well -- a record with that key might have been removed from the data base previously, and the user wants to add it back:

Remember that ":" is equivalent to THEN in an IF Uproc.

C.3.9.7  Formats for the ADDUPDATE and ADDMERGE Commands

If you intend to use the ADDUPDATE and ADDMERGE commands (or their variants, INPUT ADDUPDATE and INPUT ADDMERGE) with input formats, there are several important considerations to keep in mind as you put the formats together. All relate to the timing of the possible switch from SPIRES treating the input data as an add to treating it as an update or merge request.

As the previous sentence indicates, SPIRES (or SPIBILD) begins with the assumption that the record is being added. The first data frame of USAGE = FULL begins executing, and the value of $COMMAND is set to "ADD" at this point. (Under ADDMERGE, it may change later; see below.)

What happens next depends on the command issued. For ADDUPDATE, after format processing finishes, the record closeout processing occurs. Next, SPIRES tries to add the record; if that fails because a record with that key exists already, SPIRES tries to replace that record with the new record, i.e., changing to an UPDATE command. The $COMMAND variable is then reset to "UPDATE".

For ADDMERGE, when format processing is completed, SPIRES determines whether the record key was input, and if so, whether it matches the key of a record already in the subfile. If it does, then SPIRES transforms the record input into data to be merged into that existing record and applies it, creating the updated record. At that point, $COMMAND is set to "MERGE", and record closeout processing takes place.

An important aspect to notice is that format processing all happens prior to any test to determine whether the data is for an ADD or for either an UPDATE or MERGE. Hence the format thinks that an ADD is taking place, so it executes frames of usage FULL, even if the data will turn into a MERGE. The $COMMAND variable will always be set to "ADD" during format processing under ADDUPDATE or ADDMERGE.

Use $RECTEST function to determine if record exists

In many situations, SPIRES will handle the data appropriately regardless of whether it turns into an ADD, UPDATE or MERGE. However, there may also be situations in which you need to know during format processing whether the ADD will turn into an UPDATE or MERGE later. The very best way to determine what will happen is to use the $RECTEST function to tell you whether a record with the given key already exists in the subfile. Obviously, if it does, then the transaction will convert from an ADD to an UPDATE or MERGE later.

Knowing that may be useful, for example, if you need to reference the existing record to retrieve data from it before replacing it with the new version. [See C.5.2.]

Some technical details about the REMELEM statement and about occurrence numbers in PUTELEM statements in ADDMERGE formats will be discussed later in this manual, in the chapter on MERGE formats. [See C.5.1.3, C.5.1.4.]

C.3.10  A Sample Input Format Definition

The following input format definition demonstrates many of the statements and techniques described in this chapter. It was written for the BLOOD DONORS subfile, the same subfile for which the report format in Part B was designed. [See B.10.11.]

Like many input formats, it was written for trained data entry operators -- not every problem that could possibly arise is handled by the format. For example, if the user starts off with the wrong element, it will be interpreted as the value for the NAME element, regardless of its value. Code could be added to check that the first line does not begin with a label such as "P:" or "C:" or one of the others, but the designer did not think the additional code was worthwhile, either in terms of the design time or the processing time that would be required to execute all the data-verification statements for each record. In other words, you can overdesign your input format, trying to anticipate every possible problem that could arise -- if the data entry operators are well-trained, the additional overhead of all the extra checking may be unnecessary.

Here is a sample of input data, with notes to the right describing it:

The first lines are reserved for the NAME, ADDRESS, CITY and ZIPCODE elements, with additional occurrences of the ADDRESS element indicated by starting them with a plus sign. From that point on, elements are identified by labels, such as "B:" for BLOOD.TYPE. Multiple occurrences of all these elements are allowed in the format, with the exception of COMMENTS, which is assumed to be the end of the input if it occurs. Only the occurrence of the COMMENTS element may wrap around into multiple rows.

Below is a format definition that will process this data. Notes about specific label groups and statements appear at the end of the entire definition.

The Identification Section

 1.   ID = GQ.JNK.DONORS.INPUT;
 2.   MODDATE = THUR. JAN. 20, 1983;
 3.   DEFDATE = THUR. JAN. 20, 1983;
 4.   FILE = GQ.JNK.BLOOD.DONORS;
 5.   RECORD-NAME = REC01;

Vgroup Definitions

 6.   VGROUP = GQ.JNK.LOCAL;
 7.     VARIABLE = ELEMNAME;
 8.       OCC = 1;
 9.       TYPE = STRING;

Frame Definitions

10.   FRAME-ID = ADD;
11.     DIRECTION = INPUT;
12.     FRAME-DIM = 0,132;
13.     USAGE = FULL;
14.     LABEL = NAME;
15.       GETDATA;
16.       PUTELEM;
17.     LABEL = ADDRESS;
18.       START = X,1;
19.       GETDATA;
20.       UPROC = IF $LOOPCT > 0 : IF $LSTR($UVAL,1) ~= '+' : JUMP;
21.       UPROC = IF $LOOPCT > 0 THEN SET CVAL = $SUBSTR($CVAL,1,-1);
22.       UPROC = IF $LOOPCT > 2 THEN * 'Too many ADDRESS lines';
23.       UPROC = THEN ABORT;
24.       PUTELEM;
25.       LOOP;
26.     LABEL = CITY;
27.       MAXROWS = 1;
28.       START = *,1;
29.       SCAN = END,(0123456789);
30.       GETDATA = 6;
31.       PUTELEM;
32.     LABEL = ZIPCODE;
33.       START = *,*+1;
34.       LENGTH = 5;
35.       GETDATA = 5, 6;
36.       PUTELEM;
37.     LABEL = WHICH.ELEMENT;
38.       SCAN = END,(:);
39.       GETDATA;
40.       INPROC = $CAP;
41.       UPROC = IF $ENDATA THEN RETURN;
42.       UPROC = IF $CVAL = 'D' THEN JUMP DONATION;
43.       UPROC = IF $CVAL = 'CM' THEN JUMP COMMENTS;
44.       UPROC = IF $CVAL = 'P' THEN LET ELEMNAME = 'PHONE.NUMBER';
45.       UPROC = ELSE IF $CVAL = 'B' THEN LET ELEMNAME = 'BLOOD.TYPE';
46.       UPROC = ELSE LET ELEMNAME = 'CAN.BE.CALLED';
47.     LABEL = PUTELEM;
48.       START = *,*+2;
49.       GETDATA;
50.       PUTELEM = #ELEMNAME;
51.     LABEL;
52.       UPROC = JUMP WHICH.ELEMENT;
53.     LABEL = DONATION;
54.       PUTELEM;
55.     LABEL = DATE;
56.       START = *,*+2;
57.       SCAN = END,(,);
58.       GETDATA;
59.       PUTELEM;
60.     LABEL = LOCATION;
61.       START = *,*+2;
62.       GETDATA;
63.       PUTELEM;
64.     LABEL;
65.       UPROC = JUMP WHICH.ELEMENT;
66.     LABEL = COMMENTS;
67.       MAXROWS = 50;
68.       START = *,*+2;
69.       GETDATA;
70.       PUTELEM;

The Format Declaration Section

71.   FORMAT-NAME = INPUT;
72.     ALLOCATE = GQ.JNK.LOCAL;
73.     FRAME-NAME = ADD;
74.       FRAME-TYPE = DATA;

Notes:

14. This simple label group retrieves the value from the default row and assigns it to the NAME element.

17. The ADDRESS element will usually have only one occurrence, but it can have as many as three. The second and third occurrences are assumed to begin with a plus sign, which is stripped off by the $SUBSTR function. When SPIRES finds a line that does not begin with a plus sign, it assumes that that line holds the CITY and ZIPCODE values.

26. The CITY element will end when a numeral (the first digit of the ZIPCODE) is scanned.

37-65. These label groups comprise a large loop that handles the elements indicated by the labels preceding each value. Usually such a loop requires at least three label groups: the first to retrieve the label and set a variable containing the element name, the second to retrieve the value and put it into the named element, and the third to restart the LOOP. (Be aware that the JUMP statement that comprises the third label group cannot appear in the second, or the PUTELEM statement would not be executed.)

37. This label group retrieves the label from the next line and chooses the label group that should process it. First, if the $ENDATA flag is set, no more data is available, and no further format processing should be done. If the label is "D" for DONATION, the label groups processing the DONATION structure are executed. [See C.3.9.2.] If the label is "CM" for COMMENTS, the label group processing the COMMENTS element is executed. (Those two elements must be handled specially, the first because it is a structure, and the second because its value, which may wrap into multiple rows, is retrieved differently from the others.)

44. The user variable ELEMNAME is set to the appropriate value based on the value of the label.

47. This label group handles any of the elements PHONE.NUMBER, BLOOD.TYPE or CAN.BE.CALLED, depending on which is the current value of #ELEMNAME.

53. The DONATION structure processing begins with the PUTELEM statement to open the structure.

65. After structure processing, execution returns to the WHICH.ELEMENT label group.

66. Because the COMMENTS element is assumed to be the last element retrieved (if it occurs at all), execution does not return to the WHICH.ELEMENT loop.

71. Though the Format Declaration section for an input format has not been discussed yet, you can see that it uses the same statements (for the same purposes) as an output format's would. [See C.6.]

C.4  Prompting Input Formats

Frame definitions for prompting input formats look quite different from those for formats that read input from a data set, although many of the statements discussed in the last chapter are relevant. For example, PUTELEM statements are still necessary if you want to put element values into the record. However, since you "get data" by prompting for it, you do not code GETDATA statements but code ASK Uprocs instead.

Most of the material in the previous chapter is relevant to prompting input formats. In particular, you should be familiar with the information on the VALUE, PUTELEM, and INPROC statements, on the SET CVAL Uproc, on error handling and on structure handling.

Here is a simple example of a session using a prompting input format:

The example demonstrates how values may be requested (here shown by lines beginning with a colon) and responses from the format may be issued based on the user's responses.

Another example of a prompting input format is the system format $PROMPT. If you are not familiar with it, you should read section D.6 of the SPIRES manual "Searching and Updating".

The first section of this chapter discusses the FRAME-DIM statement for prompting input frames. Subsequent sections discuss the label groups.

C.4.1  The Frame Identification Statements

All of the frame identification statements discussed earlier [See C.2.] are the same for prompting input frames, with the exception of the FRAME-DIM statement, which is not coded in this case. The absence of the FRAME-DIM statement tells SPIRES not to read data from an external data set (such as the active file). Instead, the data can be provided from the user at the terminal through the ASK Uproc, or it can be created by the format itself, if desired. [See C.4.2.]

Omitting the FRAME-DIM statement is not the same as coding:

which requests line-by-line processing and the default buffer width. [See C.2.3.]

C.4.2  Label Groups for Prompting Input Formats

The major difference between label groups for prompting input formats and label groups discussed in the last chapter is that the SET UVAL and DOPROC Uprocs are necessary to properly process a single element in a prompting input label group:

SPIRES does not execute the INPROC rules till the DOPROC Uproc tells it to, which occurs after the user has been prompted for a value and that value has been established in $UVAL. [See C.3.4.3.]

The reason those two Uprocs are necessary is best shown by trying to rewrite the label group without them:

The problem with that label group is that the value to be stored is received from the user after INPROC (though not INCLOSE) rules have been executed, and are thus not applied to the value. In fact, it is a null value, the value of $UVAL at the start of the label group, that would be processed through the INPROC string.

To get the benefit of processing rules, you should tell SPIRES not to execute them until $UVAL has been set to the user's input value, which means coding the DOPROC Uproc. Alternatively, you could use two label groups, one to prompt for the user's value and the other to put the value into the record:

This method often becomes awkward to use when error handling is involved or when the element processing is more complex -- using the SET UVAL and DOPROC team is preferable. [See C.4.2.1.]

C.4.2.1  Error Handling

You will probably want your pair of label groups to be a bit more complicated than those shown above. For example, you might want to handle errors properly, announcing the error and allowing a reprompt:

The SET TESTFAIL Uproc tells SPIRES not to set the $GPROCERR flag when a serious error occurs unless a PUTELEM is executed for that element. It is quite important for prompting input formats since it prevents invalid values from setting $GPROCERR, instead allowing you to reprompt for a better value. Remember that $TESTFAIL should be set before entering any label group in which it is needed; also remember that it remains in effect for the duration of the format execution, unless turned off by the SET NOTESTFAIL Uproc. [See C.3.6.2.]

If desired, you could replace the error message shown with $UCODE, as established in the element's INPROC string:

The $UCODE variable was discussed earlier. [See C.3.6.3.]

C.4.2.2  Null and Attention Responses

Another area of concern for prompting input formats is how to handle BREAK/ATTN and null responses. By default, SPIRES will execute the ASK Uproc again when a null response is received (i.e., only the return key is pressed); when the ATTN/BREAK key is pressed in response to an ASK prompt, SPIRES continues executing the label group. These defaults can be changed by including ATTN and NULL clauses on the ASK Uproc. [See B.4.8.11.]

As a reminder, here is the syntax of the ASK Uproc:

The easiest way to deal with null and attention responses is with the XEQ PROC Uproc. In the following example, the XEQ PROC Uproc appears in both the NULL and ATTN clauses, referring to procs at the end of the frame definition. Note that the particular element processed by the first label group is controlled by the user variable ELEMNAME:

Notes on the example:

C.4.2.3  Providing Help to the User

When the data entry involves more than name, rank and serial number, format definers may want to provide help to the user entering records. Generally this help takes the form of providing additional information about an element being prompted for. For example, consider the following prompt for information:

What is the desired response: a number, a list of names, or simply a YES or NO? Of course, the prompt could be improved to make the desired type of response clear. But the format definer could also look for a particular value from the user that indicates the user wants additional information or help.

The user could, for instance, be allowed to respond to the prompt with the word HELP or with a question mark:

The format could check the user's response and respond accordingly:

Uprocs intercept the user's "?" responses, providing more information and then re-entering the label group to reprompt the user for another value.

You might want to take advantage of help already provided by the file owner in the file definition. If the file owner created DESCRIPTION info-elements for the elements of the file (either by creating them directly or by using the EXPLAIN attribute for elements when using File Definer), those explanations can be displayed from a format. You can retrieve the info-element values using the $ELEMINFO function.

Displaying the DESCRIPTION info-elements for an element can pose some tricky problems, however. The primary complication, in many cases, is that the DESCRIPTION info-element is meant to hold a "paragraph" (i.e., multiple lines of output) of information, while the "*" Uproc, which is the easiest way to display information to the terminal during input operations, is meant for single lines. Multiple lines of information will wrap around in the middle of words when displayed with the "*" Uproc.

Both the $PROMPT system format and the $PROMPT-like formats generated by the GENERATE FORMAT command [See D.4.1.] solve this problem in a clever way, using a device services area called TER, for "terminal". When help for a specific element is requested, the formats execute an indirect frame of direction OUTPUT that places the DESCRIPTION info-element values into the TER area, formatting it properly; from there it is sent to the terminal. This is one important use for the technique of creating output during input, discussed earlier in the manual. [See B.16.1.]

You might want to see some sample code for that technique. The best way to do that is to generate a $PROMPT-like format yourself, using the GENERATE FORMAT command. [See D.4.1.] Look for the indirect frame ELEM.HELP and the label groups that call it.

Other ways of helping the user are usually based on the above methods: 1) intercepting a special data value from the user that indicates help is desired, via the ASK Uproc and the $ASK variable; then 2) executing other statements, usually "*" Uprocs, to give the help; and then 3) returning to the original prompt. Variations may include: 2) branching to other subroutines, to give more help or allow the user to exit the format processing completely; and 3) proceeding to a different set of prompts, perhaps with help provided automatically.

C.4.3  A Sample Prompting Input Format Definition

Below is the definition for a prompting input format for the BLOOD DONORS subfile, which has been used as the basis for several other sample formats in this manual. [See B.10.11, C.3.10.] Following the definition is a sample session showing its use, followed by some notes about the definition.

Be aware that this format, though it does accomplish its purpose, is not nearly as sophisticated or as simple to set up and use as the system format $PROMPT or a custom-generated version of $PROMPT, created with the GENERATE FORMAT command. [See D.4.1.]

  1.  ID = GQ.DOC.DONORS.PROMPT;
  2.  FILE = GQ.JNK.BLOOD.DONORS;
  3.  RECORD-NAME = REC01;
  4.  VGROUP = LOCAL;
  5.    VARIABLE = COUNTER; OCC = 1; TYPE = INT;
  6.  FRAME-ID = PROMPT;
  7.    DIRECTION = INPUT;
  8.    USAGE = FULL;
  9.    LABEL = INTRODUCTION;
 10.      UPROC = * 'Prompts preceded by (R) indicate required elements.';
 11.      UPROC = * 'Pressing RETURN for a non-(R) element will skip it.';
 12.      UPROC = * ' ';
 13.      UPROC = SET TESTFAIL;
 14.    LABEL = NAME;
 15.      UPROC = ASK PROMPT '(R) Donor''s name?' ATTN='XEQ PROC ATTN';
 16.      UPROC = SET UVAL = $ASK;
 17.      UPROC = DOPROC;
 18.      UPROC = IF $PROCERR THEN JUMP NAME;
 19.      PUTELEM;
 20.    LABEL = ADDRESS1;
 21.      UPROC = ASK PROMPT '(R) Address line 1?' ATTN='XEQ PROC ATTN';
 22.      UPROC = SET UVAL = $ASK;
 23.      UPROC = DOPROC;
 24.      UPROC = IF $PROCERR THEN JUMP ADDRESS1;
 25.      PUTELEM = ADDRESS;
 26.    LABEL;
 27.      UPROC = LET COUNTER = 2;
 28.    LABEL = ADDRESS2;
 29.      UPROC = SET PROMPT = 'Address line ' #COUNTER ' (RETURN=No more)?';
 30.      UPROC = ASK NULL='JUMP CITY' ATTN='XEQ PROC ATTN';
 31.      UPROC = SET UVAL = $ASK;
 32.      UPROC = DOPROC;
 33.      UPROC = IF $PROCERR THEN JUMP ADDRESS2;
 34.      PUTELEM = ADDRESS;
 35.    LABEL;
 36.      UPROC = LET COUNTER = #COUNTER + 1;
 37.      UPROC = IF #COUNTER > 3 THEN JUMP CITY;
 38.      UPROC = JUMP ADDRESS2;
 39.    LABEL = CITY;
 40.      UPROC = ASK PROMPT '(R) City?' ATTN='XEQ PROC ATTN';
 41.      UPROC = SET UVAL = $ASK;
 42.      UPROC = DOPROC;
 43.      UPROC = IF $PROCERR THEN JUMP CITY;
 44.      PUTELEM;
 45.    LABEL = STATE;
 46.      UPROC = SET PROMPT = '(R) State (RETURN=CA)?';
 47.      UPROC = ASK NULL='JUMP ZIPCODE' ATTN='XEQ PROC ATTN';
 48.      UPROC = SET UVAL = $ASK;
 49.      UPROC = DOPROC;
 50.      UPROC = IF $PROCERR THEN JUMP STATE;
 51.      PUTELEM;
 52.    LABEL = ZIPCODE;
 53.      UPROC = SET PROMPT = 'Zipcode?';
 54.      UPROC = ASK NULL='JUMP PHONE' ATTN='XEQ PROC ATTN';
 55.      UPROC = SET UVAL = $ASK;
 56.      UPROC = DOPROC;
 57.      UPROC = IF $PROCERR THEN JUMP ZIPCODE;
 58.      PUTELEM;
 59.    LABEL = PHONE;
 60.      UPROC = SET PROMPT 'Phone Number (RETURN=No more)?';
 61.      UPROC = ASK NULL='JUMP CAN.BE.CALLED' ATTN='XEQ PROC ATTN';
 62.      UPROC = SET UVAL = $ASK;
 63.      UPROC = DOPROC;
 64.      UPROC = IF $PROCERR THEN JUMP PHONE;
 65.      PUTELEM;
 66.    LABEL;
 67.      UPROC = JUMP PHONE;
 68.    LABEL = CAN.BE.CALLED;
 69.      UPROC = SET PROMPT 'Can we call to request a donation?';
 70.      UPROC = ASK NULL='JUMP BLOOD.TYPE' ATTN='XEQ PROC ATTN';
 71.      UPROC = SET UVAL = $ASK;
 72.      UPROC = DOPROC;
 73.      UPROC = IF $PROCERR THEN JUMP CAN.BE.CALLED;
 74.      PUTELEM;
 75.    LABEL = BLOOD.TYPE;
 76.      UPROC = SET PROMPT = 'Blood type?';
 77.      UPROC = ASK NULL='JUMP DONATION' ATTN='XEQ PROC ATTN';
 78.      UPROC = SET UVAL = $ASK;
 79.      UPROC = DOPROC;
 80.      UPROC = IF $PROCERR THEN JUMP BLOOD.TYPE;
 81.      PUTELEM;
 82.    LABEL = DONATION;
 83.      UPROC = SET PROMPT = 'Date of donation (RETURN=No more)?';
 84.      UPROC = ASK NULL='JUMP COMMENTS' ATTN='XEQ PROC ATTN';
 85.      UPROC = SET UVAL = $ASK;
 86.      UPROC = DOPROC;
 87.      UPROC = IF $PROCERR THEN JUMP DONATION;
 88.      PUTELEM;
 89.    LABEL = LOCATION;
 90.      UPROC = SET PROMPT = 'Location of Donation?';
 91.      UPROC = ASK NULL='JUMP DONATION' ATTN='XEQ PROC ATTN';
 92.      UPROC = SET UVAL = $ASK;
 93.      UPROC = DOPROC;
 94.      UPROC = IF $PROCERR THEN JUMP LOCATION;
 95.      PUTELEM;
 96.    LABEL;
 97.      UPROC = JUMP DONATION;
 98.    LABEL = COMMENTS;
 99.      UPROC = ASK PROMPT 'Comments?' NULL='RETURN' ATTN='XEQ PROC ATTN';
100.      UPROC = SET UVAL = $ASK;
101.      UPROC = DOPROC;
102.      UPROC = IF $PROCERR THEN JUMP COMMENTS;
103.      PUTELEM;
104.    LABEL;
105.      UPROC = JUMP COMMENTS;
106.    LABEL = ATTN;
107.      UPROC = ASK PROMPT 'Abort this request?' ATTN='ABORT';
108.      UPROC = IF $PMATCH($CAP($ASK),YES,OK?) THEN ABORT;
109.      UPROC = ELSE RETURN;
110.  FORMAT-NAME = PROMPT.INPUT;
111.    ALLOCATE = LOCAL;
112.    FRAME-NAME = PROMPT;
113.      FRAME-TYPE = DATA;

Here is the sample session using this format:

Notes on the format definition (numbers refer to line numbers):

C.5  Merge Formats

Input formats for adding and updating records must supply all the values that will be stored in the record, except for any created by INCLOSE rules. "Merge" input formats, used in connection with the MERGE command, are different -- they will provide new values or replacement values or remove old values from a record that already exists, and they do not need to concern themselves with elements that are to be left unchanged. Moreover, they can read element values from the old record and make changes based on the older values.

When you issue a MERGE command, SPIRES retrieves the desired record and merges the given data into it. The new element values are processed through any INPROCs; old element values being removed are simply discarded. After the format processing ends, record closeout occurs -- INCLOSE rules for all record-level elements are executed, whether or not data was merged into them. INCLOSE rules for elements within occurrences of structures that have changed will also be executed. The revised record will then be placed in the deferred queue as an updated record.

A format with one or more frames of USAGE = MERGE can be used to merge records. The format may read data from a data set, or it may prompt for data, or it may itself create the data for merging. If desired, data can be read from the original record and then changed, applying the new data to the old data. This capability requires "referenced record processing".

The procedures for defining and creating an input format discussed thus far in Part C also apply to merge formats. In addition, some new statements and values are available for new functions. For example, the USAGE statement has two more values: MERGE, which was mentioned above, and REFERENCE, which allows you to create a frame with GETELEMs to retrieve the current values of elements in a record being merged. (Otherwise, GETELEM statements are restricted to output frames.) The REMELEM statement and the SET REMOVEL Uproc are introduced here. They let you remove element occurrences from the record being merged.

The first section of this chapter will discuss merge formats that add, replace or remove data from records without considering the current values in the record. [See C.5.1.] The next section discusses referenced record merging, in which SPIRES can retrieve values from the record being merged. [See C.5.2.]

(*) Formats for updating records: MERGE vs. UPDATE

In SPIRES, two different commands can be used to make changes to goal records in a subfile: the UPDATE command and the MERGE command. Both of these commands may employ formats in their processing. If a format is set, SPIRES will look for data frames of usage FULL to use when the UPDATE command is issued, and for data frames of usage MERGE when the MERGE command is issued. [See C.2.4.] Both commands put a "new record" into the deferred queue to replace an older record when the file is processed, but how the new record is created varies. How do you decide which command to use and, perhaps, which command to write an input format for?

Consider how these two commands work when formats are not involved. The UPDATE command, generally used with the TRANSFER command, presents SPIRES with a complete record to replace the old version of the record in the data base. SPIRES does not examine the old version at all -- whatever is presented as the "updated record" (along with values provided by INCLOSE rules) completely replaces the old version. All values are processed through their INPROC rule strings, whether or not they have actually changed -- since SPIRES considers the input record to be a complete replacement, it has no idea (and does not care) whether any given value has changed or not.

On the other hand, the MERGE command merges the new data into the old version of the record. The user provides only part of the record data (the element or elements to be changed) and SPIRES creates the new record by combining that data with the old data. Only the INPROCs of the new data are executed (though most INCLOSE rules are executed; see above). Thus, the MERGE command is generally more efficient than UPDATE because only those elements that are actually changed are handled by the processing rules; moreover, the user only needs to present the changed data, rather than the entire replacement record, to SPIRES.

These characteristics hold true when formats are also involved. A format used with the UPDATE command needs to handle all the elements desired in the record. Each element in the final record (again, with the exception of elements whose values may be provided by INCLOSE rules) must be placed using a PUTELEM statement. Referenced record processing may be used to retrieve the element values in the old version of the record for comparison or for placing in the new version. But again, the significant fact to remember is that the entire record must be dealt with in the format.

Merge formats generally deal with only a few elements. They are more frequently created and used than formats for the UPDATE command because they are simpler and more efficient, handling just the elements the user wants to change.

Some other considerations regarding formats for the UPDATE command are discussed elsewhere. [See C.3.9.6, D.1.1.3.]

C.5.1  Merge Frames

This section will discuss the statements specified in frames used for record merges.

C.5.1.1  The USAGE Statement

To be used when a MERGE command is issued, a frame must have a usage of MERGE:

When a MERGE command is issued, SPIRES will check the set format for the first (or only) data frame with USAGE = MERGE coded, and will execute it unless it is also declared as NAMED. [See D.1.1.1.]

MERGE is the default usage if the USAGE statement is omitted from a frame of direction INPUT.

Input Formats and Data Frames

Important note: when processing a MERGE (or ADD, UPDATE, or ADDUPDATE) request in an input format, SPIRES only executes a single data frame, even if you have coded and declared more than one data frame of the appropriate DIRECTION and USAGE in the format. This is quite different from the situation with output formats, where you can code multiple data frames, to be executed in the order in which they are declared. [See B.5.2.] The restriction to a single data frame helps SPIRES process update requests more efficiently.

C.5.1.2  Handling Elements in Merge Frames: PUTELEM and REMELEM Statements

The type of processing done to any given element in a merge frame depends on the PUTELEM or REMELEM statement in the label group. The PUTELEM statement tells SPIRES to add or replace an element occurrence with the current label group's value, while the REMELEM statement tells SPIRES to remove a particular element (or structure) occurrence or all occurrences. Occurrence numbers, when multiply occurring elements are involved, are much more frequently used in merge formats than in non-merge input formats -- they also are more complicated. A discussion of the occurrence counting methods involved in merge formats appears in the next subsection. [See C.5.1.3.]

The syntax of the PUTELEM statement was given earlier. [See C.3.3.] The syntax of the REMELEM statement is identical:

All occurrences of the element involved will be removed, using the forms shown above; for removal of a particular occurrence, you must specify an occurrence number (see below). In the first form, "element.name" represents the name of the element being stored. The second form, in which no element name is given, may be used when the element name is given in the LABEL statement. [See B.4.1.] The third form may be used when the element name is stored in a variable. Information about the last two forms, as well as more details about the first three, appeared in the sections on the GETELEM statement. [See B.4.2, B.4.2.1.]

To remove a particular occurrence rather than all occurrences, you may use the following form:

where "n" is an integer or a variable with an integer value representing the occurrence number, counting from 0. [See C.5.1.3.] Any of the values shown for REMELEM in the syntax shown above may be used in place of "element.name", with the exception of the second form, which does not have a value. The occurrence number may also be controlled by the SET PUTOCC Uproc. [See C.3.3.1.]

Remember that INCLOSE processing rules will be executed for all record-level elements and elements within structures that have been changed when the format execution is completed, including elements whose occurrences were all removed by the format.

A REMELEM statement coded in a frame of usage FULL is usually ignored; basically, it has an effect only in frames of usage MERGE. The exception to this rule is under formatted ADDMERGE processing when the operation becomes a MERGE rather than an ADD -- REMELEM statements in a frame of usage FULL become effective.

C.5.1.3  Occurrence Numbers in Merge Frames

Occurrence numbers for elements work differently in different situations. This section will consider some of the differences in their use between frames of usage FULL and usage MERGE, differences between those and the standard SPIRES format, and differences when the ADDMERGE command is used.

In the section on the PUTELEM statement in frames of usage FULL, a warning to think of occurrence numbers as relative rather than absolute appeared. For example, if multiple occurrences of an element were processed via the statement:

each new occurrence processed would follow the previous one rather than replace it. [See C.3.3.] (But see below how that can change under ADDMERGE.)

However, in a frame of usage MERGE, the same statement would cause each new occurrence processed to replace the previous one. In other words, the occurrence numbers are absolute in a frame of usage MERGE.

That rule applies even if you are adding element occurrences to the end of previous ones:

In the example, the occurrence number is controlled by the variable #OCCNUM. It is set arbitrarily high (1000) to ensure that the new occurrences will be placed at the end of the occurrences already there. [But don't set it too high -- the limit for an element is 16382, and for a structure, 8190.] In a frame of usage FULL, the same occurrence number could be used for all values being added at the end. However, in a frame of usage MERGE, if the same occurrence number were used each time, the value for that occurrence number would be replaced each time; hence, it is important to increment it each time, as shown in the example.

Occurrence numbers in FULL frames under ADDMERGE

For ADDMERGE processing, where all work is done in a FULL frame, the significance of any occurrence numbers used changes depending on whether the record ends up being added or updated. If the data becomes an added record, then the occurrence numbers become "relative", as described above. But if the data ends up being merged into a record that already exists, the occurrence numbers become "absolute" -- all references to a given occurrence number affect that specific occurrence.

How PUTELEMs and REMELEMs Work Together

In some situations, it is important to understand the order in which PUTELEMs and REMELEMs affect the record being merged. Specifically, when format processing is completed and SPIRES is applying all the requested transactions to the record, all the REMELEMs are applied first, followed by the PUTELEMs. [Under ADDMERGE processing, if the data becomes an ADD, any REMELEMs are ignored completely. [See C.5.1.4.]]

Consider the following pair of label groups:

As in the earlier example, the occurrence number 1000 was assigned in the PUTELEM statement to ensure that the value would be put as the last occurrence of the DATE.UPDATED element. However, the next label group attempts to remove that occurrence. It would not succeed in its intention though, because SPIRES will remove the occurrences specified in REMELEMs before adding or replacing occurrences specified in PUTELEMs. Thus, the second label group would have no effect (as long as the record being merged did not actually have a "thousandth" occurrence of the element); the record after the merge was completed would have the "thousandth" occurrence following all other occurrences of the element.

The above example points out that you cannot remove (via REMELEM statements) element values that are new occurrences -- once a new occurrence has been created, its value can be replaced by other PUTELEM statements, but REMELEM statements will not affect it. Once a REMELEM has been applied to an element, it cannot be "dequeued" -- you can either abort the format processing, or replace all the removed occurrences with PUTELEM statements.

Remember that you are dealing with absolute occurrence numbers when a looping label group contains a REMELEM statement. Generally, what's true for PUTELEM statements here is true for REMELEM statements too: In a looping label group, the occurrence number in the REMELEM statement should be different each time; otherwise, the same removed element value will be "removed again", i.e., there will be no effect after the first time.

But be careful when creating loops with label groups containing REMELEM statements, even if you do increment the occurrence number each time. As suggested above, a REMELEM statement applied to an occurrence that does not exist will not cause an error (i.e., break the loop). Be sure that you have some other way of controlling the loop, such as a numeric value on the LOOP statement. [See B.4.8.4.] For example,

Remember, calling the occurrence numbers in merge frames "absolute" does not mean that they will be stored that way. If a merge frame processes the "1000th occurrence" of an element, placing it in the record, and the record only contains 5 occurrences, the value will actually be stored as the sixth -- non-existent occurrences (though not null ones), such as the seventh through 999th in this example, will be discarded when format processing ends. The point is that during merge format processing, all PUTELEM references to the 1000th occurrence will refer to that specific occurrence.

Occurrence Number Differences Between Merge Formats and the Standard Format

The SPIRES standard format is frequently used for merge processing. The procedure for using it is taught in the SPIRES manual, "Searching and Updating", section D.2.5.4. With respect to occurrence numbers, it works quite differently than merge input formats.

First, and most importantly, element occurrence numbers are counted from "0" within a format. For merge processing in the standard format, they are counted from "1". Thus,

   (merge format definition)            (standard-format input)
     VALUE = value;              and      SUBTITLE(4) = value;
     PUTELEM = SUBTITLE(3);

would produce the same result in a merge operation if the SUBTITLE element had four occurrences.

Note too that removing element occurrences is done quite differently with the standard format:

   (merge format definition)            (standard-format input)
     REMELEM = SUBTITLE(3);      and      SUBTITLE(-4);

The negative occurrence number used in the standard format to indicate which element should be removed is not allowed in a merge format. For example, you might be tempted to try these invalid statements:

Within a merge format, elements can be removed only with the REMELEM statement. If you decide within a label group having a PUTELEM statement that you want to remove the element instead, use the SET REMOVEL Uproc. [See C.5.1.4.]

Note that the occurrence numbers are absolute for standard format merges exactly as they are for merge format merges.

C.5.1.4  The SET REMOVEL Uproc

Sometimes after testing an element value to be merged into a record, you want to reject the new value and discard the old value too. Or the user has input some special character to indicate that removal rather than replacement is desired. You could jump from the label group containing the PUTELEM to one containing a REMELEM statement, but an easier method that allows you to stay in the same label group uses the SET REMOVEL Uproc:

This Uproc changes the label group's PUTELEM to a REMELEM. Note that any INPROCs for the value will have already taken place before this flag can be set. Once the flag is set, it cannot be unset for the current label group -- if execution reaches the PUTELEM statement, the specified occurrence (or all occurrences if no particular one is specified) will be removed instead.

Here is a set of label groups from a prompting input format that uses the SET REMOVEL Uproc:

If the user responds to the DATE prompt with NONE, then the SET REMOVEL Uproc is executed in the next label group. When the PUTELEM is executed, it will change to a REMELEM, and the value(s) of DATE in the record being merged will be removed.

The INPROC rules before $DATE were added so that the value NONE would not cause a $DATE processing rule error to set $PROCERR. Note that the last two Uprocs handle processing rule errors when the $PROCERR flag has been set, displaying the $UCODE error message and causing SPIRES to reprompt for another value.

SET REMOVEL can also be invoked automatically by SPIRES, when a null value is retrieved by GETDATA and "GETDATA = 8" is coded. [See C.3.1.]

If you want to avoid both a PUTELEM and a REMELEM, you can code the SET SKIPEL Uproc. [See C.3.3.2.] In input frames of usage FULL when the transaction is either an add or update, the SET REMOVEL Uproc is equivalent to SET SKIPEL, telling SPIRES to skip the PUTELEM statement. However, if the FULL frame is being executed for the ADDMERGE command, and the transaction will actually become a MERGE, the SET REMOVEL Uproc does not change to SET SKIPEL.

Note that there is not a $REMOVEL flag variable.

The SET REMOVEL Uproc and Structures

It is possible to decide within a merge format to remove an occurrence of a structure rather than merge data into it. You might consider trying it this way:

The format definer wants SPIRES to retrieve a data value and based on the first character of that value, decide whether to delete the structural occurrence before opening it. Technically, however, SPIRES opens the structure at the point when it would execute the INPROC statement, which of course occurs before any Uprocs are executed. Hence, as coded in the example, the SET REMOVEL Uproc will not remove the entire structure.

The way to accomplish the desired goal is to postpone the execution of the INPROC, using the DOPROC Uproc:

With the delay of any INPROC execution, the SET REMOVEL Uproc wll apply to the structure as a whole, if it is executed. After the possible execution of the SET REMOVEL Uproc, the DOPROC Uproc officially opens the structural occurrence so that subsequent label groups can process individual elements of the structure. [See C.3.4.3.]

When DOPROC Uproc is executed following SET REMOVEL, it does NOT do INPROC rules because the element's $CVAL isn't needed by PUTELEM.

C.5.1.5  (*) Formats Used with the ADDMERGE Command

When SPIRES or SPIBILD executes a format under the ADDMERGE command, it begins by assuming that an ADD will take place. Hence, it executes the frames of usage FULL, not those of usage MERGE. However, if you need to, you can determine whether the transaction will turn into a MERGE by using the $RECTEST function to see whether the record already exists in the subfile. [See C.3.9.7.] The format could then go off in a different direction depending on the value from $RECTEST.

Though the REMELEM statement has previously been described in the context of frames of usage MERGE [See C.5.1.2.] it is allowed in frames of usage FULL. However, it will be ignored in that context unless the command being processed is ADDMERGE and the transaction actually turns into a MERGE.

Note too, as described in the previous section, that occurrence numbers in a FULL frame may behave differently depending on whether the transaction remains an ADD or changes to a MERGE. Under ADD (and UPDATE) processing, occurrence numbers are relative -- a PUTDATA to the same occurrence number of an element simply adds the value after the occurrence that already exists for that occurrence number. Under MERGE processing, occurrence numbers are absolute -- a PUTDATA to the same occurrence number of an element replaces the existing value with the new value. [See C.5.1.3.]

C.5.1.6  Filtering Merge Input

A merge format lets you update part of a record's data, while leaving the rest of the record's data unaltered. A handy way to take advantage of this capability is to use a merge filter to tell your format explicitly which elements or structures should be modified during the merge processing. (Incidentally, merge-type filters are available for merging in SPIBILD or non-formatted MERGE requests in SPIRES.)

To filter record-merging, use the "merge" option on the SET FILTER command, [For a detailed discussion of the SET FILTER command, see the manual "SPIRES Technical Notes".] as in these examples:

  set filter (merge) for order where status = paid
                   OR
  set filter (merge,display) for order where status = paid

Note that the second command causes filtering of display requests as well as record-merging -- if you want filtering to affect more than one type of activity, specify each of them explicitly.

To see some of the benefits of merge-filters, consider a file with a multiply occurring order structure, within which you want to modify occurrences of the order structure, but only if the order has been paid:

  ORDER
   ITEM
   STATUS         <--Value is either PAID or UNPAID

A compact solution to this task is to set a merge-type filter, then invoke a merge frame to modify -- perhaps to remove or archive -- only occurrences passing the filter. (The filter saves you the considerable trouble of using a tool like Partial FOR to navigate to the appropriate occurrences.)

You might set this filter in the protocol that is driving your application, as in the example below:

  ++MERGE.STACK

  Set Filter (Merge) for Order Where Status = Paid
  For Subfile

  ++MERGE.LABEL

  Using MERGE.ORD Merge, End = 'Jump NEXT.LABEL'
  Jump MERGE.LABEL
  Return

  ++NEXT.LABEL
     :

Above, the MERGE.ORD frame will only modify orders that are paid -- other occurrences of ORDER will be completely unaffected by the frame. More precisely, PUTDATA statements and indirect structure calls which take place within the MERGE frame will only operate against elements that meet the filter criteria.

Instead of issuing the SET FILTER command in the protocol, you might use the SET FILTER Uproc in your format. [See B.4.8.5a.] Note that if you do so, you must include a BUILD RECORD Uproc, so that it is executed before you exit the format. BUILD RECORD forces SPIRES to perform all of the INCLOSE processing, merging, and final record build while still under format control. Be sure to do all of your PUTELEMs prior to doing the BUILD RECORD. [See C.3.4.4.]

There is currently no particular advantage to using one method over another, although there are likely to be enhancements associated with the BUILD RECORD Uproc to provide more format-level control over the closing out of structures and records.

Cautions and Considerations

Remember that filtering only affects certain commands and does not affect others. In particular, a merge-type filter does not filter element occurrences retrieved from a referenced record during a merge request. It filters in the other direction -- during merging back into the record. This means that in the formats for "referenced record processing" discussed in the next section, [See C.5.2.] neither merge-type nor display-type filters affect retrieval of elements by the REFERENCE frame's Uprocs. Your format can "filter" element retrieval using features such as the SET STARTOCC Entry-Uproc, [See B.4.8.5.] or it can test the elements' values in other ways.

C.5.2  Referenced Record Processing

Using referenced record processing, you can retrieve element values from existing records while executing an input frame. The record retrieved comes from the record-type named in the RECORD-NAME statement in the format definition. One frame "references" the record (usually the record that will have values merged into it) and retrieves element values via GETELEM statements, placing them in user variables; another frame uses that data in combination with new data to merge new element values into the record.

Referenced record processing is primarily useful for record updating and merging, since it allows you to read data from the old version of the record in order to make intelligent changes to the data. It may also be used with frames of usage FULL, perhaps in a format used by the UPDATE command. [See C.5.] Referenced record processing is occasionally useful when adding records, though the referenced record must be a record other than the one being added, of course.

The most common ingredients of a format for referenced record processing are:

Here is an example of part of a format for updating a record using MERGE, where element values from the old version must be extracted for computations of the new values.

The MERGE1 frame calls the REF frame to reference the merged record so that data element values ITEMNAME and CURRENTSTOCK can be retrieved. The MERGE1 frame then requests the number of ITEMNAME items sold, subtracting the response from the current value of CURRENTSTOCK and merging it back into the record. Here is a sample session using the format:

The REFERENCE Uproc tells the format processor to bring the referenced record into memory. Only one REFERENCE Uproc is allowed per execution of the format (but see below); the REFERENCE Uproc should not be coded multiple times nor should it be executed, due to looping, more than once per record processed. Otherwise, an S830 error will occur.

When the REFERENCE Uproc is executed, SPIRES creates a copy of the record in memory. That copy is accessed independently of the copy being merged into. In a sense, you have both an output record and an input record. The frames that process the output record are similar to output frames, even though they are called from input frames. SPIRES maintains your processing positions in the input and output records independently.

The external form of the record key must be given in the REFERENCE Uproc; here it is supplied by $KEY. In SPIRES, $KEY is available after any MERGE or UPDATE command, whether or not you are in Global FOR mode. However, in SPIBILD, $KEY is available under Global FOR only. If records are being updated in SPIBILD using the MERGE command not under Global FOR, the key must be read from the input data, using a GETDATA statement in the input format. The key value can then be assigned to a variable for use in a "REFERENCE #variable" Uproc. [See C.9.]

If the referenced record does not exist or some other error occurs when SPIRES tries to reference the record (e.g., an INPROC error when SPIRES processes the key), an S256 error occurs and the format is exited. To avoid this situation, code the SET TESTREF Uproc ahead of the REFERENCE Uproc, as shown in the example above. SPIRES will test for the existence of the referenced record before actually referencing it; if the record exists and no error occurs, then the record is referenced and the $REFERENCED flag variable will be set. The flag may then be tested to determine whether the REFERENCE Uproc succeeded, and appropriate action may be taken. If desired, you can try referencing another record if the previous REFERENCE Uproc failed. Note that the DEFAULT statement, useful in subgoal processing, cannot be used to test for the existence of a referenced record.

Retrieving Element Values from a Referenced Record

Element values can be read from the record after the "REFERENCE key" Uproc, as shown above, using the GETELEM statement. Both $UVAL and $CVAL are available. Usually the element value is then assigned to a variable, and computations involving the new element values being input will be done later in the format.

Only data element retrieval can be done from a frame of usage REFERENCE; data element storage can only take place in a frame of usage MERGE. If desired, the reference frame can call the merge frame, rather than the other way around. In other words, the merge frame would be the indirect frame. However, the usage of the reference frame would have to be "MERGE, REFERENCE" -- otherwise, when the MERGE command was issued, SPIRES would not find the appropriate frame to execute. The method introduced first, where the frame of usage MERGE calls the frame of usage REFERENCE, seems more straightforward.

SPIRES will always retrieve the latest copy of the referenced record, even if you are merging records under a FOR TREE command. Unlike subgoal formats, where the NODEFQ statement can eliminate accessing the deferred queue, referenced record processing cannot bypass the deferred queue.

If elements within a structure must be retrieved from the referenced record, an indirect frame must be used. It must have a USAGE statement matching the frame that calls it and usually has a SUBTREE statement. [See B.8.2.]

The REFERENCE Uproc may be used to reference other records in the record-type -- it is not restricted to the record being merged.

An Example of a Format Using Referenced Record Processing

Below is most of a merge format definition for a subfile of COMMITTEES. Each goal record includes occurrences of a structure called MEMBER, which contains the name and address of a member of the committee. The format allows changes to be made to each occurrence of the structure after displaying its current values. The format retrieves one structural occurrence, updates it, then retrieves and updates the next, etc.

   1.   ID = GQ.JNK.COMMITTEES.UPDATE;
   2.   FILE = GQ.JNK.COMMITTEES;
   3.   RECORD-NAME = REC01;
   4.   VGROUP = GQ.JNK.COMMITTEES;
   5.     VARIABLE = STRUCOCC;
   6.       OCC = 1;
   7.       TYPE = INT;
   8.     VARIABLE = NOMORE;
   9.       OCC = 1;
  10.       TYPE = FLAG;
  11.       VALUE = 0;
  12.     VARIABLE = NAME;
  13.       OCC = 1;
  14.       TYPE = STRING;
  15.     VARIABLE = ADDRESS.LINE;
  16.       OCC = 3;
  17.       TYPE = STRING;

Data frame definition: DRIVER

  18.   FRAME-ID = DRIVER;
  19.     DIRECTION = INPUT;
  20.     LABEL = PRE.LOOP;
  21.       UPROC = EVAL $VGROUPINIT(GQ.JNK.COMMITTEES);
  22.     LABEL = REFERENCE;
  23.       IND-FRAME = REFERENCE;
  24.       UPROC = IF #NOMORE THEN RETURN;
  25.     LABEL = PUT.STRUCTURE;
  26.       IND-STRUCTURE = MEMBER(#STRUCOCC);
  27.       IND-FRAME = PUT.STRUCTURE;
  28.       UPROC = LET STRUCOCC = #STRUCOCC+1;
  29.     LABEL;
  30.       UPROC = JUMP REFERENCE;

Indirect frame definition: REFERENCE

  31.   FRAME-ID = REFERENCE;
  32.     DIRECTION = INPUT;
  33.     USAGE = REFERENCE;
  34.     LABEL;
  35.       UPROC = IF #STRUCOCC = 0 THEN REFERENCE $KEY;
  36.     LABEL = MEMBER;
  37.       IND-STRUCTURE = MEMBER(#STRUCOCC);
  38.       IND-FRAME = REF.MEMBER;
  39.       DEFAULT;
  40.       UPROC = IF $DEFAULT THEN LET NOMORE = $TRUE;

Indirect frame definition: REF.MEMBER

  41.   FRAME-ID = REF.MEMBER;
  42.     DIRECTION = INPUT;
  43.     SUBTREE = MEMBER;
  44.     USAGE = REFERENCE;
  45.     LABEL;
  46.       GETELEM = NAME;
  47.       UPROC = LET NAME = $CVAL;
  48.     LABEL;
  49.       GETELEM = ADDRESS.LINE;
  50.       UPROC = LET ADDRESS.LINE::$LOOPCT = $CVAL;
  51.       LOOP = 2;

Indirect frame definition: PUT.STRUCTURE

  52.   FRAME-ID = PUT.STRUCTURE;
  53.     DIRECTION = INPUT;
  54.     SUBTREE = MEMBER;
  55.     USAGE = MERGE;
  56.       ...
  57.       (These label groups might display the current values
  58.       of the NAME and ADDRESS.LINE variables, and prompt
  59.       for new values, using PUTELEM and/or REMELEM
  60.       statements to change the structural occurrence.)
  61.       ...

Format Declaration section

  62.   FORMAT-NAME = MERGE;
  63.     ALLOCATE = GQ.JNK.COMMITTEES;
  64.     FRAME-NAME = REF.MEMBER;
  65.       FRAME-TYPE = INDIRECT;
  66.     FRAME-NAME = REFERENCE;
  67.       FRAME-TYPE = INDIRECT;
  68.     FRAME-NAME = PUT.STRUCTURE;
  69.       FRAME-TYPE = INDIRECT;
  70.     FRAME-NAME = DRIVER;
  71.       FRAME-TYPE = DATA;

The format definition demonstrates one method of moving back and forth between the reference frames and the merge frames. It also shows the use of indirect reference frames to retrieve elements within structures. Below are notes on specific details of the definition:

C.5.3  (*) The SET PUTSTRUC Uproc

Occasionally, situations arise where you want to move entire occurrences of a structure. For example, suppose a banking record for a patron contains these two structures:

There may be only one occurrence of CURRENT.TRANSACT, but multiple occurrences of PREV.TRANSACT. When a patron's record is updated, a new occurrence of the CURRENT.TRANSACT structure is entered, while the previous occurrence of that structure is moved over to the PREV.TRANSACT structure.

To handle this situation in a merge format, you would first have to reference the record. [See C.5.2.] Next you could retrieve the current values of all the elements within the CURRENT.TRANSACT structure, placing them each in variables, and then place them individually into a new occurrence of the PREV.TRANSACT structure in the merge frame. However, another method, which employs the SET PUTSTRUC Uproc, allows you to avoid the overhead of dealing with the elements within the structure individually -- you handle the entire structural occurrence as a single value.

The syntax of the SET PUTSTRUC Uproc is uncomplicated:

It is placed in a label group before the label group containing the PUTELEM for the structure (see example below).

Below is the code for the label group in the reference frame that would retrieve the CURRENT.TRANSACT structure, followed by the label groups in the merge frame that would place it as an occurrence of the PREV.TRANSACT structure:

In the reference frame, the entire structure is retrieved as if it were a single element occurrence (which it is, of course, to some degree). It is stored in the user variable STRUCTUREVAL, which should be declared type STRING. The $RETYPE function should be used to tell SPIRES to treat the value of $UVAL as if it were a string without converting it to string. (If $UVAL were converted to a string, any trailing blanks, which might be an important part of the value, would be lost.)

Later, in the merge frame, the variable value is assigned to $CVAL in the VALUE statement. Because the SET PUTSTRUC Uproc has been executed, SPIRES realizes that the structure PREV.TRANSACT is not being "opened up" but that the value assigned to the label group (in the VALUE statement) is to be treated as the complete structural occurrence already in its internal form for storage. Hence, no INPROC rules are executed for the elements within the structure; moreover, no INCLOSE rules will be executed for them either, though those for the structure itself will be.

The INPROC rules for the structure would normally be executed too. In the example above, they were overridden by the "INPROC;" statement because the format definer did not want the INPROC rule string in the file definition to be executed for the structure. In this particular case, the file definition's INPROC contained a $STRUC.IN proc (A33 rule) to split the input value into the multiple elements of the structure, which should definitely be avoided when "PUTSTRUC processing" is occurring.

Though it is generally used to move occurrences of a structure to another structure in merge processing, the SET PUTSTRUC Uproc can be used in other circumstances if you can construct the internal (i.e., the final stored) form of a structure. That involves taking header information stored with most elements into account, information such as the number of occurrences of the element, the length of the element, etc.

If the structure contains only Fixed elements (not Required or Optional), no element headers are involved. For example, if a structure consists of three Fixed elements, each with a length of three bytes, then a value nine bytes long could be stored as the value of the structure:

For more information on the type of header information involved with Required and Optional elements, see the SPIRES manual "File Definition", sections B.6.2 and B.6.7.

C.6  The Format Declaration Section

The Format Declaration section of an input format is practically identical to that of an output format. The statements within it were all explained in detail in Part B. Below are summaries of them, with references to the sections in Part B that have more information:

FORMAT-NAME = format.name;

This statement specifies the name of the format, which will be used in the SET FORMAT command. It may be from one to sixteen alphanumeric characters long; the special characters period, underscore and hyphen are allowed and, though not recommended, so is a blank, as long as the format is not a general file format. [See C.10.] The name should be unique among all formats for the record-type. [See B.5.1.]

SORT-SPACE = n;

This optional statement declares how much internal memory should be reserved for sorting when the variables $SORTKEY and $SORTDATA are used. If you do not use them, do not code this statement. The value of "n" is an integer from 1 to 64, representing 1K to 64K bytes of memory. [See B.10.8.]

ALLOCATE = vgroupname;

This multiply occurring, optional statement names a variable group to be allocated when the format is set. [See B.9.3.2.]

FRAME-NAME = frame.name; FRAME-TYPE = frame.type; UPROC = uproc;

The FRAME-NAME statement introduces a group of statements naming a frame that constitutes part of the format. There may be several groups for any given format. The value for the FRAME-NAME statement is the name given in a FRAME-ID statement earlier in the format definition.

The FRAME-TYPE statement has one of the following four values for an input format:

Remember that an indirect frame must be declared here before the frame that calls it. [See B.5.2.]

Uproc statements may be included in a group as well, though the set of Uprocs allowed is limited:

COMMENTS statements are also allowed here. [See B.5.2.]

ACCOUNTS = accounts;

This statement may be used to limit the use of the format to particular accounts or groups of accounts. If the ACCOUNTS statement is not coded, anyone with access to the set of goal records may set the format for it. The standard account form, "gg.uuu", is the basic one allowed. [See B.5.3 for other forms.]

C.7  Compiling and Testing Your Format Definition

The next steps are identical for input and output formats. You add the complete format definition to the public subfile FORMATS, then compile the format definition. For details on this procedure, please refer back to chapter B.6.

Testing the format will probably require more care than it did for output formats because you are altering the data base. If the data base is an established one, rather than one currently "under construction", you might consider creating a small test file to use your test formats with. For example, you might get the original file definition, make some minor changes to the ID and SUBFILE-NAME statements, add it back to the FILEDEF subfile, and compile it anew. Then, by simply changing your format definition's FILE statement, you could add, update, remove and dequeue records without worrying about altering your original file during format tests.

To test the format, select the appropriate subfile and issue the appropriate SET FORMAT command. [See B.7.1.] If your input format reads data from your active file, collect the data there. Then, issue the ADD command. A prompting input format will begin prompting you for values; the other kind will begin reading the data in your active file. Once the format finishes executing, SPIRES will begin the closeout procedures for the record and then place it in the deferred queue. If the record-type is slot, the message "-Added record: n" will be displayed at the terminal, where "n" is the newly assigned slot key; if it is not slot, the returned command prompt will signal the successful completion of the ADD command.

Of course, that is the process if all goes well. Errors may occur along the way: processing rule errors because the data is invalid, format errors because of a violation of the frame dimensions, variable conversion errors, and so forth, are all possibilities.

Debugging input formats is usually harder than debugging output ones. This is because most file owners want to carefully control the values going into the data base, not the ones coming out. Thus, the format designer usually tries to anticipate the problems that arise when a user is inputting records, and anticipating problems can be a very time-, energy- and resource-consuming part of input format designing and testing. However, it is worthwhile to make as many different data entry errors as you can think of that are likely to occur during input, to see if the format handles them correctly. In other words, test the format from the standpoint of the person (or program) that will be using it.

Remember that the FTRACE facility is available to help you debug formats. [See B.7.2.] That, in combination with the * Uproc, is the best debugging tool when you have problems with input formats. To some extent, it is more useful with input formats than output ones because an output format shows you its finished product by displaying it at your terminal or in your active file. On input, however, the record SPIRES constructs from your data with your format is not available for you to view in toto; if SPIRES discards it because of a serious error, you cannot see exactly what value caused that to happen, unless you display the value with the * Uproc.

A Troubleshooter's Checklist for Input Format Debugging

Be sure that you have not disabled the display of system diagnostics, e.g., by issuing the Uproc "SET MESSAGES = 0".

If you get an S822 error as soon as you issue the ADD command when your prompting input format is set:

If elements not given values are appearing in the stored record as null elements rather than not appearing at all:

If the SCAN strings are not appearing in the input value when they should be:

C.8  Retrieving Data in Other Record-Types from Input Formats

In chapter B.12, you were introduced to "phantom structures" and "subgoal processing", methods whereby data in other records in the same or other record-types in the same or other files could be accessed. Remember that phantom structures consist of a pointer in the goal record that refers to a subgoal record in another record-type or another subfile -- the subgoal record acts like an occurrence of a structure (the "phantom structure") in the goal record. The specific relationship between the goal and the subgoal is defined in the goal-record definition. Subgoal processing, on the other hand, allows a similar relationship to be established between the goal record-type handled by the format and the subgoal record, but the relationship is established in the format definition. [See B.12.1, B.12.2.]

Subgoal processing is also allowed in input formats, although not for inputting data into the subgoal records. Rather, it may be used in a way similar to referenced record processing -- to retrieve data from those records to use in input processing of the current goal record. [See C.5.2.]

Phantom structures may be retrieved from a referenced record or from a subgoal record. However, phantom structures are not available in the goal record being updated, unless that record is also referenced or accessed via subgoal. In other words, in input frames you can retrieve data through phantom structures only if the record containing the phantom structure is referenced or accessed via subgoal. If phantom structures are retrieved from a referenced or subgoal record, the rules given earlier regarding their use should be followed.

Below is a comparison of subgoal and referenced record processing used in input formats:

Referenced Record Processing

Subgoal Processing

If you are retrieving values from a record before updating or merging that record, referenced record processing is the most efficient method to use. Subgoal processing is primarily used when other record-types and subfiles are involved.

The next section of this chapter discusses the details of coding input format definitions using subgoal processing. [See C.8.1.]

C.8.1  Subgoal Processing for Input Frames

As with subgoal processing during output, the subgoal frame is an indirect frame called from a label group containing a pointer (either the record key or the locator) for the subgoal record, e.g.,

That looks identical to the calling label group for an output frame, including the default handling if no record is retrieved. (If the locator is used, remember to code the SET VIALCTR Uproc prior to the label group calling the subgoal frame.) [See B.12.2.] However, the subgoal frame will look somewhat different, as shown by this example for "same-file" subgoal processing:

The DIRECTION statement changes, and a USAGE statement with REFERENCE as the value must be coded, as shown above. You can then issue GETELEM requests (usually associated with DIRECTION = OUTPUT) within the label-groups of the subgoal frame. Usually the values retrieved from GETELEM requests (both $UVAL and $CVAL are available) are then assigned to variables, using LET statements in UPROCs. These variables are used later while processing the record or the parts of a record being input, generally for computation or verification of the incoming element values.

Elements within structures may also be retrieved, usually through an indirect frame containing appropriate subtree information. [See B.8.2.] You may retrieve some elements within structures without the indirect frame -- the same rules given for retrieving elements within structures in output frames apply here. [See B.8.] If you are retrieving elements from phantom structures, be sure to follow the rules presented earlier. [See B.12.1.]

Subfile subgoal processing, allowing you access to other subfiles, is allowed in input frames; it requires that a load format be written, just as in output. [See B.12.3.] The desired frame in the load format should be coded similarly to the one above, specifying a direction of INPUT and a usage of REFERENCE.

Subgoal retrieval during input processing can be limited to TREE access, using the NODEFQ statement. If you use subgoal processing to retrieve element values from a record during an UPDATE or MERGE request against that record, you should not code the NODEFQ statement. Otherwise, if you update the same record twice in one day, the second time you update it you will be updating on the basis of the TREE copy rather than the latest copy.

C.9  Multiple-Record Input Formats for SPIRES and SPIBILD

Both SPIRES and SPIBILD, the program that handles SPIRES file maintenance, can be used for multiple record input. Using the SPIRES "standard batch format" (consisting of records in the standard format with a "command" preceding each, such as "ADD;" or "UPDATE KL366;", and an extra semicolon at the end of each), you can add, update, merge or remove multiple records at one time. More information about this facility using the standard format appears in other manuals. [See "SPIRES Searching and Updating", section D.8.1, for more information about the standard batch input format, and other sections of chapter D.8 for information about the INPUT BATCH command of SPIRES. See "SPIRES File Definition", section B.10.13, for more information about the INPUT BATCH command in SPIBILD, the command that handles these requests.]

Formatted input for multiple records may also be "batched into" a subfile in SPIRES and SPIBILD. SPIRES or SPIBILD acts as the driver, calling the format repeatedly until the end of the input file is reached or a STOPRUN Uproc is executed. [See B.4.8.8.]

In both SPIRES and SPIBILD, with custom-designed formats, a group of records may be:

Formats for multiple-record input may be set in online and batch SPIBILD, in online SPIRES and the batch program BATSPI, and in FASTBILD.

This chapter begins with a general discussion of multiple record input formats, covering their similarities to and differences from single-record input formats. [See C.9.1.] The section after that discusses the INPUT ADD, INPUT ADDUPDATE and INPUT ADDMERGE commands with input formats, which are used when the data in your active file will be used to create added records. [See C.9.2.]

Merge requests are generally handled in either of two ways in SPIBILD:

Of the two, the situation outside of Global FOR will be discussed first. That is also the only way the INPUT MERGE command works; it does not work under Global FOR. [See C.9.3.] The combination of Global FOR, SPIBILD merges and input formats will be covered in the following section. [See C.9.4.]

C.9.1  General Rules about Batch Input Formats

Multiple-record input formats for use in SPIRES are similar to those for single-record input. Most of the features discussed in this manual for single-record input are available, with the notable exception of the BACKOUT Uproc, which does not work for multiple-record input.

Input formats for use in SPIBILD are also very similar to those for single record input in SPIRES. Most of the features discussed in this manual are available here with a few exceptions:

However, many of the features already discussed are worth mentioning again in the context of SPIBILD:

The SET FORMAT command, as mentioned above, is used to set the desired format in SPIBILD as well as in SPIRES.

The set format is cleared after the INPUT command finishes execution, so it must be reset before being used again.

If desired, you can add a parameter to the SET FORMAT command:

If you use the command with a format name and a parameter, e.g.:

SPIBILD sets the format but does not execute the startup frame of the format, the usual place where the parameter (in $PARM) is handled in formats for SPIRES.

However, SPIBILD does execute the startup frame right away if you issue a subsequent SET FORMAT * command:

In other words, if you need to process a parameter, you have a couple of choices:

The latter choice is certainly preferable when you need to pass multiple parameters on several SET FORMAT * commands.

Frame Dimensions for Multiple-Record Input Formats

Multiple-record input formats, whether for SPIRES or SPIBILD, for adds or merges, may have line-by-line frames or frames with fixed dimensions. Fixed-dimension frames are easier to use but they require the input file to have a fixed number of lines for each record. If the data elements vary in length and occurrence, that might mean having to leave many blank rows in the input file, which could be an inconvenience during data entry.

When fixed frame dimensions are specified, that number of rows will be read from the input file for each record. The format executes, processing the data with GETDATA and PUTELEM statements, and then the record is added or merged into the subfile. The format then begins executing again, reading the next "n" lines of data, as specified by the frame dimensions. If desired, the SET FLUSH Uproc may be included in the format declaration section to tell SPIRES or SPIBILD to switch to line-by-line processing (see below) when a GETDATA statement tries to retrieve data beyond the frame dimensions. [See C.2.3.]

Line-by-line processing is trickier to handle. A new line of data is read each time a GETDATA tries to retrieve data from "the next row". Trying to tell SPIRES or SPIBILD where one record ends and another begins can be somewhat complicated. The best methods for determining the end of a record are described below:

Ending each record with a required element

An easy solution is to have a required element, probably singly occurring and of a set length (so that it does not wrap around into some unknown number of additional rows) at the end of each input record. Its processing would signal the end of the record.

Separating each record with a given string

Each record could end with a particular character or character string on a row of its own: "END OF RECORD" perhaps, though something shorter to type might be preferable. When the format detects that value, the format processing could end:

If the value END is picked up as a value for the CHILDREN element, the RETURN Uproc is executed.

Using the HOLD Uproc to save the first row of the next record

One way to detect the end of the record is by determining that another one is starting. In input frames, the HOLD Uproc can be coded in the Uprocs of the format declaration section to tell SPIRES or SPIBILD to hold the current frame across multiple records. When used in line-by-line frames (which is the only way it should be used), it keeps the current row in the buffer until another row is referenced by a GETDATA statement, even if that GETDATA statement does not occur until the format is called again by SPIRES/SPIBILD.

Suppose that each new record begins with an asterisk in column 1 and ends with occurrences of the CHILDREN element. Code the HOLD Uproc in the format declaration section like this:

and the last label group in the frame like this:

When the last label group retrieves the first row of the next record (the one beginning with the asterisk), the RETURN Uproc would be executed. Because of the HOLD Uproc, the row would remain in the buffer as the format began re-executing, allowing values to be retrieved from row "*". That is, suppose the following is the beginning of the DATA.INPUT frame:

When the frame begins re-executing, the row with the asterisk is still in the buffer, so the GETDATA here will retrieve the name from that row, beginning the new record. You may wonder about using "*" for the starting row in the first label group -- no data has been requested yet, so why would there be anything in the buffer for the first record processed? This is not a problem because SPIRES/SPIBILD will read the first row of the input into the buffer to create a current row if there is not one already.

Note that exception file processing is not currently supported when the HOLD Uproc is in effect.

The maximum value of "ncols" in the FRAME-DIM statement is about 4000 in batch SPIBILD and BATSPI, matching the maximum value of LRECL. In SPIBILD, the default value of "ncols" is 130, whether online or batch SPIBILD is used. In SPIRES, the default value of "ncols" is $LENGTH, whether SPIRES or BATSPI is used.

C.9.2  Multiple-Record Input Formats for Adding Records

Multiple-record input formats for adding records are very similar to their single-record counterparts. For each new record, SPIRES or SPIBILD reads into the buffer a given number of lines from your active file, according to the specification of the FRAME-DIM statement. When the format finishes executing, the record is added to the subfile (assuming no serious errors were detected) and SPIRES/SPIBILD begins processing the next one. If an error does occur to prevent the record from being added, SPIRES/SPIBILD goes on to the next record in the input file (unless the format handles the error in some other fashion).

The INPUT ADD Command in SPIRES

To use a compiled format for batch input in online SPIRES, put the data into your active file and follow the procedure below:

You may also use your format in the batch program BATSPI, though the INPUT BATCH command is slightly different:

where "ddname" is the ddname given in the DD statement of the JCL. You do not have to use the FROM option; in BATSPI you can issue WYLBUR commands such as USE to bring a data set into your batch job's active file, and then just issue the INPUT BATCH command without FROM. Of course, you would need to use the FROM option if the input data set is larger in width (LRECL) or length (number of lines) than WYLBUR can handle. [See the manual "SPIRES Searching and Updating", section E.3.3, for more information on BATSPI.]

Either INPUT ADDUPDATE or INPUT ADDMERGE can be used instead of INPUT ADD. Their bonus is that if SPIBILD determines that a record with a key matching the input data already exists in the file, it will treat the input as an update or merge request respectively, rather than reject the data as they would for an ADD request. [See C.9.2.1 for additional information about formats used by INPUT ADDUPDATE and INPUT ADDMERGE commands.]

The INPUT ADD Command in SPIBILD

To use a compiled format for batch input in online SPIBILD, put the data in your active file and then follow the procedure below:

You may also use your format in batch SPIBILD, though the ESTABLISH and INPUT ADD commands are somewhat different:

where "ddname" is the ddname given in the DD statement of the JCL -- that is where the input data should be. The "filename" option may be used if the subfile name is not unique, i.e., if the account running the job can select several subfiles having that name. The "filename" is as much of the name of the appropriate file containing the desired subfile as is necessary to uniquely identify the subfile.

The ACCT (or ACCOUNT) option may be used to specify an account whose subfile privileges are to be considered when the processing occurs. For example, if "priv-tags" or a subcode is set when that account selects the subfile (or if that account can only add or update its own records because of $TEST.ACCT or A53 processing rules), those conditions will be set when the ESTABLISH command is executed. If the ACCT option is omitted, the subfile privileges of the account under which the job is run will be used. [See the manual "SPIRES File Definition", section C.6.3, for more information about batch SPIBILD.]

C.9.3  Multiple-Record Merge Formats Outside of Global FOR

Many applications require a merge format for a situation like this: your input file has a row of data for each course being taught. The row contains the course number and the number of students enrolled:

The first item on the row is the key of the record; the second is the data you want merged into the record, either to replace or enter a value for the NUM.STUDENTS element.

To tell SPIRES or SPIBILD in the format which record should be merged, you need to code a label group containing a PUTELEM for the key:

SPIRES/SPIBILD reads the key from the input data; the PUTELEM on the key element "positions" the program to merge the proper record. After format processing, if there is no record with that key, an error will occur (error S256) and processing of that record will stop. (The entire format definition appears below.)

The label group containing the PUTELEM for the key may appear anywhere in the data frames -- it does not have to be the first element processed by the format, since SPIRES/SPIBILD will not be looking for the record for merging until after the format processing is completed. However, if the format will be referencing the record via $KEY, the label group containing the PUTELEM for the key must occur before the "REFERENCE $KEY" Uproc; otherwise, $KEY will not be set before the REFERENCE is executed.

If the input data will be used to compute new values from the old values in the record being merged, you will need to use referenced record processing in combination with this procedure. [See C.5.2.]

The INPUT MERGE Command in SPIRES

Below are the commands to issue to use a merge format for multiple-record input in SPIRES. Be sure to have the input data in your active file before issuing the INPUT MERGE command.

You may also use the command in the batch program BATSPI, though its form is slightly different:

where "ddname" matches the "ddname" in the DD statement of the JCL, which names the data set that has the input data. You do not have to use the FROM option, however; you may include a WYLBUR command such as USE in the command stream, ahead of the INPUT MERGE command, to bring the data into the batch job's active file. Of course, you would need to use the FROM option if the input data set is larger in width (LRECL) or length (number of lines) than WYLBUR can handle. [See the manual "SPIRES Searching and Updating", section E.3.3, for more information on BATSPI.]

The INPUT MERGE Command in SPIBILD

Here are the commands you should issue to use a merge format in online SPIBILD:

The input data must be in your active file.

You may use your format in batch SPIBILD if you want, though different forms of the ESTABLISH and MERGE commands are used:

where "ddname" is the ddname specified in the DD card for the input data set (there is no active file in batch SPIBILD). The other options were discussed in the previous section. [See C.9.2.] [More information about batch SPIBILD can be found in section C.6.3 in "SPIRES File Definition".]

A Complete Format Definition for Multiple-Record Merging

Here is the entire format definition for merging class enrollments, introduced in the example above:

This simple format retrieves the record key from the data (COURSE.NUMBER), and, by executing the PUTELEM, tells SPIRES/SPIBILD that this is the record to be merged. The next label group processes the data being merged into the record. If referenced record processing were needed, say, to retrieve the previous value of ENROLLMENT from the record being merged, the indirect frame call would appear between the two label groups. That is, the reference frame containing the "REFERENCE $KEY" Uproc would be called after the COURSE.NUMBER label group has established the value of $KEY.

C.9.4  SPIBILD Merge Formats Under Global FOR Processing

Sometimes you have a couple of pieces of data that you want merged into many records. Or, alternatively, you may want many records to have data created by a format to be merged into them (for example, to have the value of the DELIVERY.DATE element in all records to be incremented by five days, which would not require any data at all from the active file).

In such situations, some Global FOR commands are available in SPIBILD to determine the records that should be processed by the format. (You cannot use Global FOR in SPIRES for multiple-record merging.) The format, if properly coded, then executes for each record identified by the Global FOR processing. Typically, these might be all the records in the subfile, or all the records with a particular element value, or all the records with a particular number of occurrences of an element.

Both the format definition and the procedure in SPIBILD are different from the SPIBILD merge format processing discussed in the preceding sections. The remainder of this section will discuss those two subjects.

The Format Definition

The way in which the data to be merged is provided to the format will naturally affect the way the definition is written. The easiest methods avoid putting the merge data into the active file or other data sets -- instead, the merge data is:

If you use these methods, you should omit the FRAME-DIM statement from your frame definitions, since the data is not being read from an external data set. Within the label groups, VALUE statements or SET CVAL Uprocs should be used in conjunction with the PUTELEM statements to put the merge data into the record.

If you must read the merge data from the active file (or from an external data set if using batch SPIBILD), the format controls how the data is read. In most cases, the data is read in only once and then applied to each record. This may be done in either of two ways:

1) Code an initial frame (allowed in SPIBILD input formats). This frame will execute before the first record is processed. The frame should be defined to read the data from the data set, placing the values in variables:

The variables are then used in the data frame, which is executed each time a new record is retrieved by the Global FOR processing.

2) Code the HOLD Uproc to use the same buffer for multiple records. In SPIBILD input formats, the HOLD Uproc can be added to the format declaration section to tell SPIRES to hold the current buffer across multiple records. [See C.9.1.] Assuming the HOLD Uproc is coded appropriately, consider this data frame:

Since the HOLD Uproc prevents SPIBILD from reading lines of data whenever the format re-executes, the data is read and put into variables only once.

The Commands Issued in SPIBILD

To use the format in SPIBILD, you must follow the procedure outlined below. Each command is listed, most with some detail given. Note that the commands issued must be issued in the order in which they are listed below; optional ones, indicated by "(*)", may be omitted.

To begin with, you must be in SPIBILD (i.e., issue the CALL SPIBILD command). Then:

SET FORMAT format.name

As in other SPIBILD merge format situations, you identify the format before identifying the subfile. [See C.9.1.]

ESTABLISH subfile.name

This command causes the file containing the named subfile to be processed by SPIBILD and then left in a state ready for Global FOR processing in SPIBILD.

FOR SUBFILE [WHERE criteria-clause]

The FOR SUBFILE command (or alternately, FOR TREE, which has the identical effect in this context) identifies which records are to be processed. If no WHERE clause is appended, all the records in the subfile will be processed. The usual form of the WHERE clause is:

Details of other forms are provided in the SPIRES manual "Global FOR".

(*) SET SCAN scan.options

For example, SET SCAN LIMIT 10. Consult the manual "Global FOR".

(*) SKIP [n|NEXT|FIRST]

This command tells SPIRES to skip that number of records (or the first record in the case of NEXT or FIRST) meeting the criteria clause before beginning format processing. If no option is specified, NEXT will be used.

MERGE [ALL|NEXT|n|REST] [FROM {ACTIVE|ddname}]

This command causes the processing to begin. SPIBILD will now check for the existence of the format named in the SET FORMAT command at this point; if it does not exist, an error message will be displayed and execution will stop. If it does exist, SPIBILD will begin retrieving records and processing them according to the format definition. When all the records to be processed (according to the MERGE command) have been processed, format execution ends. Note that indexes will be rebuilt during the process as the pass stack fills up. The default, if no option is specified, is REST.

If the data to be merged is not provided by the format but is read from the active file (for online SPIBILD) or a data set (for batch SPIBILD), the FROM clause should be used. If the data is in your active file, add the FROM ACTIVE option. If you are using batch SPIBILD and you have a data set to be used by the format, you must save it in a data set that is named in the DD statement of the JCL. The "FROM ddname" option is then added to the MERGE command in the command stream, as in the MERGE command outside of Global FOR in SPIBILD. [See C.9.3.]

C.10  General File Formats for Input

Like their counterparts for output, general file formats for input come in two varieties: formats for record-types in several files, all of which have the same design; and formats that may be used for any record-type of any file, such as the system format $PROMPT. The second variety is not commonly designed by SPIRES users but instead by SPIRES system programmers, so it will not be discussed in detail here. [See B.14.]

The GEN-FILE statement is used to declare the formats in a format definition to be general file formats. It is coded near the end of the format definition, before any proc definitions and the PROCDEF statement. [See B.14.]

For the first type of general file format, the FILE and RECORD-NAME statements contain the names of one file and one of its record-types that can use the formats. SPIRES will use that record-type as a template when compiling the format definition, verifying the existence of the named file, record-type, elements, etc.

To set a general file format, use this special form of the SET FORMAT command:

where "format.name" is the name of the format as it appears in the FORMAT-NAME statement; it may not contain internal blanks. The value of "parm", which need not be set apart from the "format.name" by commas, apostrophes or quotation marks, as it must for other formats, is passed to the system variable $PARM. [See E.2.3.9.] You only need to use the fully qualified form (using the format definer's account number) if you are not the format definer.

C.11  Input Formats for Partial Record Processing

Under partial record processing, initiated by referencing a record (the REFERENCE command) and issuing a "FOR element" command, you may add, replace or remove element or structure occurrences in a single record through format control. In general, partial record processing during input is used for updating records rather than adding them. (The exception to this generalization is when partial FOR processing under the "FOR *" command is used to construct records across multiple screens in full-screen applications -- see the SPIRES manual "Device Services" for details.)

Input frames used to process structures under partial FOR processing must be assigned a frame-type of STRUCTURE in the frame declaration of the format declaration section:

The STRUCTURE1 frame is declared an indirect frame to be called from the RECORD.INPUT frame and is also declared a structure frame that may be used independently under partial FOR processing. (Because a frame of type STRUCTURE was desired, the structure processing was handled by an indirect frame. Remember though that for input frames in general, structures do not have to be processed in indirect frames.)

The SUBTREE statement must be coded in the structure frame definition so that SPIRES knows which structure the frame is for. [See B.8.2.] The structure frames must also have the appropriate DIRECTION and USAGE statements, depending on how they will be used and on what commands should cause their execution.

For more information about formats with partial record processing, please consult the chapter in Part B on output formats and partial FOR. [See B.15.] For more information about partial record processing in general, see section 12 of the manual "SPIRES Technical Notes".

C.12  Input Formats for Full-Screen Applications

Full-screen applications may require input formats as well as output formats. [See B.13.] An input format might construct a screen that looks like a blank form. The user would enter data into the various fields of that blank form, moving the cursor around the screen, and then submit the entire screen back to SPIRES at one time. The contents of the screen would be placed in the format buffer, and the data retrieved as other input formats would retrieve it. If the format detected errors, SPIRES could place an error message right next to the error on the user's input screen, telling the user directly what element value needed correction.

In terms of updating, a record could be displayed on the terminal screen. The user could then change the values on the screen, skipping the cursor around as desired, and then resubmitting the screen to SPIRES. The format can detect whether a given value has been changed or not and act accordingly. [See C.3.1 for the "7" and "5,7" options on the GETDATA statement..]

As noted in the chapter on full-screen output formats, formats for a full-screen application are usually designed and coded at the same time as protocols, global vgroups and even file definitions. The manual "SPIRES Device Services", in particular, the chapter called "Full-Screen Programming", is the primary source for information on designing and coding full-screen formats, in conjunction with the other pieces of the application. If you are interested in such applications, you should read that manual. If you want to write input formats for full-screen processing, you should be familiar with the first seven sections of Part C of this manual. An understanding of XEQ frames is also useful. [See D.2.]

C.13  Special Format Techniques for Updating Records

An output format used to display records can actually cause the records themselves to be updated, on a limited basis. For example, an application may want to keep a DATE.EXAMINED and WHO.EXAMINED element that contains the date and account of the last user to see each record in the data base, for audit and/or security reasons. Each time a record is displayed, SPIRES should update those elements.

The feature that allows record updates from output formats is called "WITH UPDATE", because the WITH UPDATE prefix must appear on the TYPE or DISPLAY command to cause the update to happen. That implies that applications using this feature generally control the user's environment, particularly in record display -- users of the application's subfiles who issue regular SPIRES commands are unlikely to add the WITH UPDATE prefix to their TYPE and DISPLAY commands for the convenience of the file owner. Thus, record displays are usually handled by protocols that issue the WITH UPDATE TYPE or WITH UPDATE DISPLAY commands, when this feature is desired.

Besides allowing you to update records you are displaying, WITH UPDATE can also let you update other records in other record-types, in combination with subgoal processing. In other words, while you are displaying or even updating records in the goal record-type, you can be simultaneously updating records in other record-types.

Before learning the coding techniques, you should be aware of other restrictions concerning the WITH UPDATE feature:

Some extra restrictions apply when you use subgoal processing to update records during output. (That allows you to update records other than those being displayed.) Those restrictions are discussed later. [See C.13.2.] Subgoal processing also allows you to update multiple records during input as well. [See C.13.3.]

This chapter continues with instructions on coding input formats to update the same record being displayed. [See C.13.1.]

C.13.1  Updating the Same Record Being Displayed

The simplest use of the WITH UPDATE facility is to update the same record you are displaying. Using subgoal processing, you can update records other than the one you are displaying, but that method builds on the techniques discussed in this section. [See C.13.2.]

There are basically three ingredients needed to take advantage of the WITH UPDATE facility:

Code for the Output Frame

To put data into the record, the output frame must have a label group with a PUTELEM statement. This "input label group" may appear among the other output label groups more common to an output frame. For example, here is part of an output frame that displays the DATE.EXAMINED element and then updates it:

The second label group does the input, replacing the current value of the DATE.EXAMINED element with the current value of the system variable $DATE. This example shows that you cannot retrieve and replace element values in one label group -- GETELEM and PUTELEM statements may not appear in the same label group.

When the TYPE or DISPLAY command issued does not have the WITH UPDATE prefix, the PUTELEM in the second label group has no effect. Specifically, SPIRES sets the $SKIPEL system variable for that label group, meaning that the PUTELEM is to be ignored, but the rest of the label group should be executed. [See E.2.1.21.]

You do not have to use the VALUE statement to establish the input value for the element. Instead, you may use the SET CVAL Uproc, or the combination of SET UVAL and DOPROC Uprocs. [See C.3.4.2, C.3.4.3.] Be aware that $UVAL and $CVAL change meaning, depending on whether the label group is doing input or output: for input, $UVAL represents the external form, and $CVAL represents the internal form; but for output, the reverse is true.

The input label group may also contain an INPROC statement, to override the processing rules specified in the file definition. Of course, the file definition may use the INPROC-REQ statement to forbid such overriding. [See C.3.4.1.] But regardless of what INPROCs are coded, no INCLOSE rules will be executed for the changed elements. Moreover, INCLOSE rules for other elements (e.g., a DATE-UPDATED element) will not be executed either.

A PUTELEM statement changes the element's value right away; subsequent GETELEM statements for that element will retrieve the new value. That is different from the way merged record processing works with referenced records. [See C.5.2.]

The length of the new value in its internal form (i.e., after INPROC processing) must match that of the value being replaced. Also, you cannot add new occurrences or remove old ones -- you can only replace existing ones. If SPIRES detects that any of these rules are being broken, format processing will stop for that record.

You must also respect structural boundaries. If you want to update elements within an occurrence of a structure, you must update the elements within an indirect frame that names the structure in the IND-STRUCTURE statement. You cannot open occurrences of a structure with the PUTELEM statement as you would do in input frames. [See C.3.9.2.] Note too that if the file definition has specified the structure as INPROC-REQ, then no elements within the structure can be updated with the WITH UPDATE technique. On the other hand, you could perhaps replace the entire occurrence of the structure, using the SET PUTSTRUC Uproc or possibly the processing rules for handling structures (e.g., $STRUC.IN), treating the structure as a single element value. [See C.3.9.2, C.5.3.]

Remember that you cannot use this technique to change the key of a record, nor can you change the value of elements in a phantom structure (use subgoal processing instead).

Secure-Switch 14

You must code secure-switch 14 in the subfile section of the file definition, setting it for those accounts whom you wish to allow to use WITH UPDATE. For example,

An error message (S881) will appear if you use the WITH UPDATE prefix when secure-switch 14 is not set. See the manual "SPIRES File Definition" for more information on secure-switches.

The WITH UPDATE [WIT UPD] Prefix

The WITH UPDATE prefix may be added to the TYPE command or the DISPLAY command or input commands, in some circumstances. [See C.13.3.] Though it may be added to other commands, including TRANSFER, it has no effect on them. In fact, it has no effect on TYPE and DISPLAY unless the format currently set has output frames containing PUTELEM statements.

The WITH UPDATE prefix may be used in conjunction with other command prefixes, such as IN ACTIVE or "THROUGH path". Commas may be used to separate prefixes for clarity, if desired, e.g.:

WITH UPDATE and Filters

An unusual aspect of the WITH UPDATE feature is that you can control the element occurrences you update by using filters. Filters do not generally affect input, but since the input is occurring through an output frame, the filters apply. So, for example, if you filter an element such that only the last occurrence can be displayed, e.g.,

then, if the "with update" format changes the value of an occurrence of PARTNER, it will be the last occurrence that is changed. (The format would be written such that if the filter were not set, the first occurrence of the element would be changed.)

C.13.2  Updating Other Records from an Output Format

Besides updating the record being displayed, the WITH UPDATE feature can update other records too, in combination with subgoal processing. The same subgoal procedures used to retrieve data from records in other record-types can be used to update those records. You can, for instance, use same-file subgoal to update records in other record-types of the file containing the current goal record-type. Alternatively, you can use subfile subgoal to update records in other subfiles. [See B.12.2, B.12.3.] Subgoal processing and WITH UPDATE can be used together when the goal record is being displayed or is itself being updated. Though it uses most of the same coding techniques, the latter situation is discussed in the next section. [See C.13.3.]

Besides combining the WITH UPDATE techniques from the previous section with the subgoal processing discussed earlier, this procedure uses a new statement in the format definition, WITH-UPDATE. It also has some restrictions in addition to those discussed earlier. [See C.13.]

The frame that handles the input into the subgoal record looks like the one shown in the previous section:

In this example, we update the TOTAL element in the subgoal record (record-type REC02), retrieving the old value in the first label group, changing it, and then replacing it in the second. Only the SUBTREE statement is new in this frame.

However, the label group that calls that indirect frame will be somewhat different than the standard one for subgoal:

The WITH-UPDATE statement tells SPIRES that the subgoal frame being called may be used for record updates. It does not say that it necessarily will be -- the WITH UPDATE prefix on the TYPE or DISPLAY command is still necessary to cause the updating of the record. Thus, both the WITH UPDATE prefix and the WITH-UPDATE statement in the label group that calls the indirect frame for subgoal processing are needed.

You should be aware of the following restrictions to this procedure:

Remember that these restrictions are in addition to those mentioned at the beginning of this chapter for the WITH UPDATE feature.

C.13.3  Updating Other Records During Record Input

The previous sections have described the procedure for using WITH UPDATE to change records from an output format. By adding another statement, IN-AREA, you can extend the capability of updating other records to input formats. The IN-AREA statement, in association with the WITH-UPDATE statement introduced in the previous section, allows an input frame to call an output frame for subgoal purposes, and that output frame may change data in the subgoal record.

In general, you follow the same procedure discussed in the previous section. Either subfile or same-file subgoal techniques must be used. The label group that calls the indirect frame for subgoal looks different again, however:

The IN-AREA statement allows the input frame to call an output frame containing GETELEM statements. The value NULL tells SPIRES to direct any output from the subgoal frame to the NULL area, in essence discarding it. (If desired, you could direct it to some other area, if your intent was to create output during input. [See B.16.1.] Here, however, you are simply using the IN-AREA statement to allow you to call an indirect output frame that contains GETELEMs.)

The indirect frame that is called begins like this:

For same-file subgoal, you use the SUBTREE statement, while for subfile subgoal, you use the SUBFILE and then the LOAD-FORMAT statements. [See B.12.2, B.12.3.]

To use the feature, you add the WITH UPDATE prefix to the record input command (ADD, UPDATE, ADDUPDATE or MERGE). If the WITH UPDATE prefix is omitted, the PUTELEM statements are not executed. Remember that Secure-Switch 14 must also be set.

The restrictions that apply to updating records during record input are the same as those described for updating records during record output. [See C.13, C.13.2.] One additional restriction is that this feature is not currently available in SPIBILD.

It is important to realize that the indirect frame doing the subgoal must be declared as an output frame, not an input frame (i.e., using the DIRECTION statement). Only output frames are allowed to change values in other records using the WITH UPDATE techniques, not input frames. Hence, do not follow the procedure for subgoal access from input frames as described earlier in Part C. [See C.8.1.] That procedure describes coding the indirect frame as an input frame of usage REFERENCE -- but the procedure will not allow you to code PUTELEM statements in that frame.

D  Assorted Topics of Formats

D.1  Combining Frames into Formats: Multiple-Purpose Format Definitions

Although emphasis so far has been on format definitions containing frames for a single format used for a specific input or output task, the definition may define multiple formats, each with multiple purposes (e.g., to handle both input and output) and sharing multiple frames. This chapter will consider how such formats may be put together. Specifically, it will begin with some sections concerning formats that allow both input and output -- that is, formats having input and output frames together. Among the topics to be discussed are "INOUT" frames and the SHOW FRAMES command. The last section will cover multiple formats within the same format definition.

D.1.1  Input and Output Frames in a Single Format

Any given format, as defined in the format declaration section of the format definition, may have up to 100 frames declared for it. Often however, not all frames are executed for any given command. SPIRES determines which frames should be executed for a given command by examining several different criteria:

the FRAME-TYPE statement

The chart below shows what type of commands or situations can cause a frame of the given type to execute:

The same frame may be declared multiple times in the format declaration section, each time with a different frame type.

the DIRECTION statement

The DIRECTION statement, coded in the frame definition, tells SPIRES which type of record processing commands (input or output) can invoke the frame:

the USAGE statement

This statement, coded in the frame definition, tells SPIRES more specifically which commands can invoke the frame:

To summarize, each command can cause execution of only certain frames. The chart below shows the values of the FRAME-TYPE, DIRECTION and USAGE statements for frames that can be executed for a given command. Numbers in parentheses refer to the notes that follow:

If no appropriate frame is found for a particular command, the standard SPIRES format will be used (except for the XEQ FRAME command, which will invoke an error message).

Notes:

A format then may be constructed for multiple purposes, executing different frames for different circumstances. Whether or not this is an advantage to you depends on how you would use it. A possible advantage is that the SET FORMAT command only needs to be issued once, with that one format handling several purposes. The user does not need to switch formats (or go through a separate path by attaching the "THROUGH path" prefix to the command) to "switch purposes".

Another advantage is that a single format allows you to share vgroup variables and their values through different types of processing, if that is important in your application. Setting different formats, even if they share the same vgroups, will reset the variable values each time (unless the ALLOCATE command was issued before the first format was set). Thus, if you want a TRANSFER and UPDATE procedure using formats to use the same variable values, you probably want to handle them in a single format.

The disadvantages are minor. One disadvantage might be that the format becomes too large to compile, forcing you to split it into multiple formats or to put some of the code into load formats. [See B.11.] Another disadvantage might be that the size of the format definition makes it unwieldy to handle when it contains too many frames, but that would be a subjective judgment, of course.

D.1.1.1  The "USING frame" Command Prefix

The charts in the preceding section show how SPIRES determines which frames in the set format to execute for any given command. [See D.1.1.] Adding the "USING frame" prefix (usually called "the USING prefix") to a command, generally in conjunction with the NAMED option in the named frame's USAGE statement (see below), gives you more flexibility in controlling frame execution for a given command.

In essence, the USING prefix names a frame in the set format to be executed by SPIRES as it processes the command, regardless of what frames would have been executed otherwise. For example, USING GRAPH DISPLAY ALL would cause SPIRES to execute the frame GRAPH (and any indirect frames it might call) for each record being displayed. The frame or frames that would have been executed if the command DISPLAY ALL had been issued instead will not be executed, unless of course they are the same frames.

The syntax of the USING prefix is:

where "frame.name" is the name of a frame in the set format, and "command" is one of the following:

The USING prefix may be used with other command prefixes, such as IN ACTIVE or VIA. The order is inconsequential -- for example, the following produce equivalent results:

You cannot use the USING prefix to circumvent frame-type restrictions; for instance, the command USING HEADER ADD will cause an error if the frame called HEADER is indeed declared a header frame-type, since header frames are not used during input processing. Direction and usage restrictions for the frame named in the USING prefix must also be observed.

In full-screen programs, another option is available on the USING prefix when it precedes the DISPLAY command:

The NODATA option tells SPIRES to display an "empty" record, i.e., the appropriate output frame is executed, putting out TITLEs, DEFAULTs and VALUEs, but no record data is processed. This empty record on the terminal screen is then filled in by the user, and the screen is read back later through an input frame. [See the manual "SPIRES Device Services", section 4.1.1.1, for more information on this option.]

The USING Command Prefix and the NAMED Usage Statement

The USING prefix is most often used with frames whose USAGE statements declare them NAMED, e.g.,

A NAMED frame cannot be executed unless it is specifically named, either by the invoking command in the USING prefix or by the IND-FRAME statement or by the USING-FRAME statement if it is in a load format.

A typical use of NAMED frames is when a single format contains two or more frames or groups of frames that could be executed for a given command. For example, suppose a format has a frame to display all the elements of a record and another frame to display only a few. You do not want both to execute for any given record, so you declare one of them, say, the condensed version, a NAMED frame. Then, a command such as USING CONDENSED TYPE would produce the shorter version, but a TYPE command without that prefix would produce the full version. Thus, no setting and resetting of formats would be needed to switch back and forth.

Formats designed to be used solely by protocols often have NAMED frames exclusively -- if a user sets the format and tries to use it without including the USING prefix, the standard format will be used instead.

D.1.1.2  The SHOW FRAMES Command

The SHOW FRAMES command may be issued anytime a format is set to show you information about all the frames in the format that may be directly invoked by commands. Its syntax is simply:

The IN ACTIVE prefix may be added to it to direct the information to your active file.

For example,

Note that the SHOW FRAMES command shows only those frames that can be directly executed or called upon from a command. It does not show startup or indirect frames, or the report-specific frames: initial, ending, header, footer, group-start and group-end frames.

The SHOW FRAMES command is often helpful when you have forgotten the names of frames you need to specify in a "USING frame" prefix or the XEQ FRAME command. [See D.1.1.1, D.2.1.]

Note that the ID of the format also appears in the SHOW FRAMES display. This is useful when you want to modify the format, but can't remember the ID under which it is stored, or perhaps when you need to contact the owner of a format.

D.1.1.3  Formats for TRANSFER/UPDATE Processing

The TRANSFER command places a particular record in the active file, the user makes changes to it using text-editing commands, and then the UPDATE command reads the changed record from the active file, placing the new version in the deferred queue to replace the tree copy of the record when the file is processed.

The TRANSFER/UPDATE record-updating procedure is most commonly done with the standard SPIRES format. However, a format can be created for use in this procedure. It must have at least two data frames, one of direction INPUT and one of OUTPUT. The output data frame must have USAGE = TRANSFER; the input data frame must have USAGE = FULL. [See B.3.4, C.2.4.]

The frames should be designed to work together. For example, the output frame will probably be designed so that the data it places in the active file can be directly read back into the data base by the input frame.

Remember that an UPDATE command completely replaces the old record with the new data, meaning that your format should process all the elements that you want to be put in the data base for each record. In other words, you probably want the output frame to place all the elements that could possibly appear in each record into the active file, and want the input frame to read all of them back in; after all, you usually do not want to lose any data from the old record during the TRANSFER/UPDATE process.

One possible exception might be elements whose values are provided by INCLOSE rules, such as a DATE.UPDATED element. However, an element such as DATE.ADDED would need to be handled by the format -- SPIRES will assign the current date to a "date.added" element if it does not already have one, and it would not already have one unless the format handled the old value explicitly.

D.1.2  Sharing Subroutines Between Input and Output Frames

When a format is created to handle both input and output, it may require subroutines that are to be used during both input and output. Such subroutines may be placed in a separate frame to be called as an indirect frame. As long as the frame does not have label groups with GETELEM, PUTELEM, REMELEM, IND-STRUCTURE, PUTDATA, GETDATA, ELENGTH, ESTART, or OUTPROC statements, the frame may be given any direction and may be called indirectly from any frame, regardless of its direction. In essence, the direction of the indirect frame is ignored, providing the only case in which input frames may call output frames and vice versa. Any frame may contain INPROC statements regardless of its direction.

Here is a sample input frame containing subroutines that may be called from either an input or an output frame:

Before the input frame is invoked as an indirect frame, the integer variable SUBCHOICE is set to the number ("n") of the desired subroutine by the calling frame. The CASE Uproc tells SPIRES to jump to the "nth" label group from the first one. Each subroutine then uses one label group, ending with a RETURN Uproc to cause execution to return to the calling frame.

The first subroutine apparently deals with the user's response to some question asked from the format. (The $YESNO system proc converts YES or OK to "Y" and NO to "N".)

Note that indirect frames are not necessary just because you want to use an INPROC during output processing -- under some conditions, INPROC statements may appear in output frame label groups. [See B.4.5.2.]

INOUT Frames

If you code "DIRECTION = INOUT" in a frame definition, that frame may also be called from either an input frame or an output frame. An INOUT frame may also be useful for holding subroutines that are to be used by both input and output frames in a format or both input and output formats defined in the same format definition. [See D.1.3.] It also has important uses in full-screen applications (see below).

The syntax of the DIRECTION statement for INOUT frames is simply:

In "full-screen" formats, INOUT frames are used in input formats to read the data on the screen and incorporate it into the record (the input part) and then display error messages on the screen if appropriate (the output part). Thus, an INOUT frame is both for input and output in full-screen applications.

INOUT frames are basically input frames with additional capabilities that are mostly useful in full-screen applications. Any input frame can be declared an INOUT frame, and it can be called from other input frames. In addition, if the INOUT frame does not contain statements regarding data movement (GETELEM, PUTELEM, REMELEM, IND-STRUCTURE, GETDATA, PUTDATA, ELENGTH or ESTART) and if it does not have frame dimensions, the INOUT frame becomes "directionless" and can be called from input, output or INOUT frames.

Directionless INOUT frames may contain VALUE, INPROC, OUTPROC, and UPROC statements (though INPROC and OUTPROC statements should not both appear in the same label group). The example frame above could be declared an INOUT frame, for instance. No other changes would be needed.

INOUT frames may be used quite differently (and usually are) in full-screen applications. See the SPIRES manual "Device Services" for details.

D.1.3  Multiple Formats in a Single Format Definition

Many different formats may appear in one format definition, just as many different frames invoked by different commands may appear in one format. The appearance of another FORMAT-NAME statement is all that is needed to signal the start of another format declaration section, naming frames and vgroups for another format.

The reasons why you might want to have multiple formats in a single definition are similar to the reasons why you might want to have multiple frames (invoked by different commands) in one format:

A different reason might be that different formats might use the same frames but allocate different vgroups, perhaps with the same variables but with those having different initial values assigned by their VALUE statements. [See B.9.3.1.]

Here is the format declaration part of a format definition defining several formats that demonstrates yet another reason:

The same DATA1 frame and the same vgroup are used by both formats, but only accounts of group GQ may set the FULL.DISPLAY format that executes frame DATA2.

D.2  XEQ Frames

XEQ frames are a special type of format frame that can only be invoked by an XEQ FRAME command. The XEQ FRAME command is not commonly invoked by users; it shows up much more frequently in application protocols. XEQ frames serve a variety of purposes; typical uses include:

The name of the command, XEQ FRAME, only identifies the tool you will be using; what you do with it is all up to you. This chapter will show how XEQ frames are constructed and invoked; the examples will demonstrate some of their uses. [See D.2.1, D.2.2.] Also, a special type of format called a "global format", consisting entirely of XEQ frames, will be introduced. [See D.2.5.]

D.2.1  The XEQ FRAME Command

The XEQ FRAME command may be issued when a format having XEQ frames is set. It causes the named XEQ frame to be executed. Its syntax is:

where "frame.name" is the name of a frame of frame-type XEQ that is defined in the currently set format. The IN ACTIVE prefix (or "IN area" prefix) may precede the XEQ FRAME command if the named frame will read data from or write data to the active file or some other device services area. [See D.2.2.1, D.2.2.2.] The WITH REPORT prefix may also precede the XEQ FRAME command to request report processing, meaning that the output from the command will include Initial and Ending frame activity, as well as Header and Footer processing. [See B.10.2.]

You will receive an error message if the named frame does not exist, if it is not an XEQ frame, or if no format is set. To see a list of the XEQ frames in the format that can be invoked by the XEQ FRAME command, issue the SHOW FRAMES command. [See D.1.1.2.]

Although the XEQ FRAME command is usually issued from a protocol, here is a sample session in which it is executed:

The HELP frame provided some information on how to use the format, that is, what commands to issue to take advantage of its design. The displayed information was placed in a buffer and then sent to the terminal; if the XEQ FRAME command had been prefixed by IN ACTIVE, the information would have been placed there. Note that had the information been displayed by star (*) Uprocs instead of being buffered, it would be displayed at the terminal even if the IN ACTIVE prefix were issued. [See D.2.2.1.]

Particular XEQ frames will be executed, if they exist, by the SHOW FORMAT INFORMATION and SHOW FORMAT INPUT commands. Those commands provide an easier way for users to get additional information about the format, if you create the appropriate frames. [See D.2.3, D.2.4.]

D.2.2  XEQ Frame Definitions

Coding frame definitions for XEQ frames is easier than coding most other types because they cannot directly reference the data base. An XEQ frame does not process records, so GETELEM, PUTELEM, REMELEM and IND-STRUCTURE are not allowed.

An XEQ frame definition may contain many other types of statements in its label groups, including PUTDATA, GETDATA, VALUE, START, INPROC, OUTPROC, LOOP, UPROC and the others not listed above as forbidden. User and system variables may be tested and changed. Indirect frames and XEQ frames in load formats may be called, as long as they do not contain data base references either (unless subgoal processing is being done -- that is allowed via load formats).

An XEQ frame must be either an input or an output frame. By default, if no DIRECTION statement is coded, it will be an output frame. [See B.3.2.] For many XEQ frames, the direction does not make much difference. However, if data will be read from or written to the active file or some other device services area, the DIRECTION statement is important, as is the FRAME-DIM statement, which is otherwise omitted. The two sections that follow will discuss the additional aspects of those kinds of XEQ frames. [See D.2.2.1, D.2.2.2.]

Generally the USAGE statement is unnecessary too -- an XEQ frame can only be executed when an XEQ FRAME command is issued. However, there is one situation where USAGE = NAMED might need to be coded: if the format may be called as a load format by another XEQ frame and you want to be sure that this XEQ frame is not executed unless it is named in the USING-FRAME statement, make this XEQ frame a NAMED frame. [See D.1.1.1.]

Below is a sample XEQ frame that provides the user with a menu of activities to choose from and then receives the user's response:

The command XEQ FRAME CHOOSE.ACTION is issued from a protocol to invoke this frame. The frame asks the user what action to take, the user responds with a number from the menu, and the frame sets the CMD variable appropriately and returns execution control to the protocol.

There is a fine line between what is handled by a protocol and what is handled by a format in any given application -- this process could be handled by the protocol if desired. The Uprocs in the MENU label group and the last five label groups easily convert to protocol commands, but how would you rewrite the ASK.CHOICE and PROCESS.CHOICE label groups? Remember, the CASE Uproc is not available as a command. Here is one possible solution:

Which is better, to handle the user's "commands" in a protocol or in an XEQ frame? Neither way is recommended unconditionally over the other -- there are good reasons for choosing either approach. On the one hand, a protocol may keep all the code together in one program, but on the other, it may not be able to parse input from the user (like the response to the ASK command above) as easily as a format using processing rules can. Because the format must be compiled, it can be more efficient than (but will be at least as efficient as) a protocol, which may be compiled.

If a format is part of your application anyway and an XEQ frame might simplify the coding, by all means take advantage of this feature.

D.2.2.1  XEQ Frames for Output

If you want an XEQ frame to be able to place data into the active file (or any other given area), it must be declared a frame of direction OUTPUT having frame dimensions. Either fixed-frame or line-by-line processing is allowed. [See B.3.3.]

Values to be placed in the buffer are usually handled by VALUE and PUTDATA statements. [See B.4.3, B.4.4.] Remember that data base references are not explicitly allowed in the frame, although subgoal processing via load formats is permitted. [See B.11, B.12.]

When the frame executes and the buffer is ready to be moved, it is sent by default to the terminal. If the "IN area" prefix is added to the XEQ FRAME command, the data will go to the named area, such as your active file (e.g., IN ACTIVE XEQ FRAME DEBUG). Values processed by the "*" (star) Uproc will be displayed at the terminal, whether the "IN area" prefix is used or not.

A sample output frame is shown in the section on XEQ frames for the SHOW FORMAT INPUT command. [See D.2.4.]

D.2.2.2  XEQ Frames for Input

XEQ frames can be created that will read data from the active file or from some other area for processing by the format. For example, a full-screen application may leave a portion of the terminal screen for user commands. An XEQ frame of direction INPUT or INOUT could read the user's input from the screen, parse it if necessary, and then pass it to the driving protocol. [See the SPIRES manual "Device Services" for a full-screen application example that uses XEQ frames that way.]

Either INPUT or INOUT must be coded for the DIRECTION statement of an XEQ frame if it will be used to read data from an area. [See C.2.2, D.1.2.] A FRAME-DIM statement is also necessary. [See C.2.3.] The data read from the area is placed in the buffer whose dimensions are defined by the FRAME-DIM statement -- it may then be accessed by GETDATA statements for further processing by the XEQ frame. Remember, however, that the XEQ frame may not directly access the data base -- no PUTELEM or REMELEM statements are allowed, for instance. Subgoal processing, which allows you to retrieve data element values from various goal record-types, is allowed, via load formats. [See C.8.]

The data will be read from the area named in the "IN area" prefix of the XEQ FRAME command. [See D.2.1.] If the "IN area" prefix is omitted, the data will be read from the user's active file.

D.2.3  The SHOW FORMAT INFORMATION (SHO FOR INF) Command

The SHOW FORMAT INFORMATION command is equivalent to the command "XEQ FRAME SHOW.INFO". That is, if the currently set format has an XEQ frame named SHOW.INFO, it will be executed when the SHOW FORMAT INFORMATION command is issued. Like SHOW FORMAT INPUT, the SHOW FORMAT INFORMATION facility can be used to provide the user with appropriate information about the set format as determined by the format definer. [See D.2.4.]

The syntax of the command is:

It may be preceded by the "IN area" prefix if you want the output from the command placed in a particular device services area (but see below) -- IN ACTIVE is the most commonly used prefix.

If no XEQ frame named SHOW.INFO can be found in the set format, the error message SHOW NOT POSSIBLE will be returned by SPIRES. If no format is set when the command is issued, the error message NO FORMAT SET will be returned.

All of the system formats, such as $PROMPT and $REPORT, have such frames. Because each of the formats has its own SHOW.INFO frame, the information displayed is different, chosen by the format designer for its particular usefulness. Try setting these formats and issuing the SHOW FORMAT INFORMATION command to see the type of information presented and to get an idea of how the SHOW.INFO frame is typically designed and used.

If you want to create a SHOW.INFO frame to be executed when a user sets your format and issues the SHOW FORMAT INFORMATION command, write an XEQ frame definition of direction OUTPUT, with or without frame dimensions. If you use frame dimensions, you may code PUTDATA statements in the label groups -- the output from the command may then be directed by the user to an area, such as the active file, by preceding the SHOW FORMAT INFORMATION command with the "IN area" prefix. (The system formats $PROMPT and $REPORT are constructed that way.)

On the other hand, if you just want to send information to the terminal, you may omit the frame dimensions and code "*" (star) Uprocs.

D.2.4  The SHOW FORMAT INPUT (SHO FOR INP) Command

This command works like the SHOW FORMAT INFORMATION command. [See D.2.3.] When it is issued, SPIRES examines the currently set format for an XEQ frame called SHOW.INPUT -- if it exists, SPIRES will execute it.

The command's syntax is:

It may be preceded by the "IN area" prefix, if desired.

The SHOW FORMAT INPUT command is generally used to display to the user the various SET FORMAT and SET FORMAT * commands issued to set the current format. From the user's perspective, it is very useful with the $PROMPT and $REPORT formats, e.g.,

The $REPORT format stores the value of $PARM in a variable in an array each time the SET FORMAT command executes the startup frame. The XEQ frame SHOW.INPUT displays the variable values, preceded by the "SET FORMAT $REPORT" or "SET FORMAT *" command. The user can save the command stream by preceding the SHOW FORMAT INPUT command with the IN ACTIVE prefix, and then storing the commands. At some later time, the commands may be retrieved from the stored data set and then executed via the XEQ command, thus relieving the user of retyping the entire set of SET FORMAT commands.

If you want your format to take advantage of this feature, you should remember to include a FRAME-DIM statement in the frame definition. Then VALUE and PUTDATA statements should be used to place the desired data into the buffer created by the FRAME-DIM statement, e.g.,:

Remember, if you were to use the * Uproc to display the information, the user could not send it to the active file.

D.2.5  Global Formats

As you have seen, XEQ frames have many different purposes: to handle command parsing for applications; to read data from and write data into device services areas, such as the active file; to manipulate and test user variables, etc. In complex applications involving multiple files and subfiles, a protocol may need to take advantage of these capabilities regardless of which, if any, subfile is currently selected. This is a problem for regular "file formats", which are cleared whenever the current subfile is cleared.

A special type of format called a "global format" can solve this problem. Global formats contain nothing but XEQ frames (and any indirect frames they may call), which are invoked by the XEQ GLOBAL FRAME command. Global formats are thus not tied to any particular file, since no data base references are allowed (but see below).

A global format is defined like any other type of format, except that:

It is added to the FORMATS subfile and compiled like any other type of format.

Global formats may call load formats, if desired. [See B.11.] Those load formats may even do subgoal processing, which provides a way to associate a global format with a particular file, at least for output. [See B.12.3.]

To use a global format, you generally just add the term GLOBAL to the various commands:

You do not need to specify the fully-qualified name in the SET GLOBAL FORMAT command unless the format is not kept under your account. Unless limited by the ACCOUNTS statement, a global format may be set by any account. [See B.5.3.]

The system variable $GLOFORMAT contains the name of the currently set global format in its fully qualified form, and the $GLOFORMATID variable contains the ID of the format. [See E.2.3.11, E.2.3.12.]

The SHOW FORMATS command will show any global formats defined under your account; the command will not indicate whether any of them are set. Check the value of $GLOFORMAT to see what is set.

Do not confuse global formats with general file formats. General file formats cannot be set unless a subfile is selected; moreover, they usually contain references to the selected data base, in GETELEM or PUTELEM statements, for example. Global formats are completely independent of the currently selected subfile and may be set whether or not a subfile is selected. Clearing the selected subfile will not clear a global format.

D.3  Startup Frames

Startup frames for a given format are executed when the format is set. They are similar in effect to the SELECT-COMMAND statement in file definitions. They are most often used to display information to the user, such as instructions on the format's use, or to initialize variable values.

They are similar to XEQ output frames in that statements referencing the data base (GETELEM, PUTELEM, IND-STRUCTURE, etc.) are not allowed. They may have frame dimensions, however -- values may be placed into a buffer for output with VALUE and PUTDATA statements. Startup frames may only be output frames; they may not have a DIRECTION value of INPUT or they will not be executed.

Multiple startup frames may be coded; indirect frames (possibly calling load formats, which could possibly be used for subgoal processing) are also allowed. No startup frame of a load format will be executed when the format is loaded unless such a frame is named in the USING-FRAME statement. [See B.11.2.] Startup frames are not executed in SPIBILD either. [See C.9.2.]

A format's startup frames are executed each time the format is set or reset -- that is, any form of the SET FORMAT command, including SET FORMAT *, will cause execution of the startup frames. [See B.7.1.] Thus, the frames are often used to handle the $PARM variable, set by the SET FORMAT command. [See E.2.3.9.]

A frame is declared a startup frame in the FRAME-TYPE statement of the format declaration section:

If you do not intend to place values in a buffer or call an indirect frame, you do not need to code a frame definition for the startup frame; instead, you may code all your startup statements in Uprocs following the FRAME-TYPE statement. In that case, you do not name a frame in the FRAME-NAME statement:

SPIRES will execute these Uprocs when the SET FORMAT command is issued even though no frame is actually called. [See B.5.2.]

On the other hand, if you do need to write a startup frame definition, it might look like this:

The startup frame is commonly used to set report mode when a report format is coded; it prevents the user from having to remember to set it manually. Only in a startup frame can the SET REPORT Uproc appear. [See B.10.2, B.10.5.]

D.4  Generating Formats Code from System Formats

The GENERATE FORMAT command creates format definitions and related code based on the currently set system format. Currently the command works with the two system formats, $PROMPT and $REPORT: generating a $PROMPT format gets you started on a customized prompting input format, while generating a $REPORT format gets you started on a customized report format. [See B.10.]

This chapter will discuss code generation with the $PROMPT format; code generation from $REPORT is described in Part C of the "SPIRES Searching and Updating" manual. [The more basic uses of these two system formats are also described in "Searching and Updating".]

Benefits of Generated System Formats

There are several good reasons for generating your own customized format from system format code, instead of coding a format from scratch or using the system format as is:

Generating Code from $PROMPT

The next sections of this chapter will discuss the specific procedure used to generate $PROMPT statements into formats code [See D.4.1.] and will also present technical information about the generated code that could be useful if you decide to customize it further. [See D.4.2.]

D.4.1  The GENERATE FORMAT Command

This section will discuss the steps to follow to create a customized format from the $PROMPT format. Several steps are involved:

First, select the subfile for which you intend to generate the format. Then set the $PROMPT format, using any options you want to be part of the generated format. For example, you might want only a few elements to be displayed and prompted for by the generated format, or you might want to change the escape character, etc.

Whatever $PROMPT options are in effect will become permanent features of the generated format. That is, you will not be able to change options (such as UCODE to NOUCODE) in the generated format the way you can for $PROMPT. So be certain you set the options you want before you issue the GENERATE FORMAT command. It is a good idea to use the $PROMPT format to add, update or display a few goal records before you generate the format, just to make certain that it is set the way you want.

Second, issue the GENERATE FORMAT command:

The format generator uses your active file, so it first asks you for permission to clear it, unless it is empty already or you have added the CLEAR option to the GENERATE FORMAT command.

The format generator will then ask you some questions, which you should answer. You may respond with a question mark (?) to get more information about any question. Here is a sample session (the $PROMPT format has already been set):

-> generate format
-Welcome to the $PROMPT code generator.  This utility will create
 a format that mimics the $PROMPT format you currently have set.
 You may type ? to receive help when responding to generator questions.
 Note: If you do not include it, the generator will add your account
 name to all IDs.

:INPUT format ID? gq.doc.shoes.input
:INPUT format name (Return = SHOES.INPUT)? input

:Do you also wish to create a display load-format? yes
:DISPLAY format ID (Return = GQ.DOC.SHOES.INPUT.DIS)?
:DISPLAY format name (Return = SHOES.INPUT.DIS)? display

:VGROUP ID (Return = SHOES.INPUT)?

-Please wait while code generation takes place...
-Vgroup GQ.DOC.SHOES.INPUT has been added to the VGROUPS subfile.
-Format GQ.DOC.SHOES.INPUT has been added to the FORMATS subfile.
-Format GQ.DOC.SHOES.INPUT.DIS has been added to the FORMATS subfile.

-All source code has been generated and added to the VGROUPS and FORMATS
 subfiles.  The code may now be modified and compiled.
->

If you make a mistake answering any of the questions, you should press the ATTN/BREAK key to stop, and then issue the GENERATE FORMAT command to start over again.

Please be aware that the format generator may take several minutes to generate all the code, so patience will be required. In addition, you will probably not be able to abort the process by pressing ATTN/BREAK once the code generation has begun.

As the session shows, the format generator will create two or three (possibly even four) different pieces of code:

If you want to use the code as written, then compile the formats and the vgroup:

The vgroup must be compiled first; the formats may then be compiled in any order. [See B.6.2.]

Though it is very unlikely, the formats may not compile. One possible reason is that you named a "floating structure" in the $PROMPT element list, which will require you to make changes to one or more frames in the format definitions before they can compile. [See D.4.2.] If that is not the reason, please see your SPIRES consultant.

You should then select the file and test the format. Set the format you named as the input format, and issue various commands such as ADD, MERGE, DISPLAY, etc., to be sure that it works the same way $PROMPT would.

Output commands such as DISPLAY will cause the output format to be loaded as a load format. If you are interested only in displaying records, you can set that format instead.

If you want to customize the format, you are welcome to make any changes you want to the format definitions or vgroup definitions. The next section discusses customization. [See D.4.2.]

D.4.2  Customizing a Generated Format Further

Once you have generated an input format using the GENERATE FORMAT command, the code belongs to you, and if you want, you may use it or alter it in any way you want. This section will discuss the structure of the generated code, considering what you may want to change, what you probably should not change, and what you can expect if you do make changes.

First of all, if you do want to make changes, you should determine whether the changes fall into one of these categories:

If they do fall into one of those categories, the easiest method is to make the change in the element information packets or in the $PROMPT format, and then generate new formats with the GENERATE FORMAT command. [See D.4.1.] SPIRES will replace the old formats and vgroup in the FORMATS and VGROUPS subfiles with the new ones if you give them the same IDs as the old ones.

Other changes may require changing the format definition code yourself. Though any formats generated through the GENERATE FORMAT command should compile and work like the $PROMPT format, the same guarantee cannot be made if you make your own changes to the code.

The GENERATE FORMAT command will create two to four pieces of code:

1) The Input Format Definition

The input format definition and the vgroup definition will always be created. The input format definition controls format execution -- it is the driver. It basically contains two types of frames: "user frames" and "system frames". The user frames contain most of the subfile-specific code for the format, while the system frames are indirect frames containing common routines (e.g., prompting, displaying) that are called by the user frames. If more than 20 elements are being handled by the format, most of the system frames are put into a load format (item 3 in the list above).

The possible system frames in your format are:

In general, you (the applications programmer) should leave these system frames alone. You should make changes to them only with the greatest of care and the most compelling of reasons.

On the other hand, you are welcome to make changes to the user frames. The user frame that handles record-level elements is called RECORD; other user frames handle structures, and each is named for the structure it handles. [In one rare situation, SPIRES may require you to make changes to one or more user frames: if you have a floating structure, i.e., two or more structures that have the same name and identical definitions, then two or more frames with the same name will be created. You will have to resolve this problem yourself, either by changing the names of some of those frames or by removing those that are exact duplicates and adding appropriate SUBTREE statements to those you keep. [See B.8.2.]]

Each element processed in a user frame is handled by a set of five label groups. Here is an example of a set for a singly occurring, required element:

Each set of label groups begins and ends with an "INIT.n" and an "END.n" label group, where "n" is an integer counter; each element processed by the frame increments the counter by 1. Hence, in the example above, the DATE.UPDATED element is the fifth element processed in the frame. The "INIT.n" label group also sets some variables used by the indirect frames called in the following label groups: #HEADING, used for the element prompt, and #ELEMNAME, the name of the element being prompted for.

The two system frames SETUP.SING and ASK are then executed, which display previous element values and prompt for new ones respectively (see above). The new value is placed in the #PUTVAR variable, which is put into the record in the fourth label group.

What types of changes are typically made to the user frames? You might want to add label groups that display additional information about an element before prompting for it. Or you might want to conduct further tests of the value of #PUTVAR sometime after the ASK indirect-frame call but before the PUTELEM in the fourth label group. Other tests might change the processing flow, as in the following example, which shows the "END.n" label group in another set:

If the input value is N or NO (or a lowercase version), then the next six sets of label groups are skipped.

This next example shows how you could allow a user to input a code instead of a longer value, if the file definition does not provide such a capability. The lines marked with asterisks were added to the generated code.

     LABEL = INIT.8;
       UPROC = LET REQUIRED = $TRUE;
       UPROC = LET HEADING = 'Recording Type';
       UPROC = LET ELEMNAME = 'RECORDING.TYPE';
     LABEL;
       IND-FRAME = SETUP.SING;
  *    UPROC = * 'Type the appropriate number:';
  *    UPROC = * '1 - Phonograph Record';
  *    UPROC = * '2 - Cassette';
  *    UPROC = * '3 - Compact Disc';
     LABEL = RECORDING.TYPE;
       IND-FRAME = ASK;
       UPROC = IF #RETURN THEN RETURN;
       UPROC = IF #NEXTELEM THEN JUMP END.8;
  *    UPROC = IF ~$MATCH(#PUTVAR,1,2,3) THEN '* Value must be 1,
                  2 or 3';
  *    UPROC = THEN JUMP RECORDING.TYPE;
  *  LABEL;
  *    VALUE = #PUTVAR;
  *    INPROC = $CHANGE.LIST('1,RECORD, 2,CASSETTE, 3,COMPACT DISC');
  *    UPROC = LET PUTVAR = $CVAL;
     LABEL;
       VALUE = #PUTVAR;
       UPROC = IF $PROCERR THEN JUMP RECORDING.TYPE;
       PUTELEM = RECORDING.TYPE;
     LABEL = END.8;

In another situation, you may want to provide a default value to be used for an element if the user does not enter a value. The generated code includes a variable called #USERDEFAULT that may be set to the default value desired. The value of #USERDEFAULT will be assigned to #PUTVAR in the ASK frame if the user does not supply a value for the element. If you want to assign a value to #USERDEFAULT, do it after the SETUP frame, but before the ASK frame. (The SETUP system frames reset #USERDEFAULT to null.) For example, here are the second and third label groups for the DATE.UPDATED element, including the setting of #USERDEFAULT:

Thus, if the user does not enter a value for DATE.UPDATED, the default value will be that day's date. The default value will be shown to the user in the prompt:

Using #USERDEFAULT with multiply occurring elements is not recommended. The user, assuming that RETURN can be pressed to end prompting for the multiply occurring element, will instead be adding an occurrence of the element with the default value. He or she will then be prompted for another occurrence, ad infinitum, until some escape sequence (like /E) is entered to end it. If you want to use #USERDEFAULT with multiply occurring elements, you should use a counter or test some other condition to determine when to stop prompting for another value.

It is also important to understand how #USERDEFAULT interacts with #INFODEFAULT. The #INFODEFAULT variable holds the value of the DEFAULT info-element (from the element information packet in the file definition) for the element being processed, if it existed when the input format was generated. The #INFODEFAULT value is shown to the user in the element prompt as a description of the default value. Thus, it overrides the value of #USERDEFAULT for the prompt. However, when the input value is actually placed in the record, #USERDEFAULT will be used if no input value was given, not #INFODEFAULT.

In fact, #INFODEFAULT will never be used as the actual default value put in the record because the DEFAULT info-element is meant to describe the default value, not to be that value. The DEFAULT info-element usually describes a default value supplied by the file definition, using the $DEFAULT (A123) inclose rule. Hence, if it shows up in the generated format as #INFODEFAULT, it probably does represent the value that would be put in the record automatically, according to the file definition.

If you want to use #USERDEFAULT but #INFODEFAULT is set in the label group where you would set #USERDEFAULT, you should either replace the Uproc that sets #INFODEFAULT with one that sets #USERDEFAULT, or not set #USERDEFAULT at all. If you were to use both of them, the element prompt would display the value of #INFODEFAULT as the default but would use #USERDEFAULT if no value was input.

Note that the DEFAULT info-element and the HEADING info-element for an element are compiled into the input format for the #INFODEFAULT and #HEADING variables -- their values at the time the GENERATE FORMAT command was issued are used. If the file owner changes them later, the format definition should also be changed.

On the other hand, the DESCRIPTION info-element, which is displayed when the user requests help for an element, is not compiled into the format definition, but is retrieved from the file itself. If the file owner changes the DESCRIPTION info-element, the changed version will appear when the user requests help.

2) The Output Format Definition

The data frame for the output format definition is called DISPLAY. All other frames are indirect frames that process structures. In general, all the frames of the output format definition can be considered "user frames" -- change them as you desire.

3) The System-Frames Load Format Definition

System frames were discussed under the "Input Format Definition" above. Remember for both this load format and the output load format that they must also be compiled. If you make changes to their names, be sure that changes are also made appropriately to the calling frame(s).

4) The Vgroup Definition

If you need to use other variables, you may add them to the end of the vgroup definition. You should not, however, remove variables created by the report generator unless you are extremely careful and have a strong reason.

The variable arrays FILLER, LMAR and DEPTHIND are used to control some of the formatting of the input prompts, as the comments in the vgroup definition indicate. If you change their assigned values in the definition (the VALUE statements), you change the appearance of the input prompts.

Of course, as stated earlier, all the code is yours to do with as you please. You may have other types of changes to make besides those suggested in this section, and you are welcome to make them. Remember though that once you start making changes to these formats, you are on your own!

E  Appendices

E.1  Annotated List of Elements (Statements) in a Format Definition Record

Below is a list of all the possible elements in a format definition record. Structures are indicated and the elements within them are indented. Most of the structures have key elements, so the structure names seldom are seen in format definitions. [See B.1.3.]

In cases where multiple mnemonics are allowed for an element, the first name given is the primary mnemonic, which will appear when the record is displayed in the standard SPIRES format. Not necessarily all the aliases for all elements are listed here, but certainly the most common are.

Detailed information for each statement can be found in this manual or through the EXPLAIN command online.

ID  (key)                     - format definition identification
                                name, in the form "gg.uuu.name"
COMMENTS, COM, COMMENT        - comments, usually about the format
                                definition as a whole
AUTHOR                        - name and phone number of the person
                                owning and/or defining the format
DEFDATE                       - date the format definition was
                                added (system-generated)
MODDATE                       - date the format definition was last
                                updated (system-generated)
MODTIME                       - time the format definition was last
                                updated (system-generated)
FILE                          - name of the file for which the
                                format is being written
RECORD-NAME, RECORD, REC      - name of the record-type in the file

GEN-FILE                      - declares the formats in the format
                                definition to be general file formats

Structure: VGROUPS            - each structural occurrence represents
                                one variable group definition
  VGROUP  (key)               - the name of the variable group

  COMMENTS, COM, COMMENT      - comments about the vgroup as a whole

  Structure: VARIABLES        - each structural occurrence represents
                                one variable or array
    VARIABLE, VAR  (key)      - the name of the variable

    COMMENTS, COM, COMMENT    - comments about the variable

    OCC, OCCURS, OCCURRENCES  - the number of occurrences of the
                                variable; > 1 indicates an array
    LEN, LENGTH               - length in bytes to be reserved for
                                each occurrence
    TYPE                      - the type of variable, e.g., STRING,
                                INT, PACKED, FLAG
    REDEFINES, REDEFINE, RED  - the name of another variable in the
                                array that this variable redefines
    REDEFINES-AT              - not currently implemented; for
                                future development
    INDEXED-BY, INDEXES, IND  - names of variables used as indexes
                                for the current variable
    VALUE, VAL, VALUES        - values to be assigned to the array
                                when they are initiated
    DISPLAY-DECIMALS, DEC     - number of decimal places displayed
                                for packed decimal variables

Structure: FRAME-DEF, FRAMES  - each occurrence represents a single
                                frame definition
  FRAME-ID  (key)             - the name of the frame

  COMMENTS, COM, COMMENT      - comments on the frame

  DIRECTION, DIR              - direction of data processing for the
                                frame: INPUT, OUTPUT, INOUT
  SUBTREE                     - structure containing the elements
                                processed by this frame (indirect)
  FRAME-DIM                   - frame dimensions describing the size
                                of the buffer to hold the data
  USAGE                       - the type of commands that will cause
                                this frame to be executed

  Structure: LABEL-GROUP      - each occurrence represents a single
                                label group definition
    LABEL, NAME  (key)        - the name of the label group; it may
                                be null, e.g., LABEL;
    COMMENTS, COM, COMMENT    - comments

    ENTRY-UPROC               - "commands" to be executed first time
                                the label group is entered
    TSTART                    - the position where placement of
                                the TITLE should begin
    TITLE                     - a character string to be displayed
                                along with the output value
    GETELEM                   - name of the element to be retrieved
                                for output processing
    VALUE, VAL                - the value to be processed by the
                                label group
    IND-STRUCTURE, IND-STRUC  - name of a data structure processed
                                by the IND-FRAME statement below
    IND-FRAME                 - the name of an indirect frame to be
                                executed at this point
    DEFAULT                   - provides a default value for output
                                if no value is retrieved by GETELEM
    ESTART                    - starting position for an error
                                message; restricted to INOUT frames
    ELENGTH, ELEN             - length in bytes reserved for the
                                error message; INOUT frames only
    PADCHAR                   - sets a pad character for this label
                                group only; overrides SET PADCHAR
    MARGINS, MAR              - left and right column margins in the
                                buffer for data placement or retrieval
    MAXROWS, ROWS             - maximum number of rows in the buffer
                                for the value
    LENGTH, LEN               - maximum length in bytes for the
                                current value
    START                     - starting position for the value

    SCAN                      - method of controlling data retrieval
                                from the buffer (input only)
    GETDATA                   - "command" telling SPIRES to retrieve
                                data from the buffer (input only)
    INPROC, IN, INP           - INPROC processing rule string through
                                which the value should be processed
    OUTPROC, OUT              - OUTPROC processing rule string through
                                which the value should be processed
    INSERT, INS               - character string to precede, follow or
                                be inserted into the value
    BREAK, TRUNC              - specifies how long values can wrap
                                around into the next row (output only)
    ADJUST, ADJ               - specifies value adjustment within the
                                field (use SET ADJUST Uproc instead)
    UPROC, P, U               - "commands" to set variables, change
                                execution flow, stop execution, etc.
    DISPLAY                   - attributes concerning full-screen
                                processing
    PUTDATA                   - places value into the buffer in
                                output frames
    PUTELEM                   - stores the value as the named element
                                (input only)
    REMELEM                   - removes the element from the record
                                (MERGE frames for input only)
    SORT                      - "command" to initiate a sort area or
                                to invoke the sorting
    LOOP                      - returns execution control to the
                                beginning of the current label group
    XSTART                    - starting position of subsequent values
                                in the buffer when "looping"
    XESTART                   - starting position of subsequent error
                                messages; see ESTART above
    IN-AREA                   - device services area to which the
                                frame being called is to be sent
    WITH-UPDATE               - allows elements in subgoal records
                                to be updated
    ELEM-USE, USE             - names element referred to in some
                                label group statements (virtual)
  LOAD-FORMAT, LOAD           - name of another format to be invoked
                                at this point
  LOAD-OPTIONS                - options relating to the named load
                                format, e.g., UNLOAD and NODEFQLOAD.
  USING-FRAME, USING          - name of the specific frame in the load
                                format to be executed
  SUBFILE, SUBF               - name of the subfile being accessed by
                                subfile subgoal processing
  NODEFQ                      - states that the deferred queue is not
                                to be searched in subgoal processing

Structure: FORMAT-DEC,FORMATS - each occurrence defines another
                                format, with all its frames
  FORMAT-NAME, FORMAT-ID(key) - the name of the format, used in the
                                SET FORMAT command
  COMMENTS, COM COMMENT       - comments about the format declaration

  ACCOUNTS, ACCOUNT, ACCT     - list of accounts allowed to set the
                                format
  SORT-SPACE                  - amount of space to be reserved for
                                internal sorting, e.g., 64 for 64K.
  ALLOCATE, ALLOC             - name of a vgroup to be allocated for
                                the format, either global or local
  PERMIT                      - not currently used

  Structure: FRAME-DEC,FRNAMES- each occurrence names a frame defined
                                above to be used by the format
    FRAME-NAME, FRAME  (key)  - the name of the frame being declared

    COMMENTS, COM COMMENT     - comments about the frame declaration

    FRAME-TYPE                - determines how the frame can be
                                invoked; e.g., DATA, INDIRECT, XEQ
    BREAK-LEVEL, LVL          - integer specifying break level for
                                frames of type Summary
    BREAK-ELEM, BRK           - data element to be examined before
                                Summary frame execution
    UPROC, U, P               - "commands" requesting processing to
                                occur before frame execution begins
    IN-AREA                   - device services area to which the
                                frame being called is to be sent
Structure: PROCS, XGROUP      - each occurrence represents a proc
                                called by INPROC or OUTPROC above
  PROC  (key)                 - the name of the proc

  RULE, RULES                 - the string of actions comprising
                                the proc
  Structure: YGROUP,          - each occurrence represents
             PROC-PARMS         parameters for the proc
    PARM  (key)               - the name of the parameter used in
                                the proc
    DEFAULT                   - a default value for the parameter

    Structure: ZGROUP,        - each occurrence represents a symbol
              PROC-SYMBOLS      that may be used for the parm value
      SYMBOL  (key)           - the name of the symbol

      VALUE                   - the value represented by the symbol

      COMMENTS, COMMENT, COM  - comments

    COMMENTS, COMMENT, COM    - comments

  COMMENTS, COMMENT, COM      - comments

  REF-TO                      - names another proc that defines
                                the current one
EXTDEF-ID, EXTDEF, PROCDEF    - name of a record in the EXTDEF
                                subfile containing procs used here
Structure: VERSION-STR        - contains version information for
                                the format
  VERSION-NUMBER (key)        - the version number for the current
                                version of the record
  VERSION-ACCT                - when this value matches the account
                                in the key, VERSION-NUMBER must
                                match the stored value

E.2  SPIRES System Variables Specific to Formats

The system variables discussed in this section are meaningful when a format has been set with the SET FORMAT command. In general, they have no meaning unless a format is set; many have no meaning unless the format is executing. Note that format users have access to all system variables and functions (described in the manual "SPIRES Protocols") as well as these format variables.

E.2.1  Flag Variables

There are two categories of flag variables. Those in the first category are not reset whenever a frame is processed and hence are global within a format; they are called "global flag variables". Those in the second category are reset whenever a frame begins executing and hence are useful only within frames or between frames (i.e., in the format declaration section), not across frames; they are called "frame flag variables". Flags are set with a Uproc or command of the form "SET flagname". Most can be cleared with a command of the form "SET NOflagname".

E.2.1.1  $NEWPAGE and $FRONTPAGE

$NEWPAGE is a global flag, used in report mode, that may be set to tell SPIRES to place the current buffer on the next page. The flag is cleared once the buffer has been flushed to the new page. The page eject does not occur when this flag is set, but rather as the buffer is being output. To cause an immediate eject, the EJECT PAGE Uproc may be coded within a label group. [See B.10.3.5.]

$FRONTPAGE may be set in conjunction with $NEWPAGE in order to cause the carriage control character "1" to be replaced by "F" when a page eject occurs (to force printing on the front of a page). The flag is turned off once the "F" is generated. [See B.10.3.5.]

E.2.1.2  $NOBREAK

The $NOBREAK flag may be set to indicate that the current buffer is not to be broken across page boundaries in a report. Therefore, if the number of rows specified by the FRAME-DIM statement does not fit at the end of the current page (according to the value of the $LLEFT variable), a new page will be started and the buffer will start on that page. This global flag is cleared when the new page is started. The SET NOBREAK Uproc has no effect when writing records in line-by-line mode, that is, when the FRAME-DIM statement has a row dimension of 0. [See B.10.3.4.]

E.2.1.3  $FREC

This global flag is set when the first record is being processed during a multiple-record processing command, such as TYPE or DISPLAY REST; the flag is subsequently cleared. You may wish to test this flag for special first-record handling. It is set whether or not report mode processing is in effect -- that is, it may be used by non-report output formats. The $FREC flag may also be used in SPIBILD input formats.

E.2.1.4  $GPROCERR and $WPROCERR

The $GPROCERR flag is set if a processing rule error of level S or E occurs in any label group of the currently executing format. Similarly, the $WPROCERR flag is set if a warning-level processing rule error occurs. As global flags, they are reset to $FALSE only at the start of format execution (as a result of a format invocation command or a REPROMPT Uproc). [See C.3.6.1.]

E.2.1.4a  $GCHANGED and $GNCHANGED

The $GCHANGED flag is set when the $CHANGED flag is set in any label group of the currently executing format. ($CHANGED indicates that a data value has been changed on the screen in a full-screen application.) [See E.2.1.24.] $GCHANGED is set whenever any field has changed during any iteration of screen processing.

The $GNCHANGED flag is set when any field is changed during the current iteration of screen processing, whether through a REPROMPT Uproc or not (i.e. a "newly changed" field in the current screen iteration).

As global flags, they are reset to $FALSE only at the start of format execution (as a result of a format invocation command).

E.2.1.5  $LRGNXT, $LRGLCTR and the SET LARGE Uproc

The SET LARGE Uproc is useful in situations where you are handling index records as goal records. If an index record gets very large (over 12,000 bytes) it may be split into multiple index records. If your format tries to retrieve the pointer group data from these indexes without doing "SET LARGE" processing, only the first of the multiple records will be processed for a given key.

To request SET LARGE processing, you first need to issue the SET LARGE Uproc for the data frame in the Frame Declaration section.

When SET LARGE processing is in effect, SPIRES will examine the record being processed to determine whether it points to another "segment" for that indexed value. If so, that segment will be treated like another record -- when the format's execution for the current segment stops, the next segment will be retrieved and the format will begin executing again, as if another complete record were being processed.

The flag variable $LRGNXT will be set to true if another segment will be retrieved. This is useful for testing in situations where you want the multiple segments to appear as a single record:

When the second and subsequent segments are retrieved, you do not want the key to be repeated. In the startup frame, the flag #PRINTKEY is set to true. When a record is processed that has $LRGNXT set, #PRINTKEY is set to false (in the final label group), the new segment is retrieved, the format starts executing again, and the PUTDATA of the first label group is skipped. Without this handling of $LRGNXT, the key would be displayed again each time a new segment was processed.

Another system variable, $LRGLCTR, is a four-byte hexadecimal variable containing the locator of the next record segment when $LRGNXT is set. [See B.12.2.]

E.2.1.6  $SORTKEND

$SORTKEND (alias $SORTVEND), a global flag, is cleared when format sorting is done (using the "SORT = A" or "SORT = D" statement). It is set when the last sort key has been retrieved. If the flag is set, no sort keys may be retrieved by referencing the $SORTKEY variable. [See B.10.8.]

E.2.1.7  $SORTDEND

This global flag is cleared when format sorting is done (by a "SORT = D" or "SORT = A" statement). It is set when the last value of $SORTDATA has been retrieved. [See B.10.8.]

E.2.1.8  $NEWCOL

You may set the $NEWCOL flag to tell the report processor to start the current buffer in the next column of the page if $COLUMNS is greater than one. Note that ejection to a new column does not take place at the time the flag is set, but when the buffer is flushed. For an immediate column eject, use the EJECT COLUMN Uproc. [See B.10.3.8.]

E.2.1.9  $REPORT and $NOREPORT

This global flag may be tested to see if multi-record processing is being done in report mode, meaning initial, header, footer, ending, group-start and group-end frames, as well as pagination, will take place if the format code requests them. When $REPORT is set to $TRUE, $NOREPORT is set to $FALSE, and vice versa. These global flag variables should only be tested within a format; when no format is being executed, the value of $REPORT is always "0" (FALSE) and the value of $NOREPORT is always "1" (TRUE).

E.2.1.10  $ENDATA

The global flag $ENDATA is set whenever a GETDATA statement attempts to access beyond the bounds of the character array (such as the active file) from which input data is being read. A subsequent GETDATA will cause an S808 error to stop execution of the format immediately. [See C.2.3.]

E.2.1.11  $ABORT

The global flag $ABORT is set when either an ABORT or STOPRUN Uproc is executed in the format. The format immediately stops executing. The $ABORT flag is cleared the next time a format begins executing -- in other words, this flag can only be tested outside of the format.

E.2.1.12  $REPROMPT

The global flag $REPROMPT is set whenever a REPROMPT Uproc is executed in a frame designated as INOUT. The flag is cleared the next time a format begins executing. [See the manual "SPIRES Device Services" for more information about this flag.]

E.2.1.13  $PROTECT

Setting the global flag $PROTECT causes SPIRES to move all data to the output character array in PROTECT mode prior to writing the array out to the CRT screen. Single PUTDATA requests may be performed unprotected by coding a DISPLAY=U statement in a label group.

If this flag is not set in a format or if SET NOPROTECT is coded, all data is moved in UNPROTECT mode, whether or not protection was defined by the user for the output area. [See the manual "SPIRES Device Services" for more information about this flag.]

E.2.1.14  $TESTFAIL

The global flag $TESTFAIL, if set, causes the input format processor to test and bypass the failure condition caused by a SERious INPROC error if no PUTELEM is done for that element. That usually means the value of $PROCERR will be tested with appropriate branching in a Uproc. [See C.3.6.2.]

E.2.1.15  $ATTN

$ATTN is a global flag variable that is true whenever the ATTN/BREAK key is pressed; however, it is turned off immediately before the next command is processed. Hence, its main value is to test whether the ATTN/BREAK key has been pressed during format processing. When the key is depressed while a display format is being processed, the display to the terminal is stopped but the format continues processing to the end of the format. If the format is long and does complicated processing, you might want to check $ATTN within the format, and if $ATTN is true, then issue a STOPRUN or ABORT.

Notice that any command or response from the terminal will reset $ATTN, including a response to an ASK UPROC within the format. [See C.3.6.5.]

E.2.1.16  $SKIPF

You may set this frame flag to indicate that SPIRES should completely bypass further processing of this frame during this call after completing the current label group. It is often tested in a frame declaration within the Format Declaration section:

telling SPIRES not to execute the declared frame if the record being processed is not the first record processed by the command. [See B.10.3.1.]

E.2.1.17  $SUPPRESS

You may set the frame flag $SUPPRESS in an output frame to indicate that the data should be processed by the frame but not placed in the buffer. That is, in an output frame, PUTDATA statements are ignored after the SET SUPPRESS Uproc is executed, and the buffer will be discarded at the end of the frame's processing. In an input frame, GETDATA statements are ignored, and input lines are not read.

E.2.1.18  $FLUSH

The $FLUSH frame flag, if set, tells SPIRES how to handle the case where the buffer has been completely filled but more data is to be output. Upon detection of this overflow condition, the current buffer is flushed; the rest of the frame is processed in line-by-line mode. Processing continues until all frame elements have been handled. [See B.3.3.]

For input frames, the SET FLUSH Uproc allows the format processor to access the data source (such as the active file) for more data after the amount defined by the frame dimensions has been processed. [See C.2.3.]

E.2.1.19  $PROCERR

The frame flag $PROCERR is set if an error designated as serious (S or E) occurred during the execution of an OUTPROC or INPROC string. You may test this flag in order to take corrective action in the format. The variable $PROCERR is cleared upon entry to each label group. [See C.3.6.1.]

E.2.1.20  $DEFAULT

The frame flag $DEFAULT is set whenever default data element processing occurs in an output label group. This flag is set only if the DEFAULT statement is specified in the label group and a GETELEM or IND-STRUC statement fails to find a value in the record. [See B.4.5.1.] It is cleared at the start of each label group.

E.2.1.21  $SKIPEL

The frame flag $SKIPEL may be set by a SET Uproc in input frame label groups to bypass execution of PUTELEM or REMELEM statements. [See C.3.3.2.] The flag is cleared at the start of each label group.

E.2.1.22  $APROCERR

This "label group" flag is set whenever any error occurs during INPROC or OUTPROC execution, even a default (D) or warning (W) error. Note that $PROCERR [See E.2.1.19.] and $GPROCERR [See E.2.1.4.] are set only when an E or S level error occurs. [See C.3.6.1.] The flag is reset for each label group executed.

E.2.1.23  $OVERFLOW

The flag $OVERFLOW may only be set by SPIRES and only when "PUTDATA = 1" is coded for an output frame label group. If a fixed-dimension frame overflows and there is no automatic flushing in effect [See E.2.1.18.] then $OVERFLOW is set to $TRUE. The PUTDATA is not executed, meaning that the value that caused the overflow condition is not placed in the buffer.

SPIRES will not execute any more label groups if $OVERFLOW is set. When it tries to execute the next label group, an automatic RETURN occurs. If this happens in an indirect frame, SPIRES will return to the calling label group of the calling frame, complete the execution of that label group (though no more LOOPing will occur) and RETURN from there. This return process continues until command level is reached.

E.2.1.24  $CHANGED

The $CHANGED flag can be tested in a label group for which the GETDATA=7 option has been specified. [See C.3.1.] The $CHANGED flag is set depending on whether a data value has been changed by the user on the CRT screen. It is meaningful only for full-screen CRT applications. If the data has been changed, then $CHANGED is set to true. This flag is set to false whenever a label group is entered. [See the SPIRES manual "Device Services" for more information about full-screen applications.]

E.2.1.25  $TESTREF and $REFERENCED

The $TESTREF variable is set when a SET TESTREF Uproc is executed. When a subsequent REFERENCE Uproc is executed, SPIRES will test for the existence of the referenced record before actually referencing it. If the record does exist, the $REFERENCED flag is set; if the record does not exist or if an error occurs, the $REFERENCED flag is not set.

If a REFERENCE Uproc attempts to reference a non-existent record and $TESTREF has not been set by the format, an S256 error will occur. [See C.5.2.]

The $TESTREF flag can be reset with the SET NOTESTREF Uproc. If not reset, it remains set for the current execution of the format. Remember that only one record can be successfully referenced per execution of the format, however.

E.2.1.26  $LREC

This flag is set after the final data frame has executed for the last record during report mode processing. It may be tested in group-end frames if special last-record handling is desired in the group-end frames (cf. ending frames).

E.2.1.27  $NODATA

The $NODATA flag variable is set when a format is executing data frames (or indirect frames called from data frames) but no data is actually being handled. There are three specific situations in which SPIRES sets $NODATA:

E.2.1.28  $FTRACE

The $FTRACE flag variable is set when either a SET FTRACE or SET GLOBAL FTRACE command is issued. [See B.7.2.1.] It is useful when you want to display debugging information at the terminal during format execution, e.g.,

That Uproc would print the current label group and the current value of $CVAL when the format tracing facility is in effect.

Note that if you want to direct custom messages to the trace log (with the SET TLOG command) you should use the $ISSUEMSG function rather than the * Uproc, as in this example:

For most tracing situations, $ISSUEMSG is preferable to using the "*" command, since it will only appear on the terminal screen and never as part of the trace log. [EXPLAIN SET TLOG for information about trace logs. EXPLAIN $ISSUEMSG for an explanation of the function.]

SPIRES changes the value of $FTRACE appropriately, depending on what type of format is being executed. For instance, if you issue the SET GLOBAL FTRACE command and then execute a non-global format, the value of $FTRACE within that format will be "0" ($FALSE), though it will be "1" ($TRUE) within the global format when a frame of the global format is executed.

E.2.1.29  $PUTFLAG

The flag variable $PUTFLAG in a format normally tells whether a PUTDATA statement has been executed since the beginning of the current frame. $PUTFLAG is set to $FALSE at the beginning of every frame (except a header frame or an overflow frame, for reasons described below). The variable becomes set to $TRUE as soon as a PUTDATA successfully completes processing, and stays true until the frame has finished executing, unless you reset its value with the SET NOPUTFLAG Uproc.

The variable is useful in reports as a way of communicating whether a segment of data that is output at the top of a page should be treated as a continuation of previous data or not. For instance, suppose one of your data frames contains a series of label groups that together make up a discrete segment of data. Just before the first label group executes, you might issue a SET NOPUTFLAG Uproc to set $PUTFLAG to false. If a header or overflow frame executes before any PUTDATA statements execute from those label groups, the frame will know by checking $PUTFLAG that no part of the segment has been output. [See B.10.3.1.]

For more information, EXPLAIN SET NOPUTFLAG UPROC. [See B.10.3.7a.]

E.2.1.29a  $NOACCESS

The flag variable $NOACCESS is set by SPIRES if you try to enter a subfile subgoal but do not have access to select the subfile. This helps distinguish whether a subfile subgoal attempt failed because of a select problem or because a record was not found. In the latter case, $NOACCESS will be false, as it usually is.

This variable can be tested effectively only if you have SET TESTSUBG prior to the subgoal attempt. [See B.12.2.2.]

E.2.2  Integer Variables

As with the flag variables, the format system variables of type INTEGER are in two categories: those that are useful throughout a format and those that are generally useful only within the confines of a frame or even a single label group.

E.2.2.1  $RECNO ($RECNUM)

The integer variable $RECNO (alias $RECNUM) holds the number of formatted records processed thus far, including the current record. $RECNO is local to a format, and is reset any time the format is set. It is also reset each time a multiple-record processing command is issued, unless the WITH FAST prefix is included in the command. It is not reset if a single-record processing command (such as DISPLAY FIRST) is issued. The value of $RECNO is incremented when the format is entered to begin processing the current record, and is immediately available for use in header, footer and overflow frames. The value of $RECNO is also incremented when the format is re-entered to begin processing the next record (same multi-record processing command).

The $RECNO variable is also available for use in batch input formats for SPIBILD, holding the number of records processed thus far, including the current record, for the current command. [See C.9.]

The SET RECNO Uproc can be used to set the value of $RECNO as desired. You can use the Uproc to set $RECNO in any data or report frame other than a header or footer.

E.2.2.2  $PAGECTL

The following table shows the values and functions of $PAGECTL. This global variable may be set with the SET PAGECTL Uproc in an initial frame of a format. [See B.10.3.3.]

The variable may also be set at the command level, using the SET PAGECTL command after setting the format and setting report mode. It should not be changed in the middle of report processing, e.g., in a data or ending frame.

E.2.2.3  $LLEFT

The global integer variable $LLEFT holds the number of lines left for data on the current page of the report being produced. It does not include the rows to be used for footer frames, as given by $FTRLEN. Also, if the header frame has not yet been executed for the page, $LLEFT does not include the rows to be used for it (or them, if there are more than one header frame), as given by $HDRLEN.

The value of $LLEFT changes only if the report is going to the active file or some other external data set (i.e., if the IN ACTIVE or "IN file" prefix is included on the output command).

Note that if you check $LLEFT in an indirect frame that has a PUTDATA on its calling label group, $LLEFT does not include the data put out by the indirect frame since it has not yet been placed in the calling frame.

E.2.2.4  $LINEPP

This global variable may be set to indicate the number of lines per page in a report. $LINEPP includes the total number of lines on the page, including lines set aside for footers by the SET FTRLEN Uproc. It is an integer variable whose default value is 60. You normally set it in the initial frame.

E.2.2.5  $HDRLEN

$HDRLEN is a global variable used to declare the number of rows that will be output as a header for the top of each page of the report. You usually set it in the initial frame. [See B.10.5, D.3.]

If this variable is set to a number less than the number of rows in the header frame dimensions, the header will be output with the number of rows specified by $HDRLEN. (If there are multiple header frames, this variable should be set to the maximum number of rows to be output by all of the frames together.)

If $HDRLEN is greater than or equal to the total number of rows specified in the header frame dimensions, the regular data will begin on the next row after the frame dimension's number of rows. In other words, SPIRES will output either $HDRLEN number of header rows or the number of rows as declared by the header frame dimensions, whichever is less.

E.2.2.6  $FTRLEN

$FTRLEN is a global integer variable used to specify the number of rows to be output as a footer to the pages in a report. You usually set it in the initial frame. [See B.10.5, D.3.]

If you set $FTRLEN to a number less than the number of rows specified in the frame dimensions for the footer frame, only that number of rows will be output. (If there are multiple footer frames, then this variable is usually set to the total number of rows to be reserved for all frames.) If you set $FTRLEN to a number higher than that of the footer frame dimensions, that number of rows will be left available for the footer frame (though the footer frame will of course only use the number of rows specified in its frame dimensions). If $LINEPP is to take into account (i.e., include) the number of rows for the footer frame(s), then this variable must be set; if it is not set, then $LINEPP does not include the number of rows for footer frames and the correct number of lines will not be left at the bottom of each page for them.

E.2.2.7  $MARGIN

This global variable can be set to the number of blank characters desired as a default left margin in report generation.

The current value of the variable at the time a buffer (whether a frame-dimension of data if in fixed-frame processing, or a line of data if in line-by-line mode) is written out is the number of blank characters inserted. So, for example, if a SET MARGIN Uproc is executed several times during the processing of a fixed-dimension frame of data, the value set at the last execution of the Uproc before the frame is flushed is used for the number of blank characters.

Here is a peculiar situation, however: In a report format, suppose that SPIRES is holding a buffer of lines with a given $MARGIN value but, before flushing that buffer, has to execute a header and/or footer in which a different value is set for $MARGIN. In that case, the header/footer will use the new value of $MARGIN, but the buffer, when flushed, will use the old value. However, once the buffer is completely flushed, the new value set in the header or footer will take effect. This situation can arise when the header/footer frames have different values for $MARGIN than the data frames -- you don't want data frames that are broken across a page to be affected by the value of $MARGIN set for the header/footer.

E.2.2.8  $PAGENO ($PAGENUM)

The global variable $PAGENO (alias $PAGENUM) contains the current page number of a report. It includes all pages written since the start of the report, including pages produced by the initial frame(s). The $PAGENO variable may be set to the initial page number desired in a report. The default initial value is 1.

Note: if you do not set $PAGENO yourself, it will always start at 1 when the format is executed. However, if you set $PAGENO explicitly, whether in a Uproc or by a SET PAGENO command, then the variable will not be reset if the format is executed again; instead, $PAGENO will continue counting from that number. For example, if the format does not contain a SET PAGENO Uproc but the user sets it himself:

Instead of resetting $PAGENO, SPIRES uses the current value since it was explicitly set by the user using the format. However, setting $PAGENO in the initial frame would prevent this problem from occurring, since each time the format executed, the variable would be reset to the desired value.

The upper limit on the value of $PAGENO is 65535.

E.2.2.9  $COLUMNS

This global variable may be set to the number of columns desired per report page. The default value is 1. [See B.10.3.8.]

E.2.2.10  $COLWDTH

This variable may be set to the number of bytes to be reserved for each report column. [See B.10.3.8.]

E.2.2.11  $CURCOL

This variable contains the current column on a report page if $COLUMNS has been set to a value greater than one. [See B.10.3.8.]

E.2.2.12  $ULEN

$ULEN is an integer variable containing the unconverted length of the current label group's value, i.e., its length before application of any INPROC, OUTPROC or INSERT statements. The variable is useful only within label groups, and it is reset each time a new label group begins execution, even if no value is explicitly processed by that label group.

E.2.2.13  $CLEN or $VLEN

$CLEN (also known as $VLEN) is an integer variable containing the converted length of the current label group's value, i.e., the length after application of any INPROC, OUTPROC and INSERT statements in the label group. The variable is useful only within label groups, and it is reset each time a new label group begins execution, even if no value is explicitly processed by that label group.

E.2.2.14  $CROW

This is the current row position in the frame, i.e., '*'. In output frames, its value represents the row of the buffer into which the previous value was successfully placed (the row in which the value ended, that is). [See B.4.6.1.]

In input frames, the value of $CROW represents the row of the buffer from which the previous value was successfully retrieved (again, the row in which the value ended.) [See C.3.9.4.]

E.2.2.15  $CCOL

This is the current column position in the frame, i.e., '*'. It is similar to $CROW described above, except that it refers to a column rather than a row. [See E.2.2.14.]

E.2.2.16  $LOOPCT

$LOOPCT is an integer variable containing the value of the loop counter for the current label group. It is only useful in label groups containing the LOOP statement. [See B.4.8.4.] The variable $LOOPCT is zero the first time the label group is processed.

The $LOOPCT variable is reset at the entry of every label group; thus, the loop count for a previous label group cannot be determined from $LOOPCT.

E.2.2.17  $NROWS

The $NROWS integer variable contains the number of rows assigned to the current buffer, as determined by the frame dimensions. Under some circumstances, it may be altered by the SET NROWS Uproc. [See B.3.3.1.]

E.2.2.18  $NCOLS

The $NCOLS integer variable contains the number of columns assigned to the current buffer, as determined by the frame dimensions. Under some circumstances, it may be changed by the SET NCOLS Uproc. [See B.3.3.1.]

E.2.2.19  $SROW

$SROW is an integer variable containing the starting row number of the last value placed in or retrieved from the buffer.

E.2.2.20  $SCOL

The $SCOL integer variable contains the starting column number of the last value placed in or retrieved from the buffer.

In input frames, it represents the column number of the position where SPIRES started "looking" for the value, not where the value actually began. Suppose, for instance, that the starting position is declared as "START = 1,20" for a length of 10 characters. Even if leading blanks appear from columns 20 to 24 and the actual value to be stored begins in column 25, the value of $SCOL will be 20. Similarly, if a starting scan string is involved, the value of $SCOL represents the point where SPIRES began scanning, not where the value was actually found.

E.2.2.21  $LROW

$LROW is an integer variable containing the number of the highest row used in the frame (i.e., X-1). [See B.4.6.1.]

E.2.2.22  $LCOL

$LCOL is an integer variable containing the number of the last column processed in the buffer. [See B.4.6.1.]

E.2.2.23  $EROW

The $EROW integer variable represents the starting row number defined by the last execution of the ESTART statement, used in full-screen applications. It is reset to the position of the first error message field after the format completes execution.

E.2.2.24  $ECOL

The integer variable $ECOL contains the starting column number defined by the last execution of the ESTART statement, used in full-screen applications. It is reset to the position of the first error message field after the format completes execution.

E.2.2.25  $FLINES

$FLINES is an integer variable containing the total number of rows of the frame that have been flushed to the output device since frame processing began. For example, if a frame is flushed, say by the FLUSH Uproc, after two rows of data have been put into it, the value of $FLINES is 2. If two more rows are then processed and flushed as the frame continues, the value of $FLINES would be increased to 4. [See B.10.3.6.]

$FLINES is reset when a new frame begins executing, with one exception: if the HOLD Uproc is in effect, $FLINES contains the total number of rows flushed since it went into effect. [See B.10.3.7.]

E.2.2.26  $ELOCC and $LASTOCC

$ELOCC is an integer variable containing the number of occurrences of the data element specified by the GETELEM statement in an output label group. (Note that if the element is in a structure, then $ELOCC is the count of the number of occurrences of the element in the current occurrence of the structure.)

$LASTOCC is an integer variable holding the occurrence number of the final occurrence of the data element accessed by the GETELEM statement in an output label group. $LASTOCC is usually equal to $ELOCC-1, except that $LASTOCC is 0 if there is one occurrence or if there are no occurrences. This variable is intended to help you loop through an occurrence list when it may be inconvenient to use $ELOCC; the $LOOPCT variable [See E.2.2.16.] can be compared directly to $LASTOCC rather than to $ELOCC-1:

E.2.2.27  $CURROCC and $TRUEOCC

$CURROCC (or $CUROCC) and $TRUEOCC are integer variables containing the occurrence number of the element retrieved by the current iteration of a label group. (The value is 0 for the first occurrence, 1 for the second occurrence, and so on.) In a label group that retrieves unfiltered occurrences of an element with a LOOP statement, the two variables contain the same value for each occurrence retrieved.

Within an input format, in a label group affected by either the SET STARTOCC Entry-Uproc or the SET LOOP BACKWARDS Entry-Uproc or by both, [See B.4.8.5.] these variables indicate an occurrence's number (its true position in its record) more reliably than the $LOOPCT variable does.

$CURROCC, $TRUEOCC, and Element Filters

When a filter has been applied to an element's occurrences, the two variables $CURROCC and $TRUEOCC take on different values. In that case, $TRUEOCC contains the "true" (unfiltered) occurrence number, reflecting the occurrence's actual position in the record, while $CURROCC contains an occurrence number reflecting the filters that have been set. For more information on element filters, see Chapter 21 of the manual "SPIRES Technical Notes" or EXPLAIN ELEMENT FILTERS.

If a GETELEM should fail and $DEFAULT processing takes place, both $CURROCC and $TRUEOCC take on the value -1.

E.2.3  Variables of other Types

Most of the system variables in this section return values of type STRING or HEX. The types of some, such as $CVAL, may vary depending on usage. It is up to you to handle them properly.

E.2.3.1  $UVAL

The $UVAL variable contains the unconverted value of the current label group, i.e., the value prior to any INPROC, OUTPROC or INSERT statement processing specified by the label group. The value of this variable is not useful outside of a label group.

The variable type of $UVAL varies, depending on the type of value assigned to it by the VALUE, IND-STRUCTURE or GETELEM statement.

E.2.3.2  $CVAL

The $CVAL variable contains the converted value of the current label group, i.e., the value after any INPROC, OUTPROC or INSERT statements have been applied as specified in the label group. Its value is undefined if a processing rule fails with an E or S level error. [See C.3.6.] The value of this variable is not useful outside of a label group.

The variable type of $CVAL varies, depending on the type of value assigned to it by the VALUE, IND-STRUCTURE or GETELEM statement, and on the type of processing done by INPROC, OUTPROC or INSERT statements.

E.2.3.3  $PVAL

The variable $PVAL contains the first 256 characters of the unconverted value of the most recently executed label group. Its type varies depending on the value assigned to it. Its value outside of label groups is unpredictable.

E.2.3.4  $SORTKEY

$SORTKEY (alias $SORTV) represents the next value in the sort array. Each time it is referenced, the next value in the array is retrieved. [See B.10.8.]

E.2.3.5  $SORTDATA

The variable $SORTDATA (a.k.a. $SORTD) represents the next value in the "companion" array to the sort array. Each time the variable is referenced, the next occurrence of the array is retrieved. [See B.10.8.]

E.2.3.6  $STARTCHAR

The $STARTCHAR variable contains the character value that caused completion of a SCAN = (chars) or ~(chars). For example, if SCAN = (ABC) were in effect and scanning found the value A, then $STARTCHAR would contain A. [See C.3.5.5.]

E.2.3.7  $ENDCHAR

The $ENDCHAR variable contains the character that caused completion of a SCAN=END, (chars) or END,~(chars). For example, if "SCAN = END, ~(0123456789)" were in effect as the value was being retrieved and then the character Z was scanned, the scanning would end, and the character Z would appear in $ENDCHAR. [See C.3.5.5.]

E.2.3.8  $LABEL

$LABEL (or $L) is a string variable containing the name of the currently executing label group. It is null if the value of the LABEL statement for the currently executing label group is null. [See B.4.1.]

E.2.3.9  $PARM

$PARM is a string variable containing the parameter list specified on a SET FORMAT, SET GLOBAL FORMAT or SELECT command. It provides a simple way for the user to pass parameter values into a format or a subfile's select commands. For example:

The value of $PARM would become "(ring) name phone.number". $PARM is reset to null when the format is cleared.

When $PARM is set by the SET FORMAT, SET GLOBAL FORMAT or SELECT command, its allowed length is virtually unlimited. It may be set explicitly during format execution with the SET PARM Uproc or the SET PARM command, in which case its value is restricted to 256 characters or less:

If you need to use the parameters passed to a format on a SET FORMAT command, it is recommended that you save the value of $PARM in a user variable by coding a LET Uproc in a startup frame. Because the variable is reset by either a SET FORMAT, a SET GLOBAL FORMAT or a SELECT command, any of which may be issued at any time, its value might change between the time the SET FORMAT (or SET GLOBAL FORMAT) command is issued and the format is executed.

Likewise, if your subfile's SELECT-COMMANDs include a SET FORMAT command, be careful to save the value of $PARM each time it is reset, if you need to use those values. That is, a $PARM value set by a parameter list on the SELECT command will be reset by the SET FORMAT command in your SELECT-COMMANDs.

E.2.3.10  $ELEMID

$ELEMID is a four-byte hexadecimal variable identifying an element within the given record-type. It can be used to access data within a record more efficiently than can the element name.

Whenever a $ELEMTEST, $ELNOTEST, $ELIDTEST, $ELEMINFO or $ELIDINFO function is executed, or whenever a "GETELEM = #variable-name;" statement is executed, the value of $ELEMID is reset to the "ID" of that element. (There is one exception to this rule, described later.) The value of $ELEMID can be saved in a variable and later used to access the element, for example, "GETELEM = #element.id;" where "#element.id" is a four-byte hex element ID picked up earlier.

The variable is most commonly used in general file applications that would save off element names for later processing or repeated processing. Retrieving element ID variables is more efficient than retrieving the element name variables (four-byte hex variables as opposed to varying-length string variables for element names), as is using the element IDs in GETELEM, PUTELEM, REMELEM and IND-STRUCTURE statements.

When STR is the code specified in a $ELEMTEST, $ELNOTEST or $ELIDTEST function, and the named element is a dynamic element, the value of $ELEMID is the value for the primary element in the dynamic element definition. Also when STR is the code in one of those functions, if the named element is structurally bound, then $ELEMID is set to the element ID of the element's "parent" structure.

When a $ELEMINFO or $ELIDINFO function returns the default value given in the function (meaning that the requested info-element does not exist), the $ELEMID variable is not set to the ID of the named element -- it is left untouched, in fact.

E.2.3.11  $FORMAT, $SETFORMAT, and $GLOFORMAT

The three string variables $FORMAT, $SETFORMAT and $GLOFORMAT contain the names of various formats currently set.

The $FORMAT variable contains the name of the currently set format. It is null if no format is set. Its value is the name given in the FORMAT-NAME statement of the format definition.

The $SETFORMAT variable contains the fully-qualified name of the currently set format. It has the same value as $FORMAT above, except that for general-file formats it includes the prefix "**ORV.gg.uuu." where "gg.uuu" is the account number of the format definer. In other words, $SETFORMAT is the complete format name as given in the SET FORMAT command that set the format. It is not the ID value of the relevant format definition (except perhaps by coincidence).

The $GLOFORMAT variable contains the fully-qualified name of the global format currently set, including the account prefix.

Note for system general-file formats and system global formats (such as $REPORT), the prefix "ORV.GG.SPI." and "**ORV.GG.SPI." will be replaced by the "$" prefix in $SETFORMAT and $GLOFORMAT.

Below is a sample session showing the various values of these three variables after SET FORMAT and SET GLOBAL FORMAT commands have been issued:

Before the formats are set, the values of the variables are null.

E.2.3.12  $FORMATID and $GLOFORMATID

The string variable $FORMATID contains the ID of the currently set format (that is, the name given in the ID statement of the format definition). It is null if no format is set.

If the currently set format is a global format [See D.2.5.] then the format ID is contained in the $GLOFORMATID variable.

E.2.3.13  $FRAME

The string variable $FRAME contains the name of a specific named frame, or it is null if no specific frame is named. Only the "USING frame" command prefix, and XEQ FRAME command cause this variable to be non-null.

:

:29  SPIRES Documentation

I. Primers

II. User Language

III. Application Development