Database Programmer's Toolkit

Language Reference and Compatibility Guide

Contents

• Notation Conventions in this Document
  
  
• Part 1: Commands
• Part 2: User Language
• Part 3: Sirius Features

Notation Conventions in this Document

In simple cases, the common backus-naur style symbols are used to denote the required syntax of commands and statements, similar to the Model 204 manuals, viz:

• Optional elements are enclosed in square brackets
  TIME [REQUEST]
• Alternatives are separated with the "|" sign, with any default underlined
  CREATE [TEMP | PERM] GROUP groupname
• Compulsory elements are enclosed in curly brackets
  BUMP {user-number | user-name | ALL}
• Keywords are in capitals, variable components are in lowercase, and single characters are sometimes enclosed in quotes where there might be confusion over whether it's part of the notation or part of the command.
  MONITOR ["(" user-number ")"]
• Options which are accepted for readability reasons but do not have an effect they might be expected to have are in italics.
  BROADCAST [URGENT] [message]

Part 1: Commands

• General Compatibility and Processing Notes
• Individual M204 Commands - Processing and Syntax Differences
• DPT Custom Commands

General Command Compatibility and Processing Notes

• Line continuations
• IN filename clauses
• Quote characters in commands on DPT
• Custom options to standard commands

Line continuations
All command lines can be continued with a hyphen on DPT. This differs from M204, where only some commands can be continued.

IN filename
As with User Language, IN clauses can not include the "AT location" phrase, since there are no scattered groups on DPT.

Quote characters in commands on DPT
Some commands take options which can be surrounded by quotes. On Model 204 the only accepted quote character anywhere (including User Language) is the single-quote, and this is respected. However, with some commands that interact with the operating system, the double-quote character is also allowed by DPT since it is commonly used, sometimes even compulsory, on many platforms. For example:

ALLOCATE F1 'MY FILES/F1.DPT'    /? usual M204 convention        ?/
ALLOCATE F1 "MY FILES/F1.DPT"    /? equivalent on DPT            ?/
ALLOCATE F1 'MY FILES/F1.DPT"    /? no good - must not mix types ?/

=MIME DEFINE .txt 'text/plain; charset="UTF-8" '
=MIME DEFINE .txt "text/plain; charset=""UTF-8"" "   /? equivalent but less neat ?/

Custom options
Some of the M204 commands have DPT custom options. Many of these can be disabled using the CSTFLAGS parameter, but a few are essential for system operation and cannot be disabled. In any case all custom options are covered in the by-command discussions below.

Individual M204 Commands - Processing and Syntax Differences

• ALLOCATE filename {osfilename | TEMP} [DIRECT | SEQ[UENTIAL] | PROCDIR] [OLD | NEW | COND | MOD | READONLY] [LRECL [=] value [PAD [=] value]] [MAXSIZE [=] value] [ALTNAME [=] value] [PRSUFFIX [=] [.]extension]
• See usage notes in the DBA Guide for some comments re. the huge range of missing options compared to Model 204.
• Quotes are only required on the osfilename if it's a path containing spaces. The quotes may be single or double. The osfilename is passed to the OS exactly as specified, so can be, and often will be, just be a relative path to the host's working directory.
• MOD, READONLY, TEMP, LRECL, PAD and MAXSIZE are only valid on sequential files. Some are DPT custom options, covered here in the DBA guide.
• The default disp is OLD and not NEW like on M204.
• With PROCDIR you must, naturally enough, specify a directory and not a file.
• ALTNAME and PRSUFFIX are special options for use with proc directories.
• To specify a null string for PRSUFFIX, use a dot, as you would when resetting the global PRSUFFIX parameter.

• ANALYZE [(DESCENDING)] [[LEAF]PAGES] [fieldname]
• This command reports information about the b-trees and inverted lists maintained for indexed fields, as on Model 204. There are some custom extensions to the command which add functionality, as covered below.
• Firstly the basic command. This is the same as on Model 204, with the difference in output format really just reflecting differences in low-level details of the index data structures.
• PAGES and LEAFPAGES are DPT custom options which cause a different and very interesting report to be generated. Information is shown for every node in the b-tree, giving a comprehensive picture of current node utilization: space available, range of values present, the active compression level, etc. For string nodes where prefix compression is active, the prefix values not actually stored are shown in brackets. LEAFPAGES is the same as PAGES except that nodes below leaf level are not shown.
• The DPT custom DESCENDING bracketed option does not affect the output but merely tells the system to perform the analysis starting at the high end of the b-tree. This option is really for technical diagnostic purposes, to ensure that descending-cursor processing such as used by descending value loops, works correctly.
• The PAGES option could be useful for anyone on a file-management training course or learning about b-trees for the first time, especially when used in conjunction with the custom BTREEPAD file parameter, which is mainly there for such purposes. Setting this parameter to a non-zero value causes DPT to think b-tree pages are fuller than they are, and you can make node splits occur at will, observing the effects on the tree structure, string compression, BX**** statistics produced by UL requests, etc.
• Likewise see the custom =ILINFO command later.

• CHECKPOINT [cpto-override]
• The difference from Model 204 here is that you can supply an override to the CPTO parameter. The default (-2) is to use the current value of CPTO. A value of -1 means abort immediately if quiesce fails.
• See the System Config Guide for more on checkpointing and CPTO.
• Most commonly this command would be issued periodically by a system daemon (see =PSTCHECK).

• COPY [DATASET | STREAM] dd1 TO [DATASET | STREAM] dd2 [COUNT=n | UNFORMATTED]
• The STREAM and DATASET keywords are optional and interchangeable. Using one or the other does not affect the operation of the command at all.
• Both files must be allocated as SEQUENTIAL.
• If the DPT custom option UNFORMATTED is specified, the file copy is performed a great deal faster, in 64K blocks, but any LRECL, RECFM and MAXSIZE specifications given on the two files at ALLOCATE time are ignored. If the destination file is allocated as MOD, this is respected. COUNT=n is incompatible with the UNFORMATTED option.
• Without the UNFORMATTED option, data is copied a "record" at a time, obeying the allocation specifications of the two files, truncating and padding the records as necessary and stopping when either COUNT records have been processed, or DD2's MAXSIZE is reached.
• A per-record copy involves a lot of processing overhead, and if you know the two files have the same format can be a waste of time. (For interest, the system takes this view when performing COPY PROC - see below - and does an unformatted-style copy of one proc to the other).
• See also the custom command =SEQREAD.

• COPY PROC procname1 TO context-specification [REPLACE] [OLDDATE] [NEWNAME=procname]
• There is no ALIAS option, since we are limited by the information maintained for files be the OS.
• On which subject, if you use the OLDDATE option on an OS (e.g Windows) which maintains file created-time, you can easily create a situation where a proc appears to have been modified before it was created.
• For multiple copies (the LIKE option is also unsupported) OS facilities are probably more appropriate - e.g. a command-line COPY or Windows Explorer drag-and-drop. You can easily SHELL a COPY command.
• To copy to a temporary procedure use e.g. COPY PROC -1 TO TEMPPROC NEWNAME -4. I.e. 'TEMPPROC' stands in for the destination file name.

• CREATE [FILE] filename
• ... parameters ...
• END
• There is no option to create a file from multiple OS files like with M204, hence no FROM option.
• The FORMAT/NOFORMAT option is not provided.
• The supported file organisations are entry order (FILEORG=0) and unordered, which must also be RRN (FILEORG=X'24').
• Only one parameter may be specified per line (i.e. much like "CCAIN"). Of course semicolons can be used to denote newlines if desired for readability.
• Usage notes.

• CREATE [TEMP|PERM] GROUP groupname FROM f1 [[,] f2...]
• ... parameters ...
• END
• OPTIONAL/MANDATORY only applies to PQO on M204, and is not valid on DPT.
• All groups are potentially procedure file groups, so the PROCFILE parameter is not valid (it is always effectively "*").
• BLDGFT is always effectively YES, so that parameter is not valid either.
• All the security-related parameters are invalid.
• In short, the only parameter you can specify is UPDTFILE. If this is not specified, you won't be able to use STORE RECORD in group context, and $UPDATE will not work either, (but IN GROUP MEMBER %xxx and IN $CURFILE will work as normal).
• See the DBA Guide for more on groups, and for more on procedure processing in group context.

• DEFINE FIELD fieldname [(attribute [[,] attribute...])]
• DEFINE FIELD fieldname WITH [attribute [[,] attribute...]]

• The rules for valid field names are pretty much the same as on Model 204. Specifically they must begin with a capital letter and be from 1 to 255 characters long. (Note that the M204 manuals are not totally accurate in this area - DPT aims to agree with actual behaviour). See also the section on using of reserved words in field names.
• An additional DPT-specific restriction is that field names containing the ASCII characters 0-31 are disallowed, even if quoted. The issue of ASCII control characters is discussed elsewhere.
• Field attributes: In all cases the equivalent M204 abbreviations can be used, or the full keywords. Only the following attributes are allowed.

• Storage attributes: [STRING | FLOAT | INVISIBLE]. FLOAT always means FLOAT LEN 8, namely a floating point value as per the usual UL rules (see UL number handling).
• FLOAT and STRING fields can be: [UPDATE IN PLACE | UPDATE AT END]
• STRING fields can also be: [BLOB | NBLOB]
• FLOAT and ORD NUM fields will not accept non-numeric values like Model 204 sometimes can (FILEMODL allowing). They are either rejected or converted to zero.

• Indexing attributes: [NORD | ORD CHAR | ORD NUM]. One of the latter is required if the field is invisible.
• Model 204 disallows the combination FLOAT + ORD CHAR, but you can do it on DPT if you really must.
• The only M204 b-tree control parameter for ORDERED fields is [SPLITPCT[=]S] where S defaults to 50. Use 99 when loading a file which will be read-only.
• DPT also has a custom field attribute NO MERGE (abbreviation NM) which affects the single-step load process. This attribute is one that can be REDEFINEd, in which case use "DU MERGE" to revert to the default.

• D[ISPLAY] FIELD {etc...all M204 options}
• The main difference with this command from Model 204 is that it is allowed in group context. The group field table is reported as if it were a normal file. Field attributes that can differ across files in a group are shown as [grp] rather than a normal value.

• D[ISPLAY] FILE {[(option [[,] option...])] filename [[,] filename...] | ALL | ALLG}
• This command on DPT has to cope with the fact that the user may have two types of “files” open, namely procedure directories and data files. The display can be limited to one or the other by using the new options PROC and DB. The default is to show both, and using either option turns off the other one.
• For data files, the PARMS option shows all parameters and values, (amalgamating the FPARMS and TABLES parameter categories), whereas for procedure directories it shows the number of procedures in the directory. In both cases the OS file path is shown.
• In both cases, the custom option GROUPINFO shows information about whether the file is open in single file and/or group context. FULL is a shorter way of specifying both PARMS and GROUPINFO.
• Specifying ALL instead of a list of file names shows all open files and/or directories. The custom option ALLG includes ALL, but also shows a list of the user’s open groups.
• To see a list of all files allocated to the system, but not necessarily open, see =LISTALC.
• When a proc directory has an ALTNAME, the file is listed by this command under the name it was opened as, which may be either the real DD name or the ALTNAME.

• D[ISPLAY] [TEMP | PERM] GROUP {groupname [[,] groupname...] | ALL | ALL2}
• No bracketed display options are allowed with this command on DPT (NAMES/DEFINITIONS). The display is always as per DEFINITIONS on M204 (the default).
• There is also a DPT custom option to include ad-hoc groups (which might be hanging around after END MORE). These are otherwise suppressed from the display - use ALL2 instead of ALL to include them.

• D[ISPLAY] GTBL {LIST [LIKE pattern] | [NAME=] globalname [[,] globalname...]}
• This is pretty much the same, except that since the globals always come out in alphabetical order with LIST, there is no IN ORDER option.
• The global names may be surrounded with single quotes to cater for names which might contain spaces, commas or the words LIST, LIKE and NAME.

• D[ISPLAY] GTBL OBJECTS
• This is a custom variation of the command which lists all the global objects currently in the global table. The report shows each object's name, type and permanence, plus a small amount of object-specific information such as for example with an image whether it's defined or just declared.

• D[ISPLAY] [PROC[EDURE]] [(option [[,] option...])] LIST [ALL | TEMP | LIKE pattern]
• Of the M204 options, BRIEF, COMPACT or VERBOSE are supported, and ALIAS, HDR and NOUSE are not. HDR is always on for BRIEF and VERBOSE. VERBOSE is the default, as on M204.
• Only files with an extension equal to the current setting of PRSUFFIX are shown, unless PRSUFFIX is set to a null string (C''), in which case all files are shown.

• D[ISPLAY] [PROC[EDURE]] [(option [[,] option...])] {[NAME [=]] {name|number} [[,] {name|number}...] | ALL | TEMP}
• The only option supported on the "text" form of the command is LABEL. ALIAS, COMPACT, NOUSE, NUMBER and PRIVILEGES are unsupported for reasons outlined elsewhere. The display is always effectively COMPACT, since most commands do not invoke output paging on DPT. With LABEL, the IN FILENAME clause prefixes the procedure name only in group context.

• TEMP is a DPT custom alternative to ALL on both flavours of the command, and displays temporary procedures, starting at 0 and working backwards.

• EDIT [ (ASYNCH | SYNCH) ] procname
• This command is only allowed on the IODev7 client.
• No second "destination" procedure name can be specified. Users sometimes do this on M204 to effect a procedure copy programmatically - use COPY PROC instead then. Otherwise use "Host Save Copy" in the editor.
• A more important difference, especially when using EDIT programmatically, is the ability to specify synchronous or asynchronous processing of the command, and this option is a custom extension. The default is ASYNCH when EDIT is issued from the command line and SYNCH when EDIT is issued from inside a procedure. ASYNCH is invalid from within a procedure, since the editor needs access to the command line to work.
• If the procedure is already being edited on the IODev7 client, focus is simply switched to it. In such cases the existing edit must have been ASYNCH, but if the new command specifies SYNCH, this is respected.
• Since there is no line editor, the Model 204 TERM[INAL] and LINE options are redundant. TERM and SCREEN are ignored, LINE is disallowed.
• Editing issues are convered in more detail in the IDE User Guide.

• FREE filename
• The difference here is that discontinuities are never allowed, so FREEing a file is not allowed if a file has been updated but not checkpointed. This restriction is implemented by simply taking an implied checkpoint as part of the FREE command. This is unlike a normal checkpoint in that it "times out" if all updates can not be quiesced immediately. This is similar to issuing "CHECKPOINT -1" (see above).

• HALT
• There is no need to specify a console message or the desired reply on DPT, and those options are disallowed.
• The System Config Guide gives full details on starting and stopping the system.

Nearly all the supported options have a wider range of allowable values than on Model 204, as follows.

• First line:
• LOGCTL {A | C | D} userid
• No VM/CMS options (P etc.) are supported.
• File (:) and group (,) variations of the command are not supported.
• User IDs can be up to 30 characters long on DPT.
• Second line:
• password, privileges
• Priority is not supported.
• On DPT passwords can be any non-zero-length string.
• Privileges are held as a 32-bit number, unlike Model 204's 8 bits. You can use the extra bits for any custom processing.
• The privileges can be given in decimal form, such as 255, or hex form, when X'FF' would be the equivalent.
• Third line:
• Terminal list is not supported, so DPT will not even request a third line.

• LOGKEY
• This command is supported only for the purpose of reporting the user ID and time of the last update of the password file.
• No "key" is actually stored anywhere so there is no parameter on the command.
• See also the System Config Guide section on this.

• LOGWHO [option [[,] option...]]
• This command shows both less and more than the equivalent M204 command.
• The "more" is with the options, which are all custom extensions. The command can take any combination of the options NOPROC, NODB and DEVICE.
• NOPROC means *don't* show the user's open procedure directories. The default is to show them, and they will be indistinguishable from data files (see below). Files which are open in both "ways" only feature once in the output.
• NODB means *don't* show the user's open files. The default is to show them. They are displayed in the same way as on M204, namely that if a file is only open as part of a group, there is an asterisk after the name. Files open in their own right as well as via a group do not have the asterisk.
• The DEVICE option means show the values the user has for the parameters IODEV, INPUT and OUTPUT. The default is not to show them.
• These are the only things that LOGWHO shows on DPT. In other words there is no information about dead terminals etc., as described in the M204 Command Reference Manual.

• MONITOR [subcommand-type-and-options] [[EVERY] n]
• In other words all monitor commands can be given an EVERY value, which makes the display repeat after that number of seconds. If omitted, a single display is generated. This is much the same as on M204, in that on file-based devices like User 0 or a daemon, the repeated display forms an ever-lengthening report until the thread is bumped, and on an interactive device like the DPT client, the user has to press enter to get the next display.
• On the DPT client (IODev7) a useful option is to get the front end to "press enter" automatically at these new page prompts, so you can get a self-updating monitor display.
• The default if no subcommand is given is USERS. That is, something similar to Model 204 (see below for more details).

• MONITOR [SYS[TEM] | USER[S] | FSTAT] [STATS]
• If the keyword STATS is also given it means present the statistics in an unformatted line as per the audit trail. The default is to present a subset of those statistics in a formatted columnar report. Specifying STATS therefore gives more information, but is less readable in the most common cases where CPU, DKPR etc. are of interest.
• Examples:
• MONITOR
• MONITOR USERS (same meaning as MONITOR)
• MONITOR FSTAT STATS EVERY 5 (repeating unformatted file statistics)

• MONITOR [USER[S]] [(usernum [,usernum]...)] {SL | FILE[S]}
• The SL option means show since-last stats instead of logout stats.
• When displaying SL stats, each user's current activity and/or procedure name is shown, if known.
• The default is to show all users, but a list of user numbers can be given just like on M204.
• The FILE option actually generates a display much like LOGWHO, and perhaps should not be a MONITOR command at all. Just copying Model 204 though.
• Note that in the formatted report, the column showing "Thread ID" is sometimes referred to by low-level DPT diagnostics. See the system configuration guide for more details on thread scheduling on DPT.
• Likewise, the WT/FLGS column shows estimated "nearest equivalents" to the M204 values. Because of the total difference in scheduling philosophy on DPT, these can not always, if ever, mean exactly the same thing. . In some cases, the custom command MONITOR RESOURCE (see later) will give more detailed information than WT/FLGS on who is holding and waiting for resources.
• Examples:
• MONITOR USER SL STATS EVERY 5 (repeating unformatted since-last stats for all users)
• MONITOR (5,6) (formatted logout stats just for those users)
• MONITOR FILES (open files for each user)

• MONITOR FSTAT [(filename [,filename]...)]
• With the obvious meaning.

• MONITOR {CHECKPOINT | CHKP}
• This command shows the same information as on M204, plus the information that M204 would present in response to the CHKMSG command.
• When a checkpoint is currently being inhibited, only the user threads are shown, not the procedure names they are executing. Use MONITOR SL (see below) to get that information if desired.
• The "extended quiesce" information alluded to in the M204 manuals is irrelevant on DPT, as checkpointing is somewhat simpler.

• MONITOR CFR
• This is implemented using the more general custom MONITOR RESOURCE command.

• MONITOR DISKBUFF
• Similar to M204 (more detail actually).
• There is also a custom command, =BUFFDUMP, which gives a different lower-level spin on the buffer contents.

• MONITOR SUBSYS
• This is implemented using the more general custom MONITOR RESOURCE command.

• MSGCTL DPT.number [TERM | NOTERM] [AUDIT | NOAUDIT] [CLASS[=]{E|I}] [RETCODE[O|B]=number]
• Sets up a user-specific override to the system settings for the message. These are the only options supported.
• Any options not specified on the command are left as they are.
• TERM and NOTERM can effectively be managed for all messages at once using the MSGCTL parameter, which is user-specific.
• The CLASS setting (E) controls what is considered a "counting error" for all purposes, namely the aborting of a compilation after ERMX; whether to run a request at the end of compilation; invocation of subsystem error procedures (CNT); retaining message text for access via $FSTERR, $ERRMSG and VIEW ERRORS. As with M204, the error count is zeroed whenever the user returns to command level, at the start of a new or continued request at any include level, and when a new subsystem procedure is included.
• There is no difference between "batch" and "online" processing in the current release of DPT, so the RETCODEO and RETCODEB options are interchangeable. Just RETCODE can also be used.
• Remember that DPT's message codes are not the same as on Model 204, so any MSGCTL commands in ported applications will need attention.
• The message with the highest RETCODE setting issued during a run is the return code that's set by the host application when it ends. All messages issued to the audit trail that have non-zero a RETCODE setting are highlighted as such to make them easier to find. A message need not be issued to the audit trail to set a non-zero return code however, (and also need not be issued to the user terminal, or even be set as an error).
• User-specific message code high-water-marks are also kept, for use in some situations. For example the return value from synchronous $SPAWN and to control the web server automatic error processing.

• OPEN file-context
• OPEN file-context, ddn, dda [, num-pad-length [, format-options]]
• OPEN file-context, 1STEPLOAD
  
  
• General notes:
• With the plain command, the file context can be a group or single file context, as on M204. With the parameterized (deferred update) variations the context must be a single file.
• This command will open both a procedure directory context and a data file context, if there is one of each with the supplied name. The command is only considered to have failed if both these attempts fail.
  
  
• Format 2 for multi-step deferred updates:
• The optional parameters are the deferred update file names which need not begin with "TAPE" on DPT. If these are specified, the context may not be a group. Both file names are required, and both must be sequential files that are currrently allocated. {Note about defer DD names: The Z command which will be required later assumes the names "TAPEA" and "TAPEN", so it's simplest to just use those. Unless you want to run several files at the same time with deferred updates turned on.}
• You should not specify LRECL on the allocations for these files, as that could truncate the deferred update records and thereby corrupt the indexes.
• The optional "num-pad-length" parameter affects the format of the deferred update records for ORD NUM fields, if there are any. Any positive value here causes these numeric values to be stringized for the sort and padded on the left (i.e. right-justified) to the specified length, rather than the default which is floating point representation (see also NOPAD option below). Field values encountered which exceed the specified length cause load failure. Try to give the lowest value possible when using this option in order to save disk I/O during the sort. At the end of a load DPT issues a message saying the longest value encountered, so you can reduce the allowance next time if appropriate.
• Further formatting options for the deferred update sequential files are keywords and can be added, separated by commas, after the above 3 values. Accepted keywords are:
  • NOCRLF:
    With the variable-lengthed formats (ORD CHAR or stringized ORD NUM) field values are preceded with a length byte instead of using a CRLF separator at the end of the record. Allows X'0D' and X'0A' in values, at the cost of a more unconventional record format.
  • NOPAD:
    Stringized ORD NUM values are written unpadded, which saves disk I/O if your sort utulity can handle numeric string values laid out like this. The NOPAD option triggers the use of stringized format for the ORD NUM values, in the same way as does a positive value for the "num-pad-length" parameter above. If both are given, the pad length is ignored.
  
  
• Format 3 for one-step deferred updates:
• After the file name the keyword "1STEPLOAD" must be given.
• The values of the LOADMEMP and LOADDIAG parameters at the time of issuing the OPEN command are used throughout the deferred update processing.
• Note: the two optional parameters that used to go with this command are no longer valid - their job is done by the above LOADxxxx parameters.

Examples

*One-step:
OPEN SALES, 1STEPLOAD                        //Default
OPEN SALES, 1STEPLOAD, 50%                   //Invalid from V2.19 - use LOADMEMP parameter

*Multi-step:
OPEN SALES, TAPEN, TAPEA                     //use default formats
OPEN SALES, TAPEN, TAPEA, 6                  //stringize numbers - pad to length 6
OPEN SALES, TAPEN, TAPEA, , NOPAD            //stringize unpadded - length irrelevant
OPEN SALES, TAPEN, TAPEA, , NOCRLF           //binary format numbers, length byte strings
OPEN SALES, TAPEN, TAPEA, , NOPAD, NOCRLF    //length byte on both strings and stringized numbers

• PROCEDURE procname
• Aliases and security classes are not supported, so those options are disallowed.
• This command is not allowed in group context, because of the differences in procedure handling from M204.
• The rules for valid proc names also differ slightly.

REDEFINE FIELD

• REDEFINE FIELD {options as per define field}
• The following do not require structural changes to existing data, and redefining them takes next to no time. Any number of these options can be redefined on the same command.
  • UP/UE for visible fields
  • SPLITPCT for ordered fields
  • [NO] MERGE for ordered fields (DPT custom option)
• The following involve changes to information throughout the file and may take a long time to perform. Only one of these changes may be performed per command.
  • FLOAT to STRING or vice versa (reads all of table B, possibly writes all too, probably slow)
  • FLOAT or STRING to INVISIBLE (as above)
  • NBLOB to BLOB or vice versa (as above, plus probably table E work too, so even slower)
  • INVISIBLE to FLOAT or STRING (reads entire index, retrieves and writes to perhaps every record, perhaps several times, probably very slow)
  • ORDERED to NON-ORDERED (quickish - just freeing up pages in table D)
  • ORD CHAR to ORD NUM or vice versa (extracts existing b-tree, resorts and builds a new one, inverted lists are touched little if at all, shouldn't take too long)
  • NON-ORDERED to ORDERED (reads all of table B, builds b-tree and inverted lists, probably quite slow)
  
  
• Other notes:
• The processing takes place as a single, non-backoutable update, so for non-trivial changes a file backup beforehand is a good idea.
• When changing the storage or index format from string to numeric, or adding an ORD NUM index to a STRING field, there might be issues with number validity, possibly causing REDEFINE to fail, in which case the file will be left physically inconsistent.
• Likewise, when changing a BLOB STRING field to a plain STRING field, if any value is longer than 255 bytes, the command will fail in the same way. STRING to BLOB will cause no such problems of course.
• Any change between STRING and BLOB will be moving a potentially large amount of data between table B and table E (table D), so it's a good idea to check their sizes beforehand and INCREASE if appropriate.
• SEQOPT will often be beneficial to this type of processing, and is worth experimenting with, especially for the commands which walk through table B (NON-ORDERED to ORDERED, any table B format change). {Note: incidentally, the same consideration applies to DELETE FIELD.}
• The final option above (create a new index from scratch) works under the covers by using the same fast index-build code used in the single-step deferred update process, and many of the same discussion points regarding memory chunks etc. apply. Likewise some of the internal processing can be affected by the LOADxxxx group of parameters.
• In cases where several changes are to be made, a reorg (i.e. unload and freshly reload the whole file) might save a lot of IO etc. compared to multiple passes over the same data pages with several REDEFINE commands.

REORGANIZE

The other main difference from M204's REORG OI command is that on DPT the unload and the load phases of the reorg happen with a single invocation of the command, whereas on M204 output files are created that must be sorted and then loaded by the Z command. The DPT scheme saves all the hassle of configuring JCL and tracking separate job steps at run time.

A third, more low-level, difference is that on Model 204 the output of the REORG IO command consists of sequential files identical to those from a deferred update run. On DPT the intermediate sequential files are a special custom format used by this command. The older deferred update file format is completely different.

DBAs often want to make file or field changes as part of a full-file reorg (see recipes in the DBA guide), and in such cases using explicit =UNLOAD and =LOAD commands with an intervening INITIALIZE gives greater control than a single REORGANIZE command. The underlying processing is virtually identical, but it may be slightly slower overall because DPT is given less chance to take shortcuts, cache data in memory, etc. For example when using separate unload+load, the intermediate data will always be written out to disk, even for a smallish file which would all fit in RAM.

• REORG[ANIZE] [OI [[FOR] fieldname]]
• If the OI keyword is not given, the entire file is reorganized, including the data area (table B) too.
• The OI keyword with no field name means just reorganize the indexes (don't touch table B), and do it for all indexed fields.
• Giving a field name further restricts processing to that field's index. The keyword "FOR" is optional and does nothing. To process more than one field, but not all, issue a separate command for each.
• There is no way to just do a table B reorg on a file which also has indexes. (Doing so would usually involve lots of changes to inverted lists that would make an index reorg advisable anyway).
• Other notes:
  • The table B part of a reorg results in efficiently-packed records as per the new file parameters and field definitions. Free record slots are reclaimed, etc., etc., etc. Note that internal record numbers may well change during this, although record order is preserved.
  • Reorganizing field indexes reloads their b-trees according to the current SPLITPCT value, and repacks inverted lists (bitmaps etc.) appropriately according to the current population density of the field values.
  • A full-file reorg has an implicit INITIALIZE at the halfway stage, before reloading, which includes the table-defragmentation processing invoked by the INITIALIZE command.
  • In most ways the underlying operation of this command is the same as if you'd issued separate =UNLOAD and =LOAD commands, so those doc sections should be consulted for detailed processing notes.
  • Any temporary files generated are placed in the #REORG directory, which is cleaned up automatically.

• R[ESET] COMMAND commandname USE=use-option
• The USE option can be one of RESPECT, IGNORE, NOCLOSE, NOOPEN or NEVER
• The privilege option is not allowed, as there are no proc access privileges on DPT.
• Another way this command may differ from M204 is that the command name must be given in its "canonical form", for example "DISPLAY FIELD" rather than any abbreviation "D FIELD". The VIEW COMMAND ALL ALIASES command produces a listing of all the command canonical forms and aliases for each. Note that in some cases such as MONITOR and DISPLAY, different flavours of the command are treated as different commands for the purposes of their USE behaviour.

• USE $EMAIL [WITH] [options]...
• See examples in DEMOPROC.
• This mechanism is provided on DPT to be somewhat similar to the "USE PRINTER ... WRITER=SMTP" scheme used on Model 204.
• The body of the message will be made up of the next item of output, as per a regular USE command.
• The special USE command options SUBJECT, TO, FROM, OPTIONS and DIAGS can be given in any order. They match those on the User Language $EMAIL function and their meaning is discussed there, along with some more general notes on the email facility currently available on DPT.
• Blat *does* have a facility to insert custom SMTP header lines into emails, but it does not work the same as you may be familiar with from M204, and existing code of that kind will need alteration to work. DPT provides a workaround to keep things roughly familiar though: While the USE $EMAIL is active, starting any print line with "^USE^ " followed by the option as it would appear on the USE command causes that line *not* to appear in the message but set an override to whatever was given when issuing the USE command. You can set all the options this way if you like:
  PRINT '^USE^ SUBJECT = ' WITH %SUBJECT
• Remember that an email is not actually sent until the complete message text is ready, so in many cases if there are going to be problems with the combination of options, the error will not be flagged up until the "NUMBER OF UDD LINES OUTPUT" message is issued.
• As ever, if parameter values contain spaces, enclose them in quotes. Sometimes you may need quotes within the quotes. The inner quotes in that case should be double (DOS-style) quotes. For example
  USE $EMAIL WITH OPTIONS = '-ATTACH "C:\\A PICTURE.JPG"'
• Note also we had to use a double backslash there to keep Blat happy.

• V[IEW] [parameter | parameter-set] [[,]...]
• The only real difference in this command is the range of parameters and parameter sets that can be used, and which parameters fall into which ranges. See the Master M204 X-reference tables for details on valid parameters.
• The valid parameter sets are ALL (the default if just VIEW is specified), ERRORS, FPARMS, TABLES, FILE (= FPARMS+TABLES), SYSTEM, USER and UNSUPPORTED.
• This this means that CWAIT, STORCUR, STORINIT, STORMAX and UTABLE are not supported.
• ALL does not include the "UNSUPPORTED" group.
• DPT maintains lookup table entries for the "UNSUPPORTED" group so that for example if you VIEW MAXBUF it can give you its lame excuse for not using that parameter, and in some cases suggest a similar custom alternative.
• This group is not comprehensive, since M204 is always adding new parameters.

• V[IEW] COMMAND {commandname | ALL | UNSUPPORTED} [ALIAS[ES]]
• This command has a couple of extra options over M204
• The ALL option lists all commands that there is a DPT equivalent for, (or a rough equivalent anyway). The UNSUPPORTED option lists those M204 commands that aren't supported at all.
• As with parameters (see above) the UNSUPPORTED group is not necessarily comprehensive.
• The ALIAS option shows any aliases, abbreviations or variations of the command. All the aliases grouped under the same "primary" form of the command will be affected by using RESET COMMAND on the primary form.

Loads post-sort deferred intex updates for a file, as per the Model 204 command. The file whose indexes are to be loaded must be open, and should have been opened with a "basic" OPEN command, that is without specifying the TAPEx DD names. See the load job in the DPT download for an example.

The index load process performed by this command works by collecting deferred updates for each field/value into a bitmap in memory before storing them in a single trip to the index on disk. This contrasts with early DPT versions where repeated index retrievals and bit-flippings were performed. With most types of field this in-memory method has a dramatic effect on load times. It can be thought of as performing a small number of User Language "FILE RECORDS UNDER" statements rather than a large number of "ADD" statements, although that is not quite accurate as FILE RECORDS would replace any existing set, whereas Z augments it.

The improvement is only effective if the input file is well sorted, and DPT provides a command option which can help ensure that is the case.

See also the DBA guide for more about loading files,

• Z [forgivingness]
• The forgivingness option causes the load to fail if the input file is not "sufficiently" sorted.
• A value of -1 means infinitely forgiving and the input file can be in any order for the command still to work.
• A value of zero means totally unforgiving. Higher numerical values specify how many input records are allowed to be out of order before abandoning the load. So for example you may have an input file with a few awkward or missing values that your sort utility cannot completely handle, but that you still wanted loaded for some reason.
• The default is zero, enforcing complete sorting. That is the ideal position that you should start from, and then slacken off if necessary.

DPT Custom Commands

Many of these commands have been left in the system from when they were used during development and testing, like the custom options on actual M204 commands discussed above. They have been left in in case somebody finds them useful. The discussion (given below the table) is sketchy in some cases - further details are available if anything sounds interesting and you'd like to know more.

Most are prefixed with equals signs, to prevent conflict with existing APSY names ('=' can not be used in APSY names). It has been suggested that asterisk would have been a better choice and maybe that's true. In any case if M204 compatibility is an issue most of them can be disabled using the CSTFLAGS parameter.

List of custom commands (usage notes follow for many)

Notes about some custom commands

=BUFFTIDY

• =BUFFTIDY [age]
• The parameter given simulates the BUFAGE parameter, and if omitted the value of the actual BUFAGE parameter is used. Note therefore that in the common case where BUFAGE is zero, meaning no pseudo-user is started, and you are using the command occasionally by hand, every deletable buffer page would get deleted by the command with no parameter. This is probably what you want - to free up the maximum amount of physical memory.

CREATE SUBSYS

• CREATE SUBSYS subsysname
• ... parameters in any order ...
• END
• Parameters specified as PARAMETER name[=]value, as with CREATE FILE etc. They're not viewable parameters as such, which raises the question of whether the word PARAMETER is appropriate. It was decided to stick with it, as it's also used in the same way in CREATE GROUP (e.g. UPDTFILE is not a viewable parameter either).
• The command can be aborted with *CANCEL.
• The following parameters are COMPULSORY
• LOGPROC: the login procedure name
• COMMGLOB: communications global
• EXITVAL: exit value of COMMGLOB
• FILES: a comma separated string of file or group names, which will be opened, in order, when invoking the apsy, and closed when exiting. If there are temporary or permanent groups they will be opened in preference to files, as per normal OPEN and CLOSE commands (there are no special considerations for temporary groups). The default context, and the one from which all the procedures are expected to come, is the first one specified in this list (as per SUBSYSMGMT on M204), and therefore must name (at least) a procedure directory. All opens must be successful for the subsystem to be used. If any can't be opened, those already opened are closed, and the user is returned to command level. This is another way of saying that all files are "mandatory" to the subsystem, which also has the effect that CLOSE commands issued for any of these files from one of the subsystem procedures are disallowed. At subsystem exit, the files are closed in reverse order, even if they were open normally before the apsy was entered (this agrees with M204 behaviour).
• The following parameters are OPTIONAL
• ERRGLOB: error global name. There is no default.
• ERRPROC: the error procedure. This has one or two special considerations on DPT. The default is for there to be no error procedure, which means that situations where it would be invoked instead result in the user leaving the subsystem.
• CMDGLOB: command line global (no default - extra text after the subsystem name is ignored)
• MSGERR and MSGINFO: These cause the MSGCTL parameter to be set at the start of execution of the login proc, and to be set back to its previous value when the thread leaves the sybsystem. Specify Y to show messages and anything else non-blank to suppress them in both cases. (Although there's nothing to stop one of the procedures in the apsy setting MSGCTL to something else during processing, to turn messaging back on again). The defaults for these are both Y (i.e. all messaging is active, even if MSGCTL was set to turn it off before the apsy was invoked).
• MSGDISC: whether a message is issued saying that the user has left the subsystem. The default is Y.
• MAXITER: maximum iterations of the same proc in a row before it is considered an error situation, and the error proc included with error global value "INCLUDE MAX".
• AUTOCOMMIT: Whether the system should issue a DBMS transaction commit when chaining to the next procedure. It can be Y or N. The default is Y.
• LOGOUT: Whether the user is automatically logged out of DPT after they leave the subsystem. Does not prevent XFER to another subsystem. Often used in conjunction with AUTOSYS. See also related options on the $SUBSYS function. It can be Y or N. The default is N.

DISPLAY AUDIT

This command is particularly useful when DPT is running in NOGUI mode, and/or the user is working on a remote machine from the host. In the first case the simple alternative is notepad, and in the second case there may be no simple alternative (short of writing your own audit file scan program using e.g. $FREAD and $TOKENIZE).

• D[ISPLAY] AUDIT [FROM[=]timevalue] [TO[=]timevalue]] [LIKE[=]pattern [CASE]] [MAX[LINES][=]n]
• All parameters are optional. See below for parameter format examples.
• Time and pattern selection criteria are "AND"ed together, so lines must pass all given tests to make it to the final report.
• Time selection:
  • If no FROM is given, the start of the file is assumed. If no TO is given, the current end of the file is assumed.
  • FROM and TO can be given as a minus sign followed by some number of seconds, meaning relative to the current time.
  • Alternatively you can give absolute time values, optionally including the date too. Each date+time value must be a single string containing no spaces, where the date and time components are as they appear in the audit trail. So that's YYYY/MM/DD for dates and HH:MM:SS for times. The date and time should either be concatenated together with a non-blank character, or if a blank is used, enclose the pair in quotes.
  • Shorter time specifications can also be given. Omitting the date means the current date is assumed. Leaving the MM and/or the SS off the time means 00 is assumed for either or both. So the shortest possible time specification is just HH for HH-o'clock today.
• Text pattern selection:
  • If no LIKE pattern is given, every line is selected.
  • The pattern is a normal M204-style pattern and is assumed to have a leading and trailing wildcard, so you don't have to specify those.
  • Audit trail line comparisons are not case-sensitive, unless the CASE option is given, when they are. (Entering mixed-case patterns would of course also require a prior *LOWER command unless the DISPLAY AUDIT command was in a procedure).
  • Patterns can be quoted. They must be quoted if they contain spaces or equals signs.
• Other notes:
  • MAXLINES stops the scan when a certain number of lines have been selected. (Default = 1000, -1 = no limit.)
  • If the audit file might be large it's best to give at least a FROM time, and even better a TO time. This gives DPT a chance to avoid scanning the whole file.
  • Large reports in any case run more quickly to USE destinations, for example USE PROC and then edit the proc.
  • Line selection from the audit file is dependent on the relevant information actually being present in the file, as partially controlled by the AUDCTL parameter. Specifically for example the scan on user number shown below will only work if the X'08' bit is set. In the case of date and time values, the command will tell you if it can't perform the scan, as will be the case for date-specific scans with the default audit trail layout, which does not show the date on every line (X'01' bit). In that situation scans with no explicit date selection component do work, but they assume the file all relates to the same day. For 24/7 DPT host runs, make sure to turn on AUDTCL=X'01' if you anticipate extensive use of this command.
  • The millisecond component of the audit trail time, if shown, is ignored.

D AUDIT                                         /? Show first 1000 lines             ?/
D AUDIT MAXLINES -1                             /? Show whole file                   ?/
D AUDIT FROM -300                               /? The last 5 minutes                ?/
D AUDIT LIKE ' 05 '                             /? Activity for user 5               ?/
D AUDIT LIKE ERR FROM 9 TO 10:25                /? Errors today from 9am to 10:25am  ?/
D AUDIT FROM 2010/09/10/18:37:12                /? Since a very specific time        ?/

=ECB

• =ECB [LIST]
• =ECB DISPLAY ecbname
• =ECB {TEXT | NUM | FLAG} ecbname new-value
• Plain =ECB prints a tabular listing showing all info held for all ECBs.
• =ECB DISPLAY ecbname displays the same information for just the requested ECB.
• The third form of the command changes the specified item to a new value, and prints the result (there is an implied DISPLAY).
• Text can be anything. Num must be an integer from 0 to 16777215 (as per $POST). Flag can be Y/N/POSTED/UNPOSTED/NOT POSTED.

=EMAIL

Consider using this command for short ad-hoc messages entered by hand, like where you might use the MSG command on Model 204.

• =EMAIL [options...] message
• The options are exactly the same as the USE $EMAIL command.
• Any text after the options is taken to be the message, unless you use the keyword "MESSAGE", in which case it is treated just like another option and can be in any order. So the following two commands do the same thing:
  =EMAIL MESSAGE HELLO TO THEQUEEN@PALACE.GOV
  =EMAIL TO THEQUEEN@PALACE.GOV HELLO
• Use pipe ("|") characters in the message to specify line breaks:
  =EMAIL TO ANATOLY@GMAIL.COM SUBJECT 'MY MOVE' P-K4|HOW IS THE WEATHER IN MOSCOW?

=LISTALC

• =LISTALC [ALL | DB | SYS | PROC | SEQ | PROCDIR]
• Entries shown for PROC refer to the actual procedures themselves, not directories. PROCDIR vice versa.
• This command is how the navigator populates its lists of proc directories and data files.

=LOAD

This command efficiently loads field descriptions and/or data and/or index information into a DPT file, so long as that information is presented in one of the acceptable formats. Typically that means that it was either generated during an earlier unload of the same file during a reorg, or from a Model 204 file on the mainframe.

• =LOAD [ALL INFORMATION] [(options)]
• The keywords ALL INFORMATION are completely cosmetic, and optional.
• The kind of information loaded by the command (field defs, data, indexes) is determined by what input files are present in the #FASTIO directory (or other specified directory - see options below). All input files are optional.
• The format of the incoming data is specified by options in the metadata header of input files themselves, and not specified on this command. The =UNLOAD command notes describe those formatting options.
• Bracketed load options:
  • Options can be separated with spaces and/or commas.
  • PREVIEW: Tells the command not to make any updates to the database file, but read through the input files and print them to the terminal (or USE destination) for the user to have a look at. This is handy because of the various ways in which the actual file contents might not be easily readable using standard file browser tools, especially if they're in EBCDIC. Numerical values are printed out as literals and EBCDIC translation is applied if appropriate.
    The optional subparameter says how many entries to print (fields/records/values/inverted lists). Zero (the default) means preview mode is not active, and the file is loaded. Just "PREVIEW" entered with no parameter is taken to mean "PREVIEW=10", since often you just want to take a glance. Any negative number or "ALL" means show everything (best if sent to a USE file or proc).
  • DIR: An override for the directory which will searched for the input files. e.g. DIR=SALESDATA, DIR="C:\TEMP", etc. The default is #FASTIO in the host working directory. Either single or double quotes should be used to surround the name if it contains spaces. There is no way to change the expected file names.
  • CLEANAFTER: Deletes the input files at the end if there were no errors. This can be convenient in routine situations.
  • NOACTION: Locates all eligible input files, but neither loads nor previews the contents.
  • CLEANONLY: The same as (NOACTION, CLEANAFTER).
• Other comments:
  • The context in which the command is issued must not be a group. Exclusive access is required to the file.
  • The process is a single, non-backoutable, bumpable, update unit. Any error (or bump) leaves the file physically inconsistent.
  • Minimal diagnostics are issued by default, but increasing the LOADDIAG to a higher value will probably be useful the first few times you use fast load.
  • Indexes are built under the covers using the same fast index-build code used in the single-step deferred update process, and many of the same points regarding memory chunks etc. apply. Likewise some of the internal processing can be affected by the LOADxxxx group of parameters.
  • Depending on the FMODLDPT parameter setting the =LOAD command might fail from data conversion errors if field definitions in the incoming data differ from those in the file.

Brief load processing overview:

Stage 1. Preparation

• Search for and open "file_TAPE*.DAT" input files.
• If index information is in a single file, this file is split into separate files, one per indexed field, in the same directory.

Stage 2. Loading field information:

• When field definition processing is performed, any field definitions present in the input file and not present in the database file are added. The simplest thing is often to issue an INITIALIZE command before the load and let the fields be defined again from the input file.
• Any existing definitions are left as they are. That is to say, the =LOAD command does not itself ever perform an implied REDEFINE.
• So if you unload a file, initialize it, then manually DEFINE one or more fields before reloading, they will end up with the manually-added definitions rather than those they had before the unload (see reorg recipes).
• If field codes are supplied, DPT makes a note of the actual codes that fields end up with after they're defined, so that if the record input stage uses field codes, the actual new ones can be substituted in.

Stage 3. Loading record data:

• Input records are read and stored in table B.
• If there were existing records in the file, the new records receive record numbers ascending from the current highest in the file, according to BRECPPG at load time.
• During a reorg if a record had extensions before the unload, they are consolidated at reload time down to a minimum number, ideally of course a single extent.
• If TAPED contains field codes instead of names, the cross-reference table built in the field stage is used to convert to the correct ones.
• If the file has indexed fields and no index input file is present for any of those fields, the data phase builds index entries as if the file were in single-step deferred update mode. There is no need to OPEN the file into DU mode beforehand: the =LOAD command does that automatically.
• If index input files *are* present for the indexed fields, DPT assumes they will contain correct index information corresponding to the data information in the data input file, and it therefore does not generate indexes from the data during this phase. This applies even if the index input file(s) later turn out to be empty. DPT is willing to trust you here - see comment below about this feature.
• If index input files are present, DPT maintains a log during the data load phase, cross-referencing the record numbers in the input versus those new ones it stores. This allows the index information to be adjusted in the next phase to point to the correct records. This is the only reason the input TAPED file has to contain the old record numbers - loaded records do *not* keep their old numbers.

Stage 4. Loading indexes:

• B-tree entries and inverted lists are stored afresh or added to those existing in the file as appropriate.
• B-trees are built as per SPLITPCT at load time, and inverted lists are stored with the appropriate level of compression taking into account which records now possess each value.
• If there was a data phase in the same overall load operation, the record numbers in incoming inverted lists are changed to the correct new ones as stored. (A sanity check is performed - see LOADCTL X'02').
• Starting with empty indexes is much faster, unless the incoming value entries are all new and located at the high or low end of the b-tree, in which case no b-tree node splits or inverted list modifications are required.

Stage 5. Completion:

• Regardless of which combination of field/data/index input files is processed, at the end of this command the database file is set to FISTAT=0 to indicate that as far as DPT is concerned it is internally consistent.
• Note: This applies regardless of whether the input files are logically consistent with each other, although DPT does issue warnings in some more obvious cases. This freedom of operation gives a database adminstrator some opportunities for creativity in performing certain file maintenance operations, for example on very large files or where unusual data is involved, although generally speaking such sleight-of-hand should not be necessary.

Examples:

=LOAD
IN SALES =LOAD ALL INFORMATION
=LOAD (PREVIEW)
=LOAD (DIR = "Fastload Inputs\Sales\June 2010", CLEANAFTER)

=MIME

DPT maintains a lookup table keyed by extension, with each entry corresponding to a MIME type. This both enables the system to serve fully-described static resources like images and applets, as well as allowing the User Language programmer to use the extension as shorthand in code when specifying a MIME type.

When the system starts, the table is empty, so DPT would be leaving everything to the browser. However, you may find it useful to set up some standard types, if only to be able to use the UL shorthand when creating dynamic content that isn't HTML (the default). The web server daemon initialization procedure is a good place to place such commands (as on the demo system).

• =MIME {LIST | CLEAR | DEFINE} [.]extension [=] mime-type }
• The LIST option displays all entries.
• The CLEAR option deletes all entries.
• The DEFINE option either adds a new entry, or updates an existing one if it already exists for the given extension.
• You can use quotes to specify values containing spaces and/or semicolons, allowing more rigorously complete MIME type definitons like e.g.

  =MIME DEFINE .txt 'text/plain; charset="UTF-8" '
  

• Since MIME types are conventionally in lowercase, you might often want to issue *LOWER before using this command and *UPPER again afterwards, since the default line-input processing uppercases everything before parsing and executing commands. Uppercase type values will however be recognized by most clients. The case in which you give the file extension is irrelevant - file name extensions are always case-insensitive.

MONITOR RESOURCE

• MONITOR RESOURCE [LIKE pattern]
• The "MONITOR CFR [LIKE cfrpatt]" command on DPT is simply shorthand for typing "MONITOR RESOURCE LIKE CFR_cfrpatt". In other words it just generates a subset of the full MONITOR RESOURCE display.
• The same goes for MONITOR SUBSYS, where any LIKE pattern given is appended to "SUBSYS_", in the same way.
• In both cases if no pattern is given, '*' is assumed - namely all resources of that type.

=PAI

• =PAI [LOB-option] [count|ALL]
• The LOB options are the same as on the UL statement.
• The only difference in processing from the statement is that the command does not bother trying to avoid table E disk reads for BLOB fields where LOB_NONE or LOB_NO_DATA are used. Hence a little less efficient in those cases.

=PAV

• =PAV [COUNT | RECNUMS] field [([DESC[ENDING]],[STOP[AFTER][=]n])]
• The basic command "=PAV field" just lists the values from the ordered index for the field.
• =PAV COUNT additionally counts the inverted list associated with each value and prints the count.
• =PAV RECNUMS not only counts the inverted list but prints out all the record numbers in it too.
• The bracketed option DESC[ENDING] means start at the high-value end of the ordered index and work back.
• The bracketed option STOP[AFTER] means only process n values of the field.

=PSTCHECK or =PSTTIDY

Normal users can issue these PSTxxx commands if they really think there's a good reason to do so. Normally there would not be any good reason.

=PSTFUN

• =PSTFUN [number]
• The single optional parameter is for use when a terminal user wants to become a $function-support daemon temporarily, and indicates the number of $functions to handle before returning to the chevron.
• In most aspects this command and its parameter are handled the same as =PSTWEB - see below for notes. The relevant control parameter in this case is FUNNDAEM instead of WEBNDAEM.

=PSTWEB

• =PSTWEB [number]
• The single parameter is for use when a terminal user wants to simulate the action of a server daemon, usually in order to debug a handler procedure in an "as-live" situation.
• The specified number of HTTP request/response exchanges are performed before the thread returns to its previous non-server-daemon state.
• If a number less than or equal to zero is specified, the thread will enter an infinite loop which can only be terminated by bumping it, or resetting WEBNDAEM to a lower value.
• When issued from an interactive thread (IODev7 or 15) it is assumed that temporary daemon-ness is required, and a parameter value of 1 is the default, causing a single HTTP request to get handled. Otherwise the default is zero.
• Such "simulated" daemons ignore the WEBNDAEM setting as they wait for incoming HTTP requests. In other words you can set WEBNDAEM to zero and all the "real" web daemons will log off, thus ensuring any incoming work gets picked up by your terminal session.
• Both dynamic and static HTTP requests count towards the 'number' value given. Therefore if you are debugging, say a page of HTML which contains 3 <IMG SRC=....> tags, you would need to say "=PSTWEB 4" to serve the entire page.

=SHELL

See also =SPAWN.

• =SHELL [(verb)] application-name-and-or-command-line
• The value of 'verb' can be one of the operations you typically see on the "right-click" menus in Windows Explorer. The allowable range therefore depends on the file associations defined on your machine, and the type of file specified on the command.
• For example OPEN (supported by most entity types), PRINT (usually only documents), and EXPLORE (directories).
• A special verb, and the default, is APP, which means that the command line is interpreted as an application name followed by an optional set of arguments to be passed to it. For example:

  =SHELL NOTEPAD MYPROC.TXT
  =SHELL (APP) WINWORD.EXE REPORT.DOC
  

• Depending on your current OS file associations the above calls might do the same thing as =SHELL (OPEN) ...etc.
• Another special verb is CMD (or synonym DOS) which means the entire line is just passed through to the DOS command interpreter CMD.EXE, and is a tidier way to invoke DOS commands. For example:

  =SHELL (CMD) MKDIR TESTRUNS
  =SHELL (DOS) DEL TESTRUNS/TEMP*.TXT
  

• This option is processed by invoking CMD.EXE with the "/c" switch. If you need finer control over the switches passed to CMD.EXE, just invoke it explicitly as an application. For example:

  =SHELL (DOS) SOMEJOB.BAT              /? might be too quick to see    ?/
  =SHELL CMD.EXE /T:0A /K SOMEJOB.BAT   /? review results when finished ?/
  

• In some cases where you might think of using a DOS command, the $FILE_xxx and $xxDIR set of UL functions offer more control over what happens and give better diagnostics.
• =SHELL can be used to simulate USE $JOB, by first writing the DOS batch commands to a sequential file with extension ".bat" and then invoking it for example as above.
• There is no direct $function equivalent to the =SHELL command, but it is easy to perform effectively the same thing, using either $SPAWN or $COMMBG and then issuing the =SHELL command from the daemon.

=SOCKET

If the listener threads crash it may also be possible to use the SOCKET command to restart them without bouncing the whole system.

• =SOCKET {START | STOP | STATUS} [TERM[INAL] | WEB]
• The optional parameter can be used to direct the command at either of the two listeners.
• Note that this command has no effect on User Language applications listening using the $SOCKET function, although you can still BUMP such threads.

=SPAWN

Daemon threads are operationally very similar to IODev3 threads, and allow all the same commands, parameter settings etc.

Note the difference from =SHELL which kicks off an OS process outside the host, and also the difference from UL $functions which make brief use of existing support daemon threads rather than spawning fresh ones.

• =SPAWN [ASYNCH | SYNCH] {IDstring | *} commandline
• The SYNCH/ASYNCH option, if given, must be the first word after the command name.
• The IDstring can be any string up to 5 characters long. If an asterisk is given the system generates a 5 digit value starting at 00001. The spawned thread logs on with a user name of "Daemon_xxxxx" where xxxxx is the ID string.
• The input for the spawned thread is limited when you use the =SPAWN command to a single physical line (but see $SPAWN which can take a $LIST). The single line can consist of a number of logical lines (e.g. B;PRINT 'HELLO';END) which is often a decent substitute for a real multi-line capability. However, since semicolons are interpreted as newlines on the issuing user's device, it is necessary to use an alternative convention to pass them to the daemon if required. The convention used by the BROADCAST command is followed, namely that you should specify slashes instead of semicolons. To actually include a slash, use "!/". For example =SPAWN * B/PRINT 'The new millennium began on 01!/01!/2001'/END.
• More usually a daemon would open a directory and include a procedure, in which case the above issue is not a problem. In any case in version 2 a call to the $SPAWN function giving multiple "real" lines in a $list is possible, and probably more familiar to M204 users.
• There is no need to include a LOGOFF command - this is assumed, unlike on an IODev3.
• The output device of the daemon thread is a sequential file of the form IDstring.txt which is stored in the #DAEMON directory. If the file already exists, it is overwritten.
• In synchronous mode, the issuing thread enters a DPT custom wait state of WT=96 until the daemon finishes.
• Also in synchronous mode the return code of the =SPAWN command is set to the highest return code encountered by the daemon thread, so the "parent" thread can be considered to have had the error too. This is relevant in various situations, such as if a SETGRC command comes next.
• Creation of the spawned thread goes hand in hand with the creation of a system resource (see also =RESxxx) named SPAWN_xxxxx, where xxxxx is the ID string. This resource is automatically acquired by the spawned thread when it starts, and released when it terminates. This is most relevant in asynchronous mode, where for example the calling thread could spawn many background tasks and then wait for them to finish by using some combination of MONITOR RESOURCE, =RESTRY, =RESGET, =RESINFO, $WAIT, $ECBDGET or =SPAWNINFO.
• If the specified ID string corresponds to a still-running daemon thread (i.e. a resource exists and is still enqueued) the =SPAWN command fails. If the ID string has been used before, but the daemon has finished (i.e. a resource exists but it is no longer enqueued) the resource is re-used and the output file overwritten.
• If a thread issues =SPAWN ASYNCH..., closely followed by one of the monitoring commands below, the daemon can fail to start because the spawning thread will be examining the resource. In a programmatic setting, you would usually use synchronous spawn.

=SPAWNINFO

• =SPAWNINFO [IDstring]
• The ID string defaults to the last one generated by the system (see =SPAWN), which means it would usually not be useful to accept the default if several threads had been spawned.
• If the specified daemon thread is still active, an error message is issued.

=SPAWNPRINT

• =SPAWNPRINT [IDstring]
• The ID string defaults to the last one generated by the system (see =SPAWN), which means it would usually not be useful to accept the default if several threads had been spawned.
• If the specified daemon thread is still active, an error message is issued.
• The corresponding resource is enqueued whilst the file is being read, so that a new daemon thread with the same ID can not be started until =SPAWNPRINT has finished. This is more relevant to the $function context, where the request might wait for unpredictable lengths of time between reading each line.

SPY

Generally you would not need to issue this command by hand, as Ctrl+F1 in the GUI toggles the debugger on and off. The following notes however may be useful to explain the meaning of some of the GUI debugger options.

• SPY [[ON | OFF | LINE] | [spymode] [rttrace] [spyflags]]
• SPY sets the values of the RTTRACE, SPYFLAGS and SPYMODE parameters, and is the only way of affecting those parameters. Most commonly the values for these parameters would be passed as client preferences rather than typing them by hand.
• SPY ON is shorthand for setting the default values for the thread type the user is currently running on, as covered below. SPY is shorthand for SPY ON.
• SPY OFF is shorthand for setting all three parameters to zero.
• SPY LINE can be used on IODev7 to set SPYMODE to 2 instead of the default 3 if you want to use line mode for some reason.
• If ON or OFF is not specified, one, two or all three of the parameters can be specified, in the above order, with missing values taking their defaults.
• SPY ON and SPY OFF are all you will usually need, but details of other possibilities are given in the IDE user's guide.

=UNHALT

• =UNHALT
• When issued from a user thread, this command, like M204's EOJ, initiates a "Do you really want to" exchange.

=UNLOAD

Efficiently extracts all or part of the information in a file. The results might be the halfway stage of a reorg, or be used used for any other purpose such as loading into another DPT file or passing to an external system. By default the command processes all records in the file and/or index values for all records, but if you want to specify a subset you can use the partner $UNLOAD User Language function, or the GUI interface in the File Wizard utility.

The output of the processing is information in the standard layouts as used by various internal operations as well as user-invoked functions like REORGANIZE and =LOAD. If extract data is required in other significantly different formats, apart from the simple options shown below, you will have to either fall back on a UL/image program, or start with =UNLOAD and then reformat the output using external tools of your choice. Put another way, DPT has no equivalent of the complex Sirius 'FUEL'.

The ordering of information extracted is as it is in the database: data records in ascending record number order, and index values in whichever collating order the field is defined as (string or numeric). There is currently no way to get the =UNLOAD command to change these extract orderings.

• =UNLOAD [what] [[EXCLUDING|FOR] [FIELD[S]] field [,field...]] [(options)]
• The 'what' clause indicates which parts of the file to extract. Specify it as a series of keywords, optionally separated by commas, spaces or the word "AND". The keywords are FDEFS, DATA, INDEX[ES], and ALL which means all 3 and is the default. ALL INFORMATION is the same as ALL.
• You can unload information for a subset of fields by giving the desired list of field names separated by commas. The list of fields must be preceded by at least one of the introductory keywords above. The default (FOR) includes only the specified fields. The alternative (EXCLUDING) will unload all fields except the ones specified. Fields not participating in the unload get no information put into any of the output files.
• The bracketed options can appear anywhere in the command and affect the formatting/encoding of the output, as follows. Multiple options should be separated with spaces and/or commas. All options are off by default.
  • PAI: Affects TAPED and TAPEI files. Makes the contents of these files human-readable, like the output of simple User Language programs, as covered here (D) and here (I). This format is acceptable to the fast load process, and in many cases can be a more convenient format for both unload and load, particularly loading data extracted from Model 204 if the highest performance is not required.
  • EBCDIC: Translate all string values into EBCDIC. Use this option if the extract is going to the mainframe. Note that when the extract contains binary data like field IDs or bitmaps it isn't possible to leave this issue to the file-transfer engine (e.g. FTP), since that would corrupt the binary data. PAI-mode jobs can either use this option or let FTP do it.
  • FNAMES: Populate the TAPED file with field names instead of codes. The lookups required to do this mean the unload is slightly less efficient, plus the data extract file is almost certainly larger. This option is always effectively turned on in PAI mode.
  • NOFLOAT: Extract numeric field values as strings rather than the 8 byte IEEE 754 FP bit patterns that are stored in the DPT file. This adds some conversion overhead, but the effect on extract size may be positive or negative depending on the data. Remember that Model 204 on the mainframe by default does not understand the IEEE 754 format. This option is always effectively turned on in PAI mode.
  • ENDIAN: Causes binary items in the output to be written in reverse-endian order. Affects all integer numbers such as record numbers, other assorted integer control fields, plus floating point values too if included. If byte ordering is treated differently on the target machine for integer and floating point, use IENDIAN and FENDIAN separately to achieve the desired combination. This option is irrelevant in PAI mode.
  • CRLF: Affects TAPED and TAPEI files. Appends an extra CRLF byte pair after each record (TAPED) or each segment inverted list (TAPEI). This can make unloaded data more palatable to e.g. sort utilities later, and likewise can also make it easier to create custom load input from external sources.
  • DIR: An override for the directory which will receive the output files. e.g. DIR=SALESDATA, DIR="C:\TEMP", etc. The default is #FASTIO in the host working directory. This directory is created if it doesn't exist already. Either single or double quotes should be used to surround the name if it contains spaces.
  • REPLACE: Since you will usually want to do something with the extract files before creating more of them, the command by default will not overwrite existing output files if they exist. This is intended to prevent earlier extract processing being wasted if you just forget to pass the extracts on to whatever their next destination is. Use the REPLACE option to disable this check. See also the related CLEANxx options on =LOAD.
• Other comments:
  • The context in which the command is issued must not be a group.
  • File updates are blocked on other user threads for the duration of the unload. In fact the =UNLOAD command itself may fail with record locking conflict if other users have in-flight transactions.
  • The processing is bumpable.
  • Minimal diagnostics are issued by default.
  • DPT can use parallel-processing during unloads. See the LOADTHRD parameter notes for more details.
  • By default, index entries which point to nonexistent records *are* unloaded, but see LOADCTL X'04'.

=UNLOAD
IN SALES =UNLOAD ALL INFORMATION
=UNLOAD FDEFS (FNAMES DIR='C:\Mystats')
=UNLOAD INDEXES FOR FIELDS SALES_Q1, SALES_Q2
=UNLOAD (REPLACE, CRLF) ALL
=UNLOAD FDEFS AND DATA (EBCDIC FNAMES) EXCLUDING AGENT.COMMISSION

=VSTAT

• =VSTAT [ALL | statname} [ALL | SL | USER | FILE | SYSTEM]
• The stat name and level, if given, must be specified in this order, i.e. name followed by level.
• If a named stat is not collected at a named level, an error message results, as it does with $VIEW and $STAT (which also return a null string).
• To view stats at FILE level, the command must be issued in some file context.
• Both "ALL" options will show all stats and/or levels that are collected. For example =VSTAT DKRD ALL would show 4 figures, and =VSTAT ALL FILE would show all the file statistics. The master M204 X-reference document gives details on which stats are collected at which levels.

=Z1

The main time this command would be useful is if one file is in DU mode and hogging a lot of memory with its deferred updates but neither getting closed nor receiving any more updates to trigger the offload. To other actively-updating files in DU mode it would then appear as if available memory was low, and they would keep flushing far too soon and spoiling some of the benefit of their DU mode. If it's not possible to get all users to close the first file, issue the =Z1 command on it.

• =Z1
• The file context must be a non-group that is currently in single-step deferred udpate mode.
• The file *remains* in single-step deferred update mode after this command.

Part 2: User Language

• General Compatibility and Processing Notes
• Statements
  • M204
  • DPT Custom Statements
• $Functions
  • M204
  • DPT Custom $Functions
  • (See also Sirius $Functions)

General UL Compatibility and Processing Notes

In some cases there are small intentional differences for the various reasons mentioned in the emulation strategy document. More details on some of these, plus other issues of general interest, are given below.

Model 204 statements that aren't supported are discussed in the compile-time messaging section below.

• Version compatibility and obsolete M204 features
• Number handling
• Pattern matching
• Context specifications
• Find Processing
• List Processing
• Subroutines
• On Units
• Callable $functions
• Variables
• Images
• Screens
• Redefining Labels
• Compile-time messaging
• Run-time messaging
• Reserved words (and characters)
• ASCII/EBCDIC
• Case handling in UL

Version compatibility and obsolete M204 features

Unbalanced brackets may not be used to continue lines. Since there is no INCCC parameter, continuation can not be implied with a non-blank character in this column. Hyphens, unfinished quoted strings and unfinished block comments are the only way to continue lines. Note that the FIND expression

FD
    (AGE > 18
     SEX = M) OR -
     RANK = CORPORAL
END FIND

The "pre V2.1" issues mentioned in the UL manual appendix are taken into account, so that the "previously-allowed" statement forms are not allowed on DPT.

Number handling

Floating point
This is a complex subject, but one thing we can say is that completely emulating Model 204's (and by implication MVS's) handling of floating point numbers is outside the scope of what we are trying to do. DPT's floating point handling is based on the IEEE 754 standard functionality commonly supported by compilers and co-processor chips. This standard is available on S/390 nowadays but M204 (being an old application) almost certainly uses the earlier "hexadecimal floating point" (HFP) format. This means the underlying notions of accuracy and precision are going to be subtly different between the two platforms.

The question is, how far should we go in making things work like they do on M204? The happy answer is that in many cases things should work the same because of the UL convention of *rounding* to 15 significant decimal digits for many operations. DPT follows this convention, as described in some detail in the UL manual, and since both IEEE 754 and long HFP are guaranteed to convert reversibly at this precision, any differences get glossed over when these roundings occur.

Rounding occurs in nearly all situations, including but not limited to: Storing in FLOAT variables, in the PRINT statement for any numeric value, storing in both table B and table D for FLOAT/ORD NUM fields, intermediate expression results after addition/subtraction and all situations where strings are used as numbers (e.g. in the compiler and when STRING variables are used in arithmetic). Rounding is skipped in very few places, namely after multiplication/division and before/after $function calls, since in these cases the extra precision might often be useful. This should be close to how Model 204 behaves, but may not be exact (interestingly the UL manuals themselves aren't completely consistent on what they say). Further refinements can be made in future releases if required.

There may be subtle differences in what is considered a valid numeric literal when exponents are used (e.g. 1.5E3 / 1.5 E 3 / 1.5E+3 etc).

This topic is also relevant to the math $functions, where the potentially different expansions used by Microsoft C and the mainframe libraries could conceivably add further rounding differences in the small digits. It's also relevant to the bit string generated by $FLOAT.

Fixed point arithmetic in UL
When doing arithmetic with FIXED and STRING variables (and image items) that have DP specifications, DPT should behave the same as M204, as per the rules set out in the UL manual 10-23...25, since in this case the behaviour is based on truncation at the point specified by the DP specification, which maxes out at 9. DPT obeys all the rules concerning scale of intermediate and final results, except one. It does not look ahead to see if the next operation is a subtraction (which in any case as the manual says ends up giving an incorrect result).

Pattern matching

However, without knowing how the M204 matcher works it's likely that highly complex patterns will work slower on DPT, but they might be faster, you never know! In particular the common cases of patterns with a leading and/or trailing wildcard and no other specifications such as '*ABC', 'ABC*' or '*ABC*' all work very quickly, being internally translated into the equivalent of $SCAN.

Context specifications

Unlike M204, global foundsets and positions can be created in ad-hoc group context on DPT, but they are deleted at end-of-request, and so do not realistically give rise to a situation where an application will work on DPT but not on Model 204. Ultimately the same result is achieved and it was much easier to implement this way internally.

The following are not allowed in FIND and FR WHERE statements, because the corresponding features are not themselves supported.

• LOCATION$ (no scattered groups)
• Equals sign in FILE$ (no scattered groups)
• SFGE$ (no sorted files)
• SFL$ (no sorted files)

Find Processing

It's also possible that DPT does things in a slightly different way when evaluating finds in group context. DPT basically just performs the same find once for each member, which means for example that you could get more than one record locking conflict or MBSCAN-exceeded message during the same find. This is something that will be easy to correct if required in future releases.

List Processing

Subroutines

In some cases it's possible there may be a difference in the handling of file context where list parameters are allowed to be implicitly defined (the M204 manuals don't make it clear what's supposed to happen). In any case if contexts don't match there will be a compile-time error and this will not cause subtle misbehaviour.

On Units

Secondly, consider that ON units can invoke and also install/deinstall themselves and other ON units, both in their own scope and the scope where the condition was raised. This means it's not always easy to see which ON units are in effect at any given time.

Combine the above points with the fact that activation and de-activation of ON units across calls to subroutines (and across "calls" to ON units) is not completely consistent in practice on M204, and things can get really confusing. DPT should work the same as M204 in all simple cases and most if not all complex cases.

When the BYPASS statement is used after ATTENTION in the test element of an IF or loop construct, M204's behaviour seems a little strange (try it!) DPT is more sensible and bypasses the entire loop or IF block.

If an ON unit is defined inside a FR loop, and later invoked from a point outside that loop, field references in the ON unit give a "no current record" error on DPT. This may be slightly different behaviour to M204, which if memory serves shows the value from the last record processed by the loop. Hopefully an obscure situation though.

Callable $functions

%X = $SETG('A', 'B')
CALL $SETG('A', 'B')
$SETG('A', 'B')

This feature can be disabled for stricter M204 compliance using the CSTFLAGS parameter. See also some related notes about syntax-colouring of $function names in the DPT client front end.

Variables

COMMON variables are an exception to this of course. You have the option of fully defining COMMON variables in each scope they are used, as was required pre-M204-V5, in which case each part of the definition must match the original, or simply declaring the variable name as COMMON, in which case the other characteristics get picked up from the original definition if there was one. The "only one INITIAL clause" error can also be turned on or off as desired.

LVARTBL is registered once per scalar variable, once for every array element, and once for any subroutine parameter of either type (since dimensions are not given on array parameters). This is all regardless of data type (STRING/FLOAT/FIXED).

DPT supports arbitrary expressions as INITIAL values (that is, the Model 204 V5 feature). The expressions are evaluated at the start of running a request, before anything else happens. By default the debugger doesn't stop on the declarations while this happens, but you can make it with an option.

Images

• STRING items behave similarly to STRING %variables.
• FLOAT items behave similarly to FLOAT %variables.
• EFORMAT items behave similarly to STRING %variables for truncation and padding, and FLOATs for computation.
• BINARY, PACKED and ZONED items behave similarly to FIXED %variables - importantly for runtime DP handling (see notes elsewhere).
• Messages are issued for truncation, variable-too-small etc. in similar situations as with %variables.
• However bear in mind the different storage format. For example the "variable too small for numeric result" error is more of an issue than with %variables, since numeric type items may be quite small (e.g. PACKED LEN 1 can only hold a number from -9 to +9). On the other hand the types which remove decimal points for storage thereby offer a "bonus" available digit!
• BINARY is an interesting case where we must decide how many BP equals each DP. This is handled by treating e.g. a BINARY BP 3 the same as FIXED DP 3 and so on. Going from binary to decimal the number of places required is the same (e.g. 110.101 = 6.625) so it's not as ridiculous a convention as it sounds.
• The high 8 bytes of a FLOAT 16 are set to zeros when writing to the item, but ignored when reading it.
• Accessing a numeric item which contains data in the wrong format is a request-cancelling error.
• LEN UNKNOWN items identified at run time as length zero return a null string or zero when accessed.

The dummy sign character used in UNSIGNED PACKED numbers is 'F' for visual familiarity to mainframe users (incidentally $PACK and $UNPACK are handled the same way). However, with an UNSIGNED ZONED item, the character is '3', which is to say no dummy character. This is more natural in an ASCII environment where the decimal digit "zone" of the character set is X'30' to X'39' instead of X'F0' to X'F9'.

Structure of images
Like ON units, User Language images offer the chance to create infinitely complicated structures, using for example large numbers of staggered redefines in conjunction with UNKNOWN or DEPENDING arrays. Also like ON units, the degree of confidence to place in the accuracy of DPT's emulation of Model 204 depends on the complexity of the situation. DPT should work fine in most realistic scenarios.

DEPENDING and UNKNOWN arrays are two ways Model 204 offers to provide mapping of variable structures using what is essentially a fixed structure, namely an image. With an UNKNOWN array certain item position checks are deferred till run time: a classical approach which simply puts some limits on what can be checked at compile time. A DEPENDING array on the other hand corrupts the clean "mapping" function of an image by inserting/removing fillers into/from external records during READ/WRITE, so that for example byte 25 of an incoming record might end up at byte 35 in the image. In this sense it is an ugly solution to the problem, and poses some tricky questions to the compiler. There are a number of restrictions M204 puts on the positioning of other items in an image which contains one or more DEPENDING arrays, presumably kluges intended to minimize the effect of the filler insertion/removal on those other items. Not all these restrictions are implemented on DPT (I don't even know what they all are, and in any case some would probably be unnecessary given the different underlying implementation). So, while existing code written on M204 should work, there is a small window of opportunity to write incompatible new code.

Screens

• User-defined screen items and ITEMNAME behave similarly to STRING %variables.
• The numeric system-defined items (PFKEY, ROW, COLUMN and ITEMID) behave similarly to FIXED DP 0, (except via screen item variables - see below).
• Messages are issued for truncation, variable-too-small etc. in similar situations as with %variables.

• The "source" variable can be a variable (:%V), an image item (:%IMAGE:ITEM) or other screen item (:%SCREEN:ITEM) or an array element (:%V(2), :%IMAGE:ARRAY(3,7) etc.)
• Unlike field name variables (%%V) screen item variable references can be stacked (e.g. :::%V = the item named in the item named in the item named in %V). This may or may not be allowed on M204, but in any case coding like that is probably not a good idea.
• The source variable of a field name variable can be a screen item variable (%:%V).
• All screen item variables are considered as strings by the compiler when used in expressions, even if they eventually resolve to ROW, COLUMN or ITEMID. This is for internal convenience.
• The DP specification of the screen item ultimately accessed at run time is preserved when used in an expression. With the system-generated items like PFKEY, DP 0 is assumed.
• There may be places M204 accepts SIVs where DPT doesn't, and vice versa. For example the DEPENDING variable in an image cannot be a SIV on DPT.

Display considerations
The ways in which the view on DPT Client differs from a 3270 are now discussed in the IDE User's Guide.

Screen processing on non-IODev 7 devices
Full screen related statements compile on any thread type, but READ and REREAD fail at run time unless the user is running on an IODev 7 (i.e. there is no support for the clunky line-at-a-time screen read processing you can do on M204). PRINT SCREEN on a non-IODev 7 device generates a formatted text-only version of what the screen would look like, minus colours, underline, blink etc. You can also force this to come out on an IODev 7.

Miscellaneous screen issues
The MUST FILL autovalidation option is treated similarly to REQUIRED (i.e. implicit AND, not OR). This disagrees with the manual but seems more logical.

Redefining Labels

DPT does have internal support for redefined labels, but it is currently disabled because its behaviour may be sufficiently different from M204 to be confusing. The following notes are left in here for interest, and to make it clear what a potentially confusing area it is. The ability to redefine labels on DPT may be reinstated if and when it is possible to confirm exactly how M204 handles things.

-----------Start of provisional notes about the disabled feature

When the current request just contains just one occurrence of a label, but there is another identical label in the global table or remaining from an earlier portion of a continued request, there is hopefully no confusion over what should happen, either with jumps to the label or with references by other statements.

However, when a label is redefined in the same request (or more specifically the same request continuation portion) it is allowed so long as the statement type (FD, SORT, COUNT etc.) is the same in all cases, and the file context is the same too where appropriate. This means other statements referring to the label can be confident of the type of "contents". The contents of the label as extracted by other statements reflect the contents as set by the most recent statement executed at run time (loops set the VALUE IN or OCC IN at the top of the loop). Jumps to redefined labels on the other hand are resolved at compile time. If the label has already been defined (backward jumps) the jump is to the most recently-encountered version. Otherwise (forward jumps) the jump has to be resolved in a final pass of the compiler, and is therefore always to the last version of the label encountered in the code. To illustrate these points:

B
A: JUMP TO V                       //invalid jump (final V is nested)

V: FRV SURNAME
     PRINT VALUE IN V              //current value of outer loop

V:   FRV CITY                      //redefinition OK - FRV
       PRINT VALUE IN V            //current value of inner loop
     END FOR

     PRINT VALUE IN V              //final value of inner loop
  END FOR

  JUMP TO A                        //back to the top

A:                                 //redefinition OK - "vanilla" statement
  PRINT VALUE IN V                 //final value of final run of inner loop
  JUMP TO A                        //previous statement
END

-----------End of provisional notes

Compile-time messaging

One of the key objectives DPT was written to achieve was to provide much more accurate and helpful compiler messages, so the first thing to point out here is that in most cases the message text doesn't match that on Model 204. Having said that, in some cases the M204 text is so familiar that it is used but with extra information added. In many cases the message is somewhat longer - a example typical of the DPT philosophy here is:

M204:
FIELD NAME REFERENCE NOT WITHIN A FOR LOOP

DPT:
Field name SOMESTRING does not exist in the file context active at this point (TEMP GROUP MYDATA). If you meant a string literal, put quotes round it.

In any case, broadly speaking there are three things that can happen if a statement doesn't compile.

Firstly, as with Model 204, there are a handful of situations where the compiler will issue a message but otherwise compile the statement in question successfully. They can be thought of as helpful advice from the compiler, although you can upgrade them to errors - see below. Most examples of this are listed here.

Secondly there is the most common situation where a statement is invalid. The compiler issues an error message, ignores the line in question, and then carries on compiling. Typically the request as a whole will fail compilation with these because by default the messages are CLASS=E. In other words they are counting errors, which means the error count will be non-zero at the end of compilation, which in turn means the evaluator will not kick in. Compilation may also be aborted prematurely if the error count reaches ERMX. If these messages are made CLASS=I, the request will be evaluated as if the source code line was not present (it will be highlighted in the debugger source code view as having failed compilation). As on Model 204, this is not usually advisable, and may cause unpredictable behaviour.

Finally there are errors which for various reasons mean the compilation cannot continue at all. In such cases the error message is followed by "Compilation aborted" and nothing is run. This category covers things like memory exhaustion and (hopefully rare) internal bugs.

DPT by default should issue messages in roughly the same situations as Model 204. The handling of individual messages in the first two categories above can be altered using the MSGCTL command, for example to force a failed compile by changing an "informational" message to TERM CLASS=E.

Messaging for unsupported M204 statements
In most cases where a Model 204 statement is not emulated, the compiler by default issues an informational message and compiles a "stub" operation into the executable program, so that during execution another informational message gets issued if that operation is hit. The intention of this is to allow existing User Language programs to be debugged and tested. If control doesn't go near the unsupported statement at run time, there is no problem. Even if control does pass through a stub operation, you might use the debugger to break there and tweak variables and image items etc. to pretend that the unsupported statement had been executed. You can upgrade either the CT or RT message to CLASS=E, to force an appropriate error response if desired.

Run-time messaging

A whole family of error messages, namely those issued from inside $functions (e.g. negative parameter to $SQRT to take an obvious example), deserve special mention. On DPT most of these $function parameter-validation messages just use the same message code with differing message text, unlike on M204 where there is a range of separate codes around M204.0745 onwards. Clearly this reduces the precision with which MSGCTL can be applied, and if this is something that causes problems for people, separate message codes can easily be reintroduced.

Additionally, DPT addresses its development and unit-testing mission by issuing messages in a few additional situations where it might highlight a potential UL coding error. A good example of this is when a string %variable is truncated on assignment - a M204 feature that can be overused and sometimes leads to nasty bugs. Similarly when assigning invalid numeric literals to FIXED and FLOAT variables. These considerations also apply to intermediate results when, for example, concatenated strings are truncated or numeric overflow occurs.

In all cases run time messages can be upgraded or downgrded to CLASS=E/I or suppressed using MSGCTL. CLASS=E messages invoke the usual chain of events for a run time error (ON ERROR unit if possible, otherwise request cancellation; then APSY error proc with error global = BUG).

As with compile-time messaging, the default should be for messages to cause errors and be printed on the terminal in roughly the same cases as on M204. All run time messages are by default printed to the audit trail at least.

See also the above comment about messaging for unsupported M204 statements.

Reserved words (and characters)

This section only concerns field names and values, not procedure names. It is an area where the Model 204 manuals are, at the time of writing this, incomplete and/or inaccurate and/or unclear. DPT aims to allow similar usage as on Model 204, but overall is probably somewhat more forgiving, which is to say it requires quotes in a smaller range of situations. The following notes describe DPT's behaviour.

In general, the approach to the treatment of reserved words is to assume the programmer knows what they're doing, and where unquoted reserved words are used to assume the reserved meaning. For example the following command does not contain any ambiguity and requires no quotes.

DEFINE FIELD GIN AND TONIC

PRINT GIN AND TONIC

PRINT G'IN AND TONIC'
PRINT GIN' AND 'TONIC

The set of reserved words and characters that can cause problems depends on the circumstances. For example the following statement will not work since the keyword WITH introduces the field attributes, (CHEESE would get flagged as an invalid attribute).

DEFINE FIELD BURGER WITH CHEESE

DEFINE FIELD 'BURGER WITH CHEESE'

FD BURGER WITH CHEESE = 3.99
ADD BURGER WITH CHEESE = 3.99

%X = B'URGER WITH CHEESE'
PRINT BURGER 'W'ITH CHEESE

When defining fields the following points also apply, not for the benefit of the DEFINE command parser, but as sanity checks to help reduce later ambiguous situations in User Language.

• Field names must not consist of a single reserved word. This test considers the entire set of reserved words that applies in all situations (see the M204 file manager's guide).
• Reserved characters must be hidden with quotes

When using quotes in field names they can go anywhere as long as they hide or break up the reserved component. In the specific cases of use in arithmetic and print expressions, the quotes cannot enclose the first character, since then it looks like a quoted string. In other cases such as commands like the one above and record-access statements like ADD and CHANGE, the entire field name can be quoted. A commonly-used convention on M204, which works in all situations regardless of whether there are any reserved words, is to leave the first character of a field name unquoted and quote the rest.

Similar considerations apply with field values in statements such as ADD, CHANGE and FIND, which do not usually need quotes, and as with field names the set of keywords causing problems will be different in each situation. For example here is a common situation where the keyword OR should obviously take its operator meaning:

FD NAME = SMITH OR JONES

FD NAME = 'SMITH OR JONES'
FD NAME = SMITH' OR 'JONES

ADD NAME = RANDALL AND HOPKIRK (DECEASED)
ADD NAME = 'RANDALL AND HOPKIRK (DECEASED)'

ASCII/EBCDIC

From DPT version 2.12, the $ASCII and $EBCDIC functions provide the capability in User Language to convert between character sets if data is being exchanged with a mainframe, and it isn't otherwise converted during transit.

Case handling in User Language

Top

M204 Statements - Processing and Syntax Differences

As with commands, custom options can be disabled using the CSTFLAGS parameter.

FOR RECORD NUMBER

• LOB_NONE: BLOB fields are omitted altogether. No table E disk reads are performed.
• LOB_DATA: BLOB fields are printed as if they were STRING fields, only longer. Non-BLOB field values print as normal. Also this option populates PAI mode result-array-3, if given, with the "raw" descriptors for BLOB fields.
• LOB_SHORT_DATA: As LOB_DATA, except truncating to 255 bytes per value.
• LOB_NO_DATA: If the statement is a PAI INTO, nothing is put into result-array-2 for BLOB fields, but the "raw" descriptors are still placed into result-array-3, if given. In non-INTO mode, BLOB descriptors are formatted into a readable format and printed as if they were the data. In neither mode are any table E disk reads performed.
• LOB_FLOD (or LOB_LOAD): Similar to Model 204, but the information is printed in a way DPT fast load will accept, rather than a way Model 204 FLOD will accept. If the statement is a PAI INTO, this option works the same as LOB_DATA.
• LOB_LOAD_ALL: (DPT custom option). As LOB_LOAD but all fields, not just BLOB fields, are printed in the alternate format. Perhaps of interest when regular STRING fields contain newline characters.

V2: SORT VALUES IN V        * V is unchanged, V2 contains fresh sorted set

SORT VALUESB IN V           * Sorts set V in place - doesn't need a label

M204 $function Differences

• General notes
• Bit string mapping functions
• Text case processing functions
• Date functions
• Thread synchronization functions
• Character set functions (ASCII/EBCDIC)
• Other functions

General notes

The following notes describe non-trivial differences in behaviour of Model 204 $functions on DPT. Most of these differences reflect differences in file handling, system config issues, etc. A catalog of supported $functions is given in the features cross-reference document.

A few of the $functions have been embellished with custom options. As with all custom options these can be disabled by resetting the CSTFLAGS parameter.

In one or two cases (e.g. $EDIT), the $function is listed here as an admission that it may not work right in 100% of cases.

In many cases Model 204 handles errors in $functions by setting a default return code (often zero or null string) and then continuing. In these situations DPT often issues an informational message before continuing. The same message code is used for all $functions so it is easy to suppress it with MSGCTL if you don't like it. In other cases it's possible that Model 204 may issue an informational message and DPT does not. If any of these are a problem for people, please say so.

Bit string data mapping functions
($BINARY, $FLOAT, $FLOATD, $UNBIN, $UNFLOAT)

When using these functions remember that the assumed byte order of the bit string reflects the architecture of the platform on which DPT is running. At the time of writing that will mostly be little-endian (e.g. an Intel PC). Most (all?) mainframes use big-endian byte ordering.

The DPT versions of $BINARY/$UNBIN and $FLOAT/$UNFLOAT interact correctly with each other, and in that sense they can be said to "work correctly". In addition, these functions use the same data formats as the BINARY and FLOAT image item types.

There are two main implications worth noting however. Firstly if you browse the data in a file generated using say $BINARY, it will not look how you expect based on mainframe experience. For example the number 255 written out as a two-byte unscaled binary value will usually read as FF00 little-endian and 00FF big-endian. The second implication is that these functions can not be used when transferring data between platforms with different endianism.

Other specific notes about these functions:

• $BINARY returns -1 ("overflow") when insufficient bits are specified for the result, rather than truncating/masking, which might also make sense. This behaviour may or may not agree with M204 - please someone confirm.
• $FLOAT truncates its 64 bit input value to 32 bits according to IEEE 754 rules, not any specified in the M204 manuals. (NB this function, as well as $FLOATD, is deprecated on M204 anyway).

Text case processing functions
($ALPHA, $ALPHNUM, $LOWCASE, $UPCASE)

Thread synchronization functions
($ECBDGET, $ECBDSET, $ECBTEST, $POST, $UNPOST, $WAIT)

• Return codes 3 and 5 are never issued. Both fall under RC=4 since if NUSERS=1 no ECBs are created at system start-up time.
• Extended quiesce is not supported on DPT. Return code 9 covers all cases where QZSIG or CPQZ are supplied as the ECB name. There is no $STATUSD information associated with RC=9. RC=11,12,13 are never issued.
• The SWAP/NOSWAP parameter of $WAIT has no effect, as all threads are permanently "swappable" on DPT. RC=6 is therefore never issued. The parameter value, if specified, is however validated as one of these two values, and RC=17 *is* issued if appropriate.
  
  
• Like all resource waits on DPT, $WAIT starts off spinning aggressively. However, since it might be called when no other users are logged on, after a second or so the wait becomes less agressive. This is to give your PC's idle process a chance to run.
• The wait interval given to $WAIT is only accurate to plus or minus one second. In other words a value of 2 might wait for 2 seconds, or 1.01 seconds or 2.99 seconds.
• See also the =ECB command.

Date functions

All internal handling of dates on DPT is based on the underlying date-handling facilities of the local operating system. What this means that there are almost certainly differences in the ranges of dates which are regarded as valid compared to Model 204. To be specific, the date functions on DPT handle dates in the range Jan 1 0100 to Dec 31 9999.

All the date functions on DPT have the (disablable) option to specify the date format as one of two special values. These formats are mainly intended for use with the web server functionality, and are denoted by the format strings 'UTC' and 'HTTP'. To work properly these must be supplied as the entire format parameter, and not mixed with other components. For example:

B
PRINT $DATECNV('DD MON YYYY', 'UTC', '01 JAN 1970')
PRINT $DATECNV('UTC', 'DD MON YY', 0)
PRINT $DATECNV('DD MON YYYY', 'UTC', '02 JAN 1970')
PRINT $DATECNV('DD MON YYYY', 'HTTP', '02 JAN 1970')
PRINT $DATECNV('DD MON YYYY', 'DD UTC YY', '02 JAN 1970')
PRINT $DATECNV('DD MON YY', 'UTC', '30 FEB 70')
END
0
01 JAN 70
86400
Fri, 02 Jan 1970 00:00:00 GMT
02 UTC 70
***

This simple numeric format is quite an efficient way of manipulating dates as it is often the "halfway house" in conversions between more readable formats. It is also ideal for comparisons, or doing non-whole-day DATECHG style processing such as adding 1 hour. If the date is given as a standard M204 format (i.e. no time component) the result in this format is always a multiple of 86400 since that's how many seconds there are in a standard day - the time of day is assumed to be midnight. Negative numbers represent times before 1/1/70.

HTTP format
This is the standard date+time format used in HTTP messaging, and any time you put a date or time value into an HTTP header it should be in this format. It is also a good format to use for display in an international setting such as on web pages. When converting from this format or using it as input to, say, DATEDIF, it is not case sensitive, so you can type values in UL code without worrying about case. It can also be in any of the 3 accepted HTTP 1.1 date/time styles. On the other hand all function outputs in this format are given in the standard mixedcase style, as shown above.

The time of day resolution of DPT's internal date-storage format means that for dates before 1899, not every second of every day can be represented, and 'HTTP' format dates will show the closest second they can. This should be no inconvenience in realistic applications.

GMT and local time
Unlike 'UTC', the 'HTTP' format explicitly professes to be GMT, although of course you could set it to any value you like. You should therefore take care that when requesting the time from the local machine and converting it into this format you request GMT and not local time. The standard M204 $DATE and $TIME functions return local time, so there is a DPT custom $function, $NOW, which you can use not only to get GMT, as adjusted for your local time zone and daylight savings, but also to get a combined date+time instead of the two things separately.

Character set functions

These functions make use of the character code pages definded in the CODESA2E and CODESE2A parameters.

• $EBCDIC = Returns the input string converted as per the code page CODESA2E.
• $ASCII = Returns the input string converted as per the code page CODESE2A. (Note: before V2.12 $ASCII returned the input string unchanged).

Other Functions

$ALPHA
$ALPHNUM

B
PRINT $STRIP('00123')
PRINT $STRIP('  0123', ' 0')
PRINT $STRIP('0.12500',, 'RIGHT')
PRINT $STRIP(' "c:/config.sys" ', ' "', 'ENDS')
PRINT $STRIP('&$% ABC +*/ DEF []}', '&$%+*/[]{} ', 'ALL')
END
123
123
0.125
c:/config.sys
ABCDEF

blat -install mail.isp.com john@smithfamily.net - - - john@smithfamily.net sm1th

blat -install mail.isp.com "John Smith <john@smithfamily.net>" - - - john@smithfamily.net sm1th

B
%X = $LOAD
%X = $LOAD (%FILENAME, %OPTS)
%X = $LOAD ('SALES', 'PREVIEW ' WITH %PCOUNT)
END

B
%X = $UNLOAD
%X = $UNLOAD (%FILENAME, %WHAT, %FIELDSPEC, %OPTS, %SETNAME)
%X = $UNLOAD (,'ALL INFORMATION')
%X = $UNLOAD ('SALES',,'AGENT_NAME')
%X = $UNLOAD (,'INDEXES','EXCLUDING AGENT_NAME, AGENT_DISTRICT')
%X = $UNLOAD (,'DATA',,'PAI REPLACE DIR="C:\MYEXTRACTS"')
%X = $UNLOAD (,'FDEFS AND INDEXES',,'EBCDIC NOFLOAT','LIST JULY.SALES')
END

B
$WEB_PATCH_URI_PARM('NAME', 'SMITH')
END

* Main handler code
B
%NAME = $WEB_GET_URI_PARM('NAME')

* etc.
END

$WEB_PUT_DATATYPE('text/css; charset=UTF-8')   /? wordy  ?/
$WEB_PUT_DATATYPE('CSS')                       /? tidier ?/

Sirius Case {ToUpper|Leave}