Release Notes DPT V2RX
Release Notes for DPT Versions 2.xx
Contents
- 2.30: 25th October 2010
- 2.29: 10th September 2010
- 2.28: 27th August 2010
- 2.27: 6th June 2010
- 2.26: 15th April 2010
- 2.25: 5th March 2010
- 2.24: 8th January 2010
- 2.23: 16th November 2009
- 2.22: 7th October 2009
- 2.21: 20th August 2009
- 2.20: 11th August 2009
- 2.19: 23rd June 2009
- 2.18: 31st May 2009
- 2.17.1: 19th May 2009
- 2.17: 11th May 2009
- 2.16: 25th April 2009
- 2.15: 3rd April 2009
- 2.14: 23rd March 2009
- 2.13: 9th Dec 2008
- 2.12: 31st July 2008
- 2.11: 14th Feb 2008
- 2.10: 10th Dec 2007
- 2.09: 13th Nov 2007
- 2.08: 28th Oct 2007
- 2.07: 17th Sept 2007
- 2.06: 19th July 2007
- 2.05: 23rd April 2007
- 2.04: 12th April 2007
- 2.03: 22nd Feb 2007
- 2.02: 12th Feb 2007
- 2.01: 8th Feb 2007
Further details (appendixes):
Summary of changes per release
In addition to the following, most releases contain assorted minor internal tidy-ups. It's possible that if you had a problem that you never reported, it is now fixed anyway.
2.01
Enhancements/changes:
Better call conventions on =SHELL, specifically for when issuing DOS commands.
Fixes:
Numeric $EDIT now drops input DP if the mask has no DP.
2.02
Enhancements/changes:
INITIAL values for %variables can be arbitrary expressions (i.e. the Model 204 Version 5 feature). There is also a new debugger option (SPYFLAGS X'0200') related to this.
SYSPORT can be given as -1 meaning terminal users will not be able to connect. (E.g. for a web server only run).
2.03
Enhancements/changes:
Full screen IO processing tailored for slow connections. Response is better, and diagnostics are more informative.
You can customize the $function information used by the editor.
The MSGCTL parameter has a new switch enabling more error diagnostics to the audit trail.
You can edit the ini files and browse the host's current working directory direct from the DPT host console.
Improvements to deferred-update file loads - see notes and benchmarks below. NB. Backward incompatibility issue for existing deferred-update jobs.
Fixes:
Editor $function assistance accidental deactivation loophole removed.
$DAYI no longer requires CUSTOM = 1 to make it accept input dates with no "C" component. The YY component is also optional.
Very long PRINT lines (e.g. infinite ellipsis) can no longer cause buffer overflow on IODev7 devices.
MSGCTL = 4 setting no longer causes an error message on host shutdown.
2.04
Enhancements/changes:
NOFLD option on INITIALIZE can be invoked with "KEEPDEFS" for more M204 compatibility.
This release is also the first one where the DBMS can be used in an API configuration.
DPT is also open source software as of this release. The downloads all come with a more standard licence now.
2.05
API/Source code:
Assorted source code changes for ISO C++ compliance and wider compiler compatibility (specifically gcc).
2.06
Enhancements/changes:
The FMODLDPT parameter has a new X'02' bit to allow nonstandard numerics for STRING ORD NUM fields.
The installed application name and the default installation directory name have changed, to be shorter and no longer contain an apostrophe (now DPT instead of Database Programmer's Toolkit). Note that this means the installer will not recognize any older version you have and will not prompt to uninstall it. You can use Control Panel "Remove Programs" to invoke the uninstall for "Database Programmer's Toolkit" by hand.
The host starts up with a new proc file allocated "USERPROC" containing user code samples, utilities and other good stuff.
Fixes:
Editor prompt texts for $ONEOF corrected.
D GROUP command no longer crashes if issued on a non-terminal thread.
One or two math $function parameter validations altered in boundary cases to match the latest reports about M204 behaviour.
Some $function handling of very high input parameters like 15 digit integers changed to be more reasonable (e.g. $IXPI, $MOD).
API/Source code:
Explicit DatabaseServices closedown function provided.
The wrapped FindRecords() now correctly uses a wrapped FieldValue parameter.
More descriptive field attribute specifications, e.g. for use in DefineField().
Some new Record:: methods provided to avoid output parameters in garbage-collected languages.
DirectValueCursor: API wrapper class - missing functions added.
New repositioning and bookmarking methods on DirectValueCursor.
Cursors in general can be optionally told to not "run off the end" of their set.
SequentialFileServices::Allocate() parameter order changed.
2.07
Enhancements/changes:
Info message is now issued whenever the PRIORTY command is used.
Improved handling of very large web pages when testing at the "chevron".
Fixes:
A fix for the long-standing (and very annoying) intermittent hang and/or user restart problem which occurred under certain multi-user conditions.
Bump now works correctly for a user who is already in the subsystem error proc.
MONITOR FILES is now OK for MONITOR FILE. Also the display now includes the "group" asterisk like on LOGWHO.
OUTLPP/UDDLPP now correctly observed in the situation where a subsystem generates a report over the course of several included procs.
2.08
Enhancements/changes:
New messages clarifying what's going on during the deferred update process, and preventing some common errors.
Fixes:
Corrected error introduced in the last version where file close stats would always show as zero.
OPEN command parsing is less strict about the placement of commas vs. spaces in the "deffered update" flavour of the command.
2.09
Enhancements/changes:
$WEB_GET_POST_PARM has a new parameter keyword to obtain the file size when using INPUT TYPE=FILE on an HTML form.
Fixes:
The browser-local file name returned by the above function can contain spaces.
Right click (floating popup menu) in the editor no longer moves the cursor if some text is selected. So now "cut" and "copy" are not always greyed out on the popup menu.
2.10
Enhancements/changes:
Some new deferred update intermediate file formats.
The FREE command now waits instead of fails if another user (typically the checkpoint daemon) is taking a checkpoint.
API: Several Python-targeted additions.
Fixes:
Corrected a bug which could cause file corruption in large uncommitted update units involving more than one file.
2.11
Fixes:
A handful of minor User Language compatibility fixes as per the latest information from users, as follows.
Support for archaic syntax when implicitly declaring a variable in User Language. For example if %V has not yet been encountered in a request, the statement
%V(5) = 'ABCDE' declares a string len 5 rather than giving an error message saying arrays can not be implicitly declared.
In an expression, the keyword operators GT, LT, WITH, AND etc. can be delimited with a plus or minus sign or bracket, as well as a space. For example the statement
IF %X LT-1 OR+%X EQ(%Y+%Z) THEN is OK. Likewise in a FIND expression like:
FD PRODUCT_CODE IS LE-1
String conversion to number in expressions: trailing spaces are now allowed. For example
%AREA = 2 * '3.1415 ' * %R would previously always equal zero.
The PRESENT operator in math expressions has been tightened up to require the IS keyword prefix. For example
IF %%FID PRESENT now fails to compile. Note that this is now in line with the use of PRESENT in FIND statements, which has always required IS. Note also however that the LIKE operator in both situations can be used *without* "IS"!
API:
Assigning floating point zero to an APIRoundedDouble object is no longer broken when SetNumRangeThrowOption() is turned on.
A new copy of Roger Marsh's notes and sample code is included in the download.
2.12
Enhancements/changes:
The Sirius way of supporting mixedcase User Language code is now emulated in most respects - see notes in the Language Guide. This new facility consists of a handful of small changes, as follows.
Mixedcase "BEGIN" is now always allowed.
New parameter "COMPOPT" as per the Sirius M204 parameter.
"Sirius" compiler directive is now recognized.
The editor has a new option to allow mixed-case keywords and $function names.
Other changes:
The $EBCDIC function is now provided for character-set translation, approximately the reverse of $ASCII.
$ASCII now works as per the M204 manual. That is, it converts from EBCDIC to ASCII, rather than "host character set" (here ASCII) to ASCII, which was previously a null operation.
Fixes:
Complex subroutine parameter INPUT/OUTPUT specifications can now appear anywhere rather than having to be the last word for each parameter. For example this one has it several ways and no longer gives a compilation error:
SUBROUTINE S (%V1 STRING LEN 3 INOUT, %V2 IS OUTPUT FLOAT, %V3 FIXED INPUT DP 2)
In a find statement on Model 204 the comparison operators (EQ/LE/> etc.) are only valid if prefixed by (at least) "IS", so as to specify the comparison type (alpha or numeric). The exceptions are "=" and "EQ" which are allowed without an "IS" part, in which case the comparison type is determined from the field attributes. Prior to this release of DPT, only the symbol "=" was treated this way, and the keyword equivalent "EQ" required "IS" like the rest. Now "EQ" is treated exactly the same as "=", including all the various ordering permutations that are allowed with "NOT"/field name/value.
Fixed the bug on some machines where after installing a new DPT version, the client would crash straight away the first time it was started.
API:
Somewhat rearranged package and one or two new source files added. Additional features in Roger's chess application.
2.13
API:
A wider range of overloads for the FindRecords(...) function. The referback examples in the doc all compile now!
2.14
DBMS:
DPT can now load data much faster than before, using a new "single-step" deferred update process, in many ways similar to Model 204's. The DBA Guide describes the process, and the release notes appendix (below) gives some performance benchmarks.
The old multi-step Z process is no longer recommended, but since it shares some of the tuned code, existing multi-step load jobs will show some improvement just by installing the new version.
The DPT download contains a sample single-step load job.
Fixed bug affecting the ANALYZE command where incorrect record counts might be shown.
User Language:
The $SUBREP function now correctly handles non-default parameter 3 (start position). Previously it would use a position one to the right of the position given.
The $WEB_GET_HEADER function now provides "pseudo" headers named METHOD, URL and VERSION. In other words the three items from the HTTP request message title line. This is for some measure of compatibility with the Sirius equivalent function $WEB_HDR_PARM.
Other:
The host console now has an option to change the font of the displayed audit trail and user 0 output.
API:
The OpenContext() calls for deferred update runs now use explicit function names. The overloaded version was acquiring a ridiculous number of optional parameters.
2.15
Assorted:
Record loop skip processing now works correctly. For example if set S consists of records A,B,C,D,E,F, the loop FOR EACH RECORD IN S BY 2 now processes B,D,F and not A,C,E.
An error message is now issued if deferred update mode is activated for a non-initialized file. It's not really an error, but is a meaningless thing to do and it's confusing if the DBMS allows it.
The error introduced in V2.14 where multiply-occurring fields might be loaded in the wrong order has been fixed.
The single-step deferred update process is better at preventing heap fragmentation, and is therefore less likely to give up due to apparent memory shortage.
API: RoundedDouble objects can no longer be assigned an unrounded value.
API: The #SEQTEMP directory is now cached at startup time, so API programs can change the OS current directory at will during a run without causing misdirected DISP=TEMP allocations.
2.16
Fixes:
User Language FRN bug fixed.
Load: Includes workaround for the missing information on [Linux + the current version of Wine], namely the available-memory figure.
Load: Fixed intermittent bug with scratch files that ended in X'14' (EOF) character.
New:
There is now a limited selection of M204-style security support, allowing for user ID/password maintenance and lookup with the LOGCTL command etc., plus $function equivalents ($LOGCTL etc).
$ENCRYPT is now supported.
Support for Sirius functions: $A2E, $E2A (synonyms for $EBCDIC and $ASCII), $X2D, $D2X, $HEXA, $IHEXA (last two synonyms for $X2C and $C2X).
A new hashing $function $SHA1.
Other:
New 'QUIET' parameter for reducing the audit trail output on big one-step data loads.
Correct handling now for API programs that shut down DPT (call Destroy()) and then restart without starting a new OS process.
2.17
Fixes:
Fixed file handle leak during index build merge.
The use of OS file handles is now parameterized to allow the index build merge phase to work if their supply is limited (e.g. smaller machines or OS configurations).
API:
New wrapper class for password file.
2.17.1
Source code:
The 2.17 changes should now compile with mingw/gcc.
Warning messages on 279 source files (missing final EOL) should no longer occur.
Updated bundle of implementation notes, scripts etc. from Roger.
2.18
Record store overheads have been reduced somewhat in the common case where the same fields are used repeatedly.
User Language:
STORE RECORDS statements now execute a tad faster, but only when all field names are known at compile time (i.e. there are no %% field name variables).
API:
API programs now have the option to skip the preparation of the field names when they're the same each time.
It is now essential to explicitly use an APIStoreRecordTemplate object (it's no longer just a synonym for RecordCopy).
2.19
DBA commands
DELETE FIELD is now supported.
REDEFINE FIELD support now covers the full range of options. The Language Guide contains assorted notes about how this command is processed on DPT when the different options are used.
Single-step deferred updates
Control and diagnostics have been tidied up somewhat. The main things are: follows.
There are two new parameters to control the process - LOADMEMP and LOADDIAG. Note that the two optional arguments that were previously supplied on the OPEN command (or equivalent API call) are no longer valid.
Message number DPT.3110 no longer exists, and therefore cannot be MSGCTL'd as previously recommended.
Index build stats are now more accurate in one or two cases where estimates are given.
Diagnostics and message texts in general may differ slightly compared to the last version. For example "single-step deferred update mode" is now sometimes referred to as "fast index build mode" reflecting its multi-purpose usage.
Since-last stat groups (diagnostics level 4 only) are back to the M204-like "BLDX" rather than than "1SLX".
API
A small extension to V2.18 aimed at handling multiply-occurring field groups.
Non-standard "const const" declaration from V2.18 removed.
2.20
Server daemons
Fixed bug where there was a small window of opportunity for a daemon with a particular ID to be restarted before it had completely logged off. Taken together with the next point this exposure caused intermittent corruption of daemon-related control blocks, and sometimes could have led to the entire web server crashing.
There is now better control of server daemon auto-respawning. On systems with a heavy turnover of these daemons the auto-respawn mechanism would prior to this version occasionally get stuck for a while repeatedly starting and stopping daemons but unhappy with the exact number it had. This would cause a flurry of unnecessary daemon user logon/logoff activity. (NB. Under ideal conditions you would hope the same set of daemons would run all day without auto-respawn coming into play at all, but realistically there will always be the occasional restart/logoff for one reason or another, triggering the process).
Daemon control codes (as issued to e.g $SPAWN) are now always uppercase. This is hopefully of minor user impact, but ensures the windows file system, which is case-insensitive, will not have problems identifying auto-generated output files.
API
The DeleteField() function from 2.19 now has an API hook.
Cosmetics
Windows Vista for some reason did not show the DPT host console "operator" menu in the same colour as the other items at startup time. Now it does.
2.21
Fixes
Fixed bug in V2.18 where null %variable in User Language STORE RECORD would crash the request.
The web server now is *even less* likely to crash than it was made in V2.20 last week. (A benign daemon-spawning error was incorrectly being treated as serious enough to terminate the server).
2.22
Email
$EMAIL function for sending email directly from User Language.
Two new commands, USE $EMAIL and =EMAIL, which hook into the same underlying code as the UL $function. The USE method provides similar functionality
to "USE PRINTER ... WRITER=SMTP" on Model 204.
All the above features rely on the freeware "Blat" mailer utility being present on the DPT host machine, and that it has been given the location of a mail server. The $function notes describe the simple process of doing that.
Other enhancements
The $FILE_OPEN function now supports 'TEMP' disposition.
There is a new flavour of the "command line history" feature on the client. Previously ALT+uparrow invoked something similar to RETRVKEY on Model 204. That facility remains but now you can add SHIFT into the mix (ALT+SHIFT+uparrow) to select quickly from all previously-issued commands.
Misc
Client-host connection is now faster if a fresh $function catalog is downloaded at each connection time (the default). The ~0.5 second delay appeared to be exclusive to systems running on Windows Vista, but in any case it's gone now. The transfer is done in a more efficient way.
2.23
Fixes
When using REDEFINE FIELD to convert from ORD CHAR to ORD NUM, the dbms now correctly merges index entries for string values which convert to the same numeric value, such as "100" and "+100".
It is also now a valid option in the same situation to allow invalid numeric values through as zeros. In other words the same as during a fresh data load.
Fixed the bug where the system would crash at file-close time if multiple files had been open in single-step deferred update mode.
Resetting FISTAT to clear the deferred update bit is not recommended, but it does work now.
Fixed the incorrect reporting of a file as physically inconsistent which occurred on small-buffer-pool onlines when opening a file in the middle of another user's transaction.
Parameter 2 of the $COMMBG function now sets the default file. In other words it does an automatic OPEN rather than OPENC.
Other
The API OpenContext...() function for multi-step deferred-update mode has its format option back again (e.g. NOCRLF). This parameter was accidentally removed in V2.14.
Clarified the informational message "Too many users of the same buffer page?!" which occured in certain API runs in garbage-collected languages such as Python, or more improbably in contrived User Language situations such as record loops nested over a hundred deep.
New load-related parameter LOADCTL. DPT no longer tries to auto-detect whether it is running on Linux/Wine.
2.24
Source code
A range of source code rejigs to allow clean compilation with current gcc and its default warning settings (very fussy!) Not all are listed here but these are the main ones. Thanks to Roger M for assistance with this. {Note - actually the next version below is the one that finally compiles cleanly.}
Assorted minor changes to cater for features deprecated in recent gcc versions, such as assigning string literals to char* (i.e. not const char*) and "friend injection".
Explicit casts from double to int inserted rather than having it done automatically. This is fairly common as all the numeric work in the DBMS and the UL compiler is floating point, even when only integer is required.
Numerous constructor initializer lists reordered to match the member order within objects (i.e. actual initialization order).
Several casts inserted to quieten gcc's worries about printf() "%d" format items not matching supplied arguments.
bbfloat.cpp now #includes <math.h> to cater for the bunch of functions that were moved there from <float.h> in recent gcc. The relevant function here is _nextafter().
All source code files now have lowercase names, and all corresponding #include statements that were previously uppercase or mixedcase are now lowercase. This allows a wider range of utility tools to be used on the source code as some tools are case-sensitive. The affected modules with this are: CFR, Cmds_OS, FLGS, ulstmt_IO, randomMT and 14 separate page_* (page_A, page_F etc).
2.25
Database C API and DLL
A new usage option is to call the DBMS from C or one of the many languages that can do C-style function linkage, linking dynamically to a pre-prepared DLL (no DPT source compiliation required!)
The tidied up and expanded API programming guide has full details.
The downloads page has a package containing the DLL, C headers and a load of sample C code.
User Language Fixes
FIND WITHOUT LOCKS previously sometimes placed an exclusive lock on the found set (better than the other way round!) Now FDWOL always works correctly.
$LOWCASE and $UPCASE correctly process complete strings containing X'00' characters, rather than only going as far as at the first X'00'.
FIXED DP 0 variables no longer overflow/wrap when assigned a value requiring more than 31 bits.
DBMS Fixes
The DIRRCD stat now updates correctly after table B searches.
Source code
Well it seems there actually were a few more things to do after the last version to get a complete clean compile with default gcc. The download contains a brief walkthrough of how to get it to compile and run in the free Eclipse IDE.
Specifically a huge number of int variables (i.e. signed) replaced with C++ STL size_t, (i.e. unsigned) when indexing into STL strings, vectors, etc., to stop complaints about unsafe comparison between signed and unsigned. Where usage is not trivial, the existing ints are left in place and cast to size_t
(i.e. the standard ANSI behaviour), which is wordier but prevents obscure bugs
creeping in from this exercise.
Miscellaneous extra functionality
UL $LOWCASE has an extra parameter to optionally lowercase the first character too.
UL $ALPHA and $ALPHANUM each have an extra parameter to optionally allow lowercase characters through.
The help buttons, menus etc. throughout the apps should now route to the correct documentation chapters on a wider range of browsers, rather than just opening at the top of the relevant documents. The change takes account of more recent browser-control-mechanism conventions for targeting HTML anchors. DPT allows it to be done two different ways now.
2.26
This version contains a small but significant change to the build settings for the downloadable .exe and .dll files. The change addresses a variety of intermittent crashes some users were seeing, especially on high-throughput systems on multi-core hardware. There are no other major changes in this version.
Details
The change is necessary because of the Microsoft C++ standard library problem described here: KB 813810. In summary, the old standard library <string> class had an implementation peculiarity which meant that such objects did not behave "reasonably". More specifically, when variables are declared as constants to the C++ compiler and are only ever read from, it's reasonable to assume that more than one thread can read them safely without risk of corruption. With this library however, hidden performance-related code inside the <string> class made modifications to the underlying data even when just reading a variable's value, resulting in possible corruption if two threads did so at the same time.
Effects of the change on existing installations
Of the various Microsoft recommended solutions, the one used is the only one which can be applied in a realistic time frame, namely to remain using the same compiler but disable this performance-related feature of C++ standard library <string> variables. Or at least partly so: To counteract the detrimental effect it had on the performance of User Language programs doing a lot of string work, most (the single-threaded parts) of the UL compiler and evaluator remain using the old C++ <string>.
DPT as a whole is therefore not too badly affected
performance-wise by this change, although it's still possible users may find specific operations which are unusually slow. Such cases can easily be addressed later if required so please report any you find, or indeed any other strange behaviour of the system in this version.
2.27
Installer
The downloadable ".exe" file which unzips and installs the DPT demo system is now prepared using a different scripter tool, addressing some problems with the old installer. At the same time the installation reference section in the system config guide has been clarified and expanded with more information and more discussion of the subject.
The main changes in the install procedure from V2.26 can be summarised as follows.
- The installer wizard offers more helpful and relevant comments/suggestions at each stage.
- The default install location is now "C:\DPT" rather than a subdirectory of "C\Program Files". This is to make it so both the installer and DPT itself can be run without administrator privileges on Windows Vista (if you are to avoid the confusing and non-intuitive UAC/folder virtualization behaviour).
- You are now offered the option to install various subsets of the download package, for example during a version upgrade choosing just the .exe files.
- The wizard no longer always creates a new directory for "DPT Version XX", which often resulted in an untidy build-up of old version directories. Instead, the same directory is always used by default, with an optional backup being taken of any existing files that get overwritten.
- Uninstall can now more completely remove DPT, in simple cases leaving no trace at all, rather than an assortment of old audit trails and ini files.
- An uninstall.exe file is now created as part of the install. This just offers another quicker way of removing DPT, in addition to the usual method of going via Control Panel.
- As a bonus, the DPT content is all zipped much more effectively (LZMA algorithm) inside the install package, which is now a lot smaller.
Note however that because it's a brand new install script, it will not uninstall any DPT versions 2.26 or before. You should do this separately via Control Panel before installing V2.27, unless you want to keep the old version hanging around to confuse you. Then in V2.28.... etc. things should work more smoothly.
For interest, the old scripter tool was called "GKSetup", and the new one used for this version is "Inno Setup". Unless anybody has any problems with it, Inno Setup will be used for the foreseeable future because it offers much better functionality.
Other miscellaneous changes in this version
The ERMX parameter can now be reset using a value of -1, disabling its effect. Previously -1 was taken to mean zero (i.e. super-harsh effect).
Clearer messages are issued when attempting to open non-DPT files.
Client
The editor "Find and Replace" dialog is now mapped more conventionally to Alt+E+E instead of Alt+E+R.
The fullscreen view now supports simulated PF key presses in response to mouse clicks. More details...
".js" is now one of the default "non-UL" file extensions for files opened in the editor.
Database C API DLL
Null string parameters can now be supplied to obtain the default action in a handful of places where "" was previously required.
2.28
Commands and User Language
Parsing of a stand-alone DECLARE IMAGE statement no longer "eats" a following label if there is one on the very next line. E.g. this now works instead of the compiler complaining that there is no label:
DECLARE IMAGE SOMEIMAGE COMMON
L:
NOTE 'SOMETHING'
Fixed bug which caused a crash in the pattern matcher. For example patterns with leading wildcards, tested against strings >128 characters long. This will not crash now:
IF $PAD('','A', 129) IS LIKE '*A*' THEN PRINT 'LIKE'
Messages are now more logical when a USE command is issued while another USE destination is already open. For example the familiar "output set to normal" message was misleading if the previous USE destination remained open. This is what it says now:
USE PROC -6
USE OUTJUNK
*** 1 DPT.1504 Sequential file is not allocated: OUTJUNK - previous USE destination remains active
DBMS
Multi-step deferred updates with NOCRLF + NOPAD options. Zero length field values no longer crash the Z step.
Client
The find and replace dialogs remember the last string searched for if there is no text selected in the editor.
The debugger command WATCH F.xxxx to manually add a watch on the value of field "xxxx" now works again. Generally the PAI check box is much more useful anyway.
Better handling of middle mouse click in fullscreen view.
Documentation
The CSS definitions have been tidied up, so certain parts of the docs (e.g. code samples scattered throughout) display more clearly on more browsers.
Further tweaks to F1/help handling within apps (see also V2.25) to support more browsers, and more potential help doc directory name unusual variations.
2.29
System Management
DPT host can now be run without its GUI if the appropriate command line parameter is given when starting it up.
It also now has all the necessary internal hooks to be invoked as a Windows Service, so long as the aforementioned NOGUI mode is used. The subject of running DPT host as a service is covered in more detail in the system config guide.
The DPT host console has a new tools dialog for conveniently installing DPT services.
Commands
A new DPT custom command DISPLAY AUDIT for accessing the audit trail. This is obviously useful in the new NOGUI mode, but also offers simple "SirScan" like functionality for general use.
2.30
Client
Large procedures edited remotely (not on the same machine) now open and save in the editor in a fraction of the time. (It no longer bothers to ACK every single line back and forth over the socket.)
Procedure directories containing a large number of procs, and field listings with a large number of fields, now open much quicker in the navigator view. Same reason.
Further usability tweaks to the editor find/replace dialog.
Similarly to the host (see below) there is a new option to make it so the client can be closed without a prompt, even in the middle of a debug session or fullscreen application. Previously the user would be prompted to terminate the debug/fullscreen session, which is still the default behaviour.
Host console
When run in GUI mode (the usual) DPT host by default now interrupts Windows closedown, as if the user had hit Alt-F4. The prompt in both cases can be disabled with a new option. This is irrelevant in service mode, where closedown is initiated via the windows service manager, and would never prompt.
User Language
Some fixes for obscure situations:
The IS PRESENT and IS LIKE operators can now be terminated in expressions by right bracket or +/-, (these two slipped through the net when the fix for the other "keyword" operators in V 2.11 was done). Example: %HASCLAIMS = CLAIMS06 IS PRESENT+(CLAIMS07 IS PRESENT).
In fullscreen processing, very rapid multiple PF key presses (or any presses during $FAKEENT) no longer trigger "attention", unless the key actually was "attention". In the case of $FAKEENT in particular this may cause subtly different behaviour for auto-refresh programs, since a preemptive PF key press by the user now returns normally (with %SCR:PFKEY set) rather than driving the ON ATTENTION unit.
In certain cases ELSEIF statements which failed to compile because of invalid conditional expressions would cause user restart. This is now fixed. e.g. ELSEIF %X EQ GARBAGE where there is no such field as GARBAGE.
It was previously possible to contrive a case where a complex subroutine would be called recursively, recycling its input parameters but with the order mixed round, where one or more incorrect values would be passed into the inner call frame.
Miscellaneous new functionality
New $LZW function providing mild but convenient data compression.
New =POST command, a wrapper to the Windows PostMessage API.
Download latest version....
Further details about specific releases
V2.03: Improvements to multi-step deferred-update loads
One ongoing plan for versions 2.xx is the improvement of DPTs support for DBA activities such as loading, reorganizing and field maintenance. The change in V2.03 is in one sense a piece of preparatory work for a specialized fast-load facility coming in the near future, but it also significantly speeds up existing "simple" User Language loads (provided that the deferred update feature is enabled). See benchmarks below.
The changes can be summarised as:
- Reduction in size of the deferred update records significantly speeds up the sort, regardless (in theory) of the sort utility used. It also reduces I/O during all phases.
- The Z command now loads indexes much faster in typical cases, by building inverted lists in memory.
Note that any existing deferred-update load decks you have will need changes to the sort parameters to handle the new shorter records. The sample load job distributed with the DPT download shows the new parameters with the old commented out.
Benchmarks
The following loads were run on a 1Ghz Celeron laptop, W2K SP2, CMSort with its default settings, and with DPT MAXBUF set to 100. The job was to load 1 million records as per the sample deferred-update code distributed with the download. All times are min:sec.
A:
5 indexed fields on every record, with a variety of typical value distribution characteristics.
Version | Data | Sort | Indexes |
2.02 |
5:32 |
20:54 |
11:35 |
2.03 |
4:39 |
17:23 |
7:38 |
B:
Just load one specific field on each record. The few-valued fields give large improvements as we would expect from the new Z processing. The many-valued fields smaller improvements just from the reduced sort record size.
Field/Version | Data | Sort | Indexes |
Few valued alpha (STADIUM_NAME) |
2.02 |
1:44 |
2:07 |
1:50 |
2.03 |
1:31 |
1:34 |
0:25 |
Few valued numeric (FOUNDED_YEAR) |
2.02 |
2:01 |
4:10 |
1:01 |
2.03 |
1:43 |
3:32 |
0:31 |
Many valued alpha (NAME) |
2.02 |
1:55 |
2:59 |
3:32 |
2.03 |
1:50 |
2:39 |
3:33 |
Many valued numeric (TID) |
2.02 |
1:58 |
3:42 |
2:01 |
2.03 |
1:45 |
3:06 |
1:55 |
V2.14: One-step loads (etc.)
Continuing the ongoing V2.x exercise of improving the speed of data loads, DPT now provides similar functionality to M204's one-step ("FILELOAD") process. In addition, the internal index-build routines have received some significant performance tuning which is beneficial in various situations, including the old multi-step load process.
The following tests tests were run on the same 1Ghz Celeron laptop used above, which has 250Mb of memory. The single-step loads used the default of 75% of the available memory, and to make for a more reasonable comparison CMSORT in the multi-step runs was configured to use 75% as well (hence the sort is somewhat quicker than it was in the V2.04 benchmarks above).
A:
5 indexed fields on every record, with a variety of typical value distribution characteristics. (Data generated by the LOAD.RANDOM.DATA program supplied in DEMOPROC).
Remarks:
- The single-step process is clearly the fastest by a long way.
- It becomes even better with larger data volumes, as sort-related overheads start to dominate the multi-step runs.
- Even discounting the cost of the sort, both the data load component and the index load component are faster using single-step.
- Comparing just the multi-step index load figures between this version and the last highlights the tuned index-load code which is part of this release.
- The bottleneck in these figures overall is actually now the User Language runtime during the data load phase. This will be addressed with a special fast load utility in a future release. API users (i.e. bypassing UL) should see even better results with the current release though.
Run type | Data | Sort | Indexes | Total |
1 million records |
Multi-step V2.12 |
4:25 |
11:32 |
8:34 |
24:31 |
Multi-step V2.14 |
4:44 |
11:22 |
3:28 |
19:34 |
Single-step |
3:04 |
- |
1:08 |
4:12 |
5 million records |
Multi-step V2.12 |
22:16 |
1:32:07 |
46.52 |
2.41.13 |
Multi-step V2.14 |
21.55 |
1.32.44 |
31.17 |
2.25.56 |
Single-step |
16:45 |
- |
4:37 |
21:12 |
B:
5 million records containing a single field, FOUNDED_YEAR (few-valued).
Remarks:
- Few-valued fields like this (just 50 values) gain most from the use of inverted lists and show the greatest improvement from using the single-step process.
Run type | Data | Sort | Indexes | Total |
Multi-step V2.12 |
8:35 |
13:53 |
2:30 |
24:58 |
Multi-step V2.14 |
8:24 |
13:51 |
2:31 |
24:55 |
Single-step |
5:46 |
- |
0:03 |
5:49 |
C:
5 million records containing a single field, TID (many-valued).
Remarks:
- The single-step run is much faster, although not to quite such a great extent as with the few-valued field. The benefit here is not from using inverted lists (there are none) but from sequential I/O savings.
- With a large b-tree to build (5 million entries), the tuning in this area of the code shows up in the multi-step index component timings.
Run type | Data | Sort | Indexes | Total |
Multi-step V2.12 |
8:29 |
12:51 |
12:41 |
34:01 |
Multi-step V2.14 |
8:17 |
12:44 |
5:52 |
26:53 |
Single-step |
6:03 |
- |
0:58 |
7:01 |