Change History

Old History ChessTab Top

Changes are listed in reverse chronological order. Followed by version comments and general comments added from time to time.

Changes

 

15 July 2024

ChessTab version 8.0 released.

The import process is replaced.

Imports can be intentionally interrupted at any time. Repeating the import will verify what has already been done and resume the import from there.

A choice between two import strategies is offered once the number of games to be imported has been counted: one quicker for small imports and the other quicker for large imports.

The large import strategy is available only for databases on the Berkeley DB, SQlite 3, and Symas LMDB, database engines via a supported Python interface.

New versions of dependencies chessql, pgn-read, solentware-base, solentware-grid, and solentware-misc.

This version of ChessTab is incompatible with earlier versions. Existing databases must be exported using the old version and imported using this version.

 
 

30 November 2023

ChessTab version 7.1.3 released.

Improve PGN error dialogues for insert and edit of games and repertoires. These now mention if the errors are in Tag Pairs or Movetext, or both.

New versions of dependencies chessql and pgn-read.

 
 

27 November 2023

ChessTab version 7.1.2 released.

Fix export of PGN files from Symas LMDB databases.

 
 

26 November 2023

ChessTab version 7.1.1 released.

Fix display of game records with errors and no movetext.

Usually imported from non-PGN files.

 
 

8 October 2023

ChessTab version 7.1 released.

Add Symas LMMD to databases which can be used by ChessTab.

Add utility to trim PGN file for resuming import after failure. Import progress reports adjusted to assist control of trimming process.

Improvements to import process.

Temporary backups created before import and deleted afterwards.

 
 

27 April 2023

ChessTab version 7.0.2 released.

Prevent drag relocation of insertion cursor and insertion of newline by Control-o when editing game scores.

This fixes some exceptions involving typed-in moves and avoids unwanted visible effects in selections on Microsoft Windows.

 
 

26 April 2023

ChessTab version 7.0.1 released.

Add fixes for two problems seen when adding RAVs concurrently with engine analysis. It proved impossible to determine actions to reliably cause the problems to test the fixes.

Opportunity taken to fix placement of RAVs for last move in game and for a black move.

 
 

31 March 2023

ChessTab version 7.0 released.

New versions of dependencies: solentware_base, solentware_grid, and solentware_misc.

Tool chesstab.tools.berkeleydb_du_splice_fix is provided to repair databases broken by doing a games import to a populated database.

An interface added to solentware_base can be used to store ChessTab databases on Symas LMMD.

Scrolling the main list of games by dragging the scrollbar slider does not work properly yet: the outcome is scrolling one game at a time. Scrolling an index list of games this way works fine.

An interface added to solentware_base can be used to access ChessTab databases on Berkeley DB via Tcl via tkinter if Berkeley DB is configured to support Tcl access. Berkeley DB databases earlier than version 5 cannot be accessed with Python 3.10 or later with the Python bsddb3 or berkeleydb packages

The import games action cannot be used in the Tcl interface. Two sample modules, dbtkinterdu_dir and dbtkinterdu_file, provide a workaround but give no indication of progress.

 
 

11 January 2023

ChessTab version 6.0.1 released.

Fix menu option 'Engines | Show Engines' problems added at version 6.0.

 
 

19 November 2022

ChessTab version 6.0 released.

Fix PGN import to Berkeley DB databases at Python 3.10 and later.

Use unicode chess pieces by default.

Fix evaluation of CQL ray filters containing just piece designators; and evaluation of empty square piece designators.

Many coding style changes prompted by tools such as pylint.

New version of dependencies: chessql, pgn-read, solentware-base, solentware-grid, solentware-misc, and uci-net.

New dependency solentware-bind.

 
 

14 October 2022

ChessTab version 5.0.6.1 released.

Fix deletion of items which are not moves, such as comments, from PGN movetext.

The problem was introduced at version 5.0.

Editing and inserting these items was not affected.

 
 

7 April 2022

ChessTab version 5.0.6 released.

New version of dependency: solentware-misc.

 
 

6 March 2022

ChessTab version 5.0.5 released.

New version of dependencies: chessql, pgn-read, solentware-base, solentware-grid, solentware-misc, and uci-net.

[tool.black] section added to pyproject.toml file.

 
 

8 October 2021

ChessTab version 5.0.4 released.

New version of dependencies: chessql, pgn-read, solentware-base, solentware-grid, selentware-misc, and uci-net.

Installation from wheel (*.whl) format by 'pip' supported.

 
 

3 October 2021

ChessTab version 5.0.3 released.

New version of dependencies: chessql, pgn-read, solentware-base, solentware-grid, selentware-misc, and uci-net.

Installation instructions are changed: use 'pip' to install ChessTab.

 
 

29 September 2021

ChessTab version 5.0.2 released.

New version of dependencies: chessql, pgn-read, solentware-base, solentware-grid, selentware-misc, and uci-net.

Apply black formatting, pydocstyle hints, and many pylint hints.

Remove *.html files derived from *.rst files.

Fix 'Ctrl Enter' binding for listing games matching queries.

Prefer berkelelydb package over bsddb3 package if both are installed. bsddb3 is not supported at Python 3.10 and after. berkeleydb is supported at Python 3.6 and after.

 
 

2 May 2021

ChessTab version 5.0.1 released.

New version of dependency uci-net.

Fix connecting to a chess engine using TCP on a LAN. Connecting to a chess engine on same host as ChessTab, without TCP, was not affected.

 
 

29 April 2021

ChessTab version 5.0 released.

New versions of dependencies: pgn-read, solentware-base, and solentware-grid.

The options for generating PGN output are collected together in the 'Export' submenu of the relevant popup menus. Previously these were usually, but not always, in the 'Database' submenu. The names in the menu entries are changed to a more correct description of the content generated. Two options are added where relevant: a PGN export format containing all tags and movetext excluding recursive annotation variations, but no comments of any kind; and a PGN import format containing the whole game score.

Games which do not conform to the export format in the PGN specification, or the PGN input format defined in ChessTab, are not output to *.pgn files. The ability to output game scores to *.txt files is added where not already present alongside existing PGN output options.

A problem highlighting entries in lists of games after discarding the corresponding displayed game score is fixed.

Fix problem, a program crash, when function key F11 is used to display a selection rule or position query and the record is not yet selected.

Fix problem, incorrect records listed, when a list of games selected by index is filtered by a value where no entries exist starting with the value.

Fix problem, a program crash, when quitting with both a database open and two selection rules open, one loaded from database and used to list games and the other opened by 'Selection | Rule' menu option but not used yet.

Fix problem inserting a ';' comment or '%' escape sequence with the editor. The terminating newline is now accepted and a warning given if the newline is not entered or extra text exists after it. The valid escape sequence is not stored on database, which is what happens to any found in PGN files.

Fix problem, a program crash, when a move is done, or undone, with the 'Down' or 'Up' key after exiting from selecting a variation with pointer action on the board or game score.

The available insert and delete PGN Tag options are included in the 'PGN' submenu of the relevant popup menus.

The position reached by the move to the left of reserved, '<..>', or escaped line, '\n%...\n', tokens when current is displayed, rather than the starting position of a game.

The popup menus with no commands in selection rule and CQL query records displayed from a list with the 'Show', 'Insert', 'Edit', 'Edit and show', and 'Delete', options are not posted.

The popup menu posted by right-click when a choice between variations is available now shows variation selection options, rather than options to go to locations in the score.

Fix regression, introduced at ChessTab-4.3.2, where a crash occurs if the move before an illegal move becomes the current move while chess engine analysis is displayed.

Variation selection highlighting in the game score is removed consistently on leaving situations where it should be highlighted.

Change pointer action for switching to and from engine analysis in a game or repertoire to 'Alt Left click'. The keystoke equivalent remains 'Alt F8'.

The pointer and keystroke options for navigating a game or repertoire displayed by the 'Show', 'Insert', 'Edit', 'Edit and show', or 'Delete', actions are available from start, rather than after first use of popup menu equivalent.

Fix 'Run Engine' commands available from menu option 'Engines | Show Engines' so engines which require command line arguments work correctly. Running such engines from the 'Engines | Start Engine' menu option was, and is, fine.

 
 

2 January 2021

ChessTab version 4.3.2 released.

New version of dependency pgn-read.

Fix problem wrapping '{..}' comments inside an '{Error}' comment when an error is found in a game in a PGN file.

Fix problem wrapping '--' movetext in the main line of a game in an {Error} comment. '--' movetext in Recursive Annotion Variations '(...)' is not affected.

Fix problems displaying records generated from text with little resemblance to valid PGN. These problems usually caused ChessTab to crash.

 
 

20 October 2020

ChessTab version 4.3.1 released.

New versions of dependencies pgn-read, solentware-base, solentware-grid, and solentware-misc.

A correction to pgn-read allows incomplete moves at the end of a game or Recursive Annotation Variation to be stored, when editing or inserting, as errors within a '{}' comment. This happens when importing too, but implies a problem in creating the affected PGN file.

Chess engines which need command line options, such as '.../gnuchess --uci', can be started from the 'Engines | Show Engines' menu option. The 'Engines | Start Engine' menu option did not suffer the problem.

The 'Tools | Board ...' menu options work again after being broken at version 4.3.

More detail is available, shown if requested, when reporting errors.

Dialogues are always closed if their parent widget is closed.

A database file created by the dbm.ndbm database engine is now deleted by the 'Database | Delete' menu option. The other database engines did not suffer the problem.

 
 

2 September 2020

ChessTab version 4.3 released.

New version of dependency pgn-read.

Adjustments to fit the new interface provided by pgn-read-2.0.

Move numbers are displayed by default, but can be switched on or off.

Recursive Annotation Variations (RAV) and comments always start on a new line when displayed.

PGN errors are put inside a comment which continues to the end of the RAV, or the end of game if the error is not in a RAV. If the error is fixed by editing the game the sequence is restored to the game outside the comment.

The list of games matching the current position in the active game shows just the move into, and the move out of, the position along with any '(' and ')' marking RAVs. Colour highlighting to show difference from the active game is retained. This summary game score appears before the player names rather than after.

On Microsoft Windows when using the pointer to enable scrolling of the list of games matching a position, it is necessary to double-click rather than click when done over game details.

 
 

26 July 2020

ChessTab version 4.2 released.

New version of dependency chessql.

ChessQL statements now follow the CQL version 6.0.4 syntax rather than the version 5.1 syntax.

The set of supported filters is unchanged: 'piece designators', 'and', 'or', 'flip', 'fliphorizontal', 'flipvertical', 'flipcolor', 'shift', 'shifthorizontal', 'shiftvertical', and '{}'. Other filters continue to be ignored.

Punctuation is more in line with CQL: for example 'shift{k q}' is accepted where 'shift { k q }' was needed before.

Chessql queries including empty square range specifications such as _d3 or _d3-6 or _g-h1 or _c-f2-3 now give correct answers when evaluated by ChessTab. An abbreviated form of the translated Chessql query was generated but a problem in solentware-base means the full form must be used.

 
 

13 May 2020

ChessTab version 4.1.2 released.

New versions of dependencies chessql, pgn-read, and solentware-base.

Positioning the main list of games at end of list no longer crashes ChessTab when using the SQLite3 database engine via either apsw or sqlite3, after a problem in solentware-base is fixed. Lists sorted by index were not affected.

The castling option problem in pgn-read had a minor impact on ChessTab: the analysis widget for the affected positions incorrectly stated a move to be invalid, but the list of games containing the position was shown.

Now the list of games for an affected position is not shown: because the search key calulated, although correct chess, is not the one referencing the list of games.

The tool chesstab-4-1-1_castling-option-correction.py is provided to the fix the search keys.

Chessql queries including empty square range specifications such as .d3-6 or .g-h1 or .c-f2-3 no longer crash ChessTab but state the query cannot be evaluated. Queries including simple empty square specifications such as .d3 still give incorrect answers rather than state the answer would be wrong.

Chess engine analysis for affected positions replaces an invalid move report when a position is analysed to greater depth or with more variations.

 
 

24 April 2020

ChessTab version 4.1.1 released.

New version of dependency solentware-base.

The mistake that prevented the create_chesstab_database tool working is fixed.

ChessTab was not affected. In particular it would create databases using the database engine chosen from the internal priority list.

 
 

14 April 2020

ChessTab version 4.1 released.

New version of dependency solentware-base.

Chesstab databases can be created with the UnQLite, Vedis, dbm.gnu, and dbm.ndbm, database engines.

 
 

15 November 2019

ChessTab version 4.0.1 released.

New versions of dependencies chessql and pgn-read.

Games containing valid PGN movetext such as Qb2c3 or Qb2xc3 are now displayed in reasonable time when selected from a database, rather than taking a few hours. A similar improvement is seen when navigating the game. See comment dated 14 November 2019.

The effect was definitely noticable when a game score contains 500 full-moves or more. A typical game has less than 60 full-moves, but annotations (Recursive Annotation Variations) may bring the effect into play.

Games with more than 500 full-moves are displayed with one full-move per line, and are prefixed with the move number. No attempt has been made to present RAVs in an orderly manner.

Chess engine analysis is not affected, unless a depth approching or more than 1000 (half-moves) is requested, because each variation is given on one line which might wrap depending on the size of the application window.

The move number of the next move played is always given in the analysis area.

Do not forget the upgrade step described in ChessTab version 4.0 release note if it is needed.

 
 

11 November 2019

ChessTab version 4.0 released.

New versions of dependencies.

Databases using the SQLite3 database engine, via either apsw or sqlite3, or the Berkeley DB database engine, via bsdbd3, created by earlier versions of ChessTab have to be upgraded by:

'python -m chesstab.tools.upgrade_3_to_4'

after which the database will be accessed correctly by ChessTab 4.0 but not by earlier versions.

Databases using the DPT database engine created by earlier versions of ChessTab are already in the form needed at ChessTab 4.0.

Project directory structure changed from chesstab to chesstab/chesstab where the implementation moves to chesstab/chesstab and project descriptions and setup.py remain in chesstab.

 
 

19 October 2019

ChessTab version 3.0.7 released.

Selection query phrases with keywords 'from', 'above', 'to', and 'below' are handled correctly again after a problem introduced at ChessTab version 1.0 was fixed.

Fix problem which prevented a database being opened when selection queries or Chess Query Language (CQL) queries which were displayed from a database, now closed, are still displayed.

 
 

2 October 2019

ChessTab version 3.0.6 released.

Games containing valid PGN movetext such as Qb2c3 or Qb2xc3 are now displayed when selected from a database, rather than causing a program crash. See comment dated 2 October 2019.

 
 

21 May 2019

ChessTab version 3.0.5 released.

New versions of dependencies.

When entering or editing a game score the popup menu option 'PGN | O-O-O' now works at the current end of game score or variation. 'PGN | O-O-O' continues to work where a new variation is started. The keystoke equivalent, Ctrl o, continues to work in these cases.

 
 

5 March 2019

ChessTab version 3.0.4 released.

New versions of dependencies.

Fix two problems at solentware-base-3.0.1.

 
 

22 February 2019

ChessTab version 3.0.3 released.

Fix problem where no games are exported to a PGN file from a sorted filtered list of games if the database engine is Berkeley DB.

The problem did not occur when the list of games was created by the equivalent selection rule.

 
 

21 February 2019

ChessTab version 3.0.2 released.

PGN files of subsets of the games on a database are created when requested from a list. Previously PGN files for the whole database or single games were the only ones created successfully, with a program crash the likely outcome in other cases.

New version of dependency pgn_read.

 
 

10 January 2019

ChessTab version 3.0.1 released.

New version of dependency pgn_read.

PGN pawn capture sequences such as '1. e4 d5 2. cxd5', where cxd5 is illegal, were treated as the legal '1. e4 d5 2. exd5' sequence but the illegal sequence was recorded. ChessTab was not affected internally by the problem, but the illegal sequence was put in PGN files generated by ChessTab.

 
 

18 November 2018

ChessTab version 3.0 released.

New versions of dependencies.

Databases using the SQLite3 database engine created by earlier versions of ChessTab, via either apsw or sqlite3, have to be upgraded by:

'python -m chesstab.tools.chesstab_sqlite_table_name_upgrade'

after which the database will be accessed correctly by ChessTab 3.0 but not by earlier versions.

Databases using the Berkeley DB or DPT database engines created by earlier versions of ChessTab are already in the form needed at ChessTab 3.0.

 
 

17 November 2018

ChessTab version 2.2.3 released.

Package README file brought up to date.

 
 

9 November 2018

ChessTab version 2.2.2 released.

A problem evaluating partial position search when using DPT database engine is fixed.

 
 

8 November 2018

ChessTab version 2.2.1 released.

Two problems when filtering the displayed list of games are fixed.

 
 

8 November 2018

ChessTab version 2.2 released.

New versions of dependencies due to renaming several projects. See ChessTab version comments for detail.

 
 

31 August 2018

ChessTab version 2.1.2 released.

New versions of dependencies.

Corrections to the basesup dependency allow the bsddb3 interface to Berkeley DB, and the apsw interface to SQLite3, to be used to do large updates to databases. The sqlite3 interface to SQLite3 did not suffer the problem.

 
 

31 August 2018

ChessTab version 1.0.2 released.

New versions of dependencies.

The latest ChessTab version 2.n.n release should be used. This note just records the change to the old version.

Corrections to the basesup dependency allow large updates to databases using the bsddb3 interface to Berkeley DB in this old major version of ChessTab. The apsw and sqlite3 interfaces to SQLite3 did not suffer the problem.

 
 

31 August 2018

ChessTab version 0.41.11 released.

New versions of dependencies.

The latest ChessTab version 2.n.n release should be used. This note just records the change to the old version.

Corrections to the basesup dependency allow large updates to databases using the apsw interface to SQLite3 in this old major version of ChessTab. The sqlite3 interface to SQLite3 did not suffer the problem. The bsddb3 interface to Berkeley DB did not suffer the problem.

 
 

12 June 2018

ChessTab version 2.1.1 released.

Insert the PGN text of new moves at the end of the game or a variation in the correct place when the current end move has variations.

 
 

30 October 2017

ChessTab version 2.1 released.

Improvements to PGN error handling based on changes to pgn package. The selection rule interface is made more like the partial position (chessql) interface.

 
 

4 October 2017

ChessTab version 2.0.1 released.

New versions of dependencies.

 
 

4 October 2017

ChessTab version 1.0.1 released.

New versions of dependencies.

 
 

2 October 2017

ChessTab version 2.0 released.

The concurrent changes to basesup are sufficient to justify changing the major version number. Follow change of major version number in basesup.

 
 

2 October 2017

ChessTab version 1.0 released.

The partial position searches are replaced by Chess Query Language (CQL) searches. CQL support is incomplete but the scope of partial position searches is covered in a superior way.

The database indicies are changed to support the CQL searches.

The concurrent changes to basesup are sufficient to justify changing the major version number, as are the partial position and database index changes. Follow change of major version number in basesup.

Chess engines running on other computers can be used to analyze positions.

Move calculation of database index values here from the pgn package.

This version was released at same time as ChessTab version 2.0.

 
 

6 August 2017

ChessTab version 0.41.10 released.

New version of dependency which stops PGN Tags with '"'s in the value being treated as an error.

 
 

31 January 2017

ChessTab version 0.41.9 released.

Fix a problem in exception reporting.

 
 

31 January 2017

ChessTab version 0.41.8 released.

New versions of dependencies.

 
 

23 December 2016

ChessTab version 0.41.7 released.

New versions of dependencies.

Many problems in selecting games using a game selection rule such as 'White from Brown to Green' are fixed. Simpler rules like 'White gt Jones' did not suffer problems.

 
 

27 November 2016

ChessTab version 0.41.6 released.

Fix problem which prevented analysis of multiple games at once. Fix problem which prevented use of a running chess engine after it had responded with a 'forced checkmate' analysis or had been asked to analyse a position which is checkmate.

 
 

9 November 2016

ChessTab version 0.41.5 released.

Improve handling of typing O-O-O moves.

 
 

21 October 2016

ChessTab version 0.41.4 released.

Documentation changes only.

 
 

18 October 2016

ChessTab version 0.41.3 released.

Add function to analyse all positions in a game or repertoire using the active chess engines.

Improve interface to start chess engines.

 
 

2 October 2016

ChessTab version 0.41.2 released.

New versions of dependencies.

 
 

28 September 2016

ChessTab version 0.41.1 released.

Fix problems inserting and editing games, repertoires, game selection rules, and partial positions.

 
 

18 September 2016

ChessTab version 0.41 released.

Consultation games are indexed under the name of each player involved.

 
 

18 September 2016

ChessTab version 0.40.5 released.

Program crashes using the Berkeley DB or Sqlite3 database engines whenever bitarray-0.8.1 is installed are fixed.

The text of game scores, chess engine analysis, repertoires, selection rules, and partial positions is split into multiple lines between words rather than characters when displaying a particular item.

The 'Move To' and 'Filter' buttons ignore commas and periods entered in the search value when applied to player's names.

The game selection rule text is no longer included in the list of these rules, leaving just the name of the rule.

Lists of games are displayed correctly when a database is opened after game scores, repertoires, and partial positions, are left open when a database is closed.

 
 

14 September 2016

ChessTab version 0.40.4 released.

Documentation changes only.

 
 

9 September 2016

ChessTab version 0.40.3 released.

The restrictions on evaluating selection rules when using the DPT database engine are removed.

 
 

7 September 2016

ChessTab version 0.40.2 released.

A number of program crashes when using the DPT database engine are fixed.

 
 

5 September 2016

ChessTab version 0.40.1 released.

Chess engine analysis presented with sensible line breaks.

 
 

5 September 2016

ChessTab version 0.40 released.

Databases updated by this version of ChessTab cannot be accessed by earlier versions because the player name format in indexes has been changed.

The database engine specific deferred update modules for use on OpenBSD are replaced by a test for available memory in the default modules, following enabling changes in the basesup component.

The chess engine analysis functions cannot be used when running under Wine. A message is displayed at start-up if this seems to be the case and the functions do nothing rather than fail.

 

Version 7.1

Temporary backups while doing an import can take many minutes. Large imports to a database can take several hours: plenty of time for some incident to force a repeat of work already done. Small imports can go wrong: hopefully restoring from backup is a lot quicker than repeating the imports to reconstruct the database.

Version 6.0

The changes prompted by pylint, pycodestyle, and pydocstyle, led to changing the version number to 6.0 rather than 5.1 because there are a number of module name changes and public method and attribute name changes.

There are a number of fixes included but even the largest of these, which involved the addition of a new dependency solentware_bind, could reasonably have been done within changing the version to 5.1.

Version 5.0.3

The distutils package is deprecated at Python 3.10 and scheduled for removal at Python 3.12. The Python Packaging Authority assumes the use of pip supported by setuptools and packaging. The use of the setup.py module provided by setuptools, which replaces the version in distutils, is deprecated in the pip package.

Thus the 'python setup.py ...' command is replaced by the 'python -m pip ...' command in the installation instructions in ChessTab's README. At time of writing both commands work for installing ChessTab.

Version 4.3.1

Games can be displayed in the main widget by 'F11', or in a separate widget by 'Enter', for example. If the separate widget is closed while a related dialogue is displayed that dialogue is now destroyed too. Previously separate action was needed to destroy the dialogue.

Version 4.3

Starting Recursive Annotation Variations (RAV) and comments on a new line, together with indenting the moves, looks nicer in my opinion.

Moves are wrapped to a new line depending on the width of the display area. If a sequence of 20 fullmoves occur without a RAV or comment the next fullmove is forced to a new line. This seems a good compromise between tidiness and response times navigating large games. See comments dated 15 November 2019 about response times.

When moves are deleted while editing a game, the move numbers are not deleted leading to move numbers referencing no move and possibly new moves out of sequence with their move numbers. The move numbers are ignored when updating the game so all will be correct when the game is opened in a new widget. While editing, the game may look less untidy if move numbers are turned off.

Version 4.1.2

It is not clear when deleting games from a Berkely DB database on OpenBSD became a problem, or if it has always been one. The reported error led to finding the maximum allowed locks is the problem. Games with a small number of moves can be deleted but at some point the extra indicies assocatied with more moves need too many locks.

The default value of 'maxlocks' is 1000 on OpenBSD; but 0, zero, on FreeBSD where there is no problem. Setting 'maxlocks' to 0, zero, on OpenBSD does not fix the problem, but setting to 10000 has been high enough so far.

The ability to set 'maxlocks' for OpenBSD has been added to solentware-base and this is used in ChessTab to set it to 10000.

Version 4.1

Modules are added to ChessTab which use the new modules in solentware-base supporting the UnQLite, Vedis, dbm.gnu, and dbm.ndbm, database engines.

None perform as well in ChessTab as the other database engines; so some protection against accidental use is given by requiring command line options to create or open databases with them.

Version 4.0

ChessTab version changed from 3 to 4 for three reasons. The table naming convention for databases using the SQLite3 database engine or the Berkeley DB database engine is changed. When using the Berkeley DB database engine all tables are on one file rather than each table being on a separate file. Much code specific to a particular database engine is moved to the appropriate module in solentware-base (see Components page): several chesstab modules are retired and replaced by modules mixing classes providing ChessTab logic with solentware-base modules providing the database interface.

(Here, table names are equivalent to names of primary and secondary databases in Berkeley DB terminology.)

Upgrade of a ChessTab database using Berkeley DB will take a lot longer than a database, containing the same data, using SQLite3 because all the data is moved to a new location.

Some details of the code changes are given in comments dated 1 November 2019.

Version 3.0

ChessTab version changed from 2 to 3 because the table naming convention for databases using the SQLite3 database engine is changed.

Version 2.2.1

Filtering a sorted list of games now displays the filtered list immediately rather than an empty list which is replaced with the correct list after the first scrolling action.

A warning is given about a program crash which will occur, unless the specified avoiding action is taken, when an '*' is included in the filter if the database engine is SQLite3.

Version 2.2

The basesup, gridsup, pgn, rmappsup, and uci, dependencies have been renamed solentware-base, solentware-grid, pgn-read, solentware-misc, and uci-net.

Version 2.1.2

Python3.6 or later should be used to run ChessTab if the database is accessed with the sqlite3 interface.

Version 2.0.1

This version was released so downloading the version of the gridsup dependency with a part missing can be avoided. ChessTab does not need the missing part of gridsup. In other words ChessTab-2.0 works exactly like this version.

This version has not replaced version 2.0 as the current version on PyPI.

Version 2.0

The scope of the chess engine analysis functions is extended to allow use of chess engines running on other computers. Chess engines running on the same computer as ChessTab may be treated like this too. The functions to run chess engines have been moved to the new uci package which also provides a way of communicating with these engines.

Firewalls may prevent communication with chess engines on other computers unless you change settings to allow it.

ChessTab cannot run under Wine at time of writing for a variety of reasons, see Comments dated 2 October 2017, but can now get analysis done by chess engines running under Wine without itself running under that Wine environment.

ChessTab uses Chess Query Language (CQL) as a way of expressing database searches. ChessTab does not attempt to replicate the report generation done by the program provided with the CQL specification. The CQL website provided version 5.1 of the query language at time of writing (the obvious links in the contemporary Wikipedia entry for CQL were to version 3.01 which is significantly different).

At present ChessTab supports CQL's 'piece designators' and 'transform filters'. This is sufficient to replace the partial position specifications used by earlier versions of ChessTab.

The indicies which supported partial positions are extended and modified to support CQL. The number of index references per typical game is doubled, with noticable effect on updating a game and significant effect on the time taken to import large numbers of games to a database (see Comments dated 2 October 2017).

Version 0.41.9

Fix problem which prevented exception reports being written to error log when importing chess games, and prevented the occurrence being reported in the progress window for the import.

Version 0.41.7

Serious problems in the query language processing of the basesup component have been fixed.

The simplest clauses, like 'field le value', were fine. When a negation was added, 'field not eq value' for example, or a range clause was used, 'field above value1 to value2' for example, the request did not work.

Mis-spelling the 'and', 'or', and 'nor', keywords remains a problem. The query is processed but the clause following the mis-spelt keyword is ignored: 'White eq Jones ot White like Smi or Date is 2003.01.24' is treated as 'White eq Smi or Date is 2003.01.24' having typed 'ot' for 'or' for example.

Version 0.41.6

Fix the problems in analyze multiple games or analyse checkmate and avoid the work-around.

The sign either problem had occurred was new analysis would no longer appear while navigating games. The counts shown by menu option 'Engines | Position Queues' would be stuck at some probably non-zero values.

The work-around was quit the engines and restart them (and try to avoid doing whatever caused the problem).

Version 0.41.5

Add a menu action and keyboard equivalent for the O-O-O move.

When typing O-O-O there is usually a point where O-O, itself a possible legal move, has been typed. The change allows O-O-O to be input directly in cases where both O-O and O-O-O are legal moves. It remains possible, for example, to type O--O and then fill in the third O to get O-O-O.

Version 0.41.4

Put correct version numbers in list of installed packages in "Help | About".

Version 0.41.3

Analysis of all positions in a game or repertoire may be requested. All active engines when the request is made are used. The order in which a games' positions are analysed will vary between runs of ChessTab, so analysis for the board position could appear almost immediately or after some longer time.

The commands to start chess engines may be started from a list on a database in addition to a 'run program' dialogue.

A couple of faults in translating moves in the analysis produced by chess engines to PGN are fixed.

Version 0.41.2

ChessTab does not use the functions changed in dependencies.

Version 0.41.1

Error detection and reporting improved. Popup menus advertize the insert, edit, and delete actions correctly.

Version 0.41

All previous versions used the value given in the White and Black tags as a player's name for indexing purposes. This version accepts multiple player names, in either tag, separated by colons as stated in the PGN specification. See comment dated 18 September 2016.

Consultation games must not be added to a database using earlier versions.

The selection rule "White like : or Black like :" will give an empty list of games if there are no consultation games on a database: if the earlier version of ChessTab supports selection rules.

You may need to export all the games using the earlier version of ChessTab, then import them to a new database using ChessTab-0.41. Certainly if the earlier version of ChessTab is before 0.40, but only if the database contains consultation games otherwise.

Version 0.40.5

The program crashes related to using bitarray-0.8.1 were introduced in changes to the basesup component concurrent with ChessTab-0.39.1 and are fixed by changes to the basesup component concurrent with ChessTab-0.40.5.

The 'Move To' and 'Filter' buttons apply the rules introduced in ChessTab-0.40 on commas and periods to the search value entered. No need to remember not to type commas and periods which appear in player's names.

The amount of text in game selection rules is likely too variable for the list to be easily read, so show the rule's name only.

The list of games matching a position in a game or repertoire, or a partial position is always displayed, rather than being left blank until an action which causes the list to be refreshed is done. Colour highlighting is corrected in cases where a game, repertoire, selection rule, or partial position is closed while the database is closed. Closing a database, and then all the games, repertoires, and partial positions, leaves a blank screen rather than a screen showing some or all the list of games matching a partial position.

Version 0.40.4

Redundant lists of items removed from object docstrings.

Version 0.40.3

The restrictions on evaluating selection rules when using the DPT database engine have been removed as a consequence of the concurrent changes in the basesup component.

Version 0.40.2

Hide selection, repertoire, and position, actions when using the DPT database engine do not crash program as a consequence of the concurrent changes in the basesup component.

Evaluation of selection rules when using the DPT database engine do not crash program. An information dialogue states evaluation has been suppresed.

Version 0.40.1

Revert to starting each variation of chess engine analysis on a new line.

Version 0.40

New component versions pgn-0.10.2 basesup-0.21 gridsup-0.19.4 rmappsup-0.38.4.

Paragraph removed because change left out in error. See comment dated 18 September 2016.

The values in the White and Black tags are stored as given but sequences of periods, commas, and spaces, are replaced by a single space for indexing by player name. PGN files often use consistent but conflicting ways of following the layout recommended in the PGN specification for commas and periods in these tags. The change attempts to achieve more consistent indexing of games from different sources.

See these nabble.com and stackexchange.com forum questions for ways to increase the memory available to processes on OpenBSD.

Comments

5 September 2016

A 2 million game deferred update using sqlite3 takes over 9.5 hours on FreeBSD, 15.5 hours on MS Windows XP, and 45 hours on OpenBSD. The differences are likely due to the memory available. Runs using bsddb3 on FreeBSD and OpenBSD take similar times to the sqlite3 runs.

DPT takes 11.5 hours to do the 2 million game deferred update on Windows XP, but 150 hours under Wine on FreeBSD. DPT seems to be unsupported now, but the scale of the discrepancy makes it worth mentioning. DPT's way of determining memory use does not work under Wine on FreeBSD and the work-around turns out to be slow.

Python's multiprocessing.Queue is used to communicate with chess engines but it does not work under Wine. Wine HQ Bugzilla bug 17195 and Python Issue Tracker issue 9099 are the closest references found which could be relevant.

When one of the chess fonts, Chess Cases for example, is added to the font directory for Wine in FreeBSD, ~/.wine/drive_c/windows/Fonts, it gets used as the default font for Text widgets created by the MS Windows version of Python tkinter. The same happens if instead the font is put in /usr/local/share/fonts so it is available for the FreeBSD version of Python as well.

But the font does not get used as the default font for Text widgets created by the FreeBSD version of Python. Nor does it get used in this way by the MS Windows version of Python tkinter running on MS Windows when the font is installed on MS Windows.

Versions used are FreeBSD 10.1, Wine 1.8.1, and MS Windows XP, Python 3.3.

Reading about fonts in the Tcl/Tk manual suggests the effect is not an error, but a consequence of the font selection rules.

The problem is resolved within ChessTab by stating the font required explicitely under Wine, taking advantage of the multiprocessing.Queue problem. However under Wine ChessTab is best run in a dedicated user, with the font installed in ~/.wine/drive_c/windows/Fonts but not in /usr/local/share/fonts, to avoid polluting the widgets of other programs running under Wine with the chess fonts.

Concurrent changes in the pgn package turned the minor problem of putting a space at the start of each variation's line into the major one of not putting each variation on a separate line. Fixing the major problem revealed the minor problem had been fixed too.

7 September 2016

The selection rule evaluator was written to use data stored using the apsw, bsddb3, or sqlite3 database interfaces without regard for the differences to the DPT database engine. Hence the problems evaluating selection rules when using the DPT database engine.

14 September 2016

The module and class docstrings look nicer when viewed with pydoc now the lists have been removed, but otherwise their content is the same.

18 September 2016

This paragraph was removed from the "Version 0.40" description because the change was left out in error: "All previous versions used the value given in the White and Black tags as a player's name for indexing purposes. This version accepts multiple player names, in either tag, separated by colons as stated in the PGN specification.". This change is applied in "Version 0.41".

18 October 2016

It became clear that both a ucinewgame and a setoption clear hash command are needed between analysis of each position by a chess engine to make the analysis consistent between runs of ChessTab. The random order in which positions are analysed is assumed to be why setoption clear hash is needed as well.

Positions are not sorted into game order for analysis because navigation of any displayed game or repertoire causes the most recent position displayed to be put at the front of the queue for analysis. It is not yet clear how Recursive Annotation Variations should be fitted into game order for analysis.

13 November 2016

It is possible to claim all the conditions mentioned in the comment dated 6 November 2011, see old history, have been met. DPT is no longer supported but the important element in DPT, bitmapped record numbers, has been transplanted to the interfaces to sqlite3 and bsddb3. Display of chess engine analysis is supported but the chess engines must be running on the same computer as ChessTab.

Problems found in ChessTab-0.41.5 will be fixed in ChessTab-0.41.6 and so forth, but new work will appear in ChessTab-0.42 and beyond. All earlier versions of ChessTab which survived previous culls have been removed from the website.

These things will be done before replacing ChessTab-0.41 as the stable version:

Fix the bug which prevents queries like 'White eq Smith or White eq Jones' being expressed as 'White eq Smith or Jones'.

Support parts of Chess Query Language (CQL) relevant to partial positions as understood in ChessTab.

Change the user interface for ChessTab partial positions to fit the 'White eq Smith or Jones' pattern, and be compatible with CQL.

Support display of chess engine analysis done on other computers in the Local Area Network.

ChessTab-0.41.5 will be put on PyPI shortly.

2 October 2017

CQL distinguishes the moves played in a game from variations, a distinction not made in the partial position scheme. The decision to support this distinction forces a change to the content of index values, to refer to the variation indexed, making ChessTab-1.0 incompatible with earlier versions. Thus a new major version number is needed.

ChessTab-1.0 does not run under Wine at time of writing on my environment for these reasons:

The Microsoft Windows installers for Python34, Python35, and Python36, fail to install Python under wine-2.0_3,1 on FreeBSD 10.3.

The 'embed' zipfile Python downloads available for Python35 and Python36 do not include the tkinter module which is required by ChessTab.

The Microsoft Windows installer for Python33 installs Python under wine-2.0_3,1 on FreeBSD 10.3 but ChessTab-1.0 uses Python's asyncio module which was added to Python at Python34. Earlier versions of ChessTab do not use the asyncio module.

The reason for running ChessTab under Wine is to use the dptdb database interface away from Microsoft Windows.

The number of index entries per game is more than doubled, from about 1800 to about 4200, to support CQL statements. On a computer with a 3Ghz CPU with 2Gb 800MHz DDR2 SDRAM and a SSD importing about 1.5 million games took about 826 minutes on ChessTab-2.0 compared with about 397 minutes on the previous version. This gives an estimate of about 75 minutes to do everything except the indexing, which does not seem inconsistent with what my notes of several years ago say about timings on a slower computer but I would have liked 120 minutes as the estimate here.

29 June 2018

The fix applied today to basesup-2.0 does not affect ChessTab, which will move on to use a later basesup version at a convenient time.

31 August 2018

A misleading error report is made at the at the end of game imports done by ChessTab-2 versions with the sqlite3 interface on Python3.5 or earlier. The import has completed successfully but the last line in the error report is 'sqlite3.OperationalError: cannot commit - no transaction is active'.

All the work to convert the original ChessTab-0 to ChessTab-1 and ChessTab-2 was done using Python3.6 and, for convenience, comparisons runs were done with apsw on Python3.5 and sqlite3 on Python3.6. For this change the combinations were apsw on Python3.6 and sqlite3 on Python3.5 and the problem got noticed.

ChessTab-0 treats the apsw and sqlite3 interfaces as unrelated code which happens to look very similar. ChessTab-2 treats the apsw and sqlite3 interfaces as the same code which uses either the apsw module or the sqlite3 module.

All the differences are implemented in basesup versions, and mainly means not using the features added by sqlite3 to standard SQLite in basesup-2 versions. The definition of a transaction in sqlite3 changes between Python3.5 and Python3.6, and ChessTab-2 needs the definition used at Python3.6.

Python3.5 and earlier are removed from the list of Programming Languages in setup classifiers.

8 November 2018

The dependencies were renamed because the old names seem poor for use on PyPI.

An '*' character used in a filter string seems to get treated as a 0-n wildcard character when used with the SQLite3 database engine, but as a literal when used with the Berkeley DB or DPT database engines. The behaviour with SQLite3 breaks the DataGrid class in the solentware-grid component but fortunately the program crash is delayed till the next scrolling action giving time for a warning and avoiding action.

2 October 2019

The problem fixed at ChessTab-3.0.6 was found after importing the PGN file referenced in the 'General Chat | 5949 moves : is this a maximum ?' thread on English Chess Forum at https://www.ecforum.org.uk in the post by Jesper Norgaard dated Thursday September 19, 2019 5:25 pm.

At time of writng my browser gives the thread reference as: https://www.ecforum.org.uk/viewtopic.php?f=2&t=10411 and the thread gives the PGN file reference as: https://drive.google.com/open?id=1v2V20kyQ-cyfAgGfc4okeV4xgY16aHRo and a reference to a discussion of the maximum number of moves as: http://www.chesschat.org/showthread.php?12548-Longest-possible-chess-game.

Importing the game into an empty database takes about 10 seconds. Probably reasonable as the game is indexed by 11790 positions and, on average, 19 pieces per position. Displaying the game from the database and navigating from one position to another takes a very long time: up to 3 hours for each action!

Moves '2692. Qb2c3', '4449... Qg1f2', and '5591. Qe6e7', were the only examples of this type of movetext found while investigating the problem.

1 November 2019

The table name changes for SQLite3 and Berkeley DB ChessTab databases, and the merging of tables for Berkeley DB ChessTab databases, were prepared first but delivery of ChessTab-4.0 depended on the successful implementation of '&', '^', '|', '&=', '^=', and '|=', operators for DPT's APIRecordSet class. The interfaces to SQLite3 and Berkeley DB databases already used shared code, in particular for these operations on their equivalents of APIRecordSet, and the interface to DPT had to be able to use this code too.

Two versions of APIRecordSet exist. APIFoundSet is immutable. APIRecordList can be changed using the Place() and Remove() methods; and these methods can be used to make the six operators used in the other interfaces.

The intial attempt to provide '&', '^', '|', '&=', '^=', and '|=', operators in the interface to DPT built with SWIG did not work, although these operators still exist in the DPT interface available for download.

It proved possible to provide the '&', '^', '|', '&=', '^=', and '|=', operators in the code that uses the DPT interface. Then use of shared code in all the interfaces just fell into place.

15 November 2019

The moves of a game are normally displayed as a single line, which will wrap depending on the size of the widget in which it appears.

The documentation for Tcl/Tk warns about the performance impact of large lines when tagged extensivly. Typical games of chess do not lead to large lines, and it was the longest possible game, see note 2 October 2019, which gave an idea on what large is in this context.

Displaying each full-move on it's own line counts as a bug-fix, hence ChessTab version 4.0.1, while changing things to always display games in a multi-line structure is definitely a new major release, ChessTab version 5.0 perhaps.

14 April 2020

Support for UnQLite and Vedis databases was added to test the tree implemented in the solentware-base.core.tree module: or how can a tree be implemented in a key-value structure. ChessTab seems to work but importing large numbers of games in one go does not seem possible. In favourable conditions half a million can be imported with 8G memory.

The Python dbm.gnu and dbm.ndbm databases were added, because it was now quick and easy to do so, to see if the absence of transaction support makes a difference. Not really, because dbm.gnu slows down by a factor of 3 after half a million games, possibly with more to come, while Vedis just runs out of memory.

These four engines are kept against the possibility replacing ordered keys by non-ordered keys will improve the performance in ChessTab.

28 April 2020

It is assumed the absence of problems, with maximum locks, importing games on OpenBSD is because transactions are not used. Editing games may not be a problem either because only the changed indicies are updated. The problem is exposed by running the 'castling options correction' tool because here the entire game, including all indicies, has to be deleted with code from one version of ChessTab and re-inserted with another version.

26 July 2020

The change in documentation style at CQL version 6.0.4 compared with version 5.1 made it practical to parse all filters. It is now clear there are many filters which ChessTab will never be able to evaluate because it is necessary to play through the game. The best ChessTab can do is reduce the number of games which have to be processed by CQL.

Empty square piece designators such as _d3 are now tranlated to the full form 'not squaremove eq 2320d3A and not squaremove eq 2320d3a' rather than the abbreviated form 'not squaremove eq 2320d3A or 2320d3a'. The abbreviated form does not work because there is a problem in the query languange provided by solentware-base.

2 September 2020

The rewrite of pgn-read at version 2.0 was prompted by 'http://portablegamenotation.com/LittlePGN.html', which I cannot access at time of writing. It included a sample game with lots of things wrong compared with the PGN standard. The game is in the pgn-read unittests as Little_01.pgn along with the minimum changes which allow the GameTextPGN class in pgn-read to handle it.

ChessTab-4.3 does not use GameTextPGN so it decides Little_01.pgn and it's derivatives have PGN errors.

On Microsoft Windows double click is needed to activate scrolling in the list of games matching a position when done over a row listing a game, but on OpenBSD a single click is sufficient.

A mixture of Text and Label widgets is used to display the lists of games, and the mix is changed between ChessTab versions 4.2 and 4.3 after remaining the same since the beginning. For example at version 4.2 the Tags column in the main list of games was a Text widget and needs the double click but is a Label now. The list of games matching a position uses a Text widget for each game at version 4.3 and it can be annoying to avoid the double click by clicking on the header row or the empty area.

Before ChessTab version 4.3 any Text widgets were to the right in a row, and scrolling was usually needed to get there, and it never became obvious the problem existed.

20 October 2020

Investigating why ChessTab's game and repertoire editor was not handling incomplete moves as expected led to finding the problem in pgn-read.

2 January 2021

The 'crafty' PGN files which used to be available at ftp://ftp.cis.uab.edu/pub/hyatt/pgn were a good source of noise between game scores in PGN format, usually in the form of email headers.

The collection of California games at www.chessdryad.com/games/CalBase.pgn has a good number of instances of the '--' move in PGN movetext. Most are in recursive annotation variations but a few are in the game line. References justifying use of '--' are hard to find, well I found it so, but the existence of 'chess.com/forum/view/general/what-is-a-null-move-and-how-to-make-it' gives '--' plausibility as an accepted extension to the PGN standard. '--' is mentioned as a minor point in a comment on 'reddit.com/r/chess/comments/47fexz/towards_a_replacement_for_the_pgn_format'.

ChessTab turns '--' and following moves in the variation into a '{...}' comment to avoid indexing the game for these moves. At some point in future I may change the indexing code to ignore '--' and following moves so these moves can be stored as found. One problem is some sources use '--' as a placeholder for an unknown move which may cause a move actually played later in a game to look like a stupid or illegal move.

3 January 2021

The first thing I do after publishing ChessTab-4.3.2 is concatenate some PGN files downloaded from 4ncl.co.uk and find the first game from the second and subsequent files marked as invalid PGN because of the end of file marker, Ctrl-Z, at the end of the preceding file. 4NCL PGN files published before 2012, which I downloaded many years ago, do not have the end of file marker.

18 November 2022

Most of the pylint suggestions on coding style were implemented, except for those saying 'too many ..' or 'too few ..' of something. Perhaps I chose the wrong way of dividing stuff up into classes, but the choices seemed reasonable back then (and still do).

Attribute names were given a "_" prefix unless pylint objected about access to a private attribute. Thinking the attribute should be public was not allowed as a reason to make it public.

There was, and is, a risk that accepting pylint's view on names of less than three characters has introduced as yet undetected problems. Quite a lot of changes were devoted to this.

31 March 2023

The Tcl via tkinter interface to Berkeley DB is a solution to the problem on OpenBSD where the available versions of Berkeley DB as ports are 3.1.17 and 4.6.21. Neither are supported by PyPI package berkeleydb, and bsddb3 which does support them is not supported at Python 3.10 or later.

The PyPI package lmdb was noticed, and appreciated, while working on the Tcl interface. It is seen as a long-term replacement for Berkeley DB as far as ChessTab is concerned.

The Python interface to Symas LMMD, lmdb, is based on PyPI package cffi, which sparked interest in rescuing the interface to DPT via cffi. The Swig interface to DPT stopped working at Python 3.8.

8 October 2023

Little progress on interface to DPT via cffi. The Swig interface to DPT has been revived on Developer Command Prompt for Visual Studio 2017 and Msys2. The x86 version seems fine but there are several unresolved problems with the x64 version.

15 July 2024

The large import strategy introduced at ChessTab 8.0 is not appropriate for the gnu.dbm, ndbm, unqlite, and vedis, database engines because they do not support BTree.

I have not yet seen a way of making the large import strategy usable on the DPT database engine. Partly because deferred index updates have to be done with transactions turned off, and at least hindered by DPT's fastload feature not working in x64 compilations.

I have not yet tried resuming an import after cutting power to the box running the import. It is unlikely I will test this on an OpenBSD box, but will see what happens on either or both FreeBSD and Windows 10.