• MariaDB 11.0 is now supported. (963f0e3, 2b112b6)
• skeema diff and skeema push can now generate supported ALTER TABLE statements even for tables that already have unsupported features, as long as the new modification to the table does not affect those unsupported features. (#214, 7b5e86d)
  • For example, these commands can now add columns or indexes to pre-existing tables that already happen to use unsupported features such as spatial indexes, sub-partitioning, application time periods, etc. (Previously, Skeema could not generate an ALTER TABLE for such tables under any circumstances.)
  • In these cases, Skeema always verifies that the generated DDL has the correct effect by testing it in a workspace, even if skip-verify has been configured.
  • This verification step is conservative, and some edge cases may fail verification, particularly on MySQL 8 tables with non-default collations. In that situation, the operation remains marked as unsupported and the table is skipped with a warning message.
• When using workspace=docker, Skeema will attempt to automatically use a session sql_mode equivalent to that of your "real" database server (assuming a database host is configured and reachable). This solves common strict-mode problems on database systems that often use a nonstandard sql_mode, such as AWS RDS. (cfc11a0)
• Bug fix: On Mac and Windows client systems, a change in Skeema v1.10.0 inadvertently caused skeema init and skeema pull to always downcase the name of any newly-created .sql files. This has now been fixed, and the casing of the original table/object will be retained when naming the .sql file. (f0e52b7)
  • Regardless of this change, users can always freely rename .sql files or move statements between .sql files in the same directory. Skeema never requires that you keep statements in the exact location that skeema init or skeema pull put them in.
• Bug fix: a regression in Skeema v1.8.1 inadvertently reduced the effectiveness of the strip-definer option specifically for procs/funcs, when using an unprivileged database user and workspace=temp-schema. In that situation, if the CREATE FUNCTION or CREATE PROCEDURE statement in the .sql file contained a DEFINER clause, workspace errors could result. This has now been fixed, and DEFINER clauses are now properly stripped from procs and funcs. (The bug did not affect views or triggers; those already worked properly and this remains the case.)
• Bug fix: On Windows, when skeema pull needs to delete a .sql file because the corresponding table/object no longer exists in the database, file locking errors could sometimes occur on Skeema v1.10.0. This has now been fixed. (061f478)
• Improvements to logging and error messages in several situations:
  • When host and/or schema are not supplied in the appropriate location, or host-wrapper doesn't output any hostnames (0825d1a, 1055b2d)
  • Common mistakes with the password option (80635d9)
  • Workspace locking conflicts and timeouts (f8f0745)
  • Problems with the connect-options configuration (4d218c1)

For a summary of the most important changes in this release, see the Skeema v1.10.0 release announcement blog post.

• The Premium edition CLI now supports management of seed data using INSERT statements, for populating new tables or spinning up new environments. (#22)
  • The new inserts option controls how skeema push handles INSERT statements found in your *.sql files. The default setting, inserts=seed, will execute INSERTs whenever the target table is empty.
  • skeema pull's new update-seed-inserts option provides a mechanism for selectively dumping row data from an environment using view logic, automatically creating or updating INSERT statements in your *.sql files.
• All logic for processing .sql files has been completely rewritten. The new implementation provides an 8x to 40x speedup on parsing time, while also using substantially less memory. This was motivated by the requirement to handle arbitrarily-large seed data INSERT statements in the Premium edition, but the performance improvements are generally beneficial to all users, and are present in the Community edition as well. (e625c97)
• MariaDB 10.11 is now marked as supported by Skeema. (5b0cc10)
• New linter rule lint-pk-type provides a way to restrict which column data types may be used in primary keys. The list of allowed data types is configurable using the new allow-pk-type companion option. (#204, #205)
• New linter rule lint-reserved-word flags names of identifiers (tables, columns, routines, etc) which match reserved words in any known version of your database server. (#120, #207, 4c2ddd0)
• Linter improvement: lint-has-time now provides more descriptive messages that differ for timestamp vs datetime/time. The timestamp message now includes a warning about Y2K38 problems with this type, in addition to other improvements. (#209, #210)
• skeema diff --brief now automatically disables linter checks and omits all INFO-level logging. These changes are designed to make the output more easily human-readable, while still surfacing any unusual errors or warnings. (310164a)
• If STDERR isn't a terminal (e.g. being piped to another program or redirected to a file), all error/warning linter annotation output from skeema lint, skeema diff, and skeema push now omits embedded newlines. Previously, each annotation message could span multiple lines, which made automated parsing and processing more difficult. Now, each individual annotation message will consist of a single line when STDERR isn't a terminal. (#211)
• Bug fix: previously, skeema init would hang for a long period if a large .sql file was present in the working directory, for example mysqldump output from a large database. The contents of the working directory no longer affect skeema init. (#190, 2481bed)
• Bug fix: Partitioned tables using the LIST COLUMNS partitioning method were previously unsupported for diff operations in many cases. Now most LIST COLUMNS tables are supported, as long as the partitioning key column names don't conflict with MySQL keywords. (#199, 3313bfc)
• Bug fix: On MySQL 8, if a table contained at least one BINARY or VARBINARY column with a literal (non-expression) DEFAULT value, previously the table could be marked as unsupported for diff operations due to a MySQL information_schema bug. Skeema now implements a workaround for this MySQL bug. (0177ef9)

Thank you to all code contributors and issue reporters!

For a summary of the most important changes in this release, see the Skeema v1.9.0 release announcement blog post.

• Environment variables may be used as option values in .skeema option files for options user, password, host, schema, port, socket, and temp-schema. This provides a flexible mechanism for dynamic configuration. (16b54e4, #101)
• Premium CLI now includes the ability to offload the temp-schema workspace to a different database server, via the new temp-schema-environment option. The option value is set to the name of an environment (.skeema file section) which can override some or all connectivity options (host, port, socket, user, password, host-wrapper, SSH options, SSL options, etc) for determining what host to place the temp-schema workspace on. (#94)
• Improvements to the various ignore-* options (ignore-table, ignore-proc, ignore-func, ignore-view, ignore-trigger)
  • The ignore-proc and ignore-func options, which previously were only available in the Premium edition (since v1.6.0-premium one year ago), are now available in the Community edition as well. (0de3c57)
  • The ignore options now take effect earlier in the processing flow for all commands. Workspaces skip ignored statements entirely now, improving performance and ensuring that execution cannot be affected by SQL errors from ignored statements. (b35f065)
• Several MariaDB-specific improvements and fixes
  • MariaDB 10.10 is now fully supported, including special conversion logic for the new inet4 column type. (53de093)
  • MariaDB's November 2022 point releases (MariaDB 10.3.37, 10.4.27, 10.5.18, 10.6.11, 10.7.7, 10.8.6, 10.9.4) are now supported by Skeema. These MariaDB releases unexpectedly changed the output logic for collations in SHOW CREATE TABLE, which broke compatibility with a core safety check in Skeema. (f2dcaf8, #193, #194, #195)
  • When modifying existing procs/funcs in MariaDB, atomic CREATE OR REPLACE syntax is now used. This functionality was originally introduced in Skeema v1.6.0-premium one year ago, but is now present in the Community Edition as well. (412a326)
• When interacting with multiple hosts (running skeema from a parent directory), it is now possible to interactively prompt for different passwords for each host, by configuring the bare password option with no =value in the same .skeema file as the host. (ab4a84c, 5e35143, b004616)
  • Previously, bare password option with no =value was only supported in global option files. This still works, and still prompts for a single "global" password rather than host-specific ones.
  • For an alternative mechanism for passing different passwords to each host, also see the new ENV variable support, mentioned above.
• When using workspace=docker with MySQL 8, whenever Skeema creates a new ephemeral containerized DB, binary logging will be disabled automatically, despite MySQL 8 normally defaulting to enabling binlogging. This improves workspace performance, and prevents potential issues with non-deterministic CREATE FUNCTION definitions. (2804a44)

• MariaDB 10.9 is now supported. (9262e29)
• When using an interactive password prompt (--password or -p with no value), the prompt text is now normally displayed on STDERR instead of STDOUT, so that it is visible even if STDOUT is being redirected to a file. This also helps ensure that STDOUT only contains valid SQL, which can be piped/redirected to other client programs as-is if desired. (c0267f4, #188)
• MySQL 8 cross-schema foreign key safety improvement: when using the default of workspace=temp-schema, if any foreign keys cross schema boundaries (the referenced "parent" table is in a different database than the "child" table defining the FK constraint), this can cause metadata locking conflicts in MySQL 8, since the database extends metadata locks to both sides of the FK. Skeema now automatically applies a low session-level lock_wait_timeout for its workspace=temp-schema operations in MySQL 8 to limit any performance impact from such metadata locking conflicts. Note that only cross-schema FKs (and not same-schema FKs) are affected by the root issue, and only under MySQL 8 and workspace=temp-schema. See commit message for more details. (762da89)
• Bug fix: a nil pointer panic could occur if the network connection to the DB was extremely unstable/flaky (initial connections succeeded, but some subsequent connection attempts in the same operation failed in one particular code path). Skeema will now gracefully handle this situation with a normal error message. (d07d106)

• Tables with explicit TABLESPACE innodb_system or TABLESPACE innodb_file_per_table clauses are now supported for diff operations. These clauses permit overriding the server's innodb_file_per_table setting on an individual table basis. This syntax is only available in MySQL 5.7+, and is relatively rare. (9ed3f60)
• MySQL 8 bug fix: under some rare conditions, tables can report different next AUTO_INCREMENT value in information_schema vs SHOW CREATE TABLE, even with information_schema_stats_expiry=0 which Skeema automatically uses for its sessions. When this occurred, tables were unsupported for diff operations. To solve this, Skeema now relies on the SHOW CREATE TABLE value exclusively, and these tables are now properly supported for diffs. (677d8de)
• Internal concurrency improvements should provide performance benefits in some scenarios, such as running Skeema with high network latency, a large count of procs/funcs, and/or operating on multiple schemas at once. (f9a6bb2)
• When Skeema is unable to generate diff DDL for a table, the extra information logged by --debug now makes it clearer which side of the diff was problematic. A similar improvement has been made to the output of --verify in the rare case of verification failing. (#52, 677d8de)
• Whenever skeema pull or skeema add-environment needs to rewrite a .skeema file, some option values are now rewritten differently:
  • String or enum type options that have been set to an empty-string value are now rewritten to use the "skip" prefix instead. For example, lint-pk='' is now rewritten as skip-lint-pk. These configurations have the exact same effect (and in this specific example both are equivalent to lint-pk=ignore), but the empty-string value was less clear. (cf19abd)
  • Premium CLI bug fix: If a .skeema file contained strip-definer with no value (implying =true), previously these commands would incorrectly rewrite this as strip-definer= (empty value implying false). Now fixed.
• In a sharded environment in which shards are running different server versions (e.g. some shards on MySQL 8.0.28 and others on 8.0.30), differences in the server's metadata representation of the legacy three-byte "utf8" vs "utf8mb3" charset (and corresponding collations) are now ignored by default in skeema diff and skeema push. (894530f)
• Some unit tests and integration tests required minor improvements to handle MySQL 8.0.30's metadata changes regarding legacy three-byte utf8mb3 collations. Aside from test suite adjustments, Skeema itself was already fully compatible with MySQL 8.0.30. (28d2b0a)

For a summary of the most important changes in this release, see our blog post.

• Premium CLI now includes built-in support for SSH tunnels via the new ssh and ssh-to-db options. (#119)
  • These options cover a wide range of SSH use-cases: bastion security servers / jump boxes, localhost admin database connections, running online schema change scripts from a unified Linux server, and more.
  • Fully supports several OpenSSH configuration options from ~/.ssh/config, interacting with OpenSSH ssh-agent via UNIX socket (Linux and MacOS) or named pipe (Windows), known_hosts management, automatic keep-alives, multi-hop tunnels without need for agent forwarding, and more.
• MariaDB 10.8 is now fully supported, including descending indexes (DESC index parts), as well as functions with IN / OUT / INOUT param qualifiers (1454f21)
• Skeema now fully supports database servers running on Windows or MacOS, regardless of object name casing or the database server's lower_case_table_names setting
  • Previously, Skeema had issues with database servers using lower_case_table_names=1 (Windows default) or lower_case_table_names=2 (MacOS default) if any table, view, or database names used uppercase characters. These issues -- which related to the database's special handling of case-insensitive filesystems -- are now fixed, and Skeema's integration testing suite covers these situations. (#65, bdc279e)
  • New linter option lint-name-case can be used to enforce an all-lowercase table/view naming scheme, if desired for portability reasons (8cb30d0)
  • Premium CLI contains view-specific fixes relating to name casing, in addition to the fixes already made in v1.7.1.
• MySQL 8 bug fix: due to a regression in Skeema v1.7.1, tables with nonstandard default collations (meaning the table's default charset has a different default collation than the table's chosen default collation) would inadvertently cause failures in the diff verify step. This was caused by unusual/inconsistent behavior in MySQL 8's SHOW CREATE TABLE and is now fixed by using a deeper structural comparison in the verify logic. Thank you to Etsy for the report! (#184, 8bde55a)
• skeema pull now logs several problem conditions -- query failures, file I/O errors, etc -- as an error with exit code 2, instead of a warning with exit code 1. This is more consistent with how other commands handle these conditions. (2d3d7c4)
• Other minor internal refactors and test suite fixes (0d5d476, d4022fc)

• Premium CLI bug fix: When the database server uses lower_case_table_names=1 (e.g. server running on Windows, including Azure Database for MySQL) or lower_case_table_names=2 (server running on MacOS), and CREATE VIEW statements in *.sql files had view names containing uppercase characters, nil pointer panics would occur in most Skeema commands due to case sensitivity mismatches. View name casing is now handled correctly even with lower_case_table_names=1 or 2.
  • Skeema still does not officially support database servers running on Windows or MacOS yet, and several other known issues remain, but this fixes the most significant problem.
  • Official support for database servers running on Windows or MacOS is currently planned for Skeema v1.8.0 (both Premium Edition and Community Edition) in a few months.
• Premium CLI bug fix: In MySQL 8, if a CREATE VIEW statement supplied an explicit column list prior to the SELECT, the view was inadvertently unsupported by Skeema. This was caused by MySQL 8’s inclusion of these column list overrides in SHOW CREATE VIEW, unlike previous MySQL versions and all MariaDB versions. This view syntax (which is fairly uncommon) is now fully supported for all database versions.
• Premium CLI license activation key timeout has been increased, and if a timeout occurs, the error is now clearer.
• Bug fix: Tables containing generated column expressions using LIKE, or check constraints using LIKE, were inadvertently unsupported due to a logic error in Skeema’s previous prohibition of CREATE TABLE ... LIKE statements. (#181, 6cbff9c)
  • CREATE TABLE ... LIKE is now permitted for use in .sql files, and works properly in skeema diff and skeema push as long as the target table is a normal table in the same schema. The statement is evaluated dynamically, meaning any changes in the target table will be reflected properly in diffs for any tables that point to it.
  • skeema pull, skeema lint, and skeema format may overwrite CREATE TABLE ... LIKE with a normal non-dynamic CREATE TABLE. This functionality may be improved in a future release.
• An upcoming release of MySQL 8 is expected to replace occurrences of "utf8" with "utf8mb3" in information_schema and SHOW command output. Skeema charset logic has now been improved to be more robust, in order to ensure compatibility with this upcoming change. (a03718c)
• When manually creating or editing *.sql files, identifiers may now contain non-Latin unicode characters without requiring backticks around the identifier. Skeema’s identifier quoting rules should now match MySQL and MariaDB properly. (3e40b6c)
• workspace=docker now logs a warning if docker-cleanup=stop or docker-cleanup=destroy fails, for example due to the Docker daemon returning an error upon attempting to stop or remove the container. (5e971e0)

An installation guide and full documentation are available on our website skeema.io.

For a summary of the most important changes in this release, please see our blog post.

• AWS Aurora v3 (MySQL 8) is now supported
  • The Premium edition of the Skeema CLI offers full support for Aurora, and is extensively tested against all three major versions of Aurora (AWS Aurora for MySQL 5.6, 5.7, and now 8.0).
  • Although the Community edition of the Skeema CLI is not tested against Aurora and does not officially support Aurora, it has no known incompatibilities with Aurora MySQL 8 so far.
• MariaDB 10.7 is now supported (8e7551c)
  • The new uuid column type has special handling in diff/push operations: uuid columns may safely be converted to/from the binary(16), char(32), varchar(32), char(36), and varchar(36) column types without requiring the allow-unsafe option.
• Linux arm64 (aarch64) builds are now available for all editions of the Skeema CLI. As with linux/amd64 (x86_64), downloads are available for linux/arm64 in formats tar.gz, rpm, deb, and apk. (#179, 3d47592)
• Premium CLI now includes self-update functionality
  • When a newer version of the Skeema Premium CLI is available, an INFO log message will indicate this. (To reduce noise, the message only appears once every few days per installation.)
  • New command skeema update updates your installation of Skeema Premium CLI to the latest release for your operating system and CPU architecture. This is intended for use only for installations that do not use a package manager.
• MySQL 8 edge case fixes: some tables using generated columns, DEFAULT expressions, or functional indexes were previously unsupported for diff operations. The root causes were bugs in MySQL 8's information_schema representations of expressions containing string literals. Skeema now parses expressions from SHOW CREATE TABLE when necessary to work-around the MySQL 8 information_schema problems. (bf38edb, 27b379b)
• Files containing a UTF8 byte-order marker (BOM) are now processed correctly. This commonly occurs on Windows systems with some text editors. All editions of Skeema now properly handle UTF8 BOMs in .sql files, as well as .skeema option files. (03ef009, 77f12b1)
• Several problem conditions in .sql files, which previously generated warnings, now emit fatal errors instead. This change is being made to improve safety, to prevent risk of Skeema misinterpreting CREATE statements which contain unfortunately-placed typos. The affected situations include .sql files containing unterminated strings/identifiers/comments; invalid characters outside of strings/identifiers/comments; unsupported statement forms CREATE TABLE ... SELECT or CREATE TABLE ... LIKE; and OS/fs/io errors opening or reading a file. (#143, 22b0b49)
• Premium CLI views improvement: when using skeema pull --format, skeema format, or skeema lint --format without enabling the update-views option, previously only a DEBUG level log was emitted for each skipped view. A summary INFO debug log is also now emitted with a count of skipped views.
• The alter-algorithm option now permits a value of "nocopy", for use on MariaDB 10.3+. (0916bb0)
• In diff/push operations, the MariaDB 10.5+ inet6 column type may now be safely converted to/from char(39) or varchar(39) column types without requiring the allow-unsafe option. (0adaabd)
• Bug fix: previously, attempting to transition between a single-char DELIMITER to a doubled version of that same delimiter (e.g., DELIMITER ;; when the previous delimiter was the default ;) did not work properly. Skeema commands do not ever automatically use this style of DELIMITER, but some users may insert it manually for multi-line procs/funcs. Skeema's .sql file processor now handles this situation correctly. (0cc8432)

For a summary of the most important changes in this release, please see our blog post.

• New global option ssl-mode provides a way of configuring connection SSL/TLS preferences with identical semantics to the MySQL 5.7+ command-line client (432944e)
  • All editions support values "disabled" (don't use TLS), "preferred" (use TLS if server supports it - default value), "required" (use TLS, throw an error if server doesn't support it)
  • Premium CLI supports additional values "verify_ca" (require TLS and verify the server cert against the supplied CA) and "verify_identity" (require TLS, verify server cert, and perform strict hostname matching)
  • The previous way of configuring TLS modes via connect-options is still permitted, but now deprecated
• Premium CLI: Several new options provide support for client-side SSL certs. These options have all been designed to mimic the exact behavior of the corresponding options in the standard mysql client. This way, if you already have them configured in ~/.my.cnf, they'll seamlessly work as-is in Skeema. (#130)
  • ssl-ca supplies a certificate authority file, for verifying the server and protecting against MITM attacks
  • ssl-cert and ssl-key specify client-side certs for auth, permitting connection with accounts that have REQUIRE X509 / REQUIRE ISSUER / REQUIRE SUBJECT attributes
  • server-public-key-path improves security and performance of unencrypted connections in recent versions of MySQL
  • ssl-verify-server-cert is equivalent to ssl-mode=verify_identity, provided for option compatibility with MariaDB's version of the command-line client
• Premium CLI: new global option strip-definer to remove DEFINER clauses from *.sql files, and ignore them in diff comparisons, for all stored procedures, functions, views, and triggers (#146)
  • With default value of "auto", definer stripping is enabled automatically whenever Skeema's user does not have SUPER / SET_USER_ID permissions to override definers
  • Designed to simplify workflows on database-as-a-service platforms (RDS, Cloud SQL, etc) where SUPER privileges are not available; situations where each environment has a different administrative user; and companies where developers each have their own non-root MySQL user
• Premium CLI: new global regex options ignore-proc and ignore-func permit ignoring stored procedures and functions (respectively) based on name pattern-matching (#171)
• Premium CLI: new global regex option ignore-view may be used to ignore views based on name pattern-matching
  • Note that ignore-table still matches both tables and views, since they share a namespace; the new ignore-view option just provides an alternative which only matches views and never tables
  • If you wish to ignore all views, use ignore-view=., since this regex matches all view names
• Premium CLI: In MariaDB 10.1+, when modifying an existing stored procedure or function, MariaDB's CREATE OR REPLACE syntax is now used automatically; since this operation permits modifying a routine in a single statement, it is not considered an unsafe action
• Mac arm64 (Apple Silicon / M1 chip) native builds are now provided for both editions of Skeema CLI, and workspace=docker is now usable on arm64 systems (aab006f)
• An APK package for Alpine Linux is now provided for both editions of Skeema CLI; this is especially useful in custom Dockerfiles when building small Alpine-based containers (3c4a5df)
• If a non-InnoDB table is unsupported for diff operations, the warning message now mentions the table's storage engine as the most likely culprit (aa143db)
• Foreign key clauses RESTRICT and NO ACTION are now treated completely equivalently by Skeema, and diffs between these two clauses will be suppressed by default (59d62f1)
• Bug fix: foreign keys with explicit ON {UPDATE|DELETE} RESTRICT clauses were erroneously marked as unsupported-for-diff by Skeema; this especially affected in-place upgrades from pre-8.0 to 8.0 (#170, 3b52e87)
• Bug fix: when an existing MySQL database was upgraded in-place from pre-8.0 to 8.0, some tables could be erroneously marked as unsupported-for-diff by Skeema in very specific situations involving non-default charsets and collations (150348a, 911c2ca)

• Premium CLI bug fix: skeema push now always creates any functions prior to views, to permit creation of views which call newly-created functions in the same push operation (#168)
• Premium CLI bug fix: skeema push now orders creation of multiple views appropriately, to permit creation of views which query other newly-created views in the same push operation
• Premium CLI bug fix: if a table is ignored due to ignore-table, any trigger definitions associated with the table are now automatically ignored as well
• Connection pool management improvements should provide a performance benefit in some scenarios (e636fa6)
• Releases are now built with Golang 1.17, which slightly reduces binary size and improves performance
• Minor logging changes (9d64b43)
  • In skeema diff and skeema push, unsafe statement errors are now logged in a way that makes the commented-out unsafe statement easier to read
  • In skeema diff and skeema push, in situations when a table cannot be diff'ed by Skeema, the warning message text has been clarified
  • When STDERR log messages span multiple lines, and STDERR isn't a TTY, each subsequent line will now repeat the log level and timestamp for easier automated parsing

Don't forget to check out our new Premium edition of the Skeema CLI -- with added support for views, triggers, AWS Aurora, native Windows exe, and more! Along with our SaaS CI offering Skeema Cloud Linter for GitHub, these premium products help fund ongoing development of the Community edition CLI as well.