Skip to content

v1.2.1 (ACID support, GDPR, Golang stdlib support and improved SQL compatibility)
Compare
Choose a tag to compare
@mmeloni mmeloni released this 14 Dec 12:20
· 1899 commits to master since this release
v1.2.1
1585f5e

Immudb 1.2.1 highlights

This release of immudb comes with many new features and improvements. The most important ones are highlighted below.

Improved transaction support

Starting with this version of immudb, SQL clients can now use true SQL transactions. It is now possible to start a transaction, perform multiple read and write operations and then commit or rollback changes. Before that each transaction had to be executed within one command sent to the server.

Logical deletion and expiration

KV and SQL records can now be deleted. This was enabled by introduction of metadata attached to entries that can store information about deletion or expiration. Such deleted entries are no longer available when trying to read KV or SQL entries but one can still lookup the history of a key or perform an SQL statement using time travel to fetch historical values before deletion.

In adddition to deletion, KV entries also support setting data expiration. Once expired, values of KV entries can no longer be retrieved. Expiration is a much stronger concept than deletion and once expired, immudb will protect against reading of the value for given key even if the history of an entry is retrieved.

Much improved SQL support

Due to introduction of metadata, immudb now supports the DELETE statement. Deleted rows do not appear in standard SQL queries anymore, however one can still query them using time travel feature.

This release also add support for a standard UPDATE statement that enables modifying only some columns of a row, before that the only way to update an existing entry was with the UPSERT statement that required values for all columns and could only be used with a primary key.

An improvement to the UPSERT allows updating rows for tables that have secondary indexes.

There are many more smaller improvements in the SQL engine for this release such as: indexes for columns with NULL values, native TIMESTAMP type, SELECT DISTINCT, IS [NOT] NULL, IN, basic CAST, LIKE NOT. We also improved SQL compatibility for table aliases and COUNT function.

Important notes

Breaking changes on the SQL layer

In order to support nullable columns, a breaking change for the on-disk format was introduced. After upgrading to immudb 1.2.1, the old data will not be available through the SQL interface even though immudb still contains the low-level KV representation of those entries. If you need to migrate the old data to the new format, please contact us on our discord channel.

Breaking changes in the SDK

One of the strengths of immudb is that immudb can prove to the client that the record was immutably stored in the database and that it was not tampered with. Verification of such proof is done on the client side.

Introduction of the entry metadata requires a slightly modified version of the proof that also needs to consider such metadata. An updated version of the SDK will be needed to correctly verify proofs for new entries.

There are also many changes in the API related to new transaction support. Because transaction requires server-side state, the client can now operate in a context of a server-side session. There are also many new commands related to transaction support.

Avoiding downtime during the upgrade

To avoid downtime during the upgrade, components should be upgraded in the following order:

  • immudb clients
  • immudb follower replica instances
  • immudb main database

Changelog

[v1.2.1] - 2021-12-13

Bug Fixes

  • protobuf: Fix compatibility with 1.1 version
  • database: Internal consistency check on data reads
  • database/meta: Do not crash on history with deleted items
  • pkg/database: history skipping not found entries

Changes

  • cmd/immuadmin/command: add super user login hint
  • embedded/sql: use sql standard escaping with single quotes
  • embedded/sql: support for escaped strings
  • embedded/store: prevent value reading of expired entries
  • embedded/store: fix expiration error declaration
  • embedded/store: dedicated expiration error
  • embedded/store: improve metadata serialization/deserialization methods
  • embedded/store: validations during metadata deserialization
  • embedded/store: return data corrupted error when deserialization cannot proceed
  • embedded/store: use fixed time during the lifespan of a tx
  • embedded/store: easily extendable meta attributes
  • embedded/store: mandatory expiration filter
  • embedded/store: reduce attribute code size
  • pkg/auth: require admin permission to export and replicate txs
  • pkg/integration: remove useless compilation tag on tests
  • pkg/server: deprecate GetAuth and WithAuth
  • pkg/server/sessions: session max inactivity time set to 3m and minor stat collecting fix
  • pkg/server/sessions: tuning sessions params
  • pkg/server/sessions: session timeout set to 2 min

Features

  • embedded/store: logical entries expiration
  • pkg/api: logical entries expiration
  • pkg/client: expirable set

[v1.2.0-RC1] - 2021-12-07

Bug Fixes

  • Update jaswdr/faker to v1.4.3 to fix build on 32-bit systems
  • Makefile: Fix building immudb for specific os/arch
  • Makefile: Use correct version of the grpc-gateway package
  • embedded/sql: ensure determinism and no value overlaps distinct rows
  • embedded/sql: fix inserting calculated null values
  • embedded/sql: correct max key length validation based on specified col max length
  • embedded/sql: fix rollback stmt
  • embedded/sql: normalize parameters with lower case identifiers
  • embedded/sql: param substitution in LIKE expression
  • embedded/sql: Use correct statement for subquery
  • embedded/sql: Do not modify value returned by colsBySelector
  • embedded/sql: Fix SELECT * when joining with subquery
  • embedded/sql: distinct row reader with limit argument
  • embedded/store: release lock when tx has a conflict
  • embedded/store: read conflict validation
  • embedded/store: typo in error message
  • pkg/auth: Fix password tests
  • pkg/client: fix database name saving on token service
  • pkg/database: sql exec on provided tx
  • pkg/server: fix keep alive session interceptor
  • testing: using pointers for job channels
  • webconsole: Fix html of the default missing page.

Changes

  • refining sdk client constructor and add readOnly tx guard
  • Update build/RELEASING.md documentation.
  • fix more tests
  • decoupled token service
  • token is handled internally by sdk. Remove useless code
  • remove token service from client options and fix tests
  • cmd/immuadmin/command: fix immuadmin token name on client creation
  • cmd/immuclient: deleteKeys functioin and updates after metadata-related changes
  • cmd/immuclient: temporary disable displaying hash in non-verified methods
  • embeddded/tbtree: leverage snapshot id to identify it's the current unflushed one
  • embedded/multierr: minor code simplification
  • embedded/sql: set INNER as default join type
  • embedded/sql: minor update after rebasing
  • embedded/sql: Alter index key prefixes
  • embedded/sql: wip rw transactions
  • embedded/sql: remove opt_unique rule to ensure proper error message
  • embedded/sql: minor code simplification
  • embedded/sql: kept last snapshot open
  • embedded/sql: Simplify row_reader key selection
  • embedded/sql: Better error messages when (up|in)serting data
  • embedded/sql: standardized datasource aliasing
  • embedded/sql: wip sqlTx
  • embedded/sql: de-duplicate tx attributes using tx header struct
  • embedded/sql: fix nullable values handling
  • embedded/sql: rollback token
  • embedded/sql: set parsing verbose mode when instantiating sql engine
  • embedded/sql: unsafe snapshot without flushing
  • embedded/sql: reusable index entries and ignore deleted index entries
  • embedded/sql: bound stmt execution to a single sqltx
  • embedded/sql: delay index sync until fetching row by its pk
  • embedded/sql: changes on tx closing
  • embedded/sql: leverage metadata for logical deletion
  • embedded/sql: use order type in scanSpecs
  • embedded/sql: cancel non-closed tx
  • embedded/sql: expose Cancel method
  • embedded/sql: wip interactive sqltx
  • embedded/sql: limit row reader
  • embedded/sql: return map with last inserted pks
  • embedded/sql: use int type for limit arg
  • embedded/sql: defer execution of onClose callback
  • embedded/sql: sql engine options and validations
  • embedded/sql: standard count(*)
  • embedded/sql: ddl stmts not counted in updatedRows
  • embedded/sql: method to return sql catalog
  • embedded/sql: non-thread safe tx
  • embedded/sql: use current db from ongoing tx
  • embedded/sql: postponing short-circuit evaluation for safetiness
  • embedded/sql: wip sql tx preparation
  • embedded/store: entryDigest calculation including key len
  • embedded/store: conservative read conflict validation
  • embedded/store: non-thread safe ongoing tx
  • embedded/store: wip tx header versioning
  • embedded/store: expose ExistKeyWithPrefix in OngoingTx
  • embedded/store: set header version at commit time
  • embedded/store: set tx as closed even on failed attempts
  • embedded/store: strengthen tx validations
  • embedded/store: GetWith method accepting filters
  • embedded/store: handle watchersHub closing error
  • embedded/store: remove currentShapshot method
  • embedded/store: tx header version validations and increased max number of entries
  • embedded/store: threadsafe tx
  • embedded/store: filter out entries when filter evals to true
  • embedded/store: ongoing tx api
  • embedded/store: early tx conflict checking
  • embedded/store: simplified ExistKeyWithPrefix in current snapshot
  • embedded/store: reorder tx validations
  • embedded/tbtree: remove ts from snapshot struct
  • embedded/tools: upgrade sql stress tool
  • embedded/tools: update stress_tool after metadata-related changes
  • embedded/tools: upgrade stress tool using write-only txs
  • pkg/api: consider nil case during tx header serialization
  • pkg/api: changes in specs to include new metadata records
  • pkg/api/schema: increase supported types when converting to sql values
  • pkg/client: updates after metadata-related changes
  • pkg/client: avoid useless tokenservice call and add tests
  • pkg/client: check if token is present before injecting it
  • pkg/client: omit deleted flag during value decoding
  • pkg/client/clienttest: fix immuclient mock
  • pkg/client/tokenservice: handlig error properly on token interceptor and fix leftovers
  • pkg/database: updates after metadata-related changes
  • pkg/database: improve readability of Database interface
  • pkg/database: snapshots should be up to current committed tx
  • pkg/database: revised locking so to ensure gracefully closing
  • pkg/database: implement current functionality with new tx supportt
  • pkg/database: return a specific error in querying
  • pkg/database: enforce verifiableSQLGet param validation
  • pkg/database: use new transaction support
  • pkg/database: limit query len result
  • pkg/errors: invalid database name error converted to immuerror
  • pkg/errors: useDatabase returns notFound code when error
  • pkg/integration: updates after metadata-related changes
  • pkg/server: use upgraded database apis
  • pkg/server: updates after metadata-related changes
  • pkg/server: error when tx are not closed
  • pkg/server/sessions: polish logger call
  • pkg/server/sessions: add sessions counter debug messages
  • pkg/stdlib: general improvements and polishments
  • pkg/stdlib: remove context injection when query or exec
  • pkg/stdlib: improve connection handling and allow ssl mode in connection string
  • pkg/stdlib: fix unit testing
  • pkg/stdlib: handling nil pointers when converting to immudb named params
  • pkg/stdlib: increase pointer values handling and testing
  • stress_tool_sql: add sessions and transaction mode
  • stress_tool_worker_pool: add long running stress tool
  • test: test backward compatibility with previous release (v1.1.0)
  • tsting: add index compactor in long running stress tool

Features

  • helm chart for deploying immudb on kubernetes (#997)
  • embedded/appendable: method for reading short unsigned integer
  • embedded/sql: wip sqltx at engine with autocommit
  • embedded/sql: support for not like
  • embedded/sql: increased expression power in LIKE and IN clauses
  • embedded/sql: null values for secondary indexes
  • embedded/sql: engine as tx executor
  • embedded/sql: support value expression in like pattern
  • embedded/sql: create index if not exists
  • embedded/sql: Detect ambigous selectons on joins
  • embedded/sql: WIP un-restricted upsert
  • embedded/sql: support for IN clause
  • embedded/sql: delete from statement
  • embedded/sql: sql update statement
  • embedded/sql: distinct row reader
  • embedded/store: initial commit towards full tx support
  • embedded/store: wip enhanced tx support
  • embedded/store: included filters in key readers
  • embedded/store: including metadata records
  • embedded/store: conservative tx invalidation
  • embedded/store: functional constraints
  • embedded/store: logical key deletion api
  • embedded/store: keyReader in tx scope
  • embedded/tbtree: read as before returns history count
  • embedded/tbtree: implements ExistKeyWithPrefix in snapshots
  • sql: Add support for IS NULL / IS NOT NULL expressions
  • sql/index-on-nulls: Update on-disk format to support nullable values
  • sql/timestamp: Add timestamp support to embedded/sql
  • sql/timestamp: Add timestamp to protobuf definition
  • sql/timestamp: Add timestamp to stdlib
  • sql/timestamp: Add CAST from varchar and integer to timestamp

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File SHA256
immudb-v1.2.1-darwin-amd64 7e86b86568350fa4f8581e43e22de6672dd26bf7cb41f5cd2ffb46d007073849
immudb-v1.2.1-darwin-arm64 57b7be3d22ddb9727d6ec1b8427a06d92074013b2566e52c9df27d5f4a36da77
immudb-v1.2.1-freebsd-amd64 e39c938202d5847ba2ae1a90083d738bedab53c1233cd10cd4f27c1f46ee6ffb
immudb-v1.2.1-linux-amd64 96b229bcad96e87be1e76d4afae49f40a1ac7a28c4b30d7101c3162d4709004d
immudb-v1.2.1-linux-amd64-static d6223f21910a7a35f578c07f5552bc0bb5ecbb794379073199607803a07bc3cc
immudb-v1.2.1-linux-arm64 7390cc5965703232a2656ce6f95bf24e7b18e09377a3e70de61d71c931fbd001
immudb-v1.2.1-linux-s390x 355210329acf78ad0f6a2bcef0834be4a2b953d9c08bbac91441be888d1d0144
immudb-v1.2.1-windows-amd64.exe 7159fb1636882d5a87023d8148cf29f56c5d467dcc510a2e9831d8dccc780fcc

Immuclient Binaries

File SHA256
immuclient-v1.2.1-darwin-amd64 64d3f44349880f828e44445eaa489e5264c07e319721d696cc0c107cb6479890
immuclient-v1.2.1-darwin-arm64 ea47fae6cda9dbedb41149171d0e66fc1cff833099330b0392657e13a9be86dd
immuclient-v1.2.1-freebsd-amd64 7f65de5dfa3b25f13fdc163507412f7e9f30d9bb4ba1a4ae4e39c9106be20059
immuclient-v1.2.1-linux-amd64 2a85728cfe3f168bea059f5ee16884fcd61ae14bb558b7117cd42cd49020f938
immuclient-v1.2.1-linux-amd64-static 212d2d6df50db7355f5f3ab8eb982a7e3b652f8aaaa9e3561d86f3a2408e6106
immuclient-v1.2.1-linux-arm64 d7754842843a796971407fa609719322929f11d605fb2fcd6cf5ab41c71d2802
immuclient-v1.2.1-linux-s390x fca1a57447310581445bd80f0c698b0e478e8c72aa822ef861394b4f1c0a1054
immuclient-v1.2.1-windows-amd64.exe bc906aab78ea4cd5896426f0a480a35d718da8ea419d143fbb1a2c078c41b6c8

Immuadmin Binaries

File SHA256
immuadmin-v1.2.1-darwin-amd64 9e6c14da2a48c5013e7bb4cfee36ea26516e1cb488cdf498e9073b727e180d8d
immuadmin-v1.2.1-darwin-arm64 77cb6b4f74c09f50617a937dd79f4734f412a4123b95649d594d461aa538cc97
immuadmin-v1.2.1-freebsd-amd64 0654cc6b4fae92883420c8b4beecacf7debc77ff25de306cd5d66c56cfc7a014
immuadmin-v1.2.1-linux-amd64 27de34932d4ccb1dd647d12b92f362612542006d0941e050d3556281b08aac09
immuadmin-v1.2.1-linux-amd64-static 28725456d7e0c4234a349714cdd2fafedd78507872f4d6abe23daecd051672c6
immuadmin-v1.2.1-linux-arm64 6c7018f0d6abee32499664c5e9a7686d9ff8de0098fc806a601bb09cb8f421cf
immuadmin-v1.2.1-linux-s390x 2e29736e401d2195eb50f685ec995770dce6ab6ccb856baf1fabade8b1bb2cca
immuadmin-v1.2.1-windows-amd64.exe a354b71c37b62fd9eb9d7d960412b77bf710d6f8c463ef58aa65f09562cc3599
Assets 26
Loading