Khepri 0.15.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.15.0

Queries are always executed locally

So far, Khepri relied directly on the query mechanism of Ra: the query function could be evaluated by the local Ra server (regardless of its state) or the leader. There was even a "consistent" query type where the query was evaluated on the leader, but after ensuring that a quorum number of members were availble. Khepri exposed the favor query option to let the caller tweak the underlying type of query. By default, it balanced between simple leader queries and consistent queries.

Unfortunately, the fact that the query function was possibly executed on a remote node caused several issues:

• The function reference might be invalid on the remote node because of a difference in the version of Erlang/OTP.
• The function reference might be invalid on the remote node because the module hosting the function is not the same.
• The function reference might be invalid on the remote node because the module hosting the function is missing.
• Using an exported MFA instead of a function reference doesn't solve the problem because the remote copy of that MFA may still be missing or simply be incompatible (different code/behavior). This might be even more difficult to debug.

This release will now always use local queries by default to avoid them. However, there is the risk that the query works on out-of-date data if the local Ra server is behind the leader. To avoid this, we made the following related changes:

• Synchronous puts, deletes and transactions use the reply_from Ra option to make sure that the local node got the update before returning.
• The favor query option stays but it only takes two values now: low_latency and consistency. In the case of consistency, the local query uses the new condition option introduced in Ra 2.13.0: it will wait for the local Ra server to catch up with the leader before evaluating the query.

If you are using the favor => compromise value, you will need to update your code to pick between the other two values.

The application parameter consistent_query_interval_in_compromise was removed in #276 as part of this change.

Note that the throughput of consistent queries was divided by 2 in the process. It should be ok for most use cases as consistent queries are rarely needed.

See #238, #260 and #283.

Projections restored after snapshot install

This release contains two bug fixes that caused inconsistencies between the store and the projections ETS tables; the ETS tables har wrong data.

• Projections were not restored if a follower caught up with the leader using a snapshot; see #259.
• Projections were not taking into account changes to projections themselves or deletions from the store that could happen in the newly installed snapshot; see #290.

Replace khepri:unregister_projection() by khepri:unregister_projections()

The goal is to allow the caller to remove many or all projections in a single call.

Here is an example of the change to make to your code:

• Up to Khepri 0.14.x:

  khepri:unregister_projection(StoreId, ProjectionName)

• Starting from Khepri 0.15.0:

  khepri:unregister_projections(StoreId, [ProjectionName])

See #282 and #286.

Other changes

• Add khepri_cluster:wait_for_leader/{0,1,2} (#289).
• Fixes to khepri_cluster member queries (#287).
• Make ?IS_KHEPRI_CONDITION/1 stricter (#265).
• Expose ?khepri_error/2 and ?khepri_exception/2 in the public API (#266).
• Fix the use of reply_from when (un)registering projections (#270).
• Make is_store_running/1 and get_store_ids/0 robust against Ra servers stopping behind the scene (#271, #275).
• Optimize transactions by using lists prepend instead of ++ (#291).
• Ra was bumped from 2.10.1 to 2.14.0 (#263, #267, #273, #280, #285, #290).
• Horus was bumped from 0.2.5 to 0.3.0 (#254, #268).
• Fix two test cases in cluster_SUITE (#269, #272).
• Rename the internal macro ?NOPROC_RETRY_INTERVAL to ?TRANSIENT_ERROR_RETRY_INTERVAL (#288).

Download

Khepri 0.15.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.15.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.15.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.15.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.15.0 is available on Hex.pm.