Merge pull request #49 from asafch/v2.0.0

V2.0.0

.github/workflows/build_test_pr.yml

@@ -66,5 +66,5 @@ jobs:
           key: ${{ runner.os }}-maven-${{ hashFiles( 'project.clj' ) }}
           restore-keys: |
             ${{ runner.os }}-maven-
-      - name: Unit tests
-        run: lein test
+      - name: Unit and integration tests
+        run: lein test :all

.github/workflows/ci_branch.yml

@@ -71,9 +71,9 @@ jobs:
           key: ${{ runner.os }}-maven-${{ hashFiles( 'project.clj' ) }}
           restore-keys: |
             ${{ runner.os }}-maven-
-      - name: Unit tests
-        run: lein eftest
-      - name: Publish Unit Test Results
+      - name: Unit and Integration tests
+        run: lein eftest :all
+      - name: Publish unit and integration test results
         uses: EnricoMi/[email protected]
         if: always()
         with:

.github/workflows/ci_master.yml

@@ -71,9 +71,9 @@ jobs:
           key: ${{ runner.os }}-maven-${{ hashFiles( 'project.clj' ) }}
           restore-keys: |
             ${{ runner.os }}-maven-
-      - name: Unit tests
-        run: lein eftest
-      - name: Publish Unit Test Results
+      - name: Unit and Integration tests
+        run: lein eftest :all
+      - name: Publish unit and integration test results
         uses: EnricoMi/[email protected]
         if: always()
         with:

CHANGELOG.md

@@ -1,6 +1,46 @@
 ## This library follows [Semantic Versioning](https://semver.org).
 ## This CHANGELOG follows [keepachangelog](https://keepachangelog.com/en/1.0.0/).
 
+### VERSION 2.0.0
+#### Added
+ * Aerospike `Key` - can now coerce `java.util.UUID` into keys alongside byte arrays, 
+  ints, longs, strings and `com.aerospike.client.Value`.
+ * Created the `protocols` namespace which now holds a myriad of protocols.
+   * This includes new protocols that group Aerospike operations by CRUD/admin semantics.
+ * Can explicitly specify the port in the host string that is passed to the client
+ constructor `init-simple-aerospike-client`.
+* Integration test namespace now has the `^:integration` metadata:
+  * Run unit tests with `lein test`
+  * Run integration tests that require a locally-running Aerospike client via `lein test :integration`.
+#### Changed
+* Artifact coordinates in [Clojars](https://clojars.org/) have changed from `aerospike-clj/aerospike-clj`
+to `com.appsflyer/aerospike-clj`.
+* Upgraded dependency on [`promesa`](https://github.com/funcool/promesa) from `5.1.0` to `6.0.0`.
+* Implementations of `ClientEvents` protocol will no longer get the DB instance 
+for runtime parameters. Instead, they should be pre-configured at instance construction time.
+* Cleaned up the `client` namespace:
+  * Removed the `IAerospikeClient` protocol it can create a collision with `com.aerospike.client.IAerospikeClient`. 
+  Abstracting over the Java client instance selection is of no concern to a simple 
+  client that interacts with a single cluster.
+    * As a result `SimpleAerospikeClient` now directly uses the vars passed in 
+    construction time instead of fetching them from the `client` with keywords, e.g. `(:el client)`.
+    * The return type of `get-cluster-stats` is no longer a triply-nested vector, 
+    but a doubly-nested vector.
+  * All protocols moved to `protocols` namespace.
+  * `SimpleAerospikeClient` record now implements the protocols mentioned above.
+* Mock client
+  * The `MockClient` record now implements the protocols mentioned above, so 
+  production code could now have its `SimpleAerospikeClient` swapped with a mock 
+  client in-place and __without__ using `with-redefs`.
+  * Functionality that is needed for unit testing purposes is defined in the 
+  `Stateful` protocol and `MockClient` instances are extended to this protocol.
+* Logging via [`tools.logging`](https://github.com/clojure/tools.logging) as a façade.
+* CI
+  * No longer runs the lein command `compile` - it would be executed implicitly by `test`
+#### Removed
+ * The function `get-multiple` was removed in favor of the protocol method `get-batch`.
+ * Dependency on [`timbre`](https://github.com/ptaoussanis/timbre).
+
 ### VERSION 1.0.2
 #### Added:
 * This CHANGELOG now follows [keepachangelog](https://keepachangelog.com/en/1.0.0/).

README.md

@@ -7,53 +7,55 @@ An opinionated Clojure library wrapping Aerospike Java Client.
 [![Build Status](https://img.shields.io/github/workflow/status/AppsFlyer/aerospike-clj/ci%20branch?event=push&branch=master&label=build%20%26%20test)](https://github.com/AppsFlyer/aerospike-clj/actions)
 # Docs:
 [Generated docs](https://appsflyer.github.io/aerospike-clj/)
-## Tutorial:
+
+## Tutorial
 [here.](https://appsflyer.github.io/aerospike-clj/tutorial.html)
-## More advanced docs:
+## More advanced docs
 * [Advanced asynchronous hooks.](https://appsflyer.github.io/aerospike-clj/advanced-async-hooks.html)
-* [Implementing your own client.](https://appsflyer.github.io/aerospike-clj/implementing-clients.html)
 
-# Requirements:
+# Requirements
 - Java 8
 - Clojure 1.8
 
-# Features:
+# Features
 - Converts Java client's callback model into Java(8) `CompletableFuture` based API.
 - Expose passing functional (asynchronous) transcoders over payloads (both put/get).
 - Health-check utility.
 - Functions return Clojure records.
 
-# Stability:
-- stable. Although not 1.x, the API will remain stable.
-- GraalVM compatible [used here](https://github.com/ashwinbhaskar/aerospike-migration).
-# Maturity:
+# Maturity
 - Feature completeness: ~~mostly~~ near complete.
 - Stability: production ready. Actively and widely used in production.
 
-# Opinionated:
+# Opinionated
 - Non blocking only: Expose only the non-blocking API. Block with `deref` if you like.
 - Futures instead of callbacks. Futures (and functional chaining) are more composable and less cluttered.
-If a synchronous behaviour is still desired, the calling code can still `deref` (`@`) the returned future object. For a more sophisticated coordination, a variety of control mechanisms can be used by directly using Java's `CompletableFuture` API or the more Clojure friendly [promesa](https://github.com/funcool/promesa) (which is also used internallly), or via the library using [transcoders](https://appsflyer.github.io/aerospike-clj/index.html) or [hooks](https://appsflyer.github.io/aerospike-clj/advanced-async-hooks.html).
+If synchronous behaviour is still desired, the calling code can still `deref` (`@`) the returned future object.
+For a more sophisticated coordination, a variety of control mechanisms can be used by directly using Java's
+`CompletableFuture` API or the more Clojure friendly [promesa](https://github.com/funcool/promesa) (which is also used internally),
+or via the library using [transcoders](https://appsflyer.github.io/aerospike-clj/index.html) or
+[hooks](https://appsflyer.github.io/aerospike-clj/advanced-async-hooks.html).
 - Follows the method names of the underlying Java APIs.
-- TTLs should be explicit, and developers should think about them. Forces passing a TTL and not use the cluster default (This can be still achieved by passing the [special values](https://www.aerospike.com/apidocs/java/com/aerospike/client/policy/WritePolicy.html#expiration) -2,-1 or 0).
+- TTLs should be explicit, and developers should think about them. Forces passing a TTL and not use the cluster default
+(This can be still achieved by passing the [special values](https://www.aerospike.com/apidocs/java/com/aerospike/client/policy/WritePolicy.html#expiration) -2,-1 or 0).
 - Minimal dependencies.
-- Single client per Aerospike namespace. Namespaces in Aerospike usually indicate different cluster configurations. In order to reduce overhead for clusters with more than a single namespace create 2 client objects and share an event loop between them.
-
-# Limitations/ caveats
+- Single client per Aerospike namespace. Namespaces in Aerospike usually indicate different cluster configurations.
+In order to reduce overhead for clusters with more than a single namespace create 2 client instances and share an event
+loop between them.
 
-# TBD
-- Support Java 11
+## Usage
 
-## Usage:
-#### Most of the time just create a simple client (single cluster)
 ```clojure
 user=> (require '[aerospike-clj.client :as aero])
 nil
 user=> (def c (aero/init-simple-aerospike-client
   #_=>          ["aerospike-001.com", "aerospik-002.com"] "my-ns" {:enable-logging true}))
 ```
 
-It is possible to inject additional asynchronous user-defined behaviour. To do that add an instance of `ClientEvents`. Some useful info is passed in in-order to support metering and to read client configuration. `op-start-time` is `(System/nanoTime)` [more here](https://appsflyer.github.io/aerospike-clj/advanced-async-hooks.html).
+It is possible to inject additional asynchronous user-defined behaviour. To do that add an implementation of  the
+`ClientEvents` protocol.
+Some useful info is passed in-order to support metering and to read client configuration. `op-start-time` is
+`(System/nanoTime)`, see more [here](https://appsflyer.github.io/aerospike-clj/advanced-async-hooks.html).
 
 ```clojure
 (let [c (aero/init-simple-aerospike-client
@@ -67,7 +69,7 @@ It is possible to inject additional asynchronous user-defined behaviour. To do t
                               (println "oh-no" op-name "failed on index" index)))})]
 
   (get-single c "index" "set-name"))
-; for better performence, a `deftype` might be preferred over `reify`, if possible.
+; for better performance, a `deftype` might be preferred over `reify`, if possible.
 ```
 
 ### Query/Put
@@ -104,35 +106,46 @@ user=> @(aero/get-single c "index" "set-name")
 ```
 
 #### Unix EPOCH TTL
-Aerospike returns a TTL on the queried records that is Epoch style, but with a different "beginning of time" which is "2010-01-01T00:00:00Z". Call `expiry-unix` with the returned TTL to get a UNIX TTL if you want to convert it later to a more standard timestamp.
+Aerospike returns a TTL on the queried records that is epoch style, but with a different "beginning of time" which is "2010-01-01T00:00:00Z".
+Call `expiry-unix` with the returned TTL to get a TTL relative to the UNIX epoch.
 
 ## Testing
-Testing is performed against a local Aerospike running in the latest docker
+### Unit tests
+Executed via running `lein test`.
+
+### Integration tests
+Testing is performed against a local Aerospike instance. You can also run an instance inside a docker container:
 
 ```shell
 $ sudo docker run -d --name aerospike -p 3000:3000 -p 3001:3001 -p 3002:3002 -p 3003:3003 aerospike
-$ lein test
+$ lein test :integration
 ```
 
 #### Mocking in application unit tests
-When performing unit tests in application code, it is most times undesirable to launch a full Aerospike container to 
-run tests against. For those cases the library exposes a mock client that replaces all the calls to `aerospike-clj.client`.  
+For unit tests purposes you can use a mock client that implements the client protocols: `MockClient`.
 
 Usage:
 
 ```clojure
 (ns com-example.app 
   (:require [clojure.test :refer [deftest use-fixtures]]
-            [aerospike-clj.mock-client :refer [init-mock]]))
+            [aerospike-clj.protocols :as pt]
+            [aerospike-clj.mock-client :as mock])
+  (:import [aerospike_clj.client SimpleAerospikeClient]))
+
+(def ^:dynamic ^SimpleAerospikeClient client nil)
+
+(defn- bind-client-to-mock [test-fn]
+  (binding [client (mock/create-instance)]
+    (test-fn)))
 
-(use-fixtures :each init-mock)
+(use-fixtures :each bind-client-to-mock)
 
 (deftest ...) ;; define your application unit tests as usual
 ```
 
-The sample code executes on every test run. It initializes the mock and runs
-the test within a `with-redefs` - rebinding all the calls to functions
-in `aerospike-clj.client` to the mock.
+The sample code executes on every test run. It initializes the mock with a proper type hint
+so you can just invoke all client protocol methods on it.
 
 Note: If the production client is initiated using a state management framework,
 you would also need to stop and restart the state on each test run.

doc/advanced-async-hooks.md

@@ -1,44 +1,55 @@
 ## Advanced asynchronous hooks:
-Since `aerospike-clj` uses a future based model instead of a callback based model, it is convenient to compose complex asynchronous logic using [manifold](https://github.com/ztellman/manifold).
+Since `aerospike-clj` uses a future based model instead of a callback based model,
+it is convenient to compose complex asynchronous logic using [promesa](https://github.com/funcool/promesa).
 
-By implementing `ClientEvents` 2 hooks are exposed that are called for each API call: `on-success` and `on-failure`.
+By implementing the `ClientEvents` protocol 2 hooks are exposed that are called
+for each API call: `on-success` and `on-failure`.
 
-Those hooks are called with valuable information that can be used, for example to configure automatic logging, or telemtry on your client. Here is an example of a such code, that is reporting useful metrics to [statsd](https://github.com/etsy/statsd). So assuming you have some `statsd` namespace tha can connect and report a statsd server, and some `metrics` namespace that is used to properly format the metric names:
+Those hooks are called with valuable information that can be used, for example
+to configure automatic logging or metrics on your client. Here is an example of
+such code that is reporting useful metrics to [statsd](https://github.com/etsy/statsd).
+So assuming you have some `statsd` namespace tha can connect and report to a `statsd`
+server, and some `metrics` namespace that is used to properly format the metric names:
 ```clojure
 (ns af-common-rta-aerospike.core
-  (:require [aerospike-clj.client :as aero]
+  (:require [aerospike-clj.protocols :as pt]
             [statsd.metrics :as metrics]
             [statsd.core :as statsd]
-            [manifold.deferred :as d]))
+            [promesa.core :as p]))
 
-(defrecord DBMeter []
-  client/ClientEvents
-  (on-success [_ op-name op-result _index op-start-time client]
-    (statsd/send-timing (metrics/format-statsd-metric (:cluster-name client) op-name "latency")
+(defrecord DBMeter [cluster-name]
+  pt/ClientEvents
+  (on-success [_ op-name op-result _index op-start-time]
+    (statsd/send-timing (metrics/format-statsd-metric cluster-name op-name "latency")
                         (micros-from op-start-time)
                         STATSD-RATE)
-    (statsd/inc-metric (metrics/format-statsd-metric (:cluster-name client) op-name "success"))
+    (statsd/inc-metric (metrics/format-statsd-metric cluster-name op-name "success"))
     (when (= "read" op-name)
       (if (some? op-result)
-         (statsd/inc-metric (metrics/format-statsd-metric (:cluster-name client) "read" "hit"))
-         (statsd/inc-metric (metrics/format-statsd-metric (:cluster-name client) "read" "miss"))))
+         (statsd/inc-metric (metrics/format-statsd-metric cluster-name "read" "hit"))
+         (statsd/inc-metric (metrics/format-statsd-metric cluster-name "read" "miss"))))
     op-result)
-  (on-failure [_ op-name op-ex index op-start-time client]
-    (statsd/send-timing (metrics/format-statsd-metric (:cluster-name client) op-name "latency")
+  (on-failure [_ op-name op-ex index op-start-time]
+    (statsd/send-timing (metrics/format-statsd-metric cluster-name op-name "latency")
                         (micros-from op-start-time)
                         STATSD-RATE)
     (statsd/inc-metric (metrics/format-statsd-metric-fail-aerospike op-ex (:cluster-name client) op-name))
-    (d/error-deferred op-ex)))
+    (p/rejected! op-ex)))
 ```
 A few notes on the above code:
 1. Passed arguments:
-  * `op-name`, `op-result` and `index` are strings. They partially used for metrics generation in our case.
-  * `client` here is the `IAerospikeClient` instace. You can use its fields here, or you can even `assoc` more keys on it when you create it, to be later used here.
-  * `op-start-time` is `(System/nanoTime)`, converted here to microseconds and used to measure latency.
-2. The code is using the passed arguments to measure latency, format metrics names. You can easily do other stuff like logging etc.
-3. Both `on-success` and `on-failure` return the results passed in. Although this logic is the last logic that happens to the operations results (e.g. after trascoders are being run), the returned result will be what the calling code gets as a returned value.
+  * `op-name`, `op-result` and `index` are strings. They are partially used for
+  metrics generation in our case.
+  * `op-start-time` is `(System/nanoTime)`, converted here to microseconds and
+  used to measure latency.
+2. The code is using the passed arguments to measure latency and format metrics'
+names. You can easily do other stuff like logging, etc.
+3. Both `on-success` and `on-failure` return the results passed in. Although this
+logic is the last logic that happens to the operations' results (e.g. after
+transcoders are called), the returned result will be what the calling code gets
+as a returned value.
 
 Finally, hook it to your client:
 ```clojure
-user=> (def c (aero/init-simple-aerospike-client ["localhost"] "test" {:client-events (->DBMeter)}))
+user=> (def c (aero/init-simple-aerospike-client ["localhost"] "test" {:client-events (->DBMeter "test-cluster")}))
 ```