Skip to content

Commit

Permalink
Merge pull request #15 from JMTamayo/feature/v0.5.0
Browse files Browse the repository at this point in the history
Feature/v0.5.0
  • Loading branch information
JMTamayo authored Oct 26, 2024
2 parents 1b4fa37 + 6a922c0 commit 25ae598
Show file tree
Hide file tree
Showing 10 changed files with 723 additions and 169 deletions.
29 changes: 15 additions & 14 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,35 +7,36 @@ on:
branches: [ "main" ]

jobs:
build:
CI:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Install Rust
- name: "Setup: Install Rust"
uses: actions-rust-lang/[email protected]
with:
components: llvm-tools-preview

- name: Install cargo-llvm-cov
- name: "Setup: Install cargo-llvm-cov"
uses: taiki-e/[email protected]
with:
tool: cargo-llvm-cov

- name: Build project
- name: Checkout
uses: actions/checkout@v3

- name: Build
run: cargo build --verbose

- name: Verify project formatting
run: cargo fmt --all --check
- name: Format
run: cargo fmt --package redsumer --all --check

- name: Check for possible errors and coding suggestions
run: cargo clippy --all-features
- name: Lints
run: cargo clippy --package redsumer --all-features --verbose

- name: Run project tests
run: cargo test --verbose
- name: Unit Tests
run: cargo test --package redsumer --verbose

- name: Run tests with coverage for all features
- name: Coverage
env:
MIN_LINE_COVERAGE_TARGET: 80
run: cargo llvm-cov --workspace --all-features --summary-only --fail-under-lines ${{ env.MIN_LINE_COVERAGE_TARGET }}
run: cargo llvm-cov --package redsumer --all-features --summary-only --fail-under-lines ${{ env.MIN_LINE_COVERAGE_TARGET }}
10 changes: 8 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,20 @@ fmt:
fmt-check:
cargo fmt --all --check

check:
cargo check --all-features

clippy-check:
cargo clippy --all-features

install-llvm-cov:
cargo install cargo-llvm-cov

test-llvm-cov:
cargo llvm-cov --workspace --all-features --summary-only --fail-under-lines 80
test-llvm-cov-report:
cargo llvm-cov --html --workspace --all-features

test-llvm-cov-target:
cargo llvm-cov --workspace --all-features --summary-only --fail-under-lines 70

test:
cargo test --all-features
Expand Down
54 changes: 27 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,32 +3,32 @@
A lightweight implementation of Redis Streams for Rust, allowing you to manage streaming messages in a simplified way. With redsumer you can:

- **Produce** new messages in a specific *stream*.
- **Consume** messages from specific *stream*, setting config parameters that allow you a flexible implementation. It also provides an option to minimize the possibility of two or more consumers from the same consumer group consuming the same message simultaneously.
- **Consume** messages from specific *stream*, setting config parameters that allow you a flexible implementation. It also provides an option to minimize the possibility of consuming the same message simultaneously by more than one consumers from the same consumer group.

To use ***redsumer*** from GitHub repository with specific version, set the dependency in Cargo.toml file as follows:

```ini
[dependencies]
redsumer = { git = "https://github.com/enerBit/redsumer-rs.git", package = "redsumer", version = "0.5.0-beta.1" }
redsumer = { git = "https://github.com/enerBit/redsumer-rs.git", package = "redsumer", version = "0.5.0" }
```

You can depend on it via cargo by adding the following dependency to your `Cargo.toml` file:

```ini
[dependencies]
redsumer = { version = "0.5.0-beta.1" }
redsumer = { version = "0.5.0" }
```

## Basic Usage

#### Produce a new stream message:

Create a new producer instance and produce a new stream message from a [BTreeMap](`std::collections::BTreeMap`):
Create a new producer instance and produce a new stream message from a **BTreeMap**:

```rust,no_run
use std::collections::BTreeMap;
use redsumer::*;
use redsumer::prelude::*;
use time::OffsetDateTime;
use uuid::Uuid;
Expand Down Expand Up @@ -68,29 +68,29 @@ async fn main() {
message_2.push(("id".to_string(), Uuid::new_v4().to_string()));
message_2.push(("started_at".to_string(), OffsetDateTime::now_utc().to_string()));
let id_1: Id = producer.produce_from_map(message_1).await.unwrap_or_else(|error| {
let reply_1: ProduceMessageReply = producer.produce_from_map(message_1).await.unwrap_or_else(|error| {
panic!("Error producing stream message from BTreeMap: {:?}", error.to_string());
});
let id_2: Id = producer.produce_from_items(message_2).await.unwrap_or_else(|error| {
let reply_2: ProduceMessageReply = producer.produce_from_items(message_2).await.unwrap_or_else(|error| {
panic!("Error producing stream message from Vec: {:?}", error.to_string());
});
println!("Message 1 produced with id: {:?}", id_1);
println!("Message 2 produced with id: {:?}", id_2);
println!("Message 1 produced with id: {:?}", reply_1);
println!("Message 2 produced with id: {:?}", reply_2);
}
```

Similar to the previous example, you can produce a message from a [HashMap](std::collections::HashMap) or a [HashSet](std::collections::HashSet).
Similar to the previous example, you can produce a message from a **HashMap** or a **HashSet**.

The [produce_from_map](Producer::produce_from_map) and [produce_from_items](Producer::produce_from_items) methods accepts generic types that implements the [ToRedisArgs](redis::ToRedisArgs) trait. Take a look at the documentation for more information.
The **produce_from_map** and **produce_from_item** methods accepts generic types that implements the **ToRedisArgs** trait. Take a look at the documentation for more information.

#### Consume messages from a stream:

Create a new consumer instance and consume messages from stream:

```rust,no_run
use redsumer::*;
use redsumer::prelude::*;
use redsumer::redis::StreamId;
#[tokio::main]
Expand Down Expand Up @@ -145,25 +145,25 @@ async fn main() {
});
loop {
let messages: Vec<StreamId> = consumer.consume().await.unwrap_or_else(|error| {
let consume_reply: ConsumeMessagesReply = consumer.consume().await.unwrap_or_else(|error| {
panic!("Error consuming messages from stream: {:?}", error);
});
for message in messages {
for message in consume_reply.get_messages() {
if consumer.is_still_mine(&message.id).unwrap_or_else(|error| {
panic!(
"Error checking if message is still in consumer pending list: {:?}", error
);
}) {
}).is_still_mine() {
// Process message ...
println!("Processing message: {:?}", message);
// ...
let ack: bool = consumer.ack(&message.id).await.unwrap_or_else(|error| {
let ack_reply: AckMessageReply = consumer.ack(&message.id).await.unwrap_or_else(|error| {
panic!("Error acknowledging message: {:?}", error);
});
if ack {
if ack_reply.was_acked() {
println!("Message acknowledged: {:?}", message);
}
}
Expand All @@ -172,29 +172,29 @@ async fn main() {
}
```

In this example, the [consume](Consumer::consume) method is called in a loop to consume messages from the stream.
The [consume](Consumer::consume) method returns a vector of [StreamId](redis::StreamId) instances. Each [StreamId](redis::StreamId) instance represents a message in the stream.
The [is_still_mine](Consumer::is_still_mine) method is used to check if the message is still in the consumer pending list.
If it is, the message is processed and then acknowledged using the [ack](Consumer::ack) method.
The [ack](Consumer::ack) method returns a boolean indicating if the message was successfully acknowledged.
In this example, the **consume** method is called in a loop to consume messages from the stream.
The **consume** method returns a vector of **StreamId** instances. Each **StreamId** instance represents a message in the stream.
The **is_still_mine** method is used to check if the message is still in the consumer pending list.
If it is, the message is processed and then acknowledged using the **ack** method.
The **ack** method returns a boolean indicating if the message was successfully acknowledged.

The main objective of this message consumption strategy is to minimize the possibility that two or more consumers from the same consumer group operating simultaneously consume the same message at the same time.
Knowing that it is a complex problem with no definitive solution, including business logic in the message processing instance will always improve results.

#### Utilities from [redis] crate:

The [redis] module provides utilities from the [redis](https://docs.rs/redis) crate. You can use these utilities to interact with Redis values and errors.
The **redis** module provides utilities from the **redis** crate. You can use these utilities to interact with Redis values and errors.

#### Unwrap [Value](redis::Value) to a specific type:
#### Unwrap **Value** to a specific type:

The [Value](redis::Value) enum represents a Redis value. It can be converted to a specific type using the [from_redis_value](redis::from_redis_value) function. This function can be imported from the [redis] module.
The **Value** enum represents a Redis value. It can be converted to a specific type using the **from_redis_value** function. This function can be imported from the **redis** module.

## Contributing

We welcome contributions to `redsumer-rs`. Here are some ways you can contribute:
We welcome contributions to **redsumer**. Here are some ways you can contribute:

- **Bug Reports**: If you find a bug, please create an issue detailing the problem, the steps to reproduce it, and the expected behavior.
- **Feature Requests**: If you have an idea for a new feature or an enhancement to an existing one, please create an issue describing your idea.
- **Pull Requests**: If you've fixed a bug or implemented a new feature, we'd love to see your work! Please submit a pull request. Make sure your code follows the existing style and all tests pass.

Thank you for your interest in improving `redsumer-rs`!
Thank you for your interest in improving **redsumer**!
62 changes: 35 additions & 27 deletions redsumer-rs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,58 +5,66 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## ✨ v0.5.0-beta.1 [2024-09-30]
## ✨ v0.5.0 [2024-10-26]

### Added:

- ⚡ Implement `Debug` for `ClientCredentials`.
- ⚡ Implement `CommunicationProtocol` type to define Redis Protocol version.
- ⚡ Implement `ClientArgs` and `RedisClientBuilder` to build Redis Client.
- ⚡ Implement `VerifyConnection` trait and `ping()` function to verify connection to Redis Server.
- ⚡ Implement `produce_from_map()`, `produce_from_items()` and `ProducerCommands` in producer core module.
- ⚡ Implement `ProducerConfig` to manage the configuration parameters for `Producer`. Implement `ClientArgs` in `Producer`. **[BreakingChange]**
- ⚡ Implement `ConsumerConfig` to manage the configuration parameters for `Consumer`. Implement `ClientArgs` in `Consumer`. Implement `ReadNewMessagesOptions` , `ReadPendingMessagesOptions` and `ClaimMessagesOptions` in `ConsumerConfig` **[BreakingChange]**
- ⚡ Implement `Debug` for `ClientCredentials`. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `CommunicationProtocol` type to define Redis Protocol version. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `ClientArgs` and `RedisClientBuilder` to build Redis Client. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `VerifyConnection` trait and `ping()` function to verify connection to Redis Server. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `produce_from_map()`, `produce_from_items()` and `ProducerCommands` in producer core module. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `ProducerConfig` to manage the configuration parameters for `Producer`. Implement `ClientArgs` in `Producer` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `ProduceMessageReply` to handle the response from `produce_from_map()` and `produce_from_items()` functions in `Producer`. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `ConsumerConfig` to manage the configuration parameters for `Consumer`. Implement `ClientArgs` in `Consumer`. Implement `ReadNewMessagesOptions` , `ReadPendingMessagesOptions` and `ClaimMessagesOptions` in `ConsumerConfig` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement types `LastDeliveredMilliseconds` and`TotalTimesDelivered` to handle the response from the `is_still_mine()` core function. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `ConsumeMessagesReply` to handle the response from `consume()` function in `Consumer`. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `AckMessageReply` to handle the response from `ack()` function in `Consumer`. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Implement `IsStillMineReply` to handle the response from `is_still_mine()` function in `Consumer`. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ Refactor in the import of library modules: The prelude, consumer, producer and client modules were implemented **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).

### Changed:

- 🚀 Rename `RedsumerProducer` to `Producer`. **[BreakingChange]**
- 🚀 Rename `RedsumerProducer` to `Producer` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 Include minimum line coverage target as a env variable in CI pipeline. By [@JMTamayo](https://github.com/JMTamayo).

### Removed:

- ❌ Remove `FromRedisValueHandler` from crate. **[BreakingChange]**
- ❌ Remove internal function `get_redis_client()` from client module.
- ❌ Remove `FromRedisValueHandler` from crate **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- ❌ Remove internal function `get_redis_client()` from client module. By [@JMTamayo](https://github.com/JMTamayo).
- ❌ Remove step `Upload coverage to Codecov` from CI pipeline. By [@JMTamayo](https://github.com/JMTamayo).

## ✨ v0.4.1 [2024-06-13]

### Fixed:

- 🛠 Fixing BUG reported in [issue #15](https://github.com/enerBit/redsumer-rs/issues/15) with arguments in function xclaim.
- 🛠 Fixing BUG reported in [issue #15](https://github.com/enerBit/redsumer-rs/issues/15) with arguments in function xclaim. By [@JMTamayo](https://github.com/JMTamayo).

## ✨ v0.4.0 [2024-04-23]

### Added:

- ⚡ Implementation of new types: `RedsumerResult`, `RedsumerError` and `Id`. **[BreakingChange]**
-`Debug` and `Clone` implementation in `RedsumerProducer` and `RedsumerConsumer`.
- ⚡ The consumer configuration parameters were implemented directly in `RedsumerConsumer`. **[BreakingChange]**
- ⚡ Implementation of new types: `RedsumerResult`, `RedsumerError` and `Id` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
-`Debug` and `Clone` implementation in `RedsumerProducer` and `RedsumerConsumer`. By [@JMTamayo](https://github.com/JMTamayo).
- ⚡ The consumer configuration parameters were implemented directly in `RedsumerConsumer` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).

### Fixed:

- 🛠 General refactoring of the package in order to improve performance.
- 🛠 General refactoring of the package in order to improve performance. By [@JMTamayo](https://github.com/JMTamayo).

### Changed:

- 🚀 New project structure as workspace.
- 🚀 Update dependencies and documentation.
- 🚀 Library modules reorganization. **[BreakingChange]**
- 🚀 `FromRedisValueImplHandler` was changed to `FromRedisValueHandler`. **[BreakingChange]**
- 🚀 The `produce_from_map()` method was replaced by the `produce()` method in `RedsumerProducer`. **[BreakingChange]**
- 🚀 The `validate_pending_message_ownership()` method was replaced by `is_still_mine()` in `RedsumerConsumer`. **[BreakingChange]**
- 🚀 The acknowledge method was replaced by ack in `RedsumerConsumer`. **[BreakingChange]**
- 🚀 The consume method was refactored in `RedsumerConsumer` in order to implement a new consumption methodology that allows scalability in distributed systems. To understand this new implementation in detail, take a look at the project https://github.com/elpablete/refactored-computing-machine.
- 🚀 New project structure as workspace. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 Update dependencies and documentation. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 Library modules reorganization **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 `FromRedisValueImplHandler` was changed to `FromRedisValueHandler` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 The `produce_from_map()` method was replaced by the `produce()` method in `RedsumerProducer` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 The `validate_pending_message_ownership()` method was replaced by `is_still_mine()` in `RedsumerConsumer` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 The acknowledge method was replaced by ack in `RedsumerConsumer` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- 🚀 The consume method was refactored in `RedsumerConsumer` in order to implement a new consumption methodology that allows scalability in distributed systems. To understand this new implementation in detail, take a look at the project https://github.com/elpablete/refactored-computing-machine. By [@JMTamayo](https://github.com/JMTamayo).

### Removed:

- ❌ The *stream_information.rs* module was removed from the project: `StreamInfo` and `StreamConsumersInfo` implementations were removed. **[BreakingChange]**
-`RedsumerConsumerOptions` was removed. **[BreakingChange]**
- ❌ The `produce_from_items()` method was removed from `RedsumerProducer`. **[BreakingChange]**
- ❌ The *stream_information.rs* module was removed from the project: `StreamInfo` and `StreamConsumersInfo` implementations were removed **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
-`RedsumerConsumerOptions` was removed **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
- ❌ The `produce_from_items()` method was removed from `RedsumerProducer` **[BreakingChange]**. By [@JMTamayo](https://github.com/JMTamayo).
5 changes: 0 additions & 5 deletions redsumer-rs/src/core/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,6 @@ impl ClientCredentials {
///
/// # Returns:
/// A new instance of [`ClientCredentials`].
///
/// ```rust,no_run
/// use redsumer::ClientCredentials;
/// let credentials = ClientCredentials::new("user", "password");
/// ```
pub fn new(user: &str, password: &str) -> ClientCredentials {
ClientCredentials {
user: user.to_owned(),
Expand Down
Loading

0 comments on commit 25ae598

Please sign in to comment.