Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(code/test): Generalize the test framework to make it work with any app #836

Merged
merged 74 commits into from
Feb 12, 2025

Conversation

romac
Copy link
Member

@romac romac commented Feb 6, 2025

Closes: #XXX

Summary

With this PR, any app can now use the test framework and re-use the existing tests (sort of).

For this, an application should first preferably create a new crate holding the tests, eg. myapp-test:

❯ tree myapp-test/
myapp-test/
├── Cargo.toml
└── src
    └── lib.rs

Then add dependencies on the following crates:

  • myapp: Assuming that this crate is where the Context is defined
  • malachitebft-test-framework: The test framework itself

In this crate, they can copy over the code/crates/starknet/test/src/tests folder from the Malachite source tree in this PR to myapp/src/tests:

❯ cp -R code/crates/starknet/test/src/tests myapp-test/src/tests
❯ tree myapp-test/
myapp-test/
├── Cargo.toml
└── src
    ├── lib.rs
    └── tests
        ├── full_nodes.rs
        ├── mod.rs
        ├── n3f0.rs
        ├── n3f0_consensus_mode.rs
        ├── n3f0_pubsub_protocol.rs
        ├── n3f1.rs
        ├── value_sync.rs
        ├── vote_sync.rs
        └── wal.rs

In myapp-test/src/lib.rs, paste the following and fill in the methods with todo!() (insert owl drawing here):

use std::collections::HashMap;
use std::path::PathBuf;

use async_trait::async_trait;
use tempfile::TempDir;
use tokio::task::JoinHandle;

use malachitebft_test_framework::HasTestRunner;
use malachitebft_test_framework::{NodeRunner, TestNode};

pub use malachitebft_test_framework::TestBuilder as GenTestBuilder;
pub use malachitebft_test_framework::{
    init_logging, EngineHandle, HandlerResult, Handles, Node, NodeId, TestParams,
};

use myapp::{MyAppContext, Height, PrivateKey, ValidatorSet};


#[cfg(test)]
pub mod tests;

pub type TestBuilder<S> = GenTestBuilder<MyAppContext, S>;

pub struct Handle {
    pub handle: JoinHandle<()>,
    pub tx_event: TxEvent<MockContext>,
}

#[async_trait]
impl NodeHandle<MyAppContext> for Handle {
    fn subscribe(&self) -> RxEvent< MyAppContext > {
        self.tx_event.subscribe()
    }

    async fn kill(&self, _reason: Option<String>) -> eyre::Result<()> {
        self.handle.abort();
        Ok(())
    }
}


impl HasTestRunner<TestRunner> for MyAppContext {
    type Runner = TestRunner;
}

#[derive(Clone)]
pub struct TestRunner {
    pub id: usize,
    pub params: TestParams,
    pub nodes_count: usize,
    pub start_height: HashMap<NodeId, Height>,
    pub home_dir: HashMap<NodeId, PathBuf>,
    pub private_keys: HashMap<NodeId, PrivateKey>,
    pub validator_set: ValidatorSet,
    pub consensus_base_port: usize,
    pub mempool_base_port: usize,
    pub metrics_base_port: usize,
}

fn temp_dir(id: NodeId) -> PathBuf {
    TempDir::with_prefix(format!("myapp-test-{id}-"))
        .unwrap()
        .into_path()
}

#[async_trait]
impl NodeRunner<MyAppContext> for TestRunner {
    type NodeHandle = Handle;

    fn new<S>(id: usize, nodes: &[TestNode<MyAppContext, S>], params: TestParams) -> Self {
        let nodes_count = nodes.len();
        let base_port = 20_000 + id * 1000;

        let (validators, private_keys) = make_validators(nodes);
        let validator_set = ValidatorSet::new(validators);

        let start_height = nodes
            .iter()
            .map(|node| (node.id, node.start_height))
            .collect();

        let home_dir = nodes
            .iter()
            .map(|node| (node.id, temp_dir(node.id)))
            .collect();

        Self {
            id,
            params,
            nodes_count,
            start_height,
            home_dir,
            private_keys,
            validator_set,
            consensus_base_port: base_port,
            mempool_base_port: base_port + 100,
            metrics_base_port: base_port + 200,
        }
    }

    /// Spawn a new node with the given id and return a set of handles 
    /// for the test framework to be able to kill the node
    async fn spawn(&self, id: NodeId) -> eyre::Result<Handle> {
        todo!()
    }

    /// Reset the database of the node with the given id.
    /// Must reset the WAL, the block store, etc.
    async fn reset_db(&self, id: NodeId) -> eyre::Result<()> {
        todo!()
    }
}

Notes

  • Calling Handle::kill should stop both the application and consensus right away, and wait for both to be stopped before returning.
  • Handle::tx_event must be a valid TxEvent instance which broadcasts events from the Consensus actor

PR author checklist

For all contributors

For external contributors

@romac romac added work in progress Work in progress code Code/implementation related labels Feb 6, 2025
@informalsystems informalsystems deleted a comment from codecov bot Feb 6, 2025
Base automatically changed from romac/app-channel-test to main February 11, 2025 13:37
@romac romac marked this pull request as ready for review February 11, 2025 15:35
Copy link
Collaborator

@ancazamfir ancazamfir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great! 🚀

@romac romac added this pull request to the merge queue Feb 12, 2025
Merged via the queue into main with commit 490766e Feb 12, 2025
22 checks passed
@romac romac deleted the romac/generic-test-framework branch February 12, 2025 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code Code/implementation related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants