Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import CPU code #292

Merged
merged 8 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
319 changes: 150 additions & 169 deletions Cargo.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
[workspace]
members = [
"iris-mpc",
"iris-mpc-cpu",
"iris-mpc-gpu",
"iris-mpc-common",
"iris-mpc-upgrade",
Expand Down
22 changes: 22 additions & 0 deletions iris-mpc-cpu/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
[package]
name = "iris-mpc-cpu"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html


[dependencies]
aes-prng = "0.2"
bytes = "1.7"
bytemuck.workspace = true
eyre.workspace = true
futures.workspace = true
iris-mpc-common = { path = "../iris-mpc-common" }
num-traits = "0.2"
rand.workspace = true
rayon.workspace = true
serde.workspace = true
static_assertions.workspace = true
thiserror = "1.0"
tokio.workspace = true
tracing.workspace = true
5 changes: 5 additions & 0 deletions iris-mpc-cpu/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pub(crate) mod networks;
pub mod prelude;
pub(crate) mod protocol;
pub(crate) mod shares;
pub(crate) mod utils;
2 changes: 2 additions & 0 deletions iris-mpc-cpu/src/networks/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
pub(crate) mod network_trait;
pub(crate) mod test_network;
42 changes: 42 additions & 0 deletions iris-mpc-cpu/src/networks/network_trait.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
use bytes::{Bytes, BytesMut};
use eyre::{Error, Result};
use iris_mpc_common::id::PartyID;
pub type IoError = std::io::Error;

#[allow(async_fn_in_trait)]
pub trait NetworkTrait: Send + Sync {
fn get_id(&self) -> PartyID;

async fn shutdown(self) -> Result<(), IoError>;

async fn send(&mut self, id: PartyID, data: Bytes) -> Result<(), IoError>;
async fn send_next_id(&mut self, data: Bytes) -> Result<(), IoError>;
async fn send_prev_id(&mut self, data: Bytes) -> Result<(), IoError>;

async fn receive(&mut self, id: PartyID) -> Result<BytesMut, IoError>;
async fn receive_prev_id(&mut self) -> Result<BytesMut, IoError>;
async fn receive_next_id(&mut self) -> Result<BytesMut, IoError>;

async fn broadcast(&mut self, data: Bytes) -> Result<Vec<BytesMut>, IoError>;
//======= sync world =========
fn blocking_send(&mut self, id: PartyID, data: Bytes) -> Result<(), IoError>;
fn blocking_send_next_id(&mut self, data: Bytes) -> Result<(), IoError>;
fn blocking_send_prev_id(&mut self, data: Bytes) -> Result<(), IoError>;

fn blocking_receive(&mut self, id: PartyID) -> Result<BytesMut, IoError>;
fn blocking_receive_prev_id(&mut self) -> Result<BytesMut, IoError>;
fn blocking_receive_next_id(&mut self) -> Result<BytesMut, IoError>;

fn blocking_broadcast(&mut self, data: Bytes) -> Result<Vec<BytesMut>, IoError>;
Comment on lines +12 to +30
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should also maybe consider reducing this down to the actual networking impl set that we want to have (async/sync)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, there is no concrete impl here other than the test-network right? What is the plan here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, there's not concrete impl other than that. We were thinking that the plan could be to get in the Aby3Store implementation which uses the TestNetwork and then refactor the network according to needs.
See https://github.com/Inversed-Tech/iris-hawk/blob/main/src/aby3_store.rs for reference

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is something for the infra team to dictate, in our impl we used QUIC using the quinn library as a concrete impl, but I guess just standard TLS over TCP is also fine. I think there was some talk of actually using NCCL here, but I don't think the ergonomics (and probably also perf) are very nice

Copy link
Collaborator

@rdragos rdragos Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think why I like using &self instead of &mut self is that it gets rid of the locks when doing operations on a global network that I'm doing: https://github.com/Inversed-Tech/iris-hawk/blob/main/src/aby3_store.rs#L146

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, mostly because the interface of hawk-pack is &self. But there would still need to be a lot of syncing internally otherwise you cannot use the networking impl for MT code anyway, as concurrent calls to less_than would use the same networking and therefore may receive messages in the wrong order. So a correct implementation would need to separate the channels/multiplex the networking correctly anyway using internal locking etc. I think it would be easier to have the lock in there, and e.g., grab a new channel from the "global" network manager that can be used for the local execution here. The "global" network manager can then internally use the locks to allow a &self bound, get a new channel for the protocol and execute it on that Channel it exclusively owns.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I agree. The global network manager should have the locking logic inside of it, rather than spilling into other parts of the code.
Is this already done somewhere, or do you reckon this would be part of a large refactoring?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no sadly this is not implemented right now

}

#[allow(async_fn_in_trait)]
pub trait NetworkEstablisher<N: NetworkTrait> {
fn get_id(&self) -> PartyID;
fn get_num_parties(&self) -> usize;
async fn open_channel(&mut self) -> Result<N, Error>;
async fn shutdown(self) -> Result<(), Error>;
//======= sync world =========
fn print_connection_stats(&self, out: &mut impl std::io::Write) -> std::io::Result<()>;
fn get_send_receive(&self, i: usize) -> std::io::Result<(u64, u64)>;
}
Loading
Loading