Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dry run xcms on bridge hubs to get forwarded xcms across the W<>R bridge #6002

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
0a74c6e
fix(parachain-system): not include messages if empty
franciscoaguirre Oct 3, 2024
4ba9883
fix: exporters already add their message to xcmp-queue
franciscoaguirre Oct 3, 2024
b9755bf
doc: prdoc
franciscoaguirre Oct 3, 2024
4594acb
Merge branch 'master' into xcm-dry-run-redundant-forwarded-xcms
franciscoaguirre Oct 3, 2024
08dad22
test(bridge-hub-westend-integration-tests): add dry run test
franciscoaguirre Oct 4, 2024
196d265
Merge branch 'master' into xcm-dry-run-redundant-forwarded-xcms
franciscoaguirre Oct 4, 2024
5914f23
test(bridge-hub-rococo-integration-tests): add same test as in westend
franciscoaguirre Oct 4, 2024
84e60a0
test(emulated-integration-tests-common): refactor rococo and westend …
franciscoaguirre Oct 4, 2024
72d1a00
Merge branch 'master' into xcm-dry-run-redundant-forwarded-xcms
franciscoaguirre Oct 4, 2024
aab1b1d
doc: update prdoc
franciscoaguirre Oct 4, 2024
4cddc77
doc: add a note of why some impls are empty
franciscoaguirre Oct 4, 2024
250c987
".git/.scripts/commands/fmt/fmt.sh"
Oct 4, 2024
ddc07f3
Merge branch 'master' into xcm-dry-run-redundant-forwarded-xcms
franciscoaguirre Oct 4, 2024
5fab2ab
Merge branch 'master' into xcm-dry-run-redundant-forwarded-xcms
franciscoaguirre Oct 7, 2024
096b855
feat: allow dry-running on bridge hubs and getting xcms over the bridge
franciscoaguirre Oct 9, 2024
daa1d8a
doc: add prdoc
franciscoaguirre Oct 9, 2024
671a326
Merge branch 'master' into xcm-dry-run-pk-bridge
franciscoaguirre Oct 10, 2024
be83b0f
Merge branch 'master' into xcm-dry-run-pk-bridge
franciscoaguirre Oct 11, 2024
a6f8b11
Merge branch 'master' into xcm-dry-run-pk-bridge
bkontur Oct 15, 2024
840ab25
chore: some feedback
franciscoaguirre Oct 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions bridges/modules/messages/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,23 @@ codec = { workspace = true }
log = { workspace = true }
scale-info = { features = ["derive"], workspace = true }

# Bridge dependencies
# Bridge dependencies.
bp-header-chain = { workspace = true }
bp-messages = { workspace = true }
bp-runtime = { workspace = true }

# Substrate Dependencies
# Substrate dependencies.
frame-benchmarking = { optional = true, workspace = true }
frame-support = { workspace = true }
frame-system = { workspace = true }
sp-runtime = { workspace = true }
sp-std = { workspace = true }
sp-trie = { optional = true, workspace = true }

# Polkadot dependencies.
xcm = { workspace = true, package = "staging-xcm" }
xcm-builder = { workspace = true, package = "staging-xcm-builder" }

[dev-dependencies]
bp-runtime = { features = ["test-helpers"], workspace = true }
bp-test-utils = { workspace = true }
Expand Down
58 changes: 55 additions & 3 deletions bridges/modules/messages/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ use bp_messages::{
DeliveryPayments, DispatchMessage, FromBridgedChainMessagesProof, MessageDispatch,
ProvedLaneMessages, ProvedMessages,
},
ChainWithMessages, DeliveredMessages, InboundLaneData, InboundMessageDetails, MessageKey,
MessageNonce, MessagePayload, MessagesOperatingMode, OutboundLaneData, OutboundMessageDetails,
UnrewardedRelayersState, VerificationError,
ChainWithMessages, DeliveredMessages, InboundLaneData, InboundMessageDetails, LaneState,
MessageKey, MessageNonce, MessagePayload, MessagesOperatingMode, OutboundLaneData,
OutboundMessageDetails, UnrewardedRelayersState, VerificationError,
};
use bp_runtime::{
AccountIdOf, BasicOperatingMode, HashOf, OwnedBridgeModule, PreComputedSize, RangeInclusiveExt,
Expand All @@ -71,6 +71,8 @@ use bp_runtime::{
use codec::{Decode, Encode};
use frame_support::{dispatch::PostDispatchInfo, ensure, fail, traits::Get, DefaultNoBound};
use sp_std::{marker::PhantomData, prelude::*};
use xcm::prelude::*;
use xcm_builder::{BridgeMessage, InspectMessageQueues};

mod call_ext;
mod inbound_lane;
Expand Down Expand Up @@ -666,6 +668,56 @@ pub mod pallet {
}
}

impl<T: Config<I>, I: 'static> InspectMessageQueues for Pallet<T, I> {
fn clear_messages() {
// Best effort.
let _ = OutboundMessages::<T, I>::clear(u32::MAX, None);
// Need to reset the lane data because it's checked in some tests
// after sending a message.
OutboundLanes::<T, I>::translate::<(), _>(|_, _| {
Some(OutboundLaneData {
oldest_unpruned_nonce: 0,
latest_received_nonce: 0,
latest_generated_nonce: 0,
state: LaneState::Opened,
})
franciscoaguirre marked this conversation as resolved.
Show resolved Hide resolved
});
}

fn get_messages() -> Vec<(VersionedLocation, Vec<VersionedXcm<()>>)> {
use xcm::prelude::*;

let mut messages: Vec<(VersionedLocation, VersionedXcm<()>)> =
OutboundMessages::<T, I>::iter()
.map(|(_key, message)| {
let mut data = &message[..];
let payload = StoredMessagePayload::<T, I>::decode(&mut data).unwrap();
let BridgeMessage { universal_dest, message } =
BridgeMessage::decode(&mut &payload[..]).unwrap();
let destination_latest: InteriorLocation = universal_dest.try_into().unwrap();
let destination: VersionedLocation =
Location::new(0, destination_latest).into();
franciscoaguirre marked this conversation as resolved.
Show resolved Hide resolved
(destination, message)
})
.collect();
messages.sort_by(|a, b| a.0.cmp(&b.0));
bkontur marked this conversation as resolved.
Show resolved Hide resolved

let mut result: Vec<(VersionedLocation, Vec<VersionedXcm<()>>)> = Vec::new();
for (destination, message) in messages.into_iter() {
if let Some((last_destination, vector)) = result.last_mut() {
if *last_destination == destination {
vector.push(message);
continue;
}
}

result.push((destination, vec![message]));
}

result
}
}

/// Structure, containing a validated message payload and all the info required
/// to send it on the bridge.
#[derive(Debug, PartialEq, Eq)]
Expand Down
25 changes: 22 additions & 3 deletions cumulus/parachains/integration-tests/emulated/common/src/macros.rs
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ macro_rules! test_chain_can_claim_assets {

#[macro_export]
macro_rules! test_dry_run_transfer_across_pk_bridge {
( $sender_asset_hub:ty, $sender_bridge_hub:ty, $destination:expr ) => {
( $destination_network_id:ty, $sender_asset_hub:ty, $sender_bridge_hub:ty, $destination:expr, $destination_bridge_hub:expr ) => {
$crate::macros::paste::paste! {
use frame_support::{dispatch::RawOrigin, traits::fungible};
use sp_runtime::AccountId32;
Expand All @@ -418,10 +418,13 @@ macro_rules! test_dry_run_transfer_across_pk_bridge {
let initial_balance = transfer_amount * 10;

// Bridge setup.
$sender_bridge_hub::fund_para_sovereign($sender_asset_hub::para_id(), 10_000_000_000_000u128);
$sender_asset_hub::force_xcm_version($destination, XCM_VERSION);
$sender_bridge_hub::force_xcm_version($destination_bridge_hub, XCM_VERSION);
open_bridge_between_asset_hub_rococo_and_asset_hub_westend();

<$sender_asset_hub as TestExt>::execute_with(|| {
let mut remote_message = VersionedXcm::from(Xcm::<()>::new());
$sender_asset_hub::execute_with(|| {
type Runtime = <$sender_asset_hub as Chain>::Runtime;
type RuntimeCall = <$sender_asset_hub as Chain>::RuntimeCall;
type OriginCaller = <$sender_asset_hub as Chain>::OriginCaller;
Expand All @@ -446,7 +449,23 @@ macro_rules! test_dry_run_transfer_across_pk_bridge {
// We assert the dry run succeeds and sends only one message to the local bridge hub.
assert!(result.execution_result.is_ok());
assert_eq!(result.forwarded_xcms.len(), 1);
assert_eq!(result.forwarded_xcms[0].0, VersionedLocation::from(Location::new(1, [Parachain($sender_bridge_hub::para_id().into())])));
let (destination, messages) = result.forwarded_xcms[0].clone();
assert_eq!(destination, VersionedLocation::from(Location::new(1, [Parachain($sender_bridge_hub::para_id().into())])));
assert_eq!(messages.len(), 1);
remote_message = messages[0].clone();
});

$sender_bridge_hub::execute_with(|| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@franciscoaguirre
I think we should be able to test or dry-run the full route: AHK -> BHK ... BHP -> AHP.

This way, in the end, we'll be able to write or run a periodic TypeScript/PAPI/whatever script that tests/simulates/dry-runs an asset transfer from AHK to AHP live, without the need to manually intervene (no chopsticks required).

I think the only missing piece is the DescendOrigin I mentioned here. We need to trigger some pre-dispatch, fake-message-proof-delivery, or whatever runtime API on the bridged BridgeHub, that will ultimately append the DescendOrigin. Let me think about it more.

Copy link
Contributor

@acatangiu acatangiu Oct 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't the dispatch (and DescendOrigin appending) happen before sending to outbound HRMP queue? I mean outbound HRMP Q of BH will contain full XCM exactly as it goes to AH, so I don't see the problem

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, not exactly, bridged/target BridgeHub appends to the xcm DescendOrigin when dispatching: https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/xcm-builder/src/universal_exports.rs#L433-L439

so the XCM in outbound Q on sender BridgeHub is not exactly the same as it comes to the target AH

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but the appending DescendOrigin we can do "offchain" (in the test, PAPI call), so this PR should be enough probably

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking of adding chopsticks tests for this under integration-tests/chopsticks/ but in another PR

type Runtime = <$sender_bridge_hub as Chain>::Runtime;
type RuntimeCall = <$sender_bridge_hub as Chain>::RuntimeCall;
type OriginCaller = <$sender_bridge_hub as Chain>::OriginCaller;

let xcm_program =
VersionedXcm::from(Xcm::<RuntimeCall>::from(remote_message.clone().try_into().unwrap()));
let asset_hub_as_seen_by_bridge_hub = $sender_bridge_hub::sibling_location_of($sender_asset_hub::para_id());
let result = Runtime::dry_run_xcm(asset_hub_as_seen_by_bridge_hub.clone().into(), xcm_program).unwrap();
assert_eq!(result.forwarded_xcms.len(), 1);
assert_eq!(result.forwarded_xcms[0].0, VersionedLocation::from(Location::new(0, [GlobalConsensus(NetworkId::$destination_network_id), Parachain($sender_asset_hub::para_id().into())])));
});
}
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -538,8 +538,10 @@ fn send_back_wnds_from_penpal_rococo_through_asset_hub_rococo_to_asset_hub_weste
#[test]
fn dry_run_transfer_to_westend_sends_xcm_to_bridge_hub() {
test_dry_run_transfer_across_pk_bridge!(
Westend,
AssetHubRococo,
BridgeHubRococo,
asset_hub_westend_location()
asset_hub_westend_location(),
bridge_hub_westend_location()
);
}
Original file line number Diff line number Diff line change
Expand Up @@ -558,8 +558,10 @@ fn send_back_rocs_from_penpal_westend_through_asset_hub_westend_to_asset_hub_roc
#[test]
fn dry_run_transfer_to_rococo_sends_xcm_to_bridge_hub() {
test_dry_run_transfer_across_pk_bridge!(
Rococo,
AssetHubWestend,
BridgeHubWestend,
asset_hub_rococo_location()
asset_hub_rococo_location(),
bridge_hub_rococo_location()
);
}
Original file line number Diff line number Diff line change
Expand Up @@ -869,7 +869,7 @@ impl_runtime_apis! {
}

fn dry_run_xcm(origin_location: VersionedLocation, xcm: VersionedXcm<RuntimeCall>) -> Result<XcmDryRunEffects<RuntimeEvent>, XcmDryRunApiError> {
PolkadotXcm::dry_run_xcm::<Runtime, xcm_config::XcmRouter, RuntimeCall, xcm_config::XcmConfig>(origin_location, xcm)
PolkadotXcm::dry_run_xcm::<Runtime, (xcm_config::XcmRouter, BridgeWestendMessages), RuntimeCall, xcm_config::XcmConfig>(origin_location, xcm)
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -801,7 +801,7 @@ impl_runtime_apis! {
}

fn dry_run_xcm(origin_location: VersionedLocation, xcm: VersionedXcm<RuntimeCall>) -> Result<XcmDryRunEffects<RuntimeEvent>, XcmDryRunApiError> {
PolkadotXcm::dry_run_xcm::<Runtime, xcm_config::XcmRouter, RuntimeCall, xcm_config::XcmConfig>(origin_location, xcm)
PolkadotXcm::dry_run_xcm::<Runtime, (xcm_config::XcmRouter, BridgeRococoMessages), RuntimeCall, xcm_config::XcmConfig>(origin_location, xcm)
}
}

Expand Down
5 changes: 4 additions & 1 deletion polkadot/xcm/pallet-xcm/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2559,7 +2559,10 @@ impl<T: Config> Pallet<T> {
XcmDryRunApiError::VersionedConversionFailed
})?;
let mut hash = xcm.using_encoded(sp_io::hashing::blake2_256);
frame_system::Pallet::<Runtime>::reset_events(); // To make sure we only record events from current call.
// Clear other messages in queues...
Router::clear_messages();
// ...and reset events to make sure we only record events from current call.
frame_system::Pallet::<Runtime>::reset_events();
let result = xcm_executor::XcmExecutor::<XcmConfig>::prepare_and_execute(
origin_location,
xcm,
Expand Down
22 changes: 22 additions & 0 deletions prdoc/pr_6002.prdoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Schema: Polkadot SDK PRDoc Schema (prdoc) v1.0.0
# See doc at https://raw.githubusercontent.com/paritytech/polkadot-sdk/master/prdoc/schema_user.json

title: Dry run xcms on bridge hubs to get forwarded xcms across the W<>R bridge

doc:
- audience: Runtime User
description: |
It's now possible to dry-run XCMs on Bridge Hubs and get the forwarded XCM to the other
side of the W<>R bridge.

crates:
- name: pallet-bridge-messages
bump: patch
- name: emulated-integration-tests-common
bump: patch
- name: bridge-hub-rococo-runtime
bump: patch
- name: bridge-hub-westend-runtime
bump: patch
- name: pallet-xcm
bump: patch
Loading