From a752337876e75039b391d8f619a524be9f1ffd66 Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Mon, 22 Jul 2024 17:51:54 +0100 Subject: [PATCH 01/14] docs: ahead of v1.3.1 Signed-off-by: Enrique Lacal --- doc-site/docs/releasenotes/index.md | 7 +++ doc-site/docs/tutorials/chains/index.md | 5 -- .../tutorials/custom_contracts/ethereum.md | 21 +++++--- .../docs/tutorials/custom_contracts/fabric.md | 50 +++++++++++++------ 4 files changed, 54 insertions(+), 29 deletions(-) delete mode 100644 doc-site/docs/tutorials/chains/index.md diff --git a/doc-site/docs/releasenotes/index.md b/doc-site/docs/releasenotes/index.md index 01abc7415f..5b3fededa0 100644 --- a/doc-site/docs/releasenotes/index.md +++ b/doc-site/docs/releasenotes/index.md @@ -4,6 +4,13 @@ title: Release Notes [Full release notes](https://github.com/hyperledger/firefly/releases) +## [v1.3.1 - , 2024](https://github.com/hyperledger/firefly/releases/tag/v1.3.1) + +What's New: + +- Enable contract listeners with multiple filters +- New multiparty status API at `/status/multiparty` + ## [v1.3.0 - April 25, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.1.0) [Migration guide](1.3_migration_guide.md) diff --git a/doc-site/docs/tutorials/chains/index.md b/doc-site/docs/tutorials/chains/index.md deleted file mode 100644 index 792e6a71e9..0000000000 --- a/doc-site/docs/tutorials/chains/index.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Connecting to other blockchains ---- - -Write interesting stuff here diff --git a/doc-site/docs/tutorials/custom_contracts/ethereum.md b/doc-site/docs/tutorials/custom_contracts/ethereum.md index 52e91f936e..aec21eb533 100644 --- a/doc-site/docs/tutorials/custom_contracts/ethereum.md +++ b/doc-site/docs/tutorials/custom_contracts/ethereum.md @@ -648,7 +648,7 @@ Here is an example of sending 100 wei with a transaction: Now that we've seen how to submit transactions and preform read-only queries to the blockchain, let's look at how to receive blockchain events so we know when things are happening in realtime. -If you look at the source code for the smart contract we're working with above, you'll notice that it emits an event when the stored value of the integer is set. In order to receive these events, we first need to instruct FireFly to listen for this specific type of blockchain event. To do this, we create an **Event Listener**. The `/contracts/listeners` endpoint is RESTful so there are `POST`, `GET`, and `DELETE` methods available on it. To create a new listener, we will make a `POST` request. We are going to tell FireFly to listen to events with name `"Changed"` from the FireFly Interface we defined earlier, referenced by its ID. We will also tell FireFly which contract address we expect to emit these events, and the topic to assign these events to. Topics are a way for applications to subscribe to events they are interested in. +If you look at the source code for the smart contract we're working with above, you'll notice that it emits an event when the stored value of the integer is set. In order to receive these events, we first need to instruct FireFly to listen for this specific type of blockchain event. To do this, we create an **Event Listener**. The `/contracts/listeners` endpoint is RESTful so there are `POST`, `GET`, and `DELETE` methods available on it. To create a new listener, we will make a `POST` request. We are going to tell FireFly to listen to events with name `"Changed"` from the FireFly Interface we defined earlier, referenced by its ID. We will also tell FireFly which contract address we expect to emit these events, and the topic to assign these events to. You can specify multiple filters for a listener, in this case we only specify one for our event. Topics are a way for applications to subscribe to events they are interested in. ### Request @@ -656,13 +656,17 @@ If you look at the source code for the smart contract we're working with above, ```json { - "interface": { - "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" - }, - "location": { - "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" - }, - "eventPath": "Changed", + "filters": [ + { + "interface": { + "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" + }, + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "eventPath": "Changed" + } + ], "options": { "firstEvent": "newest" }, @@ -671,6 +675,7 @@ If you look at the source code for the smart contract we're working with above, ``` ### Response +// TODO update this response ```json { diff --git a/doc-site/docs/tutorials/custom_contracts/fabric.md b/doc-site/docs/tutorials/custom_contracts/fabric.md index 2b4723ca1a..037a3a19e5 100644 --- a/doc-site/docs/tutorials/custom_contracts/fabric.md +++ b/doc-site/docs/tutorials/custom_contracts/fabric.md @@ -576,16 +576,20 @@ The `/contracts/listeners` endpoint is RESTful so there are `POST`, `GET`, and ` ```json { - "interface": { - "id": "f1e5522c-59a5-4787-bbfd-89975e5b0954" - }, - "location": { - "channel": "firefly", - "chaincode": "asset_transfer" - }, - "event": { - "name": "AssetCreated" - }, + "filters": [ + { + "interface": { + "id": "f1e5522c-59a5-4787-bbfd-89975e5b0954" + }, + "location": { + "channel": "firefly", + "chaincode": "asset_transfer" + }, + "event": { + "name": "AssetCreated" + } + } + ], "options": { "firstEvent": "oldest" }, @@ -597,25 +601,39 @@ The `/contracts/listeners` endpoint is RESTful so there are `POST`, `GET`, and ` ```json { - "id": "6e7f5dd8-5a57-4163-a1d2-5654e784dc31", + "id": "d6b5e774-c9e5-474c-9495-ec07fa47a907", "namespace": "default", - "name": "sb-2cac2bfa-38af-4408-4ff3-973421410e5d", - "backendId": "sb-2cac2bfa-38af-4408-4ff3-973421410e5d", + "name": "sb-44aa348a-bafb-4243-594e-dcad689f1032", + "backendId": "sb-44aa348a-bafb-4243-594e-dcad689f1032", "location": { "channel": "firefly", "chaincode": "asset_transfer" }, - "created": "2022-05-02T17:19:13.144561086Z", + "created": "2024-07-22T15:36:58.514085959Z", "event": { "name": "AssetCreated", "description": "", "params": null }, - "signature": "AssetCreated", + "signature": "firefly-asset_transfer:AssetCreated", "topic": "assets", "options": { "firstEvent": "oldest" - } + }, + "filters": [ + { + "event": { + "name": "AssetCreated", + "description": "", + "params": null + }, + "location": { + "channel": "firefly", + "chaincode": "asset_transfer" + }, + "signature": "firefly-asset_transfer:AssetCreated" + } + ] } ``` From d04c1e8780a50ae3752c63cbcd9ee36a00cfcf4a Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Tue, 30 Jul 2024 19:09:00 +0100 Subject: [PATCH 02/14] add a note on backwards compatibility Signed-off-by: Enrique Lacal --- doc-site/docs/releasenotes/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc-site/docs/releasenotes/index.md b/doc-site/docs/releasenotes/index.md index 5b3fededa0..c1c399f892 100644 --- a/doc-site/docs/releasenotes/index.md +++ b/doc-site/docs/releasenotes/index.md @@ -8,7 +8,7 @@ title: Release Notes What's New: -- Enable contract listeners with multiple filters +- Enable contract listeners with multiple filters, old format is persevered for backwards compatibility - New multiparty status API at `/status/multiparty` ## [v1.3.0 - April 25, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.1.0) From b8c87b00a2a7cf17c811b184cbb5b82fd2e37ddd Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Tue, 30 Jul 2024 19:13:28 +0100 Subject: [PATCH 03/14] Update response of multiple filters for ethereum Signed-off-by: Enrique Lacal --- .../tutorials/custom_contracts/ethereum.md | 55 ++++++++++++++++--- 1 file changed, 47 insertions(+), 8 deletions(-) diff --git a/doc-site/docs/tutorials/custom_contracts/ethereum.md b/doc-site/docs/tutorials/custom_contracts/ethereum.md index aec21eb533..26de3d47c4 100644 --- a/doc-site/docs/tutorials/custom_contracts/ethereum.md +++ b/doc-site/docs/tutorials/custom_contracts/ethereum.md @@ -675,21 +675,20 @@ If you look at the source code for the smart contract we're working with above, ``` ### Response -// TODO update this response ```json { - "id": "1bfa3b0f-3d90-403e-94a4-af978d8c5b14", + "id": "e7c8457f-4ffd-42eb-ac11-4ad8aed30de1", "interface": { - "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" + "id": "55fdb62a-fefc-4313-99e4-e3f95fcca5f0" }, "namespace": "default", - "name": "sb-66209ffc-d355-4ac0-7151-bc82490ca9df", - "protocolId": "sb-66209ffc-d355-4ac0-7151-bc82490ca9df", + "name": "019104d7-bb0a-c008-76a9-8cb923d91b37", + "backendId": "019104d7-bb0a-c008-76a9-8cb923d91b37", "location": { "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" }, - "created": "2022-02-17T22:02:36.34549538Z", + "created": "2024-07-30T18:12:12.704964Z", "event": { "name": "Changed", "description": "", @@ -717,9 +716,49 @@ If you look at the source code for the smart contract we're working with above, } ] }, + "signature": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]", + "topic": "simple-storage", "options": { - "firstEvent": "oldest" - } + "firstEvent": "newest" + }, + "filters": [ + { + "event": { + "name": "Changed", + "description": "", + "params": [ + { + "name": "from", + "schema": { + "type": "string", + "details": { + "type": "address", + "internalType": "address", + "indexed": true + } + } + }, + { + "name": "value", + "schema": { + "type": "integer", + "details": { + "type": "uint256", + "internalType": "uint256" + } + } + } + ] + }, + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "interface": { + "id": "55fdb62a-fefc-4313-99e4-e3f95fcca5f0" + }, + "signature": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]" + } + ] } ``` From 99b85de75cb5e28fb4e5f454671a2cdff8b86917 Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Wed, 31 Jul 2024 13:06:18 +0100 Subject: [PATCH 04/14] Add date to release Signed-off-by: Enrique Lacal --- doc-site/docs/releasenotes/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc-site/docs/releasenotes/index.md b/doc-site/docs/releasenotes/index.md index c1c399f892..59306c69f2 100644 --- a/doc-site/docs/releasenotes/index.md +++ b/doc-site/docs/releasenotes/index.md @@ -4,7 +4,7 @@ title: Release Notes [Full release notes](https://github.com/hyperledger/firefly/releases) -## [v1.3.1 - , 2024](https://github.com/hyperledger/firefly/releases/tag/v1.3.1) +## [v1.3.1 - July 31, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.3.1) What's New: From b742776c744bf660e357755bfc8a572c60814536 Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Thu, 1 Aug 2024 17:00:01 +0100 Subject: [PATCH 05/14] Add more explanation for contract listeners Signed-off-by: Enrique Lacal --- doc-site/docs/reference/events.md | 3 + .../docs/reference/types/contractlistener.md | 157 +++++++++++++++++- 2 files changed, 159 insertions(+), 1 deletion(-) diff --git a/doc-site/docs/reference/events.md b/doc-site/docs/reference/events.md index faf51cc390..ee3ba1f71b 100644 --- a/doc-site/docs/reference/events.md +++ b/doc-site/docs/reference/events.md @@ -187,6 +187,9 @@ Once you have configured the blockchain event listener, every event detected from the blockchain will result in a FireFly event delivered to your application of type `blockchain_event_received`. +As of 1.3.1 a group of event filters can be established under a single topic when supported by the connector, which has benefits for ordering. +See [Contract Listeners](../reference/types/contractlistener.md) for more detail + Check out the [Custom Contracts Tutorial](../tutorials/custom_contracts/index.md) for a walk-through of how to set up listeners for the events from your smart contracts. diff --git a/doc-site/docs/reference/types/contractlistener.md b/doc-site/docs/reference/types/contractlistener.md index 68c2be28db..8defa162db 100644 --- a/doc-site/docs/reference/types/contractlistener.md +++ b/doc-site/docs/reference/types/contractlistener.md @@ -3,7 +3,162 @@ title: ContractListener --- {% include-markdown "./_includes/contractlistener_description.md" %} -### Example +See below a deep dive into the format of contract listeners and important concepts to understand when managing them. + +### Multiple filters + +From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. + +Before this change each contract listener would only support listening to one specific event from an interface previously defined. A connector specific event signature would be created to easily identify and search for the contact listener for that event. Within that listener, you could optionally specify a location to listen from. Alongside declaring what and where to listen from, you need to specify a `topic` as describe in (TODO LINK) for the ordering constraint of your application. See below for an example with multiple filters. This format is still supported by the API. + +Now from v1.3.1, we have extended the contract listeners API to enable users to supply multiple filters where each contains the event to listen to and the optional location. Each filter will also contain a new connector specific signature which is constructed from the location and the event signature. FireFly will restrict the creation of a contract listener with duplicate filters or supersets. For example, if two filters are listening to the same event but one has specified a location and the other hasn't then the latter will be a superset and already be listening to all the events matching the first filter. + +FireFly made a conscious decision to restricts the creation of duplicate contract listeners and would use the values mentioned earlier to create a unique combination of signature + topic + location. If you tried to create the same listener, you would get a 409. This combination is super useful to a developer to asset that there listener exists. When using muliple filters we can no longer rely on the signature of one event and the location of one filters to calculate uniqueness of a contract listener. As part of v1.3.1, we have come up with a new format where we create a new signature format per filter of location + event signature and then concatinate all of these signatures to build the contract listener signature. This contract listener signature containing all the locations and event signtures combined with the topic will guarantee uniqueness of the contract listener. + +### Signature enhancements + +As mentioned above, we have introduced a new format for signatures of contract listener and filter signature: +- Each filter signature will be a combination of the location and the specific connector event signature +- Each contract listener signature will be a concatination of all the filter signatures + +Furthemore, we noticed that specifically for Ethereum the ABI does not include a differenciation between indexed and non-indexed fields so we included a new section to the Ethereum specific event signature to add the index of the indexed fields on the signature, as such a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma separate list of the index of those indexed fields such as `[i=0,2,3]` specifies that fields at index 0, 2, and 3 are indexed fields. + +### Example with multiple filters + +These examples are using the Ethereum connector. + +Request payload + +```json +{ + "filters": [ + { + "interface": { + "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" + }, + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "eventPath": "Changed" + } + ], + "options": { + "firstEvent": "newest" + }, + "topic": "simple-storage" +} +``` + + +Response payload + +Notice that both the +```json +{ + "id": "e7c8457f-4ffd-42eb-ac11-4ad8aed30de1", + "interface": { + "id": "55fdb62a-fefc-4313-99e4-e3f95fcca5f0" + }, + "namespace": "default", + "name": "019104d7-bb0a-c008-76a9-8cb923d91b37", + "backendId": "019104d7-bb0a-c008-76a9-8cb923d91b37", + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "created": "2024-07-30T18:12:12.704964Z", + "event": { + "name": "Changed", + "description": "", + "params": [ + { + "name": "from", + "schema": { + "type": "string", + "details": { + "type": "address", + "internalType": "address", + "indexed": true + } + } + }, + { + "name": "value", + "schema": { + "type": "integer", + "details": { + "type": "uint256", + "internalType": "uint256" + } + } + } + ] + }, + "signature": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]", + "topic": "simple-storage", + "options": { + "firstEvent": "newest" + }, + "filters": [ + { + "event": { + "name": "Changed", + "description": "", + "params": [ + { + "name": "from", + "schema": { + "type": "string", + "details": { + "type": "address", + "internalType": "address", + "indexed": true + } + } + }, + { + "name": "value", + "schema": { + "type": "integer", + "details": { + "type": "uint256", + "internalType": "uint256" + } + } + } + ] + }, + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "interface": { + "id": "55fdb62a-fefc-4313-99e4-e3f95fcca5f0" + }, + "signature": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]" + } + ] +} +``` + + +### Example with single event (old format) + +Request payload + +```json +{ + "interface": { + "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" + }, + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "eventPath": "Changed", + "options": { + "firstEvent": "newest" + }, + "topic": "simple-storage" +} +``` ```json { From 97a5f29d64381bfeffb6415c6ffc848050122a94 Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Thu, 1 Aug 2024 17:14:26 +0100 Subject: [PATCH 06/14] fix make reference and move description to contract listener description Signed-off-by: Enrique Lacal --- .../_includes/contractlistener_description.md | 21 ++ .../docs/reference/types/contractlistener.md | 185 +++--------------- internal/reference/reference.go | 27 ++- 3 files changed, 74 insertions(+), 159 deletions(-) diff --git a/doc-site/docs/reference/types/_includes/contractlistener_description.md b/doc-site/docs/reference/types/_includes/contractlistener_description.md index e6c1721b8f..f4f2e1642e 100644 --- a/doc-site/docs/reference/types/_includes/contractlistener_description.md +++ b/doc-site/docs/reference/types/_includes/contractlistener_description.md @@ -4,3 +4,24 @@ of the interface for that event. Check out the [Custom Contracts Tutorial](../../tutorials/custom_contracts/index.md) for a walk-through of how to set up listeners for the events from your smart contracts. + + +See below for a deep dive into the format of contract listeners and important concepts to understand when managing them. + +### Multiple filters + +From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. + +Before this change each contract listener would only support listening to one specific event from an interface previously defined. A connector specific event signature would be created to easily identify and search for the contact listener for that event. Within that listener, you could optionally specify a location to listen from. Alongside declaring what and where to listen from, you need to specify a `topic` as describe in (TODO LINK) for the ordering constraint of your application. See below for an example with multiple filters. This format is still supported by the API. + +Now from v1.3.1, we have extended the contract listeners API to enable users to supply multiple filters where each contains the event to listen to and the optional location. Each filter will also contain a new connector specific signature which is constructed from the location and the event signature. FireFly will restrict the creation of a contract listener with duplicate filters or supersets. For example, if two filters are listening to the same event but one has specified a location and the other hasn't then the latter will be a superset and already be listening to all the events matching the first filter. Notice that for backwards compatibility the response from the API will populate the `event` and `location` with the contents of the first event filter in the array. + +FireFly made a conscious decision to restricts the creation of duplicate contract listeners and would use the values mentioned earlier to create a unique combination of signature + topic + location. If you tried to create the same listener, you would get a 409. This combination is super useful to a developer to asset that there listener exists. When using muliple filters we can no longer rely on the signature of one event and the location of one filters to calculate uniqueness of a contract listener. As part of v1.3.1, we have come up with a new format where we create a new signature format per filter of location + event signature and then concatinate all of these signatures to build the contract listener signature. This contract listener signature containing all the locations and event signtures combined with the topic will guarantee uniqueness of the contract listener. + +### Signature enhancements + +As mentioned above, we have introduced a new format for signatures of contract listener and filter signature: +- Each filter signature will be a combination of the location and the specific connector event signature +- Each contract listener signature will be a concatination of all the filter signatures + +Furthemore, we noticed that specifically for Ethereum the ABI does not include a differenciation between indexed and non-indexed fields so we included a new section to the Ethereum specific event signature to add the index of the indexed fields on the signature, as such a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma separate list of the index of those indexed fields such as `[i=0,2,3]` specifies that fields at index 0, 2, and 3 are indexed fields. \ No newline at end of file diff --git a/doc-site/docs/reference/types/contractlistener.md b/doc-site/docs/reference/types/contractlistener.md index 8defa162db..9b5b13415a 100644 --- a/doc-site/docs/reference/types/contractlistener.md +++ b/doc-site/docs/reference/types/contractlistener.md @@ -3,162 +3,7 @@ title: ContractListener --- {% include-markdown "./_includes/contractlistener_description.md" %} -See below a deep dive into the format of contract listeners and important concepts to understand when managing them. - -### Multiple filters - -From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. - -Before this change each contract listener would only support listening to one specific event from an interface previously defined. A connector specific event signature would be created to easily identify and search for the contact listener for that event. Within that listener, you could optionally specify a location to listen from. Alongside declaring what and where to listen from, you need to specify a `topic` as describe in (TODO LINK) for the ordering constraint of your application. See below for an example with multiple filters. This format is still supported by the API. - -Now from v1.3.1, we have extended the contract listeners API to enable users to supply multiple filters where each contains the event to listen to and the optional location. Each filter will also contain a new connector specific signature which is constructed from the location and the event signature. FireFly will restrict the creation of a contract listener with duplicate filters or supersets. For example, if two filters are listening to the same event but one has specified a location and the other hasn't then the latter will be a superset and already be listening to all the events matching the first filter. - -FireFly made a conscious decision to restricts the creation of duplicate contract listeners and would use the values mentioned earlier to create a unique combination of signature + topic + location. If you tried to create the same listener, you would get a 409. This combination is super useful to a developer to asset that there listener exists. When using muliple filters we can no longer rely on the signature of one event and the location of one filters to calculate uniqueness of a contract listener. As part of v1.3.1, we have come up with a new format where we create a new signature format per filter of location + event signature and then concatinate all of these signatures to build the contract listener signature. This contract listener signature containing all the locations and event signtures combined with the topic will guarantee uniqueness of the contract listener. - -### Signature enhancements - -As mentioned above, we have introduced a new format for signatures of contract listener and filter signature: -- Each filter signature will be a combination of the location and the specific connector event signature -- Each contract listener signature will be a concatination of all the filter signatures - -Furthemore, we noticed that specifically for Ethereum the ABI does not include a differenciation between indexed and non-indexed fields so we included a new section to the Ethereum specific event signature to add the index of the indexed fields on the signature, as such a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma separate list of the index of those indexed fields such as `[i=0,2,3]` specifies that fields at index 0, 2, and 3 are indexed fields. - -### Example with multiple filters - -These examples are using the Ethereum connector. - -Request payload - -```json -{ - "filters": [ - { - "interface": { - "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" - }, - "location": { - "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" - }, - "eventPath": "Changed" - } - ], - "options": { - "firstEvent": "newest" - }, - "topic": "simple-storage" -} -``` - - -Response payload - -Notice that both the -```json -{ - "id": "e7c8457f-4ffd-42eb-ac11-4ad8aed30de1", - "interface": { - "id": "55fdb62a-fefc-4313-99e4-e3f95fcca5f0" - }, - "namespace": "default", - "name": "019104d7-bb0a-c008-76a9-8cb923d91b37", - "backendId": "019104d7-bb0a-c008-76a9-8cb923d91b37", - "location": { - "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" - }, - "created": "2024-07-30T18:12:12.704964Z", - "event": { - "name": "Changed", - "description": "", - "params": [ - { - "name": "from", - "schema": { - "type": "string", - "details": { - "type": "address", - "internalType": "address", - "indexed": true - } - } - }, - { - "name": "value", - "schema": { - "type": "integer", - "details": { - "type": "uint256", - "internalType": "uint256" - } - } - } - ] - }, - "signature": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]", - "topic": "simple-storage", - "options": { - "firstEvent": "newest" - }, - "filters": [ - { - "event": { - "name": "Changed", - "description": "", - "params": [ - { - "name": "from", - "schema": { - "type": "string", - "details": { - "type": "address", - "internalType": "address", - "indexed": true - } - } - }, - { - "name": "value", - "schema": { - "type": "integer", - "details": { - "type": "uint256", - "internalType": "uint256" - } - } - } - ] - }, - "location": { - "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" - }, - "interface": { - "id": "55fdb62a-fefc-4313-99e4-e3f95fcca5f0" - }, - "signature": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]" - } - ] -} -``` - - -### Example with single event (old format) - -Request payload - -```json -{ - "interface": { - "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" - }, - "location": { - "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" - }, - "eventPath": "Changed", - "options": { - "firstEvent": "newest" - }, - "topic": "simple-storage" -} -``` +### Example ```json { @@ -189,11 +34,35 @@ Request payload } ] }, - "signature": "Changed(uint256)", + "signature": "0x596003a91a97757ef1916c8d6c0d42592630d2cf:Changed(uint256)", "topic": "app1_topic", "options": { "firstEvent": "newest" - } + }, + "filters": [ + { + "event": { + "name": "Changed", + "description": "", + "params": [ + { + "name": "x", + "schema": { + "type": "integer", + "details": { + "type": "uint256", + "internalType": "uint256" + } + } + } + ] + }, + "location": { + "address": "0x596003a91a97757ef1916c8d6c0d42592630d2cf" + }, + "signature": "0x596003a91a97757ef1916c8d6c0d42592630d2cf:Changed(uint256)" + } + ] } ``` diff --git a/internal/reference/reference.go b/internal/reference/reference.go index 28fc1346df..699c6aab1b 100644 --- a/internal/reference/reference.go +++ b/internal/reference/reference.go @@ -389,11 +389,36 @@ func GenerateObjectsReferenceMarkdown(ctx context.Context) (map[string][]byte, e }, }, }, - Signature: "Changed(uint256)", + Signature: "0x596003a91a97757ef1916c8d6c0d42592630d2cf:Changed(uint256)", Topic: "app1_topic", Options: &core.ContractListenerOptions{ FirstEvent: "newest", }, + Filters: core.ListenerFilters{ + { + Event: &core.FFISerializedEvent{ + FFIEventDefinition: fftypes.FFIEventDefinition{ + Name: "Changed", + Params: fftypes.FFIParams{ + { + Name: "x", + Schema: fftypes.JSONAnyPtr(`{ + "type": "integer", + "details": { + "type": "uint256", + "internalType": "uint256" + } + }`), + }, + }, + }, + }, + Signature: "0x596003a91a97757ef1916c8d6c0d42592630d2cf:Changed(uint256)", + Location: fftypes.JSONAnyPtr(`{ + "address": "0x596003a91a97757ef1916c8d6c0d42592630d2cf" + }`), + }, + }, }, &core.TokenPool{ From 7f5be6fcd847c4c724cb86c5194a864f76d75681 Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Thu, 1 Aug 2024 17:15:31 +0100 Subject: [PATCH 07/14] update release notes Signed-off-by: Enrique Lacal --- doc-site/docs/releasenotes/index.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc-site/docs/releasenotes/index.md b/doc-site/docs/releasenotes/index.md index 59306c69f2..2cffbe61d1 100644 --- a/doc-site/docs/releasenotes/index.md +++ b/doc-site/docs/releasenotes/index.md @@ -4,11 +4,12 @@ title: Release Notes [Full release notes](https://github.com/hyperledger/firefly/releases) -## [v1.3.1 - July 31, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.3.1) +## [v1.3.1 - Aug 1, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.3.1) What's New: -- Enable contract listeners with multiple filters, old format is persevered for backwards compatibility +- Enable contract listeners with multiple filters + See [Contract Listeners](../reference/types/contractlistener.md) for details - New multiparty status API at `/status/multiparty` ## [v1.3.0 - April 25, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.1.0) From e9fd4bb329074a66e40fcba8f2968747f41fc38d Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Thu, 1 Aug 2024 17:17:55 +0100 Subject: [PATCH 08/14] fix topic explanation Signed-off-by: Enrique Lacal --- .../reference/types/_includes/contractlistener_description.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc-site/docs/reference/types/_includes/contractlistener_description.md b/doc-site/docs/reference/types/_includes/contractlistener_description.md index f4f2e1642e..6264e78bf4 100644 --- a/doc-site/docs/reference/types/_includes/contractlistener_description.md +++ b/doc-site/docs/reference/types/_includes/contractlistener_description.md @@ -12,9 +12,9 @@ See below for a deep dive into the format of contract listeners and important co From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. -Before this change each contract listener would only support listening to one specific event from an interface previously defined. A connector specific event signature would be created to easily identify and search for the contact listener for that event. Within that listener, you could optionally specify a location to listen from. Alongside declaring what and where to listen from, you need to specify a `topic` as describe in (TODO LINK) for the ordering constraint of your application. See below for an example with multiple filters. This format is still supported by the API. +Before this change each contract listener would only support listening to one specific event from an interface previously defined. A connector specific event signature would be created to easily identify and search for the contact listener for that event. Within that listener, you could optionally specify a location to listen from. Alongside declaring what and where to listen from, you need to specify a `topic` which determines the ordered stream there events are part of. This format is still supported by the API. -Now from v1.3.1, we have extended the contract listeners API to enable users to supply multiple filters where each contains the event to listen to and the optional location. Each filter will also contain a new connector specific signature which is constructed from the location and the event signature. FireFly will restrict the creation of a contract listener with duplicate filters or supersets. For example, if two filters are listening to the same event but one has specified a location and the other hasn't then the latter will be a superset and already be listening to all the events matching the first filter. Notice that for backwards compatibility the response from the API will populate the `event` and `location` with the contents of the first event filter in the array. +Now from v1.3.1, we have extended the contract listeners API to enable users to supply multiple filters where each contains the event to listen to and the optional location. This new feature will allow to better manage contract listeners and group togethere events your application cares about. Each filter will also contain a new connector specific signature which is constructed from the location and the event signature. FireFly will restrict the creation of a contract listener with duplicate filters or supersets. For example, if two filters are listening to the same event but one has specified a location and the other hasn't then the latter will be a superset and already be listening to all the events matching the first filter. Notice that for backwards compatibility the response from the API will populate the `event` and `location` with the contents of the first event filter in the array. FireFly made a conscious decision to restricts the creation of duplicate contract listeners and would use the values mentioned earlier to create a unique combination of signature + topic + location. If you tried to create the same listener, you would get a 409. This combination is super useful to a developer to asset that there listener exists. When using muliple filters we can no longer rely on the signature of one event and the location of one filters to calculate uniqueness of a contract listener. As part of v1.3.1, we have come up with a new format where we create a new signature format per filter of location + event signature and then concatinate all of these signatures to build the contract listener signature. This contract listener signature containing all the locations and event signtures combined with the topic will guarantee uniqueness of the contract listener. From d94b7553fb68cc847953127d4444ba45c07e8168 Mon Sep 17 00:00:00 2001 From: Enrique Lacal Date: Thu, 1 Aug 2024 17:21:42 +0100 Subject: [PATCH 09/14] add both examples Signed-off-by: Enrique Lacal --- .../_includes/contractlistener_description.md | 58 ++++++++++++++++++- 1 file changed, 56 insertions(+), 2 deletions(-) diff --git a/doc-site/docs/reference/types/_includes/contractlistener_description.md b/doc-site/docs/reference/types/_includes/contractlistener_description.md index 6264e78bf4..725eb8d059 100644 --- a/doc-site/docs/reference/types/_includes/contractlistener_description.md +++ b/doc-site/docs/reference/types/_includes/contractlistener_description.md @@ -5,7 +5,6 @@ of the interface for that event. Check out the [Custom Contracts Tutorial](../../tutorials/custom_contracts/index.md) for a walk-through of how to set up listeners for the events from your smart contracts. - See below for a deep dive into the format of contract listeners and important concepts to understand when managing them. ### Multiple filters @@ -24,4 +23,59 @@ As mentioned above, we have introduced a new format for signatures of contract l - Each filter signature will be a combination of the location and the specific connector event signature - Each contract listener signature will be a concatination of all the filter signatures -Furthemore, we noticed that specifically for Ethereum the ABI does not include a differenciation between indexed and non-indexed fields so we included a new section to the Ethereum specific event signature to add the index of the indexed fields on the signature, as such a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma separate list of the index of those indexed fields such as `[i=0,2,3]` specifies that fields at index 0, 2, and 3 are indexed fields. \ No newline at end of file +Furthemore, we noticed that specifically for Ethereum the ABI does not include a differenciation between indexed and non-indexed fields so we included a new section to the Ethereum specific event signature to add the index of the indexed fields on the signature, as such a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma separate list of the index of those indexed fields such as `[i=0,2,3]` specifies that fields at index 0, 2, and 3 are indexed fields. + + +### Formats supported + +As described above, there are two formats supported by the API for backwards compatibility with previous releases. + +*** Muliple Filters *** + +```json +{ + "filters": [ + { + "interface": { + "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" + }, + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "eventPath": "Changed" + }, + { + "interface": { + "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" + }, + "location": { + "address": "0xa4ea5d0b6b2eaf194716f0cc73981939dca27da1" + }, + "eventPath": "AnotherEvent" + }, + ], + "options": { + "firstEvent": "newest" + }, + "topic": "simple-storage" +} +``` + +*** One filter with old format *** + + +```json +{ + "interface": { + "id": "8bdd27a5-67c1-4960-8d1e-7aa31b9084d3" + }, + "location": { + "address": "0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1" + }, + "eventPath": "Changed", + "options": { + "firstEvent": "newest" + }, + "topic": "simple-storage" +} +``` \ No newline at end of file From 4981c41642b13ac6ef622f27130a08f703bbeb81 Mon Sep 17 00:00:00 2001 From: Andrew Richardson Date: Thu, 1 Aug 2024 14:58:08 -0400 Subject: [PATCH 10/14] Additional changes to ContractListener documentation Signed-off-by: Andrew Richardson --- .../_includes/contractlistener_description.md | 54 +++++++++++++------ 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/doc-site/docs/reference/types/_includes/contractlistener_description.md b/doc-site/docs/reference/types/_includes/contractlistener_description.md index 725eb8d059..0f5e68072a 100644 --- a/doc-site/docs/reference/types/_includes/contractlistener_description.md +++ b/doc-site/docs/reference/types/_includes/contractlistener_description.md @@ -4,33 +4,56 @@ of the interface for that event. Check out the [Custom Contracts Tutorial](../../tutorials/custom_contracts/index.md) for a walk-through of how to set up listeners for the events from your smart contracts. - -See below for a deep dive into the format of contract listeners and important concepts to understand when managing them. -### Multiple filters +See below for a deep dive into the format of contract listeners and important concepts to understand when managing them. -From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. +### Multiple filters -Before this change each contract listener would only support listening to one specific event from an interface previously defined. A connector specific event signature would be created to easily identify and search for the contact listener for that event. Within that listener, you could optionally specify a location to listen from. Alongside declaring what and where to listen from, you need to specify a `topic` which determines the ordered stream there events are part of. This format is still supported by the API. +From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. -Now from v1.3.1, we have extended the contract listeners API to enable users to supply multiple filters where each contains the event to listen to and the optional location. This new feature will allow to better manage contract listeners and group togethere events your application cares about. Each filter will also contain a new connector specific signature which is constructed from the location and the event signature. FireFly will restrict the creation of a contract listener with duplicate filters or supersets. For example, if two filters are listening to the same event but one has specified a location and the other hasn't then the latter will be a superset and already be listening to all the events matching the first filter. Notice that for backwards compatibility the response from the API will populate the `event` and `location` with the contents of the first event filter in the array. +Before this change, each contract listener would only support listening to one specific event from an interface previously defined. Each listener would be comprised of: -FireFly made a conscious decision to restricts the creation of duplicate contract listeners and would use the values mentioned earlier to create a unique combination of signature + topic + location. If you tried to create the same listener, you would get a 409. This combination is super useful to a developer to asset that there listener exists. When using muliple filters we can no longer rely on the signature of one event and the location of one filters to calculate uniqueness of a contract listener. As part of v1.3.1, we have come up with a new format where we create a new signature format per filter of location + event signature and then concatinate all of these signatures to build the contract listener signature. This contract listener signature containing all the locations and event signtures combined with the topic will guarantee uniqueness of the contract listener. +- a reference to a specific blockchain event to listen for +- (optional) a specific location/address to listen from +- a connector-specific signature (generated from the event), which allows you to easily identify and search for the contact listener for an event +- a `topic` which determines the ordered stream that these events are part of + +This format is still supported by the API. However, it may not fully guarantee accurate ordering of events coming from multiple listeners, even if they share the same `topic` (such as during cases of blockchain catch-up, or when one listener is created much later than the others). + +From v1.3.1, you can supply multiple filters on a single listener. Each filter contains: + +- a reference to a specific blockchain event to listen for +- (optional) a specific location/address to listen from +- a connector-specific signature (generated from the event and the location) + +In addition to this list of multiple filters, the listener specifies a single `topic` to identify the stream of events. This new feature will allow better management of contract listeners and strong ordering of all of the events your application cares about. + +Note: For backwards compatibility, the response from the API will populate top-level `event` and `location` fields with the contents of the first event filter in the array. + +### Duplicate filters + +FireFly will restrict the creation of a contract listener with duplicate filters or superset filters. For example, if two filters are listening to the same event, but one has specified a location and the other hasn't, then the latter will be a superset, and already be listening to all the events matching the first filter. Creation of duplicate or superset filters will be blocked. + +### Duplicate listeners + +As of v1.3.1, each filter on a listener includes a signature generated from the filter location + event, and the listener concatenates all of these signatures to build the overall contract listener signature. This contract listener signature - containing all the locations and event signatures combined with the listener topic - will guarantee uniqueness of the contract listener. If you tried to create the same listener again, you would receive HTTP 409. This combination can allow a developer to assert that their listener exists, without the risk of creating duplicates. + +Note: Prior to v1.3.1, FireFly would detect duplicates simply by requiring a unique combination of signature + topic + location for each listener. When using muliple filters, we cannot rely on the signature of one event and the location of one filter to calculate uniqueness of a contract listener, hence the need for the more sophisticated uniqueness checks described here. ### Signature enhancements As mentioned above, we have introduced a new format for signatures of contract listener and filter signature: -- Each filter signature will be a combination of the location and the specific connector event signature -- Each contract listener signature will be a concatination of all the filter signatures -Furthemore, we noticed that specifically for Ethereum the ABI does not include a differenciation between indexed and non-indexed fields so we included a new section to the Ethereum specific event signature to add the index of the indexed fields on the signature, as such a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma separate list of the index of those indexed fields such as `[i=0,2,3]` specifies that fields at index 0, 2, and 3 are indexed fields. +- Each filter signature will be a combination of the location and the specific connector event signature +- Each contract listener signature will be a concatenation of all the filter signatures +Furthermore, because Ethereum ABI does not include a differentiation between indexed and non-indexed fields, we have included a new section in the Ethereum-specific event signature to add the index of the indexed fields. As such, a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma-separated list of the index of those indexed fields such as `[i=0,2,3]` (specifies that fields at index 0, 2, and 3 are indexed fields). ### Formats supported -As described above, there are two formats supported by the API for backwards compatibility with previous releases. +As described above, there are two input formats supported by the API for backwards compatibility with previous releases. -*** Muliple Filters *** +**Muliple Filters** ```json { @@ -52,7 +75,7 @@ As described above, there are two formats supported by the API for backwards com "address": "0xa4ea5d0b6b2eaf194716f0cc73981939dca27da1" }, "eventPath": "AnotherEvent" - }, + } ], "options": { "firstEvent": "newest" @@ -61,8 +84,7 @@ As described above, there are two formats supported by the API for backwards com } ``` -*** One filter with old format *** - +**One filter (old format)** ```json { @@ -78,4 +100,4 @@ As described above, there are two formats supported by the API for backwards com }, "topic": "simple-storage" } -``` \ No newline at end of file +``` From 318849c6e2dd8f18af1b994b74d9152d09cd3882 Mon Sep 17 00:00:00 2001 From: Andrew Richardson Date: Mon, 5 Aug 2024 10:13:04 -0400 Subject: [PATCH 11/14] Update release date for v1.3.1 Signed-off-by: Andrew Richardson --- doc-site/docs/releasenotes/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc-site/docs/releasenotes/index.md b/doc-site/docs/releasenotes/index.md index 2cffbe61d1..40b349b7a2 100644 --- a/doc-site/docs/releasenotes/index.md +++ b/doc-site/docs/releasenotes/index.md @@ -4,7 +4,7 @@ title: Release Notes [Full release notes](https://github.com/hyperledger/firefly/releases) -## [v1.3.1 - Aug 1, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.3.1) +## [v1.3.1 - Aug 5, 2024](https://github.com/hyperledger/firefly/releases/tag/v1.3.1) What's New: From cb73c3ec9103d12990bc4a5b0628e0c60b60a2dd Mon Sep 17 00:00:00 2001 From: Andrew Richardson Date: Mon, 5 Aug 2024 15:56:28 -0400 Subject: [PATCH 12/14] Additional info and clarification in listener docs Signed-off-by: Andrew Richardson --- .../_includes/contractlistener_description.md | 70 ++++++++++++++----- 1 file changed, 51 insertions(+), 19 deletions(-) diff --git a/doc-site/docs/reference/types/_includes/contractlistener_description.md b/doc-site/docs/reference/types/_includes/contractlistener_description.md index 0f5e68072a..d1a459a89b 100644 --- a/doc-site/docs/reference/types/_includes/contractlistener_description.md +++ b/doc-site/docs/reference/types/_includes/contractlistener_description.md @@ -9,7 +9,13 @@ See below for a deep dive into the format of contract listeners and important co ### Multiple filters -From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. +From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. Each filter contains: + +- a reference to a specific blockchain event to listen for +- (optional) a specific location/address to listen from +- a connector-specific signature (generated from the event and the location) + +In addition to this list of multiple filters, the listener specifies a single `topic` to identify the stream of events. This new feature will allow better management of contract listeners, and strong ordering of events matching multiple filters. Before this change, each contract listener would only support listening to one specific event from an interface previously defined. Each listener would be comprised of: @@ -18,40 +24,66 @@ Before this change, each contract listener would only support listening to one s - a connector-specific signature (generated from the event), which allows you to easily identify and search for the contact listener for an event - a `topic` which determines the ordered stream that these events are part of -This format is still supported by the API. However, it may not fully guarantee accurate ordering of events coming from multiple listeners, even if they share the same `topic` (such as during cases of blockchain catch-up, or when one listener is created much later than the others). +For backwards compatibility, this format is still supported by the API. -From v1.3.1, you can supply multiple filters on a single listener. Each filter contains: +### Signature strings -- a reference to a specific blockchain event to listen for -- (optional) a specific location/address to listen from -- a connector-specific signature (generated from the event and the location) +Each filter is identified by a generated `signature` that matches a single event, and each contract listener is identified by a `signature` computed from its filters. + +Ethereum provides a string standard for event signatures, of the form `EventName(uint256,bytes)`. Prior to v1.3.1, the signature of each Ethereum contract listener would exactly follow this Ethereum format. + +As of v1.3.1, Ethereum signature strings have been changed, because this format does not fully describe the event - particularly because each top-level parameter can in the ABI definition be marked as `indexed`. For example, while the following two Solidity events have the same signature, they are serialized differently due to the different placement of `indexed` parameters, and thus a listener must define both individually to be able to process them: -In addition to this list of multiple filters, the listener specifies a single `topic` to identify the stream of events. This new feature will allow better management of contract listeners and strong ordering of all of the events your application cares about. +- ERC-20 `Transfer` -Note: For backwards compatibility, the response from the API will populate top-level `event` and `location` fields with the contents of the first event filter in the array. + ```solidity + event Transfer(address indexed _from, address indexed _to, uint256 _value) + ``` + +- ERC-721 `Transfer` + + ```solidity + event Transfer(address indexed _from, address indexed _to, uint256 indexed _tokenId); + ``` + +The two above will now be expressed in the following manner by the Ethereum blockchain connector: + +```solidity +Transfer(address,address,uint256) [i=0,1] +Transfer(address,address,uint256) [i=0,1,2] +``` + +The `[i=]` listing at the end of the signature indicates the position of all parameters that are marked as `indexed`. + +Building on the blockchain-specific signature format for each event, FireFly will then compute the final signature for each filter and each contract listener as follows: + +- Each filter signature will be a combination of the location and the specific connector event signature, such as `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` +- Each contract listener signature will be a concatenation of all the filter signatures, separated by `;` ### Duplicate filters -FireFly will restrict the creation of a contract listener with duplicate filters or superset filters. For example, if two filters are listening to the same event, but one has specified a location and the other hasn't, then the latter will be a superset, and already be listening to all the events matching the first filter. Creation of duplicate or superset filters will be blocked. +FireFly will restrict the creation of a contract listener containing duplicate filters or superset filters. For example, if two filters are listening to the same event, but one has specified a location and the other hasn't, then the latter will be a superset, and already be listening to all the events matching the first filter. Creation of duplicate or superset filters within a single listener will be blocked. ### Duplicate listeners -As of v1.3.1, each filter on a listener includes a signature generated from the filter location + event, and the listener concatenates all of these signatures to build the overall contract listener signature. This contract listener signature - containing all the locations and event signatures combined with the listener topic - will guarantee uniqueness of the contract listener. If you tried to create the same listener again, you would receive HTTP 409. This combination can allow a developer to assert that their listener exists, without the risk of creating duplicates. - -Note: Prior to v1.3.1, FireFly would detect duplicates simply by requiring a unique combination of signature + topic + location for each listener. When using muliple filters, we cannot rely on the signature of one event and the location of one filter to calculate uniqueness of a contract listener, hence the need for the more sophisticated uniqueness checks described here. +As noted above, each listener has a generated signature. This signature - containing all the locations and event signatures combined with the listener topic - will guarantee uniqueness of the contract listener. If you tried to create the same listener again, you would receive HTTP 409. This combination can allow a developer to assert that their listener exists, without the risk of creating duplicates. -### Signature enhancements +**Note:** Prior to v1.3.1, FireFly would detect duplicates simply by requiring a unique combination of signature + topic + location for each listener. The updated behavior for the listener signature is intended to preserve similar functionality, even when dealing with listeners that contain many event filters. -As mentioned above, we have introduced a new format for signatures of contract listener and filter signature: +### Backwards compatibility -- Each filter signature will be a combination of the location and the specific connector event signature -- Each contract listener signature will be a concatenation of all the filter signatures +As noted throughout this document, the behavior of listeners is changed in v1.3.1. However, the following behaviors are retained for backwards-compatibility, to ensure that code written prior to v1.3.1 should continue to function. -Furthermore, because Ethereum ABI does not include a differentiation between indexed and non-indexed fields, we have included a new section in the Ethereum-specific event signature to add the index of the indexed fields. As such, a filter listening to the event `Changed(address indexed from, uint256 value)` at address `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1` will result in `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` where `[i=0]` specifies that the first field is indexed. If there were more indexed fields, it will be a comma-separated list of the index of those indexed fields such as `[i=0,2,3]` (specifies that fields at index 0, 2, and 3 are indexed fields). +- The response from all query APIs of `listeners` will continue to populate top-level `event` and `location` fields + - The first entry from the `filters` array is duplicated to these fields +- On input to create a new `listener`, the `event` and `location` fields are still supported + - They function identically to supplying a `filters` array with a single entry +- The `signature` field is preserved at the listener level + - The format has been changed as described above -### Formats supported +### Input formats -As described above, there are two input formats supported by the API for backwards compatibility with previous releases. +The two input formats supported when creating a contract listener are shown below. **Muliple Filters** From 8a38fce4a9bb90c11ccdb7ab1ea0026e38a0c21c Mon Sep 17 00:00:00 2001 From: Andrew Richardson Date: Mon, 5 Aug 2024 16:07:41 -0400 Subject: [PATCH 13/14] Subdivide listener docs in a few more places Signed-off-by: Andrew Richardson --- .../_includes/contractlistener_description.md | 26 ++++++++++++------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/doc-site/docs/reference/types/_includes/contractlistener_description.md b/doc-site/docs/reference/types/_includes/contractlistener_description.md index d1a459a89b..b4dc562eb5 100644 --- a/doc-site/docs/reference/types/_includes/contractlistener_description.md +++ b/doc-site/docs/reference/types/_includes/contractlistener_description.md @@ -7,17 +7,23 @@ a walk-through of how to set up listeners for the events from your smart contrac See below for a deep dive into the format of contract listeners and important concepts to understand when managing them. -### Multiple filters +### Event filters -From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic when supported by the connector. Each filter contains: +#### Multiple filters + +From v1.3.1 onwards, a contract listener can be created with multiple filters under a single topic, when supported by the connector. Each filter contains: - a reference to a specific blockchain event to listen for - (optional) a specific location/address to listen from - a connector-specific signature (generated from the event and the location) -In addition to this list of multiple filters, the listener specifies a single `topic` to identify the stream of events. This new feature will allow better management of contract listeners, and strong ordering of events matching multiple filters. +In addition to this list of multiple filters, the listener specifies a single `topic` to identify the stream of events. + +Creating a single listener that listens for multiple events will allow for the easiest management of listeners, and for strong ordering of the events that they process. -Before this change, each contract listener would only support listening to one specific event from an interface previously defined. Each listener would be comprised of: +#### Single filter + +Before v1.3.1, each contract listener would only support listening to one specific event from a contract interface. Each listener would be comprised of: - a reference to a specific blockchain event to listen for - (optional) a specific location/address to listen from @@ -28,6 +34,8 @@ For backwards compatibility, this format is still supported by the API. ### Signature strings +#### String format + Each filter is identified by a generated `signature` that matches a single event, and each contract listener is identified by a `signature` computed from its filters. Ethereum provides a string standard for event signatures, of the form `EventName(uint256,bytes)`. Prior to v1.3.1, the signature of each Ethereum contract listener would exactly follow this Ethereum format. @@ -46,7 +54,7 @@ As of v1.3.1, Ethereum signature strings have been changed, because this format event Transfer(address indexed _from, address indexed _to, uint256 indexed _tokenId); ``` -The two above will now be expressed in the following manner by the Ethereum blockchain connector: +The two above are now expressed in the following manner by the Ethereum blockchain connector: ```solidity Transfer(address,address,uint256) [i=0,1] @@ -57,14 +65,14 @@ The `[i=]` listing at the end of the signature indicates the position of all par Building on the blockchain-specific signature format for each event, FireFly will then compute the final signature for each filter and each contract listener as follows: -- Each filter signature will be a combination of the location and the specific connector event signature, such as `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` -- Each contract listener signature will be a concatenation of all the filter signatures, separated by `;` +- Each filter signature is a combination of the location and the specific connector event signature, such as `0xa5ea5d0a6b2eaf194716f0cc73981939dca26da1:Changed(address,uint256) [i=0]` +- Each contract listener signature is a concatenation of all the filter signatures, separated by `;` -### Duplicate filters +#### Duplicate filters FireFly will restrict the creation of a contract listener containing duplicate filters or superset filters. For example, if two filters are listening to the same event, but one has specified a location and the other hasn't, then the latter will be a superset, and already be listening to all the events matching the first filter. Creation of duplicate or superset filters within a single listener will be blocked. -### Duplicate listeners +#### Duplicate listeners As noted above, each listener has a generated signature. This signature - containing all the locations and event signatures combined with the listener topic - will guarantee uniqueness of the contract listener. If you tried to create the same listener again, you would receive HTTP 409. This combination can allow a developer to assert that their listener exists, without the risk of creating duplicates. From 5504c0164b4a7df4e1ffeb1f7613ee24302866cc Mon Sep 17 00:00:00 2001 From: Andrew Richardson Date: Mon, 5 Aug 2024 16:09:00 -0400 Subject: [PATCH 14/14] Clarification on superset filters Signed-off-by: Andrew Richardson --- .../types/_includes/contractlistener_description.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc-site/docs/reference/types/_includes/contractlistener_description.md b/doc-site/docs/reference/types/_includes/contractlistener_description.md index b4dc562eb5..5027b71ade 100644 --- a/doc-site/docs/reference/types/_includes/contractlistener_description.md +++ b/doc-site/docs/reference/types/_includes/contractlistener_description.md @@ -70,7 +70,11 @@ Building on the blockchain-specific signature format for each event, FireFly wil #### Duplicate filters -FireFly will restrict the creation of a contract listener containing duplicate filters or superset filters. For example, if two filters are listening to the same event, but one has specified a location and the other hasn't, then the latter will be a superset, and already be listening to all the events matching the first filter. Creation of duplicate or superset filters within a single listener will be blocked. +FireFly restricts the creation of a contract listener containing duplicate filters. + +This includes the special case where one filter is a superset of another filter, due to a wildcard location. + +For example, if two filters are listening to the same event, but one has specified a location and the other hasn't, then the latter will be a superset, and already be listening to all the events matching the first filter. Creation of duplicate or superset filters within a single listener will be blocked. #### Duplicate listeners