Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

imp: adding 08-wasm build opts with libwasmvm linking disabled #5923

Merged
merged 25 commits into from
Apr 2, 2024

Conversation

damiannolan
Copy link
Contributor

@damiannolan damiannolan commented Mar 4, 2024

Description

Tested adding custom build directive nolink_libwasmvm in rollkit app binary compilation.
See: chatton/gm-demo#2 (with successful build discarding libwasmvm dependencies - NOTE: uses fork of wasmvm with build directives)

panic: not implemented, please build with cgo enabled or nolink_libwasmvm disabled

goroutine 1 [running]:
github.com/cosmos/ibc-go/modules/light-clients/08-wasm/keeper.NewKeeperWithVM(...)
        github.com/cosmos/ibc-go/modules/light-clients/[email protected]/keeper/keeper_no_vm.go:27
gm/app.RegisterIBC({0x10f524678, 0xc0004a2000})
        gm/app/ibc.go:222 +0x50b
gm/cmd/gmd/cmd.NewRootCmd()
        gm/cmd/gmd/cmd/root.go:110 +0x3d7
main.main()
        gm/cmd/gmd/main.go:14 +0x17

When the application binary is compiled removing the calls to wasmkeeper.NewKeeperWithVM and wasmkeeper.NewKeeperWithConfig the docker image can be run and the entrypoint executes successfully returning the simd help text.

closes: #5903


Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Targeted PR against the correct branch (see CONTRIBUTING.md).
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Code follows the module structure standards and Go style guide.
  • Wrote unit and integration tests.
  • Updated relevant documentation (docs/) or specification (x/<module>/spec/).
  • Added relevant godoc comments.
  • Provide a commit message to be used for the changelog entry in the PR description for review.
  • Re-reviewed Files changed in the Github PR explorer.
  • Review Codecov Report in the comment section below once CI passes.

Summary by CodeRabbit

  • New Features

    • Introduced a Docker build target for WebAssembly simulation applications.
    • Enhanced the Wasm module by integrating WebAssembly VM functionality for better interface and management within Cosmos SDK applications.
    • Added documentation on configuring the Wasm module for different operational modes, including options to disable CGO or selectively link with libwasmvm.
  • Refactor

    • Streamlined Keeper initialization in the Wasm module by consolidating logic into a single struct initialization approach.
    • Removed redundant interface declaration in the Wasm engine code to simplify interface definitions.
  • Chores

    • Updated the Makefile to include a new build target and improved the help target output.

@chatton
Copy link
Contributor

chatton commented Mar 5, 2024

One thing that would be nice to test before merging this is maybe we can try building the gm-demo importing this commit?

@DimitrisJim
Copy link
Contributor

One thing that would be nice to test before merging this is maybe we can try building the gm-demo importing this commit?

which depends on some types in 08-wasm, right? We could also check binary size for sanity sake

Copy link
Contributor

coderabbitai bot commented Mar 6, 2024

Walkthrough

The changes involve enhancing the 08-wasm module by adding new functionalities and modifying existing ones to better manage Wasm modules in a Cosmos SDK application. Key updates include the introduction of a new Makefile target for building wasm simapp Docker images, changes in the handling of Wasm engines, and the addition of scripts and configurations to support different build environments. These updates aim to improve the flexibility and usability of the 08-wasm module, particularly in scenarios involving WebAssembly virtual machines and Docker.

Changes

File Path Change Summary
Makefile Added build-docker-wasm target; Updated help target.
modules/.../internal/ibcwasm/expected_interfaces.go Removed _ WasmEngine = (*wasmvm.VM)(nil) affecting WasmEngine interface.
modules/.../internal/ibcwasm/wasm_cgo.go Introduced interfacing with WebAssembly VM using CGO.
modules/.../internal/ibcwasm/wasm_vm.go Added, defining _ of type WasmEngine pointing to wasmvm.VM.
modules/.../keeper/keeper.go Unified Keeper initialization; Removed NewKeeperWithVM and NewKeeperWithConfig.
scripts/build-wasm-simapp-docker.sh Introduced script for building Docker image for specific libwasm version.
modules/.../doc.go Added info on configuring 08-wasm module to use types without cgo or libwasmvm dependencies.
modules/.../keeper/keeper_no_vm.go Introduced for creating Keeper instance with different Wasm configurations.

Assessment against linked issues

Objective Addressed Explanation
Build 08-wasm with libwasmvm linking disabled (#5903)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 6

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 511236b and 6f9f2a7.
Files selected for processing (7)
  • Makefile (2 hunks)
  • modules/light-clients/08-wasm/internal/ibcwasm/expected_interfaces.go (1 hunks)
  • modules/light-clients/08-wasm/internal/ibcwasm/wasm_cgo.go (1 hunks)
  • modules/light-clients/08-wasm/keeper/keeper.go (2 hunks)
  • modules/light-clients/08-wasm/keeper/keeper_cgo.go (1 hunks)
  • modules/light-clients/08-wasm/keeper/keeper_no_cgo.go (1 hunks)
  • scripts/build-wasm-simapp-docker.sh (1 hunks)
Additional comments: 11
modules/light-clients/08-wasm/internal/ibcwasm/wasm_cgo.go (3)
  • 1-1: The build tag //go:build cgo is correctly used to conditionally compile this file when CGO is enabled. This is a standard practice for Go projects that have CGO-specific code.
  • 5-5: The import of wasmvm from github.com/CosmWasm/wasmvm is appropriate for working with WebAssembly virtual machines. Ensure that the version of wasmvm used is compatible with the rest of the project dependencies.
  • 7-7: The declaration var _ WasmEngine = (*wasmvm.VM)(nil) is a common Go idiom to ensure at compile time that wasmvm.VM implements the WasmEngine interface. This is a good practice to catch any interface implementation issues early.
scripts/build-wasm-simapp-docker.sh (3)
  • 3-3: The use of set -eou pipefail is a best practice in bash scripting as it causes the script to exit on error, treats unset variables as an error, and causes pipelines to fail on the first error. This makes the script more robust and easier to debug.
  • 5-10: The build_wasm_image function correctly extracts the version and checksum for libwasm using a Python script and then uses these values to build a Docker image. Ensure that the Python script get-libwasm-version.py is present in the scripts directory and functions correctly.
  • 13-16: Setting a default tag for the Docker image if none is provided (TAG="${1:-ibc-go-wasm-simd:latest}") is a user-friendly feature. It ensures that the script can be run without specifying a tag, using ibc-go-wasm-simd:latest as the default. This is a good practice for scripts intended to be used in various environments.
modules/light-clients/08-wasm/keeper/keeper_no_cgo.go (1)
  • 1-1: The build tag //go:build !cgo is correctly used to conditionally compile this file when CGO is disabled. This is a standard practice for Go projects that need to differentiate behavior based on the availability of CGO.
modules/light-clients/08-wasm/keeper/keeper_cgo.go (1)
  • 1-1: The build tag //go:build cgo is correctly used to conditionally compile this file when CGO is enabled. This ensures that the file's contents are only included in builds where CGO is available, aligning with the project's requirements.
modules/light-clients/08-wasm/keeper/keeper.go (2)
  • 3-8: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [1-1]

The file keeper.go correctly defines the Keeper struct and its methods, which are crucial for the management of wasm code storage, migration, and client state within the 08-wasm module. This centralization of functionality within the Keeper struct is a good practice for maintainability and clarity of the module's operations.

  • 3-8: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [1-1]

Given the AI-generated summary's mention of the removal of NewKeeperWithVM and NewKeeperWithConfig functions, it's important to ensure that the consolidated logic in the Keeper struct initialization does not negatively impact the module's functionality or introduce any regressions. This would require thorough testing, especially in scenarios where these functions were previously used.

modules/light-clients/08-wasm/internal/ibcwasm/expected_interfaces.go (1)
  • 8-13: > 📝 NOTE

This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [1-1]

The file expected_interfaces.go correctly defines the WasmEngine, QueryRouter, and QueryPluginsI interfaces, which are crucial for the operation of the 08-wasm module. These interfaces abstract the interaction with wasm code and provide a framework for query functionality, contributing to the modularity and flexibility of the module.

Makefile Show resolved Hide resolved
modules/light-clients/08-wasm/keeper/keeper_no_cgo.go Outdated Show resolved Hide resolved
modules/light-clients/08-wasm/keeper/keeper_no_cgo.go Outdated Show resolved Hide resolved
modules/light-clients/08-wasm/keeper/keeper_cgo.go Outdated Show resolved Hide resolved
modules/light-clients/08-wasm/keeper/keeper_cgo.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yml

Commits Files that changed from the base of the PR and between 6f9f2a7 and cbf297f.
Files selected for processing (1)
  • Makefile (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • Makefile

@damiannolan
Copy link
Contributor Author

Let's change this to not use the cgo build tags and instead use a separately defined one. Similarly to how we use build tags in e2e.

@charleenfei discovered when building binaries that require some cgo dependencies for keys etc that some code is not available, and re-enabling the cgo build tag would then require the wasmvm dlls to be available. I think we can use a separate tag here to instruct the compiler to use the files when buillding - making it possible to have CGO_ENABLED=1 but 08-wasm using types only.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yml

Commits Files that changed from the base of the PR and between ee4ac4d and 003fa1b.
Files selected for processing (1)
  • modules/light-clients/08-wasm/doc.go (1 hunks)
Files skipped from review as they are similar to previous changes (1)
  • modules/light-clients/08-wasm/doc.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@damiannolan
Copy link
Contributor Author

Happy to consider this good to go if there is agreement to go ahead with CosmWasm/wasmvm#527

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 1

Configuration used: .coderabbit.yml

Commits Files that changed from the base of the PR and between 003fa1b and 22dab08.
Files selected for processing (1)
  • modules/light-clients/08-wasm/keeper/keeper_vm.go (1 hunks)
Additional comments: 2
modules/light-clients/08-wasm/keeper/keeper_vm.go (2)
  • 18-64: The implementation of NewKeeperWithVM correctly checks for nil arguments and panics with appropriate messages. This aligns with the PR objectives and ensures early failure with clear error messaging when the function is used without the necessary wasmvm dependencies. Good practice!
  • 67-84: The implementation of NewKeeperWithConfig effectively handles the instantiation of a new Wasm VM instance and delegates to NewKeeperWithVM for keeper initialization. This approach is consistent with the PR objectives and ensures clear error messaging and early failure when necessary. Well done!

modules/light-clients/08-wasm/keeper/keeper_vm.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: .coderabbit.yml

Commits Files that changed from the base of the PR and between 22dab08 and fffd115.
Files selected for processing (1)
  • modules/light-clients/08-wasm/keeper/keeper_vm.go (1 hunks)
Files skipped from review as they are similar to previous changes (1)
  • modules/light-clients/08-wasm/keeper/keeper_vm.go

Copy link
Contributor

@DimitrisJim DimitrisJim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this definitely sounded like it was a pain! glad to go with this if wasmvm PR lands.

@@ -4,5 +4,12 @@ ClientMessage and types for the proxy light client module communicating
with underlying Wasm light clients.
This implementation is based off the ICS 08 specification
(https://github.com/cosmos/ibc/blob/main/spec/client/ics-008-wasm-client)

By default the 08-wasm module requires cgo and libwasmvm dependencies available on the system.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

➕ 🙏

@@ -0,0 +1,44 @@
//go:build !cgo || nolink_libwasmvm
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can rename these files to cgo & no_cgo again?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think no_vm is more representative of the purpose of these build tags

@crodriguezvega crodriguezvega added the priority PRs that need prompt reviews label Mar 25, 2024
Copy link

sonarqubecloud bot commented Apr 2, 2024

Quality Gate Passed Quality Gate passed for 'ibc-go'

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

Copy link
Contributor

@chatton chatton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@damiannolan
Copy link
Contributor Author

Will require bump of next wasmvm release. Merging! 🚀

@damiannolan damiannolan merged commit 031c2b8 into main Apr 2, 2024
84 checks passed
@damiannolan damiannolan deleted the damian/5903-wasm-build-no-cgo branch April 2, 2024 14:59
mergify bot pushed a commit that referenced this pull request Jun 10, 2024
* wip: messing with build options for wasm cgo

* chore: mv type assertion to wasm_cgo with build flags

* chore: mv make target to build section

* chore: revert cgo enabled 0 build opt for testing

* chore: rm unneeded file

* update build tag

* linter

* update panic message

* add Codec back

* refactor: adapt build tags to match wasmvm

* Update modules/light-clients/08-wasm/doc.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update modules/light-clients/08-wasm/keeper/keeper_vm.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* chore: make lint-fix

* chore: make lint-fix

---------

Co-authored-by: Charly <[email protected]>
Co-authored-by: Charly <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Cian Hatton <[email protected]>
(cherry picked from commit 031c2b8)

# Conflicts:
#	Makefile
#	modules/light-clients/08-wasm/keeper/keeper.go
damiannolan added a commit that referenced this pull request Jun 12, 2024
…ort #5923) (#6548)

* imp: adding 08-wasm build opts with libwasmvm linking disabled (#5923)

* wip: messing with build options for wasm cgo

* chore: mv type assertion to wasm_cgo with build flags

* chore: mv make target to build section

* chore: revert cgo enabled 0 build opt for testing

* chore: rm unneeded file

* update build tag

* linter

* update panic message

* add Codec back

* refactor: adapt build tags to match wasmvm

* Update modules/light-clients/08-wasm/doc.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update modules/light-clients/08-wasm/keeper/keeper_vm.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* chore: make lint-fix

* chore: make lint-fix

---------

Co-authored-by: Charly <[email protected]>
Co-authored-by: Charly <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Cian Hatton <[email protected]>
(cherry picked from commit 031c2b8)

# Conflicts:
#	Makefile
#	modules/light-clients/08-wasm/keeper/keeper.go

* chore: resolve merge conflicts from backport

* chore: update changelog

---------

Co-authored-by: Damian Nolan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build 08-wasm with libwasmvm linking disabled
5 participants