Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add [lambdex].layout to opt-in to soon-to-be-deprecated FaaS behaviour #19072

Conversation

huonw
Copy link
Contributor

@huonw huonw commented May 21, 2023

This adds a layout options to the [lambdex] scope, so that current users can explicitly opt-in to the current behaviour for building FaaS artifacts (python_awslambda, python_google_cloud_function), to allow us to change the default behaviour without breaking them.

The current behaviour uses Lambdex to adjust a PEX to be runnable in the FaaS environments, but it's still a PEX under the hood, with dynamic dependency selection on start-up. The new behaviour (#18879, first attempted in #19022) will be to lay out a "normal" zip with the dependencies statically included at the top level, since the serverless environments are fixed and known at build-time. We plan to switch the default to the new behaviour in 2.18, and remove the lambdex option in 2.19.

The suggestion is to cherry-pick this back to 2.17.x, to allow us to deprecate and remove Lambdex sooner. See #19032 (comment) for discussion.

(This PR is currently stacked on top of #19071.)

@huonw
Copy link
Contributor Author

huonw commented May 21, 2023

I've tested this manually, and it's noisy, but that's probably okay (these targets are already very noisy on macOS), e.g. building three lambdas:

$ PANTS_SOURCE=~/projects/pantsbuild/pants pants package ...

...

10:09:32.11 [WARN] `python_awslambda` targets built on macOS may fail to build. If your function uses any third-party dependencies without binary wheels (bdist) for Linux available, it will fail to build. If this happens, you will either need to update your dependencies to only use dependencies with pre-built wheels, or find a Linux environment to run pants package. (See https://realpython.com/python-wheels/ for more about wheels.)

(If the build does not raise an exception, it's safe to use macOS.)
10:09:32.11 [WARN] `python_awslambda` targets built on macOS may fail to build. If your function uses any third-party dependencies without binary wheels (bdist) for Linux available, it will fail to build. If this happens, you will either need to update your dependencies to only use dependencies with pre-built wheels, or find a Linux environment to run pants package. (See https://realpython.com/python-wheels/ for more about wheels.)

(If the build does not raise an exception, it's safe to use macOS.)
10:09:32.11 [WARN] `python_awslambda` targets built on macOS may fail to build. If your function uses any third-party dependencies without binary wheels (bdist) for Linux available, it will fail to build. If this happens, you will either need to update your dependencies to only use dependencies with pre-built wheels, or find a Linux environment to run pants package. (See https://realpython.com/python-wheels/ for more about wheels.)

(If the build does not raise an exception, it's safe to use macOS.)
10:09:32.11 [WARN] DEPRECATED: using the Lambdex layout by default for `python_awslambda` targets is scheduled to be removed in version 2.18.0.dev0.

To prepare for the transition to the new layout for `python_awslambda` targets in Pants 2.18, set the `[lambdex].layout` option to `lambdex` explicitly, in `pants.toml`:

    [lambdex]
    layout = "lambdex"
10:09:32.11 [WARN] DEPRECATED: using the Lambdex layout by default for `python_awslambda` targets is scheduled to be removed in version 2.18.0.dev0.

To prepare for the transition to the new layout for `python_awslambda` targets in Pants 2.18, set the `[lambdex].layout` option to `lambdex` explicitly, in `pants.toml`:

    [lambdex]
    layout = "lambdex"
10:09:32.11 [WARN] DEPRECATED: using the Lambdex layout by default for `python_awslambda` targets is scheduled to be removed in version 2.18.0.dev0.

To prepare for the transition to the new layout for `python_awslambda` targets in Pants 2.18, set the `[lambdex].layout` option to `lambdex` explicitly, in `pants.toml`:

    [lambdex]
    layout = "lambdex"

...

and setting the option silences the layout warnings. I don't know if this is the best way to communicate this deprecation.

@huonw huonw marked this pull request as ready for review May 21, 2023 00:20
@huonw huonw requested a review from benjyw May 21, 2023 00:22
@huonw huonw deleted the branch pantsbuild:revert-19022-feature/18879-lambda-layout May 21, 2023 02:12
@huonw huonw closed this May 21, 2023
@huonw
Copy link
Contributor Author

huonw commented May 21, 2023

Oops the stacking on #19071 meant this was closed when I deleted that PR's branch. Reopened in #19074.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants