-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Executor configuration extended .. #1099
Conversation
will be finalised once #1096 gets merged |
83fc925
to
647f426
Compare
647f426
to
82ad6a4
Compare
... this way we can configure way more options
cb82c7b
to
5d990a7
Compare
5d990a7
to
9f8bc2f
Compare
please have a look when you get chance @andygrove |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks @milenkovicm
Thanks @andygrove, I need another pass to clean it up and put documention. Will let you know when ready |
so executors can configure runtime per task, and session config they have
8194b5a
to
c83a158
Compare
... process (logical, and physical codec)
i believe this is ready now @andygrove |
in default configuration as ballista disables them
... this provide much more options to configure executors.
Which issue does this PR close?
Closes #1098.
Rationale for this change
expose more configuration options for executor and scheduler, this way users can set up their own instances with custom configuration (object stores, configuration, udfs ...)
What changes are included in this PR?
testcontainers
optionBALLISTA_PLANNER_OVERRIDE
config was removed, there is equivalent method inSessionConfigExt
BallistaFunctionRegistry
was promoted a bit, cleaning up executor codeto follow up
Are there any user-facing changes?
there are slight changes in original API, but not drastic