Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support experiment enablement in Taskfile schema #1978

Open
nathanperkins opened this issue Dec 30, 2024 · 5 comments · May be fixed by #1982
Open

Support experiment enablement in Taskfile schema #1978

nathanperkins opened this issue Dec 30, 2024 · 5 comments · May be fixed by #1982

Comments

@nathanperkins
Copy link

nathanperkins commented Dec 30, 2024

Description

Experiments like #1038 cause major changes to the behavior for a task. Given that the taskfile would have to be written to make use of this experiment, it is dangerous that the experiment has to be enabled using env vars for every engineer and every automation flow which runs that task. If the task were accidentally run without the experiment, it could lead to undesired behavior, like deploying remote environments with settings based on a developer's local environment.

The risk of dangerous, undesired behavior makes it practically impossible to use the experiment, so we have to continue using a workaround.

Ideally, we would be able to enable this experiment within the taskfile itself so that we have consistent behavior across our team and automation without any risk.

@task-bot task-bot added the state: needs triage Waiting to be triaged by a maintainer. label Dec 30, 2024
@pd93
Copy link
Member

pd93 commented Dec 30, 2024

(Was just about to reply in #1038), but may as well have this discussion here now.

This is why we have option 3 in our docs for enabling experiments. This allows you to commit the .env file which enables it for anyone checking out your repo. I do understand that a .env file isn't always ideal though.

My main concern with adding them to the Taskfile schema itself is that it would (ironically) be a backwards-compatible change to remove the experiments later which we need to be able to do.

Maybe a compromise could be to add support for a dedicated .task-experiments.yaml file or something similar.

@nathanperkins
Copy link
Author

This is why we have option 3 in our docs for enabling experiments. This allows you to commit the .env file which enables it for anyone checking out your repo. I do understand that a .env file isn't always ideal though.

Ah, I see. I thought the .env file would only be used if we sourced it ourselves or if we included it in dotenv but I didn't want to replace any other vars.

Based on my experimentation, it seems like the experiment vars are loaded from .env regardless of whether you put it in dotenv, while other variables are ignored unless you have it in dotenv. It might be worth clarifying this further in the docs.

If so, that's good enough for my team.

My main concern with adding them to the Taskfile schema itself is that it would (ironically) be a backwards-compatible change to remove the experiments later which we need to be able to do.

That makes sense but also maybe this kind of change should break visibly. I'd rather task exit with an error than for it to do something unexpected.

Maybe a compromise could be to add support for a dedicated .task-experiments.yaml file or something similar.

That sounds good, although I'm fine with the current .env solution as I understand it.

@nathanperkins
Copy link
Author

Possibly related: #1979

@gedw99
Copy link

gedw99 commented Dec 31, 2024

I like the .task-experiments.yaml idea. It almost like task feature flags.

separating it from the .env makes a lot of sense.

@vmaerten
Copy link
Member

My main concern with adding them to the Taskfile schema itself is that it would (ironically) be a backwards-compatible change to remove the experiments later which we need to be able to do.

Adding that handling remote experiment directly in the Taskfile schema could be tricky.

I wonder if it would be better to have a more global YAML configuration file, like .task-config.yml or .task-settings.yaml. This could help address issues like this one #1589
Any thoughts @pd93 / @andreynering ?

@vmaerten vmaerten removed the state: needs triage Waiting to be triaged by a maintainer. label Dec 31, 2024
@vmaerten vmaerten linked a pull request Dec 31, 2024 that will close this issue
@vmaerten vmaerten linked a pull request Dec 31, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants