Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify job manager: add_backend without name #663

Open
soxofaan opened this issue Nov 18, 2024 · 6 comments
Open

simplify job manager: add_backend without name #663

soxofaan opened this issue Nov 18, 2024 · 6 comments

Comments

@soxofaan
Copy link
Member

To add a backend to MultiBackendJobManager, one has to give it a name, e.g.

job_manager.add_backend("foo", connection)

But that name has very little purpose. Can't we drop the requirement to provide a name, and just fall back on the connection url?

@HansVRP
Copy link
Contributor

HansVRP commented Jan 29, 2025

@manugv best to first discuss with stefaan as we need to remain backwards compatible.

Sounds like an easy python-client issue wherein the name in add_backend should be made optional.

This ticket also allows you to explore the multijobmanager. Examples on its usage can be found in the community examples

@jdries
Copy link
Collaborator

jdries commented Jan 31, 2025

@soxofaan isn't the purpose of the name to allow tracking jobs on multiple backends? Or do you suggest to generate an identifier internally?

@soxofaan
Copy link
Member Author

Indeed, I understand it is important internally, but requiring the user to give it a name seems unnecessary, especially for users that only use one backend anyway.
I think the backend URL itself could indeed act as fallback identifier

@manugv
Copy link

manugv commented Jan 31, 2025

I have another question: If we remove the identifier then will it not remove a feature where someone is working on two backends and specify different jobs using the identifier (in pandas Dataframe)? Or this feature is unnecessary?
If unnecessary, I will remove the identifier and if not then I will make it optional.

Second: in both cases removing the identifier or making it optional, will break backward compatibility unless a new function is created. So how important is backward compatibility in this case?

@soxofaan
Copy link
Member Author

Maybe I was not clear in my original description, but I would just remove the requirement for the user to provide a name (and use the backend URL as fallback), but keep it as an option if they want it (and to stay backward compatible )

soxofaan added a commit that referenced this issue Jan 31, 2025
@soxofaan
Copy link
Member Author

As discussed earlier today: this is pretty low priority ticket (to simplify the user experience a bit), we probably have higher prio tasks to tackle first.
(also note that it's probably not trivial to do this in a backwards-compatible way, so the cost/benefit ratio is a bit off in that sense)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants