JuliaPOMDP Registry for POMDP Problems

[dylan-asmar]

As the number of problems being developed within the JuliaPOMDP ecosystem continues to grow, I’d like to open a discussion about the possibility of creating a separate registry specifically for POMDP problems.

Currently, most of our packages are registered in the main Julia registry. This setup works well for core packages and solvers. However, I don’t think this model is ideal for packages that primarily implement problems using the POMDPs.jl framework. While having easily installable problems is important, it may not meet the criteria for inclusion in the main Julia registry, especially as the Julia ecosystem continues to expand.

The General Julia registry README explicitly states that it is not intended for "personal packages" or those only useful to closed groups, and suggests setting up a custom registry for such cases. While many of the problems we are registering are widely applicable within the POMDP community, having our own POMDPModels registry could provide greater flexibility in managing problem-specific packages and streamline development.

We already have the POMDPModels.jl package, which includes several smaller problems. My understanding for not including all problems in POMDPModels.jl has been concerns about increased loading times and dependencies. By transitioning to a dedicated registry for problems, we can ensure new problem packages remain accessible without overloading the core framework or causing dependency issues.

At one point, there was a registry maintained within the POMDPs.jl project. However, with the move to POMDPs.jl and POMDPTools.jl, the number of packages requiring registration was reduced, and registering main packages in the General registry became easier and more consistent with the broader Julia ecosystem.

Some key questions I’d like to raise:

• Were there any specific challenges maintaining a separate registry in the past?
• What potential drawbacks are there to not having problem packages registered in the General registry?
• While solvers likely meet the criteria for standalone packages on the General registry, should we also consider moving solvers to a separate POMDP Solvers registry?

Looking forward to hearing your thoughts on this!

[zsunberg]

Thanks for bringing this up! We can definitely do a better job of having a vibrant collection of benchmark problems. For the moment, I'll just tag @kylewray because he also mentioned it to me.

Were there any specific challenges maintaining a separate registry in the past?

I don't remember specifics at the moment, but I do remember strongly never wanting to do it again 😂

it may not meet the criteria for inclusion in the main Julia registry

My assessment is that packages like RockSamplePOMDP do meet the criteria for the main Julia registry, so I don't think this should be a big factor.

having our own POMDPModels registry could provide greater flexibility in managing problem-specific packages and streamline development.

Could you give some specific examples or describe the cases where we need greater flexibility and how operating our own registry would streamline it?

[zsunberg]

I don't quite understand the goal yet :) I don't think we should take on the significant maintenance, documentation, and difficulty-of-use burden that comes with running our own registry unless we have a very clear need.

Here are some goals that I thought of. Does your goal fit in one of these categories?

1. Making it easier for newcomers to the POMDPs.jl ecosystem to find POMDP models
2. Making it easier to reproduce results in papers
3. Making it easier for beginners (e.g. DMU students) to get their POMDP models out in public

[zsunberg]

Yeah, I think non-general purpose packages don't really belong in General.

Are there any other examples of domain-specific registries in the wild? I know that was a theoretical goal of the Pkg designers, but has it ever been put into practice?

[dylan-asmar]

I didn’t really have a well-defined goal per se—I just wanted to start a discussion around the idea that non-general packages probably shouldn’t be in the General registry.

As far as the goals you mentioned:

1. Making it easier for newcomers to the POMDPs.jl ecosystem to find POMDP models

I don’t think maintaining our own registry would change this significantly. Whether it’s on General, a set of non-registered repos, or a JuliaPOMDP registry, we could always provide support tools, documentation, or other resources to help guide users to different problems.

2. Making it easier to reproduce results in papers

Similar to the first point.

3. Making it easier for beginners (e.g., DMU students) to get their POMDP models out in public

I know there are a few problems around SISL that haven’t made it to General yet. I’m not 100% sure what the main barriers were/are. I suppose there might be less hesitation to share a problem if it’s not on General, which could potentially encourage more open sharing.

Are there any other examples of domain-specific registries in the wild? I know that was a theoretical goal of the Pkg designers, but has it ever been put into practice?

I don’t know of any outside of the HolyLabRegistry. There does seem to be some use for custom registries, given that tools like LocalRegistry.jl and RegistryCI.jl have active maintainers.

[dylan-asmar]

@mossr you just set up a SISL registry recently, right? Do you have any thoughts/opinions on this with that recent experience?

[mossr]

@dylan-asmar I think Maxime set it up, but I've been using it a lot recently. I think it's great to have! Super easy to configure and manage. I'd be all for a separate POMDP problems registry.

[mykelk]

I think @rejuvyesh might have some input here.