Introduce tags to magic castle's non essential components

[victorusu]

First of all, I am not sure if this question should go here or in the puppet repo. So, I apologize in advance.

According to the documentaiton, magic castle supports the following tags:

• login
• proxy
• public
• puppet
• spot
• efa
• ssl

with the additional tags supported by the official puppet repo

• login
• mgmt
• nfs
• node

By default, this provides an out-of-the-box experience that contains the following packages that are not fundamental for a magic castle cluster to work

• jupyterhub
• squid server
• cvmfs
• singularity
• globus

I would like to propose adding additional tags for each component, similarly to the mfa component, so that the end-user can select what to install in their cluster.

This would allow more generic clusters (with fewer components) to be built using magic castle and would allow other components to be added without interfering with the standards defined by ComputeCanada for their magic castle installations.

What do you think about it?

[cmd-ntrf]

I agree that we should have a mechanism to turn off some optional components. However, I am not convinced tag is the right mechanism mainly because working with tags interdependency in Terraform would hard, and allowing any tags to be assigned to any instance would potentially be really complex to support in Terraform and Puppet.

So for example, if we define a tag for jupyterhub, what does it mean if it is assigned to a compute node? Should the compute node be assigned a public ip address? Should jupyterhub represents the concept and if it is assigned to a login node, it means the login runs a hub and when it is assigned to a compute node, it can run notebook jobs? Then comes the difficult task of documenting the effect of each tag on each instance type.

There might be some components that would fit properly if assigned a tag, but apart for singularity. I think the four components you mention are too complex to be associated with tags.

One change I am considering for future releases is separating site.pp and the puppet classes in two different git repo. The site.pp repo that handles the tags would then be really small and easy to customize while it will still be able to depends on Puppet classes that are defined in a separate repository. We could envision having a the slurm cluster puppet environment use the Magic Castle puppet environment template defined here : https://github.com/MagicCastle/puppet-environment.