Feature request manager for puppet modules metadata.json

[TheMeier]

Tell us more.

Puppet modules have a file metadata.json which, among other things, describes dependencies on other modules with a version specifier. These version specifiers need to be updated quite often and for many repositories, so an automation via renovate would be quite helpful.

The version specifier is using npm smever (at least claimed here https://github.com/puppetlabs/semantic_puppet/blob/main/lib/semantic_puppet/version_range.rb#L6) so I assume the code in https://github.com/renovatebot/renovate/tree/main/lib/modules/versioning/npm can be re-used for parsing the version specs.

[github-actions[bot]]

Hi there,

You're asking us to support a new package manager. We need to know some basic information about this package manager first. Please copy/paste the new package manager questionnaire, and fill it out in full.

Once the questionnaire is filled out we'll decide if we want to support this new manager.

Good luck,

The Renovate team

[TheMeier]

New package manager questionnaire

Did you read our documentation on adding a package manager?

• I've read the adding a package manager documentation.

Basics

What's the name of the package manager?

puppet-module

What language(s) does this package manager support?

puppet DSL

How popular is this package manager?

I would describe it as fairly popular. The puppetlabs registry (https://forge.puppet.com/modules) lists > 2000 modules with > 1 billion downloads. On top of that users usually have their own in-house modules which would also benefit from this.

Does this language have other (competing?) package managers?

• Yes (give names).
• No.

What are the big selling points for this package manager?

When handling puppet modules a common maintenance task is to check and update the dependencies. Renovate could be a great help here. IMHO this is a low hanging fruit since the versioning logic (npm) and datasource (puppet-forge) required for this are already part of renovate. So very little implementation would be needed to add some very useful functionality for puppet users.

Detecting package files

What kind of package files, and names, does this package manager use?

Each puppet module contains a single file at the top level called metadata.json

Which fileMatch pattern(s) should Renovate use?

^metadata.json$

Do many users need to extend the fileMatch pattern for custom file names?

• Yes, provide details.
• No.

Is the fileMatch pattern going to get many "false hits" for files that have nothing to do with package management?

Since the name metadata.json is very generic there is some potential for the match to trigger on repos that actually have nothing to do with puppet at all. That being said, I have never seen this name being used elsewhere. When searching for possible other uses I found a product "IBM Cloud Pak System" which also uses that name.

IMHO the probability for false positives is low to very-low.

Parsing and Extraction

Can package files have "local" links to each other that need to be resolved?

No. They just can contain puppet module names that need to be looked up in the registry

Package file parsing method

The package files should be:

• Parsed together (in serial).
• Parsed independently.

Since there is only one package file, this does not apply.

Which format/syntax does the package file use?

• JSON
• TOML
• YAML
• Custom (explain below)

How should we parse the package files?

• Off the shelf parser.
• Using regex.
• Custom-parsed line by line.
• Other.

Does the package file have different "types" of dependencies?

• Yes, production and development dependencies.
• No, all dependencies are treated the same.

List all the sources/syntaxes of dependencies that can be extracted

All extracted dependencies have the form namespace/name which describe a puppet module and can directly be looked up in the (existing) datasource (puppet-forge)

Describe which types of dependencies above are supported and which will be implemented in future

The dependencies are all puppet modules themselves.

Versioning

What versioning scheme does the package file(s) use?

Does this versioning scheme support range constraints, like ^1.0.0 or 1.x?

• Supports range constraints (for example: ^1.0.0 or 1.x), provide details.
• No.

The versioning scheme is npm (with ranges) which is already present in lib/modules/versioning/npm/

Lookup

Is a new datasource required?

• Yes, provide details.
• No.

Will users want (or need to) set a custom host or custom registry for Renovate's lookup?

• Yes, provide details.
• No.

Users often have custom in-house puppet-forge compatible registries. Support for this is already implemented in the existing puppet-forge datasource.

Where can Renovate find the custom host/registry?

• No custom host or registry is needed.
• In the package file(s), provide details.
• In some other file inside the repository, provide details.
• User needs to configure Renovate where to find the information, provide details.

If the user wants to use a custom registry he needs to specify that via config or environment variable. Support for this is already implemented in the existing puppet-forge datasource.

Are there any constraints in the package files that Renovate should use in the lookup procedure?

• Yes, there are constraints on the parent language (for example: supports only Python v3.x), provide details.
• Yes, there are constraints on the parent platform (for example: only supports Linux, Windows, etc.), provide details.
• Yes, some other kind of constraint, provide details.
• No constraints.

Will users need the ability to configure language or other constraints using Renovate config?

• Yes, provide details.
• No.

Artifacts

Does the package manager use a lock file or checksum file?

• Yes, uses lock file.
• Yes, uses checksum file.
• Yes, uses lock file and checksum file.
• No lock file or checksum.

Is the locksum or checksum mandatory?

• Yes, locksum is mandatory.
• Yes, checksum is mandatory.
• Yes, lock file and checksum are mandatory.
• No mandatory locksum or checksum.
• Package manager does not use locksums or checksums.

If lockfiles or checksums are used: what tool and exact commands should Renovate use to update one (or more) package versions in a dependency file?

not applicable

Package manager cache

Does the package manager use a cache?

• Yes, provide details.
• No.

If the package manager uses a cache, how can Renovate control the cache?

• Package manager does not use a cache.
• Controlled via command line interface, provide details.
• Controlled via environment variables, provide details.

Should Renovate keep a cache?

• Yes, ignore/disable the cache.
• No.

Generating a lockfile from scratch

Renovate can perform "lock file maintenance" by getting the package manager to generate a lockfile from scratch.
Can the package manager generate a lockfile from scratch?

• Yes, explain which command Renovate should use to generate the lockfile.
• No, the package manager does not generate a lockfile from scratch.
• No, the package manager does not use lockfiles.

Other

Maybe this does not need to be a completely new package manager but could be an extension to the existing puppet package manager if that makes any sense.

What else should we know about this package manager?

[ekohl]

All extracted dependencies have the form namespace/name which describe a puppet module and can directly be looked up in the (existing) datasource (puppet-forge)

Some use namespace-name and I think by now that's the recommended form. It's also the slug on the Puppet forge. The / is translated internally in Puppet.

The dependencies are all puppet modules themselves.

You can take supported Puppet versions into account. Under requirements you can list those and compare it with other modules. Though it's not commonly done automatically today, it could be a nice future extension.

Maybe this does not need to be a completely new package manager but could be an extension to the existing puppet package manager if that makes any sense.

To add to this, the existing puppet package manager only supports Puppetfile but those are for control repositories. Modules list the same modules with the same version requirements in their metadata.json.

[rarkins]

Converted to issue: #33994