Allow multiple repo

[alasdairwilson]

This moves the repo configuration from an env variable to the config yaml file.

repos are configured by:

• [repo]:
  • the repo name, use a unique identifier for each repo you wish to fetch.
  • path:
    : Unique identifier for the repo, this is used for subdpath in the /material path on the deployed site.
  • url:
    : The URL of the material repo, e.g. https://github.com/UNIVERSE-HPC/course-material.
  • exclude:
    : You can exclude certain sections and courses from the material repo, this is useful if you want to include a subset of the material in your deployment.
    • theme:
      : A list of themes to exclude from the material repo, this should match the "id" field in the theme markdown page.
    • course:
      : A list of courses to exclude from the material repo, this should match the "id" field in the course markdown page.
    • section:
      : A list of sections to exclude from the material repo, this should match the "id" field in the section markdown page.

Support for multiple repos:

• by adding multiple entries to the config file, multiple repos can be used as sources
• all repos are combined as cards on /material
• subsequent urls have the url config var prepended such that it would be /materia/urlA/[repoA material] etc.

Support for excluding parts of the repo:

• All levels of material can be excluded, by putting an exclude in the config of a repo at the theme, course or section level, those themes, courses and sections and all child parts are excluded.

TODO:

This isn't complete:

• urls returned from search are busted, this needs reworked
• pullmaterial, the nightly refresh needs fixed: it can be ported to use the new initrepos function/package.json script (one of)
• there are no tests: follow up, no idea how we can test this.
  • I suspect we should have a test repo and check it pulls correctly, and check some excludes. I think in general there should be a test.yaml in the config folder for various reasons. This is probably extremely low priority
• The logic for filtering the nodes in the treeview is overly complex, there are redundant sections (this should be spun off:
  • The issue here is that the component call chain is very very very large for these graphs and that some of the graph by necessity is populated by the material object (which we have successfully excluded) but other times parts are populated by requirements defined in the material files themselves. If these have been excluded then this will cause issues so at any point we are traversing nodes or making edges/links we need to check the exclude list to make sure we are not traversing a forbidden part of the tree. This has been done absolutely everywhere but there are some parts of the tree which are only reached with the safe method (filtered material object).

[martinjrobins]

This looks great, happy to merge as is, thanks @alasdairwilson!