How to dynamically override templates with hooks?

[kamilkrzyskow]

Disclaimer: This guide expects a basic understanding of the Python programming language, which is used in the solution. However, the explanation describes a reusable script which could be altered to the users needs without full understanding. This free course can also get you up to speed with using Python.

Another disclaimer: This guide expects a basic understanding of normal static template overriding. You can learn more about it here and here2.

---

Context

Whenever our team requires some functionality adjustment in MkDocs or this theme, I typically add some extra JavaScript solution that runs after the site loads or do some changes in a block inside the main.html override. I always preferred this approach, because it is dynamic and separated. A static template replacement is nice and quick, but possibly requires maintenance even when some unrelated part of the file gets updated.

This time the team decided to add country flags to the language selector. I could add a JavaScript solution with a static mapping of languages to flag <svg>/<img> tags and run it after the page loads, but I would have to add new flags for each new language we decide to add in the future. This idea didn't sit well with me,

So I started with generating a list of flags in the main.html to pass it over to JavaScript, but it turned out that the :flag_pl: 🇵🇱 emojis get parsed only for the markdown content 😳, so this idea didn't work out. Also overriding the template file was a no-go too, because it wouldn't parse the emojis either.

Therefore I needed something that would parse the emoji codes and inject them into the header selector...

Solution

Since version 8.5.5, this theme works on top of MkDocs 1.4.0, which added support for hooks. In short hooks are scripts that mimic the functionality of MkDocs plugins, but do not need to be packaged and installed to the Python environment. Having access to the plugin events we can run operations during the build process.

First idea (not implemented)

Before a page is saved to disk, the built HTML can be modified in the on_post_page event. It is probably the most straight-forward idea, as I could either replace the selector part or inject the emojis, and then process them via JavaScript. However, I decided against it, because the processing time would grow together with the amount of pages in the docs and therefore could slow down the build process in the future.

Second idea (implementation)

If I don't want to process each page I need to influence how they're created, but after reading the documentation about events it turned out there is no direct event that processes theme files 😳, so is this the end?
Of course not! Each module in Python is a file, and we can access the path to the file like so module_name.__file__.

The solution description is separated into 6 sections:

• Constants
• Backup Management
• Core Logic Event
• Core Logic Override
• Closing Events
• Extra CSS

Constants

The constants include a mapping of languages (used in the language selector) to country codes (used in flag emojis) and a copy of the Twemoji index. Adapted respectively from the translations.py hook script and from materialx.emoji module. Apart from that there is a logger instance which is used to log messages to the terminal window, and the name of the hook which is used in the log messages.

Backup management

I won't go into the details of the backup management, but will explain the process of how it works:

Backup creation:

1. Check if there is a backup of the file, restore it, compare file to assert a valid restoration.
2. Create backup, and assert that it was correctly created via a file comparison.

Backup restoration:

1. Check if backup still exists, if not send a warning log message.
2. Restore backup, and assert that it was correctly restored via a file comparison.
3. Delete the backup file.

Any failed assertion will crash MkDocs, but this should never happen because we copy the files and immediately compare them.

Possible issues:

• If there is a backup file left over after a bad crash (that lead to the file not being restored, which is unlikely), and you later update the theme without uninstalling it first (which is also unlikely) and it will have a new version of the template - running the script would restore the old backup overwriting the newer original version. This should never happen, but it is a possibility.
• The backup system was designed around a "1 script processes 1..n unique templates" idea, so if 2 scripts process the same file, there could be some race condition when it comes to the backup restoration. It might restore a different version of the file than the original, and as the restoration deletes that backup file, each subsequent script will log a warning, because the backup doesn't exist anymore. This can be fixed in multiple ways. The first idea that comes to mind is the creation of an empty partial.html.backup.lock file after creating the first backup, that would limit the backup handling to only the first script.

Core Logic (Hook Event)

The main solution is located in the on_env event, which is the last event that runs before the build begins. This assures that most things have been already configured in other events that run before that. The logic is separated into different parts:

Disclaimer: Best to open the text editor with the file, linked at the bottom and follow along the guide with the file.

Add this to your mkdocs.yml config to enable the hook (your path may be different):

hooks:
  - overrides/.hooks/language_flags.py

1. Make sure the script can run:

   @plugins.event_priority(100)
   def on_env(*_, config: MkDocsConfig, **__) -> Optional[jinja2.Environment]:
       """
       Main function. Triggers just before the build begins. 
       Priority 100 to promote having access to the files first
       """
   
       if config["theme"]["name"] != "material":
           LOG.info(f'{HOOK_NAME}: Only the "material" theme is supported')
           return None
      
       if "extra" not in config or "alternate" not in config["extra"]:
           LOG.info(f'{HOOK_NAME}: "extra.alternate" not detected')
           return None
      
       if len(config["extra"]["alternate"]) < 2:
           LOG.info(f"{HOOK_NAME}: Not enough languages")
           return None

   My hook won't run if MkDocs uses another theme, if there is missing configuration in mkdocs.yml or
   if the configuration not the way I like. Think of cases where your hook might crash or is not needed and prevent it from running like shown above.

2. Fill out the list of source and backup path pairs, then backup those files:

   partials: Path = Path(material.__file__).parent / "partials"
   header: Path = partials / "header.html"
   another_partial: Path = partials / "another_partial.html"
   
   # Fill out the list of source and backup file pairs
   if not BackupManager.backups: 
       BackupManager.backups.append((header, Path(f"{header}.backup")))
       BackupManager.backups.append((another_partial, Path(f"{another_partial}.backup")))
   
   BackupManager.create_backup()

   Using the pathlib.Path class I got to the header.html partial which contains the language selector template.
   The conditional if not BackupManager.backups is there to prevent adding duplicate entries.
   It means it will only append new entries when the list was initially empty (so only during the 1st build).

3. Process the files:

   # Process template
   _add_flags(partial=header, config=config)
   _process_another_partial(partial=another_partial, ...)

   I separated the event invocation from the template processing to keep the function shorter.
   You can do as you prefer.

Core Logic (Overriding the template)

1. Configure the tokens that will be replaced:

   def _add_flags(*, partial: Path, config: MkDocsConfig) -> None:
       """Process the "header.html" partial and add flags to the language selector template"""    
   
       # Configure the tokens
       tokens: Dict[str, str] = {
           "START": "{% if config.extra.alternate %}",
           "CONFIG": '{% set icon = config.theme.icon.alternate or "material/translate" %}',
           "SELECTOR": '{% include ".icons/" ~ icon ~ ".svg" %}',
           "LINK": "{{ alt.name }}",
           "NEW_IDENTIFIER": "...",
           "END": "{% endif %}",
       }
   
       # A negative number means the same level as the START token
       end_indent_level: int = -1

   This is a mapping of token identifiers and their respective strings in the file.
   START and END are mandatory the rest is custom up to the template / usage.
   The end_indent_level variable can be adjusted to change the scope of the loaded section.

2. Load the section in the template file.

   There is nothing you need to change in this part, so I'll omit the code example.
   It scans the file looking for the START and END tokens. It logs an error if either was not found.
   Additionally it writes a file with the loaded section, so you can find out what the issue was.

   The loaded content is stored in the loaded_content variable, and the loaded section in loaded_section.

3. Validate the loaded section.

   There is nothing you need to change in this part, so I'll omit the code example.
   It asserts that every token is in the loaded section. It logs an error if any was not found.
   Additionally it writes a file with the loaded section, so you can find out what the issue was.

   This is the part which distinguishes this dynamic method from simple file replacement,
   because I'm comparing only the needed lines and the rest are always up-to-date.
   If you prefer to compare the relevant section you could save a original.html file
   and load it for comparison too.

4. Process anything required before overriding the template:

   # Load all flags relevant to the current build
   flag_mapping: Dict[str, str] = {
       alt["lang"]: _flag_svg(alt["lang"]) for alt in config["extra"]["alternate"]
   }
   
   # Set the "flag_mapping" selector
   selector: str = "config.theme.language"
   
   # Change selector for the "i18n" plugin
   if "i18n" in config.plugins:
       selector = "i18n_page_locale"

   In my case I've created a mapping of the alternate languages to their respective emojis.
   The logic of _flag_svg is irrelevant to this guide.
   Apart from that I choose the selector of the mapping depending on the presence of the i18n plugin.

5. Override the template:

   # Modify the partial
   with open(partial, "w", encoding="utf8") as file:
       file.write(
           loaded_content.replace(
               loaded_section,
               loaded_section.replace(
                   tokens["CONFIG"], "{% set flag_mapping = " + str(flag_mapping) + " %}"
               )
               .replace(tokens["SELECTOR"], "{{ flag_mapping[" + selector + "] }}")
               .replace(
                   tokens["LINK"],
                   "{{ flag_mapping[alt.lang] }} "
                   + '{{ alt.name | replace(alt.lang, "") | replace(" ", "") | replace("-", "") }}',
               ),
           )
       )

   Replace each token with another value, you can limit of replacements like so replace(old, new, 1).
   However, it will always start from the first occurrence. After that replace the loaded section in the loaded content and write the modified content to file.
   The LINK token's replacement is like that to support both name patterns in the alternate config:

   • "English"
   • "en - English"

Closing Events

For the closing events I chose on_build_error, on_post_build and on_shutdown. I'm using 3 different events, because there are cases where 1 of them might not run. All of them try to run the restore backup logic, if the files are already restored from a previous trigger it won't log any warnings.

Extra CSS

By default no emoji will show up. That's because the built-in CSS works only with inline SVG's. Therefore add this to your extra CSS file:

/* override language selector */
.md-select button img.twemoji {
    height: 1.2rem;
    vertical-align: middle;
}

.md-select__link img.twemoji {
    height: 1rem;
    vertical-align: text-bottom;
    margin-right: 1%;
}

---

I tried to be brief and describe how it works, and I accomplished none of that 😅. Nevertheless, I hope it provided some
value. The newest version of the hook can always be found here.

Here is the version, this guide was basing on language_flags.zip

Before posting this guide I looked over the discussions and only found this suggestion which contains an example of how to change the language selector with static template overrides, and SVG files. The current version of the hook doesn't support inline SVG's, and there is no fallback for the case when the remote images won't load, so the selector button will be empty when that happens. In that regard the static override solution might be better as it hosts the images in the docs too😄.

The script hook works on both Windows and Linux (tested only on the GitHub Actions runner Ubuntu).
Tested with Python 3.11 and Material for MkDocs 9.0.12

Example of a valid INFO log when running MkDocs with the i18n plugin:

$ mkdocs build -s
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /site
INFO     -  language_flags: Backing up 1 file...
INFO     -  Building en documentation
INFO     -  Building de documentation
INFO     -  language_flags: Restoring 1 file...
INFO     -  Documentation built in 5.55 seconds

Before and After pictures:

[squidfunk]

Many thanks for sharing!