Add embedded language support for markdown in @docs and @moduledocs

[Terbium-135]

Right now I am extracting the text into a markdown file, edit and copy back.

It would be very nice to get syntax highlighting for @doc and @moduledoc parts. Since these are markdown the readability might increase.

Instead of this:

one might see:

[Blond11516]

@Terbium-135 This should be fixed on main. Please pull the latest main and try it out when you get the chance. Thanks!

[lukaszsamson]

@Blond11516 I tried that in ElixirLS v0.15 and had to revert as markdown rules would leak out of context and break highlighting of everything following. See elixir-lsp/vscode-elixir-ls#315 (comment)

[Terbium-135]

I tried it as well and facing:

I changed package.json and set:

  "engines": {
    "vscode": "^1.79.0-insider"
  },

I see no leakage so far. But only had a quick glance.

and

I am not using ElixirLS.

[Terbium-135]

Sadly I discovered a file which leaks markdown - perhaps something can be done to better (start and) terminate the heredoc area?

taken from this part:

The 2nd next """ ends markdown and switches back to elixir:

[Blond11516]

So from what I've seen the issue of leakage is caused by the additional indentation that @doc and friends add at the beginning of the markdown. This causes vscode to misinterpret the markdown as soon as it detects some sort of indented element (non-fenced code block, element within a list, etc.)

For example, as soon as the heredoc contains a list, vscode interprets everything coming after it as a list element, because the markdown is indented as such:

This then appears to cause the next closing heredoc to be missed for whatever reason.

A custom markdown grammar might indeed be able to fix this, but that's obviously not a very desirable solution. I'll dig a bit into TextMate to see if there's any way to have it ignore this identation but I wouldn't bet on that being the case. Ideally I'd like to see how other extensions handle this; it would need to be a grammar that injects a whitespace-sensitive language (markdown, python, etc.) with some implicit indentation but I don't have any ideas off the top of my head. I'm all ears if anyone knows of something like this 🤷

I've pushed a small reproduction repository with examples of list and nested module failures.

[Blond11516]

I've reverted the changes from main and pushed them back in a new branch.

[Terbium-135]

Thank you for all your work and investigation so far!

[SteffenDE]

I think a while pattern should do the trick, see elixir-lsp/vscode-elixir-ls#315 (comment).

Confirmed working (with ElixirLS) for the examples from https://github.com/Blond11516/markdown-leakage-repro