Syntax highlighting of intrapackage @testitems

[MilesCranmer]

I have been loving TestItems so far. One thing that would make it even better would be to have some way to change the syntax highlighting of a test item block to help visually disentangle it from my source code.

Skimming through my project, it is hard for me to identify source code from test code:

@test looks like every other macro, so unless I read the actual symbol, I might not be able to tell what part is test and what part is code.

Thus I'd like to propose displaying the intra-package test blocks as slightly transparent (user-configurable), so it would look like this:

Or, perhaps leaving the @testitem ... line itself opaque with the rest of the test slightly transparent. Anyways, it is the same sort of VSCode mechanism as when you have a block of code in Python that is inaccessible – it will turn transparent.

Then, only when I run or edit a @testitem block would it become opaque again. Thus, I can still skim my codebase without worrying about cluttering it up with tests.

One other option would be to have automatic folding for the test blocks. I'm not sure what would be more work.

[davidanthoff]

I like the idea, but it should probably be behind a configurable option flag, I can also see that some folks wouldn't want this

[MilesCranmer]

Yep sounds good to me.

[MilesCranmer]

I really really want this for my own use so maybe you could point me to how I could go about doing the work? Even if it’s just something I need to configure manually (and even without the feature of becoming opaque during a test), that would be great.

[davidanthoff]

Ha, I'm actually not sure :) I think it probably would have to be added in the language server, as that is the only place where we really have the data structures that fully understand the content of a Julia source file.

But then it gets complicated... I think the language server protocol does have messages that handle syntax highlighting stuff, but I'm not sure whether that is an opt-in-entirely story, or whether one could do that just for this... But I might also misremember whether the LS supports this at all. The other option would be that the LS sends custom messages back to the extension and then the extension uses some VS Code API to change things.

Both of these options touch a lot of stuff...

CC @pfitzseb he might also have ideas.

[pfitzseb]

Our best bet here would be to use the semantic tokens spec. That allows augmenting or replacing the regex based grammar with additional LS provided info. IIRC we don't have any of the relevant infrastructure in place now, so it'd be somewhat involved to set that up.

The easy short-term solution is to add an additional token to the grammar that you can then use to customize the appearance of test items (by adding to editor.tokenColorCustomizations).

[MilesCranmer]

The easy short-term solution is to add an additional token to the grammar that you can then use to customize the appearance of test items (by adding to editor.tokenColorCustomizations).

Sounds promising. Where do I start? I see that support.function.macro.julia detects a macro call by itself, although I guess we want to detect the inner block as well.

[davidanthoff]

The easy short-term solution

Would that allow us to make this optional, though? I don't think this would be a good default... I can see the transparency being useful for some, but the default interpretation for that in VS Code seems to be something like "disabled by a compile flag" or something along those lines, which is not really a great match for our situation...

[MilesCranmer]

I think @pfitzseb is simply suggesting adding @testitem to the grammar of the LSP, so you could do stuff with it. But the default “stuff” would treat it like any other macro that gets detected.

Then for the time being those interested can modify their VSCode settings to make such a token induce some transparency on its captured block (or different color, or whatever else they want).

I don’t think we should worry about the right default behavior now, that can be a different discussion after this is made possible in the first place.

[MilesCranmer]

To add, though:

"disabled by a compile flag" or something along those lines

I think this might actually be what we want? Those @testitems are disabled during regular execution of the package. Which is just like if you had a release build that disabled blocks of unused code (which VSCode shows as transparent).

(in any case this seems like something to think about later, after trying to get it working)

[MilesCranmer]

Okay @pfitzseb @davidanthoff I have made a PR to add it to the grammar: JuliaEditorSupport/atom-language-julia#281

The tricky part which I left as-is for now is I didn't know how to capture a matching begin ... end block of the @testitem. So the hacky way I did it for now is to refuse to capture the @testitem unless it appears at the start of a newline. And the matching end will simply be the first that appears at the start of a newline.

I was not sure how to get around this with just regex. But I think for an experimental addition it's fine as it would be optional to turn on.

[MilesCranmer]

I put this in my VSCode settings.json: (and copied in the grammar.. I can verify it is indeed changing the color of the testitem blocks!)

    "editor.semanticTokenColorCustomizations": {
        "enabled": true,
        "rules": {
            "keywords.other.testitem.julia": "#afafaf"
        }
    }

but doesn't seem to work. I assume I'm doing this completely wrong... Pointers would be much appreciated :)

[davidanthoff]

I took a look at the LSP section for semantic tokens. At some level it might actually be much easier to do this via the language server: we already have the entire machinery to detect the full range of @testitems in there, so all we would have to do is send these tokens back. And if we can do this as augmenting and overlapping, then maybe this could be relatively easy?

https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide is useful for this.

[MilesCranmer]

That sounds good to me. @pfitzseb how can I add this?

[davidanthoff]

I started with some of the infrastructure over at julia-vscode/julia-vscode#3606, but it isn't working yet...

Also, are we positive that the rust extension (which does something similar in terms of UI) is actually using semantic tokens for this? I can't find anything in https://github.com/rust-lang/rust-analyzer/blob/master/editors/code/package.json that matches. Maybe they are doing it in a different way?

Also, it seems that VS Code does not support multiline semantic tokens, which would be a problem?

[MilesCranmer]

Hm.. indeed if you turn on the VSCode developer mode see what tokens/scopes are active, you can see that the greyed out code has the exact same TextMate scopes as non-greyed out code. So it sounds like it’s something else the LS is sending to demarcate unused snippets?