Skip to content

Commit

Permalink
_
Browse files Browse the repository at this point in the history
  • Loading branch information
@jeff-hykin
jeff-hykin committed Jul 4, 2019
1 parent 2d593d3 commit c7794cd
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 5 deletions.
4 changes: 2 additions & 2 deletions package.json
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{
"name": "vscode-textmate",
"name": "vscode-textmate-experimental",
"version": "4.1.1",
"description": "VSCode TextMate grammar helpers",
"author": {
"name": "Microsoft Corporation"
"name": "Jeff Hykin"
},
"main": "./release/main.js",
"typings": "./release/main.d.ts",
Expand Down
9 changes: 6 additions & 3 deletions release/main.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2025,11 +2025,12 @@ var BeginEndRule = /** @class */ (function (_super) {
exports.BeginEndRule = BeginEndRule;
var BeginWhileRule = /** @class */ (function (_super) {
__extends(BeginWhileRule, _super);
function BeginWhileRule($location, id, name, contentName, begin, beginCaptures, _while, whileCaptures, patterns) {
function BeginWhileRule($location, id, name, contentName, begin, beginCaptures, overshoot, _while, whileCaptures, patterns) {
var _this = _super.call(this, $location, id, name, contentName) || this;
_this._begin = new RegExpSource(begin, _this.id);
_this.beginCaptures = beginCaptures;
_this.whileCaptures = whileCaptures;
_this.overshoot = overshoot;
_this._while = new RegExpSource(_while, -2);
_this.whileHasBackReferences = _this._while.hasBackReferences;
_this.patterns = patterns.patterns;
Expand Down Expand Up @@ -2101,7 +2102,7 @@ var RuleFactory = /** @class */ (function () {
return new IncludeOnlyRule(desc.$vscodeTextmateLocation, desc.id, desc.name, desc.contentName, RuleFactory._compilePatterns(desc.patterns, helper, repository));
}
if (desc.while) {
return new BeginWhileRule(desc.$vscodeTextmateLocation, desc.id, desc.name, desc.contentName, desc.begin, RuleFactory._compileCaptures(desc.beginCaptures || desc.captures, helper, repository), desc.while, RuleFactory._compileCaptures(desc.whileCaptures || desc.captures, helper, repository), RuleFactory._compilePatterns(desc.patterns, helper, repository));
return new BeginWhileRule(desc.$vscodeTextmateLocation, desc.id, desc.name, desc.contentName, desc.begin, RuleFactory._compileCaptures(desc.beginCaptures || desc.captures, helper, repository), desc.overshoot || 0, desc.while, RuleFactory._compileCaptures(desc.whileCaptures || desc.captures, helper, repository), RuleFactory._compilePatterns(desc.patterns, helper, repository));
}
return new BeginEndRule(desc.$vscodeTextmateLocation, desc.id, desc.name, desc.contentName, desc.begin, RuleFactory._compileCaptures(desc.beginCaptures || desc.captures, helper, repository), desc.end, RuleFactory._compileCaptures(desc.endCaptures || desc.captures, helper, repository), desc.applyEndPatternLast, RuleFactory._compilePatterns(desc.patterns, helper, repository));
});
Expand Down Expand Up @@ -2782,9 +2783,11 @@ function _checkWhileConditions(grammar, lineText, isFirstLine, linePos, stack, l
});
}
}
var overshoot = 0;
for (var whileRule = whileRules.pop(); whileRule; whileRule = whileRules.pop()) {
var ruleScanner = whileRule.rule.compileWhile(grammar, whileRule.stack.endRule, isFirstLine, anchorPosition === linePos);
var r = ruleScanner.scanner.findNextMatchSync(lineText, linePos);
overshoot = whileRule.rule.overshoot;
if (debug_1.IN_DEBUG_MODE) {
console.log(' scanning for while rule');
console.log(debugCompiledRuleToString(ruleScanner));
Expand Down Expand Up @@ -2812,7 +2815,7 @@ function _checkWhileConditions(grammar, lineText, isFirstLine, linePos, stack, l
break;
}
}
return { stack: stack, linePos: linePos, anchorPosition: anchorPosition, isFirstLine: isFirstLine };
return { stack: stack, linePos: overshoot + linePos, anchorPosition: anchorPosition, isFirstLine: isFirstLine };
}
function _tokenizeString(grammar, lineText, isFirstLine, linePos, stack, lineTokens) {
var lineLength = lineText.content.length;
Expand Down
1 change: 1 addition & 0 deletions release/types.d.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ export interface IRawRule extends ILocatable {
readonly beginCaptures?: IRawCaptures;
readonly end?: string;
readonly endCaptures?: IRawCaptures;
readonly overshoot?: number;

This comment has been minimized.

Sign in to view

Copy link
@matter123

matter123 Jul 4, 2019

Collaborator

why is overshoot a number?

This comment has been minimized.

Sign in to view

Copy link
@jeff-hykin

jeff-hykin Jul 5, 2019

Author Owner

the idea was for it to be the number of lines to advance (or go back on) after the last while-pattern match.
e.g. match the

// comment \
     next line \
     last line

the line with the \ would be the last one matched, and it would have an overshoot of 1 so the final part of the comment was included.

I was thinking it could also be negative as a kind of muilt-linelookahead

I'm still not sure if this is a good idea, or even if it's possible.

This comment has been minimized.

Sign in to view

Copy link
@matter123

matter123 Jul 5, 2019 •
edited
Loading

Collaborator

That can be accomplished with an end pattern of lookBehindToAvoid(/[\\]/).then(@end_of_line) when there is something like atom/first-mate#90 to match the end pattern first.

Alternatively, vscode-textmate could treat line-continuation characters as combining the two lines. That way the grammar see //comment next line last line

Edit: combining on line continuation won't work because vscode-textmate only gets a single pass at each line.

This comment has been minimized.

Sign in to view

Copy link
@matter123

matter123 Jul 5, 2019

Collaborator

From how I understand while patterns work, a negative overshoot is not possible. Namely, because tokenizeLine is called in sequence for each line, the grammar cannot backtrack. So if overshoot was -2. vscode-textmate has no way of reparsing next line \ as not a comment.

This comment has been minimized.

Sign in to view

Copy link
@jeff-hykin

jeff-hykin Jul 5, 2019 •
edited
Loading

Author Owner

sadly lookBehindToAvoid(/[\\]/).then(@end_of_line) doesn't work when theres another internal pattern that is looking for it that also consumes more characters. It also doesn't force-close all the pattern ranges the way the while method would.

That makes sense with the backtracking.

This comment has been minimized.

Sign in to view

Copy link
@matter123

matter123 Jul 5, 2019 •
edited
Loading

Collaborator

I mentioned that the first-mate PR would have to be implemented for that to work.
#1 has a rough description of how that works.

This comment has been minimized.

Sign in to view

Copy link
@jeff-hykin

jeff-hykin Jul 5, 2019

Author Owner

Ah thanks, I completely missed that. Yeah thats basically exactly the kind of functionality thats needed that I hope we can implement.

This comment has been minimized.

Sign in to view

Copy link
@jeff-hykin

jeff-hykin Jul 5, 2019 •
edited
Loading

Author Owner

its probably best to scrap the overshoot completely and just go straight to trying to implement the atom/first-mate#90 feature.

This comment has been minimized.

Sign in to view

Copy link
@matter123

matter123 Jul 5, 2019

Collaborator

Probably, but overshoot should be relatively simple to implement (at least with while).

This comment has been minimized.

Sign in to view

Copy link
@jeff-hykin

jeff-hykin Jul 5, 2019

Author Owner

Thats true, so maybe it is worth exploring. There is the problem of the existing issues with while though (the issue blocking numbers from showing up in C++ embedded in markdown)
readonly while?: string;
readonly whileCaptures?: IRawCaptures;
readonly patterns?: IRawRule[];
Expand Down

0 comments on commit c7794cd

Please sign in to comment.