Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dispatch shouldn't require comma after {} arm #344

Open
2 tasks done
adamchalmers opened this issue Oct 7, 2023 · 1 comment
Open
2 tasks done

dispatch shouldn't require comma after {} arm #344

adamchalmers opened this issue Oct 7, 2023 · 1 comment
Labels
A-combinator Area: combinators C-bug Category: Things not working as expected

Comments

@adamchalmers
Copy link
Contributor

adamchalmers commented Oct 7, 2023

Please complete the following tasks

rust version

1.73

winnow version

0.5.16

Minimal reproducible code

use winnow::prelude::*;
use winnow::token::any;
use winnow::combinator::{success, fail, dispatch, preceded};

fn escaped(input: &mut &str) -> PResult<char> {
    preceded('\\', escape_seq_char).parse_next(input)
}

fn escape_seq_char(input: &mut &str) -> PResult<char> {
    dispatch! {any;
        'b' => {
            success('\u{8}')
        }
        'f' => success('\u{c}'),
        _ => fail::<_, char, _>,
    }
    .parse_next(input)
}

Steps to reproduce the bug with the above code

cargo check

Actual Behaviour

Program does not compile, because the arm with 'b' doesn't have a trailing comma.

error: no rules expected the token `'f'`
  --> src/main.rs:14:9
   |
13 |         }
   |          - help: missing comma here
14 |         'f' => success('\u{c}'),
   |         ^^^ no rules expected this token in macro call
   |
   = note: while trying to match sequence start

If you add a trailing comma after, it compiles:

        'b' => {
            success('\u{8}')
        },

Expected Behaviour

dispatch! should work like Rust's standard match construct, which requires a trailing comma after non-{} arms, but allows omitting the trailing comma after {} arms.

This way dispatch! will be more familiar to new users of the library

Additional Context

This is a very minor low-priority issue, but the current macro behaviour surprised me. I figured it was worth tracking.

@adamchalmers adamchalmers added the C-bug Category: Things not working as expected label Oct 7, 2023
@adamchalmers adamchalmers changed the title Winnow dispatch shouldn't require comma after {} arm dispatch shouldn't require comma after {} arm Oct 7, 2023
@epage epage added the A-combinator Area: combinators label Oct 9, 2023
@epage
Copy link
Collaborator

epage commented Oct 9, 2023

Thanks for reporting! macro-rules can be a pain to get right in all of these corner cases.

adamchalmers added a commit to KittyCAD/modeling-app that referenced this issue Oct 12, 2023
* New parser built with Winnow

This new parser uses [winnow](docs.rs/winnow) to replace the handwritten recursive parser.

## Differences

I think the Winnow parser is more readable than handwritten one, due to reusing standard combinators. If you have a parsre like `p` or `q` you can combine them with standard functions like `repeat(0..4, p)`, `opt(p)`, `alt((p, q))` and `separated1(p, ", ")`. This IMO makes it more readable once you know what those standard functions do.

It's also more accurate now -- e.g. the parser no longer swallows whitespace between comments, or inserts it where there was none before. It no longer changes // comments to /* comments depending on the surrounding whitespace. 

Primary form of testing was running the same KCL program through both the old and new parsers and asserting that both parsers produce the same AST. See the test `parser::parser_impl::tests::check_parsers_work_the_same`. But occasionally the new and old parsers disagree. This is either:

- Innocuous (e.g. disagreeing on whether a comment starts at the preceding whitespace or at the //)
- Helpful (e.g. new parser recognizes comments more accurately, preserving the difference between // and /* comments)
- Acceptably bad (e.g. new parser sometimes outputs worse error messages, TODO in #784)

so those KCL programs have their own unit tests in `parser_impl.rs` demonstrating the behaviour.

If you'd like to review this PR, it's arguably more important to review changes to the existing unit tests rather than the new parser itself. Because changes to the unit tests show where my parser changes behaviour -- usually for the better, occasionally for the worse (e.g. a worse error message than before). I think overall the improvements are worth it that I'd like to merge it without spending another week fixing it up -- we can fix the error messages in a follow-up PR.

## Performance

| Benchmark | Old parser (this branch) | New parser (this branch) | Speedup |
| ------------- | ------------- | ------------- | ------------- |
| Pipes on pipes  | 922 ms  | 42 ms | 21x |
| Kitt SVG  | 148 ms  | 7 ms | 21x |

There's definitely still room to improve performance:

- #839
- #840

## Winnow

Y'all know I love [Nom](docs.rs/nom) and I've blogged about it a lot. But I'm very happy using Winnow, a fork. It's got some really nice usability improvements. While writing this PR I found some bugs or unclear docs in Winnow:

- winnow-rs/winnow#339
- winnow-rs/winnow#341
- winnow-rs/winnow#342
- winnow-rs/winnow#344

The maintainer was quick to close them and release new versions within a few hours, so I feel very confident building the parser on this library. It's a clear improvement over Nom and it's used in toml-edit (and therefore within Cargo) and Gitoxide, so it's becoming a staple of the Rust ecosystem, which adds confidence.

Closes #716 
Closes #815 
Closes #599
silvanshade added a commit to silvanshade/winnow that referenced this issue May 28, 2024
silvanshade added a commit to silvanshade/winnow that referenced this issue May 28, 2024
silvanshade added a commit to silvanshade/winnow that referenced this issue May 28, 2024
silvanshade added a commit to silvanshade/winnow that referenced this issue May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-combinator Area: combinators C-bug Category: Things not working as expected
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants