Identify words that may contain numbers without selecting numbers

[LordGolias]

I have a set of rules to identify tokens, that are used for more complex statements.

string    = @{ ("\"" ~ ("\"\"" | (!"\"" ~ ANY))* ~ "\"") | ("'" ~ ("''" | (!"'" ~ ANY))* ~ "'") }

// allow .1, 1e6, etc. negatives are handled as unary ops somewhere else
number    = @{
    "."? ~ ASCII_DIGIT+ ~ ("."? ~ ASCII_DIGIT+)? ~ (^"e" ~ ("+" | "-")? ~ ASCII_DIGIT+)?
}
word      = @{ (ASCII_ALPHA | "_") ~ (ASCII_ALPHANUMERIC | "_")* }
symbol    = @{ "&&" | "||" | "==" | "<=" | ">=" | "!=" | "<<" | ">>" | SYMBOL | PUNCTUATION | NUMBER }
base_term = @{ string | word | number | symbol }

however, this breaks under examples where the word starts with a number and contains more letters, such as 3a33aa?

I placed the word | number to avoid the number from capturing the first number from a word, but I have been unable to re-write the word to make support this use-case without capturing numbers.

Example of what I tried, but is not sufficient

word_prefix  = @{ (NUMBER | "_") }
word_primary = @{ (ASCII_ALPHA | "_")+ }
word         = @{ word_prefix? ~ word_primary ~ (word_prefix ~ word_primary)* }

Any ideas on how to extend the word rule to allow things like 3a33aa without capturing numbers like 3?

[notpeter]

Based on the first version of your code word doesn't match 3a33aa because word specifically begins with a required (ASCII_ALPHA|"_") so it's never going to match things that start with a number. Change that to be more permissive:

word      = @{ (ASCII_ALPHANUMERIC | "_")+ }

If you put number before word in base_term it will greedily capture numbers (3 and 33) and things like 3a33aa will fall through and be captured by word.

base_term = @{ string | number | word | symbol }

If that isn't enough, can you provide a little more detail:

1. Examples of what you are trying to parse? (both success and failure)
2. Whether you're using implicit whitespace (and if so what value you have for WHITESPACE)

---

Totally unrelated your number rule has a couple issues. It'll match .1.1 (not a number) and won't match 1.E6 or 1. (valid floats in most languages).

I wrote recipes for numbers for parsing Lua that you are free to steal:

number = @{ HEX_INTEGER | FLOAT | INTEGER }
FLOAT = @{
    ( ASCII_DIGIT+ ~ "." ~ ASCII_DIGIT* ~ FLOAT_LITERAL_EXPONENT? )
    | ( "." ~ ASCII_DIGIT+ ~ FLOAT_LITERAL_EXPONENT? )
    | (ASCII_DIGIT+ ~ FLOAT_LITERAL_EXPONENT)
}
FLOAT_LITERAL_EXPONENT = { ("e" | "E") ~ ("+" | "-")? ~ ASCII_DIGIT+ }
HEX_INTEGER = { ("0x" | "0X") ~ ASCII_HEX_DIGIT+ }
INTEGER = { ASCII_DIGIT+ }

Feel free to tip you found this helpful.