You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I originally created this to create tokenz, but I'm finding myself more and more using this to walk through text without generating tokenz.
Because of this I might want to name this project as if it walks through text, not just a tokenizer library.
Because of that I'm not entirely sure I should rename this to walk-, but the one thing preventing me from doing so is that I have a package called tokenz-html. I like the idea of having different tokenizers based on this tokenz library.
Maybe I could rename it to a name that implies it deals with strings.
The text was updated successfully, but these errors were encountered:
I originally created this to create tokenz, but I'm finding myself more and more using this to walk through text without generating tokenz.
Because of this I might want to name this project as if it walks through text, not just a tokenizer library.
Because of that I'm not entirely sure I should rename this to walk-, but the one thing preventing me from doing so is that I have a package called tokenz-html. I like the idea of having different tokenizers based on this tokenz library.
Maybe I could rename it to a name that implies it deals with strings.
The text was updated successfully, but these errors were encountered: