lexer-generator

It would be nice if subpatterns could be used within subpatterns.

#[derive(Logos, Copy, Clone, Debug)]
#[logos(subpattern a = r"[a-zA-Z!$%&*/:<=>?^_~]")]
#[logos(subpattern b = r"(?&a)|[0-9+\-.@]")]
pub enum Token {
    #[error]
    Error,
    // ...
}

Currently, this is not supported and causes a regex parse error.

Add an option to NimlLexer to lexNext / lexIter provide a specified token when the lexer reaches EOF only once.

Example:

# lexer setup
lexer.ignoreIf = someProc
lexer.produceEOFToken(tokenForEOF) # if it is not given, reaching EOF produces no token.

# use lexer
...

related: #70

to make the circle complete, support constructing a regular expression out of a DFA.

This is an implementation detail, and may be nice for future features.