Something like the classic [lex](https://en.wikipedia.org/wiki/Lexical_analysis). Probably tricky to get right but feasible with current tools.
Something like the classic lex. Probably tricky to get right but feasible with current tools.