Regexps with Unicode flag

[gcampax]

Hi! Thanks for this great library.

I'm wondering if it would be possible to add support for Unicode regular expressions? Currently, it seems that the "u" flag is discarded when the lexer compiles a rule. Unicode regular expressions are necessary to properly match non BMP characters (e.g. emojis) as well as use Unicode properties as character classes.

[sormy]

It should work well with regexps like /something/u.
Please provide minimal working bug reproduction example.

[gcampax]

It doesn't work once you start using definitions: the u flag is not allowed in the regexp passed to addDefinition. The syntax for definitions is also problematic, as it relies on a deprecated syntax that Unicode regexps don't allow.

[sormy]

I guess we should use u flag by default. PR is welcome.