Every parse() call makes a handful of full-log regex passes - a per-line search for each of the 9 latest-match patterns, a whole-log re.findall per log-category, and PATTERN_LOG_ENDING / scrape-item scans from the start - even though each one only needs a small, locatable slice of the log. On large logs this dominates the runtime.
They can all be narrowed with cheap literal checks (in / str.find, C-level) without changing the output - the regex still decides every match, and the literal only decides whether it's worth running. Measured (CPython 3.12):
| log |
now |
patched |
speedup |
| 142 KB |
13.7 ms |
1.8 ms |
7.6× |
| 1.1 MB |
108 ms |
15 ms |
7.3× |
| 10 MB |
896 ms |
126 ms |
7.1× |
| 1.1 MB (many error/retry/redirect lines) |
98 ms |
14 ms |
7.2× |
Output is byte-identical and the existing test suite passes unchanged.
Every
parse()call makes a handful of full-log regex passes - a per-line search for each of the 9 latest-match patterns, a whole-logre.findallper log-category, andPATTERN_LOG_ENDING/ scrape-item scans from the start - even though each one only needs a small, locatable slice of the log. On large logs this dominates the runtime.They can all be narrowed with cheap literal checks (
in/str.find, C-level) without changing the output - the regex still decides every match, and the literal only decides whether it's worth running. Measured (CPython 3.12):Output is byte-identical and the existing test suite passes unchanged.