Tokenizer refactor #12
Description
although they work alright (hopefully lol) the tokenizers are still somewhat rudimentary. all of them are brute force solutions to do stuff (greedy encoding during code point level BPE for example). also maybe clean up the codebase a little, especially for fallback-to-unknown-char cases