Skip to content

docs(nlp): state that Tokenizer.encode adds no special tokens - #1633

Merged
barhanc merged 3 commits into
software-mansion:mainfrom
Rasaboun:docs/tokenizer-encode-special-tokens
Nov 12, 2026
Merged

barhanc merged 3 commits into
software-mansion:mainfrom
Rasaboun:docs/tokenizer-encode-special-tokens

Commits

  1. Commits on Nov 12, 2026