Skip to content

docs(nlp): state that Tokenizer.encode adds no special tokens - #1633

Merged
barhanc merged 3 commits into
software-mansion:mainfrom
Rasaboun:docs/tokenizer-encode-special-tokens
Nov 12, 2026
Merged

barhanc merged 3 commits into
software-mansion:mainfrom
Rasaboun:docs/tokenizer-encode-special-tokens

docs(nlp): apply the encode special-token fix to the 0.10.x and 0.11.…

f7710ce
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

1 notice
check
succeeded Nov 12, 2026 in 7m 7s