Skip to content

fix: count literal special-token strings instead of crashing - #13014

Closed
milangeorge2000 wants to merge 2 commits into
deepset-ai:mainfrom
milangeorge2000:fix/12869-tiktoken-special-tokens
Closed

milangeorge2000 wants to merge 2 commits into
deepset-ai:mainfrom
milangeorge2000:fix/12869-tiktoken-special-tokens

Merge branch 'main' into fix/12869-tiktoken-special-tokens

d486999
Select commit
Loading
Failed to load commit list.
Sign in for the full log view