Skip to content

fix: count literal special-token strings in TiktokenCounter - #12879

Closed
burakeyler wants to merge 1 commit into
deepset-ai:mainfrom
burakeyler:fix/tiktoken-counter-special-tokens
Closed

fix: count literal special-token strings in TiktokenCounter#12879
burakeyler wants to merge 1 commit into
deepset-ai:mainfrom
burakeyler:fix/tiktoken-counter-special-tokens

fix: count literal special-token strings in TiktokenCounter

71feaf3
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

1 warning
limit
succeeded Sep 23, 2026 in 8s