fix: count literal special-token strings instead of crashing

[milangeorge2000]

Haystack PR — Fixes #12869

TiktokenCounter.count() used Encoding.encode(), which raises ValueError on text containing special-token markers like <|endoftext|>. Since allowed_special is empty at this call site, the strict check can never emit a special token — it only turns ordinary source text into a crash.

Changes

• Count with encode_ordinary() instead of encode(), matching the earlier #12853 fix for DocumentSplitter/RecursiveDocumentSplitter.
• Updated the _FakeEncoder test double (it now stubs encode_ordinary) and added a regression test with a literal <|endoftext|> marker.
• Added a release note per repo process.

Verification (observed)

• Real encoder: encode() raises ValueError: ... disallowed special token ...; encode_ordinary() returns 9 tokens for the same string.
• test/token_counters/: 45 passed, 1 skipped via Hatch.
• hatch run fmt: all checks passed.

[vercel]

Someone is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

Hi @milangeorge2000, thanks for your interest in contributing to Haystack! 🙏

⚠️ Issue #12869 is already being addressed by open pull request(s) #12871, #12876, #12879, #12888, #13009. Before opening a PR for an issue, please check whether a PR is already linked to it, and consider contributing to the existing PR instead. We may close duplicate PRs to keep the review queue manageable.

This is an automated message to help us keep the review queue healthy.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ milangeorge2000
❌ milan varghese george

---

milan varghese george seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[HaystackBot]

Hi @milangeorge2000, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

[julian-risch]

Thank you for your efforts! We're closing this PR because it duplicates #13009, which makes the same one-line fix to TiktokenCounter. This PR is also still a draft with the CLA unsigned, and its release note fails our backtick check.