fix: count literal special tokens in TiktokenCounter

[yusei21]

Related Issues

• Fixes TiktokenCounter raises ValueError on literal special-token strings like <|endoftext|> #12869

Proposed Changes

Use encode_ordinary() in TiktokenCounter so literal strings resembling tiktoken special tokens in messages and tool descriptions are counted as ordinary text. Add unit coverage, a real-encoder integration regression test, and a release note.

How did you test it?

• hatch run test:unit test/token_counters/test_tiktoken_counter.py -q — 11 passed.
• hatch run test:integration test/token_counters/test_tiktoken_counter.py -q — 4 passed.
• hatch run fmt — passed.

Notes for the reviewer

This PR was prepared with an AI assistant. Focused tests and formatting checks were run; please review the changes before merging.

[vercel]

@yusei21 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

Hi @yusei21, thanks for your interest in contributing to Haystack! 🙏

⚠️ Issue #12869 is already being addressed by open pull request(s) #12871, #12876, #12879, #12888, #13009. Before opening a PR for an issue, please check whether a PR is already linked to it, and consider contributing to the existing PR instead. We may close duplicate PRs to keep the review queue manageable.

This is an automated message to help us keep the review queue healthy.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}