Skip to content

Add unit tests for the Cohere buildCohereCountTokensText helper #4

Description

@deepintai

Context

buildCohereCountTokensText flattens Responses messages into a single plain-text payload for Cohere tokenization. It handles several message shapes (plain text, content blocks, reasoning summaries) but has no dedicated unit tests, and there is no count_tokens_test.go next to the implementation yet.

What to do

Create deepintshield_server/core/providers/cohere/count_tokens_test.go (using the same style as rerank_test.go) with cases for:

  1. A single text message.
  2. Multiple sequential messages.
  3. An empty message slice (must not panic; returns empty string).
  4. Messages whose content is a list of content blocks containing text.
  5. Messages carrying reasoning summaries.
  6. Mixed content (text + blocks + reasoning).
  7. Messages with nil fields.
  8. Messages with empty/whitespace strings (verify trimming behavior).

Acceptance criteria

  • New file deepintshield_server/core/providers/cohere/count_tokens_test.go.
  • All 8 cases present and passing via go test ./core/providers/cohere/....
  • Uses standard Go testing + testify assertions, matching rerank_test.go.
  • Each case asserts the exact expected flattened string (or that it is empty where appropriate).
  • No production-code changes required.

File pointers

  • Function under test: deepintshield_server/core/providers/cohere/count_tokens.go:89 (buildCohereCountTokensText)
  • Related schemas: deepintshield_server/core/providers/cohere/count_tokens.go (request/response types near the top)
  • Test-style reference: deepintshield_server/core/providers/cohere/rerank_test.go

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions