Skip to content

Add unit tests for custom model endpoint with Anthropic thinking blocks#176

Open
okottorika wants to merge 1 commit intoboldsoftware:mainfrom
okottorika:fix-custom-model-thinking-test
Open

Add unit tests for custom model endpoint with Anthropic thinking blocks#176
okottorika wants to merge 1 commit intoboldsoftware:mainfrom
okottorika:fix-custom-model-thinking-test

Conversation

@okottorika
Copy link
Copy Markdown

Problem

The existing tests for issue #146 (custom Anthropic models with ThinkingLevel enabled) were integration tests requiring a real ANTHROPIC_API_KEY, making them impossible to run in CI or for most contributors.

Solution

This PR adds two unit tests using a mock SSE server to verify the custom model test endpoint behavior:

Tests Added

  1. TestCustomModelTestEndpointWithMockAnthropic — Verifies that responses with thinking blocks followed by text content are handled correctly. The mock server returns a realistic Anthropic SSE stream with a thinking block followed by a text block, confirming the endpoint correctly skips thinking blocks and finds the text response.

  2. TestCustomModelTestEndpointOnlyThinkingBlocks — Verifies that responses containing ONLY thinking blocks (no text) are properly rejected with the "empty response from model" error. This ensures the validation logic doesn't false-positive on thinking-only responses.

Context

Issue #146 was previously fixed in commit f01b163, which updated the response validation logic to iterate through all content blocks instead of only checking the first one. However, no test coverage existed without a real API key. These tests provide that coverage using a mock HTTP server.

Testing

  • All custom model tests pass locally
  • Tests run without requiring ANTHROPIC_API_KEY
  • No changes to production code — test coverage only

Fixes #146

@cla-bot cla-bot bot added the cla-signed label Apr 6, 2026
…g blocks

The existing tests for issue boldsoftware#146 were integration tests requiring a real
ANTHROPIC_API_KEY, making them impossible to run in CI or for most
contributors.

Add two unit tests using a mock SSE server:
- TestCustomModelTestEndpointWithMockAnthropic: verifies that responses
  with thinking blocks followed by text are handled correctly
- TestCustomModelTestEndpointOnlyThinkingBlocks: verifies that responses
  with only thinking blocks (no text) are properly rejected

Fixes boldsoftware#146

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
@okottorika okottorika force-pushed the fix-custom-model-thinking-test branch from 810a534 to aa2d260 Compare April 6, 2026 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix empty response error when using Anthropic models with ThinkingLevel

1 participant