Add unit tests for custom model endpoint with Anthropic thinking blocks#176
Open
okottorika wants to merge 1 commit intoboldsoftware:mainfrom
Open
Add unit tests for custom model endpoint with Anthropic thinking blocks#176okottorika wants to merge 1 commit intoboldsoftware:mainfrom
okottorika wants to merge 1 commit intoboldsoftware:mainfrom
Conversation
…g blocks The existing tests for issue boldsoftware#146 were integration tests requiring a real ANTHROPIC_API_KEY, making them impossible to run in CI or for most contributors. Add two unit tests using a mock SSE server: - TestCustomModelTestEndpointWithMockAnthropic: verifies that responses with thinking blocks followed by text are handled correctly - TestCustomModelTestEndpointOnlyThinkingBlocks: verifies that responses with only thinking blocks (no text) are properly rejected Fixes boldsoftware#146 Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
810a534 to
aa2d260
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The existing tests for issue #146 (custom Anthropic models with
ThinkingLevelenabled) were integration tests requiring a realANTHROPIC_API_KEY, making them impossible to run in CI or for most contributors.Solution
This PR adds two unit tests using a mock SSE server to verify the custom model test endpoint behavior:
Tests Added
TestCustomModelTestEndpointWithMockAnthropic — Verifies that responses with thinking blocks followed by text content are handled correctly. The mock server returns a realistic Anthropic SSE stream with a thinking block followed by a text block, confirming the endpoint correctly skips thinking blocks and finds the text response.
TestCustomModelTestEndpointOnlyThinkingBlocks — Verifies that responses containing ONLY thinking blocks (no text) are properly rejected with the "empty response from model" error. This ensures the validation logic doesn't false-positive on thinking-only responses.
Context
Issue #146 was previously fixed in commit f01b163, which updated the response validation logic to iterate through all content blocks instead of only checking the first one. However, no test coverage existed without a real API key. These tests provide that coverage using a mock HTTP server.
Testing
ANTHROPIC_API_KEYFixes #146