[python] mark llmobs anthropic tests as missing feature#6484
Conversation
Yun-Kim
requested review from
juanjux and
sabrenner
and removed request for
a team
|
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b3536885c7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
manifests/python.yml
Outdated
| tests/ffe/test_exposures.py: v4.2.0-dev | ||
| tests/integration_frameworks/llm/anthropic/test_anthropic_apm.py::TestAnthropicApmMessages: v3.16.0 | ||
| tests/integration_frameworks/llm/anthropic/test_anthropic_llmobs.py::TestAnthropicLlmObsMessages: v3.16.0 | ||
| tests/integration_frameworks/llm/anthropic/test_anthropic_llmobs.py::TestAnthropicLlmObsMessages::test_create: missing_feature # ephemeral cache TTL metrics not yet released |
There was a problem hiding this comment.
These new entries mark the tests as missing_feature for all Python versions, but the inline reason says the cache TTL metrics are "not yet released"; this means CI will also treat current released versions as xfail, reducing signal and potentially masking real regressions in latest runs. Use a version-scoped declaration (with component_version) so only the unreleased range is deactivated.
Useful? React with 👍 / 👎.
## Description Ran ./scripts/update-system-tests-version.py to update system tests version following the missing_feature marking in DataDog/system-tests#6484 <!-- Provide an overview of the change and motivation for the change --> ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers --> Co-authored-by: yun.kim <yun.kim@datadoghq.com>
2 participants
Motivation
Marks anthropic llmobs python tests as missing feature to reflect the incoming cache ttl metrics from DataDog/dd-trace-py#16843. Once this is merged and nodejs is also adjusted accordingly, we'll revert this.
Changes
Workflow
🚀 Once your PR is reviewed and the CI green, you can merge it!
🛟 #apm-shared-testing 🛟
Reviewer checklist
tests/ormanifests/is modified ? I have the approval from R&P teambuild-XXX-imagelabel is present