-
Notifications
You must be signed in to change notification settings - Fork 14
[Tracing] Implement initial OTLP traces weblog tests #6363
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
zacharycmontoya
wants to merge
14
commits into
main
Choose a base branch
from
zach.montoya/weblog-traces-otlp
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
dd04bd9
Implement initial OTLP traces + trace stats with the .NET weblog exam…
zacharycmontoya ebbbb5a
Skip assertions for the Trace Metrics, as we don't have a spec for th…
zacharycmontoya 7fee5a2
Run formatter
zacharycmontoya 5f8b2a8
Remove "/api/v0.2/stats" subpath for the OpenTelemetry interface so w…
zacharycmontoya 221b031
Add manifest entries for languages
zacharycmontoya 86b284d
Merge branch 'main' into zach.montoya/weblog-traces-otlp
zacharycmontoya cacbab5
Remove DD_TRACE_OTEL_ENABLED=true from the APM_TRACING_OTLP weblog en…
zacharycmontoya 14b1995
Remove all helper methods and setup related to OTLP trace stats, sinc…
zacharycmontoya 57ed6bc
Add more comprehensive testing, especially around the http/json encod…
zacharycmontoya 6a1df07
Small refactoring of code and comments
zacharycmontoya f032d9d
Merge branch 'main' into zach.montoya/weblog-traces-otlp
zacharycmontoya 746148b
Add test case asserting that an incoming trace context's sampling dec…
zacharycmontoya 919b5ef
Move enums to dd_constants.py
zacharycmontoya a63042c
Merge branch 'main' into zach.montoya/weblog-traces-otlp
zacharycmontoya File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| # Unless explicitly stated otherwise all files in this repository are licensed under the the Apache License Version 2.0. | ||
| # This product includes software developed at Datadog (https://www.datadoghq.com/). | ||
| # Copyright 2024 Datadog, Inc. | ||
|
|
||
| import time | ||
| import re | ||
| from utils import weblog, interfaces, scenarios, features | ||
| from utils.dd_constants import SpanKind, StatusCode | ||
| from typing import Any | ||
| from collections.abc import Iterator | ||
|
|
||
|
|
||
| def _snake_to_camel(snake_key: str) -> str: | ||
| parts = snake_key.split("_") | ||
| return parts[0].lower() + "".join(p.capitalize() for p in parts[1:]) | ||
|
|
||
|
|
||
| def get_otlp_key(d: dict[str, Any] | None, snake_case_key: str, *, is_json: bool, default: Any = None) -> Any: # noqa: ANN401 | ||
| """Look up a field by its snake_case name when is_json is false, or its camelCase equivalent when is_json is true. | ||
| Fields must be camelCase for JSON Protobuf encoding. See https://opentelemetry.io/docs/specs/otlp/#json-protobuf-encoding | ||
| """ | ||
| if d is None: | ||
| return default | ||
| key = _snake_to_camel(snake_case_key) if is_json else snake_case_key | ||
| return d.get(key, default) | ||
|
|
||
|
|
||
| def get_keyvalue_generator(attributes: list[dict]) -> Iterator[tuple[str, Any]]: | ||
| for key_value in attributes: | ||
| if key_value["value"].get("string_value"): | ||
| yield key_value["key"], key_value["value"]["string_value"] | ||
| elif key_value["value"].get("stringValue"): | ||
| yield key_value["key"], key_value["value"]["stringValue"] | ||
| elif key_value["value"].get("bool_value"): | ||
| yield key_value["key"], key_value["value"]["bool_value"] | ||
| elif key_value["value"].get("boolValue"): | ||
| yield key_value["key"], key_value["value"]["boolValue"] | ||
| elif key_value["value"].get("int_value"): | ||
| yield key_value["key"], key_value["value"]["int_value"] | ||
| elif key_value["value"].get("intValue"): | ||
| yield key_value["key"], key_value["value"]["intValue"] | ||
| elif key_value["value"].get("double_value"): | ||
| yield key_value["key"], key_value["value"]["double_value"] | ||
| elif key_value["value"].get("doubleValue"): | ||
| yield key_value["key"], key_value["value"]["doubleValue"] | ||
| elif key_value["value"].get("array_value"): | ||
| yield key_value["key"], key_value["value"]["array_value"] | ||
| elif key_value["value"].get("arrayValue"): | ||
| yield key_value["key"], key_value["value"]["arrayValue"] | ||
| elif key_value["value"].get("kvlist_value"): | ||
| yield key_value["key"], key_value["value"]["kvlist_value"] | ||
| elif key_value["value"].get("kvlistValue"): | ||
| yield key_value["key"], key_value["value"]["kvlistValue"] | ||
| elif key_value["value"].get("bytes_value"): | ||
| yield key_value["key"], key_value["value"]["bytes_value"] | ||
| elif key_value["value"].get("bytesValue"): | ||
| yield key_value["key"], key_value["value"]["bytesValue"] | ||
| else: | ||
| raise ValueError(f"Unknown attribute value: {key_value['value']}") | ||
|
|
||
|
|
||
| # @scenarios.apm_tracing_e2e_otel | ||
| @features.otel_api | ||
| @scenarios.apm_tracing_otlp | ||
| class Test_Otel_Tracing_OTLP: | ||
| def setup_single_server_trace(self): | ||
| self.start_time_ns = time.time_ns() | ||
| self.req = weblog.get("/") | ||
| self.end_time_ns = time.time_ns() | ||
|
|
||
| def test_single_server_trace(self): | ||
| """Validates the required elements of the OTLP payload for a single trace""" | ||
| data = list(interfaces.open_telemetry.get_otel_spans(self.req)) | ||
|
|
||
| # Assert that there is only one OTLP request containing the desired server span | ||
| assert len(data) == 1 | ||
| request, content, span = data[0] | ||
|
|
||
| # Determine if JSON Protobuf Encoding was used for the OTLP request (rather than Binary Protobuf) | ||
| # We need to assert that we match the OTLP specification, which has some odd encoding rules when using JSON: https://opentelemetry.io/docs/specs/otlp/#json-protobuf-encoding | ||
| request_headers = {key.lower(): value for key, value in request.get("headers")} | ||
| is_json = request_headers.get("content-type") == "application/json" | ||
|
|
||
| # Assert that there is only one resource span (i.e. SDK) in the OTLP request | ||
| resource_spans = get_otlp_key(content, "resource_spans", is_json=is_json) | ||
| expected_key = _snake_to_camel("resource_spans") if is_json else "resource_spans" | ||
| assert resource_spans is not None, f"missing '{expected_key}' on content: {content}" | ||
| assert len(resource_spans) == 1, f"expected 1 resource span, got {len(resource_spans)}" | ||
| resource_span = resource_spans[0] | ||
|
|
||
| attributes = { | ||
| key_value["key"]: get_otlp_key(key_value["value"], "string_value", is_json=is_json) | ||
| for key_value in resource_span.get("resource").get("attributes") | ||
| } | ||
|
|
||
| # Assert that the resource attributes contain the service-level attributes and tracer-level attributes we expect | ||
| # TODO: Assert the following attributes: runtime-id, git.commit.sha, git.repository_url | ||
| assert attributes.get("service.name") == "weblog" | ||
| assert attributes.get("service.version") == "1.0.0" | ||
| assert ( | ||
| attributes.get("deployment.environment.name") == "system-tests" | ||
| or attributes.get("deployment.environment") == "system-tests" | ||
| ) | ||
| assert attributes.get("telemetry.sdk.name") == "datadog" | ||
| assert "telemetry.sdk.language" in attributes | ||
| assert "telemetry.sdk.version" in attributes | ||
|
|
||
| # Assert that the `traceId` and `spanId` JSON fields are valid case-insensitive hexadecimal strings, not base64-encoded strings as defined in the standard Protobuf JSON Mapping. | ||
| # See https://opentelemetry.io/docs/specs/otlp/#json-protobuf-encoding | ||
| # TODO: Assert against trace_id and span_id fields in the protobuf encoding as well | ||
| if is_json: | ||
| assert re.match(r"^[0-9a-fA-F]{32}$", span.get("traceId")), ( | ||
| f"traceId is not a valid case-insensitive hexadecimal string, got {span.get('traceId')}" | ||
| ) | ||
| assert re.match(r"^[0-9a-fA-F]{16}$", span.get("spanId")), ( | ||
| f"spanId is not a valid case-insensitive hexadecimal string, got {span.get('spanId')}" | ||
| ) | ||
|
|
||
| # Assert that the span fields match the expected values | ||
| span_start_time_ns = int(get_otlp_key(span, "start_time_unix_nano", is_json=is_json)) | ||
| span_end_time_ns = int(get_otlp_key(span, "end_time_unix_nano", is_json=is_json)) | ||
| assert span_start_time_ns >= self.start_time_ns | ||
| assert span_end_time_ns >= span_start_time_ns | ||
| assert span_end_time_ns <= self.end_time_ns | ||
|
|
||
| assert get_otlp_key(span, "name", is_json=is_json) | ||
| assert get_otlp_key(span, "kind", is_json=is_json) == SpanKind.SERVER.value | ||
| assert get_otlp_key(span, "attributes", is_json=is_json) is not None | ||
| assert ( | ||
| get_otlp_key(span, "status", is_json=is_json) is None | ||
| or get_otlp_key(span, "status", is_json=is_json).get("code") == StatusCode.STATUS_CODE_UNSET.value | ||
| ) | ||
|
|
||
| # Assert HTTP tags | ||
| # Convert attributes list to a dictionary, but for now only handle key_value objects with stringValue | ||
| span_attributes = dict(get_keyvalue_generator(get_otlp_key(span, "attributes", is_json=is_json))) | ||
| method = span_attributes.get("http.method") or span_attributes.get("http.request.method") | ||
| status_code = span_attributes.get("http.status_code") or span_attributes.get("http.response.status_code") | ||
| assert method == "GET", f"HTTP method is not GET, got {method}" | ||
| assert status_code is not None | ||
| assert int(status_code) == 200, f"HTTP status code is not 200, got {int(status_code)}" | ||
|
|
||
| def setup_unsampled_trace(self): | ||
| self.req = weblog.get("/", headers={"traceparent": "00-11111111111111110000000000000001-0000000000000001-00"}) | ||
|
|
||
| def test_unsampled_trace(self): | ||
| """Validates that the spans from a non-sampled trace are not exported.""" | ||
| data = list(interfaces.open_telemetry.get_otel_spans(self.req)) | ||
|
|
||
| # Assert that the span from this test case was not exported | ||
| assert len(data) == 0, f"Expected no weblog spans in the OTLP trace payload, got {data}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel that data should be properly deserialized by the proxy, WDYT ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer this approach where we add workarounds in order to access the request properly, because then we can still store the original contents of the request. And if there are any invalid fields that we try to assert against, it should be easy for the developer to debug this (the original request is saved to disk via the interface). Do you feel strongly that we should change this at the moment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes I do :) Let me explain :
The philosophy for those files is to have data ready to tests, and easy to read for humans. The result is expected to be something clear and functional, expunged from any shenanigans made for transport purpose.
Those transformations are made by the proxy, as long as :
For the data you're working one, they are already deserialized, they are far away from the original content. But the job is not fully done. So for instance what is done for v1 traces : https://github.com/DataDog/system-tests/blob/main/utils/proxy/traces/trace_v1.py
And I can see many similarities with your code 😉
Now, the needs for having the real raw request content (maybe with only a very basic deserialization) is definitly something we should have, if ever we need to investigate on the raw data. It's quite easy to do, and I can work on that if you'd like to have it.
But for the
jsonfiles, I'd like to keep the original philosophy, since it's a cornerstone to have tests easy to write, andeasynot too hard debug sessions 😄