feat(go): Enable flag evaluation metrics E2E tests for Go; fix reason=static#6410
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 14 commits intomainfrom Mar 12, 2026
Merged
Conversation
Contributor
|
|
|
✅ Tests 🎉 All green!❄️ No new flaky tests detected 🔗 Commit SHA: 0951079 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback! |
leoromanovsky
requested review from
brettlangdon,
claponcet,
manuel-alvarez-alvarez,
r1viollet,
sameerank,
typotter and
xlamorlette-datadog
and removed request for
a team
…test The Go weblog was calling ofClient.Object() for all evaluations, ignoring the variationType field. This meant type conversion errors could never occur, unlike Python/Node.js which dispatch to the type-specific methods (BooleanValue, StringValue, etc.). Fix the Go weblog to dispatch based on variationType, matching the behavior of other language weblogs. Add Test_FFE_Eval_Metric_Type_Mismatch: configures a STRING flag but evaluates it as BOOLEAN, triggering a type conversion error that happens after the core evaluate() returns. This test would fail with the old evaluate()-level metric recording (which would see targeting_match / no error) and only passes when metrics are recorded via a Finally hook (which sees error / type_mismatch).
Only Go supports flag evaluation metrics via OTel so far. Without this, the test file runs for all FFE-enabled languages and fails.
Replace hardcoded time.sleep(25) in each test setup with agent_interface_timeout=30 on the FFE scenario. The container shutdown flushes metrics; the timeout gives the agent time to receive and process them.
Assert that feature_flag.result.allocation_key tag is present with value "default-allocation" on successful flag evaluations.
- Enable tests/ffe/test_flag_eval_metrics.py for PHP (>=1.16.0) and Node.js (express4 v6.0.0-pre)
- Fix reason assertion: UFC engine returns AssignmentReason::Static for a 100%
catch-all allocation (rules:[], splits:[{shards:[]}]), not TargetingMatch
- Add type annotations to test helpers (mypy compliance)
leoromanovsky
force-pushed
the
leo.romanovsky/ffe-eval-metrics
branch
from
d342bb1 to
a33486a
Compare
leoromanovsky
changed the title
…=static
- Enable tests/ffe/test_flag_eval_metrics.py for Go only (PHP and Node.js remain missing_feature)
- Fix reason assertion: UFC engine returns AssignmentReason::Static for a 100%
catch-all allocation (rules:[], splits:[{shards:[]}]), not TargetingMatch
- Add type annotations to test helpers (mypy compliance)
leoromanovsky
changed the title
gh-worker-dd-devflow-36fce6
bot
added
mergequeue-status: removed
and removed
mergequeue-status: waiting
labels
Contributor
Author
|
/merge |
|
View all feedbacks in Devflow UI.
This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
The expected merge time in
|
gh-worker-dd-mergequeue-cf854d
bot
merged commit 93ab576
into
main
2721 of 2726 checks passed
gh-worker-dd-mergequeue-cf854d
bot
deleted the
leo.romanovsky/ffe-eval-metrics
branch
gh-worker-dd-devflow-36fce6
bot
added
mergequeue-status: done
and removed
mergequeue-status: in_progress
labels
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Per the RFC "Flag evaluations tracking for APM tracers": collect a
feature_flag.evaluationsOTel counter metric on each flag evaluation to track SDK usage. These system tests validate the end-to-end pipeline: evaluation in the weblog → OTel SDK aggregation → OTLP export to agent → agent forwards to proxy → system tests assert oninterfaces.agent.get_metrics().Changes
tests/ffe/test_flag_eval_metrics.py: 5 E2E test classes for Go:Test_FFE_Eval_Metric_Basic: metric exists with correctfeature_flag.key,variant,reason=static,allocation_keytagsTest_FFE_Eval_Metric_Count: same flag evaluated 5× → metric count ≥ 5Test_FFE_Eval_Metric_Different_Flags: two flags → two separate metric seriesTest_FFE_Eval_Metric_Error: non-existent flag →reason=error,error.type=flag_not_foundTest_FFE_Eval_Metric_Type_Mismatch: STRING flag evaluated as BOOLEAN →reason=error,error.type=type_mismatchmanifests/golang.yml: Enable metrics tests for Go atv2.7.0-dev.missing_feature(pending tracer implementation).utils/_context/_scenarios/__init__.py: AddDD_METRICS_OTEL_ENABLED=true,OTEL_EXPORTER_OTLP_METRICS_ENDPOINT, andagent_interface_timeout=30to the FFE scenario.utils/build/docker/golang/app/_shared/common/ffe.go: Dispatch/ffebyvariationTypeinstead of always callingofClient.Object(), enabling type mismatch errors.Decisions
reason=staticnottargeting_match. The UFC engine returnsAssignmentReason::Staticfor a 100% catch-all allocation (emptyrulesarray, single split withshards:[]). The test fixtures use this shape, so the assertion reflects the actual engine output.agent_interface_timeout=30at scenario level. The OTLP pipeline (OTel SDK export + agent flush + proxy buffer) needs up to 30s to propagate. Moving the wait to the scenario level viaagent_interface_timeoutreplaces per-testsleep()calls.OTLP metrics endpoint direct to agent. The FFE scenario's
DD_TRACE_AGENT_URLpoints to the proxy, which has no OTLP receiver.OTEL_EXPORTER_OTLP_METRICS_ENDPOINTis set directly toagent:4318/v1/metricsto bypass the proxy for metric export. The agent then forwards processed metrics to/api/v2/series, where system tests capture them.