Skip to content

feat(openfeature): implement flag evaluation metrics#7993

Merged
dd-oleksii merged 1 commit into
masterfrom
oleksii/jj-lwzwlonrxrpr
Apr 28, 2026
Merged

feat(openfeature): implement flag evaluation metrics#7993
dd-oleksii merged 1 commit into
masterfrom
oleksii/jj-lwzwlonrxrpr

Conversation

@dd-oleksii
Copy link
Copy Markdown
Member

@dd-oleksii dd-oleksii commented Apr 14, 2026

What does this PR do?

Add an OpenFeature finally hook that emits a feature_flag.evaluations OTel counter for every flag evaluation, including error and short-circuit cases. The hook is a no-op when otelMetricsEnabled is false.

Motivation

FFL-1944

Additional Notes

Tested locally against system-tests and it passes except a couple of cases affected by FFL-2064.

@dd-oleksii dd-oleksii requested a review from a team as a code owner April 14, 2026 11:30
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 14, 2026

Overall package size

Self size: 5.56 MB
Deduped: 6.4 MB
No deduping: 6.4 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.1 | 82.56 kB | 817.39 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@datadog-prod-us1-5
Copy link
Copy Markdown

datadog-prod-us1-5 Bot commented Apr 14, 2026

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 31.99% (-27.39%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 43bb33c | Docs | Datadog PR Page | Give us feedback!

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 14, 2026

Codecov Report

❌ Patch coverage is 24.13793% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.50%. Comparing base (df1f326) to head (43bb33c).
⚠️ Report is 9 commits behind head on master.

Files with missing lines Patch % Lines
...ages/dd-trace/src/openfeature/eval-metrics-hook.js 22.22% 21 Missing ⚠️
...ages/dd-trace/src/openfeature/flagging_provider.js 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7993      +/-   ##
==========================================
- Coverage   73.70%   73.50%   -0.21%     
==========================================
  Files         783      784       +1     
  Lines       36369    36398      +29     
==========================================
- Hits        26805    26753      -52     
- Misses       9564     9645      +81     
Flag Coverage Δ
aiguard-macos 36.38% <ø> (-0.09%) ⬇️
aiguard-ubuntu 36.49% <ø> (-0.09%) ⬇️
aiguard-windows 36.29% <ø> (-0.09%) ⬇️
apm-capabilities-tracing-macos 48.12% <24.13%> (-0.03%) ⬇️
apm-capabilities-tracing-ubuntu-active 48.14% <24.13%> (-0.03%) ⬇️
apm-capabilities-tracing-ubuntu-latest 48.11% <24.13%> (-0.03%) ⬇️
apm-capabilities-tracing-ubuntu-maintenance 48.14% <24.13%> (-0.02%) ⬇️
apm-capabilities-tracing-ubuntu-oldest 48.13% <24.13%> (-0.03%) ⬇️
apm-capabilities-tracing-windows 47.90% <24.13%> (-0.08%) ⬇️
apm-integrations-child-process 36.03% <ø> (-0.10%) ⬇️
apm-integrations-couchbase-18 35.04% <ø> (-0.09%) ⬇️
apm-integrations-couchbase-eol 35.09% <ø> (-0.09%) ⬇️
apm-integrations-oracledb 35.10% <ø> (-0.09%) ⬇️
appsec-express 52.73% <ø> (-0.07%) ⬇️
appsec-fastify 49.21% <ø> (-0.07%) ⬇️
appsec-graphql 49.50% <ø> (-0.07%) ⬇️
appsec-kafka 41.97% <ø> (-0.08%) ⬇️
appsec-ldapjs 41.31% <ø> (-0.08%) ⬇️
appsec-lodash 41.34% <ø> (-0.08%) ⬇️
appsec-macos 56.75% <ø> (-0.07%) ⬇️
appsec-mongodb-core 45.63% <ø> (-0.07%) ⬇️
appsec-mongoose 46.51% <ø> (-0.07%) ⬇️
appsec-mysql 48.69% <ø> (-0.07%) ⬇️
appsec-node-serialize 40.51% <ø> (-0.08%) ⬇️
appsec-passport 44.52% <ø> (-0.08%) ⬇️
appsec-postgres 48.28% <ø> (-0.17%) ⬇️
appsec-sourcing 40.01% <ø> (-0.08%) ⬇️
appsec-stripe 42.24% <ø> (-0.08%) ⬇️
appsec-template 40.68% <ø> (-0.08%) ⬇️
appsec-ubuntu 56.83% <ø> (-0.07%) ⬇️
appsec-windows 56.63% <ø> (-0.07%) ⬇️
instrumentations-instrumentation-bluebird 29.75% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-body-parser 37.64% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-child_process 35.41% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-cookie-parser 31.68% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-express 31.90% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-express-mongo-sanitize 31.80% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-express-session 37.27% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-fs 29.42% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-generic-pool 30.53% <ø> (ø)
instrumentations-instrumentation-http 36.88% <ø> (-0.08%) ⬇️
instrumentations-instrumentation-knex 29.72% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-light-my-request 37.20% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-mongoose 30.82% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-multer 37.41% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-mysql2 35.38% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-passport 41.17% <ø> (?)
instrumentations-instrumentation-passport-http 40.87% <ø> (-0.08%) ⬇️
instrumentations-instrumentation-passport-local 41.38% <ø> (-0.08%) ⬇️
instrumentations-instrumentation-pg 34.91% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-promise 29.68% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-promise-js 29.69% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-q 29.72% <ø> (-0.09%) ⬇️
instrumentations-instrumentation-url 29.68% <ø> (-0.10%) ⬇️
instrumentations-instrumentation-when 29.70% <ø> (-0.10%) ⬇️
llmobs-ai 38.35% <ø> (-0.09%) ⬇️
llmobs-anthropic 37.82% <ø> (-0.09%) ⬇️
llmobs-bedrock 37.06% <ø> (-0.08%) ⬇️
llmobs-google-genai 37.49% <ø> (-0.08%) ⬇️
llmobs-langchain 36.97% <ø> (-0.07%) ⬇️
llmobs-openai 41.18% <ø> (-0.08%) ⬇️
llmobs-vertex-ai 37.67% <ø> (-0.09%) ⬇️
platform-core 30.03% <ø> (ø)
platform-esbuild 32.83% <ø> (ø)
platform-instrumentations-misc 40.01% <ø> (ø)
platform-shimmer 35.72% <ø> (ø)
platform-unit-guardrails 31.39% <ø> (ø)
platform-webpack 20.73% <ø> (ø)
plugins-azure-durable-functions 25.36% <ø> (ø)
plugins-azure-event-hubs 25.51% <ø> (ø)
plugins-azure-service-bus 24.92% <ø> (ø)
plugins-bullmq 40.55% <ø> (-0.21%) ⬇️
plugins-cassandra 35.24% <ø> (-0.09%) ⬇️
plugins-cookie 26.47% <ø> (ø)
plugins-cookie-parser 26.28% <ø> (ø)
plugins-crypto 25.70% <ø> (ø)
plugins-dd-trace-api 35.34% <ø> (-0.09%) ⬇️
plugins-express-mongo-sanitize 26.42% <ø> (ø)
plugins-express-session 26.24% <ø> (ø)
plugins-fastify 39.25% <ø> (-0.09%) ⬇️
plugins-fetch 35.75% <ø> (-0.09%) ⬇️
plugins-fs 35.63% <ø> (-0.10%) ⬇️
plugins-generic-pool 25.40% <ø> (ø)
plugins-google-cloud-pubsub 42.99% <ø> (-0.08%) ⬇️
plugins-grpc 38.01% <ø> (-0.09%) ⬇️
plugins-handlebars 26.46% <ø> (ø)
plugins-hapi 37.25% <ø> (-0.09%) ⬇️
plugins-hono 37.50% <ø> (-0.09%) ⬇️
plugins-ioredis 35.68% <ø> (?)
plugins-knex 26.14% <ø> (ø)
plugins-langgraph ?
plugins-ldapjs 24.02% <ø> (ø)
plugins-light-my-request 25.88% <ø> (ø)
plugins-limitd-client 29.98% <ø> (?)
plugins-lodash 25.47% <ø> (ø)
plugins-mariadb 36.55% <ø> (-0.09%) ⬇️
plugins-memcached 35.33% <ø> (-0.09%) ⬇️
plugins-microgateway-core 36.34% <ø> (-0.09%) ⬇️
plugins-modelcontextprotocol-sdk ?
plugins-moleculer 38.03% <ø> (-0.09%) ⬇️
plugins-mongodb 36.50% <ø> (-0.09%) ⬇️
plugins-mongodb-core 36.15% <ø> (-0.09%) ⬇️
plugins-mongoose 36.10% <ø> (ø)
plugins-multer 26.24% <ø> (ø)
plugins-mysql 36.42% <ø> (+0.04%) ⬆️
plugins-mysql2 36.40% <ø> (-0.09%) ⬇️
plugins-node-serialize 26.51% <ø> (ø)
plugins-opensearch 35.00% <ø> (-0.09%) ⬇️
plugins-passport-http 26.30% <ø> (ø)
plugins-pino 31.78% <ø> (-0.09%) ⬇️
plugins-postgres 34.41% <ø> (-0.09%) ⬇️
plugins-process 25.70% <ø> (ø)
plugins-pug 26.47% <ø> (ø)
plugins-redis 35.89% <ø> (-0.09%) ⬇️
plugins-router 39.90% <ø> (-0.09%) ⬇️
plugins-sequelize 25.18% <ø> (ø)
plugins-test-and-upstream-amqp10 35.65% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-amqplib 40.82% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-apollo 36.51% <ø> (-0.08%) ⬇️
plugins-test-and-upstream-avsc 35.41% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-bunyan 31.13% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-connect 37.84% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-graphql 37.18% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-koa 37.45% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-protobufjs 35.63% <ø> (-0.09%) ⬇️
plugins-test-and-upstream-rhea 40.92% <ø> (-0.09%) ⬇️
plugins-undici 36.51% <ø> (-0.09%) ⬇️
plugins-url 25.70% <ø> (ø)
plugins-valkey 35.24% <ø> (-0.09%) ⬇️
plugins-vm 25.70% <ø> (ø)
plugins-winston 31.58% <ø> (-0.09%) ⬇️
plugins-ws 38.97% <ø> (-0.09%) ⬇️
profiling-macos 37.82% <ø> (-0.09%) ⬇️
profiling-ubuntu 37.99% <ø> (-0.54%) ⬇️
profiling-windows 39.35% <ø> (-0.09%) ⬇️
serverless-azure-functions-client 25.25% <ø> (ø)
serverless-azure-functions-eventhubs 25.25% <ø> (ø)
serverless-azure-functions-servicebus 25.25% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

sameerank
sameerank previously approved these changes Apr 15, 2026
Copy link
Copy Markdown

@sameerank sameerank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI we are currently ignoring this following system test for node: Test_FFE_Eval_Targeting_Key_Optional

https://github.com/DataDog/system-tests/blob/7d79f7af78d86a8e43723e6c27854ca368cfd70c/manifests/nodejs.yml#L1579

At some point we may want to think about making it match the other tracers

Comment thread packages/dd-trace/src/openfeature/eval-metrics-hook.js
Comment thread packages/dd-trace/src/openfeature/eval-metrics-hook.js
@dd-oleksii
Copy link
Copy Markdown
Member Author

FYI we are currently ignoring this following system test for node: Test_FFE_Eval_Targeting_Key_Optional

Yeah. My understanding is that this should have been fixed in FFL-1730 but it was not

Comment thread packages/dd-trace/src/openfeature/eval-metrics-hook.js
@dd-oleksii dd-oleksii requested a review from rochdev April 16, 2026 11:31
@dd-oleksii dd-oleksii force-pushed the oleksii/jj-lwzwlonrxrpr branch from 7605a55 to 873af1c Compare April 16, 2026 11:59
@dd-oleksii dd-oleksii requested a review from a team as a code owner April 16, 2026 11:59
@dd-oleksii dd-oleksii requested review from greghuels and leoromanovsky and removed request for a team April 16, 2026 11:59
@dd-oleksii
Copy link
Copy Markdown
Member Author

(rebased on latest master to run system-tests)

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Apr 16, 2026

Benchmarks

Benchmark execution time: 2026-04-27 18:18:30

Comparing candidate commit 43bb33c in PR branch oleksii/jj-lwzwlonrxrpr with baseline commit df1f326 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1347 metrics, 97 unstable metrics.

leoromanovsky
leoromanovsky previously approved these changes Apr 16, 2026
Copy link
Copy Markdown
Contributor

@leoromanovsky leoromanovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid tests.

@dd-oleksii dd-oleksii dismissed rochdev’s stale review April 17, 2026 20:07

This comment is the last thing blocking the merge and @rochdev is unresponsive on both github and slack.

FFE team's current consensus is that it is a misreading of the code and does not warrant the block. We also use the same approach in Go, Java, Python, and .NET tracers.

I'm dismissing the review to unblock the progress. I'll wait for Monday eve before merging unless I hear any objections 🙏

Comment thread packages/dd-trace/src/openfeature/eval-metrics-hook.js
@dd-oleksii dd-oleksii requested a review from rochdev April 20, 2026 16:35
Add an OpenFeature `finally` hook that emits a `feature_flag.evaluations`
OTel counter for every flag evaluation, including error and short-circuit
cases. The hook is a no-op when `otelMetricsEnabled` is false.
@dd-oleksii dd-oleksii dismissed stale reviews from leoromanovsky and sameerank via 43bb33c April 27, 2026 18:06
@dd-oleksii dd-oleksii force-pushed the oleksii/jj-lwzwlonrxrpr branch from 873af1c to 43bb33c Compare April 27, 2026 18:06
@dd-oleksii
Copy link
Copy Markdown
Member Author

(rebased to fix merge conflict in tests — no code changes)

@dd-oleksii dd-oleksii merged commit 6044071 into master Apr 28, 2026
1179 of 1190 checks passed
@dd-oleksii dd-oleksii deleted the oleksii/jj-lwzwlonrxrpr branch April 28, 2026 15:29
dd-octo-sts Bot pushed a commit that referenced this pull request Apr 29, 2026
Add an OpenFeature `finally` hook that emits a `feature_flag.evaluations`
OTel counter for every flag evaluation, including error and short-circuit
cases. The hook is a no-op when `otelMetricsEnabled` is false.
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Apr 29, 2026
BridgeAR pushed a commit that referenced this pull request Apr 30, 2026
Add an OpenFeature `finally` hook that emits a `feature_flag.evaluations`
OTel counter for every flag evaluation, including error and short-circuit
cases. The hook is a no-op when `otelMetricsEnabled` is false.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants