Skip to content

Per meterid metrics distribution.percentiles-histogram expose incorrectly active metrics #48410

@jtorkkel

Description

@jtorkkel

When using explicit serviceLevelObjectiveBoundaries in prometheus metrics like

management.prometheus.metrics.export.enabled=true
management.metrics.distribution.slo.http=30,50,100,150,200,300,500,700,1s,1500,2s,3s,4s,5s,7s,10s,15s,20s,30s

Then only "http_server_requests_seconds_bucket" and "http_client_requests_seconds_bucket" are correctly created.

http_server_requests_seconds_bucket{app="test-v1-v",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/test",le="0.03"} 0
http_server_requests_seconds_bucket{app="test-v1-v",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/test",le="0.05"} 0

However when using "percentiles-histogram" the output contains also histogram 28 buckets for "active" metrics (from 120ms to 2h) like "http_server_requests_active_seconds_bucket" and "http_client_requests_active_seconds_bucket"

http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="120.0"} 1
http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="137.438953471"} 1
.......
http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="7200.0"} 1
http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="+Inf"} 1

Trying to exclude "active" is not possible. Limiting buckets to metric id "http.server.requests" still include active "http_server_requests_active_seconds_bucket" buckets.

management.metrics.distribution.percentiles-histogram.http.server.requests=true
management.metrics.distribution.minimum-expected-value.http=40
management.metrics.distribution.maximum-expected-value.http=30s

And not possible to add "seconds", because "seconds" is not metrics id, it is added automatically for legacy histograms

management.metrics.distribution.percentiles-histogram.http.server.requests.seconds=true # not working
management.metrics.distribution.minimum-expected-value.http=40
management.metrics.distribution.maximum-expected-value.http=30s

Now slo and percentiles-histogram behave inconsistently.

It should be possible to control if active metrics are included into the buckets.
Or have possibility to exclude active metrics.

Tested in:
SpringBoot version: 3.5.8
Windows 11
Java 17

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions