Fixes flaky server metrics collector integration tests by TinaHeiligers · Pull Request #121469 · elastic/kibana

TinaHeiligers · 2021-12-16T23:11:54Z

Resolves #59234

Increases requestWaitDelay from 25 to 35
Adds statusCodes to metrics.requests assertion
Improves response times assertions.

Unit or functional tests were updated

…mes assertions

elasticmachine · 2021-12-17T18:40:44Z

Pinging @elastic/kibana-core (Team:Core)

TinaHeiligers · 2021-12-20T01:51:56Z

@elasticmachine merge upstream

…-test

pgayvallet · 2021-12-20T07:47:48Z

 import { setTimeout as setTimeoutPromise } from 'timers/promises';

-const requestWaitDelay = 25;
+const requestWaitDelay = 35;


Was this run against the flaky runner?

In the failing test, we are explicitly waiting until all the requests have at least been reaching the handler, via

kibana/src/core/server/metrics/integration_tests/server_collector.test.ts

Lines 97 to 102 in a7556f4

await hitSubject

.pipe(

filter((count) => count >= 2),

take(1)

)

.toPromise();

So I'm not quite sure just increase the delay where we were already waiting should really resolves the flakiness, because in theory, we should not get to the assertions because all requests have reached the server? So I'm suspecting the flakiness is appearing from elsewhere.

Maybe we should also be adding an await delay(requestWaitDelay); between await hitSubject... and let metrics = await collector.collect(); to some times for the HAPI internals?

Was this run against the flaky runner?

The buildkite flaky test runner doesn't run jest tests yet but our Jenkins runner does 😄
https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/2162/
The metrics integration tests passed 42/42 times on 7f89ebd

So I'm suspecting the flakiness is appearing from elsewhere

I'll dig deeper...

So, after some digging it seems like the flakiness is because the requests aren't always being triggered with supertest(hapiServer.listener).get('/disconnect).end() before the assertions.

The /disconnect router hangs on new Promise((resolve) => undefined) which is how we're mimicking the disconnection.

Even though the callback to .end is optional, I've added one anyway since we're not awaiting the request. That and the added delay should hopefully help stabilize the tests.

Flaky test runner job with these changes: https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/2163/
passed 10/10 times.

TinaHeiligers · 2021-12-21T02:02:00Z

@elasticmachine merge upstream

…-test

kibana-ci · 2021-12-21T03:20:59Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💔 Build #14406 failed c9a9760
💚 Build #14104 succeeded 7f89ebd
💚 Build #13957 succeeded c685643
💔 Build #13804 failed c685643
💔 Build #13781 failed c685643

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

kibanamachine · 2021-12-21T15:02:38Z

💔 Backport failed

Status	Branch	Result
✅	8.0
❌	7.17	Commit could not be cherrypicked due to conflicts
❌	7.16	Commit could not be cherrypicked due to conflicts

Successful backport PRs will be merged automatically after passing CI.

To backport manually run:
node scripts/backport --pr 121469

…1776) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> # Conflicts: # src/core/server/metrics/integration_tests/server_collector.test.ts

…1799) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> # Conflicts: # src/core/server/metrics/integration_tests/server_collector.test.ts

…1800) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> # Conflicts: # src/core/server/metrics/integration_tests/server_collector.test.ts

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Increases requestWaitDelay, improves metrics requests and response_ti…

c685643

…mes assertions

TinaHeiligers added 8.0.1 v8.0.0 v7.16.2 v7.17.0 v8.1.0 auto-backport Deprecated - use backport:version if exact versions are needed release_note:skip Skip the PR/issue when compiling release notes and removed 8.0.1 labels Dec 16, 2021

TinaHeiligers marked this pull request as ready for review December 17, 2021 18:40

TinaHeiligers requested a review from a team as a code owner December 17, 2021 18:40

TinaHeiligers added the Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// label Dec 17, 2021

Merge branch 'main' into kbn-59234-flaky-server-metrics-collector-int…

7f89ebd

…-test

pgayvallet reviewed Dec 20, 2021

View reviewed changes

Adds delay to wait for request to reach the server

c9a9760

Merge branch 'main' into kbn-59234-flaky-server-metrics-collector-int…

e2749af

…-test

pgayvallet approved these changes Dec 21, 2021

View reviewed changes

TinaHeiligers merged commit 8d621b7 into elastic:main Dec 21, 2021

TinaHeiligers deleted the kbn-59234-flaky-server-metrics-collector-int-test branch December 21, 2021 14:59

kibanamachine mentioned this pull request Dec 21, 2021

[8.0] Fixes flaky server metrics collector integration tests (#121469) #121776

Merged

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Dec 21, 2021

Fixes flaky server metrics collector integration tests (elastic#121469)

6c25971

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

TinaHeiligers mentioned this pull request Dec 21, 2021

[7.17] Fixes flaky server metrics collector integration tests (#121469) #121799

Merged

TinaHeiligers mentioned this pull request Dec 21, 2021

[7.16] Fixes flaky server metrics collector integration tests (#121469) #121800

Merged

patrykkopycinski pushed a commit to patrykkopycinski/kibana that referenced this pull request May 6, 2026

Fixes flaky server metrics collector integration tests (elastic#121469)

f6a418a

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Navigation Menu

Repository navigation

Fixes flaky server metrics collector integration tests - #121469

Fixes flaky server metrics collector integration tests#121469
TinaHeiligers merged 4 commits into
elastic:mainfrom
TinaHeiligers:kbn-59234-flaky-server-metrics-collector-int-test

TinaHeiligers commented Dec 16, 2021

Uh oh!

elasticmachine commented Dec 17, 2021

Uh oh!

TinaHeiligers commented Dec 20, 2021

Uh oh!

pgayvallet Dec 20, 2021

Uh oh!

TinaHeiligers Dec 20, 2021 •

edited

Loading

Uh oh!

TinaHeiligers Dec 21, 2021 •

edited

Loading

Uh oh!

TinaHeiligers commented Dec 21, 2021

Uh oh!

kibana-ci commented Dec 21, 2021

Uh oh!

kibanamachine commented Dec 21, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	await hitSubject
	.pipe(
	filter((count) => count >= 2),
	take(1)
	)
	.toPromise();

Conversation

TinaHeiligers commented Dec 16, 2021

Uh oh!

elasticmachine commented Dec 17, 2021

Uh oh!

TinaHeiligers commented Dec 20, 2021

Uh oh!

pgayvallet Dec 20, 2021

Choose a reason for hiding this comment

Uh oh!

TinaHeiligers Dec 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TinaHeiligers Dec 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TinaHeiligers commented Dec 21, 2021

Uh oh!

kibana-ci commented Dec 21, 2021

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

kibanamachine commented Dec 21, 2021

💔 Backport failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

TinaHeiligers Dec 20, 2021 •

edited

Loading

TinaHeiligers Dec 21, 2021 •

edited

Loading