[Ceph] Add Integration Package with Pool Disk data stream by harnish-crest-data · Pull Request #5070 · elastic/integrations

harnish-crest-data · 2023-01-20T12:32:01Z

Enhancement

What does this PR do?

Added 1 data stream (Pool Disk metrics).
Added data collection logic for the data streams.
Added the ingest pipeline for the data streams.
Mapped fields according to the ECS schema and added Fields metadata in the appropriate yaml files.
Added system test cases for the data stream.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Clone integrations repo.
Install elastic-package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/ceph directory.
Run the following command to run tests.
elastic-package test

Related issues

elasticmachine · 2023-01-20T12:35:08Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-02-07T13:13:42.055+0000
Duration: 16 min 7 sec

Test stats 🧪

Test	Results
Failed	0
Passed	26
Skipped	0
Total	26

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

kush-elastic · 2023-01-20T12:41:10Z

/test

elasticmachine · 2023-01-20T12:54:33Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`5/5`)	💚
Files	100.0% (`5/5`)	💚 6.25
Classes	100.0% (`5/5`)	💚 6.25
Methods	100.0% (`41/41`)	💚 13.869
Lines	92.57% (`299/323`)	👍 6.644
Conditionals	100.0% (`0/0`)	💚

kush-elastic

LGTM!

agithomas

{\"stored\":2142673,\"objects\":4,\"kb_used\":6336,\"bytes_used\":6488064,\"percent_used\":8.4362458437681198e-05,\"max_avail\":25633505280}},

can we have these fields recorded and mapped ?

agithomas · 2023-01-30T11:41:04Z

+          description: Available bytes of the pool.
+          metric_type: gauge
+          unit: byte
+        - name: id


please change to pool_id

In this data stream, we have pool_disk as parent field. So the current output will be like ceph.pool_disk.id. Based on the suggested scenario, we will have ceph.pool_disk.pool_id. So I think this change would make pool redundant. WDYT?

Please keept is pool_id

agithomas · 2023-01-30T11:41:16Z

+        - name: id
+          type: long
+          description: Id of the pool.
+        - name: name


please change to pool_name

In this data stream, we have pool_disk as parent field. So the current output will be like ceph.pool_disk.name. Based on the suggested scenario, we will have ceph.pool_disk.pool_name. So I think this change would make pool redundant. WDYT?

Please keep it as pool_name

harnish-crest-data · 2023-01-30T12:25:30Z

{\"stored\":2142673,\"objects\":4,\"kb_used\":6336,\"bytes_used\":6488064,\"percent_used\":8.4362458437681198e-05,\"max_avail\":25633505280}},

can we have these fields recorded and mapped ?

Based on the response we have ingested objects, bytes_used and max_avail fields. For the kb_used and bytes_used both fields are having redundant data. Hence we can drop kb_used. We will include stored and percent_used fields in data colection.

…into package-ceph_pool_disk

agithomas

LGTM

elasticmachine · 2023-02-08T11:28:30Z

Package ceph - 0.5.0 containing this change is available at https://epr.elastic.co/search?package=ceph

* Add pool disk data stream * update changelog.yml * remove dashboard * update readme

Add pool disk data stream

9ebe066

update changelog.yml

79c4992

kush-elastic self-requested a review January 20, 2023 12:39

kush-elastic assigned harnish-crest-data Jan 20, 2023

kush-elastic added enhancement New feature or request Integration:ceph Ceph Team:Service-Integrations Label for the Observability Service Integrations team labels Jan 20, 2023

kush-elastic requested review from SubhrataK, agithomas and lalit-satapathy January 20, 2023 12:41

kush-elastic marked this pull request as ready for review January 20, 2023 12:42

kush-elastic approved these changes Jan 24, 2023

View reviewed changes

harnish-crest-data mentioned this pull request Jan 27, 2023

[Ceph] Add Integration Package with OSD Pool Stats data stream #5128

Merged

4 tasks done

agithomas reviewed Jan 30, 2023

View reviewed changes

remove dashboard

692d708

update readme

2dfb88a

harnish-crest-data requested review from agithomas and removed request for SubhrataK and lalit-satapathy February 3, 2023 05:22

kush-elastic requested review from SubhrataK and lalit-satapathy February 3, 2023 05:23

agithomas reviewed Feb 4, 2023

View reviewed changes

Comment thread packages/ceph/data_stream/pool_disk/fields/fields.yml

harnish-crest-data added 2 commits February 7, 2023 18:35

Merge branch 'main' of https://github.com/harnish-elastic/integrations …

b73d7f1

…into package-ceph_pool_disk

Merge branch 'main' of https://github.com/harnish-elastic/integrations …

ff17161

…into package-ceph_pool_disk

harnish-crest-data requested a review from a team as a code owner February 7, 2023 13:13

agithomas approved these changes Feb 7, 2023

View reviewed changes

kush-elastic merged commit 1784d35 into elastic:main Feb 8, 2023

harnish-crest-data mentioned this pull request Feb 8, 2023

[Ceph] Add Integration Package with Cluster Status data stream #5216

Merged

4 tasks done

orestisfl pushed a commit to orestisfl/integrations that referenced this pull request May 15, 2026

[Ceph] Add Integration Package with Pool Disk data stream (elastic#5070)

4b94afb

* Add pool disk data stream * update changelog.yml * remove dashboard * update readme

Conversation

harnish-crest-data commented Jan 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

How to test this PR locally

Related issues

Uh oh!

elasticmachine commented Jan 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

kush-elastic commented Jan 20, 2023

Uh oh!

elasticmachine commented Jan 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

kush-elastic left a comment

Choose a reason for hiding this comment

Uh oh!

agithomas left a comment

Choose a reason for hiding this comment

Uh oh!

agithomas Jan 30, 2023

Choose a reason for hiding this comment

Uh oh!

harnish-crest-data Jan 30, 2023

Choose a reason for hiding this comment

Uh oh!

agithomas Feb 4, 2023

Choose a reason for hiding this comment

Uh oh!

agithomas Jan 30, 2023

Choose a reason for hiding this comment

Uh oh!

harnish-crest-data Jan 30, 2023

Choose a reason for hiding this comment

Uh oh!

agithomas Feb 4, 2023

Choose a reason for hiding this comment

Uh oh!

harnish-crest-data commented Jan 30, 2023

Uh oh!

Uh oh!

agithomas left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Feb 8, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

harnish-crest-data commented Jan 20, 2023 •

edited

Loading

elasticmachine commented Jan 20, 2023 •

edited

Loading

elasticmachine commented Jan 20, 2023 •

edited

Loading