Skip to content

[Kubernetes] Add support for TSDB - set dimension fields for state data streams #5621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
constanca-m merged 5 commits into from
Mar 22, 2023
Merged

[Kubernetes] Add support for TSDB - set dimension fields for state data streams #5621

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ac87cf7
Update dimension fields.
constanca-m Mar 21, 2023
ab0e2e3
Update dimension fields.
constanca-m Mar 21, 2023
ca2aa68
Update changelog.
constanca-m Mar 21, 2023
ab33916
Merge branch 'main' into update-tsdb-states
constanca-m Mar 21, 2023
3b1ff96
Merge branch 'main' into update-tsdb-states
constanca-m Mar 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/kubernetes/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.34.0"
changes:
- description: Add support for TSDB on all state metric data streams
type: enhancement
link: https://github.com/elastic/integrations/pull/5621
- version: "1.33.0"

@MichaelKatsoulis MichaelKatsoulis Mar 22, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, you should go with the 1.33.0 version. If the other commit gets merged first you will rebase/merge on this one and increment the version.

@constanca-m constanca-m Mar 22, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, I chose it assuming the other one was being merged first. But I will adjust this if it is not case.

changes:
- description: Add `data_stream.dataset` setting and custom yaml input.
Expand Down
1 change: 0 additions & 1 deletion packages/kubernetes/data_stream/state_container/fields/agent.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,6 @@
type: group
fields:
- name: id
dimension: true
level: core
type: keyword
ignore_above: 1024
Expand Down
3 changes: 0 additions & 3 deletions packages/kubernetes/data_stream/state_container/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
type: group
fields:
- name: pod.name
dimension: true
type: keyword
description: >
Kubernetes pod name
Expand All @@ -31,7 +30,6 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace
Expand Down Expand Up @@ -136,7 +134,6 @@
Name of the CronJob to which the Pod belongs

- name: container.name
dimension: true
type: keyword
description: >
Kubernetes container name
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_container/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
Expand All @@ -10,3 +11,4 @@
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
3 changes: 3 additions & 0 deletions packages/kubernetes/data_stream/state_container/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@
type: group
fields:
- name: id
dimension: true
type: keyword
description: Container id
- name: status
type: group
fields:
- name: phase
dimension: true
type: keyword
description: |
Container phase (running, waiting, terminated)
Expand All @@ -21,6 +23,7 @@
description: |
Container restarts count
- name: reason
dimension: true
type: keyword
description: |
Waiting (ContainerCreating, CrashLoopBackoff, ErrImagePull, ImagePullBackoff) or termination (Completed, ContainerCannotRun, Error, OOMKilled) reason.
Expand Down
5 changes: 1 addition & 4 deletions packages/kubernetes/data_stream/state_cronjob/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,11 @@
type: group
fields:
- name: pod.name
dimension: true
type: keyword
description: >
Kubernetes pod name

- name: pod.uid
dimension: true
type: keyword
description: >
Kubernetes pod UID
Expand All @@ -31,12 +29,12 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down Expand Up @@ -95,7 +93,6 @@
Kubernetes statefulset name

- name: container.name
dimension: true
type: keyword
description: >
Kubernetes container name
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_cronjob/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 0 additions & 2 deletions packages/kubernetes/data_stream/state_cronjob/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,9 @@
type: keyword
description: Cronjob name
- name: schedule
dimension: true
type: keyword
description: Cronjob schedule
- name: concurrency
dimension: true
type: keyword
description: Concurrency policy
- name: active.count
Expand Down
2 changes: 1 addition & 1 deletion packages/kubernetes/data_stream/state_daemonset/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_daemonset/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 1 addition & 1 deletion packages/kubernetes/data_stream/state_deployment/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
Kubernetes namespace

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down Expand Up @@ -77,7 +78,6 @@
Kubernetes Service selectors map

- name: replicaset.name
dimension: true
type: keyword
description: >
Kubernetes replicaset name
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_deployment/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 1 addition & 1 deletion packages/kubernetes/data_stream/state_job/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_job/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_node/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_persistentvolume/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 1 addition & 1 deletion packages/kubernetes/data_stream/state_persistentvolumeclaim/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down
2 changes: 0 additions & 2 deletions packages/kubernetes/data_stream/state_pod/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
type: group
fields:
- name: pod.name
dimension: true
type: keyword
description: >
Kubernetes pod name
Expand All @@ -31,7 +30,6 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_pod/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
Expand All @@ -10,3 +11,4 @@
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
3 changes: 1 addition & 2 deletions packages/kubernetes/data_stream/state_replicaset/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down Expand Up @@ -84,7 +84,6 @@
Kubernetes replicaset name

- name: deployment.name
dimension: true
type: keyword
description: >
Kubernetes deployment name
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_replicaset/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_resourcequota/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 1 addition & 1 deletion packages/kubernetes/data_stream/state_service/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace
Expand All @@ -49,6 +48,7 @@
Kubernetes annotations map

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_service/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 1 addition & 1 deletion packages/kubernetes/data_stream/state_statefulset/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@
Kubernetes pod IP

- name: namespace
dimension: true
type: keyword
description: >
Kubernetes namespace

- name: namespace_uid
dimension: true
type: keyword
description: >
Kubernetes namespace UID
Expand Down
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_statefulset/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
2 changes: 2 additions & 0 deletions packages/kubernetes/data_stream/state_storageclass/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
name: ecs.version
- external: ecs
name: service.address
dimension: true
- external: ecs
name: service.type
- external: ecs
name: orchestrator.cluster.name
- external: ecs
name: orchestrator.cluster.url
dimension: true
Loading