Skip to content

[Security Solution] Add create_index to kibana_system role for Elastic Defend indices #115241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
paul-tavares merged 16 commits into from
Oct 23, 2024
Merged

[Security Solution] Add create_index to kibana_system role for Elastic Defend indices #115241

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
cacf96b
Add `create_index` to `kibana_system` role for index `.logs-endpoint.…
paul-tavares Oct 21, 2024
72b6a20
Update docs/changelog/115241.yaml
paul-tavares Oct 21, 2024
9746357
Add `create_index` for endpoint diagnostics and heartbeat indices
paul-tavares Oct 22, 2024
c3e0d71
Merge remote-tracking branch 'upstream/main' into task/olm-10840-upda…
paul-tavares Oct 22, 2024
e279e9f
Merge remote-tracking branch 'origin/task/olm-10840-update-kibana-sys…
paul-tavares Oct 22, 2024
453d2d0
Correct misspelling
paul-tavares Oct 22, 2024
e8c899d
Correct misspelling (again)
paul-tavares Oct 22, 2024
d189ddf
Formatted code
paul-tavares Oct 22, 2024
1761322
Fix test
paul-tavares Oct 23, 2024
8fa5f85
Merge remote-tracking branch 'upstream/main' into task/olm-10840-upda…
paul-tavares Oct 23, 2024
962921f
format files
paul-tavares Oct 23, 2024
6cd5983
Merge remote-tracking branch 'upstream/main' into task/olm-10840-upda…
paul-tavares Oct 23, 2024
1112c67
Fix test (maybe?)
paul-tavares Oct 23, 2024
9b12f77
Fix test (again - maybe?)
paul-tavares Oct 23, 2024
96cb2c1
Fix tests - this time for real?
paul-tavares Oct 23, 2024
016851f
Merge remote-tracking branch 'upstream/main' into task/olm-10840-upda…
paul-tavares Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/changelog/115241.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
pr: 115241
summary: "[Security Solution] Add `create_index` to `kibana_system` role for index/DS\
\ `.logs-endpoint.action.responses-*`"
area: Authorization
type: enhancement
issues: []
19 changes: 12 additions & 7 deletions ...org/elasticsearch/xpack/core/security/authz/store/KibanaOwnedReservedRoleDescriptors.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,11 @@ static RoleDescriptor kibanaSystem(String name) {
// Data telemetry reads mappings, metadata and stats of indices
RoleDescriptor.IndicesPrivileges.builder().indices("*").privileges("view_index_metadata", "monitor").build(),
// Endpoint diagnostic information. Kibana reads from these indices to send
// telemetry
RoleDescriptor.IndicesPrivileges.builder().indices(".logs-endpoint.diagnostic.collection-*").privileges("read").build(),
// telemetry and also creates the index when policies are first created
RoleDescriptor.IndicesPrivileges.builder()

@kc13greiner kc13greiner Oct 23, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heya @paul-tavares

Question: It looks like the kibana_system user has had read access for a bit, can you provide any context as to why it needs to create these indexes now?

@paul-tavares paul-tavares Oct 23, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kc13greiner for looking at this PR.
This change is necessary in order to address the issues found last week when an attempt was made (and later backed out) to restrict DOT indices for serverless. I have updated the description of this PR to more clearly state the need for the changes, which I will include here now:

This change is required in support of restricting DOT indexes from being created by non-privileged users. Up until now, when the Elastic Defend package is installed via Fleet, the Datastream templates for a few DOT prefixed indices are installed. The indexes are then created the first time the Elastic Defend integration executable (running on host machines) streams it first document - which under the restriction (in serverless) the Elastic Defend executable will no longer be given privileges to do. The alternative is to have Kibana create the indexes prior to the endpoint streaming any documents to elastic search.

Let me know if you have any other questions...

(now: on to figure out whats wrong with the failing test... I may need some help there... 😁 )

@kc13greiner kc13greiner Oct 23, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the information!

.indices(".logs-endpoint.diagnostic.collection-*")
.privileges("read", "create_index")
.build(),
// Fleet secrets. Kibana can only write to this index.
RoleDescriptor.IndicesPrivileges.builder()
.indices(".fleet-secrets*")
Expand Down Expand Up @@ -277,17 +280,19 @@ static RoleDescriptor kibanaSystem(String name) {
)
.build(),
// Endpoint specific action responses. Kibana reads and writes (for third party
// agents) to the index
// to display action responses to the user.
// agents) to the index to display action responses to the user.
// `create_index`: is necessary in order to ensure that the DOT datastream index is
// created by Kibana in order to avoid errors on the Elastic Defend side when streaming
// documents to it.
RoleDescriptor.IndicesPrivileges.builder()
.indices(".logs-endpoint.action.responses-*")
.privileges("auto_configure", "read", "write")
.privileges("auto_configure", "read", "write", "create_index")
.build(),
// Endpoint specific actions. Kibana reads and writes to this index to track new
// actions and display them.
RoleDescriptor.IndicesPrivileges.builder()
.indices(".logs-endpoint.actions-*")
.privileges("auto_configure", "read", "write")
.privileges("auto_configure", "read", "write", "create_index")
.build(),
// Legacy Osquery manager specific action responses. Kibana reads from these to
// display responses to the user.
Expand Down Expand Up @@ -475,7 +480,7 @@ static RoleDescriptor kibanaSystem(String name) {
RoleDescriptor.IndicesPrivileges.builder().indices(".slo-observability.*").privileges("all").build(),
// Endpoint heartbeat. Kibana reads from these to determine metering/billing for
// endpoints.
RoleDescriptor.IndicesPrivileges.builder().indices(".logs-endpoint.heartbeat-*").privileges("read").build(),
RoleDescriptor.IndicesPrivileges.builder().indices(".logs-endpoint.heartbeat-*").privileges("read", "create_index").build(),
// For connectors telemetry. Will be removed once we switched to connectors API
RoleDescriptor.IndicesPrivileges.builder().indices(".elastic-connectors*").privileges("read").build() },
null,
Expand Down
34 changes: 17 additions & 17 deletions .../test/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStoreTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -801,7 +801,7 @@ public void testKibanaSystemRole() {
assertThat(kibanaRole.indices().allowedIndicesMatcher(GetIndexAction.NAME).test(mockIndexAbstraction(index)), is(true));
assertThat(
kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(mockIndexAbstraction(index)),
is(false)
is(true)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportIndexAction.NAME).test(mockIndexAbstraction(index)), is(false));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportDeleteAction.NAME).test(mockIndexAbstraction(index)), is(false));
Expand Down Expand Up @@ -949,7 +949,7 @@ public void testKibanaSystemRole() {
);
});

// read-only index for Endpoint and Osquery manager specific action responses
// Elastic Defend internal index for response actions results
Arrays.asList(".logs-endpoint.action.responses-" + randomAlphaOfLength(randomIntBetween(0, 13))).forEach((index) -> {
final IndexAbstraction indexAbstraction = mockIndexAbstraction(index);
assertThat(kibanaRole.indices().allowedIndicesMatcher("indices:foo").test(indexAbstraction), is(false));
Expand All @@ -959,10 +959,7 @@ public void testKibanaSystemRole() {
is(false)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(GetIndexAction.NAME).test(indexAbstraction), is(true));
assertThat(
kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(indexAbstraction),
is(false)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(indexAbstraction), is(true));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportIndexAction.NAME).test(indexAbstraction), is(true));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportDeleteAction.NAME).test(indexAbstraction), is(true));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportSearchAction.TYPE.name()).test(indexAbstraction), is(true));
Expand Down Expand Up @@ -1069,10 +1066,7 @@ public void testKibanaSystemRole() {
is(false)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(GetIndexAction.NAME).test(indexAbstraction), is(true));
assertThat(
kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(indexAbstraction),
is(false)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(indexAbstraction), is(true));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportIndexAction.NAME).test(indexAbstraction), is(true));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportDeleteAction.NAME).test(indexAbstraction), is(true));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportSearchAction.TYPE.name()).test(indexAbstraction), is(true));
Expand All @@ -1097,10 +1091,7 @@ public void testKibanaSystemRole() {
is(false)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(GetIndexAction.NAME).test(indexAbstraction), is(true));
assertThat(
kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(indexAbstraction),
is(false)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(indexAbstraction), is(true));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportIndexAction.NAME).test(indexAbstraction), is(false));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportDeleteAction.NAME).test(indexAbstraction), is(false));
assertThat(kibanaRole.indices().allowedIndicesMatcher(TransportSearchAction.TYPE.name()).test(indexAbstraction), is(true));
Expand Down Expand Up @@ -1319,12 +1310,21 @@ public void testKibanaSystemRole() {

final boolean isAlsoAutoCreateIndex = indexName.startsWith(".logs-endpoint.actions-")
|| indexName.startsWith(".logs-endpoint.action.responses-");

final boolean isAlsoCreateIndex = indexName.startsWith(".logs-endpoint.actions-")
|| indexName.startsWith(".logs-endpoint.action.responses-")
|| indexName.startsWith(".logs-endpoint.diagnostic.collection-")
|| indexName.startsWith(".logs-endpoint.heartbeat-");

assertThat(
kibanaRole.indices().allowedIndicesMatcher(TransportCreateIndexAction.TYPE.name()).test(indexAbstraction),
is(false)
is(isAlsoCreateIndex)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(AutoCreateAction.NAME).test(indexAbstraction), is(isAlsoCreateIndex));
assertThat(
kibanaRole.indices().allowedIndicesMatcher(CreateDataStreamAction.NAME).test(indexAbstraction),
is(isAlsoCreateIndex)
);
assertThat(kibanaRole.indices().allowedIndicesMatcher(AutoCreateAction.NAME).test(indexAbstraction), is(isAlsoAutoCreateIndex));
assertThat(kibanaRole.indices().allowedIndicesMatcher(CreateDataStreamAction.NAME).test(indexAbstraction), is(false));
assertThat(
kibanaRole.indices().allowedIndicesMatcher(TransportIndexAction.NAME).test(indexAbstraction),
is(isAlsoAutoCreateIndex)
Expand Down