[ML] Adding new jobs to siem module by jgowdyelastic · Pull Request #43783 · elastic/kibana

jgowdyelastic · 2019-08-22T15:25:27Z

Adding new jobs to the Siem Auditbeat and Winlogbeat modules.

siem_auditbeat_ecs
linux_anomalous_network_activity
linux_anomalous_network_port_activity
linux_anomalous_network_service
linux_anomalous_network_url_activity
linux_anomalous_process_all_hosts
linux_anomalous_user_name
rare_process_by_host_linux_ecs
suspicious_login_activity_ecs

siem_winlogbeat_ecs
rare_process_windows_ecs
windows_anomalous_network_activity
windows_anomalous_path_activity
windows_anomalous_process_all_hosts
windows_anomalous_process_creation
windows_anomalous_script
windows_anomalous_service
windows_anomalous_user_name

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

This was checked for cross-browser compatibility, including a check against IE11
Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility

For maintainers

This was checked for breaking API changes and was labeled appropriately
This includes a feature addition or change that requires a release note and was labeled appropriately

elasticmachine · 2019-08-22T16:57:48Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-08-23T10:00:53Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-08-23T10:13:47Z

Pinging @elastic/ml-ui

elasticmachine · 2019-08-23T10:13:48Z

Pinging @elastic/siem

cwurm · 2019-08-23T12:03:54Z

Looking through the queries, a few questions:

must vs filter: In boolean queries we use must, sometimes filter. I think we should always use the faster (because non-scoring) filter.
match vs term We sometimes use match and sometimes term. I'm not sure the behavior is different on keyword fields where match cannot do any text analysis - but regardless maybe we should be consistent and use term everywhere?
minimum_should_match We often exclude multiple values using should clauses with multiple sub-clauses. In such cases, I've often seen "minimum_should_match" : 1 used since only one match is enough to exclude a document. So I wonder, would using it give us a performance advantage? Or is the boolean query smart enough to figure out that a should inside a must_not already implies it?

I focus on the queries mainly because with an increasing number of built-in jobs the query performance might matter, esp. when a user starts them all at once.

sophiec20 · 2019-08-23T13:43:08Z

@blaklaybul along with the above, can we please consider "model_memory_limit": "256mb" ... for count by field detectors this is way too high and will prevent multiple jobs being opened in quick succession on small nodes (especially a cloud trial). Ref #41135

blaklaybul · 2019-08-23T16:25:19Z

@cwurm thanks for the feedback on the queries. I'll continue to refine them using your suggestions.

@sophiec20 absolutely. I was using 256mb as a placeholder while I go through the model_memory_stats from the jobs, which is ongoing.

Some of the linux jobs consumed a lot of memory, some as high as 3.5gb. Also, we do not have figures for the windows jobs. Being that the windows jobs were not developed on a cluster, all we have for estimating model size is using the data from another cluster. This has more data than what we used to develop the initial set of SIEM jobs.

So based on seen or assumed high memory sizes, I think we should hold the following jobs so we can continue to refine them and test them on real world datasets:

file_deletes_linux_ecs.json (3.5GB)
file_events_linux_ecs.json (1.5 GB)
file_events_windows_ecs.json (assumed 1.5GB)
rare_destination_port_process_windows_ecs.json (potential high cardinality partition field)
rare_process_tree_windows_ecs.json (potential high cardinality partition field)

cc @randomuserid

randomuserid · 2019-08-23T18:56:03Z

On the proposed cuts:

file_deletes_linux_ecs.json (3.5GB)
file_events_linux_ecs.json (1.5 GB)

I'm OK with holding these

file_events_windows_ecs.json (assumed 1.5GB)
rare_process_tree_windows_ecs.json (potential high cardinality partition field)

-these are canonical hunts; the second is actually taught in threat hunting school , so losing them would be a bit of a shame.

rare_destination_port_process_windows_ecs.json (potential high cardinality partition field)

this could be replaced by the rare network process job

They have asked for a number of things and I'd like to include two more jobs I have tested using their ideas - rare network process and rare process arguments. These jobs are testing OK and are good hunts so it would be valuable to include them.

blaklaybul · 2019-08-27T01:19:28Z

Thanks for the feedback everyone. @randomuserid and I have finalized the jobs configs, datafeed queries, custom urls, model memory limits, and naming - bringing us to a total of 13 new jobs. The final jobs can be found here:

@jgowdyelastic note that the final list of jobs (and their names) have changed for consistency and ease of use within the SIEM app.

Additionally, we will need to update the manifest descriptions and queries to reflect the new jobs:

siem-auditbeat:

{
"description": "Detect unusual behavior in Auditbeat ECS data (beta)"`, 
"query": {
    "bool": {
      "filter": [
        { "term": { "agent.type": "auditbeat" } }
      ]
    }
  }
}

siem-winlogbeat:

{
"description": "Detect unusual behavior in Winlogbeat ECS data (beta)"`, 
"query": {
    "bool": {
      "filter": [
        { "term": { "agent.type": "winlogbeat" } }
      ]
    }
  }
}

elasticmachine · 2019-08-27T09:33:17Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-08-27T11:04:22Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

cwurm · 2019-08-27T11:27:00Z

I still see one must clause and several match queries. Can we change those to filter and term?

elasticmachine · 2019-08-28T21:41:50Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-08-29T09:30:04Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

* [ML] Adding new jobs to siem module * updating descriptions * removing new jobs * updating manifests * adding updated modules * updating queries * fixing detector * updating job descriptions * updating datafeed ids * changing duplicate description * changing match for term * adding (beta) to descriptions

@benskelker

## Summary Adds the below new ML Jobs from #43783 and #44383 for the SIEM ML Integration. Note: This also removes the `siem-api-` prefix added when jobs are initially created. As a result, upgrading users who have any of the three original jobs (`rare_process_linux_ecs`, `suspicious_login_activity_ecs`, `rare_process_windows_ecs`) installed, will have the latest version of them installed without the prefix. The old `siem-api-` prefixed jobs will then show up in the `Custom jobs` tab within the UI. This was going to happen with the `rare_process_linux_ecs` anyway, as it was renamed to `rare_process_by_host_linux_ecs`, so we took the opportunity to clean up naming while the job count is low. Job versioning/migration will be addressed in 7.5. ``` ts { name: 'siem_auditbeat_ecs', defaultIndexPattern: 'auditbeat-*', jobs: [ 'rare_process_by_host_linux_ecs', 'suspicious_login_activity_ecs', 'linux_anomalous_network_activity_ecs', 'linux_anomalous_network_port_activity_ecs', 'linux_anomalous_network_service', 'linux_anomalous_network_url_activity_ecs', 'linux_anomalous_process_all_hosts_ecs', 'linux_anomalous_user_name_ecs', ], }, { name: 'siem_winlogbeat_ecs', defaultIndexPattern: 'winlogbeat-*', jobs: [ 'rare_process_by_host_windows_ecs', 'windows_anomalous_network_activity_ecs', 'windows_anomalous_path_activity_ecs', 'windows_anomalous_process_all_hosts_ecs', 'windows_anomalous_process_creation', 'windows_anomalous_script', 'windows_anomalous_service', 'windows_anomalous_user_name_ecs', ], }, ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Will work with @benskelker to update the job list in the [SIEM Guide](https://www.elastic.co/guide/en/siem/guide/current/machine-learning.html). @randomuserid, is there any additional documentation you would like to add? - [ ] ~[Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~ - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

@benskelker

## Summary Adds the below new ML Jobs from elastic#43783 and elastic#44383 for the SIEM ML Integration. Note: This also removes the `siem-api-` prefix added when jobs are initially created. As a result, upgrading users who have any of the three original jobs (`rare_process_linux_ecs`, `suspicious_login_activity_ecs`, `rare_process_windows_ecs`) installed, will have the latest version of them installed without the prefix. The old `siem-api-` prefixed jobs will then show up in the `Custom jobs` tab within the UI. This was going to happen with the `rare_process_linux_ecs` anyway, as it was renamed to `rare_process_by_host_linux_ecs`, so we took the opportunity to clean up naming while the job count is low. Job versioning/migration will be addressed in 7.5. ``` ts { name: 'siem_auditbeat_ecs', defaultIndexPattern: 'auditbeat-*', jobs: [ 'rare_process_by_host_linux_ecs', 'suspicious_login_activity_ecs', 'linux_anomalous_network_activity_ecs', 'linux_anomalous_network_port_activity_ecs', 'linux_anomalous_network_service', 'linux_anomalous_network_url_activity_ecs', 'linux_anomalous_process_all_hosts_ecs', 'linux_anomalous_user_name_ecs', ], }, { name: 'siem_winlogbeat_ecs', defaultIndexPattern: 'winlogbeat-*', jobs: [ 'rare_process_by_host_windows_ecs', 'windows_anomalous_network_activity_ecs', 'windows_anomalous_path_activity_ecs', 'windows_anomalous_process_all_hosts_ecs', 'windows_anomalous_process_creation', 'windows_anomalous_script', 'windows_anomalous_service', 'windows_anomalous_user_name_ecs', ], }, ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Will work with @benskelker to update the job list in the [SIEM Guide](https://www.elastic.co/guide/en/siem/guide/current/machine-learning.html). @randomuserid, is there any additional documentation you would like to add? - [ ] ~[Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~ - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

@benskelker

## Summary Adds the below new ML Jobs from elastic#43783 and elastic#44383 for the SIEM ML Integration. Note: This also removes the `siem-api-` prefix added when jobs are initially created. As a result, upgrading users who have any of the three original jobs (`rare_process_linux_ecs`, `suspicious_login_activity_ecs`, `rare_process_windows_ecs`) installed, will have the latest version of them installed without the prefix. The old `siem-api-` prefixed jobs will then show up in the `Custom jobs` tab within the UI. This was going to happen with the `rare_process_linux_ecs` anyway, as it was renamed to `rare_process_by_host_linux_ecs`, so we took the opportunity to clean up naming while the job count is low. Job versioning/migration will be addressed in 7.5. ``` ts { name: 'siem_auditbeat_ecs', defaultIndexPattern: 'auditbeat-*', jobs: [ 'rare_process_by_host_linux_ecs', 'suspicious_login_activity_ecs', 'linux_anomalous_network_activity_ecs', 'linux_anomalous_network_port_activity_ecs', 'linux_anomalous_network_service', 'linux_anomalous_network_url_activity_ecs', 'linux_anomalous_process_all_hosts_ecs', 'linux_anomalous_user_name_ecs', ], }, { name: 'siem_winlogbeat_ecs', defaultIndexPattern: 'winlogbeat-*', jobs: [ 'rare_process_by_host_windows_ecs', 'windows_anomalous_network_activity_ecs', 'windows_anomalous_path_activity_ecs', 'windows_anomalous_process_all_hosts_ecs', 'windows_anomalous_process_creation', 'windows_anomalous_script', 'windows_anomalous_service', 'windows_anomalous_user_name_ecs', ], }, ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Will work with @benskelker to update the job list in the [SIEM Guide](https://www.elastic.co/guide/en/siem/guide/current/machine-learning.html). @randomuserid, is there any additional documentation you would like to add? - [ ] ~[Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~ - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

@benskelker

## Summary Adds the below new ML Jobs from #43783 and #44383 for the SIEM ML Integration. Note: This also removes the `siem-api-` prefix added when jobs are initially created. As a result, upgrading users who have any of the three original jobs (`rare_process_linux_ecs`, `suspicious_login_activity_ecs`, `rare_process_windows_ecs`) installed, will have the latest version of them installed without the prefix. The old `siem-api-` prefixed jobs will then show up in the `Custom jobs` tab within the UI. This was going to happen with the `rare_process_linux_ecs` anyway, as it was renamed to `rare_process_by_host_linux_ecs`, so we took the opportunity to clean up naming while the job count is low. Job versioning/migration will be addressed in 7.5. ``` ts { name: 'siem_auditbeat_ecs', defaultIndexPattern: 'auditbeat-*', jobs: [ 'rare_process_by_host_linux_ecs', 'suspicious_login_activity_ecs', 'linux_anomalous_network_activity_ecs', 'linux_anomalous_network_port_activity_ecs', 'linux_anomalous_network_service', 'linux_anomalous_network_url_activity_ecs', 'linux_anomalous_process_all_hosts_ecs', 'linux_anomalous_user_name_ecs', ], }, { name: 'siem_winlogbeat_ecs', defaultIndexPattern: 'winlogbeat-*', jobs: [ 'rare_process_by_host_windows_ecs', 'windows_anomalous_network_activity_ecs', 'windows_anomalous_path_activity_ecs', 'windows_anomalous_process_all_hosts_ecs', 'windows_anomalous_process_creation', 'windows_anomalous_script', 'windows_anomalous_service', 'windows_anomalous_user_name_ecs', ], }, ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Will work with @benskelker to update the job list in the [SIEM Guide](https://www.elastic.co/guide/en/siem/guide/current/machine-learning.html). @randomuserid, is there any additional documentation you would like to add? - [ ] ~[Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~ - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

@benskelker

## Summary Adds the below new ML Jobs from #43783 and #44383 for the SIEM ML Integration. Note: This also removes the `siem-api-` prefix added when jobs are initially created. As a result, upgrading users who have any of the three original jobs (`rare_process_linux_ecs`, `suspicious_login_activity_ecs`, `rare_process_windows_ecs`) installed, will have the latest version of them installed without the prefix. The old `siem-api-` prefixed jobs will then show up in the `Custom jobs` tab within the UI. This was going to happen with the `rare_process_linux_ecs` anyway, as it was renamed to `rare_process_by_host_linux_ecs`, so we took the opportunity to clean up naming while the job count is low. Job versioning/migration will be addressed in 7.5. ``` ts { name: 'siem_auditbeat_ecs', defaultIndexPattern: 'auditbeat-*', jobs: [ 'rare_process_by_host_linux_ecs', 'suspicious_login_activity_ecs', 'linux_anomalous_network_activity_ecs', 'linux_anomalous_network_port_activity_ecs', 'linux_anomalous_network_service', 'linux_anomalous_network_url_activity_ecs', 'linux_anomalous_process_all_hosts_ecs', 'linux_anomalous_user_name_ecs', ], }, { name: 'siem_winlogbeat_ecs', defaultIndexPattern: 'winlogbeat-*', jobs: [ 'rare_process_by_host_windows_ecs', 'windows_anomalous_network_activity_ecs', 'windows_anomalous_path_activity_ecs', 'windows_anomalous_process_all_hosts_ecs', 'windows_anomalous_process_creation', 'windows_anomalous_script', 'windows_anomalous_service', 'windows_anomalous_user_name_ecs', ], }, ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Will work with @benskelker to update the job list in the [SIEM Guide](https://www.elastic.co/guide/en/siem/guide/current/machine-learning.html). @randomuserid, is there any additional documentation you would like to add? - [ ] ~[Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~ - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

* [ML] Adding new jobs to siem module * updating descriptions * removing new jobs * updating manifests * adding updated modules * updating queries * fixing detector * updating job descriptions * updating datafeed ids * changing duplicate description * changing match for term * adding (beta) to descriptions

@benskelker

## Summary Adds the below new ML Jobs from elastic#43783 and elastic#44383 for the SIEM ML Integration. Note: This also removes the `siem-api-` prefix added when jobs are initially created. As a result, upgrading users who have any of the three original jobs (`rare_process_linux_ecs`, `suspicious_login_activity_ecs`, `rare_process_windows_ecs`) installed, will have the latest version of them installed without the prefix. The old `siem-api-` prefixed jobs will then show up in the `Custom jobs` tab within the UI. This was going to happen with the `rare_process_linux_ecs` anyway, as it was renamed to `rare_process_by_host_linux_ecs`, so we took the opportunity to clean up naming while the job count is low. Job versioning/migration will be addressed in 7.5. ``` ts { name: 'siem_auditbeat_ecs', defaultIndexPattern: 'auditbeat-*', jobs: [ 'rare_process_by_host_linux_ecs', 'suspicious_login_activity_ecs', 'linux_anomalous_network_activity_ecs', 'linux_anomalous_network_port_activity_ecs', 'linux_anomalous_network_service', 'linux_anomalous_network_url_activity_ecs', 'linux_anomalous_process_all_hosts_ecs', 'linux_anomalous_user_name_ecs', ], }, { name: 'siem_winlogbeat_ecs', defaultIndexPattern: 'winlogbeat-*', jobs: [ 'rare_process_by_host_windows_ecs', 'windows_anomalous_network_activity_ecs', 'windows_anomalous_path_activity_ecs', 'windows_anomalous_process_all_hosts_ecs', 'windows_anomalous_process_creation', 'windows_anomalous_script', 'windows_anomalous_service', 'windows_anomalous_user_name_ecs', ], }, ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials * Will work with @benskelker to update the job list in the [SIEM Guide](https://www.elastic.co/guide/en/siem/guide/current/machine-learning.html). @randomuserid, is there any additional documentation you would like to add? - [ ] ~[Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~ - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

jgowdyelastic force-pushed the adding-new-siem-jobs branch from 77dcf5d to 218d884 Compare August 23, 2019 08:42

jgowdyelastic requested review from alvarezmelissa87, blaklaybul, peteharverson and walterra August 23, 2019 10:12

jgowdyelastic self-assigned this Aug 23, 2019

jgowdyelastic added :ml release_note:enhancement review Team:SIEM v7.4.0 v8.0.0 labels Aug 23, 2019

cwurm reviewed Aug 23, 2019

View reviewed changes

Comment thread ...er/models/data_recognizer/modules/siem_auditbeat_ecs/ml/datafeed_file_deletes_linux_ecs.json Outdated

cwurm reviewed Aug 23, 2019

View reviewed changes

Comment thread ...models/data_recognizer/modules/siem_winlogbeat_ecs/ml/datafeed_rare_service_windows_ecs.json Outdated

jgowdyelastic force-pushed the adding-new-siem-jobs branch from f66fd5f to e1deb52 Compare August 27, 2019 12:32

jgowdyelastic marked this pull request as ready for review August 27, 2019 12:36

jgowdyelastic requested a review from a team as a code owner August 27, 2019 12:36

jgowdyelastic force-pushed the adding-new-siem-jobs branch from e1deb52 to dff1a98 Compare August 27, 2019 13:06

jgowdyelastic added 12 commits August 29, 2019 08:39

[ML] Adding new jobs to siem module

5f8986c

updating descriptions

7d47643

removing new jobs

f72887e

updating manifests

dfbce4a

adding updated modules

11b1da1

updating queries

8c80103

fixing detector

3d0a80a

updating job descriptions

46baf31

updating datafeed ids

972f15d

changing duplicate description

c81552a

changing match for term

4b1c127

adding (beta) to descriptions

2ea172f

jgowdyelastic force-pushed the adding-new-siem-jobs branch from 87979c6 to 2ea172f Compare August 29, 2019 07:39

jgowdyelastic merged commit 64ec780 into elastic:master Aug 29, 2019

jgowdyelastic deleted the adding-new-siem-jobs branch August 29, 2019 09:32

jgowdyelastic mentioned this pull request Aug 29, 2019

[7.x] [ML] Adding new jobs to siem module (#43783) #44365

Merged

jgowdyelastic mentioned this pull request Aug 29, 2019

[ML] Adding ecs to SIEM jobs names #44383

Merged

Conversation

jgowdyelastic commented Aug 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

For maintainers

Uh oh!

elasticmachine commented Aug 22, 2019

💚 Build Succeeded

Uh oh!

elasticmachine commented Aug 23, 2019

💚 Build Succeeded

Uh oh!

elasticmachine commented Aug 23, 2019

Uh oh!

elasticmachine commented Aug 23, 2019

Uh oh!

Uh oh!

Uh oh!

cwurm commented Aug 23, 2019

Uh oh!

sophiec20 commented Aug 23, 2019

Uh oh!

blaklaybul commented Aug 23, 2019 • edited by cwurm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

randomuserid commented Aug 23, 2019 • edited by cwurm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blaklaybul commented Aug 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

siem-auditbeat:

siem-winlogbeat:

Uh oh!

elasticmachine commented Aug 27, 2019

💚 Build Succeeded

Uh oh!

elasticmachine commented Aug 27, 2019

💚 Build Succeeded

Uh oh!

cwurm commented Aug 27, 2019

Uh oh!

elasticmachine commented Aug 28, 2019

💔 Build Failed

Uh oh!

elasticmachine commented Aug 29, 2019

💚 Build Succeeded

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

jgowdyelastic commented Aug 22, 2019 •

edited

Loading

blaklaybul commented Aug 23, 2019 •

edited by cwurm

Loading

randomuserid commented Aug 23, 2019 •

edited by cwurm

Loading

blaklaybul commented Aug 27, 2019 •

edited

Loading