[Detection Rules] Resolves regression where Elastic Endgame rules would warn about unmapped timestamp override field by brokensound77 · Pull Request #96394 · elastic/kibana

brokensound77 · 2021-04-07T11:40:39Z

Summary

Endgame promotion rules in Kibana/7.12 are at version 5 and have timestamp_override defined (which should not be). These same rules are at version 4 in the detection-rules repo 7.12 branch and kibana/master and timestamp_override is not defined. These updates are targeted for 7.12.1

There most likely was an issue with the maze of backports and interlaced updates.

To fix the rules, they need to be reconciled across:

detection-rules 7.12 & main
kibana 7.12.1 and master
bump detection-rules/7.12 to v6 -> PR to kibana/master -> backport to 7.12.1

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support

For maintainers

This was checked for breaking API changes and was labeled appropriately

spong

LGTM!

Verified the following, e.g. for endgame_adversary_behavior_detected.json (7.12.x+) / endpoint_adversary_behavior_detected.json (7.11.x)

Released

7.11.2 (source)

Version: 4
No timestamp override

7.12.0 (source)

Version: 5
Timestamp override: event.ingested

Unreleased

7.12.1 (source) (Same as 7.12.0 till 7.12.1 FF next week)

Version: 5
Timestamp override: event.ingested

7.x (source)

Version: 5
Timestamp override: event.ingested

master (source)

Version: 4
No timestamp override

This PR 🎉

Version 6
No timestamp override

So just need to ensure we backport to both 7.12 and 7.x, and also ensure timestamp_override is removed in those backports as well (will have to do this manually as the auto-backport will only see the version diff).

As far as communication/docs, we'll need to add a known-issue to the 7.12 release notes (done!) for the regression around #91597 (i.e. * Warning is shown for 7.12 Elastic Endgame rules, can be ignored), and also let's update this PR to be release_notes:fix detailing that it addresses that regression (so it's automatically included in 7.12.1 note). I can work with the @elastic/security-docs folks one getting these changes in. Thank you so much @brokensound77 for catching and addressing this! 🙂

elasticmachine · 2021-04-07T17:07:37Z

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine · 2021-04-07T17:07:38Z

Pinging @elastic/security-solution (Team: SecuritySolution)

spong · 2021-04-08T01:01:09Z

@elasticmachine merge upstream

kibanamachine · 2021-04-08T03:06:13Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💔 Build #117806 failed 401d878

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2021-04-08T05:55:32Z

💔 Backport failed

❌ 7.12: Commit could not be cherrypicked due to conflicts
❌ 7.x: Commit could not be cherrypicked due to conflicts

To backport manually run:
node scripts/backport --pr 96394

…ld warn about unmapped timestamp override field (elastic#96394) related to elastic/detection-rules#1082 ## Summary Endgame promotion rules in Kibana/7.12 are at version 5 and have timestamp_override defined (which should not be). These same rules are at version 4 in the detection-rules repo 7.12 branch and kibana/master and timestamp_override is not defined. These updates are targeted for 7.12.1 There most likely was an issue with the maze of backports and interlaced updates. To fix the rules, they need to be reconciled across: detection-rules 7.12 & main kibana 7.12.1 and master bump detection-rules/7.12 to v6 -> PR to kibana/master -> backport to 7.12.1 ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) # Conflicts: # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_adversary_behavior_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_dumping_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_dumping_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_manipulation_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_manipulation_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_exploit_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_exploit_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_malware_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_malware_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_permission_theft_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_permission_theft_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_process_injection_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_process_injection_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_ransomware_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_ransomware_prevented.json

…ld warn about unmapped timestamp override field (#96394) (#96528) related to elastic/detection-rules#1082 ## Summary Endgame promotion rules in Kibana/7.12 are at version 5 and have timestamp_override defined (which should not be). These same rules are at version 4 in the detection-rules repo 7.12 branch and kibana/master and timestamp_override is not defined. These updates are targeted for 7.12.1 There most likely was an issue with the maze of backports and interlaced updates. To fix the rules, they need to be reconciled across: detection-rules 7.12 & main kibana 7.12.1 and master bump detection-rules/7.12 to v6 -> PR to kibana/master -> backport to 7.12.1 ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) # Conflicts: # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_adversary_behavior_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_dumping_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_dumping_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_manipulation_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_manipulation_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_exploit_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_exploit_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_malware_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_malware_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_permission_theft_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_permission_theft_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_process_injection_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_process_injection_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_ransomware_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_ransomware_prevented.json Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

…ld warn about unmapped timestamp override field (#96394) (#96529) related to elastic/detection-rules#1082 ## Summary Endgame promotion rules in Kibana/7.12 are at version 5 and have timestamp_override defined (which should not be). These same rules are at version 4 in the detection-rules repo 7.12 branch and kibana/master and timestamp_override is not defined. These updates are targeted for 7.12.1 There most likely was an issue with the maze of backports and interlaced updates. To fix the rules, they need to be reconciled across: detection-rules 7.12 & main kibana 7.12.1 and master bump detection-rules/7.12 to v6 -> PR to kibana/master -> backport to 7.12.1 ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) # Conflicts: # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_adversary_behavior_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_dumping_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_dumping_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_manipulation_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_cred_manipulation_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_exploit_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_exploit_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_malware_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_malware_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_permission_theft_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_permission_theft_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_process_injection_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_process_injection_prevented.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_ransomware_detected.json # x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules/endgame_ransomware_prevented.json Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>

…ld warn about unmapped timestamp override field (elastic#96394) related to elastic/detection-rules#1082 ## Summary Endgame promotion rules in Kibana/7.12 are at version 5 and have timestamp_override defined (which should not be). These same rules are at version 4 in the detection-rules repo 7.12 branch and kibana/master and timestamp_override is not defined. These updates are targeted for 7.12.1 There most likely was an issue with the maze of backports and interlaced updates. To fix the rules, they need to be reconciled across: detection-rules 7.12 & main kibana 7.12.1 and master bump detection-rules/7.12 to v6 -> PR to kibana/master -> backport to 7.12.1 ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

[Detection Rules] Add 7.12 rules

401d878

brokensound77 added the v7.12.1 label Apr 7, 2021

brokensound77 requested a review from a team as a code owner April 7, 2021 11:40

brokensound77 added the release_note:skip Skip the PR/issue when compiling release notes label Apr 7, 2021

brokensound77 requested review from rw-access and spong April 7, 2021 11:41

brokensound77 added Feature:Detection Rules Security Solution rules and Detection Engine auto-backport Deprecated - use backport:version if exact versions are needed labels Apr 7, 2021

rw-access approved these changes Apr 7, 2021

View reviewed changes

spong approved these changes Apr 7, 2021

View reviewed changes

spong added release_note:fix v7.13.0 and removed release_note:skip Skip the PR/issue when compiling release notes labels Apr 7, 2021

spong changed the title ~~[Detection Rules] Add 7.12.1 rule changes~~ [Detection Rules] Resolves regression where Elastic Endgame rules would warn about unmapped timestamp override field Apr 7, 2021

spong added Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detections and Resp labels Apr 7, 2021

spong added the v8.0.0 label Apr 7, 2021

Merge branch 'master' into rules/7.12.1

cd32efa

spong merged commit bc14f24 into elastic:master Apr 8, 2021

spong mentioned this pull request Apr 8, 2021

[7.x] [Detection Rules] Resolves regression where Elastic Endgame rules would warn about unmapped timestamp override field (#96394) #96528

Merged

spong mentioned this pull request Apr 8, 2021

[7.12] [Detection Rules] Resolves regression where Elastic Endgame rules would warn about unmapped timestamp override field (#96394) #96529

Merged

brokensound77 deleted the rules/7.12.1 branch April 14, 2021 13:19

Navigation Menu

Repository navigation

[Detection Rules] Resolves regression where Elastic Endgame rules would warn about unmapped timestamp override field - #96394

[Detection Rules] Resolves regression where Elastic Endgame rules would warn about unmapped timestamp override field#96394
spong merged 2 commits into
elastic:masterfrom
brokensound77:rules/7.12.1

brokensound77 commented Apr 7, 2021

Uh oh!

spong left a comment •

edited

Loading

Uh oh!

elasticmachine commented Apr 7, 2021

Uh oh!

elasticmachine commented Apr 7, 2021

Uh oh!

spong commented Apr 8, 2021

Uh oh!

kibanamachine commented Apr 8, 2021

Uh oh!

kibanamachine commented Apr 8, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

brokensound77 commented Apr 7, 2021

Summary

Checklist

For maintainers

Uh oh!

spong left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Released

Unreleased

Uh oh!

elasticmachine commented Apr 7, 2021

Uh oh!

elasticmachine commented Apr 7, 2021

Uh oh!

spong commented Apr 8, 2021

Uh oh!

kibanamachine commented Apr 8, 2021

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

kibanamachine commented Apr 8, 2021

💔 Backport failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

spong left a comment •

edited

Loading