Repository navigation
[Final][Cases] Cases analytics v2 - Attachments Index (.cases-attachments) - #276117
Merged
michaelolo24 merged 21 commits intoJul 8, 2026
Merged
michaelolo24 merged 21 commits into
michaelolo24 merged 21 commits into
Conversation
Contributor
|
Pinging @elastic/kibana-cases (Feature:Cases) |
Contributor
|
Pinging @elastic/kibana-cases (Team:Cases) |
…attachments-surface-pr3-clean
…attachments-surface-pr3-clean
michaelolo24
force-pushed
the
cases/analytics-v2-attachments-surface-pr3-clean
branch
from
July 3, 2026 02:19
04434da to
15a71aa
Compare
michaelolo24
force-pushed
the
cases/analytics-v2-attachments-surface-pr3-clean
branch
from
July 3, 2026 04:40
597d60e to
88e56c4
Compare
michaelolo24
force-pushed
the
cases/analytics-v2-attachments-surface-pr3-clean
branch
from
July 3, 2026 11:32
23a63de to
fd8d596
Compare
10 tasks
…attachments-surface-pr3-clean
michaelolo24
force-pushed
the
cases/analytics-v2-attachments-surface-pr3-clean
branch
from
July 3, 2026 12:36
8176b33 to
dbe2a25
Compare
…ertion The managed cases-analytics data view now spans three indices (.cases,.cases-activity,.cases-attachments); the per-space data views FTR test still expected only the first two, so it failed on the new attachments index. Update the expected title to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…attachments-surface-pr3-clean
The .cases-attachments analytics doc records the UNIFIED attachment type,
so a legacy `user` comment lands with `attachment.type: 'comment'`, not
'user'. The test matched the doc by the legacy `AttachmentType.user` enum,
so `docs.find(...)` returned undefined and `expect(userDoc).to.be.an('object')`
failed. `--bail` stopped the run at the first test, hiding the same latent
mismatch in the updateComment test. Match on the unified `'comment'` type
at both doc-lookup sites; the updateComment request payload keeps the
legacy `AttachmentType.user` (correct for the SO/API).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
|
When |
…attachments-surface-pr3-clean
| // Mirroring the partial `update` response directly would drop the | ||
| // immutable creation fields (`created_at` → `@timestamp`) and flicker | ||
| // the edited attachment off time-filtered views until reconciliation. | ||
| this.mirrorUpdatedAttachments([{ type: CASE_ATTACHMENT_SAVED_OBJECT, id: savedObjectId }]); |
Contributor
There was a problem hiding this comment.
mirrorUpdatedAttachments is called unconditionally from update and bulkUpdate, and it always does a bulkGet re-read before touching the writer
…attachments-surface-pr3-clean
christineweng
approved these changes
Jul 7, 2026
michaelolo24
enabled auto-merge (squash)
July 8, 2026 00:32
Contributor
💛 Build succeeded, but was flaky
Failed CI Steps
Test Failures
Metrics [docs]Unknown metric groupsESLint disabled line counts
Total ESLint disabled count
History
|
28 of 29 tasks
patrykkopycinski
pushed a commit
to patrykkopycinski/kibana
that referenced
this pull request
Aug 5, 2026
…ents) (elastic#276117) ## Summary Feature Flag: `xpack.cases.analyticsV2.enabled` This is a follow up to elastic#269581 and elastic#275686, which introduced the cases as data V2 work and its first two indices, `.cases` and `.cases-activity`. This PR adds the third and final one, `.cases-attachments`, which holds the case comments and attachments data (one document per comment / attachment). The `Cases Analytics` data view added in the first PR already points at all three indices, so this data can be joined back to its case via `cases.id` using an ES|QL `LOOKUP JOIN`. Like the other two indices, `.cases-attachments` follows the same approach set up in the previous PRs: - Whenever a comment or attachment is created, updated, or deleted, we asynchronously write the same change to the `.cases-attachments` index. As with the other indices, this *will not* block the primary write from happening if the analytics write fails. - A reconciliation task runs periodically to check all comments and attachments since the last run and make sure they've been written to `.cases-attachments`. Each index is reconciled independently, so an issue with one won't hold up the others. - The `/reset` and `/reconcile/run_soon` admin routes and the initial backfill now cover all three indices. The backfill and `/reset` rebuild the three indices in parallel, and read saved objects in larger batches, so a full rebuild finishes noticeably faster than doing them one after another. Of note: - This index holds both the older case comments (cases-comments) and the newer unified attachments (cases-attachments), normalized into a single document shape, so it works whether or not a deployment has moved to the new attachments saved object. Both source types are always mirrored and reconciled, since the unified attachments saved object is always registered. `xpack.cases.attachments.enabled` only affects which saved object new attachments are written to — not what the analytics index reads. - Deleting a case also deletes its comments and attachments, so when a case is deleted we make sure to remove its `.cases-attachments` docs as well. With all three indices now in place, a final PR will follow to remove the existing V1 implementation. The current cases as data v1 work is unaffected. ## Technical Implementation For a complete set of details, please see: `x-pack/platform/plugins/shared/cases/server/cases_analytics_v2/README.md` ## Testing ### Setup Add to `config/kibana.dev.yml`: ```yaml xpack.cases.analyticsV2.enabled: true xpack.cases.analyticsV2.enableAdminRoutes: true # Optional — makes NEW attachments write to the unified `cases-attachments` # saved object (default is the legacy `cases-comments`). The analytics index # mirrors/reconciles both source types either way. xpack.cases.attachments.enabled: true ``` Restart Kibana, then open **Dev Tools → Console**. ### 1. Real-time writes Create a case and add a couple of things to it (a comment, and an attachment if you enabled `attachments.enabled`). Then confirm the data landed: ``` GET .cases-attachments/_search { "query": { "term": { "cases.id": "<case id>" } } } ``` Each comment / attachment should have a matching doc, with the attachment `type` and a few of the important fields pulled out for easier analysis. Editing or deleting a comment should be reflected on the next search (upsert / delete path). ### 2. Deleting a case Delete the case, then re-run the search above → 0 hits (its attachment docs are removed along with the case). ### 3. Reconciliation + reset - `GET kbn:/internal/cases/_analyticsV2/state` now reports all three indices (`.cases`, `.cases-activity`, `.cases-attachments`) and includes `attachments_last_run_at` alongside the existing reconciliation info. - `POST kbn:/internal/cases/_analyticsV2/reset` rebuilds all three indices. Poll `/state` and confirm they all come back. While a reset is running, `/state` reports a single `running` phase with the per-index processed counts (`cases_processed`, `activity_processed`, `attachments_processed`) climbing together, since the three rebuilds run in parallel. #### Automated coverage ```bash node scripts/functional_tests \ --config x-pack/platform/test/cases_api_integration/spaces_only/config_analytics_v2.ts yarn jest --config x-pack/platform/plugins/shared/cases/jest.config.js cases_analytics_v2 ``` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. - [ ] All changes are net new and behind `xpack.cases.analyticsV2.enabled` (off by default); the primary case / comment / attachment writes are never blocked by an analytics write, and the reconciliation task backstops any missed writes. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Feature Flag:
xpack.cases.analyticsV2.enabledThis is a follow up to #269581 and #275686, which introduced the cases as data V2 work and its first two indices,
.casesand.cases-activity. This PR adds the third and final one,.cases-attachments, which holds the case comments and attachments data (one document per comment / attachment). TheCases Analyticsdata view added in the first PR already points at all three indices, so this data can be joined back to its case viacases.idusing an ES|QLLOOKUP JOIN.Like the other two indices,
.cases-attachmentsfollows the same approach set up in the previous PRs:.cases-attachmentsindex. As with the other indices, this will not block the primary write from happening if the analytics write fails..cases-attachments. Each index is reconciled independently, so an issue with one won't hold up the others./resetand/reconcile/run_soonadmin routes and the initial backfill now cover all three indices. The backfill and/resetrebuild the three indices in parallel, and read saved objects in larger batches, so a full rebuild finishes noticeably faster than doing them one after another.Of note:
This index holds both the older case comments (cases-comments) and the newer unified attachments (cases-attachments), normalized into a single document shape, so it works whether or not a deployment has moved to the new attachments saved object. Both source types are always mirrored and reconciled, since the unified attachments saved object is always registered.
xpack.cases.attachments.enabledonly affects which saved object new attachments are written to — not what the analytics index reads.Deleting a case also deletes its comments and attachments, so when a case is deleted we make sure to remove its
.cases-attachmentsdocs as well.With all three indices now in place, a final PR will follow to remove the existing V1 implementation. The current cases as data v1 work is unaffected.
Technical Implementation
For a complete set of details, please see:
x-pack/platform/plugins/shared/cases/server/cases_analytics_v2/README.mdTesting
Setup
Add to
config/kibana.dev.yml:Restart Kibana, then open Dev Tools → Console.
1. Real-time writes
Create a case and add a couple of things to it (a comment, and an attachment if you enabled
attachments.enabled). Then confirm the data landed:Each comment / attachment should have a matching doc, with the attachment
typeand a few of the important fields pulled out for easier analysis. Editing or deleting a comment should be reflected on the next search (upsert / delete path).2. Deleting a case
Delete the case, then re-run the search above → 0 hits (its attachment docs are removed along with the case).
3. Reconciliation + reset
GET kbn:/internal/cases/_analyticsV2/statenow reports all three indices (.cases,.cases-activity,.cases-attachments) and includesattachments_last_run_atalongside the existing reconciliation info.POST kbn:/internal/cases/_analyticsV2/resetrebuilds all three indices. Poll/stateand confirm they all come back. While a reset is running,/statereports a singlerunningphase with the per-index processed counts (cases_processed,activity_processed,attachments_processed) climbing together, since the three rebuilds run in parallel.Automated coverage
Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
Identify risks
Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.
xpack.cases.analyticsV2.enabled(off by default); the primary case / comment / attachment writes are never blocked by an analytics write, and the reconciliation task backstops any missed writes.