Skip to content

ti_*: fix transform destination to accept multiple namespaces - #20002

Merged
efd6 merged 1 commit into
elastic:mainfrom
efd6:s7331-ti
Jul 7, 2026
Merged

ti_*: fix transform destination to accept multiple namespaces#20002
efd6 merged 1 commit into
elastic:mainfrom
efd6:s7331-ti

Conversation

@efd6

@efd6 efd6 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Proposed commit message

ti_*: fix transform destination to accept multiple namespaces

The transform destination index for all threat intel packages defines
data_stream.namespace as constant_keyword without an explicit value.
The first document locks the constant to its namespace, and any
subsequent document from a different namespace is permanently
rejected, halting the transform entirely. This breaks dashboards and
detection rules that depend on the transform output.

Change data_stream.namespace to keyword in the transform destination
field definitions. Bump destination index suffixes and
fleet_transform_version to force recreation on upgrade. Namespace is
deliberately excluded from unique_key because threat indicators are
global: the same indicator from different namespaces should
deduplicate to one entry.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 self-assigned this Jul 6, 2026
@efd6 efd6 added bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Jul 6, 2026
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@efd6 efd6 added Integration:ti_misp MISP Integration:ti_cybersixgill Cybersixgill Integration:ti_abusech abuse.ch Integration:ti_otx AlienVault OTX Integration:ti_recordedfuture Recorded Future Integration:ti_anomali Anomali ThreatStream Integration:ti_maltiverse Maltiverse (Partner supported) Integration:ti_threatq ThreatQuotient (Partner supported) Integration:ti_cif3 Collective Intelligence Framework v3 (Community supported) Integration:ti_rapid7_threat_command Rapid7 Threat Command (Partner supported) Integration:ti_opencti OpenCTI Integration:ti_eclecticiq EclecticIQ (Partner supported) Integration:ti_threatconnect ThreatConnect Integration:ti_eset ESET Threat Intelligence (Partner supported) Integration:ti_custom Custom Threat Intelligence Integration:ti_domaintools DomainTools Feeds (Partner supported) Integration:ti_google_threat_intelligence Google Threat Intelligence (Partner supported) Integration:ti_greynoise GreyNoise (Community supported) Integration:ti_cyware_intel_exchange Cyware Intel Exchange Integration:ti_anyrun ANY.RUN Threat Intelligence Feeds (Partner supported) Integration:ti_flashpoint Flashpoint Integration:ti_socradar_taxii SOCRadar Threat Intelligence (TAXII) (Partner supported) Integration:ti_socradar_feeds SOCRadar Threat Feeds (Partner supported) labels Jul 6, 2026
@efd6
ti_*: fix transform destination to accept multiple namespaces
c39391b 
The transform destination index for all threat intel packages defines
data_stream.namespace as constant_keyword without an explicit value.
The first document locks the constant to its namespace, and any
subsequent document from a different namespace is permanently
rejected, halting the transform entirely. This breaks dashboards and
detection rules that depend on the transform output.

Change data_stream.namespace to keyword in the transform destination
field definitions. Bump destination index suffixes and
fleet_transform_version to force recreation on upgrade. Namespace is
deliberately excluded from unique_key because threat indicators are
global: the same indicator from different namespaces should
deduplicate to one entry.
@vera-review-bot

vera-review-bot Bot commented Jul 7, 2026

Copy link
Copy Markdown

No issues across the latest commits c39391b.

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

ShourieG
ShourieG approved these changes Jul 7, 2026
View reviewed changes

@ShourieG ShourieG left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mergify

mergify Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

kcreddy
kcreddy approved these changes Jul 7, 2026
View reviewed changes

@kcreddy kcreddy left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Non-blocking:
ti_eclecticiq's README hardcodes logs-ti_eclecticiq_latest.observables-1 (used in a dashboard _index filter). That name doesn't match the actual dest index and predates this PR.

@efd6
efd6 merged commit 5b0f778 into elastic:main Jul 7, 2026
11 checks passed
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_abusech - 4.1.2 containing this change is available at https://epr.elastic.co/package/ti_abusech/4.1.2/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_anomali - 2.8.1 containing this change is available at https://epr.elastic.co/package/ti_anomali/2.8.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_anyrun - 1.2.2 containing this change is available at https://epr.elastic.co/package/ti_anyrun/1.2.2/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_cif3 - 1.20.1 containing this change is available at https://epr.elastic.co/package/ti_cif3/1.20.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_custom - 1.8.1 containing this change is available at https://epr.elastic.co/package/ti_custom/1.8.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_cybersixgill - 1.36.1 containing this change is available at https://epr.elastic.co/package/ti_cybersixgill/1.36.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_cyware_intel_exchange - 0.4.1 containing this change is available at https://epr.elastic.co/package/ti_cyware_intel_exchange/0.4.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_domaintools - 1.5.1 containing this change is available at https://epr.elastic.co/package/ti_domaintools/1.5.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_eclecticiq - 1.7.1 containing this change is available at https://epr.elastic.co/package/ti_eclecticiq/1.7.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_eset - 1.11.1 containing this change is available at https://epr.elastic.co/package/ti_eset/1.11.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_flashpoint - 0.3.1 containing this change is available at https://epr.elastic.co/package/ti_flashpoint/0.3.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_google_threat_intelligence - 1.1.1 containing this change is available at https://epr.elastic.co/package/ti_google_threat_intelligence/1.1.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_greynoise - 0.9.2 containing this change is available at https://epr.elastic.co/package/ti_greynoise/0.9.2/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_maltiverse - 1.8.1 containing this change is available at https://epr.elastic.co/package/ti_maltiverse/1.8.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_misp - 1.44.1 containing this change is available at https://epr.elastic.co/package/ti_misp/1.44.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_opencti - 2.15.1 containing this change is available at https://epr.elastic.co/package/ti_opencti/2.15.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_otx - 1.32.1 containing this change is available at https://epr.elastic.co/package/ti_otx/1.32.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_rapid7_threat_command - 2.10.2 containing this change is available at https://epr.elastic.co/package/ti_rapid7_threat_command/2.10.2/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_recordedfuture - 2.6.1 containing this change is available at https://epr.elastic.co/package/ti_recordedfuture/2.6.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_socradar_feeds - 0.1.1 containing this change is available at https://epr.elastic.co/package/ti_socradar_feeds/0.1.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_socradar_taxii - 0.2.1 containing this change is available at https://epr.elastic.co/package/ti_socradar_taxii/0.2.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_threatconnect - 2.2.1 containing this change is available at https://epr.elastic.co/package/ti_threatconnect/2.2.1/

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Package ti_threatq - 1.39.1 containing this change is available at https://epr.elastic.co/package/ti_threatq/1.39.1/

@github-actions github-actions Bot mentioned this pull request Jul 7, 2026
Merged
1 of 5 tasks
@andrewkroh andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kcreddy kcreddy kcreddy approved these changes
@ShourieG ShourieG ShourieG approved these changes

Assignees

@efd6 efd6

Labels

bugfix Pull request that fixes a bug issue documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. Integration:ti_abusech abuse.ch Integration:ti_anomali Anomali ThreatStream Integration:ti_anyrun ANY.RUN Threat Intelligence Feeds (Partner supported) Integration:ti_cif3 Collective Intelligence Framework v3 (Community supported) Integration:ti_custom Custom Threat Intelligence Integration:ti_cybersixgill Cybersixgill Integration:ti_cyware_intel_exchange Cyware Intel Exchange Integration:ti_domaintools DomainTools Feeds (Partner supported) Integration:ti_eclecticiq EclecticIQ (Partner supported) Integration:ti_eset ESET Threat Intelligence (Partner supported) Integration:ti_flashpoint Flashpoint Integration:ti_google_threat_intelligence Google Threat Intelligence (Partner supported) Integration:ti_greynoise GreyNoise (Community supported) Integration:ti_maltiverse Maltiverse (Partner supported) Integration:ti_misp MISP Integration:ti_opencti OpenCTI Integration:ti_otx AlienVault OTX Integration:ti_rapid7_threat_command Rapid7 Threat Command (Partner supported) Integration:ti_recordedfuture Recorded Future Integration:ti_socradar_feeds SOCRadar Threat Feeds (Partner supported) Integration:ti_socradar_taxii SOCRadar Threat Intelligence (TAXII) (Partner supported) Integration:ti_threatconnect ThreatConnect Integration:ti_threatq ThreatQuotient (Partner supported) Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@efd6 @kcreddy @ShourieG @andrewkroh