[Fortinet Fortigate] Prevent pipeline failures from oversized duration and non-IP address values by robester0403 · Pull Request #19893 · elastic/integrations

robester0403 · 2026-06-30T21:29:18Z

Two FortiGate log conditions currently fail the whole document in the log data stream:

duration overflow — some FortiOS IPsec tunnel-stats events emit an unsigned 64-bit duration (e.g. 18446744073700579894) that exceeds long, so the duration script throws and the entire event (incl. advpnsc, byte counts, etc.) is dropped.
Non-IP source.ip / destination.ip — malformed values (e.g. dstip=authid="...") cause network_direction/geoip to hard-fail and the field to be _ignored at index time.

Proposed commit message

Prevent pipeline failures from oversized duration and non-IP address values

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
~~- [ ] I have verified that all data streams collect metrics or logs.~~
I have added an entry to my package's changelog.yml file.
~~- [ ] I have verified that Kibana version constraints are current according to guidelines.~~
~~- [ ] I have verified that any added dashboard complies with Kibana's Dashboard good practices~~

Related issues

Partially Resolves: #19832

Screenshots

…d drop non-IP values from and

infra-vault-gh-plugin-prod · 2026-06-30T21:29:22Z

Pinging @elastic/integration-experience (Team:Integration-Experience)

github-actions · 2026-06-30T21:30:40Z

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

vera-review-bot · 2026-06-30T21:35:04Z

👀 I have started reviewing the PR

vera-review-bot · 2026-06-30T22:01:00Z

👀 I have started reviewing the PR

vera-review-bot · 2026-06-30T22:05:09Z

Vera Review Bot

For the current commit state, I did not find any issues.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

elastic-vault-github-plugin-prod · 2026-06-30T22:17:40Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

andrewkroh · 2026-07-01T01:13:51Z

      target_field: network.direction
      ignore_missing: true
      if: ctx.network?.direction == null
+  - grok:


You should use a convert processor with type: ip to validate fields are IP addresses and will index in elasticsearch. The processor doesn't actually convert the field value or type (it's still a string), but you can attach an on_failure to remove invalid IPs.

github-actions · 2026-07-03T14:09:36Z

TL;DR

Buildkite failed during the pipeline upload because the PR can no longer be auto-merged with main. The conflict is in packages/fortinet_fortigate/changelog.yml: main already has a top 1.36.7 FortiGate entry from PR #19935, while this PR also adds 1.36.7.

Remediation

Rebase or merge latest main, resolve packages/fortinet_fortigate/changelog.yml by keeping both changelog entries, and bump this PR's package version/changelog entry to the next release version (likely 1.36.8) instead of reusing 1.36.7.
Update packages/fortinet_fortigate/manifest.yml to the same version, then rerun CI.

Investigation details

Root Cause

Buildkite's repository post-checkout hook checks out the PR, fetches main, creates pr_merge_19893, and attempts an automatic merge before uploading the dynamic pipeline. That merge failed because both branches modify the top of packages/fortinet_fortigate/changelog.yml.

Current main has this at the top of packages/fortinet_fortigate/changelog.yml:1:

# newer versions go on top
- version: "1.36.7"
  changes:
    - description: Support plain word identifiers in login source field, such as FortiClient Flexible Communication (fgfm_fgc).
      type: enhancement
      link: https://github.com/elastic/integrations/pull/19935

This PR adds a different 1.36.7 entry at the same location and also changes packages/fortinet_fortigate/manifest.yml from 1.36.6 to 1.36.7, so after rebasing it should use the next package version.

Evidence

Build: https://buildkite.com/elastic/integrations/builds/45473
Job/step: :pipeline::arrow_up: Upload Pipeline: .buildkite/pipeline.yml
Key log excerpt:

Auto-merging packages/fortinet_fortigate/changelog.yml
CONFLICT (content): Merge conflict in packages/fortinet_fortigate/changelog.yml
Automatic merge failed; fix conflicts and then commit the result.
Merge failed: 1
Error: running "repository post-checkout" shell hook: The repository post-checkout hook exited with status 1

Verification

Not run: the failure occurs before Buildkite uploads/generated package test steps, so there are no package test logs for this build.

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

# Conflicts: # packages/fortinet_fortigate/changelog.yml

elastic-vault-github-plugin-prod · 2026-07-06T12:34:06Z

✅ All changelog entries have the correct PR link.

vera-review-bot · 2026-07-06T12:39:26Z

 <185>date=2026-03-31 time=07:48:26 devname="MYDEV" devid="MYDEVID" eventtime=1773733509096718719 tz="-0500" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="root" severity="high" srcip=10.11.12.1 dstip=10.7.1.5 srcintf="eth0" srcintfrole="undefined" dstintf="eth1" dstintfrole="undefined" sessionid=1234567 action="detected" proto=6 service="HTTP" vrf=2 policyid=11 poluuid="07689140-fad1-4833-93ff-a5fcfbdc3041" policytype="policy" attack="HTTP.X-Forwarded-For.Header.XSS" srcport=50123 dstport=443 hostname="10.5.0.1" url="/?sample=1" agent="curl/7.47.0" httpmethod="GET" direction="outgoing" attackid=55823 profile="Profile1" ref="http://www.fortinet.com/ids/VID55823" incidentserialno=12345678 msg="web_misc: HTTP.X-Forwarded-For.Header.XSS" forwardedfor="abcd0e.xff\"></script><script>alert(document.domain);</script>" crscore=30 craction=8192 crlevel="high"
 <185>date=2026-03-31 time=07:48:26 devname="MYDEV" devid="MYDEVID" eventtime=1773733509096718719 tz="-0500" logid="0419016384" type="utm" subtype="ips" eventtype="signature" level="alert" vd="root" severity="high" srcip=10.11.12.1 dstip=10.7.1.5 srcintf="eth0" srcintfrole="undefined" dstintf="eth1" dstintfrole="undefined" sessionid=1234567 action="detected" proto=6 service="HTTP" vrf=2 policyid=11 poluuid="07689140-fad1-4833-93ff-a5fcfbdc3041" policytype="policy" attack="HTTP.X-Forwarded-For.Header.XSS" srcport=50123 dstport=443 hostname="10.5.0.1" url="/?sample=1" agent="curl/7.47.0" httpmethod="GET" direction="outgoing" attackid=55823 profile="Profile1" ref="http://www.fortinet.com/ids/VID55823" incidentserialno=12345678 msg="web_misc: HTTP.X-Forwarded-For.Header.XSS\\" forwardedfor="abcd0e.xff\"></script><script>alert(document.domain);</script>" crscore=30 craction=8192 crlevel="high"
+<189>date=2026-06-25 time=17:42:42 devname="MYDEV" devid="MYDEVID" eventtime=1782409391234567890 tz="+0000" logid="0419016384" type="event" subtype="vpn" level="notice" vd="root" logdesc="IPsec tunnel statistics" msg="IPsec tunnel statistics" action="tunnel-stats" remip=192.0.2.10 locip=192.0.2.20 remport=500 locport=500 outintf="wan1" srccountry="Reserved" cookies="1111111122222222/3333333344444444" user="192.0.2.10" group="N/A" useralt="N/A" xauthuser="N/A" xauthgroup="N/A" assignip=N/A vpntunnel="VPNTUNNEL" tunnelip=N/A tunnelid=1234567890 tunneltype="ipsec" duration=18446711111123456789 sentbyte=0 rcvdbyte=0 nextstat=600 fctuid="N/A" advpnsc=0
+<189>date=2026-06-25 time=13:43:13 devname="MYDEV" devid="MYDEVID" eventtime=1782409391234567890 tz="-0400" logid="0419016384" type="event" subtype="user" level="notice" vd="root" logdesc="Explicit proxy authentication successful" srcip=192.0.2.202 dstip=authid="AUTH-ID" user="user1" authproto="HTTP(192.0.2.202)" action="NEGO-auth" status="success" reason="Authentication succeeded" msg="User user1 succeeded in authentication"


Severity: 🔵 Low confidence: medium path: packages/fortinet_fortigate/data_stream/log/_dev/test/pipeline/test-fortinet-fix.log:23

The new test fixtures exercise the destination.ip drop path but not the symmetric source.ip drop path; add a log line with a non-IP srcip so remove_source_ip_a34d8ade is covered.

Details

This PR adds two symmetric convert-then-remove branches: convert_source_ip_d1d4ef25 (on_failure remove_source_ip_a34d8ade) and convert_destination_ip_3561ff45 (on_failure remove_destination_ip_4e25a91b). The added auth event line has a malformed dstip (dstip=authid="AUTH-ID"), so it exercises and verifies the destination.ip removal branch. No added fixture line contains a non-IP srcip, so the source.ip removal branch has no test coverage and a regression in that path would not be caught.

Recommendation:

Add a fixture line whose srcip is a non-IP value (mirroring the malformed dstip case) so the source.ip convert failure and removal are exercised, e.g.:

<189>date=2026-06-25 time=13:44:00 devname="MYDEV" devid="MYDEVID" eventtime=1782409391234567890 tz="-0400" logid="0419016384" type="event" subtype="user" level="notice" vd="root" logdesc="Explicit proxy authentication successful" srcip=notanip dstip=192.0.2.50 user="user2" action="NEGO-auth" status="success" reason="Authentication succeeded" msg="User user2 succeeded in authentication"

and add the matching entry to test-fortinet-fix.log-expected.json (regenerated via elastic-package test pipeline -g) confirming source.ip is absent.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

vera-review-bot · 2026-07-06T12:39:28Z

Review summary

Issues found across the latest commits 02664d3…fde90d3 (43 commits) — 1 low

🔵 The new test fixtures exercise the destination.ip drop path but not the symmetric source.ip drop path (link) (Unresolved)

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

infra-vault-gh-plugin-prod · 2026-07-06T12:54:50Z

💚 Build Succeeded

Buildkite Build
Commit: fde90d3

History

💔 Build #45473 failed 1b1380e
💚 Build #45281 succeeded c03bf61
💔 Build #45278 failed 3e7e946

cc @robester0403

mergify · 2026-07-06T20:02:20Z

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

Queue this pull request

elastic-vault-github-plugin-prod · 2026-07-06T21:28:15Z

Package fortinet_fortigate - 1.36.8 containing this change is available at https://epr.elastic.co/package/fortinet_fortigate/1.36.8/

fix: Prevent pipeline failures when exceeds the maximum long value an…

3e7e946

…d drop non-IP values from and

robester0403 self-assigned this Jun 30, 2026

robester0403 added the bugfix Pull request that fixes a bug issue label Jun 30, 2026

robester0403 requested a review from a team as a code owner June 30, 2026 21:29

robester0403 added Integration:fortinet_fortigate Fortinet FortiGate Firewall Logs Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience] labels Jun 30, 2026

vera-review-bot Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread packages/fortinet_fortigate/changelog.yml Outdated

chore: shorten changelog message and add in pr link

c03bf61

infra-vault-gh-plugin-prod Bot deleted a comment from elasticmachine Jun 30, 2026

andrewkroh reviewed Jul 1, 2026

View reviewed changes

fix: Use convert processors instead of grok to convert IP's and validate

1b1380e

Merge branch 'main' into fortinet-fortigate-duration-and-ip-issues

fde90d3

# Conflicts: # packages/fortinet_fortigate/changelog.yml

vera-review-bot Bot reviewed Jul 6, 2026

View reviewed changes

robester0403 requested a review from andrewkroh July 6, 2026 13:00

andrewkroh approved these changes Jul 6, 2026

View reviewed changes

robester0403 merged commit 696f576 into elastic:main Jul 6, 2026
10 checks passed

robester0403 deleted the fortinet-fortigate-duration-and-ip-issues branch July 6, 2026 21:10

Conversation

robester0403 commented Jun 30, 2026

Proposed commit message

Checklist

Related issues

Screenshots

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

✅ Elastic Docs Style Checker (Vale)

Uh oh!

vera-review-bot Bot commented Jun 30, 2026

Uh oh!

Uh oh!

vera-review-bot Bot commented Jun 30, 2026

Uh oh!

vera-review-bot Bot commented Jun 30, 2026

Vera Review Bot

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Uh oh!

andrewkroh Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 3, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jul 6, 2026

Uh oh!

vera-review-bot Bot Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

vera-review-bot Bot commented Jul 6, 2026

Review summary

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jul 6, 2026

💚 Build Succeeded

History

Uh oh!

mergify Bot commented Jul 6, 2026

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

elastic-vault-github-plugin-prod Bot commented Jun 30, 2026 •

edited

Loading