Skip to content

Tomcat and Redis: Update input.log to match existing regex #2011

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jul 31, 2025
Merged

Tomcat and Redis: Update input.log to match existing regex #2011

merged 3 commits into from
Jul 31, 2025

Conversation

@dyl10s
Copy link
Collaborator

@dyl10s dyl10s commented Jul 30, 2025

Description

Updated the input logs for golden files to match the existing regex parsers for Tomcat System and Redis receivers

Related issue

How has this been tested?

Checklist:

  • Unit tests
    • Unit tests do not apply.
    • Unit tests have been added/modified and passed for this PR.
  • Integration tests
    • Integration tests do not apply.
    • Integration tests have been added/modified and passed for this PR.
  • Documentation
    • This PR introduces no user visible changes.
    • This PR introduces user visible changes and the corresponding documentation change has been made.
  • Minor version bump
    • This PR introduces no new features.
    • This PR introduces new features, and there is a separate PR to bump the minor version since the last release already.
    • This PR bumps the version.

franciscovalentecastro
franciscovalentecastro reviewed Jul 30, 2025
View reviewed changes
transformation_test/testdata/logging_processor-tomcat-system/input.log Outdated
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit : Is it possible to add one last "log" line after the "multiline" example ? This will make sure the "multiline" example is completed.

Now i'm realizing that MultilineRule works as a state machine and it keeps looking for the next cont line different until it doesn't find it, so there is no ending "state" for this log.

ops-agent/apps/tomcat.go

Lines 93 to 104 in cda3830

Rules: []confgenerator.MultilineRule{
{
StateName: "start_state",
NextState: "cont",
Regex: `^\d{2}-[A-Z]{1}[a-z]{2}-\d{4}\s\d{2}:\d{2}:\d{2}.\d{3}`,
},
{
StateName: "cont",
NextState: "cont",
Regex: `^(?!\d{2}-[A-Z]{1}[a-z]{2}-\d{4}\s\d{2}:\d{2}:\d{2}.\d{3})`,
},
},

Copy link
Collaborator Author

@dyl10s dyl10s Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some additional logs after and that looks like it parsed it correctly. Seems like a bug that could be addressed in the future.

Copy link
Contributor

@franciscovalentecastro franciscovalentecastro Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update! I don't think this is a "bug" per-se, since that is how Fluent-bit multiline parsing works . Otel Logging multiline parsing may work a bit differently when we implement it.

After this update, now this is PR is facing the issue that is going to be fixed in #2010 . Will merge this after the fix.

transformation_test/testdata/logging_processor-tomcat-system/output_fluentbit.yaml Outdated
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit : I'm realizing that [ContainerBackgroundProcessor[StandardEngine[Catalina]]] won't be parsed correctly by \[(?<module>[^\]]+)\] due to the nested brackets.

ops-agent/apps/tomcat.go

Lines 79 to 83 in cda3830

// Sample line: at org.apache.catalina.util.LifecycleBase.invalidTransition(LifecycleBase.java:430)
// Sample line: at org.apache.catalina.util.LifecycleBase.destroy(LifecycleBase.java:316)
Regex: `^(?<time>\d{2}-[A-Z]{1}[a-z]{2}-\d{4}\s\d{2}:\d{2}:\d{2}.\d{3})\s(?<level>[A-Z]+)\s\[(?<module>[^\]]+)\]\s(?<message>(?<source>[\w\.]+)[\S\s]+)`,
Parser: confgenerator.ParserShared{
TimeKey: "time",

I don't think we should update/address/fix in this PR. This is more of a curiosity about Apache Tomcat versions. Does the application version of matter when getting this type of "nested bracket" logs ?

Copy link
Collaborator Author

@dyl10s dyl10s Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will leave this as is for now

Copy link
Contributor

@franciscovalentecastro franciscovalentecastro Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree! We should leave it as is for now. I was mostly curious if this was due to the logs from a different Tomcat version.

@franciscovalentecastro franciscovalentecastro force-pushed the update-regex-parse-tests-missing-fields branch from 2c821c2 to c962589 Compare July 30, 2025 20:03
franciscovalentecastro
franciscovalentecastro approved these changes Jul 30, 2025
View reviewed changes
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@franciscovalentecastro franciscovalentecastro merged commit 5c97254 into GoogleCloudPlatform:master Jul 31, 2025
53 of 56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@franciscovalentecastro franciscovalentecastro franciscovalentecastro approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dyl10s @franciscovalentecastro