Skip to content

feat: Update Jetty to use LoggingProcessorMacro #2001

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Aug 18, 2025
Merged

feat: Update Jetty to use LoggingProcessorMacro #2001

merged 5 commits into from
Aug 18, 2025

Conversation

@Caleb-Hurshman
Copy link
Collaborator

@Caleb-Hurshman Caleb-Hurshman commented Jul 23, 2025

Description

Refactor the Jetty receiver logging to be sub-agent agnostic.

Related issue

How has this been tested?

Updated transformation_test and confgenerator_test, confirmed the output of output_fluentbit.yaml was unchanged after refactoring the receiver.

Checklist:

  • Unit tests
    • Unit tests do not apply.
    • Unit tests have been added/modified and passed for this PR.
  • Integration tests
    • Integration tests do not apply.
    • Integration tests have been added/modified and passed for this PR.
  • Documentation
    • This PR introduces no user visible changes.
    • This PR introduces user visible changes and the corresponding documentation change has been made.
  • Minor version bump
    • This PR introduces no new features.
    • This PR introduces new features, and there is a separate PR to bump the minor version since the last release already.
    • This PR bumps the version.

@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 23, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 23, 2025
@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/jetty-otel-logging branch from 038f2c8 to cb8d63d Compare July 23, 2025 20:39
@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 23, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 23, 2025
@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/jetty-otel-logging branch from cb8d63d to 21c2a55 Compare July 31, 2025 13:58
@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
franciscovalentecastro
franciscovalentecastro reviewed Jul 31, 2025
View reviewed changes
@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/jetty-otel-logging branch from 21c2a55 to 8cdc422 Compare July 31, 2025 15:53
@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
franciscovalentecastro
franciscovalentecastro reviewed Jul 31, 2025
View reviewed changes
@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/jetty-otel-logging branch from a29f8d2 to dc378d3 Compare July 31, 2025 17:46
@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Jul 31, 2025
franciscovalentecastro
franciscovalentecastro requested changes Aug 8, 2025
View reviewed changes
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR LGTM!

One thing before merging is that after rebasing to HEAD, we would need to run make transformation_test_update again (also may need to run make rebuild_sumodules) to get the changes after the new pase_regex support (#1930).

@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/jetty-otel-logging branch from dc378d3 to 203c980 Compare August 12, 2025 13:12
@Caleb-Hurshman
Copy link
Collaborator Author

Caleb-Hurshman commented Aug 12, 2025

One thing before merging is that after rebasing to HEAD, we would need to run make transformation_test_update again (also may need to run make rebuild_sumodules) to get the changes after the new pase_regex support (#1930).

@franciscovalentecastro After rebasing and running make transformation_test_update, many of the output_otel.yaml are changed from a set of entries to something like this (apache error log example):

- exit_error: exit status 1
- collector_errors:
  - stderr: |
      Error: invalid configuration: processors::transform/input_0: unable to parse OTTL statement "set(log.cache[\"__parsed_regex\"], ExtractPatternsRubyRegex(log.body[\"message\"], \"^\\\\[(?<time>[^\\\\]]+)\\\\] \\\\[(?:(?<module>\\\\w+):)?(?<level>[\\\\w\\\\d]+)\\\\](?: \\\\[pid (?<pid>\\\\d+)(?::tid (?<tid>[0-9]+))?\\\\])?(?: (?<errorCode>[^\\\\[:]*):?)?(?: \\\\[client (?<client>[^\\\\]]*)\\\\])? (?<message>.*)$\")) where (log.body != nil and log.body[\"message\"] != nil)": error while parsing arguments for call to "set": invalid argument at position 1: undefined function "ExtractPatternsRubyRegex"
      2025/08/12 06:26:19 collector server run finished with error: invalid configuration: processors::transform/input_0: unable to parse OTTL statement "set(log.cache[\"__parsed_regex\"], ExtractPatternsRubyRegex(log.body[\"message\"], \"^\\\\[(?<time>[^\\\\]]+)\\\\] \\\\[(?:(?<module>\\\\w+):)?(?<level>[\\\\w\\\\d]+)\\\\](?: \\\\[pid (?<pid>\\\\d+)(?::tid (?<tid>[0-9]+))?\\\\])?(?: (?<errorCode>[^\\\\[:]*):?)?(?: \\\\[client (?<client>[^\\\\]]*)\\\\])? (?<message>.*)$\")) where (log.body != nil and log.body[\"message\"] != nil)": error while parsing arguments for call to "set": invalid argument at position 1: undefined function "ExtractPatternsRubyRegex"

Is this expected?
Note: this occurs regardless of whether I run make rebuild_submodules

@franciscovalentecastro
Copy link
Contributor

franciscovalentecastro commented Aug 12, 2025 •
edited
Loading

Re @Caleb-Hurshman :

Is this expected?

The error undefined function "ExtractPatternsRubyRegex" happens when the built local version of the Otel Collector (otelopscol) is still pointing to an older version, e.g. before the parse_regex support.

My guess is that the local repo submodule opentelemetry-operations-collector in ops-agent/tree/master/submodules is still pointing to a previous commit. Some ways to address this are :

  • git submodule update to update the opentelemetry-operations-collector to the commit version in master.
  • make otelopscol_local to force a new binary rebuild of the otel collector.
  • make rebuild_submodules.

franciscovalentecastro
franciscovalentecastro approved these changes Aug 12, 2025
View reviewed changes
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/jetty-otel-logging branch from b455388 to d3601e6 Compare August 18, 2025 15:44
@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Aug 18, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Aug 18, 2025
@franciscovalentecastro franciscovalentecastro merged commit a35bc2c into GoogleCloudPlatform:master Aug 18, 2025
37 checks passed
@Caleb-Hurshman Caleb-Hurshman deleted the feat/jetty-otel-logging branch August 18, 2025 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@franciscovalentecastro franciscovalentecastro franciscovalentecastro approved these changes

Assignees

No one assigned

Labels

review-focus

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Caleb-Hurshman @franciscovalentecastro @stackdriver-instrumentation-release