feat: Refactor IIS to use `LoggingProcessorMacro` #2014

Caleb-Hurshman · 2025-07-31T18:17:25Z

Description

Refactor the IIS receiver logging to be sub-agent agnostic.
I am considering this a "rough draft". I believe the Lua functionality has been preserved. IIS is Windows-only, but the RegisterLoggingFilesProcessorMacro function doesn't handle a platform argument yet.

Related issue

How has this been tested?

Updated transformation_test and confgenerator_test, confirmed the output of output_fluentbit.yaml was unchanged after refactoring the receiver.

Checklist:

Unit tests
- Unit tests do not apply.
- Unit tests have been added/modified and passed for this PR.
Integration tests
- Integration tests do not apply.
- Integration tests have been added/modified and passed for this PR.
Documentation
- This PR introduces no user visible changes.
- This PR introduces user visible changes and the corresponding documentation change has been made.
Minor version bump
- This PR introduces no new features.
- This PR introduces new features, and there is a separate PR to bump the minor version since the last release already.
- This PR bumps the version.

franciscovalentecastro

@Caleb-Hurshman This is a good "rough draft". It really brings to light all the technical considerations around refactoring IIS. I added some comments. Thank you for the effort in this so far!

apps/iis.go

transformation_test/testdata/logging_processor-iis_access/output_otel.yaml

apps/iis.go

transformation_test/testdata/logging_processor-iis_access/output_otel.yaml

franciscovalentecastro · 2025-08-23T04:00:53Z

confgenerator/logging_modify_fields.go

After further thought, i think we shouldn't add CustomLuaFunc / CustomOTTLFunc to ModifyFields. Some reasons for this :

ModifyFields current abstraction is to work with the fields <dest> , <source>, but this new features extend this to an arbitrary list of fields.

ModifyFields is also a feature exposed to users.

This is required by only the IIS processor so adding complexity here is unnecessary for only one case.

The only required operation missing from ModifyFields is string concatenation. Using CustomConvertFunc + Concat ¹ we can do this in Otel.

For fluent-bit we can keep a similar Lua script for the string concatenation and simplify the steps that can be done with ModifyFields (e.g. ommiting fields).

Please remove CustomLuaFunc / CustomOTTLFunc and do the following instead :

Create an InternalLoggingProcessor (interface) called IISConcatFields which implements the methods Components (fluent-bit) and Processors (otel).

Use ModifyFields + CustomConvertFunc + Concat ¹ for the Otel implementation. CustomConvertFunc can access any field in the log.

Use LuaFilterComponents for fluent-bit implementation of concatenation.

Simplify/remove the existing LuaScripts if possible.

Let me know if this is possible or if there any specific limitations.

Footnotes

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/ottlfuncs/README.md#concat ↩ ↩²

@franciscovalentecastro Most of this sounds good, but the otel output has not been working properly after changing to Processors:
- config_error: "processor \"processor0\" has invalid configuration: unimplemented"

I believe there might be a mismatch with the processor interface in OTel, let me know what you think.

I think the reason of that failure is that the signature of the Processors method should return ([]otel.Component, error) :

ops-agent/confgenerator/config.go

Lines 799 to 803 in c6005d9

}

type InternalOTelProcessor interface {

Processors(context.Context) ([]otel.Component, error)

}

For this particular implementation it seems you would be able to call the Processors method of the modifyFields processor defined within the IISConcatFields.Processors method and "return" the result.

I did try the return that way, but encountered the same result. I'll change the return type back, but there's more to this issue. I'll dig in further once the rest of the receivers are up to date.

Yes, you are right, there is more to this issue. This is a bug i've encountered before, but failed to fix in other PR's. I can fix this in a separate PR or we can do it here. Let me know what do you think its best.

[Optional]

In logging_macros.go, all the Pipelines () / Processors() methods should use InternalOtelReceiver / InternalOtelProcessor instead of OtelReceiver / OtelProcessor. This should fix the unimplemented error.

ops-agent/confgenerator/logging_macros.go

Lines 64 to 66 in 5171236

receiver, processors := cr.Expand(ctx)

if r, ok := any(receiver).(OTelReceiver); ok {

rps, err := r.Pipelines(ctx)

ops-agent/confgenerator/logging_macros.go

Lines 71 to 73 in 5171236

for _, p := range processors {

if p, ok := p.(OTelProcessor); ok {

c, err := p.Processors(ctx)

ops-agent/confgenerator/logging_macros.go

Lines 126 to 128 in 5171236

for _, lp := range cp.Expand(ctx) {

if p, ok := any(lp).(OTelProcessor); ok {

c, err := p.Processors(ctx)

I'm glad to hear you know what the issue is! I think a separate PR would be good since a lot is going on in this PR already.

apps/iis.go

github-actions · 2025-10-28T03:20:27Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

Caleb-Hurshman requested a review from franciscovalentecastro July 31, 2025 20:25

Caleb-Hurshman force-pushed the feat/iis-otel-logging branch from 31303da to 8adee32 Compare August 18, 2025 16:02

franciscovalentecastro reviewed Aug 19, 2025

View reviewed changes

franciscovalentecastro added the review-focus label Aug 19, 2025

Caleb-Hurshman force-pushed the feat/iis-otel-logging branch from 8adee32 to 816ca88 Compare August 25, 2025 13:25

franciscovalentecastro reviewed Aug 25, 2025

View reviewed changes

Caleb-Hurshman force-pushed the feat/iis-otel-logging branch from 811d2d3 to 75332f6 Compare August 25, 2025 15:39

Caleb-Hurshman requested a review from franciscovalentecastro August 25, 2025 15:39

franciscovalentecastro mentioned this pull request Aug 25, 2025

feat: Update Mongodb receiver to use LoggingReceiverMacro #2003

Open

9 tasks

Caleb-Hurshman added 11 commits August 28, 2025 12:57

testdata + refactor

51b5621

WIP, wait for feedback

288dfe9

WIP

01e0d8c

rough draft

f8c30a8

rebase, update goldens

36b9e25

fix timeformat space

93ea0cb

refactor, add IISConcatFieldsProcessor

5a398e4

expose platforms for macro processors

b76a6d2

rebase, update confgenerator

5b4f1f2

fix Processor() return signature

799b4c9

rebase

11bade5

Caleb-Hurshman force-pushed the feat/iis-otel-logging branch from deda40c to 11bade5 Compare August 28, 2025 13:02

github-actions bot added the Stale label Oct 28, 2025

	}
	type InternalOTelProcessor interface {
	Processors(context.Context) ([]otel.Component, error)
	}

	receiver, processors := cr.Expand(ctx)
	if r, ok := any(receiver).(OTelReceiver); ok {
	rps, err := r.Pipelines(ctx)

	for _, p := range processors {
	if p, ok := p.(OTelProcessor); ok {
	c, err := p.Processors(ctx)

	for _, lp := range cp.Expand(ctx) {
	if p, ok := any(lp).(OTelProcessor); ok {
	c, err := p.Processors(ctx)

feat: Refactor IIS to use LoggingProcessorMacro #2014

Are you sure you want to change the base?

feat: Refactor IIS to use LoggingProcessorMacro #2014

Uh oh!

Conversation

Caleb-Hurshman commented Jul 31, 2025

Description

Related issue

How has this been tested?

Checklist:

Uh oh!

franciscovalentecastro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

franciscovalentecastro Aug 23, 2025

Choose a reason for hiding this comment

Footnotes

Uh oh!

Caleb-Hurshman Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

franciscovalentecastro Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

Caleb-Hurshman Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

franciscovalentecastro Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

Caleb-Hurshman Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: Refactor IIS to use `LoggingProcessorMacro` #2014

feat: Refactor IIS to use `LoggingProcessorMacro` #2014